Mmmmm, Metrics
A few days ago Devon Smith announced she’s been working on quantifying how well LORT theaters use Twitter.
This is neat. I like this idea, and in the spirit of public feedback about it, here’s, uh, some public feedback:
The Metrics I Generally Dig
@mentions — Measuring mentions captures something about both re-tweets and conversations. Both of those things feel very important.
Followers — Measuring the number of follower certainly seems, on the face of it, to be a good yardstick. But: it only captures one level. I suspect this metric could be improved by factoring in the 2nd degree followers, i.e. how many followers do your followers have? My own Twitter account doesn’t have that many followers, but when I wrote a proposal for a new funding model for theater it reached the eyeballs (and struck the fancy) of Jess Hutchinson. It was Jess’s tweet, not mine, that gave that post traction. At the time Jess was a 2nd degree follower, through Nick Keenan. So I’d like to see a more sophisticated model for measuring followers.
Web Badge location — I don’t now how to weight this, but I’m so glad Devon tried. I know it can take time to modify a website, and maybe you want to test the twitter waters gently at first, but eventually, if you’re in, then freaking go in all the way. Make the choice. Commit. Don’t go weaksauce on us.
Twitter Name — Again, I don’t know how to weight it, but kudos to Devon for trying. It’s not just a branding thing, it’s a user interface thing. Think like a software developer and imagine what it will be like to actually use your Twitter name. I bet you a lot of money that a lot of people misspell GLTFCleveland.
The Metrics I Generally Don’t Buy
Frequency — Proof by counter-example: I have no qualms about un-following Twitter accounts that won’t shut up, even if they’re great tweets. In my experience quality and quantity don’t seem closely correlated on Twitter.
Total Tweets — See above.
Time in existence — I mean, if you were on the ball early on, cool, but I don’t think you get extra points for this. Maybe you knew you didn’t know how to use Twitter, in which case you should get extra points for not putzing around. Late to the party is no big deal if you come out swinging.
Client — Devon describes this metric as follows: “Included under the assumption that theatres using desktop applications (like TweetDeck) are able to better manage their Twitter presence”. I think that’s a bad assumption, and anyway, don’t grade the tools, grade how they’re used. A great foley artist could beat a lousy QLab user without much trouble.
Running with it
I’d love to see Devon’s metrics refined and extended. I’d also like to find a way to close the loop on evaluating the metrics. Can we connect these numbers to, say, ticket sales? Or volunteer hours clocked for the theater? Until we do something like that, it’s all speculation.
Speculating is fun, though
After reading Devon’s analysis, I wanted to play with some numbers too. However, I don’t really know anything about most of those LORT theaters (with one huge exception). Instead, I wanted to play with numbers for which I have some real-life context. To do that, I browsed through the Baltimore theaters I currently follow. Here they be, ordered by number of followers:
| Theater | Followers | Following | ERS/ING Ratio | Tweets |
| @SingleCarrot | 739 | 900 | 0.82 | 143 |
| @BIGimprov | 721 | 186 | 3.88 | 323 |
| @StrandTheater | 518 | 771 | 0.67 | 154 |
| @CENTERSTAGE_MD | 478 | 206 | 2.32 | 221 |
| @EverymanTheatre | 391 | 104 | 3.76 | 152 |
| @TheatreProject | 62 | 12 | 5.17 | 14 |
What to make of this, Armchair Edition
First off, what’s that “ERS/ING Ratio” thing? I propose that it’s one way to measure the strength of your magnet. If that number is high, your followers sought you out. A high ratio means you didn’t just troll for followers as a Twitter whore. (The tactic of following every single account you stumble on and hoping for a tag-back.)
The trouble with this ratio is that a high number is good, but a low number isn’t necessarily bad. For example, you yourself might be a tag-back follower. That’s not necessarily a bad thing, at least for an organization. If your style is to tag-back your own followers, then they might have all clicked “Follow” before you returned the favor, in which case you’ve still got a great magnet even though your ratio is diluted.
“I hate quotation. Tell me what you know.” ~ Ralph “Doomed to Ironic Appropriation” Emerson
Another thing worth noting: Lots of tweets don’t translate to lots of followers. Yes, I’m looking at you, CENTERSTAGE_MD. Yes, I know you’re running The Importance of Being Earnest, aka “the mildly amusing play that theaters will NOT FREAKING STOP PRODUCING”. Yes, I know Oscar Wilde was a clever fellow. Now stop using Twitter to quote him every single day, because no one cares.
Is it just me, or is it getting young in here?
Maybe this means nothing, or maybe it’s just to be expected, but I’d like to note that the theaters run by younger people (Single Carrot, BIG Improv, The Strand) are all kicking the Twitter asses of the theaters run by older people (CENTERSTAGE, Everyman, Theatre Project).
I have gone on record respectfully needling the older theaters about their relationship to Twitter. I don’t think anything I said in that post has really changed.
You Don’t Have To Twitter
Look, I’m big on Twitter. I think it’s the best, cleanest, coolest combination of personal and practical social networking that we’ve seen so far. But I can dig that it may not be your style. I genuinely don’t care if you use Twitter or not. I’d much rather see an organization use one kind of marketing really really well, than ten kinds poorly.
One thing I get from these numbers is that the bigger, older theaters maybe shouldn’t be jumping on the Twitter bandwagon. That would be okay. No, seriously, I’m a huge technology geek and I’m telling you: it’s okay to not use technology. The marketing that works best is the kind that comes from your heart. Find out what that means for you. If that means marketing a romantic show solely with the stunning use of letter-press printed postcards that double as a buy-one-get-one free coupon, which is the single way in which anyone can get a ticket for your event, which in turn leads to a massive “date night” for your show and you never even think about Twitter at any step of that process, dude, go for it. That would be so hott.
We need data
Despite all that stuff I just wrote, my biggest realization from looking at this table is that I simple don’t know what these numbers really mean. I want tools to give us more data. I want to see follower break-downs by locality (near/far). I want to see multi-level follower counts (1st degree/2nd degree/3rd degree). I want to track the effect of tweets on ticket sales, or volunteer hours, or something else I care about. I want to keep a running tab of how many local actors, designers, carpenters, or directors were found through Twitter connections. (I got my first acting gig in Baltimore because of Twitter.) I want to quantify the strength of the relationship on a per-follower basis (how did they start following? how often do they get into a conversation? how often do they re-tweet?). I want, in a word, more data, with more granularity. But we’ll need some tools to gather that stuff. (Do they exist already? Anyone know?)
Final Thought
Dude, Theatre Project. I love you. I am literally wearing your t-shirt right now. But come on, guys. You didn’t even try. You just gave up.
November 4th, 2009 at 6:16 pm
As usual: a gorgeous post.
I think you’re right: The trick here is finding metrics that measure traction. And most of those are not transparent enough until we can break in to the box office, cross check patron contact information with twitter usernames, and physically measure how many people were not only talking about a show before seeing it: but how many people IN THOSE PEOPLE’S network also went to see the show WITHOUT seeing it. That’s a lot of work for a marketing department to do.
It’s what NOT doing that and not improving these metrics means that scares me. This is a time when we should be pulling apart the traditional metrics of theatre ’sales’ and recalibrating our efforts to the metrics that actually matter. Measuring tweet frequency doesn’t measure quality of tweets — but measuring quality, depth, and breadth of communication across the surface area of the organization does indicate whether we are doing our job as theater artists: Telling stories that change the world.
November 5th, 2009 at 12:01 am
Just the kind of feedback I was looking for! A heartfelt thanks.
I’ll try a point/counterpoint:
Followers: I’d LOVE to capture 2nd/3rd degree social network. Have you by chance found a tool out there that will size up a user’s total network?
Frequency: So if the concern is about theatres being too prolific and users dropping them because of it, maybe I should think of ‘negative’ points if that number goes too high. For now, if a theatre manages to tweet once/day on average, I consider that successful.
Total Tweets: this is where the full data table would be helpful for you to see as proof of concept…I’ll be posting the entire spreadsheet online soon. Until then, I’m trying to see what kind of statistical impact # of tweets has on @mentions…in other words, it’s multi-variate regression time!
Time in Existence: a few months from now, perhaps even now, I agree this will become meaningless. I just wanted to give a little boost to people who jumped into the fray early. I could instead give these 5 points over to number of lists a theatre appears on. Thoughts?
Client: definitely something I’m still struggling with. I’m baffled by how some very effective twits are still using web as their interface. My interest is actually in “punishing” people posting to Twitter via Facebook. These are different platforms, and thus should have different uses/messages. My own habits unfortunately are a terrible example. But in truth, I definitely am mixing ‘encouraging best practices’ and ‘rewarding for good results.’
ROI: I’m trying to turn my attention to this now. It’s a little complex because everyone’s using it a little differently, and I don’t feel qualified to prescribe a single best-solution. Thinking hard about how to best adapt the brilliant ideas in http://www.slideshare.net/mzkagan/what-the-fk-is-social-media-one-year-later.
ERS/ING: I had at one point an “Authority” category that was this same measure. Unfortunately, it didn’t pan out in a predictable way. Some theatres follow 1:1, others as high as 10:1, but that number turned out not to be a good predictor of anything. It was also difficult to scale, so a theatre that might have 20 followers, and only follow 2 people was being rewarded.
More tools: I’m barely keeping up with you on this one. Check out the latest post: http://docs.google.com/View?id=dfv2gkvv_10g2ntr8ch
November 6th, 2009 at 5:21 pm
Hi Devon!
Cool stuff. I definitely am just armchair speculating here; I don’t know what tools are available.
I’m really interested to watch you continue to hash this out. (And reporting it to us all on Twitter, natch.) :-)
November 9th, 2009 at 8:39 pm
I think the Minnesota Timberwolves have been reading your blog, Chris: http://www.beastpieces.com/2009/11/timberwolves-basketball-invitation/
January 16th, 2010 at 5:19 pm
[...] to quantify the theatre field’s use of social media. What am I missing? Link to more. Also, Chris Ashworth responds. Share [...]