Ask good e-commerce or search marketing professionals how they build successful programs and you’ll hear them discuss things like conversion rates. Conversion rates are the percentage of a total audience that takes an action. I’ll take that a step further toward clarity and say that “conversion rate” is reserved for a monetary transaction. For other activities you try to motivate and measure (filling out forms, social sharing, answering questions and etc.), I prefer to use the term “action rate.” 

So, if you have 1,000 visitors to a web page and 200 of them fill out a form, then 20 of them actually purchase the product in question, you have a 20 percent action rate (200 of the 1,000 filled out the form … they took the action) and a two percent conversion rate (20 of the 1,000 actually “converted” to a customer.)

Search engine optimization and e-commerce experts live in this math. They know how many visitors to a given site or page it takes to produce one action. They then know how many of those actions they need to produce one conversion. They focus their energies then on two areas:

• Drive the requisite numbers to the page to result in the projected conversions for their business’s success
• Optimize the site or page so it takes fewer visitors and/or actions to achieve the same outcome

The more efficient the action and conversion rates on the back side of the site or page, the less work there is on the front side to get people there. So, in essence, this math is a microcosm of an effective business: Reduce costs and maximize revenue.

Your challenge this week is to determine what your action rate and conversion rate for your website, landing page or even social sites are. Then you can start reducing costs and maximizing revenue to optimize what you’re doing.

Metavana is a new-on-the-scene semantic-analysis vendor whose core science invokes a supposed universal descriptive pattern, the Maximum Information Principle. MIP, Metavana explains, describes the distribution of galaxy sizes and, as exploited by Metavana’s software, the distribution of multi-term, natural-language “n-plets.”

Interesting, but there’s plenty of computational-linguistics and semantic-science mojo in a host of established, competing text and sentiment analysis offerings, developed by smart people. The real question is this one: Does MIP make for great “solutions that measure customer satisfaction,” capable of “taming the chaos of the social Web”?

The answer is unclear, not yet established, despite an argument by authority, that the company was “founded by a renowned physicist,” Dr. Minh Duong-van (who speaks about the science and technology in a long video posted by Dana Stanley at his Research Access site, which Metavana CMO Romi Mahajan blogs for), and given only a couple of customer wins.

One very notable industry partnership does speak in Metavana’s favor, an alliance with Satmetrix to produce a simple, consumable number, a SparkScore, that seeks to lend new life to SatMetrix’ venerable but tired Net Promoter Score (NPS). Metavana, however, is only the latest of several text analysis companies that complement NPS with text-extracted information — Attensity, Clarabridge, Kana, Lexalytics, and others have supported the practice for years — and I even have a speaker lined up on the topic for the next Sentiment Analysis Symposium (which I organize), Bill Tuohig from J.D. Power and Associates. Yet the SatMetrix link is significant and will certainly help Metavana make sales. Read Dana Stanley’s GreenBook blog interview with Satmetrix CEO Richard Owen for detailed background. It’s interesting reading, but do keep in mind that Stanley apparently has a business connection with Metavana a Metavana insider.

Metavana’s published materials lack detail that would allow for assessment of company claims, and there’s no testable public interface. Further — a bad sign — Metavana co-founder Spencer Trask, a private-equity firm that is seeking to sell a $6 million Metavana stake, is disseminating information that unfairly slurs competitors and contains a number of factually incorrect statements. But getting back to what can be learned, let’s examine my earlier question, restated –

Is Metavana that innovative?

Metavana’s “search for meaning” technology extracts entities, attributes, topics, and sentiment from text. The company compares its Data Feed offering to Thomson Reuters’ streaming of news and financial information (without providing information on content or coverage) and offers (or will soon offer) an as-a-service sentiment scoring engine, which Erick Watson, Metavana director of product management compared, in a June 25 briefing, to ViralHeat’s. Our briefing centered on Watson’s showing me Metavana’s self-service social-intelligence application. It’s a graphical interface for exploration of social-harvested reviews and opinions, with the ability to associate polar (positive/negative) sentiment to star ratings and hooks for a pending influence measure. See it for yourself in a demo recorded by Watson.

The MIP approach seems quite reminiscent of natural-language understanding efforts that date to the late 1950s. Metavana’s application of the Maximum Information Principle (described in a company white paper) applies the same term-frequency-as-an-indicator-of-significance seen in Hans-Peter Luhn’s 1958 “The Automatic  Creation of Literature Abstracts.” Other, latter-day technologies break text into n-grams, which are multi-word sequences. Metavana uses the words “singlet,” “doublet,” and “triplet” where a linguist would refer to an “n-gram” (n=1, 2, or 3), and then Metavana assembles its n-gram-equivalents into unordered sets, borrowing the term “n-plet” from the physics world of the company’s founding scientist. Frankly, I don’t see a meaningful, practical difference between Metavana’s “n-plets” and the sets of terms that variations of a well-established probabilistic technique, latent semantic analysis, will adduce as typical of different text clusters. Documents that contain “dog,” “cat,” and “goldfish” might typify one cluster while “mustang,” “prius,” and “escalade” typify another.

According to President Michael Tupanjanin, Metavana applies machine-learning technology to refine classifications, and also creates models specific to different business domains such as smartphones, printers, hotels, and airlines. (Tupanjanin presented a 5-minute lightning talk at the May 8, 2012 Sentiment Analysis Symposium.)

There must be nuances to Metavana’s mixture-decomposition approach, but is the company’s approach better? I don’t know, but I do know that it’s not as good as asserted. Michael Tupanjanin states, “the algorithms that we have written have taken accuracy to a whole new level, up to over 95%.” This claim is repeated by founder Dr. Minh Duong-van, who puts Metavana sentiment resolution accuracy is “95, 96 per cent accurate,” in the video I linked to above. It is not supportable. Metavana’s measurement method is flawed, as I described in Never Trust Sentiment Accuracy Claims. Do not accept that 95% figure, but refocusing, let me restate the main point of that article, that –

Accuracy isn’t enough

The insights delivered by any worthwhile data analysis have to be useful and consumable. In my opinion, sentiment analysis that assigns a positive/negative/neutral score, at the document, sentence, or phrase level — that’s what Metavana does — are rarely sufficiently useful. Those crude tools are accurate enough to help you hit the “broad side of the barn” — I’ll give them that much — but they’re of little help when your business decision-making requires guidance that is highly specific, when the broad side of the barn isn’t good enough.

Metavana targets the customer-experience space. Better tools in that space (and in others), for instance from Clarabridge, extract sentiment at the feature level — for names of companies, products, and brands and for features such as, using hospitality as an example, a hotel’s cleanliness, staff friendliness, location, comfort, and value — and further analyze sentiment according to categories such as emotion (e.g., angry, sad, happy), or even in Crimson Hexagon‘s case, categories set up by the end-user business analyst, which may be much more useful than positive/negative assignments.

Metavana’s materials, devoid of detail, do not explain or justify ing the software’s apparent limitations. Given Metavana’s ability to extra topics from documents, why is the company content with cruder sentiment analysis, capable only of sentence- and phrase-level resolution and of extraction of only positive/negative/neutral sentiment? Should it boast about high-precision but crude sentiment analysis when rivals resolve sentiment at the more-granular (and higher-recall) feature level, and of polar sentiment scoring when others can handle arbitrary categorizations, such as emotion categories, that are more business-outcome aligned? But maybe all this detail doesn’t matter. Perhaps the answer is that, so often and in Metavana’s case –

Simplicity sells

You can check out the SatMetrix-Metavana SparkScore via a demo site. According to that site, “The SparkScore Sentiment Engine, powered by Metavana, combines the methodology of Net Promoter with customer sentiments from the social graph to deliver an integrated view of customer experience without the noise.” Metavana compares the SparkScore with the Klout score, which Metavana (very justifiable) derides as limited and simplistic, but which provides an attractively simple mechanism for scoring social influence.

In the end, Metavana’s messaging is similarly, attractively simple: Authority, science, and in SparkScore, a special insights-delivery capability and route to market. Will a reliance on idiosyncratic accuracy claims sell product? Blogger Vikki Chowney of eConsultancy is skeptical is skeptical of SparkScore, and so am I. I further see the apparent crudeness of the Metavana’s sentiment resolution as a competitive disadvantage that may discourage adoption of Metavana’s as-a-service and social-intelligence workbench offering, that is, unless the company fights a price war to wrest the quick-and-dirty market from rivals such as ViralHeat and the plethora of low-end social-intelligence dashboard vendors.

Will a haze of science and authority nonetheless create magic that convinces other skeptics? We’ll see.

I once provided counsel for a company that had little confidence in social media marketing. It begrudgingly decided it would hire a consultant, go through some strategic exercises and probably prove that social media was bullshit and it would be better off sticking with its traditional guns.

The brand’s insistence with my work, as is with most clients I’ve dealt with, was that we measure everything as infinitely as possible. Skeptics are like that. They want to know how many clicks, how many re-tweets, how many milliseconds the time-on-site number increases each month, even if none of those measures really matter. Social Media Explorer has always been more partial to sales and lead conversion or at least metrics that match the client’s business goals. We reported those as well and met the client’s expectations.

Picture of a common measuring tape in inches. It is divided into 1/32nd of an inch. (Photo credit: Wikipedia)

And after a few months and some short-term measurable impact through social media, they cut the cord. The metrics were good, but not eye-popping. And probably not as good as their last ad campaign. Skeptics don’t care that social media is not advertising and there needs to be time spent building a relationship with customers there. They want a 5.9 rating overnight or 150,000 impressions … or at least a promise that 150,000 issues of the magazine their quarter-page ad appeared in were distributed somehow.

Don’t get me wrong. I’m not upset the client decided social wasn’t for them. Best to explore and find out than to A) Not know or B) Explore, hate it and keep on banging your head against the wall. Some brands … or more specifically brand managers or CMOs … are just better suited to put social media off for the next guy or gal in charge. And maybe they’ll hear me yammering on about approaching social media marketing strategically and that you can measure if you plan to do so and give Social Media Explorer a shot.

But here’s the rub with companies like the one described above: A friend recently saw an advertisement for them … trailing the back of a bi-plane at a large, outdoor event.

And what, pray tell, infinite metrics did they get from that?

The digital space offers unprecedented access to lots of things. None is more compelling than the measures we can produce for clients. But those metrics are going to offer brands one thing they didn’t expect: honesty.

They’ve never gotten that before.

No matter how many of anything we report, no matter what percentage increase month-over-month, no matter what definitive conversion rate averages … nothing, and I mean nothing, will ever beat, “50,000 people may have seen your ad!”

Have You Registered For Explore Minneapolis?

Don’t miss two days of intensive learning with some of the leading thinkers and practitioners in the digital marketing and social media marketing space. Join SME’s Jason Falls and Nichole Kelly, The Now Revolution co-author Jay Baer, Edison Research’s Tom Webster, Ad Contrarian Bob Hoffman, Neil Patel of Kissmetrics and more at one of the leading digital and social media marketing events of 2012, August 16-17 in Minneapolis, Minn. DON’T WAIT TO REGISTER! Seats are filling fast! Reserve yours today!

Editor’s Note: The following is a guest post from Tristan Handy from Argyle Social, a service SME uses and a company that sponsors our events. But I think you’ll agree, disclosures aside, it’s awesomeness. 

You’ve spent all year writing and sharing great content. And from time to time, you check to see how your posts perform. But when you look at that report—whether it’s in bit.ly, Google Analytics, or your social media management tool of choice—how do you interpret it? Is 100 clicks good? Is 1,000 clicks good?

If you don’t frame your performance correctly, your boss won’t recognize your impact. If you want to demonstrate success, you need to go further than simply reporting on the raw traffic you drove.

Fortunately, demonstrating your success can be easy. Just follow these three steps:

1. Measure the right stuff
2. Know how you stand up to your peers
3. Choose outliers to tell a story

Read on as I explore each step in detail.

1. Change your metric—Measure clicks per follower

The core measure of performance in social media marketing is clicks. A count of clicks answers the all-important question, “How many people read the content I shared?” But the number of clicks is an insufficient measure when it comes to evaluating your performance, as it just doesn’t provide enough information on its own. Obviously, getting 100 clicks is pretty impressive if you have 200 followers. It’s less impressive if you have 100,000.

To give your metrics more weight, start measuring clicks per follower (CPF). Just take that click data you already have and divide by the number of fans or followers for that account. If your tweet had 200 clicks and you have 7,000 followers, your CPF is 200/7,000, or 2.86%.

Normalizing by audience size is important for two reasons. First, hopefully your audience will grow over time, and it’s important that your idea of “good performance” scales with your audience size. If your audience has tripled in the past year, a post that got 100 clicks last year is more impressive than a post that got 100 clicks today. Second, it’s important if you want to compare your performance to your peers.

2. Benchmark your performance against your peers

The best way to determine what performance is “good” is to compare yourself against your peers. But this is often easier said than done. Fortunately, I’ve done all the hard work for you.

It turns out that audience size is the single biggest and most reliable factor that affects engagement (CPF). This is reflected in a recent study published by my company, Argyle Social, and seconded by a recent report from EdgeRank Checker.

If you compare the average CPF performance of hundreds of business social media accounts of varying sizes, smaller accounts perform better than large ones. This relationship holds true for both Facebook and Twitter.

(source: Argyle Social)

When you stop and think about it, this makes intuitive sense. At Ma & Pa’s Corner Store, the owner likely has a personal relationship with much of their audience. At a global online retailer, this clearly isn’t true. Closer relationships translate into greater engagement.

Does this mean that you should try to keep your audience size as low as possible? Probably not. If your goal is to maximize engagement, then larger accounts still get more clicks per post, even though followers engage less frequently.

(source: Argyle Social)

So, if your peer group is based on how many followers or fans you currently have, what we need now is a set of performance benchmarks for your peer group. The table below is just that.

(source: Argyle Social)

Let’s run through a quick example so that this makes a little more sense.

You’re a regional B2C retailer with 12,000 Twitter followers. You tweet a link that gets 12 clicks. That’s a CPF of .1%, which puts that tweet right at the 50^th percentile in your peer group. Not great, but not too bad either. Then, you tweet another link that gets 180 clicks. That’s a CPF of 1.5%, which puts you in the 95^th percentile. Awesome!

Running the numbers on your annual performance works the same way. Put together a big spreadsheet with all your posts, all of their clicks, and get an average CPF for the year. (Or use your social media management tool of choice.) Then compare yourself against the benchmarks above.

How do you perform? If you’re at the top of your peer group, great! Go brag about it and demand a raise. If you’re not, that’s OK—use this process to inform your 2012 goals. Create an action plan that will have you ahead of your peer group by this time next year.

 3. Select outliers

Now that you’ve diligently measured your CPF and normed it against your peer group, it’s time to select outliers. Rank your posts from 2011 by CPF. What do the top ten have in common? The bottom ten? Pull these out as success stories to celebrate and epic fails to vilify.

This step is particularly important in your year-end review because it helps you tell stories. Numbers are important, but human brains are wired for storytelling. If you can support your industry-benchmarked numbers from step 2 with some really excellent success stories and lessons learned, you’ll make a much bigger impact.

Want some of our success stories? Here are three of the best performing posts in the data set used for this research:

• Who’s been more influential in 2011, @PiersMorgan, @TinieTempah or @CharlieSheen? Vote for the Top 49 Men UK Edition now %link%
• Everyone #likes to WIN – Win A 25th Anniversary Nintendo Wii Bundle! #Wii #Retweet #Vancouver  %link%
• Facebook Achieves Majority – NEW Research from Edison and Arbitron: %link%

As you can see, looking at individual outliers is quite powerful. These three posts immediately beg the question, “Should I be posting more surveys, contests, and research?” Use anecdotal evidence like this to frame the larger conversation and to ask questions that should be looked into further.

Dominating your performance review

There you have it—three easy steps for success when it comes time to summarize your performance.

Do you use this process already or have a favorite of your own? Let us know in the comments.

Methodology

The data behind this analysis comes from a sample of customers’ activity on Argyle Social, a social media marketing software provider (and my employer!).

The selected sample included more than 150,000 posts from more than 1,000 Twitter, Facebook, and LinkedIn accounts between November 2010 and December 2011. Our customers are professional marketers representing a range of company sizes across all major industries.

Have You Registered For Explore Dallas-Fort Worth?

Don’t miss a day of intensive learning with some of the leading thinkers and practitioners in the digital marketing and social media marketing space. AT&T’s Chris Baccus, Radio Shack’s Adrian Parker, Copyblogger’s Brian Clark, Edison Research’s Tom Webster, Edelman Digital’s Zena Weist and more headline one of the leading digital and social media marketing events of 2012, February 17 in Dallas, Texas! DON’T WAIT TO REGISTER! The first 100 to do so get an incredible discount! Reserve your seat today!

Tristan Handy is Director of Operations for Argyle Social, a social media marketing dashboard that helps businesses create real returns from the social channel. Follow him on Twitter @jthandy.