Last week at Emetrics I was speaking with a company doing vertical search. We discussed metrics like number of “next page” clicks, time to first click, and lots more, in order to measure the user experience. Metrics often take the place of real data, e.g. for inferring things like relevance of the search results.
So, thought experiment: Visitor A does a search, clicks on 15 links. Visitor B does a search, doesn’t click on any links. Visitor C does a search, clicks on 2 links. Average clicks per search is 5.66. Which visitor had the best experience?
The answer is that it depends on what the search term was.
- Visitor A typed santa barbara photos. They want to see lots of ’em. The fact that they looked at 15 is a good sign.
- Visitor B typed time in santa barbara. Poof, the time is on the results page. No need to click further.
- Visitor C typed santa barbara tourism, and was perhaps confused by both http://www.santabarbaraca.com/ and http://www.santabarbaraca.gov/. Still, the first two results seem to indicate a success.
So, it would be misleading to compute an average, and compare each result against the average. If you categorized your searches into terms like
general, you could group your searches and compute your metrics by category. You’d want your information searches to have low clicks, and browsing searches to have high clicks.