On surveys

Got mail from Stowe Boyd today, with whom I had the pleasure of working back in the early 80s. Today brings the news that he’s leaving the LinkedIn social network. No reason was given, but he did point to a survey he was conducting. He posted about his decision at Corante where he indicates he’s extracting himself from some social networking applications, although he doesn’t list which.

This post isn’t about Stowe’s decision; it’s about his survey. He admits it’s not scientific, but wow, there’s unscientific, and there’s biased. There were questions like

The social networking applications I have used are lacking
critical features and are therefore have not become essential to me.

(You can see the poll results including the questions and answers.)

Stowe’s no dummy; he knows what he’s doing. He also admits he has a bias. I just found it odd that he’d rig a survey to support the bias.

I took the survey, because it’s Stowe. But as a rule, I don’t take surveys. The reasoning is simple – most people don’t know how to correctly build a survey, or how to analyze it. They think that a few questions with an assortment of answers, and an ability to tabulate the answers, is sufficient. It’s not. It requires:

  • building single-concept questions, without bias
  • providing a specific set of answers that also do not indicate a bias (e.g. three different “yes” answers and one “no”)
  • a proper sampling methodology so you know that the survey is representative of the population you really want to understand
  • and finally actual analysis of the results – rather than simply regurgitating the tabulation – to extract meaning from the survey.

Here’s a survey from me to you:
Q: Don’t you hate surveys, since they cause you high blood pressure?

  • A1: Yes!
  • A2: Surveys just tell the pollsters what they want to hear
  • A3: My doctor would like me to be more active
  • A4: I think there should be less surveys and more leadership!
  • A5: I hadn’t realized my high blood pressure was due to surveys – thanks!
On surveys