Startups Choose Doing Surveys Over Customer Interviews At Their Own Peril

Alternative title:  Surveys make entrepreneurs lazy.


Entrepreneurs tend to be skeptical about the notion of doing qualitative research as means to validate their business hypotheses and establish if there’s a real business opportunity.  This attitude comes from the notion that, in order for information to be reliable, it needs to come in the form of tables and charts derived from quantitative research.  “In ‘hard data’ we trust!” seems to be the mantra.  This is particularly true for those working in the consumer space as tabulated data provides a condensed view of thousands of people at once, which is understandably, music to their ears.  And what’s the best way to obtain the numbers that feed into those lovely pie charts?  You guessed it: surveys.


The allure of surveys

Surveys make an appealing case: they allow us to gather data from thousands of people with relatively little effort.   In the case of online surveys the temptation is irresistible as with just a few clicks we can setup a questionnaire, distribute it, tabulate the incoming data and visualize the results.  What’s not to like?!  Well, surveys are not the right tool for the type of job that needs to be done at early stage startups.


We don’t know what we don’t know

As obvious as it may sound this riddle-esque idea has an implication so subtle it escapes many: we can only inquire directly about known unknowns.


Think about it this way: we can only ask questions about the things we know we don´t know.  When it comes to survey questionnaires, the range of themes addressed in those yes/no and multiple-choice questions (and their answer choices) are limited by our current knowledge of what the range of possible themes is.  We are, by definition, leaving out key questions that could potentially unearth breakthrough insights simply because we are not aware of them (as if they were out of sight).

But wait! That’s why we have open-ended questions in surveys, so people can speak their minds, no?

In theory: yes.  In practice: rarely.

Who hasn’t designed a survey questionnaire that included questions like “Why?” to cast a wide net and added “Other.  Please Explain.” at the end of a set of answer choices (of a multiple-choice question) to cover their back?  But after going through the rather bland and generic responses from those few kind people who after rolling their eyes made the effort of providing an answer, it becomes rather obvious that we were being naïve.  By asking these tell-me-everything-I-don’t-know-about-you questions we assume that people are willing (or capable) of analyzing themselves on a given situation, and then articulate conclusion in a tidy sentence.  And all this on a split second.  There’s no way than will happen!  It’s worth saying that using open-ended questions in survey questionnaires is not the issue, but rather how we use them.   Open-ended questions yield better results when they focus on having people describing past events (e.g. “What happened that time when…?”) or behavior (“How do you…?”).  But even the responses from questions such as these should be treated with a pinch of salt, especially those inquiring about people’s problems (but that’s the subject of another post).


Bumping into unknown unknowns

This is precisely where surveys fail and qualitative research has the upper hand.  Bumping into unknown unknowns is what early stage startups should learn to do in a systematic manner.  And there’s no better way to break free from the cage that represents our current knowledge of things (i.e. both known-knowns and known-unknowns) than by talking to people in the form of customer development interviews.  It’s through (structured) conversations that we give ourselves the chance to be surprised (and many times inspired) by something someone says.  It’s the ‘randomness’ of people’s stories that allows those I-would-have-never-thought-of-that-in-a-million-years! moments to happen, opening in front of our eyes a window of possibility which we didn’t know was there.


The right time for doing surveys

Simply put, surveys are most effective when we want to define the relative importance of things and to establish priorities.  This means that we make the most of surveys only when we know (based on the learning from customer development interviews) which are the right questions to ask, and ideally, what are the most likely answers.  For example, through surveys we can refine the demographic and psychographic profile of our target segments (from interviews we learned there was more than one), identify how widespread is a behavior (which unexpectedly emerged in the stories of a few people) and establish who are the main competitors (including those makeshift solutions people showed us).


The many excuses for not doing interviews

At the top of the list there is laziness.  Plain and simple.  It’s undeniable that preparing and conducting customer development interviews require much more legwork than running online surveys from a laptop, and sadly, this is one of the main reasons why entrepreneurs  (even if they fail to admit it) prefer to look the other way.  Others who consider themselves domain experts (usually due to their previous work experience) say that talking to people is something they’ve already done in the past or they believe there’s nothing new they could learn thus they would rather spend the time doing something else.  A few others are simply unaware of how they could ‘do research’ besides the techniques they see being used by big established companies.  But the truth is that no matter the excuse, failing to spend time face to face with prospects represents a risk very few startups can afford i.e. missing out on bumping into the nugget of information that  will change their fate.

Startups Choose Doing Surveys Over Customer Interviews At Their Own Peril

7 Responses

  1. The essential is that unlike past treatments, the tinnitus is not being masked and not being hidden. When the mucus is not drained, it
    accumulates and brings pressure to your middle ear, preventing you from hearing properly and sometimes producing sounds that aren’t really there.
    Patients should implement for it and it truly is proposed that they already be below the consideration of
    a physician, however the grant is for $1500. November 15, 2014 at 11:15 am #
  2. Soundscapes played out by means of an MP3 or CD player.
    When the mucus is not drained, it accumulates and brings pressure to your middle
    ear, preventing you from hearing properly
    and sometimes producing sounds that aren’t really there.
    Tinnitus is an auditory hallucination, a noise abnormality that originates in the ear. November 16, 2014 at 3:17 am #
  3. I used to be able to find good advioce from your content.

    recommended reading November 18, 2014 at 4:05 am #
  4. An impressive share! I have just forwarded this onto a coworker who had been doing a little research on this.
    And he in fact bought me lunch due to the fact that I stumbled upon it for
    him… lol. So let me reword this…. Thanks for the meal!!
    But yeah, thanx for spending some time to discuss
    this issue here on your web page.

    identity theft protection services December 7, 2014 at 11:17 pm #
  5. She Deaг One Who Follօwed Me – It was she, blesѕ her,who followed me.

    However, the foundation, in 1963, of tɦe Bedford Park
    Society led to the listing of 356 houses by the government, and so,
    much of the estate becoming part of the Вedford Park Conservation Area.

    As a good example of the observer, let’s look at a moose walking through a forest,
    just going along its merry way.

    world abdominal percussion sounds December 28, 2014 at 10:33 am #
  6. It’s great that you are getting thoughts from this post as well as from our argument made at this place. January 2, 2015 at 7:38 pm #
  7. And once you learn about the law of attraction,
    you will begin to see why this is so. The real problem is that the whole reason we have things like mp3 players and headphones is because we want to take
    our music wherever we want, and outdoors is where all the noise is.
    We need that time to maintain some personal freedom and to allow our minds to take
    a breather, unwind, and just relax. June 10, 2015 at 6:55 am #

Leave a Reply