Some thoughts on Internet Surveys
Internet surveys seem to go though a cycle of popularity. First everyone wants to have them, then every one thinks they are a budgetary waste. I thought I would take a little time to explain what they are good for, and what problems can arise from them.
A brief definition
Lets first define what I mean when I say “Internet Survey”. Internet surveys are usually emails or web pages that are sent to a vast number of people to get their opinion. Two common examples are surveys that are sent via email to a member list, and surveys that appear between page navigation on websites. Usually the questions asked are very short and relate to a product or service. If you have used Netflix in the past you have almost certainly received one of their “How was the quality?” surveys.
The largest problem with Internet surveys is that they do not provide a good bases for making business decisions when dealing with smaller issues. With larger issues their is almost always a better indicator of a problem. This comes from a term called “selection bias”. In the case of Internet surveys it stems from the fact that the people most likely to respond are those with a strong opinion on the topic (they either really like your product, or they hate it), or those that are “board” and are filling in the survey because it’s something to do. A direct quote from Mind on Statistics (a text book) states it best; “Surveys that simply use those who respond voluntarily are sure to be biased in favor of those with strong opinions or with time on their hands.”
So when you send an Internet survey you are usually getting only the people that have polarized opinions about the subject of the survey. This is even more true with email surveys as they are very easy to ignore (just hit the delete button), and people generally check their email more actively when they are about to do something.
What not to use Internet surveys for
So given the information above, there are some pretty big things Internet surveys should not be used for. First do not use them as a basis for a business decision. If your having a product problem, then the only people likely to respond are people that hate your product. Conversely, if your product is doing well, your likely to get only results that enforce that. In both cases you miss the “middle of the road” people that your trying to entice to your product.
You should also avoid using them as a feature planing tool. Customer feedback is important for designing new features, but again, your not going to get very accurate results. Your new feature, recommended by 90% of the survey responders, may only impress 5% of your actual user base, while angering 30% (that never bothered to respond to the survey). It is much better to use split testing, direct feedback, or other methods to gauge customer response to a proposed new feature.
What Internet Surveys are good for
That doesn’t mean Internet surveys have no use. They excel at providing your customers a place to lodge their complaints. You can look at the survey data as a way to gauge your least satisfied customer’s reasons for being dissatisfied. Giving your customers a place to lodge their complaints is important, and well help you provide better customer service. You much more likely to not loose a customer, if when their having a problem you ask them “what could we do better?” as opposed to just ignoring them, or quietly fixing their problems with no interaction.
Internet surveys are also great for “marketing”. They bring people to your site and ask them questions about your product. By the time they are done with the survey they have a vested interest in your product and are more likely to give it a try, or continue using it.
They can also be used to measure the “polarization” of your user base. If you send 1,000 users a survey and 500 respond. Then 50% of your user base (roughly, don’t forget that board people respond as well), have stronger feelings about your product. This means that your product is one that people either love or hate. That can be a good thing, but can also cause problems.
Making Internet surveys more effective
To enable you to get the best results from an Internet survey, there are a few things you should do. Start by keeping the survey short. Ask simple, pointed questions. People are much more likely to answer a 2 question survey then a 10 question survey. If the questions are simple enough you may even overcome some of the “selection bias”. Think about a survey that only asks one question. “Do you like my product?” Then only give two possible answers “Yes” or “No”. You will get far more responses, then if you ask more questions or have more possible answers.
Remember that most people that are answering already have strong feelings about your product. Try to ask questions that gauge that strong feeling. Do they like it, or hate it. Do not try to fully explore the 500 reasons someone may not like your product.
Narrow your questions to specific topics. Keep them focused on one thing. If you can only ask 2 questions, what two questions would you ask? Some good examples are “Do you like this new feature?” or “Would new colors make the image easier to see?”.
Finally keep a list of people that respond and use that to narrow down your future surveys. If you send 100 surveys and 50 people respond, then next time only send to those 50 people. If 10 of those 50 people “hate” your product. Then send just those 10 the “Why do you hate it survey”. The same is try for the 20 people that “like” your product. The 30 people that “are in the middle” should get a third set of surveys aimed at figuring out how to get them into the “like” category.
Internet surveys can be useful, if you remember the selection bias, and focus them correctly. When used as customer feedback path, they can be very powerful. Just remember the “powerful” does not always mean “good”, you have to decide how to use the information in the best way to effect your product.
Coteyr.net Programming LLC. is about one thing. Getting your project done the way you like it. Using Agile development and management techniques, we are able to get even the most complex projects done in a short time frame and on a modest budget.
Feel free to contact me via any of the methods below. My normal hours are 10am to 10pm Eastern Standard Time. In case of emergency I am available 24/7.
Phone: (813) 421-4338