Newseum: Do You Trust Blogs?I had an idea to write some common guidances for the online surveying process a long time ago but whenever I was putting down some notes I found myself with a list of very intuitive common-sense advice, and playing the role of Captain obvious is never the goal of my posts.

However last night I stumbled upon the survey offered on behalf of a massive international enterprise on their website and funny enough, within the first 10 minutes of filling in the survey, I spotted 4 issues I had thought were easy to avoid for everybody. If one of the most recognised brands in the world, with an enormous budget and army size team of marketers, have succeeded at combining almost all possible mistakes in one survey then perhaps the pitfalls are not so obvious any more.

4 tips on how to waste a budget with an online survey

1. Lie to your visitors

9 out of 10 surveys state the amount of time it takes for a user to complete; fair point as you always should state how much effort you require from your visitor so they can compare the benefits of completing the survey with amount of friction requiring.

The survey I mentioned clearly stated it would take 5–15 minutes of your time (unfortunately I forgot to take screenshot and now can’t access it) when in fact it took me at least 25 and, if I were to read every question thoroughly and spend some time on thinking, this number could easily double.

Is it so hard to ask your friend/colleague/cleaning lady to complete the survey in front of you so you can track the actual time required?  What do you think a person who was promised to waste 5 mins of his time would do after finding himself spending 30 min and still being far away from the end?

The same rule applies to showing the ‘% complete’ figure; why does answering 3 simple demographic questions show that I’ve completed 23% so far, but then subsequent multiple-choice and really time-consuming questions add only 3%? Why not just state the number of questions left? It’s clear and easy.

(click on the image to enlarge)

These issues don’t just result in very high-abandonment rates or plenty of unqualified participants (quite often your users would just randomly click on any answer in order to complete as fast as possible), they also seriously damage the brand of the organisation this survey was created for. Why should I believe that an organisation’s service or product delivers when they just shamelessly lied to me in order to get the info they need?

2. Ask questions you already have answers for

At least half of the questions asked in the survey could be answered just by spending 2 minutes in whatever web analytics solution you use.

Some version of this can be spotted in almost every online survey – usually the reason behind this mistake is broken communication between a company’s departments, or commonly they’ve employed traditional old-school market researchers to do the survey online. I am not saying traditional marketing research is ineffective; all I want to say is that old-school marketers rarely have any idea on what is currently possible within the internet marketing industry.

(click on the image to enlarge)

And you don’t need expensive corporate web analytics solutions (usually used by huge companies as in our example, often starting from 10k per annum) as your simple Google Analytics would answer the questions without any segmentation or customisation.

(click on the image to enlarge)

In this case integrating a research team with the analytics team would not just answer half the questions, simplifying the process by a mile, but would produce credible and reliable insights in place of doubtful users’ opinions.

3. Ask useless questions

This point combines both way-too-general questions like:

(click on the image to enlarge)

Rate exactly what? User experience? Colour choice? Web copy style?

Always make sure to specify exact information you are looking for. More specific questions will eliminate “Don’t know” variance and mostly importantly will actually highlight the problems rather than give a really opinionated mark.

And questions not providing any insights:

(click on the image to enlarge)

First of all, please have a look at the questions “This website inspired me to do something new” and “The website reflects British Culture”. What kind of British culture or inspiration are we talking about? The site is a web-player-style tool where you can catchup with TV and radio shows you missed. It’s just a tool not a guru.

What about the slightly differently-worded questions like “The website is up to date” and “The website is fresh and new”? For sure it is. It’s a website providing opportunity to catch up with recent shows you missed. Being up to date is the primary function, otherwise it would be completely useless.

Every single question should be built up as a micro-hypothesis. In simple terms you should have a clear idea of the precise actions you are going to take in case any answer shows statistically significant popularity. If you are not able to do that then forget about the question. The primary purpose of your questionnaire is improving user experience based on the findings, not a 70 page dust-collecting report on a how a web player represents British culture.

4. Make the process look complicated

What would you think if you saw something like this?

(click on the image to enlarge)

Any user would quietly swear after seeing something similar. Is it so hard to divide it into 3 parts? Visually heavy elements will always look more complicated than they actually are, so why multiply already massive friction? Adding 1 or 2 more steps would not make any severe difference to the already long and time-consuming process but could ease the user experience by a mile.

Make sure to make every step of your multi-stage questionnaire easy to digest and scan. It’s not  the number of steps responsible for friction, it’s the amount of time and effort it takes to complete.

Follow up

If you are willing to learn more about the benefits of using online surveys (I can be blunt) as the part of Conversion Rate Optimisation join me this Friday at New Media Breakfast or subscribe for our webinar.

Tags: , , ,