Before the emergence of the Internet, there were two major conventional ways of doing intellectual property consumer surveys — mall intercept surveys and telephone surveys.   Mall intercepts work best for branded, consumer products where there is a visual element to be tested. They are moderately expensive and require some incentive. Telephone interviews are good for brand names, genericness studies or other types of research where the respondent does not need to view a visual. Most telephone research requires no incentives.

The Internet, in theory, combines the best of both worlds. Internet surveys not only permit the asking of verbal questions and recording verbatim answers, they also permit transmission of visual images such as products, labels, logos and packaging. Internet technology also permits sound transmission. Transmission costs are minimal with an e-mail blast of 5,000 names costing about $800 or $160 per thousand. (Typical mall costs are $30-$40 per interview). Unfortunately, there is no telephone book for e-mail addresses, and in order to use this medium you have to hook into a vendor that has large opt-in consumer panel data bases. By using opt-in panels, you will bypass all the SPAM filers and anti-SPAM on-line watchdogs. Moreover, you have an instant, real-time tabulation process.

One methodology I have recently used combines Internet with telephone. The Web is used to recruit, but the actual interview is administered over the phone. Visuals are used in that the respondent is required to access a Website and questions are based on what the respondent views on the site. For this type of research an incentive is needed. The real-time tabulation benefit is part of the Internet/Telephone hybrid method.

Another major plus with using on-line surveys is that, despite low response rates, research shows “conducting surveys on the web has proven to be just as effective as other methods, but if offers quicker turnaround, less biased responses, more representative sample and lower cost,” according to an April, 2005, Market Tool White Paper issued by San Francisco-based MarketTools, entitled “Why Online? New Benefits and Possibilities.”  The MarketTools White Paper goes on to report, “A number of studies have shown that data collected via the Web very closely matches data collected through other media.  A report in a study conducted for a MarketTools client in 2004 indicates: ”We compared the results of 90 concept tests. The results…indicate a very strong correlation between on-line and off-line scores.”

However, Internet surveys have their problems. For one thing, response rates are abysmal. This clearly is reflective of the high noise level in e-mail in-boxes. InfoUSA, a leading consumer panel name provider, reports the average consumer Internet survey produces a click-through rate of 0.5 to 1 percent and your average business-to-business survey produces a click-through rate of 0.5 to 2%. Also incentives are required. (Click through is the percentage of people who get the survey, open it and complete it.) Getting a sufficient number of responses is a pure “numbers game” based on your response rate. If your response rate is 2 percent and you need 200 completed interviews, you will have to send out 10,000 e-mails to obtain a valid number of responses. If your click-through rate is 1 percent it will require 20,000 e-mails.

Bounce-back rates are another problem. Consumers are fickle. They move and change e-mail addresses. Typical bounce rates for consumer e-mail surveys, according to InfoUSA, is 10 to 35 percent and typical bounce-back rates for business-to-business e-mail surveys are 2-8 percent. 

A far bigger problem is the respondents versus non-respondents dilemma. If 2 percent of a universe responds, what about the other 98 percent? To help rectify this problem, some kind of back-up or verification study might be needed, like a small mall intercept survey. Telephone would work if there isn’t a visual element to the survey. 

Any researcher will admit there is no such thing as THE perfect survey — or, for that matter, a TOTALLY useless survey. Every survey method has its pluses and minuses. The key is to understand the limitations of the survey method before embarking on the research effort.

By James T. Berger