A Hunt for Reality
Key concepts behind polling and survey research

Overview
At CommunityPoll, we have come to see and appreciate that research is all about a "hunt for reality," reality about what employees, customers, members, or constituents truly want, think, and believe. Often, the louder voices in a community create an exaggerated perception about what the whole of that community believes. Surveys can provide quantitative and scientifically projectable data that describe the true opinions in a community. The more that is known about the reality of "wants" of a constituency, the better are the decisions that can made.

Beginning: The qualitative stage
Management's initial questions to itself: Why conduct a poll? What are the objectives? How will the data support specific decision-making, and respective actions, in a meaningful way? Is the poll being conducted to prove a reality which management already believes exists, or is true objectivity desired?

An inclusive process supports ownership of, and higher quality, findings
Include key stakeholders at the qualitative stage so that the process benefits from true expertise and genuine acceptance of findings. This inclusiveness results in a form of "listening" that identifies the core issues of most importance. The process can sometimes result in a re-evaluation of what really are the objectives for conducting this poll, which in turn will increase the poll's final value.

This initial preparation can result in a listing of possible answer choices for each question, thereby producing a multiple choice poll and statistically meaningful quantitative data, versus many open-ended questions which must be re-interpreted at the end of the survey (and be subject to the interpretation of the reviewer). This initial preparation to consider the possible answer choices will generally identify 80-99% of the likely responses from respondents while also providing substantially increased confidence in the final numeric findings.

Inviting in chaos to maximize creativity
The qualitative and brainstorming stage has an order to it; it just looks like chaos. The important topics, questions, and the possible answers are collected, but they are not yet organized into neat and clean survey questions. Trust that a refined survey instrument will indeed be developed and refined.

"Actionable" versus "nice to know" and the effects on survey length
In almost all situations choose to write, or edit, questions so that the answers to each question can lead to action in support of the initially developed objectives. Calls for action will be present when the differences between the answer options are statistically significant (substantially greater than the margin of error for the survey as a whole, or among selected cells of the survey). There is may also be information that you may might want to know about your respondents, but which won't lead to an action. Using the attention of your poll respondents to ask these "nice to know" questions is just not productive; it can reduce the number of completed surveys, and can make respondents less likely to take your poll the next time. An exception to the rule is when you are developing benchmarks for future comparisons of increases/decreases in behaviors or opinions, which will then lead to action.

Precise survey instrument language for each question
All the planning and prep to reach the stage of writing the survey instrument can be lost if the questions are written ambiguously, or if the question language is not understood by the respondents in the targeted population. This is the kind of thing that can sneak up on you unexpectedly and bite. Consider, for example, the phrase "a challenging job." Is it challenging in that it is exciting and invigorating, or is the job challenging in that it is too difficult or over the respondent's head? How would you interpret the actual survey results should they overwhelming indicate that the employees "definitely agreed" that their jobs were challenging? A similar situation can occur when two questions are asked in the same question: "Please rate the quality of our food and service." Does "excellent" apply to the food or the service?

Non-response bias
This bias occurs because not every one that is contacted at random will actually speak to the phone interviewer. This produces survey results based only on those people who are willing to give their time to answer the questions. The same is true with Internet-based surveys. This bias effects the extent to reach one can reasonably project the findings to the full universe of the defined population (such as the unique characteristics of a population of a particular city). A random and truly balanced survey population sample can go a long way to reducing response bias.

Balanced and random population samples
Balanced and random population samples Usually the organization conducting the poll will know the percentages of the key demographic variables of its "marketplace"; for example, the percentages of ethnicity categories or household income categories. If this information is not yet known, it can be acquired by the CommunityPoll team. To the extent practical, endeavor to have the population of your survey sample match these key demographic variables which already describe the population you want to survey.

2001 CommunityPoll.com, Inc.



Tell us what you have in mind and we'll respond quickly with a firm quote.

Home   Contact Us   Project Quote   How We Work   Price List & Ordering   Examples of Clients   Summary of System Features
Easy Self Tour   Advanced Tour   Key Concepts in Polling    Friendly Service    Excellent Value    Accuracy



We look forward to working with you.