At Survata, we’re committed to spreading research best practices to help with survey design, avoiding inadvertently biased surveys, and ensuring all our clients are getting optimal results from their studies. From this experience, we’ve put together some resources to help our clients avoid common issues, both when working with us and as they continue to conduct research in the future.
Before getting into survey design and analysis, there is often prep-work needed to fully take advantage of the insights a research study can offer. A few tools and methods for this are described here.
When considering the best methodology for an upcoming research project, one of the first big steps is deciding whether to do qualitative or quantitative research.
Typing Tools are spreadsheet files that allocate respondents to a particular segment based on their answers to the Segmentation questions.
Segmentations are a popular market research solution for companies seeking to better understand their consumer audience by isolating how needs and behaviors differ among the consumers who buy their product.
After identifying the problem to be addressed, the next step is to design a survey that targets the correct respondents, asks appropriate questions, and minimizes biases.
Mutually Exclusive, Collectively Exhaustive (MECE) refers to the concept of organizing information such that it is separated into buckets that are both mutually exclusive.
Survey Logic controls how questions within a survey relate to one another, allowing a respondent’s answer to a question to affect other questions they will or will not be shown.
Most research requires responses from certain groups, rather than an entirely random sampling of the population. In this situation, studies use quotas to cap certain categories of respondents.
Used to ensure that the distribution of respondents accurately reflects the population of the country.
Randomization is used to remove or reduce serial position biases that can result in over or underrepresentation of certain answers based on their position within a list.
A Likert scale is a widely-used rating system for survey research, consisting of a symmetrical scale of positive and negative responses.
Monadic Testing is an experimental design that allows researchers to create a single testing instrument that systematically assesses multiple different concepts and controls for the potential for bias.
Semantic Differential Scales are used to quantitatively measure attitudes and perceptions regarding people, brands, activities, etc. A typical semantic differential scale is a 7-point scale, with a neutral point in the middle.
Even though the large step of survey design is complete, there are a few challenges that present themselves in the fielding stage as well.
Acquiescence bias is the type of response error that stems from the tendency of respondents to agree with research questions or statements.
Incidence rate refers to the frequency of something occurring within the population.
Nonresponse bias is the type of survey error that stems from respondents not taking the survey. Factors including lifestyle, personal values, demographics, etc. can be noticeably different among survey respondents and all other potential respondents who chose not take the survey, and these differences bias the survey data.
Social Desirability Bias is a type of response bias that occurs when respondents feel pressured, either internally or externally, to provide a socially desired response. Thus, the data from these respondents is biased and does not accurately reflect the target population.
Response Error occurs when respondents do not provide accurate answers to survey questions. It is impossible to entirely eliminate this type of error, but careful consideration during the survey design process can help mitigate its effects.
The specific components in an analysis toolkit will depend on the project, but there are a few commonly used tools and methods to keep in mind.
A crosstab is a frequency table that shows data for more than one variable.
Cluster analysis is a data reduction method that focuses on respondents as opposed to variables. It reduces a large pool of respondents into groups (or clusters) based on common themes. The members of each cluster group are more similar within a group than they are between groups.
Conjoint Analysis is a type of analysis based on a Conjoint module in a quantitative survey. The Conjoint module asks respondents assess their interest in a wide range of variations on a particular product or service offering.
In situations where many variables are at play, factor analysis identifies the key underlying factors or constructs within a set of data. A factor analysis may start with twenty known (or manifest) variables from a questionnaire and reduce them to, for instance, five constructs, or latent factors.
Kano analysis measures customer perceptions of product or service features, and classifies them based on satisfaction with the product according to the presence or absence of a feature.
MaxDiff is a form of analysis where survey respondents are shown a subset of possible items, often three to five at a time, and asked to indicate the two elements of the subset shown that are the most and least important to them. This forces respondents to make choices between options, in contrast to standard rating scales.
Multiple regression is an extension of simple regression that is used to predict the value of an unknown variable based on the value of two or more known variables.
Regression analysis is a technique used to understand the relationship between two variables and can reveal how closely two factors are related, as well as how certain one can be about the resulting predictions.
Weighting is a data processing technique that allows researchers to change the value of a particular respondent’s answers relative to the rest of the responses in a dataset. This allows researchers to correct for sampling imbalances by adjusting the data to be more representative of their desired target population.
Some additional tips to best design your survey for optimal performance.
Collection of institutional terms used in the market research field.
Calculate the margin of error in your error using our calculator.
Survata uses the latest web technologies to offer the best possible user experience. Unfortunately, your browser does not support the required technologies. To use Survata, please upgrade to one of the following browsers.