The Business Owner’s Guide to Survey Data Analysis

In this guest post,  Heidi Thiel from siegemedia.com outlines the main aspects of a successful survey from writing the questions to their analysis.

Customer surveys are one of the most valuable tools business owners can use. Since survey data is straight from the source, business owners can get unbiased and accurate insights about customers, things that need work, and ways to improve. Though this method of data collection is helpful, if the survey data analysis is done improperly or inaccurately, the information gathered is useless. Designing and analyzing surveys is not a difficult process but there is a specific science to it. Chattermill has created an in-depth guide to teach business owners to consider variables, best practices, and analyzing techniques to ensure every survey results in usable data.

Write the Questions

The very first thing business owners should think about is writing questions that will result in success. Here are a few things that should be taken into consideration:

Finite Metrics: When writing the questions, survey creators should stick to finite metrics for measuring survey responses especially when other variables change. For example, if a survey is designed to find out what women think about sustainability in different cities, the survey could be geared towards women of similar age.

Appropriate Descriptors: When looking at different demographics, some questions may need to be re-written to cater to the target demographic. As the descriptors change, make sure that the answers are still measurable by the finite metrics.

Quantifiable Scales: In customer surveys, yes/no and true/false questions should be avoided if possible. Questions that are formatted with 1–10 answers or strongly agree to strongly disagree options provide much more insight. 

Multiple-Choice vs. Open-ended questions: Both multiple-choice and open-ended questions have their place in customer surveys. Open-ended questions can give business owners a more in-depth understanding of customer opinions in fewer questions. The issue with open-ended surveys is that it is more difficult to code the answers if businesses are looking for more of a high-level look at customer approval. For those surveys, multiple-choice questions may be a better option.

Ask the Questions

After the questions are designed for success, survey conductors should carefully consider the placement of these questionnaires. Here are specific points in the customer buying process that are best for surveys:

Milestones in the Customer Journey: There are many key milestones in the customer journey that are natural placements for surveys. Many of these milestones are often positive and usually lead to constructive survey responses. Here are a few milestones where surveys can be placed: 

• Initial sign-up 
• Membership renewal
• Product conversions 

The Bumps in the Journey: Like the positive moments in the customer journey, the negative milestones are also beneficial places to give a customer feedback survey. During these times customers are likely to be brutally honest and can give insight to any pain points that exist.

• Canceling a subscription
• Unsubscribing to emails
• Absence in purchasing 

Customer Service Experience: One of the best places to present a questionnaire is during customer service experiences. Since the audience is already engaging with the service, they can provide insight into shortcomings in products and services. These surveys can appear on:

• Live chat services 
• Customer service calls
• Email correspondence 
• Social media

Refusal to Convert: Many customers will spend time on company sites and choose not to convert. Surveys sent at this point can help remind the customer of your services and give you insight into how to increase conversions moving forward. Some of these points are: 

• Left shopping cart 
• Refusal to sign up for service
• Opting for the free product versus paid services

Analyze the Data

Once business owners have obtained data that is accurate and usable, the next step is analyzing it and making it valuable. These steps will ensure that the maximum value is collected from the data.

1. Determine the Hypothesis

Before survey conductors release the questions, the basic goal of the survey should be determined. A great way to analyze whether or not those questions were answered is by creating a set of research questions and a list of the corresponding “expected” answers to those queries. Once the responses come in, the original list can be used to compare to evaluate how effective the survey was.

2. Identify Any Variables 

If the basic answers are met, surveys can then be analyzed further by drilling down into the specific demographics that answered the questions. This can be helpful for customers who are experiencing negative feedback. The exact demographic who is unhappy can be identified and the company can brainstorm ways to solve their problems. For example, if a business notices that people are unsubscribing from their email at a high rate, the survey could reveal that users 65+ were unable to read the text and were unsubscribing because of the user experience. With that information, emails can be crafted differently to serve an older audience.

3. Notice the Patterns 

Once the various demographic responses have been analyzed, patterns will begin to emerge. These patterns may be vague as positive or negative, or they may highlight specific pain points in the customer journey. Once these themes start to show, survey analysts can identify where more research is needed. Here are a few additional questions survey analysts can use as jumping-off points for finding patterns:

• Are the responses negative or positive?
• How do different demographics compare?
• Did the responses line up with the initial hypothesis?
• Are the negative responses about a specific product or service?

4. Test the Data

One of the biggest mistakes that companies make when conducting surveys is ignoring statistical significance. This happens when the sample size is too small to reflect the general population and assumptions are made based on the answers of a few people. For example, if an apartment complex takes a tenant survey but only one person on the top floor answers, it would be unfair to assume that everyone on the top floor shared the same opinion. 

There are many calculations and associated terms that are helpful when analyzing data. Here are a few jumping-off points to help business owners get started:

Variance: This number illustrates the degree of difference between all of the data points. To find how much they vary:

1. Take the average of all data 
2. Subtract that number from each data point
3. Take each of those numbers and square them
4. Find the mean of all those squared differences to find the variance

Standard Deviation: The standard deviation points out how far the data points are from the mean. To find this number, take the square root of the variance that you found in the step above.

Z-Score: The Z-score shows how much standard deviation a data point has from the average. To calculate the Z-score:

1. Take the data point 
2. Subtract the mean
3. Divide by the standard deviation

Confidence Level: The confidence level is the probability that your results could be repeated if the survey ran again. A good confidence level for a survey is at least 95 percent. To avoid complicated calculations, survey analysts can research the targeted confidence level and what Z-score corresponds to that number.

Margin of Error: Margin of error is important to note when analyzing data because it gives an accurate estimate of customer opinion. To find this number:

1. Divide the standard deviation by the square root of the sample size
2. Multiply by the desired Z-score

Statistical Significance: Finally, statistical significance is the probability that the results of the survey were affected by something other than chance. Calculating statistical significance can be a complicated process of setting different hypotheses and running multiple calculations. For business owners who don’t have the experience, calculators can help provide quick answers.

5. Use the Results

Once the survey is written and distributed and data is gathered and analyzed, the next step is using the results to improve the business. Survey analysts can return to the original hypothesis and use the new data to back up the claim or refute it. Either way, the results can be used across the board to improve customer experience.

Once the analysis is complete, the results of the survey can then be shared internally so that a plan of action can be made to correct any issues or strategies for steering future business initiatives. If the data revealed that the website is slow, developers can be assigned to fix the issue. Or if young women reported that the product wasn’t attractive, advertising teams can brainstorm ways to cater to that specific audience. Once the company has a chance to respond and implement solutions to these changes, a second survey can be used to see if customers respond differently.

Customer surveys can provide businesses with insight about their customers that is unbiased and tailored to provide the maximum information. Whether customers are answering questions about user experience, product quality, or customer service, the data that can be gathered can help businesses constantly improve. For business owners who are looking to harness the power of customer surveys, these tips are a helpful guide for analyzing data so that the study is accurate and usable.