How to perform data checks after Soft Launch

The Soft Launch phase has concluded and survey fielding has paused. We have a set of data from real respondents. Now what?

Data cleaning

Let us start with the basics, we do not want to analyze low quality data so the first thing to do is see how many qc flags each respondent has raised. If the amount of failed flags is above the threshold you have set with the panel(s) those respondents should be removed and replaced through additional fielding.

Another basic check that most panels will do is reviewing the open-ended text input responses. Anyone filling out "gibberish" should be removed. Lastly, check the completion time or Length of Interview (LOI) if a respondent is speeding through the survey chance are high they the quality of the data from them is low.

Pathway checks

Beyond the basics one of the first checks is making sure a respondent is seeing all the questions they should have seen. Most surveys go out to multiple respondent groups like User and Non-user, Current and Lapsed customer, etc. These are referred to as pathways or profiles. Go through the various pathways and make sure that each audience is seeing the correct questions. This can be time consuming because of branching logic but in the end it is worth it.

Industry background checks

Does the data align with your secondary research of the industry this survey is based on? Are the most popular vendors also the most utilized by this respondent base? Does the data support or refute the hypotheses formed prior to launching the survey?

Analysis Data point checks

Do you have all the data you need? Perhaps you have already built your model or maybe you are using IncQuery's data analysis tool? Are there any gaps in the data points? Now is the time to add questions to fill those gaps.