Split Ballot Technique

What is split ballot technique?

The split ballot technique is a survey method where a group of respondents are divided in half (or several smaller groups). Each group receives a questionnaire that asks for the same information but using slightly different questions.

Using a half split technique can help reduce the effects of position bias in multiple-choice survey questions. For example, a survey may ask a question that lists answers as:

1. X
2. Y
3. Z

In another version of the survey, respondents may be asked the same questions with answers listed as:

1. Y
2. Z
3. X

The question hasn’t changed, but survey takers may have a subconscious bias based on the position of the answer. Some respondents might be prone to choosing the first answer more often than other responses, even if they’re not aware of it.

Other ways split surveys reduce potential for bias is to use experimental manipulation. This takes two versions of the same question to assess participants’ willingness to express opinions. For example, one set of surveys may use the phrase “greater than” listed first, while the other set lists “less than” first. Having two differently worded questions that ask for the same answer helps reduce potential biases.

Although basic questions are asked of all participants, the split half technique in research may have slightly mismatched data when comparing certain variables to total completed surveys. Since some survey respondents had different questions than the other half, you could have a difference between the number of survey participants and the totals for some of the variables.

For example, if you have two surveys of 5 questions each, question number 5 may have two versions for the two surveys:

1. X
2. Y
3. Z

Or:

1. A
2. B
3. C

When calculating the total value of respondents who choose “1. X” in your survey, you’ll have fewer responses than the number of surveys you sent out. Half of your participants did not see the same question with that answer available.

QuestionPro uses a split ballot technique to measure the effects of two frames: regulation versus non-regulation. A secondary manipulation, varying the order of schemata measures (frame first versus schema measures first), tests the role of schemata, or behaviors, in framing effects. Split ballot technique allows you to try out different versions of the same questions to see which version seems to result in the most accurate reporting. It also ensures that the response frequencies will be stable enough for you to draw the necessary conclusions. It allows researchers to make causal interpretations and helps them to be more certain about the reasons for any effects they obtain.

One example of using the split half technique in research is to find the general public’s opinion on a topic. Three versions of the most important question on an issue are used in agenda-setting research to measure issue salience, or level of importance, among the public. A split ballot in a state-wide survey compared multiple versions of the public agenda with a social frame of reference versus a personal frame of reference, the traditional term problem versus issue, and the effects of question order. High correlations between the different versions were found in all three sets of comparisons.

What is Survey Router / Split Ballot Testing?

Split Ballot Testing is when you randomly divide your sample into two or more sub-samples and perform an experiment. The experiment could be different versions of a survey, different surveys altogether, or a combination of these. The goal is to see if any differences exist between the versions of the survey or how users behave differently.

How do I set up Survey Router / Split Ballot Testing?

Go To:

• $crumb.fullText

  Select the Router Mode and assign surveys and click on the Save Changes button.

Screenshot

A URL will be provided which can be used for distribution to your respondents or sample panel. Depending on the mode selected the surveys will be displayed.

Screenshot

Target Survey: With target survey you can set up a URL and assign a survey for the URL.

Screenshot

Split Ballot: With split ballot you can set up a URL and assign Multiple Surveys. Randomly one of the selected surveys will be picked.

Screenshot