Back Checks

This guide covers IPA’s format for backchecks, which are essential quality control processes in survey data collection. It provides detailed protocols for planning, executing, and analyzing backchecks to ensure data accuracy, verify survey reliability, and improve fieldwork performance through systematic monitoring and feedback.

Key Takeaways
  • Backchecks are essential for ensuring data quality by verifying survey accuracy and identifying discrepancies.
  • Proper planning, execution, and analysis of backchecks can significantly improve survey reliability and fieldwork performance.

What are Backchecks?

Backchecks, also known as field audits or reinterviews, are a quality control process in survey data collection to ensure accuracy and reliability. They involve re-contacting a subset of respondents to administer a mini-survey with selected questions from the original questionnaire. The responses are then compared to the initial survey to identify discrepancies, assess surveyor performance, and evaluate the robustness of the survey instrument. Backchecks help hold surveyors accountable and improve data quality by verifying how questions are asked and detecting potential errors in data collection.

This critical step in data quality enables researchers to monitor both the performance of the survey team and the questionnaire in the field. Back checks are just as critical to producing high-quality data as double entry of data is to ensuring data entry accuracy. Well-done back checks can lead to:

  • Questionnaire revisions.
  • Improved training.
  • Additional rounds of surveying.
  • Changes to the survey team.

Why is important to Backcheck?

Surveying can be arduous and exhausting. To get out of the heat and back to the office, surveyors are tempted to cut corners in the field. The most common shortcuts are:

  • Skipping sections or entire surveys.
  • Failing to prompt correctly.
  • Modifying examples or informed consent scripts.
  • Prematurely classifying respondents as missing or away.

In your mission to collect high-quality data, you need to develop a systematic way to detect poor surveying and incentivize high-quality fieldwork. Essentially, researchers need a disincentive for surveyors to take shortcuts and falsify data.

Beyond poor administration, researchers also want to methodically monitor how well your questionnaire is performing in the field. Researchers need to determine

  • Are respondents changing their answers to questions that shouldn’t change?
  • Do key outcomes vary significantly?

To understand whether your questionnaire accurately captures the key outcomes of your study, you need a tool that measures the quality of your measures.

IPA Sample Selection Protocols

Making it Happen: How to Design the Back Check Survey

Develop Your Plan Before You Start Surveying

The key to a quality back check is planning. Multiple analyses conducted by IPA staff have shown that the time elapsed since the original survey is the most common statistically significant predictor of the number of errors in back checks across projects and contexts. The back check team should be ready to begin fieldwork on day one of the survey period.

Craft your back check plan at the same time as your survey work plan and maintain it as a running document, updating details as the survey work plan evolves. This ensures that at the end of the survey, you have an accurate description of how the back checks were carried out and any actions taken.

When developing your plan, address the following questions:

  • How will you conduct your back checks? Can you do them in person?
  • How many surveys will you back check? 10% or more? Will you change the percentage during the survey period?
  • How will you spread them across your enumeration areas, surveyors, and survey period?
  • How many questionnaires will you use? Multiple questionnaires are optimal, especially for longer, more complex surveys.
  • How much money do you have for your back check? Note: Back checks should be budgeted during the project development stage.
  • If your budget is tight, can you implement phone back checks or other low-cost strategies?
  • When will the back checks be done? Ideally, within 1-2 days. One week is the maximum.
  • How big will your back check team be, and how will you train them?
  • Are there enough back checkers to check all enumerators each week? Do you need to hire more back checkers during the first few weeks of surveying?
  • How will the team get to the field? If conducting back checks by phone, do you have enough phones?
  • How will you define and calculate errors?
  • How will you deal with discrepancies? Create a plan with your field management team.
  • How will you log back check errors? Aim to integrate this into your master tracking system.

Selecting and Managing the Back Check Team

Selecting the Right Team

A back check team member should be more qualified than your regular surveyors. In detail, a member of the back check team should be:

  • Experienced: Consider retaining the team of surveyors used during the pilot as the back check team.
  • Trustworthy: Prefer someone you’ve worked with before.
  • Independent and Enterprising: Members may have to travel to villages individually.

Training

The back check team should attend the main training with the entire team to ensure everyone understands the questions in the same way. Consider using them as leaders at the training of the main survey team.

Executing the Back Check

Timing and Revisits

  • Aim to complete back checks within 1-2 days of the original survey.
  • Decide on the number of revisits the back check team should aim to do. Ideally, the team will do the same number of revisits as the original surveyors.

Selecting Respondents

  • The back check team should never select the households. Use Stata or Excel to select respondents to back check using random selection, stratified by surveyor.
  • If your project requires replicable randomization for back checks, use Stata. If not, use SurveyCTO randomization.

Producing and Acting on Results

Analysis Framework

  • During questionnaire design, create your analysis framework and define what you consider an error.
  • Use IPA’s user-written Stata command bcstats to compare survey data and back check data.

Acceptable Range of Deviation

  • Establish a range of acceptable deviation for every back check question. For example, age +/-5 years.

Analysis for Type 1, Type 2, Type 3

  • Type 1: Look at the overall error rate. If it is greater than 10%, this is a red flag.
  • Type 2: Perform the same analysis as Type 1, but examine these variables individually.
  • Type 3: Examine the overall error rates by question and perform stability checks.

When and How to Take Action on Results

  • Results of the back check comparison should be discussed with the field leadership team.
  • Set clear standards for when to fire surveyors, when to follow up with respondents, and when to re-do surveys.

Digital Back Checks

For CATI surveys, audio audits can be used to monitor enumerator performance. However, audio audits will not fulfill the back check’s function of testing the reliability of a survey question. If your PI is concerned with the stability and quality of the survey questions, a back check is required.

Back Checks in practice

Check out the IPA High Frequency Check Exercise! These exercises help familiarize users with the setup and use of IPA’s Data Management System, which includes High-Frequency Checks (HFCs), Survey Tracking, and Back Check comparison. Learn more

Conclusion

Back checks are a critical component of ensuring high-quality data collection. By following the guidelines outlined in this manual, you can effectively implement back checks, analyze the results, and take appropriate actions to improve your survey process.

Back checks are a critical component of ensuring high-quality data collection. By following the guidelines outlined in this manual, researchers can implement back checks, analyze the results, and take appropriate actions to improve the survey process.

Back to top