IPA’s Data Management System

IPA’s Data Management System, or DMS, is a package of Stata and Excel tools that automate and assist your project’s data flow.

Key Takeaways
  • The DMS automates data quality checks, file organization, and reporting throughout your research project’s data collection process.
  • Use the DMS for duplicate detection, high-frequency checks, backcheck comparisons, and enumerator performance tracking.

What is the DMS?

IPA’s Data Management System, or DMS, is a Stata package that automates and assists your project’s data flow. The DMS assists you with the following functionality:

  • Creating folders and do-files: Automatically creates a folder structure, Stata files and documentation, including for projects with multiple surveys—creates standardized folder structure, input files, do-files and readme files
  • Deciding which quality checks are important to your project: A simple global do-file with a detailed description of the different check features as well as a section to turn on and off specific checks based on your own needs—backchecks, field comments, survey tracking, duplicates, missing values, outliers, text audits, and more
  • De-duplicating: Automatically flags all duplicates, and provides a summary of the differences of each duplicate group. Also, saves a new de-duplicated clean dataset so high-frequency checks can run.
  • Checking and reporting on data quality: Runs high-frequency checks to flag issues with data quality. The system asks you to identify important checks and variables in the global do-file and Excel-based inputs sheets, then automatically generates reports for interpretation and follow-up actions to improve data quality.
  • Checking and reporting on inconsistencies in the dataset: Compares and reports on the differences in responses in the Survey and Backchecks datasets.
  • Corrections to the dataset: Includes an easy-to-use excel-based tool for making and logging changes to data including replacements and values, dropping of observations and flagging HFC outliers values as correct.
  • Reporting on enumerator performance: Generates enumerator reports with indicators of enumerator productivity and other general enumerator performance including Don’t Know/Refuse/Missing/Other specify responses rate and summary statistics by specified variables.
  • Reporting on survey progress: Generates progress reports on survey completion, by day/week/month and by a customizable filter variable—progress reports by region, district, village, treatment status

How to install the DMS?

To use the DMS, you will need Stata 17 or above. If you are installing the DMS for the first time, use this command in Stata:

* install ipacheck
net install ipacheck, from("https://raw.githubusercontent.com/PovertyAction/high-frequency-checks/master") replace
ipacheck update

* after initial installation, ipacheck can be updated at any time via
ipacheck update

To start your project’s folder structure, use:

ipacheck new, surveys("NAME OF SURVEY") folder(“path/to/file/destination")

Note that if you do not include the folder option, the system will create the folder structure in your current working directory. There is also an option to make subfolders for multiple surveys:

ipacheck new, surveys("Survey1" "Survey2") subfolders

Learning to use the DMS

The DMS includes a practice exercise. This exercise can help you learn how to fill out the globals do-file and the input sheets of the DMS. You can also work through the different output sheets and make corrections. If you have already updated your ipacheck package using the code in the previous section, you can do the exercise on your computer using this code:

ipacheck new, exercise folder("path/to/file/destination")

Note that if you do not include the folder option, the system will create the folders in your current directory. This code will download the folder structure and input and replacement files as ipacheck new would, as well as:

  • Practice survey data (in 4_data/2_survey folder)
  • Sample data (in 4_data/2_survey)
  • Backcheck data (in 4_data/3_backcheck)
  • Text audit media files (in 4_data/2_survey/media)
  • Comment media files (in 4_data/2_survey/media)
  • Audio Audit files (in 4_data/2_survey/media)

If you already use an existing file structure, you can download the files only using:

ipacheck new, files folder(“path/to/file/destination")

The folders and input structure of the DMS have changed in version 4.0. You will need to make some changes to the folder structure of do-files.

Adapting the DMS to your project

Table 1 describes some features that comprise the DMS, and when you should incorporate them based on the stage of your IPA project

Table 1: Example DMS features by data collection stage
Feature Description Before data collection During data collection After data collection
ipacheck new Creates a folder structure and/or input files for DMS. Also used to create DMS exercise X X
README files Every folder has a readme file to list files, authors, changes, and anything necessary for a successful handover X X X
DMS inputs file Initializes inputs for outliers, other specify and enumerator stats checks X X

DMS Training

See the following excerpt for a demonstration of the DMS!

Do you need support with the DMS?

If your research project has limited time or capacity to set up or run the DMS, the Global Research and Data Science team is available to provide direct technical support to any project. If interested or need support installing the DMS, email researchsupport@poverty-action.org.

Back to top