3 Cool Things You Can Do With RapidPro Data / Experience from Pakistan
Launched: November 2015
By Elliot McBride – Innovations Unit, Pakistan
Post was originally posted on the unicefstories.org.
The potential that the U-Report system has for monitoring and evaluation in developing parts of the world is nothing short of revolutionary. At the very beginning of my development career, I undertook an M&E assignment for an implementing partner in Islamabad. There were 68 participants in the study who filled out surveys comprised of yes/no and short sentence answers. As I was new to development and new to M&E, I was immediately struck with the joys of M&E manual labor. The marking took about 6 minutes/survey and therefore:
6*68 (total minutes)/60 (total hours) = 6.8 hours (it still gives me a headache just thinking about it).
Data entry was easily another 2 hours, given that a lot of the answers were handwritten sentences and were sometimes illegible and difficult to understand. From conversations with other M&E professionals in development, this is a common and often unavoidable problem associated with typical survey style feedback on programs. Using this method, you are 9-10 hours behind in work hours before crunching your first number (plus whatever time you have spent in the field ensuring all forms are completed, focus groups are filled with participants etc etc.)
However, the biggest problem any stats nerd has with the data produced in this way is the shallow amount of analysis that you are able to play with afterwards. If an applicant forgets to write down their district or town, this distorts the data. If a participant does not put in their gender – you cannot give their answers gender representation for the remainder of their responses. Considering that typically a large study has a total sample size between 60 and 80 – these manifestations can completely change the overall conclusions of your final report. Frustrating times indeed.
Enter RapidPro, sit back, and hold on tight.
In conjunction with the WASH section at UNICEF Pakistan, we distributed 6000 WASHkits with items including a bucket, towel, jerry cans and soap. Our goal was to successfully use the U-Report function as a program monitoring tool. The premise for this program was simple, apply a sticker asking people to text us with their feedback. We designed a simple sticker which was applied to every WASHkit. The sticker displayed the following text:
“Please help us improve this kit for FREE.
Text ‘kit’ to 8623”
After 6 days and only a quarter of our WASHkits distributed, we have over 250 subscribers, more than any M&E response I have ever worked on before. However, the most exciting thing I found after downloading the results spreadsheet was the freedom I had to explore what was happening. This post is simply to provide inspiration for the cool things that you can do with the RapidPro data once you have downloaded it. Not all of it will apply to every type of poll – but hopefully it can provide insight to the level of accuracy and limitless creativity attainable with the data from this system. And goodness me, what a system.
1. Scatter plot of real-time registration mapping across districts.
This one blew my noodle, and I’m only guessing here, but I’m not sure there is another product out there that can get this information from rural areas in a developing or sensitive region. This shows you each registration, in real time, according to the beneficiaries location. *mic drop*.
If your RapidPro account has a webhook built in to identify districts/regions/towns etc. you now are able to track registrations OF INDIVIDUAL BENEFICIARIES, in real-time (if you don’t have a district webhook, Issmail has written an easy-to-use guide on how to achieve such a thing here). You’ve got to give this system some credit….this is justcool – Here you can see which districts were targeted when and identify how long a respondent typically has before they register.
The end product looks something like this:
2. Using gender as the independent variable – Getting down with Bar graphs
The next way to present your data with the output given to us by the gods of RapidPro are results using gender as the independent variable. A typical registration flow will include the participant’s particulars – name, age, gender, district etc. Once you have the portion of the flow that assigns gender into the results you can use this information to break any type of response into a male/female independent variable, data graph.
See below for the finished graphs. This has been done many times before in development surveys, but remember from start to finish RapidPro allows you to make this in less than 5 minutes, with automatic data entry.
3. Graphing the Response times for completed polls. Frequency Histograms: an exposé
This last one will become more useful to you the more polls you put out. If you have ever been curious about how long it takes our U-Reporters to finish a poll this next function will help you get a handle on that. Out of all of the finished poll responses you can graph the time it takes to successfully complete each poll. For example if your poll only has 36% completion and the last survey had 71% running this test will allow you to determine if there is a higher dropout rate because the poll is taking too long to complete.
This data can be displayed in any format from a simple average response time i.e. 14 minutes 3 seconds. This does not take into account the outliers that will dramatically increase the average time. I like to use a frequency histogram; this way the outliers don’t give you an inaccurate reading of how long it takes the average respondent. The average time to complete this poll was 16 minutes 26 seconds, but as you can see, most of the respondents completed the poll in under 10 minutes. Histograms – the better way to gram.
RapidPro completely changes the monitoring and evaluation landscape by shortening the distance between UNICEF and the most disadvantaged beneficiary, and then giving them a voice. It’s up to us to squeeze everything out of this technology that we can to amplify that voice. If there are any questions or comments on any of this please feel free to email me at firstname.lastname@example.org. I would also be excited to learn from others and see more creative ways to use the data from RapidPro.
Stay safe out there,