Sunday, June 19, 2011

What I Didn’t Learn in Graduate School, or, Survey Research is Hard!

Though I remain somewhat skeptical of the experimental turn in Political Science, I must confess that I have also grown a bit contemptuous of papers that are only based on survey methods. Well, in the past two weeks my esteem for survey methods has risen considerably. Survey-based research, particularly in a developing country context, is hard!!!

Logistics, logistics, logistics

For the most part, I was not privy to the development of the survey instruments we’re using, so I can’t speak to the challenges that were surely involved in developing them. Rather, I’ve been more aware of the logistical challenges associated with actually implementing the surveys in the field. We’re using a paper-based survey,* so I’ve witnessed and taken part in epic amounts of photocopying and stapling. Transporting the survey instruments to Rongo involved purchasing new suitcases and filling a number of cardboard boxes, which are all now littered about (er, organized perfectly, if my boss is reading this) my hotel room. We’ve warned the third PI on the project to pack light so he can transport the completed surveys back to the states for double data entry.

In addition to the papers and the paperwork, there are all the logistics of implementing the surveys in the field. Organizing transportation, making sure the enumerators are going to the right places, making sure they have sufficient credit on their phones to check in constantly with their field coordinators, buying refreshments for meetings. If you check out Kenya’s rating on the World Bank’s Ease of Doing Business survey, you can imagine that performing the simple transactions required for such activities is non-trivial.

Beyond Research Fatigue

In a previous blog post I worried that the respondents we were interviewing might be tired of being the subject of yet another research project, which does not obviously benefit them. The situation I have encountered in the villages I have visited so far has not exactly matched this prediction, but it has made me even more aware of conflicted nature of conducting research in a place like rural country. In a nutshell, we are getting a lot of requests for financial assistance. It makes sense. After people spend hours telling us about the enormous challenges they face in their daily lives, they wonder what we can do to help. My PI recently blogged about this particular challenge, in a more elegant manner than I am doing here so you should check out his post. (The post also contains a photo of me looking like a Mormon on a mission trip, for added incentive).

So bottom line: survey-based research is hard. It’s a little depressing that our fairly small-N research this summer is unlikely to have very much “power” to shed statistically significant light on the questions we’re asking. But hopefully the open-ended nature of the survey tools will provide a wealth of qualitative data, which will be interesting in and of itself, and also help to inform the next round of larger-N (and thus even harder!) survey research.


I was just brushing my teeth after having posted this and realized that it may come across as rather whiney. So I figured I should add that I’m also finding this particular experience with survey research to be incredibly fulfilling. Spending my days outside in a beautiful place, talking to people and learning out their lives is pretty amazing. As is the T-shirt tan I’ve acquired in the process.

*I’ve been lobbying my PI’s to adopt a smartphone-based survey, such as those one can create with Open Data Kit, in future rounds of this research project. They have raised a number of challenges associated with adopting more advanced technology (e.g. power outages, difficulty in entering free-form responses, intimidation of survey respondents) but I’m not sure these trump the challenges associated with the paper-based format. With this in mind I’m planning to try and develop a smartphone-based version by the end of the summer if and when I find the time.


  1. Keep these posts coming. I'm totally engrossed.

  2. If love is a mountain, then if men go up, more women they will see while
    women will see fewer men.
    Tera Items