“Identifying causal effects involves assumptions, but it also requires a particular kind of belief about the work of scientists. Credible and valuable research requires that we believe that it is more important to do our work correctly than to try and achieve a certain outcome (e.g., confirmation bias, statistical significance, stars). The foundations of scientific knowledge are scientific methodologies. Science does not collect evidence in order to prove what we want to be true or what people want others to believe. That is a form of propaganda, not science. Rather, scientific methodologies are devices for forming a particular kind of belief. Scientific methodologies allow us to accept unexpected, and sometimes, undesirable answers. They are process oriented, not outcome oriented. And without these values, causal methodologies are also not credible.”
Dev links coming to you weekly from now on!
Week #6: Oct 17
“How does a locally-managed conditional cash transfer program impact trust in government?”
- Cash transfers increased trust in leaders and perceptions of leaders’ responsiveness and honesty
- Beneficiaries reported higher trust in elected leaders but not in appointed bureaucrats
- Government record-keeping on health and education improved in treatment communities
2. Kinda random: sand dams
Read a WB blogpost on sand dams as a method for increasing water sustainability in arid regions … but that did not explain how the heck you store water in sand, so watched this cool video from Excellent Development, a non-profit that works on sand dam projects.
3. USAID increasingly using “geospatial impact evaluations” ft. MAPS!
Outlines example of a GIE on USAID West Bank/Gaza’s recent $900 million investment in rural infrastructure
BONUS: In other geospatial news
LSE blog post on the work of ground-truthing spatial data in Kenya
Week #5: Oct 10
Health Round-Up Edition
1. Dashboards for decisions: Immunization in Nigeria
A new dashboard is being used to improve data on routine immunizations … but doesn’t look like the underlying data quality has been improved. Is this just better access to bad data?
2. Norway vs. Thailand vs. US
A comparative study of health services for undocumented migrants
3. Traditional Midwives in Guatemala
Aljazeera on the complicated relationship between traditional midwives providing missing services and the gov’t trying to provide those services in health centers
BONUS: Visualizing fires + “good”
- Satellite imagery of crop burning in India in 2017 vs 2018
- How good is good? 6.92/10. The YouGov visualization on how people rate different descriptors on a 0-10 scale is really interesting if you look at the distributions – lots of agreement on appalling, average (you’d hope there would be clustering around 5!), and perfect. Then, pretty wide variance for quite bad, pretty bad, somewhat bad, great, really good, and very good. Shows how you should cut out generic good/bad descriptions in your writing and use words like appalling or abysmal that are more universally evocative.
Week #4: Oct 3
“Consider a simple policy rule: if a government’s statistics cannot be questioned, they shouldn’t be trusted. By that rule, the Bank and Fund would not report Tanzania’s numbers or accept them in determining creditworthiness—and they would immediately withdraw the offer of foreign aid to help Tanzania produce statistics its citizens cannot criticize.”
2. 12 Things We Can Agree On About Global Poverty?
In August, a CGDev post proposed 12 universally agreed-upon truths about global poverty. Do you agree? Are there other truths we should all agree on?
3. Food for thought on two relevant method issues
- Peter Hull released a two-page brief on controlling for propensity scores instead of using them to match or weight observations
- Spillover and estimands: “The key issue is that the assumption of no spillovers runs so deep that it is often invoked even prior to the definition of estimands. If you write the “average treatment effect” estimand using potential outcomes notation, as E(Y(1)−Y(0))E(Y(1)−Y(0)), you are already assuming that a unit’s outcomes depend only on its own assignment to treatment and not on how other units are assigned to treatment. The definition of the estimand leaves no space to even describe spillovers.”
BONUS: New head of IMF
Dr. Gita Gopinath takes over.
My final week of taking over IDinsight’s internal development links.
1. Development myths: debunked
Rachel Glennerster asked for examples of development myths, resulting in a list development myths along with debunking sources / evidence against. Some of the myths shared, with accompanying evidence:
- Food aid causes conflict in Africa: Famous paper discredited based on methodological issues
- Better educated women will automatically make more gender-equitable choices (even in the absence of norm change): Discredited due to paper on more selective abortion of girls by more educated mothers in India
- Service delivery will result in state legitimacy in war-affected contexts: Study in Afghanistan refutes this
- Surgery is too expensive and not cost-effective in LMICs: Evidence that it is comparably cost effective to other public health interventions
2. Traditional local governance systems (autocratic) underutilize local human capital
A new paper by Katherine Casey, Rachel Glennerster, Ted Miguel, and Maarten Voors. “We experimentally evaluate two solutions to these problems [autocratic local rule by old, uneducated men] in rural Sierra Leone: an expensive long-term intervention to make local institutions more inclusive; and a low-cost test to rapidly identify skilled technocrats and delegate project management to them. In a real-world competition for local infrastructure grants, we find that technocratic selection dominates both the status quo of chiefly control and the institutional reform intervention, leading to an average gain of one standard deviation unit in competition outcomes. The results uncover a broader failure of traditional autocratic institutions to fully exploit the human capital present in their communities.“
3. Aggressive U.S. recruitment of nurses from Philippines did not result in brain drain / negative health impacts
A new paper by Paolo Abarcar and Caroline Theoharides. “For each new nurse that moved abroad, approximately two more individuals with nursing degrees graduated. The supply of nursing programs increased to accommodate this. New nurses appear to have switched from other degree types. Nurse migration had no impact on either infant or maternal mortality.”
BONUS. Data viz: Poverty persists in Africa, falls in other regions
Justin Sandefur shared that the Economist much improved a World Bank graphic to more clearly visualize how the number of people living in poverty has risen slightly in Africa while other regions have seen sharp decreases in # of people in poverty over time. (Wonder how the graphic would like stacked Africa, South Asia, then East Asia & Pacific? Less dramatic contrast between Africa and the other regions? Number of poor in South Asia hasn’t decreased as dramatically as East Asia, would look more similar to Africa trend than East Asia trend until about 2010 I think.)
This is part 2 of me taking over IDinsight’s internal development link round-up.
1. This week in gender & econ
- Increasing property rights increased son preference in India
- When Mexican children went to school, their mothers’ labor force participation, # hours worked, and income went up
2. Two papers on p-hacking or bad reporting in econ papers
- Comparing p-hacking across research design methods: Diff-in-Diff and IV distribution of p-values suggests p-hacking, while RCT shows a little evidence and RDD looks good – see p. 15 graphic
- Unreported balance results: Estimates 45% of statistically significant, “failed” balance tests for treatment/control groups post-randomization are unreported
3. Mapping trade routes Tilman Graff shared some really cool visualizations of trade routes, aid, and infrastructure in several Africa countries. They were created as part of his MPhil thesis.
Lant Pritchett and other researchers often argue that development economists are too focused on one-off, micro interventions and fail to see the big picture. They are highly critical of the hype that develops around specific interventions following the release of studies using RCTs or other quasi-experimental methods to measure the impact of a specific program – microfinance, for example, had a big moment and, more recently, cash transfers have dominated many discussions of economic development.
Pritchett’s scorecard comparing first generation RCT practice to the approach of the non-RCT crowd is an especially brutal assessment of the micro development literature (second table in the link). He writes, “National Development leads to better well being. National development is ontologically a social process (markets, politics, organizations, institutions). RCTs have focused on topics that account for roughly zero of the observed variation in human development outcomes.”
There’s a lot that’s valid about this line of critique, although I think it’s more a call to be sure to contextualize learnings, ideally with qualitative research to investigate the how and why of a quantitative claim, rather than motivation to throw out the micro development approach altogether.
Besides, there is something so satisfying about how a small intervention can have a big impact.
Small bridges, big deal
Brooks and Donovan’s recent paper (full PDF here) found that building footbridges in Northern Nicaragua protected local workers from the typical wage loss seen during flooding, when travel routes are cut off, and even led to increased profits of local farmers.
Their primary finding is best seen through two graphics from the paper. The first shows the distribution of wage earnings before footbridge construction, and you can clearly see a massive disadvantage to those experiencing flooding. In the second, the gap has disappeared.
Figures 1 & 2: Distribution of wage earnings BEFORE footbridge construction
They also find positive spillover effects. First, rural villagers were able to take higher paying jobs in nearby towns, increasing their wages and increasing the wages of those left behind, who faced less competition in the local labor market. (A similar mechanism to that found in the No Lean Season research, which offered select villagers incentives to migrate to cities for work and found positive income effects for those households and neighboring non-study households.)
Second, farmer profits increased. Not because of lower trade costs that allowed farmers to buy cheaper inputs, but because they were able to access new purchasing markets for their goods and diversify their income sources.
This paper is amazing because the data viz communicates clearly, the findings are meaningful and positive, and the idea for the research design had to have come from an intimate knowledge of the challenges facing rural citizens of Northern Nicaragua.
A national and local development tool
Infrastructure studies connect easily to those big questions about national development that anti-randomistas would prefer to focus on.While it won’t be footbridges in every location, there are lots of countries where road and transport infrastructure solutions are needed to promote both local and national development.
Papers like this one show how connectivity and access can be an important determinant of economic welfare via multiple mechanisms. Besides income effects like those measured in the Brooks and Donovan paper, there are possible effects for access to credit, healthcare, or other public services that isolated communities would otherwise miss out on.
Gaining entitlements with infrastructure and cash
There’s a seriously inspiring narrative in there – a simple change that leads to more options, more opportunities, more connectivity. As my colleague Sindy was discussing today, there is a pattern that interventions about increasing options and expanding opportunity, such as infrastructure improvements or cash transfers, seem more powerful to affect broad change than interventions targeting very narrow and specific goals.
Although, there is probably a gain in using both types of interventions at different times, or concurrently.
McIntosh and Zeitlin’s new paper compares a cash transfer program directly with a child nutrition program.The final line of their abstract made me think about paternalism and beneficiary preferences: “The results indicate that programs targeted towards driving specific outcomes can do so at lower cost than cash, but large cash transfers drive substantial benefits across a wide range of impacts, including many of those targeted by the more tailored program.”
People spend their money with different priorities than programs dictate and seem to get more out of it. That suggests to me that cash transfers (or infrastructure improvements) are a way to improve this baseline ability to provide for your household (“entitlements” à la Amartya Sen), while specific health or education interventions are more useful as public service-style campaigns to promote undervalued goods, such as immunizations.
A final thought
I’m generally curious how often Sen’s entitlements approach is explicitly applied to non-famine topics in development research. I’m guessing often. (A two-minute google led me to a PhD thesis called “Poverty as entitlement failures” that sounds interesting.)
Each Wednesday at IDinsight, one of our tech team members, Akib Khan, posts a few links (mostly from Twitter!) to what he’s been reading in development that week. For the next three weeks, he’s on leave and I am taking over! Thought I should cross-post my selections (also mostly curated from #EconTwitter):
Cash Transfer Bonanza: The details matter
Blattman et al. just released a paper following up on previous 4-year results from a one-time cash transfer of $400, now reporting 9-year results (see first 3 links). To liven up the internal discussion, I’m adding critiques by Ashu Handa (UNC Transfer Project / UNICEF-Innocenti economist and old family friend), who has cautioned against lack of nuance in interpretation of CT study results, esp. around program implementation details like who is distributing grants, the size of the grants, and how frequently they are given – he studies social protection programs giving repeat cash transfers.
- Blattman, Fiala, Martinez (2018)
- Vox coverage of paper
- Berk Ozler DIME coverage
- NYT coverage of USAID studies directly comparing cash to other interventions
- Handa critical comments on Ozler CT blogpost from earlier this year
- He also tweets about cash transfer studies a lot!
- Transfer project
Diff-in-diff treatment timing paper… with GIFs!
Andrew Goodman-Bacon (what a name!) has a new paper that all of #EconTwitter is going crazy over. It deals with some methodological issues using diff-in-diff when treatment turns on at different times for different groups, and other scenarios where timing becomes important. Real paper not for the faint hearted, but the Twitter thread has some great GIFs!
African debt to China: reality doesn’t match the hype
Bonus link: Eritrea & Ethiopia border opening party
In order to have a high-quality writing sample for the RA jobs I’m applying to this fall, I am revamping my thesis! Joy of joys!
I thought about doing this earlier in the year and even created a whole plan to do it, but ended up deciding to work on this blog, learning to code, and other, less horrifying professional development activities.
I say horrifying because the thesis I submitted was HORRIBLY WRITTEN. So so so bad. I cringe every time I look back over it. I had tackled a 6-year project (the length of time it took to write the paper I was basing my thesis on, I later found out) in four months time. Too little of the critical thinking I had done on how to handle the piles and piles of data I needed to answer my research question actually ended up in writing.
I thought it would be a drag to fix up the paper. I didn’t expect to still be as intrigued by my research topic (democracy and health in sub-Saharan Africa!) or to be as enthusiastic about practicing my economic writing. I’m taking the unexpected enjoyment as a positive sign that life as a researcher will be awesome.
I’ve been thinking critically about the question of democracy and health and how they’re interrelated and how economic development ties into each. I’ve read (skimmed) a few additional sources that I didn’t even think to look for last time and I already have some good ideas for a new framing of why this research is interesting and important. The first time around, I focused a lot on the cool methodology (spatial regression discontinuity design) because that’s what I spent most of my time working on.
My perspective on the research question has been massively refreshed by time apart from my thesis, new on-the-ground development experience, and the papers I’ve read in the interim.
My first tasks have been to re-read the thesis (yuck), and then gather the resources I need to re-write at least the introduction. I am focusing on the abstract and introduction as the first order of business because some of the writing samples I will need to submit will be or can be shorter and the introduction is as far as most people would get anyways.
To improve my writing and the structure of my introduction, my thesis advisor – who I can now call Erick instead of Professor Gong – recommended reading some of Ted Miguel’s introductions. I printed three and all were well-written and informative in terms of structure; one of them (with Pascaline Dupas) even helped me rethink the context around my research question and link it more solidly to the development economics literature.
The next move is to outline the introduction by writing the topic sentence of each paragraph (a tip taken from my current manager at IDinsight, Ignacio, who is very into policy memo-style writing) using a Miguel-type structure. I’ll edit that structure a bit, then add the text of the paragraphs.