A new paper by Jakiela and Ozier sounds like an insane amount of data work to classify 4,336 languages by whether they gender nouns. For example, in French, a chair is feminine – la chaise.
They find, across countries:
- Gendered language = greater gaps in labor force participation between men and women (11.89 percentage point decline in female labor force participation)
- Gendered language = “significantly more regressive gender norms … on the magnitude of one standard deviation”
Within-country findings from Kenya, Niger, Nigeria, and Uganda – countries with sufficient and distinct in-country variation in language type – further show statistically significant lower educational attainment for women who speak a gendered language.
(Disclaimer: The results aren’t causal, as there are too many unobserved variables that could be at play here.)
As the authors say: “individuals should reflect upon the social consequences of their linguistic choices, as the nature of the language we speak shapes the ways we think, and the ways our children will think in the future.”
3ie wrote on June 11 about why you may need a pilot study to improve power calculations:
- Low uptake: “Pilot studies help to validate the expected uptake of interventions, and thus enable correct calculation of sample size while demonstrating the viability of the proposed intervention.”
- Overly optimistic MDEs: “By groundtruthing the expected effectiveness of an intervention, researchers can both recalculate their sample size requirements and confirm with policymakers the intervention’s potential impact.” It’s also important to know if the MDE is practically meaningful in context.
- Underestimated ICCs: “Underestimating one’s ICC may lead to underpowered research, as high ICCs require larger sample sizes to account for the similarity of the research sample clusters.”
The piece has many strengths, including that 3ie calls out one of their own failures on each point. They also share the practical and cost implications of these mistakes.
At work, I might be helping develop an ICC database, so I got a kick out of the authors’ own call for such a tool…
“Of all of the evaluation design problems, an incomplete understanding of ICCs may be the most frustrating. This is a problem that does not have to persist. Instead of relying on assumed ICCs or ICCs for effects that are only tangentially related to the outcomes of interest for the proposed study, current impact evaluation researchers could simply report the ICCs from their research. The more documented ICCs in the literature, the less researchers would need to rely on assumptions or mismatched estimates, and the less likelihood of discovering a study is underpowered because of insufficient sample size.”
…although, if ICCs are rarely reported, I may have my work cut out for me!
I was supposed to read Wanderlust for my first year seminar, but it ended up being one of the first school assignments I did not fully complete. Three years later, my next creative writing teacher recommended A Field Guide to Getting Lost to all of us new graduates. I read it, my first Rebecca Solnit book, at the end of March this year. I immediately wanted to read everything Solnit had ever written, and shipped The Faraway Nearby home so my mother could bring it with her to France. I’m 50 pages from the end now and, like all her books, it’s a wondrous journey across many geographies, stories, and histories.
“Fairy tales are children’s stories not in who they were made for but in their focus on the early stages of life, when others have power over you and you have power over no one.
In them, power is rarely the right tool for survival anyway. Rather the powerless thrive on alliances, often in the form of reciprocated acts of kindness – from beehives that were not raided, birds that were not killed but set free or fed, old women who were saluted with respect. Kindness sewn among the meek is harvested in crisis.”
From the internet
The LSE Impact Blog shared a fun way to spread academic learnings to a wider audience: comics!
“By turning research into an illustrated abstract it’s possible to make academic work more accessible to a non-technical audience and people outside of the discipline. Alpa and Karen have even been approached by an author who would like to include the image in a forthcoming anthropology textbook, while Alpa hopes the article will also reach students, other academics and even non-academics, to explain the value of anthropology and long-term participant observation.”
My mother and I listened to “Episode 77: Kalevala (with Elena Varg)” from Spirits on the train to Versailles. The Finnish epic is bonkers, featuring birds nesting on the limbs of goddesses, the Devil’s own personal petting zoo, badass mothers, and a river of death. Elena Varg’s accent is wonderful and her excitement for the story makes this one of my favorite Spirits episodes ever.
Haribos Sour Rainbow Strips are one of a variety of rainbow-colored, sour/sugar-coated gummy candies, nearly all of which I love. While other brands can be too licorice-flavored, gummy, sweet, or unwieldly, Haribos’ colors can easily be stripped apart and each belt of tart candy is mouthwatering. Unfortunately unavailable in Nairobi. I should have brought more than one bag back from France, since I’ve already devoured the one I did bring.
Watch Them Again
You can really see the drawings behind the animation in the original Lion King movie. Rafiki is more bonkers than I remembered, and Nala a more beautiful lioness.
Also watched some Modern Family in the evenings with my mother on our vacation. Like most family drama/comedy shows I love (see also Reba), everyone makes mistakes and is flawed but they choose to do the right thing in the end. Heartwarming and cozy, with lots of silly in between.
I was reading about the new African journal – Scientific African – that will cater specifically to the needs of African scientists. Awesome!
Among the advantages of the new journal is the fact that “publication in Scientific African will cost $200, around half of what it costs in most recognised journals.”
You have to pay to be published in an academic journal? Dang.
I guess that cost is probably built into whatever research grant you’re working on, but in most other publications, I thought writers got paid to contribute content. I guess it’s so that there’s not a direct incentive to publish as much as possible, which could lead to more falsified results? Although it seems like the current model has a lot of messed up incentives, too.
I’ve only hiked one brief section of the Appalachian Trail. It was just over the NC border in Tennessee, it was steep, and it was beautiful.
I’ve been hiking twice in my now 7 months of living in Nairobi. One of those hikes lasted 2.5 hours longer than it should have and was essentially straight up and then straight down a river of mud – and I got altitude sickness.
And yet I’ve gotten a strong itch to hike the entire Appalachian Trail as a thru hiker.
Thru-hiking the Appalachian Trail, all the way from Georgia to Maine, has always appealed to me. But I’ve never seriously thought about training for it and fitting it into my plans. I want to do so much (I want to do about 10 different grad programs), and taking 5 months “off” for an expensive trek in the wilderness isn’t very convenient.
I grew up in a well-treed town, with a whole network of trails and a creek just behind my house. I went to college in a small town nestled among the Green Mountains and the Adirondacks. I miss the East Coast trees and mountains. Trekking through that landscape, getting intimate with the dirt and rocks and trees, getting offline – that sounds like the best way to return home.
Plus, I think it’d be pretty badass to carry your life on your back for 2,181 miles.
Hiking the AT requires a lot of prep and money. It costs about $5,000 not including gear. And it takes about 5 months: April-August if you’re hiking from Georgia to Maine (NOBO) or June/July-October/November if you’re hiking south (SOBO).
Most people hike NOBO, ending the hike with the most challenging sections in the White Mountains before summiting Mt. Katahdin for the finish. So many people hike the AT every year now that the NOBO route can be pretty crowded. As of May 16, 2,508 people were registered to thru-hike NOBO, while only 218 were registered for SOBO hiking.
The idea of doing SOBO appeals to me just on that basis, but I’d have to weigh a lot of variables carefully before choosing.
The timeline alone makes SOBO a difficult choice. Unless I’m going to do it starting next July. Which I am wholly unqualified to do.
I have never been backpacking. I’m working on planning my first trip soon, but I’d need to go on plenty of longer trips before heading off for 5 months. (Assuming I like backpacking!)
I’d also want to get trained in wilderness first aid, wear in my gear beforehand, and coordinate how friends can join me for sections of the trail. I’m fine to hike alone if no one else wants to take 5 months off to join me, but I’d want my favorite humans to join for at least a little bit each!
I know my father and brother would join for at least a week each. And the boyfriend. And I have several friends who I could probably convince to do a few weeks each.
Fitting the hike into my own schedule will be hard enough… although the NOBO timing probably works pretty well for right before starting a grad school program.
It’s likely that I won’t end up thru-hiking the AT. I’ve never taken such a big risk to do something so outside the normal career path. But I’m going to start doing some backpacking, just in case I fall in love.
I recently went through a long period of time (about 5/6 weeks) where I wasn’t running and I wasn’t even going to volleyball.
When I miss a planned workout or opportunity to exercise, I get a lot of mixed feelings. I feel good that I’m letting my body rest or, as is often the case, investing time in my personal projects and intellectual growth instead. But I also feel guilty about not exercising, and then guilty for feeling guilty – like I’m doing a bad job being body-positive or self-loving enough.
And then I feel guilty about feeling guilty about feeling guilty because I shouldn’t expect myself to be 100% self-loving all the time. So it’s rare for me to actually feel guiltless about missing exercise – usually, I’m judging myself under the surface and just ignoring those feelings.
I also know that if I miss one opportunity, I’m more likely to skip the next as well. I think that’s in large part because of the guilt cycle I get into. I start to feel defiant, like I shouldn’t even have to “force” myself to exercise if I’m not feeling into it. Because I’ve always been an athlete on sports teams, I’ve rarely had to make the decision to work out myself – it was simply demanded of me by my responsibilities to the team. Now, on my own, I have to actively make the choice to exercise each time and that’s way harder than I expected it to be.
The defiance also comes from the feeling that I don’t want societal pressures to look a certain way, or pressures I’ve put on myself to be seen as an athlete, to control my actions. It’s good to be aware of those pressures, but if I react so strongly in the opposite direction, am I acting any more independently?
Ultimately, I believe it’s good to take breaks and not beat myself up too much about “skipping days.” But it’s also really important to me to exercise. Exercise has a lot of benefits past my physical strength. It makes me happier, more energized.
The first challenge is recognizing when I’m taking a healthy break vs. feeling defiant and making an unhealthy, guilt-driven choice. Although, there might not always be a strict division between those two mentalities.
The second challenge is choosing the best action: do I “force” myself to do some kind of exercise even if I’m not really feeling it, so that I can have more motivation to work out tomorrow? How can I ensure I do workout the day after I take a break?
It’s a lot easier to think through these issues when I have been exercising. When I haven’t, I feel pessimistic about my ability to fix the problems. When I start exercising again, it’s more obvious that I can fix them – hey, I already started working out again, right?
This week, I ran on Sunday and Monday, played volleyball Monday night, and went rock climbing on Tuesday. That feels good, although I’m a bit worried I’ve swung too far in the other direction. Today, I’m planning to just go on a chill, short run when I get home from work or do a yoga session, depending on the rain situation.
I’m proud of myself for ending the exercise drought. And now I’m trying to figure out a strategy for getting back at it earlier in the future.
Two weeks ago, I had my first opportunity to climb outdoors. My friends were going to Hell’s Gate – the national park two hours outside Nairobi that inspired much of the Lion King film.
I have been afraid of heights my whole life. That fear is one of the main reasons I’ve never gone rock climbing outdoors. In a rock climbing gym, the controlled environment feels like a pretty safe space to dangle from a rope two to three stories up. But when I get near the edge of a cliff, I feel like I suddenly have to fight the urge to leap into the void.
This really freaked me out when I was younger, even though the fear was tinged with a sense of exhilaration, too.
Aside: Recently (read: five minutes ago), I learned this urge is called the “high place phenomenon.” In one study on the feeling, researchers found the desire to jump wasn’t correlated with suicidal thoughts and was instead correlated with “anxiety sensitivity.” Anxiety sensitivity is essentially how anxious being anxious makes you – specifically how reactive you are to the physical sensations of your body telling you it’s in danger (like the quaking in your knees as you look over a cliff).
It was one of the most beautiful days I have ever experienced. It had been rainy all week (we’re just wrapping up the rainy season in Kenya), but the day we were climbing was all sunshine and scudding clouds.
We hired a climbing guide to set up two top ropes so that we could belay ourselves after that. We had one easier climb and one harder one. Later, another climber showed up and set up another climb and we moved the easier rope to another wall. I was able to try all four and got to the top of three. The fourth had an overhang and was the last one I attempted. I got my arms onto the overhang but couldn’t haul myself over the top that late in the day.
After we “cleaned” the routes (removed the equipment we had used for top-roping), we drove through the rest of the park to reach some sulphur hot springs on the opposite side. The whole landscape was wide open and gorgeous.
At one point a giraffe just started running alongside our car. It was magical.
10/10 experience and made me really want to climb more!
Andrew Gelman’s recent blog post responding to a Berk Özler hypothetical about data collection costs and survey design raised a good point about counterfactuals that I theoretically knew, but was phrased in a way that brought new insight:
“A related point is that interventions are compared to alternative courses of action. What are people currently doing? Maybe whatever they are currently doing is actually more effective than this 5 minute patience training?”
It was the question “What are people currently doing?” that caught my attention. It reminded me that one key input for interpreting results of an RCT is what’s actually going on in your counterfactual. Are they already using some equivalent alternative to your intervention? Are they using a complementary or incompatible alternative? How will the proposed intervention interact with what’s already on the ground – not just how will it interact in a hypothetical model of what’s happening on the ground?
This blogpost called me to critically investigate what quant and qual methods I could use to understand the context more fully in my future research. It also called me to invest in my ability to do comprehensive and thorough literature reviews and look at historical data – both of which could further inform my understanding of the context. And, even better, to always get on the ground and talk to people myself. Ideally, I would always do this in-depth research before signing onto the kind of expensive, large-scale research project Özler and Gelman are considering in the hypothetical.
Academic writing is full of bad habits. For example, using words like “obviously,” “clearly,” or “of course.” If the author’s claim or reasoning really is obvious to you, these words make you feel like you’re in on the secret; you’re part of the club; you’ve been made a part of the “in” group.
But when you don’t know what they’re talking about, the author has alienated you from their work. They offer no explanation of the concept because it seems so simple to them that they simply won’t deign to explain themselves clearly to those not already “in the know.”
Part of an academic’s job is to clearly explain every argument in their papers. It is lazy and exclusionary to imply readers should already understand a concept or a path of reasoning.
At worst, it just makes you sound rude and superior:
“Advertising is, of course, the obvious modern method of identifying buyers and sellers.” – Stigler, “The Economics of Information”
He really doubled-down on how evident this fact is, which only tells the reader how smart he thinks he is. The sentence could have read, “Advertising is the preferred modern method of identifying buyers and sellers,” and could have included a citation.
On the other hand, a non-exclusionary use of “obviously”:
“Obviously, rural Ecuador and the United States are likely to differ in a large number of ways, but the results in this (and other recent) papers that show a shifting food Engel curve point to the risks inherent in assuming that the Engel curve is stable.” – Shady & Rosero paper on cash transfers to women
The authors had previously compared two papers from two very different contexts; they use “obviously” to acknowledge the potential issues with comparing these two settings. This is an acceptable use case because the statement that follows actually is obvious and is bringing any reader on board by acknowledging a possible critique of the argument. It is an acknowledgement of possible lack on the author’s part, rather than a test of the reader’s intelligence or prior knowledge.
It was my friend Jayne’s sister’s wedding this weekend.
we performed our dances, then danced the rest of the night even when the lights went out and it was pouring rain.
I knew so many of the songs and it was so great to dance to them again (including ones I danced to with Midd Masti! – the Humma Song and Nachde me Saare). The dance’s choreographer and cousin to the bride said he’s going to quit his life as a banker and come back to Nairobi and start a dance crew. Vas and I signed up to be the first students if it ever really happens.
I would love to spend all my free time learning Bollywood dances.
Vas, Maddy, and me at the dancing/cocktail party on Sunday.
On Monday night…
we went to the Oshwal Centre for a quick greet with the families (lots of namaste-ing to adorable teeny grannies in beautiful saris) and then a thali-style dinner. The pistachio burfi was so good.
Me, Vas, Jayne (sister of the bride), and Maddy after dinner Monday night. I’m always the short one in volleyball team pics but the towering giant in pics with other friends.
The first night, we wore kurtis for the dancing and on Friday everyone was in a mix of Indo-Western styles.
was a public holiday in Kenya, and the date of the final ceremony. We started our morning early with some ball gowns in Java.
We drove out into the countryside to the gorgeous Fuscia Gardens our among the tea farms. First, we welcomed the groom’s party with dancing and also the bride’s family stole his shoes?! He’s supposed to pay to get them back but I’m not sure how that worked out in the end. “London Thumakda” got way stuck in my head, and we had a reprisal of Bollywood “Shape of You.”
The bride’s family blessed the groom and his family as they entered – during the blessing, the bride’s auntie tried to grab the groom’s nose, like you play when you’re little with your parents. He couldn’t move away, but his friends and family would pull him away if she got close. Whether she grabbed it was supposed to symbolize whether the bride’s family will have much say in the bride’s new married life.
Then we had chai (mmmmm) and noms, including poori, the best breakfast food of all time. There was a break as we waited for the bride to officially arrive, so we took advantage of the ominous clouds and our swirly dresses.
There was a cute moment after the bride’s party officially entered where the groom was “revealed” to the bride by popping up from behind a banner before both were seated on the ceremony dais. They were draped in enormous garlands of white and pink flowers and a thin chain that went over both their heads, linking them together. The bride’s skirt was also tied to a piece of cloth draped over the groom’s neck.
There were a lot of prayers and throwing things into the fire and tying their hands together during the ceremony. I couldn’t quite follow since it was in Gujarati. Got some good Hindi word recognition practice out of it though!
The most famous and exciting part of the ceremony was when the bride led the groom around the fire. She led him three times, each time greeting and accepting blessings from different family members: her parents, her father’s eldest relatives, her mother’s eldest relatives. The fourth and final time around the fire, he led her and they were greeted and blessed by the groom’s family.
Before each circle around the fire, there were prayers and the couple threw rice into the flames. After each circle, the man presiding over the ceremony would count to three and they would sit as fast as possible – the one who sat first would signify who would have more control over the household.
The whole wedding was full of fun and silly traditions, vibrant colors, and lots of involvement from all friends and family.
Two weekends ago, I went on one of the most beautiful hikes I’ve ever been on – Elephant Hills in the Aberdares.
Unfortunately, the hike was 3 hours longer than advertised, we ran out of water, we had to hike in the dark (with hyenas??), and I got altitude sickness and threw up. Otherwise, best hike ever.
“The Aberdares? In the rainy season? Are you crazy?!” – My boss when she heard about our trip at work the next day
Mikala found a group hiking Elephant Hills – a hike we’d all heard was crazy beautiful – and invited Brooke and me. I added Nick and Alice from work to the hiking crew. Women Who Hike Kenya organized the buses, park fees, park rangers, and “personal photographer” (which is why I have so many pics from the hike – I didn’t get my phone out to take any pics until we reached the peak).
The best section going up and down was the bamboo forest we passed through. On the way up, we thought this was the middle third of the hike up – it was more like one of the middle tenths of the climb.
We started out by criss-crossing a muddy road, then followed the road through potato farms, through a pine forest, through an electric fence to keep the game out, through more trees, and along another open grassy section before reaching the bamboo part. On the way down, these pre-bamboo sections all merged into one quick burst in our minds… instead, it kept going and going and going. The bamboo section was just a vertical shoot up a mud slide between gorgeous bamboo shoots – lots more falling.
After the long bamboo section, we took a quick break on a grassy knoll. A bunch of us thought it was the elephant’s head, so we ate all our lunch. But then on we went.
The next section was just a muddy stream of squelching mud sprinkled with safari ants (vicious biters, but I thankfully wasn’t bit). We also thought this was the last section…
…until we arrived at another grassy section leading us up into the clouds.
After that, I started feeling really sick and decided I had to just power through to the peak. The clouds faked us out at least 5 or 6 times before I finally made it. As soon as I reached the top, the clouds parted and we were treated to incredible 360 views. I was too busy dying of altitude sickness to notice at first. I recovered a bit and ate some of my leftover “I’m not going to Liberia for two months!”chocolate cake from Java.
I was desperate to get down from altitude (about 12,500 ft at the peak, up nearly 3000 ft from the trailhead), so I tried to keep up with Alice and Nick. They hadn’t been feeling the altitude sickness like Brooke, Mikala, and me. Pretty quickly, though, I had to stop and almost started crying my head hurt so much. I let the rest of the summiters pass me by until Brooke and Mikala reached me and rallied my spirits.
Still, Mikala and I were hit pretty hard and it was slow going. Meanwhile, Brooke was calculating how long it would take to get down, comparing that timeline to when it would get dark, and debating whether it was better to push us to go faster when we were feeling so crappy or to be in a national park with wild animals after dark. Actually, at that point, we were more worried about being in that dense, slippery bamboo in the dark.
My head was pounding, I felt hopeless but knew I had to keep going, and my legs were quaking. I’ve never seen my legs shake as hard as they did each time I paused to take a sip of water or breath more deeply. I’d been sitting on the idea that I needed to throw up for about 15 minutes when I finally sped up a bit, turned off the trail and puked. After that, I felt AWESOME. My legs were still shaking, but now my head wasn’t pounding.
We made it to the first grassy knoll where we’d eaten lunch. Happily, another group of friends on the hike had over-prepared with extra gatorades and lent us some. We refueled and then plunged back into the bamboo forest.
We had some great jungle-woman moments slipping and sliding down the increasingly dimly lit bamboo section. The bamboo on the edge of the path were key; we swung between them rather than trust our weight on the muddy slope.
By the time we made it to the end of the bamboo, we were euphoric and wanted to power through the final spurt. But by the time we hit the pine tree forest again, it was already dark. At that point, one of the more experienced hikers who had done Elephant Hills seven or eight times before started getting antsy. He kept hiking super fast but also yelling at the few stragglers to stay with the group and warning everyone about hyenas.
At that point, though, I was actually in a great mood. I had a stick to wave menacingly at the dark edges of the path, I wasn’t at a crazy altitude anymore, and I knew where we were and how to get back.
In all, we hiked 12 miles, straight up and straight down through deep mud. No switchbacks in the Aberdares, apparently! I was out on the trail from 9 am to 7 pm.
10/10 would do again… but maybe only through the bamboo section. And probably not in the rainy season!
Also adding videos of the final dances and more pics soon.
Today was the first day I used my new weekly professional development time block.
I had hoped to spend 2 hours reading and summarizing an academic article, and then 1 hour researching geospatial data in Africa.
Instead, I spent all three hours summarizing and digesting some research on grounded theory methodology I did two weeks ago. I had hoped to accomplish that writing and then read and write about an academic paper on GT and economics.
One of my PD goals is to track my predictions of how I will spend my time and evaluate whether those predictions were accurate. In this case, not so much. I feel good about how I spent the time today. I learned a lot by reviewing and digesting the notes I had previously taken.
I learn a lot every time I try to describe something I’ve read. Not sure if I will write Grounded Theory, Part 2: Grounded Theory & Economics next week during my PD session, or if I will use it for something else. Another related topic I want to understand is what the difference between heterodox and traditional economics really is.
Maybe some of it will happen in my PD session, maybe I’ll do one or both in my own time throughout the week and use next week for looking into the GIS/African data thing. TBD, but I feel good about my time spent this morning.
Brené Brown quotes this poem by 20th century Spanish poet Antonio Machado in describing her research methodology.
I recently read Brené Brown’s Daring Greatly. The book presents Brown’s research, but it can feel more like a personal guidebook to tackling issues of vulnerability and shame.
Because the research has a conversational feel, it’s hard to understand how much of the book is based in research and how much in Brown’s individual experiences. She weaves in personal stories frequently, often to demonstrate a prickly emotional experience that was common across her interviews. But when I reached the end of the book, I wanted to know how she drew these theories from the data. I’ve only worked sparingly with qualitative data: how does one “code” qualitative data? How do you analyze it without bringing in all sorts of personal biases? How do you determine its replicability, internal and external validity, and generalizability?
Ingeniously, Brown grounds the book in her research methods with a final chapter on grounded theory methodology. Her summary (also found online here) was a good introduction to how using grounded theory works and feels. But I still didn’t “get” it.
So I did some research.
Brown quotes 20th century Spanish poet Antonio Machado at the top of her research methods page:
“Traveler, there is no path. / The path must be forged as you walk.”
This sentiment imbued the rest of the grounded theory (GT) research I did. Which seemed bizarre to a quant-trained hopeful economist. I’m used to pre-analysis plans, testing carefully theorized models, and starting with a narrow question.
Grounded theory is about big questions and a spirit of letting the data talk to you.
Founded by Barney Glaser and Anselm Strauss in 1967, GT is a general research methodology for approaching any kind of research, whether qual- or quant-focused. When using GT, everything is data – your personal experiences, interviews, mainstream media, etc. Anything you consume can count, as long as you take field notes.
Writing field notes is one of the key steps of GT: coding those notes (or the data themselves – I’m still a little blurry on this) line-by-line is another. The “codes” are recurring themes or ideas that you see emerging from the data. It is a very iterative methodology: you collect initial data, take field notes, code the notes/data, compile them into memos summarizing your thoughts, collect more data based on your first learnings, code those, compile more memos, collect more data…
Throughout the whole process, you are theorizing and trying to find emergent themes and ideas and patterns, and you should actively seek new data based on what your theories are. You take a LOT of written notes – and it sounds like in the Glaserian tradition, you’re supposed to do everything by hand. (Or is it just not using any algorithms?)
Brown describes the data she collected and her coding methodology:
“In addition to the 1,280 participant interviews, I analyzed field notes that I had taken on sensitizing literature, conversations with content experts, and field notes from my meetings with graduate students who conducted participant interviews and assisted with the literature analysis. Additionally, I recorded and coded field notes on the experience of taking approximately 400 master and doctoral social-worker students through my graduate course on shame, vulnerability, and empathy, and training an estimated 15,000 mental health and addiction professionals.
I also coded over 3,500 pieces of secondary data. These include clinical case studies and case notes, letters, and journal pages. In total, I coded approximately 11,000 incidents (phrases and sentences from the original field notes) using the constant comparative method (line- by- line analysis). I did all of this coding manually, as software is not recommended in Glaserian-grounded theory.” [emphasis mine]
The ultimate goal is to have main concepts and categories emerge from the data, “grounded” in the data, that explain what main problem your subjects are experiencing and how they are trying to solve that problem. For example, Brown’s work centers on how people seek connection through vulnerability and try to deal with shame in various health and unhealthy ways. She started with this big idea of connection and just started asking people about what that meant, what issues there were around it, etc. until a theory started to arise from those conversations.
You’re not supposed to have preexisting hypotheses, or even do a literature review to frame specific questions, because that will bias how you approach the data. You’re supposed to remain open and let the data “speak to you.” My first instinct on this front is that it’s impossible to be totally unbiased in how you collect data. Invariably, your personal experience and background determine how you read the data. Which makes me question – how can this research be replicable? How can a “finding” be legitimate as research?
My training thus far has focused on quantitative data, so I’m primed to preference research that follows the traditional scientific method. Hypothesize, collect data, analyze, rehypothesize, repeat. This kind of research is judged on:
- Replicability: If someone else followed your protocol, would they get the same result?
- Internal validity: How consistent, thorough, and rigorous is the research design?
- External validity: Does the learning apply in other similar populations?
- Generalizability: Do the results from a sample of the population also apply to the population as a whole?
GT, on the other hand, is judged by:
- Fit: How closely do concepts fit the incidents (data points)? (aka how “grounded” is the research in the data?)
- Relevance: Does the research deal with the real concerns of participants and is it of non-academic interest?
- Workability: Does the developed theory explain how the problem is being solved, accounting for variation?
- Modifiability: Can the theory be altered as new relevant data are compared to existing data?
I also read (on Wikipedia, admittedly), that Glaser & Strauss see GT as never “right” or “wrong.” A theory only has more or less fit, relevance, workability, or modifiability. And the way Brown describes it, I had the impression that GT should be grounded in one specific researcher’s approach:
“I collected all of the data with the exception of 215 participant interviews that were conducted by graduate social-work students working under my direction. In order to ensure inter-rater reliability, I trained all research assistants and I coded and analyzed all of their field notes.”
I’m still a bit confused by Brown’s description here. I didn’t know what inter-rater reliability was, so I had assumed it meant that the study needed to have internal consistency in who was doing the coding. But when I looked it up online, it appears to be the consistency of different researchers to code the same data in the same way. So I’m not sure how having one person do all of the research enables this kind of reliability. Maybe if your GT research is re-done (replicated) by an independent party?
My initial thoughts are that all GT research sound like they should have two authors that work in parallel but independently, with the same data. Each develops separate theories and then at the end, the study can compare the two parallel work streams to identify what both researchers found in common and where they differed. I still have a lot of questions about how this works, though.
A lot of my questions are functional. How do you actually DO grounded theory?
- How does GT coding really work? What does “line-by-line” coding mean? Does it mean you code each sentence or literally each line of written text?
- Do these ever get compiled in a database? How do you weight data sources by their expertise and quality (if you’re combining studies and interviews with average Joes, do you actively weight the studies)? -> Can you do essentially quantitative analysis on a dataset based on binary coding of concepts and categories?
- How do you “code” quantitative data? If you had a dataset of 2000 household surveys, would you code each variable for each household as part of your data? How does this functionally work?
- If you don’t do a literature review ahead of time, couldn’t you end up replicating previous work and not actually end up contributing much to the literature?
And then I also wondered: how is it applicable in my life?
- Is GT a respected methodology in economics? (I’d guess not.)
- How could GT enhance quant methods in econ?
- Has GT been used in economic studies?
- What kinds of economic questions can GT help us answer?
- Should I learn more about GT or learn to use it in my own research?
Coming up: Part 2, Grounded Theory & Economics
To answer some of my questions, I want to do an in-depth read of a paper from the 2005 Grounded Theory Review by Frederick S. Lee: “Grounded Theory and Heterodox Economics.” (The journal has another article from 2017 entitled “Rethinking Applied Economics by Classical Grounded Theory: An invitation to collaborate” by Olavur Christiansen that I hope to read, too.)
I wanted to read more in 2018. I also wanted to read some classic lit that my education has neglected to this point.
So I decided to read 52 books in 2018. I’m at 12, including a few books from 2017 I snuck in there since I read them after deciding to do the challenge.
|Week||Title||Author||Date Started||Date Finished|
|Dec 31||Lolita||Vladimir Nabokov||Oct. 15, 2017||Dec. 13, 2017|
|Jan 7||American Gods||Neil Gaiman||Nov. 16, 2017||Nov. 19, 2017|
|Jan 14||The Curious Incident of the Dog in the Night||Mark Haddon||Nov. 23, 2017||Nov. 25, 2017|
|Jan 21||Jane Eyre||Charlotte Bronte||Dec. 17, 2017||Dec. 20, 2017|
|Jan 28||Woman at Point Zero||Nawal El Saadawi||Dec. 21, 2017||Dec. 22, 2017|
|Feb 4||Doing Good Better||William MacAskill||Jan. 28, 2018||Feb. 10, 2018|
|Feb 11||Turtles All the Way Down||John Green||Feb. 1, 2018||Feb. 1, 2018|
|Feb 18||The Fifth Season||N. K. Jemisin||Feb. 2, 2018||Feb. 3, 2018|
|Feb 25||La Belle Sauvage||Philip Pullman||Feb. 9, 2018||Feb. 9, 2018|
|Mar 4||Red Queen||Victoria Aveyard||March 16, 2018||March 17, 2018|
|Mar 11||A Field Guide to Getting Lost||Rebecca Solnit||March 29, 2018||April 3, 2018|
|Mar 18||Daring Greatly||Brene Brown||April 4, 2018||April 14, 2018|
|Mar 25||The Power||Naomi Alderman|
|Apr 1||The Unbearable Lightness of Being||MIlan Kundera||April 16, 2018|
|Apr 8||The Girls (?)||Emma Cline||April 7, 2018|
|Apr 15||Poor Economics||
Esther Duflo & Abhijit Banerjee
|Apr 22||Thinking, Fast and Slow||Daniel Kahneman||Feb. 18, 2018|
|Apr 29||Her Body and Other Parties||Carmen Maria Machado|
|May 6||The Faraway Nearby||Rebecca Solnit|
|May 13||Wanderlust||Rebecca Solnit|
|May 20||1984||George Orwell|
|May 27||If On a Winter’s Night A Traveler||Italo Calvino|
|Jun 3||A Room of One’s Own||Virginia Woolf|
|Jun 10||Franny and Zooey||J. D. Salinger|
|Jun 17||Capital in the Twenty-First Century||Thomas Picketty|
|Jun 24||To the Lighthouse||Virginia Woolf|
|Jul 1||The Hitchhiker’s Guide to the Galaxy||Douglas Adams|
|Jul 8||Notorious RBG: The Life and Times of Ruth Bader Ginsburg||Irin Carmon|
|Jul 15||The Core of the Sun||Joanna Sinisalo|
|Jul 22||The Dead||James Joyce|
|Jul 29||The Ethical Slut: A Guide to Infinite Sexual Possibilities||Dossie Easton|
|Aug 5||Midnight’s Children||Salman Rushdie|
|Aug 12||Catch-22||Joseph Heller|
|Aug 19||Anna Karenina||Leo Tolstoy|
|Aug 26||Mountains Beyond Mountains||Tracy Kidder|
|Sep 2||The Metamorphosis||Kafka|
|Sep 9||Slaughterhouse Five||Kurt Vonnegut|
|Sep 16||Song of Solomon||Toni Morrison||Dec. 20, 2017|
|Sep 23||Why Nations Fail||Acemoglu & Robinson|
|Sep 30||Neverwhere||Neil Gaiman|
|Oct 7||Lord of the Flies||William Golding|
|Oct 14||Daughters of the North||Sarah Hall|
|Oct 21||The Captain Class (?)||Sam Walker|
|Oct 28||The Obelisk Gate||N. K. Jemisin|
|Nov 4||The Stone Sky||N. K. Jemisin|
|Nov 11||Binti||Nnedi Okorafor|
|Nov 18||As Eve Said to the Serpent||Rebecca Solnit|
|Nov 25||The Mother of All Questions||Rebecca Solnit|
|Dec 2||Hope in the Dark||Rebecca Solnit|
|Dec 9||The Challenge for Africa||Wangari Maathai|
|Dec 16||A Paradise Built in Hell||Rebecca Solnit|
|Dec 23||The Signature of All Things (?)||Elizabeth Gilbert|
Will most likely keep moving things around and taking books on and off the list, but it’s an outline.
In total, I read 37 books over my 2018+ “year” of reading. An awesome increase from previous years.
The additional books I ended up reading:
|The Faraway Nearby||Rebecca Solnit||June 8, 2018|
|The Two Princesses of Bamarre||Gail Carson Levine||June 12, 2018|
|The Power||Naomi Alderman||July 10, 2018|
|Love in the Time of Dragons||Katie MacAlister||July 11, 2018|
|The Story of O||Anne Desclos||July 14, 2018|
|The Obelisk Gate||N. K. Jemisin||July 15, 2018|
|Her Body and Other Parties||Carmen Maria Machado||July 23, 2018|
|Glass Sword||Victoria Aveyard||July 31, 2018|
|What Got You Here Won’t Get You There||Marshall Goldsmith||August 5, 2018|
|The Stone Sky||N. K. Jemisin||August 8, 2018|
|Who Fears Death||Nnedi Okorafor||August 16,2018|
|Binti||Nnedi Okorafor||August 18, 2018|
|What Sunny Saw in the Flames||Nnedi Okorafor||August 21, 2018|
|Lagoon||Nnedi Okorafor||August 23, 2018|
|On Writing||Stephen King||September 3, 2018|
|Lady Chatterley’s Lover||D. H. Lawrence||September 16, 2018|
|Bad Feminist||Roxane Gay||September 26, 2018|
|The Wife||Meg Wolitzer||September 26, 2018|
|An Untamed State||Roxane Gay||September 29, 2018|
|Avalon High: The Merlin Prophecy||Meg Cabot||September 29, 2018|
|Kingdom of Ash||Sarah J. Maas||November 4, 2018|
|Succubus Dreams||Richelle Mead||November 25, 2018|
|King’s Cage||Victoria Aveyard||December 1, 2018|
|Deep Work||Cal Newport||December 8, 2018|
|The Power of Habit||Charles Duhigg||December 18, 2018|
CW: sexual assault
Partners with different sex drives
This week, one of my favorite podcasts – AdultSh1t from Kate Peterman and Kelsey Darragh of Buzzfeed – answered a question from a listener whose sex drive is much higher than that of her long-time girlfriend. The listener said she felt bad and uncomfortable asking for more sex because she doesn’t want to feel “rape-y” or like she’s pressuring her girlfriend into doing sexual things she doesn’t want to do. But she’s not getting the sexual fulfillment she needs right now.
Kate and Kelsey advise even more communication, but also to make sure you’re getting what you need in the relationship. I would add a few things to what they shared:
- Start the conversation from a point of, “I love you and I want to figure this out together. We seem to need different things and I want to understand why you need what you need and explain why I need what I need.” It is super scary to start a conversation with a long-time partner that you know could lead to some really painful and possibly break-up-inducing discussions. Phrasing it like you’re tackling an issue together makes it seem less intimidating to get into it at a deeper level.
- Be careful to not attack her perspective/experience. I think this is especially challenging because there’s an assumption that everyone wants to have sex inherent in U.S. culture. Being sensitive to the fact that she might be ready to get defensive about that assumption (just like you could be sensitive about the assumption that having a higher sex drive is “sinful”) can help. Avoid approaching it like there’s a problem with her. It’s a problem between the two of you.
- Share that you don’t want to feel like you’re pressuring her, that you’re struggling to approach the issue, and that that’s a point of pain for you. Specifically share that it can make you feel unwanted, unsexy, and confused. Maybe this can lead to her sharing ways that you can introduce sexy times without it being pressuring, or what specific language y’all can use to discuss whether to get it on in the future. Also, ask her to tell you how these situations make her feel – is she uncomfortable? Annoyed? Sad? Frustrated? It might also be hard for her to know she’s not fulfilling all your sexual needs. My good friend Annaji introduced me to the powers of the “I” statement. If you keep it about how you feel, that acknowledges there’s room for misinterpretation and offers the other person space to clarify where they’re coming from, too.
- Once you’ve both been able to share how y’all are feeling about your sex life, then you can talk about next steps more freely and as a team. Maybe you can brainstorm some solutions or compromises. Maybe, even though it’s scary, you both will find you really do need a partner whose sex drive matches your own more closely. Even though it’s a tough decision to make after so many years together.
I was once in a relationship where I found out almost 2 years in that my partner was unhappy with how much sex we’d been having. He felt like we had sex too often. He told me that our intimacy cut into his time to do personal projects that were important for him to be fulfilled. This was partially because we would sleep over in each others’ dorm rooms a lot and go to bed early, cutting into hours he would have previously used to create.
It was really painful to hear that he felt pressure to have sex, even if he said that pressure wasn’t coming from me so much as from society. I still worry that he did feel pressure from me. Maybe, as I discuss below, he just couldn’t tell me he was uncomfortable.
We did break up shortly after this revelation, which came at a complicated time in our relationship for other reasons. I’m not sure how we would have dealt with our mismatched sex drives given more time. Thankfully, we still have a good relationship as friends. And I think that’s in large part due to the fact that we did have honest conversations about our sex life before the break-up.
Men’s experiences: the same, but different
This episode also deepened my thinking on the Beautiful / Anonymous episode I shared earlier this week, where a man called in to talk about how he was sexually assaulted three times in his life, twice by women. I had a moment at work today to discuss it with my colleague who recommended the episode. One of our big takeaways was that it was amazing how familiar the caller’s description of his doubt, shame, and struggle sounded.
Most women have heard female friends’ stories of sexual assault or experienced it themselves. Through most stories, there are common threads: feeling unable to react fast enough or strongly enough, doubting whether we truly couldn’t have done anything, wondering if the other person genuinely thought it was okay and just missed our signals, whether the signals we sent were strong enough. All of that self-doubt I have heard expressed by numerous women – it was a revelation to hear a man share the same doubts.
Anyone can have their bodily autonomy and safety violated. Even if you know that in your brain, hearing this man’s story makes it stick in your soul in a new way. I think all of us still carry with us ingrained messaging about men’s relationship to intimacy, sex, and violence. Those narratives lead us to make assumptions that can really hurt other people; this podcast invites us (women) to examine our part in perpetuating these assumptions about gender and sex.
It’s hard to think about, because it really complicates narratives that are easier to keep clean cut. Women are victims, men are violent, sex-driven animals. We (feminists) know it’s a gross simplification, but it’s still so tempting when the statistics are that most victims of sexual violence are women, and most perpetrators are men. That narrative can even be comforting/validating on some level for women who are violated in that way – you’re not alone, this happened to us too, we’re here for you, it’s not your fault.
While the male caller shared these same doubts and feelings of shame and guilt as so many women, the way he expressed the doubts was also telling. My colleague and I both marked that he was bewildered by those feelings and without a narrative to put them into that accounted for both his masculinity and his vulnerability.
My female friends and I, on the other hand, can see how our own experiences fit into the larger societal phenomenon of violence against women. Each of our experiences is intensely personal and can feel isolating. Yet when we’ve been able to talk to each other about those experiences, we can wryly see it as part of “womanhood” in our culture. A terrible yet shared burden. We all contribute a piece to the larger narrative. And, we can also share in the new narratives that are rising about self-care, how to survive & thrive after sexual violence, how to find sisterhood in this massive, horrifying phenomenon.
It is both to men’s privilege and to the caller’s disadvantage that he is not part of this narrative. This podcast called me to think about ways in which I perpetuate bad myths about gender, sex and violence, and ways in which I can bring non-female survivors of sexual assault into the sisterhood component of the narrative I hold.
So I was phoneless in Kenya…
After my iPhone took a dive onto the floor of my shower room (showers have their own rooms in Nairobi), it finally powered off for good.
The shower incident was this particular phone’s second aquatic misadventure, and despite a full recovery that lasted nearly a month, it stopped charging properly and I judged it good and dead this time.
So I was phoneless in Kenya, where people buy SIM cards and phones separately, and don’t buy longterm contracts like we do in the States. This meant that a new iPhone would cost out-of-pocket nearly $1000.
I had to get some kind of phone, though, because Uber or Taxify apps are really the only way to get around without a car, especially at night, unless you have a “guy,” a word which here means someone you trust to drive you around at all hours and who can usually take your calls.
People have a different “guy” (see also: “fundi”) for every need. There’s the chair fundi, the boda guy (boda = motorcycle), and even the guy guy, who can find you the right guy for whatever needs getting done.
At the Safaricom shop, I perused a wall of smartphones. I left ten minutes later with a fully functioning smartphone, just $50 lighter. I was euphoric — a smartphone for $50!? And I can download Medium, Facebook, a podcast app, Audible, even Spotify? Maybe I won’t get a new iPhone when I’m home for Christmas. What on earth are we shelling out a thousand dollars to Apple for?
Mmm…. quite a lot it turns out. While technically the phone does everything an iPhone would do, it does it all worse and at one tenth the speed.
Still, it’s incredible that Tecno has made something so functional — if not smooth — for $50. Really, I got exactly what I was looking for: a cheap phone that works for calls and texts, and close enough for Uber.
So what else can you get in Nairobi for $50?
5 months worth of plenty of data.
25 days worth of “Traditional Veg Mix + Chapati (x2)” delivered lunches.
5/6 of an amazing Festive Cheetah “explorer pillow” from artist Kanagrui. (Okay, seriously, how great are these cheetahs?)
2500 trips to work from the closest matatu stop (if I take the cheaper matatu).
5x$10 knock-off iPhone charging cords that are not broken and which you can buy to find out that your iPhone chargers were all broken (including the brand new one you bought 3 weeks ago, somehow), and that your iPhone is actually fine.
Ran into my colleague Hanna at our corner veggie market. Bought 2 green peppers, 3 tomatoes, and 3 onions to make dal, plus 3 bananas to add to smoothies; all together, that cost 140 KES, which is about $1.40.
Zell Kravinsky risked his life to donate his healthy kidney to a complete stranger. Would you do the same?
Kravinsky is a radical altruist. He believes in giving away as much as possible to others, including his nearly $45 million fortune and his own body parts. Most people would consider donating a kidney as going above and beyond, but Kravinsky told the New Yorker in 2004 that he considers anyone who doesn’t donate their extra kidney a murderer.
We probably don’t, as individuals, have a moral responsibility to donate our organs, but maybe we do have a societal responsibility to find a system by which we can match kidney donors and recipients so that no one has to die just because there isn’t a transplant available. In 2012, there were 95,000 Americans on the wait list for a life-saving kidney, according to economists Gary Becker and Julio Elias. The average wait time for a kidney in 2012 was over four years.
Becker and Elias are proponents of creating a formal, legal market for organs to eliminate long wait times and better match recipients with donors. Right now, it is illegal to sell your organs in most of the world, including in the U.S.
The main risks of monetary compensation for organ donations are the coercion of unwilling donors, the potentially unequal distribution of donors — poor people would be more likely to become donors, and the moral question of whether or not it is okay to sell body parts, even if they are our own.
Purely moral arguments aside for a moment, there are ways to alleviate the risks of a market for organs. Waiting periods between registration and donation, psychiatric evaluation ahead of registration as an organ donor, and strict identification requirements or even background checks can all combat coercion in the market for organs, while saving the lives of the many Americans who die on an organ waitlist. Becker and Elias also point to the fact that people in lower income brackets are disproportionately affected by long waitlists: the wealthy can fly abroad to obtain a healthy organ or manipulate the current waitlist system in their favor, while poorer Americans face longer wait times. While donors may be disproportionately poor, which raises concerns of implicit economic coercion, the lower income brackets also benefit disproportionately from the policy.
Even more powerful than a legal market alone would be a combination of a legal market for organs and an implied consent law, which would mean people would have to opt out of being an organ donor, rather than the U.S. standard of opting into being a donor. A 2006 study by economists Alberto Abadie and Sebastien Gay found that implied consent laws have a positive impact on organ donations. Under a combination of these two initiatives, essentially all organ donor needs might be met, and a person’s will might come to include provisions for their organs to be harvested and family members to be compensated.
While Kravinsky donated his kidney for free, he once offered a journalist $10,000 to donate a kidney to a stranger, according to Philadelphia Magazine. But the journalist backed out of the deal he struck with Kravinsky after his wife and friends convinced him not to go through with it. They convinced him that the risk of surgery, though relatively minor, was not worth saving a life. But if a safe, legal market for organ sales is established, perhaps the establishment of a market price for organ donation and a normalization of the procedure will allow Americans to save lives and make money, without requiring Kravinsky’s extreme, and perhaps aggressive, sort of altruism.
Originally written for my Economics of Sin senior seminar, spring 2017; previously published at the Unofficial Economist on Medium.
CW: sexual assault
A colleague recommended I listen to this latest episode (April 23, 2018).
The man who calls had been molested by a male babysitter as a child, and then raped/sexually assaulted by two different women in the course of his high school years. It was an intense soundtrack to my first run in weeks. The caller is raw and open with his vulnerability.
It really helps you understand viscerally the broad spectrum of sexual experiences, traumas, and approaches people may have. The whole episode is a clear, loud call for more communication, more openness, and more thoughtfulness in our sexual lives.
If I continue on my current career path, I may end up arbitrating who lives and who dies. (And maybe I’ll tell their story in an economics journal and make a living doing so.)
I am planning on pursuing a career in development work, specifically in the evaluation of development programs. The “gold standard” for evaluating programs is a Randomized Control Trial (RCT).
Consider a non-profit distributing books to children with the goal of improving literacy. The non-profit wants to know whether their books really have any impact on children’s literacy. Ideally, they could look at what happens when they give a group of children the books and also what happens when they don’t give the same children books.
However, due to thus far unchangeable time-space continuum properties, this isn’t possible. So, in order to confidently say that their books had an impact, the non-profit needs to compare the literacy scores of children who received the books with other very similar children who didn’t get books. Let’s say they hire me to run an RCT for this very purpose.
To determine which children will get the books (the treatment group) and which children will serve as the comparison group (the control group), I take a list of 100 schools and randomly assign half of them to receive the extra books program. After the books are distributed and some time has passed, I go back to the schools and I have all the children take literacy tests. I compare the test scores of children in each group, and find that, on average, children who received books did much better on the literacy tests.
The non-profit is very happy and uses the results to convince more people to donate to their program. Now they can give books to many more children, and presumably those children’s literacy scores will also increase.
This is all good and well. Even if some children in the study were chosen not to receive books, there are several commonly accepted justifications for why we studied them without providing a service:
- The non-profit did not have enough money to give books to all the schools anyway. Randomly determining which schools received the books makes it as fair as possible.
- While the books program was unlikely to have negative effects on children, we didn’t know if it would have no effect or a positive effect at the start. So we didn’t know if we were really depriving children of a chance to improve their literacy.
- Being able to conduct the evaluation could inform policy and global knowledge on effective ways to improve literacy, and could improve decision-making at the non-profit.
- In this case, maybe the control group children were the first to receive books when the non-profit’s funding increased.
These are common justifications for development evaluations. They seem quite reasonable — randomly giving out benefits might be the fairest option, we don’t know what the effect really is, and the study will contribute to our shared knowledge and lead to better decisions and even better outcomes in the future.
What if, instead of working on literacy, the non-profit wanted to reduce deaths from childbirth by improving access to and use of health facilities by pregnant women?
Suddenly, so much more is at stake.
If I randomly assign half a county to have access to a special taxi service that drives pregnant women to hospitals for safer deliveries, and one of the women who was assigned NOT to receive the taxi service dies because she gave birth at home, is the evaluation immoral? Am I morally culpable for her death?
Because I work with numbers and data, it is easy to separate myself from the potential negative consequences of the work. I didn’t choose her to die — the random number generator made me do it.
So what if we’re in a situation where a randomized control trial seems immoral? How can we still learn about what works and what doesn’t?
There are other evaluation methods that can give us an idea of what programs work and which don’t. For example, quasi-experimental methods look at situations where comparable control and treatment groups are incidentally defined by the implementation of a policy. Then we can compare two groups without having to be responsible for directly assigning some people to receive a program while others go without.
Qualitative or other non-experimental methods involve gathering data by talking to people, doing research, and meeting with different groups to get various opinions on what’s happening. These methods can also help paint a picture of whether a program is having a positive effect.
But the RCT is the gold standard for a reason. A well-designed RCT can tell us what the effect of a program is with much higher confidence and precision than other methods.
UNICEF Social Policy Specialist Tia Palermo recently wrote a post titled “Are Randomized Control Trials Bad for Children?” for UNICEF’s Evidence for Action blog. She makes a powerful point to consider: What are the alternatives to running RCTs? Are they better or worse?
Palermo sees the alternative as worse: “Is it ethical to pour donor money into projects when we don’t know if they work? Is it ethical not to learn from the experience of beneficiaries about the impacts of a program?” she asks.
Her most convincing argument is that there are ethical implications every research method we might choose:
“A non-credible or non-rigorous evaluation is a problem because underestimating program impacts might mean that we conclude a program or policy doesn’t work when it really does (with ethical implications). Funding might be withdrawn and an effective program is cut off. Or we might overestimate program impacts and conclude that a program is more successful than it really is (also with ethical implications). Resources might be allocated to this program over another program that actually works, or works better.”
And there are ethical implications to not evaluating programs at all. If non-profits aren’t held to any standard and don’t measure the effect of their program at all, there’s no way to tell which interventions and which non-profits are helping, having no effect on, or even harming the program recipients.
In the case of the woman who died because she didn’t get to a health facility, if the study had never taken place, would she have gotten to a health facility or not? It is impossible to know what would have happened, but it’s not impossible to minimize the risk of harm and maximize the benefits to all study participants.
Ultimately, RCTs generate important evidence when they are well executed. The findings from such studies can be used to make better decisions at non-profits, at big donor foundations like the Gates Foundation or GiveWell, and at government agencies. All of which can lead to more lives saved, which is the ultimate goal.
So what to do about the ethical implications of randomly determining who gets access to a potentially life-saving program? Or any program that could have a positive impact on people’s lives?
There are a variety of measures in place to ensure ethical conduct in research and many more ~official~ economists are thinking about these ideas.
The 1979 Belmont Report in helped establish criteria for ethics in human research, focusing on respect for people’s right to make decisions freely, maximizing benefits and doing no harm, and fairness in who bears any risks or benefits. Institutional Review Boards (IRBs) are governing bodies that ensure these principles are being upheld for all research.
Economists Rachel Glennerster & Shawn Powers wrote a highly recommended piece on these ethical considerations, “Balancing Risk and Benefit: Ethical Tradeoffs in Running Randomized Evaluations,” which I’m currently reading.
Yet persistent concerns about how to run ethical evaluations suggest that there is more work to do.
Taking the time to consider the ethical implications of each project is key. And I think there is more room for evaluators to read deeply on the subject and really dig into how to make evaluations more just and more beneficial to even those in the control group who don’t receive the program.
A driving principle, especially for researchers running RCTs in the development field, could be that an evaluation must have a direct positive impact on all study participants, either during the study or immediately following its completion. There are a variety of ways, some more commonly used than others, that researchers can apply this principle:
- If we truly don’t know whether the effect of the program is positive or negative, we can make plans to provide the program to control households if it is found to have a positive effect.
- If we suspect the program has a positive effect, the control group can be offered the program immediately after the study period has ended.
- We can offer everyone in the study a base service, while the study tests the effectiveness of an additional service provided only to the treatment group. This way, everyone who is contributing time and information to the study receives some benefit in return.
- Extensive piloting (testing different ideas and aspects of the evaluation before the start of the study) can also reveal potential moral dilemmas to evaluating any particular program.
- Community interest meetings can be held before the study is implemented to gain community-level consent to participate in the study. These meetings could also be held quite early on to inform research designs and improve the quality of the study results. For example, in some cultures, it is not appropriate for a man to be alone with a woman he is not related to. If this is the case in a study area, then hiring male staff to conduct surveys would lead to a less successful study.
- Local staff can be hired to conduct any surveys or data collection to ensure that the surveys are culturally appropriate.
- We always obtain full and knowledgable consent from participants, which may require translating surveys into participants’ native language.
- If study participation requires much time or effort from control group individuals, they can be appropriately compensated.
- All reports on evaluations (RCTs and other designs) can be fully transparent about research decisions and how ethical concerns were addressed. This will contribute to the international research community’s combined knowledge of how to ensure the rights of participants are provided for in RCTs and other research.
- The learnings from the study can also be shared with the participating community and should add to their knowledge about their own lives; contributing to the abstract “international research community” is not enough.
Enacting these measures requires more of researchers: some have the potential to affect the legitimacy of the evaluation results if they are not properly accounted for in analysis. But a strong sense of ethics and a dedication to the population being served (often low-income individuals from the Global South, contrasted with well-off researchers from the West) demand that we take the extra time in our research to consider all ethical implications.
So much learning I want to do
- Coding classes: Advanced R, Intro to Python, Machine Learning
- Eventually: ML online competitions!
- Reading academic articles
- In economics
- In global health sector
- Reading development-related articles
- Summarizing/critiquing work-related articles
- And post online
- And share on internal knowledge management channels
- Read Poor Economics for real (embarrassed I’ve only read half of it though)
- Read Field Experiments book
- Stata challenges from work
- Plan a brown bag lunch presentation or chai & chat on a topic that interests me
- An opportunity to practice presenting a slide deck
- Read/plan for Tech Team Bookclub meetings on Machine Learning
- Create a mapping portfolio by doing GIS challenges (possible??)
So little time…
I have blocked out three hours a week for my own PD. What do I want to prioritize? How scheduled/organized should I be about this?
I want to use the time for a mix of projects. This week, I want to read and write about one academic article related to health care and economics. It’s something I’ve been meaning to do. That should take 2 hours – I’ll polish and post my thoughts on my own time. Then, I can use the rest of the time to investigate what kind of mapping questions I can start looking into. I’m very excited about maps.
Long-term, I can plan to split it up into 2-3 chunks so I can make some progress on each of my projects across time.
- I’ll come back to the coding and the books later
- I’ll do Tech Team Bookclub and Stata challenges as they arise at work
- I’ll plan for a mix of reading & writing about articles and GIS for now
- Maybe once I have some cool maps made, I’ll do a brown-bag and an internal blog post about spatial data in Africa and how it’s relevant for IDinsight
2 hours: I want to break down what heterodox vs. pluralistic vs. mainstream economics are. The idea of alternative economic models really appeals to me, but I don’t know what the big distinctions or points of conflict are. I’ll find some sources on my own time this week, and on Friday, read them and write up a summary for here.
1 hour: Investigate spatial data available for Kenya, maybe read an article on general spatial data quality in Africa.
With much awkwardness and also appreciation from Vas :)