August 15, 2019
Last week we discussed The Lab’s second step in our project cycle: Design. This week we explore step three: Do Something.
On the go? Listen to this blog as a podcast.
We piloted a new program… we tested a new design… we implemented a new study. If you look at the tidy wording on our project pages, testing something new sounds so simple. It’s kind of like being served a fancy dish with the action of the bustling restaurant kitchen out of view. Of course, behind closed doors, the kitchen is hot, loud, perhaps even chaotic. Sometimes dishes drop or burn. A flood of orders seem to come in at once.
Doing something new—in any sector—can feel like that chaotic kitchen. But what we see on the other side as consumers of those dishes—or in The Lab’s case, project reports and published papers—can lead us to forget the heat of “the kitchen.”
If we’re being honest about what it takes for governments to do—and test—their work, we have to take this behind-the-scenes look. There are items that have to be procured, daily operations that have to be adjusted, regulations written, and data agreements that have to be squared away. Unexpected hurdles arise. Elbow grease needs applying. And evaluating doesn’t make it easier.
And if not for the commitment of agency partners to serve up the best for DC residents, we’d have little to show. In honor of them, we’re talking about a few “in the kitchen” moments we’ve shared together.
Senior Operations Analyst, Karissa Minnich, tugs on a city trash can for the Smartcans project to see if it’s bolted to the sidewalk. It is. (Credit: The Lab @ DC)
Trying something for the first time presents unexpected hurdles.
DC Government is full of new ideas to try. What if motivational messaging on public trash could reduce littering? That’s when the “do something” sets in. How do we get signage on 400 of the city’s trash cans with no additional budget?
Our team hit the streets to inspect the cans. What material should the signs be made of so they could bend to the can while also withstanding the elements? Where should the sign be positioned so it didn’t get in the way of being emptied? And how often are they emptied? Together with the Department of Public Works (DPW), we came up with a plan: a soft metal sign with zip-ties that could be easily removed in a few months, placed on the door of the can so it could be emptied, and installed during trash pick-up at night by DPW collectors.
It might require some additional elbow grease.
In the example above, DPW tried something labor-intensive and new to improve the lives of residents. As they work to get something new off the ground, they may not yet have the service contracts and people they need in place to test the best way to, for example, install a sign on a trash can. Or to train their staff in a new protocol. Every agency is trying to squeeze the most out of their budget and resources, so adding something new can be difficult.
Back in 2017, the Department for Human Services (DHS) worked with The Lab to test additional reminder letters to help families navigate the process of renewing their eligibility for vital government support. DHS wanted to figure out quickly whether these letters worked better than their usual, legally required mailers, so we worked together to design something that could be sent in-house. The Lab prepared the letters with recipients’ addresses, appointment dates, renewal deadlines, and nearest service centers for DHS to print. Then we went over to DHS for an afternoon of sealing and stamping the letters for mailing.
Sometimes what’s needed to try something new is just some additional people for a few hours—or months—to get it done. Lending a hand is the least we could do, given the extra commitment our evaluations call for.
Evaluating that new thing throws in a few more hurdles.
Our best tool for finding out if something new is working is a randomized evaluation. This means that participants are assigned to get the new program completely randomly, similar to flipping a coin: heads, you get assigned to the program; tails, nothing changes for you.
In rare cases—like when we’re testing a new training for all Metropolitan Police Department (MPD) personnel—we know everyone who’s eligible for the program ahead of time, allowing us to randomly assign—in this example—officers to participate or not all at once, as if we were pulling names out of a hat.
But most of the time, it’s not so simple. For example, in 2018, nurses joined the 911 call center to speak to callers who didn’t need emergency services. To test the impact of the nurses on unnecessary emergency room visits, we randomly assigned non-emergency calls to either receive the typical 911 response or be connected to a nurse. We had to figure out a way to randomize callers in the moment without interfering with the 911 operators’ ability to respond to callers as quickly as they always do. An evaluation should never interfere with vital services.
How did we do it? The Office of Unified Communications worked with us to include randomization as they rolled out new software and protocols for 911 operators. As operators entered information about callers, they were automatically instructed to redirect the call to a nurse if a client was eligible and had been randomly assigned to receive the new program.
What’s next?
You know that dish you love from your favorite restaurant? Chances are that it wasn’t the first time the chef made it. Most of the time it’s a process of testing and refining it until they think it’s worthy of serving. We think of government in much the same way. Come back next week to learn about the next step in The Lab’s project cycle: Test.
Nellie Moore is an Operations Analyst at The Lab @ DC. Outside of work, Nellie can be found in her Brookland kitchen finding new ways to cook, bake, and store her weekly produce share from a local urban farm.
Karissa Minnich is a Senior Operations Analyst at The Lab @ DC. She recently turned 17 lbs of farmers market tomatoes into pasta sauce in her Ward 2 kitchen.