1 Introduction

This report is a product of the collaboration between ideas42 and the Golub Capital Social Impact Lab to advance practical applications of machine learning to behavioral science policy and field experimentation.

The contents of this report are designed for any audience looking to learn more about how machine learning can add new techniques to behavioral design, causal inference, and experimentation. A big thank you to Schmidt Futures and The Alfred P. Sloan Foundation for their generous contributions to this innovative collaboration and research. This report is written in RMarkdown with bookdown.

1.1 Who should read this report?

  • You’re an economist, researcher, policy designer, et cetera who uses evidence from randomized controlled trials or A/B tests to make decisions about what works.
  • You’re a student of computer science who cares about social impact.
  • You’re a behavioral scientist or practitioner who wants to understand how increasingly large data can be used for good.

1.2 Why did we pursue this research?

Behavioral science interventions have already generated widespread, positive social impact – but the field has yet to reach its full potential.

A vision for bigger, better impact

Typically, behavioral scientists design and test interventions to learn how they work on average, but we can achieve greater impact if we use data to learn about what works for whom. With new computational methods, we can uncover unexpected insights about who does (and does not) respond well to specific treatments. Equipped with predictions about how interventions are likely to perform across groups, we can decide who should receive which treatment and select policies that meet our goals in a given context (e.g. reduce cost, reduce disparities between groups, maximize the number of people who follow through on their intentions).

From vision to action

Our goal for this research was to translate the vision articulated above into actionable insights for practitioners. We applied cutting-edge machine learning methods (causal forests, optimal policy estimation, and bandit algorithms) to a set of real-world behavioral interventions – and in the process advanced the methods themselves. This report reviews the findings from our research, provides links to supplemental materials and tutorials, and offers suggestions for how practitioners can scope and design projects to incorporate these methods most effectively. As these methods continue to evolve, we hope you will apply them in new contexts and share your insights so we can continue to innovate and positively impact people’s lives.

1.4 About us

1.4.1 ideas42

We’re a non-profit using deep insights into human behavior—into why people do what they do—to help improve lives, build better systems, and drive social change. Working globally, we reinvent the practices of institutions and create better products and policies that can be scaled for maximum impact.

We also teach others, ultimately striving to generate lasting social impact and create a future where the universal application of behavioral science powers a world with optimal health, equitable wealth, and environments and systems that are sustainable and just for all.

For more than a decade, we’ve been at the forefront of applying behavioral science in the real world. And as we’ve developed our expertise, we’ve helped to define an entire field. Our efforts have so far extended to 40 countries as we’ve partnered with governments, foundations, NGOs, private enterprises, and a wide array of public institutions—in short, anyone who wants to make a positive difference in peoples’ lives.

Visit ideas42.org and follow us on Twitter to learn more about our work. Contact us at with questions.

1.4.2 The Golub Capital Social Impact Lab

We use digital technology and social science research to improve the effectiveness of leading social sector organizations. We drive measurable social impact through two key activities: collaborating on original research and sharing best practices.

Based out of Stanford Graduate School of Business, the lab is a research initiative of affiliated academics, staff, and student researchers who are passionate about conducting research that guides and improves the process of innovation, as well as applying and disseminating that research to achieve social impact at large scale. We collaborate with a wide range of organizations, including large firms and smaller startups, for-profits and nonprofits, as well as NGOs and governments.

Through research projects, the lab works to promote, develop, and apply innovative technology solutions and best-practice methodologies at social impact organizations. We create and disseminate the insights to students and practitioners, thus guiding future innovation in more productive directions and improving the efficiency of the innovation process.

1.4.3 Acknowledgements

We would like to thank Schmidt Futures and the Alfred P. Sloan Foundation for funding this research into the intersection of two innovative fields. We would also like to thank the following individuals for their invaluable support in launching and sustaining this effort.

  • Our advisors: Guido Imbens, Jens Ludwig, Sendhil Mullainathan, Ted Robertson
  • Our research team: Vitor Hadad, Niall Keleher, Matthew Schaelling, Henrike Steimer
  • Our collaborators: Michael Stern, Charlie Connell, Carolyn Silverman, Gregory Stoddard, Everet Rummel, Nicole Bellettiere, Colin Chellman, Tom Tasche, Peter Polga-Hecimovich, Imanol Arrieta Ibarra