13 Applying machine learning requires institutional buy-in, where there is meaningful policy relevance

Some contexts may be better suited than others for personalized policy. There are cases in which the differences between treatment effects, while statistically robust, are not large enough to have implications for policy decisions. For example, if we increase the number of families who fill out a financial aid form by two-points on average, and meaningful subgroups show robust heterogeneous effects ranging from 0.9-points to 2-points, the application of that finding is not particularly impactful in practice. This is especially true for an intervention like a text message or email that has little-to-no cost after its initial production.

Institutional buy-in is much easier to build when personalization has meaningful policy relevance, meaning findings can improve welfare. City agencies and organizations are much more likely to spend their resources to build personalized designs into their systems when the benefits of doing so are clear. Estimating the actual impacts of a policy, given the constraints of each context, becomes crucial to conversations about real implementation and adoption. This is how personalized policies will realize their potential impact.