Good Targeting Doesn’t Require Bad Privacy

Brad Sagara (Research Project Manager, Data Privacy Lab, CEGA) summarizes key takeaways from Ben Jacobsen (Department of Computer Sciences, University of Wisconsin-Madison) and Nitin Kohli’s (Staff Scientist, Data Privacy Lab, CEGA) latest paper where they evaluate the tradeoffs between privacy, efficiency and precision in targeting welfare improving interventions. 

Governments, non-profits and private sector actors implementing welfare-enhancing interventions are all confronted with the challenge of how to identify who benefits from the intervention. Two common approaches to targeting or allocating those benefits is individual and group level allocation. In individual level targeting, individual beneficiaries are identified based on their observed or predicted characteristics (e.g. poverty status) whereas in group level targeting, aggregate statistics about the group (e.g. prevalence of poverty) are used to allocate resources to the entire group. Both approaches incur data collection and analysis costs, and social scientists have extensively studied the choice between individual and group level targeting of different interventions (e.g., cash transfers, graduation programs). Research has shown that group-based targeting works as well or better than individual targeting whenever group membership already reveals a lot about how much help an individual needs. 

But what happens to this tradeoff when you introduce robust privacy protections for vulnerable populations? CEGA researchers have already studied this tradeoff directly in one real-world context. In Togo’s COVID-era Novissi cash-transfer program, researchers built a new form of differential privacy for mobile-phone-based targeting and found that strong privacy protections could be added to preserve individual privacy from data analysts doing the targeting with only a modest cost to targeting accuracy. Ben Jacobsen and Nitin Kohli’s new paper expands on this work by evaluating privacy protections designed to preserve individual privacy from other community members. They find that in many realistic settings, strong privacy protection costs almost nothing in terms of targeting precision and efficiency. 

Privacy and Targeting

There is an inherent tension between privacy and determining whether an individual is eligible to receive benefits from welfare-enhancing interventions: the implementor must learn information about the potential beneficiaries to effectively and efficiently allocate resources. But privacy concerns also arise throughout the targeting process; this paper is concerned with the privacy of receiving the benefit itself. For example, when targeting criteria include poverty status, receipt of an anti-poverty benefit makes that information public, which has been shown to provoke shame or jealousy among peers. However, it is unclear how decision-makers should weigh privacy protections when deciding whether to spend their budget collecting more data versus helping more people. 

To evaluate these tradeoffs, Jacobsen and Kohli turned to differential privacy: a mathematical framework for carefully adding statistical noise to data so that no one can learn too much about any single individual from downstream analytical outputs like aggregate statistics or machine learning models, even with strong background knowledge. Authors developed and tested formal models of both individual- and group-based targeting under differential privacy across three common scenarios: 

Scenario 1: Decision-makers already know everyone’s true welfare scores; e.g. government programs using administrative data

Scenario 2: Decision-makers incurs the costs of collecting information regarding welfare status of the population; e.g. through a household survey

Scenario 3: Decision-makers predict people’s welfare status from proxy data, e.g. mobile phone records

In each scenario privacy-preserving targeting strategies’ performance were compared to the best possible allocation if the decision-makers had perfect, identifiable information.

Results

Introducing robust privacy protection has negligible effects on the efficiency of individual or group-based targeting: across all three scenarios, both targeting approaches can be made rigorously private with minimal loss in performance among sufficiently large populations. Additionally, consistent with existing research,  the authors find that structural aspects, i.e. predictive model performance, cost of data collection, average welfare, and existing levels of inequality dictate which targeting approach outperforms the other. For instance, in Scenario 3, group-based targeting is mathematically guaranteed to outperform individual-based targeting whenever (i) welfare is hard to predict accurately from the proxy data, (ii) data acquisition costs are high, (iii) average welfare in the population is low, and (iv) the existing inequality between groups is high. This scenario arises whenever the proxy data used to predict welfare is not as informative as observable group membership, and the cost of acquiring new data to improve a predictive model would be too high and better spent allocating aid to more people. This implies that individual-based targeting shines when the proxy information can be used to construct a sufficiently accurate model to improve the targeting of benefits relative to the existing welfare and inequality among groups.  Decision-makers may select whichever targeting approach best fits their situation based on predictive model performance, data collection costs, average welfare, and levels of inequality, and incorporate robust privacy protections with little loss of targeting precision.

Why this matters

Privacy protection is not an unaffordable luxury: even under common real-world constraints, adding strong, mathematically guaranteed privacy protection does not incur any additional cost in terms of targeting efficiency. This finding matters most in contexts where personal data is now routinely used to decide who receives assistance (e.g. social protection programs, humanitarian response) many of them in LMICs where the people being targeted have the least power to contest how their data is used. 

Read Privacy, Prediction, and Allocation (full technical version available here) to go deeper into the mathematics behind these results. 

This is just one example of how Nitin Kohli and the Data Privacy Lab are bridging rigorous privacy research and real-world challenges. To stay up to date on the latest news, please subscribe to the Data Science for Development mailing list, explore our resources, and learn more about how we are deploying privacy enhancing technologies in LMICs. 

If you are interested in learning more or collaborating with the Data Privacy Lab, please contact: dataprivacylab@berkeley.edu.