23  Use cases of causal inference in industry

This section is for keeping track of blog articles and papers related to use cases of causal inference in industry. Many of these methods use meta-learners (Künzel et al. 2019) or doubly-robust estimators (Funk et al. 2011) which are not covered in this course.

23.1 Matching

  • At Uber, researchers studied the effect of using Uber Eats, in addition to Uber Rides, on the amount spent on Uber Rides. They used propensity score matching with 100+ covariates to match each user who used both Uber Eats and Uber Rides to a user who only used Uber Rides. Their results suggest that using UberEats drives up spending on Uber Rides (Python tutorial).

23.2 Instrumental variables

  • Researchers at Twitch studied the effect of number of friends on the user’s likelihood to return to the site. Their instrumental variable is the random assignment of receiving a prompt to add more friends (Forter 2017).
  • Researchers at Roblox studied the impact of the Avatar Shop on the community engagement. Their instrumental variable is the random assignment of getting a recommendation for items in the Avatar Shop (Kharel 2021).
  • Farre-Mensa, Hegde, and Ljungqvist (2017) studied the impact of having patents on a startup’s subsequent growth. The judges are the examiners of the patents, with different levels of leniency, and the instrumental variable is the random assignment of judges to the patent applications.
  • Researchers at TripAdvisor studied the effect of being a membership on user engagement. Since it is not possible to force users to comply and become members, they instead used a instrumental variable design in which a randomized group of users were provided with a single-click sign-up button, which was much easier than the previous sign-up process. In this study, the instrumental variable was whether the user was offered the easier sign-up option (Python tutorial).

23.3 Difference-in-differences

  • At Spotify, researchers studied the treatment effects of adopting a streaming service on the total music consumption. The study was performed using difference-in-differences on a unique panel data that contains individual-level music consumption. Since the sampled consumers may not be representative of the larger population of potential adopters, they instead studied local average treatment effects (LATE) among those consumer segments who adopt streaming (Datta, Knox, and Bronnenberg 2018).

23.4 Panel data

  • Researchers at Uber studied the impact of increased pricing during high demand on the supply of Uber rides. They fitted a fixed effect model on panel data of drivers’ trips, using the hourly fare multiplier (depending of the supply and demand) as the treatment variable, an indicator of whether the trip was the driver’s last one of the session as the outcome variable, and individual driver, day of week, hour of day, and region of city as fixed effects. The results of the model suggest that a surge in hourly fares significantly increases the supply of rides on the Uber platform (Chen 2016).

23.5 Synthetic control

  • Researchers at Spotify proposed Bayesian structural time-series model (Brodersen et al. 2015) to constructs a counterfactual artist popularity outcome using a set of synthetic controls. Their findings suggest that releasing a new track has a positive impact on the popularity of other tracks by the same artist, and other related and competing artists also benefit from a new track release (Mehrotra, Bhattacharya, and Lalmas 2020).

References

Brodersen, Kay H., Fabian Gallusser, Jim Koehler, Nicolas Remy, and Steven L. Scott. 2015. “INFERRING CAUSAL IMPACT USING BAYESIAN STRUCTURAL TIME-SERIES MODELS.” The Annals of Applied Statistics 9 (1): 247–74. http://www.jstor.org/stable/24522418.
Chen, M. Keith. 2016. “Dynamic Pricing in a Labor Market.” In Proceedings of the 2016 ACM Conference on Economics and Computation. ACM. https://doi.org/10.1145/2940716.2940798.
Datta, Hannes, George Knox, and Bart J. Bronnenberg. 2018. “Changing Their Tune: How Consumers’ Adoption of Online Streaming Affects Music Consumption and Discovery.” Marketing Science 37 (1): 5–21. https://doi.org/10.1287/mksc.2017.1051.
Farre-Mensa, Joan, Deepak Hegde, and Alexander Ljungqvist. 2017. “What Is a Patent Worth? Evidence from the U.S. Patent "Lottery".” Working {Paper}. Working Paper Series. National Bureau of Economic Research. https://doi.org/10.3386/w23268.
Forter, Carson. 2017. “Two-Stage Least Squares For A/B Tests.” Twitch Blog. https://blog.twitch.tv/en/2017/06/30/two-stage-least-squares-for-a-b-tests-669d07f904f7/.
Funk, Michele Jonsson, Daniel Westreich, Chris Wiesen, Til Stürmer, M. Alan Brookhart, and Marie Davidian. 2011. “Doubly Robust Estimation of Causal Effects.” American Journal of Epidemiology 173 (7): 761–67. https://doi.org/10.1093/aje/kwq439.
Kharel, Ujwal. 2021. “Causal Inference Using Instrumental Variables.” Roblox Blog. https://blog.roblox.com/2021/09/causal-inference-using-instrumental-variables.
Künzel, Sören R., Jasjeet S. Sekhon, Peter J. Bickel, and Bin Yu. 2019. “Metalearners for Estimating Heterogeneous Treatment Effects Using Machine Learning.” Proceedings of the National Academy of Sciences 116 (10): 4156–65. https://doi.org/10.1073/pnas.1804597116.
Mehrotra, Rishabh, Prasanta Bhattacharya, and Mounia Lalmas. 2020. “Inferring the Causal Impact of New Track Releases on Music Recommendation Platforms Through Counterfactual Predictions.” In Proceedings of the 14th ACM Conference on Recommender Systems, 687–91. RecSys ’20. New York, NY, USA: Association for Computing Machinery. https://doi.org/10.1145/3383313.3418491.