Menu

1 / 8

Difference-in-Differences vs Lagged-Outcome Regression

Reading 0:00
15s threshold

Most treatments in product analytics are chosen by the user. Someone turns on a feature, joins a community, or starts a free trial of a premium tier. There is no randomisation, but under the right assumptions we can still estimate the effect of these self-selected treatments. Without a valid instrument or a cut-off to exploit (the settings for instrumental variables or regression discontinuity), analysts tend to fall into one of two camps:

Lagged-outcome regression , usually estimated with double machine learning. This camp learned from Causal Inference for the Brave and True and uses libraries like EconML and CausalML .

Difference-in-differences , learned from The Mixtape and brought up to date by papers like Sant'Anna and Zhao's Doubly Robust Difference-in-Differences Estimators (2020).

Both are reasonable, and both can be estimated with the same doubly robust machinery ( DoubleML , for example, implements both).…

Continue reading — create a free account

Join HashtagPLUS to read full articles, follow hashtags, vote, and join the conversation.