YXYang XiaoResearch portfolio
Yang XiaoExperiments & incentives
Research question

Who should get the offer?

A discount can increase activity. The harder question is whether a model can identify who will respond—and spend the same budget better.

01 · Method

I test effect estimates against known answers, then compare spending rules at the same budget in simulated markets held out from training.

02 · Conclusion

The complex targeting model has not earned its advantage.

The learner misses the recovery benchmark. In the budget experiment, a uniform allocation also produces more incremental trips than the model-based rule.

Scope of the evidence. Semi-synthetic validation, not a measured effect of a real NYC promotion. The oracle knows the true average effect; it is a test benchmark, not a deployable policy.

03 · Direct evidence

See what supports the finding.

Open interactive Space ↗

The model misses the simple benchmark

Effect-estimation error · lower is better

The model misses the simple benchmark

What this shows. The targeting model has more error than a constant benchmark that knows the true average effect. This test does not support a claim that the model reliably identifies who benefits more.

View exact values & source
EstimatorRMSE
Targeting model0.0417
Constant oracle0.0247

Source: CasualLab/artifacts/benchmarks/heterogeneity/hte_recovery.json

Download figure (SVG)

Equal budgets, no targeting advantage

Incremental trips · eight held-out simulated markets

Equal budgets, no targeting advantage

What this shows. With a budget of 1,000 in each market, the model-based policy averages 64.50 extra trips versus 68.50 for uniform allocation. These are simulated policy outcomes, not field-experiment estimates.

View exact values & source
Allocation ruleMean incremental trips
Model-based64.49589085777113
Random65.5022313812971
Rule-based66.08454126083961
Uniform68.49668956342123

Source: CasualLab/artifacts/benchmarks/policy_results.csv

Download figure (SVG)
Research notes

Design, sources & limitations

What I built

I built the path from estimand and experiment design to known-truth estimator checks, policy simulation, and inspectable release artifacts.

0.0417learner HTE RMSE
0.0247constant-oracle RMSE
Gate stoppedweaker HTE claim withheld
2.006own effect vs 2.000 truth

Bottom line: the known-truth benchmark measured the HTE learner at 0.0417 RMSE versus a 0.0247 constant oracle, and the predeclared gate correctly stopped the weaker model from being promoted.

Research question and relevance

Question: How do causal mechanisms and heterogeneous treatment effects shape market outcomes and policy interventions in field-like datasets?

Credible policy evaluation requires more than a point estimate. The project connects identification assumptions, estimator recovery, heterogeneous effects, intervention allocation, and reporting so a decision can be traced back to evidence.

Original paper or report: CasualLab is a project-originated research toolkit; this release does not claim to reproduce a specific third-party paper without a documented citation and benchmark.

Data and sample

The public GitHub copy contains synthetic fixtures, NYC-oriented samples, schemas, and tests. The full versioned research content is stored under CasualLab/ in the Dataset.

RecordCurrent evidence
Original sample periodNot applicable to synthetic fixtures; source-specific periods are documented per run.
Updated sample periodPublished with the current checksummed public-data and known-truth validation artifacts.
Data availabilityCompact fixtures on GitHub; full research payload on Hugging Face.

Capabilities demonstrated

  1. Define the estimand and identification assumptions.
  2. Load an authorized source or documented fixture.
  3. Estimate average and heterogeneous effects.
  4. Run estimator-recovery and sensitivity checks.
  5. Publish outputs with provenance and a result manifest.

Robustness work includes alternative specifications, heterogeneous-effect recovery, placebo or synthetic checks where applicable, and explicit release boundaries.

Validation evidence

Validated componentEvidenceInterpretation
HTE recovery audit0.0417 learner RMSE vs 0.0247 oracleThe predeclared gate correctly withheld an unsupported heterogeneous-effect claim.
Controlled spillover recovery2.006 / 1.495 / 0.691 recoveredOwn, neighbor, and history effects closely match known truths of 2.000 / 1.500 / 0.700 in the controlled slice.
Public-data system19.7M records → 194,928 cellsChecksummed ingestion, conservation checks, and deterministic artifacts verify the engineering path.

These are known-truth and public-data validation results, not a live NYC treatment-effect estimate.

Reproduction and citation

Run make reproduce from the CasualLab/ folder of the open-economic-quant repository. Cite the portfolio release, Dataset revision, CasualLab, and every original data provider used in a run.

Evidence boundary

The published HTE comparison is semi-synthetic and evaluated against known truth; the public trip panel is descriptive. Neither is presented as a live marketplace treatment effect.