[go: up one dir, main page]
IDEAS home Printed from https://ideas.repec.org/p/sin/wpaper/17-a011.html
   My bibliography  Save this paper

Estimating Counterfactual Treatment Effects to Assess External Validity

Author

Abstract
We propose statistical methods for assessing the external validity of treatment effect estimates obtained in a specific status-quo environment. In particular, we estimate counterfactual quantile treatment effects that would obtain if one were to change the composition of the population targeted by the status-quo treatment. Assuming unconfoundedness, and the invariance of the conditional distributions of the potential outcomes, the parameter of interest is identified and can be nonparametrically estimated by a kernel-based method. Viewed as a random function over the continuum of quantile indices, the estimator converges weakly to a zero mean Gaussian process at the parametric rate. Exploiting this result, we propose a multiplier bootstrap procedure to construct uniform confidence bands. We provide similar results for the counterfactually treated subpopulation and the average effect. As an application, we estimate the counterfactual quantile treatment effect of the Job Corps training program in the U.S. under various scenarios. The results suggest that strong economic conditions and the skill hypotheses both help explain the earlier finding in the literature that the program was ineffective at low quantiles of the earnings distribution.

Suggested Citation

  • Yu-Chin Hsu & Tsung-Chih Lai & Robert P. Lieli, 2017. "Estimating Counterfactual Treatment Effects to Assess External Validity," IEAS Working Paper : academic research 17-A011, Institute of Economics, Academia Sinica, Taipei, Taiwan.
    • Handle: RePEc:sin:wpaper:17-a011
    as

    Download full text from publisher

    File URL: https://www.econ.sinica.edu.tw/~econ/pdfPaper/17-A011.pdf
    Download Restriction: no
    ---><---

    References listed on IDEAS

    as
    1. Sergio Firpo & Nicole M. Fortin & Thomas Lemieux, 2009. "Unconditional Quantile Regressions," Econometrica, Econometric Society, vol. 77(3), pages 953-973, May.
    Full references (including those not matched with items on IDEAS)

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Korom, Philipp, 2016. "Inherited advantage: The importance of inheritance for private wealth accumulation in Europe," MPIfG Discussion Paper 16/11, Max Planck Institute for the Study of Societies.
    2. Joanna Tyrowicz & Lucas van der Velde, 2017. "When the opportunity knocks: large structural shocks and gender wage gaps," GRAPE Working Papers 2, GRAPE Group for Research in Applied Economics.
    3. Michele Raitano & Francesco Vona & Claudia Vittori, 2015. "The effect of parental background along the son's earnings distribution : does one model fit for all?," Working Papers hal-03459749, HAL.
    4. Huong Thu Le & Ha Trong Nguyen, 2018. "The evolution of the gender test score gap through seventh grade: new insights from Australia using unconditional quantile regression and decomposition," IZA Journal of Labor Economics, Springer;Forschungsinstitut zur Zukunft der Arbeit GmbH (IZA), vol. 7(1), pages 1-42, December.
    5. Holmes, Craig & Mayhew, Ken, 2015. "Have UK Earnings Distributions Polarised?," INET Oxford Working Papers 2015-02, Institute for New Economic Thinking at the Oxford Martin School, University of Oxford.
    6. Francis,David C. & Kubinec ,Robert, 2022. "Beyond Political Connections : A Measurement Model Approach to Estimating Firm-levelPolitical Influence in 41 Economies," Policy Research Working Paper Series 10119, The World Bank.
    7. Xiaoying Liu & Jere R. Behrman & Emily Hannum & Fan Wang & Qingguo Zhao, 2022. "Same environment, stratified impacts? Air pollution, extreme temperatures, and birth weight in south China," Papers 2204.00219, arXiv.org.
    8. Katie Meara & Francesco Pastore & Allan Webster, 2020. "The gender pay gap in the USA: a matching study," Journal of Population Economics, Springer;European Society for Population Economics, vol. 33(1), pages 271-305, January.
    9. Sergio Longobardi & Margherita Maria Pagliuca & Andrea Regoli, 2018. "Can problem-solving attitudes explain the gender gap in financial literacy? Evidence from Italian students’ data," Quality & Quantity: International Journal of Methodology, Springer, vol. 52(4), pages 1677-1705, July.
    10. Victor Chernozhukov & Iván Fernández‐Val & Blaise Melly, 2013. "Inference on Counterfactual Distributions," Econometrica, Econometric Society, vol. 81(6), pages 2205-2268, November.
    11. Jaanika Meriküll & Maryna Tverdostup, 2020. "The Gap That Survived The Transition: The Gender Wage Gap Over Three Decades In Estonia," University of Tartu - Faculty of Economics and Business Administration Working Paper Series 127, Faculty of Economics and Business Administration, University of Tartu (Estonia).
    12. Brahim Boudarbat & Marie Connolly, 2013. "The gender wage gap among recent post‐secondary graduates in Canada: a distributional approach," Canadian Journal of Economics/Revue canadienne d'économique, John Wiley & Sons, vol. 46(3), pages 1037-1065, August.
    13. Cupák, Andrej & Fessler, Pirmin & Hsu, Joanne W. & Paradowski, Piotr R., 2022. "Investor confidence and high financial literacy jointly shape investments in risky assets," Economic Modelling, Elsevier, vol. 116(C).
    14. Daniel Fackler & Lisa Hölscher & Claus Schnabel & Antje Weyh, 2022. "Does working at a start-up pay off?," Small Business Economics, Springer, vol. 58(4), pages 2211-2233, April.
    15. Männasoo, Kadri, 2022. "Working hours and gender wage differentials: Evidence from the American Working Conditions Survey," Labour Economics, Elsevier, vol. 76(C).
    16. Martínez-Iriarte, Julián & Montes-Rojas, Gabriel & Sun, Yixiao, 2024. "Unconditional effects of general policy interventions," Journal of Econometrics, Elsevier, vol. 238(2).
    17. Vincenzo Carrieri & Francesco Principe & Michele Raitano, 2018. "What makes you ‘super-rich’? New evidence from an analysis of football players’ wages," Oxford Economic Papers, Oxford University Press, vol. 70(4), pages 950-973.
    18. Jean-Marc Fournier & Isabell Koske, 2012. "The determinants of earnings inequality: evidence from quantile regressions," OECD Journal: Economic Studies, OECD Publishing, vol. 2012(1), pages 7-36.
    19. German Blanco & Carlos A. Flores & Alfonso Flores-Lagunes, 2013. "Bounds on Average and Quantile Treatment Effects of Job Corps Training on Wages," Journal of Human Resources, University of Wisconsin Press, vol. 48(3), pages 659-701.
    20. Aswini Kumar Mishra & Vedant Bhardwaj, 2021. "Wealth distribution and accounting for changes in wealth inequality: empirical evidence from India, 1991–2012," Economic Change and Restructuring, Springer, vol. 54(2), pages 585-620, May.

    More about this item

    Keywords

    counterfactual analysis; external validity; program evaluation; multiplier bootstrap; Job Corps JEL Classification: C13; C31; J24; J30;
    All these keywords.

    JEL classification:

    • C13 - Mathematical and Quantitative Methods - - Econometric and Statistical Methods and Methodology: General - - - Estimation: General
    • C31 - Mathematical and Quantitative Methods - - Multiple or Simultaneous Equation Models; Multiple Variables - - - Cross-Sectional Models; Spatial Models; Treatment Effect Models; Quantile Regressions; Social Interaction Models
    • J24 - Labor and Demographic Economics - - Demand and Supply of Labor - - - Human Capital; Skills; Occupational Choice; Labor Productivity
    • J30 - Labor and Demographic Economics - - Wages, Compensation, and Labor Costs - - - General

    NEP fields

    This paper has been announced in the following NEP Reports:

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:sin:wpaper:17-a011. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: HsiaoyunLiu (email available below). General contact details of provider: https://edirc.repec.org/data/sinictw.html .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.