[go: up one dir, main page]
IDEAS home Printed from https://ideas.repec.org/a/taf/houspd/v33y2023i2p453-486.html
   My bibliography  Save this article

Evaluating Contradictory Experimental and Nonexperimental Estimates of Neighborhood Effects on Economic Outcomes for Adults

Author

Listed:
  • David J. Harding
  • Lisa Sanbonmatsu
  • Greg J. Duncan
  • Lisa A. Gennetian
  • Lawrence F. Katz
  • Ronald C. Kessler
  • Jeffrey R. Kling
  • Matthew Sciandra
  • Jens Ludwig
Abstract
Although nonexperimental studies find robust neighborhood effects on adults, such findings have been challenged by results from the Moving to Opportunity (MTO) residential mobility experiment. Using a within-study comparison design, this article compares experimental and nonexperimental estimates from MTO and a parallel analysis of the Panel Study of Income Dynamics (PSID). Striking similarities were found between nonexperimental estimates based on MTO and PSID. No clear evidence was found that different estimates are related to duration of adult exposure to disadvantaged neighborhoods, nonlinear effects of neighborhood conditions, magnitude of the change in neighborhood context, frequency of moves, treatment effect heterogeneity, or measurement, although the uncertainty bands around our estimates were sometimes large. Another possibility is that MTO-induced moves might have been unusually disruptive, but results are inconsistent for that hypothesis. Taken together, the findings suggest that selection bias might account for evidence of neighborhood effects on adult economic outcomes in nonexperimental studies.

Suggested Citation

  • David J. Harding & Lisa Sanbonmatsu & Greg J. Duncan & Lisa A. Gennetian & Lawrence F. Katz & Ronald C. Kessler & Jeffrey R. Kling & Matthew Sciandra & Jens Ludwig, 2023. "Evaluating Contradictory Experimental and Nonexperimental Estimates of Neighborhood Effects on Economic Outcomes for Adults," Housing Policy Debate, Taylor & Francis Journals, vol. 33(2), pages 453-486, March.
    • Handle: RePEc:taf:houspd:v:33:y:2023:i:2:p:453-486
    DOI: 10.1080/10511482.2021.1881985
    as

    Download full text from publisher

    File URL: http://hdl.handle.net/10.1080/10511482.2021.1881985
    Download Restriction: Access to full text is restricted to subscribers.
    File URL: https://libkey.io/10.1080/10511482.2021.1881985?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    As the access to this document is restricted, you may want to look for a different version below or search for a different version of it.

    Other versions of this item:

    References listed on IDEAS

    as
    1. Richard K. Crump & V. Joseph Hotz & Guido W. Imbens & Oscar A. Mitnik, 2009. "Dealing with limited overlap in estimation of average treatment effects," Biometrika, Biometrika Trust, vol. 96(1), pages 187-199.
    2. Brian A. Jacob, 2004. "Public Housing, Housing Vouchers, and Student Achievement: Evidence from Public Housing Demolitions in Chicago," American Economic Review, American Economic Association, vol. 94(1), pages 233-258, March.
    3. LaLonde, Robert J, 1986. "Evaluating the Econometric Evaluations of Training Programs with Experimental Data," American Economic Review, American Economic Association, vol. 76(4), pages 604-620, September.
    4. Raj Chetty & Nathaniel Hendren, 2018. "The Impacts of Neighborhoods on Intergenerational Mobility II: County-Level Estimates," The Quarterly Journal of Economics, President and Fellows of Harvard College, vol. 133(3), pages 1163-1228.
    5. David McKenzie & John Gibson & Steven Stillman, 2006. "How Important is Selection? Experimental vs Non-experimental Measures of the Income Gains of Migration," Working Papers 06_02, Motu Economic and Public Policy Research.
    6. McKenzie, David & Gibson, John & Stillman, Steven, 2006. "How important is selection ? Experimental versus non-experimental measures of the income gains from migration," Policy Research Working Paper Series 3906, The World Bank.
    7. Hilary Williamson Hoynes, 2000. "Local Labor Markets And Welfare Spells: Do Demand Conditions Matter?," The Review of Economics and Statistics, MIT Press, vol. 82(3), pages 351-368, August.
    8. Raj Chetty & Nathaniel Hendren, 2018. "The Impacts of Neighborhoods on Intergenerational Mobility I: Childhood Exposure Effects," The Quarterly Journal of Economics, President and Fellows of Harvard College, vol. 133(3), pages 1107-1162.
    9. Silver, Eric & Mulvey, Edward P. & Swanson, Jeffrey W., 2002. "Neighborhood structural characteristics and mental disorder: Faris and Dunham revisited," Social Science & Medicine, Elsevier, vol. 55(8), pages 1457-1470, October.
    10. Katherine M. O'Regan & John M. Quigley, 1996. "Teenage Employment and the Spatial Isolation of Minority and Poverty Households," Journal of Human Resources, University of Wisconsin Press, vol. 31(3), pages 692-702.
    11. Dionissi Aliprantis & Francisca G.-C. Richter, 2020. "Evidence of Neighborhood Effects from Moving to Opportunity: Lates of Neighborhood Quality," The Review of Economics and Statistics, MIT Press, vol. 102(4), pages 633-647, October.
    12. George Galster, 2017. "People Versus Place, People and Place, or More? New Directions for Housing Policy," Housing Policy Debate, Taylor & Francis Journals, vol. 27(2), pages 261-265, March.
    13. Sean F. Reardon & Kendra Bischoff & Ann Owens & Joseph B. Townsend, 2018. "Has Income Segregation Really Increased? Bias and Bias Correction in Sample-Based Segregation Estimates," Demography, Springer;Population Association of America (PAA), vol. 55(6), pages 2129-2160, December.
    14. Katz, Lawrence & Duncan, Greg J. & Kling, Jeffrey R. & Kessler, Ronald C. & Ludwig, Jens & Sanbonmatsu, Lisa & Liebman, Jeffrey B., 2008. "What Can We Learn about Neighborhood Effects from the Moving to Opportunity Experiment?," Scholarly Articles 2766959, Harvard University Department of Economics.
    15. Roberto Agodini & Mark Dynarski, 2004. "Are Experiments the Only Option? A Look at Dropout Prevention Programs," The Review of Economics and Statistics, MIT Press, vol. 86(1), pages 180-194, February.
    16. Charles Michalopoulos & Howard S. Bloom & Carolyn J. Hill, 2004. "Can Propensity-Score Methods Match the Findings from a Random Assignment Evaluation of Mandatory Welfare-to-Work Programs?," The Review of Economics and Statistics, MIT Press, vol. 86(1), pages 156-179, February.
    17. David McKenzie & John Gibson & Steven Stillman, 2010. "How Important Is Selection? Experimental vs. Non-Experimental Measures of the Income Gains from Migration," Journal of the European Economic Association, MIT Press, vol. 8(4), pages 913-945, June.
    18. Philip Oreopoulos, 2003. "The Long-Run Consequences of Living in a Poor Neighborhood," The Quarterly Journal of Economics, President and Fellows of Harvard College, vol. 118(4), pages 1533-1575.
    19. Eric Chyn, 2018. "Moved to Opportunity: The Long-Run Effects of Public Housing Demolition on Children," American Economic Review, American Economic Association, vol. 108(10), pages 3028-3056, October.
    20. Jessie P H Poon, 2005. "Women in Economic Geography," Environment and Planning A, , vol. 37(5), pages 765-768, May.
    21. George Galster & Anna Maria Santiago, 2017. "Do Neighborhood Effects on Low-Income Minority Children Depend on Their Age? Evidence From a Public Housing Natural Experiment," Housing Policy Debate, Taylor & Francis Journals, vol. 27(4), pages 584-610, July.
    22. Virginia Parks, 2004. "Access to Work: The Effects of Spatial and Social Accessibility on Unemployment for Native-Born Black and Immigrant Women in Los Angeles," Economic Geography, Taylor & Francis Journals, vol. 80(2), pages 141-172, April.
    23. Gurmu, Shiferaw & Ihlanfeldt, Keith R. & Smith, William J., 2008. "Does residential location matter to the employment of TANF recipients Evidence from a dynamic discrete choice model with unobserved effects," Journal of Urban Economics, Elsevier, vol. 63(1), pages 325-351, January.
    24. Raj Chetty & Nathaniel Hendren & Lawrence F. Katz, 2016. "The Effects of Exposure to Better Neighborhoods on Children: New Evidence from the Moving to Opportunity Experiment," American Economic Review, American Economic Association, vol. 106(4), pages 855-902, April.
    25. David A. Reingold & Gregg G. Van Ryzin & Michelle Ronda, 2001. "Does Urban Public Housing Diminish the Social Capital and Labor Force Activity of Its Tenants?," Journal of Policy Analysis and Management, John Wiley & Sons, Ltd., vol. 20(3), pages 485-504.
    26. Tara Watson, 2009. "Inequality And The Measurement Of Residential Segregation By Income In American Neighborhoods," Review of Income and Wealth, International Association for Research in Income and Wealth, vol. 55(3), pages 820-844, September.
    27. Elizabeth Ty Wilde & Robinson Hollister, 2007. "How close is close enough? Evaluating propensity score matching using data from a class size reduction experiment," Journal of Policy Analysis and Management, John Wiley & Sons, Ltd., vol. 26(3), pages 455-477.
    28. Brian A. Jacob & Jens Ludwig, 2012. "The Effects of Housing Assistance on Labor Supply: Evidence from a Voucher Lottery," American Economic Review, American Economic Association, vol. 102(1), pages 272-304, February.
    29. Diez Roux, A.V., 2001. "Investigating neighborhood and area effects on health," American Journal of Public Health, American Public Health Association, vol. 91(11), pages 1783-1789.
    30. Brian A. Jacob & Max Kapustin & Jens Ludwig, 2015. "The Impact of Housing Assistance on Child Outcomes: Evidence from a Randomized Housing Lottery," The Quarterly Journal of Economics, President and Fellows of Harvard College, vol. 130(1), pages 465-506.
    31. John F. Kain, 1968. "Housing Segregation, Negro Employment, and Metropolitan Decentralization," The Quarterly Journal of Economics, President and Fellows of Harvard College, vol. 82(2), pages 175-197.
    32. George Galster & Lina Hedman, 2013. "Measuring Neighbourhood Effects Non-experimentally: How Much Do Alternative Methods Matter?," Housing Studies, Taylor & Francis Journals, vol. 28(3), pages 473-498, April.
    33. Harry J. Holzer, 1991. "The Spatial Mismatch Hypothesis: What Has the Evidence Shown?," Urban Studies, Urban Studies Journal Limited, vol. 28(1), pages 105-122, February.
    34. Thomas Fraker & Rebecca Maynard, 1987. "The Adequacy of Comparison Group Designs for Evaluations of Employment-Related Programs," Journal of Human Resources, University of Wisconsin Press, vol. 22(2), pages 194-227.
    35. Duncan, Greg J. & Katz, Lawrence F. & Kessler, Ronald & Kling, Jeffrey R. & Gennetian, Lisa & Adam, Emma & Ludwig, Jens & Sanbonmatsu, Lisa & Tessler, Stacy & McDade, Thomas W. & Whitaker, Robert C., 2011. "Neighborhoods, Obesity and Diabetes –-- A Randomized Social Experiment," Scholarly Articles 8642951, Harvard University Department of Economics.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Jo Blanden & Matthias Doepke & Jan Stuhler, 2022. "Education inequality," CEP Discussion Papers dp1849, Centre for Economic Performance, LSE.
    2. Hu, Jinhua & Mu, Fan & Jiang, Xinling & Wu, Zhong'an & Olasehinde, Toba & Fan, Yubing & Wang, Tong, 2024. "Herders’ willingness to adopt Climate-Smart Grassland Agriculture: Evidence from the Qilian Mountain region of Northwestern China," 2024 Annual Meeting, July 28-30, New Orleans, LA 343634, Agricultural and Applied Economics Association.
    3. Belchior, Carlos Alberto & Gonzaga, Gustavo & Ulyssea, Gabriel, 2023. "Unpacking Neighborhood Effects: Experimental Evidence from a Large-Scale Housing Program in Brazil," IZA Discussion Papers 16113, Institute of Labor Economics (IZA).
    4. Austin, Wes & Figlio, David & Goldhaber, Dan & Hanushek, Eric A. & Kilbride, Tara & Koedel, Cory & Sean Lee, Jaeseok & Lou, Jin & Özek, Umut & Parsons, Eric & Rivkin, Steven G. & Sass, Tim R. & Strunk, 2023. "Academic mobility in U.S. public schools: Evidence from nearly 3 million students," Journal of Public Economics, Elsevier, vol. 228(C).
    5. Sandher, Jeevun, 2022. "Familiar Faces, Worn Out Places: The Effect of Personal and Place Prosperity On Well-Being," SocArXiv am6gq, Center for Open Science.

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Eric Chyn & Lawrence F. Katz, 2021. "Neighborhoods Matter: Assessing the Evidence for Place Effects," Journal of Economic Perspectives, American Economic Association, vol. 35(4), pages 197-222, Fall.
    2. Lorenzo Neri, 2020. "Moving Opportunities: The Impact of Public Housing Re-generations on Student Achievement," Working Papers 907, Queen Mary University of London, School of Economics and Finance.
    3. Fredrik Andersson & John C. Haltiwanger & Mark J. Kutzbach & Giordano Palloni & Henry O. Pollakowski & Daniel H. Weinberg, 2013. "Childhood Housing and Adult Earnings: A Between-Siblings Analysis of Housing Vouchers and Public Housing," Working Papers 13-48, Center for Economic Studies, U.S. Census Bureau.
    4. Cassidy, Michael T., 2020. "A Closer Look: Proximity Boosts Homeless Student Performance in New York City," IZA Discussion Papers 13558, Institute of Labor Economics (IZA).
    5. Jipeng Zhang & Lue Zhan & Chong Lu, 2020. "The long-run effects of poverty alleviation resettlement on child development: Evidence from a quasi-experiment in China," Demographic Research, Max Planck Institute for Demographic Research, Rostock, Germany, vol. 43(10), pages 245-284.
    6. Ferraro, Paul J. & Miranda, Juan José, 2014. "The performance of non-experimental designs in the evaluation of environmental programs: A design-replication study using a large-scale randomized experiment as a benchmark," Journal of Economic Behavior & Organization, Elsevier, vol. 107(PA), pages 344-365.
    7. ANDREOLI Francesco & MUSSINI Mauro & PRETE Vincenzo, 2019. "Urban poverty: Theory and evidence from American cities," LISER Working Paper Series 2019-07, Luxembourg Institute of Socio-Economic Research (LISER).
    8. Francesco Andreoli & Mauro Mussini & Vincenzo Prete & Claudio Zoli, 2021. "Urban poverty: Measurement theory and evidence from American cities," The Journal of Economic Inequality, Springer;Society for the Study of Economic Inequality, vol. 19(4), pages 599-642, December.
    9. Vivian C. Wong & Peter M. Steiner & Kylie L. Anglin, 2018. "What Can Be Learned From Empirical Evaluations of Nonexperimental Methods?," Evaluation Review, , vol. 42(2), pages 147-175, April.
    10. Tarek Azzam & Michael Bates & David Fairris, 2019. "Do Learning Communities Increase First Year College Retention? Testing Sample Selection and External Validity of Randomized Control Trials," Working Papers 202002, University of California at Riverside, Department of Economics.
    11. Simen Markussen & Knut Røed, 2023. "Are richer neighborhoods always better for the kids?," Journal of International Economic Law, Oxford University Press, vol. 23(3), pages 629-651.
    12. Chong Lu, 2022. "The effect of migration on rural residents’ intergenerational subjective social status mobility in China," Quality & Quantity: International Journal of Methodology, Springer, vol. 56(5), pages 3279-3308, October.
    13. Dionissi Aliprantis & Hal Martin & Kristen Tauber, 2020. "What Determines the Success of Housing Mobility Programs?," Working Papers 20-36R, Federal Reserve Bank of Cleveland, revised 19 Oct 2022.
    14. Bryan A. Stuart, 2022. "The Long-Run Effects of Recessions on Education and Income," American Economic Journal: Applied Economics, American Economic Association, vol. 14(1), pages 42-74, January.
    15. Ludwig, Jens & Duncan, Greg J. & Katz, Lawrence F. & Kessler, Ronald & Kling, Jeffrey R. & Gennetian, Lisa A. & Sanbonmatsu, Lisa, 2012. "Neighborhood Effects on the Long-Term Well-Being of Low-Income Adults," Scholarly Articles 11870359, Harvard University Department of Economics.
    16. Aarland, Kristin & Santiago, Anna Maria & Galster, George C. & Nordvik, Viggo, 2021. "Childhood Housing Tenure and Young Adult Educational Outcomes: Evidence from Sibling Comparisons in Norway," Journal of Housing Economics, Elsevier, vol. 54(C).
    17. John C. Haltiwanger & Mark J. Kutzbach & Giordano Palloni & Henry O. Pollakowski & Matthew Staiger & Daniel H. Weinberg, 2020. "The Children of HOPE VI Demolitions: National Evidence on Labor Market Outcomes," Working Papers 20-39, Center for Economic Studies, U.S. Census Bureau.
    18. Belchior, Carlos Alberto & Gonzaga, Gustavo & Ulyssea, Gabriel, 2023. "Unpacking Neighborhood Effects: Experimental Evidence from a Large-Scale Housing Program in Brazil," IZA Discussion Papers 16113, Institute of Labor Economics (IZA).
    19. Fletcher, Jason M. & Ross, Stephen L. & Zhang, Yuxiu, 2020. "The consequences of friendships: Evidence on the effect of social relationships in school on academic achievement," Journal of Urban Economics, Elsevier, vol. 116(C).
    20. Ashlesha Datar & Nancy Nicosia & Anya Samek, 2022. "Heterogeneity in Place Effects on Health: The Case of Time Preferences and Adolescent Obesity," NBER Working Papers 29935, National Bureau of Economic Research, Inc.

    More about this item

    JEL classification:

    • H0 - Public Economics - - General
    • I3 - Health, Education, and Welfare - - Welfare, Well-Being, and Poverty
    • J0 - Labor and Demographic Economics - - General

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:taf:houspd:v:33:y:2023:i:2:p:453-486. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Chris Longhurst (email available below). General contact details of provider: http://www.tandfonline.com/RHPD20 .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.