[go: up one dir, main page]
IDEAS home Printed from https://ideas.repec.org/a/bla/brjirl/v62y2024i1p3-27.html
   My bibliography  Save this article

Power and bias in industrial relations research

Author

Listed:
  • Anthony Doucouliagos
  • Hristos Doucouliagos
  • T. D. Stanley
Abstract
We survey 20,439 estimates from 64 distinct research areas to assess power, bias and statistical significance in industrial relations research. The average estimate published in industrial relations research lacks adequate power; average power is 33 per cent, and median power is only 14 per cent, much lower than the conventional 80 per cent standard. Low power means that industrial relations researchers will find it more difficult to detect important associations pertaining to workplace relations. Low power also leads to exaggerated research findings. We find substantial publication bias in industrial relations research, though nearly half of the research areas have little or no bias.

Suggested Citation

  • Anthony Doucouliagos & Hristos Doucouliagos & T. D. Stanley, 2024. "Power and bias in industrial relations research," British Journal of Industrial Relations, London School of Economics, vol. 62(1), pages 3-27, March.
    • Handle: RePEc:bla:brjirl:v:62:y:2024:i:1:p:3-27
    DOI: 10.1111/bjir.12746
    as

    Download full text from publisher

    File URL: https://doi.org/10.1111/bjir.12746
    Download Restriction: no
    File URL: https://libkey.io/10.1111/bjir.12746?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    References listed on IDEAS

    as
    1. John P. A. Ioannidis & T. D. Stanley & Hristos Doucouliagos, 2017. "The Power of Bias in Economics Research," Economic Journal, Royal Economic Society, vol. 127(605), pages 236-265, October.
    2. Abel Brodeur & Mathias Lé & Marc Sangnier & Yanos Zylberberg, 2016. "Star Wars: The Empirics Strike Back," American Economic Journal: Applied Economics, American Economic Association, vol. 8(1), pages 1-32, January.
    3. Andrew C. Chang & Phillip Li, 2017. "A Preanalysis Plan to Replicate Sixty Economics Research Papers That Worked Half of the Time," American Economic Review, American Economic Association, vol. 107(5), pages 60-64, May.
    4. Alexandre Flage, 2019. "Discrimination against gays and lesbians in hiring decisions: a meta-analysis," International Journal of Manpower, Emerald Group Publishing Limited, vol. 41(6), pages 671-691, September.
    5. Henrik Kleven & Camille Landais & Johanna Posch & Andreas Steinhauer & Josef Zweimüller, 2019. "Child Penalties across Countries: Evidence and Explanations," AEA Papers and Proceedings, American Economic Association, vol. 109, pages 122-126, May.
    6. Cristina Blanco-Perez & Abel Brodeur, 2020. "Publication Bias and Editorial Statement on Negative Findings," The Economic Journal, Royal Economic Society, vol. 130(629), pages 1226-1247.
    7. Garret Christensen & Edward Miguel, 2018. "Transparency, Reproducibility, and the Credibility of Economics Research," Journal of Economic Literature, American Economic Association, vol. 56(3), pages 920-980, September.
    8. Hristos Doucouliagos & Janto Haman & T.D. Stanley, 2012. "Pay for Performance and Corporate Governance Reform," Industrial Relations: A Journal of Economy and Society, Wiley Blackwell, vol. 51(3), pages 670-703, July.
    9. Megan Linde Leonard & T. D. Stanley & Hristos Doucouliagos, 2014. "Does the UK Minimum Wage Reduce Employment? A Meta-Regression Analysis," British Journal of Industrial Relations, London School of Economics, vol. 52(3), pages 499-520, September.
    10. de Linde Leonard, Megan & Stanley, T.D., 2015. "Married with children: What remains when observable biases are removed from the reported male marriage wage premium," Labour Economics, Elsevier, vol. 33(C), pages 72-80.
    11. Hristos Doucouliagos & Richard B. Freeman & Patrice Laroche & T. D. Stanley, 2018. "How Credible Is Trade Union Research? Forty Years of Evidence on the Monopoly–Voice Trade-Off," ILR Review, Cornell University, ILR School, vol. 71(2), pages 287-305, March.
    12. Chletsos, Michael & Giotis, Georgios P., 2015. "The employment effect of minimum wage using 77 international studies since 1992: A meta-analysis," MPRA Paper 61321, University Library of Munich, Germany.
    13. Hristos Doucouliagos & T. D. Stanley, 2009. "Publication Selection Bias in Minimum‐Wage Research? A Meta‐Regression Analysis," British Journal of Industrial Relations, London School of Economics, vol. 47(2), pages 406-428, June.
    14. Paul Gertler & Sebastian Galiani & Mauricio Romero, 2018. "How to make replication the norm," Nature, Nature, vol. 554(7693), pages 417-419, February.
    15. Hristos Doucouliagos & Richard Freeman & Patrice Laroche & T.D. Stanley, 2018. "How Credible Is Trade Union Research? Forty Years of Evidence on the Monopoly–Voice Trade-Off," Post-Print hal-02137429, HAL.
    16. Keith Whitfield & George Strauss, 2000. "Methods Matter: Changes in Industrial Relations Research and their Implications," British Journal of Industrial Relations, London School of Economics, vol. 38(1), pages 141-151, March.
    17. Dale Belman & Paul J. Wolfson, 2014. "What Does the Minimum Wage Do?," Books from Upjohn Press, W.E. Upjohn Institute for Employment Research, number wdmwd.
    18. Neal S Young & John P A Ioannidis & Omar Al-Ubaydli, 2008. "Why Current Publication Practices May Distort Science," PLOS Medicine, Public Library of Science, vol. 5(10), pages 1-5, October.
    19. Camerer, Colin & Dreber, Anna & Forsell, Eskil & Ho, Teck-Hua & Huber, Jurgen & Johannesson, Magnus & Kirchler, Michael & Almenberg, Johan & Altmejd, Adam & Chan, Taizan & Heikensten, Emma & Holzmeist, 2016. "Evaluating replicability of laboratory experiments in Economics," MPRA Paper 75461, University Library of Munich, Germany.
    20. Chris Doucouliagos & T.D. Stanley, 2013. "Are All Economic Facts Greatly Exaggerated? Theory Competition And Selectivity," Journal of Economic Surveys, Wiley Blackwell, vol. 27(2), pages 316-339, April.
    21. Jessica S. Merkle & Michelle Andrea Phillips, 2018. "The Wage Impact Of Teachers Unions: A Meta‐Analysis," Contemporary Economic Policy, Western Economic Association International, vol. 36(1), pages 93-115, January.
    Full references (including those not matched with items on IDEAS)

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Stanley, T. D. & Doucouliagos, Chris, 2019. "Practical Significance, Meta-Analysis and the Credibility of Economics," IZA Discussion Papers 12458, Institute of Labor Economics (IZA).
    2. Guillaume Coqueret, 2023. "Forking paths in financial economics," Papers 2401.08606, arXiv.org.
    3. repec:wly:econjl:v::y:2017:i:605:p:f236-f265 is not listed on IDEAS
    4. John P. A. Ioannidis & T. D. Stanley & Hristos Doucouliagos, 2017. "The Power of Bias in Economics Research," Economic Journal, Royal Economic Society, vol. 127(605), pages 236-265, October.
    5. Doucouliagos, Hristos & Hinz, Thomas & Zigova, Katarina, 2022. "Bias and careers: Evidence from the aid effectiveness literature," European Journal of Political Economy, Elsevier, vol. 71(C).
    6. Furukawa, Chishio, 2019. "Publication Bias under Aggregation Frictions: Theory, Evidence, and a New Correction Method," EconStor Preprints 194798, ZBW - Leibniz Information Centre for Economics.
    7. Jindrich Matousek & Tomas Havranek & Zuzana Irsova, 2022. "Individual discount rates: a meta-analysis of experimental evidence," Experimental Economics, Springer;Economic Science Association, vol. 25(1), pages 318-358, February.
    8. Dreber, Anna & Johannesson, Magnus, 2023. "A framework for evaluating reproducibility and replicability in economics," Ruhr Economic Papers 1055, RWI - Leibniz-Institut für Wirtschaftsforschung, Ruhr-University Bochum, TU Dortmund University, University of Duisburg-Essen.
    9. Isaiah Andrews & Maximilian Kasy, 2019. "Identification of and Correction for Publication Bias," American Economic Review, American Economic Association, vol. 109(8), pages 2766-2794, August.
    10. Tomas Havranek & Zuzana Irsova & Lubica Laslopova & Olesia Zeynalova, 2020. "Skilled and Unskilled Labor Are Less Substitutable than Commonly Thought," Working Papers IES 2020/29, Charles University Prague, Faculty of Social Sciences, Institute of Economic Studies, revised Sep 2020.
    11. Abel Brodeur & Nikolai Cook & Carina Neisser, 2024. "p-Hacking, Data type and Data-Sharing Policy," The Economic Journal, Royal Economic Society, vol. 134(659), pages 985-1018.
    12. Cazachevici, Alina & Havranek, Tomas & Horvath, Roman, 2020. "Remittances and economic growth: A meta-analysis," World Development, Elsevier, vol. 134(C).
    13. Fišar, Miloš & Greiner, Ben & Huber, Christoph & Katok, Elena & Ozkes, Ali & Management Science Reproducibility Collaboration, 2023. "Reproducibility in Management Science," Department for Strategy and Innovation Working Paper Series 03/2023, WU Vienna University of Economics and Business.
    14. Ankel-Peters, Jörg & Fiala, Nathan & Neubauer, Florian, 2023. "Do economists replicate?," Journal of Economic Behavior & Organization, Elsevier, vol. 212(C), pages 219-232.
    15. Abel Brodeur & Nikolai Cook & Anthony Heyes, 2020. "Methods Matter: p-Hacking and Publication Bias in Causal Analysis in Economics," American Economic Review, American Economic Association, vol. 110(11), pages 3634-3660, November.
    16. Nick Huntington‐Klein & Andreu Arenas & Emily Beam & Marco Bertoni & Jeffrey R. Bloem & Pralhad Burli & Naibin Chen & Paul Grieco & Godwin Ekpe & Todd Pugatch & Martin Saavedra & Yaniv Stopnitzky, 2021. "The influence of hidden researcher decisions in applied microeconomics," Economic Inquiry, Western Economic Association International, vol. 59(3), pages 944-960, July.
    17. Dimos, Christos & Pugh, Geoff & Hisarciklilar, Mehtap & Talam, Ema & Jackson, Ian, 2022. "The relative effectiveness of R&D tax credits and R&D subsidies: A comparative meta-regression analysis," Technovation, Elsevier, vol. 115(C).
    18. Brian Fabo & Martina Jancokova & Elisabeth Kempf & Lubos Pastor, 2020. "Fifty Shades of QE: Conflicts of Interest in Economic Research," Working and Discussion Papers WP 5/2020, Research Department, National Bank of Slovakia.
    19. Nazila Alinaghi & W. Robert Reed, 2021. "Taxes and Economic Growth in OECD Countries: A Meta-analysis," Public Finance Review, , vol. 49(1), pages 3-40, January.
    20. Zigraiova, Diana & Havranek, Tomas & Irsova, Zuzana & Novak, Jiri, 2021. "How puzzling is the forward premium puzzle? A meta-analysis," European Economic Review, Elsevier, vol. 134(C).
    21. Maurizio Canavari & Andreas C. Drichoutis & Jayson L. Lusk & Rodolfo M. Nayga, Jr., 2018. "How to run an experimental auction: A review of recent advances," Working Papers 2018-5, Agricultural University of Athens, Department Of Agricultural Economics.

    More about this item

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:bla:brjirl:v:62:y:2024:i:1:p:3-27. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Wiley Content Delivery (email available below). General contact details of provider: https://edirc.repec.org/data/lsepsuk.html .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.