[go: up one dir, main page]
IDEAS home Printed from https://ideas.repec.org/p/iza/izadps/dp2070.html
   My bibliography  Save this paper

An Examination of the Reliability of Prestigious Scholarly Journals: Evidence and Implications for Decision-makers

Author

Listed:
  • Oswald, Andrew J.

    (University of Warwick)

Abstract
In universities all over the world, hiring and promotion committees regularly hear the argument: “this is important work because it is about to appear in prestigious journal X”. Moreover, those who allocate levels of research funding, such as in the multi-billion pound Research Assessment Exercise in UK universities, often come under pressure to assess research quality in a mechanical way by using journal prestige ratings. The results in this paper suggest that such tendencies are dangerous. It uses total citations over a quarter of a century as the criterion. The paper finds that it is far better to publish the best article in an issue of a medium-quality journal like the Oxford Bulletin of Economics and Statistics than to publish the worst article (or often the worst 4 articles) in an issue of a top journal like the American Economic Review. Implications are discussed.

Suggested Citation

  • Oswald, Andrew J., 2006. "An Examination of the Reliability of Prestigious Scholarly Journals: Evidence and Implications for Decision-makers," IZA Discussion Papers 2070, Institute of Labor Economics (IZA).
    • Handle: RePEc:iza:izadps:dp2070
    as

    Download full text from publisher

    File URL: https://docs.iza.org/dp2070.pdf
    Download Restriction: no
    ---><---

    Other versions of this item:

    References listed on IDEAS

    as
    1. Jerry G. Thursby, 2000. "What Do We Say about Ourselves and What Does It Mean? Yet Another Look at Economics Department Research," Journal of Economic Literature, American Economic Association, vol. 38(2), pages 383-404, June.
    2. Anthony F J van Raan, 1998. "Assessment of social sciences: The use of advanced bibliometric methods as a necessary complement of peer review," Research Evaluation, Oxford University Press, vol. 7(1), pages 2-6, April.
    3. Nancy K. Bayers, 2005. "Using ISI data in the analysis of German national and institutional research output," Scientometrics, Springer;Akadémiai Kiadó, vol. 62(1), pages 155-163, January.
    4. David N. Laband, 1990. "Is There Value-Added from the Review Process in Economics?: Preliminary Evidence from Authors," The Quarterly Journal of Economics, President and Fellows of Harvard College, vol. 105(2), pages 341-352.
    5. Moore, William J & Newman, Robert J & Turnbull, Geoffrey K, 1998. "Do Academic Salaries Decline with Seniority?," Journal of Labor Economics, University of Chicago Press, vol. 16(2), pages 352-366, April.
    Full references (including those not matched with items on IDEAS)

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Goodall, Amanda H., 2009. "Highly cited leaders and the performance of research universities," Research Policy, Elsevier, vol. 38(7), pages 1079-1092, September.
    2. Michael J. Hilmer & Christiana E. Hilmer, 2009. "Fishes, Ponds, And Productivity: Student‐Advisor Matching And Early Career Publishing Success For Economics Phds," Economic Inquiry, Western Economic Association International, vol. 47(2), pages 290-303, April.
    3. Daniel S. Hamermesh, 2018. "Citations in Economics: Measurement, Uses, and Impacts," Journal of Economic Literature, American Economic Association, vol. 56(1), pages 115-156, March.
    4. Carolyn Pitchik, 2008. "Self-Promoting Investments," Journal of Institutional and Theoretical Economics (JITE), Mohr Siebeck, Tübingen, vol. 164(3), pages 381-406, September.
    5. Justus Haucap & Johannes Muck, 2015. "What drives the relevance and reputation of economics journals? An update from a survey among economists," Scientometrics, Springer;Akadémiai Kiadó, vol. 103(3), pages 849-877, June.
    6. Grijalva, Therese C. & Nowell, Clifford, 2014. "What Interests Environmental and Resource Economists? A Comparison of Research Output in Agricultural Economics versus Environmental Economics," Agricultural and Resource Economics Review, Northeastern Agricultural and Resource Economics Association, vol. 43(2), pages 1-18, August.
    7. Awadh Pratap Singh & Shiv Prasad Yadav & Preeti Tyagi, 2022. "Performance assessment of higher educational institutions in India using data envelopment analysis and re-evaluation of NIRF Rankings," International Journal of System Assurance Engineering and Management, Springer;The Society for Reliability, Engineering Quality and Operations Management (SREQOM),India, and Division of Operation and Maintenance, Lulea University of Technology, Sweden, vol. 13(2), pages 1024-1035, April.
    8. Rolf Ketzler & Klaus F. Zimmermann, 2009. "Publications: German economic research institutes on track," Scientometrics, Springer;Akadémiai Kiadó, vol. 80(1), pages 231-252, July.
    9. Mehdi Rhaiem & Nabil Amara, 2020. "Determinants of research efficiency in Canadian business schools: evidence from scholar-level data," Scientometrics, Springer;Akadémiai Kiadó, vol. 125(1), pages 53-99, October.
    10. Diana Tal & Avishag Gordon, 2017. "Publication attributes of leadership: what do they mean?," Scientometrics, Springer;Akadémiai Kiadó, vol. 112(3), pages 1391-1402, September.
    11. Bruno Frey, 2005. "Problems with Publishing: Existing State and Solutions," European Journal of Law and Economics, Springer, vol. 19(2), pages 173-190, April.
    12. Valadkhani, Abbas & Ville, Simon, 2008. "Identifying the Most Research Intensive Faculties of Business in Australia: A Multidimensional Approach," Economics Working Papers wp08-03, School of Economics, University of Wollongong, NSW, Australia.
    13. Di Vaio, Gianfranco & Waldenström, Daniel & Weisdorf, Jacob, 2012. "Citation success: Evidence from economic history journal publications," Explorations in Economic History, Elsevier, vol. 49(1), pages 92-104.
    14. Ofer H. Azar, 2006. "The Academic Review Process: How Can We Make it More Efficient?," The American Economist, Sage Publications, vol. 50(1), pages 37-50, March.
    15. James Walker & Anna Vignoles & Mark Collins, 2010. "Higher education academic salaries in the UK," Oxford Economic Papers, Oxford University Press, vol. 62(1), pages 12-35, January.
    16. John Gibson & David L. Anderson & John Tressler, 2017. "Citations Or Journal Quality: Which Is Rewarded More In The Academic Labor Market?," Economic Inquiry, Western Economic Association International, vol. 55(4), pages 1945-1965, October.
    17. Jerome K. Vanclay, 2012. "Impact factor: outdated artefact or stepping-stone to journal certification?," Scientometrics, Springer;Akadémiai Kiadó, vol. 92(2), pages 211-238, August.
    18. Fabel, Oliver & Lehmann, Erik & Warning, Susanne, 2002. "Vorträge als Qualitätsindikator: Empirische Evidenz der Jahrestagungen des Vereins für Socialpolitik," Discussion Papers, Series I 321, University of Konstanz, Department of Economics.
    19. KimMarie McGoldrick & Gail Hoyt & Dave Colander, 2008. "The Professional Development of Graduate Students in Economics," Middlebury College Working Paper Series 0811, Middlebury College, Department of Economics.
    20. Azar, Ofer H., 2008. "Evolution of social norms with heterogeneous preferences: A general model and an application to the academic review process," Journal of Economic Behavior & Organization, Elsevier, vol. 65(3-4), pages 420-435, March.

    More about this item

    Keywords

    economics journals; research productivity; citations; Research Assessment Exercise;
    All these keywords.

    JEL classification:

    • A11 - General Economics and Teaching - - General Economics - - - Role of Economics; Role of Economists
    • O3 - Economic Development, Innovation, Technological Change, and Growth - - Innovation; Research and Development; Technological Change; Intellectual Property Rights

    NEP fields

    This paper has been announced in the following NEP Reports:

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:iza:izadps:dp2070. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Holger Hinte (email available below). General contact details of provider: https://edirc.repec.org/data/izaaade.html .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.