[go: up one dir, main page]
IDEAS home Printed from https://ideas.repec.org/a/inm/oropre/v63y2015i6p1246-1261.html
   My bibliography  Save this article

OR Forum—Tenure Analytics: Models for Predicting Research Impact

Author

Listed:
  • Dimitris Bertsimas

    (Sloan School of Management and Operations Research Center, Massachusetts Institute of Technology, Cambridge, Massachusetts 02139)

  • Erik Brynjolfsson

    (Sloan School of Management, Initiative on the Digital Economy, and Operations Research Center, Massachusetts Institute of Technology, Cambridge, Massachusetts 02139)

  • Shachar Reichman

    (Recanati Business School, Tel Aviv University, Tel Aviv 6997801, Israel; and Sloan School of Management, Massachusetts Institute of Technology, Cambridge, Massachusetts 02139)

  • John Silberholz

    (Sloan School of Management and Operations Research Center, Massachusetts Institute of Technology, Cambridge, Massachusetts 02139)

Abstract
Tenure decisions, key decisions in academic institutions, are primarily based on subjective assessments of candidates. Using a large-scale bibliometric database containing 198,310 papers published 1975–2012 in the field of operations research (OR), we propose prediction models of whether a scholar would perform well on a number of future success metrics using statistical models trained with data from the scholar’s first five years of publication, a subset of the information available to tenure committees. These models, which use network centrality of the citation network, coauthorship network, and a dual network combining the two, significantly outperform simple predictive models based on citation counts alone. Using a data set of the 54 scholars who obtained a Ph.D. after 1995 and held an assistant professorship at a top-10 OR program in 2003 or earlier, these statistical models, using data up to five years after the scholar became an assistant professor and constrained to tenure the same number of candidates as tenure committees did, made a different decision than the tenure committees for 16 (30%) of the candidates. This resulted in a set of scholars with significantly better future A-journal paper counts, citation counts, and h -indexes than the scholars actually selected by tenure committees. These results show that analytics can complement the tenure decision-making process in academia and improve the prediction of academic impact.

Suggested Citation

  • Dimitris Bertsimas & Erik Brynjolfsson & Shachar Reichman & John Silberholz, 2015. "OR Forum—Tenure Analytics: Models for Predicting Research Impact," Operations Research, INFORMS, vol. 63(6), pages 1246-1261, December.
    • Handle: RePEc:inm:oropre:v:63:y:2015:i:6:p:1246-1261
    DOI: 10.1287/opre.2015.1447
    as

    Download full text from publisher

    File URL: http://dx.doi.org/10.1287/opre.2015.1447
    Download Restriction: no
    File URL: https://libkey.io/10.1287/opre.2015.1447?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    References listed on IDEAS

    as
    1. Theoharakis V. & Skordia M., 2003. "How Do Statisticians Perceive Statistics Journals?," The American Statistician, American Statistical Association, vol. 57, pages 115-123, May.
    2. Lutz Bornmann & Rüdiger Mutz & Hans‐Dieter Daniel, 2008. "Are there better indices for evaluation purposes than the h index? A comparison of nine different variants of the h index using data from biomedicine," Journal of the American Society for Information Science and Technology, Association for Information Science & Technology, vol. 59(5), pages 830-837, March.
    3. Currie, Russell R. & Pandher, Gurupdesh S., 2011. "Finance journal rankings and tiers: An Active Scholar Assessment methodology," Journal of Banking & Finance, Elsevier, vol. 35(1), pages 7-20, January.
    4. Pierre Azoulay & Joshua S. Graff Zivin & Gustavo Manso, 2011. "Incentives and creativity: evidence from the academic life sciences," RAND Journal of Economics, RAND Corporation, vol. 42(3), pages 527-554, September.
    5. repec:bla:germec:v:9:y:2008:i::p:402-430 is not listed on IDEAS
    6. Daniel E. Acuna & Stefano Allesina & Konrad P. Kording, 2012. "Predicting scientific success," Nature, Nature, vol. 489(7415), pages 201-202, September.
    7. Johannes Hönekopp & Julie Khan, 2012. "Future publication success in science is better predicted by traditional measures than by the h index," Scientometrics, Springer;Akadémiai Kiadó, vol. 90(3), pages 843-853, March.
    8. Ritzberger Klaus, 2008. "A Ranking of Journals in Economics and Related Fields," German Economic Review, De Gruyter, vol. 9(4), pages 402-430, December.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. James J. Heckman & Sidharth Moktan, 2020. "Publishing and Promotion in Economics: The Tyranny of the Top Five," Journal of Economic Literature, American Economic Association, vol. 58(2), pages 419-470, June.
    2. Zuo, Zhiya & Zhao, Kang & Ni, Chaoqun, 2019. "Standing on the shoulders of giants?—Faculty hiring in information schools," Journal of Informetrics, Elsevier, vol. 13(1), pages 341-353.
    3. Hird, Mackenzie D. & Pfotenhauer, Sebastian M., 2017. "How complex international partnerships shape domestic research clusters: Difference-in-difference network formation and research re-orientation in the MIT Portugal Program," Research Policy, Elsevier, vol. 46(3), pages 557-572.
    4. Matthias Kuppler, 2022. "Predicting the future impact of Computer Science researchers: Is there a gender bias?," Scientometrics, Springer;Akadémiai Kiadó, vol. 127(11), pages 6695-6732, November.

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Danielle H. Lee, 2019. "Predicting the research performance of early career scientists," Scientometrics, Springer;Akadémiai Kiadó, vol. 121(3), pages 1481-1504, December.
    2. Cristiano Varin & Manuela Cattelan & David Firth, 2016. "Statistical modelling of citation exchange between statistics journals," Journal of the Royal Statistical Society Series A, Royal Statistical Society, vol. 179(1), pages 1-63, January.
    3. Wohlrabe, Klaus, 2016. "Taking the Temperature: A Meta-Ranking of Economics Journals," MPRA Paper 68933, University Library of Munich, Germany.
    4. Claudio Vitari, 2014. "Can virtual currencies drive sustainable de-growth?," Grenoble Ecole de Management (Post-Print) halshs-01924234, HAL.
    5. Claudio Vitari, 2014. "Electronic currencies for purposive degrowth?," Working Papers hal-00975432, HAL.
    6. Ziqiang Zeng & Lantian Shi, 2021. "A two-dimensional journal classification method based on output and input factors: perspectives from citation and authorship related indicators," Scientometrics, Springer;Akadémiai Kiadó, vol. 126(5), pages 3929-3964, May.
    7. Lutz Bornmann & Alexander Butz & Klaus Wohlrabe, 2018. "What are the top five journals in economics? A new meta-ranking," Applied Economics, Taylor & Francis Journals, vol. 50(6), pages 659-675, February.
    8. Claudio Vitari, 2014. "Electronic currencies for purposive degrowth?," Working paper serie RMT - Grenoble Ecole de Management hal-00975432, HAL.
    9. Claudio Vitari, 2016. "Electronic Currencies: a literature review [Monnaies électroniques : a revue de la littérature]," Post-Print halshs-01924191, HAL.
    10. Samreen Ayaz & Nayyer Masood & Muhammad Arshad Islam, 2018. "Predicting scientific impact based on h-index," Scientometrics, Springer;Akadémiai Kiadó, vol. 114(3), pages 993-1010, March.
    11. Claudio Vitari, 2014. "Can virtual currencies drive sustainable de-growth?," Post-Print halshs-01924234, HAL.
    12. Zhang, Lin & Thijs, Bart & Glänzel, Wolfgang, 2011. "The diffusion of H-related literature," Journal of Informetrics, Elsevier, vol. 5(4), pages 583-593.
    13. Claudio Vitari, 2016. "Electronic Currencies: a literature review [Monnaies électroniques : a revue de la littérature]," Grenoble Ecole de Management (Post-Print) halshs-01924191, HAL.
    14. Antonio Fernandez-Cano & Inés M. Fernández-Guerrero, 2017. "A multivariate model for evaluating emergency medicine journals," Scientometrics, Springer;Akadémiai Kiadó, vol. 110(2), pages 991-1003, February.
    15. Deming Lin & Tianhui Gong & Wenbin Liu & Martin Meyer, 2020. "An entropy-based measure for the evolution of h index research," Scientometrics, Springer;Akadémiai Kiadó, vol. 125(3), pages 2283-2298, December.
    16. Sten F Odenwald, 2020. "A citation study of earth science projects in citizen science," PLOS ONE, Public Library of Science, vol. 15(7), pages 1-26, July.
    17. Waltman, L. & van Eck, N.J.P., 2009. "A Taxonomy of Bibliometric Performance Indicators Based on the Property of Consistency," ERIM Report Series Research in Management ERS-2009-014-LIS, Erasmus Research Institute of Management (ERIM), ERIM is the joint research institute of the Rotterdam School of Management, Erasmus University and the Erasmus School of Economics (ESE) at Erasmus University Rotterdam.
    18. Pantelis Kalaitzidakis & Theofanis P. Mamuneas & Thanasis Stengos, 2011. "An updated ranking of academic journals in economics," Canadian Journal of Economics, Canadian Economics Association, vol. 44(4), pages 1525-1538, November.
    19. Bartels, Charlotte & Jäger, Simon & Obergruber, Natalie, 2020. "Long-Term Effects of Equal Sharing: Evidence from Inheritance Rules for Land," IZA Discussion Papers 13665, Institute of Labor Economics (IZA).
    20. Alex Bell & Raj Chetty & Xavier Jaravel & Neviana Petkova & John Van Reenen, 2019. "Who Becomes an Inventor in America? The Importance of Exposure to Innovation," The Quarterly Journal of Economics, President and Fellows of Harvard College, vol. 134(2), pages 647-713.

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:inm:oropre:v:63:y:2015:i:6:p:1246-1261. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Chris Asher (email available below). General contact details of provider: https://edirc.repec.org/data/inforea.html .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.