[go: up one dir, main page]
IDEAS home Printed from https://ideas.repec.org/p/arx/papers/2407.14333.html
   My bibliography  Save this paper

As Generative Models Improve, We Must Adapt Our Prompts

Author

Listed:
  • Eaman Jahani
  • Benjamin S. Manning
  • Joe Zhang
  • Hong-Yi TuYe
  • Mohammed Alsobay
  • Christos Nicolaides
  • Siddharth Suri
  • David Holtz
Abstract
The recent surge in generative AI has led to new models being introduced almost every month. In light of this rapid progression, we pose and address a central question: to what extent must prompts evolve as the capabilities of generative AI models advance? To answer this question, we conducted an online experiment with N = 1,983 participants where each participant was incentivized to write prompts to reproduce a target image as closely as possible in 10 consecutive tries. Each participant was randomly and blindly assigned to use one of three text-to-image diffusion models: DALL-E 2, its more advanced successor, DALL-E 3, or a version of DALL-E 3 with automatic prompt revision. In total, we collected and analyzed over 18,000 prompts and over 300,000 images. We find that task performance was higher for participants using DALL-E 3 than for those using DALL-E 2. This performance gap corresponds to a noticeable difference in the similarity of participants' images to their target images, and was caused in equal measure by: (1) the increased technical capabilities of DALL-E 3, and (2) endogenous changes in participants' prompting in response to these increased capabilities. Furthermore, while participants assigned to DALL-E 3 with prompt revision still outperformed those assigned to DALL-E 2, automatic prompt revision reduced the benefits of using DALL-E 3 by 58%. Our results suggest that for generative AI to realize its full impact on the global economy, people, firms, and institutions will need to update their prompts in response to new models.

Suggested Citation

  • Eaman Jahani & Benjamin S. Manning & Joe Zhang & Hong-Yi TuYe & Mohammed Alsobay & Christos Nicolaides & Siddharth Suri & David Holtz, 2024. "As Generative Models Improve, We Must Adapt Our Prompts," Papers 2407.14333, arXiv.org, revised Dec 2024.
    • Handle: RePEc:arx:papers:2407.14333
    as

    Download full text from publisher

    File URL: http://arxiv.org/pdf/2407.14333
    File Function: Latest version
    Download Restriction: no
    ---><---

    References listed on IDEAS

    as
    1. Erik Brynjolfsson & Danielle Li & Lindsey R. Raymond, 2023. "Generative AI at Work," NBER Working Papers 31161, National Bureau of Economic Research, Inc.
    Full references (including those not matched with items on IDEAS)

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Carvajal, Daniel & Franco, Catalina & Isaksson, Siri, 2024. "Will Artificial Intelligence Get in the Way of Achieving Gender Equality?," Discussion Paper Series in Economics 3/2024, Norwegian School of Economics, Department of Economics, revised 31 Oct 2024.
    2. Gaétan de Rassenfosse & Adam B. Jaffe & Joel Waldfogel, 2024. "Intellectual Property and Creative Machines," NBER Chapters, in: Entrepreneurship and Innovation Policy and the Economy, volume 4, National Bureau of Economic Research, Inc.
    3. Alexander Cuntz & Carsten Fink & Hansueli Stamm, 2024. "Artificial Intelligence and Intellectual Property : An Economic Perspective," WIPO Economic Research Working Papers 77, World Intellectual Property Organization - Economics and Statistics Division.
    4. Lilia Patrignani, 2024. "Understanding digital trade," Questioni di Economia e Finanza (Occasional Papers) 841, Bank of Italy, Economic Research and International Relations Area.
    5. Amali Matharaarachchi & Wishmitha Mendis & Kanishka Randunu & Daswin De Silva & Gihan Gamage & Harsha Moraliyage & Nishan Mills & Andrew Jennings, 2024. "Optimizing Generative AI Chatbots for Net-Zero Emissions Energy Internet-of-Things Infrastructure," Energies, MDPI, vol. 17(8), pages 1-19, April.
    6. Bohren, Noah & Hakimov, Rustamdjan & Lalive, Rafael, 2024. "Creative and Strategic Capabilities of Generative AI: Evidence from Large-Scale Experiments," IZA Discussion Papers 17302, Institute of Labor Economics (IZA).
    7. Leonardo Gambacorta & Han Qiu & Shuo Shan & Daniel M Rees, 2024. "Generative AI and labour productivity: a field experiment on coding," BIS Working Papers 1208, Bank for International Settlements.
    8. Becker, Dominik & Deck, Luca & Feulner, Simon & Gutheil, Niklas & Schüll, Moritz & Decker, Stefan & Eymann, Torsten & Gimpel, Henner & Pippow, Andreas & Röglinger, Maximilian & Urbach, Nils, 2024. "Lohnt sich Microsoft 365 Copilot? Eine Potenzialanalyse für Unternehmen und Bildungseinrichtungen," Bayreuth Reports on Information Systems Management 72, University of Bayreuth, Chair of Information Systems Management.
    9. Christian Peukert & Florian Abeillon & Jérémie Haese & Franziska Kaiser & Alexander Staub, 2024. "Strategic Behavior and AI Training Data," CESifo Working Paper Series 11099, CESifo.
    10. Stankov, Petar, 2024. "The death of exams? Grade inflation and student satisfaction when coursework replaces exams," International Review of Economics Education, Elsevier, vol. 46(C).
    11. Iñaki Aldasoro & Leonardo Gambacorta & Anton Korinek & Vatsala Shreeti & Merlin Stein, 2024. "Intelligent financial system: how AI is transforming finance," BIS Working Papers 1194, Bank for International Settlements.
    12. Samuel Chang & Andrew Kennedy & Aaron Leonard & John A. List, 2024. "12 Best Practices for Leveraging Generative AI in Experimental Research," NBER Working Papers 33025, National Bureau of Economic Research, Inc.
    13. James Bono & Alec Xu, 2024. "Randomized Controlled Trials for Security Copilot for IT Administrators," Papers 2411.01067, arXiv.org, revised Nov 2024.
    14. Morgan Blangeois, 2023. "Generative AI: Revolution or Threat for Digital Service Companies ? [IA générative : révolution ou menace pour les entreprises de services du numérique ?]," Post-Print hal-04355219, HAL.
    15. Spencer Bastani & Daniel Waldenström, 2024. "AI, Automation and Taxation," CESifo Working Paper Series 11084, CESifo.
    16. Raphael Auer & David Köpfer & Josef Švéda & Raphael A. Auer, 2024. "The Rise of Generative AI: Modelling Exposure, Substitution, and Inequality Effects on the US Labour Market," CESifo Working Paper Series 11410, CESifo.
    17. Ozge Demirci & Jonas Hannane & Xinrong Zhu, 2024. "Who Is AI Replacing? The Impact of Generative AI on Online Freelancing Platforms," CESifo Working Paper Series 11276, CESifo.
    18. Aldasoro, Iñaki & Armantier, Olivier & Doerr, Sebastian & Gambacorta, Leonardo & Oliviero, Tommaso, 2024. "The gen AI gender gap," Economics Letters, Elsevier, vol. 241(C).
    19. Bastani, Spencer & Waldenström, Daniel, 2024. "AI, Automation and Taxation," IZA Policy Papers 212, Institute of Labor Economics (IZA).
    20. Suleymenova, Kamilya & Dawood, Mary & Psyllou, Maria, 2024. "Essays in economics in ICU: Resuscitate or pull the plug?," International Review of Economics Education, Elsevier, vol. 45(C).

    More about this item

    NEP fields

    This paper has been announced in the following NEP Reports:

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:arx:papers:2407.14333. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: arXiv administrators (email available below). General contact details of provider: http://arxiv.org/ .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.