Melanie Walsh

Assistant Professor | Information School | University of Washington | Seattle, USA 🌦️😎

Hi, I’m Melanie Walsh. I’m an Assistant Professor in the Information School and an Adjunct Assistant Professor in the English Department at the University of Washington. I’m also on the Executive Council for the new Center for Advances in Libraries, Archives, and Museums (CALMA).

Before UW, I was a Postdoctoral Associate in Information Science at Cornell University, where I worked with David Mimno. I received my PhD in English Literature from Washington University in St. Louis.

My research interests include data science, digital humanities/cultural analytics, literature, libraries, and social media—preferably all of the above combined. I investigate how data and computational methods shape contemporary culture — such as the publishing industry and public libraries — and how they can be used to understand culture in turn. You can download my full CV here.

I am currently at work on a book project, When Postwar American Fiction Went Viral: Protest, Profit, and Popular Readers in the 21st Century, which follows the social media afterlives of American authors.

I also designed and released a free, open-source textbook, Introduction to Cultural Analytics & Python, which introduces the programming language Python to people interested in the humanities and social science.

With wonderful collaborators, I also lead several projects that I’m proud of:

In my free time, I like to play basketball 🏀, hike ⛰️, and paddle board among seal friends 🦭 (like in Shilshole Bay pictured above!).

I am recruiting one student to join my group in 2025-2026. For more info, see “Mentorship”.

news

Jan 19, 2025	I am giving a keynote talk at the Hong Kong Association for Digital Humanities’ (HKADH) inaugural conference in Hong Kong.
Dec 10, 2024	I gave a talk at the CNRS AI for Science, Science for AI (ASSAI) Digital Humanities and AI Final Conference in Paris, France.
Dec 04, 2024	We presented our paper, “Does ChatGPT Have a Poetic Style?” at the Computational Humanities Research (CHR) conference in Aarhus, Denmark.
Nov 21, 2024	I gave a talk, “AI + DEI? Investigating Automated Diversity Audits in Public Libraries,” at UCLA—sponsored by the Critical Data Lab and with support from UCLA’s IDRE and OARC.
Nov 14, 2024	With Connor Rey, I spoke at the “When Machine Learning Meets Cultural Heritage Collections” roundtable, sponsored by the Center for Advancements in Libraries, Museums, and Archives (CALMA).

selected publications

Does ChatGPT Have a Poetic Style?

Melanie Walsh, Anna Preus, and Elizabeth Gronski

In Computational Humanities Research (CHR), Dec 2024

arXiv:2410.15299

arXiv PDF Code Copy Bibtex Citation Read the Abstract

Generating poetry has become a popular application of LLMs, perhaps especially of OpenAI’s widely-used chatbot ChatGPT. What kind of poet is ChatGPT? Does ChatGPT have its own poetic style? Can it successfully produce poems in different styles? To answer these questions, we prompt the GPT-3.5 and GPT-4 models to generate English-language poems in 24 different poetic forms and styles, about 40 different subjects, and in response to 3 different writing prompt templates. We then analyze the resulting 5.7k poems, comparing them to a sample of 3.7k poems from the Poetry Foundation and the Academy of American Poets. We find that the GPT models, especially GPT-4, can successfully produce poems in a range of both common and uncommon English-language forms in superficial yet noteworthy ways, such as by producing poems of appropriate lengths for sonnets (14 lines), villanelles (19 lines), and sestinas (39 lines). But the GPT models also exhibit their own distinct stylistic tendencies, both within and outside of these specific forms. Our results show that GPT poetry is much more constrained and uniform than human poetry, showing a strong penchant for rhyme, quatrains (4-line stanzas), iambic meter, first-person plural perspectives (we, us, our), and specific vocabulary like “heart,” “embrace,” “echo,” and “whisper.”
@inproceedings{Walsh_Preus_Gronski_2024, title = {Does ChatGPT Have a Poetic Style?}, doi = {10.48550/arXiv.2410.15299}, note = {arXiv:2410.15299}, number = {arXiv:2410.15299}, publisher = {arXiv}, author = {Walsh, Melanie and Preus, Anna and Gronski, Elizabeth}, year = {2024}, month = dec, booktitle = {Computational Humanities Research (CHR)} }

Sonnet or Not, Bot? Poetry Evaluation for Large Models and Datasets

Melanie Walsh, Maria Antoniak, and Anna Preus

In Findings of the Association for Computational Linguistics: EMNLP, Nov 2024

arXiv PDF Code & Data Copy Bibtex Citation Read the Abstract

Large language models (LLMs) can now generate and recognize poetry. But what do LLMs really know about poetry? We develop a task to evaluate how well LLMs recognize one aspect of English-language poetry—poetic form—which captures many different poetic features, including rhyme scheme, meter, and word or line repetition. By using a benchmark dataset of over 4.1k human expert-annotated poems, we show that state-of-the-art LLMs can successfully identify both common and uncommon fixed poetic forms—such as sonnets, sestinas, and pantoums—with surprisingly high accuracy. However, performance varies significantly by poetic form; the models struggle to identify unfixed poetic forms, especially those based on topic or visual features. We additionally measure how many poems from our benchmark dataset are present in popular pretraining datasets or memorized by GPT-4, finding that pretraining presence and memorization may improve performance on this task, but results are inconclusive. We release a benchmark evaluation dataset with 1.4k public domain poems and form annotations, results of memorization experiments and data audits, and code.

@inproceedings{Walsh_Antoniak_Preus_2024,
  title = {Sonnet or Not, Bot? Poetry Evaluation for Large Models and Datasets},
  abstractnote = {Large language models (LLMs) can now generate and recognize poetry.
     But what do LLMs really know about poetry? We develop a task to evaluate how well LLMs recognize one aspect of English-language poetry—poetic form—which captures many different poetic features, including rhyme scheme, meter, and word or line repetition. By using a benchmark dataset of over 4.1k human expert-annotated poems, we show that state-of-the-art LLMs can successfully identify both common and uncommon fixed poetic forms—such as sonnets, sestinas, and pantoums—with surprisingly high accuracy. However, performance varies significantly by poetic form; the models struggle to identify unfixed poetic forms, especially those based on topic or visual features. We additionally measure how many poems from our benchmark dataset are present in popular pretraining datasets or memorized by GPT-4, finding that pretraining presence and memorization may improve performance on this task, but results are inconclusive. We release a benchmark evaluation dataset with 1.4k public domain poems and form annotations, results of memorization experiments and data audits, and code.},
  booktitle = {Findings of the Association for Computational Linguistics: EMNLP},
  author = {Walsh, Melanie and Antoniak, Maria and Preus, Anna},
  year = {2024},
  month = nov,
  language = {en}
}

The Challenges and Possibilities of Social Media Data: New Directions in Literary Studies and the Digital Humanities

Melanie Walsh

Nov 2023

publisher: U of Minnesota Press

Debates in DH PDF Copy Bibtex Citation

@inbook{Walsh_2023,
  title = {The Challenges and Possibilities of Social Media Data: New Directions in Literary Studies and the Digital Humanities},
  url = {https://dhdebates.gc.cuny.edu/read/f3f87448-138c-4d19-8ff8-b06acf40ddd1/section/a57b98ab-0f10-45d0-b205-3e563aab7ea8#ch18},
  note = {publisher: U of Minnesota Press},
  booktitle = {Debates in the Digital Humanities 2023},
  publisher = {University of Minnesota Press},
  author = {Walsh, Melanie},
  year = {2023}
}

Where is all the book data?

Melanie Walsh

Public Books, Nov 2022

🎙️I spoke about this essay with Australian radio ABC.🎙️

Public Books Sydney Review of Books Interview

The Goodreads “Classics”: A Computational Study of Readers, Amazon, and Crowdsourced Amateur Criticism

Melanie Walsh, and Maria Antoniak

Post45 and Journal of Cultural Analytics, Nov 2021

Post45 Journal of Cultural Analytics PDF Interactive Plots Code & Data Goodreads Scraper Copy Bibtex Citation

@article{Walsh_Antoniak_2021,
  title = {The Goodreads “Classics”: A Computational Study of Readers, Amazon, and Crowdsourced Amateur Criticism},
  volume = {6},
  url = {https://culturalanalytics.org/article/22221.pdf},
  number = {2},
  journal = {Post45 and Journal of Cultural Analytics},
  author = {Walsh, Melanie and Antoniak, Maria},
  year = {2021},
  bibtex_show = true
}