The Research Rundown: July 2025-September 2025

Welcome to the research rundown from the UX & Discovery Department at Harvard Library. In this quarterly blog post, we will share recent research our team has conducted at Harvard Library. 

If you have any questions about the research or would like to consult with our team about helping you plan or conduct research at Harvard Library, please email libraryux@harvard.edu. 

Collections Explorer LLM Evaluations 

In July & August, UX & Discovery conducted evaluations on LLM summaries & content for Collections Explorer. The first round of testing was conducted in July with the UX & Discovery intern team, and the second round was conducted in August with the entire UX & Discovery team. 

Major Insights

  • Queries with phrases related to Harvard or Harvard Library yield less relevant results.
  • Keyword queries tend to produce more relevant results than semantic queries.
  • Testers considered including ‘Limited Results Found’ for niche queries with limited relevant matches in Harvard collections.
  • The You Might Also Try prompts can create non-Harvard-focused queries.
  • The HOLLIS catalog link can miss important keywords even in keyword searches.

UX & Discovery provided guidance on improving the system. The insights gained from this research will be used to enhance Collections Explorer further. 

Aeon Contextual Inquiry 

In August 2025, to support the reimaging discovery project, UX & Discovery conducted interviews to understand how users experience HOLLIS Special Request (Aeon). These interviews explored how users navigate the system, complete requests, and what happens after they submit requests.

Major Insights

  • Navigation within HOLLIS Special Request is overwhelming for participants because there is no clear hierarchy on the pages, which hinders their ability to parse information easily.
  • The copy formats are not always clear to participants, even in HOLLIS Special Request and HOLLIS for Archival Discovery, despite the presence of definitions.
  • Ambiguous language in the status requests led participants to be uncertain about when their materials would be available.
  • The drop-down menu format for initiating a reading room and copy request in HOLLIS for Archival Discovery is unintuitive, hindering the discoverability of the copy request option.

UX & Discovery provided guidance on improving the system. The insights gained from this research will be used to further enhance Aeon.

Accessibility Audits of Collections Explorer, Mirador, & CUROISity

During this quarter, the UX & Discovery conducted accessibility audits on three Harvard systems: Collections Explorer, Mirador, and CURIOSity. 

For all cases, UX & Discovery provided guidance on improving accessibility for the system, specifically for users of screen readers and those using keyboards. Insights from CURIOSity were integrated into a recent community sprint, making the CORE code more accessible. 

Share on: