The Research Rundown: October 2024 - December 2024

Welcome to the research rundown from the User Research Center at Harvard Library. In this quarterly blog post, the URC will share recent research our team has conducted at Harvard Library. 

If you have any questions about the research or would like to consult with our team about helping you plan or conduct research at Harvard Library, please email libraryux@harvard.edu. 

Collections Explorer First Phase User Testing

In October 2024, the URC conducted user testing on Collections Explorer. This experimental tool uses semantic search, a search technique that uses artificial intelligence (AI) to understand the meaning of a search query, to streamline access to Harvard’s special collections. Five students and four members of the public participated in structured, task-based exercises, with the URC capturing qualitative observations and quantitative metrics. Here are the insights: 

• Participants liked the AI-generated enhancements, such as the “Why Did I Get This Result?” summaries and the “You Might Also Try” suggestions, as they were approachable and helpful when searching. 
• Participants emphasized prioritizing item-level content over collection-level descriptions to enhance immediate discoverability.
• Participants wanted a more transparent display of metadata, such as resource type, digitization status, and publication dates, so they could better assess an item’s potential value without additional clicks. 
• Participants expected advanced filtering mechanisms. 
• Participants favored clear language within the interface, advising the removal of ambiguities and library-centric jargon like “HOLLIS,” which many public participants found confusing. 
• Participants suggested clarifying AI-generated features with icons and user-friendly language. 

This feedback will guide the next iteration of Collections Explorer. 

Collections Explorer Digitized Filter Design Testing

After being prompted to consider filter design from the Collection Explorer First Phase User Testing, the URC conducted a survey to investigate user perceptions of the “Digitized Only” option in Collections Explorer and potential improvements. 

Seventy-eight participants, predominantly infrequent special collections users, briefly viewed a prototype and answered questions about the filter’s functionality and label. Here are the insights: 

• Key findings indicate that the filter often goes unnoticed, suggesting a more visually prominent design is needed. 
• The “Digitized Only” label was widely understood to refer to content accessible solely online, excluding physical materials.
• Participants preferred the unchecked filter by default, allowing them to explore all materials initially. 

This feedback will guide the next iteration of Collections Explorer. 

Library Student Experience Panel

In November 2024, the URC facilitated a workshop with six students from diverse academic fields to understand their library use and research journeys better. The session included writing and journey-mapping activities focused on space use, research approaches, learning experiences, and personal enjoyment. 

When asked about their library use, students said they rely on library spaces for solo study, group projects, call booths, and daily tasks. Their research often draws on digital collections, archival resources, interlibrary loans, and librarian consultations. They appreciate the library’s role as a learning environment, hosting classes and training sessions, and enjoy events like cultural exhibits or social gatherings. 

When asked about their research journeys, the following insights were drawn: 

Behaviors

• Students find relevant research through citation chasing, which involves retracing an author's research by reviewing the author's citations.
• Students care about recent research. They filter by the newest research as a first step in their search journey.
• Students consult with outside sources for additional context during the search.
• Students use systems like ChatGPT+, Google, and WorldCat to find keywords to put into HOLLIS.
• Students save research they find outside of HOLLIS in tools like Zotero or Google Docs.
• Students use HOLLIS as a second step in their search process for access to materials, not the actual identification of research they want to consult.

Pain Points

• When HOLLIS fails to return relevant results for the keywords, students commonly associate with their topics.
• HOLLIS sometimes fails to recognize and correct slight misspellings of terms or names, leading to incomplete or missing search results.
• When students discover that a resource is checked out on a semester loan, they often struggle to find an option for Interlibrary Loan or a way to place a hold on the book.
• Students can't quickly tell when a source is digitized when searching in HOLLIS.
• When emails from access services are overly brief and lack details, students often feel they are not provided with the information they need to address the issue they sought help with.
• Students often struggle to quickly identify whether a book is checked out during their initial search.
• When terms like 'offsite storage' are not clearly explained, students may find it challenging to understand what they mean when requesting books, including how they impact the time it takes for the book to be available for pickup.

These insights will inform the upcoming redesign of library.harvard.edu.  

University Archives Record Management System Testing

In the Fall of 2024, the User Research Center (URC) tested Harvard University’s new records management system, Zasio, which stores the General Record Schedules (GRS). During the engagement, the URC performed a usability test, an accessibility audit, and a focus group on training materials for the Zasio system. 

The URC provided guidance on improving all three evaluations of the new Record Management system structure. For more information, please contact the URC. The insights from this research are helping Harvard University roll out the Zasio record management system.