HOME / BLOG /

Research Rundown: September 2023 to January 2024

Welcome to the first research rundown from the User Research Center at Harvard Library. In this quarterly newsletter, the URC will be sharing recent research that our team has conducted at Harvard Library.

If you have any questions about the research or would like to consult with our team about helping you plan or conduct research at Harvard Library please reach out to libraryux@harvard.edu.

Survey on Student Use of AI for Academic Work

This survey was conducted in Fall 2023 with 376 undergraduate (50%) and graduate (50%) students to understand their usage and perceptions of generative AI. Questions focused on student use of generative AI chatbots for academic work, their trust in the tools, and their need for support and guidance. The results will be used by library staff and others to inform services related to the use of generative AI for academic work.

Major Insights

  • Nearly 65% of students have used or plan to use generative AI chatbots for academic work, even though the majority of respondents (also 65%) do not find their outputs trustworthy enough for academic work.
  • The vast majority of students use ChatGPT.
  • The most popular tasks related to academic work that students do with generative AI include: summarizing text, getting feedback on writing, and topic exploration.
  • Students are actively using these tools but desire guidance around effective use.

Librarians must be at the forefront of understanding this technology's significant impact on information-seeking behaviors and research habits. To effectively support students, librarians must know how to use these tools to advise students on how to critically evaluate AI output and effectively incorporate it into their research.

AI for HOLLIS Usability Interviews

In January 2024, we partnered with Library Technology Services (LTS) to conduct a study on the concept of AI for HOLLIS, called “Talk with HOLLIS.” “Talk with HOLLIS” is a HUIT ETIP grant project. Participants included 11 undergraduate and graduate students familiar with HOLLIS, but with different levels of familiarity with generative AI.

Major Insights

On HOLLIS

  • HOLLIS is a secondary step in the research process for students, they go there with known item searches or intentional search goals. Doing preliminary searching when they're at the idea stage is done elsewhere, sometimes with ChatGPT.

On ChatGPT Use

  • ChatGPT is understood to be different from search. The functionality expectations for chatbots are different from search engines.
  • Students use ChatGPT as a learning partner, writing editor, translation tool, and to find “lost citations” but they don’t trust it. Say they have to “fact check” the output.
  • Students know that ChatGPT won’t connect them to Harvard resources.

Impressions of the Library Chatbot

  • Students are excited that the libraries are experimenting with generative AI; they have high expectations and specific ideas of what this tool could do.
  • They expect it to build on previous output through follow-up prompts.
  • They want more explanation of why these results are in the output.
  • They want the tool to connect with other library services (Ex. Make an appointment).
  • There will be greater trust in the tool if we share how it was trained.

These results will be used by LTS to inform future work on generative AI integration into the discovery system.

Tozzer Library Lower Level Renovation Tiny Cafe

In this FY24 project, we partnered with Susan Gilman and Julie Fiveash to understand their needs for space, ambiance, and furniture choice. Over 40 students participated in the in-person data collection. The library will use the results to inform their renovation project.

Major Insights

  • Students envision Tozzer’s lower level as a quiet study space for individuals and groups, the furniture preferences reflected this with individual study carrels receiving the most votes.
  • Students desire space sectioning and signage. Students noted that at times it was hard to know where designated spaces were in Tozzer's lower level.

Space studies are valuable tools to help create spaces students want to experience in our libraries. This research shows that students also desire signage or clear designation of types of space within libraries.

Farnsworth Room Survey

In this Winter 2023 project, we partnered with Stav Birmbas, Elizabeth Kirk, and Ardys Kozbial to understand undergraduate students' desires for ambiance, furniture, and materials in the room. Over 240 students completed the survey. The library will use the results to inform the continued improvement of the Farnsworth Room.

Major Insights

  • The majority of undergraduate student respondents love the Farnsworth room as it is and don’t want different kinds of furniture. They strongly prefer a “warm” or “calming” atmosphere with soft seating with nearby desks or tables for individual activities. The comments were very strongly against making it a more modern, colorful space like Cabot. They feel like the campus already offers enough modern study spaces.
  • Students are most interested in using the space for individual study (113) and individual leisure activities (51).
  • In addition to books, students are most interested in finding puzzles, art supplies, a record player with headphones and vinyl, handheld video games, and card games in the Farnsworth Room.

This space study will help the library prioritize new collection building and furniture choices in the Farnsworth room that align with students' needs and desires for the space.