By Meg McMahon

Analyzing the notes from a user research study is an essential part of answering the research questions that prompted the study. 

During the analysis, you or your team decide what insights are generated from the notes taken about the research studies. The insights should relate to the research questions posed at the beginning of the study or point to future research.

Before Analysis 

Become familiar with the study

Read over the research questions, script, background information on the project, and any notes moderators took during interviews.

Decide on analysis type

There are two types of analysis that the URC team uses frequently, an affinity diagramming session and thematic coding.

• Affinity diagramming: this is a process in which all of the notes are separated into individual sticky notes (virtual or physical) and grouped by question or participant at first. Then the analyzers take an even number of groups and sort them individually and then together to find themes. We at the URC use Miro or Dovetail to virtually facilitate this process. 
• Thematic coding: this is a process where the notes or a transcript are marked up using codes. Codes are like tags or hashtags. They are bits of metadata that you use to describe other data. After the codes are created the analyzers go through and draw insights from the coded information. We at the URC use Dovetail for this type of analysis.

Decide on who should help with analysis

Often it is a good idea to include stakeholders, as well as yourself, in the analysis process for two primary reasons. One, the stakeholders are able to be a subject matter expert on the system, process, or workflow you are evaluating and two, by participating in the analysis process they are more likely to feel ownership over the final findings and recommendations.

 

Analysis Basics 

 

Keep a scratch document

While analyzing your data, it can be helpful to have a scratch document (a document separate from the main notes document) open to the side. You can use this scratch document to:

• Write down quotes from notes that are especially powerful or meaningful. 
• Write down any initial insights you have. An insight informs the research questions or furthers your research goals in some way. 

Give insights specific names

Insights are created from the themes or codes created during affinity diagramming or thematic coding. Look at the connections between themes or codes, what are larger insights you can draw from them?

Give each insight a name and be as specific as possible. Insights should help your stakeholders understand their research questions.

• DO: use a strong, clear insight: Participants collaboratively search with a collection specialist as a part of the search process. This was noted to build researchers' special collections research skills.
• DON’T: use a hard to understand, non-specific insight: collaborative search

Review the data with other analyzers 

After reviewing the data on your own, discuss what you think of the first themes. Why are they important? What pattern do they surface? How do they relate to the research questions?

Back up insights with quotes

Quote or paraphrase ideas directly from the participants that relate to insights add more impact in the future report.

Look for research gaps

After having insights figured out the next step is to review the insights for any research gaps. Can you answer the research questions in your research plan? If not, what additional research is needed? Or are there other themes that don’t directly relate to the research questions which point to a future follow up study?

Prioritize insights most closely related to research questions

As a user researcher your job is to answer the research questions that started the study. If there is an insight that is only tangentially related to the research questions, it is okay to prioritize insights that are more directly related to the research questions when writing the report. 

Affinity diagramming

Take the time to set up session

Affinity diagramming takes time to set up, but it is important to have every note grouped either by participant or by question. 

Create top level themes and sub themes

Start to sort the notes into themes. It is okay to create multiple levels of themes if the data lends itself to this kind of grouping. This is helpful for reports to group like insights together for readability. 

Themes are not insights

Themes are not insights, they are building blocks for insights. When you are grouping themes it is okay to keep them more vague than you would an insight.

Collapse themes together if necessary 

Often when affinity diagramming you may find similar themes within different questions. It is okay to collapse those themes together under a unifying heading. Themes (and insights) don’t need to be based on only a single question.

Avoid creating a "group" of unrelated notes

There are times in affinity diagramming where there are notes that don’t relate easily to any other notes for the study. It is okay to have orphan notes, notes that don’t relate to any theme. If the finding is important enough, it is okay for the orphan note to be its own theme.

 

Thematic coding

Codes (tags) are not insights

Codes (tags) are not insights, they are building blocks for insights. When you are adding codes to the notes or transcripts it is okay to keep them more vague than you would an insight.

Use nested coding

When coding you may need to use nested codes. For example, say you’ve created a tag labeled “Coffee”. Now, if you are talking about how people like their coffee served you may make it a two word code/tag like “Coffee Served”.

If you want to specify that someone likes the coffee “black”, you can add a third word. However, considering “black” is a word that now describes “Coffee Served”, it would be best to create a nested tag which should look something like this: “Coffee Served: Black”.

This makes the tag structure easy to use again. For example, if you have to tag a comment later where someone mentions they like the coffee  with cream and sugar, it’s easy to add that as “Coffee Served: Cream & Sugar”.

Repeat codes if necessary 

Your codes should inform your research questions and you should repeat codes as necessary. If someone talks about the same thing at various times throughout the interview, code each of those instances with the same code.