The "First Pass, Second Pass" Strategy of AI and Traditional Methods of Data Transcription in Qualitative Research

Analeigh E. Horton
Fairleigh Dickinson University

This assignment asks researchers (most likely, graduate students, but anyone dealing with oral qualitative data) to explore traditional and modern methods of transcription. Students will learn how to work with different tools and consider the affordances and constraints of each, developing the ability to discern what approach is best for their specific study. This approach is particularly helpful for researchers who are unfamiliar with transcription processes and uncertain in determining the details they need to capture. This assignment can be modified to suit a range of transcription technologies to help students gain familiarity with various softwares.


Learning Goals

  • Strengthen skills in transcribing by hand and with AI tools
  • Discern usefulness of by-hand and by-AI transcription methods
  • Determine best transcription strategy for a qualitative research project

Original Assignment Context: Graduate-level researchers in data analysis stage of research projects

Materials Needed:

  • Recorded verbal data (e.g., interview, focus group)
  • Word processor
  • Speech-to-text transcription application using AI and machine learning (e.g., Otter.ai)

Time Frame: ~1 hour

Overview

This assignment, designed primarily for graduate students but valuable for any researcher conducting transcription, encourages researchers to consider the affordances and constraints of traditional and AI methods of data transcription. Qualitative researchers are guided through one exercise of transcribing verbal data by hand and another using generative AI. Researchers will consider how combining these methods can lead to a robust transcript.

This assignment has been taught to multiple masters and doctoral level graduate students conducting qualitative research. The outcome is that students are able to decide which methods best serve their research needs. For example, linguistics students who need detailed notes on elements like pitch, pauses, and plosives prefer to conduct their transcriptions by hand. Conversely, writing studies students who need conversation content prefer to use AI tools. By testing out both methods in this assignment, students gain discernment on what method is best for their project’s context and goals. 

Ultimately, it seems that a combination of both methods is most useful. Using AI as a “first pass” transcriber accomplishes the most tedious elements of transcribing by writing out the text. Then, transcribing by hand as a “second pass” enables the researcher to develop a closer relationship to the data by spending time reviewing all of the recordings to double check the generated text and add any transcriber notes or transcription symbols for conversation analysis. This “first pass, second pass” method brings researchers closer to saturating the data, or having a fuller understanding of the data set.


Assignment

This lesson can be used in a qualitative research methods course or between an advisor and student. The time frame can be adapted to suit the needs of the data sample size and the materials can be adapted to teach various transcription technologies. Students should have an audio recording of verbal data. Preferably, this would come from their own projects, but you could provide a sample for them if needed.

Step 1: First Pass with AI

Import the audio file into a speech-to-text transcription app. Ask the app to transcribe the audio file. Save a copy of the generated transcription.

Step 2: Second Pass by hand

Play the audio file. This time, you will transcribe the data. Although this is called “by hand,” you might do it in a word processor or computer assisted qualitative data analysis (CAQDAS) software. Save a copy of your transcription.

Step 3: Review each transcript

Review your two transcripts. Read them while playing the audio file and following along to check for accuracy. As you review each transcription, consider the following questions:

  • Which version is the most accurate? 
  • Which version best captures verbal nuances like pitch and pauses?
  • Which process gave me a deeper knowledge of the data?
  • Which process was faster?

Now, review your research questions and consider the following questions:

  • What does my overall research goal value?
  • Which process and version best supports my research goal?
  • Which process and version best supports my research context? Factors might include time, funding, and your working style. 

Asking yourself these questions will help you determine the affordances and constraints of each method. Remember that the answers to these questions might change as your research or the technology evolves. For this reason, it’s best to ask yourself these questions at the beginning of each new research project.

Step 4: Merge your transcription strategies

Go back to your AI-generated transcript. Create a copy of it that you can annotate (e.g., a PDF editor, a printed version, or in the app itself if available). Now, conduct the Second Pass strategy. This time, instead of writing out each word in a new document, follow along with the AI-generated transcript. As you listen to the audio file, use your Second Pass techniques to make any edits to the AI-generated transcript, add transcription symbols, and take notes. Be sure to annotate the transcript in ways best aligned to your research goals! 

Step 5: Review your completed transcript

You should now have a transcript that best suits your research needs. Reflect back on the process and the affordances and constraints of the First Pass and Second Pass strategies. Note how, for example, the First Pass created a transcript quickly and how the Second Pass created a transcript deeply. Consider how uniting these strategies developed a transcript that was speedier and richer than using one strategy alone. Moreover, think about how aligning your research goals to your transcription methods can support your ethical considerations of working with human subjects data and allows you to best capture and uplift the voices of your participants.

Step 6: Transcribe the rest of the data

Now that you have a transcription strategy that is tailored to your specific research goal, apply that strategy to the rest of your data set. Happy transcribing!