Automatic speech-to-text transcription: evidence from a smartphone survey with voice answers

Publikation: Beitrag in FachzeitschriftArtikelForschungPeer-Review

Autorschaft

  • Jan Karem Höhne
  • Timo Lenzner
  • Joshua Claassen

Organisationseinheiten

Externe Organisationen

  • Deutsches Zentrum für Hochschul- und Wissenschaftsforschung (DZHW)
  • GESIS - Leibniz-Institut für Sozialwissenschaften
Forschungs-netzwerk anzeigen

Details

OriginalspracheEnglisch
FachzeitschriftInternational Journal of Social Research Methodology
Frühes Online-Datum1 Jan. 2025
PublikationsstatusElektronisch veröffentlicht (E-Pub) - 1 Jan. 2025

Abstract

Advances in information and communication technology, coupled with a smartphone increase in web surveys, provide new avenues for collecting answers from respondents. Specifically, the microphones of smartphones facilitate the collection of voice instead of text answers to open questions. Speech-to-text transcriptions through Automatic Speech Recognition (ASR) systems pose an efficient way to make voice answers accessible to text-as-data methods. However, there is little evidence on the transcription performance of ASR systems when it comes to voice answers. We therefore investigate the performance of two leading ASR systems–Google’s Cloud Speech-to-Text API and OpenAI’s Whisper–using voice answers to two open questions administered in a smartphone survey in Germany. The results indicate that Whisper produces more accurate transcriptions than Google’s API. Both systems produce similar errors, but these errors are more common for the Google API. However, the Google API is faster than both Whisper and human transcribers.

ASJC Scopus Sachgebiete

Zitieren

Automatic speech-to-text transcription: evidence from a smartphone survey with voice answers. / Höhne, Jan Karem; Lenzner, Timo; Claassen, Joshua.
in: International Journal of Social Research Methodology, 01.01.2025.

Publikation: Beitrag in FachzeitschriftArtikelForschungPeer-Review

Höhne JK, Lenzner T, Claassen J. Automatic speech-to-text transcription: evidence from a smartphone survey with voice answers. International Journal of Social Research Methodology. 2025 Jan 1. Epub 2025 Jan 1. doi: 10.1080/13645579.2024.2443633
Download
@article{1f355341e8c6461b887f6e9fcdcfb3fb,
title = "Automatic speech-to-text transcription: evidence from a smartphone survey with voice answers",
abstract = "Advances in information and communication technology, coupled with a smartphone increase in web surveys, provide new avenues for collecting answers from respondents. Specifically, the microphones of smartphones facilitate the collection of voice instead of text answers to open questions. Speech-to-text transcriptions through Automatic Speech Recognition (ASR) systems pose an efficient way to make voice answers accessible to text-as-data methods. However, there is little evidence on the transcription performance of ASR systems when it comes to voice answers. We therefore investigate the performance of two leading ASR systems–Google{\textquoteright}s Cloud Speech-to-Text API and OpenAI{\textquoteright}s Whisper–using voice answers to two open questions administered in a smartphone survey in Germany. The results indicate that Whisper produces more accurate transcriptions than Google{\textquoteright}s API. Both systems produce similar errors, but these errors are more common for the Google API. However, the Google API is faster than both Whisper and human transcribers.",
keywords = "Automatic speech recognition (ASR), built-in microphone, narrative questions, smartphone survey, transcription quality",
author = "H{\"o}hne, {Jan Karem} and Timo Lenzner and Joshua Claassen",
note = "Publisher Copyright: {\textcopyright} 2024 The Author(s). Published by Informa UK Limited, trading as Taylor & Francis Group.",
year = "2025",
month = jan,
day = "1",
doi = "10.1080/13645579.2024.2443633",
language = "English",
journal = "International Journal of Social Research Methodology",
issn = "1364-5579",
publisher = "Taylor and Francis Ltd.",

}

Download

TY - JOUR

T1 - Automatic speech-to-text transcription

T2 - evidence from a smartphone survey with voice answers

AU - Höhne, Jan Karem

AU - Lenzner, Timo

AU - Claassen, Joshua

N1 - Publisher Copyright: © 2024 The Author(s). Published by Informa UK Limited, trading as Taylor & Francis Group.

PY - 2025/1/1

Y1 - 2025/1/1

N2 - Advances in information and communication technology, coupled with a smartphone increase in web surveys, provide new avenues for collecting answers from respondents. Specifically, the microphones of smartphones facilitate the collection of voice instead of text answers to open questions. Speech-to-text transcriptions through Automatic Speech Recognition (ASR) systems pose an efficient way to make voice answers accessible to text-as-data methods. However, there is little evidence on the transcription performance of ASR systems when it comes to voice answers. We therefore investigate the performance of two leading ASR systems–Google’s Cloud Speech-to-Text API and OpenAI’s Whisper–using voice answers to two open questions administered in a smartphone survey in Germany. The results indicate that Whisper produces more accurate transcriptions than Google’s API. Both systems produce similar errors, but these errors are more common for the Google API. However, the Google API is faster than both Whisper and human transcribers.

AB - Advances in information and communication technology, coupled with a smartphone increase in web surveys, provide new avenues for collecting answers from respondents. Specifically, the microphones of smartphones facilitate the collection of voice instead of text answers to open questions. Speech-to-text transcriptions through Automatic Speech Recognition (ASR) systems pose an efficient way to make voice answers accessible to text-as-data methods. However, there is little evidence on the transcription performance of ASR systems when it comes to voice answers. We therefore investigate the performance of two leading ASR systems–Google’s Cloud Speech-to-Text API and OpenAI’s Whisper–using voice answers to two open questions administered in a smartphone survey in Germany. The results indicate that Whisper produces more accurate transcriptions than Google’s API. Both systems produce similar errors, but these errors are more common for the Google API. However, the Google API is faster than both Whisper and human transcribers.

KW - Automatic speech recognition (ASR)

KW - built-in microphone

KW - narrative questions

KW - smartphone survey

KW - transcription quality

UR - http://www.scopus.com/inward/record.url?scp=85214406860&partnerID=8YFLogxK

U2 - 10.1080/13645579.2024.2443633

DO - 10.1080/13645579.2024.2443633

M3 - Article

AN - SCOPUS:85214406860

JO - International Journal of Social Research Methodology

JF - International Journal of Social Research Methodology

SN - 1364-5579

ER -