Whisper

← Back to the article

Whisper turns spoken audio into written words.

What it does

Whisper turns spoken audio into written words.

Transcribe: Turning speech to text • Robust: Working well with noise

How it works

  • Listen to the sound.
  • Find speech patterns.
  • Write the words it thinks were spoken.

Uses and applications

  • A video app can make captions.
  • A phone can turn voice notes into text.
  • A translation app can help change spoken words into another language.

In short

Whisper can turn speech into text, even with background noise.

Related methods

  • Wav2Vec 2.0: Learns speech features from unlabelled audio.
  • CTC (Alignment): A training method that aligns audio frames with text.
  • Transformer: The encoder-decoder architecture Whisper uses.

Back to Audio and speech · All topics

The interactive demonstration and the Python walkthrough for this topic need JavaScript.