Wav2Vec 2.0

← Back to the article

Wav2Vec learns from sound by guessing missing parts of audio.

What it does

Wav2Vec learns from sound by guessing missing parts of audio.

Self-Supervised: Learning without human labels • Waveform: Sound shape

How it works

  • Listen to lots of audio.
  • Hide small parts of the sound.
  • Learn by trying to guess the hidden parts.

Uses and applications

  • A speech tool can learn from audio without every second being typed by a person.
  • A caption system can improve with more sound examples.
  • A voice app can get better at different accents.

In short

It learns language from audio without needing many labels.

Related methods

  • Whisper: Trained end to end on paired audio and transcripts.
  • CTC (Alignment): Commonly used to fine-tune Wav2Vec for transcription.
  • BERT: Uses a similar masked-prediction idea on text.

Back to Audio and speech · All topics

The interactive demonstration and the Python walkthrough for this topic need JavaScript.