CLIP
CLIP learns how pictures and words match each other.
What it does
CLIP learns how pictures and words match each other.
Embedding: Math code for data • Alignment: Matching word to photo
How it works
- Look at many image-and-text pairs.
- Put matching pictures and words close together in number space.
- Use that space to compare new pictures and words.
Uses & Applications
- A photo search can find dog pictures when you type “dog.”
- A safety tool can check whether an image matches a label.
- A robot can connect written instructions to what it sees.
In short
CLIP helps AI recognize that a word can match a photo.
Try it yourself
Match a picture to its name
The computer practices giving a matching picture and caption a higher score than a wrong pair.
Your turn
Change one setting at a time. Reset example puts everything back.
Match text and visual features
Step 1Why this happens