Whether retrieval can work with images depends on the models you picked for the knowledge base. Start from what you want retrieval to do with images:
-
Text-to-image search: a text question finds images in the knowledge base by what they show.
Pick a vision-capable embedding model. A question like “the org chart” can then match a diagram by what it shows, even if it has no caption or alt text.
-
Image-to-image search: an attached image finds visually similar images in the knowledge base.
Pick a vision-capable embedding model, so similar images match even when they share no text.
-
Image-to-text search: an attached image finds relevant text chunks.
Pick a vision-capable reasoning model. It describes the image in words and searches that description against text.
For all three, pick vision-capable models for both embedding and reasoning. Either model can be changed later under Settings > Retrieval. Changing the embedding model re-indexes every document and costs tokens.
After switching from two text-only models, click Re-index on each document that has images once the change finishes.
How Models Affect Image Behavior#
| Images are extracted | Text finds image | Image finds text | Image finds image | |
|---|---|---|---|---|
| Both support vision | Yes | By appearance and by caption or alt text | Deep and Research: yes. Fast: no, image queries find images only. | Yes |
| Embedding only | Yes | By appearance and by caption or alt text | No | Yes |
| Reasoning only | Yes | By caption or alt text only | Yes | No |
| Neither | Markdown and HTML: only the caption or alt text. Office files: no. | By caption or alt text only | No. Image-only queries fail, and images on a text query are ignored. | No |
Captions and alt text on images in Markdown and HTML files are searchable regardless of model choice, so well-captioned sources improve image retrieval in every combination.
- Text inside images in PDF files is read when either model accepts images.
- Text inside images in Word, PowerPoint, and Excel files isn’t read, so put anything users will search for in the surrounding text.