For AI agents: docs.octen.ai/capabilities/vl-embedding.md
Text, images, and videos normally live in separate vector spaces: a text query can't find an image, and a product photo can't find its description. VL Embedding closes that gap: it encodes text, images, and videos into one shared vector space, so any modality can retrieve any other. For the full list of parameters, see the [VL Embedding API reference](/api-reference/vl-embedding). For the concept behind embeddings, see [What are Embeddings](/capabilities/embedding#what-are-embeddings). ## Why VL Embedding * **Cross-modal search.** Query with text and retrieve images or videos, or the other way around. * **Visual retrieval.** Find similar images for dedup, recommendations, and visual search. * **Multimodal RAG.** Ground models on knowledge that includes screenshots, diagrams, and video. For text-only workloads, use [Text Embedding](/capabilities/embedding) instead. ## Why Octen * **SOTA multimodal retrieval.** Top-ranked on MMEB-v2 for retrieval across text, images, videos, and visual documents. * **Native video embedding.** Videos embed into the same space as text and images, so video libraries become searchable instead of staying a blind spot. * **Flexible output.** One vector per element, or a single fused vector for the whole multimodal input. ## Model Choices | Model | Context (tokens) | Max dimension | Best for | | -------------------------- | ---------------- | ------------- | ------------------------- | | `octen-vl-embedding-large` | 32,000 | 4096 | Best accuracy | | `octen-vl-embedding` | 32,000 | 2048 | Most production workloads | ## How It Works 1. **Embed your content.** Send text, images, or videos, mixed freely in one request. The model encodes them into one shared vector space: one vector per element, or a single fused vector for the combined input. 2. **Build a vector index.** Store the vectors in a vector database, the same way as text-only embeddings. 3. **Embed queries and retrieve.** Encode the query, in any modality, with the same model, and compare it against the index by similarity. The closest matches come back regardless of modality, so text can find images and videos, and an image can find its description. ## Scenarios ### Index your images Embed a product catalog, photo library, or asset collection so every image can be found by meaning. ```bash theme={null} curl -X POST https://api.octen.ai/vl-embedding \ -H "Content-Type: application/json" \ -H "x-api-key: YOUR_API_KEY" \ -d '{ "model": "octen-vl-embedding", "input": { "contents": [ { "image": "https://example.com/product-1.jpg" }, { "image": "https://example.com/product-2.jpg" } ] } }' ``` ### Find images by describing them No tags, no filenames: embed a plain description in the same space, and the closest image vectors are the results. ```json theme={null} { "model": "octen-vl-embedding", "input": { "contents": [{ "text": "red sneakers on a white background" }] } } ``` ### One vector for the whole item A product is more than its photo: fuse the title and the image into a single vector that represents the item as a whole, so one entry per item lands in your index. ```json theme={null} { "model": "octen-vl-embedding", "input": { "contents": [ { "text": "Ergonomic mesh office chair, black" }, { "image": "https://example.com/chair.jpg" } ] }, "enable_fusion": true } ``` ### Make videos searchable Tutorials, demos, and recordings hold knowledge that keyword search never reaches. Embed them like any document; lower the frame sampling to cut token cost on long footage. ```json theme={null} { "model": "octen-vl-embedding-large", "input": { "contents": [{ "video": "https://example.com/demo.mp4" }] }, "fps": 0.5 } ``` ### Customize the task instruction Pass a task description to control what the embedding captures. The instruction below makes the vector represent the screenshot's UI layout style, so retrieval matches by design rather than by subject. ```json theme={null} { "model": "octen-vl-embedding", "input": { "contents": [{ "image": "https://example.com/screenshot.png" }] }, "instruct": "Represent the UI layout style of the input." } ``` ## Next Steps