Body Required

• inputs array[object] Required

  A list of multimodal inputs to be vectorized.

  A single input in the list is a dictionary containing a single key "content", whose value represents a sequence of text and images.

  • The value of "content" is a list of dictionaries, each representing a single piece of text or image. The dictionaries have four possible keys:
  • type: Specifies the type of the piece of the content. Allowed values are text, image_url, or image_base64.
  • text: Only present when type is text. The value should be a text string.
  • image_base64: Only present when type is image_base64. The value should be a Base64-encoded image in the data URL format data:[<mediatype>];base64,<data>. Currently supported mediatypes are: image/png, image/jpeg, image/webp, and image/gif.
  • image_url: Only present when type is image_url. The value should be a URL linking to the image. We support PNG, JPEG, WEBP, and GIF images.
  • Note: Only one of the keys, image_base64 or image_url, should be present in each dictionary for image data. Consistency is required within a request, meaning each request should use either image_base64 or image_url exclusively for images, not both.

  Example payload where inputs contains an image as a URL:

  The inputs list contains a single input, which consists of a piece of text and an image (which is provided via a URL).

  { "inputs": [ { "content": [ { "type": "text", "text": "This is a banana." }, { "type": "image_url", "image_url": "https://raw.githubusercontent.com/voyage-ai/voyage-multimodal-3/refs/heads/main/images/banana.jpg" } ] } ], "model": "voyage-multimodal-3.5" }
  

  Example payload where inputs contains a Base64 image:

  Below is an equivalent example to the one above where the image content is a Base64 image instead of a URL. (Base64 images can be lengthy, so the example only shows a shortened version.)

  { "inputs": [ { "content": [ { "type": "text", "text": "This is a banana." }, { "type": "image_base64", "image_base64": "data:image/jpeg;base64,/9j/4AAQSkZJRgABAQAA..." } ] } ], "model": "voyage-multimodal-3.5" }
  

  The following constraints apply to the inputs list:

  • The list must not contain more than 1000 inputs.
  • Each image must not contain more than 16 million pixels or be larger than 20 MB in size.
  • With every 560 pixels of an image being counted as a token, each input in the list must not exceed 32,000 tokens, and the total number of tokens across all inputs must not exceed 320,000.

  At least 1 but not more than 1000 elements.

  Hide inputs attribute Show inputs attribute object

  • content array[object] Required

    A sequence of text and images.

    At least 1 element.

    One of:

    object-1 object object-2 object object-3 object

    Text content

    Hide attributes Show attributes

    • type string Required

      Value is text.

    • text string Required

      The text string.

      Minimum length is 1.

    Image content via URL

    Hide attributes Show attributes

    • type string Required

      Value is image_url.

    • image_url string(uri) Required

      URL linking to the image. Supports PNG, JPEG, WEBP, and GIF.

    Image content via Base64

    Hide attributes Show attributes

    • type string Required

      Value is image_base64.

    • image_base64 string Required

      Base64-encoded image in data URL format (data:[mediatype];base64,[data]). Supported mediatypes are image/png, image/jpeg, image/webp, and image/gif.

      Minimum length is 1.

• model string Required

  The multimodal embedding model to use. Recommended model: voyage-multimodal-3.5.

  Values are voyage-multimodal-3.5 or voyage-multimodal-3.

• input_type

  Type of the input. Defaults to null. Other options: query, document.

  • When input_type is null, the embedding model directly converts the inputs into numerical vectors. For retrieval or search purposes, where a "query", which can be text or image in this case, searches for relevant information among a collection of data referred to as "documents," specify whether your inputs are queries or documents by setting input_type to query or document, respectively. In these cases, Voyage automatically prepends a prompt to your inputs before vectorizing them, creating vectors more tailored for retrieval or search tasks. Since inputs can be multimodal, "queries" and "documents" can be text, images, or an interleaving of both modalities. Embeddings generated with and without the input_type argument are compatible.
  • For transparency, the following prompts are prepended to your input:
  • For query, the prompt is "Represent the query for retrieving supporting documents: ".
  • For document, the prompt is "Represent the document for retrieval: ".

  Values are query, document, or null.

• truncation boolean

  Whether to truncate the inputs to fit within the context length. Defaults to true.

  • If true, over-length inputs are truncated to fit within the context length before vectorization by the embedding model. If the truncation happens in the middle of an image, the entire image is discarded.
  • If false, an error occurs if any input exceeds the context length.

  Default value is true.

• output_encoding

  Format in which the embeddings are encoded. Defaults to null.

  • If null, the embeddings are represented as a list of floating-point numbers.
  • If base64, the embeddings are represented as a Base64-encoded NumPy array of single-precision floats.

  Values are base64 or null.