​

Chat + Vision (Image as Input)

The Bytez API enables multimodal capabilities, allowing chat models to process both text and images. This allows models to describe, compare, and analyze images alongside user queries.

​

Code

​

Text + Image

This example sends an image with a text prompt to generate a description.

​

Text + Multiple Images

Compare and analyze multiple images using a single query.

​

Streaming

​

How Streaming Works

To enable streaming, pass true as the third argument to the model.run() function. The model will return a stream that you can read incrementally.

const stream = await model.run(textInput, params, true);

​

Node.js Example

const { Readable } = require('stream');

const stream = await model.run(textInput, params, true);

try {
  const readableStream = Readable.fromWeb(stream); // Convert Web Stream to Node.js Readable Stream
  for await (const chunk of readableStream) {
    console.log(chunk.toString()); // Handle each chunk of data
  }
} catch (error) {
  console.error(error); // Handle errors
}

​

Browser Example

const stream = await model.run(textInput, params, true);

try {
  const reader = stream.getReader(); // Get a reader for the Web Stream

  while (true) {
    const { done, value } = await reader.read(); // Read the stream chunk-by-chunk
    if (done) break; // Exit when the stream ends
    console.log(new TextDecoder().decode(value)); // Convert Uint8Array to string
  }
} catch (error) {
  console.error(error); // Handle errors
}

​

Key Points

• Node.js: Convert the Web Stream using Readable.fromWeb() for compatibility.
• Browser: Use getReader() and TextDecoder to process the stream.
• Error Handling: Both methods use try…catch to handle potential errors.
• Data Handling: Data chunks are processed as they arrive via data events or .read() calls.

​

Explore Specialized Models

• Object Detection: Models trained to detect objects in images.
• Fill Mask: Models designed to fill in missing parts of an image.
• Image Classification: Models optimized for classifying images into categories.
• Image-to-Text: Models that generate textual descriptions for images.