Completions - Bytez

Provides completions for all open source models that are text-generation, it also supports completions models from the closed source providers, openai, anthropic, mistral, cohere, and google. To specify a provider, prefix the model with the provider, e.g. davinci-002 should be passed in as openai/davinci-002 We provide access to models from openai, mistral, and google. You will need to supply a header provider-key in order to make requests to cohere models. NOTE: Logprobs are supported for all models!

Basic usage (Open Source)

import OpenAI from "openai";

const client = new OpenAI({
  apiKey: "BYTEZ_KEY",
  baseURL: "https://api.bytez.com/models/v2/openai/v1"
});

const response = await client.completions.create({
  model: "openai-community/gpt2",
  prompt: "Write a short poem about AI",
  temperature: 0.7,
  max_tokens: 150
});

console.log(response);

Streaming (Open Source)

import OpenAI from "openai";

const client = new OpenAI({
  apiKey: "BYTEZ_KEY",
  baseURL: "https://api.bytez.com/models/v2/openai/v1"
});

const stream = await client.completions.create({
  model: "openai-community/gpt2",
  prompt: "Write a short poem about AI",
  max_tokens: 150,
  temperature: 0.7,
  stream: true
});

let text = '';
for await (const event of stream) {
  if (event.choices[0].finish_reason) {
    break;
  }

  const content = event.choices[0].text;
  text += content;
  console.log(content);
}

console.log({ text });

Basic usage (Closed Source)

import OpenAI from "openai";

const client = new OpenAI({
  apiKey: "BYTEZ_KEY",
  baseURL: "https://api.bytez.com/models/v2/openai/v1"
});

const response = await client.completions.create({
  model: "openai/davinci-002",
  prompt: "Write a short poem about AI",
  temperature: 0.7,
  max_tokens: 150
});

console.log(response);

Streaming (Closed Source)

import OpenAI from "openai";

const client = new OpenAI({
  apiKey: "BYTEZ_KEY",
  baseURL: "https://api.bytez.com/models/v2/openai/v1"
});

const stream = await client.completions.create({
  model: "openai/davinci-002",
  prompt: "Write a short poem about AI",
  max_tokens: 150,
  temperature: 0.7,
  stream: true
});

let text = '';
for await (const event of stream) {
  if (event.choices[0].finish_reason) {
    break;
  }

  const content = event.choices[0].text;
  text += content;
  console.log(content);
}

console.log({ text });

Model API