Skip to content
Floopy Docs

Providers

Overview

Floopy acts as a gateway between your application and LLM providers. To route requests, you need to add at least one provider’s API key to your organization. Floopy supports the following providers:

  • OpenAI — GPT-4o, GPT-4.1, o3, o4-mini, and other OpenAI models
  • Anthropic — Claude 4, Claude 3.5 Sonnet, and other Anthropic models
  • Google Gemini — Gemini 2.5 Pro, Gemini 2.0 Flash, and other Google models
  • Groq — Llama, Mixtral, and other models on Groq’s inference engine
  • Mistral — Mistral Large, Codestral, and other Mistral AI models
  • DeepSeek — DeepSeek V3, DeepSeek R1, and other DeepSeek models
  • xAI — Grok 4.20, Grok 4.1 Fast, and other xAI models
  • Perplexity — Sonar, Sonar Pro, and other search-augmented models
  • Novita — DeepSeek, Qwen, Llama, and other open-source models via Novita inference
  • Nebius — DeepSeek, Qwen, Llama, and other open-source models via Nebius inference
  • DeepInfra — DeepSeek, Qwen, Llama, and other open-source models via DeepInfra inference
  • Azure OpenAI — GPT-4o, GPT-4.1, o3, and other OpenAI models on Azure
  • Together — Llama, Mixtral, Qwen, and other open-source models via Together inference
  • Fireworks — Llama, Mixtral, and other open-source models via Fireworks inference
  • Cohere — Command R, Command R+, and other Cohere models
  • AI21 — Jamba 1.5 and other AI21 Labs models
  • Cerebras — Llama and other models on Cerebras wafer-scale inference
  • SambaNova — Llama, DeepSeek, and other models on SambaNova accelerated inference
  • AWS Bedrock — Access AWS Bedrock models via custom base URL configuration
  • Google Vertex — Access Google Vertex AI models via custom base URL configuration

Adding a Provider

  1. Go to Settings > Providers in the dashboard.
  2. Click Add provider.
  3. Select the provider from the list.
  4. Paste your API key for that provider.
  5. Click Save.

Your provider API key is encrypted at rest and never exposed in the dashboard after saving.

How It Works

When your application sends a request to https://api.floopy.ai/v1, Floopy determines which provider to call based on:

  1. The model name in the request (e.g., gpt-4o routes to OpenAI, claude-3-5-sonnet routes to Anthropic).
  2. The routing rule assigned to the API key, if one exists.

Floopy translates the request into the provider’s native API format, forwards it, and returns the response in OpenAI-compatible format. Your application code does not need to change when switching providers.

Supported Models

Each provider exposes its full set of chat completion models through the gateway. Use the model name exactly as the provider documents it:

import { OpenAI } from "openai";


const client = new OpenAI({
  baseURL: "https://api.floopy.ai/v1",
  apiKey: process.env.FLOOPY_API_KEY,
});


// OpenAI
const a = await client.chat.completions.create({
  model: "gpt-4o",
  messages: [{ role: "user", content: "Hello" }],
});


// Anthropic (via Floopy gateway)
const b = await client.chat.completions.create({
  model: "claude-3-5-sonnet-20241022",
  messages: [{ role: "user", content: "Hello" }],
});


// Google Gemini (via Floopy gateway)
const c = await client.chat.completions.create({
  model: "gemini-2.5-pro",
  messages: [{ role: "user", content: "Hello" }],
});

Updating a Provider Key

To rotate or update a provider’s API key:

  1. Go to Settings > Providers.
  2. Find the provider and click Edit.
  3. Paste the new API key.
  4. Click Save.

The change takes effect immediately. The gateway cache is invalidated automatically, so new requests will use the updated key.

Removing a Provider

To remove a provider:

  1. Go to Settings > Providers.
  2. Find the provider and click Remove.
  3. Confirm the action.

If the removed provider is referenced in a routing rule, requests that would route to it will fail over to the next available provider (if using fallback routing) or return an error.

Multiple Providers

Adding multiple providers enables powerful routing strategies. For example, you can:

  • Set up fallback routing so requests automatically retry on a different provider if one is down.
  • Use weighted routing to split traffic 80/20 between providers for cost optimization.
  • Configure latency-based routing to always use the fastest provider.

See the Routing guide for details on configuring these strategies.