Routing Decision Flow

Floopy has many places that can decide which provider and model serve a request: the request body, headers, prompts, A/B tests, Smart Selector, routing rules, Smart Cost, and feedback-driven routing. Configure several of them and they will disagree.

This page is the tie-breaker. It lists every layer in the order it runs, states what each one can override, and answers the question directly: if everything is configured at once, what actually gets called?

The routing rule’s target list decides the final (provider, model). Everything upstream — the request body, headers, prompts, Smart Selector — feeds into it, but a routing rule dispatches its own targets, and the target’s model is the one sent to the provider.

Two things then override the target list:

1. Smart Cost, when it swaps a model, becomes the primary target and the rule’s targets become its fallbacks.
2. The MCP agent loop, when active, takes over dispatch entirely.

If no routing rule exists, there are no targets, and the model resolved upstream (request body, prompt, or provider default) is the one used.

Each layer runs after the one above it, so a later layer overwrites an earlier one.

The model in the request body is the starting point. The floopy-model-override header replaces it outright, before anything else runs — nothing downstream ever sees the original.

Both pick a variant, but they are not equal:

• Smart Selector can set a model and a provider.
• An A/B test only swaps which prompt is used. It never sets a model.

When both are configured, Smart Selector wins and the A/B test is not consulted.

A prompt can carry its own model and provider, which override the request body.

prompt_overwrite reverses that: it hands control back to the client, so the request body’s model wins over the prompt’s. It also causes the prompt’s provider to be dropped.

It is set on the prompt in the dashboard, and the floopy-prompt-overwrite header overrides that column in both directions — a request can force it on for a prompt that has it off, and vice versa.

Only used when nothing above resolved a model. It is a floor, not an override.

Smart Cost classifies prompt complexity and, on the simple and moderate tiers, swaps in a cheaper model from the providers you configured on the rule.

When it swaps, its choice becomes the rule’s primary target, and your configured targets stay behind it as fallbacks — so a failed swap still degrades to your list rather than erroring.

A complex prompt bypasses Smart Cost entirely and leaves your configured strategy untouched. So Smart Cost owns the cheap tier, and your routing strategy — including feedback-driven routing — owns everything Smart Cost declines.

This is where the model is finally decided. A routing rule dispatches its own targets, and each target carries its own (provider, model). Whichever target the strategy picks, that target’s model is the one sent to the provider — the model resolved in layers 1–7 is replaced.

Within the rule, the strategy chooses which target:

• Feedback-driven routing, if your plan enables it, replaces the rule’s configured strategy entirely. A rule set to weighted will not behave as weighted for an org on feedback-driven routing.
• Otherwise the rule’s own strategy runs: fallback, round-robin, weighted, or latency-based.

If a target is skipped (no API key, unknown provider, open circuit breaker) or fails, the next target is tried.

If your plan has MCP outbound and the org has at least one enabled outbound server, the request enters the agent loop instead of the normal strategy chain. Send floopy-mcp-disabled to opt out per request.

flowchart TD
    A[Request body model] --> B{floopy-model-override header?}
    B -->|yes| C[Header model replaces it]
    B -->|no| D[Keep body model]
    C --> E{Smart Selector?}
    D --> E

    E -->|yes| F[Smart Selector sets model + provider<br/>A/B test is skipped]
    E -->|no| G{A/B test?}
    G -->|yes| H[Swaps the prompt only<br/>model unchanged]
    G -->|no| I[No variant]

    F --> J{Prompt resolved?}
    H --> J
    I --> J

    J -->|yes| K{prompt_overwrite?}
    J -->|no| L{Model still empty?}

    K -->|false| M[Prompt model + provider win]
    K -->|true| N[Request model wins<br/>prompt provider dropped]

    M --> O
    N --> O
    L -->|yes| P[Provider default model]
    L -->|no| O[Model resolved]
    P --> O

    O --> Q{MCP agent active?}
    Q -->|yes| R[Agent loop owns dispatch]
    Q -->|no| S{Routing rule exists?}

    S -->|no| T[Legacy cascade<br/>resolved model is dispatched]
    S -->|yes| U{Smart Cost enabled<br/>and prompt not complex?}

    U -->|yes| V[Cheaper model becomes PRIMARY target<br/>configured targets become fallbacks<br/>strategy pinned to fallback]
    U -->|no| W{Feedback-driven routing?}

    W -->|yes| X[Scores targets, ignores rule strategy<br/>explores per the exploration budget]
    W -->|no| Y[Rule strategy:<br/>fallback / round-robin / weighted / latency]

    V --> Z
    X --> Z
    Y --> Z[Dispatch target<br/>TARGET MODEL replaces the resolved model]

    Z --> AA{Target OK?}
    AA -->|skipped or failed| AB[Try next target]
    AB --> Z
    AA -->|success| AC[Response]
    T --> AC
    R --> AC

A request arrives with:

• body model: "gpt-4o"
• header floopy-model-override: gpt-4o-mini
• a prompt whose model is claude-sonnet-5, with prompt_overwrite = false
• a routing rule with targets [openai/gpt-4o, anthropic/claude-3], strategy weighted
• Smart Cost enabled, with openai/gpt-4o-mini configured for the simple tier
• the org on a plan with feedback-driven routing

What runs:

1. The header replaces the body model → gpt-4o-mini.
2. The prompt sets the model → claude-sonnet-5 (prompt_overwrite is false, so the prompt wins).
3. The prompt is simple, so Smart Cost swaps in openai/gpt-4o-mini and makes it the primary target. The rule’s targets become fallbacks, and the strategy is pinned to fallback.
4. openai/gpt-4o-mini is called. If it fails, openai/gpt-4o is tried, then anthropic/claude-3.

Note what did not happen: weighted never ran, feedback-driven routing never ran, and neither gpt-4o nor claude-sonnet-5 was called. Smart Cost declines on a complex prompt — and only then would feedback-driven routing score the rule’s two targets and pick between them.

Feedback-driven routing and Smart Cost do not always pick the best-scoring model — that would mean never trying anything else, and a model that loses once would lose forever, because it never gets dispatched and so never gathers the data that would let it win.

A fraction of traffic explores instead. One draw per request, seeded on the request id so a decision is reproducible:

The untested share comes out of the exploration budget, it is not added to it. At an exploration rate of 20% and an untested share of 50%, untested models get 10% of all traffic and scored exploration the other 10%. Total exploration is still 20%.

A model with no history has no quality score, so the gate would judge it on its static benchmark — and a model with no published benchmark scores a neutral default that sits below the usual minimum. It would be filtered out before exploration could ever try it, so it could never earn the score that would let it through. The gate would be the very thing keeping it untested.

Set the untested share to 0% to disable this and only ever explore models that already clear the quality gate.

Both values are configured under Routing → Feedback-driven in the dashboard, and the effective percentage of total traffic is shown under the slider.

• Routing strategies — how fallback, round-robin, weighted and latency-based each distribute traffic
• Smart Cost Routing
• Feedback-driven routing
• Prompts
• Explaining a decision — ask the gateway why it picked what it picked, for a specific request