# Model providers

A model-provider plugin makes a vendor selectable under **AI Providers → Add provider**, next to the built-in ones.
The user adds it with its own settings (typically their API key), adds models under it, and picks those models for
agents — exactly like a native provider. Your plugin translates between AgentParley's request format and the
vendor's API.

A provider plugin can host any combination of four groups:

| Group | Handlers | What the user gets |
| --- | --- | --- |
| Chat | `complete` | models an agent can think with |
| Decision ("System One") | `noul` + `choose` + `score` — all three or none | fast typed answers for `decide` and `host.decide` |
| Voice | `speak` + `voices` — both or neither | companion voices — see [Voice and speech-to-text](plugin-voice-and-speech.md) |
| Speech-to-text | `transcribe` (+ optional `startLiveTranscription`) | companion listening — see [Voice and speech-to-text](plugin-voice-and-speech.md) |

This page covers chat and decision models.

## Declare the provider

```ts
// infrastructure/plugins/keyword-decider/index.ts
globalThis.manifest = {
  modelProvider: { kindId: "keyword-decider", displayName: "Keyword Decider (test)" },
  settings: [
    {
      key: "simulateFailure",
      title: "Simulate failure",
      type: "enum",
      options: ["none", "keyRejected", "unavailable"],
      default: "none",
    },
  ],
};
```

`manifest.modelProvider` is required as soon as you assign any `modelProvider.*` handler (see
[Manifest](plugin-manifest.md#modelprovider)). `kindId` is your provider's permanent id: lowercase letters, digits and
dashes, 2–64 characters, starting with a letter, and not the id of a built-in provider. `displayName` (up to 100
characters) is shown as "*Keyword Decider (test)* · from plugin"; `description` is up to 500.

**Settings belong to the provider the user added, not to the install.** When a user adds your provider they fill
in your settings form; `host.settings.get` / `getSecret` in a provider handler read *that provider's* configured
values, not the installation's.

## Reporting vendor failures — `tools.fail`

Every provider handler receives `tools` with one helper:

```ts
tools.fail(kind, message): ModelProviderFailure
```

| `kind` | Means | User sees |
| --- | --- | --- |
| `keyRejected` | bad or revoked key (401/403) | the red provider-failure banner |
| `noCredit` | out of credit (402) | the red banner |
| `modelNotFound` | the saved model id doesn't exist | the red banner |
| `requestRejected` | the vendor refused this request (400/422) | the red banner |
| `rateLimited` | 429 | no banner — transient |
| `unavailable` | 5xx, down | no banner — transient |

Return the value — `return tools.fail("keyRejected", "…")` — instead of throwing. A thrown error or a timeout is
treated as `unavailable`. The OpenAI Voice plugin maps HTTP status codes like this:

```ts
// infrastructure/plugins/openai-voice/index.ts
function mapFailure(status: number, tools: ModelProviderTools): ModelProviderFailure {
  if (status === 401 || status === 403) return tools.fail("keyRejected", "OpenAI rejected the API key");
  if (status === 402) return tools.fail("noCredit", "OpenAI account is out of credit");
  if (status === 429) return tools.fail("rateLimited", "OpenAI rate limit exceeded");
  if (status >= 500) return tools.fail("unavailable", `OpenAI returned ${status}`);
  return tools.fail("requestRejected", `OpenAI returned ${status}`);
}
```

## Chat models — `complete`

> Written for these docs against the SDK types — no shipped plugin implements `complete` yet.

```ts
modelProvider.complete(request: CompletionRequest, tools): CompletionResponse | ModelProviderFailure
```

The request is AgentParley's normalized chat request:

```ts
interface CompletionRequest {
  schemaVersion: number;
  model: string;                     // the model identifier the user saved
  messages: {
    role: "system" | "user" | "assistant" | "tool";
    content: string | null;
    toolCalls?: { id: string; name: string; argumentsJson: string }[];   // on assistant messages
    toolCallId?: string;                                                // on tool results
    media?: { type: "image" | "audio"; url: string; contentType: string }[];
    thinking?: { text: string | null; replayJson: string | null };
  }[];
  tools: { name: string; description: string; parametersJsonSchema: string }[];
  params: { temperature: number | null; maxTokens: number | null; topP: number | null;
            stop: string[] | null; thinking: string };
}
```

Return the assistant turn and the token usage:

```ts
interface CompletionResponse {
  schemaVersion: 1;
  assistant: { text: string | null; toolCalls: { id: string; name: string; argumentsJson: string }[];
               thinking?: { text: string | null; replayJson: string | null } };
  usage: { inputTokens: number; outputTokens: number; cachedTokens: number };
  stopReason: "stop" | "tool_calls" | "length" | "content_filter" | "error";
}
```

A minimal adapter for an OpenAI-style vendor:

```ts
globalThis.manifest = {
  egress: ["api.example-llm.com"],
  modelProvider: { kindId: "example-llm", displayName: "Example LLM", mediaSupport: ["image"] },
  settings: [{ key: "apiKey", title: "API key", type: "secret", required: true }],
};

globalThis.modelProvider.complete = async (request, tools) => {
  const apiKey = await host.settings.getSecret("apiKey");
  const response = await host.fetch({
    method: "POST",
    url: "https://api.example-llm.com/v1/chat/completions",
    headers: { Authorization: `Bearer ${apiKey}`, "Content-Type": "application/json" },
    body: JSON.stringify({
      model: request.model,
      messages: request.messages.map((m) => ({ role: m.role, content: m.content, tool_call_id: m.toolCallId })),
      tools: request.tools.map((t) => ({
        type: "function",
        function: { name: t.name, description: t.description, parameters: JSON.parse(t.parametersJsonSchema) },
      })),
      temperature: request.params.temperature ?? undefined,
      max_tokens: request.params.maxTokens ?? undefined,
    }),
  });
  if (response.status === 401) return tools.fail("keyRejected", "Example LLM rejected the API key");
  if (response.status !== 200) return tools.fail("unavailable", `Example LLM returned ${response.status}`);

  const body = JSON.parse(response.body);
  const choice = body.choices[0];
  const toolCalls = (choice.message.tool_calls ?? []).map((c: any) => ({
    id: c.id, name: c.function.name, argumentsJson: c.function.arguments,
  }));
  return {
    schemaVersion: 1,
    assistant: { text: choice.message.content, toolCalls },
    usage: { inputTokens: body.usage.prompt_tokens, outputTokens: body.usage.completion_tokens, cachedTokens: 0 },
    stopReason: toolCalls.length > 0 ? "tool_calls" : "stop",
  };
};
```

Notes:

- **No credential crosses the request.** Read your key with `host.settings.getSecret`.
- **`mediaSupport`** in the manifest says which media your models accept. The platform strips unsupported media
  from `messages` before calling you, the same as for native providers. For a call that carries media, the media
  URLs' hosts are added to your egress for that call only, so `host.fetch` can download them.
- **Usage is what the user pays.** You report tokens; the platform prices them at the per-token rates the user
  entered for the model. Plugins never report a cost. `inputTokens` is **uncached input only**: report cached
  tokens in `cachedTokens` and leave them out of `inputTokens` (an OpenAI-style `prompt_tokens` includes them, so
  send `prompt_tokens - cached_tokens`), or they are billed twice.
- `complete` is not streamed: the reply appears when your handler returns. It gets the ordinary 30 s wall clock,
  and each `host.fetch` inside it times out after **10 s** — a vendor reply that takes longer fails as
  `unavailable`.
- Your handler must return an object with `assistant` and `usage`, or the call fails as a bad request.

## Decision models — `noul`, `choose`, `score`

Decision models answer typed questions about a piece of state, fast and with calibrated numbers. Agents use them
through the built-in `decide` tool; plugins through `host.decide`; companions to pick gestures. Your plugin can
host them.

All three handlers take the same request shape — **one call answers every question**, reading `state` once:

```ts
request: {
  model: string;
  state: string | Record<string, unknown> | unknown[];
  questions: NoulQuestion[] | ChoiceQuestion[] | ScoreQuestion[];
}
returns: { answers: …[]; inputTokens: number }   // same order and count as `questions`
```

| Handler | Question | Answer |
| --- | --- | --- |
| `noul` — how likely is this true? | `{ instructions, trueMeans?, falseMeans? }` | `{ probability }` in 0–1 |
| `choose` — pick one option | `{ instructions, options: { [key]: description \| null } }` | `{ choice, confidence?, probabilities? }` — `choice` must be one of the keys |
| `score` — place on a scale | `{ instructions, levels: string[] }` (low → high) | `{ score, confidence?, probabilities? }` — `score` in 0…levels.length − 1, may be fractional |

```ts
// infrastructure/plugins/keyword-decider/index.ts (abridged)
globalThis.modelProvider = {
  noul(request, tools) {
    const failure = simulatedFailure(tools);
    if (failure) return failure;

    const text = stateText(request.state);
    const answers = request.questions.map((question) => {
      const positive = overlap(text, question.trueMeans ?? question.instructions);
      const negative = overlap(text, question.falseMeans ?? "");
      const sign = positive === negative ? 0 : positive > negative ? 1 : -1;
      return { probability: 0.5 + 0.4 * sign };
    });
    return { answers, inputTokens: Math.ceil(text.length / 4) };
  },
  choose(request, tools) { /* … */ },
  score(request, tools) { /* … */ },
};
```

- Implement **all three** or none — the platform may call any of them, and adding a model runs a test `noul`.
- 10 s wall clock per call. `inputTokens` is what the user is charged for, at their rate.

## Rules for provider plugins

- **No agent powers.** A provider translates a vendor's API; it doesn't act as an agent. Provider handlers get no
  `host.session`, `host.agents`, `host.llm` or `host.decide`, and publishing a provider plugin that calls
  `host.session.message.post`, `todo` writes, `subagent.spawn`, `peer.send`, `notify.schedule` or
  `host.agents.create/update/delete` fails. Account reads such as `host.models.list()` are allowed.
- **No silent fallback.** If your model fails, the agent's turn fails visibly (with the banner for the four
  "hard" kinds). The platform never quietly switches the user to another model.

## How a user sets it up

1. Install the plugin.
2. **AI Providers → Add provider → "*Your name* · from plugin"**, fill in your settings (their API key).
3. Add a model under it, entering the identifier your vendor uses (it reaches you as `request.model`), with its
   per-token prices.
4. Pick that model on an agent — or as the account's decision model for decision providers.