llm_wire

Prepares, runs and streams LLM calls through an application-owned http_gun.Client.

import gleam/io
import http_gun
import http_gun/config as http_config
import llm_wire
import llm_wire/openai

pub fn main() -> Nil {
  let assert Ok(client) = http_gun.start(http_config.default())
  let config = openai.new("sk-...") |> openai.config
  let request = llm_wire.request("gpt-5", [llm_wire.user("Hello")])
  let assert Ok(prepared) = llm_wire.prepare(config, request)
  case llm_wire.run(client, prepared) {
    Ok(llm_wire.Answer(text:, ..)) -> io.println(text)
    Ok(_) -> io.println("no final answer")
    Error(failure) -> io.println(llm_wire.describe_failure(failure))
  }
  http_gun.stop(client)
}

prepare is pure: it checks the configuration, messages, tools and schemas and encodes the request once, before any I/O. run executes it to an Outcome; stream returns a Stream read with next until Done. A prepared call can run again, for a retry. Structured output is one more request step, with_output(name, codec): the Answer then holds the decoded value, and invalid output fails with error.InvalidOutput, keeping the raw text.

The caller owns the conversation and the tool loop: on NeedsTools, append the turn and one tool_result per call, and prepare the next request. LLM Wire never retries; advise(failure) says whether another attempt may help and how long to wait.

Default	Value	Setter
whole call, start to final event	600 s	`with_call_timeout`
first token, start to first progress	180 s	`with_first_token_timeout`
idle gap between provider events	60 s, reset by every event	`with_idle_timeout`
byte and count bounds	see `llm_wire/limit`	`with_limit`
invalid tool calls	fail the response	`with_tool_call_checks`
plaintext `http://`	loopback addresses only	none: HTTP Gun’s destination policy

Each call’s whole-call budget replaces the HTTP Gun client’s relative request timeout and lifts its idle timeout. An explicit absolute deadline on the supplied client remains a ceiling, including with an infinite call budget. The client’s connect and pool timeouts, destination policy and trust still apply.

Types

Bound

</>

A time bound that may be lifted. Infinity must be chosen explicitly.

pub type Bound {
  After(duration.Duration)
  Infinity
}

Constructors

```
After(duration.Duration)
```
```
Infinity
```

CloseOutcome

</>

pub type CloseOutcome {
  Closed
  AlreadyEnded
}

Constructors

```
Closed
```
The call was running and is now cancelled.
```
AlreadyEnded
```
The call had already ended.

Config

</>

A provider and its execution settings, built by llm_wire/openai, llm_wire/anthropic, llm_wire/google or llm_wire/provider. It is pure data, opens nothing, and never prints its API key.

pub type Config =
  @internal Config

Event

</>

One item of a stream.

pub type Event(o) {
  Progress(message.Progress)
  Done(Result(Outcome(o), Failure))
}

Constructors

```
Progress(message.Progress)
```
```
Done(Result(Outcome(o), Failure))
```
The call ended; the stream is finished.

Failure

</>

A failed call. Read fields by label; it may gain fields.

pub type Failure {
  Failure(
    error: error.Error,
    sent: Sent,
    partial_output: Bool,
    provider: message.Provider,
    usage: option.Option(message.Usage),
  )
}

Constructors

Failure(
  error: error.Error,
  sent: Sent,
  partial_output: Bool,
  provider: message.Provider,
  usage: option.Option(message.Usage),
)

Arguments

partial_output: Whether progress, such as text, had streamed before the failure.
usage: The last usage the provider reported, for token accounting.

Outcome

</>

How a call ended without failing. Read fields by label.

pub type Outcome(o) {
  Answer(
    output: o,
    text: String,
    usage: option.Option(message.Usage),
  )
  NeedsTools(
    turn: message.AssistantTurn,
    issues: List(tool.ToolCallIssue),
    usage: option.Option(message.Usage),
  )
  OutputLimited(
    partial_text: String,
    partial_calls: List(message.ToolCall),
    usage: option.Option(message.Usage),
  )
  Refused(reason: String, usage: option.Option(message.Usage))
}

Constructors

```
Answer(
  output: o,
  text: String,
  usage: option.Option(message.Usage),
)
```
The final answer. output is the text for a plain request, or the decoded value for a request built with with_output.
```
NeedsTools(
  turn: message.AssistantTurn,
  issues: List(tool.ToolCallIssue),
  usage: option.Option(message.Usage),
)
```
The model asks for tool results. Append message.Assistant(turn) and one tool_result per call to the conversation. issues lists calls that failed admission under tool.ReportInvalidToolCalls.

OutputLimited(
  partial_text: String,
  partial_calls: List(message.ToolCall),
  usage: option.Option(message.Usage),
)

The output token limit cut the answer.

```
Refused(reason: String, usage: option.Option(message.Usage))
```
The model declined in its own words: OpenAI’s refusal content, or a custom provider’s provider.refused. A provider’s safety filter is not an outcome: the call fails with error.ContentFiltered.

Prepared

</>

A request admitted and encoded for one configuration. Run it any number of times.

pub type Prepared(o) =
  @internal Prepared(o)

ReadError

</>

Why a read returned no event.

pub type ReadError {
  StreamEnded
  ConcurrentRead
  OwnerGone
  TimedOut
}

Constructors

```
StreamEnded
```
The stream already delivered Done or was closed.
```
ConcurrentRead
```
Another read of this stream is waiting.
```
OwnerGone
```
The call’s owner process is gone.
```
TimedOut
```
next_within waited its whole duration; the stream stays readable.

Request

</>

A request whose answer decodes to o: String for plain text, or the type of the codec given to with_output.

pub type Request(o) =
  @internal Request(o)

RetryAdvice

</>

The retry decision for a failure and when to act on it. Read the fields by label; it may gain fields.

pub type RetryAdvice {
  RetryAdvice(prospect: RetryProspect, delay: RetryDelay)
}

Constructors

RetryAdvice(prospect: RetryProspect, delay: RetryDelay)

RetryDelay

</>

How long to wait before another attempt, and who chose the time. A scheduler that distinguishes the two can snooze for a provider’s delay without counting an attempt, and back off otherwise.

pub type RetryDelay {
  ProviderDelay(duration.Duration)
  Backoff
}

Constructors

```
ProviderDelay(duration.Duration)
```
The provider’s own Retry-After, read from the response headers. A scheduler maps it to a snooze: the provider chose the time, so no attempt is spent. (Not a scheduler’s “retry after”: that spends one.) LLM Wire does not cap it: bound it before sleeping or scheduling. A date in the past is a zero delay.
```
Backoff
```
The provider named no delay: choose a backoff. Also the delay of a failure that carried no response headers, such as a timeout.

RetryProspect

</>

Whether another attempt of the same prepared call may succeed.

pub type RetryProspect {
  MayHelp
  WillNotHelpUnchanged
  Unknown
}

Constructors

```
MayHelp
```
The failure may be transient. Another attempt is not guaranteed to succeed, and failure.sent says whether the first one cost tokens.
```
WillNotHelpUnchanged
```
The request, configuration, limits, account or cancellation must change first.
```
Unknown
```
The failure does not establish a prospect.

Sent

</>

Whether the request reached the provider.

pub type Sent {
  NotSent
  MaybeSent
  Completed
}

Constructors

```
NotSent
```
Nothing reached the network; another attempt cannot repeat any work.
```
MaybeSent
```
The provider may have received the request and spent tokens on it.
```
Completed
```
The provider finished its response, with an error status, a reported error or content that failed admission.

Stream

</>

An executing call. Read it with next until Done; close ends it early. Only the process that started it may read it, one read at a time.

pub type Stream(o) =
  @internal Stream(o)

Values

advise

</>

pub fn advise(failure: Failure) -> RetryAdvice

Decide whether another attempt may help, from the failure alone.

HTTP Gun failures are classified by their Kind: unavailable, network and timeout failures may help; refused destinations, invalid requests and limits will not, nor will a provider’s content filter (error.ContentFiltered): the prompt must change. HTTP statuses 408, 429, 500, 502, 503 and 504 (and Anthropic’s 529) may help. Provider error codes are matched exactly per provider; prose is never parsed. The delay is ProviderDelay when the provider sent a readable Retry-After (delay seconds or an HTTP date) and Backoff otherwise. LLM Wire never retries by itself, and MayHelp does not make a MaybeSent call free: it may have spent tokens.

case llm_wire.advise(failure) {
  RetryAdvice(MayHelp, delay: ProviderDelay(wait)) -> snooze(wait)
  RetryAdvice(MayHelp, delay: Backoff) -> retry_with_backoff()
  RetryAdvice(_, _) -> give_up(llm_wire.describe_failure(failure))
}

append

</>

pub fn append(
  request: Request(o),
  messages: List(message.Message),
) -> Request(o)

Append messages to the request’s conversation.

assistant

</>

pub fn assistant(text: String) -> message.Message

An assistant message the application wrote, without provider data.

close

</>

pub fn close(stream: Stream(o)) -> CloseOutcome

End the call early. Safe to call more than once.

collect

</>

pub fn collect(stream: Stream(o)) -> Result(Outcome(o), Failure)

Read the stream to its outcome, discarding progress.

describe_failure

</>

pub fn describe_failure(failure: Failure) -> String

A one-line description of a failure for logs.

messages

</>

pub fn messages(request: Request(o)) -> List(message.Message)

The request’s conversation.

model

</>

pub fn model(request: Request(o)) -> String

</>

pub fn next(stream: Stream(o)) -> Result(Event(o), ReadError)

Wait for the stream’s next event. The call’s timers bound the wait.

next_within

</>

pub fn next_within(
  stream: Stream(o),
  wait: duration.Duration,
) -> Result(Event(o), ReadError)

Wait at most wait for the next event. TimedOut leaves the stream readable.

prepare

</>

pub fn prepare(
  config: Config,
  request: Request(o),
) -> Result(Prepared(o), error.PrepareError)

Admit and encode request for config, without I/O.

request

</>

pub fn request(
  model: String,
  messages: List(message.Message),
) -> Request(String)

A plain request: its Answer.output is the final text.

request_json

</>

pub fn request_json(prepared: Prepared(o)) -> String

The JSON body prepared sends, for logs and tests. It never contains headers or credentials.

run

</>

pub fn run(
  client: http_gun.Client,
  prepared: Prepared(o),
) -> Result(Outcome(o), Failure)

Execute the call and wait for its outcome.

stream

</>

pub fn stream(
  client: http_gun.Client,
  prepared: Prepared(o),
) -> Result(Stream(o), Failure)

Start the call. The calling process owns the stream: if it exits, the call is cancelled and its HTTP stream released.

system

</>

pub fn system(text: String) -> message.Message

tool_result

</>

pub fn tool_result(
  call: message.ToolCall,
  content: String,
) -> message.Message

The result of call, for the request after a NeedsTools outcome.

tools

</>

pub fn tools(request: Request(o)) -> List(tool.Tool)

user

</>

pub fn user(text: String) -> message.Message

with_call_timeout

</>

pub fn with_call_timeout(config: Config, bound: Bound) -> Config

Bound the whole call, from the start of execution to the final event. Default After(duration.seconds(600)).

with_endpoint

</>

pub fn with_endpoint(config: Config, endpoint: String) -> Config

Send to another base URL, such as a compatible server or a proxy. The endpoint is checked by prepare.

with_first_token_timeout

</>

pub fn with_first_token_timeout(
  config: Config,
  bound: Bound,
) -> Config

Bound the wait from the start of execution to the first progress event, covering admission, the response head and hidden reasoning. Default After(duration.seconds(180)).

with_generation

</>

pub fn with_generation(
  request: Request(o),
  settings: generation.Settings,
) -> Request(o)

Replace all generation controls while preserving model, messages, tools and output. Later scalar setters override only their named control. Applying generation.default clears explicit controls. Validation happens in prepare, where the provider is known.

with_google_cached_content

</>

pub fn with_google_cached_content(
  request: Request(o),
  name: String,
) -> Request(o)

Set Google’s cachedContent reference. Other built-in providers refuse it in llm_wire.prepare.

with_idle_timeout

</>

pub fn with_idle_timeout(config: Config, bound: Bound) -> Config

Bound the gap between provider events after the first progress event. Every event resets it, tool-argument deltas and pings included. Default After(duration.seconds(60)).

with_limit

</>

pub fn with_limit(
  config: Config,
  which: limit.Limit,
  value: Int,
) -> Config

Set one byte or count bound; see llm_wire/limit for every bound and its default.

with_max_tokens

</>

pub fn with_max_tokens(
  request: Request(o),
  max: Int,
) -> Request(o)

Set the maximum output-token count; preparation requires a positive value.

with_openai_prompt_cache_key

</>

pub fn with_openai_prompt_cache_key(
  request: Request(o),
  key: String,
) -> Request(o)

Set OpenAI’s prompt_cache_key. Other built-in providers refuse it in llm_wire.prepare.

with_output

</>

pub fn with_output(
  request: Request(a),
  name: String,
  output: codec.Codec(o),
) -> Request(o)

Ask for structured output matching codec’s schema, named name. The answer is validated and decoded; invalid output fails the call with error.InvalidOutput. The codec must be a record at the root. OpenAI and Anthropic take a strict schema (every field required; no pair, number range or codec.value()); Gemini takes all of these too. prepare refuses what the provider cannot take with error.UnsupportedSchema.

with_stop_sequences

</>

pub fn with_stop_sequences(
  request: Request(o),
  sequences: List(String),
) -> Request(o)

Set stop sequences; provider-specific admission remains in preparation.

with_temperature

</>

pub fn with_temperature(
  request: Request(o),
  temperature: Float,
) -> Request(o)

Set sampling temperature; preparation accepts values from zero to two.

with_tool_call_checks

</>

pub fn with_tool_call_checks(
  config: Config,
  checks: tool.ToolCallChecks,
) -> Config

Choose whether a call to an unknown tool or with invalid arguments fails the response (the default) or is reported in NeedsTools.issues.

with_tools

</>

pub fn with_tools(
  request: Request(o),
  tools: List(tool.Tool),
) -> Request(o)

with_top_p

</>

pub fn with_top_p(
  request: Request(o),
  top_p: Float,
) -> Request(o)

Set nucleus sampling probability; preparation accepts zero through one.