Class: LLM::Client

Inherits:
Object
  • Object
show all
Defined in:
lib/llm/client.rb

Overview

Convenience layer over Providers::Anthropic for sending messages and handling tool execution loops.

Examples:

registry = Tools::Registry.new
registry.register(Tools::WebGet)
client.chat_with_tools(messages, registry: registry, session_id: session.id)

Constant Summary collapse

INTERRUPT_MESSAGE =

Synthetic tool_result when a tool is skipped because the human pressed Escape.

"Your human wants your attention"

Instance Attribute Summary collapse

Instance Method Summary collapse

Constructor Details

#initialize(model: Anima::Settings.model, max_tokens: Anima::Settings.max_tokens, provider: nil, logger: nil) ⇒ Client

Returns a new instance of Client.

Parameters:

  • model (String) (defaults to: Anima::Settings.model) —

    Anthropic model identifier (default from Settings)

  • max_tokens (Integer) (defaults to: Anima::Settings.max_tokens) —

    maximum tokens in the response (default from Settings)

  • provider (Providers::Anthropic, nil) (defaults to: nil) —

    injectable provider instance; defaults to a new Providers::Anthropic using credentials

  • logger (Logger, nil) (defaults to: nil) —

    optional logger for tool call tracing



29
30
31
32
33
34
 
# File 'lib/llm/client.rb', line 29

def initialize(model: Anima::Settings.model, max_tokens: Anima::Settings.max_tokens, provider: nil, logger: nil)
  @provider = build_provider(provider)
  @model = model
  @max_tokens = max_tokens
  @logger = logger
end

Instance Attribute Details

#max_tokens ⇒ Integer (readonly)

Returns maximum tokens in the response.

Returns:

  • (Integer) —

    maximum tokens in the response



22
23
24
 
# File 'lib/llm/client.rb', line 22

def max_tokens
  @max_tokens
end

#model ⇒ String (readonly)

Returns the model identifier used for API calls.

Returns:

  • (String) —

    the model identifier used for API calls



19
20
21
 
# File 'lib/llm/client.rb', line 19

def model
  @model
end

#provider ⇒ Providers::Anthropic (readonly)

Returns the underlying API provider.

Returns:



16
17
18
 
# File 'lib/llm/client.rb', line 16

def provider
  @provider
end

Instance Method Details

#chat_with_tools(messages, registry:, session_id:, first_response: nil, between_rounds: nil, **options) ⇒ Hash?

Send messages with tool support. Runs the full tool execution loop: call LLM, execute any requested tools, feed results back, repeat until the LLM produces a final text response.

Emits Events::ToolCall and Events::ToolResponse events for each tool interaction so they’re persisted and visible in the event stream.

When the user interrupts via Escape, remaining tools receive synthetic “Your human wants your attention” results and the loop exits without another LLM call.

Parameters:

  • messages (Array<Hash>) —

    conversation messages in Anthropic format

  • registry (Tools::Registry) —

    registered tools to make available

  • session_id (Integer, String) —

    session ID for emitted events

  • first_response (Hash, nil) (defaults to: nil) —

    pre-fetched first API response from AgentLoop#deliver!. Skips the first API call when provided so the Bounce Back transaction doesn’t duplicate work.

  • between_rounds (#call, nil) (defaults to: nil) —

    callback invoked after each tool round completes, before the next LLM request. Must return an Array<String> of message contents to inject (e.g. promoted pending messages). Injected as text blocks alongside tool_result blocks so the LLM sees them in the next round.

  • options (Hash) —

    additional API parameters (e.g. system:)

Returns:

  • (Hash, nil) —

    :text (String) and :api_metrics (Hash), or nil when interrupted

Raises:



60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
 
# File 'lib/llm/client.rb', line 60

def chat_with_tools(messages, registry:, session_id:, first_response: nil, between_rounds: nil, **options)
  messages = messages.dup
  rounds = 0
  last_api_metrics = nil

  loop do
    rounds += 1
    max_rounds = Anima::Settings.max_tool_rounds
    if rounds > max_rounds
      return {text: "[Tool loop exceeded #{max_rounds} rounds — halting]", api_metrics: last_api_metrics}
    end

    response = if first_response && rounds == 1
      first_response
    else
      broadcast_session_state(session_id, "llm_generating")
      provider.create_message(
        model: model,
        messages: messages,
        max_tokens: max_tokens,
        tools: registry.schemas,
        include_metrics: true,
        **options
      )
    end

    # Capture api_metrics from ApiResponse wrapper (nil for pre-fetched first_response)
    last_api_metrics = response.api_metrics if response.respond_to?(:api_metrics)

    log(:debug, "stop_reason=#{response["stop_reason"]} content_types=#{(response["content"] || []).map { |b| b["type"] }.join(",")}")

    if response["stop_reason"] == "tool_use"
      tool_results = execute_tools(response, registry, session_id)
      promoted = promote_between_rounds(between_rounds)

      # Dual injection: user messages go as text blocks within the current
      # tool_results turn (same speaker); sub-agent messages append as
      # separate assistant→user turn pairs (distinct tool invocations).
      promoted[:texts].each { |text| tool_results << {type: "text", text: text} }

      messages += [
        {role: "assistant", content: response["content"]},
        {role: "user", content: tool_results}
      ]

      messages.concat(promoted[:pairs])

      return nil if handle_interrupt!(session_id)
    else
      # Discard the text response if the user pressed Escape while
      # the API was generating it. Without this check the interrupt
      # flag set during the blocking API call would be silently
      # cleared by the ensure block in AgentRequestJob.
      return nil if handle_interrupt!(session_id)

      return {text: extract_text(response), api_metrics: last_api_metrics}
    end
  end
end