Class: LLM::Ollama

Inherits:

Provider

• Object
• Provider
• LLM::Ollama

Includes:

Format

Defined in:

lib/llm/providers/ollama.rb,
lib/llm/providers/ollama/format.rb,
lib/llm/providers/ollama/models.rb,
lib/llm/providers/ollama/error_handler.rb,
lib/llm/providers/ollama/response_parser.rb

Overview

The Ollama class implements a provider for [Ollama](ollama.ai/).

This provider supports a wide range of models, it is relatively straight forward to run on your own hardware, and includes multi-modal models that can process images and text. See the example for a demonstration of a multi-modal model by the name ‘llava`

Examples:

#!/usr/bin/env ruby
require "llm"

llm = LLM.ollama(nil)
bot = LLM::Chat.new(llm, model: "llava").lazy
bot.chat LLM::File("/images/capybara.png")
bot.chat "Describe the image"
bot.messages.select(&:assistant?).each { print "[#{_1.role}]", _1.content, "\n" }

Defined Under Namespace

Modules: Format, ResponseParser Classes: ErrorHandler, Models

Constant Summary

HOST =

"localhost"

Instance Method Summary

• #assistant_role ⇒ String

  Returns the role of the assistant in the conversation.

• #complete(prompt, role = :user, model: default_model, schema: nil, **params) ⇒ LLM::Response::Completion

  Provides an interface to the chat completions API.

• #default_model ⇒ String

  Returns the default model for chat completions.

• #embed(input, model: "llama3.2", **params) ⇒ LLM::Response::Embedding

  Provides an embedding.

• #initialize(secret) ⇒ Ollama constructor

  A new instance of Ollama.

• #models ⇒ LLM::Ollama::Models

  Provides an interface to Ollama’s models API.

Methods included from Format

#format

Methods inherited from Provider

#audio, #chat, #chat!, #files, #images, #inspect, #respond, #respond!, #responses, #schema

Constructor Details

#initialize(secret) ⇒ Ollama

Returns a new instance of Ollama.

Parameters:

• secret (String) —

  The secret key for authentication

# File 'lib/llm/providers/ollama.rb', line 32

def initialize(secret, **)
  super(secret, host: HOST, port: 11434, ssl: false, **)
end

Instance Method Details

#assistant_role ⇒ String

Returns the role of the assistant in the conversation. Usually “assistant” or “model”

Returns:

• (String) —

  Returns the role of the assistant in the conversation. Usually “assistant” or “model”

# File 'lib/llm/providers/ollama.rb', line 86

def assistant_role
  "assistant"
end

#complete(prompt, role = :user, model: default_model, schema: nil, **params) ⇒ LLM::Response::Completion

Provides an interface to the chat completions API

Examples:

llm = LLM.openai(ENV["KEY"])
messages = [
  {role: "system", content: "Your task is to answer all of my questions"},
  {role: "system", content: "Your answers should be short and concise"},
]
res = llm.complete("Hello. What is the answer to 5 + 2 ?", :user, messages:)
print "[#{res.choices[0].role}]", res.choices[0].content, "\n"

Parameters:

• prompt (String) —

  The input prompt to be completed

• role (Symbol) (defaults to: :user) —

  The role of the prompt (e.g. :user, :system)

• model (String) (defaults to: default_model) —

  The model to use for the completion

• params (Hash) —

  Other completion parameters

Returns:

• (LLM::Response::Completion)

Raises:

• (LLM::Error::PromptError) —

  When given an object a provider does not understand

• (LLM::Error::Unauthorized) —

  When authentication fails

• (LLM::Error::RateLimit) —

  When the rate limit is exceeded

• (LLM::Error::ResponseError) —

  When any other unsuccessful status code is returned

• (SystemCallError) —

  When there is a network error at the operating system level

See Also:

• Ollama docs

# File 'lib/llm/providers/ollama.rb', line 63

def complete(prompt, role = :user, model: default_model, schema: nil, **params)
  params = {model:, stream: false}
             .merge!(expand_schema(schema))
             .merge!(params)
             .compact
  req = Net::HTTP::Post.new("/api/chat", headers)
  messages = [*(params.delete(:messages) || []), LLM::Message.new(role, prompt)]
  body = JSON.dump({messages: format(messages)}.merge!(params))
  set_body_stream(req, StringIO.new(body))
  res = request(@http, req)
  Response::Completion.new(res).extend(response_parser)
end

#default_model ⇒ String

Returns the default model for chat completions

Returns:

• (String)

See Also:

• llama3.2

# File 'lib/llm/providers/ollama.rb', line 94

def default_model
  "llama3.2"
end

#embed(input, model: "llama3.2", **params) ⇒ LLM::Response::Embedding

Provides an embedding

Parameters:

• input (String, Array<String>) —

  The input to embed

• model (String) (defaults to: "llama3.2") —

  The embedding model to use

• params (Hash) —

  Other embedding parameters

Returns:

• (LLM::Response::Embedding)

Raises:

• (LLM::Error::Unauthorized) —

  When authentication fails

• (LLM::Error::RateLimit) —

  When the rate limit is exceeded

• (LLM::Error::ResponseError) —

  When any other unsuccessful status code is returned

• (SystemCallError) —

  When there is a network error at the operating system level

# File 'lib/llm/providers/ollama.rb', line 43

def embed(input, model: "llama3.2", **params)
  params   = {model:}.merge!(params)
  req      = Net::HTTP::Post.new("/v1/embeddings", headers)
  req.body = JSON.dump({input:}.merge!(params))
  res      = request(@http, req)
  Response::Embedding.new(res).extend(response_parser)
end

#models ⇒ LLM::Ollama::Models

Provides an interface to Ollama’s models API

Returns:

• (LLM::Ollama::Models)

See Also:

• Ollama docs

# File 'lib/llm/providers/ollama.rb', line 80

def models
  LLM::Ollama::Models.new(self)
end