Models and settings#

Everything that generates text goes through a named model profile. Settings (⚙) is where you create them, store credentials, choose the models folder and download weights.

Model profiles#

A profile is a name plus a provider and its configuration. The name is what you select in the UI and what scripts ask for:

print(ai.generate("Summarise this page", { profile = "Claude" }))
Provider Use it for Key fields
anthropic Claude through the official SDK model, maxTokens, system
openai anything OpenAI-compatible: OpenAI, LM Studio, vLLM, the llama.cpp server, OpenRouter, Groq baseUrl, model
ollama a local Ollama baseUrl (default http://127.0.0.1:11434), model
llamacpp GGUF files inside Ohm, no server model (a GGUF path), gpuLayers, contextSize
web transformers.js in the browser on WebGPU / WASM / WebNN model (a Hub repo or local folder), device, dtype

One profile is active; scripts can override it per call, and ai.useProfile("name") switches it.

Two provider notes worth knowing:

  • Temperature is not sent for the newest Claude models, which reject it.
  • Generation is request/response everywhere: there is no token streaming and no way to cancel a generation in flight.

Credentials#

API keys and your Hugging Face token are encrypted by the operating system — DPAPI on Windows, Keychain on macOS, libsecret on Linux — and stored separately from settings.

  • The UI can write a key or clear it, never read it back; it only sees "a key is set".
  • Scripts can create and edit profiles but there is no function that reads or writes a credential.
  • settings.json never contains a key.
  • If the OS offers no encryption, Ohm refuses to store keys rather than writing them in plaintext, and says so in Settings.

Environment variables work as a fallback: ANTHROPIC_API_KEY, OPENAI_API_KEY, HF_TOKEN.

One caveat to be aware of: a profile's endpoint is editable, and the key is filed under the profile name. A script you import can therefore point a profile that owns a key at a different server. Treat imported scripts like any other code you run.

The models folder#

Default <userData>/models, changeable in Settings (or OHM_MODELS_DIR for the first run). Downloads mirror the Hub layout:

<modelsDir>/<owner>/<repo>/<file path>

Ohm recognises two kinds automatically: a *.gguf file, and an ONNX model folder (one containing config.json plus an onnx/ directory or *.onnx files).

From the list you can create a profile in one click — GGUF becomes a llamacpp profile, ONNX becomes a web profile — or delete a model.

Downloading from Hugging Face#

Search the Hub, filter by gguf / onnx / transformers.js, pick files and download.

  • Downloads resume: partial files are kept as .part and continued with a range request; a server that ignores the range restarts the file correctly instead of corrupting it.
  • Files already present at the right size are skipped.
  • Gated or private repos work once a token is set — a refusal says so explicitly rather than failing with a bare 403.
  • Progress is live, and a download can be cancelled.

The same thing from a script:

local files = models.files("Qwen/Qwen2.5-0.5B-Instruct-GGUF")
models.download("Qwen/Qwen2.5-0.5B-Instruct-GGUF", { "qwen2.5-0.5b-instruct-q4_k_m.gguf" })

models/download-gguf.ohm and models/download-web-model.ohm in your scripts folder do the whole job — find the right file, download it, create the profile and run a test prompt.

Embedded llama.cpp#

A llamacpp profile runs GGUF weights in-process through node-llama-cpp, with prebuilt CPU binaries (and Vulkan or CUDA if the installer was built with them).

  • gpuLayers takes auto, max or a number.
  • One model is loaded at a time, and requests are serialised.
  • The model is unloaded automatically when you change the models folder or delete a model.

If the native binary is missing, the error appears when you first use the profile — note that until the app is restarted, that failure is remembered, so fix the install and restart.

Web models#

A web profile runs in the renderer, in a worker, on WebGPU by default with wasm and webnn as alternatives. Weights load from your local models folder first (ohm-models://) and fall back to the Hub. The ONNX runtime is bundled — nothing is fetched from a CDN.

If WebGPU is unavailable, Ohm says so and suggests wasm rather than failing with a library error.

Settings that are not models#

  • Folders — the data, browser-profile, scripts and models folders, each movable from Settings → Folders or with a flag; see Storage.
  • Scripts import/export and project bundles — see Storage.
  • Graph component profiles — see Graphs.

For the full data model, the threat model and the known gaps, see the models spec.