title: "The Ohm language" description: "Lua 5.4 interleaved with LaTeX math, compiled line-for-line to Lua and run in a sandboxed VM with an async host bridge." nodes: 40 progress: 80% generated: ohm spec editor#

The Ohm language#

Lua 5.4 interleaved with LaTeX math, compiled line-for-line to Lua and run in a sandboxed VM with an async host bridge.

Contents#

• Overview
  • Scope
  • Math is an expression language, not a document format
  • Line-for-line compilation
• Splitting Lua from math
  • The scanner
  • Math regions have their own skip rules
• The LaTeX subset
  • Lexical rules
  • Operators and precedence
  • Subscripts and superscripts are overloaded
  • Commands
  • Errors name the fix
• Compilation
  • Pipeline
  • Two-tier error recovery
  • Generated code
• Runtime
  • The VM
  • Sandbox
  • The async scheduler
  • Errors carry the script line
  • Output is streamed
  • A runaway script can always be stopped
  • Memory
• Host API
  • Surface
  • Matrices
  • Chatting to a page
  • Qualitative coding
  • Operator families
• Review findings
  • ~= silently compiles to ==
  • No exponent notation, and % is a comment
  • JSON null corrupts an array
  • ohm.host and debug are reachable from scripts
  • A stale host result can kill the next run
  • A failed session is cached forever
  • Chained comparison silently displays
  • Smaller observations

☑ Overview#

☑ Scope#

note

packages/ohm-lang (splitter, lexer, parser, AST, code generator, operator registry) and packages/ohm-runtime (wasmoon session, prelude, scheduler, worker runner).

☑ Math is an expression language, not a document format#

decision

$…$ is exactly one expression; $$…$$ is a sequence of statements. Everything outside them is literal Lua, copied through untouched.

The point is that notation people already read — \sum, \forall, \circ, \frac — becomes executable, and that AI operators get a compact spelling (\CoT_{\Psi}(q; C)) instead of a nest of function calls.

☑ Line-for-line compilation#

requirement #regression

Source line N compiles to Lua line N. The writer counts newlines it emits and pads forward to the source line before each statement, each math segment and each block body; padding never moves backwards, so no construct may emit more newlines than its source span occupies.

This is the only source map: the chunk is loaded as "=" .. scriptName, so a Lua runtime error reports script.ohm:12: and the editor can jump to it directly.

☑ Splitting Lua from math#

☑ The scanner#

flow

A source is split into a flat list of segments — lua, inline, display — each recording both the delimiter span and the content span.

In Lua regions the scanner must skip, so that a $ inside them stays literal: -- line comments, --[[ ]] / --[==[ ]==] long comments, "…" and '…' strings (with backslash escapes, and an unescaped newline reported as Unfinished string), and [[ ]] / [==[ ]==] long strings.

☑ Math regions have their own skip rules#

requirement

Inside a math region the scanner skips \x escape pairs (so \$, \{, \} never terminate), jumps over { … } with a brace matcher, skips "…" strings, and treats % as a comment to end of line.

The brace matcher is what makes nesting work: a { … } body is Ohm again, so $…$ may appear inside it, and the matcher applies the Lua trivia rules inside the braces as well.

Inline math additionally ends at a blank line and then reports Unclosed $ inline math at the opening $; display math scans to EOF, so a stray $$ swallows the rest of the file before diagnosing.

• node: splitter

☑ The LaTeX subset#

☑ Lexical rules#

data-model

• Whitespace: space, tab, \r, and also ~ and & (LaTeX spacing/alignment).
• \n is a significant token: it separates statements. \\ is also a separator.
• % starts a comment.
• Commands are \ + letters, or \ + one non-letter.
• Layout commands are dropped: \, \; \: \! \ \quad \qquad \left \right \big… \displaystyle \textstyle \limits \nolimits \middle.
• Numbers are digits with an optional single decimal point — no exponent, no hex, no leading dot.
• Identifiers are [A-Za-z][A-Za-z0-9]* — no underscore, because _ is the subscript operator; use \mathrm{my_var}.
• Strings are "…" with \n \t \r \" \\ \' escapes.

☑ Operators and precedence#

data-model

^ is handled outside the table by the power rule, and is non-associative: a^b^c is a parse error, braces are required.

Unary - binds a power, so -x^2 is -(x^2). Logical not (\neg, !) binds looser than comparison, so \neg x = 1 is not (x == 1).

Implicit multiplication fires before a number, identifier, primary command, or a ( when the left side is not callable — 2x, 3(x+1), \pi r^2. It is suppressed after a lambda or a string.

A newline continues the previous expression only when the next line starts with an infix operator that is not - (otherwise a statement beginning with a negative number would be swallowed).

☑ Subscripts and superscripts are overloaded#

data-model

_ means five different things, decided by what it follows:

^ is exponentiation except as a big-operator upper bound or in \for_{i=1}^{n}.

☑ Commands#

api

Values: \frac \dfrac \tfrac, \sqrt{x} and \sqrt[n]{x}, \text{…} (and \textrm/\textit/\textbf/\texttt) for strings, \mathrm{…}/\mathit/\mathbf/\mathsf/\var for identifiers that may contain _, \operatorname{Name}, \mathcal{M} (memory), \Gamma (ctx), \Theta (params), \Psi/\Sigma/\Lambda (families), \pi, \infty, \emptyset, \top/\bot/\nil, \lfloor…\rfloor, \lceil…\rceil, \sum, \prod, \max_, \min_, \forall, \lua{…}, \{a, b\} lists and \{a,\dots,b\} ranges, \mapsto lambdas, \gets/\leftarrow as synonyms for :=.

Math functions: sin cos tan arcsin arccos arctan sinh cosh tanh exp ln log lg max min gcd sqrt abs sgn, callable as \sin x, \sin(x, y) or \log_{2}(8).

Greek letters compile to plain identifiers of the same name.

Statements: \while, \if/\elseif/\else, \for, \forall, \return, \lua, and the no-ops \usepackage, \newoperator, \documentclass.

☑ Errors name the fix#

requirement

Unknown commands are rejected by name. Lua keywords used as math identifiers come with the LaTeX spelling: and → "(use \land)", or → "(use \lor)", not → "(use \neg)". Also rejected: assignment to a non-assignable left side, a lambda with a non-name parameter, \mathcal{X} for anything but M, mismatched \if branches, and a \while/\elseif with no condition group.

☑ Compilation#

☑ Pipeline#

flow

1. SourceFile precomputes line starts for offset → line/column.
2. declaredOperators pre-scans the raw text for \newoperator{X} and defop("F","X", so an operator can be used before it is declared.
3. Split into segments; copy Lua verbatim; parse each math segment.
4. Emit, padding to the source line before each statement and after each segment.
5. Return { ok, lua, diagnostics } with ok === (diagnostics.length === 0).

• node: line-preservation

☑ Two-tier error recovery#

requirement #regression

A splitter error (unclosed math, unfinished string) aborts the whole compile and returns empty Lua — the segment boundaries are no longer trustworthy.

A parser error is caught per segment: it becomes a diagnostic, inline math emits the literal nil, display math emits nothing, and compilation continues. Every broken region is therefore reported in one pass, not just the first.

☑ Generated code#

data-model

Inline math becomes (expr). The shapes worth pinning:

Big operators are IIFEs accumulating into __acc:

(function() local __acc = 0; for i = 1, n do __acc = __acc + ((i ^ 2)) end; return __acc end)()

Statements map to the obvious Lua, each suffixed with "; ". A bare expression statement becomes ohm.display(expr); a bare call does not — so a value written on its own line is shown, while f(x) stays silent.

☑ Runtime#

☑ The VM#

component

Lua 5.4 through wasmoon (Lua compiled to WebAssembly), one engine per session, with injectObjects: false and enableProxy: false — no live object proxies cross the boundary, only JSON strings.

Globals persist across runs in the same session, which is what makes the console a REPL.

☑ Sandbox#

requirement #security

The prelude removes io, dofile, loadfile, package and require, and trims os to { time, clock, date, difftime }. Filesystem access is only through the host fs API, which is confined to the scripts folder.

Still reachable: load, debug, string, table, math, utf8, coroutine — see the review findings.

• node: risk-debug

☑ The async scheduler#

flow

Host calls are asynchronous, but Lua code reads as if they were not. The mechanism:

1. Four JS globals are injected and then captured and niled by the prelude, so scripts cannot reach them: __host_call, __host_wait, __host_out, __init_json.
2. Every task is a coroutine. hcall(name, …) JSON-encodes its arguments, calls __host_call to get an id, and yields (WAIT, id).
3. The scheduler drains the ready queue, then blocks on __host_wait(), decodes the envelope {id, ok, value|error} and reschedules the matching task.
4. ohm.spawn / ohm.join / ohm.all build on the same queues; \parallel compiles to ohm.par, which is genuinely concurrent (a test asserts peak concurrency 3).
5. Deadlock is an error, not a hang: an empty waiting set with unfinished tasks raises "all tasks are blocked waiting for each other".

Both queues are reset at the start of every run, so tasks from a previous run are abandoned.

☑ Errors carry the script line#

requirement #regression

Four cooperating pieces:

• where() walks stack levels and returns the first frame that is not the prelude chunk — so the prefix names the user's line, not the prelude's.
• annotate adds a chunk:line: prefix from the coroutine traceback when the message has none.
• cleanError strips wasmoon's [string "…"]:N: wrapper and the trailing stack traceback:….
• errorLine extracts the number so the editor can jump to it.

The prelude must be loaded as chunk =ohm-prelude and the script as "=" .. name for any of this to work.

• node: line-preservation

☑ Output is streamed#

requirement

Five streams: out (print), err/warn/info (log.*), and result (ohm.display, and the script's return value). Lines flow prelude → __host_out → worker message → scripts.output event as they happen, not buffered to the end.

☑ A runaway script can always be stopped#

requirement #safety

Each run owns a worker thread. stop() clears the worker reference first, settles the pending run as { ok: false, stopped: true }, then terminates the thread — so the exit handler does not re-settle. Termination kills even a pure CPU loop (while true do end), which no cooperative scheme could.

The cost is that all session state is discarded; the next run starts a fresh worker. One run at a time per session.

• spec: 01-engine-and-shell

☑ Memory#

data-model

\mathcal{M} is a persistent table. It is re-read after every run, including a failed one, and written back only when it changed.

☑ Host API#

☑ Surface#

api

Every host object is wired so that obj.fn(x) and obj:fn(x) both work.

• browser: open(url, opts), tabs(), active(), get(id)

• tab: go text html info url title eval query queryAll click type press waitFor scroll screenshot show hide activate close

• ai: generate(prompt, opts), json(prompt, opts) (strips a code fence, then falls back to brace matching before decoding), profiles(), saveProfile(), useProfile()

• fs: read write list exists — confined to the scripts folder

• project, prompts, curves, experiments, specs, models

• mem, ctx ({script, project, args, tab}), params, json, print, log, sleep

• ohm: spawn join all sleep defop inspect text show display host plus the codegen helpers.

• spec: 04-models-and-settings

• spec: 08-projects-and-research

☑ Matrices#

data-model

A list is a matrix, and arithmetic is overloaded over it the way + already was.

• \Tab is a builtin marker for the tab opener, so \Tab \times urls reads as multiplication and opens one tab per url, hidden, all at once. Ten urls should not take over the window; \Tabs_{\Gamma}(urls; hidden=false) shows them.
• M \times f maps f across the matrix concurrently; n \times M repeats it; numbers still multiply.
• <- sends the right side into the left: a tab is asked (typed into and read back), a list of tabs gets one prompt each (cycled), a list gains an item.

Codegen routes * through ohm.mul and <- through ohm.send, so the overloading lives in the runtime rather than the parser.

Structure operators, all \Sigma and all pure: Map Shuffle Rotate Reverse Transpose Zip Chunk Pick Flatten Unique Sort. Shuffle and Pick take a seed, so a randomised run repeats.

• spec: 01-engine-and-shell — hidden tabs

☑ Chatting to a page#

flow

tab <- prompt (and \Chat_{\Lambda}) finds the largest visible text input, marks it with a data attribute, types, presses Enter, then polls the page text until it stops growing for two consecutive samples, and returns what was appended.

It is a heuristic by nature — that is the price of working on any chat site without a per-site adapter — so it fails loudly ("no text input found on …") rather than silently returning nothing, and timeout, step and settle are configurable.

• node: matrices

☑ Qualitative coding#

api

\OpenCode_{\Psi}(corpus; k) → codes with definitions, verbatim quotes and source indices; \AxialCode_{\Psi}(codes) → categories with typed relations (causal, context, strategy, consequence); \SelectiveCode_{\Psi}(axial; question) → the core category and storyline; \Codebook_{\Psi}(codes) → a markdown table.

All four ask for JSON and parse it, and share a system prompt that insists on the participants' own words and forbids invented material.

☑ Operator families#

api

Thirty built-in operators in five families:

• Ψ reasoning — Ask CoT ToT GoT SelfAsk Plan Reflect Critique Summarize
• Γ context — Retrieve Search Fetch Rank Filter Compress MergeContext Ground Cite
• ℳ memory — MemRead MemWrite MemAppend MemSearch MemSummarize MemForget
• Σ aggregate — Join Vote Debate
• Λ agent — ReAct Verify Fork

Config is read as cfg.name or cfg[n], so \CoT(q; C) and \CoT(q; context=C) are equivalent. ohm.defop(family, name, fn) adds one from a script, and the pre-scan makes it usable anywhere in the same file.

ohm.compose applies rather than composes when its right operand is not callable, which is what makes \Summarize_{\Psi} \circ \CoT_{\Psi}(q; C) work.

• spec: 09-graphs

☑ Review findings#

☐ `~=` silently compiles to `==`#

risk urgent #correctness

The lexer treats ~ as whitespace (LaTeX spacing) before consulting the symbol table, so the ~= entry can never fire: a ~= b lexes as a, =, b and compiles to (a == b).

A silent inversion of meaning, and the worst defect found in the language. != and \neq are unaffected. Fix: match ~= before the whitespace rule, or reject a bare ~ followed by =.

☐ No exponent notation, and `%` is a comment#

risk high #correctness

$1e3$ lexes as 1 then the identifier e3 and compiles to (1 * e3) — a runtime nil-arithmetic error with no compile-time diagnostic. .5 fails to parse.

$a % b$ silently discards the rest of the line, because % is a LaTeX comment; \bmod is the intended spelling but nothing says so.

Both deserve a diagnostic at minimum.

• node: lexing

☐ JSON null corrupts an array#

risk high #correctness

decode maps null to nil while still advancing the index, so [1,null,2] becomes {[1]=1,[3]=2}; the length probe then fails and encode re-emits it as the object {"1":1,"3":2}. Only the trailing-null case is tested.

☐ `ohm.host` and `debug` are reachable from scripts#

risk high #security

ohm.host(name, …) is a public global that calls any host function by name, bypassing the typed wrappers, and the debug library is left in place because the prelude itself needs it. Together they widen the sandbox well past what the fs confinement implies.

Decide explicitly: either keep them and document the trust model as "scripts are trusted code", or make the prelude capture debug the way it captures the host globals.

• node: sandbox

☐ A stale host result can kill the next run#

risk

The Lua queues are reset per run, but the JS-side completed queue is not. A result delivered after its run ended is dropped by the scheduler, which may then see an empty waiting set and raise "all tasks are blocked waiting for each other" — a deadlock error whose real cause was a message from a previous run.

• node: scheduler

☐ A failed session is cached forever#

risk

Both the worker and the in-process runner use session ??= create(…); a rejected promise is retained, so every later run fails with the same stale error until the worker is replaced. The same pattern bites the embedded llama.cpp loader.

• spec: 04-models-and-settings

☐ Chained comparison silently displays#

risk

a = b = c parses as ((a==b)==c), fails the assignability test, and falls through to an expression statement — so it prints instead of reporting an error.

☐ Smaller observations#

note low

• a^b^c is a parse error (braces required) — intended, but the message ("Expected end of statement") does not say so.
• |…| cannot nest: | |x| - |y| | mis-parses with no diagnostic.
• .. means range inside math and concatenation in the surrounding Lua — same token, two meanings, untested.
• Operator calls used as statements print their result, because only Call and raw Lua are exempt from the display rule.
• declaredOperators scans comments and strings, so a defop mentioned in a comment whitelists the name.
• \Psi.CoT(x) cannot be parsed although the runtime proxy supports it.
• The editor's HOST_API table omits project, prompts, curves and experiments, so completion under-reports the real surface.
• InProcessRunner.stop() never settles the pending run and has no already-running guard; it is test-only, but the asymmetry with the worker runner is a trap.

Links between nodes#

graph LR
  math_regions_have_their_own_skip_rules["Math regions have their own skip rules"] -.-> splitter["The scanner"]
  pipeline["Pipeline"] -.-> line_preservation["Line-for-line compilation"]
  sandbox["Sandbox"] -.-> risk_debug["`ohm.host` and `debug` are reachable from scripts"]
  errors_carry_the_script_line["Errors carry the script line"] -.-> line_preservation["Line-for-line compilation"]
  chatting_to_a_page["Chatting to a page"] -.-> matrices["Matrices"]
  no_exponent_notation_and_is_a_comment["No exponent notation, and `%` is a comment"] -.-> lexing["Lexical rules"]
  risk_debug["`ohm.host` and `debug` are reachable from scripts"] -.-> sandbox["Sandbox"]
  a_stale_host_result_can_kill_the_next_run["A stale host result can kill the next run"] -.-> scheduler["The async scheduler"]