Projects and deep research#

A project is a folder holding links, documents, data, scripts and typed items — and a research project turns a question into a cited report.

Contents#

☑ Overview#

☑ Scope#

note

packages/core/src/{projects,project-templates}.ts, the Projects panel, and the deep-research pipeline seeded into research projects.

☑ Projects are folders, not records#

decision

Everything a project holds is a plain file under projects/<id>/. That means the scripts explorer already shows it, Ohm scripts can read and write it, git can track it, and export is a convenience rather than the only way out.

  • spec: 12-storage

☑ Storage#

☑ Folder layout#

data-model

projects/<id>/
├─ ohm.project.json      { name, kind, description?, question?, createdAt }
├─ links.json            ProjectLink[]
├─ docs/  data/  scripts/  files/  annotations/
├─ prompts/*.prompt.json
├─ curves/*.curve.json
├─ experiments/*.experiment.json   (+ <name>.runs/)
├─ graphs/*.graph.json
└─ specs/*.spec.json

kind is general or research. path is synthesised on read, never stored.

☑ Names and ids are validated at one choke point#

requirement #security

  • Project id: /^[a-z0-9][a-z0-9-]{0,63}$/, produced by slugifying the name (NFKD, non-alphanumerics to dashes, 48 chars) and appending -2, -3… until unique. Directories that do not match are skipped when listing.
  • Item name: /^[\w .()-]{1,80}$/ with a leading dot rejected — so no separator and no .. can reach a path.
  • Every user-supplied relative path goes through one inside(id, sub, rel) helper that resolves and requires the result to stay under the subfolder, throwing Path is outside <sub>/.

There is no other way to build a project path, and that is the property to preserve.

requirement #regression

Every write goes to <file>.<pid>.tmp and is renamed into place. Link mutations run through a promise queue, because the research pipeline adds links from several concurrent tasks — twenty concurrent adds of ten distinct URLs must end with ten links.

Links dedupe on the normalised URL and only fill in a missing title or note; an invalid URL is rejected.

  • node: research-flow

☑ Version snapshots#

requirement

Saving an item first snapshots the previous content into <item>.versions/<ISO with : and . replaced by ->.json, but only when the file already existed and the serialised content actually changed — so a new item has no versions and a no-op save adds nothing. The newest 50 are kept.

Listing returns { id, at, size } newest-first, reconstructing the timestamp from the file name. Restore is client-side: the editor loads the snapshot and leaves the document dirty so the user decides.

This applies uniformly to prompts, curves, experiments, graphs and specs.

  • node: risk-versions-dir

☑ Deep research#

☑ The pipeline#

flow

A research project is seeded with scripts/research.ohm and an equivalent graph. The script:

  1. Takes the question from ctx.args or the project manifest; refuses to run without one.
  2. \SelfAsk_{\Psi}(question; n=3) — decompose into sub-questions.
  3. Per sub-question, concurrently (ohm.all): \Search_{\Gamma}(sub; k=2) → save every hit as a project link immediately with source = "research: <sub>" → \Fetch_{\Gamma}(url) under pcall, page text truncated to 20000 chars, falling back to the snippet → \Retrieve_{\Gamma}(sub; pages, k=6) → \CoT_{\Psi}(sub; C).
  4. \Join_{\Sigma}(notes; question) then \Cite_{\Gamma}(draft; context).
  5. project.writeDoc("report.md", …) with the question, the report and the per-sub-question findings, then print the report and the link count.

Search runs in a hidden, non-activating tab that is closed afterwards, which is the whole reason hidden tabs exist.

  • spec: 01-engine-and-shell
  • spec: 02-ohm-language

☑ Sources are saved as they are found#

requirement

Links are written during the search step, not at the end, so a run that fails half-way still leaves the user with the sources it had already found.

  • node: research-flow

☑ Templates#

component

Every new project gets docs/README.md, a sample dataset, a summarize prompt, a temperature curve (warmup-cosine), a summarize-sweep experiment and a plan spec. Research projects additionally get research.ohm and the research graph, and their spec uses the Research plan template.

  • spec: 10-specs
  • spec: 09-graphs

☑ Review findings#

☐ The versions directory is derived by a loose string replace#

risk high

The suffix is stripped with String.replace over the whole absolute path, removing the first occurrence anywhere. Two consequences: items named x and x.spec share one history, and a scripts root whose own path contained .spec.json would put the versions directory outside the item's folder. Slice from the end instead.

  • node: versions

☐ Deleting an item orphans its history#

risk

deleteItem removes the item file but leaves <item>.versions/ behind, invisible to listings. Recreating an item with the same name silently inherits the old history.

  • node: versions

☐ Item names allow Windows-hostile spellings#

risk

The name pattern permits trailing spaces and dots and the reserved device names (con, nul, aux), which fail at the filesystem with an opaque error on Windows. Reject them by name.

  • node: path-safety

☐ Snapshots can collide within a millisecond#

risk low

Two saves in the same millisecond produce the same snapshot file name and the second overwrites the first. Add a counter or a short random suffix.

  • node: versions

☐ Rename is unreachable from the UI#

note

saveItem supports renaming — refusing to clobber an existing target, deleting the old file and moving the versions folder — but the shared editor hook always sends previousName: undefined, so the path is only exercised by direct RPC. It is also non-atomic: the old file is deleted before the versions folder is moved, with no rollback.

graph LR
  writes_are_atomic_and_links_are_serialised["Writes are atomic and links are serialised"] -.-> research_flow["The pipeline"]
  versions["Version snapshots"] -.-> risk_versions_dir["The versions directory is derived by a loose string replace"]
  sources_are_saved_as_they_are_found["Sources are saved as they are found"] -.-> research_flow["The pipeline"]
  risk_versions_dir["The versions directory is derived by a loose string replace"] -.-> versions["Version snapshots"]
  deleting_an_item_orphans_its_history["Deleting an item orphans its history"] -.-> versions["Version snapshots"]
  item_names_allow_windows_hostile_spellings["Item names allow Windows-hostile spellings"] -.-> path_safety["Names and ids are validated at one choke point"]
  snapshots_can_collide_within_a_millisecond["Snapshots can collide within a millisecond"] -.-> versions["Version snapshots"]