title: "Document viewers and annotations" description: "Reading PDFs, Word documents, spreadsheets, slide decks, images and text inside the app, and annotating them in place." nodes: 32 progress: 75% generated: ohm spec editor#

Document viewers and annotations#

Reading PDFs, Word documents, spreadsheets, slide decks, images and text inside the app, and annotating them in place.

Contents#

• Overview
  • Scope
  • Read the sources where the research happens
  • Format detection is by extension
• Viewers
  • PDF (pdf.js)
  • Word (mammoth)
  • Spreadsheets (SheetJS)
  • Slides (.pptx, parsed from OOXML)
  • Image and text
  • Code annotations
  • An anchor knows when it has drifted
  • What was deliberately not ported
  • Tasks and nesting
  • The region overlay
• Annotations
  • Record
  • Anchors and their coordinate systems
  • Resolution on reopen
  • Autosave and the sidebar
• Storage and serving
  • On disk
  • Getting the bytes
• Review findings
  • The text anchor occurrence is computed wrongly
  • Selecting text creates an annotation
  • Spreadsheet cell refs can disagree with the workbook
  • Unsaved annotation text is lost on a fast switch
  • A corrupt sidecar is silently replaced
  • pptx gaps: notes, groups and the image budget
  • Binary files are rendered as text
  • Smaller observations

☑ Overview#

☑ Scope#

note

packages/docview (pure parsing, no DOM), apps/ui/src/views/DocumentViewer.tsx (all rendering and interaction), the files.*/annotations.* protocol methods and their storage in packages/core/src/projects.ts.

☑ Read the sources where the research happens#

decision

A research project's PDFs, spreadsheets and decks are part of the project, so they are read in the app and annotated in place, with the notes stored beside the documents rather than in a separate tool. Nothing is uploaded and no external viewer is launched.

• spec: 08-projects-and-research

☑ Format detection is by extension#

requirement

docKind(path) lowercases and takes the substring after the last dot:

No sniffing, no MIME. other currently falls through to the text viewer — see the findings.

☑ Viewers#

☑ PDF (pdf.js)#

component #regression

Page navigation, zoom in 0.2 steps clamped to 0.4…4, a canvas render and pdf.js's official TextLayer over it for selection. Annotate by selecting text or by dragging a box.

Two things are load-bearing and were each a bug once:

1. One canvas serves every page and zoom, so the in-flight RenderTask must be cancelled before the next render(), and RenderingCancelledException / AbortException must be swallowed rather than surfaced. Without this, paging or zooming throws "Cannot use the same canvas during multiple render() operations" and leaves a stale page on screen.
2. The text layer needs four CSS variables. --total-scale-factor and --scale-factor are set imperatively per render; --scale-round-x/--scale-round-y: 1px must come from CSS because pdf.js sizes the layer with round(down, …, var(--scale-round-x)) and its own viewer defines them on .page. Without them the layer collapses to zero width, every percentage-positioned glyph lands at the origin, and selection silently stops working. The viewer stylesheet's inset: 0 must also be overridden with inset: auto; top: 0; left: 0 so the layer is page-sized, not container-sized.

• node: anchors

☑ Word (mammoth)#

component

convertToHtml gives headings, lists, tables and images — not page layout, and the toolbar says so. Selecting a passage anchors an annotation to the quoted text.

☑ Spreadsheets (SheetJS)#

component

One button per sheet; formatted strings (raw: false), blank rows dropped, rows capped at 5000 and padded to the widest row. Sticky column letters and row numbers. Clicking a cell annotates it.

• node: risk-cell-ref

☑ Slides (.pptx, parsed from OOXML)#

component

JSZip plus a namespace-stripping XML parser. Deck size from presentation.xml (default 960×540), slides discovered by regex and sorted numerically, shapes and pictures read from the shape tree.

• EMU → px is exactly ÷9525 (96 dpi), rounded. Slide anchors are meaningless in any other space.
• Text: paragraphs with level, and bold/size/color taken from the first run; sz is hundredths of a point, rot sixtieths of a thousandth of a degree.
• Empty paragraphs are dropped, and a shape with no remaining text is skipped.
• Images are resolved through the slide's relationships, with a 30 MB budget.
• Elements are sorted top-to-bottom then left-to-right.

The UI renders positioned divs on a scaled canvas, with a thumbnail strip and speaker notes. Annotate by dragging a box.

• node: risk-pptx

☑ Image and text#

component

Images render at natural size with a region layer and no scale factor at all. Text is fetched and truncated to 2,000,000 characters, and opens in the Code view; Prose gives the old quote-annotation behaviour and, for .ohm, Math typesets the LaTeX.

• node: code-annotations
• spec: 03-scripts

☑ Code annotations#

component

A source file is annotated through a capture mode rather than only by line: cursor, line, selection, word, statement, inside/with brackets, function / class / if / while / for body, variable, declaration, assignment, call, whole file.

The engine is packages/annotate — pure functions, no DOM — ported from the 6co editor in Storyteller. It uses light language knowledge (bracket matching, block keywords, literal stripping) rather than a parser, so it still works on a file that does not compile. Block and identifier modes are supported for Lua/Ohm, JavaScript, TypeScript, Java, C, C++, Go, C# and Rust; everything else gets the positional modes and is told so.

The view previews the capture before it is committed — position, word, line count, or the reason the mode does not apply — and tints the lines that would be taken.

☑ An anchor knows when it has drifted#

requirement #regression

A code anchor stores { line, endLine?, col?, endCol?, mode, word, text } — the captured text, not just a position. checkAnchor re-reads the file and reports one of four states:

The comparison is against the first line of the capture, because that is what line means; comparing against the whole span would call a multi-line anchor "fine" while its start had slid.

The UI draws the highlight where the text actually is, lists every unhealthy anchor under the file, and offers Follow to line N — which is a click, never automatic. This is the gap in the engine this was ported from: there, the line never moved and the word was silently re-read from whatever now sat at that position, so a note quietly began describing different code.

• node: code-annotations

☑ What was deliberately not ported#

decision

The source engine also executed a JavaScript snippet per annotation through new Function, with network and DOM access, from data that round-trips through a server. Ohm already has a scripting language with a sandbox, a worker and a stop button — if annotations should run code, they should run Ohm, not arbitrary JS.

Also left out: the 3D scene, the media/stick-figure/Chinese-glyph attachments and the per-type shared code, all of which are tied to that app's document model.

Ported and fixed: an out-of-range line no longer throws and takes every highlight with it; "first annotation on a line" is computed in one index space instead of two; autoNest reads the fields the annotations actually carry; and there is one reference shape, not two with the same name.

• spec: 02-ohm-language

☑ Tasks and nesting#

data-model

An annotation may carry taskStatus (todo / doing / blocked / review / done / cancelled) and dependencies — ids of annotations that must be finished first. dependencyStatus reports { met, blockers, missing, total }.

autoNest builds a tree by range containment, smallest containing range wins, so a note on a function body owns the notes inside it; nestedOrder flattens that back to rows with depths, and items with a broken parent link are still shown.

☑ The region overlay#

component

Shared by PDF, slides and images. Pointer-based dragging normalised so any direction works, coordinates always element-local, and a 6 px minimum in both axes before an annotation is created — the only thing standing between a stray click and an empty annotation.

☑ Annotations#

☑ Record#

data-model

DocAnnotation {
  id, type, anchor, text,
  quote?, color?, status?: 'open'|'resolved', tags?, refs?: SpecRef[],
  author?, createdAt, updatedAt?
}
DocAnnotations { version: 1, file, annotations: DocAnnotation[] }

Six types, each with a default colour: comment, highlight, to do, question, issue (bug), spec note. refs is the same link type specs use, so a document note can point at a requirement.

• spec: 10-specs

☑ Anchors and their coordinate systems#

data-model #regression

Every stored rectangle is scale-free: divided by the zoom on write, multiplied on read. Changing zoom must never rewrite an anchor.

Page and slide indices are 0-based on disk and 1-based in the UI (and in pdf.js).

• node: pdf

☑ Resolution on reopen#

flow

Geometric anchors (pdf, slide, image) are filtered by page/slide and re-projected by the current scale. Cell anchors are looked up per rendered cell by sheet name and A1 ref. Text anchors are re-found by walking the rendered DOM for the quote and wrapping the hit in a <mark>; if the quote is gone or spans elements, the annotation keeps existing in the sidebar with no visual anchor.

☑ Autosave and the sidebar#

requirement

Every edit marks the document dirty and restarts a 1200 ms debounce that saves; the header button doubles as the state indicator (Save / Saved).

The sidebar offers a whole-file note, per-card type switching (which also resets the colour), resolve/reopen, delete, the quoted text, a note field and tags (committed on blur). A filter switches between all and open-only — note that the filtered list is what the viewers draw, so resolved annotations disappear from the document surface under "open only".

• node: risk-unsaved

☑ Storage and serving#

☑ On disk#

data-model

Documents live in projects/<id>/files/<rel>; annotations in projects/<id>/annotations/<rel>.annots.json — the document's full relative path plus the suffix, so sub/deck.pptx maps to annotations/sub/deck.pptx.annots.json.

Server-side rules: version and file are always rewritten from the request path (clients are not trusted for them); the list is truncated to 5000; writes are atomic; import never overwrites (a collision becomes name (2).ext); deleting a document deletes its sidecar; and the per-file annotation count is derived at list time, never cached, so it stays correct after a bundle restore or a hand edit.

• spec: 12-storage

☑ Getting the bytes#

flow #gotcha

files.url first: the Electron composition root returns ohm-file://local/<percent-encoded path>, which the scheme handler serves from the scripts root with a containment check. If no engine supplies a URL, the viewer falls back to files.read → base64 → a blob URL, which is what lets the UI work outside the shell.

The scheme must be registered { standard, secure, supportFetchAPI, corsEnabled, stream }: standard+secure make it an origin the CSP can name, supportFetchAPI+corsEnabled let pdf.js/mammoth/SheetJS fetch it, and stream lets pdf.js range-read.

The production CSP therefore needs ohm-file: in both connect-src and img-src, plus style-src 'unsafe-inline' because the pdf.js text layer writes inline styles on every span.

• spec: 01-engine-and-shell

☑ Review findings#

☐ The text anchor occurrence is computed wrongly#

risk high #correctness

On creation the Word viewer counts total occurrences in the document minus one rather than which occurrence was selected — select the first of three identical phrases and it stores occurrence: 2. The resolver then counts something else again: it increments once per text node containing the quote, not once per occurrence. The two only agree when the quote appears once. The text viewer sidesteps it by never writing occurrence, so it always highlights the first match.

Fix: compute the index of the selected match by walking the same text nodes the resolver walks, and count occurrences the same way on both sides.

• node: anchors

☐ Selecting text creates an annotation#

risk high

Word, text and PDF bind onMouseUp to the create handler as well as offering an explicit "Annotate selection" button, so merely selecting text to copy it silently creates an empty annotation and marks the document dirty. Keep the button, drop the implicit binding.

☐ Spreadsheet cell refs can disagree with the workbook#

risk high #correctness

Blank rows are dropped at parse time and the A1 ref is derived from the rendered grid position, so for a sheet with blank rows or a used range that does not start at A1, the stored ref does not name the cell the user would click in Excel. The parsed origin is captured and never used. The 5000-row cap silently hides annotations beyond it.

• node: sheets

☐ Unsaved annotation text is lost on a fast switch#

risk high

Autosave is a 1200 ms debounce with no flush on unmount, and switching files unmounts the viewer. Type a note and switch within a second and it is gone. Flush on unmount.

• node: autosave

☐ A corrupt sidecar is silently replaced#

risk high #data-loss

Reading annotations catches all errors — including a JSON parse failure and a permission error — and returns an empty set. The UI then treats "no annotations" as the truth and the first edit autosaves over the damaged but possibly recoverable file. Distinguish "missing" from "unreadable", and refuse to autosave over the latter.

• node: annots-storage

☐ pptx gaps: notes, groups and the image budget#

risk

• Speaker notes are matched positionally (notesSlide{i+1}) rather than through the slide's relationships, so a deck where only some slides have notes attaches them to the wrong slides. The relationships are already parsed.

• Only top-level shapes and pictures are walked: groups, tables and charts are dropped entirely, so those slides render blank while region annotations float over nothing. Shapes that inherit geometry from a layout or master collapse to the top-left corner with fallback sizes.

• The image budget adds a picture's bytes to the running total before deciding to skip it and never rolls back, so one oversized image poisons the budget and every later picture is dropped.

• Picture blobs are created with no MIME type, so SVG and EMF/WMF media do not render although the extension is right there in the zip path.

• node: slides

☐ Binary files are rendered as text#

risk

docKind returns other for .zip, .exe and friends, and the render ladder's final branch is the text viewer — so up to 2 MB of binary is decoded and dumped into a <pre>. SUPPORTED_DOCS exists to gate exactly this and is referenced nowhere. Offer a download/open-externally card instead.

• node: detection

☐ Smaller observations#

note low

• The base64-fallback object URL leaks when the component unmounts before files.read resolves.
• The 5000-annotation cap truncates on save with no error, so memory and disk diverge until reload.
• mammoth's HTML is injected with dangerouslySetInnerHTML and no sanitiser; the output tag set is constrained, but the trust boundary is a user-supplied .docx, so this deserves an explicit decision.
• DocAnchor.start/end, DocAnnotation.author and refs are declared but never produced or consumed.
• The UI re-implements A1 column naming instead of using the tested helper exported by @ohm/docview.
• The PDF render effect omits url from its dependencies; it is safe only because the component is keyed by path and remounts.

Links between nodes#

graph LR
  pdf["PDF (pdf.js)"] -.-> anchors["Anchors and their coordinate systems"]
  sheets["Spreadsheets (SheetJS)"] -.-> risk_cell_ref["Spreadsheet cell refs can disagree with the workbook"]
  slides["Slides (.pptx, parsed from OOXML)"] -.-> risk_pptx["pptx gaps: notes, groups and the image budget"]
  image_and_text["Image and text"] -.-> code_annotations["Code annotations"]
  drift["An anchor knows when it has drifted"] -.-> code_annotations["Code annotations"]
  anchors["Anchors and their coordinate systems"] -.-> pdf["PDF (pdf.js)"]
  autosave["Autosave and the sidebar"] -.-> risk_unsaved["Unsaved annotation text is lost on a fast switch"]
  the_text_anchor_occurrence_is_computed_wrongly["The text anchor occurrence is computed wrongly"] -.-> anchors["Anchors and their coordinate systems"]
  risk_cell_ref["Spreadsheet cell refs can disagree with the workbook"] -.-> sheets["Spreadsheets (SheetJS)"]
  risk_unsaved["Unsaved annotation text is lost on a fast switch"] -.-> autosave["Autosave and the sidebar"]
  a_corrupt_sidecar_is_silently_replaced["A corrupt sidecar is silently replaced"] -.-> annots_storage["On disk"]
  risk_pptx["pptx gaps: notes, groups and the image budget"] -.-> slides["Slides (.pptx, parsed from OOXML)"]
  binary_files_are_rendered_as_text["Binary files are rendered as text"] -.-> detection["Format detection is by extension"]