WAT-native WinHelp design

Status: implementation in progress. Phase 1's WAT-owned file buffer, bounded-reader foundation, directory B+tree parser, and raw VFS load path are implemented. Phase 2 now parses |SYSTEM, canonical |TTLBTREE topics, |CONTEXT, |CTXOMAP, and Hall |PhrIndex/|PhrImage phrase tables, including two-level B+trees, bounded LZ77 expansion, and referential validation. It also validates the complete |TOPIC link chain, binds canonical topics to their type-2 records, and decodes phrase-expanded raw LinkData2 streams. Legacy |Phrases tables are supported in HC30, HC31, and MVB forms. The bounded topic-IR layer now interleaves every non-empty, NUL-delimited text string with typed PARAGRAPH, FONT, SPACE, LINE_BREAK, BITMAP, HOTSPOT, and MACRO tokens and a terminal END_TOPIC. Formatted LinkData1 is fully bounds-walked—including compressed values, tables, paragraph metrics, tabs, fonts, pictures, hotspots, and macros—before any document is published. Each character command must pair with exactly one LinkData2 string. Stable copies of validated records and variable command payloads live in a separate caller-owned arena, never in the reusable TOPIC-block scratch buffer. The Phase 3 command engine now owns viewer session state and dispatches loaded documents transactionally: Contents/Index, Context, ContextPopup, Finder, exact Key, PartialKey prefix selection, SetContents, and owner-scoped Quit have explicit success/failure semantics. Unknown and not-yet-parsed structured commands return false without changing the visible topic. ANSI/Unicode ABI normalization now feeds the same engine. The existing help window owns exact formatted-token/payload arenas plus positioned text/space/bitmap runs, and paints visible WAT-laid-out text and embedded rasters without reparsing while scrolling. Each referenced normalized font descriptor is now realized as a WAT-owned HFONT inside the same view transaction. Layout and paint select the same face, negative character height, weight, and italic state; positioned runs retain underline/strikeout decoration bits, and replacement/Quit deletes every dynamic font after removing it from the target DC. Paragraph layout now consumes retained binary headers directly: metric conversion, margins, first-line/continuation indents, spacing, explicit line height, right/center alignment, typed tabs, and fixed/variable table cells are all WAT-owned. Asynchronous browser VFS mounting remains in progress. Fixed hotspot semantics now distinguish direct-topic E0/E1 commands from the context-hash E2/E3/E6/E7 family, with even opcodes routed to popups and odd opcodes to the main viewer. EA/EB/EE/EF structures accept only exact bounded type 0/1/4/6 payloads. The current document retains its canonical VFS path; relative type-4 filenames load mounted sibling HLP/CNT files through WAT, while failed loads or unresolved hashes restore the source document, session, view, and history. Normal external navigation and external popups suspend up to four documents for Back/dismissal. |SYSTEM type-6 records normalize into a bounded window table, so numeric type-1 and named type-6 selectors resolve against the document that owns the target topic; an unknown number or name fails explicitly without changing topic, history, or presentation. The |FONT face/descriptor table and standalone |bmN lP/lp picture headers, palettes, compressed payload slices, and hotspot slices are also normalized into bounded WAT-owned records. Picture payloads now decode through all four WinHelp packing modes into preflighted caller-owned buffers. External bmc/bml/bmr commands resolve to normalized resource indexes; referenced DDB/DIB pixels and palettes are copied transactionally into WAT-owned GDI objects, laid out at intrinsic dimensions, painted through canonical BitBlt, and released on replacement or Quit. Inline picture unions and metafiles remain explicit placeholders. Paired |KWBTREE and |KWDATA files now publish a case-folded default keyword index with canonical topic postings and explicit |Rose macro sentinels. The obsolete JavaScript HlpParser production runtime and the semantic help_open, help_get_topic, and help_get_title WASM imports have been removed. Browser and CLI production paths now expose only raw VFS bytes and presentation primitives to the WAT-owned parser/session/viewer.

This document defines the replacement for the current split WinHelp path. The target implementation parses HLP and CNT data, interprets WinHelpA/W, owns navigation state, and renders the user interface in WAT. JavaScript remains a host for file availability and presentation primitives; it does not interpret help formats or choose topics.

The archived Windows 98 winhlp32.exe under test/binaries/help/ is a reference oracle only. It is not a production dependency and must not be added to the web application or deployment manifest.

Decision

All WinHelp semantics belong in WAT:

• HLP internal-file directory and B+tree traversal;
• phrase decompression and topic record decoding;
• title, context, keyword, browse, font, bitmap, and macro metadata;
• CNT hierarchy and the in-memory search/index model;
• WinHelpA/W command dispatch and return values;
• document, window, history, selection, focus, and scroll state;
• topic layout, hotspot hit testing, and standard WinHelp UI behavior.

The host may:

• make raw files available through the VFS, including asynchronous browser fetch followed by a normal WAT resume;
• expose the existing low-level GDI/canvas presentation boundary;
• upload WAT-owned dirty rectangles and composite windows.

The host must not:

• parse HLP, CNT, or GID structures;
• return preselected or flattened help text;
• resolve a context ID, keyword, browse link, or contents node;
• own WinHelp history, selection, layout, or UI state.

flowchart LR
    G[Guest application] -->|WinHelpA/W| W[WAT WinHelp subsystem]
    W --> A[WAT API command dispatcher]
    A --> D[WAT document/session model]
    D --> P[WAT HLP/CNT parsers]
    D --> U[WAT windows, controls, layout]
    P -->|raw byte reads| V[VFS]
    V -->|fetch/mount only| J[JavaScript host]
    U -->|GDI primitives / dirty surface| R[Renderer host]

    X[Archived winhlp32.exe] -. test-only oracle .-> T[Differential tests]
    T -. verifies .-> A
    T -. verifies .-> P
    T -. verifies .-> U

Why replace the current implementation

The current path is useful as a smoke test but is not a WinHelp implementation:

WinHelpA in 09a-handlers.wat
        |
        | ignores almost every uCommand and dwData
        v
host.help_open in lib/host-imports.js
        |
        | HlpParser parses a small subset and returns flattened text
        v
help_wndproc in 09c-help.wat
        |
        +-- fixed 400x300 window
        +-- plain lines of text
        +-- synthetic [Contents] and [Back] links

Specific gaps:

• WinHelpA handles HELP_QUIT but otherwise treats commands alike.
• dwData, caller ownership, new filenames, context IDs, keywords, and popups are ignored.
• WinHelpW returns success without doing any work.
• lib/hlp-parser.js guesses topic boundaries and titles and produces visibly corrupted text for several checked-in fixtures.
• CNT, context maps, keyword indexes, browse chains, formatting, hotspots, macros, and embedded images are not part of the runtime model.
• JavaScript caches one parser independent of pathname, while WAT separately owns the visible help window and history.
• Missing or malformed files can still lead to an empty window and a success return.

The replacement must remove this split ownership instead of extending it.

Goals

1. Make F1 and context-sensitive Help open the requested topic in the checked-in Win98 applications.
2. Match Windows 98 topic, Contents, Index/Find, Back, popup, and close behavior closely enough for applications to rely on it.
3. Keep parsing deterministic, bounded, and testable without Canvas or a browser.
4. Reuse the existing WAT window/control/GDI systems rather than creating a second renderer or DOM UI.
5. Support ANSI and Unicode API entry points through one command engine.
6. Use the archived viewer and v86 captures for behavior and pixel oracles without shipping Microsoft binaries.
7. Fail cleanly on unsupported or malformed data; never silently select an unrelated topic.

Non-goals for the first complete slice

• Bit-for-bit compatibility with every WinHelp macro.
• Executing arbitrary external commands embedded in help files.
• Persisting or consuming Windows-generated GID files.
• Full-text ranking identical to the native viewer.
• Supporting WinHelp 3.x variants before the Win98 fixture set works.
• Replacing host file fetch or final Canvas composition with WAT.

GID is a generated cache, not source content. The first WAT implementation builds its own in-memory indexes from HLP and CNT data. A compatible GID cache can be added later if startup time justifies it.

Ownership boundary

flowchart TB
    subgraph Guest[Guest-visible Win32 boundary]
        WA[WinHelpA]
        WW[WinHelpW]
    end

    subgraph WAT[Canonical WAT ownership]
        C[Command normalization]
        S[HelpSession]
        HD[HelpDocument]
        HP[HLP parser]
        CP[CNT parser]
        IX[Context / keyword / contents indexes]
        LY[Topic token stream and layout]
        UI[WinHelp windows and controls]
    end

    subgraph Host[JavaScript host boundary]
        FS[Raw VFS fetch/mount]
        GDI[Primitive presentation imports]
    end

    WA --> C
    WW --> C
    C --> S
    S --> HD
    HD --> HP
    HD --> CP
    HP --> IX
    CP --> IX
    IX --> LY
    LY --> UI
    HP --> FS
    CP --> FS
    UI --> GDI

The canonical copy of loaded help bytes resides in WASM memory. Parsed structures store offsets into that buffer or WAT-owned heap pointers. A host object must never be required to interpret or resume a topic operation.

Public API behavior

Both entry points normalize into one internal call:

help_dispatch(
    caller_hwnd,
    path_wa,          ;; normalized ANSI in WAT memory, or zero
    command,
    data,
    source_is_wide
) -> BOOL

WinHelpW converts the UTF-16 pathname and any command-specific string or structure into bounded WAT-owned ANSI/UTF-8-compatible bytes before calling the same engine. There must be no separate Unicode behavior stub.

Command coverage

Unknown commands return FALSE and set a diagnostic status. They must not fall through to Contents while claiming success.

Return and lifecycle rules

• Return TRUE only after the request is accepted, including an asynchronous load that has a valid continuation.
• Return FALSE for missing paths, malformed files, unresolved context IDs, unsupported command data, or allocation failure.
• A new non-null pathname may replace the active document; it must never reuse an unrelated cached parser.
• A null pathname reuses the caller's active document only where native command semantics permit it.
• Caller HWND is retained as owner state. Destruction and HELP_QUIT must not close an unrelated application's help session.
• Repeated requests focus or navigate the existing matching window instead of creating duplicates.

Request and asynchronous-load state machine

The current yield reason 4=help_load can remain, but the continuation becomes WAT-owned. The host only finishes mounting bytes into the VFS.

stateDiagram-v2
    [*] --> Idle
    Idle --> ResolvePath: WinHelpA/W
    ResolvePath --> ParseDocument: bytes already in VFS
    ResolvePath --> WaitingForFile: browser asset not mounted
    ResolvePath --> Failed: missing or invalid path
    WaitingForFile --> ParseDocument: VFS mount completed
    WaitingForFile --> Failed: fetch failed
    ParseDocument --> Ready: required indexes valid
    ParseDocument --> Failed: malformed / unsupported core format
    Ready --> Navigate: dispatch command + data
    Navigate --> Visible: topic/dialog/popup painted
    Visible --> Navigate: another WinHelp request or UI action
    Visible --> Closing: HELP_QUIT / WM_CLOSE / owner teardown
    Closing --> Idle: last reference released
    Failed --> Idle: return FALSE and clear continuation

The pending request record contains the normalized path, caller, command, command data copy, and API return continuation. It must not retain raw guest pointers across a yield because the caller can mutate or free them.

Long term, WinHelp should use normal VFS CreateFile/ReadFile machinery. A temporary raw-byte import is acceptable during migration only if it has no HLP semantics:

vfs_request_mount(path_wa) -> READY | PENDING | NOT_FOUND

After resume, WAT opens and reads the mounted file. The existing help_open, help_get_title, and help_get_topic semantic imports are removed at the end of migration.

HLP parsing pipeline

Parsing is bottom-up and each layer consumes bounded slices rather than naked pointers.

flowchart TD
    B[Raw HLP byte buffer] --> H[Validate file header and size]
    H --> D[Parse internal-file directory B+tree]
    D --> SY[|SYSTEM]
    D --> PH[|PhrIndex + |PhrImage or |Phrases]
    D --> TO[|TOPIC]
    D --> TT[|TTLBTREE]
    D --> CX[|CONTEXT + |CTXOMAP]
    D --> KW[Keyword B+trees / data]
    D --> FO[|FONT]
    D --> BM[|bmN embedded resources]

    PH --> TD[Topic decoder]
    TO --> TD
    TT --> TI[Canonical topic index]
    CX --> TI
    KW --> KI[Keyword index]
    FO --> IR[Formatted topic IR]
    BM --> IR
    TD --> IR
    TI --> IR

Bounded reader contract

Every parser function receives a HelpSlice:

HelpSlice
  +0  base_wa       i32   start of complete HLP buffer
  +4  file_size     i32
  +8  offset        i32   slice offset from base
  +12 length        i32

Primitive readers return success separately from the value, using a shared result global or an out pointer:

help_read_u8(slice, relative_offset, out)
help_read_u16le(slice, relative_offset, out)
help_read_u32le(slice, relative_offset, out)
help_subslice(slice, relative_offset, length, out_slice)
help_read_cstring(slice, relative_offset, max_length, out_string)

Required invariants:

• validate offset <= file_size before addition;
• validate length <= file_size - offset to avoid wraparound;
• validate multiplication before page/table size calculations;
• never scan for NUL outside the current slice;
• cap B+tree depth, page count, entry count, and recursion;
• reject cyclic page links and overlapping records where the format forbids them;
• cap phrase count and total decompressed bytes;
• retain the first parse error code and source offset for diagnostics;
• do not publish partially initialized indexes.

Directory and internal files

The internal-file directory maps names such as |TOPIC to HLP file offsets. The WAT parser builds a compact sorted array:

HelpInternalFile[entry_count]
  +0  name_hash     i32
  +4  name_off      i32   offset into HLP buffer
  +8  name_len      i16
  +10 flags         i16
  +12 data_off      i32   after internal-file header
  +16 data_len      i32

Lookup verifies both hash and bytes. Hash collision must never select the wrong internal file.

Phrase decompression

Phrase tables are parsed once into {offset,length} entries. Topic decoding streams decompressed bytes into the topic-token builder; it does not allocate one unbounded copy of the entire decompressed HLP.

Both Hall |PhrIndex/|PhrImage and older |Phrases forms are separate decoders behind one interface. Tests must include malformed bit streams, truncated phrase images, maximum-length phrases, and references to missing entries.

Canonical topic identity

The current flat 1..N topic index is removed. A canonical topic reference is the validated logical topic position used by the HLP tables:

HelpTopic
  +0  topic_ref         i32
  +4  topic_record_off  i32
  +8  title_off         i32
  +12 title_len         i32
  +16 context_hash      i32
  +20 browse_prev_ref   i32
  +24 browse_next_ref   i32
  +28 flags             i32

All navigation sources resolve to topic_ref:

numeric context ID ─┐
context hash ───────┤
keyword result ─────┼──> topic_ref --> decode/layout/display
CNT leaf ───────────┤
hotspot jump ───────┤
browse/back entry ──┘

|TTLBTREE supplies titles and topic positions. The first line of decoded body text is not treated as a title unless the format genuinely lacks title metadata.

CNT and GID policy

CNT is parsed in WAT as a line-oriented source file. It supplies hierarchy and links; it is not flattened into numbered text.

HelpContentsNode
  +0  parent_index    i32   -1 for root
  +4  first_child     i32   -1 if none
  +8  next_sibling    i32   -1 if none
  +12 depth           i16
  +14 flags           i16   book, leaf, expanded, unresolved
  +16 title_ptr       i32
  +20 title_len       i32
  +24 topic_ref       i32   0 until resolved
  +28 context_string  i32   optional source link

The visual icon is derived from node state, not copied from the source:

has children + collapsed  -> closed book
has children + expanded   -> open book
no children + topic       -> topic/page icon
unresolved target         -> disabled topic icon or explicit parse failure

GID is treated as an optional generated cache. Phase one ignores it and builds the following indexes in memory:

• context ID to topic reference;
• context hash to topic reference;
• normalized keyword to one or more topic references;
• contents-node hierarchy;
• optional full-text token postings for Find.

This avoids version/staleness problems and keeps clean test runs independent of a machine-generated file. A future GID reader/writer must be an optimization with identical results, never the only path to content.

The default K-footnote index is represented by 16-byte keyword records {text_off, text_len, first_posting, posting_count} and 8-byte posting records {topic_ref, flags}. Keyword strings remain bounded offsets into the immutable HLP image; comparisons fold ASCII case and retain non-ASCII bytes exactly. Every non-macro posting must resolve to the canonical HelpTopic table before publication. A topic_ref of -1 is retained only with the macro flag for later |Rose interpretation, and exact/prefix resolution skips such sentinels.

Document and session memory model

All structures are heap-backed. Do not reserve another fixed low-memory table; the existing low-memory map is already dense.

flowchart LR
    HS[HelpSession] --> HD[HelpDocument]
    HS --> HW[Window state]
    HS --> BK[Back/forward stacks]
    HS --> PR[Pending request]

    HD --> FB[Raw HLP buffer]
    HD --> DR[Internal-file directory]
    HD --> PT[Phrase table]
    HD --> TI[Topic index]
    HD --> CI[Context indexes]
    HD --> KI[Keyword index]
    HD --> CT[CNT nodes]
    HD --> FT[Font/resource metadata]

    HW --> LY[Current layout]
    LY --> TK[Topic token arena]
    LY --> HR[Hotspot rectangles]

HelpSession

One emulated process owns a bounded set of sessions keyed by caller/application and pathname. The initial implementation may cap this at four live documents and one primary window plus popups per document.

Conceptual fields:

HelpSession
  state
  owner_hwnd
  document_ptr
  topic_hwnd
  topics_dialog_hwnd
  popup_hwnd
  current_topic_ref
  contents_selection
  active_tab
  scroll_x / scroll_y
  back_stack_ptr / count / capacity
  forward_stack_ptr / count / capacity
  pending_request_ptr
  last_error / last_error_offset

Document arena

Each HelpDocument owns an arena chain. Parser indexes, copied strings, topic tokens, and CNT nodes allocate from that arena. Closing the last session drops the chain in one operation. Transient layout and search result arenas can be reset without reparsing the document.

The original raw HLP buffer remains immutable until document teardown. Indexes prefer offsets into it over duplicated bytes.

Formatted topic intermediate representation

Flattening a topic to text loses the information required for layout, hotspots, images, and accurate navigation. The topic decoder emits a bounded WAT-owned token stream:

Each token is a 16-byte record {kind, payload_off, payload_len, value}. TEXT offsets address the caller's exact decoded LinkData2 arena. Structured offsets address a second caller-owned arena containing one exact copy of every referenced LinkData1 record. A PARAGRAPH token's offset/length identifies the exact direct paragraph-header slice, while value stores the record type in its high byte and the complete record's payload offset in its low 24 bits. The direct slice avoids rescanning intervening character commands in a multi-paragraph table, while the shared record offset keeps column geometry available without duplicating common table bytes. FONT stores the parsed font index directly. SPACE and LINE_BREAK distinguish their source command in value. Bitmap, hotspot, and macro tokens reference the exact validated command subrange, so later resource resolution and the safe macro interpreter never depend on transient parser memory.

Font faces are stored as bounded offsets into the immutable HLP buffer. Font descriptors normalize face index, height, family, style attributes, weight, foreground, and background while retaining whether metrics use half-points or twips. Bitmap records normalize resource/picture number, picture and packing type, dimensions, depth, resolution, palette, transparency, compressed data, hotspot slices, and decoded-size metadata. Every offset/length pair is checked against its containing internal file before publication. The payload decoder supports raw, RLE, LZ77, and LZ77-then-RLE streams. It derives exact WORD- or DWORD-aligned raster sizes with i64 arithmetic, caps both final and intermediate expansion, rejects output aliases into document-owned storage, and validates a whole stream without writes before filling the caller's buffer.

flowchart LR
    TR[HLP topic records] --> DC[Bounded decoder]
    DC --> TS[Topic token stream]
    TS --> LM[Line measurement]
    LM --> LN[Positioned lines/runs]
    LN --> PA[WAT GDI paint]
    LN --> HT[Hotspot hit regions]
    HT --> NV[Navigation dispatcher]

Layout is deterministic and integer-based:

1. Resolve logical fonts through the existing WAT font/GDI system.
2. Decode each retained paragraph header into pixel margins, spacing, alignment, tabs, and optional table-cell bounds.
3. Measure tokens inside the resulting first-line and continuation bounds.
4. Break lines at explicit and legal soft breaks, then align complete lines.
5. Emit positioned runs and hotspot rectangles.
6. Paint only visible runs using the top-level window back-canvas.
7. Re-layout on width/font changes; scrolling does not reparse the topic.

Installed FNT and scalable TrueType faces stay on the canonical WAT font path; the topic model and positions remain WAT-owned.

UI architecture

The implementation uses the existing WAT window table, controls, non-client painting, menu system, and top-level back-canvas.

Main help window
┌──────────────────────────────────────────────────────┐
│ caption / menu                                      │
├──────────────────────────────────────────────────────┤
│ [Help Topics] [Back] [Options]       command strip  │
├──────────────────────────────────────────────────────┤
│                                                      │
│ formatted topic viewport                             │
│ text, bitmaps, hotspots, vertical/horizontal scroll │
│                                                      │
└──────────────────────────────────────────────────────┘

Help Topics dialog
┌───────────────────────────────────────────────[?][X]┐
│ [Contents] [Index/Find]                              │
│ ┌──────────────────────────────────────────────────┐ │
│ │ closed/open books and topic leaves              │ │
│ │ selection, keyboard navigation, expansion       │ │
│ └──────────────────────────────────────────────────┘ │
│                         [Display] [Print...] [Cancel] │
└──────────────────────────────────────────────────────┘

Implementation rules:

• Use ordinary WAT Button, TreeView, tab, scrollbar, and dialog behavior where available.
• Keep custom topic layout in a WAT-native viewport wndproc.
• Draw into the owning top-level back-canvas; do not add a help-only surface.
• Derive ? caption behavior from DS_CONTEXTHELP, as the generic renderer now does.
• Keep window geometry/profile state separate from document content. Tests that compare pixels must explicitly control both.
• Treat toolbar, menu, and Topics dialog actions as commands into the same navigation engine used by WinHelpA.

Navigation transaction

sequenceDiagram
    participant E as Event/API
    participant N as Navigation engine
    participant I as Document indexes
    participant D as Topic decoder
    participant L as Layout
    participant W as Window

    E->>N: target descriptor
    N->>I: resolve to topic_ref
    alt unresolved
        I-->>N: error
        N-->>E: FALSE / visible diagnostic
    else resolved
        I-->>N: topic_ref
        N->>D: decode(topic_ref)
        D-->>N: bounded token stream
        N->>L: layout(tokens, client width)
        L-->>N: runs + hotspots + extent
        N->>N: push old topic; commit new state
        N->>W: invalidate/update scrollbars/title
        W-->>E: displayed
    end

The old topic remains visible until resolve, decode, and layout all succeed. Navigation is transactional: malformed target data must not destroy a valid current page or corrupt history.

Macro and hotspot safety

Macros are parsed into typed opcodes before execution. Phase one supports only the subset needed by fixtures and required for in-document navigation, such as jumps, popup jumps, contents, back, and safe window commands.

These actions are disabled until explicitly designed and tested:

• launching external programs or documents;
• arbitrary DLL calls;
• filesystem mutation;
• shell execution;
• network navigation outside mounted VFS assets.

Unsupported macros remain visible in diagnostics and fail that action without crashing the viewer. They must not be silently treated as successful.

Source organization

The final split should follow WAT concatenation order and keep unrelated USER tables out of the parser:

Names may change during implementation, but parser and UI code should not be folded into the already-large generic handlers file.

Testing strategy

Testing has four layers.

1. Pure parser tests

Expose test-only WAT entry points that accept a buffer already copied into WASM memory. Tests inspect WAT-owned records, not host-parser results.

Required fixtures and assertions:

• all checked-in HLP files parse or fail with a documented format gap;
• exact internal-file names, offsets, and lengths;
• exact title strings from |SYSTEM/title metadata;
• exact topic count, canonical references, and titles;
• known context IDs resolve to expected topic references;
• phrase decompression yields exact byte strings;
• malformed/truncated variants fail at exact stages without out-of-bounds access or partial published state;
• parser tests run with no renderer and no semantic help host imports.

2. API command tests

Use a tiny guest fixture or exported ABI harness to call WinHelpA/W with controlled arguments:

• context, contents, finder, key, partial key, popup, and quit;
• null versus new filename;
• ANSI versus Unicode pathname/data;
• missing file and missing context;
• two caller HWNDs and two HLP files;
• yield/resume with guest pointers mutated after the initial call;
• close/reopen without stale parser reuse;
• exact BOOL return and stack cleanup.

3. UI behavior and visual tests

Build deterministic state captures for each fixture:

main topic
Contents default
expanded book
selected leaf
Display result
Back result
Index/Find tab
context popup
context-help caption button

Assertions combine window/control dumps, selected topic/state, exact text, and small stable pixel masks. Full-image hashes are avoided where font fallback or desktop placement can vary.

4. Differential reference tests

For each scenario, provide identical HLP/CNT bytes and request semantics to Windows 98/v86 and the WAT implementation.

flowchart LR
    F[Same HLP + CNT fixture] --> N[Native Win98 / winhlp32]
    F --> O[Our WAT WinHelp]
    Q[Same command/context/interaction] --> N
    Q --> O
    N --> C[Capture normalized state]
    O --> C
    C --> R[Compare topic, hierarchy, controls, geometry, pixel masks]

The archived executable remains hash-pinned in tests. It is never copied into production assets. Native GID files are generated in isolated temporary directories and deleted after capture.

Diagnostics

Add a runtime trace category rather than ad-hoc logging:

--trace-help
  api caller=... path=... command=... data=...
  load state=ready|pending|failed size=...
  parse internal="|TOPIC" off=... len=...
  resolve kind=context id=... topic_ref=...
  navigate from=... to=... history=...
  layout tokens=... lines=... extent=...
  error code=... file_off=...

Useful test-only exports may expose counts and immutable record fields. They must not become an alternate host-owned code path.

Suggested stable error classes:

• bad outer header;
• directory/B+tree corruption;
• missing required internal file;
• phrase table corruption;
• topic record corruption;
• unresolved context/keyword;
• unsupported command or macro;
• allocation/capacity limit;
• VFS missing/fetch failure.

Capacity limits

Initial limits should be explicit constants and tested at their boundaries. Proposed starting envelope:

These are compatibility and safety bounds, not promises about native WinHelp. Raise them only with a real fixture and memory/overflow regression.

Migration plan

Phase 0: lock the oracle and expose the gap

• Keep current native/emulator screenshots and archived executable hash checks.
• Add fixture expectations for real internal-file tables and selected native topic text/context mappings.
• Add explicit tests demonstrating corruption or missing context behavior in the JS parser; do not encode corrupted output as desired behavior.

Exit criterion: the expected WAT parser outputs and native UI states are known for FreeCell plus at least Calculator, Notepad, Paint, and WordPad.

Phase 1: WAT file buffer, bounded readers, and directory

Status: implemented. The focused parser gate covers all checked-in HLP directories plus synthetic multi-page/indexed trees, source-buffer mutation, cyclic links, truncation, invalid internal-file headers, and capacity bounds.

• Mount/read HLP bytes through VFS into a document arena.
• Implement outer header, internal-file header, and directory B+tree.
• Add parser error/status exports and malformed fixtures.

Exit criterion: WAT enumerates the exact internal-file directory for every checked-in HLP with no call to HlpParser.

Phase 2: titles, phrases, topics, and context maps

Status: partially implemented. Document metadata, canonical topic/title records, signed context-hash indexes, numeric context maps, and Hall phrase tables are WAT-owned. The bounded topic-block decoder now validates physical LZ77 blocks and the complete forward/back TOPICLINK chain, binds every canonical title entry to its type-2 record, and phrase-expands each topic's raw LinkData2 stream while preserving paragraph-control bytes. A record wider than one physical block — including one whose 21-byte header itself straddles the boundary — is reassembled into a bounded owned gather buffer before it is read, so long topics no longer reject the whole file; a record larger than that buffer, or one whose continuation block is missing, still fails before publication. All checked-in fixtures have exact topic-reference, context-resolution, decompressed-phrase, raw-topic-length, and full-corpus hash coverage, supplemented by synthetic two-level trees and malformed semantic/topic inputs. The canonical phrase interface also covers uncompressed HC30 |Phrases, LZ77-compressed HC31 tables, and the extended MVB layout, including legacy topic-reference spacing semantics and malformed-table cleanup. The formatted token builder uses a two-pass transaction: it first computes exact token and payload requirements, rejects undersized, out-of-memory, or overlapping non-empty arenas without partial token/payload writes, then emits. It preserves exact LinkData2 offsets, copies validated LinkData1, and requires one bounded NUL-terminated string for every character command instead of guessing paragraph breaks from NUL bytes. Exact real-corpus token-kind and payload-byte inventories cover all six checked-in help files, including table paragraphs; synthetic fixtures cover all documented variable payload families, arena aliasing/capacity, and command/string-count mismatch. Exact normalized |FONT and lP/lp |bmN records cover every checked-in resource, with transactional malformed-offset, descriptor, hotspot, duplicate-ID, and capacity tests. Exact decoded payload hashes cover every checked-in picture, while synthetic fixtures exercise all four packing modes plus truncated RLE, invalid LZ77, alias, capacity, and integer-overflow failures without partial output. The paired default keyword tree/data parser validates multi-level pages, linked leaves, occurrence slices, case-folded ordering, canonical topic references, macro sentinels, and capacity limits transactionally. Layout remains before Phase 2 is complete.

• Parse |SYSTEM, phrase tables, |TOPIC, |TTLBTREE, |CONTEXT, and |CTXOMAP.
• Emit exact topic titles/text token streams.
• Build numeric and hashed context indexes.

Exit criterion: known context IDs resolve and decoded plain text matches native reference content for all fixtures.

Phase 3: real API dispatcher and basic topic window

Status: partially implemented. The unified WAT dispatcher can load a raw VFS path or reuse the matching owner's active document. It retains canonical topic references/indexes, dialog/popup mode, keyword selection, contents override, owner, command, and diagnostic status. Context and keyword targets are fully resolved before publication, so failed and foreign-owner requests do not mutate visible state. Matching-owner Quit releases both the session and the WAT-owned document. Focused tests cover bounded command strings, missing paths, unsupported-command failure, and every implemented transition. The WinHelpA and WinHelpW handlers now share this engine; Unicode paths and keyword/macro strings are copied through bounded temporary storage, while ANSI pointer data is normalized from guest to WAT addresses. Accepted topic requests populate the existing help window directly from the WAT-owned title and decoded LinkData2 strings, with no semantic JS callback. Topic presentation now decodes the typed IR transactionally, preflights exact viewer allocations, and publishes retained positioned runs only after layout succeeds. Text wrapping, explicit breaks, semantic spaces, realized font selection/metrics/decorations, normalized color, paragraph margins/indents/spacing/alignment/tabs, fixed and variable table cells, hotspot membership, visible-run painting, bounded scrolling, intrinsic bitmap geometry, and raster painting are WAT-owned. The current viewer path has a temporary 64 KiB decoded-topic cap; dynamic sizing remains Phase 5 work. Browser assets not already mounted in the VFS still need the raw-byte async continuation described above.

• Implement ANSI/Unicode normalization and transactional requests.
• Support P0 commands and correct return/lifecycle semantics.
• Replace flattened JS topic access with WAT document/topic state.
• Render text tokens and Back/Contents through the existing WAT window.

Exit criterion: application F1/help-menu requests open the requested topic; close/reopen and multiple paths do not reuse stale state.

Phase 4: CNT and Help Topics dialog

Status: partially implemented. A bounded two-pass WAT CNT parser owns an exact copy of the source, recognizes the standard directives, and publishes canonical 32-byte hierarchy records only after the complete file validates. It supports compact depth syntax used by HOVER!, enforces the byte, node, line, and nesting caps, computes the documented 256-entry signed context hash, binds resolvable leaves to canonical topics, and retains explicit unresolved, external, and disabled-macro flags. Exact inventories cover every checked-in CNT fixture plus the HOVER! hierarchy; synthetic tests cover parent/child and sibling construction, macros, malformed depth/title/directive data, hostile capacity, source bounds, and preservation of the previously published tree. Mounted HLP paths now derive and load an optional same-directory CNT through the raw VFS boundary, while an existing malformed companion rejects the whole document without partial publication. HELP_FINDER and HELP_PARTIALKEY feed a separate WAT-native Topics window with Contents/Index tabs, canonical visible-row expansion, selection scrolling, keyboard/mouse input, cancel-mode restoration, and Display through the shared transactional navigation engine. Native-reference geometry/icon tuning, multiple-posting selection, and the optional full-text Find model remain to complete this phase.

• Parse CNT hierarchy.
• Bind books/leaves to canonical topics.
• Implement Contents and Index/Find tabs with keyboard/mouse interaction.

Exit criterion: hierarchy, selection, expansion icons, Display, and Back match the native fixture matrix.

Phase 5: formatted topics, hotspots, popups, and images

Status: partially implemented. Typed topic arenas and deterministic positioned text runs are live in the production WinHelp window. Layout uses an exact no-write preflight, retains raw/token/payload/run state transactionally, wraps words and overlong spans, carries font/color and hotspot state into each run, and repaints visible runs without decoding again. Hotspot runs retain the exact begin-token identity, bounded hit testing accounts for scrolling, and fixed direct-topic or context-hash jumps resolve through canonical topic state, shared transactional history, and the production window-message path. Fixed popup hotspots and HELP_CONTEXTPOPUP instead create a bounded owned WAT-native popup plus shadow surface. Popup layout uses a 320-pixel maximum measure and shrinks to retained run extents, with explicit minimum/maximum geometry and screen-edge clamping. Nested, orphaned, and unterminated hotspot regions are rejected. External EA/EB/EE/EF structures are parsed by exact size/type/string grammar; current- file types resolve hashes locally, and mounted type-4 filenames resolve against the retained source directory. A four-record owning document stack makes cross-file Back and popup dismissal LIFO transactions, including restoration of the source path, session scalars, scroll, and 16-entry topic Back contents. Missing files, malformed targets, unresolved target hashes, allocation failure, and a fifth suspension leave the visible source transaction intact. Macro forms still fail explicitly without changing topic or history.

|SYSTEM type-6 records normalize into a bounded 56-byte window table holding flags, the type/name/caption slices, signed geometry, the show word, and both region colors; fixed-width fields may consume their full field, and a record whose size is not the documented 90 bytes rejects the file. A type-1 hotspot number indexes that table, with 0xFF retaining the current viewer, and a type-6 window name resolves case-insensitively inside the file it names, after that document is live. The selector rides with the session: it is captured and restored by the document snapshot stack, so cross-file Back returns to the presentation its source used, and any API-issued command returns to the canonical main viewer. Presentation applies through the ordinary viewer path — geometry scales from the |SYSTEM 1024ths coordinate space against the live screen size and moves the window only when the selector itself changed, the layout width follows the presented width, and the caption is published as an owned NUL-terminated copy that falls back to the document title. Referenced logical fonts materialize once per view as owned type-4 GDI objects; bounded face copies, half-point/twip conversion, weight/italic selection, exact selected-font measurement, repaint selection, and retained underline/strikeout geometry share the normal WAT GDI path. Allocation failure retains the prior complete view, and replacement/document/Quit teardown removes selected DC references before deleting every font. External embedded-picture references now retain exact command bytes alongside a canonical normalized bitmap index. Referenced DDB/DIB resources decode and materialize as owned GDI bitmap/source-DC state inside the same view transaction; layout uses normalized width/height, visible image runs paint via SRCCOPY, and topic/document/Quit teardown deletes every object. A failed decode or allocation leaves the prior complete view live. Inline picture unions and metafiles keep bounded placeholders pending their independent decoders. Retained paragraph slices now drive half-point/twip margins, first-line and continuation indents, before/after/line spacing, right/center alignment, left/right/center tab stops, and fixed or relative table-cell bounds. Variable tables honor their minimum width and scale their signed column metrics against the documented 32767 total without floating point. While a popup is live, the primary typed view is detached without freeing or re-decoding its raw/token/payload/run arenas or its owned font/bitmap/DC state. Escape, WM_CLOSE, focus loss, or a background click destroys both popup windows and atomically restores the exact primary pointers, scroll position, session topic, mode, and Back count. A normal jump from a popup rejoins the ordinary primary navigation transaction instead. HELP_QUIT reunites and frees both views exactly once.

• Extend raster materialization to inline picture unions and metafiles.

Exit criterion: visual topic captures and hotspot target transitions match the reference for fixtures containing |FONT and |bmN data.

Phase 6: keyword/search and safe macros

Status: partially implemented. The default exact/prefix keyword index and its canonical postings are WAT-owned. The semantic JavaScript HLP parser's browser script, async continuation, CLI require, host callbacks, and WASM imports have been deleted, so production has no alternate host-owned topic-selection path. Named keyword tables, the full-text Find model, and the fixture-driven safe macro subset remain.

• Extend the implemented default keyword exact/prefix index to named HELP_MULTIKEY tables.
• Build the in-memory Find index without requiring GID.
• Add the fixture-driven safe macro subset.

Exit criterion: all supported commands use WAT-owned state, production loads no JS HLP parser, and the archived viewer remains test-only.

Definition of done

The WAT-native WinHelp effort is complete for the supported Win98 corpus when:

• WinHelpA and WinHelpW share a real command dispatcher;
• P0 and P1 commands have exact API and lifecycle tests;
• HLP and CNT parsing is entirely WAT-owned and bounds-checked;
• context IDs, keywords, contents nodes, hotspots, and history use canonical topic references;
• topic text is no longer corrupted or guessed from arbitrary first lines;
• Contents/Find, books/leaves, Display, Back, popups, and close behavior match native reference states;
• formatted text and checked-in embedded bitmaps render through canonical WAT GDI surfaces;
• malformed files and unsupported commands fail deterministically;
• JavaScript contains no help-format parser or semantic topic callbacks;
• production assets do not include winhlp32.exe, GID caches, or other Microsoft reference binaries.