Video for Windows support — design

Status: design, 2026-09-28. The decoders exist: src/09a7e-video-codecs.wat (BI_RGB, RLE8, Cinepak) is in the build and bit-exact against ffmpeg. The ICM, AVI core, MCI and Animate layers above it are not written yet.

tl;dr

                           GUEST APPS
 ┌──────────────┬───────────────┬─────────────────┬──────────────┐
 │ direct ICM   │ AVIFile       │ MCI avivideo    │ Animate ctl  │
 │ War Wind     │ Dark Colony   │ HL Uplink       │ SysAnimate32 │
 │ Dark Colony  │ TDR2000, AVS  │ Civ2 (MCIWnd)   │ VB6 samples, │
 │ Civ2, MW3,   │ Civ2 (Win16,  │ AoE1/2 (MCIWnd) │ shell copy   │
 │ MCM, DbtS    │  exists)      │                 │ progress     │
 └──────┬───────┴───────┬───────┴────────┬────────┴──────┬───────┘
        │               │                │               │
        │       ┌───────▼────────────────▼───────────────▼──────┐
        │       │ AVI CORE (WAT, one parser for Win16 + Win32)  │
        │       │ RIFF 'AVI ' → avih, strl{strh,strf,strn},     │
        │       │ movi{##dc ##db ##wb ##pc, LIST rec}, idx1,    │
        │       │ AVIX extents; keyframe flags; sample<->time   │
        │       └───────┬──────────────────────────────┬────────┘
        │               │ video chunk                  │ audio chunk
 ┌──────▼───────────────▼─────┐               ┌────────▼──────────┐
 │ ICM LAYER (WAT)            │               │ AUDIO             │
 │ ICOpen ICLocate ICClose    │               │ PCM → waveOut mix │
 │ ICInfo ICGetInfo           │               │ ADPCM → ACM later │
 │ ICSendMessage ICDecompress │               └───────────────────┘
 │ + Win16 MSVIDEO ordinals   │
 └──────┬─────────────────────┘
        │ switch on fourcc
 ┌──────▼─────────────────────────────────────────────────────────┐
 │ CODECS (WAT): compressed frame in → DIB out (8/16/24/32 bpp)   │
 │  BI_RGB  copy / flip / 555                     done (09a7e)    │
 │  BI_RLE8 runs + delta, retained plane          done (09a7e)    │
 │  cvid    Cinepak: strips, V1/V4 2x2 codebooks  done (09a7e)    │
 │  CRAM    MS Video 1: 4x4 blocks, 1/2/8 colours done (09a7e)    │
 │  IV41    Indeo 4: wavelet bands, VLC, MC       large, phase 5  │
 └──────┬─────────────────────────────────────────────────────────┘
        ▼
  DrawDib / StretchDIBits (exist) → window surface
  MCI player clock = guest time, audio is the master clock

 NOT OURS: Smacker (smackw32.dll) and Bink (binkw32.dll) ship with the
 game and already run as x86. Deferred: DirectShow/quartz, QuickTime.

 ORDER: ICM + BI_RGB/RLE8/Cinepak → AVIFIL32 → MCI avivideo
        → Animate control → MS Video 1 → Indeo 4

What the apps actually do

Measured 2026-09-28: a static scan of all 2,523 PEs under test/binaries for the VfW entry points, ffprobe over all 145 .avi files there, then test/run.js --trace-fs --trace-api=<video APIs> for 40 s per app. The CD-only titles came from their ISOs.

The 40 s runs of AoE1, AoE2, MCM, Moorhuhn 2, TetriNET, SC2K Net and the Win98 tour/welcome made MCI calls for CD audio and MIDI only; their video APIs are imported but not reached in that window.

What exists today

Updated 2026-09-28. Phases 0-4 and MCIWnd have landed; Indeo 4 remains.

Still missing:

• Indeo 4 (IV41). This is every Civ2 MGE movie (59 files). The route being built loads the real ir41_32.dll, from the game's own disc or install, as guest x86 behind the ICM driver protocol. We do not ship it.
• The mciSendCommand binary interface to the avivideo device.
• Palette-change chunks in MCI playback, play repeat, and the MCIWnd playbar and menu.
• DrawDib with compressed input.

API surfaces

1. ICM — the codec manager (MSVFW32.DLL; Win16 MSVIDEO.DLL)

The foundation. Everything else reaches codecs through it.

Most of the vfw.h "functions" (ICDecompressBegin, ICDecompressGetFormat, ICDecompressEx…) are macros over ICSendMessage, which is why War Wind imports only ICOpen, ICSendMessage and ICClose.

Messages (ICM_USER = 0x4000):

Structs: ICDECOMPRESS { dwFlags, lpbiInput, lpInput, lpbiOutput, lpOutput, ckid }; ICDECOMPRESSEX { dwFlags, lpbiSrc, lpSrc, lpbiDst, lpDst, xDst, yDst, dxDst, dyDst, xSrc, ySrc, dxSrc, dySrc }. Guest buffers are guest pointers; decode through $guest_span_in or per-row translation, never through one $g2w of the whole frame (a 320×240×3 output spans many guest pages; see tools/check-guest-span.js).

HIC record (emulator-private table, generation-tagged handle like the SetupDi sets): {codec id, mode, state ptr, in format copy, out format copy, palette[256]}. Codec state lives in an emulator arena: a Cinepak context is 32 strips × (2 × 256 × 12 bytes of codebook) plus the retained RGB frame.

2. AVIFile (AVIFIL32.DLL; Win16 AVIFILE.DLL)

Dark Colony needs AVIFileInit, AVIFileExit, AVIStreamOpenFromFileA, AVIStreamInfoA, AVIStreamLength, AVIStreamReadFormat, AVIStreamRead, AVIStreamRelease. Round it out with AVIFileOpenA/W, AVIFileGetStream, AVIFileInfo, AVIFileRelease, AVIStreamStart, AVIStreamSampleToTime, AVIStreamTimeToSample, AVIStreamFindSample, and AVIStreamGetFrameOpen/GetFrame/GetFrameClose (Winamp AVS), which is a thin loop over the ICM layer.

A PAVIFILE/PAVISTREAM is a COM object (IAVIFile, IAVIStream, IGetFrame); apps may call through the vtable, so build them the way the DX COM objects are built (vtable in the thunk zone, 0xC0DE0000|api_id).

The Win16 AVIFILE parser already exists. Lift its RIFF walk into a shared AVI core that both front doors call, rather than writing a second one. The core owns: the header lists, a per-stream sample index (from idx1 when present, else a movi scan), keyframe flags, ##pc palette changes, and OpenDML AVIX extents. AVIStreamRead returns the raw chunk bytes; nothing in AVIFile decodes.

3. MCI avivideo (MCIAVI) and MCIWnd

The player that apps without their own decode loop use. Commands in the traced apps: open <file> type AVIVideo alias A parent H style child, window A handle H [state …], put A destination at x y w h, seek A to start, break A on <vk>, play A [wait|notify], stop, close, and status A length/position/mode, where A destination. mciSendCommand carries the same through MCI_OPEN/PLAY/WINDOW/PUT/… with MCI_DGV_* structs.

Design:

• A device record per open: AVI core handle, video and audio stream, HIC, output DIB, target hwnd + destination rect, position, mode, notify hwnd, break key.
• The device lives in WAT, not in lib/host-audio.js: it decodes, owns a window and paints. The JS MCI parser forwards avivideo/.avi opens to it, the way it already hands cdaudio to the disc.
• Painting: decode into the device DIB, then the same path as StretchDIBits onto the target window (or a child window it created under parent, class AVIWnd32).
• play … wait blocks the guest call. Park the calling thread like a blocking wait; each batch the device advances to the frame the guest clock says, paints, and resumes the guest with 0 at the end, on stop, or when the break key is pressed (Uplink passes VK_ESCAPE, 27). play … notify returns at once and posts MM_MCINOTIFY (0x3B9) to the callback hwnd when done.
• Audio: the PCM stream goes into the existing waveOut mixer as one voice (a device-owned waveOut), and its played-sample position is the master clock. Video frames are chosen from it; a late frame is decoded without being painted (Cinepak and RLE8 inter frames must still be decoded in order).
• MCIWndCreateA/W (MSVFW32) is a window class (MCIWndClass) wrapping the same device, driven by MCIWNDM_* messages; add it after the string interface works, for Civ2 MGE and AoE.

4. The Animate control (COMCTL32 SysAnimate32)

ACM_OPENA/W (WM_USER+100/+103), ACM_PLAY (+101, from/to/repeat), ACM_STOP (+102), notifications ACN_START/ACN_STOP. It plays silent RLE8 or BI_RGB AVIs from a resource or a file, looped. That covers the VB6 sample AVIs and the shell file-operation progress dialogs, which load their animations from shell32 resources. A timer-driven wndproc over the AVI core plus the RLE8/BI_RGB converters; no ICM needed.

5. DrawDib with compressed input

DrawDibBegin/DrawDibDraw accept a compressed lpbi and decompress through ICM themselves. Once ICM exists, a compressed format opens an HIC in the DDIB record and decodes into its buffer before the existing StretchDIBits step, which also makes DDF_UPDATE possible.

Formats and algorithms

Container: RIFF AVI

RIFF 'AVI ' { LIST 'hdrl' { avih; LIST 'strl' { strh; strf; strn? }… }; LIST 'movi' { chunks | LIST 'rec ' { chunks } }; idx1? }, optionally followed by RIFF 'AVIX' extents (OpenDML). Chunks are padded to even sizes. A chunk id is two stream digits plus dc (compressed video), db (uncompressed video), wb (audio) or pc (palette change). strh carries dwScale/dwRate (fps = rate/scale) and dwLength; strf is a BITMAPINFOHEADER plus palette for video, WAVEFORMATEX for audio. idx1 entries {ckid, flags, offset, size} give keyframes (AVIIF_KEYFRAME 0x10); offsets are relative to the movi list (occasionally absolute — detect by checking the first entry). A zero-length video chunk means "repeat the previous frame" (WWOPEN has 7 of them).

BI_RGB (uncompressed)

Rows padded to 4 bytes, bottom-up unless biHeight < 0. 8 bpp uses the strf palette (updated by ##pc), 16 bpp is RGB555, 24 bpp is BGR, 32 bpp is BGRX. The only work is conversion to the requested output depth.

BI_RLE8

Byte pairs (n, c): n > 0 is a run of n copies of index c; n = 0 escapes: 0 end of line, 1 end of frame, 2 dx dy skip, 3–255 that many literal bytes padded to a word. In AVI, delta frames encode only what changed, so the index plane must persist between frames and not be cleared; the 10a bitmap decoder starts from a fresh plane and needs a "retain" mode. Output is 8 bpp indices plus palette, or converted through the palette.

Cinepak ('cvid', Radius/SuperMac, iccvid.dll)

A vector quantiser. Frame = 10-byte header {flags8, size24, w16, h16, strips16} (big-endian throughout), then horizontal strips, each {id16 (0x1000 key / 0x1100 inter), size16, y1, x1, y2, x2} (a zero y1 means "relative to the previous strip") containing chunks:

A codebook entry is a 2×2 block: Y0..Y3 plus signed U and V, converted as R = Y + 2V, G = Y − U/2 − V (C truncating division), B = Y + 2U, clamped. V1 blows one entry up to 4×4 (each pixel doubled); V4 tiles four entries into the 4×4 block (TL, TR, BL, BR). Codebooks persist per strip index across frames, and strip i > 0 starts from strip i−1's books unless frame-flag bit 0 is set. Output: 24/32 bpp directly, RGB555 by truncation.

8-bit output (War Wind asks for it): iccvid decodes into a fixed palette with an ordered dither and hands that palette out on DECOMPRESS_GET_PALETTE. The exact iccvid palette and dither matrix are the one open fact here. Measure them on the Win2K QEMU 256-colour reference VM (decode a known frame to 8 bpp with a tiny ICM test program, read back palette + indices). Until then, use a 6×6×6 cube plus greys in the 236 non-system entries with a 4×4 Bayer dither, and say so in the code. Palettized Cinepak sources (biBitCount 8, codebook entries are indices) exist but are rare; decline them with ICERR_BADFORMAT until one turns up.

MS Video 1 ('CRAM'/'MSVC'/'WHAM', msvidc32.dll)

4×4 blocks with a 16-bit flags word each: a skip run, a solid fill, 2 colours

• a 16-bit mask, or 8 colours (four 2×2 quadrants with 2 colours each). 8 bpp (palette indices) and 16 bpp (RGB555) variants. Not in the corpus AVIs, but it is the other codec Win98 shipped, and it is small.

Done: $vid_cram_decode (8 bpp into the retained index plane, then $vid_index_to_bgrx; 16 bpp straight into the BGRX frame), registered in the ICM layer for 'CRAM'/'MSVC'/'WHAM' in any case, and in the standalone player. Bit order, measured against ffmpeg's msvideo1.c: flag bit k paints row k>>2 counted up from the block's bottom, column k&3, set = the first colour of the pair; eight-colour pairs are bottom-left, bottom-right, top-left, top-right; a skip counts this block, and a skip count of 0 skips the rest of the frame. 8 bpp takes eight colours for flag high byte 0x90 and up, 16 bpp when bit 15 of the first colour is set. test/test-video-cram.js checks hand-built 8 and 16 bpp streams with every block kind against the generator's own model, against ffmpeg through verify.js (bit-exact, including an ffmpeg-encoded 16 bpp movie), and through ICLocate + ICM_DECOMPRESS. ffmpeg has no 8 bpp encoder, which is why the 8 bpp fixture is hand-built.

Indeo 4 ('IV41', Intel ir41_32.ax)

Civ2's 119 movies. YVU 4:1:0: three planes, the luma plane optionally split into 4 wavelet bands (Haar or 5/3), each band divided into tiles → macroblocks (8×8/16×16) → blocks (4×4/8×8). Per band: an inverse transform (slant 8×8/4×4, 1-D slants, Haar or none), run-level coefficient coding with VLCs chosen from 8 static and custom Huffman codebooks plus run/value remapping tables, per-macroblock quantiser deltas, and half-pel motion compensation from the previous (or scalable-reference) frame. Frame types: intra, inter, scalable inter, droppable, null (repeat). An optional transparency plane. Output YUV → RGB. The reference implementation is FFmpeg's indeo4.c + ivi.c + ivi_dsp.c with their tables, several thousand lines of C. It is the largest single item here and only Civ2 needs it; check license compatibility before porting the tables.

Audio inside AVIs

Every traced AVI carries PCM (8-bit unsigned or 16-bit signed, mono or stereo), which needs no codec. IMA ADPCM (0x11) and MS ADPCM (0x02) would go through ACM (acmStreamOpen/Convert), which today refuses every non-PCM stream. Add those two converters only when a real AVI needs them.

Where the code goes

Regions: one region.declare for the codec-state arena (HIC records, Cinepak books, retained frames; ~2 MB covers several open streams at 640×480) and a small one for MCI device records. No new host imports: decoding and painting are WAT, and audio reuses the waveOut mixer.

Testing

1. One implementation, checked against ffmpeg. The decoders exist only as WAT. There is no JS copy to drift from them. tools/avi-player/codecs-wasm.js compiles src/09a7e-video-codecs.wat as a module of its own: the fragment's functions plus an exported memory, built with the vendored WATX compiler under the build's own options. It works because the fragment is pure. It runs in Node through tools/watx.js and in the page through the tools/watx-src/ scripts. node tools/avi-player/verify.js <avi…> compares that module's output with ffmpeg, frame for frame. Measured 2026-09-28, all bit-exact with worst channel delta 0:

   • every frame of War Wind's WWOPEN (1898)
   • the first 200 of LOGOS, EA1CH and TH1MS
   • all five Dark Colony movies
   • every frame of Uplink's intro (1344) and of uplink.avi
   • all 19 VB6 samples (Cinepak, RLE8, 8/16/24 bpp)

   Civ2's IV41 is reported as SKIP.

2. ICM unit test (test/test-video-codecs.js): drive ICSendMessage inside the full emulator with real frames from those files. Compare the guest output buffer with the standalone module's frame: 24 bpp must match exactly; 8 bpp is checked against the iccvid palette once that has been measured. This test covers the ICM layer's pointer translation and format conversion; the decoders themselves are covered by 1.

3. ICM surface tests: ICLocate with NULL lpbiOut, GET_FORMAT size probe, QUERY refusal of IV41, DECOMPRESSEX rectangles, HURRYUP.

4. App checks (headless --png at fixed batches): War Wind's LOGOS frame, Dark Colony past AVIStreamOpenFromFileA into its intro, Uplink's intro via MCI (and break on ESC ends it), Civ2 still skipping cleanly.

5. Watching. tools/avi-player/index.html (serve the repo root with node tools/dev-server.js, open /tools/avi-player/) plays any of the corpus movies or a dropped file, with audio, a seek bar, frame stepping and the per-frame key/delta, size and decode time. It compiles the same WAT fragment in the page. The only JS is the demux, PCM unpacking and the BGRX→RGBA copy into the canvas.

Phases

Out of scope

• Smacker / Bink. The game ships the vendor DLL (smackw32.dll, binkw32.dll) and it runs as x86 like any other code; War Wind II's cutscenes and SimGolf's .bik intro use this path.
• DirectShow / ActiveMovie (quartz.dll, MPEG-1), QuickTime, and compression (ICCompress, only VirtualDub asks) — each is a separate design if an app needs it.
• WebCodecs / <video>. Neither decodes Cinepak, RLE8 or Indeo, and a browser-only decoder would split the CLI from the browser. The decoders are WAT so both hosts run the same bytes.