test: real shader/terrain tests + headless render check

npm test was a brace counter: it checked each file was >10 chars and that {
and } counts matched. It passed on a build whose shader would not compile.

- tests/shaders.test.js compiles the game's real GLSL with glslangValidator
  (Khronos' reference compiler) under a three.js-style attribute prefix, and
  includes a canary that asserts a known-bad vec4 construction is REJECTED, so
  the suite cannot pass vacuously.
- tests/terrain.test.js covers heightfield determinism (the "persistent world"
  claim), airport flattening inside the 200 m radius with no leakage outside it,
  out-of-bounds behaviour, plausible height range, and vertex colour counts.
- scripts/headless-check.mjs (npm run test:render) serves dist/, drives the
  game over raw CDP with SwiftShader software WebGL, and fails if the canvas is
  black, the shaders fail to compile, or the scene does not change after input.
  No new dependencies: Node 22 built-in fetch/WebSocket. 120 s watchdog so it
  cannot hang a runner.
- npm test -> node --test "tests/*.test.js"; old checker kept as npm run lint:syntax.
- CI installs glslang-tools and runs the real tests; render job is
  continue-on-error because not every runner has a Chrome binary.

Verified non-vacuous: re-breaking the shader makes the unit tests fail 2/4 with
glslang's own "constructor: too many arguments" and makes the render check fail
four ways (1 distinct colour, 0/64000 lit pixels, shader error, frozen scene).
This commit is contained in:
Hermes Cron Job committed 2026-10-07 13:31:32 +00:00
1 parent 04e3400ab9
commit ee24276fd0
6 files changed
+528 -13

No files matched your search

+26 -11
View File
@@ -67,16 +67,31 @@ shove a warning in your face.
**Fix:** stall flag on the flight model → HUD warning + stick shaker / audio cue.
## T4 — The test suite is theatre
## T4 — The test suite is theatre — FIXED on branch `fix/crt-shader-and-real-tests`
`scripts/check-syntax.js` checks that each file is >10 chars and that `{` and `}` counts
match. It cannot catch T1. CI (`.gitea/workflows/*.yml`) runs it on every push, so the
green tick currently means nothing.
match. It cannot catch T1 or T8. CI (`.gitea/workflows/ci.yml`) runs it on every push, so
the green tick meant nothing.
**Fix:** real unit tests on `FlightModel` (takeoff profile, stall, ground contact) and
`Terrain` (determinism, airport flattening radius). Node has `node:test` built in — no
new dependency needed. Add a headless smoke test that asserts the game reaches the
`FLIGHT` state.
Now: `npm test` runs `node --test "tests/*.test.js"` (no new dependency) with
`tests/shaders.test.js` and `tests/terrain.test.js`; the old brace-counter survives as
`npm run lint:syntax`. `npm run test:render` runs `scripts/headless-check.mjs`, a
dependency-free CDP check that renders the built game in software WebGL and fails if the
canvas is black or the shaders fail to compile.
Both new tests were proven **non-vacuous** by re-breaking the shader and watching them
fail: the shader tests fail 2/4 with glslang's own `'constructor' : too many arguments`,
and the render check fails four ways (1 distinct colour, 0/64,000 lit pixels, shader
error, scene frozen).
Still outstanding: no `FlightModel` tests yet — those belong with T1, where the expected
behaviour has to be decided first.
Note on CI: this instance currently has **no registered Actions runners**
(`GET /repos/kevinhermes/retro-flight-sim/actions/runners` → `total_count: 0`), and
upstream's only run (id 36) ended `cancelled`, not `success`. So CI has never actually
validated this repo. The workflow is updated to install `glslang-tools` and run the real
tests, but treat CI as unproven until a runner exists.
## T5 — No LICENSE
@@ -91,7 +106,7 @@ not the same thing.
**Fix:** persist the world seed + aircraft state; add a save/load menu entry.
## T8 — The CRT shader does not compile, so the 3D canvas is black
## T8 — The CRT shader does not compile, so the 3D canvas is black — FIXED on branch `fix/crt-shader-and-real-tests`
Found by running the game under software WebGL, not by reading the code. `js/retro.js:29`:
@@ -115,10 +130,10 @@ colour, 0/64,000 non-black pixels**. Verified in isolation: `vec4(vec3, float)`
**Fix (verified):** `gl_Position = vec4(position.xy, 0.0, 1.0);` — compiles clean, and the
canvas then reports 143 distinct colours, 63,792/64,000 non-black pixels, scanline banding
visible every 25 rows, and the scene hash changing frame to frame once the aircraft moves.
Applied locally during the audit to confirm it, then reverted — not committed.
Committed on this branch.
Note this is the kind of bug the current test suite cannot see (see T4): braces balance
perfectly.
Note this is exactly the class of bug the old test suite could not see: braces balanced
perfectly. `tests/shaders.test.js` now compiles these shaders with glslang on every run.
## T7 — Minor / housekeeping