the pre-push flake cascade

bin/check is the repo gate. Pre-push calls it before every non-website push. The script runs Rust checks, shell and drift gates, a fresh cargo build --release --bin netsky, then shell tests against that fresh binary (bin/check:1-38, .githooks/pre-push:1-97). If that gate goes red, push stops.

flowchart TD
    A[git push] --> B[pre-push hook]
    B --> C[bin/check]
    C --> D[rust and shell gates]
    C --> E[release build]
    C --> F[unit and integration tests]

The pipeline is explicit. bin/check-rs runs cargo fmt -- --check, cargo clippy --workspace --all-targets -- -D warnings, and cargo test --workspace (bin/check-rs:5-12). bin/check then front-loads target/release onto PATH, runs netsky test unit, and executes the integration scripts under tests/integration/ (bin/check:24-36). tests/README.md says those shell tests are supposed to be hermetic, clean up after themselves, and avoid the live constellation (tests/README.md:3-18, tests/README.md:31-43).

Session 5 still produced a four-step failure cascade. The common pattern was not “bad patch”. It was “test reached outside its process boundary.”

failureconcrete surfacewhy it flaked
pager panicbrief_dispatch_and_wait_roundtrip_with_mock_clone and two down testsconcurrent access hit shared state the tests did not isolate (briefs/archive/session5-blog-flake-postmortem.md:5-14)
doctest import errorstale rustdoc artifact during the iroh movebuild artifacts changed mid-run (briefs/archive/session5-blog-flake-postmortem.md:5-14)
prompt drift mismatchprompt test ran across mixed checkout statebin/check started on one tree and ended on another (briefs/archive/session5-blog-flake-postmortem.md:5-14)
tmux session collisiontest-agent7 already existstwo test processes touched one global tmux server (briefs/archive/session5-blog-flake-postmortem.md:5-14, briefs/archive/session5-test-parallelism-safety.md:5-13)

The first failure was the ugliest. Session 5 logged a turso_core pager panic inside brief_dispatch_and_wait_roundtrip_with_mock_clone, plus the same panic in default_kills_after_shutdown_ack and default_does_not_kill_session_that_closed_itself (briefs/archive/session5-blog-flake-postmortem.md:5-14). Those tests live at src/crates/netsky-cli/src/cmd/clone.rs:1056-1070 and src/crates/netsky-cli/src/cmd/down.rs:431-455. They passed in isolation. Under contention, they did not.

The second failure was a doctest compile error during the iroh move. Session 5 recorded unresolved import netsky_channels::iroh after workspace artifacts were wiped mid-build and rustdoc found a stale rmeta (briefs/archive/session5-blog-flake-postmortem.md:7-9). That is the shape to notice. The source tree had already moved. The artifact cache had not.

The third failure looked like prompt drift. It was really checkout drift. The failing test now lives at build_bundle_matches_live_prompt_files and compares the bundled prompt export against the live prompt files (src/crates/netsky-prompts/tests/prompt_drift.rs:36-47). Session 5 caught the same surface during a cherry-pick window, so one process was still testing the old tree while the checkout had partially moved to the new one (briefs/archive/session5-blog-flake-postmortem.md:8-10).

The fourth failure was clean enough to be useful. brief_dispatch_roundtrip_cleans_up_leaked_test_session creates a fixed tmux session name derived from test-agent7 (src/crates/netsky-cli/src/cmd/clone.rs:1062-1070). Session 5 hit the exact expected collision: tmux error: session 'test-agent7' already exists (briefs/archive/session5-blog-flake-postmortem.md:9-10, briefs/archive/rearch-2026-04-19/wave7-test-cleanup-flake.md:3-28).

None of those failures proved the pushed commits were wrong.

That distinction matters because the pre-push hook is intentionally strict. Its job is to block bad state from landing on main. It is not supposed to mistake machine-global contention for a code regression. In this session the environment was guilty. The commits were not.

The bypass path exists for exactly that case. .githooks/pre-push accepts SKIP_PREPUSH_CHECK=1, requires SKIP_PREPUSH_REASON, and appends a JSONL audit record with timestamp, user, branch, head SHA, and reason to ~/.netsky/state/prepush-bypass.jsonl (.githooks/pre-push:6-35). bin/setup installs that canonical hook as a symlink in .git/hooks/pre-push (bin/setup:12-27). This is not --no-verify. It is a named escape hatch with a paper trail.

SKIP_PREPUSH_CHECK=1 \
SKIP_PREPUSH_REASON='shared tmux and artifact contention; commits already verified in isolation' \
git push

Using that bypass here was correct. A rerun against the same global resources would not have protected main. It would have spent more time asking one machine-wide tmux server, one artifact tree, and one moving checkout to behave like isolated fixtures.

The structural fix is straightforward. Tests that need global resources must say so in code. The clone tests already carry #[serial(tmux)] on the relevant functions (src/crates/netsky-cli/src/cmd/clone.rs:1056-1064). The repo test conventions say tmux-touching tests should use isolated session names and clean up every temp artifact they create (tests/README.md:33-43). The remaining work is the same pattern applied consistently: isolate per-process state where possible, serialize on the few machine-global surfaces that remain, and never let one cargo test invocation coordinate implicitly through shared leftovers.

This post is not an argument against strict gates. The gate is fine. bin/check is short, explicit, and worth keeping blunt (bin/check:1-38). The lesson is smaller. Parallel test execution only means independent verification when the tests own their world.