The capture contract

There is one specification of the capture pattern, and this is it. Not a description of it - an executable one, in tests/conformance/.

It exists because the capture is the thing heliograph is most careful about, and because its failure mode is invisible. A broken capture does not crash. It produces a log that reads perfectly and cannot answer the only question it was written to answer.

Run it

./tests/conformance/conformance.sh tests/conformance/drivers/bash.sh
./tests/conformance/conformance.sh tests/conformance/drivers/mutant.sh   # must FAIL

A driver supplies the implementation; the suite supplies the properties. Nothing in the suite may reference caplib.sh, run.sh or any path inside the station payload - the moment it does, it stops being a specification and becomes a second copy of one implementation.

drivers/mutant.sh is deliberately broken and the suite asserts that it fails. A test suite nobody has watched fail is a suite nobody knows works.

The ten properties

1. Every captured line carries a distinct UTC timestamp. Three lines a second apart must produce three different stamps. This is the busybox failure and the reason the whole suite exists: a sed without -u buffers, every line of the block is stamped when the buffer flushes, and the log reads perfectly while being useless.

2. A real gap in the source appears as a gap in the log. Distinct is not enough. Stamps could be distinct and still wrong, by being applied when a line is read from a buffer rather than when it was produced. A three-second sleep must show as a three-second gap.

3. The step's real exit code survives the pipeline. The capture is a pipeline, so $? is the exit of the last stage and is almost always 0. Getting this wrong publishes every failed run as a success - the most expensive possible defect here, because it wastes a round trip through somebody who cannot debug the machine. Both 42 and 0 are asserted, so the check cannot be stuck on failure.

4. A failed run still writes a complete log. Same footer, real exit code, RESULT: FAILED, and the output produced before the failure. A passing run says OK, so RESULT is not hardcoded.

5. An undeclared step refuses, exit 3, having written nothing. With a control: the same runner, the same invocation, a step that does declare itself, which must run. Without that, the property would pass just as well if the runner refused everything.

6. Running as the privileged account refuses, exit 5. Simulated rather than actually run as root, and the driver supplies the simulation - the suite asks the question and the platform answers it. On Unix that is a fake id on PATH: cap_refuse_root calls id -u rather than reading $EUID specifically so a test can, a deliberate seam. Windows has no root; it has SYSTEM, S-1-5-18 and an Administrator role, and how you pretend to be one is a property of the platform rather than of the capture.

The refusal must also precede the step. Exit 5 after running it is the whole defect wearing the right exit code, so the step writes a marker and the marker must be absent.

7. Redaction is wired into the capture. Weaker than the redaction unit tests, on purpose, and asserting something different: that masking is actually on the capture path of this implementation. A correct redactor a second implementation forgets to call is exactly the drift this suite is for, and a unit test of the function cannot see it. Over-masking is asserted too.

The cases live in tests/fixtures/redaction-corpus.txt, one file both implementations are measured against. Neither redactor is the specification; that file is.

8. A cancelled run keeps what it captured. A log that stops mid-sentence is still evidence, and usually the evidence you wanted: the last line names the probe that was in flight.

The cancel has to be proved, and the proof is asked of the file rather than of the driver: after the cancel the log must stop growing. Three seconds into a ten-second step, a log that is merely unfinished looks exactly like a cancelled one - a driver that cancelled nothing at all passed every other assertion here until the suite started watching the size.

9. The finished log reaches the far side. Read back from the receiving end, never from the working tree that wrote it.

10. The log carries text, not a terminal. ANSI escapes stripped, carriage returns gone, stderr captured, and the final line kept even with no newline after it. That last one found a real defect: the bash capture dropped it, because while read returns non-zero at EOF and the loop ended - and the line a step was mid-way through writing is the probe that was in flight, which is the most valuable line in the file.

Property 9 is the one worth explaining

Properties 1 to 4 all assert things about the log file, and the file was always correct. None of them asked whether it arrived.

It had not. On the relay and blob transports the finished log never left the machine: the runner ended at a git push, unconditionally, so a station on any other channel captured a perfect log and delivered nothing - no footer, no exit code, no RESULT. It passed every other property throughout, and it was invisible from the control side because the log was written correctly to local disk every single time.

That is the argument for the property, and for the suite: the properties you have are the defects you catch.

Every transport, not just git

Property 9 answered that question on git alone for as long as it existed, which made it a much weaker property than it looked. Properties 1 to 8 look at the captured file, and the file is written correctly to local disk on every transport - so a tp_put_log that returned success and did nothing whatsoever would have passed the entire suite.

The whole suite now runs once per transport:

The PowerShell station ships git, share and relay, and runs the suite over all three. The relay run is the one that earns its place: the station side has no binary and seals with lib/seal.psm1, while the control side of the harness opens the delivery with Go. A byte of divergence between the two implementations of the seal returns nothing, and p9 fails - which is the only way that divergence is ever visible.

Where there is no Go toolchain or no python3 to run the stub, the driver skips the relay by name rather than claiming a channel it could not read back.

The relay stub is a queue with an HTTP interface and the relay's token rules, which is the entire contract the station side depends on. It is deliberately ignorant of the payload: every body is an opaque sealed envelope, stored and handed back byte for byte. A stub that could read the messages would be one that could accept an envelope the real relay would mangle.

Its token scopes are asymmetric, because the real ones are: a station token may collect a request and publish status and logs, and may not queue a request even for itself. One token for everything would let a station that used the wrong credential pass here and be refused by a real relay. The suite asserts that boundary against the stub rather than assuming it - a double more permissive than the thing it stands in for is worse than no double.

And running it three times only proves three passes, so each transport is also checked for teeth - in both implementations, because a second implementation of delivery is a second thing that can silently stop delivering: tp_put_log is replaced with return 0 - a delivery that claims success and does nothing, the exact shape of the original defect - and property 9 must fail. If it does not, it is reading the local file again.

blob is not conformance-tested. Its far side is an Azure storage account and there is no honest way to stand one up offline. That exclusion is asserted rather than assumed: the suite lists every transport in the toolkit and fails if one is on neither the covered list nor the excluded one, so a transport added later cannot go unnoticed.

Two honest limits

A driver skips loudly rather than passing. A driver that cannot observe the far side skips property 9 and says so. A scrollback reading as though everything was checked when half was not is the same defect as a silently skipped checksum.

Nothing here exercises a capture against a real remote machine. The behaviour that matters is what a log looks like after a round trip through someone else's terminal, and no test asserts that.

Property 2 has an upper bound, and an overloaded runner can exceed it. A 3 second gap in the source must appear as a 3 to 8 second gap in the log. The lower bound is the property; the upper one guards against stamps applied at flush rather than at read, and a badly stalled CI worker could produce a legitimate gap larger than 8 seconds. It has not yet.

The second implementation

station/powershell/caplib.psm1 is the capture in PowerShell. It exists for the estate that has no bash and will not be given any, where the alternative is not a bash station but no station. A Windows box that does have Git for Windows should keep running the bash station: one implementation is better than two wherever there is a choice.

station/powershell/run.ps1 is the runner, carrying gates 1, 2 and 4 - the same three run.sh carries, with the same exit codes. station/powershell/station.ps1 is the loop, and it carries gate 3: the station must have been started with --allow-actions. That gate cannot live in a runner, because a runner invoked by hand has no station behind it to ask.

station/powershell/lib/cancel.psm1 is the cancel, start.ps1 the preflight, and transports/{git,share}.psm1 both halves of the channel - fetching a request and publishing a status, as well as delivering the log.

Together they pass all ten properties, over both transports the implementation ships, on Windows PowerShell 5.1 and on 7, with no skips at all.

That was not always true, and the way it stopped being true is the point. p8, the cancel, was permitted to skip on Windows while Git-Bash had no setsid and the Job Object did not exist. Both stopped being so, and the permission outlived its reason - which is how a property drops out unnoticed, because an exemption reads as deliberate for ever after. It is gone, and the suite now demands an answer to every property on every edition.

The suite checks which properties skipped, not how many. A count was the first version and it was wrong on Windows; loosening it to "two or three" would have accepted a third skip anywhere, including a capture property quietly dropping out.

A cancel has to take the whole tree

A step runs terraform, which runs a provider, which runs git. Signalling only what the station started leaves all of that running while the operator is told the run was cancelled - and it carries on changing the estate.

bash gets this from setsid and a negative pid. Windows has no process groups; it has Job Objects, which are better for this. A job with JOB_OBJECT_LIMIT_KILL_ON_JOB_CLOSE terminates every process in it the moment the last handle closes, so killing the station takes the whole tree - including anything started after the kill was decided.

The fallback is not cosmetic. A hardened estate can block Add-Type, by Constrained Language Mode or by policy, and that estate is exactly the one this PowerShell station exists for. Where the P/Invoke will not compile, the cancel falls back to taskkill /T /F, which walks the child tree at the moment it runs - so a grandchild started a millisecond later can survive it. That is a real gap, it is why the Job Object is preferred rather than merely tidier, and the preflight says which of the two this machine actually gets.

The twins are compared case by case

Two of the first divergences between run.ps1 and run.sh were in a security gate, and both had the same cause: PowerShell compares case-insensitively and bash does not.

• CONFIRM=YES satisfied the action gate on PowerShell and was refused by

run.sh. A state-changing step ran on one implementation and not the other, from the same request.

• # heliograph-mode: READ-ONLY did the same. A declaration that means

different things to two readers has not declared anything.

A third: ./run.ps1 ENV resolved to the env step, because a PowerShell hashtable is case-insensitive too.

Finding those by hand does not scale and does not stay found, so the same declaration now goes through both runners and the exit codes must match. The body of each step differs - one is bash, one is PowerShell - but the header is the thing under test and it is byte-identical.

It is tested on Windows PowerShell 5.1, not only on 7. That matters more than it sounds: the driver used to prefer pwsh, so on a runner with both editions the 5.1 path was never taken - and that is exactly how a first version shipped using ProcessStartInfo.ArgumentList, which .NET Framework does not have and which would have thrown on every real 5.1 station. Windows CI now runs the suite once per edition.

Two more things are asserted that no property covers:

• Both implementations write a log of the same shape. The control side

parses these logs and it parses one format. Two captures can each pass every property and still disagree about a header. The properties assert what a log means; this asserts what it looks like.

• Every .ps1 and .psm1 in the repository parses. Nothing else in CI

reads them, so without it a syntax error ships and fails on a Windows box.

What must never appear on the PowerShell capture path

The busybox failure, respelled. Each of these collects the stream before handing it on, so every line of a block gets the timestamp of the flush:

$( ), Out-String without -Stream, Select-Object, Sort-Object, Group-Object, and -Wait.

Two traps cost real time here and are worth knowing before writing any PowerShell that reads a process:

• A scriptblock attached to OutputDataReceived has no runspace. It runs on

a threadpool thread, and PowerShell terminates the entire process with "There is no Runspace available to run scripts in this thread" - not an exception a caller can catch. A capture that crashes the station is worse than one that buffers.

• Register-ObjectEvent has a runspace but the wrong clock. Its -Action

runs when the engine gets round to it, so the stamp says when PowerShell was free rather than when the line arrived. That is the buffering defect exactly, just relocated.

Neither is that the answer, and the second attempt was wrong too. One ReadLineAsync per pipe with Task.WaitAny looks correct and is not: a line that arrives while the loop is busy with the other stream sits in a completed task, unstamped, until the loop comes back for it. Under steady output on one stream the other starves - its stamp drifts from its arrival, and the child can block writing to it.

What works is what bash does: the child merges the two streams, and the parent reads one pipe with a blocking ReadLine. cmd /d /s /c on Windows, sh -c elsewhere. The ordering is the child's own, and nothing gets between the read and the clock.

Adding an implementation

A second implementation of the capture is permitted only while it passes this suite. That is the whole rule, and it replaced a blanket prohibition: the argument against a port was always about untested drift, and the suite makes it testable.

Write a driver implementing drv_name, drv_supports, drv_capture, drv_step_file, drv_step_echo, drv_step_name, drv_bootstrap, drv_step, drv_capture_bg, drv_cancel, drv_deliver and drv_delivered, plus drv_step_privileged if the platform can simulate one and drv_teardown if the far side is a process. Then make the suite pass without touching the suite.

Nothing platform-specific belongs in the suite. Three things were in it and are not now:

• the fake id that simulated root
• the sed -u probe that decides whether a cancel can keep anything - a fact

about busybox, not about the property

• every step fixture. They were heredocs beginning #!/usr/bin/env bash, so

a PowerShell driver could not have run a single property. The suite now names a step by kind and the driver writes it; for redaction the suite hands over the lines and the driver writes a step that prints them.