# Task 6 Report — Artifact handoff and governed rule evidence

Date: 2026-07-23
Branch: `codex/data-rule-execution-m3a-m5`

Lifecycle boundary: Task 6 consumes only plans already marked `published`.
It does not add, relax, or emulate the Task 7 publication gate.

## Outcome

Task 6 now provides a production-wired path from a signed Runner task to one
server-attested `rule_runs` record, optional durable violation evidence, and a
closed artifact-only handoff to the next node.

The implementation includes:

- `PostgresRuleEvidenceWriter` with canonical deployment, component, rule
  version, plan hash, DataFlow, workflow version, node, and correlation
  attestation;
- a server-derived evidence key that makes a node execution idempotent across
  different signed task tokens and Runner retries;
- reliable `running` to `success`, `failed`, `unknown`, or `cancelled`
  finalization with row counts, timings, commit outcome, and bounded public
  result;
- commit-acknowledgement rechecks for both run and sample finalization;
- a maximum 100-row violation sample, redacted before it leaves the isolated
  Polars worker and redacted again at the evidence boundary;
- a `pending` to verified MinIO upload to `ready` sample state machine with
  digest, schema hash, reference, expiry, and bounded TTL cleanup;
- no raw sample values in Runner responses or application log calls;
- correlation-scoped upstream output resolution that accepts only current,
  ready, unexpired, storage-attested artifacts;
- a closed SQL staging reference for same-source pushdown handoff instead of
  row JSON;
- Kestra single-predecessor `input_artifact` and multi-predecessor named
  `input_artifacts` expressions, with governed rule nodes receiving no
  unrelated workflow parameters;
- a top-level Runner `output_artifact` field matching the Kestra expression
  contract while retaining the full bounded result object.

## Forward-only schema change

Migration `20260723_160` extends the existing evidence tables without changing
historical migrations 110, 140, or 150.

`rule_runs` gains:

- nullable legacy-compatible `evidence_key` with a unique constraint;
- `commit_outcome`;
- bounded `public_result`;
- `failure_code` and `updated_at`.

`rule_violation_samples` gains:

- artifact and schema digests;
- `legacy`, `pending`, `ready`, and `failed` handoff states;
- failure and update fields;
- one sample per run;
- a strict 100-row ceiling and retention index.

Downgrade is deliberately rejected because deleting execution evidence would
break audit and replay guarantees.

The local Docker PostgreSQL was upgraded through the formal
`20260723_150 -> 20260723_160` Alembic path. The existing isolated old-140
migration acceptance also passes through current head.

## RED / GREEN evidence

The initial fail-first slice produced five expected failures:

- four executor evidence tests failed because `RulePlanExecutor` did not accept
  an evidence writer;
- the Kestra artifact-only test failed because downstream parameters contained
  only workflow inputs and no upstream output reference.

A second fail-first assertion proved that the Polars adapter did not expose a
bounded violation sample before the worker implementation was added.

Final verification:

- focused evidence, Runner, Polars, Kestra, migration, and real integration:
  `27 passed`;
- full suite: `586 passed, 26 skipped, 59 subtests passed`;
- Ruff across every changed Python file: `All checks passed!`;
- `git diff --check`: passed;
- Docker Compose configuration validation: passed;
- historical migration 140/150 diff check: empty.

## Real PostgreSQL and MinIO acceptance

The production-path integration uses:

- source PostgreSQL and source MySQL;
- platform PostgreSQL at migration head;
- real MinIO;
- the isolated Polars worker;
- `PostgresRulePlanRepository`;
- `PostgresArtifactResolver`;
- `PostgresRuleEvidenceWriter`;
- `RulePlanExecutor`;
- signed Runner HTTP tokens and the durable task ledger.

It proves:

1. the first node reads the PostgreSQL and MySQL artifacts, executes
   normalize/join/assert/deduplicate, publishes an output, and records one
   successful run;
2. one violating row is stored as an all-field-redacted, expiring Parquet
   sample with ready state and exact digest;
3. a second published rule node consumes the first node's exact
   `output_artifact` reference and publishes a distinct output;
4. the same correlation contains exactly two successful run records;
5. a different signed token for the same node replays the existing run and
   sample instead of executing again;
6. a failed adapter records `failed/not_committed`;
7. an uncertain write records `unknown/unknown`;
8. all test-owned PostgreSQL rows and MinIO objects are removed after
   acceptance.

## Residual boundary

Task 6 deliberately does not decide whether a compiled plan may become
published. Task 7 must require trusted generation, test, and preflight evidence
before changing plan lifecycle state. The evidence added here is the immutable
input to that gate, not a bypass around it.