# DataFlow n8n Workflow Versioning Implementation Plan

> **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:test-driven-development and superpowers:executing-plans to implement this plan task-by-task.

**Goal:** Map each governed DataFlow to multiple identifiable n8n Workflow versions while enforcing one active version per environment.

**Architecture:** Neo4j remains the DataFlow definition store; PostgreSQL owns immutable workflow version mappings and activation state. The backend coordinates n8n API calls through a state machine and outbox, with a partial unique index as the final concurrency guard.

**Tech Stack:** Flask, PostgreSQL, Neo4j, n8n REST API, pytest, Vue 2.

---

### Task 1: Create workflow version schema

**Files:**
- Create: `migrations/versions/20260716_20_workflow_versions.py`
- Create: `app/core/data_flow/workflow_models.py`
- Create: `tests/test_workflow_version_schema.py`

- [ ] Test version uniqueness and concurrent attempts to create two active rows for one `(dataflow_uid, environment)`.
- [ ] Create `dataflow_workflow_versions` with UUID ID, stable `dataflow_uid`, environment enum, monotonically increasing version number, n8n ID, immutable definition hash, status and audit fields.
- [ ] Add `UNIQUE(dataflow_uid, environment, version_no)` and the partial unique active index.
- [ ] Seed no production mappings automatically; provide an import report command for existing n8n workflows.

### Task 2: Add repository and n8n definition snapshotting

**Files:**
- Create: `app/core/data_flow/workflow_repository.py`
- Modify: `app/core/data_factory/n8n_client.py`
- Create: `tests/test_workflow_repository.py`
- Modify: `tests/test_n8n_client_config.py`

- [ ] Test creating a version from an n8n workflow, canonical JSON hashing, missing workflow, duplicate hash and redaction of credentials.
- [ ] Add n8n client methods for read/export/activate/deactivate with explicit timeouts and typed errors.
- [ ] Persist only a safe definition snapshot or hash; never store n8n credential secrets in PostgreSQL.
- [ ] Use stable DataFlow UID and reject numeric Neo4j IDs for new mappings.

### Task 3: Implement activation state machine

**Files:**
- Create: `app/core/data_flow/workflow_activation.py`
- Create: `tests/test_workflow_activation.py`

- [ ] Test draft-to-validating-to-active, old active-to-superseded, n8n failure, retry, duplicate request and two concurrent activation requests.
- [ ] Lock the environment mapping, mark the target validating, invoke n8n, then atomically switch PostgreSQL active status and append outbox events.
- [ ] If n8n succeeds but PostgreSQL finalization fails, reconcile from n8n state and correlation ID instead of blindly reactivating.
- [ ] Expose `activation_failed` with actionable error metadata that excludes API keys.

### Task 4: Add governed version APIs

**Files:**
- Modify: `app/api/data_flow/routes.py`
- Create: `app/api/data_flow/schemas.py`
- Create: `tests/test_workflow_version_api.py`

- [ ] Add list/create/detail/activate endpoints under `/api/dataflow/{dataflow_uid}/workflow-versions`.
- [ ] Require editor permission to create versions and administrator/editor-with-activation permission to activate.
- [ ] Return both DataFlow identity and n8n Workflow identity, version, environment, status and timestamps.
- [ ] Keep `/api/datafactory/workflows/*` for n8n-native diagnostics; do not rename Workflow globally.

### Task 5: Build the version management UI

**Files:**
- Modify: `frontend/src/views/dataGovernance/dataProcess/index.vue`
- Create: `frontend/src/views/dataGovernance/dataProcess/components/workflowVersions.vue`
- Modify: `frontend/src/api/dataGovernance.js`
- Create: `tests/e2e/workflow-versions.spec.js`

- [ ] Display DataFlow details separately from n8n Workflow versions and show the active version per environment.
- [ ] Require confirmation before activation and disable actions while a request is in flight.
- [ ] Show validation/activation errors and n8n execution links without exposing credentials.
- [ ] Verify viewer read-only, editor version creation and authorized activation in browser tests.

### Task 6: Add reconciliation and operational acceptance

**Files:**
- Create: `app/commands/reconcile_workflow_versions.py`
- Create: `tests/integration/test_n8n_reconciliation.py`
- Modify: `deploy/docker/README.md`

- [ ] Compare PostgreSQL active mappings with local n8n activation state and emit a report before making repairs.
- [ ] Test n8n downtime, request timeout, backend restart and out-of-band n8n deactivation.
- [ ] Run full local-stack acceptance and prove one active version per environment under concurrency.
- [ ] Regenerate OpenAPI and publish operator recovery commands.