DataOps-platform/docs/superpowers/plans/2026-07-29-wp08-quality-remediation.md

WP08 Quality Issue Remediation Implementation Plan

For Codex: Execute this plan with targeted tests only. Do not run the full regression suite.

Goal: Turn immutable WP07 device-quality violations into governed quality issues that can be assigned, remediated, independently reviewed, closed, reopened, and measured for overdue and recurrence.

Architecture: WP07 quality runs and violation samples remain immutable source evidence. WP08 adds a PostgreSQL-backed issue register, append-only remediation rounds, and append-only timeline events. An issue snapshots the bounded violation evidence so the work record remains auditable after the 30-day violation-sample retention window. State transitions are explicit and optimistic-lock protected; closure requires an accountable quality-issue reviewer who is not the remediation submitter. Statistics are derived from persisted issue history and never presented as an automated root-cause conclusion.

Technology: Flask, SQLAlchemy/PostgreSQL, Alembic, Vue 2/Vuetify, Node test runner, pytest.

---

Scope and constraints

• Implement DQA-10 issue creation/classification/impact/priority, DQA-11 assignment/remediation/review/close/reopen/overdue, and the bounded DQA-12 recurrence count/rate.
• Create issues only from persisted WP07 violation samples. Preserve source run, violation, rule, asset, field, mapping, message, and safe evidence snapshot.
• Import at most 100 violation UIDs per request. Re-importing the same unresolved recurrence identity returns the existing issue; a later violation may become a new recurrence only after the prior issue is closed.
• Use states open, assigned, in_progress, pending_review, and closed. Every mutation requires expected_version.
• Only an active platform user may be assigned. Only the current assignee or an administrator may start work and submit remediation.
• Closure requires a passed verification result from the unique accountable asset manager configured at quality_issue/DEVICE_QUALITY_ISSUES. A reviewer cannot approve their own remediation submission.
• Rejection returns the issue to in_progress; reopening a closed issue returns it to assigned when it still has an assignee.
• Store remediation summary and up to 20 bounded evidence references. Optional verification_run_uid links a real WP07 run, but no absence-in-a-sample claim is treated as automatic proof.
• Overdue is derived from due_at < now while not closed. Recurrence is derived from the deterministic identity of rule, canonical asset, and field; it is not AI root-cause analysis.
• Do not add generic ticketing, notifications, external workflow integration, automatic source repair, automatic quality re-execution, SLA escalation, or predictive maintenance.

Task 1: Domain lifecycle

Files:

• Create: app/core/data_research/quality_issues.py
• Modify: app/core/data_research/errors.py
• Test: tests/data_research/test_quality_issues.py

1. Write failing tests for violation import, open-issue deduplication, assignment, assignee-only work, submission, independent accountable review, rejection, close, reopen, stale versions, overdue and recurrence.
2. Add immutable record types, bounded input validation, deterministic recurrence identity, and the transition service.
3. Run only tests/data_research/test_quality_issues.py.

Task 2: PostgreSQL persistence

Files:

• Create: app/core/data_research/quality_issue_repository.py
• Modify: app/models/data_research.py
• Create: migrations/versions/20260729_340_quality_issue_remediation.py
• Test: tests/integration/test_quality_issue_postgres.py

1. Write failing integration tests for schema constraints, open-issue deduplication, append-only rounds/timeline, filtering, overdue, and recurrence aggregation.
2. Add issue, remediation-round, and timeline models with indexes and checks.
3. Implement repository locking, active-user validation, source evidence lookup, pagination, and statistics queries.
4. Apply migration 340 to the local PostgreSQL test database and run only the WP08 integration test.

Task 3: Governed API

Files:

• Modify: app/api/data_development/routes.py
• Modify: app/core/system/permissions.py
• Modify: tests/test_permission_matrix.py
• Create: tests/data_research/test_quality_issue_api.py

1. Write failing permission and route tests for read, import, assign, start, submit, review, reopen, timeline, and statistics.
2. Add quality-issues:edit and quality-issues:review. Editors receive edit; administrators receive edit/review; reads remain governance-read.
3. Bind review authorization to the unique accountable quality_issue/DEVICE_QUALITY_ISSUES asset manager.
4. Expose bounded list/detail/statistics/timeline reads and explicit transition writes below /api/development/v1/device-quality/issues.
5. Run only WP08 API and permission tests.

Task 4: Quality issue workbench

Files:

• Modify: frontend/src/api/dataDevelopment.js
• Modify: frontend/src/views/dataGovernance/development/deviceQuality.vue
• Modify: frontend/src/views/dataGovernance/development/deviceQualityModel.js
• Modify: frontend/tests/device-quality-model.test.mjs
• Create: tests/data_research/test_quality_issue_frontend_contract.py

1. Write failing model and source-contract tests for permissions, labels, overdue/recurrence presentation, API bindings, issue table, timeline, assignment, submission, review, and reopen actions.
2. Add an issue-register section to the existing device-quality workbench so violations and remediation remain in one governed context.
3. Allow eligible users to select WP07 violation samples and create issues, then operate the explicit lifecycle through constrained dialogs.
4. Run only the device-quality Node model test, WP08 frontend contract, ESLint on touched frontend files, and a production build.

Task 5: Contracts, ledger and targeted acceptance

Files:

• Modify: docs/architecture/OPENAPI.yaml
• Modify: docs/architecture/DATA_MODEL.md
• Modify: docs/FUNCTION_MODULE_CENSUS_20260726.md
• Modify: docs/DATAOPS_PHASE1_3_MONTH_WORK_PLAN_20260729.md
• Modify: matching files under deployment/app/

1. Generate OpenAPI from current routes and record the new operation count.
2. Copy changed runtime source into deployment/app/ and verify byte parity.
3. Update DQA-10 through DQA-12 and WP-08 maturity honestly: engineering complete but enterprise quality data and operating-role acceptance still pending.
4. Run only touched Python lint, WP08 unit/integration/API/permission/contract/architecture tests, frontend model/lint/build, migration head check, and one local browser lifecycle using real PostgreSQL.
5. Review the diff, commit WP08 on the current branch, and leave the worktree clean.