DataOps-platform/docs/superpowers/plans/2026-07-29-wp04-device-asset-catalog.md

WP-04 Device Asset Catalog Implementation Plan

For agentic workers: REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (- [ ]) syntax for tracking.

Goal: Build a canonical device asset catalog with stable platform UIDs, immutable versions, source-system code mappings, source timestamps, search, and end-to-end traceability for devices, components, measurement points, alarms, and maintenance records.

Architecture: PostgreSQL remains authoritative for canonical device assets, source mappings, and immutable versions. A normalized import service accepts bounded records from the existing ingestion boundary, never reads credentials, and never merges two source identities automatically; later WP-06 matching may explicitly link identities after review. The Vue 2 catalog is a read and trace interface, while write access is exposed through a dedicated permission-controlled API for adapters and data engineers.

Tech Stack: Flask, SQLAlchemy, PostgreSQL JSONB, Alembic, Vue 2, Vuetify, pytest.

Global Constraints

• Supported asset types are device, component, measurement_point, alarm, and maintenance_record.
• Stable asset identity is preserved by the unique tuple (source_uid, source_entity, asset_type, source_code).
• Reimporting an unchanged record updates source observation time but does not create a new asset version.
• A changed normalized record creates exactly one immutable next version.
• WP-04 does not automatically merge identities across sources; matching, review, merge, and rollback belong to WP-06.
• Credentials, connection strings, passwords, API keys, and tokens are rejected from asset attributes and never returned.
• Imports contain at most 500 records and each normalized attributes object is bounded.
• Viewer can read; editor and admin can import through device-assets:edit.
• Real enterprise device and maintenance data remains an external acceptance gate.
• Validation follows the user-approved rule: only changed functions, routes, migration, frontend files, and the local WP-04 browser flow are tested; no full repository regression is run.
• Work continues on codex/dataops-phase1-equipment-governance; no push or remote deployment is authorized.

---

Task 1: Canonical Asset Identity and Version Semantics

Files:

• Create: app/core/data_research/device_assets.py
• Modify: app/core/data_research/errors.py
• Test: tests/data_research/test_device_assets.py

Interfaces:

• Consumes: repository.source_is_active(source_uid), repository.find_mapping(...), repository.create_asset(...), repository.update_asset(...), and repository.touch_mapping(...).

• Produces: DeviceAssetService.import_records(payload, actor_uid), DeviceAssetRecord, DeviceAssetMappingRecord, DeviceAssetVersionRecord, and DeviceAssetImportResult.

• [x] Step 1: Write failing domain tests

Cover stable UID reuse, unchanged reimport without a new version, changed record with exactly one new version, five allowed asset types, inactive source rejection, duplicate source code rejection inside one request, bounded batch size, required source entity/code/name, attributes size, secret-key rejection, and no caller-supplied asset UID.

• Step 2: Verify RED

Run:

PYTHONPATH=. .venv/bin/pytest -q tests/data_research/test_device_assets.py

Expected: failure because the device asset service does not exist.

• Step 3: Implement minimal domain service

Normalize fields, compute a deterministic SHA-256 content hash, create a UUIDv7 only for a new source identity, preserve the existing UID on reimport, and classify every input as created, updated, or unchanged.

• Step 4: Verify GREEN

Run the Task 1 test command and expect all tests to pass.

Task 2: PostgreSQL Persistence and Migration

Files:

• Create: app/core/data_research/device_asset_repository.py
• Modify: app/models/data_research.py
• Create: migrations/versions/20260729_290_device_asset_catalog.py
• Modify: tests/test_database_migrations.py
• Create: tests/integration/test_device_asset_postgres.py

Interfaces:

• Produces: SqlAlchemyDeviceAssetRepository, tables device_assets, device_asset_source_mappings, and device_asset_versions.

• [x] Step 1: Write failing persistence tests

Add a migration contract test and a real PostgreSQL integration test covering source validation, one canonical asset, one unique mapping, immutable sequential versions, source timestamp refresh, search by source code/name/location/responsible person, type/status/source filters, and detail traceability.

• Step 2: Verify RED

Run:

PYTHONPATH=. .venv/bin/pytest -q \
  tests/test_database_migrations.py::test_device_asset_catalog_migration_is_versioned_and_traceable \
  tests/integration/test_device_asset_postgres.py

Expected: failure because the migration, models, and SQL repository do not exist.

• Step 3: Implement schema and SQL repository

Use unique constraints for source identity and asset version, row locking for updates, indexed search fields, source and actor timestamps, JSONB snapshots, and data-preserving downgrade behavior.

• Step 4: Upgrade local database and verify GREEN

Run:

docker exec dataops-test-backend-1 alembic upgrade head
PYTHONPATH=. .venv/bin/pytest -q \
  tests/test_database_migrations.py::test_device_asset_catalog_migration_is_versioned_and_traceable \
  tests/integration/test_device_asset_postgres.py

Expected: migration head 20260729_290; tests pass.

Task 3: Permission-Controlled Catalog API

Files:

• Modify: app/api/data_development/routes.py
• Modify: app/core/system/permissions.py
• Modify: tests/data_research/test_development_api.py
• Modify: tests/test_permission_matrix.py

Interfaces:

• Produces: POST /api/development/v1/device-assets/import, GET /api/development/v1/device-assets, GET /api/development/v1/device-assets/{asset_uid}, and GET /api/development/v1/device-assets/{asset_uid}/versions.

• [x] Step 1: Write failing API and permission tests

Cover viewer read, viewer import denial, editor import, secret-free responses, search/filter pagination, detail mappings, version history, malformed pagination, and not-found errors.

• Step 2: Verify RED

Run:

PYTHONPATH=. .venv/bin/pytest -q \
  tests/data_research/test_development_api.py \
  tests/test_permission_matrix.py::test_data_development_paths_have_specific_write_policies

Expected: failure because the routes and permission do not exist.

• Step 3: Implement API and permission boundary

Add device-assets:edit to editor/admin, keep reads on governance:read, bound page to at least 1 and page size to 1–100, serialize source mappings and versions without secret material, and return domain errors through the existing data-research error envelope.

• Step 4: Verify GREEN

Run the Task 3 test command and expect all tests to pass.

Task 4: Device Asset Catalog UI and Delivery Evidence

Files:

• Modify: frontend/src/api/dataDevelopment.js
• Modify: frontend/src/router/routes.js
• Modify: frontend/src/views/dataGovernance/development/index.vue
• Create: frontend/src/views/dataGovernance/development/deviceAssets.vue
• Modify: docs/architecture/OPENAPI.yaml
• Modify: docs/architecture/DATA_MODEL.md
• Modify: docs/FUNCTION_MODULE_CENSUS_20260726.md
• Modify: docs/DATAOPS_PHASE1_3_MONTH_WORK_PLAN_20260729.md
• Modify: deployment/app/ only for the WP-04 backend subset.

Interfaces:

• Produces: /data-governance/development/device-assets with keyword/type/status filters, stable UID display, source-code chips, source timestamps, responsibility fields, detail traceability, and immutable version history.

• [x] Step 1: Implement the catalog UI

Add API functions, hidden child route, research-center entry, responsive table, detail dialog, empty/error/loading states, source mappings, and version timeline. Do not add a generic analysis-development or raw-SQL interface.

• Step 2: Regenerate contracts and update ledgers

Run:

.venv/bin/python scripts/generate_openapi.py

Record engineering completion separately from enterprise data acceptance and keep WP-06 automatic matching outside WP-04.

• Step 3: Run targeted verification

Run only:

PYTHONPATH=. .venv/bin/pytest -q \
  tests/data_research/test_device_assets.py \
  tests/integration/test_device_asset_postgres.py \
  tests/data_research/test_development_api.py \
  tests/test_permission_matrix.py::test_data_development_paths_have_specific_write_policies \
  tests/test_database_migrations.py::test_device_asset_catalog_migration_is_versioned_and_traceable \
  tests/test_architecture_artifacts.py
npx eslint src/api/dataDevelopment.js src/router/routes.js \
  src/views/dataGovernance/development/index.vue \
  src/views/dataGovernance/development/deviceAssets.vue

Then rebuild only the affected local backend/frontend services and exercise import, search, detail, source mapping, and version history in the browser. Run git diff --check.

• Step 4: Commit

Create one independently reversible WP-04 engineering commit. Do not push.

Implementation Receipt

• Domain RED/GREEN covered stable UID reuse, unchanged import, immutable next version, five object types, secret rejection, bounded inputs, source/filter validation, source mappings, and timezone-aware catalog timestamps.
• PostgreSQL migration 20260729_290 is the local head; the real PostgreSQL integration test covers source identity, search, filters, detail, and version history.
• Four device asset endpoints use governance:read for reads and device-assets:edit for import. OpenAPI now inventories 151 operations.
• The Vue catalog, hidden child route, research-center entry, and API client passed changed-file ESLint; affected local backend/frontend images built successfully.
• Browser acceptance retained EQ-WP04-DEMO: source-code search returned one stable asset at V2, detail showed source mapping plus V1/V2 locations and correct China timestamps, with zero browser console errors.
• Enterprise device and maintenance records remain the only WP-04 acceptance gate. Cross-source matching, merge, and rollback remain WP-06.