DataOps-platform/docs/superpowers/specs/2026-08-19-enterprise-connector-metadata-boundary-design.md

Enterprise Connector and REST Catalog Navigation Boundary Design

Status

Approved design option: move database connector operations to Data Factory and integrate REST Catalog into the existing Metadata page.

Problem

The current 企业连接器 page is owned by 数据研发, but it combines two different responsibilities:

• database connector compatibility and collection runtime operations for Oracle and SQL Server;
• controlled REST Catalog ingestion for importing metadata from an external catalog API.

The database connector runtime belongs with production pipelines, observability, and connection-pool operations in 数据工厂. REST Catalog is not a general enterprise data connector: it imports governed metadata from a fixed HTTPS catalog contract and belongs with metadata governance.

Keeping both responsibilities in one page makes the navigation and the current product scope difficult to understand. It also conflicts with the product decision that this release exposes only database access through 企业连接器; file directories, object storage, APIs, and message systems remain future extensions.

Decisions

1. Move Enterprise Connectors to Data Factory

Move the existing enterpriseConnectors navigation record from 数据研发 to 数据工厂.

Preserve the existing route contract:

• path /data-governance/development/enterprise-connectors;
• route name enterpriseConnectors;
• component dataGovernance/development/enterpriseConnectors;
• permissions connectors:read, with actions continuing to use connectors:operate and connectors:manage;
• page and backend API compatibility.

The preserved path keeps bookmarks and direct links working. Navigation ownership, breadcrumb ancestry, and menu placement change to Data Factory.

Place 企业连接器 before 数据生产线投产. The Data Factory order becomes:

1. 企业连接器
2. 数据生产线投产
3. n8n 生产线管理（待下线）
4. n8n 工作流管理（待下线）
5. 数据可观测与事故
6. 数据库连接池管理

Existing route names and paths for the other entries remain unchanged.

2. Limit the Enterprise Connector Page to Database Access

The user-facing Enterprise Connector page displays only these current database connectors:

• Oracle Database;
• Microsoft SQL Server.

The page filters both manifests and run records to the database connector allowlist. It must not display REST Catalog manifests or runs.

Add a visible scope notice:

当前开放范围仅限数据库访问。文件目录、对象存储、API 与消息系统等来源将在后续版本扩展开发。

The existing enterprise-UAT notice remains accurate: Oracle and SQL Server have engineering compatibility evidence, while real enterprise accounts, versions, networks, and UAT are still external prerequisites.

This change does not claim that PostgreSQL and MySQL have versioned enterprise connector implementations. They remain data-source and connection-pool capabilities until separate connector manifests are delivered.

3. Integrate REST Catalog into Metadata Governance

The existing 数据研发 → 元数据 page gains two tabs:

1. 元数据管理 — the current metadata list, create, edit, review, and assistant behavior;
2. 外部目录接入 — the REST Catalog operational surface.

The new external-catalog tab is implemented as a dedicated component under the metadata feature. It exposes only the rest-catalog connector and retains:

• connector version and capability display;
• compatibility checks;
• controlled Dry-run using source_uid, credential_ref, HTTPS base URL, and allowed host;
• REST Catalog run records;
• checkpoint, cursor, attempt, error, cancel, and resume information.

It must not display Oracle or SQL Server connector manifests or runs.

The backend rest-catalog implementation, registry, runtime, persistence, API paths, and security controls remain unchanged. This is a frontend information-architecture split, not a duplicate backend implementation.

4. Reuse One Connector Operations Component

Extract the shared connector catalogue and run-table behavior from the existing page into one reusable frontend component. Page wrappers provide an explicit connector allowlist:

• Enterprise Connectors: oracle, sqlserver;
• Metadata external-catalog tab: rest-catalog.

The shared component continues to use the existing datasource connector APIs. It filters manifests and runs before rendering. Page-specific notices and REST-only configuration fields remain in the appropriate wrapper or mode so that database users are not shown REST URL fields.

This separation avoids copying run, checkpoint, cancel, resume, and permission behavior into two divergent implementations.

Permissions and Security

• Moving the navigation does not broaden access. The Data Factory parent is visible only when at least one permitted child remains after menu filtering.
• connectors:read is required to view connector manifests and runs.
• connectors:operate or connectors:manage remains required for human Dry-run, cancel, and resume actions.
• The Metadata page itself remains accessible under its existing policy. The 外部目录接入 tab is shown only to users with connectors:read; users without that permission retain normal metadata management access.
• REST Catalog continues to require HTTPS, an exact allowed host, a secret reference, bounded responses, schema validation, and no redirects.
• No page displays secret values or short-lived machine credentials.
• Human operations remain Dry-run only. Non-Dry-run connector execution remains on the machine-credential boundary.

Data and API Flow

Database connector flow

Data Factory → Enterprise Connectors
    → database connector manifest
    → source_uid / approved credential reference
    → database connection manager and pool
    → discover, snapshot, or incremental metadata result
    → run, checkpoint, evidence, and audit

REST Catalog flow

Data Research → Metadata → External Catalog Access
    → REST Catalog manifest
    → approved HTTPS catalog endpoint and secret reference
    → /v1/catalog cursor-based reads
    → metadata assets
    → run, checkpoint, evidence, and audit

Error Handling

• If manifest or run loading fails, each surface uses the existing snackbar error boundary and stops rendering stale results as current.
• Unknown connector IDs and versions continue to fail closed in the registry.
• The shared component must not fall back to showing all connectors when an allowlist is empty or malformed.
• If the user lacks connector read permission, the Metadata management tab remains usable and the external-catalog operational content is not requested.
• Existing UAT-blocked messages remain visible; no local compatibility result is promoted to enterprise production readiness.

Alternatives Rejected

Keep REST Catalog on the Enterprise Connector page

Rejected because the approved product scope says Enterprise Connectors currently exposes database access only. Showing REST Catalog would contradict that scope and blur metadata-governance ownership.

Add a separate left-navigation item for REST Catalog

Rejected because it would add another top-level Data Research entry and separate external metadata ingestion from the metadata objects it creates.

Move REST Catalog into the hidden Active Metadata page

Rejected because the capability would remain difficult to discover and would mix external catalog transport configuration with discovery-plan, lineage, and correction workflows.

Duplicate REST Catalog APIs under a metadata-specific backend namespace

Rejected because the existing connector runtime already supplies the required security, retry, checkpoint, evidence, and audit boundaries. A second backend path would create divergent governance behavior.

Test Plan

1. Add a failing navigation contract proving enterpriseConnectors exists exactly once under Data Factory and is absent from Data Research.
2. Prove its path, name, component, permission, and direct-link compatibility are unchanged.
3. Prove Data Factory ordering is stable and legacy n8n entries remain independently removable.
4. Add component contracts proving the database surface renders only Oracle and SQL Server manifests and runs.
5. Add component contracts proving the REST Catalog surface renders only rest-catalog manifests and runs.
6. Prove the Metadata page contains exactly the 元数据管理 and 外部目录接入 tabs while preserving existing metadata CRUD behavior.
7. Prove permission behavior for connectors:read, connectors:operate, and connectors:manage, including a metadata-only user who cannot load connector data.
8. Run existing WP03 connector, metadata, RBAC, Data Factory navigation, and architecture contracts.
9. Build the frontend production bundle.
10. Rebuild the local UAT frontend and verify menu ownership, tab rendering, preserved direct links, and refresh behavior in an authenticated browser session.

Rollback

Rollback is frontend-only:

• move the unchanged enterpriseConnectors navigation object back to Data Research;
• restore the combined page rendering;
• remove the Metadata external-catalog tab and shared wrapper.

No connector facts, data-source definitions, credentials, runs, checkpoints, metadata assets, database migrations, or backend API contracts are changed by this navigation split.