|
@@ -0,0 +1,585 @@
|
|
|
|
|
+# DataOps Platform 生产环境部署与运维手册
|
|
|
|
|
+
|
|
|
|
|
+## 目录
|
|
|
|
|
+
|
|
|
|
|
+1. [系统概述](#1-系统概述)
|
|
|
|
|
+2. [服务器环境](#2-服务器环境)
|
|
|
|
|
+3. [目录结构](#3-目录结构)
|
|
|
|
|
+4. [配置参数](#4-配置参数)
|
|
|
|
|
+5. [虚拟环境管理](#5-虚拟环境管理)
|
|
|
|
|
+6. [服务管理](#6-服务管理)
|
|
|
|
|
+7. [日志管理](#7-日志管理)
|
|
|
|
|
+8. [常见问题排查](#8-常见问题排查)
|
|
|
|
|
+9. [备份与恢复](#9-备份与恢复)
|
|
|
|
|
+10. [API 接口文档](#10-api-接口文档)
|
|
|
|
|
+
|
|
|
|
|
+---
|
|
|
|
|
+
|
|
|
|
|
+## 1. 系统概述
|
|
|
|
|
+
|
|
|
|
|
+### 1.1 项目简介
|
|
|
|
|
+
|
|
|
|
|
+DataOps Platform 是一个基于 Flask 的数据运营平台,提供数据管理、处理和分析功能。平台集成了 Neo4j 图数据库进行关系管理,并支持 n8n 工作流自动化。
|
|
|
|
|
+
|
|
|
|
|
+### 1.2 技术栈
|
|
|
|
|
+
|
|
|
|
|
+| 组件 | 技术 | 版本 |
|
|
|
|
|
+|------|------|------|
|
|
|
|
|
+| Web 框架 | Flask | 2.3.3 |
|
|
|
|
|
+| ORM | SQLAlchemy | 2.0.23 |
|
|
|
|
|
+| 关系数据库 | PostgreSQL | 14+ |
|
|
|
|
|
+| 图数据库 | Neo4j | 5.x |
|
|
|
|
|
+| 对象存储 | MinIO | - |
|
|
|
|
|
+| 工作流引擎 | n8n | 1.116.2 |
|
|
|
|
|
+| WSGI 服务器 | Gunicorn | 21.2.0 |
|
|
|
|
|
+| 进程管理 | Supervisor | - |
|
|
|
|
|
+| 反向代理 | Nginx | - |
|
|
|
|
|
+
|
|
|
|
|
+### 1.3 访问地址
|
|
|
|
|
+
|
|
|
|
|
+| 服务 | 地址 |
|
|
|
|
|
+|------|------|
|
|
|
|
|
+| **DataOps API** | `https://company.citupro.com:18183` |
|
|
|
|
|
+| **n8n 工作流** | `https://n8n.citupro.com` |
|
|
|
|
|
+| **Neo4j Browser** | `http://192.168.3.143:7474` |
|
|
|
|
|
+| **MinIO Console** | `http://192.168.3.143:9000` |
|
|
|
|
|
+
|
|
|
|
|
+---
|
|
|
|
|
+
|
|
|
|
|
+## 2. 服务器环境
|
|
|
|
|
+
|
|
|
|
|
+### 2.1 服务器信息
|
|
|
|
|
+
|
|
|
|
|
+| 项目 | 值 |
|
|
|
|
|
+|------|------|
|
|
|
|
|
+| 操作系统 | Ubuntu 20.04/22.04 LTS |
|
|
|
|
|
+| 用户 | ubuntu |
|
|
|
|
|
+| Python 版本 | 3.8+ |
|
|
|
|
|
+| 服务器 IP | 192.168.3.143 |
|
|
|
|
|
+
|
|
|
|
|
+### 2.2 依赖服务
|
|
|
|
|
+
|
|
|
|
|
+确保以下服务已安装并运行:
|
|
|
|
|
+
|
|
|
|
|
+```bash
|
|
|
|
|
+# 检查 Python 版本
|
|
|
|
|
+python3 --version
|
|
|
|
|
+
|
|
|
|
|
+# 检查 Supervisor
|
|
|
|
|
+sudo systemctl status supervisor
|
|
|
|
|
+
|
|
|
|
|
+# 检查 Nginx
|
|
|
|
|
+sudo systemctl status nginx
|
|
|
|
|
+
|
|
|
|
|
+# 检查 PostgreSQL 连接
|
|
|
|
|
+psql -h 192.168.3.143 -U postgres -d dataops -c "SELECT 1"
|
|
|
|
|
+
|
|
|
|
|
+# 检查 Neo4j 连接
|
|
|
|
|
+curl http://192.168.3.143:7474
|
|
|
|
|
+```
|
|
|
|
|
+
|
|
|
|
|
+---
|
|
|
|
|
+
|
|
|
|
|
+## 3. 目录结构
|
|
|
|
|
+
|
|
|
|
|
+### 3.1 安装路径
|
|
|
|
|
+
|
|
|
|
|
+```
|
|
|
|
|
+/opt/dataops-platform/ # 应用根目录
|
|
|
|
|
+├── app/ # 应用代码
|
|
|
|
|
+│ ├── api/ # API 路由
|
|
|
|
|
+│ │ ├── business_domain/ # 业务领域 API
|
|
|
|
|
+│ │ ├── data_factory/ # n8n 工作流 API
|
|
|
|
|
+│ │ ├── data_flow/ # 数据流 API
|
|
|
|
|
+│ │ ├── data_interface/ # 数据接口 API
|
|
|
|
|
+│ │ ├── data_source/ # 数据源 API
|
|
|
|
|
+│ │ ├── graph/ # 图数据 API
|
|
|
|
|
+│ │ ├── meta_data/ # 元数据 API
|
|
|
|
|
+│ │ └── system/ # 系统 API
|
|
|
|
|
+│ ├── config/ # 配置文件
|
|
|
|
|
+│ │ ├── config.py # 主配置
|
|
|
|
|
+│ │ └── cors.py # CORS 配置
|
|
|
|
|
+│ ├── core/ # 核心业务逻辑
|
|
|
|
|
+│ ├── models/ # 数据模型
|
|
|
|
|
+│ ├── scripts/ # 数据库脚本
|
|
|
|
|
+│ └── services/ # 共享服务
|
|
|
|
|
+├── database/ # SQL 脚本
|
|
|
|
|
+├── docs/ # 文档
|
|
|
|
|
+├── logs/ # 应用日志
|
|
|
|
|
+├── scripts/ # 运维脚本
|
|
|
|
|
+│ ├── deploy_dataops.sh # 部署脚本
|
|
|
|
|
+│ ├── start_dataops.sh # 启动脚本
|
|
|
|
|
+│ ├── stop_dataops.sh # 停止脚本
|
|
|
|
|
+│ └── restart_dataops.sh # 重启脚本
|
|
|
|
|
+├── tests/ # 测试文件
|
|
|
|
|
+├── venv/ # Python 虚拟环境
|
|
|
|
|
+├── application.py # 应用入口(开发用)
|
|
|
|
|
+├── wsgi.py # WSGI 入口(生产用)
|
|
|
|
|
+├── requirements.txt # Python 依赖
|
|
|
|
|
+└── gunicorn.conf.py # Gunicorn 配置(可选)
|
|
|
|
|
+```
|
|
|
|
|
+
|
|
|
|
|
+### 3.2 日志路径
|
|
|
|
|
+
|
|
|
|
|
+| 日志类型 | 路径 |
|
|
|
|
|
+|----------|------|
|
|
|
|
|
+| 应用日志 | `/opt/dataops-platform/flask_production.log` |
|
|
|
|
|
+| Supervisor 日志 | `/var/log/supervisor/dataops-platform.log` |
|
|
|
|
|
+| Nginx 访问日志 | `/var/log/nginx/access.log` |
|
|
|
|
|
+| Nginx 错误日志 | `/var/log/nginx/error.log` |
|
|
|
|
|
+
|
|
|
|
|
+### 3.3 配置文件路径
|
|
|
|
|
+
|
|
|
|
|
+| 配置 | 路径 |
|
|
|
|
|
+|------|------|
|
|
|
|
|
+| Supervisor | `/etc/supervisor/conf.d/dataops-platform.conf` |
|
|
|
|
|
+| Nginx | `/etc/nginx/sites-enabled/dataops-platform.conf` |
|
|
|
|
|
+| 应用配置 | `/opt/dataops-platform/app/config/config.py` |
|
|
|
|
|
+
|
|
|
|
|
+---
|
|
|
|
|
+
|
|
|
|
|
+## 4. 配置参数
|
|
|
|
|
+
|
|
|
|
|
+### 4.1 应用配置 (`app/config/config.py`)
|
|
|
|
|
+
|
|
|
|
|
+#### 生产环境配置 (ProductionConfig)
|
|
|
|
|
+
|
|
|
|
|
+```python
|
|
|
|
|
+# Flask 配置
|
|
|
|
|
+FLASK_ENV = "production"
|
|
|
|
|
+DEBUG = False
|
|
|
|
|
+PORT = 80
|
|
|
|
|
+
|
|
|
|
|
+# PostgreSQL 数据库
|
|
|
|
|
+SQLALCHEMY_DATABASE_URI = "postgresql://postgres:dataOps@192.168.3.143:5432/dataops"
|
|
|
|
|
+
|
|
|
|
|
+# Neo4j 图数据库
|
|
|
|
|
+NEO4J_URI = "bolt://192.168.3.143:7687"
|
|
|
|
|
+NEO4J_HTTP_URI = "http://192.168.3.143:7474"
|
|
|
|
|
+NEO4J_USER = "neo4j"
|
|
|
|
|
+NEO4J_PASSWORD = "cituneo4j"
|
|
|
|
|
+
|
|
|
|
|
+# MinIO 对象存储
|
|
|
|
|
+MINIO_HOST = "192.168.3.143:9000"
|
|
|
|
|
+MINIO_USER = "citu-dataops-acc-key"
|
|
|
|
|
+MINIO_PASSWORD = "citu-dataops-secret-key"
|
|
|
|
|
+MINIO_BUCKET = "dataops-bucket"
|
|
|
|
|
+
|
|
|
|
|
+# n8n 工作流引擎
|
|
|
|
|
+N8N_API_URL = "https://n8n.citupro.com"
|
|
|
|
|
+N8N_API_KEY = "<JWT_TOKEN>"
|
|
|
|
|
+N8N_API_TIMEOUT = 30
|
|
|
|
|
+
|
|
|
|
|
+# LLM 配置 (阿里云通义千问)
|
|
|
|
|
+LLM_BASE_URL = "https://dashscope.aliyuncs.com/compatible-mode/v1"
|
|
|
|
|
+LLM_MODEL_NAME = "qwen-turbo"
|
|
|
|
|
+LLM_API_KEY = "<API_KEY>"
|
|
|
|
|
+
|
|
|
|
|
+# 文件路径
|
|
|
|
|
+UPLOAD_BASE_PATH = "/data/upload"
|
|
|
|
|
+ARCHIVE_BASE_PATH = "/data/archive"
|
|
|
|
|
+
|
|
|
|
|
+# 日志配置
|
|
|
|
|
+LOG_LEVEL = "INFO"
|
|
|
|
|
+LOG_FILE = "flask_production.log"
|
|
|
|
|
+LOG_TO_CONSOLE = False
|
|
|
|
|
+```
|
|
|
|
|
+
|
|
|
|
|
+### 4.2 Supervisor 配置
|
|
|
|
|
+
|
|
|
|
|
+**文件**: `/etc/supervisor/conf.d/dataops-platform.conf`
|
|
|
|
|
+
|
|
|
|
|
+```ini
|
|
|
|
|
+[program:dataops-platform]
|
|
|
|
|
+command=/opt/dataops-platform/venv/bin/gunicorn --bind 0.0.0.0:5500 --workers 4 --timeout 120 wsgi:application
|
|
|
|
|
+directory=/opt/dataops-platform
|
|
|
|
|
+user=ubuntu
|
|
|
|
|
+autostart=true
|
|
|
|
|
+autorestart=true
|
|
|
|
|
+redirect_stderr=true
|
|
|
|
|
+stdout_logfile=/var/log/supervisor/dataops-platform.log
|
|
|
|
|
+stdout_logfile_maxbytes=50MB
|
|
|
|
|
+stdout_logfile_backups=5
|
|
|
|
|
+environment=FLASK_ENV=production,PATH="/opt/dataops-platform/venv/bin"
|
|
|
|
|
+```
|
|
|
|
|
+
|
|
|
|
|
+**配置说明**:
|
|
|
|
|
+
|
|
|
|
|
+| 参数 | 值 | 说明 |
|
|
|
|
|
+|------|------|------|
|
|
|
|
|
+| `--bind` | `0.0.0.0:5500` | 监听地址和端口 |
|
|
|
|
|
+| `--workers` | `4` | Worker 进程数 |
|
|
|
|
|
+| `--timeout` | `120` | 请求超时时间(秒) |
|
|
|
|
|
+| `autostart` | `true` | 系统启动时自动启动 |
|
|
|
|
|
+| `autorestart` | `true` | 进程异常退出时自动重启 |
|
|
|
|
|
+
|
|
|
|
|
+### 4.3 环境变量(可选)
|
|
|
|
|
+
|
|
|
|
|
+可以通过环境变量覆盖默认配置:
|
|
|
|
|
+
|
|
|
|
|
+```bash
|
|
|
|
|
+export FLASK_ENV=production
|
|
|
|
|
+export DATABASE_URL=postgresql://user:pass@host:5432/db
|
|
|
|
|
+export NEO4J_URI=bolt://host:7687
|
|
|
|
|
+export NEO4J_PASSWORD=password
|
|
|
|
|
+export N8N_API_KEY=your_api_key
|
|
|
|
|
+export LLM_API_KEY=your_llm_api_key
|
|
|
|
|
+```
|
|
|
|
|
+
|
|
|
|
|
+---
|
|
|
|
|
+
|
|
|
|
|
+## 5. 虚拟环境管理
|
|
|
|
|
+
|
|
|
|
|
+### 5.1 虚拟环境路径
|
|
|
|
|
+
|
|
|
|
|
+```
|
|
|
|
|
+/opt/dataops-platform/venv/
|
|
|
|
|
+```
|
|
|
|
|
+
|
|
|
|
|
+### 5.2 激活虚拟环境
|
|
|
|
|
+
|
|
|
|
|
+```bash
|
|
|
|
|
+cd /opt/dataops-platform
|
|
|
|
|
+source venv/bin/activate
|
|
|
|
|
+```
|
|
|
|
|
+
|
|
|
|
|
+### 5.3 查看已安装的包
|
|
|
|
|
+
|
|
|
|
|
+```bash
|
|
|
|
|
+source venv/bin/activate
|
|
|
|
|
+pip list
|
|
|
|
|
+```
|
|
|
|
|
+
|
|
|
|
|
+### 5.4 安装/更新依赖
|
|
|
|
|
+
|
|
|
|
|
+```bash
|
|
|
|
|
+source venv/bin/activate
|
|
|
|
|
+pip install -r requirements.txt
|
|
|
|
|
+```
|
|
|
|
|
+
|
|
|
|
|
+### 5.5 重建虚拟环境
|
|
|
|
|
+
|
|
|
|
|
+如果虚拟环境损坏,可以重建:
|
|
|
|
|
+
|
|
|
|
|
+```bash
|
|
|
|
|
+cd /opt/dataops-platform
|
|
|
|
|
+
|
|
|
|
|
+# 删除旧的虚拟环境
|
|
|
|
|
+rm -rf venv
|
|
|
|
|
+
|
|
|
|
|
+# 创建新的虚拟环境
|
|
|
|
|
+python3 -m venv venv
|
|
|
|
|
+
|
|
|
|
|
+# 激活并安装依赖
|
|
|
|
|
+source venv/bin/activate
|
|
|
|
|
+pip install --upgrade pip
|
|
|
|
|
+pip install -r requirements.txt
|
|
|
|
|
+
|
|
|
|
|
+# 验证安装
|
|
|
|
|
+python -c "from app import create_app; app = create_app(); print('成功')"
|
|
|
|
|
+```
|
|
|
|
|
+
|
|
|
|
|
+### 5.6 主要依赖包
|
|
|
|
|
+
|
|
|
|
|
+| 包名 | 版本 | 用途 |
|
|
|
|
|
+|------|------|------|
|
|
|
|
|
+| Flask | 2.3.3 | Web 框架 |
|
|
|
|
|
+| Flask-SQLAlchemy | 3.1.1 | ORM 扩展 |
|
|
|
|
|
+| SQLAlchemy | 2.0.23 | ORM |
|
|
|
|
|
+| psycopg2-binary | 2.9.9 | PostgreSQL 驱动 |
|
|
|
|
|
+| neo4j | 5.26.0 | Neo4j 驱动 |
|
|
|
|
|
+| minio | 7.2.10 | MinIO 客户端 |
|
|
|
|
|
+| openai | 1.58.1 | LLM 客户端 |
|
|
|
|
|
+| gunicorn | 21.2.0 | WSGI 服务器 |
|
|
|
|
|
+| pandas | 2.0.3 | 数据处理 |
|
|
|
|
|
+| loguru | 0.7.2 | 日志 |
|
|
|
|
|
+
|
|
|
|
|
+---
|
|
|
|
|
+
|
|
|
|
|
+## 6. 服务管理
|
|
|
|
|
+
|
|
|
|
|
+### 6.1 使用运维脚本
|
|
|
|
|
+
|
|
|
|
|
+```bash
|
|
|
|
|
+cd /opt/dataops-platform
|
|
|
|
|
+
|
|
|
|
|
+# 启动服务
|
|
|
|
|
+sudo ./scripts/start_dataops.sh
|
|
|
|
|
+
|
|
|
|
|
+# 停止服务
|
|
|
|
|
+sudo ./scripts/stop_dataops.sh
|
|
|
|
|
+
|
|
|
|
|
+# 重启服务
|
|
|
|
|
+sudo ./scripts/restart_dataops.sh
|
|
|
|
|
+```
|
|
|
|
|
+
|
|
|
|
|
+### 6.2 使用 Supervisor 命令
|
|
|
|
|
+
|
|
|
|
|
+```bash
|
|
|
|
|
+# 查看状态
|
|
|
|
|
+sudo supervisorctl status dataops-platform
|
|
|
|
|
+
|
|
|
|
|
+# 启动服务
|
|
|
|
|
+sudo supervisorctl start dataops-platform
|
|
|
|
|
+
|
|
|
|
|
+# 停止服务
|
|
|
|
|
+sudo supervisorctl stop dataops-platform
|
|
|
|
|
+
|
|
|
|
|
+# 重启服务
|
|
|
|
|
+sudo supervisorctl restart dataops-platform
|
|
|
|
|
+
|
|
|
|
|
+# 重新加载配置
|
|
|
|
|
+sudo supervisorctl reread
|
|
|
|
|
+sudo supervisorctl update
|
|
|
|
|
+
|
|
|
|
|
+# 查看实时日志
|
|
|
|
|
+sudo supervisorctl tail -f dataops-platform
|
|
|
|
|
+```
|
|
|
|
|
+
|
|
|
|
|
+### 6.3 健康检查
|
|
|
|
|
+
|
|
|
|
|
+```bash
|
|
|
|
|
+# 本地检查
|
|
|
|
|
+curl http://127.0.0.1:5500/api/system/health
|
|
|
|
|
+
|
|
|
|
|
+# 外部检查
|
|
|
|
|
+curl https://company.citupro.com:18183/api/system/health
|
|
|
|
|
+```
|
|
|
|
|
+
|
|
|
|
|
+**正常响应**:
|
|
|
|
|
+
|
|
|
|
|
+```json
|
|
|
|
|
+{
|
|
|
|
|
+ "code": 200,
|
|
|
|
|
+ "message": "操作成功",
|
|
|
|
|
+ "data": {
|
|
|
|
|
+ "status": "healthy",
|
|
|
|
|
+ "components": {
|
|
|
|
|
+ "database": true,
|
|
|
|
|
+ "neo4j": true,
|
|
|
|
|
+ "environment": "production",
|
|
|
|
|
+ "platform": "linux"
|
|
|
|
|
+ }
|
|
|
|
|
+ }
|
|
|
|
|
+}
|
|
|
|
|
+```
|
|
|
|
|
+
|
|
|
|
|
+### 6.4 代码更新流程
|
|
|
|
|
+
|
|
|
|
|
+```bash
|
|
|
|
|
+cd /opt/dataops-platform
|
|
|
|
|
+
|
|
|
|
|
+# 1. 拉取最新代码(如果使用 Git)
|
|
|
|
|
+git pull origin main
|
|
|
|
|
+
|
|
|
|
|
+# 2. 激活虚拟环境并更新依赖(如有变更)
|
|
|
|
|
+source venv/bin/activate
|
|
|
|
|
+pip install -r requirements.txt
|
|
|
|
|
+
|
|
|
|
|
+# 3. 重启服务
|
|
|
|
|
+sudo supervisorctl restart dataops-platform
|
|
|
|
|
+
|
|
|
|
|
+# 4. 验证服务状态
|
|
|
|
|
+sudo supervisorctl status dataops-platform
|
|
|
|
|
+curl http://127.0.0.1:5500/api/system/health
|
|
|
|
|
+```
|
|
|
|
|
+
|
|
|
|
|
+---
|
|
|
|
|
+
|
|
|
|
|
+## 7. 日志管理
|
|
|
|
|
+
|
|
|
|
|
+### 7.1 查看日志
|
|
|
|
|
+
|
|
|
|
|
+```bash
|
|
|
|
|
+# 查看应用日志
|
|
|
|
|
+tail -f /opt/dataops-platform/flask_production.log
|
|
|
|
|
+
|
|
|
|
|
+# 查看 Supervisor 日志
|
|
|
|
|
+tail -f /var/log/supervisor/dataops-platform.log
|
|
|
|
|
+
|
|
|
|
|
+# 查看最近 100 行日志
|
|
|
|
|
+tail -100 /var/log/supervisor/dataops-platform.log
|
|
|
|
|
+
|
|
|
|
|
+# 搜索错误日志
|
|
|
|
|
+grep -i "error" /var/log/supervisor/dataops-platform.log
|
|
|
|
|
+```
|
|
|
|
|
+
|
|
|
|
|
+### 7.2 日志级别
|
|
|
|
|
+
|
|
|
|
|
+生产环境默认日志级别为 `INFO`,如需调试可临时修改为 `DEBUG`:
|
|
|
|
|
+
|
|
|
|
|
+```python
|
|
|
|
|
+# app/config/config.py
|
|
|
|
|
+LOG_LEVEL = "DEBUG" # 改为 DEBUG
|
|
|
|
|
+```
|
|
|
|
|
+
|
|
|
|
|
+修改后重启服务生效。
|
|
|
|
|
+
|
|
|
|
|
+### 7.3 日志轮转
|
|
|
|
|
+
|
|
|
|
|
+Supervisor 自动管理日志轮转:
|
|
|
|
|
+- 最大文件大小:50MB
|
|
|
|
|
+- 保留备份数:5
|
|
|
|
|
+
|
|
|
|
|
+---
|
|
|
|
|
+
|
|
|
|
|
+## 8. 常见问题排查
|
|
|
|
|
+
|
|
|
|
|
+### 8.1 服务无法启动
|
|
|
|
|
+
|
|
|
|
|
+**检查步骤**:
|
|
|
|
|
+
|
|
|
|
|
+```bash
|
|
|
|
|
+# 1. 检查 Supervisor 状态
|
|
|
|
|
+sudo supervisorctl status dataops-platform
|
|
|
|
|
+
|
|
|
|
|
+# 2. 查看错误日志
|
|
|
|
|
+tail -50 /var/log/supervisor/dataops-platform.log
|
|
|
|
|
+
|
|
|
|
|
+# 3. 手动测试启动
|
|
|
|
|
+cd /opt/dataops-platform
|
|
|
|
|
+source venv/bin/activate
|
|
|
|
|
+python -c "from app import create_app; app = create_app(); print('成功')"
|
|
|
|
|
+```
|
|
|
|
|
+
|
|
|
|
|
+**常见原因**:
|
|
|
|
|
+- 虚拟环境路径错误
|
|
|
|
|
+- 依赖包缺失
|
|
|
|
|
+- 配置文件错误
|
|
|
|
|
+- 端口被占用
|
|
|
|
|
+
|
|
|
|
|
+### 8.2 数据库连接失败
|
|
|
|
|
+
|
|
|
|
|
+```bash
|
|
|
|
|
+# 测试 PostgreSQL 连接
|
|
|
|
|
+psql -h 192.168.3.143 -U postgres -d dataops -c "SELECT 1"
|
|
|
|
|
+
|
|
|
|
|
+# 测试 Neo4j 连接
|
|
|
|
|
+curl http://192.168.3.143:7474
|
|
|
|
|
+```
|
|
|
|
|
+
|
|
|
|
|
+### 8.3 n8n API 连接失败
|
|
|
|
|
+
|
|
|
|
|
+```bash
|
|
|
|
|
+# 测试 n8n API
|
|
|
|
|
+curl -H "X-N8N-API-KEY: <YOUR_API_KEY>" https://n8n.citupro.com/api/v1/workflows
|
|
|
|
|
+```
|
|
|
|
|
+
|
|
|
|
|
+### 8.4 依赖包缺失
|
|
|
|
|
+
|
|
|
|
|
+```bash
|
|
|
|
|
+source venv/bin/activate
|
|
|
|
|
+pip install <package_name>
|
|
|
|
|
+sudo supervisorctl restart dataops-platform
|
|
|
|
|
+```
|
|
|
|
|
+
|
|
|
|
|
+### 8.5 权限问题
|
|
|
|
|
+
|
|
|
|
|
+```bash
|
|
|
|
|
+# 修复文件权限
|
|
|
|
|
+sudo chown -R ubuntu:ubuntu /opt/dataops-platform
|
|
|
|
|
+chmod +x /opt/dataops-platform/scripts/*.sh
|
|
|
|
|
+```
|
|
|
|
|
+
|
|
|
|
|
+---
|
|
|
|
|
+
|
|
|
|
|
+## 9. 备份与恢复
|
|
|
|
|
+
|
|
|
|
|
+### 9.1 备份内容
|
|
|
|
|
+
|
|
|
|
|
+| 内容 | 路径/命令 |
|
|
|
|
|
+|------|------|
|
|
|
|
|
+| 应用代码 | `/opt/dataops-platform/` |
|
|
|
|
|
+| PostgreSQL 数据库 | `pg_dump -h 192.168.3.143 -U postgres dataops > backup.sql` |
|
|
|
|
|
+| Neo4j 数据库 | Neo4j Admin 导出 |
|
|
|
|
|
+| MinIO 文件 | `mc mirror minio/dataops-bucket ./backup/` |
|
|
|
|
|
+
|
|
|
|
|
+### 9.2 备份脚本示例
|
|
|
|
|
+
|
|
|
|
|
+```bash
|
|
|
|
|
+#!/bin/bash
|
|
|
|
|
+BACKUP_DIR="/data/backup/$(date +%Y%m%d)"
|
|
|
|
|
+mkdir -p $BACKUP_DIR
|
|
|
|
|
+
|
|
|
|
|
+# 备份代码
|
|
|
|
|
+tar -czf $BACKUP_DIR/dataops-code.tar.gz /opt/dataops-platform/app
|
|
|
|
|
+
|
|
|
|
|
+# 备份数据库
|
|
|
|
|
+pg_dump -h 192.168.3.143 -U postgres dataops > $BACKUP_DIR/dataops-db.sql
|
|
|
|
|
+
|
|
|
|
|
+echo "备份完成: $BACKUP_DIR"
|
|
|
|
|
+```
|
|
|
|
|
+
|
|
|
|
|
+### 9.3 恢复数据库
|
|
|
|
|
+
|
|
|
|
|
+```bash
|
|
|
|
|
+psql -h 192.168.3.143 -U postgres dataops < backup.sql
|
|
|
|
|
+```
|
|
|
|
|
+
|
|
|
|
|
+---
|
|
|
|
|
+
|
|
|
|
|
+## 10. API 接口文档
|
|
|
|
|
+
|
|
|
|
|
+### 10.1 系统接口
|
|
|
|
|
+
|
|
|
|
|
+| 接口 | 方法 | 路径 | 说明 |
|
|
|
|
|
+|------|------|------|------|
|
|
|
|
|
+| 健康检查 | GET | `/api/system/health` | 系统健康状态 |
|
|
|
|
|
+| 系统信息 | GET | `/api/system/info` | 系统配置信息 |
|
|
|
|
|
+
|
|
|
|
|
+### 10.2 工作流接口 (Data Factory)
|
|
|
|
|
+
|
|
|
|
|
+| 接口 | 方法 | 路径 | 说明 |
|
|
|
|
|
+|------|------|------|------|
|
|
|
|
|
+| n8n 健康检查 | GET | `/api/datafactory/health` | n8n 连接状态 |
|
|
|
|
|
+| 工作流列表 | GET | `/api/datafactory/workflows` | 获取工作流列表 |
|
|
|
|
|
+| 工作流详情 | GET | `/api/datafactory/workflows/{id}` | 获取工作流详情 |
|
|
|
|
|
+| 工作流状态 | GET | `/api/datafactory/workflows/{id}/status` | 获取工作流状态 |
|
|
|
|
|
+| 激活工作流 | POST | `/api/datafactory/workflows/{id}/activate` | 激活工作流 |
|
|
|
|
|
+| 停用工作流 | POST | `/api/datafactory/workflows/{id}/deactivate` | 停用工作流 |
|
|
|
|
|
+| 执行记录列表 | GET | `/api/datafactory/executions` | 获取执行记录 |
|
|
|
|
|
+| 执行详情 | GET | `/api/datafactory/executions/{id}` | 获取执行详情 |
|
|
|
|
|
+| 触发工作流 | POST | `/api/datafactory/workflows/{id}/execute` | 触发工作流执行 |
|
|
|
|
|
+
|
|
|
|
|
+### 10.3 其他 API 模块
|
|
|
|
|
+
|
|
|
|
|
+| 模块 | 前缀 | 说明 |
|
|
|
|
|
+|------|------|------|
|
|
|
|
|
+| 元数据 | `/api/meta/` | 元数据管理 |
|
|
|
|
|
+| 图数据 | `/api/graph/` | Neo4j 图数据操作 |
|
|
|
|
|
+| 数据接口 | `/api/interface/` | 数据接口管理 |
|
|
|
|
|
+| 数据源 | `/api/datasource/` | 数据源管理 |
|
|
|
|
|
+| 数据流 | `/api/dataflow/` | 数据流管理 |
|
|
|
|
|
+| 业务领域 | `/api/bd/` | 业务领域管理 |
|
|
|
|
|
+
|
|
|
|
|
+---
|
|
|
|
|
+
|
|
|
|
|
+## 附录
|
|
|
|
|
+
|
|
|
|
|
+### A. 联系方式
|
|
|
|
|
+
|
|
|
|
|
+如有问题,请联系运维团队。
|
|
|
|
|
+
|
|
|
|
|
+### B. 更新记录
|
|
|
|
|
+
|
|
|
|
|
+| 日期 | 版本 | 更新内容 |
|
|
|
|
|
+|------|------|------|
|
|
|
|
|
+| 2025-12-25 | 1.0.0 | 初始版本,包含完整部署说明 |
|
|
|
|
|
+
|
|
|
|
|
+### C. 快速命令参考
|
|
|
|
|
+
|
|
|
|
|
+```bash
|
|
|
|
|
+# 服务管理
|
|
|
|
|
+sudo supervisorctl status dataops-platform
|
|
|
|
|
+sudo supervisorctl restart dataops-platform
|
|
|
|
|
+
|
|
|
|
|
+# 日志查看
|
|
|
|
|
+tail -f /var/log/supervisor/dataops-platform.log
|
|
|
|
|
+
|
|
|
|
|
+# 健康检查
|
|
|
|
|
+curl http://127.0.0.1:5500/api/system/health
|
|
|
|
|
+
|
|
|
|
|
+# 虚拟环境
|
|
|
|
|
+source /opt/dataops-platform/venv/bin/activate
|
|
|
|
|
+```
|
|
|
|
|
+
|