Merge tag 'v0.1.90' into merge/upstream-v0.1.90

注册邮箱域名白名单策略上线,后台大数据场景性能大幅优化。

- 注册邮箱域名白名单:支持管理员配置允许注册的邮箱域名策略
- Keys 页面表单筛选:用户 /keys 页面支持按条件筛选 API Key
- Settings 页面分 Tab 拆分:管理后台设置页面按功能模块分 Tab 展示

- 后台大数据场景加载性能优化:仪表盘/用户/账号/Ops 页面大数据集加载显著提速
- Usage 大表分页优化:默认避免全量 COUNT(*),大幅降低分页查询耗时
- 消除重复的 normalizeAccountIDList,补充新增组件的单元测试
- 清理无用文件和过时文档,精简项目结构
- EmailVerifyView 硬编码英文字符串替换为 i18n 调用

- 修复 Anthropic 平台无限流重置时间的 429 误标记账号限流问题
- 修复自定义菜单页面管理员视角菜单不生效问题
- 修复 Ops 错误详情弹窗未展示真实上游 payload 的问题
- 修复充值/订阅菜单 icon 显示问题

# Conflicts:
#	.gitignore
#	backend/cmd/server/VERSION
#	backend/ent/group.go
#	backend/ent/runtime/runtime.go
#	backend/ent/schema/group.go
#	backend/go.sum
#	backend/internal/handler/admin/account_handler.go
#	backend/internal/handler/admin/dashboard_handler.go
#	backend/internal/pkg/usagestats/usage_log_types.go
#	backend/internal/repository/group_repo.go
#	backend/internal/repository/usage_log_repo.go
#	backend/internal/server/middleware/security_headers.go
#	backend/internal/server/router.go
#	backend/internal/service/account_usage_service.go
#	backend/internal/service/admin_service_bulk_update_test.go
#	backend/internal/service/dashboard_service.go
#	backend/internal/service/gateway_service.go
#	frontend/src/api/admin/dashboard.ts
#	frontend/src/components/account/BulkEditAccountModal.vue
#	frontend/src/components/charts/GroupDistributionChart.vue
#	frontend/src/components/layout/AppSidebar.vue
#	frontend/src/i18n/locales/en.ts
#	frontend/src/i18n/locales/zh.ts
#	frontend/src/views/admin/GroupsView.vue
#	frontend/src/views/admin/SettingsView.vue
#	frontend/src/views/admin/UsageView.vue
#	frontend/src/views/user/PurchaseSubscriptionView.vue
This commit is contained in:
erio
2026-03-04 19:58:38 +08:00
461 changed files with 63392 additions and 6617 deletions

View File

@@ -66,11 +66,15 @@ LOG_SAMPLING_INITIAL=100
# 之后每 N 条保留 1 条
LOG_SAMPLING_THEREAFTER=100
# Global max request body size in bytes (default: 100MB)
# 全局最大请求体大小(字节,默认 100MB
# Global max request body size in bytes (default: 256MB)
# 全局最大请求体大小(字节,默认 256MB
# Applies to all requests, especially important for h2c first request memory protection
# 适用于所有请求,对 h2c 第一请求的内存保护尤为重要
SERVER_MAX_REQUEST_BODY_SIZE=104857600
SERVER_MAX_REQUEST_BODY_SIZE=268435456
# Gateway max request body size in bytes (default: 256MB)
# 网关请求体最大字节数(默认 256MB
GATEWAY_MAX_BODY_SIZE=268435456
# Enable HTTP/2 Cleartext (h2c) for client connections
# 启用 HTTP/2 Cleartext (h2c) 客户端连接
@@ -108,7 +112,7 @@ POSTGRES_DB=sub2api
DATABASE_PORT=5432
# -----------------------------------------------------------------------------
# PostgreSQL 服务端参数(可选;主要用于 deploy/docker-compose-aicodex.yml
# PostgreSQL 服务端参数(可选)
# -----------------------------------------------------------------------------
# POSTGRES_MAX_CONNECTIONSPostgreSQL 服务端允许的最大连接数。
# 必须 >=(所有 Sub2API 实例的 DATABASE_MAX_OPEN_CONNS 之和)+ 预留余量(例如 20%)。
@@ -159,7 +163,7 @@ REDIS_PORT=6379
# Leave empty for no password (default for local development)
REDIS_PASSWORD=
REDIS_DB=0
# Redis 服务端最大客户端连接数(可选;主要用于 deploy/docker-compose-aicodex.yml
# Redis 服务端最大客户端连接数(可选)
REDIS_MAXCLIENTS=50000
# Redis 连接池大小(默认 1024
REDIS_POOL_SIZE=4096

View File

@@ -0,0 +1,78 @@
# datamanagementd 部署说明(数据管理)
本文说明如何在宿主机部署 `datamanagementd`,并与主进程联动开启“数据管理”功能。
## 1. 关键约束
- 主进程固定探测路径:`/tmp/sub2api-datamanagement.sock`
- 仅当该 Unix Socket 可连通且 `Health` 成功时,后台“数据管理”才会启用
- `datamanagementd` 使用 SQLite 持久化元数据,不依赖主库
## 2. 宿主机构建与运行
```bash
cd /opt/sub2api-src/datamanagement
go build -o /opt/sub2api/datamanagementd ./cmd/datamanagementd
mkdir -p /var/lib/sub2api/datamanagement
chown -R sub2api:sub2api /var/lib/sub2api/datamanagement
```
手动启动示例:
```bash
/opt/sub2api/datamanagementd \
-socket-path /tmp/sub2api-datamanagement.sock \
-sqlite-path /var/lib/sub2api/datamanagement/datamanagementd.db \
-version 1.0.0
```
## 3. systemd 托管(推荐)
仓库已提供示例服务文件:`deploy/sub2api-datamanagementd.service`
```bash
sudo cp deploy/sub2api-datamanagementd.service /etc/systemd/system/
sudo systemctl daemon-reload
sudo systemctl enable --now sub2api-datamanagementd
sudo systemctl status sub2api-datamanagementd
```
查看日志:
```bash
sudo journalctl -u sub2api-datamanagementd -f
```
也可以使用一键安装脚本(自动安装二进制 + 注册 systemd
```bash
# 方式一:使用现成二进制
sudo ./deploy/install-datamanagementd.sh --binary /path/to/datamanagementd
# 方式二:从源码构建后安装
sudo ./deploy/install-datamanagementd.sh --source /path/to/sub2api
```
## 4. Docker 部署联动
`sub2api` 运行在 Docker 容器中,需要将宿主机 Socket 挂载到容器同路径:
```yaml
services:
sub2api:
volumes:
- /tmp/sub2api-datamanagement.sock:/tmp/sub2api-datamanagement.sock
```
建议在 `docker-compose.override.yml` 中维护该挂载,避免覆盖主 compose 文件。
## 5. 依赖检查
`datamanagementd` 执行备份时依赖以下工具:
- `pg_dump`
- `redis-cli`
- `docker`(仅 `source_mode=docker_exec` 时)
缺失依赖会导致对应任务失败,并在任务详情中体现错误信息。

View File

@@ -19,7 +19,10 @@ This directory contains files for deploying Sub2API on Linux servers.
| `.env.example` | Docker environment variables template |
| `DOCKER.md` | Docker Hub documentation |
| `install.sh` | One-click binary installation script |
| `install-datamanagementd.sh` | datamanagementd 一键安装脚本 |
| `sub2api.service` | Systemd service unit file |
| `sub2api-datamanagementd.service` | datamanagementd systemd service unit file |
| `DATAMANAGEMENTD_CN.md` | datamanagementd 部署与联动说明(中文) |
| `config.example.yaml` | Example configuration file |
---
@@ -145,6 +148,14 @@ SELECT
(SELECT COUNT(*) FROM user_allowed_groups) AS new_pair_count;
```
### datamanagementd数据管理联动
如需启用管理后台“数据管理”功能,请额外部署宿主机 `datamanagementd`
- 主进程固定探测 `/tmp/sub2api-datamanagement.sock`
- Docker 场景下需把宿主机 Socket 挂载到容器内同路径
- 详细步骤见:`deploy/DATAMANAGEMENTD_CN.md`
### Commands
For **local directory version** (docker-compose.local.yml):
@@ -575,7 +586,7 @@ gateway:
name: "Profile 2"
cipher_suites: [4866, 4867, 4865, 49199, 49195, 49200, 49196]
curves: [29, 23, 24]
point_formats: [0]
point_formats: 0
# Another custom profile
profile_3:

View File

@@ -27,11 +27,11 @@ server:
# Trusted proxies for X-Forwarded-For parsing (CIDR/IP). Empty disables trusted proxies.
# 信任的代理地址CIDR/IP 格式),用于解析 X-Forwarded-For 头。留空则禁用代理信任。
trusted_proxies: []
# Global max request body size in bytes (default: 100MB)
# 全局最大请求体大小(字节,默认 100MB
# Global max request body size in bytes (default: 256MB)
# 全局最大请求体大小(字节,默认 256MB
# Applies to all requests, especially important for h2c first request memory protection
# 适用于所有请求,对 h2c 第一请求的内存保护尤为重要
max_request_body_size: 104857600
max_request_body_size: 268435456
# HTTP/2 Cleartext (h2c) configuration
# HTTP/2 Cleartext (h2c) 配置
h2c:
@@ -134,6 +134,12 @@ security:
# Allow skipping TLS verification for proxy probe (debug only)
# 允许代理探测时跳过 TLS 证书验证(仅用于调试)
insecure_skip_verify: false
proxy_fallback:
# Allow auxiliary services (update check, pricing data) to fallback to direct
# connection when proxy initialization fails. Does NOT affect AI gateway connections.
# 辅助服务(更新检查、定价数据拉取)代理初始化失败时是否允许回退直连。
# 不影响 AI 账号网关连接。默认 falsefail-fast 防止 IP 泄露。
allow_direct_on_error: false
# =============================================================================
# Gateway Configuration
@@ -143,9 +149,9 @@ gateway:
# Timeout for waiting upstream response headers (seconds)
# 等待上游响应头超时时间(秒)
response_header_timeout: 600
# Max request body size in bytes (default: 100MB)
# 请求体最大字节数(默认 100MB
max_body_size: 104857600
# Max request body size in bytes (default: 256MB)
# 请求体最大字节数(默认 256MB
max_body_size: 268435456
# Max bytes to read for non-stream upstream responses (default: 8MB)
# 非流式上游响应体读取上限(默认 8MB
upstream_response_read_max_bytes: 8388608
@@ -199,6 +205,83 @@ gateway:
# OpenAI 透传模式是否放行客户端超时头(如 x-stainless-timeout
# 默认 false过滤超时头降低上游提前断流风险。
openai_passthrough_allow_timeout_headers: false
# OpenAI Responses WebSocket 配置(默认开启,可按需回滚到 HTTP
openai_ws:
# 新版 WS mode 路由(默认关闭)。关闭时保持当前 legacy 实现行为。
mode_router_v2_enabled: false
# ingress 默认模式off|shared|dedicated仅 mode_router_v2_enabled=true 生效)
ingress_mode_default: shared
# 全局总开关,默认 true关闭时所有请求保持原有 HTTP/SSE 路由
enabled: true
# 按账号类型细分开关
oauth_enabled: true
apikey_enabled: true
# 全局强制 HTTP紧急回滚开关
force_http: false
# 允许在 WSv2 下按策略恢复 store=true默认 false
allow_store_recovery: false
# ingress 模式收到 previous_response_not_found 时,自动去掉 previous_response_id 重试一次(默认 true
ingress_previous_response_recovery_enabled: true
# store=false 且无可复用会话连接时的策略:
# strict=强制新建连接隔离优先adaptive=仅在高风险失败后强制新建off=尽量复用(性能优先)
store_disabled_conn_mode: strict
# store=false 且无可复用会话连接时,是否强制新建连接(默认 true优先会话隔离
# 兼容旧配置:仅在 store_disabled_conn_mode 未配置时生效
store_disabled_force_new_conn: true
# 是否启用 WSv2 generate=false 预热(默认 false
prewarm_generate_enabled: false
# 协议 feature 开关v2 优先于 v1
responses_websockets: false
responses_websockets_v2: true
# 连接池参数(按账号池化复用)
max_conns_per_account: 128
min_idle_per_account: 4
max_idle_per_account: 12
# 是否按账号并发动态计算连接池上限:
# effective_max_conns = min(max_conns_per_account, ceil(account.concurrency * factor))
dynamic_max_conns_by_account_concurrency_enabled: true
# 按账号类型分别设置系数OAuth / API Key
oauth_max_conns_factor: 1.0
apikey_max_conns_factor: 1.0
dial_timeout_seconds: 10
read_timeout_seconds: 900
write_timeout_seconds: 120
pool_target_utilization: 0.7
queue_limit_per_conn: 64
# 流式写出批量 flush 参数
event_flush_batch_size: 1
event_flush_interval_ms: 10
# 预热触发冷却(毫秒)
prewarm_cooldown_ms: 300
# WS 回退到 HTTP 后的冷却时间(秒),用于避免 WS/HTTP 来回抖动0 表示关闭冷却
fallback_cooldown_seconds: 30
# WS 重试退避参数(毫秒)
retry_backoff_initial_ms: 120
retry_backoff_max_ms: 2000
# 抖动比例0-1
retry_jitter_ratio: 0.2
# 单次请求 WS 重试总预算(毫秒);建议设置为有限值,避免重试拉高 TTFT 长尾
retry_total_budget_ms: 5000
# payload_schema 日志采样率0-1降低热路径日志放大
payload_log_sample_rate: 0.2
# 调度与粘连参数
lb_top_k: 7
sticky_session_ttl_seconds: 3600
# 会话哈希迁移兼容开关:新 key 未命中时回退读取旧 SHA-256 key
session_hash_read_old_fallback: true
# 会话哈希迁移兼容开关:写入时双写旧 SHA-256 key短 TTL
session_hash_dual_write_old: true
# context 元数据迁移兼容开关:保留旧 ctxkey.* 读取/注入桥接
metadata_bridge_enabled: true
sticky_response_id_ttl_seconds: 3600
# 兼容旧键:当 sticky_response_id_ttl_seconds 缺失时回退该值
sticky_previous_response_ttl_seconds: 3600
scheduler_score_weights:
priority: 1.0
load: 1.0
queue: 0.7
error_rate: 0.8
ttft: 0.5
# HTTP upstream connection pool settings (HTTP/2 + multi-proxy scenario defaults)
# HTTP 上游连接池配置HTTP/2 + 多代理场景默认值)
# Max idle connections across all hosts
@@ -779,12 +862,12 @@ rate_limit:
# 定价数据源(可选)
# =============================================================================
pricing:
# URL to fetch model pricing data (default: LiteLLM)
# 获取模型定价数据的 URL默认LiteLLM
remote_url: "https://github.com/Wei-Shaw/model-price-repo/raw/refs/heads/main/model_prices_and_context_window.json"
# URL to fetch model pricing data (default: pinned model-price-repo commit)
# 获取模型定价数据的 URL默认固定 commit 的 model-price-repo
remote_url: "https://raw.githubusercontent.com/Wei-Shaw/model-price-repo/c7947e9871687e664180bc971d4837f1fc2784a9/model_prices_and_context_window.json"
# Hash verification URL (optional)
# 哈希校验 URL可选
hash_url: "https://github.com/Wei-Shaw/model-price-repo/raw/refs/heads/main/model_prices_and_context_window.sha256"
hash_url: "https://raw.githubusercontent.com/Wei-Shaw/model-price-repo/c7947e9871687e664180bc971d4837f1fc2784a9/model_prices_and_context_window.sha256"
# Local data directory for caching
# 本地数据缓存目录
data_dir: "./data"

View File

@@ -1,212 +0,0 @@
# =============================================================================
# Sub2API Docker Compose Test Configuration (Local Build)
# =============================================================================
# Quick Start:
# 1. Copy .env.example to .env and configure
# 2. docker-compose -f docker-compose-test.yml up -d --build
# 3. Check logs: docker-compose -f docker-compose-test.yml logs -f sub2api
# 4. Access: http://localhost:8080
#
# This configuration builds the image from source (Dockerfile in project root).
# All configuration is done via environment variables.
# No Setup Wizard needed - the system auto-initializes on first run.
# =============================================================================
services:
# ===========================================================================
# Sub2API Application
# ===========================================================================
sub2api:
image: sub2api:latest
build:
context: ..
dockerfile: Dockerfile
container_name: sub2api
restart: unless-stopped
ulimits:
nofile:
soft: 100000
hard: 100000
ports:
- "${BIND_HOST:-0.0.0.0}:${SERVER_PORT:-8080}:8080"
volumes:
# Data persistence (config.yaml will be auto-generated here)
- sub2api_data:/app/data
# Mount custom config.yaml (optional, overrides auto-generated config)
# - ./config.yaml:/app/data/config.yaml:ro
environment:
# =======================================================================
# Auto Setup (REQUIRED for Docker deployment)
# =======================================================================
- AUTO_SETUP=true
# =======================================================================
# Server Configuration
# =======================================================================
- SERVER_HOST=0.0.0.0
- SERVER_PORT=8080
- SERVER_MODE=${SERVER_MODE:-release}
- RUN_MODE=${RUN_MODE:-standard}
# =======================================================================
# Database Configuration (PostgreSQL)
# =======================================================================
- DATABASE_HOST=postgres
- DATABASE_PORT=5432
- DATABASE_USER=${POSTGRES_USER:-sub2api}
- DATABASE_PASSWORD=${POSTGRES_PASSWORD:?POSTGRES_PASSWORD is required}
- DATABASE_DBNAME=${POSTGRES_DB:-sub2api}
- DATABASE_SSLMODE=disable
- DATABASE_MAX_OPEN_CONNS=${DATABASE_MAX_OPEN_CONNS:-50}
- DATABASE_MAX_IDLE_CONNS=${DATABASE_MAX_IDLE_CONNS:-10}
- DATABASE_CONN_MAX_LIFETIME_MINUTES=${DATABASE_CONN_MAX_LIFETIME_MINUTES:-30}
- DATABASE_CONN_MAX_IDLE_TIME_MINUTES=${DATABASE_CONN_MAX_IDLE_TIME_MINUTES:-5}
# =======================================================================
# Redis Configuration
# =======================================================================
- REDIS_HOST=redis
- REDIS_PORT=6379
- REDIS_PASSWORD=${REDIS_PASSWORD:-}
- REDIS_DB=${REDIS_DB:-0}
- REDIS_POOL_SIZE=${REDIS_POOL_SIZE:-1024}
- REDIS_MIN_IDLE_CONNS=${REDIS_MIN_IDLE_CONNS:-10}
# =======================================================================
# Admin Account (auto-created on first run)
# =======================================================================
- ADMIN_EMAIL=${ADMIN_EMAIL:-admin@sub2api.local}
- ADMIN_PASSWORD=${ADMIN_PASSWORD:-}
# =======================================================================
# JWT Configuration
# =======================================================================
# Leave empty to auto-generate (recommended)
- JWT_SECRET=${JWT_SECRET:-}
- JWT_EXPIRE_HOUR=${JWT_EXPIRE_HOUR:-24}
# =======================================================================
# Timezone Configuration
# This affects ALL time operations in the application:
# - Database timestamps
# - Usage statistics "today" boundary
# - Subscription expiry times
# - Log timestamps
# Common values: Asia/Shanghai, America/New_York, Europe/London, UTC
# =======================================================================
- TZ=${TZ:-Asia/Shanghai}
# =======================================================================
# Gemini OAuth Configuration (for Gemini accounts)
# =======================================================================
- GEMINI_OAUTH_CLIENT_ID=${GEMINI_OAUTH_CLIENT_ID:-}
- GEMINI_OAUTH_CLIENT_SECRET=${GEMINI_OAUTH_CLIENT_SECRET:-}
- GEMINI_OAUTH_SCOPES=${GEMINI_OAUTH_SCOPES:-}
- GEMINI_QUOTA_POLICY=${GEMINI_QUOTA_POLICY:-}
# Built-in OAuth client secrets (optional)
# SECURITY: This repo does not embed third-party client_secret.
- GEMINI_CLI_OAUTH_CLIENT_SECRET=${GEMINI_CLI_OAUTH_CLIENT_SECRET:-}
- ANTIGRAVITY_OAUTH_CLIENT_SECRET=${ANTIGRAVITY_OAUTH_CLIENT_SECRET:-}
# =======================================================================
# Security Configuration (URL Allowlist)
# =======================================================================
# Allow private IP addresses for CRS sync (for internal deployments)
- SECURITY_URL_ALLOWLIST_ALLOW_PRIVATE_HOSTS=${SECURITY_URL_ALLOWLIST_ALLOW_PRIVATE_HOSTS:-true}
depends_on:
postgres:
condition: service_healthy
redis:
condition: service_healthy
networks:
- sub2api-network
healthcheck:
test: ["CMD", "curl", "-f", "http://localhost:8080/health"]
interval: 30s
timeout: 10s
retries: 3
start_period: 30s
# ===========================================================================
# PostgreSQL Database
# ===========================================================================
postgres:
image: postgres:18-alpine
container_name: sub2api-postgres
restart: unless-stopped
ulimits:
nofile:
soft: 100000
hard: 100000
volumes:
- postgres_data:/var/lib/postgresql/data
environment:
# postgres:18-alpine 默认 PGDATA=/var/lib/postgresql/18/docker位于镜像声明的匿名卷 /var/lib/postgresql 内)。
# 若不显式设置 PGDATA则即使挂载了 postgres_data 到 /var/lib/postgresql/data数据也不会落盘到该命名卷
# docker compose down/up 后会触发 initdb 重新初始化,导致用户/密码等数据丢失。
- PGDATA=/var/lib/postgresql/data
- POSTGRES_USER=${POSTGRES_USER:-sub2api}
- POSTGRES_PASSWORD=${POSTGRES_PASSWORD:?POSTGRES_PASSWORD is required}
- POSTGRES_DB=${POSTGRES_DB:-sub2api}
- TZ=${TZ:-Asia/Shanghai}
networks:
- sub2api-network
healthcheck:
test: ["CMD-SHELL", "pg_isready -U ${POSTGRES_USER:-sub2api} -d ${POSTGRES_DB:-sub2api}"]
interval: 10s
timeout: 5s
retries: 5
start_period: 10s
# 注意:不暴露端口到宿主机,应用通过内部网络连接
# 如需调试可临时添加ports: ["127.0.0.1:5433:5432"]
# ===========================================================================
# Redis Cache
# ===========================================================================
redis:
image: redis:8-alpine
container_name: sub2api-redis
restart: unless-stopped
ulimits:
nofile:
soft: 100000
hard: 100000
volumes:
- redis_data:/data
command: >
redis-server
--save 60 1
--appendonly yes
--appendfsync everysec
${REDIS_PASSWORD:+--requirepass ${REDIS_PASSWORD}}
environment:
- TZ=${TZ:-Asia/Shanghai}
# REDISCLI_AUTH is used by redis-cli for authentication (safer than -a flag)
- REDISCLI_AUTH=${REDIS_PASSWORD:-}
networks:
- sub2api-network
healthcheck:
test: ["CMD", "redis-cli", "ping"]
interval: 10s
timeout: 5s
retries: 5
start_period: 5s
# =============================================================================
# Volumes
# =============================================================================
volumes:
sub2api_data:
driver: local
postgres_data:
driver: local
redis_data:
driver: local
# =============================================================================
# Networks
# =============================================================================
networks:
sub2api-network:
driver: bridge

View File

@@ -1,137 +0,0 @@
# =============================================================================
# Docker Compose Override Configuration Example
# =============================================================================
# This file provides examples for customizing the Docker Compose setup.
# Copy this file to docker-compose.override.yml and modify as needed.
#
# Usage:
# cp docker-compose.override.yml.example docker-compose.override.yml
# # Edit docker-compose.override.yml with your settings
# docker-compose up -d
#
# IMPORTANT: docker-compose.override.yml is gitignored and will not be committed.
# =============================================================================
# =============================================================================
# Scenario 1: Use External Database and Redis (Recommended for Production)
# =============================================================================
# Use this when you have PostgreSQL and Redis running on the host machine
# or on separate servers.
#
# Prerequisites:
# - PostgreSQL running on host (accessible via host.docker.internal)
# - Redis running on host (accessible via host.docker.internal)
# - Update DATABASE_PORT and REDIS_PORT in .env file if using non-standard ports
#
# Security Notes:
# - Ensure PostgreSQL pg_hba.conf allows connections from Docker network
# - Use strong passwords for database and Redis
# - Consider using SSL/TLS for database connections in production
# =============================================================================
services:
sub2api:
# Remove dependencies on containerized postgres/redis
depends_on: []
# Enable access to host machine services
extra_hosts:
- "host.docker.internal:host-gateway"
# Override database and Redis connection settings
environment:
# PostgreSQL Configuration
DATABASE_HOST: host.docker.internal
DATABASE_PORT: "5678" # Change to your PostgreSQL port
# DATABASE_USER: postgres # Uncomment to override
# DATABASE_PASSWORD: your_password # Uncomment to override
# DATABASE_DBNAME: sub2api # Uncomment to override
# Redis Configuration
REDIS_HOST: host.docker.internal
REDIS_PORT: "6379" # Change to your Redis port
# REDIS_PASSWORD: your_redis_password # Uncomment if Redis requires auth
# REDIS_DB: 0 # Uncomment to override
# Disable containerized PostgreSQL
postgres:
deploy:
replicas: 0
scale: 0
# Disable containerized Redis
redis:
deploy:
replicas: 0
scale: 0
# =============================================================================
# Scenario 2: Development with Local Services (Alternative)
# =============================================================================
# Uncomment this section if you want to use the containerized postgres/redis
# but expose their ports for local development tools.
#
# Usage: Comment out Scenario 1 above and uncomment this section.
# =============================================================================
# services:
# sub2api:
# # Keep default dependencies
# pass
#
# postgres:
# ports:
# - "127.0.0.1:5432:5432" # Expose PostgreSQL on localhost
#
# redis:
# ports:
# - "127.0.0.1:6379:6379" # Expose Redis on localhost
# =============================================================================
# Scenario 3: Custom Network Configuration
# =============================================================================
# Uncomment if you need to connect to an existing Docker network
# =============================================================================
# networks:
# default:
# external: true
# name: your-existing-network
# =============================================================================
# Scenario 4: Resource Limits (Production)
# =============================================================================
# Uncomment to set resource limits for the sub2api container
# =============================================================================
# services:
# sub2api:
# deploy:
# resources:
# limits:
# cpus: '2.0'
# memory: 2G
# reservations:
# cpus: '1.0'
# memory: 1G
# =============================================================================
# Scenario 5: Custom Volumes
# =============================================================================
# Uncomment to mount additional volumes (e.g., for logs, backups)
# =============================================================================
# services:
# sub2api:
# volumes:
# - ./logs:/app/logs
# - ./backups:/app/backups
# =============================================================================
# Additional Notes
# =============================================================================
# - This file overrides settings in docker-compose.yml
# - Environment variables in .env file take precedence
# - For more information, see: https://docs.docker.com/compose/extends/
# - Check the main README.md for detailed configuration instructions
# =============================================================================

View File

@@ -1,222 +0,0 @@
```mermaid
flowchart TD
%% Master dispatch
A[HTTP Request] --> B{Route}
B -->|v1 messages| GA0
B -->|openai v1 responses| OA0
B -->|v1beta models model action| GM0
B -->|v1 messages count tokens| GT0
B -->|v1beta models list or get| GL0
%% =========================
%% FLOW A: Claude Gateway
%% =========================
subgraph FLOW_A["v1 messages Claude Gateway"]
GA0[Auth middleware] --> GA1[Read body]
GA1 -->|empty| GA1E[400 invalid_request_error]
GA1 --> GA2[ParseGatewayRequest]
GA2 -->|parse error| GA2E[400 invalid_request_error]
GA2 --> GA3{model present}
GA3 -->|no| GA3E[400 invalid_request_error]
GA3 --> GA4[streamStarted false]
GA4 --> GA5[IncrementWaitCount user]
GA5 -->|queue full| GA5E[429 rate_limit_error]
GA5 --> GA6[AcquireUserSlotWithWait]
GA6 -->|timeout or fail| GA6E[429 rate_limit_error]
GA6 --> GA7[BillingEligibility check post wait]
GA7 -->|fail| GA7E[403 billing_error]
GA7 --> GA8[Generate sessionHash]
GA8 --> GA9[Resolve platform]
GA9 --> GA10{platform gemini}
GA10 -->|yes| GA10Y[sessionKey gemini hash]
GA10 -->|no| GA10N[sessionKey hash]
GA10Y --> GA11
GA10N --> GA11
GA11[SelectAccountWithLoadAwareness] -->|err and no failed| GA11E1[503 no available accounts]
GA11 -->|err and failed| GA11E2[map failover error]
GA11 --> GA12[Warmup intercept]
GA12 -->|yes| GA12Y[return mock and release if held]
GA12 -->|no| GA13[Acquire account slot or wait]
GA13 -->|wait queue full| GA13E1[429 rate_limit_error]
GA13 -->|wait timeout| GA13E2[429 concurrency limit]
GA13 --> GA14[BindStickySession if waited]
GA14 --> GA15{account platform antigravity}
GA15 -->|yes| GA15Y[ForwardGemini antigravity]
GA15 -->|no| GA15N[Forward Claude]
GA15Y --> GA16[Release account slot and dec account wait]
GA15N --> GA16
GA16 --> GA17{UpstreamFailoverError}
GA17 -->|yes| GA18[mark failedAccountIDs and map error if exceed]
GA18 -->|loop| GA11
GA17 -->|no| GA19[success async RecordUsage and return]
GA19 --> GA20[defer release user slot and dec wait count]
end
%% =========================
%% FLOW B: OpenAI
%% =========================
subgraph FLOW_B["openai v1 responses"]
OA0[Auth middleware] --> OA1[Read body]
OA1 -->|empty| OA1E[400 invalid_request_error]
OA1 --> OA2[json Unmarshal body]
OA2 -->|parse error| OA2E[400 invalid_request_error]
OA2 --> OA3{model present}
OA3 -->|no| OA3E[400 invalid_request_error]
OA3 --> OA4{User Agent Codex CLI}
OA4 -->|no| OA4N[set default instructions]
OA4 -->|yes| OA4Y[no change]
OA4N --> OA5
OA4Y --> OA5
OA5[streamStarted false] --> OA6[IncrementWaitCount user]
OA6 -->|queue full| OA6E[429 rate_limit_error]
OA6 --> OA7[AcquireUserSlotWithWait]
OA7 -->|timeout or fail| OA7E[429 rate_limit_error]
OA7 --> OA8[BillingEligibility check post wait]
OA8 -->|fail| OA8E[403 billing_error]
OA8 --> OA9[sessionHash sha256 session_id]
OA9 --> OA10[SelectAccountWithLoadAwareness]
OA10 -->|err and no failed| OA10E1[503 no available accounts]
OA10 -->|err and failed| OA10E2[map failover error]
OA10 --> OA11[Acquire account slot or wait]
OA11 -->|wait queue full| OA11E1[429 rate_limit_error]
OA11 -->|wait timeout| OA11E2[429 concurrency limit]
OA11 --> OA12[BindStickySession openai hash if waited]
OA12 --> OA13[Forward OpenAI upstream]
OA13 --> OA14[Release account slot and dec account wait]
OA14 --> OA15{UpstreamFailoverError}
OA15 -->|yes| OA16[mark failedAccountIDs and map error if exceed]
OA16 -->|loop| OA10
OA15 -->|no| OA17[success async RecordUsage and return]
OA17 --> OA18[defer release user slot and dec wait count]
end
%% =========================
%% FLOW C: Gemini Native
%% =========================
subgraph FLOW_C["v1beta models model action Gemini Native"]
GM0[Auth middleware] --> GM1[Validate platform]
GM1 -->|invalid| GM1E[400 googleError]
GM1 --> GM2[Parse path modelName action]
GM2 -->|invalid| GM2E[400 googleError]
GM2 --> GM3{action supported}
GM3 -->|no| GM3E[404 googleError]
GM3 --> GM4[Read body]
GM4 -->|empty| GM4E[400 googleError]
GM4 --> GM5[streamStarted false]
GM5 --> GM6[IncrementWaitCount user]
GM6 -->|queue full| GM6E[429 googleError]
GM6 --> GM7[AcquireUserSlotWithWait]
GM7 -->|timeout or fail| GM7E[429 googleError]
GM7 --> GM8[BillingEligibility check post wait]
GM8 -->|fail| GM8E[403 googleError]
GM8 --> GM9[Generate sessionHash]
GM9 --> GM10[sessionKey gemini hash]
GM10 --> GM11[SelectAccountWithLoadAwareness]
GM11 -->|err and no failed| GM11E1[503 googleError]
GM11 -->|err and failed| GM11E2[mapGeminiUpstreamError]
GM11 --> GM12[Acquire account slot or wait]
GM12 -->|wait queue full| GM12E1[429 googleError]
GM12 -->|wait timeout| GM12E2[429 googleError]
GM12 --> GM13[BindStickySession if waited]
GM13 --> GM14{account platform antigravity}
GM14 -->|yes| GM14Y[ForwardGemini antigravity]
GM14 -->|no| GM14N[ForwardNative]
GM14Y --> GM15[Release account slot and dec account wait]
GM14N --> GM15
GM15 --> GM16{UpstreamFailoverError}
GM16 -->|yes| GM17[mark failedAccountIDs and map error if exceed]
GM17 -->|loop| GM11
GM16 -->|no| GM18[success async RecordUsage and return]
GM18 --> GM19[defer release user slot and dec wait count]
end
%% =========================
%% FLOW D: CountTokens
%% =========================
subgraph FLOW_D["v1 messages count tokens"]
GT0[Auth middleware] --> GT1[Read body]
GT1 -->|empty| GT1E[400 invalid_request_error]
GT1 --> GT2[ParseGatewayRequest]
GT2 -->|parse error| GT2E[400 invalid_request_error]
GT2 --> GT3{model present}
GT3 -->|no| GT3E[400 invalid_request_error]
GT3 --> GT4[BillingEligibility check]
GT4 -->|fail| GT4E[403 billing_error]
GT4 --> GT5[ForwardCountTokens]
end
%% =========================
%% FLOW E: Gemini Models List Get
%% =========================
subgraph FLOW_E["v1beta models list or get"]
GL0[Auth middleware] --> GL1[Validate platform]
GL1 -->|invalid| GL1E[400 googleError]
GL1 --> GL2{force platform antigravity}
GL2 -->|yes| GL2Y[return static fallback models]
GL2 -->|no| GL3[SelectAccountForAIStudioEndpoints]
GL3 -->|no gemini and has antigravity| GL3Y[return fallback models]
GL3 -->|no accounts| GL3E[503 googleError]
GL3 --> GL4[ForwardAIStudioGET]
GL4 -->|error| GL4E[502 googleError]
GL4 --> GL5[Passthrough response or fallback]
end
%% =========================
%% SHARED: Account Selection
%% =========================
subgraph SELECT["SelectAccountWithLoadAwareness detail"]
S0[Start] --> S1{concurrencyService nil OR load batch disabled}
S1 -->|yes| S2[SelectAccountForModelWithExclusions legacy]
S2 --> S3[tryAcquireAccountSlot]
S3 -->|acquired| S3Y[SelectionResult Acquired true ReleaseFunc]
S3 -->|not acquired| S3N[WaitPlan FallbackTimeout MaxWaiting]
S1 -->|no| S4[Resolve platform]
S4 --> S5[List schedulable accounts]
S5 --> S6[Layer1 Sticky session]
S6 -->|hit and valid| S6A[tryAcquireAccountSlot]
S6A -->|acquired| S6AY[SelectionResult Acquired true]
S6A -->|not acquired and waitingCount < StickyMax| S6AN[WaitPlan StickyTimeout Max]
S6 --> S7[Layer2 Load aware]
S7 --> S7A[Load batch concurrency plus wait to loadRate]
S7A --> S7B[Sort priority load LRU OAuth prefer for Gemini]
S7B --> S7C[tryAcquireAccountSlot in order]
S7C -->|first success| S7CY[SelectionResult Acquired true]
S7C -->|none| S8[Layer3 Fallback wait]
S8 --> S8A[Sort priority LRU]
S8A --> S8B[WaitPlan FallbackTimeout Max]
end
%% =========================
%% SHARED: Wait Acquire
%% =========================
subgraph WAIT["AcquireXSlotWithWait detail"]
W0[Try AcquireXSlot immediately] -->|acquired| W1[return ReleaseFunc]
W0 -->|not acquired| W2[Wait loop with timeout]
W2 --> W3[Backoff 100ms x1.5 jitter max2s]
W2 --> W4[If streaming and ping format send SSE ping]
W2 --> W5[Retry AcquireXSlot on timer]
W5 -->|acquired| W1
W2 -->|timeout| W6[ConcurrencyError IsTimeout true]
end
%% =========================
%% SHARED: Account Wait Queue
%% =========================
subgraph AQ["Account Wait Queue Redis Lua"]
Q1[IncrementAccountWaitCount] --> Q2{current >= max}
Q2 -->|yes| Q2Y[return false]
Q2 -->|no| Q3[INCR and if first set TTL]
Q3 --> Q4[return true]
Q5[DecrementAccountWaitCount] --> Q6[if current > 0 then DECR]
end
%% =========================
%% SHARED: Background cleanup
%% =========================
subgraph CLEANUP["Slot Cleanup Worker"]
C0[StartSlotCleanupWorker interval] --> C1[List schedulable accounts]
C1 --> C2[CleanupExpiredAccountSlots per account]
C2 --> C3[Repeat every interval]
end
```

123
deploy/install-datamanagementd.sh Executable file
View File

@@ -0,0 +1,123 @@
#!/usr/bin/env bash
set -euo pipefail
# 用法:
# sudo ./install-datamanagementd.sh --binary /path/to/datamanagementd
# 或:
# sudo ./install-datamanagementd.sh --source /path/to/sub2api/repo
BIN_PATH=""
SOURCE_PATH=""
INSTALL_DIR="/opt/sub2api"
DATA_DIR="/var/lib/sub2api/datamanagement"
SERVICE_FILE_NAME="sub2api-datamanagementd.service"
function print_help() {
cat <<'EOF'
用法:
install-datamanagementd.sh [--binary <datamanagementd二进制路径>] [--source <仓库路径>]
参数:
--binary 指定已构建的 datamanagementd 二进制路径
--source 指定 sub2api 仓库路径(脚本会执行 go build
-h, --help 显示帮助
示例:
sudo ./install-datamanagementd.sh --binary ./datamanagement/datamanagementd
sudo ./install-datamanagementd.sh --source /opt/sub2api-src
EOF
}
while [[ $# -gt 0 ]]; do
case "$1" in
--binary)
BIN_PATH="${2:-}"
shift 2
;;
--source)
SOURCE_PATH="${2:-}"
shift 2
;;
-h|--help)
print_help
exit 0
;;
*)
echo "未知参数: $1"
print_help
exit 1
;;
esac
done
if [[ -n "$BIN_PATH" && -n "$SOURCE_PATH" ]]; then
echo "错误: --binary 与 --source 只能二选一"
exit 1
fi
if [[ -z "$BIN_PATH" && -z "$SOURCE_PATH" ]]; then
echo "错误: 必须提供 --binary 或 --source"
exit 1
fi
if [[ "$(id -u)" -ne 0 ]]; then
echo "错误: 请使用 root 权限执行(例如 sudo"
exit 1
fi
if [[ -n "$SOURCE_PATH" ]]; then
if [[ ! -d "$SOURCE_PATH/datamanagement" ]]; then
echo "错误: 无效仓库路径,未找到 $SOURCE_PATH/datamanagement"
exit 1
fi
echo "[1/6] 从源码构建 datamanagementd..."
(cd "$SOURCE_PATH/datamanagement" && go build -o datamanagementd ./cmd/datamanagementd)
BIN_PATH="$SOURCE_PATH/datamanagement/datamanagementd"
fi
if [[ ! -f "$BIN_PATH" ]]; then
echo "错误: 二进制文件不存在: $BIN_PATH"
exit 1
fi
if ! id sub2api >/dev/null 2>&1; then
echo "[2/6] 创建系统用户 sub2api..."
useradd --system --no-create-home --shell /usr/sbin/nologin sub2api
else
echo "[2/6] 系统用户 sub2api 已存在,跳过创建"
fi
echo "[3/6] 安装 datamanagementd 二进制..."
mkdir -p "$INSTALL_DIR"
install -m 0755 "$BIN_PATH" "$INSTALL_DIR/datamanagementd"
echo "[4/6] 准备数据目录..."
mkdir -p "$DATA_DIR"
chown -R sub2api:sub2api /var/lib/sub2api
chmod 0750 "$DATA_DIR"
SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
SERVICE_TEMPLATE="$SCRIPT_DIR/$SERVICE_FILE_NAME"
if [[ ! -f "$SERVICE_TEMPLATE" ]]; then
echo "错误: 未找到服务模板 $SERVICE_TEMPLATE"
exit 1
fi
echo "[5/6] 安装 systemd 服务..."
cp "$SERVICE_TEMPLATE" "/etc/systemd/system/$SERVICE_FILE_NAME"
systemctl daemon-reload
systemctl enable --now sub2api-datamanagementd
echo "[6/6] 完成,当前状态:"
systemctl --no-pager --full status sub2api-datamanagementd || true
cat <<'EOF'
下一步建议:
1. 查看日志sudo journalctl -u sub2api-datamanagementd -f
2. 在 sub2api容器部署时挂载 socket:
/tmp/sub2api-datamanagement.sock:/tmp/sub2api-datamanagement.sock
3. 进入管理后台“数据管理”页面确认 agent=enabled
EOF

View File

@@ -0,0 +1,22 @@
[Unit]
Description=Sub2API Data Management Daemon
After=network.target
Wants=network.target
[Service]
Type=simple
User=sub2api
Group=sub2api
WorkingDirectory=/opt/sub2api
ExecStart=/opt/sub2api/datamanagementd \
-socket-path /tmp/sub2api-datamanagement.sock \
-sqlite-path /var/lib/sub2api/datamanagement/datamanagementd.db \
-version 1.0.0
Restart=always
RestartSec=5s
LimitNOFILE=100000
NoNewPrivileges=true
PrivateTmp=false
[Install]
WantedBy=multi-user.target