Commit Graph

965 Commits

Author SHA1 Message Date
erio
cb3958bac3 fix: use detached context for concurrency batch query to prevent all-zero display
The upstream v0.1.90 changed GetAccountConcurrencyBatch from individual
Lua script calls (which swallowed per-account errors) to a Redis pipeline
approach that propagates errors from rdb.Time() or pipe.Exec(). When the
HTTP request context is cancelled (e.g., browser abort), the entire batch
fails and the handler silently shows all concurrency as 0.

Fix: use context.WithTimeout(context.Background(), 3s) for the Redis call
so HTTP request cancellation doesn't affect the read-only concurrency query.
2026-03-04 21:33:18 +08:00
erio
7895dd65b2 fix: remove duplicate groupExistenceBatchReader declaration in admin_service.go 2026-03-04 20:24:12 +08:00
erio
a6026e7ac4 Merge tag 'v0.1.90' into merge/upstream-v0.1.90
注册邮箱域名白名单策略上线,后台大数据场景性能大幅优化。

- 注册邮箱域名白名单:支持管理员配置允许注册的邮箱域名策略
- Keys 页面表单筛选:用户 /keys 页面支持按条件筛选 API Key
- Settings 页面分 Tab 拆分:管理后台设置页面按功能模块分 Tab 展示

- 后台大数据场景加载性能优化:仪表盘/用户/账号/Ops 页面大数据集加载显著提速
- Usage 大表分页优化:默认避免全量 COUNT(*),大幅降低分页查询耗时
- 消除重复的 normalizeAccountIDList,补充新增组件的单元测试
- 清理无用文件和过时文档,精简项目结构
- EmailVerifyView 硬编码英文字符串替换为 i18n 调用

- 修复 Anthropic 平台无限流重置时间的 429 误标记账号限流问题
- 修复自定义菜单页面管理员视角菜单不生效问题
- 修复 Ops 错误详情弹窗未展示真实上游 payload 的问题
- 修复充值/订阅菜单 icon 显示问题

# Conflicts:
#	.gitignore
#	backend/cmd/server/VERSION
#	backend/ent/group.go
#	backend/ent/runtime/runtime.go
#	backend/ent/schema/group.go
#	backend/go.sum
#	backend/internal/handler/admin/account_handler.go
#	backend/internal/handler/admin/dashboard_handler.go
#	backend/internal/pkg/usagestats/usage_log_types.go
#	backend/internal/repository/group_repo.go
#	backend/internal/repository/usage_log_repo.go
#	backend/internal/server/middleware/security_headers.go
#	backend/internal/server/router.go
#	backend/internal/service/account_usage_service.go
#	backend/internal/service/admin_service_bulk_update_test.go
#	backend/internal/service/dashboard_service.go
#	backend/internal/service/gateway_service.go
#	frontend/src/api/admin/dashboard.ts
#	frontend/src/components/account/BulkEditAccountModal.vue
#	frontend/src/components/charts/GroupDistributionChart.vue
#	frontend/src/components/layout/AppSidebar.vue
#	frontend/src/i18n/locales/en.ts
#	frontend/src/i18n/locales/zh.ts
#	frontend/src/views/admin/GroupsView.vue
#	frontend/src/views/admin/SettingsView.vue
#	frontend/src/views/admin/UsageView.vue
#	frontend/src/views/user/PurchaseSubscriptionView.vue
2026-03-04 19:58:38 +08:00
Wesley Liddick
27abae21b8 Merge pull request #724 from PMExtra/feat/registration-email-domain-whitelist
feat(registration): add email domain whitelist policy
2026-03-04 15:51:51 +08:00
shaw
0819c8a51a refactor: 消除重复的 normalizeAccountIDList,补充 PR#754 新增组件的单元测试
- 删除 account_today_stats_cache.go 中重复的 normalizeAccountIDList,统一使用 id_list_utils.go 的 normalizeInt64IDList
- 新增 snapshot_cache_test.go:覆盖 snapshotCache、buildETagFromAny、parseBoolQueryWithDefault
- 新增 id_list_utils_test.go:覆盖 normalizeInt64IDList、buildAccountTodayStatsBatchCacheKey
- 新增 ops_query_mode_test.go:覆盖 shouldFallbackOpsPreagg、cloneOpsFilterWithMode
2026-03-04 15:22:46 +08:00
Wesley Liddick
9dcd3cd491 Merge pull request #754 from xvhuan/perf/admin-core-large-dataset
perf(admin): 优化后台大数据场景加载性能(仪表盘/用户/账号/Ops)
2026-03-04 15:15:13 +08:00
xvhuan
f6fe5b552d fix(admin): resolve CI lint and user subscriptions regression 2026-03-04 14:07:17 +08:00
PMExtra
bd0801a887 feat(registration): add email domain whitelist policy 2026-03-04 13:54:18 +08:00
xvhuan
80ae592c23 perf(admin): optimize large-dataset loading for dashboard/users/accounts/ops 2026-03-04 13:45:49 +08:00
shaw
ba6de4c4d4 feat: /keys页面支持表单筛选 2026-03-04 11:29:31 +08:00
shaw
72961c5858 fix: Anthropic 平台无限流重置时间的 429 不再误标记账号限流 2026-03-04 09:36:24 +08:00
Wesley Liddick
a11ac188c2 Merge pull request #738 from DaydreamCoding/feat/ungrouped-key-setting
feat(gateway): 系统设置控制未分组 Key 调度 — Handler 层中间件拦截
2026-03-03 21:03:31 +08:00
Wesley Liddick
60350d298a Merge pull request #735 from alfadb/fix/count-tokens-default-ignore
fix(ops): 默认忽略 count_tokens 404 错误
2026-03-03 21:02:46 +08:00
shaw
838dad8759 feat: 重构 /v1/usage 端点,支持 quota_limited 和 unrestricted 双模式
- quota_limited 模式:返回 Key 级别的总额度、速率限制窗口用量和过期时间
- unrestricted 模式:返回订阅限额或钱包余额信息(向后兼容)
- 新增 model_stats 字段,支持 start_date/end_date 参数查询按模型用量统计
- 提取 buildUsageData/parseUsageDateRange 等辅助方法,减少主函数复杂度
- 新增 APIKeyService.GetRateLimitData 和 UsageService.GetAPIKeyModelStats
2026-03-03 20:59:12 +08:00
QTom
0c7cbe3566 feat(gateway): 系统设置控制未分组 Key 调度 — Handler 层中间件拦截
新增系统设置 allow_ungrouped_key_scheduling(默认关闭),
未分组的 API Key 在网关请求时直接返回 403,
由 RequireGroupAssignment 中间件统一拦截,
支持 Anthropic / Google 两种错误格式响应。

全栈实现:常量 → 结构体 → 解析/更新/初始化 → DTO → 管理接口 →
中间件 → 路由注册 → 前端设置界面 + i18n。
2026-03-03 19:56:27 +08:00
alfadb
832b0185c7 style: fix gofmt formatting in ops_settings.go
Remove extra space before inline comment to pass golangci-lint gofmt check.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-03 18:00:49 +08:00
alfadb
b1719b26d1 fix(ops): 默认忽略 count_tokens 404 错误
将 IgnoreCountTokensErrors 默认值从 false 改为 true。

count_tokens 返回 404 是预期业务行为(上游不支持 endpoint,
客户端应 fallback 到本地 tokenizer 估算),不应被视为错误。

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-03 16:50:13 +08:00
shaw
ccf6a921c7 fix: 修复 PR #723 引入的 CI lint 和 test 编译错误
- wire_gen_test.go: 补充 NewTokenRefreshService 缺失的 tempUnschedCache 参数
- config.go, token_refresh_service.go: 修复 gofmt 格式问题
2026-03-03 16:45:29 +08:00
Wesley Liddick
197c570baa Merge pull request #723 from zqq-nuli/fix/oauth-401-temp-unschedulable
fix: OAuth 401 不再永久锁死账号,改用临时不可调度实现自动恢复
2026-03-03 16:33:07 +08:00
shaw
4a91954532 fix: correct migration 061 checksum and add missing BillingCache mock methods
- Fix fileChecksum for 061 migration: use TrimSpace hash (66207e7a) instead
  of raw sha256sum (97bdd9a3), matching the actual runtime computation
- Add 222b4a09 as accepted DB checksum for 061 migration
- Add missing GetAPIKeyRateLimit/SetAPIKeyRateLimit/UpdateAPIKeyRateLimitUsage/
  InvalidateAPIKeyRateLimit methods to mock BillingCache in test stubs
- Fix NewBillingCacheService call in singleflight test (add apiKeyRepo param)
2026-03-03 16:11:05 +08:00
shaw
b8b5cec35c fix: resolve CI lint errors and test compilation failures for rate limit feature
- Fix errcheck: properly handle rows.Close() error via named return + defer closure
- Fix gofmt: auto-format billing_cache.go, api_key_service.go, billing_cache_service.go
- Add missing rate limit interface methods to 4 test stubs (GetRateLimitData, IncrementRateLimitUsage, ResetRateLimitWindows)
- Fix NewBillingCacheService calls missing the new apiKeyRepo parameter
2026-03-03 15:43:08 +08:00
Wesley Liddick
43c203333e Merge pull request #733 from DaydreamCoding/fix/group-isolation
fix(gateway): 分组隔离 — 禁止未分组账号被跨组调度
2026-03-03 15:10:30 +08:00
shaw
a80ec5d8bb feat: apikey支持5h/1d/7d速率控制 2026-03-03 15:01:10 +08:00
QTom
530a16291c fix(gateway): 分组隔离 — 禁止未分组账号被跨组调度
当 API Key 无分组时,调度仅从未分组账号池中选取。
修复 isAccountInGroup 在 groupID==nil 时的逻辑,
同时补全 scheduler_snapshot_service 和 gemini_compat_service
中的 SimpleMode 保护,确保分组隔离在所有调度路径生效。

新增 ListSchedulableUngroupedByPlatform/s 方法,
使用 Ent 的 Not(HasAccountGroups()) 谓词实现未分组账号隔离。
新增 17 个单元和端到端隔离测试,覆盖所有分支和边界条件。
2026-03-03 13:20:58 +08:00
Wesley Liddick
b7df7ce5d5 Merge pull request #726 from DaydreamCoding/feat/dual-mode-umq
feat(gateway): 双模式用户消息队列 — 串行队列 + 软性限速
2026-03-03 08:41:34 +08:00
erio
451a851118 fix: remove unused sanitizeCustomMenuItemsJSON function
Replaced by filterUserVisibleMenuItems which includes both array
validation and admin-item filtering.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-03 07:13:08 +08:00
erio
e97c376681 fix: security hardening and architectural improvements for custom menu
1. (Critical) Filter admin-only menu items from public API responses -
   both GetPublicSettings handler and GetPublicSettingsForInjection now
   exclude visibility=admin items, preventing unauthorized access to
   admin menu URLs.

2. (Medium) Validate JSON array structure in sanitizeCustomMenuItemsJSON -
   use json.Unmarshal into []json.RawMessage instead of json.Valid to
   reject non-array JSON values that would cause frontend runtime errors.

3. (Medium) Decouple router from business JSON parsing - move origin
   extraction logic from router.go to SettingService.GetFrameSrcOrigins,
   eliminating direct JSON parsing of custom_menu_items in the routing
   layer.

4. (Low) Restrict custom menu item ID charset to [a-zA-Z0-9_-] via
   regex validation, preventing route-breaking characters like / ? # or
   spaces.

5. (Low) Handle crypto/rand error in generateMenuItemID - return error
   instead of silently ignoring, preventing potential duplicate IDs.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-03 07:05:01 +08:00
erio
7541e243bc style: fix gofmt alignment in setting_service.go 2026-03-03 06:38:04 +08:00
erio
e4f8799323 fix: include custom_menu_items in GetPublicSettingsForInjection 2026-03-03 06:21:23 +08:00
erio
a50d5d351b fix: replace curly quotes with straight quotes in domain_constants.go
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-03 06:18:35 +08:00
erio
067810fa98 feat: custom menu pages with iframe embedding and CSP injection
Add configurable custom menu items that appear in sidebar, each rendering
an iframe-embedded external page. Includes shared URL builder with
src_host/src_url tracking, CSP frame-src multi-origin deduplication,
admin settings UI, and i18n support.

chore: bump version to 0.1.87.19

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-03 06:18:20 +08:00
QTom
a9285b8a94 feat(gateway): 双模式用户消息队列 — 串行队列 + 软性限速
新增 UMQ (User Message Queue) 双模式支持:
- serialize: 账号级分布式串行锁 + RPM 自适应延迟(严格限流)
- throttle: 仅 RPM 自适应前置延迟,不阻塞并发(软性限速)

后端:
- config: 新增 Mode 字段,保留 Enabled 向后兼容
- service: 新增 UserMessageQueueService(Lua 锁/延迟算法/清理 worker)
- repository: 新增 UserMsgQueueCache(Redis Lua acquire/release/force-release)
- handler: 新增 UserMsgQueueHelper(SSE ping + 等待循环 + throttle)
- gateway: 按 mode 分支集成 serialize/throttle 逻辑
- lint: 修复 gofmt rewrite rules、errcheck 类型断言、staticcheck QF1012

前端:
- 三态选择器 UI(关闭/软性限速/串行队列)替代 toggle 开关
- BulkEdit 支持 null 语义(不修改)
- i18n 中英文文案

通过 6 轮专家评审(42 次 review)、golangci-lint、单元测试、集成测试。
2026-03-03 01:05:11 +08:00
zqq61
ec6bcfeb83 fix: OAuth 401 不再永久锁死账号,改用临时不可调度实现自动恢复
OAuth 账号收到 401 时,原逻辑同时设置 expires_at=now() 和 SetError(),
但刷新服务只查询 status=active 的账号,导致 error 状态的账号永远无法
被刷新服务拾取,expires_at=now() 实际上是死代码。

修复:
- OAuth 401 使用 SetTempUnschedulable 替代 SetError,保持 status=active
- 新增 oauth_401_cooldown_minutes 配置项(默认 10 分钟)
- 刷新成功后同步清除 DB 和 Redis 中的临时不可调度状态
- 不可重试错误检查(invalid_grant 等)从 Antigravity 推广到所有平台
- 可重试错误耗尽后不再标记 error,下个刷新周期继续重试

恢复流程:
OAuth 401 → temp_unschedulable + expires_at=now → 刷新服务拾取
  → 成功: 清除 temp_unschedulable → 自动恢复
  → invalid_grant: SetError → 永久禁用
  → 网络错误: 仅记日志 → 下周期重试
2026-03-02 22:54:38 +08:00
QTom
fdcbf7aacf feat(proxy): 集中代理 URL 验证并实现全局 fail-fast
提取 proxyurl.Parse() 公共包,将分散在 6 处的代理 URL 验证逻辑
统一收敛,确保无效代理配置在创建时立即失败,永不静默回退直连。

主要变更:
- 新增 proxyurl 包:统一 TrimSpace → url.Parse → Host 校验 → Scheme 白名单
- socks5:// 自动升级为 socks5h://,防止 DNS 泄漏(大小写不敏感)
- antigravity: http.ProxyURL → proxyutil.ConfigureTransportProxy 支持 SOCKS5
- openai_oauth: 删除 newOpenAIOAuthHTTPClient,收编至 httpclient.GetClient
- 移除未使用的 ProxyStrict 字段(fail-fast 已是全局默认行为)
- 补充 15 个 proxyurl 测试 + pricing/usage fail-fast 测试
2026-03-02 16:04:20 +08:00
Wesley Liddick
445bfdf242 Merge pull request #706 from PMExtra/feat/default-subscriptions-on-user-create
feat(settings): add default subscriptions for new users
2026-03-02 11:38:26 +08:00
PMExtra
0fba1901c8 fix(ci): fix backend unit test constructor arg and gofmt issues 2026-03-02 10:54:14 +08:00
PMExtra
7e02082209 feat(settings): add default subscriptions for new users
- add default subscriptions to admin settings

- auto-assign subscriptions on register and admin user creation

- add validation/tests and align settings UI with subscription selector patterns
2026-03-02 03:59:31 +08:00
QTom
d869ac95fa feat(identity): 指纹缓存 TTL 懒续期机制
- TTL 改为 7 天,配合 24 小时自动续期保持活跃账号永不过期
- 版本升级时采用合并语义,仅更新请求中实际存在的字段
- 添加产品名验证防止浏览器 UA 误判为更新版本
2026-03-02 01:12:41 +08:00
erio
65459a99b6 feat(dashboard): add group usage distribution chart to usage page
Add a doughnut chart showing usage statistics broken down by group on
the admin usage records page. The chart appears alongside the existing
model distribution chart (2-column grid), with the token usage trend
chart moved to a separate full-width row below.

Changes:
- backend/pkg/usagestats: add GroupStat type
- backend/service: add GetGroupStatsWithFilters interface method and implementation
- backend/repository: implement GetGroupStatsWithFilters with LEFT JOIN groups
- backend/handler: add GetGroupStats handler with full filter support
- backend/routes: register GET /admin/dashboard/groups route
- backend/tests: add GetGroupStatsWithFilters stubs to contract/sora tests
- frontend/types: add GroupStat interface
- frontend/api: add getGroupStats API function and types
- frontend/components: add GroupDistributionChart.vue doughnut chart
- frontend/views: update UsageView layout and load group stats in parallel
- frontend/i18n: add groupDistribution, group, noGroup keys (zh + en)
2026-03-01 20:10:51 +08:00
erio
7c5746ffbc feat(usage): add group usage distribution chart alongside model distribution
- Add GroupStat type to usagestats package
- Add GetGroupStatsWithFilters to UsageLogRepository interface and implement with LEFT JOIN groups
- Add GetGroupStats dashboard API endpoint (GET /admin/dashboard/groups)
- Add GroupDistributionChart.vue component mirroring ModelDistributionChart
- Rearrange UsageView layout: model + group in one row, token trend full-width below
- All filters (user, api_key, account, group, model, date range) apply to group stats
2026-03-01 19:49:01 +08:00
Wesley Liddick
fd57fa4913 Merge pull request #690 from touwaeriol/pr/bulk-edit-mixed-channel-warning
feat: add mixed-channel warning for bulk account edit
2026-03-01 18:25:05 +08:00
Wesley Liddick
8c4d22b3f9 Merge pull request #685 from touwaeriol/pr/admin-create-and-redeem-docs
feat(admin): add create-and-redeem endpoint for payment integrations
2026-03-01 18:24:15 +08:00
QTom
b2141a96e2 fix(ci): 修复 golangci-lint 和 API 合约测试失败
- 修复 errcheck: singleflight 返回值类型断言添加 ok 检查
- 修复 gofmt: 格式化 setting_service.go 和 claude_code_validator_test.go
- 修复 TestAPIContracts: 在 GET /admin/settings 期望中添加 min_claude_code_version 字段
2026-03-01 16:39:21 +08:00
erio
0b96c7a65e fix(auth): replace submit turnstile widget with VerifyTurnstileForRegister
Port upstream's VerifyTurnstileForRegister which skips the duplicate
Turnstile check when email verify flow is already completed, instead of
requiring a second Turnstile widget on the verify page.
2026-03-01 16:34:14 +08:00
erio
d4231150a9 fix: add groupExistenceBatchReader interface and update test stub for release branch compatibility 2026-03-01 15:53:45 +08:00
erio
2268e93aec fix: remove unused preload/snapshot functions and fix gofmt 2026-03-01 15:49:26 +08:00
erio
c976916b6d fix: remove dead code in BulkUpdateAccounts group binding loop 2026-03-01 15:49:15 +08:00
erio
0bbd003a18 fix: use i18n for mixed-channel warning messages and improve bulk pre-check
- BulkUpdate handler: add structured details to 409 response
- BulkUpdateAccounts: simplify to global pre-check before any DB write;
  remove per-account snapshot tracking which is no longer needed
- MixedChannelError.Error(): restore English message for API compatibility
- BulkEditAccountModal: use t() with details for both pre-check and 409
  fallback paths instead of displaying raw backend strings
- Update test to verify pre-check blocks on existing group conflicts
2026-03-01 15:49:05 +08:00
QTom
4280aca82c feat(gateway): 添加 Claude Code 客户端最低版本检查功能
- 通过 User-Agent 识别 Claude Code 客户端并提取版本号
- 在网关层验证客户端版本是否满足管理员配置的最低要求
- 在管理后台提供版本要求配置选项(英文/中文双语)
- 实现原子缓存 + singleflight 防止并发问题和 thundering herd
- 使用 context.WithoutCancel 隔离 DB 查询,避免客户端断连影响缓存
- 双 TTL 策略:60s 正常、5s 错误恢复,保证性能与可用性
- 仅检查 Claude Code 客户端,其他客户端不受影响
- 添加完整单元测试覆盖版本提取、比对、上下文操作
2026-03-01 15:45:44 +08:00
erio
c08889b021 fix: remove unused preload/snapshot functions and fix gofmt 2026-03-01 15:22:27 +08:00