Author SHA1 Message Date
Jayway Liu e841af67a6 merge: 同步 main 的 CI 基建(.gitea/workflows 迁移、PG 咨询锁测试适配、artifact 容忍、audit 重试)
Verify / verify (push) Failing after 1s
Verify / verify (pull_request) Failing after 1s
AI Review / review (pull_request) Successful in 14s
2026-09-26 17:02:08 +08:00
Jayway Liu dfaecde8a3 test(worker): 适配 PostgreSQL 咨询锁下的立即重认领语义
Verify / verify (push) Failing after 1s
Verify / verify (pull_request) Failing after 0s
PG 上 worker 持咨询锁会立即重认领 processing 任务,第二次 --once 被导入任务
抢占导致导出断言失败(该测试此前从未在 PG 上真正运行过)。第一次调用后把导入
任务置为 ready 模拟完成,两种后端行为一致
2026-09-26 16:20:40 +08:00
Jayway Liu 9a5ab26014 fix(dialogs): F04 载入已保存定义后把原定义带入可见对话上下文
Verify / verify (push) Failing after 1s
Verify / verify (pull_request) Failing after 1s
- 载入定义时以“用户消息+助手确认”形式把名称、含义、范围、缺失处理和表达式
  写入对话,后续“把刚才公式改成乘以 100”等指代可关联到已载入定义;
  生成前仍需显式确认,不静默沿用旧定义
- 选择空选项时同步清理当前定义与上下文,选择状态与预览保持一致
- 服务端单条消息长度上限 1000→2000 以容纳完整定义上下文,超长仍拒绝
- E2E 覆盖:载入后请求包含原定义、修改后表达式为 (收入−成本)×100、
  空选项清理;单测覆盖 2000/2001 边界
2026-09-26 01:30:34 +08:00
Jayway Liu 6cf236a811 fix(dialogs): F03 证据弹窗同期间重复点击不再停留在加载中
- 第 0 页重复点击当前已选期间直接返回,不清空列表、不重复请求;
  第二页点击当前期间回到第一页并刷新
- 请求失败显示错误并提供“重试”按钮;空结果显示空态行而不是过期数据
- E2E 覆盖:重复点击、快速本期/基期切换、第二页切期间、失败重试、空列表
2026-09-26 01:30:05 +08:00
Jayway Liu 5ebe83afbb fix(worker): F08 执行/下载统一核验角色与账号状态,恢复卡死的 processing 任务
Verify / verify (pull_request) Failing after 25s
Verify / verify (push) Failing after 1s
- worker 执行前重新核验账号 is_active 与 admin/analyst 角色:撤权/停用后排队任务
  直接失败,不发布敏感工作簿
- formula-exports 状态与下载接口补齐与创建接口一致的角色白名单,被降为 viewer
  的账号不能再查询/下载旧导出
- import_worker 对超过 15 分钟仍处于 processing 的任务重新认领(PostgreSQL 依赖
  咨询锁立即恢复),崩溃重启不再无限停留 processing;新近 processing 不被抢占
- 测试:撤权/停用执行、viewer 下载与状态 403、半成品/缺文件下载 409/410、
  超时任务恢复且无重复临时文件、新近 processing 不被重复认领
2026-09-26 01:22:37 +08:00
Jayway Liu 54dc58d2a5 fix(cleanup): F02 清理失败记录含任务编号的告警,孤儿 xlsx 按保留期回收
- process_formula_export 失败清理受阻时记录不含敏感数据的 WARNING(任务编号+文件名),
  并保留 job.path 使残留文件仍可追踪,而不是清空后失联
- cleanup_formula_exports 新增孤儿 .xlsx 回收:超过 24h 保留期且不属于任何
  未过期成功任务的文件统一回收(覆盖失败任务残留、任务行被级联删除两种来源),
  不触碰仍处于活动任务的文件
- 测试:注入 unlink 权限错误→任务失败状态仍保存+告警可见;恢复权限后回收成功
  且其他任务文件不受影响;失败任务残留/任务行已删/未过期成功/近期文件/非 UUID 命名
2026-09-26 01:20:50 +08:00
Jayway Liu 36fd5f3461 fix(exporting): F01 路径完整验证前不登记清理目标,清理不跟随符号链接
- 根目录/用户目录/目标文件先通过 lstat 完整校验(拒绝符号链接与非目录),
  校验失败时不删除任何磁盘文件(此前父目录为链接时会沿链接 unlink 同名文件)
- 失败清理只处理本次登记的受控文件,并通过 O_NOFOLLOW 目录描述符删除;
  目录被换成符号链接时拒绝删除,保留任务编号与路径的对应关系
- 测试覆盖:用户目录链接、目标链接、运行中目录被换成链接(受保护文件字节不变)、
  失败后正常重试成功
2026-09-26 01:18:18 +08:00
31 changed files with 882 additions and 1383 deletions
-1
View File
@@ -12,5 +12,4 @@ MAX_IMPORT_ROWS=1000000
AI_ENABLED=0
OPENAI_API_KEY=
OPENAI_MODEL=
OPENAI_BASE_URL=https://api.openai.com/v1
VITE_AG_GRID_LICENSE_KEY=
+1 -1
View File
@@ -119,7 +119,7 @@ REFERENCE_DATASET_NAME='分析数据_原稿(1).xlsx' npm run test:e2e
## AI 与跨平台
默认 AI_ENABLED=0。启用外部模型有两种方式,数据库配置优先:全权限管理员在页面「管理 → AI 模型」配置服务商(OpenAI 或本地 OpenAI 兼容服务如 Ollama / LM Studio / vLLM)、Base URL、API Key 与模型,可从 `/v1/models` 一键获取本地服务模型列表多选一;或配置环境变量 `AI_ENABLED=1`、`OPENAI_API_KEY`、`OPENAI_MODEL`(可选 `OPENAI_BASE_URL`)。均需完成机构数据处理审批。密钥只在后端:读取接口不回传密钥原文,配置修改记入审计(不含密钥)。用户自由输入的问题也可能含敏感信息,不能仅凭不发送原始行就认定无隐私风险。模型不能提交 SQL、修改金额或修改账号;输出只允许受控意图,异常/拒答关闭执行。
默认 AI_ENABLED=0。启用外部模型需配置 `AI_ENABLED=1`、`OPENAI_API_KEY`、`OPENAI_MODEL`,并完成机构数据处理审批。密钥只在后端。用户自由输入的问题也可能含敏感信息,不能仅凭不发送原始行就认定无隐私风险。模型不能提交 SQL、修改金额或修改账号;输出只允许受控意图,异常/拒答关闭执行。
前端为标准 Web,API 无平台耦合,可由后续 PWA、Tauri 或 Capacitor 客户端复用。当前没有打包桌面/移动客户端,也不在浏览器离线缓存财务数据。
-149
View File
@@ -1,149 +0,0 @@
"""AI 模型配置:每个组织可在页面配置 OpenAI 或本地 OpenAI 兼容服务(Ollama / LM Studio / vLLM 等)。
数据库配置优先;未配置时回退到环境变量(AI_ENABLED / OPENAI_*),保持旧行为兼容。
API Key 只存后端,读取接口只做脱敏回显,绝不把密钥下发到浏览器。
"""
import httpx
from django.conf import settings
from django.http import JsonResponse
from .models import AIConfig
from .views import APIError, audit, body, endpoint
PROVIDERS = ("openai", "local")
OPENAI_BASE_URL = "https://api.openai.com/v1"
def require_full_admin(request):
if request.member.departments:
raise APIError("没有配置 AI 模型的权限", 403)
def resolve(org):
"""返回生效的 AI 配置字典;未启用或配置不完整时返回 None。"""
cfg = AIConfig.objects.filter(organization=org).first()
if cfg:
base_url = (cfg.base_url or "").rstrip("/") or (OPENAI_BASE_URL if cfg.provider == "openai" else "")
key_required = cfg.provider == "openai"
if not cfg.enabled or not base_url or not cfg.model or (key_required and not cfg.api_key):
return None
return {
"enabled": True,
"provider": cfg.provider,
"base_url": base_url,
"api_key": cfg.api_key,
"model": cfg.model,
}
if settings.AI_ENABLED and settings.OPENAI_API_KEY and settings.OPENAI_MODEL:
return {
"enabled": True,
"provider": "openai",
"base_url": (settings.OPENAI_BASE_URL or OPENAI_BASE_URL).rstrip("/"),
"api_key": settings.OPENAI_API_KEY,
"model": settings.OPENAI_MODEL,
}
return None
def org_models_endpoint(org):
"""组织配置或环境回退的 base_url/api_key,用于探测模型列表。"""
cfg = AIConfig.objects.filter(organization=org).first()
if cfg:
base_url = (cfg.base_url or "").rstrip("/") or (OPENAI_BASE_URL if cfg.provider == "openai" else "")
return base_url, cfg.api_key
return (settings.OPENAI_BASE_URL or OPENAI_BASE_URL).rstrip("/"), settings.OPENAI_API_KEY
@endpoint(("GET", "PUT"), roles=("admin",))
def ai_config(request):
require_full_admin(request)
org = request.member.organization
if request.method == "GET":
cfg = AIConfig.objects.filter(organization=org).first()
return JsonResponse(
{
"enabled": cfg.enabled if cfg else False,
"provider": cfg.provider if cfg else "local",
"base_url": cfg.base_url if cfg else "",
"model": cfg.model if cfg else "",
"api_key_set": bool(cfg and cfg.api_key),
"updated_at": cfg.updated_at.isoformat() if cfg else None,
}
)
data = body(request)
enabled = data.get("enabled")
provider = data.get("provider")
base_url = data.get("base_url", "")
api_key = data.get("api_key", "")
model = data.get("model", "")
if type(enabled) is not bool or provider not in PROVIDERS:
raise APIError("AI 配置参数无效")
if not isinstance(base_url, str) or len(base_url) > 500:
raise APIError("Base URL 无效")
base_url = base_url.rstrip("/")
if base_url and not base_url.startswith(("http://", "https://")):
raise APIError("Base URL 必须以 http:// 或 https:// 开头")
if not isinstance(api_key, str) or len(api_key) > 500:
raise APIError("API Key 无效")
if not isinstance(model, str) or len(model) > 200:
raise APIError("模型名无效")
cfg = AIConfig.objects.filter(organization=org).first()
if enabled:
if provider == "local" and not base_url:
raise APIError("本地服务需要填写 Base URL,例如 http://127.0.0.1:11434/v1")
if provider == "openai" and not (api_key or (cfg and cfg.api_key)):
raise APIError("OpenAI 服务需要 API Key")
if not model:
raise APIError("启用前需要填写模型名,可点击“获取可用模型”选择")
key_changed = bool(api_key)
defaults = {
"enabled": enabled,
"provider": provider,
"base_url": base_url,
"model": model,
}
if api_key:
defaults["api_key"] = api_key
if not cfg:
AIConfig.objects.create(organization=org, **defaults)
else:
for field, value in defaults.items():
setattr(cfg, field, value)
cfg.save()
audit(
request,
"ai_config.update",
details={
"provider": provider,
"model": model,
"enabled": enabled,
"key_changed": key_changed,
},
)
return JsonResponse({"ok": True})
@endpoint(roles=("admin",))
def ai_models(request):
require_full_admin(request)
base_url, api_key = org_models_endpoint(request.member.organization)
if not base_url:
raise APIError("请先保存 Base URL 再获取模型列表", 400)
headers = {"Authorization": f"Bearer {api_key}"} if api_key else {}
try:
with httpx.Client(timeout=10) as client:
response = client.get(f"{base_url}/models", headers=headers)
response.raise_for_status()
data = response.json()
items = data.get("data", [])
ids = [m.get("id") for m in items if isinstance(m, dict) and isinstance(m.get("id"), str)]
if not ids and isinstance(data.get("models"), list):
ids = [
m.get("name")
for m in data["models"]
if isinstance(m, dict) and isinstance(m.get("name"), str)
]
return JsonResponse({"items": ids[:200]})
except (httpx.HTTPError, ValueError, AttributeError) as exc:
raise APIError("无法从该服务获取模型列表,请检查 Base URL 与 API Key", 502) from exc
+13 -17
View File
@@ -1,17 +1,14 @@
import json
import httpx
from django.conf import settings
from django.http import JsonResponse
from .ai import resolve
from .analysis import METRICS, analyze, context, dimensions
from .views import APIError, audit, body, dataset, endpoint, limit, preferences
def model_intent(question, ds, ctx, cfg=None):
cfg = cfg or resolve(ds.organization)
if not cfg:
raise APIError("尚未配置外部 AI 服务", 503)
def model_intent(question, ds, ctx):
keys = [d["key"] for d in dimensions(ds)]
schema = {
"type": "object",
@@ -34,7 +31,7 @@ def model_intent(question, ds, ctx, cfg=None):
}
# Only schema, selected context and the user's question leave the server; never raw rows.
payload = {
"model": cfg["model"],
"model": settings.OPENAI_MODEL,
"store": False,
"max_output_tokens": 1200,
"input": [
@@ -58,8 +55,8 @@ def model_intent(question, ds, ctx, cfg=None):
try:
with httpx.Client(timeout=30) as client:
response = client.post(
f"{cfg['base_url']}/responses",
headers={"Authorization": f"Bearer {cfg['api_key']}"} if cfg["api_key"] else {},
"https://api.openai.com/v1/responses",
headers={"Authorization": f"Bearer {settings.OPENAI_API_KEY}"},
json=payload,
)
response.raise_for_status()
@@ -110,15 +107,12 @@ def assistant(request, dataset_id):
use_model = data.get("use_model", False)
if type(use_model) is not bool:
raise APIError("AI 模式必须为布尔值")
provider = "local"
if use_model:
cfg = resolve(request.member.organization)
if not cfg:
raise APIError("尚未配置外部 AI 服务;请管理员在「AI 模型」中配置", 503)
if not (settings.AI_ENABLED and settings.OPENAI_API_KEY and settings.OPENAI_MODEL):
raise APIError("尚未配置外部 AI 服务", 503)
if data.get("consent") is not True:
raise APIError("请先同意发送问题与当前筛选上下文", 400)
provider = cfg["provider"]
intent = model_intent(question, ds, ctx, cfg)
intent = model_intent(question, ds, ctx)
else:
action = (
"evidence"
@@ -146,19 +140,21 @@ def assistant(request, dataset_id):
else:
change = value["change"]
change_text = f"{change:+.1%}" if change is not None else "不可计算(基期为零或数据缺失)"
answer = f"当前范围{METRICS[metric]}为 {value['current']:,.2f},较基期变化 {change_text}。按当前规则识别出 {result['anomaly_count']} 个异常分组。"
answer = f"当前范围{METRICS[metric]}为 {value['current']:,.4f},较基期变化 {change_text}。按当前规则识别出 {result['anomaly_count']} 个异常分组。"
candidates = [r for r in result["rows"] if r["anomaly"]][:3]
if candidates:
answer += " 优先核查:" + "、".join(r["label"] for r in candidates) + "。"
answer += " 这是数据贡献分解,不是因果结论。"
audit(request, "assistant.query", ds.pk, {"provider": provider, "intent": intent})
audit(
request, "assistant.query", ds.pk, {"provider": "openai" if use_model else "local", "intent": intent}
)
return JsonResponse(
{
"answer": answer,
"intent": intent,
"context": result["context"],
"version": ds.sha256,
"provider": provider,
"provider": "openai" if use_model else "local",
"evidence": [
{"label": r["label"], "filters": r["filters"], "values": r["values"][metric]}
for r in result["rows"][:5]
+79 -98
View File
@@ -3,6 +3,7 @@ import hmac
import logging
import os
import secrets
import stat
import warnings
from datetime import timedelta
from decimal import Decimal, InvalidOperation
@@ -12,62 +13,74 @@ from django.conf import settings
from django.utils import timezone
from openpyxl import Workbook
from openpyxl.cell import WriteOnlyCell
from openpyxl.styles import Font, PatternFill, Protection
from openpyxl.worksheet.datavalidation import DataValidation
from openpyxl.styles import Font, PatternFill
from openpyxl.worksheet.table import Table, TableColumn, TableStyleInfo
from .analysis import METRICS, aggregations, context, dimensions, metrics, scoped_facts
from .formula_columns import (
RESERVED_DATA_HEADERS,
ensure_available_metrics,
evaluate,
metrics_used,
to_excel,
validate_column,
)
from .models import FormulaExport, Membership
# Keep the private test seam while sharing the single evaluator implementation.
_evaluate_expression = evaluate
FORMULA_VERSION = "3"
MAX_ROWS = 100_000
MAX_GROUPS = 200
RECONCILE_TOLERANCES = {
"customers": Decimal("0"),
"margin": Decimal("0.0001"),
"discount_rate": Decimal("0.0001"),
"upgrade_rate": Decimal("0.0001"),
}
DEFAULT_RECONCILE_TOLERANCE = Decimal("0.01")
logger = logging.getLogger(__name__)
def _safe_unlink_export(path, root):
"""Delete only a file below the owned export root without following links."""
path, root = Path(path), Path(root).absolute()
if not path.is_absolute() or root.is_symlink() or path.is_symlink():
logger.warning("formula export cleanup refused unsafe path: %s", path)
return False
parent = path.parent
def _validated_export_directory(path, description):
"""导出目录必须不存在或为真实目录;符号链接一律拒绝并保持文件原样。"""
try:
if not parent.is_relative_to(root):
logger.warning("formula export cleanup refused out-of-root path: %s", path)
return False
except ValueError:
return False
current = parent
while current != root:
if current.is_symlink() or current.parent == current:
logger.warning("formula export cleanup refused linked parent: %s", path)
return False
current = current.parent
info = os.lstat(path)
except FileNotFoundError:
path.mkdir(parents=True, mode=0o700)
info = os.lstat(path)
if stat.S_ISLNK(info.st_mode) or not stat.S_ISDIR(info.st_mode):
raise ValueError(f"{description}必须是真实目录,不能是符号链接")
os.chmod(path, 0o700)
return path
def _validated_export_target(user_dir, job):
"""目标文件若已存在必须是普通文件;符号链接或其他类型一律拒绝覆盖。"""
target = user_dir / f"{job.pk}.xlsx"
try:
path.unlink(missing_ok=True)
return True
info = os.lstat(target)
except FileNotFoundError:
return target
if stat.S_ISLNK(info.st_mode) or not stat.S_ISREG(info.st_mode):
raise ValueError("公式导出目标路径已存在且不是普通文件,已拒绝生成")
return target
def _unlink_created_file(path):
"""只通过不跟随链接的目录句柄删除受控目录内的普通文件。
返回 "removed"(已删除)、"missing"(本就不存在)或 "blocked"(无法安全删除,
例如目录已被换成符号链接或权限不足),调用方据此保留失败任务与磁盘的一致性。
"""
try:
dir_fd = os.open(path.parent, os.O_RDONLY | os.O_NOFOLLOW)
except OSError:
logger.warning("formula export cleanup failed for %s", path, exc_info=True)
return False
return "blocked"
try:
try:
info = os.lstat(path.name, dir_fd=dir_fd)
except FileNotFoundError:
return "missing"
if not stat.S_ISREG(info.st_mode):
return "blocked"
os.unlink(path.name, dir_fd=dir_fd)
return "removed"
except OSError:
return "blocked"
finally:
os.close(dir_fd)
def _excel_number(value):
@@ -310,19 +323,6 @@ def build_formula_workbook(ds, member, export_context, output):
),
)
data.append(headers)
editable_indices = {len(dims) + 2, len(dims) + 3, len(dims) + 4, len(dims) + 5, len(dims) + 7}
if count:
numeric_input = DataValidation(
type="decimal", operator="between", formula1="-1e15", formula2="1e15", allow_blank=True
)
numeric_input.showErrorMessage = True
numeric_input.errorStyle = "stop"
numeric_input.errorTitle = "请输入数值"
numeric_input.error = "计算输入只接受范围内的数值或空白;请勿粘贴公式或文本。"
for index in editable_indices:
column_name = _column_name(index + 1)
numeric_input.add(f"{column_name}2:{column_name}{count + 1}")
data.data_validations.append(numeric_input)
token_secret = secrets.token_bytes(32)
customer_roster = {}
gross_index = next((col["index"] for col in mapping["columns"] if col["role"] == "gross"), None)
@@ -383,12 +383,6 @@ def build_formula_workbook(ds, member, export_context, output):
f'{_column_name(len(dims) + 6)}{row_number},""))'
),
)
for index in editable_indices:
cell = row[index]
if not hasattr(cell, "protection"):
cell = WriteOnlyCell(data, value=cell)
cell.protection = Protection(locked=False)
row[index] = cell
data.append(row)
customer_headers = [
"Group",
@@ -468,13 +462,7 @@ def build_formula_workbook(ds, member, export_context, output):
],
["ROI 口径", mapping["roi_formula"], "按导入映射保存的公式分支", "否"],
["自营产品集合", _text_cell(parameters, ", ".join(products)), "升单率和自营收入口径", "否"],
["产品毛利率快照", str(rates), "导入时保存的只读口径;修改需重新确认导入映射", "否"],
[
"对账容差",
"人数 0;金额 0.01;比率 0.0001;自定义 0.01",
"服务器 Decimal 参考与 Excel 结果的绝对差;正式财务验收前需确认",
"否",
],
["产品毛利率", str(rates), "直接毛利缺失时的可编辑计算输入", "是"],
[
"Excel 数值精度",
"IEEE-754 双精度",
@@ -509,10 +497,6 @@ def build_formula_workbook(ds, member, export_context, output):
["变化率", "差额/ABS(基期);基期为0或期间缺失时不可用。"],
["精度", "原始 Decimal 以文本留存;Excel 双精度存在细微误差可能。"],
["隐私", "客户标识替换为本文件随机 token,文件仍按敏感财务信息保护。"],
[
"编辑保护",
"工作表结构、人数去重、原始证据和公式列已保护;计算明细仅收入、成本、毛利输入、毛利率和原价金额可编辑。行结构变化请重新导出。",
],
]:
instructions.append(row)
for definition in custom_columns:
@@ -629,7 +613,6 @@ def build_formula_workbook(ds, member, export_context, output):
for key, name in METRICS.items():
formula_row = metric_rows[(label, key)]
value = reference[key]
tolerance = RECONCILE_TOLERANCES.get(key, DEFAULT_RECONCILE_TOLERANCE)
reconcile.append(
[
name,
@@ -638,30 +621,24 @@ def build_formula_workbook(ds, member, export_context, output):
"" if value is None else str(value),
f"='分析汇总'!{col}{formula_row}",
f'=IFERROR(E{reconcile_row}-D{reconcile_row},"")',
f'=IF(D{reconcile_row}="","无参考",IF(ISNUMBER(F{reconcile_row}),IF(ABS(F{reconcile_row})<={tolerance},"通过","待复核"),"待复核"))',
f'=IF(D{reconcile_row}="","无参考",IF(ISNUMBER(F{reconcile_row}),IF(ABS(F{reconcile_row})<0.01,"通过","待复核"),"待复核"))',
]
)
reconcile_row += 1
for index, definition in enumerate(custom_columns):
formula_row = metric_rows[(label, f"custom_{index}")]
custom_reference = evaluate(definition["expression"], reference)
reconcile.append(
[
_text_cell(reconcile, definition["name"]),
_text_cell(reconcile, label),
side_name,
"" if custom_reference is None else str(custom_reference),
"自定义公式无服务器参考",
f"='分析汇总'!{col}{formula_row}",
f'=IFERROR(E{reconcile_row}-D{reconcile_row},"")',
f'=IF(D{reconcile_row}="","无参考",IF(ISNUMBER(F{reconcile_row}),IF(ABS(F{reconcile_row})<={DEFAULT_RECONCILE_TOLERANCE},"通过","待复核"),"待复核"))',
"",
"需业务验收",
]
)
reconcile_row += 1
for sheet in wb.worksheets:
sheet.protection.sheet = True
sheet.protection.selectLockedCells = False
sheet.protection.selectUnlockedCells = True
wb.security.lockStructure = True
wb.calculation.fullCalcOnLoad = True
wb.calculation.forceFullCalc = True
wb.calculation.calcMode = "auto"
@@ -671,42 +648,46 @@ def build_formula_workbook(ds, member, export_context, output):
def process_formula_export(export_id):
job = FormulaExport.objects.select_related("dataset", "user").get(pk=export_id)
temporary = None
target = None
root = Path(settings.MEDIA_ROOT) / "formula-exports"
created = []
try:
if root.is_symlink():
raise ValueError("公式导出目录不能是符号链接")
root.mkdir(parents=True, exist_ok=True, mode=0o700)
root.chmod(0o700)
target = root / str(job.user_id) / f"{job.pk}.xlsx"
target.parent.mkdir(parents=True, exist_ok=True, mode=0o700)
if target.parent.is_symlink() or target.is_symlink():
raise ValueError("公式导出路径不能是符号链接")
target.parent.chmod(0o700)
temporary = target.with_name(f"{job.pk}.{secrets.token_hex(8)}.tmp")
member = Membership.objects.get(user=job.user, organization=job.dataset.organization)
if not job.user.is_active or member.role not in ("admin", "analyst"):
raise PermissionError("账号已撤权,请重新提交导出")
# 执行阶段重新核验角色与账号状态:撤权或停用后排队中的任务不得继续发布敏感工作簿。
if not job.user.is_active:
raise PermissionError("账号已停用,公式导出已拒绝执行")
if member.role not in ("admin", "analyst"):
raise PermissionError("账号角色已变更,公式导出已拒绝执行")
if set(member.departments) != set(job.departments):
raise PermissionError("科室授权已变更,请重新提交导出")
root = _validated_export_directory(Path(settings.MEDIA_ROOT) / "formula-exports", "公式导出目录")
user_dir = _validated_export_directory(root / str(job.user_id), "公式导出用户目录")
target = _validated_export_target(user_dir, job)
# 路径完整验证通过后才把目标登记为可清理文件;此前任何失败都不会删除磁盘文件。
created.append(target)
temporary = target.with_name(f"{job.pk}.{secrets.token_hex(8)}.tmp")
descriptor = os.open(temporary, os.O_WRONLY | os.O_CREAT | os.O_EXCL, 0o600)
created.append(temporary)
with os.fdopen(descriptor, "wb") as file:
stats = build_formula_workbook(job.dataset, member, job.context, file)
temporary.replace(target)
os.replace(temporary, target)
created = [target]
job.path, job.status, job.error = str(target), "succeeded", ""
job.finished_at = timezone.now()
job.expires_at = timezone.now() + timedelta(hours=24)
job.save(update_fields=["path", "status", "error", "finished_at", "expires_at"])
return stats
except Exception as exc:
remaining = ""
for path in (temporary, target):
if path is not None and not _safe_unlink_export(path, root):
if path.exists() or path.is_symlink():
remaining = str(path)
job.status, job.error, job.path = "failed", str(exc)[:500], remaining
job.finished_at, job.expires_at = timezone.now(), None
job.status, job.error, job.finished_at, job.expires_at = "failed", str(exc)[:500], timezone.now(), None
# 只清理本次登记且确认位于受控目录内的文件;目录被换成符号链接时拒绝删除。
leftover = [path for path in created if _unlink_created_file(path) == "blocked"]
if leftover:
# 日志只含任务编号与文件名,不包含敏感数据;残留文件由清理命令按保留期回收。
logger.warning(
"公式导出任务 %s 失败后清理文件受阻,待清理命令回收:%s",
job.pk,
"、".join(path.name for path in leftover),
)
job.path = str(target) if target in leftover else ""
job.save(update_fields=["status", "error", "path", "finished_at", "expires_at"])
raise
-25
View File
@@ -1,5 +1,4 @@
import math
from decimal import Decimal, localcontext
from .analysis import METRICS
@@ -108,27 +107,3 @@ def to_excel(node, references):
if "number" in node:
return str(node["number"])
return f"({to_excel(node['left'], references)}{OPERATORS[node['operator']]}{to_excel(node['right'], references)})"
def evaluate(node, references):
"""Evaluate a validated expression with Decimal values and #N/A semantics."""
with localcontext() as decimal_context:
decimal_context.prec = 64
return _evaluate(node, references)
def _evaluate(node, references):
if "metric" in node:
return references.get(node["metric"])
if "number" in node:
return Decimal(str(node["number"]))
left, right = _evaluate(node["left"], references), _evaluate(node["right"], references)
if left is None or right is None:
return None
if node["operator"] == "add":
return left + right
if node["operator"] == "subtract":
return left - right
if node["operator"] == "multiply":
return left * right
return None if right == 0 else left / right
+21 -82
View File
@@ -3,14 +3,12 @@ from pathlib import Path
import httpx
from django.conf import settings
from django.db import transaction
from django.db.models import Q
from django.http import FileResponse, JsonResponse
from django.utils import timezone
from .analysis import METRICS, context, dimensions
from .formula_columns import available_metrics, ensure_available_metrics, validate_column
from .models import Dataset, FormulaDefinition, FormulaExport
from .models import FormulaDefinition, FormulaExport
from .views import APIError, audit, body, dataset, endpoint, limit
@@ -24,7 +22,8 @@ def design_column(messages, ds, ctx):
or set(item) != {"role", "content"}
or item["role"] not in ("user", "assistant")
or not isinstance(item["content"], str)
or len(item["content"]) > 1000
# 2000 字符容纳载入已保存定义后的完整对话上下文
or len(item["content"]) > 2000
for item in messages
):
raise APIError("对话消息内容无效")
@@ -192,23 +191,13 @@ def formula_definitions(request, dataset_id):
if request.member.departments:
raise APIError("科室授权账号不可管理跨范围公式定义", 403)
if request.method == "GET":
items = (
FormulaDefinition.objects.filter(dataset=ds)
.filter(Q(user=request.user) | Q(shared=True))
.select_related("user")
.order_by("normalized_name")
)
items = FormulaDefinition.objects.filter(dataset=ds, user=request.user).order_by("normalized_name")
return JsonResponse(
{
"items": [
{
"id": str(item.pk),
"definition": item.definition,
"revision": item.revision,
"shared": item.shared,
"owned": item.user_id == request.user.pk,
"owner": item.user.username,
"history": item.history,
"updated_at": item.updated_at.isoformat(),
}
for item in items
@@ -216,74 +205,28 @@ def formula_definitions(request, dataset_id):
}
)
data = body(request)
definition = validate_column(data.get("definition"))
shared = data.get("shared", False)
if type(shared) is not bool:
raise APIError("共享选项无效")
definition = validate_column(body(request).get("definition"))
try:
ensure_available_metrics(ds, definition)
except ValueError as exc:
raise APIError(str(exc)) from exc
normalized_name = definition["name"].strip().casefold()
requested_revision = data.get("revision")
with transaction.atomic():
Dataset.objects.select_for_update().get(pk=ds.pk)
existing = (
FormulaDefinition.objects.select_for_update()
.filter(dataset=ds, user=request.user, normalized_name=normalized_name)
.first()
)
if existing:
if requested_revision is not None and (
type(requested_revision) is not int or requested_revision != existing.revision
):
raise APIError("公式定义已在其他窗口更新,请重新载入", 409)
# ponytail: retain 20 revisions; use a separate history table if long-term retention is required.
existing.history = [
*existing.history,
{
"revision": existing.revision,
"definition": existing.definition,
"updated_at": existing.updated_at.isoformat(),
},
][-20:]
existing.definition = definition
existing.shared = shared if "shared" in data else existing.shared
existing.revision += 1
existing.save(update_fields=["definition", "revision", "shared", "history", "updated_at"])
item, created = existing, False
else:
if FormulaDefinition.objects.filter(dataset=ds, user=request.user).count() >= 50:
raise APIError("每个数据集最多保存 50 个公式定义")
item = FormulaDefinition.objects.create(
dataset=ds,
user=request.user,
normalized_name=normalized_name,
definition=definition,
shared=shared,
)
created = True
audit(
request,
"formula_definition.save",
item.pk,
{
"dataset": str(ds.pk),
"name": definition["name"],
"shared": item.shared,
"revision": item.revision,
},
)
existing = FormulaDefinition.objects.filter(
dataset=ds, user=request.user, normalized_name=normalized_name
).first()
if not existing and FormulaDefinition.objects.filter(dataset=ds, user=request.user).count() >= 50:
raise APIError("每个数据集最多保存 50 个公式定义")
item, created = FormulaDefinition.objects.update_or_create(
dataset=ds,
user=request.user,
normalized_name=normalized_name,
defaults={"definition": definition},
)
audit(request, "formula_definition.save", item.pk, {"dataset": str(ds.pk), "name": definition["name"]})
return JsonResponse(
{
"id": str(item.pk),
"definition": item.definition,
"revision": item.revision,
"shared": item.shared,
"owned": True,
"owner": request.user.username,
"history": item.history,
"updated_at": item.updated_at.isoformat(),
},
status=201 if created else 200,
@@ -299,10 +242,8 @@ def formula_definition_delete(request, dataset_id, definition_id):
if not item:
raise APIError("公式定义不存在", 404)
name = item.definition.get("name", "")
with transaction.atomic():
item = FormulaDefinition.objects.select_for_update().get(pk=item.pk)
audit(request, "formula_definition.delete", item.pk, {"dataset": str(ds.pk), "name": name})
item.delete()
audit(request, "formula_definition.delete", item.pk, {"dataset": str(ds.pk), "name": name})
item.delete()
return JsonResponse({"deleted": True})
@@ -339,8 +280,6 @@ def formula_export_create(request, dataset_id):
def _owned_job(request, dataset_id, export_id):
if not request.user.is_active or request.member.role not in ("admin", "analyst"):
raise APIError("没有此导出任务的访问权限", 403)
job = FormulaExport.objects.filter(
pk=export_id,
dataset_id=dataset_id,
@@ -354,7 +293,7 @@ def _owned_job(request, dataset_id, export_id):
return job
@endpoint(("GET",))
@endpoint(("GET",), roles=("admin", "analyst"))
def formula_export_status(request, dataset_id, export_id):
job = _owned_job(request, dataset_id, export_id)
return JsonResponse(
@@ -367,7 +306,7 @@ def formula_export_status(request, dataset_id, export_id):
)
@endpoint(("GET",))
@endpoint(("GET",), roles=("admin", "analyst"))
def formula_export_download(request, dataset_id, export_id):
job = _owned_job(request, dataset_id, export_id)
if job.status != "succeeded" or not job.path or not job.expires_at or job.expires_at <= timezone.now():
@@ -1,19 +1,20 @@
import time
import uuid
from datetime import timedelta
from pathlib import Path
from django.conf import settings
from django.core.management.base import BaseCommand
from django.db import transaction
from django.db.models import Q
from django.utils import timezone
from finance.exporting import _safe_unlink_export
from finance.models import FormulaExport
ORPHAN_RETENTION_HOURS = 24
class Command(BaseCommand):
help = "Remove expired formula workbooks and stale temporary files"
help = "Remove expired formula workbooks and reclaim orphan files"
def add_arguments(self, parser):
parser.add_argument("--loop", action="store_true")
@@ -21,8 +22,10 @@ class Command(BaseCommand):
def handle(self, *args, **options):
while True:
jobs, files = self.clean_expired()
self.stdout.write(f"Expired {jobs} jobs; removed {files} files")
jobs, files, orphans = self.clean_expired()
self.stdout.write(
f"Expired {jobs} jobs; removed {files} files; reclaimed {orphans} orphan files"
)
if not options["loop"]:
return
time.sleep(max(60, options["interval"]))
@@ -31,30 +34,22 @@ class Command(BaseCommand):
export_root = Path(settings.MEDIA_ROOT) / "formula-exports"
if export_root.is_symlink():
raise ValueError("公式导出目录不能是符号链接")
root = export_root.absolute()
root = export_root.resolve()
now = timezone.now()
jobs = files = 0
jobs = files = orphans = 0
jobs_qs = FormulaExport.objects.filter(
Q(status="succeeded", expires_at__lte=now) | Q(status="failed", path__gt="")
).iterator()
for job in jobs_qs:
for job in FormulaExport.objects.filter(status="succeeded", expires_at__lte=now).iterator():
path = Path(job.path) if job.path else None
resolved = path.resolve() if path and path.is_absolute() and not path.is_symlink() else None
in_root = bool(resolved and resolved.is_relative_to(root))
try:
with transaction.atomic():
deleted = bool(path and _safe_unlink_export(path, root))
if job.status == "succeeded":
if deleted or not path or not path.is_absolute() or not path.is_relative_to(root):
if deleted:
files += 1
job.status, job.path = "expired", ""
job.save(update_fields=["status", "path"])
jobs += 1
elif deleted:
if in_root and resolved.is_file():
resolved.unlink()
files += 1
job.path = ""
job.save(update_fields=["path"])
jobs += 1
job.status, job.path = "expired", ""
job.save(update_fields=["status", "path"])
jobs += 1
except OSError as exc:
self.stderr.write(f"Could not remove expired export {job.pk}: {exc}")
@@ -71,5 +66,41 @@ class Command(BaseCommand):
files += 1
except OSError as exc:
self.stderr.write(f"Could not remove stale temporary export {path.name}: {exc}")
orphans += self._reclaim_orphan_workbooks(user_dir, now)
return jobs, files
return jobs, files, orphans
def _reclaim_orphan_workbooks(self, user_dir, now):
"""回收超过保留期且不再属于任何有效导出任务的 .xlsx 孤儿文件。
有效任务指状态 succeeded、路径一致且尚未过期的任务;失败任务残留的文件和任务行
已被删除(如数据集级联删除)的文件在保留期后回收,绝不触碰活动任务的文件。
"""
retention = (now - timedelta(hours=ORPHAN_RETENTION_HOURS)).timestamp()
removed = 0
for path in user_dir.glob("*.xlsx"):
try:
if path.is_symlink() or not path.is_file() or path.stat().st_mtime > retention:
continue
job_id = uuid.UUID(path.stem)
except (OSError, ValueError):
continue
job = FormulaExport.objects.filter(pk=job_id).first()
if (
job is not None
and job.status == "succeeded"
and job.path == str(path)
and job.expires_at is not None
and job.expires_at > now
):
continue
try:
path.unlink()
removed += 1
if job is not None and job.path == str(path):
job.path = ""
job.save(update_fields=["path"])
self.stdout.write(f"Reclaimed leftover workbook of job {job.pk}")
except OSError as exc:
self.stderr.write(f"Could not reclaim orphan export {path.stem}: {exc}")
return removed
+24 -14
View File
@@ -1,13 +1,19 @@
import time
from datetime import timedelta
from django.core.management.base import BaseCommand
from django.db import connection, transaction
from django.db.models import Q
from django.utils import timezone
from finance.exporting import process_formula_export
from finance.importing import process_dataset
from finance.models import Dataset, FormulaExport
# 无咨询锁的数据库(如 SQLite)上等待该时限后才重新认领 processing 任务,
# 避免与仍在运行的 worker 抢任务;PostgreSQL 由咨询锁保证单 worker,可立即恢复。
STALE_PROCESSING_SECONDS = 15 * 60
class Command(BaseCommand):
help = "Process durable import and formula-export jobs"
@@ -24,36 +30,41 @@ class Command(BaseCommand):
self.stdout.write("Another worker owns the database lock")
return
try:
self.run_jobs(options, True)
self.run_jobs(options, postgres)
finally:
if postgres:
with connection.cursor() as cursor:
cursor.execute("SELECT pg_advisory_unlock(782146291)")
def run_jobs(self, options, recover):
def run_jobs(self, options, recover_immediately):
while True:
processed = self._next_import(recover)
processed = self._next_import(recover_immediately)
if not processed:
processed = self._next_export(recover)
processed = self._next_export(recover_immediately)
if options["once"]:
return
if not processed:
time.sleep(2)
def _claim(self, model, states):
def _claim(self, model, immediate_recovery):
stale_before = timezone.now() - timedelta(seconds=STALE_PROCESSING_SECONDS)
claimable = Q(status="queued") | Q(status="processing", created_at__lt=stale_before)
if immediate_recovery:
claimable = Q(status="queued") | Q(status="processing")
with transaction.atomic():
qs = model.objects.filter(status__in=states).order_by("created_at")
qs = model.objects.filter(claimable).order_by("created_at")
if connection.features.has_select_for_update_skip_locked:
qs = qs.select_for_update(skip_locked=True)
item = qs.first()
if not item:
return None
updated = model.objects.filter(pk=item.pk, status__in=states).update(status="processing")
return item if updated else None
# CAS 认领 queued 任务互斥;processing 仅在超时或咨询锁保护下重认领,
# 重复发布对同一目标路径写入相同内容,不会产生重复文件。
claimed = model.objects.filter(pk=item.pk, status=item.status).update(status="processing")
return item if claimed else None
def _next_import(self, recover):
states = ["queued", "processing"] if recover else ["queued"]
item = self._claim(Dataset, states)
def _next_import(self, recover_immediately):
item = self._claim(Dataset, recover_immediately)
if not item:
return False
Dataset.objects.filter(pk=item.pk).update(started_at=timezone.now())
@@ -61,9 +72,8 @@ class Command(BaseCommand):
self.stdout.write(f"{result.pk}: {result.status}, {result.row_count} rows")
return True
def _next_export(self, recover):
states = ["queued", "processing"] if recover else ["queued"]
item = self._claim(FormulaExport, states)
def _next_export(self, recover_immediately):
item = self._claim(FormulaExport, recover_immediately)
if not item:
return False
try:
-43
View File
@@ -1,43 +0,0 @@
# Generated by Django 5.2.17 on 2026-09-26 13:56
import django.db.models.deletion
from django.db import migrations, models
class Migration(migrations.Migration):
dependencies = [
("finance", "0004_formuladefinition"),
]
operations = [
migrations.CreateModel(
name="AIConfig",
fields=[
(
"id",
models.BigAutoField(
auto_created=True, primary_key=True, serialize=False, verbose_name="ID"
),
),
("enabled", models.BooleanField(default=False)),
(
"provider",
models.CharField(
choices=[("openai", "OpenAI"), ("local", "本地 OpenAI 兼容服务")],
default="local",
max_length=16,
),
),
("base_url", models.CharField(blank=True, max_length=500)),
("api_key", models.CharField(blank=True, max_length=500)),
("model", models.CharField(blank=True, max_length=200)),
("updated_at", models.DateTimeField(auto_now=True)),
(
"organization",
models.OneToOneField(
on_delete=django.db.models.deletion.CASCADE, to="finance.organization"
),
),
],
),
]
@@ -1,15 +0,0 @@
from django.db import migrations, models
class Migration(migrations.Migration):
dependencies = [
("finance", "0005_aiconfig"),
]
operations = [
migrations.AddField(
model_name="formuladefinition",
name="revision",
field=models.PositiveIntegerField(default=1),
),
]
@@ -1,15 +0,0 @@
from django.db import migrations, models
class Migration(migrations.Migration):
dependencies = [
("finance", "0006_formuladefinition_revision"),
]
operations = [
migrations.AlterField(
model_name="dataset",
name="file",
field=models.FileField(max_length=500, upload_to="sources/%Y/%m"),
),
]
@@ -1,22 +0,0 @@
# Generated by Django 5.2.17 on 2026-10-07 23:50
from django.db import migrations, models
class Migration(migrations.Migration):
dependencies = [
("finance", "0007_alter_dataset_file"),
]
operations = [
migrations.AddField(
model_name="formuladefinition",
name="history",
field=models.JSONField(default=list),
),
migrations.AddField(
model_name="formuladefinition",
name="shared",
field=models.BooleanField(default=False),
),
]
+1 -16
View File
@@ -22,7 +22,7 @@ class Dataset(models.Model):
organization = models.ForeignKey(Organization, on_delete=models.PROTECT)
created_by = models.ForeignKey(settings.AUTH_USER_MODEL, on_delete=models.PROTECT)
name = models.CharField(max_length=200)
file = models.FileField(upload_to="sources/%Y/%m", max_length=500)
file = models.FileField(upload_to="sources/%Y/%m")
sha256 = models.CharField(max_length=64)
status = models.CharField(max_length=16, default="mapping")
profile = models.JSONField(default=dict)
@@ -92,18 +92,6 @@ class Audit(models.Model):
created_at = models.DateTimeField(auto_now_add=True)
class AIConfig(models.Model):
organization = models.OneToOneField(Organization, on_delete=models.CASCADE)
enabled = models.BooleanField(default=False)
provider = models.CharField(
max_length=16, choices=[("openai", "OpenAI"), ("local", "本地 OpenAI 兼容服务")], default="local"
)
base_url = models.CharField(max_length=500, blank=True)
api_key = models.CharField(max_length=500, blank=True)
model = models.CharField(max_length=200, blank=True)
updated_at = models.DateTimeField(auto_now=True)
class Throttle(models.Model):
key = models.CharField(max_length=64, primary_key=True)
count = models.PositiveIntegerField(default=0)
@@ -133,9 +121,6 @@ class FormulaDefinition(models.Model):
user = models.ForeignKey(settings.AUTH_USER_MODEL, on_delete=models.CASCADE)
normalized_name = models.CharField(max_length=320)
definition = models.JSONField()
revision = models.PositiveIntegerField(default=1)
shared = models.BooleanField(default=False)
history = models.JSONField(default=list)
created_at = models.DateTimeField(auto_now_add=True)
updated_at = models.DateTimeField(auto_now=True)
+5 -26
View File
@@ -24,7 +24,7 @@ from openpyxl.cell import WriteOnlyCell
from .analysis import METRICS, analyze, context, default_preferences, dimensions, scoped_facts
from .importing import file_digest, profile_file, suggest_mapping, validate_mapping
from .models import AIConfig, Audit, Dataset, Investigation, Membership, Preference, Throttle
from .models import Audit, Dataset, Investigation, Membership, Preference, Throttle
class APIError(Exception):
@@ -130,20 +130,10 @@ def serialize_dataset(ds, detailed=False, restricted=False, raw_access=True):
@endpoint(public=True)
def session(request):
ai_enabled = settings.AI_ENABLED and bool(settings.OPENAI_API_KEY and settings.OPENAI_MODEL)
if request.user.is_authenticated:
try:
cfg = AIConfig.objects.filter(organization=request.user.membership.organization).first()
if cfg:
ai_enabled = (
cfg.enabled and bool(cfg.model) and (cfg.provider == "local" or bool(cfg.api_key))
)
except Membership.DoesNotExist:
pass
data = {
"csrf": get_token(request),
"user": None,
"ai_enabled": ai_enabled,
"ai_enabled": settings.AI_ENABLED and bool(settings.OPENAI_API_KEY and settings.OPENAI_MODEL),
}
if request.user.is_authenticated:
try:
@@ -297,14 +287,6 @@ def query(request, dataset_id):
return JsonResponse(analyze(ds, request.member, body(request), pref))
def amount_text(value):
"""金额展示文本:去掉定点数存储尺度产生的多余尾零(如 10.000000000000000000000000 → 10)。"""
if value is None:
return None
text = format(value.normalize(), "f")
return "0" if text == "-0" else text
@endpoint(("POST",))
def evidence(request, dataset_id):
ds = dataset(request, dataset_id, True)
@@ -322,9 +304,6 @@ def evidence(request, dataset_id):
"source_row", "period", "dimensions", "revenue", "cost", "gross", "raw"
)
)
for row in rows:
for field in ("revenue", "cost", "gross"):
row[field] = amount_text(row[field])
# Raw patient/customer identifiers require an explicit administrator role.
if request.member.role != "admin":
for row in rows:
@@ -423,9 +402,9 @@ def export(request, dataset_id):
row.source_row,
row.period,
*[row.dimensions.get(d["key"], "") for d in dims],
amount_text(row.revenue),
amount_text(row.cost),
amount_text(row.gross),
row.revenue,
row.cost,
row.gross,
]
)
if workbook:
+80 -108
View File
@@ -13,7 +13,6 @@ import {
Bell,
BookOpen,
ChevronRight,
Cpu,
Database,
Download,
Eye,
@@ -34,7 +33,6 @@ import { api, download, setCsrf } from "./api";
import { Chart } from "./Chart";
import {
AdminDialog,
AIConfigDialog,
FormulaExportDialog,
Copilot,
EvidenceDialog,
@@ -45,7 +43,6 @@ import {
import {
format,
labels,
roleLabels,
type Analysis,
type Context,
type Dataset,
@@ -86,7 +83,6 @@ export function App() {
| "dictionary"
| "users"
| "audit"
| "ai"
| null
| "formula"
>(null),
@@ -309,23 +305,6 @@ export function App() {
setPath([]);
setModal(null);
}
// 全部分组都为空的指标列(如数据集没有成本/ROI)不展示,避免占位挤压可用列。
const metricKeys = useMemo(() => {
const candidates = [
...new Set<Metric>([
"revenue",
"gross",
"cost",
"roi",
"customers",
ctx?.metric || "revenue",
]),
];
const available = candidates.filter(
(m) => m === "revenue" || (result?.rows || []).some((r) => r.values[m]?.current != null),
);
return available.length ? available : (["revenue"] as Metric[]);
}, [result, ctx]);
const columns = useMemo<ColDef<Row>[]>(
() => [
{
@@ -345,14 +324,23 @@ export function App() {
</button>
),
},
...metricKeys.map((metric) => ({
...[
...new Set<Metric>([
"revenue",
"gross",
"cost",
"roi",
"customers",
ctx?.metric || "revenue",
]),
].map((metric) => ({
colId: metric,
headerName: labels[metric],
minWidth: 130,
valueGetter: (p: { data?: Row }) =>
p.data?.values[metric]?.current == null
p.data?.values[metric].current == null
? null
: Number(p.data.values[metric]?.current),
: Number(p.data.values[metric].current),
valueFormatter: (p: { value: unknown }) =>
format(p.value as number, metric),
cellClass: metric === ctx?.metric ? "metric-column" : "",
@@ -405,34 +393,24 @@ export function App() {
),
},
],
[ds, ctx, path, metricKeys],
[ds, ctx, path],
);
const rows = (result?.rows || []).filter(
(r) => (screen !== "anomalies" || r.anomaly) && r.label.includes(search),
);
const metric = ctx?.metric || "revenue";
const compactAxis = (value: number) => {
const abs = Math.abs(value);
if (abs >= 1e8) return `${value / 1e8}亿`;
if (abs >= 1e4) return `${value / 1e4}万`;
return String(value);
};
const trend = useMemo(
() => ({
color: ["#197568", "#5794c9"],
tooltip: { trigger: "axis" },
legend: { bottom: 0 },
grid: { top: 25, left: 85, right: 25, bottom: 65 },
grid: { top: 25, left: 65, right: 25, bottom: 65 },
xAxis: {
type: "category",
data: result?.trend.map((r) => r.period),
axisLabel: { color: "#73827b", hideOverlap: true },
},
yAxis: {
type: "value",
splitLine: { lineStyle: { color: "#ecf0ee" } },
axisLabel: { hideOverlap: true },
axisLabel: { color: "#73827b" },
},
yAxis: { type: "value", splitLine: { lineStyle: { color: "#ecf0ee" } } },
series: [
{
name: "实际收入",
@@ -462,11 +440,7 @@ export function App() {
() => ({
tooltip: { trigger: "axis" },
grid: { top: 15, left: 95, right: 30, bottom: 30 },
xAxis: {
type: "value",
splitLine: { lineStyle: { color: "#ecf0ee" } },
axisLabel: { hideOverlap: true, formatter: compactAxis },
},
xAxis: { type: "value", splitLine: { lineStyle: { color: "#ecf0ee" } } },
yAxis: {
type: "category",
inverse: true,
@@ -569,10 +543,6 @@ export function App() {
<ShieldCheck />
操作审计
</button>
<button onClick={() => setModal("ai")}>
<Cpu />
AI 模型
</button>
</>
)}
</nav>
@@ -901,8 +871,6 @@ export function App() {
missing_gross: "毛利缺失",
missing_customer: "客户标识缺失",
blank_rows: "空白行",
g_l_mismatch: "应收金额两列不一致",
i_n_mismatch: "实收与实际收入不一致",
} as Record<string, string>
)[k] || k}
<strong>{v.toLocaleString()}</strong>
@@ -1055,11 +1023,16 @@ export function App() {
)?.name
}
</th>
{metricKeys
.filter((m) => m !== "customers")
.map((m) => (
<th key={m}>{labels[m]}</th>
))}
{(
[
"revenue",
"gross",
"cost",
"roi",
] as Metric[]
).map((m) => (
<th key={m}>{labels[m]}</th>
))}
<th>状态</th>
</tr>
</thead>
@@ -1067,10 +1040,15 @@ export function App() {
{result.rows.slice(0, 12).map((r) => (
<tr key={r.label}>
<th>{r.label}</th>
{metricKeys
.filter((m) => m !== "customers")
.map((m) => {
const v = r.values[m]?.change;
{(
[
"revenue",
"gross",
"cost",
"roi",
] as Metric[]
).map((m) => {
const v = r.values[m].change;
return (
<td key={m}>
<button
@@ -1084,7 +1062,7 @@ export function App() {
: "heat-positive"
}
onClick={() => {
setCtx({ ...ctx, metric: m as Metric });
setCtx({ ...ctx, metric: m });
setSelected(r);
setScreen("anomalies");
}}
@@ -1242,53 +1220,48 @@ export function App() {
</div>
</div>
<div className="grid-layout">
{rows.length ? (
<div className="grid">
<AgGridReact<Row>
theme={gridTheme}
rowData={rows}
columnDefs={columns}
defaultColDef={{
sortable: true,
filter: true,
resizable: true,
}}
pagination
paginationPageSize={20}
paginationPageSizeSelector={[20, 50, 100]}
getRowId={(p) => p.data.label}
onRowClicked={(e) => setSelected(e.data)}
localeText={{
page: "页",
of: "/",
to: "至",
more: "更多",
nextPage: "下一页",
previousPage: "上一页",
firstPage: "首页",
lastPage: "末页",
pageSizeSelectorLabel: "每页",
}}
{...(enterprise
? {
cellSelection: true,
sideBar: "columns",
statusBar: {
statusPanels: [
{
statusPanel: "agTotalRowCountComponent",
},
],
},
}
: {})}
/>
</div>
) : (
<p role="status" className="grid-empty">
没有符合条件的分组
</p>
)}
<div className="grid">
<AgGridReact<Row>
theme={gridTheme}
rowData={rows}
columnDefs={columns}
defaultColDef={{
sortable: true,
filter: true,
resizable: true,
}}
pagination
paginationPageSize={20}
paginationPageSizeSelector={[20, 50, 100]}
getRowId={(p) => p.data.label}
onRowClicked={(e) => setSelected(e.data)}
localeText={{
noRowsToShow: "没有符合条件的分组",
page: "页",
of: "/",
to: "至",
more: "更多",
nextPage: "下一页",
previousPage: "上一页",
firstPage: "首页",
lastPage: "末页",
pageSizeSelectorLabel: "每页",
}}
{...(enterprise
? {
cellSelection: true,
sideBar: "columns",
statusBar: {
statusPanels: [
{
statusPanel: "agTotalRowCountComponent",
},
],
},
}
: {})}
/>
</div>
{selected && (
<aside className="explanation">
<div className="section-heading">
@@ -1492,7 +1465,7 @@ export function App() {
<tr key={c.index}>
<td>{c.name}</td>
<td>{c.description}</td>
<td>{roleLabels[c.role] || c.role}</td>
<td>{c.role}</td>
</tr>
))}
</tbody>
@@ -1503,7 +1476,6 @@ export function App() {
{(modal === "users" || modal === "audit") && (
<AdminDialog mode={modal} onClose={() => setModal(null)} />
)}
{modal === "ai" && <AIConfigDialog onClose={() => setModal(null)} />}
</div>
);
}
+84 -240
View File
@@ -1,14 +1,16 @@
import { useEffect, useRef, useState } from "react";
import { ChevronLeft, ChevronRight, Download, Save, Trash2, X } from "lucide-react";
import {
ChevronLeft,
ChevronRight,
Download,
Save,
Trash2,
X,
} from "lucide-react";
import { api } from "./api";
import {
labels,
roleLabels,
type AIConfig,
type Context,
type FormulaColumn,
type FormulaExpression,
type SavedFormulaDefinition,
type Dataset,
type Metric,
type Preferences,
@@ -17,6 +19,21 @@ import {
type User = NonNullable<Session["user"]>;
type FormulaExpression =
| { metric: Metric }
| { number: number }
| {
operator: "add" | "subtract" | "multiply" | "divide";
left: FormulaExpression;
right: FormulaExpression;
};
type SavedFormulaDefinition = {
id: string;
definition: Record<string, unknown>;
updated_at: string;
};
function describeExpression(expression: FormulaExpression): string {
if ("metric" in expression) return labels[expression.metric];
if ("number" in expression) return String(expression.number);
@@ -24,6 +41,19 @@ function describeExpression(expression: FormulaExpression): string {
return `(${describeExpression(expression.left)} ${operators[expression.operator]} ${describeExpression(expression.right)})`;
}
function definitionSummary(definition: Record<string, unknown>): string {
const text = [
`已载入已保存定义「${String(definition.name)}」`,
`含义:${String(definition.meaning)}`,
`适用范围:${String(definition.scope)}`,
`缺失/零分母处理:${String(definition.missing)}`,
`计算表达式:${JSON.stringify(definition.expression)}`,
].join(";");
return text.length <= 2000
? text
: `${text.slice(0, 2000)}…(内容过长已截断)`;
}
export function Modal({
title,
onClose,
@@ -154,9 +184,7 @@ export function ImportDialog({
"dimension",
"ignore",
].map((r) => (
<option key={r} value={r}>
{roleLabels[r] || r}
</option>
<option key={r}>{r}</option>
))}
</select>
</td>
@@ -341,7 +369,8 @@ export function EvidenceDialog({
}>(),
[side, setSide] = useState<"current" | "baseline">("current"),
[page, setPage] = useState(0),
[error, setError] = useState("");
[error, setError] = useState(""),
[reloadKey, setReloadKey] = useState(0);
useEffect(() => {
let active = true;
setResult(undefined);
@@ -356,9 +385,14 @@ export function EvidenceDialog({
return () => {
active = false;
};
}, [ds.id, ctx, side, page]);
}, [ds.id, ctx, side, page, reloadKey]);
function selectSide(value: "current" | "baseline") {
if (value === side && page === 0) return;
if (value === side) {
// 重复点击当前期间:仅在不在第一页时回到第一页触发刷新,
// 同期间同页直接返回,避免清空列表后停在“加载中”。
if (page !== 0) setPage(0);
return;
}
setResult(undefined);
setPage(0);
setSide(value);
@@ -394,11 +428,12 @@ export function EvidenceDialog({
{user.role !== "admin" && <p>客户原始标识不包含在明细中。</p>}
{error && (
<p role="alert" className="error">
{error}
{error}{" "}
<button onClick={() => setReloadKey((n) => n + 1)}>重试</button>
</p>
)}
<div className="table-scroll">
<table className="evidence-table">
<table>
<thead>
<tr>
<th>来源行</th>
@@ -428,6 +463,11 @@ export function EvidenceDialog({
<td>{String(r.gross ?? "")}</td>
</tr>
))}
{result && result.rows.length === 0 && (
<tr>
<td colSpan={3 + ds.dimensions.length}>本期范围内没有数据</td>
</tr>
)}
</tbody>
</table>
</div>
@@ -464,153 +504,6 @@ export function EvidenceDialog({
);
}
export function AIConfigDialog({ onClose }: { onClose: () => void }) {
const [value, setValue] = useState<AIConfig>({
enabled: false,
provider: "local",
base_url: "",
model: "",
api_key_set: false,
updated_at: null,
}),
[key, setKey] = useState(""),
[models, setModels] = useState<string[]>([]),
[error, setError] = useState(""),
[busy, setBusy] = useState(false),
[listing, setListing] = useState(false);
useEffect(() => {
api<AIConfig>("/ai-config")
.then((r) => setValue(r))
.catch((e) => setError((e as Error).message));
}, []);
async function fetchModels() {
setListing(true);
setError("");
try {
const r = await api<{ items: string[] }>("/ai-models");
setModels(r.items);
if (!r.items.length) setError("该服务没有返回可用模型");
} catch (e) {
setError((e as Error).message);
} finally {
setListing(false);
}
}
async function save() {
setBusy(true);
setError("");
try {
await api("/ai-config", "PUT", {
enabled: value.enabled,
provider: value.provider,
base_url: value.base_url,
model: value.model,
api_key: key,
});
setKey("");
setValue((v) => ({ ...v, api_key_set: v.api_key_set || Boolean(key) }));
if (value.base_url) await fetchModels();
} catch (e) {
setError((e as Error).message);
} finally {
setBusy(false);
}
}
return (
<Modal title="AI 模型配置" onClose={onClose} wide>
<div className="dialog-body">
<p className="muted">
配置财务分析助手使用的模型服务。数据库配置优先;未配置时回退到部署环境变量。
</p>
<label className="check">
<input
type="checkbox"
checked={value.enabled}
onChange={(e) => setValue({ ...value, enabled: e.target.checked })}
/>
启用 AI 模型理解问题
</label>
<label>
服务商
<select
value={value.provider}
onChange={(e) =>
setValue({
...value,
provider: e.target.value as AIConfig["provider"],
})
}
>
<option value="openai">OpenAI</option>
<option value="local">
本地 OpenAI 兼容服务(Ollama / LM Studio / vLLM)
</option>
</select>
</label>
<label>
Base URL
<input
value={value.base_url}
placeholder={
value.provider === "local"
? "http://127.0.0.1:11434/v1"
: "https://api.openai.com/v1"
}
onChange={(e) => setValue({ ...value, base_url: e.target.value })}
/>
</label>
<label>
API Key
<input
type="password"
value={key}
placeholder={
value.api_key_set ? "已配置,留空保持不变" : "本地服务通常可留空"
}
onChange={(e) => setKey(e.target.value)}
/>
</label>
<label>
模型
<input
list="ai-model-list"
value={value.model}
placeholder="模型名,如 qwen2.5:7b"
onChange={(e) => setValue({ ...value, model: e.target.value })}
/>
<datalist id="ai-model-list">
{models.map((m) => (
<option key={m} value={m} />
))}
</datalist>
</label>
<footer>
<button onClick={onClose}>取消</button>
<button
type="button"
disabled={listing || busy}
onClick={fetchModels}
>
{listing ? "获取中…" : "获取可用模型"}
</button>
<button className="primary" disabled={busy} onClick={save}>
{busy ? "保存中…" : "保存配置"}
</button>
</footer>
{error && (
<p role="alert" className="error">
{error}
</p>
)}
<p className="muted">
只向模型服务发送问题与维度/当前筛选,原始数据行不出服务器;密钥仅保存在后端,
配置修改会记入操作审计。
</p>
</div>
</Modal>
);
}
export function Copilot({
ds,
ctx,
@@ -690,7 +583,7 @@ export function Copilot({
checked={useModel}
onChange={(e) => setUseModel(e.target.checked)}
/>
使用已配置的 AI 模型理解问题
使用外部 AI 理解问题
</label>
{useModel && (
<label className="check">
@@ -815,7 +708,7 @@ export function FormulaExportDialog({
{ role: "user" | "assistant"; content: string }[]
>([]),
[draft, setDraft] = useState(""),
[column, setColumn] = useState<FormulaColumn | null>(null),
[column, setColumn] = useState<Record<string, unknown> | null>(null),
[questions, setQuestions] = useState<string[]>([]),
[task, setTask] = useState<{
id: string;
@@ -824,30 +717,9 @@ export function FormulaExportDialog({
}>(),
[definitions, setDefinitions] = useState<SavedFormulaDefinition[]>([]),
[selectedDefinition, setSelectedDefinition] = useState(""),
[shared, setShared] = useState(false),
[loadedDefinition, setLoadedDefinition] = useState<FormulaColumn | null>(null),
[consent, setConsent] = useState(false),
[busy, setBusy] = useState(false),
[error, setError] = useState("");
const draftKey = `traceiq.formula-draft.${ds.id}`;
const [draftRestored, setDraftRestored] = useState(false);
useEffect(() => {
try {
const stored = sessionStorage.getItem(draftKey);
if (stored) {
const saved = JSON.parse(stored);
setDraft(saved.draft || "");
setMessages(saved.messages || []);
setColumn(saved.column || null);
}
} catch {
sessionStorage.removeItem(draftKey);
}
setDraftRestored(true);
}, [draftKey]);
useEffect(() => {
if (draftRestored) sessionStorage.setItem(draftKey, JSON.stringify({ draft, messages, column }));
}, [draftKey, draftRestored, draft, messages, column]);
useEffect(() => {
api<{ items: SavedFormulaDefinition[] }>(
`/datasets/${ds.id}/formula-definitions`,
@@ -874,10 +746,7 @@ export function FormulaExportDialog({
async function send() {
const text = draft.trim();
if (!text || !consent || userTurns >= 6) return;
const content = loadedDefinition
? `已载入的原始定义:${JSON.stringify(loadedDefinition)}\n本次补充:${text}`
: text;
const next = [...messages, { role: "user" as const, content }];
const next = [...messages, { role: "user" as const, content: text }];
setBusy(true);
setError("");
setQuestions([]);
@@ -886,7 +755,7 @@ export function FormulaExportDialog({
const result = await api<{
complete: boolean;
questions: string[];
column: FormulaColumn | null;
column: Record<string, unknown> | null;
}>(`/datasets/${ds.id}/formula-design`, "POST", {
context: ctx,
messages: next,
@@ -903,7 +772,6 @@ export function FormulaExportDialog({
]);
setQuestions(result.questions);
setColumn(result.column);
setLoadedDefinition(null);
setDraft("");
} catch (e) {
setError((e as Error).message);
@@ -912,7 +780,7 @@ export function FormulaExportDialog({
}
}
async function create() {
if (!column && !ctx.custom_columns?.length) return;
if (!column) return;
setBusy(true);
setError("");
try {
@@ -920,12 +788,7 @@ export function FormulaExportDialog({
await api<{ id: string; status: string }>(
`/datasets/${ds.id}/formula-exports`,
"POST",
{
...ctx,
custom_columns: column
? [...(ctx.custom_columns || []).filter((item) => item.name !== column.name), column]
: ctx.custom_columns,
},
{ ...ctx, custom_columns: [column] },
),
);
} catch (e) {
@@ -942,17 +805,7 @@ export function FormulaExportDialog({
const saved = await api<SavedFormulaDefinition>(
`/datasets/${ds.id}/formula-definitions`,
"POST",
{
definition: column,
shared,
revision: definitions.find(
(item) =>
(item.id === selectedDefinition && item.owned) ||
(item.owned &&
String(item.definition.name).trim().toLowerCase() ===
String(column.name).trim().toLowerCase()),
)?.revision,
},
{ definition: column },
);
setDefinitions((items) =>
[...items.filter((item) => item.id !== saved.id), saved].sort((a, b) =>
@@ -973,14 +826,24 @@ export function FormulaExportDialog({
setSelectedDefinition(id);
const saved = definitions.find((item) => item.id === id);
if (!saved) {
setLoadedDefinition(null);
// 选择空选项时同步清理当前定义,避免下拉选择与预览不一致
setColumn(null);
setMessages([]);
setQuestions([]);
return;
}
setColumn(saved.definition);
setShared(saved.owned && saved.shared);
setLoadedDefinition(saved.definition);
setMessages([]);
// 载入定义后把原定义显式放进可见、经同意的对话上下文,
// 使“把刚才公式改成乘以 100”这类指代能关联到已载入表达式;
// 修改结果仍需重新确认,不会静默沿用旧定义。
setMessages([
{ role: "user", content: definitionSummary(saved.definition) },
{
role: "assistant",
content:
"已载入该定义作为当前基准。请描述要调整的内容,我会在该定义基础上生成新的列定义,仍需你确认后再生成。",
},
]);
setQuestions([]);
setDraft("");
}
@@ -997,7 +860,6 @@ export function FormulaExportDialog({
items.filter((item) => item.id !== selectedDefinition),
);
setSelectedDefinition("");
setLoadedDefinition(null);
setColumn(null);
} catch (e) {
setError((e as Error).message);
@@ -1007,10 +869,15 @@ export function FormulaExportDialog({
}
async function getFile() {
try {
const link = document.createElement("a");
link.href = `/api/datasets/${ds.id}/formula-exports/${task!.id}/download`;
const blob = await api<Blob>(
`/datasets/${ds.id}/formula-exports/${task!.id}/download`,
),
url = URL.createObjectURL(blob),
link = document.createElement("a");
link.href = url;
link.download = "traceiq-formula-analysis.xlsx";
link.click();
window.setTimeout(() => URL.revokeObjectURL(url), 1000);
} catch (e) {
setError((e as Error).message);
}
@@ -1019,11 +886,6 @@ export function FormulaExportDialog({
<Modal title="生成公式工作簿" onClose={onClose} wide>
<div className="dialog-body">
<p>描述新列含义、计算方式、适用范围,以及缺失值或零分母处理方式。</p>
{!column && !task && !!ctx.custom_columns?.length && (
<button disabled={busy} onClick={create}>
按网页已选 {ctx.custom_columns.length} 列生成工作簿
</button>
)}
{definitions.length > 0 && !task && (
<div className="form-row">
<label>
@@ -1036,7 +898,7 @@ export function FormulaExportDialog({
<option value="">选择定义</option>
{definitions.map((item) => (
<option key={item.id} value={item.id}>
{item.definition.name}{item.owned ? "" : `(${item.owner}共享)`}
{String(item.definition.name)}
</option>
))}
</select>
@@ -1045,27 +907,13 @@ export function FormulaExportDialog({
className="icon"
aria-label="删除所选定义"
title="删除所选定义"
disabled={busy || !definitions.find((item) => item.id === selectedDefinition)?.owned}
disabled={busy || !selectedDefinition}
onClick={deleteDefinition}
>
<Trash2 />
</button>
</div>
)}
{definitions.find((item) => item.id === selectedDefinition)?.history.length ? (
<details>
<summary>版本历史(最近 20 次,可载入后另存)</summary>
{definitions.find((item) => item.id === selectedDefinition)!.history.map((version) => (
<button key={version.revision} disabled={busy} onClick={() => {
setColumn(version.definition);
setLoadedDefinition(version.definition);
setMessages([]);
}}>
载入版本 {version.revision} · {version.definition.meaning}
</button>
))}
</details>
) : null}
<label className="check">
<input
type="checkbox"
@@ -1108,10 +956,6 @@ export function FormulaExportDialog({
</p>
<p>适用范围:{String(column.scope)}</p>
<p>缺失/零分母:{String(column.missing)}</p>
<label className="check">
<input type="checkbox" checked={shared} onChange={(event) => setShared(event.target.checked)} />
保存时共享给当前数据集的其他全范围分析账号
</label>
<button
disabled={busy || Boolean(draft.trim())}
onClick={saveDefinition}
+7 -22
View File
@@ -484,7 +484,7 @@ a {
display: flex;
flex-direction: column;
align-items: stretch;
padding: 18px 16px 14px;
padding: 20px 20px 16px;
gap: 16px;
background: white;
min-width: 0;
@@ -510,13 +510,12 @@ a {
}
.kpi strong {
text-align: left;
font-size: 21px;
font-size: 24px;
line-height: 1.25;
font-weight: 620;
font-variant-numeric: tabular-nums;
white-space: nowrap;
overflow: hidden;
text-overflow: ellipsis;
white-space: normal;
overflow-wrap: anywhere;
}
.kpi-bottom {
display: flex;
@@ -1272,6 +1271,9 @@ details {
.kpi {
padding: 17px 13px;
}
.kpi strong {
font-size: 21px;
}
.chart-band {
gap: 15px;
}
@@ -1560,20 +1562,3 @@ details {
scroll-behavior: auto !important;
}
}
.grid-empty {
margin: 0;
padding: 40px 0;
text-align: center;
color: var(--muted, #5b6675);
border: 1px dashed var(--line);
border-radius: 6px;
width: 100%;
}
.evidence-table {
min-width: 960px;
}
.evidence-table th {
white-space: nowrap;
}
+2 -50
View File
@@ -32,20 +32,6 @@ export interface Column {
key: string;
confidence: number;
}
export const roleLabels: Record<string, string> = {
period: "期间",
year: "年份",
month: "月份",
dimension: "维度",
customer: "客户标识",
quantity: "数量",
revenue: "实际收入",
cost: "成本",
gross: "毛利贡献",
list_amount: "应收金额",
discount: "折扣金额",
ignore: "未使用",
};
export interface Mapping {
sheet: string;
header: number;
@@ -98,32 +84,6 @@ export interface Context {
filters: Record<string, string>;
group: string;
metric: Metric;
custom_columns?: FormulaColumn[];
}
export type FormulaExpression =
| { metric: Metric }
| { number: number }
| {
operator: "add" | "subtract" | "multiply" | "divide";
left: FormulaExpression;
right: FormulaExpression;
};
export interface FormulaColumn {
name: string;
meaning: string;
scope: string;
missing: string;
expression: FormulaExpression;
}
export interface SavedFormulaDefinition {
id: string;
definition: FormulaColumn;
revision: number;
updated_at: string;
shared: boolean;
owned: boolean;
owner: string;
history: { revision: number; definition: FormulaColumn; updated_at: string }[];
}
export interface Value {
current: Numeric;
@@ -147,23 +107,15 @@ export interface Analysis {
current: string[];
baseline: string[];
};
summary: Record<string, Value>;
summary: Record<Metric, Value>;
rows: Row[];
trend: ({ period: string } & Record<string, Numeric>)[];
trend: ({ period: string } & Record<Metric, Numeric>)[];
row_count: number;
anomaly_count: number;
comparable: boolean;
notice: string;
version: string;
}
export interface AIConfig {
enabled: boolean;
provider: "openai" | "local";
base_url: string;
model: string;
api_key_set: boolean;
updated_at: string | null;
}
export interface Session {
csrf: string;
user: {
+4 -3
View File
@@ -74,9 +74,10 @@ fi
echo "执行数据库迁移…"
$PY manage.py migrate --no-input >/dev/null
# init_local.py 以数据库为准幂等执行:缺 admin 时创建并写凭据文件,已存在则跳过
echo "初始化本地管理员(已存在则跳过)…"
PYTHONPATH="$ROOT" $PY scripts/init_local.py
if [ ! -f var/local-access.txt ]; then
echo "初始化本地管理员…"
PYTHONPATH="$ROOT" $PY scripts/init_local.py
fi
if [ ! -d frontend/dist ]; then
echo "构建前端(首次)…"
-1
View File
@@ -93,7 +93,6 @@ MAX_UPLOAD_BYTES = int(os.environ.get("MAX_UPLOAD_BYTES", 64 * 1024 * 1024))
MAX_IMPORT_ROWS = int(os.environ.get("MAX_IMPORT_ROWS", 1000000))
OPENAI_API_KEY = os.environ.get("OPENAI_API_KEY", "")
OPENAI_MODEL = os.environ.get("OPENAI_MODEL", "")
OPENAI_BASE_URL = os.environ.get("OPENAI_BASE_URL", "https://api.openai.com/v1")
AI_ENABLED = os.environ.get("AI_ENABLED", "0") == "1"
LOGGING = {
"version": 1,
-3
View File
@@ -1,7 +1,6 @@
from django.urls import path
from finance import formula_views, views
from finance.ai import ai_config, ai_models
from finance.assistant import assistant
urlpatterns = [
@@ -12,8 +11,6 @@ urlpatterns = [
path("api/datasets", views.datasets),
path("api/users", views.users),
path("api/audit", views.audit_log),
path("api/ai-config", ai_config),
path("api/ai-models", ai_models),
path("api/datasets/<uuid:dataset_id>", views.dataset_detail),
path("api/datasets/<uuid:dataset_id>/mapping", views.mapping),
path("api/datasets/<uuid:dataset_id>/query", views.query),
+39 -27
View File
@@ -59,11 +59,22 @@ test("新增列:补充信息触发追问并重新确认后生成公式工作
meaning: "实际收入减成本",
scope: "所选分组与期间",
missing: "结果不可用(#N/A);零分母不可用",
expression: {
operator: "subtract",
left: { metric: "revenue" },
right: { metric: "cost" },
},
expression:
designCalls === 5
? {
operator: "multiply",
left: {
operator: "subtract",
left: { metric: "revenue" },
right: { metric: "cost" },
},
right: { number: 100 },
}
: {
operator: "subtract",
left: { metric: "revenue" },
right: { metric: "cost" },
},
},
},
});
@@ -119,33 +130,34 @@ test("新增列:补充信息触发追问并重新确认后生成公式工作
);
await dialog.getByLabel("关闭", { exact: true }).click();
await page.getByRole("button", { name: "选择网页自定义列" }).click();
const columns = page.getByRole("dialog", { name: "选择网页自定义列" });
await columns.getByRole("checkbox", { name: /收入成本差/ }).check();
const customQuery = page.waitForResponse(
(response) =>
response.url().includes("/query") && response.request().method() === "POST",
);
await columns.getByRole("button", { name: "应用", exact: true }).click();
const customResult = await (await customQuery).json();
expect(customResult.summary.custom_0.current).toBe("1000");
expect(customResult.rows[0].values.custom_0.current).toBe("1000");
await page.getByRole("button", { name: "生成公式工作簿" }).click();
const reopened = page.getByRole("dialog", { name: "生成公式工作簿" });
await reopened.getByLabel("已保存定义").selectOption({ label: "收入成本差" });
await expect(reopened).toContainText("计算表达式:(实际收入 − 成本)");
await reopened.getByLabel("列计算描述").fill("把刚才公式乘以 100");
await expect(
reopened.getByRole("button", { name: "先发送补充内容" }),
).toBeDisabled();
await reopened
.getByLabel("同意将描述、指标定义和当前筛选范围发送至已配置的外部 AI 服务")
.check();
// F04 回归:载入定义后,原定义作为可见上下文进入对话
await expect(reopened).toContainText("已载入已保存定义「收入成本差」");
await expect(reopened).toContainText("我:已载入已保存定义");
await expect(reopened).toContainText("助手:已载入该定义作为当前基准");
// F04 回归:选择空选项时同步清理当前定义与上下文,选择状态与预览一致
await reopened.getByLabel("已保存定义").selectOption({ label: "选择定义" });
await expect(reopened.getByText("计算表达式:(实际收入 − 成本)")).toHaveCount(
0,
);
await expect(reopened.getByText("已载入已保存定义")).toHaveCount(0);
await reopened.getByLabel("已保存定义").selectOption({ label: "收入成本差" });
await reopened.getByRole("checkbox").check();
await reopened.getByLabel("列计算描述").fill("把刚才公式改成乘以 100");
await reopened.getByRole("button", { name: "发送描述" }).click();
expect(sentMessages[0].content).toContain('"name":"收入成本差"');
expect(sentMessages[0].content).toContain('"operator":"subtract"');
expect(sentMessages[0].content).toContain("把刚才公式乘以 100");
await expect(reopened).toContainText(
"计算表达式:((实际收入 − 成本) × 100)",
);
// 模型请求必须包含载入的原定义上下文和新的修改描述
expect(sentMessages.at(0)?.content).toContain(
"已载入已保存定义「收入成本差」",
);
expect(sentMessages.at(0)?.content).toContain('"operator":"subtract"');
expect(sentMessages.at(-1)?.content).toBe("把刚才公式改成乘以 100");
await reopened.getByRole("button", { name: "确认并生成公式工作簿" }).click();
await expect(reopened.getByRole("status")).toContainText("生成完成", {
timeout: 30000,
+81 -26
View File
@@ -87,21 +87,27 @@ test("桌面:真实导入、异常、下钻、证据、设置、Copilot、导
await expect(evidenceDialog).toContainText("1 条");
await expect(evidenceDialog).toContainText("2026-08");
await expect(evidenceDialog).toContainText("CSV");
await evidenceDialog
.getByRole("button", { name: "本期", exact: true })
.click();
await expect(evidenceDialog).toContainText("2026-08");
await expect(evidenceDialog).not.toContainText("加载中");
await evidenceDialog
.getByRole("button", { name: "基期", exact: true })
.click();
await expect(evidenceDialog).toContainText("2025-08");
await evidenceDialog.getByLabel("关闭", { exact: true }).click();
const evidenceOffsets: number[] = [];
let evidenceFailOnce = false;
let evidenceEmpty = false;
await page.route("**/api/datasets/*/evidence", async (route) => {
const request = route.request().postDataJSON() as { offset?: number };
const offset = request.offset || 0;
if (evidenceFailOnce) {
evidenceFailOnce = false;
await route.fulfill({ status: 500, json: { error: "服务暂不可用" } });
return;
}
evidenceOffsets.push(offset);
if (evidenceEmpty) {
await route.fulfill({ json: { total: 0, sheet: "CSV", rows: [] } });
return;
}
const count = offset === 0 ? 50 : 1;
await route.fulfill({
json: {
@@ -126,11 +132,45 @@ test("桌面:真实导入、异常、下钻、证据、设置、Copilot、导
await expect(evidenceDialog).toContainText("第 51-51 条,共 51 条");
await expect(evidenceDialog).toContainText("2026-09");
expect(evidenceOffsets).toEqual([0, 50]);
// F03 回归:第二页时重复点击当前期间应回到第一页并重新加载
await evidenceDialog
.getByRole("button", { name: "本期", exact: true })
.click();
await expect(evidenceDialog).toContainText("第 1-50 条,共 51 条");
expect(evidenceOffsets).toEqual([0, 50, 0]);
// F03 回归:第一页重复点击当前期间不清空列表、不重新请求、不停在“加载中”
await evidenceDialog
.getByRole("button", { name: "本期", exact: true })
.click();
await expect(evidenceDialog).toContainText("第 1-50 条,共 51 条");
await expect(evidenceDialog).not.toContainText("加载中");
expect(evidenceOffsets).toEqual([0, 50, 0]);
// F03 回归:快速本期/基期切换最终停留在当前期间的数据
await evidenceDialog
.getByRole("button", { name: "基期", exact: true })
.click();
await evidenceDialog
.getByRole("button", { name: "本期", exact: true })
.click();
await expect(evidenceDialog).toContainText("第 1-50 条,共 51 条");
await expect(evidenceDialog).not.toContainText("2026-09");
await expect(evidenceDialog).not.toContainText("加载中");
// F03 回归:请求失败显示错误并可重试恢复
evidenceFailOnce = true;
await evidenceDialog
.getByRole("button", { name: "下一页", exact: true })
.click();
await expect(evidenceDialog).toContainText("服务暂不可用");
await expect(evidenceDialog).toContainText("加载失败");
await evidenceDialog.getByRole("button", { name: "重试" }).click();
await expect(evidenceDialog).toContainText("第 51-51 条,共 51 条");
// F03 回归:空结果显示空态而不是过期数据或加载中
evidenceEmpty = true;
await evidenceDialog
.getByRole("button", { name: "上一页", exact: true })
.click();
await expect(evidenceDialog).toContainText("本期范围内没有数据");
await expect(evidenceDialog).toContainText("共 0 条");
await expect(evidenceDialog).not.toContainText("加载中");
await evidenceDialog.getByLabel("关闭", { exact: true }).click();
await page
.getByRole("navigation", { name: "分析路径" })
@@ -180,38 +220,53 @@ test("桌面:真实导入、异常、下钻、证据、设置、Copilot、导
await page.route("**/formula-design", async (route) => {
designCalls++;
await route.fulfill({
json: designCalls === 1
? { complete: false, questions: ["缺失值和零分母如何处理?"], column: null }
: {
complete: true,
questions: [],
column: {
name: "收入成本差",
meaning: "实际收入减成本",
scope: "所选分组与期间",
missing: "结果不可用(#N/A);零分母不可用",
expression: {
operator: "subtract",
left: { metric: "revenue" },
right: { metric: "cost" },
json:
designCalls === 1
? {
complete: false,
questions: ["缺失值和零分母如何处理?"],
column: null,
}
: {
complete: true,
questions: [],
column: {
name: "收入成本差",
meaning: "实际收入减成本",
scope: "所选分组与期间",
missing: "结果不可用(#N/A);零分母不可用",
expression: {
operator: "subtract",
left: { metric: "revenue" },
right: { metric: "cost" },
},
},
},
},
});
});
await page.getByRole("button", { name: "生成公式工作簿" }).click();
const formulaDialog = page.getByRole("dialog", { name: "生成公式工作簿" });
await formulaDialog.getByRole("checkbox").check();
await formulaDialog.getByLabel("列计算描述").fill("新增收入成本差,缺失或零分母显示不可用");
await formulaDialog
.getByLabel("列计算描述")
.fill("新增收入成本差,缺失或零分母显示不可用");
await formulaDialog.getByRole("button", { name: "发送描述" }).click();
await expect(formulaDialog).toContainText("缺失值和零分母如何处理?");
await formulaDialog.getByLabel("列计算描述").fill("缺失项和零分母都按不可用处理");
await formulaDialog
.getByLabel("列计算描述")
.fill("缺失项和零分母都按不可用处理");
await formulaDialog.getByRole("button", { name: "发送描述" }).click();
await formulaDialog.getByRole("button", { name: "确认并生成公式工作簿" }).click();
await expect(formulaDialog.getByRole("status")).toContainText("生成完成", { timeout: 30000 });
await formulaDialog
.getByRole("button", { name: "确认并生成公式工作簿" })
.click();
await expect(formulaDialog.getByRole("status")).toContainText("生成完成", {
timeout: 30000,
});
const formulaDownload = page.waitForEvent("download");
await formulaDialog.getByRole("button", { name: "下载公式工作簿" }).click();
expect((await formulaDownload).suggestedFilename()).toBe("traceiq-formula-analysis.xlsx");
expect((await formulaDownload).suggestedFilename()).toBe(
"traceiq-formula-analysis.xlsx",
);
await formulaDialog.getByLabel("关闭").click();
});
test("手机:登录、总览、设置、Copilot 布局无横向溢出", async ({ page }) => {
+4 -209
View File
@@ -1,182 +1,17 @@
import json
from unittest.mock import patch
import httpx
import pytest
from django.contrib.auth.models import User
from django.test import Client
from finance.assistant import model_intent
from finance.models import AIConfig, Audit, FormulaDefinition, FormulaExport, Membership
from finance.models import Audit, FormulaDefinition, FormulaExport, Membership
pytestmark = pytest.mark.django_db
def completed_response(intent=None):
class Response:
def raise_for_status(self):
return None
def json(self):
return {
"status": "completed",
"output": [
{
"content": [
{
"type": "output_text",
"text": json.dumps(
intent
or {
"action": "overview",
"metric": "revenue",
"group": "department",
"filters": [{"key": "department", "value": "儿科"}],
}
),
}
]
}
],
}
return Response()
def test_ai_config_requires_full_admin_and_masks_key(workspace):
ds, member, client, _, _ = workspace
url = "/api/ai-config"
viewer = User.objects.create_user("ai-viewer", password="Finance-TestPassword991!")
Membership.objects.create(user=viewer, organization=ds.organization, role="viewer")
viewer_client = Client()
assert viewer_client.login(username="ai-viewer", password="Finance-TestPassword991!")
assert viewer_client.get(url).status_code == 403
member.departments = ["儿科"]
member.save(update_fields=["departments"])
scoped = Client()
assert scoped.login(username=member.user.username, password="Finance-TestPassword991!")
assert scoped.get(url).status_code == 403
member.departments = []
member.save(update_fields=["departments"])
assert client.get(url).json() == {
"enabled": False,
"provider": "local",
"base_url": "",
"model": "",
"api_key_set": False,
"updated_at": None,
}
invalid = [
{"enabled": True, "provider": "local", "base_url": "", "model": "qwen2.5:7b"},
{"enabled": True, "provider": "openai", "base_url": "", "model": "gpt-4o"},
{"enabled": True, "provider": "local", "base_url": "ftp://x", "model": "m"},
{"enabled": True, "provider": "local", "base_url": "http://x/v1", "model": ""},
]
for payload in invalid:
assert client.put(url, data=payload, content_type="application/json").status_code == 400
saved = client.put(
url,
data={
"enabled": True,
"provider": "local",
"base_url": "http://127.0.0.1:11434/v1",
"api_key": "ollama-secret",
"model": "qwen2.5:7b",
},
content_type="application/json",
)
assert saved.status_code == 200
shown = client.get(url).json()
assert shown["api_key_set"] is True
assert shown["enabled"] is True and shown["provider"] == "local"
assert "ollama-secret" not in json.dumps(shown)
assert client.get("/api/session").json()["ai_enabled"] is True
assert Audit.objects.filter(action="ai_config.update").exists()
def test_ai_models_lists_local_models(workspace):
ds, _, client, _, _ = workspace
client.put(
"/api/ai-config",
data={
"enabled": False,
"provider": "local",
"base_url": "http://127.0.0.1:11434/v1",
"model": "",
},
content_type="application/json",
)
with patch("finance.ai.httpx.Client") as http:
http.return_value.__enter__.return_value.get.return_value = type(
"Response",
(),
{
"raise_for_status": lambda self: None,
"json": lambda self: {"data": [{"id": "qwen2.5:7b"}, {"id": "llama3.1:8b"}]},
},
)()
listed = client.get("/api/ai-models")
assert listed.status_code == 200
assert listed.json()["items"] == ["qwen2.5:7b", "llama3.1:8b"]
call = http.return_value.__enter__.return_value.get.call_args
assert call.args[0] == "http://127.0.0.1:11434/v1/models"
with patch("finance.ai.httpx.Client") as http:
http.return_value.__enter__.return_value.get.side_effect = httpx.ConnectError("boom")
failed = client.get("/api/ai-models")
assert failed.status_code == 502
def test_assistant_uses_org_db_config(workspace):
ds, _, client, *_ = workspace
client.put(
"/api/ai-config",
data={
"enabled": True,
"provider": "local",
"base_url": "http://127.0.0.1:11434/v1",
"model": "qwen2.5:7b",
},
content_type="application/json",
)
with patch("finance.assistant.httpx.Client") as http:
http.return_value.__enter__.return_value.post.return_value = completed_response()
response = client.post(
f"/api/datasets/{ds.pk}/assistant",
data={
"question": "分析儿科收入",
"context": {"start": "2026-08", "end": "2026-08", "comparison": "yoy", "group": "department"},
"use_model": True,
"consent": True,
},
content_type="application/json",
)
assert response.status_code == 200
assert response.json()["provider"] == "local"
call = http.return_value.__enter__.return_value.post.call_args
assert call.args[0] == "http://127.0.0.1:11434/v1/responses"
assert call.kwargs["json"]["model"] == "qwen2.5:7b"
assert "Authorization" not in call.kwargs["headers"]
disabled = AIConfig.objects.get(organization=ds.organization)
disabled.enabled = False
disabled.save(update_fields=["enabled"])
denied = client.post(
f"/api/datasets/{ds.pk}/assistant",
data={"question": "分析儿科收入", "use_model": True, "consent": True},
content_type="application/json",
)
assert denied.status_code == 503
def test_model_intent_accepts_only_valid_dimensions_and_sends_no_rows(workspace, settings):
ds, _, _, _, _ = workspace
settings.AI_ENABLED = True
settings.OPENAI_MODEL = "test-model"
settings.OPENAI_API_KEY = "test-key"
@@ -223,7 +58,6 @@ def test_model_intent_accepts_only_valid_dimensions_and_sends_no_rows(workspace,
def test_model_intent_rejects_unknown_filter_and_incomplete_provider_response(workspace, settings):
ds, *_ = workspace
settings.AI_ENABLED = True
settings.OPENAI_MODEL = "test-model"
settings.OPENAI_API_KEY = "test-key"
@@ -331,34 +165,19 @@ def test_formula_definition_library_is_private_validated_and_audited(workspace):
"right": {"metric": "cost"},
},
}
saved = client.post(url, data={"definition": definition, "shared": True}, content_type="application/json")
saved = client.post(url, data={"definition": definition}, content_type="application/json")
assert saved.status_code == 201
assert saved.json()["shared"] is True
definition_id = saved.json()["id"]
assert client.get(url).json()["items"][0]["definition"] == definition
updated = client.post(
url,
data={
"definition": {**definition, "name": " 收入成本差 ", "meaning": "更新说明"},
"shared": True,
"revision": saved.json()["revision"],
},
data={"definition": {**definition, "name": " 收入成本差 ", "meaning": "更新说明"}},
content_type="application/json",
)
assert updated.status_code == 200
assert FormulaDefinition.objects.filter(dataset=ds, user=ds.created_by).count() == 1
assert updated.json()["definition"]["meaning"] == "更新说明"
assert updated.json()["revision"] == 2
assert len(updated.json()["history"]) == 1
stale = client.post(
url,
data={"definition": {**definition, "meaning": "旧标签页覆盖"}, "revision": 1},
content_type="application/json",
)
assert stale.status_code == 409
assert FormulaDefinition.objects.get(pk=definition_id).definition["meaning"] == "更新说明"
other = Client()
other_user = ds.created_by.__class__.objects.create_user(
@@ -366,8 +185,7 @@ def test_formula_definition_library_is_private_validated_and_audited(workspace):
)
Membership.objects.create(user=other_user, organization=ds.organization, role="analyst")
assert other.login(username=other_user.username, password="Formula-Library-TestPassword991!")
shared = other.get(url).json()["items"]
assert len(shared) == 1 and shared[0]["owned"] is False and shared[0]["shared"] is True
assert other.get(url).json()["items"] == []
assert other.delete(f"{url}/{definition_id}").status_code == 404
deleted = client.delete(f"{url}/{definition_id}")
@@ -401,26 +219,3 @@ def test_formula_definition_library_rejects_unavailable_metric(workspace):
)
assert response.status_code == 400
assert "成本" in response.json()["error"]
def test_formula_definition_audit_failure_rolls_back_definition(workspace, monkeypatch):
ds, _, client, *_ = workspace
definition = {
"name": "审计回滚",
"meaning": "实际收入",
"scope": "所选分组与期间",
"missing": "结果不可用(#N/A);零分母不可用",
"expression": {"metric": "revenue"},
}
def fail(*_args, **_kwargs):
raise RuntimeError("audit unavailable")
monkeypatch.setattr("finance.formula_views.audit", fail)
with pytest.raises(RuntimeError, match="audit unavailable"):
client.post(
f"/api/datasets/{ds.pk}/formula-definitions",
data={"definition": definition},
content_type="application/json",
)
assert not FormulaDefinition.objects.filter(dataset=ds, user=ds.created_by).exists()
+2 -31
View File
@@ -133,35 +133,6 @@ def test_evidence_pagination_returns_stable_pages_and_rejects_invalid_bounds(wor
assert response.status_code == 400
def test_evidence_amounts_trim_storage_scale(workspace):
from decimal import Decimal
from finance.views import amount_text
assert amount_text(Decimal("10.000000000000000000000000")) == "10"
assert amount_text(Decimal("10.500000000000000000000000")) == "10.5"
assert amount_text(Decimal("0E-24")) == "0"
assert amount_text(None) is None
ds, _, client, *_ = workspace
response = client.post(
f"/api/datasets/{ds.pk}/evidence",
data={
"start": "2026-08",
"end": "2026-08",
"comparison": "yoy",
"group": "department",
"side": "current",
"limit": 1,
},
content_type="application/json",
)
row = response.json()["rows"][0]
assert row["revenue"] == "80"
assert row["cost"] == "30"
assert row["gross"] == "50"
def test_settings_revision_and_invalid_metric_are_rejected(workspace):
ds, _, client, *_ = workspace
url = f"/api/datasets/{ds.pk}/settings"
@@ -198,8 +169,8 @@ def test_csv_export_neutralizes_formula_injection_and_preserves_exact_values(wor
assert response.streaming
content = b"".join(response.streaming_content).decode("utf-8-sig")
assert "'=1+1" in content
assert ",'-10,5,'-15" in content
assert "00000000000000" not in content
assert ",-10.000000000000000000000000," in content
assert "'-10.000000000000000000000000" not in content
assert "traceiq-evidence.csv" in response["Content-Disposition"]
xlsx = client.post(
+76 -2
View File
@@ -1,4 +1,5 @@
import os
import uuid
from datetime import timedelta
import pytest
@@ -27,6 +28,12 @@ def cleanup_root(settings, tmp_path):
return tmp_path / "formula-exports", dataset, user
def _aged(path, hours=25):
stamp = (timezone.now() - timedelta(hours=hours)).timestamp()
os.utime(path, (stamp, stamp))
return path
def test_cleanup_expires_jobs_and_removes_files(cleanup_root):
root, dataset, user = cleanup_root
path = root / str(user.pk) / "expired.xlsx"
@@ -55,8 +62,7 @@ def test_cleanup_removes_only_stale_temporary_files(cleanup_root):
stale, recent = user_dir / "stale.tmp", user_dir / "recent.tmp"
stale.touch()
recent.touch()
old = timezone.now().timestamp() - 25 * 3600
os.utime(stale, (old, old))
_aged(stale)
call_command("cleanup_formula_exports")
@@ -81,3 +87,71 @@ def test_cleanup_does_not_delete_files_outside_export_root(cleanup_root, tmp_pat
job.refresh_from_db()
assert job.status == "expired"
assert outside.read_bytes() == b"keep"
def test_cleanup_reclaims_failed_job_leftover_after_retention(cleanup_root):
root, dataset, user = cleanup_root
job = FormulaExport.objects.create(dataset=dataset, user=user, status="failed")
path = root / str(user.pk) / f"{job.pk}.xlsx"
path.parent.mkdir(parents=True)
path.write_bytes(b"leftover")
job.path = str(path)
job.save(update_fields=["path"])
_aged(path)
call_command("cleanup_formula_exports")
job.refresh_from_db()
assert not path.exists()
assert job.path == ""
def test_cleanup_keeps_workbook_of_unexpired_succeeded_job(cleanup_root):
root, dataset, user = cleanup_root
job = FormulaExport.objects.create(
dataset=dataset,
user=user,
status="succeeded",
expires_at=timezone.now() + timedelta(hours=1),
)
path = root / str(user.pk) / f"{job.pk}.xlsx"
path.parent.mkdir(parents=True)
path.write_bytes(b"active")
job.path = str(path)
job.save(update_fields=["path"])
_aged(path)
call_command("cleanup_formula_exports")
assert path.read_bytes() == b"active"
job.refresh_from_db()
assert job.path == str(path)
def test_cleanup_reclaims_orphan_whose_job_row_was_deleted(cleanup_root):
root, _, user = cleanup_root
user_dir = root / str(user.pk)
user_dir.mkdir(parents=True)
orphan = user_dir / f"{uuid.uuid4()}.xlsx"
orphan.write_bytes(b"orphan")
recent = user_dir / f"{uuid.uuid4()}.xlsx"
recent.write_bytes(b"recent orphan")
_aged(orphan)
call_command("cleanup_formula_exports")
assert not orphan.exists()
assert recent.exists()
def test_cleanup_ignores_non_uuid_workbook_names(cleanup_root):
root, _, user = cleanup_root
user_dir = root / str(user.pk)
user_dir.mkdir(parents=True)
odd = user_dir / "notes.xlsx"
odd.write_bytes(b"keep")
_aged(odd)
call_command("cleanup_formula_exports")
assert odd.exists()
+50 -24
View File
@@ -97,30 +97,8 @@ def test_formula_workbook_exposes_custom_metric_as_recalculable_column(formula_d
custom = workbook["自定义指标"]
assert custom["B1"].value == "收入成本差(本期)"
assert custom["B2"].value == f"='分析汇总'!C{metric_row}"
instruction_text = " ".join(str(cell.value) for row in workbook["使用说明"].iter_rows() for cell in row)
assert "含义:实际收入减成本" in instruction_text
assert "范围:按所选分组及本期、基期计算" in instruction_text
def test_formula_workbook_protects_structure_and_unlocks_calculation_inputs(formula_data):
ds, member = formula_data
output = BytesIO()
build_formula_workbook(
ds,
member,
{"start": "2026-08", "end": "2026-08", "comparison": "yoy", "group": "department"},
output,
)
workbook = load_workbook(BytesIO(output.getvalue()), data_only=False)
data = workbook["计算明细"]
assert data.protection.sheet
assert data["D2"].protection.locked is False
assert data["F2"].protection.locked is False
assert data["G2"].protection.locked is False
assert data["I2"].protection.locked is False
assert data["C2"].protection.locked is True
assert data["M2"].protection.locked is True
assert workbook["原始证据"].protection.sheet
assert "含义:实际收入减成本" in workbook["使用说明"]["B12"].value
assert "范围:按所选分组及本期、基期计算" in workbook["使用说明"]["B12"].value
def test_custom_expression_rejects_formula_text_and_unknown_metrics():
@@ -222,6 +200,54 @@ def test_design_column_preserves_clarification_branch_and_full_conversation(form
assert payload["input"][1:] == messages
def test_design_column_accepts_loaded_definition_context(formula_data, settings):
ds, _ = formula_data
settings.AI_ENABLED = True
settings.OPENAI_API_KEY = "test-key"
settings.OPENAI_MODEL = "test-model"
context = "已载入已保存定义「收入成本差」;计算表达式:" + '{"operator":"subtract",' * 45
assert 1000 < len(context) <= 2000
messages = [
{"role": "user", "content": context},
{"role": "assistant", "content": "已载入该定义作为当前基准。"},
{"role": "user", "content": "把刚才公式改成乘以 100"},
]
clarification = {
"status": "completed",
"output": [
{
"content": [
{
"type": "output_text",
"text": '{"complete":false,"questions":["要乘在整个表达式上吗?"],"column":null}',
}
]
}
],
}
with patch("finance.formula_views.httpx.Client") as http:
http.return_value.__enter__.return_value.post.return_value = type(
"Response",
(),
{"raise_for_status": lambda self: None, "json": lambda self: clarification},
)()
result = design_column(
messages,
ds,
{"current": ["2026-08"], "baseline": ["2025-08"], "filters": {}, "group": "department"},
)
assert result["complete"] is False
payload = http.return_value.__enter__.return_value.post.call_args.kwargs["json"]
assert payload["input"][1:] == messages
with pytest.raises(Exception, match="对话消息内容无效"):
design_column(
[{"role": "user", "content": "y" * 2001}],
ds,
{"current": ["2026-08"], "baseline": ["2025-08"], "filters": {}, "group": "department"},
)
def test_design_column_returns_only_validated_formula_definition(formula_data, settings):
ds, _ = formula_data
settings.AI_ENABLED = True
+135 -48
View File
@@ -1,3 +1,4 @@
import os
import stat
from decimal import Decimal
from io import BytesIO
@@ -198,22 +199,25 @@ def test_formula_export_worker_records_path_preparation_failure(workspace, setti
assert job.status == "failed"
def test_formula_export_failure_does_not_follow_a_symlinked_user_directory(workspace, settings):
def _queued_job(ds):
return FormulaExport.objects.create(
dataset=ds,
user=ds.created_by,
context={"start": "2026-08", "end": "2026-08", "comparison": "yoy", "group": "department"},
)
def test_failed_export_rejects_symlinked_user_directory_without_touching_files(workspace, settings):
from finance.exporting import process_formula_export
ds, _, client, *_ = workspace
response = client.post(
f"/api/datasets/{ds.pk}/formula-exports",
data={"start": "2026-08", "end": "2026-08", "comparison": "yoy", "group": "department"},
content_type="application/json",
)
job = FormulaExport.objects.get(pk=response.json()["id"])
ds, *_ = workspace
job = _queued_job(ds)
root = Path(settings.MEDIA_ROOT) / "formula-exports"
root.mkdir(parents=True)
protected = Path(settings.MEDIA_ROOT) / "protected"
protected.mkdir()
target = protected / f"{job.pk}.xlsx"
target.write_bytes(b"keep")
root.mkdir()
victim = protected / f"{job.pk}.xlsx"
victim.write_bytes(b"protected workbook")
(root / str(job.user_id)).symlink_to(protected, target_is_directory=True)
with pytest.raises(ValueError, match="符号链接"):
@@ -221,42 +225,142 @@ def test_formula_export_failure_does_not_follow_a_symlinked_user_directory(works
job.refresh_from_db()
assert job.status == "failed"
assert target.read_bytes() == b"keep"
assert not job.path
assert victim.read_bytes() == b"protected workbook"
assert list(protected.glob("*.tmp")) == []
def test_formula_export_failed_cleanup_is_observable_and_reclaimed(workspace, monkeypatch):
def test_failed_export_rejects_symlinked_target_and_keeps_link(workspace, settings):
from finance.exporting import process_formula_export
ds, *_ = workspace
job = _queued_job(ds)
user_dir = Path(settings.MEDIA_ROOT) / "formula-exports" / str(job.user_id)
user_dir.mkdir(parents=True)
protected_file = Path(settings.MEDIA_ROOT) / "protected.xlsx"
protected_file.write_bytes(b"protected workbook")
(user_dir / f"{job.pk}.xlsx").symlink_to(protected_file)
with pytest.raises(ValueError, match="普通文件"):
process_formula_export(job.pk)
job.refresh_from_db()
assert job.status == "failed"
assert protected_file.read_bytes() == b"protected workbook"
assert (user_dir / f"{job.pk}.xlsx").is_symlink()
assert list(user_dir.glob("*.tmp")) == []
def test_failed_export_refuses_cleanup_when_parent_directory_swapped_to_symlink(
workspace, settings, monkeypatch
):
from finance.exporting import process_formula_export
ds, *_ = workspace
job = _queued_job(ds)
root = Path(settings.MEDIA_ROOT) / "formula-exports"
user_dir = root / str(job.user_id)
user_dir.mkdir(parents=True)
protected = Path(settings.MEDIA_ROOT) / "protected"
protected.mkdir()
victim = protected / f"{job.pk}.xlsx"
victim.write_bytes(b"protected workbook")
def swap_and_fail(*args, **kwargs):
args[-1].write(b"partial")
user_dir.rename(user_dir.with_name("swapped"))
user_dir.symlink_to(protected, target_is_directory=True)
raise ValueError("controlled failure after path validation")
monkeypatch.setattr("finance.exporting.build_formula_workbook", swap_and_fail)
with pytest.raises(ValueError, match="controlled failure"):
process_formula_export(job.pk)
job.refresh_from_db()
assert job.status == "failed"
assert victim.read_bytes() == b"protected workbook"
assert user_dir.is_symlink()
# 本次创建的临时文件残留在被换名的原目录中,由清理命令按保留期回收。
assert list((root / "swapped").glob(f"{job.pk}.*.tmp"))
def test_formula_export_retry_after_failure_succeeds(workspace, monkeypatch):
from finance.exporting import process_formula_export
ds, *_ = workspace
job = _queued_job(ds)
def fail(*args, **kwargs):
args[-1].write(b"partial")
raise ValueError("controlled export failure")
monkeypatch.setattr("finance.exporting.build_formula_workbook", fail)
with pytest.raises(ValueError, match="controlled export failure"):
process_formula_export(job.pk)
monkeypatch.undo()
stats = process_formula_export(job.pk)
job.refresh_from_db()
assert stats["rows"] == 5
assert job.status == "succeeded"
assert Path(job.path).read_bytes()[:2] == b"PK"
assert not list(Path(job.path).parent.glob(f"{job.pk}.*.tmp"))
def test_cleanup_failure_is_logged_and_reclaimed_after_permissions_restored(
workspace, settings, monkeypatch, caplog
):
import logging as logging_module
from datetime import timedelta
from django.core.management import call_command
from django.utils import timezone
from finance.exporting import process_formula_export
ds, _, client, *_ = workspace
response = client.post(
f"/api/datasets/{ds.pk}/formula-exports",
data={"start": "2026-08", "end": "2026-08", "comparison": "yoy", "group": "department"},
content_type="application/json",
ds, *_ = workspace
job = _queued_job(ds)
user_dir = Path(settings.MEDIA_ROOT) / "formula-exports" / str(job.user_id)
user_dir.mkdir(parents=True)
target = user_dir / f"{job.pk}.xlsx"
target.write_bytes(b"stale workbook")
other = FormulaExport.objects.create(
dataset=ds,
user=ds.created_by,
context={"start": "2026-08", "end": "2026-08", "comparison": "yoy", "group": "department"},
status="succeeded",
expires_at=timezone.now() + timedelta(hours=1),
)
job = FormulaExport.objects.get(pk=response.json()["id"])
target = Path(job.dataset.file.storage.location) / "formula-exports" / str(job.user_id) / f"{job.pk}.xlsx"
target.parent.mkdir(parents=True)
target.write_bytes(b"orphan")
monkeypatch.setattr(
"finance.exporting.build_formula_workbook",
lambda *_a, **_k: (_ for _ in ()).throw(ValueError("retry failed")),
)
real_unlink = Path.unlink
monkeypatch.setattr(Path, "unlink", lambda *_a, **_k: (_ for _ in ()).throw(OSError("read-only")))
kept = user_dir / f"{other.pk}.xlsx"
other.path = str(kept)
other.save(update_fields=["path"])
kept.write_bytes(b"active workbook")
with pytest.raises(ValueError, match="retry failed"):
def fail(*args, **kwargs):
args[-1].write(b"partial")
os.chmod(user_dir, 0o500)
raise ValueError("controlled export failure")
monkeypatch.setattr("finance.exporting.build_formula_workbook", fail)
with pytest.raises(ValueError, match="controlled export failure"):
process_formula_export(job.pk)
job.refresh_from_db()
assert job.status == "failed"
assert job.path == str(target)
warnings = [r for r in caplog.records if r.levelno == logging_module.WARNING]
assert warnings and all(str(job.pk) in record.getMessage() for record in warnings)
monkeypatch.setattr(Path, "unlink", real_unlink)
os.chmod(user_dir, 0o700)
old = (timezone.now() - timedelta(hours=25)).timestamp()
for leftover in [target, *user_dir.glob(f"{job.pk}.*.tmp"), kept]:
os.utime(leftover, (old, old))
call_command("cleanup_formula_exports")
job.refresh_from_db()
assert not target.exists()
assert not list(user_dir.glob(f"{job.pk}.*.tmp"))
assert job.path == ""
assert target.exists() is False
assert kept.read_bytes() == b"active workbook"
def test_formula_workbook_rejects_unclassified_label_collision(workspace):
@@ -435,20 +539,3 @@ def test_formula_workbook_rejects_dimension_keys_that_collide_with_reserved_head
{"start": "2026-08", "end": "2026-08", "comparison": "yoy", "group": "revenue"},
BytesIO(),
)
def test_custom_reconcile_reference_preserves_large_decimal_and_zero_division():
from finance.exporting import _evaluate_expression
expression = {
"operator": "divide",
"left": {"metric": "revenue"},
"right": {"number": 0},
}
assert (
_evaluate_expression(expression, {"revenue": Decimal("1.2345678901234567890123456789012345")}) is None
)
assert _evaluate_expression(
{"operator": "add", "left": {"metric": "revenue"}, "right": {"number": 1}},
{"revenue": Decimal("1234567890123456789012345678901234")},
) == Decimal("1234567890123456789012345678901235")
+1 -41
View File
@@ -46,23 +46,6 @@ def test_worker_processes_queued_import_and_formula_export(workspace):
assert job.status == "processing"
def test_worker_reclaims_processing_formula_export_after_restart(workspace):
ds, _, _, *_ = workspace
job = FormulaExport.objects.create(
dataset=ds,
user=ds.created_by,
context={"custom_columns": []},
status="processing",
)
from unittest.mock import patch
with patch(
"finance.management.commands.import_worker.process_formula_export", return_value={"rows": 5}
) as process:
call_command("import_worker", once=True)
process.assert_called_once_with(job.pk)
def test_formula_export_status_download_and_access_controls(workspace, settings, tmp_path):
ds, _, client, _, other_client = workspace
url = f"/api/datasets/{ds.pk}"
@@ -75,7 +58,7 @@ def test_formula_export_status_download_and_access_controls(workspace, settings,
job.status = "succeeded"
job.expires_at = timezone.now() + timedelta(hours=1)
target = tmp_path / "formula-exports" / str(ds.created_by_id) / f"{job.pk}.xlsx"
target.parent.mkdir(parents=True, exist_ok=True)
target.parent.mkdir(parents=True)
target.write_bytes(b"xlsx")
job.path = str(target)
job.save()
@@ -102,29 +85,6 @@ def test_formula_export_status_download_and_access_controls(workspace, settings,
assert client.get(status_url).status_code == 403
def test_formula_export_rechecks_role_when_worker_runs_and_file_is_downloaded(workspace, settings, tmp_path):
from finance.exporting import process_formula_export
ds, member, client, _, _ = workspace
job = FormulaExport.objects.create(dataset=ds, user=ds.created_by, context={}, status="processing")
member.role = "viewer"
member.save(update_fields=["role"])
with pytest.raises(PermissionError, match="撤权"):
process_formula_export(job.pk)
job.refresh_from_db()
assert job.status == "failed"
job.status = "succeeded"
job.expires_at = timezone.now() + timedelta(hours=1)
target = tmp_path / "formula-exports" / str(ds.created_by_id) / f"{job.pk}.xlsx"
target.parent.mkdir(parents=True, exist_ok=True)
target.write_bytes(b"xlsx")
job.path = str(target)
job.save(update_fields=["status", "expires_at", "path"])
assert client.get(f"/api/datasets/{ds.pk}/formula-exports/{job.pk}").status_code == 403
assert client.get(f"/api/datasets/{ds.pk}/formula-exports/{job.pk}/download").status_code == 403
def test_formula_export_create_validates_definition_and_records_job(workspace):
ds, _, client, *_ = workspace
url = f"/api/datasets/{ds.pk}/formula-exports"
+118
View File
@@ -1,7 +1,10 @@
from datetime import timedelta
from pathlib import Path
from unittest.mock import patch
import pytest
from django.core.management import call_command
from django.utils import timezone
from finance.models import Dataset, FormulaExport
@@ -30,3 +33,118 @@ def test_worker_processes_queued_import_and_reports_export_failure(workspace):
call_command("import_worker", once=True)
export.refresh_from_db()
assert export.status == "processing"
def _export_job(ds, **kwargs):
return FormulaExport.objects.create(
dataset=ds,
user=ds.created_by,
context={"start": "2026-08", "end": "2026-08", "comparison": "yoy", "group": "department"},
**kwargs,
)
def test_worker_refuses_export_after_role_revocation(workspace):
from finance.exporting import process_formula_export
ds, member, *_ = workspace
job = _export_job(ds, status="processing")
member.role = "viewer"
member.save(update_fields=["role"])
with pytest.raises(PermissionError, match="角色"):
process_formula_export(job.pk)
job.refresh_from_db()
assert job.status == "failed"
assert not job.path
assert not list(Path(ds.file.storage.location).rglob(f"{job.pk}*"))
def test_worker_refuses_export_after_account_deactivation(workspace):
from finance.exporting import process_formula_export
ds, _, _, _, _ = workspace
user = ds.created_by
user.is_active = False
user.save(update_fields=["is_active"])
job = _export_job(ds, status="processing")
with pytest.raises(PermissionError, match="停用"):
process_formula_export(job.pk)
job.refresh_from_db()
assert job.status == "failed"
def test_status_and_download_deny_revoked_viewer(workspace, settings):
ds, member, client, *_ = workspace
job = _export_job(
ds,
status="succeeded",
finished_at=timezone.now(),
expires_at=timezone.now() + timedelta(hours=1),
)
target = Path(settings.MEDIA_ROOT) / "formula-exports" / str(job.user_id) / f"{job.pk}.xlsx"
target.parent.mkdir(parents=True)
target.write_bytes(b"PK workbook")
job.path = str(target)
job.save(update_fields=["path"])
assert client.get(f"/api/datasets/{ds.pk}/formula-exports/{job.pk}").status_code == 200
assert (
client.get(f"/api/datasets/{ds.pk}/formula-exports/{job.pk}/download").status_code == 200
)
member.role = "viewer"
member.save(update_fields=["role"])
assert client.get(f"/api/datasets/{ds.pk}/formula-exports/{job.pk}").status_code == 403
assert (
client.get(f"/api/datasets/{ds.pk}/formula-exports/{job.pk}/download").status_code == 403
)
def test_half_finished_export_cannot_be_downloaded(workspace, settings):
ds, _, client, *_ = workspace
for status, code in (("processing", 409), ("failed", 409), ("queued", 409)):
job = _export_job(ds, status=status)
assert (
client.get(f"/api/datasets/{ds.pk}/formula-exports/{job.pk}/download").status_code
== code
), status
succeeded = _export_job(
ds,
status="succeeded",
path=str(Path(settings.MEDIA_ROOT) / "nowhere.xlsx"),
finished_at=timezone.now(),
expires_at=timezone.now() + timedelta(hours=1),
)
assert (
client.get(f"/api/datasets/{ds.pk}/formula-exports/{succeeded.pk}/download").status_code
== 410
)
def test_stale_processing_export_is_reclaimed_after_timeout(workspace):
ds, *_ = workspace
job = _export_job(ds, status="processing")
FormulaExport.objects.filter(pk=job.pk).update(
created_at=timezone.now() - timedelta(minutes=20)
)
call_command("import_worker", once=True)
job.refresh_from_db()
assert job.status == "succeeded"
assert Path(job.path).read_bytes()[:2] == b"PK"
assert not list(Path(job.path).parent.glob(f"{job.pk}.*.tmp"))
def test_recent_processing_export_is_not_reclaimed(workspace):
ds, *_ = workspace
job = _export_job(ds, status="processing")
call_command("import_worker", once=True)
job.refresh_from_db()
assert job.status == "processing"