docs: rewrite all markdown documentation for views/tools package refactoring

- Update README.md: new architecture (views/ 12 modules, tools/ 17 modules), remove simple_analysis refs, 30 MCP tools, 8GB RAM spec - Update AGENTS.md: project structure tree, 30 tools organized by implementation module, remove entity_detector/entity_aggregator refs - Update docs/工作流.md: full pipeline v3.0, code references to tools/ package, remove simple_analysis section (note moved to master) - Update docs/故障诊断手册.md: fix code location references, add WebGL2 tips, remove Leaflet/simple_analysis refs Branch: beta-clean
cleanup: consolidate TLS_HEX_MAP, extract constants, remove dead code, add traceback logging
2026-07-23 22:51:36 +08:00 · 2026-07-23 22:44:42 +08:00 · 2026-07-23 22:31:17 +08:00 · 2026-07-23 22:22:28 +08:00 · 2026-07-23 22:10:26 +08:00 · 2026-07-23 20:51:32 +08:00
85 changed files with 11730 additions and 7975 deletions
@@ -40,3 +40,6 @@ RUNTIME_CHANGED.txt
 # Test data (auto-generated CSV files)
 data/*.csv
 data/multi_upload/
+
+# User config (contains API keys)
+config/config.yaml
@@ -0,0 +1 @@
+11572
@@ -13,11 +13,10 @@
 | 机器学习 | scikit-learn (HDBSCAN, KMeans, MiniBatchKMeans, IsolationForest), UMAP, TruncatedSVD, StandardScaler |
 | Web 框架 | Django 4.2 + SQLite WAL |
 | 前端 3D | Three.js (603KB, 离线, 地球贴图 1.4MB) |
-| 前端图表 | 纯 Canvas 2D 散点图 / Leaflet 地图 / Canvas 地理分布 |
-| 前端地图 | Leaflet (简易分析模块) |
+| 前端图表 | 纯 Canvas 2D 散点图 / Canvas 地理分布 |
 | LLM 协议 | MCP (stdio transport) + OpenAI 兼容 API |
-| 启动方式 | `run.bat` (双击即用) / `runtime\python\python.exe manage.py runserver` (开发) |
-| 目标设备 | Windows 10 1902+, 4GB RAM, 无独立显卡 (核显) |
+| 启动方式 | `run.bat` (双击即用, 非阻塞, 打印 PID 后退出) / `runtime\python\python.exe manage.py runserver` (开发) |
+| 目标设备 | Windows 10 1909+, 8GB RAM, 无独立显卡 (核显), 支持 2500 CSV × 10000 行 × 50~58 列 |

 ## Project Structure

@@ -26,59 +25,92 @@
 ├── tianxuan/                     Django 项目配置
 │   ├── settings.py               ALLOWED_HOSTS 自动检测, LOGGING 文件+stderr
 │   ├── wsgi.py                   SQLite 启动自修复
-│   ├── urls.py                   路由汇总 (含 simple_analysis 路由)
-│   └── llm_orchestrator.py       LLM 编排 (32B/284B 双兼容, 策略式提示词)
+│   ├── urls.py                   路由汇总
+│   └── llm_orchestrator.py       LLM 编排 (策略式提示词, 自动错误恢复)
 ├── analysis/                     核心分析模块 (Django app)
-│   ├── data_loader.py            CSV 加载, BOM/编码检测, schema 容错, 递归 glob, ZIP 解压
-│   ├── data_profiler.py          列统计 + 相关性矩阵 (numpy 无 pandas 回退)
-│   ├── entity_detector.py        实体列自动检测 (tuple 关键词 + unique_ratio + null 惩罚)
-│   ├── entity_aggregator.py      实体聚合 (tuple 关键词, lat/lon 检测, 多列 group_by, IP 子网)
-│   ├── data_validator.py         列校验 (缺失/异常值/IP 有效性)
-│   ├── type_classifier.py        值优先类型检测 (MAC/端口/IPv4/URL/HEX/ENUM/LAT_LON/BOOL_ENUM)
-│   ├── geoip.py                  GeoIP 经纬度查询 (data/geoip_data.txt)
-│   ├── ip_clustering.py          IP 子网聚类与转换
-│   ├── session_store.py          线程安全单例内存存储 + 特征结果
-│   ├── tool_registry.py          30 个 MCP 工具注册 + 处理函数 (sync_to_async DB)
+│   ├── views/                    视图包 (12 模块, 原 views.py 拆分)
+│   │   ├── __init__.py           重导出所有视图函数
+│   │   ├── helpers.py            辅助函数
+│   │   ├── dashboard.py          首页 + 运行记录
+│   │   ├── pipeline.py           管道执行器
+│   │   ├── clustering.py         聚类概览/详情/地球数据
+│   │   ├── entity.py             实体画像页
+│   │   ├── upload.py             CSV 上传
+│   │   ├── manual.py             手动分析工作流构建器
+│   │   ├── auto.py               LLM 自动分析
+│   │   ├── globe.py              3D 地球数据接口
+│   │   ├── config.py             配置编辑 + LLM 测试
+│   │   ├── log_viewer.py         日志查看器
+│   │   └── tools.py              工具实验室
+│   ├── tools/                    工具包 (17 模块, 30 个 MCP 工具实现)
+│   │   ├── __init__.py           重导出所有 handler
+│   │   ├── _registry.py          工具元数据 (Tool descriptor + get_tools_meta)
+│   │   ├── _dispatch.py          工具调用分发 (handle_call)
+│   │   ├── _helpers.py           共享辅助 (过滤表达式构建/响应截断/数据集决议/数值列检测)
+│   │   ├── load_data.py          工具 1: load_data
+│   │   ├── profile.py            工具 2: profile_data
+│   │   ├── filter.py             工具 3: filter_data
+│   │   ├── preprocess.py         工具 4: preprocess_data
+│   │   ├── clustering.py         工具 5+8: run_clustering, filter_and_cluster
+│   │   ├── evaluate.py           工具 6: evaluate_clustering
+│   │   ├── features.py           工具 7: extract_features (+ UMAP 嵌入)
+│   │   ├── entities.py           工具 9+10: build_entity_profiles, compute_scores
+│   │   ├── anomalies.py          工具 11+12: detect_anomalies, visualize_anomalies
+│   │   ├── export.py             工具 13: export_results
+│   │   ├── analysis.py           工具 14-19: analyze_patterns/temporal/fft/tls_health/geo_distribution/entity_detail
+│   │   ├── diagnostics.py        工具 20-26: validate_data/explore_distributions/find_outliers/diagnose_clustering/compare_datasets/export_debug_sample/repair_schema
+│   │   ├── data_mgmt.py          工具 27-29: list_datasets/drop_dataset/clone_dataset
+│   │   └── distance_matrix.py    工具 30: compute_distance_matrix
+│   ├── tool_registry.py          旧编排层 (薄包装, 重导出 tools/)
 │   ├── mcp_server.py             MCP stdio server
+│   ├── data_loader.py            CSV 加载底层 (BOM/编码/schema 容错/递归 glob/ZIP 解压)
+│   ├── data_profiler.py          列统计 + 相关性矩阵
+│   ├── data_validator.py         列校验 (缺失/异常值/IP 有效性)
+│   ├── type_classifier.py        值优先类型检测 (MAC/端口/IPv4/URL/HEX/ENUM/LAT_LON/BOOL_ENUM/TIMESTAMP)
+│   ├── geoip.py                  GeoIP 经纬度查询
+│   ├── ip_clustering.py          IP 子网聚类与转换
+│   ├── session_store.py          线程安全单例内存存储
+│   ├── distance.py               距离计算 (IP 子网/地理/Levenshtein/Hamming/FFT 相位)
 │   ├── models.py                 ORM: AnalysisRun, ClusterResult, EntityProfile, ClusterFeature
-│   ├── views.py                  所有 Django 视图 (20+ 路由, 上传/手动/LLM/配置/地球/简易)
 │   ├── urls.py                   路由注册
-│   ├── admin.py                  Django admin
+│   ├── nl_describe.py            自然语言数据描述
+│   ├── profile_util.py           画像工具函数
+│   ├── tls_ref.py                TLS 版本/加密套件参考数据
+│   ├── db_utils.py               SQLite WAL 工具 + 锁重试
+│   ├── constants.py              全局常量
+│   ├── appdata.py                应用数据路径 (%APPDATA%/TianXuan)
 │   └── management/commands/
 │       ├── start_mcp.py          MCP 服务器启动命令
-│       └── run_pipeline.py       一键 CLI 管道命令
-├── simple_analysis/              简易分析模块 (独立 Django app)
-│   ├── views.py                  上传 CSV → 按目标筛选 → 地理/IP 子网聚类
-│   ├── urls.py                   路由: /simple/
-│   ├── templates/simple_analysis/ Leaflet 地图展示页面
-│   └── static/simple_analysis/   模块静态资源
+│       ├── run_pipeline.py       一键 CLI 管道命令
+│       └── import_tlsdb.py       TLS 数据库导入
 ├── config/                       配置管理
 │   ├── config.yaml               自动生成默认配置 (entity, server, data, clustering, llm)
 │   ├── loader.py                 Pydantic 配置加载 + 文件修改检测 + 缓存
 │   └── __init__.py
-├── templates/                    页面模板
+├── templates/                    页面模板 (13 个 HTML)
 │   ├── base.html                 导航: 首页/上传/手动/LLM/记录/地球/配置
-│   └── analysis/                 全部 11 个页面模板
-│       ├── dashboard.html        首页 - 最近运行
-│       ├── upload.html            CSV 上传 - 拖拽+多文件删除+进度
-│       ├── manual.html            手动分析 - 工作流构建器 (添加/排序/保存/执行)
-│       ├── auto.html              LLM 自动分析 - 实时日志, thinking panel, 工具调用折叠
-│       ├── config.html            配置编辑 - LLM 连通性测试
-│       ├── run_list.html          运行记录列表 (含 retry 按钮)
-│       ├── run_detail.html        运行详情摘要
-│       ├── cluster_overview.html  聚类概览 - Canvas 散点图 (PC1/PC2) + 地理分布 + Silhouette
-│       ├── cluster_detail.html    簇详情 - 特征实体列表
-│       ├── entity_profile.html    实体画像 - 特征偏离 + Canvas 单点
-│       ├── globe.html             3D 地球 - Three.js 流量弧线 + 经纬线 + 国境线
-│       └── log_viewer.html        日志查看
+│   ├── analysis/                 分析页面 (6 个)
+│   │   ├── dashboard.html        首页 — 最近运行
+│   │   ├── run_list.html         运行记录列表
+│   │   ├── run_detail.html       运行详情摘要
+│   │   ├── cluster_overview.html 聚类概览 — Canvas 散点图 (PC1/PC2) + 地理分布 + Silhouette
+│   │   ├── cluster_detail.html   簇详情 — 特征实体列表
+│   │   └── entity_profile.html   实体画像 — 特征偏离 + Canvas 单点
+│   └── tianxuan/                 功能页面 (6 个)
+│       ├── upload.html           CSV 上传 — 拖拽+多文件删除+进度
+│       ├── manual.html           手动分析 — 工作流构建器 (添加/排序/保存/执行)
+│       ├── auto.html             LLM 自动分析 — 实时日志, thinking panel, 工具调用折叠
+│       ├── globe.html            3D 地球 — Three.js 流量弧线 + 经纬线 + 国境线
+│       ├── config.html           配置编辑 — LLM 连通性测试
+│       └── log_viewer.html       日志查看
 ├── static/tianxuan/              离线静态资源
 │   ├── three.min.js              Three.js 3D 引擎 (603KB)
 │   ├── earth_atmos_2048.jpg      地球贴图 (1.4MB)
 │   └── world_borders.js          177 国完整国境线 (269KB)
 ├── runtime/python/               便携 Python 3.12 运行时 (~593MB)
 ├── scripts/                      工具脚本
-│   ├── gen_test_data.py          简单测试数据生成 (--globe 模式使用真实 GeoIP 范围)
-│   ├── gen_complex_test.py       复杂数据生成 (5000 行 4 列含 55% 缺失)
+│   ├── gen_test_data.py          简单测试数据生成
+│   ├── gen_complex_test.py       复杂数据生成 (5000 行 含 55% 缺失)
 │   ├── column_survey.py          CSV 列结构调查
 │   ├── diagnose_compare.py       跨机日志对比诊断
 │   └── debug_db.py               DB 持久化调试
@@ -87,6 +119,10 @@
 │   ├── test_e2e_batch.py         批量端到端测试
 │   ├── start_srv.py              测试服务器启动辅助
 │   └── _test_hdbscan.py          HDBSCAN 零样本降级测试
+├── data/                         数据目录
+│   └── geoip_data.txt            GeoIP 离线数据库
+├── logs/                         日志输出
+│   └── tianxuan.log              主日志文件
 ├── docs/
 │   ├── 工作流.md                  完整分析流水线文档
 │   └── 故障诊断手册.md            跨机问题诊断指南
@@ -103,59 +139,65 @@

 ## 30 MCP Tools

-系统现有 **30 个 MCP 工具**，分为四层：
+系统现有 **30 个 MCP 工具**, 分四层, 实现于 `analysis/tools/` 包的 14 个 handler 模块:

-### ⚡ 核心工具 (13 个) — 执行分析，可修改数据
+### ⚡ 核心工具 (14 个) — 执行分析, 可修改数据

-| # | 工具 | 功能 | 必需参数 |
-|---|------|------|----------|
-| 1 | `load_data` | 加载 CSV 文件 (glob/递归/ZIP 解压/schema 容错) | `csv_glob` |
-| 2 | `profile_data` | 数据集概要统计 + 相关性矩阵 | `dataset_id` |
-| 3 | `filter_data` | 按条件过滤 (12 操作符 + AND/OR 逻辑) | `dataset_id`, `filters` |
-| 4 | `preprocess_data` | 预处理 (StandardScaler/OneHot/填充/列删除) | `dataset_id`, `columns` |
-| 5 | `run_clustering` | 执行聚类 (HDBSCAN/KMeans + 自动特征过滤 + 质量评估) | `dataset_id`, `cluster_columns` |
-| 6 | `evaluate_clustering` | 评估聚类质量 (Silhouette/DB/CH/噪声比/簇大小) | `cluster_result_id`, `dataset_id` |
-| 7 | `extract_features` | 提取各聚类区分特征 (Z-Score/ANOVA) + UMAP-2D 嵌入 | `dataset_id`, `cluster_result_id` |
-| 8 | `filter_and_cluster` | 一步完成: 过滤 + 自动选数值列 + 聚类 | `dataset_id`, `filters` |
-| 9 | `build_entity_profiles` | 自动检测实体列 + 聚合特征 (多列复合键支持) | `dataset_id` |
-| 10 | `compute_scores` | 自适应代理/异常/威胁评分 (SVD 权重+ IQR 阈值) | `dataset_id` |
-| 11 | `detect_anomalies` | Isolation Forest 异常检测 (分块处理大数据集) | `dataset_id` |
-| 12 | `visualize_anomalies` | UMAP-2D 嵌入 + 聚类可视化 (按风险等级/聚类着色) | `dataset_id` |
-| 13 | `export_results` | 导出结果到磁盘 (CSV/Parquet/JSON) | `result_id`, `output_path` |
+| # | 工具 | 实现模块 | 功能 | 必需参数 |
+|---|------|---------|------|----------|
+| 1 | `load_data` | tools/load_data.py | 加载 CSV 文件 (glob/递归/ZIP 解压/schema 容错) | `csv_glob` |
+| 2 | `profile_data` | tools/profile.py | 数据集概要统计 + 相关性矩阵 | `dataset_id` |
+| 3 | `filter_data` | tools/filter.py | 按条件过滤 (12 操作符 + AND/OR 逻辑) | `dataset_id`, `filters` |
+| 4 | `preprocess_data` | tools/preprocess.py | 预处理 (StandardScaler/OneHot/填充/列删除) | `dataset_id`, `columns` |
+| 5 | `run_clustering` | tools/clustering.py | 执行聚类 (HDBSCAN/KMeans + 自动特征过滤 + 质量评估) | `dataset_id`, `cluster_columns` |
+| 6 | `evaluate_clustering` | tools/evaluate.py | 评估聚类质量 (Silhouette/DB/CH/噪声比/簇大小) | `cluster_result_id`, `dataset_id` |
+| 7 | `extract_features` | tools/features.py | 提取各聚类区分特征 (Z-Score/ANOVA) + UMAP-2D 嵌入 | `dataset_id`, `cluster_result_id` |
+| 8 | `filter_and_cluster` | tools/clustering.py | 一步完成: 过滤 + 自动选数值列 + 聚类 | `dataset_id`, `filters` |
+| 9 | `build_entity_profiles` | tools/entities.py | 自动检测实体列 + 聚合特征 (多列复合键支持) | `dataset_id` |
+| 10 | `compute_scores` | tools/entities.py | 自适应代理/异常/威胁评分 (SVD 权重 + IQR 阈值) | `dataset_id` |
+| 11 | `detect_anomalies` | tools/anomalies.py | Isolation Forest 异常检测 (分块处理大数据集) | `dataset_id` |
+| 12 | `visualize_anomalies` | tools/anomalies.py | UMAP-2D 嵌入 + 聚类可视化 (按风险等级/聚类着色) | `dataset_id` |
+| 13 | `export_results` | tools/export.py | 导出结果到磁盘 (CSV/Parquet/JSON) | `result_id`, `output_path` |
+| - | `export_entities_json` | tools/export.py | 导出实体画像为 JSON (附加工具) | `dataset_id`, `output_path` |

-### 🔍 分析工具 (6 个) — 只读深潜，安全随时调用
+### 🔍 分析工具 (6 个) — 只读深潜, 安全随时调用

-| # | 工具 | 功能 |
-|---|------|------|
-| 14 | `analyze_patterns` | 流量模式: top 源/目标 IP、端口/TLS 版本/协议分布 |
-| 15 | `analyze_temporal` | 时间维度流量分析: 小时/日流量计数、繁忙/空闲时段 |
-| 16 | `analyze_fft` | FFT 频谱分析: 周期模式、频率特征、周期性评分 |
-| 17 | `analyze_tls_health` | TLS 安全态势: 旧版比例、弱加密、证书问题、SNI 异常 |
-| 18 | `analyze_geo_distribution` | 地理分布: top 来源国家、不可能旅行、ISP 多样性 |
-| 19 | `analyze_entity_detail` | 单实体深度调查: 流/版本/目标/时间模式/异常评分 |
+| # | 工具 | 实现模块 | 功能 |
+|---|------|---------|------|
+| 14 | `analyze_patterns` | tools/analysis.py | 流量模式: top 源/目标 IP、端口/TLS 版本/协议分布 |
+| 15 | `analyze_temporal` | tools/analysis.py | 时间维度流量分析: 小时/日流量计数、繁忙/空闲时段 |
+| 16 | `analyze_fft` | tools/analysis.py | FFT 频谱分析: 周期模式、频率特征、周期性评分 |
+| 17 | `analyze_tls_health` | tools/analysis.py | TLS 安全态势: 旧版比例、弱加密、证书问题、SNI 异常 |
+| 18 | `analyze_geo_distribution` | tools/analysis.py | 地理分布: top 来源国家、不可能旅行、ISP 多样性 |
+| 19 | `analyze_entity_detail` | tools/analysis.py | 单实体深度调查: 流/版本/目标/时间模式/异常评分 |

 ### 🩺 诊断工具 (7 个) — 排查问题

-| # | 工具 | 功能 |
-|---|------|------|
-| 20 | `validate_data` | 数据质量: schema 一致性、缺失率、列类型冲突 |
-| 21 | `explore_distributions` | 列分布统计: 唯一值、空值率、min/max/mean/std、直方图 |
-| 22 | `find_outliers` | IQR 方法检测数值列统计异常值 |
-| 23 | `diagnose_clustering` | 聚类效果差时诊断: 方差/相关分析 + 参数建议 |
-| 24 | `compare_datasets` | 对比两数据集: schema 差异、行数变化、值分布偏移 |
-| 25 | `export_debug_sample` | 导出原始数据样本 JSON 供外部调试 |
-| 26 | `repair_schema` | 修复 schema 不匹配: 列名对齐、大小写合并、填充缺失 |
+| # | 工具 | 实现模块 | 功能 |
+|---|------|---------|------|
+| 20 | `validate_data` | tools/diagnostics.py | 数据质量: schema 一致性、缺失率、列类型冲突 |
+| 21 | `explore_distributions` | tools/diagnostics.py | 列分布统计: 唯一值、空值率、min/max/mean/std、直方图 |
+| 22 | `find_outliers` | tools/diagnostics.py | IQR 方法检测数值列统计异常值 |
+| 23 | `diagnose_clustering` | tools/diagnostics.py | 聚类效果差时诊断: 方差/相关分析 + 参数建议 |
+| 24 | `compare_datasets` | tools/diagnostics.py | 对比两数据集: schema 差异、行数变化、值分布偏移 |
+| 25 | `export_debug_sample` | tools/diagnostics.py | 导出原始数据样本 JSON 供外部调试 |
+| 26 | `repair_schema` | tools/diagnostics.py | 修复 schema 不匹配: 列名对齐、大小写合并、填充缺失 |

-### 工具 (4 个) — 数据管理
+### 📦 数据管理 (3 个)

-| # | 工具 | 功能 |
-|---|------|------|
-| 27 | `list_datasets` | 列出所有活跃数据集和结果 |
-| 28 | `drop_dataset` | 删除数据集释放内存 (支持 dataset/cluster/feature 三种类型) |
-| 29 | `clone_dataset` | 浅拷贝数据集 (LazyFrame 查询计划, 不复制数据) |
-| 30 | `compute_distance_matrix` | LLM 驱动: 用户编写 Python 距离函数, 逐行执行并返回评分摘要 |
+| # | 工具 | 实现模块 | 功能 |
+|---|------|---------|------|
+| 27 | `list_datasets` | tools/data_mgmt.py | 列出所有活跃数据集和结果 |
+| 28 | `drop_dataset` | tools/data_mgmt.py | 删除数据集释放内存 (支持 dataset/cluster/feature 三种类型) |
+| 29 | `clone_dataset` | tools/data_mgmt.py | 浅拷贝数据集 (LazyFrame 查询计划, 不复制数据) |

-所有工具通过 `analysis/tool_registry.py` 注册，`analysis/mcp_server.py` 暴露标准 MCP stdio 接口。
+### 🧠 LLM 驱动 (1 个)
+
+| # | 工具 | 实现模块 | 功能 |
+|---|------|---------|------|
+| 30 | `compute_distance_matrix` | tools/distance_matrix.py | LLM 编写 Python 距离函数, 逐行执行并返回评分摘要 |
+
+**工具注册流程**: `tools/_registry.py` (Tool 元数据) → `tools/_dispatch.py` (分发到对应 handler) → `analysis/mcp_server.py` (暴露 MCP stdio 接口)

 ### 手动分析工作流

@@ -169,10 +211,10 @@
 ### 标准分析流程

 ```
-load_data → profile_data → build_entity_profiles → compute_scores → run_clustering → extract_features
+load_data → filter_data → profile_data → build_entity_profiles → compute_scores → run_clustering → extract_features
 ```

-LLM 自动模式由 `tianxuan/llm_orchestrator.py` 驱动，策略式提示词: 先 profile 了解数据 → 根据数据决策 → 失败自动诊断 → 重复调用检测 + 自动错误恢复。
+LLM 自动模式由 `tianxuan/llm_orchestrator.py` 驱动, 策略式提示词: 先 profile 了解数据 → 根据数据决策 → 失败自动诊断 → 重复调用检测 + 自动错误恢复。

 ## Config.Entity

@@ -180,8 +222,8 @@ LLM 自动模式由 `tianxuan/llm_orchestrator.py` 驱动，策略式提示词:

 ```python
 class Entity(BaseModel):
-    subnet_masks: list[int] = []           # 子网掩码列表，如 [24, 28]
-    ip_columns: list[str] = ['src_ip', 'dst_ip']  # 需聚合的 IP 列
+    subnet_masks: list[int] = []                     # 子网掩码列表, 如 [24, 28]
+    ip_columns: list[str] = ['src_ip', 'dst_ip']     # 需聚合的 IP 列
 ```

 config.yaml 默认配置:
@@ -191,17 +233,14 @@ entity:
  ip_columns: ["src_ip", "dst_ip"]
 ```

-`entity_aggregator.ip_to_subnet()` 负责 IP 到 CIDR 子网前缀的转换，
-`apply_subnet_aggregation()` 在 group_by 前对 IP 列应用子网掩码:
-```python
-ip_to_subnet("192.168.1.42", 24)  # → "192.168.1.0/24"
-ip_to_subnet("192.168.1.42", 28)  # → "192.168.1.32/28"
-```
+IP 子网聚合逻辑位于 `analysis/ip_clustering.py`, `tools/entities.py::_handle_build_entity_profiles` 在 group_by 前对 IP 列自动应用子网掩码。

 ---

 ## Architecture

+- **views 包拆分**: 原单体 `views.py` (2000+ 行) 拆分为 12 个模块: dashboard, pipeline, clustering, entity, upload, manual, auto, globe, config, log_viewer, tools, helpers
+- **tools 包拆分**: 原 `tool_registry.py` 拆分为 17 个模块 (14 个 handler + 3 个基础设施: _registry/_dispatch/_helpers)
 - **tuple 替代 frozenset**: 避免跨机 PYTHONHASHSEED 差异
 - **PYTHONUTF8=1**: 跨机编码一致性 (run.bat 内置)
 - **类型安全聚合**: 聚合前检查 dtype, 非数值列 cast
@@ -215,7 +254,7 @@ ip_to_subnet("192.168.1.42", 28)  # → "192.168.1.32/28"
 - **UMAP 降维**: extract_features 中为实体数据计算 UMAP-2D 嵌入, 存储到 EntityProfile 表 (embedding_x/embedding_y), 训练 10K 样本 + 批量变换 1K 批次
 - **TruncatedSVD**: 聚类时特征 > 50 维自动降维; compute_scores 中学习自适应评分权重
 - **LLM 驱动距离计算**: compute_distance_matrix 工具允许 LLM 编写 Python 函数在受限命名空间 (math, numpy) 中逐行执行
- **简单分析模块**: `simple_analysis/` 独立 Django app, 提供简化的上传→筛选→Leaflet 地图工作流, 零依赖外部地图服务 (OSM 瓦片离线缓存)
+- **距离计算多策略**: tools/distance_matrix.py 中的 compute_distance_matrix 和 analysis/distance.py 中的 IP 子网/地理/Levenshtein/Hamming/FFT 相位距离

 ---

@@ -254,7 +293,6 @@ runtime\python\python.exe scripts\debug_db.py

 ```bash
 runtime\python\python.exe scripts\test_llm_full.py
-# 或 runtime\python\python.exe -c "from tianxuan.llm_orchestrator import run_llm_pipeline, LLMConfig; ..."
 ```

 ### 生成测试数据
@@ -302,14 +340,23 @@ runtime\python\python.exe scripts\column_survey.py --csv "data/*.csv"

 ## Operating Procedures

-1. **不要阻塞式启动后端** — Start Django backend in background (use `start /B` or threading), never block the terminal. Use `start /B runtime\python\python.exe manage.py runserver` to avoid hanging the shell.
+1. **非阻塞启动后端** — `run.bat` 使用 `start /B` 启动 Django, 打印 PID 后退出, 不阻塞终端。日志写入 `logs/tianxuan.log`。

-2. **全量测试** — After all changes, run full E2E test: generate 2000 CSVs, upload through frontend, verify complete pipeline. This validates the entire system end-to-end before considering work complete.
+2. **全量测试** — 所有改动完成后, 运行完整 E2E 测试: 生成测试 CSV, 上传前端, 验证全流水线。

-3. **更新文档和 Git** — After tests pass: update AGENTS.md, commit all changes, push, clean up temp files and running processes. Do not leave dirty state (orphan processes, temp files, uncommitted changes).
+3. **更新文档和 Git** — 测试通过后: 更新 AGENTS.md, 提交所有更改, 清理临时文件和残留进程。不留脏状态 (孤儿进程、临时文件、未提交更改)。

-4. **同步依赖** — Use `git pull` to sync latest code, then `runtime\python\python.exe -m pip install -r requirements.txt` to install any new packages. Do this before starting any new work session.
+4. **同步依赖** — 使用 `git pull` 同步最新代码, 然后 `runtime\python\python.exe -m pip install -r requirements.txt` 安装新依赖。每次新工作会话前执行。

-5. **编码前缀** — Prefix ALL terminal commands with `chcp 65001` to prevent Opencode encoding errors. PowerShell/CMD encoding issues are the #1 source of inexplicable command failures.
+5. **编码前缀** — 所有终端命令加 `chcp 65001` 避免 Opencode 编码错误。PowerShell/CMD 编码问题是命令失败的首要原因。

-6. **Python UTF-8** — When running Python, always add `set PYTHONUTF8=1` before the command. This ensures consistent Unicode handling across all machines and prevents cross-machine encoding drift.
+6. **Python UTF-8** — 运行 Python 时始终加 `set PYTHONUTF8=1`。确保跨机 Unicode 处理一致, 防止编码漂移。
+
+## 近期变更 (beta-clean 分支)
+
+- views.py (2000+ 行单体) → `analysis/views/` 包 (12 模块)
+- tool_registry.py → `analysis/tools/` 包 (17 模块)
+- 删除 `simple_analysis/` 模块 (已移至 master 分支独立维护)
+- 工具数量从 27 升至 30 (新增 export_entities_json, diagnostic toolbox 扩展)
+- 模板目录重组: 6 个分析页面在 `analysis/`, 6 个功能页面在 `tianxuan/`
+- 新增 `analysis/distance.py` (距离计算) 和 `analysis/nl_describe.py` (自然语言描述)
@@ -1,6 +1,6 @@
-# 天璇 (TianXuan) — TLS流数据分析平台
+# 天璇 (TianXuan) — TLS 流数据分析平台

-基于 Polars + scikit-learn + Three.js 的 TLS 流量分析工具。CSV 上传后支持手动分步分析（工作流构建器）、LLM 全自动分析（MCP 工具链）、聚类 3D 地球可视化，以及简易即用的一键上传→筛选→聚类→地图模块。全离线运行，内嵌 Python 3.12 运行时。
+基于 Polars + scikit-learn + Three.js 的 TLS 流量分析工具。CSV 上传后支持手动分步分析（工作流构建器）、LLM 全自动分析（MCP 工具链）、聚类 3D 地球可视化。全离线运行，内嵌 Python 3.12 运行时。

 ## 功能概述

@@ -8,13 +8,12 @@
 |------|------|
 | **CSV 上传** | 多文件拖拽、BOM 自动检测、schema 容错、ZIP 自动解压 |
 | **手动分析** | 工作流构建器：添加/排序/保存工具链，串行执行 |
-| **LLM 自动分析** | OpenAI 兼容 API，自动编排 profile→entity→cluster→feature |
+| **LLM 自动分析** | OpenAI 兼容 API，自动编排 filter→profile→entity→cluster→feature |
 | **聚类可视化** | HDBSCAN/KMeans，Silhouette/DB 评估，Canvas 散点图 + 地理分布 |
 | **3D 地球** | Three.js 流量弧线，TLS 版本着色，经纬线+国境线 |
 | **降维** | UMAP + TruncatedSVD 用于高维数据降维与可视化 |
-| **实体画像** | 聚合特征 + Z-Score 簇偏离量，自定义实体列/IP子网掩码 |
-| **简易分析** | 独立模块: 上传 CSV → 按目标筛选 → 地理/IP子网聚类 → Leaflet 地图 |
-| **MCP 协议** | 27 个 JSON-RPC 工具供外部 LLM 编排调用 |
+| **实体画像** | 聚合特征 + Z-Score 簇偏离量，自定义实体列/IP 子网掩码 |
+| **MCP 协议** | 30 个 JSON-RPC 工具供外部 LLM 编排调用 |
 | **增量更新** | update.bat + rollback.bat，用户数据/代码分离 |

 ## 技术栈
@@ -23,12 +22,12 @@
 |------|------|
 | 后端 | Django 4.2 + SQLite WAL |
 | 数据处理 | Polars 1.42 (LazyFrame + streaming) |
-| 机器学习 | scikit-learn (HDBSCAN, KMeans, StandardScaler), UMAP, TruncatedSVD |
+| 机器学习 | scikit-learn (HDBSCAN, KMeans, MiniBatchKMeans, IsolationForest), UMAP, TruncatedSVD, StandardScaler |
 | 前端 3D | Three.js (603KB, 离线, 地球贴图 1.4MB) |
-| 前端图表 | 纯 Canvas 2D / Leaflet |
+| 前端图表 | 纯 Canvas 2D (散点图/地理分布/Silhouette) |
 | 配置管理 | Pydantic + YAML |
 | LLM 协议 | MCP SDK (stdio) + OpenAI 兼容 API |
-| 运行时 | 内嵌 Python 3.12 (593MB, Win7 兼容版) |
+| 运行时 | 内嵌 Python 3.12 (~593MB) |

 ## 安装

@@ -43,53 +42,149 @@ runtime\python\python.exe manage.py migrate

 ```bat
 :: 双击启动 Web 界面 (http://127.0.0.1:8000/)
-run.bat
+run.bat          :: 非阻塞启动，打印 PID 后退出，日志写入 logs/tianxuan.log

 :: 命令行一键管道
 runtime\python\python.exe manage.py run_pipeline "data/*.csv"

-:: 简易分析模块入口: http://127.0.0.1:8000/simple/
+:: 开发模式启动
+set PYTHONUTF8=1
+runtime\python\python.exe manage.py runserver
 ```

 **Web 工作流:** 上传 CSV → 手动分析（工作流构建器）或 LLM 自动分析 → 检视聚类概览/簇详情/实体画像/3D 地球

-**CLI 管道:** 加载 CSV → 实体检测 → 聚合 → 聚类 → 特征提取 → 写入数据库
+**CLI 管道:** 加载 CSV → 类型检测 → 实体聚合 → 聚类 → 特征提取 → 写入数据库

 ## 项目结构

 ```
 天璇/
-├── analysis/              核心分析模块 (Django app)
-│   ├── data_loader.py     CSV 加载、BOM/编码检测、schema 容错
-│   ├── entity_detector.py  实体列自动检测 (关键词+唯一值比率打分)
-│   ├── entity_aggregator.py 按实体聚合特征 (含 IP 子网)
-│   ├── tool_registry.py   27 个 MCP 工具注册 + 处理函数
-│   ├── mcp_server.py      MCP stdio 服务器
-│   ├── type_classifier.py 值优先类型检测 (MAC/端口/IPv4/URL/HEX/经纬度)
-│   ├── ip_clustering.py   IP 子网聚类与转换工具
-│   ├── data_profiler.py   数据集概要 + 相关性矩阵
-│   ├── geoip.py           GeoIP 经纬度查询
-│   ├── session_store.py   线程安全内存会话存储
-│   └── views.py           Django 视图 (20+ 路由)
-├── simple_analysis/       简易分析模块 (独立上传→筛选→聚类→地图)
-├── tianxuan/              Django 项目配置 + LLM 编排器
-├── config/                配置 (config.yaml + Pydantic 加载器)
-├── templates/             页面模板 (analysis/ tianxuan/ base.html)
-├── static/                离线静态资源 (Three.js 地球贴图 国境线)
-├── runtime/python/        内嵌 Python 3.12 运行时
-├── scripts/               工具脚本 (测试数据生成/跨机诊断/增量更新)
-├── tests/                 单元 + 集成测试
-├── docs/                  工作流文档
-├── run.bat                用户启动入口
-├── update.bat             增量更新
-└── rollback.bat           更新回滚
+├── analysis/                    核心分析模块 (Django app)
+│   ├── views/                   视图包 (12 模块)
+│   │   ├── __init__.py          重导出所有视图函数
+│   │   ├── helpers.py           辅助函数 (_extract_lat, _extract_lon)
+│   │   ├── dashboard.py         首页 + 运行记录
+│   │   ├── pipeline.py          管道执行器 (_run_pipeline_worker)
+│   │   ├── clustering.py        聚类概览/详情/3D 地球数据
+│   │   ├── entity.py            实体画像页
+│   │   ├── upload.py            CSV 上传 (拖拽/多文件/进度)
+│   │   ├── manual.py            手动分析工作流构建器
+│   │   ├── auto.py              LLM 自动分析 (SSE 流式)
+│   │   ├── globe.py             3D 地球数据接口
+│   │   ├── config.py            配置编辑 + LLM 连通性测试
+│   │   ├── log_viewer.py        日志查看器
+│   │   └── tools.py             工具实验室
+│   ├── tools/                   工具包 (17 模块, 30 个 MCP 工具实现)
+│   │   ├── __init__.py          重导出所有 handler
+│   │   ├── _registry.py         工具元数据定义 (Tool schema)
+│   │   ├── _dispatch.py         工具调用分发器
+│   │   ├── _helpers.py          共享辅助 (过滤/截断/数据集决议)
+│   │   ├── load_data.py         CSV 加载 (glob/递归/ZIP/schema 容错)
+│   │   ├── profile.py           数据概要统计 + 相关性矩阵
+│   │   ├── filter.py            数据过滤 (12 操作符 + AND/OR)
+│   │   ├── preprocess.py        预处理 (StandardScaler/OneHot/填充)
+│   │   ├── clustering.py        聚类执行 (HDBSCAN/KMeans + 自动降维)
+│   │   ├── evaluate.py          聚类质量评估 (Silhouette/DB/CH)
+│   │   ├── features.py          特征提取 (Z-Score/ANOVA/UMAP)
+│   │   ├── entities.py          实体画像 + 自适应评分 (SVD 权重)
+│   │   ├── anomalies.py         异常检测 (IsolationForest) + UMAP 可视化
+│   │   ├── export.py            结果导出 (CSV/Parquet/JSON)
+│   │   ├── analysis.py          只读分析 (6 工具: 模式/时序/FFT/TLS/地理/实体)
+│   │   ├── diagnostics.py       诊断 (7 工具: 校验/分布/异常值/聚类诊断/对比/导出/修复)
+│   │   ├── data_mgmt.py         数据管理 (列表/删除/克隆)
+│   │   └── distance_matrix.py   LLM 驱动距离计算
+│   ├── tool_registry.py         旧编排层 (薄包装，重导出 tools/)
+│   ├── mcp_server.py            MCP stdio 服务器
+│   ├── data_loader.py           CSV 加载底层实现
+│   ├── data_profiler.py         列统计 + 相关性矩阵
+│   ├── data_validator.py        列校验 (缺失/异常值/IP 有效性)
+│   ├── type_classifier.py       值优先类型检测 (MAC/端口/IPv4/URL/HEX/经纬度)
+│   ├── geoip.py                 GeoIP 经纬度查询
+│   ├── ip_clustering.py         IP 子网聚类与转换
+│   ├── session_store.py         线程安全内存会话存储
+│   ├── distance.py              距离计算 (IP 子网/地理/Levenshtein/Hamming/FFT)
+│   ├── models.py                ORM: AnalysisRun, ClusterResult, EntityProfile, ClusterFeature
+│   ├── urls.py                  路由注册
+│   ├── nl_describe.py           自然语言数据描述
+│   ├── profile_util.py          画像工具函数
+│   ├── tls_ref.py               TLS 参考数据
+│   ├── db_utils.py              SQLite 工具
+│   ├── constants.py             全局常量
+│   ├── appdata.py               应用数据路径管理
+│   └── management/commands/     CLI 命令
+│       ├── start_mcp.py         MCP 服务器启动
+│       ├── run_pipeline.py      一键 CLI 管道
+│       └── import_tlsdb.py      TLS 数据库导入
+├── tianxuan/                    Django 项目配置
+│   ├── settings.py              ALLOWED_HOSTS 自动检测, LOGGING 文件+stderr
+│   ├── wsgi.py                  SQLite 启动自修复
+│   ├── urls.py                  路由汇总
+│   └── llm_orchestrator.py     LLM 编排 (策略式提示词, 自动错误恢复)
+├── config/                      配置管理
+│   ├── config.yaml              自动生成默认配置 (entity, server, data, clustering, llm)
+│   ├── loader.py                Pydantic 配置加载 + 文件修改检测
+│   └── __init__.py
+├── templates/                   页面模板 (13 个 HTML)
+│   ├── base.html                导航: 首页/上传/手动/LLM/记录/地球/配置
+│   ├── analysis/                分析页面 (6 个)
+│   │   ├── dashboard.html       首页
+│   │   ├── run_list.html        运行记录列表
+│   │   ├── run_detail.html      运行详情
+│   │   ├── cluster_overview.html 聚类概览
+│   │   ├── cluster_detail.html  簇详情
+│   │   └── entity_profile.html  实体画像
+│   └── tianxuan/                功能页面 (6 个)
+│       ├── upload.html          CSV 上传
+│       ├── manual.html          手动分析
+│       ├── auto.html            LLM 自动分析
+│       ├── globe.html           3D 地球
+│       ├── config.html          配置编辑
+│       └── log_viewer.html      日志查看
+├── static/tianxuan/             离线静态资源
+│   ├── three.min.js             Three.js 3D 引擎 (603KB)
+│   ├── earth_atmos_2048.jpg     地球贴图 (1.4MB)
+│   └── world_borders.js         177 国完整国境线 (269KB)
+├── runtime/python/              便携 Python 3.12 运行时 (~593MB)
+├── scripts/                     工具脚本
+│   ├── gen_test_data.py         测试数据生成
+│   ├── gen_complex_test.py      复杂数据生成
+│   ├── column_survey.py         CSV 列结构调查
+│   ├── diagnose_compare.py      跨机日志对比诊断
+│   └── debug_db.py              DB 持久化调试
+├── tests/                       端到端测试
+│   ├── test_e2e_full.py
+│   ├── test_e2e_batch.py
+│   └── _test_hdbscan.py
+├── docs/                        文档
+│   ├── 工作流.md                 完整分析流水线
+│   └── 故障诊断手册.md            问题排查指南
+├── run.bat                      用户启动入口 (含 PYTHONUTF8=1)
+├── shell.bat                    Django shell
+├── update.bat                   增量更新
+├── rollback.bat                 更新回滚
+├── build.bat                    构建打包
+└── manage.py                    Django 管理入口
 ```

+## 架构要点
+
+- **views 包**: 原单体 `views.py` (2000+ 行) 拆分为 12 个模块，按功能域分离 (dashboard, pipeline, clustering, entity, upload, manual, auto, globe, config, log_viewer, tools, helpers)
+- **tools 包**: 原 `tool_registry.py` 拆分为 17 个模块 (14 个 handler 模块 + 3 个基础设施模块)，每个工具领域独立文件
+- **30 个 MCP 工具** 分四层: 核心 (14 个)、只读分析 (6 个)、诊断 (7 个)、数据管理 (3 个)
+- **纯 Canvas 2D 图表**: 零外部 JS 依赖，散点图/地理分布/Silhouette 均纯 Canvas 绘制
+- **tuple 替代 frozenset**: 避免跨机 PYTHONHASHSEED 差异
+- **值优先类型检测**: config → 值采样 → 名称推断 → STRING 回退 (type_classifier.py)
+- **TruncatedSVD 降维**: 特征 >50 维时自动降维至 50；compute_scores 中学习自适应评分权重
+- **UMAP 2D 嵌入**: 训练 10K 样本 + 批量变换 1K 批次，存入 EntityProfile 表
+- **上传目录隔离**: `%APPDATA%/TianXuan/data/uploads/`，与项目路径无关
+
 ## 目标设备

 | 条件 | 规格 |
 |------|------|
-| 操作系统 | Windows 10 1902+ |
-| 内存 | 4 GB |
+| 操作系统 | Windows 10 1909+ |
+| 内存 | 8 GB |
 | GPU | 无独立显卡（核显可运行 Three.js） |
 | 磁盘 | SSD 推荐（SQLite WAL 优化） |
+| 数据规模 | 支持 2500 个 CSV × 10000 行 × 50~58 列 |
@@ -0,0 +1,96 @@
+"""Centralized constants for the TianXuan analysis module.
+
+All magic numbers from analysis/tools/ and analysis/views/ are defined here
+so they can be updated in one place. Import with::
+
+    from analysis.constants import RANDOM_SEED, MAX_SAMPLE_ROWS, ...
+"""
+
+import polars as pl
+
+# ---------------------------------------------------------------------------
+#  Clustering / UMAP
+# ---------------------------------------------------------------------------
+
+MAX_SAMPLE_ROWS = 50000
+"""Maximum rows for clustering sample (downsampling threshold)."""
+
+MAX_DISTRIBUTION_SAMPLE = 10000
+"""Default sample size for distribution exploration and scoring."""
+
+UMAP_TRAIN_SAMPLE = 10000
+"""Max rows used to train a UMAP reducer in extract_features."""
+
+UMAP_BATCH_SIZE = 1000
+"""Batch size for UMAP transform when dataset exceeds UMAP_TRAIN_SAMPLE."""
+
+LOW_MEMORY_THRESHOLD_GB = 2
+"""If available memory < this (in GiB), enable low-memory clustering path."""
+
+LOW_MEMORY_THRESHOLD_ROWS = 100000
+"""Row count above which low-memory downsampling is triggered."""
+
+RANDOM_SEED = 42
+"""Default random seed for all sklearn/numpy operations."""
+
+# ---------------------------------------------------------------------------
+#  Globe
+# ---------------------------------------------------------------------------
+
+GLOBE_MAX_ROWS = 300
+"""Max rows to load per run for 3D globe visualization."""
+
+# ---------------------------------------------------------------------------
+#  Filter
+# ---------------------------------------------------------------------------
+
+FILTER_MAX_ROWS = 5
+"""Maximum visible filter rows in the manual analysis UI (deprecated / reserved)."""
+
+MAX_COLUMNS_PROFILE = 200
+"""Max columns to include in a dataset profile summary."""
+
+# ---------------------------------------------------------------------------
+#  File upload limits
+# ---------------------------------------------------------------------------
+
+FILE_UPLOAD_MAX_SIZE = 500 * 1024 * 1024  # 500 MB
+"""Maximum size for a single uploaded file."""
+
+TOTAL_UPLOAD_MAX_SIZE = 50 * 1024 * 1024 * 1024  # 50 GB
+"""Maximum total upload size across all files."""
+
+# ---------------------------------------------------------------------------
+#  Tool limits (numeric type tuples used across tools/)
+# ---------------------------------------------------------------------------
+
+NUMERIC_DTYPES: tuple[type[pl.DataType], ...] = (
+    pl.Int8, pl.Int16, pl.Int32, pl.Int64,
+    pl.UInt8, pl.UInt16, pl.UInt32, pl.UInt64,
+    pl.Float32, pl.Float64,
+)
+"""Polars numeric data types — used for column selection in clustering,
+entity aggregation, anomaly detection, and feature extraction."""
+
+# ---------------------------------------------------------------------------
+#  Diagnostics / Validation
+# ---------------------------------------------------------------------------
+
+MAX_VALIDATE_SAMPLE = 1000
+"""Max rows sampled during validate_data."""
+
+MAX_DIAGNOSE_SAMPLE = 5000
+"""Max rows sampled for clustering diagnosis (SVD)."""
+
+MAX_CLUSTER_FEATURES = 50
+"""Feature count threshold above which TruncatedSVD is applied."""
+
+CLUSTER_MAX_FEATURES_TOP = 10
+"""Max feature columns selected by auto-detection in clustering tools."""
+
+# ---------------------------------------------------------------------------
+#  Globe / Flow extraction
+# ---------------------------------------------------------------------------
+
+GLOBE_FLOW_MAX_ROWS = 300
+"""Alias for GLOBE_MAX_ROWS — max rows loaded for flow extraction."""
@@ -1,14 +1,18 @@
 """
-离线 GeoIP 查询。内嵌 IPv4 区间 → 城市/经纬度映射。
+离线 GeoIP 查询。纯本地运行，无网络请求。

-从 ``data/geoip_data.txt`` 加载 IP 段数据，在模块首次导入时构建
-排序索引，提供 O(log N) 的 IP 查询能力。
+三级存储：
+1. 快速缓存 (data/geoip_cache.json) — 已查过的 IP，磁盘持久化
+2. 离线数据库 (data/geoip_data.txt) — 手动整理的优先 IP 段
+3. MMDB 数据库 — 若 python_geoip_geolite2 包已安装，加载其
+   内置的 GeoLite2-City.mmdb 作为最全面回退。

-数据约 200KB，覆盖 23 个主要城市 IP 段。
-可通过编辑 ``data/geoip_data.txt`` 扩展。
+查询链：缓存 > geoip_data.txt > MMDB > 写入缓存。
 """
 import bisect
+import json
 import logging
+import os
 from pathlib import Path
 from typing import Optional

@@ -18,32 +22,22 @@ logger = logging.getLogger(__name__)
 #  Data structures
 # ---------------------------------------------------------------------------

-# [(start_int, end_int, lat, lon, city, country), ...]
-# Built at module load time from data/geoip_data.txt
 _GEO_DATA: list[tuple[int, int, float, float, str, str]] = []
-
-# Sorted list of start integers (for bisect lookup) + parallel data list
 _STARTS: list[int] = []
 _RANGES: list[tuple[int, int, float, float, str, str]] = []

+_MMDB_READER = None
+
+_CACHE: dict[str, Optional[dict]] = {}
+_CACHE_DIRTY = False
+_CACHE_PATH = Path(__file__).resolve().parent.parent / 'data' / 'geoip_cache.json'
+

 # ---------------------------------------------------------------------------
 #  IP conversion
 # ---------------------------------------------------------------------------

 def ip_to_int(ip_str: str) -> Optional[int]:
-    """Convert an IPv4 dotted-string to its 32-bit integer representation.
-
-    Returns ``None`` if *ip_str* is not a valid IPv4 address (including
-    when any octet is outside 0-255).
-
-    Examples
-    --------
-    >>> ip_to_int('8.8.8.8')
-    134744072
-    >>> ip_to_int('1.0.0.0')
-    16777216
-    """
    parts = ip_str.strip().split('.')
    if len(parts) != 4:
        return None
@@ -57,78 +51,144 @@ def ip_to_int(ip_str: str) -> Optional[int]:


 # ---------------------------------------------------------------------------
-#  Index building
+#  geoip_data.txt loader
 # ---------------------------------------------------------------------------

-def _build_index() -> tuple[list[int], list[tuple[int, int, float, float, str, str]]]:
-    """Build sorted parallel lists for bisect-based IP lookup.
-
-    Processes ``_GEO_DATA``, sorts by start integer, and returns
-    ``(starts, ranges)`` where *starts* is a sorted list of start integers
-    and *ranges* is the corresponding data list at the same indices.
-    """
+def _build_index():
    if not _GEO_DATA:
        return [], []
-
    sorted_data = sorted(_GEO_DATA, key=lambda x: x[0])
    starts = [entry[0] for entry in sorted_data]
    return starts, sorted_data


 def _load_data_file() -> None:
-    """Load IP range data from ``data/geoip_data.txt`` into ``_GEO_DATA``.
-
-    Called once at module import time. Each non-blank, non-comment line
-    in the file should contain six comma-separated fields::
-
-        start_ip,end_ip,lat,lon,city,country
-    """
-    data_path = Path(__file__).parent.parent / 'data' / 'geoip_data.txt'
-
+    data_path = Path(__file__).resolve().parent.parent / 'data' / 'geoip_data.txt'
    if not data_path.exists():
        logger.warning('[GEOIP] data file not found: %s', data_path)
        return
-
-    loaded: list[tuple[int, int, float, float, str, str]] = []
+    loaded = []
    skipped = 0
-
    with open(data_path, 'r', encoding='utf-8') as f:
-        for line_no, raw in enumerate(f, start=1):
+        for raw in f:
            stripped = raw.strip()
            if not stripped or stripped.startswith('#'):
                continue
-
            parts = [p.strip() for p in stripped.split(',')]
            if len(parts) != 6:
                skipped += 1
                continue
-
-            start_ip_str, end_ip_str, lat_str, lon_str, city, country = parts
-
-            start_int = ip_to_int(start_ip_str)
-            end_int = ip_to_int(end_ip_str)
-
-            if start_int is None or end_int is None:
+            sip, eip, lat_s, lon_s, city, country = parts
+            si = ip_to_int(sip)
+            ei = ip_to_int(eip)
+            if si is None or ei is None or si > ei:
                skipped += 1
                continue
-
            try:
-                lat = float(lat_str)
-                lon = float(lon_str)
+                lat = float(lat_s)
+                lon = float(lon_s)
            except ValueError:
                skipped += 1
                continue
-
-            loaded.append((start_int, end_int, lat, lon, city, country))
-
+            loaded.append((si, ei, lat, lon, city, country))
    _GEO_DATA.clear()
    _GEO_DATA.extend(loaded)
-
    global _STARTS, _RANGES
    _STARTS, _RANGES = _build_index()
+    logger.info('[GEOIP] loaded %d ranges (%d skipped)', len(loaded), skipped)

-    logger.info('[GEOIP] loaded %d ranges from %s (%d skipped)',
-                len(loaded), data_path, skipped)
+
+# ---------------------------------------------------------------------------
+#  MMDB reader (GeoLite2-City.mmdb bundled with python_geoip_geolite2)
+# ---------------------------------------------------------------------------
+
+def _get_mmdb_reader():
+    global _MMDB_READER
+    if _MMDB_READER is not None:
+        return _MMDB_READER
+    # Try multiple locations for the MMDB file
+    candidates = []
+    try:
+        import _geoip_geolite2
+        candidates.append(Path(_geoip_geolite2.__file__).parent / 'GeoLite2-City.mmdb')
+    except Exception:
+        pass
+    # Fallback: temp directory without Chinese chars in path
+    candidates.append(Path(os.environ.get('TEMP', 'C:/temp')) / 'opencode' / 'GeoLite2-City.mmdb')
+    candidates.append(Path.home() / 'AppData' / 'Local' / 'Temp' / 'opencode' / 'GeoLite2-City.mmdb')
+
+    for mmdb_path in candidates:
+        if mmdb_path.exists():
+            try:
+                import maxminddb
+                _MMDB_READER = maxminddb.open_database(mmdb_path)
+                logger.info('[GEOIP] loaded MMDB: %s (%.1f MB)',
+                            mmdb_path.name, mmdb_path.stat().st_size / 1e6)
+                break
+            except Exception as exc:
+                logger.debug('[GEOIP] failed to open %s: %s', mmdb_path, exc)
+                continue
+
+    if _MMDB_READER is None:
+        logger.debug('[GEOIP] MMDB not available — falling back to txt database only')
+    return _MMDB_READER
+
+
+def _mmdb_lookup(ip_str: str) -> Optional[dict]:
+    reader = _get_mmdb_reader()
+    if reader is None:
+        return None
+    try:
+        result = reader.get(ip_str)
+        if result is None:
+            return None
+        location = result.get('location', {})
+        city_data = result.get('city', {})
+        country_data = result.get('country', {})
+        lat = location.get('latitude')
+        lon = location.get('longitude')
+        if lat is None or lon is None:
+            return None
+        city = ''
+        if city_data and 'names' in city_data:
+            city = city_data['names'].get('zh-CN') or city_data['names'].get('en', '')
+        country = ''
+        if country_data and 'names' in country_data:
+            country = country_data['names'].get('zh-CN') or country_data['names'].get('en', '')
+        return {'lat': float(lat), 'lon': float(lon), 'city': city, 'country': country}
+    except Exception as exc:
+        logger.debug('[GEOIP] MMDB lookup failed for %s: %s', ip_str, exc)
+        return None
+
+
+# ---------------------------------------------------------------------------
+#  Disk cache
+# ---------------------------------------------------------------------------
+
+def _load_cache() -> None:
+    if not _CACHE_PATH.exists():
+        return
+    try:
+        data = json.loads(_CACHE_PATH.read_text(encoding='utf-8'))
+        _CACHE.update(data)
+        logger.info('[GEOIP] loaded %d cached entries', len(data))
+    except Exception:
+        pass
+
+
+def _save_cache() -> None:
+    global _CACHE_DIRTY
+    if not _CACHE_DIRTY:
+        return
+    try:
+        _CACHE_PATH.parent.mkdir(parents=True, exist_ok=True)
+        _CACHE_PATH.write_text(
+            json.dumps(_CACHE, ensure_ascii=False, indent=2),
+            encoding='utf-8',
+        )
+        _CACHE_DIRTY = False
+    except Exception as exc:
+        logger.warning('[GEOIP] cache write failed: %s', exc)


 # ---------------------------------------------------------------------------
@@ -136,55 +196,44 @@ def _load_data_file() -> None:
 # ---------------------------------------------------------------------------

 def lookup(ip_str: str) -> Optional[dict]:
-    """查询 IP 对应的地理位置。
+    """查询 IP 地理位置。纯本地，无网络。

-    Uses binary search (``bisect_right``) on the sorted start-IP list to
-    locate the range *ip_str* falls within, then checks the end bound.
-
-    Parameters
-    ----------
-    ip_str:
-        IPv4 address as a dotted string (e.g. ``'8.8.8.8'``).
-
-    Returns
-    -------
-    dict or None
-        A dictionary with keys ``lat``, ``lon``, ``city``, ``country``
-        if the IP is found in the database; ``None`` otherwise.
-
-    Examples
-    --------
-    >>> result = lookup('8.8.8.8')
-    >>> result['city']
-    'Mountain View'
-    >>> lookup('10.0.0.1') is None
-    True
+    查询链：磁盘缓存 > geoip_data.txt (O(log N)) > MMDB (GeoLite2) > 写入缓存。
    """
+    # 1. Cache hit
+    cached = _CACHE.get(ip_str)
+    if cached is not None:
+        return cached
+    if ip_str in _CACHE:
+        return None
+
+    result = None
+
+    # 2. geoip_data.txt
    target = ip_to_int(ip_str)
-    if target is None:
-        return None
+    if target is not None and _STARTS:
+        idx = bisect.bisect_right(_STARTS, target) - 1
+        if idx >= 0:
+            start, end, lat, lon, city, country = _RANGES[idx]
+            if start <= target <= end:
+                result = {'lat': lat, 'lon': lon, 'city': city, 'country': country}

-    if not _STARTS:
-        return None
+    # 3. MMDB fallback (GeoLite2, ~3M entries world-wide)
+    if result is None:
+        result = _mmdb_lookup(ip_str)

-    idx = bisect.bisect_right(_STARTS, target) - 1
-    if idx < 0:
-        return None
+    # 4. Cache for next time
+    global _CACHE_DIRTY
+    _CACHE[ip_str] = result
+    _CACHE_DIRTY = True
+    _save_cache()

-    start, end, lat, lon, city, country = _RANGES[idx]
-    if start <= target <= end:
-        return {
-            'lat': lat,
-            'lon': lon,
-            'city': city,
-            'country': country,
-        }
-
-    return None
+    return result


 # ---------------------------------------------------------------------------
-#  Module-level initialisation
+#  Module-level init
 # ---------------------------------------------------------------------------

 _load_data_file()
+_load_cache()
@@ -1,11 +1,19 @@
-# Generated by Django 4.2.30 on 2026-07-13 14:36
+# Generated by Django 4.2.30 on 2026-07-23 11:15

 from django.db import migrations, models
+from django.db.models import F
+import django.db.migrations.operations.special
 import django.db.models.deletion


+def populate_display_id(apps, schema_editor):
+    AnalysisRun = apps.get_model('tianxuan_analysis', 'AnalysisRun')
+    AnalysisRun.objects.filter(display_id__isnull=True).update(display_id=F('id'))
+
 class Migration(migrations.Migration):

+    replaces = [('tianxuan_analysis', '0001_initial'), ('tianxuan_analysis', '0002_analysisrun_progress_msg_analysisrun_progress_pct_and_more'), ('tianxuan_analysis', '0003_analysisrun_sqlite_table'), ('tianxuan_analysis', '0004_tls_ref_data'), ('tianxuan_analysis', '0005_remove_entity_column'), ('tianxuan_analysis', '0006_analysisrun_display_id_analysisrun_llm_thinking_and_more'), ('tianxuan_analysis', '0007_populate_display_id'), ('tianxuan_analysis', '0008_entityprofile_embedding_z_and_more')]
+
    initial = True

    dependencies = [
@@ -26,6 +34,10 @@ class Migration(migrations.Migration):
                ('entity_count', models.IntegerField(blank=True, null=True)),
                ('cluster_count', models.IntegerField(blank=True, null=True)),
                ('entity_column', models.CharField(blank=True, default='', help_text='Auto-detected entity column name', max_length=256)),
+                ('progress_msg', models.CharField(blank=True, default='', max_length=256)),
+                ('progress_pct', models.IntegerField(default=0)),
+                ('run_log', models.TextField(blank=True, default='')),
+                ('sqlite_table', models.CharField(blank=True, default='', help_text='SQLite table name for persisted raw data', max_length=256)),
            ],
            options={
                'ordering': ['-created_at'],
@@ -81,4 +93,51 @@ class Migration(migrations.Migration):
                'unique_together': {('cluster', 'feature_name')},
            },
        ),
+        migrations.RunSQL(
+            sql="\nCREATE TABLE IF NOT EXISTS tls_ref_data (\n    id          INTEGER PRIMARY KEY AUTOINCREMENT,\n    field_code  TEXT    NOT NULL UNIQUE,\n    meaning     TEXT    NOT NULL DEFAULT '',\n    data_type   TEXT    NOT NULL DEFAULT ''\n);\n",
+            reverse_sql='\nDROP TABLE IF EXISTS tls_ref_data;\n',
+        ),
+        migrations.RemoveField(
+            model_name='analysisrun',
+            name='entity_column',
+        ),
+        migrations.AddField(
+            model_name='analysisrun',
+            name='display_id',
+            field=models.IntegerField(blank=True, help_text='User-facing numeric ID (recycles gaps)', null=True, unique=True),
+        ),
+        migrations.AddField(
+            model_name='analysisrun',
+            name='llm_thinking',
+            field=models.TextField(blank=True, default='', help_text='Accumulated LLM reasoning / thinking text'),
+        ),
+        migrations.AddField(
+            model_name='analysisrun',
+            name='run_type',
+            field=models.CharField(choices=[('upload', '手动上传'), ('manual', '手动分析'), ('auto', 'LLM自动')], default='upload', max_length=16),
+        ),
+        migrations.AddField(
+            model_name='analysisrun',
+            name='tool_calls_json',
+            field=models.JSONField(blank=True, default=list, help_text='Structured tool call list: [{step, name, input, output}, ...]'),
+        ),
+        migrations.RunPython(
+            code=populate_display_id,
+            reverse_code=django.db.migrations.operations.special.RunPython.noop,
+        ),
+        migrations.AddField(
+            model_name='entityprofile',
+            name='embedding_z',
+            field=models.FloatField(blank=True, help_text='UMAP-3D Z coordinate', null=True),
+        ),
+        migrations.AlterField(
+            model_name='entityprofile',
+            name='embedding_x',
+            field=models.FloatField(blank=True, help_text='UMAP-2D X coordinate', null=True),
+        ),
+        migrations.AlterField(
+            model_name='entityprofile',
+            name='embedding_y',
+            field=models.FloatField(blank=True, help_text='UMAP-2D Y coordinate', null=True),
+        ),
    ]
@@ -1,28 +0,0 @@
-# Generated by Django 4.2.30 on 2026-07-16 05:11
-
-from django.db import migrations, models
-
-
-class Migration(migrations.Migration):
-
-    dependencies = [
-        ('tianxuan_analysis', '0001_initial'),
-    ]
-
-    operations = [
-        migrations.AddField(
-            model_name='analysisrun',
-            name='progress_msg',
-            field=models.CharField(blank=True, default='', max_length=256),
-        ),
-        migrations.AddField(
-            model_name='analysisrun',
-            name='progress_pct',
-            field=models.IntegerField(default=0),
-        ),
-        migrations.AddField(
-            model_name='analysisrun',
-            name='run_log',
-            field=models.TextField(blank=True, default=''),
-        ),
-    ]
@@ -1,18 +0,0 @@
-# Generated by Django 4.2.30 on 2026-07-20 05:11
-
-from django.db import migrations, models
-
-
-class Migration(migrations.Migration):
-
-    dependencies = [
-        ('tianxuan_analysis', '0002_analysisrun_progress_msg_analysisrun_progress_pct_and_more'),
-    ]
-
-    operations = [
-        migrations.AddField(
-            model_name='analysisrun',
-            name='sqlite_table',
-            field=models.CharField(blank=True, default='', help_text='SQLite table name for persisted raw data', max_length=256),
-        ),
-    ]
@@ -1,38 +0,0 @@
-"""Migration 0004 — create raw SQLite table ``tls_ref_data``.
-
-Replaces the ``TlsDB.csv`` reference file with a proper SQLite table.
-The table stores TLS field-code metadata: human-readable meaning and
-data type for each field code used in TLS flow data.
-"""
-
-from django.db import migrations
-
-# Raw SQL for creating the tls_ref_data table
-CREATE_TLS_REF_DATA = """
-CREATE TABLE IF NOT EXISTS tls_ref_data (
-    id          INTEGER PRIMARY KEY AUTOINCREMENT,
-    field_code  TEXT    NOT NULL UNIQUE,
-    meaning     TEXT    NOT NULL DEFAULT '',
-    data_type   TEXT    NOT NULL DEFAULT ''
-);
-"""
-
-DROP_TLS_REF_DATA = """
-DROP TABLE IF EXISTS tls_ref_data;
-"""
-
-
-class Migration(migrations.Migration):
-    """Create ``tls_ref_data`` table via raw SQL — no Django model needed."""
-
-    dependencies = [
-        ('tianxuan_analysis', '0003_analysisrun_sqlite_table'),
-    ]
-
-    operations = [
-        migrations.RunSQL(
-            sql=CREATE_TLS_REF_DATA,
-            reverse_sql=DROP_TLS_REF_DATA,
-            state_operations=None,
-        ),
-    ]
@@ -1,17 +0,0 @@
-# Generated by Django 4.2.30 on 2026-07-20 05:20
-
-from django.db import migrations
-
-
-class Migration(migrations.Migration):
-
-    dependencies = [
-        ('tianxuan_analysis', '0004_tls_ref_data'),
-    ]
-
-    operations = [
-        migrations.RemoveField(
-            model_name='analysisrun',
-            name='entity_column',
-        ),
-    ]
@@ -1,33 +0,0 @@
-# Generated by Django 4.2.30 on 2026-07-20 05:25
-
-from django.db import migrations, models
-
-
-class Migration(migrations.Migration):
-
-    dependencies = [
-        ('tianxuan_analysis', '0005_remove_entity_column'),
-    ]
-
-    operations = [
-        migrations.AddField(
-            model_name='analysisrun',
-            name='display_id',
-            field=models.IntegerField(blank=True, help_text='User-facing numeric ID (recycles gaps)', null=True, unique=True),
-        ),
-        migrations.AddField(
-            model_name='analysisrun',
-            name='llm_thinking',
-            field=models.TextField(blank=True, default='', help_text='Accumulated LLM reasoning / thinking text'),
-        ),
-        migrations.AddField(
-            model_name='analysisrun',
-            name='run_type',
-            field=models.CharField(choices=[('upload', '手动上传'), ('manual', '手动分析'), ('auto', 'LLM自动')], default='upload', max_length=16),
-        ),
-        migrations.AddField(
-            model_name='analysisrun',
-            name='tool_calls_json',
-            field=models.JSONField(blank=True, default=list, help_text='Structured tool call list: [{step, name, input, output}, ...]'),
-        ),
-    ]
@@ -1,20 +0,0 @@
-# Generated by Django 4.2.30 on 2026-07-20 05:26
-
-from django.db import migrations
-from django.db.models import F
-
-
-def populate_display_id(apps, schema_editor):
-    AnalysisRun = apps.get_model('tianxuan_analysis', 'AnalysisRun')
-    AnalysisRun.objects.filter(display_id__isnull=True).update(display_id=F('id'))
-
-
-class Migration(migrations.Migration):
-
-    dependencies = [
-        ('tianxuan_analysis', '0006_analysisrun_display_id_analysisrun_llm_thinking_and_more'),
-    ]
-
-    operations = [
-        migrations.RunPython(populate_display_id, migrations.RunPython.noop),
-    ]
@@ -1,28 +0,0 @@
-# Generated by Django 4.2.30 on 2026-07-23 04:21
-
-from django.db import migrations, models
-
-
-class Migration(migrations.Migration):
-
-    dependencies = [
-        ('tianxuan_analysis', '0007_populate_display_id'),
-    ]
-
-    operations = [
-        migrations.AddField(
-            model_name='entityprofile',
-            name='embedding_z',
-            field=models.FloatField(blank=True, help_text='UMAP-3D Z coordinate', null=True),
-        ),
-        migrations.AlterField(
-            model_name='entityprofile',
-            name='embedding_x',
-            field=models.FloatField(blank=True, help_text='UMAP-2D X coordinate', null=True),
-        ),
-        migrations.AlterField(
-            model_name='entityprofile',
-            name='embedding_y',
-            field=models.FloatField(blank=True, help_text='UMAP-2D Y coordinate', null=True),
-        ),
-    ]
@@ -0,0 +1,91 @@
+"""Line profiling utility for the TianXuan backend.
+
+Usage in any backend module::
+
+    from analysis.profile_util import profile
+
+    @profile
+    def my_function(...):
+        ...
+
+When ``TIANXUAN_PROFILE=1`` env var is set AND line_profiler is installed,
+the decorator activates real line-level profiling and prints stats at exit.
+Otherwise the decorator is a no-op.
+
+To auto-profile ALL public functions in a module automatically, add at the
+bottom of the file::
+
+    from analysis.profile_util import auto_profile_module
+    auto_profile_module(__name__)
+
+Auto-profiling is also gated by ``TIANXUAN_PROFILE=1``.
+"""
+import os
+import sys
+import logging
+import atexit
+
+logger = logging.getLogger(__name__)
+
+_PROFILER_ACTIVE = os.environ.get('TIANXUAN_PROFILE', '') == '1'
+
+if _PROFILER_ACTIVE:
+    try:
+        from line_profiler import profile as _line_profile
+    except ImportError:
+        logger.warning('TIANXUAN_PROFILE=1 but line_profiler not installed')
+        _PROFILER_ACTIVE = False
+
+
+def profile(func):
+    """Decorate a function for line-level profiling.
+
+    Active only when ``TIANXUAN_PROFILE=1`` env var is set and
+    line_profiler is installed.
+    """
+    if _PROFILER_ACTIVE:
+        return _line_profile(func)
+    return func
+
+
+def auto_profile_module(module_name: str) -> int:
+    """Apply @profile to every public (non-underscore) function in *module_name*.
+
+    Must be called at module level, typically at the bottom of the file::
+
+        from analysis.profile_util import auto_profile_module
+        auto_profile_module(__name__)
+
+    Active only when ``TIANXUAN_PROFILE=1`` env var is set and
+    line_profiler is installed.  Returns number of functions wrapped, or 0
+    when inactive.
+    """
+    if not _PROFILER_ACTIVE:
+        return 0
+
+    try:
+        from line_profiler import LineProfiler
+    except ImportError:
+        return 0
+
+    module = sys.modules.get(module_name)
+    if module is None:
+        logger.warning('auto_profile_module: module %r not found', module_name)
+        return 0
+
+    lp = LineProfiler()
+    count = 0
+    for name in dir(module):
+        if name.startswith('_'):
+            continue
+        obj = getattr(module, name)
+        if callable(obj):
+            try:
+                lp.add_function(obj)
+                count += 1
+            except (TypeError, ValueError):
+                pass  # skip non-Python or special objects
+
+    atexit.register(lambda: lp.print_stats(output_unit=1e-6))
+    logger.info('auto_profile: wrapped %d functions in %s', count, module_name)
+    return count
@@ -323,12 +323,20 @@ class SessionStore:
    def drop_dataset(self, dataset_id: str) -> bool:
        """Remove a dataset from the store and force garbage collection.

+        Cascades to child datasets (e.g. filtered results) via parent_id tracking.
        Returns True if the dataset existed.
        """
        with self._datalock:
            existed = dataset_id in self._stores and self._stores[dataset_id].get('type') == 'dataset'
            if existed:
                del self._stores[dataset_id]
+            # Cascade: drop all child datasets that reference the dropped one as parent
+            child_ids = [
+                k for k, v in self._stores.items()
+                if v.get('type') == 'dataset' and v.get('parent_id') == dataset_id
+            ]
+            for cid in child_ids:
+                del self._stores[cid]
        if existed:
            gc.collect()
            self._save_to_disk()
@@ -0,0 +1,62 @@
+"""Tool registry package — split from analysis/tool_registry.py.
+
+Re-exports all tool metadata, dispatcher, and handlers.
+"""
+
+# ── Helpers ────────────────────────────────────────────────────────────────
+from ._helpers import (
+    _json_size,
+    _truncate_response,
+    _build_filter_expr,
+    _resolve_dataset,
+    _count_rows,
+    _get_numeric_columns,
+    NUMERIC_DTYPES,
+    NUMERIC_TYPE_NAMES,
+)
+
+# ── Registry + dispatch ────────────────────────────────────────────────────
+from ._registry import get_tools_meta
+from ._dispatch import handle_call
+
+# ── Handlers (tool implementations) ────────────────────────────────────────
+from .load_data import _handle_load_data
+from .profile import _handle_profile_data
+from .filter import _handle_filter_data
+from .preprocess import _handle_preprocess_data
+from .clustering import _handle_run_clustering, _handle_filter_and_cluster
+from .evaluate import _handle_evaluate_clustering
+from .features import _handle_extract_features, _save_features_to_db
+from .export import _handle_export_results
+from .entities import (
+    _handle_build_entity_profiles,
+    _handle_compute_scores,
+    _add_adaptive_scores,
+    _ENTITY_KEYWORDS,
+    _NUMERIC_TYPES_FOR_AGG,
+    _ANOMALY_FEATURE_COLS,
+)
+from .anomalies import _handle_detect_anomalies, _handle_visualize_anomalies
+from .diagnostics import (
+    _handle_validate_data,
+    _handle_explore_distributions,
+    _handle_find_outliers,
+    _handle_diagnose_clustering,
+    _handle_compare_datasets,
+    _handle_export_debug_sample,
+    _handle_repair_schema,
+)
+from .analysis import (
+    _handle_analyze_patterns,
+    _handle_analyze_temporal,
+    _handle_analyze_fft,
+    _handle_analyze_tls_health,
+    _handle_analyze_geo_distribution,
+    _handle_analyze_entity_detail,
+)
+from .data_mgmt import (
+    _handle_list_datasets,
+    _handle_drop_dataset,
+    _handle_clone_dataset,
+)
+from .distance_matrix import _handle_compute_distance_matrix
@@ -0,0 +1,79 @@
+"""Tool call dispatcher — maps tool names to handler functions."""
+from ._helpers import _truncate_response
+from .load_data import _handle_load_data
+from .profile import _handle_profile_data
+from .filter import _handle_filter_data
+from .preprocess import _handle_preprocess_data
+from .clustering import _handle_run_clustering, _handle_filter_and_cluster
+from .evaluate import _handle_evaluate_clustering
+from .features import _handle_extract_features
+from .export import _handle_export_results
+from .entities import _handle_build_entity_profiles, _handle_compute_scores
+from .anomalies import _handle_detect_anomalies, _handle_visualize_anomalies
+from .diagnostics import (
+    _handle_validate_data, _handle_explore_distributions, _handle_find_outliers,
+    _handle_diagnose_clustering, _handle_compare_datasets,
+    _handle_export_debug_sample, _handle_repair_schema,
+)
+from .analysis import (
+    _handle_analyze_patterns, _handle_analyze_temporal, _handle_analyze_fft,
+    _handle_analyze_tls_health, _handle_analyze_geo_distribution,
+    _handle_analyze_entity_detail,
+)
+from .data_mgmt import _handle_list_datasets, _handle_drop_dataset, _handle_clone_dataset
+from .distance_matrix import _handle_compute_distance_matrix
+
+
+async def handle_call(name: str, arguments: dict) -> dict:
+    """Dispatch a tool call to the appropriate handler.
+
+    Args:
+        name: Tool name (must match one of the 30 tools).
+        arguments: Tool-specific parameters.
+
+    Returns:
+        A JSON-serialisable dict.
+
+    Raises:
+        ValueError: If *name* is not a recognised tool.
+    """
+    _handlers = {
+        'load_data': _handle_load_data,
+        'profile_data': _handle_profile_data,
+        'filter_data': _handle_filter_data,
+        'preprocess_data': _handle_preprocess_data,
+        'run_clustering': _handle_run_clustering,
+        'evaluate_clustering': _handle_evaluate_clustering,
+        'extract_features': _handle_extract_features,
+        'export_results': _handle_export_results,
+        'list_datasets': _handle_list_datasets,
+        'drop_dataset': _handle_drop_dataset,
+        'clone_dataset': _handle_clone_dataset,
+        'build_entity_profiles': _handle_build_entity_profiles,
+        'compute_scores': _handle_compute_scores,
+        'filter_and_cluster': _handle_filter_and_cluster,
+        'detect_anomalies': _handle_detect_anomalies,
+        'visualize_anomalies': _handle_visualize_anomalies,
+        'validate_data': _handle_validate_data,
+        'explore_distributions': _handle_explore_distributions,
+        'find_outliers': _handle_find_outliers,
+        'diagnose_clustering': _handle_diagnose_clustering,
+        'compare_datasets': _handle_compare_datasets,
+        'export_debug_sample': _handle_export_debug_sample,
+        'repair_schema': _handle_repair_schema,
+        'analyze_patterns': _handle_analyze_patterns,
+        'analyze_temporal': _handle_analyze_temporal,
+        'analyze_fft': _handle_analyze_fft,
+        'analyze_tls_health': _handle_analyze_tls_health,
+        'analyze_geo_distribution': _handle_analyze_geo_distribution,
+        'analyze_entity_detail': _handle_analyze_entity_detail,
+        'compute_distance_matrix': _handle_compute_distance_matrix,
+    }
+    handler = _handlers.get(name)
+    if handler is None:
+        raise ValueError(f"Unknown tool: '{name}'")
+    result = await handler(**arguments)
+    # Always ensure truncated key exists
+    if 'truncated' not in result:
+        result['truncated'] = False
+    return _truncate_response(result)
@@ -0,0 +1,138 @@
+"""Shared utilities for tool handlers.
+
+Provides:
+- NUMERIC_DTYPES / NUMERIC_TYPE_NAMES: shared constants for dtype filtering
+- _json_size, _truncate_response: LLM context-window helpers
+- _build_filter_expr: Polars filter expression builder
+- _resolve_dataset: one-liner SessionStore lookup (replaces 30+ repeated blocks)
+- _count_rows: fast streaming row count
+- _get_numeric_columns: extract numeric column names from schema
+"""
+import json
+import logging
+import traceback
+from typing import Any
+
+import polars as pl
+
+from ..session_store import SessionStore
+logger = logging.getLogger(__name__)
+
+
+# ═══════════════════════════════════════════════════════════════════════
+#  Shared constants
+# ═══════════════════════════════════════════════════════════════════════
+
+NUMERIC_DTYPES = (
+    pl.Int8, pl.Int16, pl.Int32, pl.Int64,
+    pl.UInt8, pl.UInt16, pl.UInt32, pl.UInt64,
+    pl.Float32, pl.Float64,
+)
+
+NUMERIC_TYPE_NAMES = {
+    'Int8', 'Int16', 'Int32', 'Int64',
+    'UInt8', 'UInt16', 'UInt32', 'UInt64',
+    'Float32', 'Float64',
+}
+
+
+# ═══════════════════════════════════════════════════════════════════════
+#  LLM response helpers
+# ═══════════════════════════════════════════════════════════════════════
+
+_MAX_RESPONSE_KB = 64
+
+
+def _json_size(obj: Any) -> int:
+    """Estimate JSON byte size of *obj*."""
+    return len(json.dumps(obj, default=str, ensure_ascii=False))
+
+
+def _truncate_response(result: dict, max_kb: int = _MAX_RESPONSE_KB) -> dict:
+    """Truncate large lists/keys in *result* to stay under *max_kb*."""
+    if _json_size(result) <= max_kb * 1024:
+        result.setdefault('truncated', False)
+        return result
+
+    # Drop verbose keys first
+    for costly_key in ('column_stats', 'features', 'correlation_matrix',
+                        'labels', 'cluster_labels'):
+        if costly_key in result and isinstance(result[costly_key], dict):
+            result[costly_key] = {'__truncated__': True,
+                                  'key_count': len(result[costly_key])}
+        elif costly_key in result and isinstance(result[costly_key], list):
+            lst = result[costly_key]
+            if len(lst) > 10:
+                result[costly_key] = lst[:10] + [{'__truncated__': True,
+                                                   'omitted': len(lst) - 10}]
+    result['truncated'] = True
+    return result
+
+
+# ═══════════════════════════════════════════════════════════════════════
+#  Filter expression builder
+# ═══════════════════════════════════════════════════════════════════════
+
+def _build_filter_expr(column: str, op: str, value: Any) -> pl.Expr:
+    """Build a single Polars expression from a filter descriptor."""
+    col = pl.col(column)
+    op_map = {
+        'eq': col == value,
+        'neq': col != value,
+        'gt': col > value,
+        'gte': col >= value,
+        'lt': col < value,
+        'lte': col <= value,
+        'contains': col.str.contains(value) if value is not None else col.cast(str).str.contains(''),
+        'not_contains': ~(col.str.contains(value) if value is not None else col.cast(str).str.contains('')),
+        'in': col.is_in(value) if isinstance(value, (list, tuple)) else col.is_in([value]),
+        'not_in': ~(col.is_in(value) if isinstance(value, (list, tuple)) else col.is_in([value])),
+        'is_null': col.is_null(),
+        'is_not_null': col.is_not_null(),
+    }
+    expr = op_map.get(op)
+    if expr is None:
+        raise ValueError(f"Unsupported filter operator: {op}")
+    return expr
+
+
+# ═══════════════════════════════════════════════════════════════════════
+#  SessionStore helpers
+# ═══════════════════════════════════════════════════════════════════════
+
+def _resolve_dataset(dataset_id: str) -> tuple:
+    """Resolve a dataset from SessionStore.
+
+    Returns:
+        (entry, None) on success, where *entry* is the SessionStore entry dict.
+        (None, error_dict) if the dataset is not found.
+    """
+    store = SessionStore()
+    entry = store.get_dataset(dataset_id)
+    if entry is None:
+        return None, {'error': f'Dataset not found: {dataset_id}', 'truncated': False}
+    return entry, None
+
+
+def _count_rows(lf: pl.LazyFrame) -> int:
+    """Count rows in a LazyFrame via fast streaming count."""
+    try:
+        return lf.select(pl.len()).collect(streaming=True).item()
+    except Exception:
+        logger.error("_count_rows failed: {}".format(traceback.format_exc()))
+        return 0
+
+
+def _get_numeric_columns(lf: pl.LazyFrame, schema: dict, max_cols: int = 10) -> list[str]:
+    """Get numeric column names from a dataset.
+
+    Prefers *schema* (string-keyed); falls back to LazyFrame collect_schema().
+    """
+    if schema:
+        return [c for c in schema
+                if schema[c].split('(')[0].strip() in NUMERIC_TYPE_NAMES
+                and not c.startswith('_')][:max_cols]
+    # Fallback to LazyFrame schema
+    available = lf.collect_schema()
+    return [available.names()[i] for i, dt in enumerate(available.dtypes())
+            if dt in NUMERIC_DTYPES and not available.names()[i].startswith('_')][:max_cols]
@@ -0,0 +1,754 @@
+"""Tool metadata definitions for the MCP server.
+
+All 30 Tool(...) descriptor objects and the get_tools_meta() function.
+"""
+from mcp.types import Tool
+
+
+def get_tools_meta() -> list[Tool]:
+    """Return the list of Tool metadata objects for MCP server registration.
+
+    Call this from :meth:`TLSAnalyzerMCPServer._setup_tools`.
+    """
+    return [
+        Tool(
+            name="load_data",
+            description=(
+                "加载匹配glob模式的CSV文件。自动检测每个文件的BOM，验证schema一致性，"
+                "纵向合并，并将结果作为惰性数据集存储在会话存储中。"
+            ),
+            inputSchema={
+                "type": "object",
+                "properties": {
+                    "csv_glob": {
+                        "type": "string",
+                        "description": "Glob pattern for CSV files (e.g. 'data/*.csv')",
+                    },
+                    "config_path": {
+                        "type": "string",
+                        "description": "Optional path to a YAML config file",
+                    },
+                    "encoding": {
+                        "type": "string",
+                        "description": "Fallback encoding when no BOM is found",
+                        "default": "utf-8",
+                    },
+                    "delimiter": {
+                        "type": "string",
+                        "description": "Column delimiter",
+                        "default": ",",
+                    },
+                    "schema_strict": {
+                        "type": "boolean",
+                        "description": "When True, raise on column mismatch; when False, merge leniently with nulls",
+                        "default": False,
+                    },
+                    "recursive": {
+                        "type": "boolean",
+                        "description": "When True, use recursive glob for **/*.csv patterns",
+                        "default": False,
+                    },
+                },
+                "required": ["csv_glob"],
+            },
+        ),
+        Tool(
+            name="profile_data",
+            description=(
+                "计算数据集的每列统计信息和数值相关矩阵。结果自动截断至约16 KB以"
+                "节省LLM上下文窗口空间。"
+            ),
+            inputSchema={
+                "type": "object",
+                "properties": {
+                    "dataset_id": {
+                        "type": "string",
+                        "description": "Dataset identifier from a previous load_data call",
+                    },
+                    "sample_size": {
+                        "type": "integer",
+                        "description": "Maximum rows to profile (sampled if larger)",
+                        "default": 1000,
+                    },
+                    "columns": {
+                        "type": "array",
+                        "items": {"type": "string"},
+                        "description": "Columns to profile (default: all)",
+                    },
+                },
+                "required": ["dataset_id"],
+            },
+        ),
+        Tool(
+            name="filter_data",
+            description=(
+                "对数据集应用列级过滤器，将结果存储为新的派生数据集。支持eq/neq/gt/gte/"
+                "lt/lte/contains/not_contains/in/not_in/is_null/is_not_null运算符。"
+            ),
+            inputSchema={
+                "type": "object",
+                "properties": {
+                    "dataset_id": {
+                        "type": "string",
+                        "description": "Source dataset identifier",
+                    },
+                    "filters": {
+                        "type": "array",
+                        "items": {
+                            "type": "object",
+                            "properties": {
+                                "column": {"type": "string"},
+                                "op": {
+                                    "type": "string",
+                                    "enum": [
+                                        "eq", "neq", "gt", "gte", "lt", "lte",
+                                        "contains", "not_contains",
+                                        "in", "not_in",
+                                        "is_null", "is_not_null",
+                                    ],
+                                },
+                                "value": {"description": "Operand value"},
+                            },
+                            "required": ["column", "op"],
+                        },
+                        "description": "List of filter expressions",
+                    },
+                    "logic": {
+                        "type": "string",
+                        "enum": ["and", "or"],
+                        "description": "Combine filters with AND or OR logic",
+                        "default": "and",
+                    },
+                },
+                "required": ["dataset_id", "filters"],
+            },
+        ),
+        Tool(
+            name="preprocess_data",
+            description=(
+                "预处理选定列：标准化（StandardScaler）、独热编码、缺失值填充和列删除。"
+            ),
+            inputSchema={
+                "type": "object",
+                "properties": {
+                    "dataset_id": {
+                        "type": "string",
+                        "description": "Source dataset identifier",
+                    },
+                    "columns": {
+                        "type": "array",
+                        "items": {"type": "string"},
+                        "description": "Columns to preprocess",
+                    },
+                    "config": {
+                        "type": "object",
+                        "properties": {
+                            "standardize": {
+                                "type": "boolean",
+                                "description": "Apply StandardScaler to numeric columns",
+                                "default": False,
+                            },
+                            "onehot": {
+                                "type": "boolean",
+                                "description": "One-hot encode categorical columns",
+                                "default": False,
+                            },
+                            "fillna": {
+                                "type": "string",
+                                "enum": ["mean", "median", "zero", "drop"],
+                                "description": "Missing value strategy",
+                            },
+                            "drop": {
+                                "type": "array",
+                                "items": {"type": "string"},
+                                "description": "Columns to drop entirely",
+                            },
+                        },
+                    },
+                },
+                "required": ["dataset_id", "columns"],
+            },
+        ),
+        Tool(
+            name="run_clustering",
+            description=(
+                "对选定列执行聚类。使用HDBSCAN（默认）或KMeans。特征在聚类前进行"
+                "标准化缩放。返回聚类标签和质量指标。"
+            ),
+            inputSchema={
+                "type": "object",
+                "properties": {
+                    "dataset_id": {
+                        "type": "string",
+                        "description": "Dataset identifier",
+                    },
+                    "cluster_columns": {
+                        "type": "array",
+                        "items": {"type": "string"},
+                        "description": "Columns to use as clustering features",
+                    },
+                    "algorithm": {
+                        "type": "string",
+                        "enum": ["agglomerative", "hdbscan", "kmeans"],
+                        "description": "Clustering algorithm (default: agglomerative/Ward)",
+                        "default": "agglomerative",
+                    },
+                    "params": {
+                        "type": "object",
+                        "description": (
+                            "Algorithm parameters. Agglomerative: n_clusters (5), metric ('euclidean'), "
+                            "linkage ('ward'). HDBSCAN: min_cluster_size (5), "
+                            "min_samples (None), metric ('euclidean'). "
+                            "KMeans: n_clusters (3), random_state (42)."
+                        ),
+                    },
+                    "random_state": {
+                        "type": "integer",
+                        "description": "Random seed for reproducibility",
+                        "default": 42,
+                    },
+                },
+                "required": ["dataset_id", "cluster_columns"],
+            },
+        ),
+        Tool(
+            name="evaluate_clustering",
+            description=(
+                "评估数据集的聚类结果。当前支持轮廓系数（silhouette）、Davies-Bouldin指数、"
+                "Calinski-Harabasz指数、噪声比（noise_ratio）和簇大小（cluster_sizes）指标。"
+            ),
+            inputSchema={
+                "type": "object",
+                "properties": {
+                    "cluster_result_id": {
+                        "type": "string",
+                        "description": "Result ID from run_clustering",
+                    },
+                    "dataset_id": {
+                        "type": "string",
+                        "description": "Original dataset for metric computation",
+                    },
+                    "metrics": {
+                        "type": "array",
+                        "items": {"type": "string"},
+                        "description": (
+                            "Metrics to compute. Options: silhouette, "
+                            "davies_bouldin, calinski_harabasz, noise_ratio, "
+                            "cluster_sizes (default: all)"
+                        ),
+                    },
+                },
+                "required": ["cluster_result_id", "dataset_id"],
+            },
+        ),
+        Tool(
+            name="extract_features",
+            description=(
+                "提取每个聚类的区分特征并自动保存到Django ORM（ClusterFeature模型）。"
+                "使用z-score或ANOVA对每个聚类的特征重要性进行排序。"
+            ),
+            inputSchema={
+                "type": "object",
+                "properties": {
+                    "dataset_id": {
+                        "type": "string",
+                        "description": "Dataset containing the feature columns",
+                    },
+                    "cluster_result_id": {
+                        "type": "string",
+                        "description": "Result ID from run_clustering",
+                    },
+                    "top_k": {
+                        "type": "integer",
+                        "description": "Top distinguishing features per cluster",
+                        "default": 10,
+                    },
+                    "method": {
+                        "type": "string",
+                        "enum": ["zscore", "anova"],
+                        "description": "Method to rank feature importance",
+                        "default": "zscore",
+                    },
+                    "save_to_db": {
+                        "type": "boolean",
+                        "description": "Save features to Django ORM if True",
+                        "default": True,
+                    },
+                },
+                "required": ["dataset_id", "cluster_result_id"],
+            },
+        ),
+        Tool(
+            name="export_results",
+            description=(
+                "将数据集或聚类结果以CSV或Parquet格式导出到磁盘。数据集物化后写入；"
+                "聚类结果序列化为JSON。"
+            ),
+            inputSchema={
+                "type": "object",
+                "properties": {
+                    "result_id": {
+                        "type": "string",
+                        "description": "Dataset ID or cluster result ID to export",
+                    },
+                    "output_path": {
+                        "type": "string",
+                        "description": "Output file path or directory",
+                    },
+                    "format": {
+                        "type": "string",
+                        "enum": ["csv", "parquet", "json"],
+                        "description": "Output format",
+                        "default": "csv",
+                    },
+                    "overwrite": {
+                        "type": "boolean",
+                        "description": "Overwrite existing file",
+                        "default": False,
+                    },
+                },
+                "required": ["result_id", "output_path"],
+            },
+        ),
+        Tool(
+            name="list_datasets",
+            description="列出会话存储中的所有活跃数据集和结果。",
+            inputSchema={
+                "type": "object",
+                "properties": {},
+            },
+        ),
+        Tool(
+            name="drop_dataset",
+            description=(
+                "从会话存储中移除数据集（或结果）并通过垃圾回收释放内存。"
+            ),
+            inputSchema={
+                "type": "object",
+                "properties": {
+                    "dataset_id": {
+                        "type": "string",
+                        "description": "Dataset or result identifier to remove",
+                    },
+                },
+                "required": ["dataset_id"],
+            },
+        ),
+        Tool(
+            name="clone_dataset",
+            description=(
+                "浅拷贝数据集。由于LazyFrame是查询计划，不会复制实际数据。适用于分支分析。"
+            ),
+            inputSchema={
+                "type": "object",
+                "properties": {
+                    "dataset_id": {
+                        "type": "string",
+                        "description": "Source dataset identifier",
+                    },
+                },
+                "required": ["dataset_id"],
+            },
+        ),
+        Tool(
+            name="filter_and_cluster",
+            description=(
+                "一步完成：先应用列级过滤器，再对过滤结果执行聚类。支持eq/neq/gt/gte/lt/lte/"
+                "contains/not_contains/in/not_in/is_null/is_not_null运算符和HDBSCAN/KMeans聚类。"
+                "自动选择数值列作为聚类特征。将过滤后的数据集和聚类结果都存储在会话中。"
+            ),
+            inputSchema={
+                "type": "object",
+                "properties": {
+                    "dataset_id": {
+                        "type": "string",
+                        "description": "Source dataset identifier",
+                    },
+                    "filters": {
+                        "type": "array",
+                        "items": {
+                            "type": "object",
+                            "properties": {
+                                "column": {"type": "string"},
+                                "op": {
+                                    "type": "string",
+                                    "enum": [
+                                        "eq", "neq", "gt", "gte", "lt", "lte",
+                                        "contains", "not_contains",
+                                        "in", "not_in",
+                                        "is_null", "is_not_null",
+                                    ],
+                                },
+                                "value": {"description": "Operand value"},
+                            },
+                            "required": ["column", "op"],
+                        },
+                        "description": "List of filter expressions",
+                    },
+                    "logic": {
+                        "type": "string",
+                        "enum": ["and", "or"],
+                        "description": "Combine filters with AND or OR logic",
+                        "default": "and",
+                    },
+                    "algorithm": {
+                        "type": "string",
+                        "enum": ["agglomerative", "hdbscan", "kmeans"],
+                        "description": "Clustering algorithm (default: agglomerative/Ward)",
+                        "default": "agglomerative",
+                    },
+                    "params": {
+                        "type": "object",
+                        "description": (
+                            "Algorithm parameters. Agglomerative: n_clusters (5), metric ('euclidean'), "
+                            "linkage ('ward'). HDBSCAN: min_cluster_size (5), "
+                            "min_samples (None), metric ('euclidean'). "
+                            "KMeans: n_clusters (3), random_state (42)."
+                        ),
+                    },
+                },
+                "required": ["dataset_id", "filters"],
+            },
+        ),
+        Tool(
+            name="build_entity_profiles",
+            description=(
+                "按实体列（源IP、SNI等）分组原始流数据。自动检测实体列（如 src_ip, "
+                "dst_ip, sni, host, mac, domain），按列分组并计算每个实体的聚合特征："
+                "流数量、唯一端口数、TLS版本分布、字节总量、唯一目标数。"
+                "生成的数据集每个实体一行。结果存储在会话中，可供后续 compute_scores 使用。"
+            ),
+            inputSchema={
+                "type": "object",
+                "properties": {
+                    "dataset_id": {
+                        "type": "string",
+                        "description": "Dataset ID from load_data or profile_data",
+                    },
+                    "entity_column": {
+                        "type": "string",
+                        "description": "Specific entity column to group by (e.g. 'src_ip')",
+                    },
+                    "entity_columns": {
+                        "type": "array",
+                        "items": {"type": "string"},
+                        "description": "Multiple entity columns to group by",
+                    },
+                    "auto_detect": {
+                        "type": "boolean",
+                        "description": "Auto-detect entity columns (IP, SNI, host, MAC, domain)",
+                        "default": True,
+                    },
+                },
+                "required": ["dataset_id"],
+            },
+        ),
+        Tool(
+            name="compute_scores",
+            description=(
+                "在实体数据上计算自适应代理/异常/威胁评分。从数据自动学习"
+                "特征权重，计算加权代理评分（proxy_score）、基于IQR的风险等级"
+                "（normal/watch/suspicious/critical）和威胁评分（threat_score）。"
+                "返回评分摘要与阈值。"
+            ),
+            inputSchema={
+                "type": "object",
+                "properties": {
+                    "dataset_id": {
+                        "type": "string",
+                        "description": "Entity dataset ID from filter_and_cluster or a scored dataset",
+                    },
+                },
+                "required": ["dataset_id"],
+            },
+        ),
+        Tool(
+            name="detect_anomalies",
+            description=(
+                "对已评分的实体数据运行孤立森林（Isolation Forest）异常检测。识别偏离"
+                "正常模式的实体。分块处理以应对大数据集。返回异常实体ID列表。"
+            ),
+            inputSchema={
+                "type": "object",
+                "properties": {
+                    "dataset_id": {
+                        "type": "string",
+                        "description": "Scored entity dataset ID (after compute_scores)",
+                    },
+                    "contamination": {
+                        "type": "number",
+                        "description": "Expected proportion of anomalies (0.0-0.5)",
+                        "default": 0.05,
+                    },
+                },
+                "required": ["dataset_id"],
+            },
+        ),
+        Tool(
+            name="visualize_anomalies",
+            description=(
+                "为高风险实体生成UMAP-2D嵌入和聚类可视化数据。按风险等级和聚类着色。"
+                "返回用于前端渲染的散点图JSON数据。"
+            ),
+            inputSchema={
+                "type": "object",
+                "properties": {
+                    "dataset_id": {
+                        "type": "string",
+                        "description": "Scored entity dataset ID (after compute_scores)",
+                    },
+                },
+                "required": ["dataset_id"],
+            },
+        ),
+        # ── Edge-case / Diagnostic tools (unexpected situations) ────────
+        Tool(
+            name="validate_data",
+            description=(
+                "诊断工具：检查数据质量问题——文件间schema一致性、缺失值比例、列类型冲突、"
+                "编码问题和空列。当流水线步骤因schema错误失败，或怀疑CSV数据损坏/格式异常时使用。"
+            ),
+            inputSchema={
+                "type": "object",
+                "properties": {
+                    "dataset_id": {"type": "string", "description": "Dataset ID from load_data"},
+                    "fix_issues": {"type": "boolean", "description": "Auto-fix schema inconsistencies", "default": False},
+                },
+                "required": ["dataset_id"],
+            },
+        ),
+        Tool(
+            name="explore_distributions",
+            description=(
+                "诊断工具：计算每列分布统计（唯一值计数、空值比例、最小/最大/均值/标准差、"
+                "前10高频值、直方图分箱）。当聚类产生意外结果、实体检测失败，或需要在选择"
+                "分析策略前了解数据时使用。"
+            ),
+            inputSchema={
+                "type": "object",
+                "properties": {
+                    "dataset_id": {"type": "string", "description": "Dataset ID from load_data or entity data"},
+                    "columns": {"type": "array", "items": {"type": "string"},
+                                "description": "Specific columns to analyze (default: auto-detect numeric)"},
+                    "max_sample": {"type": "integer", "description": "Max rows to sample", "default": 10000},
+                },
+                "required": ["dataset_id"],
+            },
+        ),
+        Tool(
+            name="find_outliers",
+            description=(
+                "诊断工具：使用IQR方法检测数值列中的统计异常值。返回任何数值列超过中位数3倍IQR的行。"
+                "当怀疑数据质量问题或极端值偏离分析时使用。"
+            ),
+            inputSchema={
+                "type": "object",
+                "properties": {
+                    "dataset_id": {"type": "string", "description": "Dataset ID from load_data"},
+                    "columns": {"type": "array", "items": {"type": "string"},
+                                "description": "Columns to check (default: all numeric)"},
+                    "threshold": {"type": "number", "description": "IQR multiplier (default 3.0)", "default": 3.0},
+                },
+                "required": ["dataset_id"],
+            },
+        ),
+        Tool(
+            name="diagnose_clustering",
+            description=(
+                "诊断工具：当聚类产生太少/太多聚类、全部为噪声或轮廓系数低时——诊断原因。"
+                "检查：特征方差、相关矩阵、特征降维分析、数据稀疏性，并建议参数调整"
+                "（min_cluster_size、算法选择、特征选择）。"
+            ),
+            inputSchema={
+                "type": "object",
+                "properties": {
+                    "dataset_id": {"type": "string", "description": "Entity dataset ID"},
+                    "cluster_result_id": {"type": "string", "description": "Cluster result ID from previous run_clustering"},
+                },
+                "required": ["dataset_id", "cluster_result_id"],
+            },
+        ),
+        Tool(
+            name="compare_datasets",
+            description=(
+                "诊断工具：比较两个数据集（如处理前/后，或两次不同运行）。报告schema差异、"
+                "行数变化、列值分布偏移。用于验证处理步骤未损坏数据或比较不同运行的结果。"
+            ),
+            inputSchema={
+                "type": "object",
+                "properties": {
+                    "dataset_id_a": {"type": "string", "description": "First dataset ID"},
+                    "dataset_id_b": {"type": "string", "description": "Second dataset ID"},
+                },
+                "required": ["dataset_id_a", "dataset_id_b"],
+            },
+        ),
+        Tool(
+            name="export_debug_sample",
+            description=(
+                "诊断工具：导出原始数据样本为JSON用于外部调试。当流水线深处发生错误且需要"
+                "检查实际数据值时使用。返回前N行所有列。"
+            ),
+            inputSchema={
+                "type": "object",
+                "properties": {
+                    "dataset_id": {"type": "string", "description": "Dataset ID"},
+                    "max_rows": {"type": "integer", "description": "Rows to export", "default": 50},
+                },
+                "required": ["dataset_id"],
+            },
+        ),
+        Tool(
+            name="repair_schema",
+            description=(
+                "诊断工具：通过对齐列名（不区分大小写合并、下划线/连字符规范化）并用空值填充"
+                "缺失列，尝试修复文件间的schema不匹配。当load_data因CSV文件具有不同列集而"
+                "失败时使用。"
+            ),
+            inputSchema={
+                "type": "object",
+                "properties": {
+                    "dataset_id": {"type": "string", "description": "Failed dataset ID from load_data"},
+                },
+                "required": ["dataset_id"],
+            },
+        ),
+        # ── Read-only analysis tools (LLM deep-dive, no side effects) ────
+        Tool(
+            name="analyze_patterns",
+            description=(
+                "分析工具：检测流量模式——top源/目标IP、端口分布、TLS版本分布、协议混合。"
+                "返回汇总统计。只读，不修改数据。用于了解高层流量结构。"
+            ),
+            inputSchema={
+                "type": "object",
+                "properties": {
+                    "dataset_id": {"type": "string", "description": "Dataset ID from load_data"},
+                    "top_n": {"type": "integer", "description": "Number of top items per category", "default": 10},
+                },
+                "required": ["dataset_id"],
+            },
+        ),
+        Tool(
+            name="analyze_temporal",
+            description=(
+                "分析工具：分析随时间变化的流量——小时/日流量计数、繁忙时段、空闲时段。"
+                "需要时间戳列。只读。用于了解时间相关模式和周期性。"
+            ),
+            inputSchema={
+                "type": "object",
+                "properties": {
+                    "dataset_id": {"type": "string", "description": "Dataset ID from load_data"},
+                    "timestamp_column": {"type": "string", "description": "Column name for timestamps (auto-detected if omitted)"},
+                },
+                "required": ["dataset_id"],
+            },
+        ),
+        Tool(
+            name="analyze_fft",
+            description=(
+                "FFT频谱分析——从时间戳列提取周期模式和频率特征。适用于检测时间周期性"
+                "（如每日/每周流量模式）。返回频率、幅度、top周期、频谱质心、周期性评分。"
+                "只读，无副作用。"
+            ),
+            inputSchema={
+                "type": "object",
+                "properties": {
+                    "dataset_id": {
+                        "type": "string",
+                        "description": "Dataset ID from load_data",
+                    },
+                    "timestamp_column": {
+                        "type": "string",
+                        "description": "Column name containing timestamps or numeric time values",
+                    },
+                    "top_n": {
+                        "type": "integer",
+                        "description": "Number of top frequency peaks to return",
+                        "default": 5,
+                    },
+                },
+                "required": ["dataset_id", "timestamp_column"],
+            },
+        ),
+        Tool(
+            name="analyze_tls_health",
+            description=(
+                "分析工具：TLS安全态势评估——检查过时的TLS版本（1.0/1.1）、弱加密套件、"
+                "证书问题、SNI异常。只读。用于评估流量的TLS安全性。"
+            ),
+            inputSchema={
+                "type": "object",
+                "properties": {
+                    "dataset_id": {"type": "string", "description": "Dataset ID from load_data"},
+                },
+                "required": ["dataset_id"],
+            },
+        ),
+        Tool(
+            name="analyze_geo_distribution",
+            description=(
+                "分析工具：流量地理分布——top源/目标国家、异常地点对（如不可能的行程）、"
+                "ISP多样性。只读。需要国家或经纬度列。用于检测地理异常。"
+            ),
+            inputSchema={
+                "type": "object",
+                "properties": {
+                    "dataset_id": {"type": "string", "description": "Dataset ID from load_data"},
+                },
+                "required": ["dataset_id"],
+            },
+        ),
+        Tool(
+            name="analyze_entity_detail",
+            description=(
+                "分析工具：深入分析单个实体——其所有流、使用的TLS版本、目标多样性、时间模式、"
+                "异常评分。只读。在compute_scores后调查特定可疑实体时使用。"
+            ),
+            inputSchema={
+                "type": "object",
+                "properties": {
+                    "dataset_id": {"type": "string", "description": "Entity dataset ID (after filter_and_cluster or compute_scores)"},
+                    "entity_value": {"type": "string", "description": "Entity value to investigate (e.g. an IP address or SNI)"},
+                },
+                "required": ["dataset_id", "entity_value"],
+            },
+        ),
+        Tool(
+            name="compute_distance_matrix",
+            description=(
+                "LLM驱动工具：对数据集的每一行执行用户编写的Python函数，返回距离/相似度评分摘要。"
+                "LLM提供函数体，签名格式为`def distance_fn(row: dict) -> float:`。执行器将其包装，"
+                "在受限命名空间（仅math、numpy）中逐行运行，返回得分的最小/最大/均值/标准差以及"
+                "评分行样本。用于现有工具无法表达的一次性度量、过滤或评分任务。"
+            ),
+            inputSchema={
+                "type": "object",
+                "properties": {
+                    "dataset_id": {
+                        "type": "string",
+                        "description": "Source dataset identifier",
+                    },
+                    "python_function": {
+                        "type": "string",
+                        "description": (
+                            "Body of a Python function with signature `def distance_fn(row: dict) -> float:`. "
+                            "Example: \"return abs(row.get('col1', 0)) + row.get('col2', 0)\". "
+                            "Available namespace: math, numpy (as np). No dangerous modules."
+                        ),
+                    },
+                    "columns": {
+                        "type": "array",
+                        "items": {"type": "string"},
+                        "description": "Columns to pass to the function (default: auto-detect numeric columns)",
+                    },
+                    "max_sample": {
+                        "type": "integer",
+                        "description": "Maximum rows to evaluate (default 10000 for performance)",
+                        "default": 10000,
+                    },
+                },
+                "required": ["dataset_id", "python_function"],
+            },
+        ),
+    ]
@@ -0,0 +1,335 @@
+"""Read-only analysis handlers — patterns, temporal, FFT, TLS health, geo, entity detail."""
+import traceback
+from analysis.constants import LOW_MEMORY_THRESHOLD_ROWS
+import numpy as np
+import polars as pl
+
+from ..session_store import SessionStore
+from ._helpers import _resolve_dataset, _count_rows, NUMERIC_TYPE_NAMES
+import logging
+logger = logging.getLogger(__name__)
+
+
+async def _handle_analyze_patterns(dataset_id: str, top_n: int = 10) -> dict:
+    """Read-only: traffic pattern analysis."""
+    entry, err = _resolve_dataset(dataset_id)
+    if err:
+        return err
+    lf = entry['lazyframe']
+    schema = entry.get('schema', {})
+    n = _count_rows(lf)
+    df = lf.head(LOW_MEMORY_THRESHOLD_ROWS).collect(streaming=True)
+
+    result = {'total_rows': n, 'sampled': min(n, LOW_MEMORY_THRESHOLD_ROWS)}
+
+    for col_name, label in [(':ips', 'src_ips'), (':ipd', 'dst_ips'),
+                              ('snam', 'snis'), ('cnam', 'cert_names')]:
+        if col_name in df.columns:
+            try:
+                top = df[col_name].value_counts(sort=True).head(top_n)
+                result[label] = {
+                    'top': [{'value': str(r[col_name]), 'count': int(r['count'])}
+                            for r in top.iter_rows(named=True)],
+                    'unique': int(df[col_name].n_unique()),
+                }
+            except Exception:
+                logger.error("_handle_analyze_patterns failed: {}".format(traceback.format_exc()))
+                pass
+
+    for port_col in [':prs', ':prd']:
+        if port_col in df.columns:
+            try:
+                top = df[port_col].value_counts(sort=True).head(top_n)
+                result[f'{port_col}_dist'] = {
+                    'top': [{'port': int(r[port_col]), 'count': int(r['count'])}
+                            for r in top.iter_rows(named=True)],
+                    'unique': int(df[port_col].n_unique()),
+                }
+            except Exception:
+                logger.error("unknown failed: {}".format(traceback.format_exc()))
+                pass
+
+    for tls_col in ['0ver']:
+        if tls_col in df.columns:
+            try:
+                vc = df[tls_col].value_counts(sort=True)
+                result['tls_versions'] = [{'version': str(r[tls_col]), 'count': int(r['count'])}
+                                          for r in vc.iter_rows(named=True)]
+            except Exception:
+                logger.error("unknown failed: {}".format(traceback.format_exc()))
+                pass
+
+    return result
+
+
+async def _handle_analyze_temporal(dataset_id: str, timestamp_column: str = None) -> dict:
+    """Read-only: temporal traffic pattern analysis."""
+    entry, err = _resolve_dataset(dataset_id)
+    if err:
+        return err
+    lf = entry['lazyframe']
+    schema = entry.get('schema', {})
+
+    if not timestamp_column:
+        for cand in ['timestamp', 'time', '8dbd', '8did']:
+            if cand in schema:
+                timestamp_column = cand
+                break
+    if not timestamp_column:
+        return {'error': 'No timestamp column found', 'truncated': False}
+
+    n = _count_rows(lf)
+    return {'total_rows': n, 'timestamp_column': timestamp_column,
+            'note': 'Temporal analysis requires parsed timestamps; raw values shown in explore_distributions'}
+
+
+async def _handle_analyze_fft(
+    dataset_id: str,
+    timestamp_column: str,
+    top_n: int = 5,
+) -> dict:
+    """FFT spectrum analysis on a timestamp column to detect periodic patterns."""
+    entry, err = _resolve_dataset(dataset_id)
+    if err:
+        return err
+
+    lf: pl.LazyFrame = entry['lazyframe']
+    schema = entry.get('schema', {})
+
+    col = None
+    if timestamp_column in schema:
+        col = timestamp_column
+    else:
+        col_lower = timestamp_column.lower()
+        for c in schema:
+            if c.lower() == col_lower:
+                col = c
+                break
+    if col is None:
+        return {'error': f"Timestamp column '{timestamp_column}' not found in dataset. Available: {list(schema.keys())[:20]}",
+                'truncated': False}
+
+    df = lf.select(pl.col(col).alias('__ts__')).collect(streaming=True)
+    n_raw = len(df)
+    if n_raw < 4:
+        return {'error': f'Need at least 4 samples for FFT; got {n_raw}', 'truncated': False}
+
+    ts_series = df['__ts__']
+    dtype = ts_series.dtype
+    str_dtypes = (pl.Utf8, pl.String)
+
+    if dtype in str_dtypes:
+        try:
+            ts_parsed = ts_series.str.strptime(pl.Datetime, format=None, strict=False)
+            ts_numeric = ts_parsed.cast(pl.Int64) / 1_000_000_000.0
+        except Exception:
+            logger.error("unknown failed: {}".format(traceback.format_exc()))
+            try:
+                ts_numeric = ts_series.cast(pl.Float64)
+            except Exception as e:
+                return {'error': f'Cannot convert string column to numeric or datetime: {e}',
+                        'truncated': False}
+    elif dtype in (pl.Datetime, pl.Date):
+        ts_numeric = ts_series.cast(pl.Int64) / 1_000_000_000.0
+    else:
+        try:
+            ts_numeric = ts_series.cast(pl.Float64)
+        except Exception as e:
+            return {'error': f'Cannot cast column to float: {e}', 'truncated': False}
+
+    df_sorted = pl.DataFrame({'__val__': ts_numeric}).drop_nulls().sort('__val__')
+    n = len(df_sorted)
+    if n < 4:
+        return {'error': f'After removing nulls, only {n} samples remain (need ≥4)', 'truncated': False}
+
+    values = df_sorted['__val__'].to_numpy().astype(np.float64)
+
+    values_centered = values - np.mean(values)
+    fft_complex = np.fft.fft(values_centered)
+    fft_magnitudes = np.abs(fft_complex)
+    freqs = np.fft.fftfreq(n)
+
+    pos_mask = freqs > 0
+    freqs_pos = freqs[pos_mask]
+    mags_pos = fft_magnitudes[pos_mask]
+
+    if len(freqs_pos) == 0:
+        return {'error': 'Insufficient positive frequencies for FFT analysis', 'truncated': False}
+
+    peak_indices = np.argsort(mags_pos)[::-1][:min(top_n, len(mags_pos))]
+    top_periods = []
+    for idx in peak_indices:
+        freq = float(freqs_pos[idx])
+        mag = float(mags_pos[idx])
+        period = float(1.0 / freq) if freq > 1e-10 else float('inf')
+        if n > 1:
+            time_span = float(values[-1] - values[0])
+            sample_period = time_span / max(n - 1, 1)
+            time_period = period * sample_period
+        else:
+            time_period = float('inf')
+        top_periods.append({
+            'period': round(period, 2),
+            'magnitude': round(mag, 4),
+            'frequency': round(freq, 6),
+            'time_domain_period_seconds': round(time_period, 2) if time_period != float('inf') else None,
+        })
+
+    mag_sum = float(np.sum(mags_pos))
+    if mag_sum > 0:
+        spectral_centroid = float(np.sum(freqs_pos * mags_pos) / mag_sum)
+    else:
+        spectral_centroid = 0.0
+
+    mean_mag = float(np.mean(mags_pos)) if len(mags_pos) > 0 else 1.0
+    max_mag = float(mags_pos[peak_indices[0]]) if len(peak_indices) > 0 else 0.0
+    periodicity_score = round(max_mag / max(mean_mag, 1e-10), 4)
+
+    decimate_step = max(1, len(freqs_pos) // 200)
+    freqs_decimated = freqs_pos[::decimate_step].tolist()
+    mags_decimated = mags_pos[::decimate_step].tolist()
+
+    return {
+        'frequencies': [round(float(f), 6) for f in freqs_decimated],
+        'magnitudes': [round(float(m), 4) for m in mags_decimated],
+        'top_periods': top_periods,
+        'spectral_centroid': round(spectral_centroid, 6),
+        'periodicity_score': periodicity_score,
+        'n_samples': n,
+        'n_raw': n_raw,
+        'nan_dropped': n_raw - n,
+        'timestamp_column': col,
+    }
+
+
+async def _handle_analyze_tls_health(dataset_id: str) -> dict:
+    """Read-only: TLS security assessment."""
+    entry, err = _resolve_dataset(dataset_id)
+    if err:
+        return err
+    lf = entry['lazyframe']
+    schema = entry.get('schema', {})
+    n = _count_rows(lf)
+    df = lf.head(LOW_MEMORY_THRESHOLD_ROWS).collect(streaming=True)
+
+    result = {'total_rows': n, 'sampled': min(n, LOW_MEMORY_THRESHOLD_ROWS)}
+
+    if '0ver' in df.columns:
+        vc = df['0ver'].value_counts(sort=True)
+        versions = {}
+        for r in vc.iter_rows(named=True):
+            v = str(r['0ver']).replace(' ', '')
+            if v in ('0303', '0304'):
+                versions['tls_1_2_plus'] = versions.get('tls_1_2_plus', 0) + int(r['count'])
+            else:
+                versions['tls_legacy'] = versions.get('tls_legacy', 0) + int(r['count'])
+        total = sum(versions.values())
+        result.update({
+            'tls_1_2_plus_pct': round(versions.get('tls_1_2_plus', 0) / max(total, 1) * 100, 1),
+            'tls_legacy_pct': round(versions.get('tls_legacy', 0) / max(total, 1) * 100, 1),
+            'tls_legacy_versions': [{'version': str(r['0ver']), 'count': int(r['count'])}
+                                     for r in vc.iter_rows(named=True)
+                                     if str(r['0ver']).replace(' ', '') not in ('0303', '0304')],
+        })
+
+    if 'snam' in df.columns:
+        empty_sni = df['snam'].is_null().sum() + df['snam'].cast(pl.Utf8).is_in(['', 'null', 'None']).sum()
+        result['sni_missing_pct'] = round(float(empty_sni) / max(len(df), 1) * 100, 1)
+
+    if 'cipher-suite' in df.columns:
+        weak_ciphers = ['rc4', 'cbc', 'null', 'export']
+        total_ciphers = len(df)
+        weak_count = 0
+        for w in weak_ciphers:
+            try:
+                weak_count += df['cipher-suite'].str.to_lowercase().str.contains(w).sum()
+            except Exception:
+                logger.error("unknown failed: {}".format(traceback.format_exc()))
+                pass
+        result['weak_cipher_pct'] = round(float(weak_count) / max(total_ciphers, 1) * 100, 1) if total_ciphers else 0
+
+    if result.get('tls_legacy_pct', 0) > 20:
+        result['warning'] = 'High proportion of legacy TLS — possible downgrade attacks'
+    if result.get('sni_missing_pct', 0) > 50:
+        result['warning'] = 'Most flows lack SNI — possible IP-direct connections (proxy/VPN)'
+    return result
+
+
+async def _handle_analyze_geo_distribution(dataset_id: str) -> dict:
+    """Read-only: geographic traffic analysis."""
+    entry, err = _resolve_dataset(dataset_id)
+    if err:
+        return err
+    lf = entry['lazyframe']
+    schema = entry.get('schema', {})
+    n = _count_rows(lf)
+    df = lf.head(LOW_MEMORY_THRESHOLD_ROWS).collect(streaming=True)
+
+    result = {'total_rows': n, 'sampled': min(n, LOW_MEMORY_THRESHOLD_ROWS)}
+
+    for geo_col, label in [('scnt', 'src_countries'), ('dcnt', 'dst_countries')]:
+        if geo_col in df.columns:
+            try:
+                vc = df[geo_col].value_counts(sort=True).head(15)
+                result[label] = [{'country': str(r[geo_col]), 'count': int(r['count'])}
+                                 for r in vc.iter_rows(named=True)]
+                result[f'{label}_unique'] = int(df[geo_col].n_unique())
+            except Exception:
+                logger.error("_handle_analyze_geo_distribution failed: {}".format(traceback.format_exc()))
+                pass
+
+    if 'scnt' in df.columns and 'dcnt' in df.columns:
+        try:
+            same = (df['scnt'] == df['dcnt']).sum()
+            result['domestic_pct'] = round(float(same) / max(len(df), 1) * 100, 1)
+            result['international_pct'] = round(100.0 - result['domestic_pct'], 1)
+        except Exception:
+            logger.error("_handle_analyze_geo_distribution failed: {}".format(traceback.format_exc()))
+                pass
+
+    return result
+
+
+async def _handle_analyze_entity_detail(dataset_id: str, entity_value: str) -> dict:
+    """Read-only: deep-dive into a single entity."""
+    entry, err = _resolve_dataset(dataset_id)
+    if err:
+        return err
+    lf = entry['lazyframe']
+    schema = entry.get('schema', {})
+
+    entity_col = None
+    for c in schema:
+        if c.startswith('_') or schema[c].split('(')[0].strip() in NUMERIC_TYPE_NAMES or '.' in c:
+            continue
+        entity_col = c
+        break
+    if not entity_col:
+        entity_col = list(schema.keys())[0]
+
+    filtered = lf.filter(pl.col(entity_col).cast(pl.Utf8).str.contains(entity_value, literal=True))
+    n = _count_rows(filtered)
+    if n == 0:
+        return {'error': f"Entity '{entity_value}' not found in column '{entity_col}'", 'truncated': False}
+    df = filtered.collect(streaming=True)
+
+    result = {
+        'entity_value': entity_value,
+        'entity_column': entity_col,
+        'flow_count': n,
+        'columns': list(df.columns[:15]),
+    }
+
+    for c in df.columns:
+        if df[c].dtype in (pl.Float32, pl.Float64, pl.Int32, pl.Int64):
+            try:
+                result[f'{c}_stats'] = {
+                    'mean': round(float(df[c].mean()), 2),
+                    'min': float(df[c].min()) if df[c].min() is not None else None,
+                    'max': float(df[c].max()) if df[c].max() is not None else None,
+                }
+            except Exception:
+                logger.error("unknown failed: {}".format(traceback.format_exc()))
+                pass
+
+    return result
@@ -0,0 +1,94 @@
+"""Anomaly detection + visualization handlers."""
+import numpy as np
+from analysis.constants import MAX_DISTRIBUTION_SAMPLE, RANDOM_SEED
+import polars as pl
+
+from ..session_store import SessionStore
+from ._helpers import _resolve_dataset, NUMERIC_TYPE_NAMES
+import logging
+logger = logging.getLogger(__name__)
+
+
+async def _handle_detect_anomalies(dataset_id: str, contamination: float = 0.05) -> dict:
+    """Run Isolation Forest on scored entity data."""
+    from sklearn.ensemble import IsolationForest
+import traceback
+    entry, err = _resolve_dataset(dataset_id)
+    if err:
+        return err
+
+    lf: pl.LazyFrame = entry['lazyframe']
+    schema = entry.get('schema', {})
+    num_cols = [c for c in schema if schema[c].split('(')[0].strip() in NUMERIC_TYPE_NAMES and not c.startswith('_')]
+    if not num_cols:
+        return {'error': 'No numeric columns for anomaly detection', 'truncated': False}
+
+    df = lf.select(num_cols).collect(streaming=True)
+    mat = df.to_numpy()
+    mat = np.nan_to_num(mat, nan=0.0)
+
+    n = len(mat)
+    chunk_size = min(MAX_SAMPLE_ROWS, n)
+    labels = np.zeros(n, dtype=int)
+    for start in range(0, n, chunk_size):
+        end = min(start + chunk_size, n)
+        iso = IsolationForest(n_estimators=100, max_samples=min(256, chunk_size),
+                              contamination=contamination, random_state=RANDOM_SEED, n_jobs=-1)
+        labels[start:end] = iso.fit_predict(mat[start:end])
+
+    anomaly_count = int((labels == -1).sum())
+    return {
+        'status': 'completed',
+        'total_entities': n,
+        'anomaly_count': anomaly_count,
+        'anomaly_rate': round(anomaly_count / max(n, 1), 4),
+        'dataset_id': dataset_id,
+    }
+
+
+async def _handle_visualize_anomalies(dataset_id: str) -> dict:
+    """Generate UMAP-2D embedding for anomaly visualization."""
+    from sklearn.preprocessing import StandardScaler
+    import umap
+    entry, err = _resolve_dataset(dataset_id)
+    if err:
+        return err
+
+    lf: pl.LazyFrame = entry['lazyframe']
+    schema = entry.get('schema', {})
+    num_cols = [c for c in schema if schema[c].split('(')[0].strip() in NUMERIC_TYPE_NAMES and not c.startswith('_')]
+    if len(num_cols) < 2:
+        return {'error': 'Need at least 2 numeric columns for UMAP', 'truncated': False}
+
+    try:
+        n_total = lf.select(pl.len()).collect(streaming=True).item()
+    except Exception:
+        logger.error("_handle_visualize_anomalies failed: {}".format(traceback.format_exc()))
+        n_total = 0
+    sample_lf = lf if n_total <= MAX_DISTRIBUTION_SAMPLE else lf.collect(streaming=True).sample(n=MAX_DISTRIBUTION_SAMPLE, seed=RANDOM_SEED).lazy()
+    df = sample_lf.select(num_cols).collect(streaming=True)
+    mat = StandardScaler().fit_transform(df.to_numpy())
+    mat = np.nan_to_num(mat, nan=0.0)
+    reducer = umap.UMAP(n_components=2, random_state=RANDOM_SEED,
+                        n_neighbors=15, min_dist=0.1, metric='euclidean')
+    coords = reducer.fit_transform(mat)
+
+    risk_col = next((c for c in schema if 'risk' in c.lower()), None)
+    score_col = next((c for c in schema if 'proxy_score' in c.lower() or 'threat' in c.lower()), None)
+
+    non_num = [c for c in df.columns if c not in num_cols and not c.startswith('_')]
+    ent_col = non_num[0] if non_num else df.columns[0]
+    entities = df.select(ent_col).to_series().to_list() if ent_col in df.columns else []
+
+    scatter = [
+        {'x': float(coords[i, 0]), 'y': float(coords[i, 1]),
+         'entity': str(entities[i]) if i < len(entities) else '',
+         'score': round(float(mat[i].mean()), 4)}
+        for i in range(len(coords))
+    ]
+    return {
+        'status': 'completed',
+        'scatter_points': len(scatter),
+        'scatter_data': scatter[:200],
+        'dataset_id': dataset_id,
+    }
@@ -0,0 +1,458 @@
+"""Clustering handlers — HDBSCAN, KMeans, AgglomerativeClustering.
+
+Contains:
+- _handle_run_clustering: cluster selected columns
+- _handle_filter_and_cluster: filter then auto-cluster
+- _cluster_core: shared clustering pipeline extracted from both handlers
+"""
+import logging
+from analysis.constants import LOW_MEMORY_THRESHOLD_GB, LOW_MEMORY_THRESHOLD_ROWS, MAX_CLUSTER_FEATURES, MAX_SAMPLE_ROWS, RANDOM_SEED
+from typing import Optional
+
+import numpy as np
+import polars as pl
+
+from ..session_store import SessionStore, _generate_id
+from ._helpers import (
+logger = logging.getLogger(__name__)
+    NUMERIC_DTYPES,
+    _build_filter_expr,
+    _resolve_dataset,
+    _count_rows,
+)
+
+_clust_logger = logging.getLogger(__name__)
+
+
+def _cluster_core(
+    data: np.ndarray,
+    feature_cols: list[str],
+    algorithm: str,
+    params: dict,
+    random_state: int = RANDOM_SEED,
+):
+    """Shared clustering pipeline: scale, filter, reduce, fit, score.
+
+    Args:
+        data: Numpy array (n_samples, n_features).
+        feature_cols: Column names matching data columns.
+        algorithm: 'hdbscan', 'kmeans', or 'agglomerative'.
+        params: Algorithm-specific parameters dict.
+        random_state: Random seed.
+
+    Returns:
+        (labels_array, n_clusters_found, noise_count, quality_metrics, _ds_idx)
+    """
+    p = dict(params)
+
+    # ── Low memory check ───────────────────────────────────────────────
+    low_memory = False
+    try:
+        import psutil
+        mem = psutil.virtual_memory()
+        low_memory = mem.available < LOW_MEMORY_THRESHOLD_GB * 1024 ** 3  # 2 GB
+    except ImportError:
+        pass
+
+    if low_memory and len(data) > LOW_MEMORY_THRESHOLD_ROWS:
+        rng = np.random.default_rng(RANDOM_SEED)
+        idx = rng.choice(len(data), MAX_SAMPLE_ROWS, replace=False)
+        data = data[idx]
+
+    # ── Downsampling to 50K max ────────────────────────────────────────
+    _ds_idx = None
+    if len(data) > MAX_SAMPLE_ROWS:
+        rng = np.random.default_rng(RANDOM_SEED)
+        _ds_idx = rng.choice(len(data), MAX_SAMPLE_ROWS, replace=False)
+        data = data[_ds_idx]
+
+    # ── Variance filtering ─────────────────────────────────────────────
+    if data.shape[1] > 1:
+        variances = np.var(data, axis=0)
+        keep_var = np.where(variances >= 1e-10)[0]
+        if len(keep_var) == 0:
+            keep_var = np.arange(data.shape[1])
+        if len(keep_var) < data.shape[1]:
+            data = data[:, keep_var]
+            feature_cols = [feature_cols[i] for i in keep_var]
+
+    # Log clustering input
+    _clust_logger.info(
+        '[CLUSTER_INPUT] rows=%d cols=%d feature_cols=%s',
+        len(data), data.shape[1] if len(data.shape) > 1 else 1, feature_cols,
+    )
+
+    # Standard scale
+    from sklearn.preprocessing import StandardScaler
+    _clust_logger.info('[CLUSTER_SCALE] Starting StandardScaler...')
+    data_scaled = StandardScaler().fit_transform(data)
+    _clust_logger.info('[CLUSTER_SCALE] Done. shape=%s', data_scaled.shape)
+
+    # ── Correlation filtering ──────────────────────────────────────────
+    if data_scaled.shape[1] > 1:
+        corr = np.corrcoef(data_scaled.T)
+        upper = np.triu(np.abs(corr), 1)
+        to_drop = [j for j in range(upper.shape[1]) if any(upper[:, j] > 0.95)]
+        if to_drop:
+            keep = [j for j in range(data_scaled.shape[1]) if j not in to_drop]
+            data_scaled = data_scaled[:, keep]
+            feature_cols = [feature_cols[j] for j in keep]
+
+    # ── SVD dimensionality reduction (if >50 features) ─────────────────
+    n_features = data_scaled.shape[1]
+    if n_features > MAX_CLUSTER_FEATURES:
+        from sklearn.decomposition import TruncatedSVD
+        svd = TruncatedSVD(n_components=MAX_CLUSTER_FEATURES, random_state=random_state)
+        data_scaled = svd.fit_transform(data_scaled)
+
+    # ── Hyperparameter auto-tuning ─────────────────────────────────────
+    n = len(data_scaled)
+    if algorithm == 'hdbscan':
+        min_cluster = p.get('min_cluster_size', max(3, min(MAX_CLUSTER_FEATURES, n // 20)))
+        p['min_cluster_size'] = min_cluster
+        p['min_samples'] = p.get('min_samples', min_cluster)
+        p['cluster_selection_epsilon'] = p.get('cluster_selection_epsilon', 0.0)
+    else:
+        p['min_cluster_size'] = min(
+            p.get('min_cluster_size', 5),
+            max(n // 2, 2),
+        )
+
+    # ── Model fit ──────────────────────────────────────────────────────
+    from sklearn.cluster import HDBSCAN, MiniBatchKMeans, AgglomerativeClustering
+    if algorithm == 'kmeans':
+        n_clusters_param = p.get('n_clusters', 3)
+        model = MiniBatchKMeans(
+            n_clusters=n_clusters_param, random_state=random_state,
+            batch_size=1024, n_init='auto',
+        )
+    elif algorithm in ('agglomerative', 'ward'):
+        n_clusters_param = p.get('n_clusters', 5)
+        model = AgglomerativeClustering(
+            n_clusters=n_clusters_param,
+            metric=p.get('metric', 'euclidean'),
+            linkage=p.get('linkage', 'ward'),
+            distance_threshold=None,
+        )
+    else:
+        model = HDBSCAN(
+            min_cluster_size=p['min_cluster_size'],
+            min_samples=p.get('min_samples', None),
+            metric=p.get('metric', 'euclidean'),
+            cluster_selection_epsilon=p.get('cluster_selection_epsilon', 0.0),
+        )
+
+    labels_array = model.fit_predict(data_scaled)
+    n_clusters_found = int(len(set(labels_array)) - (1 if -1 in labels_array else 0))
+    noise_count = int((labels_array == -1).sum()) if algorithm == 'hdbscan' else 0
+    noise_ratio = round(noise_count / len(labels_array), 4) if algorithm == 'hdbscan' else 0.0
+
+    # ── Quality metrics ────────────────────────────────────────────────
+    quality = {
+        'n_clusters': n_clusters_found,
+        'n_noise': noise_count,
+        'noise_ratio': noise_ratio,
+        'algorithm': algorithm,
+    }
+
+    if n_clusters_found > 1:
+        try:
+            from sklearn.metrics import silhouette_score
+            score = silhouette_score(data_scaled, labels_array, random_state=random_state)
+            quality['silhouette_score'] = round(float(score), 4)
+        except Exception:
+            logger.error("unknown failed: {}".format(traceback.format_exc()))
+            quality['silhouette_score'] = None
+        try:
+            from sklearn.metrics import davies_bouldin_score
+            dbs = davies_bouldin_score(data_scaled, labels_array)
+            quality['davies_bouldin_score'] = round(float(dbs), 4)
+        except Exception:
+            logger.error("unknown failed: {}".format(traceback.format_exc()))
+            quality['davies_bouldin_score'] = None
+        try:
+            from sklearn.metrics import calinski_harabasz_score
+import traceback
+            chs = calinski_harabasz_score(data_scaled, labels_array)
+            quality['calinski_harabasz_score'] = round(float(chs), 4)
+        except Exception:
+            logger.error("unknown failed: {}".format(traceback.format_exc()))
+            quality['calinski_harabasz_score'] = None
+
+    # Cluster sizes
+    cluster_sizes = {}
+    for label in np.unique(labels_array):
+        cluster_sizes[int(label)] = int((labels_array == label).sum())
+    quality['cluster_sizes'] = cluster_sizes
+
+    return labels_array, n_clusters_found, noise_count, quality, _ds_idx
+
+
+# ═══════════════════════════════════════════════════════════════════════
+#  _handle_run_clustering
+# ═══════════════════════════════════════════════════════════════════════
+
+async def _handle_run_clustering(
+    dataset_id: str,
+    cluster_columns: list[str],
+    algorithm: str = 'agglomerative',
+    params: Optional[dict] = None,
+    random_state: int = RANDOM_SEED,
+) -> dict:
+    """Run HDBSCAN, KMeans, or Agglomerative clustering on selected columns."""
+    entry, err = _resolve_dataset(dataset_id)
+    if err:
+        return err
+
+    lf: pl.LazyFrame = entry['lazyframe']
+    p = params or {}
+
+    # Collect cluster features - numeric only
+    available = lf.collect_schema()
+    available_names = available.names()
+    available_dtypes = available.dtypes()
+    feature_cols = [
+        c for c in cluster_columns
+        if c in available_names
+        and available_dtypes[available_names.index(c)] in NUMERIC_DTYPES
+    ]
+    if not feature_cols:
+        # Fall back: auto-select all numeric columns (exclude internal ones)
+        feature_cols = [
+            available_names[i] for i, dt in enumerate(available_dtypes)
+            if dt in NUMERIC_DTYPES and not available_names[i].startswith('_')
+        ][:10]
+    if not feature_cols:
+        # Fallback: try coercing Utf8 columns to Float64
+        utf8_cols = [
+            available_names[i] for i, dt in enumerate(available_dtypes)
+            if dt == pl.Utf8 and not available_names[i].startswith('_')
+        ][:10]
+        if utf8_cols:
+            for col in utf8_cols:
+                try:
+                    casted = lf.select(
+                        pl.col(col).cast(pl.Float64, strict=False).alias(col)
+                    ).collect(streaming=True)
+                    null_ratio = casted[col].is_null().sum() / max(len(casted), 1)
+                    if null_ratio < 0.9:
+                        lf = lf.with_columns(
+                            pl.col(col).cast(pl.Float64, strict=False)
+                        )
+                        feature_cols.append(col)
+                except Exception:
+                    logger.error("unknown failed: {}".format(traceback.format_exc()))
+                    pass
+        if not feature_cols:
+            all_numeric = [f"{available_names[i]}" for i, dt in enumerate(available_dtypes)
+                           if dt in NUMERIC_DTYPES]
+            return {
+                'error': f'No numeric cluster columns found in dataset. Available: {all_numeric}',
+                'truncated': False
+            }
+
+    # Downsample before collect
+    row_count = _count_rows(lf)
+    if row_count > LOW_MEMORY_THRESHOLD_ROWS:
+        lf = lf.collect(streaming=True).sample(n=LOW_MEMORY_THRESHOLD_ROWS, seed=RANDOM_SEED).lazy()
+
+    df_features = lf.select(feature_cols).collect(streaming=True)
+
+    # Drop all-NaN columns
+    for col in df_features.columns:
+        if df_features[col].is_null().all():
+            df_features = df_features.drop(col)
+            feature_cols = [c for c in feature_cols if c != col]
+
+    if df_features.width == 0:
+        return {'error': 'No valid feature columns after dropping all-NaN columns',
+                'truncated': False}
+
+    data = df_features.to_numpy()
+
+    # Fill remaining NaN with column mean
+    if data.dtype.kind == 'f':
+        col_mean = np.nanmean(data, axis=0)
+        col_mean = np.nan_to_num(col_mean, nan=0.0)
+        data = np.where(np.isnan(data), col_mean, data)
+
+    # Zero-sample guard
+    if len(data) < 3:
+        return {
+            'cluster_result_id': _generate_id('clust'),
+            'n_clusters': 0,
+            'n_noise': 0,
+            'quality_metrics': {'note': 'too few samples'},
+            'truncated': False,
+        }
+
+    labels_array, n_found, noise_count, quality, _ds_idx = _cluster_core(
+        data, feature_cols, algorithm, p, random_state,
+    )
+
+
+    result_id = _generate_id('clust')
+    store = SessionStore()
+    store.store_cluster_result(
+        result_id=result_id,
+        labels=labels_array.tolist(),
+        n_clusters=n_found,
+        params={
+            'algorithm': algorithm,
+            'params': p,
+            'random_state': random_state,
+            'feature_columns': feature_cols,
+            '_ds_idx': _ds_idx.tolist() if _ds_idx is not None else None,
+        },
+        parent_dataset_id=dataset_id,
+    )
+
+    return {
+        'cluster_result_id': result_id,
+        'n_clusters': n_found,
+        'n_noise': noise_count,
+        'labels_sample': labels_array[:50].tolist(),
+        'quality_metrics': quality,
+    }
+
+
+# ═══════════════════════════════════════════════════════════════════════
+#  _handle_filter_and_cluster
+# ═══════════════════════════════════════════════════════════════════════
+
+async def _handle_filter_and_cluster(
+    dataset_id: str,
+    filters: list[dict],
+    logic: str = 'and',
+    algorithm: str = 'hdbscan',
+    params: dict | None = None,
+) -> dict:
+    """Apply column-level filters then run clustering on the filtered result."""
+    entry, err = _resolve_dataset(dataset_id)
+    if err:
+        return err
+
+    lf: pl.LazyFrame = entry['lazyframe']
+    full_schema = entry.get('schema', {})
+
+    # ── Step 1: Apply filters ─────────────────────────────────────────────
+    exprs = [_build_filter_expr(
+        f['column'], f.get('op', 'eq'), f.get('value'),
+    ) for f in filters]
+
+    if exprs:
+        if logic == 'and':
+            combined = exprs[0]
+            for e in exprs[1:]:
+                combined = combined & e
+        else:
+            combined = exprs[0]
+            for e in exprs[1:]:
+                combined = combined | e
+        filtered_lf = lf.filter(combined)
+    else:
+        filtered_lf = lf
+
+    try:
+        row_count = _count_rows(filtered_lf)
+    except Exception:
+        logger.error("unknown failed: {}".format(traceback.format_exc()))
+        row_count = None
+
+
+    # ── Step 2: Auto-detect numeric feature columns ───────────────────────
+    available = filtered_lf.collect_schema()
+    available_names = available.names()
+    available_dtypes = available.dtypes()
+    feature_cols = [
+        available_names[i] for i, dt in enumerate(available_dtypes)
+        if dt in NUMERIC_DTYPES and not available_names[i].startswith('_')
+    ][:10]
+
+    if not feature_cols:
+        return {
+            'error': 'No numeric columns found for clustering after filtering',
+            'filtered_dataset_id': None,
+            'truncated': False,
+        }
+
+    # ── Step 3: Store filtered dataset ────────────────────────────────────
+    filtered_dataset_id = _generate_id('flt')
+    store = SessionStore()
+    store.store_dataset(
+        dataset_id=filtered_dataset_id,
+        lazyframe=filtered_lf,
+        schema=full_schema,
+        metadata={
+            'row_count': row_count,
+            'parent_dataset_id': dataset_id,
+            'filter_count': len(filters),
+            'logic': logic,
+        },
+        parent_id=dataset_id,
+    )
+
+    # ── Step 4: Run clustering ────────────────────────────────────────────
+    p = params or {}
+    total_count = _count_rows(filtered_lf)
+    clamp_lf = filtered_lf
+    if total_count > LOW_MEMORY_THRESHOLD_ROWS:
+        clamp_lf = filtered_lf.collect(streaming=True).sample(n=LOW_MEMORY_THRESHOLD_ROWS, seed=RANDOM_SEED).lazy()
+
+    df_features = clamp_lf.select(feature_cols).collect(streaming=True)
+
+    for col in df_features.columns:
+        if df_features[col].is_null().all():
+            df_features = df_features.drop(col)
+            feature_cols = [c for c in feature_cols if c != col]
+
+    if df_features.width == 0:
+        return {'error': 'No valid feature columns after dropping all-NaN columns',
+                'filtered_dataset_id': filtered_dataset_id, 'truncated': False}
+
+    data = df_features.to_numpy()
+
+    if data.dtype.kind == 'f':
+        col_mean = np.nanmean(data, axis=0)
+        col_mean = np.nan_to_num(col_mean, nan=0.0)
+        data = np.where(np.isnan(data), col_mean, data)
+
+    if len(data) < 3:
+        return {
+            'cluster_result_id': _generate_id('clust'),
+            'n_clusters': 0,
+            'n_noise': 0,
+            'quality_metrics': {'note': 'too few samples'},
+            'filtered_dataset_id': filtered_dataset_id,
+            'truncated': False,
+        }
+
+    labels_array, n_found, noise_count, quality, _ds_idx = _cluster_core(
+        data, feature_cols, algorithm, p, RANDOM_SEED,
+    )
+
+
+    result_id = _generate_id('clust')
+    store.store_cluster_result(
+        result_id=result_id,
+        labels=labels_array.tolist(),
+        n_clusters=n_found,
+        params={
+            'algorithm': algorithm,
+            'params': p,
+            'random_state': RANDOM_SEED,
+            'feature_columns': feature_cols,
+            '_ds_idx': _ds_idx.tolist() if _ds_idx is not None else None,
+        },
+        parent_dataset_id=filtered_dataset_id,
+    )
+
+    return {
+        'cluster_result_id': result_id,
+        'filtered_dataset_id': filtered_dataset_id,
+        'n_clusters': n_found,
+        'n_noise': noise_count,
+        'labels_sample': labels_array[:50].tolist(),
+        'quality_metrics': quality,
+        'feature_columns': feature_cols,
+    }
@@ -0,0 +1,36 @@
+"""Data management handlers — list, drop, clone datasets."""
+from typing import Optional
+
+from ..session_store import SessionStore
+
+
+async def _handle_list_datasets(
+    _filter_type: Optional[str] = None,
+) -> dict:
+    """List all active datasets and results."""
+    store = SessionStore()
+    datasets = store.list_datasets()
+    return {'datasets': datasets, 'count': len(datasets)}
+
+
+async def _handle_drop_dataset(dataset_id: str) -> dict:
+    """Remove a dataset/result and release memory."""
+    store = SessionStore()
+
+    dropped = (store.drop_dataset(dataset_id)
+               or store.drop_cluster_result(dataset_id)
+               or store.drop_feature_result(dataset_id))
+
+    if dropped:
+        return {'dropped': True, 'dataset_id': dataset_id}
+    return {'error': f'No dataset or result found: {dataset_id}',
+            'truncated': False}
+
+
+async def _handle_clone_dataset(dataset_id: str) -> dict:
+    """Shallow-clone a dataset."""
+    store = SessionStore()
+    new_id = store.clone_dataset(dataset_id)
+    if new_id is None:
+        return {'error': f'Dataset not found: {dataset_id}', 'truncated': False}
+    return {'original_dataset_id': dataset_id, 'cloned_dataset_id': new_id}
@@ -0,0 +1,253 @@
+"""Diagnostic tool handlers — validate, explore, diagnose, compare, repair."""
+import traceback
+from analysis.constants import LOW_MEMORY_THRESHOLD_ROWS, RANDOM_SEED
+import polars as pl
+
+from ..session_store import SessionStore
+from ._helpers import _resolve_dataset, _count_rows, NUMERIC_TYPE_NAMES
+import logging
+logger = logging.getLogger(__name__)
+
+
+async def _handle_validate_data(dataset_id: str, fix_issues: bool = False) -> dict:
+    """Check data quality: schema consistency, missing values, type conflicts."""
+    entry, err = _resolve_dataset(dataset_id)
+    if err:
+        return err
+    lf = entry['lazyframe']
+    schema = entry.get('schema', {})
+    issues = []
+    df_sample = lf.head(1000).collect(streaming=True)
+    for col in schema:
+        null_count = df_sample[col].is_null().sum()
+        if null_count > 500:
+            issues.append(f'{col}: {null_count/10:.0f}% null')
+        try:
+            unique = df_sample[col].n_unique()
+            if unique <= 1 and null_count == 0:
+                issues.append(f'{col}: constant value (no variation)')
+        except Exception:
+            logger.error("_handle_validate_data failed: {}".format(traceback.format_exc()))
+            pass
+    total_rows = _count_rows(lf)
+    return {
+        'status': 'ok',
+        'total_rows': total_rows,
+        'columns': len(schema),
+        'issues_found': len(issues),
+        'issues': issues[:20],
+        'recommendation': 'Try explore_distributions for detailed column analysis' if issues else 'No issues found',
+    }
+
+
+async def _handle_explore_distributions(dataset_id: str, columns: list[str] = None,
+                                        max_sample: int = 10000) -> dict:
+    """Per-column distribution statistics."""
+    entry, err = _resolve_dataset(dataset_id)
+    if err:
+        return err
+    lf = entry['lazyframe']
+    schema = entry.get('schema', {})
+
+    if not columns:
+        columns = [c for c in schema if schema[c].split('(')[0].strip() in NUMERIC_TYPE_NAMES][:10]
+    if len(columns) > 20:
+        columns = columns[:20]
+
+    try:
+        n = _count_rows(lf)
+        if n > max_sample:
+            sample_lf = lf.select(columns).sample(n=max_sample, seed=RANDOM_SEED)
+        else:
+            sample_lf = lf.select(columns)
+        df = sample_lf.collect(streaming=True)
+    except Exception as exc:
+        return {'error': f'explore_distributions failed: {exc}', 'truncated': False}
+
+    stats = {}
+    for col in columns:
+        try:
+            s = df[col]
+            stats[col] = {
+                'dtype': str(s.dtype),
+                'null_pct': round(float(s.is_null().mean() * 100), 2),
+                'unique': int(s.n_unique()),
+                'top_values': [str(v) for v in s.head(5).to_list()],
+            }
+            if s.dtype in (pl.Float32, pl.Float64, pl.Int32, pl.Int64):
+                stats[col]['min'] = float(s.min()) if s.min() is not None else None
+                stats[col]['max'] = float(s.max()) if s.max() is not None else None
+                stats[col]['mean'] = round(float(s.mean()), 4) if s.mean() is not None else None
+                stats[col]['std'] = round(float(s.std()), 4) if s.std() is not None else None
+        except Exception:
+            logger.error("unknown failed: {}".format(traceback.format_exc()))
+            stats[col] = {'error': 'failed to compute'}
+    return {'columns': stats, 'total_rows': n, 'sampled': min(n, max_sample)}
+
+
+async def _handle_find_outliers(dataset_id: str, columns: list[str] = None,
+                                threshold: float = 3.0) -> dict:
+    """Find statistical outliers using IQR method."""
+    entry, err = _resolve_dataset(dataset_id)
+    if err:
+        return err
+    lf = entry['lazyframe']
+    schema = entry.get('schema', {})
+    if not columns:
+        columns = [c for c in schema if schema[c].split('(')[0].strip() in NUMERIC_TYPE_NAMES]
+    n = _count_rows(lf)
+    sample_lf = lf if n <= 100000 else lf.collect(streaming=True).sample(n=LOW_MEMORY_THRESHOLD_ROWS, seed=RANDOM_SEED).lazy()
+    df = sample_lf.select(columns).collect(streaming=True)
+
+    outlier_counts = {}
+    for col in columns:
+        try:
+            s = df[col]
+            q1, q3 = s.quantile(0.25), s.quantile(0.75)
+            iqr = q3 - q1
+            lo = q1 - threshold * iqr
+            hi = q3 + threshold * iqr
+            n_out = ((s < lo) | (s > hi)).sum()
+            if n_out > 0:
+                outlier_counts[col] = int(n_out)
+        except Exception:
+            logger.error("_handle_find_outliers failed: {}".format(traceback.format_exc()))
+            pass
+    return {'outlier_counts': outlier_counts, 'total_sampled': len(df), 'threshold_iqr': threshold}
+
+
+async def _handle_diagnose_clustering(dataset_id: str, cluster_result_id: str) -> dict:
+    """Diagnose clustering quality issues."""
+    from sklearn.decomposition import TruncatedSVD
+    store = SessionStore()
+    entry = store.get_dataset(dataset_id)
+    cluster_entry = store.get_cluster_result(cluster_result_id)
+    if entry is None:
+        return {'error': f'Dataset not found: {dataset_id}', 'truncated': False}
+    if cluster_entry is None:
+        return {'error': f'Cluster result not found: {cluster_result_id}', 'truncated': False}
+    lf = entry['lazyframe']
+    schema = entry.get('schema', {})
+    num_cols = [c for c in schema if schema[c].split('(')[0].strip() in NUMERIC_TYPE_NAMES and not c.startswith('_')]
+
+    n_clusters = cluster_entry.get('n_clusters', 0)
+    n_noise = cluster_entry.get('n_noise', 0)
+    quality = cluster_entry.get('quality_metrics', {})
+    labels = cluster_entry.get('labels', [])
+    params = cluster_entry.get('params', {})
+    algo = params.get('algorithm', 'unknown')
+    feature_cols = params.get('feature_columns', [])
+
+    diagnosis = []
+    if n_clusters <= 1:
+        diagnosis.append('Only 1 cluster — data may lack structure or need feature scaling')
+    if n_noise > 0.5 * len(labels) if labels else False:
+        diagnosis.append(f'High noise ratio ({n_noise}/{len(labels)}) — try HDBSCAN with larger min_cluster_size')
+    sil = quality.get('silhouette_score')
+    if sil and sil < 0.2:
+        diagnosis.append(f'Low silhouette ({sil:.3f}) — clusters overlap, try KMeans or feature selection')
+    if len(feature_cols) < 3:
+        diagnosis.append(f'Only {len(feature_cols)} features — add more columns for meaningful clusters')
+
+    try:
+        df_sample = lf.select(num_cols).head(5000).collect(streaming=True)
+        mat = df_sample.to_numpy()
+        n_comp = min(mat.shape)
+        svd = TruncatedSVD(n_components=n_comp, random_state=RANDOM_SEED).fit(mat)
+        var_explained = svd.explained_variance_ratio_
+        if var_explained[0] > 0.95:
+            diagnosis.append(f'First SVD component explains {var_explained[0]:.0%} variance — data nearly 1D')
+    except Exception:
+        logger.error("unknown failed: {}".format(traceback.format_exc()))
+        pass
+
+    return {
+        'diagnosis': diagnosis,
+        'n_clusters': n_clusters,
+        'n_noise': n_noise,
+        'silhouette': sil,
+        'algorithm': algo,
+        'n_features': len(feature_cols),
+        'suggestions': [
+            f'Try algorithm={"kmeans" if algo == "hdbscan" else "hdbscan"}',
+            'Increase min_cluster_size for fewer, larger clusters',
+            'Use explore_distributions to check feature quality',
+        ],
+    }
+
+
+async def _handle_compare_datasets(dataset_id_a: str, dataset_id_b: str) -> dict:
+    """Compare two datasets."""
+    store = SessionStore()
+    entry_a, entry_b = store.get_dataset(dataset_id_a), store.get_dataset(dataset_id_b)
+    if entry_a is None or entry_b is None:
+        return {'error': 'One or both datasets not found', 'truncated': False}
+
+    def _get_info(entry):
+        schema = entry.get('schema', {})
+        lf = entry['lazyframe']
+        try:
+            n = _count_rows(lf)
+        except Exception:
+            logger.error("_get_info failed: {}".format(traceback.format_exc()))
+            n = -1
+        return {'schema': set(schema.keys()), 'row_count': n, 'columns': len(schema)}
+
+    info_a, info_b = _get_info(entry_a), _get_info(entry_b)
+    common_cols = info_a['schema'] & info_b['schema']
+    only_a = info_a['schema'] - info_b['schema']
+    only_b = info_b['schema'] - info_a['schema']
+    return {
+        'dataset_a': {'id': dataset_id_a, **info_a},
+        'dataset_b': {'id': dataset_id_b, **info_b},
+        'common_columns': len(common_cols),
+        'columns_only_in_a': list(only_a)[:10] if only_a else [],
+        'columns_only_in_b': list(only_b)[:10] if only_b else [],
+        'row_diff': info_a['row_count'] - info_b['row_count'],
+        'identical_schema': len(only_a) == 0 and len(only_b) == 0,
+    }
+
+
+async def _handle_export_debug_sample(dataset_id: str, max_rows: int = 50) -> dict:
+    """Export raw data sample as JSON for debugging."""
+    entry, err = _resolve_dataset(dataset_id)
+    if err:
+        return err
+    df = entry['lazyframe'].head(max_rows).collect(streaming=True)
+    rows = df.to_dict(as_series=False)
+    result = []
+    for i in range(len(df)):
+        row = {}
+        for col in df.columns:
+            val = rows[col][i]
+            if isinstance(val, (float, int, str, bool)):
+                row[col] = val
+            elif val is None:
+                row[col] = None
+            else:
+                row[col] = str(val)
+        result.append(row)
+    return {'rows': result, 'count': len(result), 'columns': df.columns}
+
+
+async def _handle_repair_schema(dataset_id: str) -> dict:
+    """Attempt to repair schema mismatches by normalising column names."""
+    entry, err = _resolve_dataset(dataset_id)
+    if err:
+        return err
+    lf = entry['lazyframe']
+    schema = entry.get('schema', {})
+    col_map = {}
+    for c in schema:
+        norm = c.lower().replace('-', '_').replace('.', '_')
+        col_map[c] = norm
+    if len(set(col_map.values())) < len(col_map):
+        return {'status': 'conflict', 'message': 'Normalised names conflict', 'column_map': col_map}
+    renames = {old: new for old, new in col_map.items() if old != new}
+    if renames:
+        lf = lf.rename(renames)
+        new_schema = {col_map.get(c, c): dtype for c, dtype in schema.items()}
+        store = SessionStore()
+        store.store_dataset(dataset_id, lf, schema=new_schema, metadata=entry.get('metadata', {}))
+        return {'status': 'repaired', 'renamed': len(renames), 'column_map': col_map}
+    return {'status': 'no_change', 'message': 'Column names already normalised'}
@@ -0,0 +1,146 @@
+"""compute_distance_matrix handler — LLM-driven per-row evaluation."""
+import traceback
+from analysis.constants import RANDOM_SEED
+from typing import Optional
+
+import numpy as np
+import polars as pl
+
+from ..session_store import SessionStore
+from ._helpers import _resolve_dataset, NUMERIC_DTYPES
+import logging
+logger = logging.getLogger(__name__)
+
+
+def _indent_function_body(body: str, indent: str = '    ') -> str:
+    """Indent each non-empty line of a code block for wrapping in a def."""
+    return '\n'.join(
+        indent + line if line.strip() else line
+        for line in body.split('\n')
+    )
+
+
+async def _handle_compute_distance_matrix(
+    dataset_id: str,
+    python_function: str,
+    columns: Optional[list[str]] = None,
+    max_sample: int = 10000,
+) -> dict:
+    """Evaluate a user-provided Python function on each row of a dataset."""
+    entry, err = _resolve_dataset(dataset_id)
+    if err:
+        return err
+
+    lf: pl.LazyFrame = entry['lazyframe']
+    df = lf.collect(streaming=True)
+
+    if columns is None:
+        columns = [c for c in df.columns
+                   if df[c].dtype in NUMERIC_DTYPES]
+
+    available = set(df.columns)
+    columns = [c for c in columns if c in available]
+    if not columns:
+        return {'error': 'No valid columns found in dataset',
+                'truncated': False}
+
+    n_total = len(df)
+
+    if n_total > max_sample:
+        df_sampled = df.sample(n=max_sample, seed=RANDOM_SEED)
+        n_evaluated = max_sample
+    else:
+        df_sampled = df
+        n_evaluated = n_total
+
+    rows = df_sampled.select(columns).to_dicts()
+
+    safe_builtins = {
+        'abs': abs, 'len': len, 'range': range,
+        'min': min, 'max': max, 'sum': sum, 'round': round,
+        'int': int, 'float': float, 'str': str, 'bool': bool,
+        'list': list, 'dict': dict, 'tuple': tuple,
+        'isinstance': isinstance, 'type': type,
+        'True': True, 'False': False, 'None': None,
+        'enumerate': enumerate, 'zip': zip, 'map': map,
+        'filter': filter, 'sorted': sorted, 'reversed': reversed,
+    }
+
+    safe_ns = {
+        '__builtins__': safe_builtins,
+        'math': __import__('math'),
+        'np': __import__('numpy'),
+    }
+
+    body = python_function.strip()
+    if body.startswith('def '):
+        full_code = body
+    else:
+        indented = _indent_function_body(body)
+        full_code = f'def distance_fn(row):\n{indented}'
+
+    try:
+        exec(full_code, safe_ns)
+        distance_fn = safe_ns.get('distance_fn')
+        if distance_fn is None:
+            for _name, _obj in safe_ns.items():
+                if callable(_obj) and _name not in ('math', 'np', '__builtins__'):
+                    distance_fn = _obj
+                    break
+        if distance_fn is None:
+            return {'error': 'Could not locate the compiled function',
+                    'truncated': False}
+    except SyntaxError as e:
+        return {'error': f'Function syntax error: {e}', 'truncated': False}
+    except Exception as e:
+        return {'error': f'Function compilation failed: {e}',
+                'truncated': False}
+
+    scores: list[float | None] = []
+    errors = 0
+    for row in rows:
+        try:
+            score = distance_fn(row)
+            if isinstance(score, (int, float)):
+                scores.append(float(score))
+            else:
+                scores.append(None)
+                errors += 1
+        except Exception:
+            logger.error("unknown failed: {}".format(traceback.format_exc()))
+            scores.append(None)
+            errors += 1
+
+    valid_scores = [s for s in scores if s is not None]
+    if not valid_scores:
+        return {'error': 'No valid scores computed — all rows raised errors',
+                'truncated': False}
+
+    arr = np.array(valid_scores)
+    summary = {
+        'min': float(np.min(arr)),
+        'max': float(np.max(arr)),
+        'mean': float(np.mean(arr)),
+        'std': float(np.std(arr)),
+        'count': len(arr),
+        'errors': errors,
+        'total_rows_in_sample': len(rows),
+        'matrix_shape': [len(rows), len(columns)],
+    }
+
+    scored_sample = []
+    for i, row in enumerate(rows[:30]):
+        if i < len(scores):
+            scored_sample.append({
+                'index': i,
+                'values': {c: row.get(c) for c in columns[:3]},
+                'score': scores[i],
+            })
+
+    return {
+        'summary': summary,
+        'scored_sample': scored_sample,
+        'columns_used': columns,
+        'total_rows_in_dataset': n_total,
+        'rows_evaluated': n_evaluated,
+    }
@@ -0,0 +1,256 @@
+"""Entity profile building + adaptive scoring handlers."""
+import traceback
+from analysis.constants import MAX_DISTRIBUTION_SAMPLE, RANDOM_SEED
+import logging
+from typing import Optional
+
+import numpy as np
+import polars as pl
+
+from ..session_store import SessionStore, _generate_id
+from ..profile_util import profile
+from ._helpers import _resolve_dataset, _count_rows, NUMERIC_DTYPES
+logger = logging.getLogger(__name__)
+
+
+# ═══════════════════════════════════════════════════════════════════════
+#  Constants
+# ═══════════════════════════════════════════════════════════════════════
+
+_ENTITY_KEYWORDS = [
+    'src_ip', 'dst_ip', 'source_ip', 'destination_ip', 'src_addr', 'dst_addr',
+    'sip', 'dip', 'ip_src', 'ip_dst', 'saddr', 'daddr',
+    'sni', 'host', 'domain', 'uri', 'url', 'fqdn', 'server_name',
+    'mac', 'mac_addr', 'src_mac', 'dst_mac',
+    'user_agent', 'ua', 'asn', 'organization', 'org',
+]
+
+_NUMERIC_TYPES_FOR_AGG = (pl.Int8, pl.Int16, pl.Int32, pl.Int64,
+                           pl.UInt8, pl.UInt16, pl.UInt32, pl.UInt64,
+                           pl.Float32, pl.Float64)
+
+_ANOMALY_FEATURE_COLS = [
+    'non_std_port_rate', 'modern_tls_rate', 'sni_missing_ratio',
+    'recoverable_ratio', 'flow_count', 'total_bytes_sent',
+    'total_packets', 'unique_dst_ips', 'unique_dst_ports',
+    'unique_protocols', 'has_sni', 'avg_sni_length',
+    'countries_visited',
+]
+
+
+# ═══════════════════════════════════════════════════════════════════════
+#  _handle_build_entity_profiles
+# ═══════════════════════════════════════════════════════════════════════
+
+async def _handle_build_entity_profiles(
+    dataset_id: str,
+    entity_column: Optional[str] = None,
+    entity_columns: Optional[list[str]] = None,
+    auto_detect: bool = True,
+) -> dict:
+    """Group raw flow data by entity column(s) and compute per-entity aggregate features."""
+    entry, err = _resolve_dataset(dataset_id)
+    if err:
+        return err
+
+    lf: pl.LazyFrame = entry['lazyframe']
+    schema = entry.get('schema', {})
+    col_names = list(schema.keys())
+
+    # ── Resolve entity columns ──────────────────────────────────────────
+    if entity_column and entity_column in col_names:
+        group_cols = [entity_column]
+    elif entity_columns:
+        group_cols = [c for c in entity_columns if c in col_names]
+        if not group_cols:
+            return {'error': f'None of entity_columns {entity_columns} exist in dataset. Available: {col_names[:20]}',
+                    'truncated': False}
+    elif auto_detect:
+        matched = []
+        for kw in _ENTITY_KEYWORDS:
+            found = [c for c in col_names if kw.lower() in c.lower()]
+            matched.extend(found)
+        seen = set()
+        group_cols = [c for c in matched if not (c in seen or seen.add(c))]
+        if not group_cols:
+            for c in col_names:
+                if c.startswith('_'):
+                    continue
+                group_cols = [c]
+                break
+        group_cols = group_cols[:3]
+    else:
+        return {'error': 'No entity column specified and auto_detect is disabled',
+                'truncated': False}
+
+    if not group_cols:
+        return {'error': 'Could not determine entity column(s) for grouping',
+                'truncated': False}
+
+    # ── Build aggregation expressions ──────────────────────────────────
+    agg_exprs: list[pl.Expr] = [
+        pl.len().alias('flow_count'),
+        pl.col(group_cols[0]).n_unique().alias(f'unique_{group_cols[0]}'),
+    ]
+
+    numeric_cols = [c for c in col_names if c not in group_cols
+                    and schema[c].startswith(('Int', 'UInt', 'Float'))]
+    for c in numeric_cols[:20]:
+        agg_exprs.append(pl.sum(c).alias(f'total_{c}'))
+        agg_exprs.append(pl.mean(c).alias(f'avg_{c}'))
+
+    str_cols = [c for c in col_names if c not in group_cols
+                and schema[c].startswith(('Utf8', 'String', 'Cat'))]
+    for c in str_cols[:10]:
+        agg_exprs.append(pl.col(c).n_unique().alias(f'unique_{c}'))
+
+    for kw in ['dst_ip', 'dip', 'dst_addr', 'daddr', 'ip_dst', 'dest_ip',
+               'dst_port', 'dport', 'destination_port', 'dest_port',
+               'port', 'protocol', 'proto', 'sni', 'server_name',
+               'cipher_suite', 'cipher', 'tls_version', 'version', '0ver',
+               'country', 'scnt', 'dcnt', 'asn', 'organization']:
+        found = [c for c in col_names if c not in group_cols and kw.lower() in c.lower()]
+        for c in found:
+            alias_name = f'unique_{c}'
+            if alias_name not in [str(a.meta.output_name()) if hasattr(a, 'meta') else '' for a in agg_exprs]:
+                agg_exprs.append(pl.col(c).n_unique().alias(alias_name))
+
+    # ── Execute grouping ───────────────────────────────────────────────
+    entity_lf = lf.group_by(group_cols).agg(agg_exprs)
+    n_entities = _count_rows(entity_lf)
+
+    # ── Store result ───────────────────────────────────────────────────
+    new_schema = {name: str(dtype) for name, dtype in
+                  zip(entity_lf.collect_schema().names(), entity_lf.collect_schema().dtypes())}
+    new_id = f'entity_{_generate_id("ds")}'
+    store = SessionStore()
+    store.store_dataset(
+        dataset_id=new_id,
+        lazyframe=entity_lf,
+        schema=new_schema,
+        metadata={
+            'row_count': n_entities,
+            'entity_columns': group_cols,
+            'parent_dataset_id': dataset_id,
+            'n_agg_features': len(agg_exprs),
+        },
+        parent_id=dataset_id,
+    )
+
+    return {
+        'dataset_id': new_id,
+        'entity_columns': group_cols,
+        'n_entities': n_entities,
+        'n_features': len(agg_exprs),
+        'aggregated_columns': list(new_schema.keys())[:20],
+    }
+
+
+# ═══════════════════════════════════════════════════════════════════════
+#  Adaptive scoring helpers
+# ═══════════════════════════════════════════════════════════════════════
+
+@profile
+def _add_adaptive_scores(lf: pl.LazyFrame, feature_names: list[str]) -> pl.LazyFrame:
+    """Add anomaly scores using data-driven weights (SVD + IQR + percentile)."""
+    _agg_cols = lf.collect_schema().names()
+    avail = [c for c in _ANOMALY_FEATURE_COLS if c in _agg_cols]
+    if not avail:
+        return lf
+
+    try:
+        n_rows = _count_rows(lf)
+        sample_lf = lf if n_rows <= MAX_DISTRIBUTION_SAMPLE else lf.sample(n=MAX_DISTRIBUTION_SAMPLE, seed=RANDOM_SEED)
+        df_sample = sample_lf.select(avail).collect(streaming=True)
+        mat = df_sample.to_numpy()
+        mat = np.nan_to_num(mat, nan=0.0)
+
+        from sklearn.decomposition import TruncatedSVD
+        n_comp = min(1, min(mat.shape) - 1)
+        if n_comp < 1 or mat.shape[1] < 2:
+            return lf
+        svd = TruncatedSVD(n_components=1, random_state=RANDOM_SEED).fit(mat)
+        raw_weights = np.abs(svd.components_[0])
+        weights = dict(zip(avail, raw_weights / raw_weights.sum()))
+
+        expr_parts = []
+        for col, w in weights.items():
+            if col == 'modern_tls_rate':
+                expr_parts.append((1.0 - pl.col(col).fill_null(0.5)) * w)
+            else:
+                expr_parts.append(pl.col(col).fill_null(0.0) * w)
+        if expr_parts:
+            proxy_expr = expr_parts[0]
+            for p in expr_parts[1:]:
+                proxy_expr = proxy_expr + p
+        else:
+            return lf
+
+        lf = lf.with_columns(proxy_expr.clip(0, 1).alias('_raw_score'))
+
+        sample_scores = df_sample.select(
+            pl.sum_horizontal([
+                (1.0 - pl.col('modern_tls_rate').fill_null(0.5)) * weights.get('modern_tls_rate', 0)
+                if c == 'modern_tls_rate'
+                else pl.col(c).fill_null(0.0) * weights.get(c, 0)
+                for c in avail
+            ]).clip(0, 1)
+        ).to_series().to_numpy()
+
+        q25, q50, q75 = np.percentile(sample_scores, [25, 50, 75])
+        iqr = q75 - q25
+        upper_fence = float(q75 + 1.5 * iqr) if iqr > 0 else 0.9
+        extreme_fence = float(q75 + 3.0 * iqr) if iqr > 0 else 0.99
+
+        lf = lf.with_columns([
+            pl.col('_raw_score').alias('proxy_score'),
+            pl.when(pl.col('_raw_score') >= extreme_fence)
+            .then(pl.lit('critical'))
+            .when(pl.col('_raw_score') >= upper_fence)
+            .then(pl.lit('suspicious'))
+            .when(pl.col('_raw_score') >= q50)
+            .then(pl.lit('watch'))
+            .otherwise(pl.lit('normal'))
+            .alias('risk_level'),
+            (pl.col('_raw_score') * 0.6 + 0.4 * (pl.col('_raw_score') > q75).cast(pl.Float64))
+            .alias('threat_score'),
+        ]).drop('_raw_score')
+
+        valid_counts = {c: int((mat[:, i] != 0).sum()) for i, c in enumerate(avail)}
+        logger = logging.getLogger(__name__)
+        logger.info(
+            '[SCORE] adaptive weights from SVD: %s | thresholds: p50=%.3f upper=%.3f extreme=%.3f | sample=%d/%d',
+            {c: f'{w:.3f}' for c, w in sorted(weights.items(), key=lambda x: -x[1])},
+            float(q50), upper_fence, extreme_fence,
+            min(n_rows, MAX_DISTRIBUTION_SAMPLE), n_rows,
+        )
+
+    except Exception as exc:
+        logging.getLogger(__name__).warning('[SCORE] adaptive scoring skipped: %s', exc)
+
+    return lf
+
+
+# ═══════════════════════════════════════════════════════════════════════
+#  _handle_compute_scores
+# ═══════════════════════════════════════════════════════════════════════
+
+async def _handle_compute_scores(dataset_id: str) -> dict:
+    """Compute adaptive proxy/anomaly/threat scores on entity data."""
+    entry, err = _resolve_dataset(dataset_id)
+    if err:
+        return err
+
+    lf = entry['lazyframe']
+    lf = _add_adaptive_scores(lf, [])
+    try:
+        updated_schema = lf.collect_schema()
+        new_schema = {name: str(dtype) for name, dtype in
+                      zip(updated_schema.names(), updated_schema.dtypes())}
+    except Exception:
+        logger.error("_handle_compute_scores failed: {}".format(traceback.format_exc()))
+        new_schema = entry.get('schema', {})
+    store = SessionStore()
+    store.store_dataset(dataset_id, lf, schema=new_schema,
+                        metadata=entry.get('metadata', {}))
+    return {'status': 'scored', 'dataset_id': dataset_id}
@@ -0,0 +1,76 @@
+"""evaluate_clustering handler — compute clustering quality metrics."""
+from typing import Optional
+from analysis.constants import RANDOM_SEED
+
+import numpy as np
+import polars as pl
+
+from ..session_store import SessionStore
+
+
+async def _handle_evaluate_clustering(
+    cluster_result_id: str,
+    dataset_id: str,
+    metrics: Optional[list[str]] = None,
+) -> dict:
+    """Evaluate clustering result against the dataset."""
+    store = SessionStore()
+    cluster_entry = store.get_cluster_result(cluster_result_id)
+    if cluster_entry is None:
+        return {'error': f'Cluster result not found: {cluster_result_id}',
+                'truncated': False}
+
+    dataset_entry = store.get_dataset(dataset_id)
+    if dataset_entry is None:
+        return {'error': f'Dataset not found: {dataset_id}', 'truncated': False}
+
+    labels = cluster_entry['labels']
+    default_metrics = ['silhouette', 'davies_bouldin', 'calinski_harabasz',
+                       'noise_ratio', 'cluster_sizes']
+    requested = metrics or default_metrics
+
+    scores: dict = {'n_clusters': cluster_entry['n_clusters']}
+
+    lf: pl.LazyFrame = dataset_entry['lazyframe']
+    params = cluster_entry.get('params', {})
+    feature_cols = params.get('feature_columns', [])
+    if not feature_cols and 'feature_columns' in params:
+        pass
+    elif feature_cols:
+        df_features = lf.select(feature_cols).collect(streaming=True)
+        data = df_features.to_numpy()
+
+        # Align labels with data (handle missing rows)
+        valid_mask = ~np.isnan(data).any(axis=1) if data.dtype.kind == 'f' else np.ones(len(data), dtype=bool)
+        data_clean = data[valid_mask]
+        labels_clean = np.array(labels)[:len(data_clean)]
+
+        unique_labels = set(labels_clean)
+        n_clusters_actual = len(unique_labels - {-1})
+
+        for m in requested:
+            if m == 'silhouette' and n_clusters_actual >= 2:
+                from sklearn.metrics import silhouette_score
+                scores['silhouette'] = round(float(
+                    silhouette_score(data_clean, labels_clean, random_state=RANDOM_SEED)), 4)
+            elif m == 'davies_bouldin' and n_clusters_actual >= 2:
+                from sklearn.metrics import davies_bouldin_score
+                scores['davies_bouldin'] = round(float(
+                    davies_bouldin_score(data_clean, labels_clean)), 4)
+            elif m == 'calinski_harabasz' and n_clusters_actual >= 2:
+                from sklearn.metrics import calinski_harabasz_score
+                scores['calinski_harabasz'] = round(float(
+                    calinski_harabasz_score(data_clean, labels_clean)), 4)
+            elif m == 'noise_ratio':
+                noise = int((labels_clean == -1).sum())
+                scores['noise_ratio'] = round(noise / len(labels_clean), 4)
+            elif m == 'cluster_sizes':
+                sizes = {}
+                for label in unique_labels:
+                    sizes[int(label)] = int((labels_clean == label).sum())
+                scores['cluster_sizes'] = sizes
+
+    scores['n_clusters'] = cluster_entry['n_clusters']
+
+
+    return {'scores': scores}
@@ -0,0 +1,91 @@
+"""export_results handler — write datasets/results to disk."""
+import json
+from pathlib import Path
+
+import polars as pl
+
+from ..session_store import SessionStore
+
+
+async def _handle_export_results(
+    result_id: str,
+    output_path: str,
+    format: str = 'csv',
+    overwrite: bool = False,
+) -> dict:
+    """Export a dataset (or result) to disk."""
+    store = SessionStore()
+
+    out = Path(output_path)
+    file_paths: list[str] = []
+
+    # Check if it's a dataset
+    dataset_entry = store.get_dataset(result_id)
+    if dataset_entry is not None:
+        lf: pl.LazyFrame = dataset_entry['lazyframe']
+        df = lf.collect(streaming=True)
+        if out.suffix == '':
+            out = out / f"export_{result_id}.{format}"
+
+        if out.exists() and not overwrite:
+            return {'error': f'Output exists and overwrite=False: {out}',
+                    'truncated': False}
+
+        out.parent.mkdir(parents=True, exist_ok=True)
+
+        if format == 'csv':
+            df.write_csv(out)
+        elif format == 'parquet':
+            df.write_parquet(out)
+        elif format == 'json':
+            df.write_json(out)
+        else:
+            return {'error': f'Unsupported format: {format}',
+                    'truncated': False}
+
+        file_paths.append(str(out.resolve()))
+
+        return {'file_paths': file_paths, 'format': format,
+                'row_count': len(df)}
+
+    # Check if it's a cluster result
+    cluster_entry = store.get_cluster_result(result_id)
+    if cluster_entry is not None:
+        if out.suffix == '':
+            out = out / f"cluster_{result_id}.json"
+        out.parent.mkdir(parents=True, exist_ok=True)
+
+        if out.exists() and not overwrite:
+            return {'error': f'Output exists and overwrite=False: {out}',
+                    'truncated': False}
+
+        export_data = {
+            'result_id': result_id,
+            'n_clusters': cluster_entry['n_clusters'],
+            'params': cluster_entry['params'],
+            'labels_sample': cluster_entry['labels'][:1000],
+        }
+        with open(out, 'w', encoding='utf-8') as f:
+            json.dump(export_data, f, default=str)
+        file_paths.append(str(out.resolve()))
+
+        return {'file_paths': file_paths, 'format': 'json'}
+
+    # Check feature result
+    feat_entry = store.get_feature_result(result_id)
+    if feat_entry is not None:
+        if out.suffix == '':
+            out = out / f"features_{result_id}.json"
+        out.parent.mkdir(parents=True, exist_ok=True)
+
+        if out.exists() and not overwrite:
+            return {'error': f'Output exists and overwrite=False: {out}',
+                    'truncated': False}
+
+        with open(out, 'w', encoding='utf-8') as f:
+            json.dump(feat_entry['features'], f, default=str)
+        file_paths.append(str(out.resolve()))
+        return {'file_paths': file_paths, 'format': 'json'}
+
+    return {'error': f'No dataset or result found for ID: {result_id}',
+            'truncated': False}
@@ -0,0 +1,314 @@
+"""extract_features handler — per-cluster feature ranking + ORM persistence."""
+import sys
+import traceback
+
+import numpy as np
+import polars as pl
+
+from ..session_store import SessionStore, _generate_id
+from ..profile_util import profile
+from analysis.constants import RANDOM_SEED, UMAP_BATCH_SIZE, UMAP_TRAIN_SAMPLE
+import logging
+logger = logging.getLogger(__name__)
+
+
+async def _handle_extract_features(
+    dataset_id: str,
+    cluster_result_id: str,
+    top_k: int = 10,
+    method: str = 'zscore',
+    save_to_db: bool = True,
+    run_id: int | None = None,
+) -> dict:
+    """Compute per-cluster distinguishing features and optionally persist."""
+    store = SessionStore()
+    dataset_entry = store.get_dataset(dataset_id)
+    if dataset_entry is None:
+        return {'error': f'Dataset not found: {dataset_id}', 'truncated': False}
+
+    cluster_entry = store.get_cluster_result(cluster_result_id)
+    if cluster_entry is None:
+        return {'error': f'Cluster result not found: {cluster_result_id}',
+                'truncated': False}
+
+    lf: pl.LazyFrame = dataset_entry['lazyframe']
+    labels = cluster_entry['labels']
+
+    # D5: If clustering downsampled the data, filter matching rows
+    _ds_idx = cluster_entry.get('params', {}).get('_ds_idx')
+    if _ds_idx is not None:
+        df = (lf.with_row_index('__ds_idx')
+              .filter(pl.col('__ds_idx').is_in(_ds_idx))
+              .drop('__ds_idx')
+              .collect(streaming=True))
+    else:
+        df = lf.collect(streaming=True)
+
+    # Truncate labels if df was filtered but labels weren't
+    if len(labels) != len(df):
+        labels = labels[:len(df)]
+
+    # Only use numeric columns for feature extraction
+    numeric_cols = [c for c in df.columns
+                    if df[c].dtype in (pl.Float32, pl.Float64,
+                                        pl.Int32, pl.Int64,
+                                        pl.UInt32, pl.UInt64)]
+    if len(labels) != len(df):
+        labels = labels[:len(df)]
+
+    # Global statistics per column
+    global_stats = {}
+    for col in numeric_cols:
+        s = df[col].drop_nulls()
+        if len(s) > 0:
+            global_stats[col] = {
+                'mean': float(s.mean()),
+                'std': float(s.std()) if s.std() is not None else 1.0,
+            }
+
+    unique_labels = sorted(set(labels))
+    all_features = []
+
+    for label in unique_labels:
+        mask = np.array(labels) == label
+        if mask.sum() < 2:
+            continue
+
+        cluster_df = df.filter(pl.Series('__mask__', mask))
+        cluster_features = []
+
+        for col in numeric_cols:
+            c_series = cluster_df[col].drop_nulls()
+            gs = global_stats.get(col)
+            if gs is None or gs['std'] == 0 or len(c_series) == 0:
+                continue
+
+            c_mean = float(c_series.mean())
+            c_std = float(c_series.std()) if c_series.std() is not None else 0.0
+
+            if method == 'zscore':
+                score = (c_mean - gs['mean']) / max(gs['std'], 1e-10)
+            else:
+                score = abs(c_mean - gs['mean']) / max(c_std + gs['std'], 1e-10)
+
+            cluster_features.append({
+                'feature_name': col,
+                'cluster_label': int(label),
+                'mean': round(c_mean, 4),
+                'std': round(c_std, 4),
+                'global_mean': round(gs['mean'], 4),
+                'global_std': round(gs['std'], 4),
+                'distinguishing_score': round(float(score), 4),
+                'distinguishing_method': method,
+            })
+
+        cluster_features.sort(key=lambda x: abs(x['distinguishing_score']), reverse=True)
+        all_features.extend(cluster_features[:top_k])
+
+    # ── Compute UMAP-2D embedding ──────────────────────────────────
+    if len(numeric_cols) >= 2 and len(df) > 2:
+        try:
+            from sklearn.preprocessing import StandardScaler
+            import umap
+            MAX_UMAP_TRAIN = UMAP_TRAIN_SAMPLE
+            BATCH_SIZE = UMAP_BATCH_SIZE
+            data_matrix = df.select(numeric_cols).to_numpy()
+            if len(df) > MAX_UMAP_TRAIN:
+                idx_sample = np.random.RandomState(RANDOM_SEED).choice(
+                    len(df), size=MAX_UMAP_TRAIN, replace=False)
+                mat_sample = data_matrix[idx_sample]
+                scaler = StandardScaler().fit(mat_sample)
+                mat_sample_scaled = np.nan_to_num(scaler.transform(mat_sample),
+                                                  nan=0.0, posinf=0.0, neginf=0.0)
+                reducer = umap.UMAP(n_components=2, random_state=RANDOM_SEED,
+                                    n_neighbors=15, min_dist=0.1,
+                                    metric='euclidean')
+                reducer.fit(mat_sample_scaled)
+                coords_list = []
+                for start in range(0, len(df), BATCH_SIZE):
+                    end = min(start + BATCH_SIZE, len(df))
+                    mat_batch = scaler.transform(data_matrix[start:end])
+                    mat_batch = np.nan_to_num(mat_batch, nan=0.0, posinf=0.0, neginf=0.0)
+                    coords_list.append(reducer.transform(mat_batch))
+                coords = np.vstack(coords_list)
+            else:
+                data_scaled = StandardScaler().fit_transform(data_matrix)
+                data_scaled = np.nan_to_num(data_scaled, nan=0.0, posinf=0.0, neginf=0.0)
+                reducer = umap.UMAP(n_components=2, random_state=RANDOM_SEED,
+                                    n_neighbors=15, min_dist=0.1,
+                                    metric='euclidean')
+                coords = reducer.fit_transform(data_scaled)
+            non_numeric = [c for c in df.columns if c not in numeric_cols and not c.startswith('_')]
+            entity_col_umap = non_numeric[0] if non_numeric else df.columns[0]
+            labels_list = cluster_entry.get('labels', [])
+            from analysis.models import EntityProfile as EP
+            from analysis.models import ClusterResult as CR
+            from analysis.models import AnalysisRun as AR
+            from asgiref.sync import sync_to_async
+            csv_glob = dataset_entry.get('metadata', {}).get('csv_glob', 'manual')
+            run_obj = await sync_to_async(
+                AR.objects.filter(csv_glob=csv_glob).order_by('-id').first,
+                thread_sensitive=True
+            )()
+            if run_obj:
+                run_id_for_ep = run_obj.id
+                profiles = []
+                cr_cache = {}
+                for i, row in enumerate(df.iter_rows(named=True)):
+                    if i >= len(coords): break
+                    ev = str(row.get(entity_col_umap, ''))
+                    if not ev: continue
+                    lbl = labels_list[i] if i < len(labels_list) else -1
+                    cache_key = f'{run_id_for_ep}_{lbl}'
+                    if cache_key not in cr_cache:
+                        cr_cache[cache_key] = CR.objects.filter(run_id=run_id_for_ep, cluster_label=lbl).first()
+                    profiles.append(EP(
+                        entity_value=ev, run_id=run_id_for_ep, cluster_label=lbl,
+                        cluster=cr_cache[cache_key],
+                        embedding_x=float(coords[i, 0]),
+                        embedding_y=float(coords[i, 1]),
+                    ))
+                if profiles:
+                    EP.objects.bulk_create(profiles, ignore_conflicts=True, batch_size=1000)
+        except Exception:
+            print(f'[UMAP_ERR] {traceback.format_exc()}', file=sys.stderr)
+
+    # Persist to Django ORM
+    db_saved = False
+    if save_to_db:
+        try:
+            from asgiref.sync import sync_to_async
+            from analysis.models import AnalysisRun as AnalysisRunModel
+            ce = cluster_entry
+            async def _do_save(ce_inner):
+                if run_id is not None:
+                    run = await sync_to_async(AnalysisRunModel.objects.get)(id=run_id)
+                else:
+                    csv_glob_fallback = dataset_entry.get('metadata', {}).get('csv_glob', 'manual')
+                    run = await sync_to_async(
+                        AnalysisRunModel.objects.filter(csv_glob=csv_glob_fallback).order_by('-id').first,
+                        thread_sensitive=True
+                    )()
+                    if run is None:
+                        run = await sync_to_async(AnalysisRunModel.objects.create)(
+                            csv_glob=csv_glob_fallback, status='running', total_flows=0,
+                        )
+                run.status = 'completed'
+                run.cluster_count = ce_inner['n_clusters']
+                await sync_to_async(run.save)()
+                return await sync_to_async(_save_features_to_db)(cluster_result_id, all_features, run_id=run.id)
+            db_saved = await _do_save(ce)
+        except Exception as e:
+            print(f'[DB_SAVE_ERROR] {e}\n{traceback.format_exc()}', file=sys.stderr)
+            db_saved = False
+
+
+    result_id = _generate_id('feat')
+    store.store_feature_result(
+        result_id=result_id,
+        features=all_features,
+        parent_cluster_result_id=cluster_result_id,
+    )
+
+    return {
+        'feature_result_id': result_id,
+        'features': all_features,
+        'n_features': len(all_features),
+        'db_saved': db_saved,
+    }
+
+
+@profile
+def _save_features_to_db(
+    cluster_result_id: str,
+    features: list[dict],
+    run_id: int | None = None,
+) -> bool:
+    """Save extracted features to the Django ORM ClusterFeature model."""
+    if run_id is not None:
+        from analysis.models import AnalysisRun as AnalysisRun2
+        run = AnalysisRun2.objects.filter(id=run_id).first()
+        if run is None:
+            return False
+    else:
+        store = SessionStore()
+        cluster_entry = store.get_cluster_result(cluster_result_id)
+        if cluster_entry is None:
+            return False
+
+        parent_ds_id = cluster_entry.get('parent_dataset_id', '')
+        from analysis.models import AnalysisRun
+        ds_entry = store.get_dataset(parent_ds_id)
+        if ds_entry is None:
+            return False
+
+        csv_glob = ds_entry.get('metadata', {}).get('csv_glob', '')
+        unique_global = sorted(set(f['cluster_label'] for f in features))
+        n_clusters_global = len([l for l in unique_global if l >= 0])
+        try:
+            run = AnalysisRun.objects.filter(csv_glob=csv_glob).order_by('-created_at').first()
+            if run is None:
+                run = AnalysisRun.objects.create(
+                    csv_glob=csv_glob,
+                    status='completed',
+                    cluster_count=n_clusters_global,
+                )
+        except Exception:
+            logger.error("unknown failed: {}".format(traceback.format_exc()))
+            return False
+
+    from analysis.models import ClusterResult as ClusterResultModel
+
+    store = SessionStore()
+    cluster_entry = store.get_cluster_result(cluster_result_id)
+    label_counts = {}
+    if cluster_entry:
+        full_labels = cluster_entry.get('labels', [])
+        for lbl in full_labels:
+            label_counts[lbl] = label_counts.get(lbl, 0) + 1
+
+    unique_labels = sorted(set(
+        f['cluster_label'] for f in features
+    ))
+    n_clusters = len([l for l in unique_labels if l >= 0])
+    for label in unique_labels:
+        if label == -1:
+            continue
+        label_features = [f for f in features if f['cluster_label'] == label]
+        if not label_features:
+            continue
+
+        cluster_db, _ = ClusterResultModel.objects.get_or_create(
+            run=run,
+            cluster_label=label,
+            defaults={'size': label_counts.get(label, 1)},
+        )
+        if label_counts.get(label) is not None:
+            ClusterResultModel.objects.filter(
+                run=run, cluster_label=label
+            ).update(size=label_counts[label])
+
+        from analysis.models import ClusterFeature as ClusterFeatureModel
+        for feat in label_features:
+            try:
+                ClusterFeatureModel.objects.get_or_create(
+                    cluster=cluster_db,
+                    feature_name=feat['feature_name'],
+                    defaults={
+                        'mean': feat.get('mean'),
+                        'std': feat.get('std'),
+                        'median': None,
+                        'p25': None,
+                        'p75': None,
+                        'missing_rate': None,
+                        'distinguishing_score': feat.get('distinguishing_score'),
+                        'distinguishing_method': feat.get('distinguishing_method', 'zscore'),
+                    }
+                )
+            except Exception:
+                logger.error("unknown failed: {}".format(traceback.format_exc()))
+                continue
+
+    run.cluster_count = n_clusters
+    run.save(update_fields=['cluster_count'])
+    return True
@@ -0,0 +1,68 @@
+"""filter_data handler — apply column-level filters to a dataset."""
+import traceback
+import polars as pl
+
+from ..session_store import SessionStore, _generate_id
+from ._helpers import _build_filter_expr, _resolve_dataset
+import logging
+logger = logging.getLogger(__name__)
+
+
+async def _handle_filter_data(
+    dataset_id: str,
+    filters: list[dict],
+    logic: str = 'and',
+) -> dict:
+    """Apply filters and store filtered dataset."""
+    entry, err = _resolve_dataset(dataset_id)
+    if err:
+        return err
+
+    lf: pl.LazyFrame = entry['lazyframe']
+    full_schema = entry.get('schema', {})
+
+    exprs = [_build_filter_expr(
+        f['column'], f.get('op', 'eq'), f.get('value'),
+    ) for f in filters]
+
+    if not exprs:
+        return {'error': 'No filter expressions could be built', 'truncated': False}
+
+    if logic == 'and':
+        combined = exprs[0]
+        for e in exprs[1:]:
+            combined = combined & e
+    else:
+        combined = exprs[0]
+        for e in exprs[1:]:
+            combined = combined | e
+
+    filtered_lf = lf.filter(combined)
+
+    # Estimate row count (fast streaming count)
+    try:
+        row_count = filtered_lf.select(pl.len()).collect(streaming=True).item()
+    except Exception:
+        logger.error("unknown failed: {}".format(traceback.format_exc()))
+        row_count = None
+
+    new_id = _generate_id('flt')
+    store = SessionStore()
+    store.store_dataset(
+        dataset_id=new_id,
+        lazyframe=filtered_lf,
+        schema=full_schema,
+        metadata={
+            'row_count': row_count,
+            'parent_dataset_id': dataset_id,
+            'filter_count': len(filters),
+            'logic': logic,
+        },
+        parent_id=dataset_id,
+    )
+
+    return {
+        'filtered_dataset_id': new_id,
+        'row_count': row_count,
+        'column_deltas': {'kept': list(full_schema.keys()), 'dropped': []},
+    }
@@ -0,0 +1,58 @@
+"""load_data handler — load CSV files and store as dataset."""
+import os
+from pathlib import Path
+from typing import Optional
+
+import polars as pl
+
+from ..data_loader import load_csv_directory
+from ..session_store import SessionStore, _generate_id
+from analysis.constants import MAX_DISTRIBUTION_SAMPLE
+
+
+async def _handle_load_data(
+    csv_glob: str,
+    config_path: Optional[str] = None,
+    encoding: str = 'utf-8',
+    delimiter: str = ',',
+    schema_strict: bool = False,
+    recursive: bool = False,
+) -> dict:
+    """Load, validate, merge CSVs, and store in session."""
+    try:
+        lf, schema, row_count, file_count, memory_mb = load_csv_directory(
+            glob_pattern=csv_glob,
+            encoding=encoding,
+            delimiter=delimiter,
+            config_path=config_path,
+            sample_rows=MAX_DISTRIBUTION_SAMPLE,
+            schema_strict=schema_strict,
+            recursive=recursive,
+        )
+    except FileNotFoundError as e:
+        return {'error': f'File not found: {e}', 'truncated': False}
+    except ValueError as e:
+        return {'error': f'Schema validation failed: {e}', 'truncated': False}
+
+    store = SessionStore()
+    dataset_id = _generate_id('ds')
+    store.store_dataset(
+        dataset_id=dataset_id,
+        lazyframe=lf,
+        schema=schema,
+        metadata={
+            'row_count': row_count,
+            'file_count': file_count,
+            'memory_mb': round(memory_mb, 2),
+            'csv_glob': csv_glob,
+            'encoding': encoding,
+        },
+    )
+
+    return {
+        'dataset_id': dataset_id,
+        'schema': schema,
+        'row_count': row_count,
+        'file_count': file_count,
+        'memory_mb': round(memory_mb, 2),
+    }
@@ -0,0 +1,107 @@
+"""preprocess_data handler — scale, encode, impute, drop columns."""
+from typing import Optional
+
+import polars as pl
+
+from ..session_store import SessionStore, _generate_id
+from ._helpers import _resolve_dataset
+
+
+async def _handle_preprocess_data(
+    dataset_id: str,
+    columns: list[str],
+    config: Optional[dict] = None,
+) -> dict:
+    """Preprocess selected columns (scale, encode, impute, drop)."""
+    entry, err = _resolve_dataset(dataset_id)
+    if err:
+        return err
+
+    lf: pl.LazyFrame = entry['lazyframe']
+    cfg = config or {}
+    column_mapping: dict[str, str] = {}
+
+    df = lf.collect(streaming=True)
+
+    # Resolve columns that actually exist
+    available = set(df.columns)
+    target_cols = [c for c in columns if c in available]
+    if not target_cols:
+        return {'error': 'None of the specified columns exist in the dataset',
+                'truncated': False}
+
+    # Drop columns
+    drop_cols = cfg.get('drop', [])
+    if drop_cols:
+        df = df.drop([c for c in drop_cols if c in df.columns])
+        for c in drop_cols:
+            if c in available:
+                column_mapping[c] = f'<dropped:{c}>'
+
+    # Fill missing values
+    fill_strategy = cfg.get('fillna')
+    if fill_strategy and fill_strategy != 'drop':
+        for col in target_cols:
+            if col not in df.columns:
+                continue
+            if df[col].null_count() == 0:
+                continue
+            if fill_strategy == 'mean':
+                mean_val = df[col].drop_nulls().mean()
+                if mean_val is not None:
+                    df = df.with_columns(pl.col(col).fill_null(mean_val))
+            elif fill_strategy == 'median':
+                med_val = df[col].drop_nulls().median()
+                if med_val is not None:
+                    df = df.with_columns(pl.col(col).fill_null(med_val))
+            elif fill_strategy == 'zero':
+                df = df.with_columns(pl.col(col).fill_null(0))
+    elif fill_strategy == 'drop':
+        df = df.drop_nulls(subset=target_cols)
+
+    # Standard scaling (z-score)
+    if cfg.get('standardize'):
+        from sklearn.preprocessing import StandardScaler
+        num_cols = [c for c in target_cols if c in df.columns
+                    and df[c].dtype in (pl.Float32, pl.Float64, pl.Int32, pl.Int64)]
+        if num_cols:
+            scaler = StandardScaler()
+            scaled = scaler.fit_transform(df.select(num_cols).to_numpy())
+            for i, col in enumerate(num_cols):
+                col_scaled = f'{col}_scaled'
+                df = df.with_columns(pl.Series(col_scaled, scaled[:, i]))
+                column_mapping[col] = col_scaled
+
+    # One-hot encoding
+    if cfg.get('onehot'):
+        str_cols = [c for c in target_cols if c in df.columns
+                    and df[c].dtype in (pl.Utf8, pl.String, pl.Categorical)]
+        if str_cols:
+            df = df.to_dummies(columns=str_cols, drop_first=False)
+            for c in str_cols:
+                column_mapping[c] = f'<onehot:{c}>'
+
+    new_id = _generate_id('pp')
+    new_lf = df.lazy()
+    new_schema = {name: str(dtype) for name, dtype in
+                  zip(new_lf.collect_schema().names(), new_lf.collect_schema().dtypes())}
+
+    store = SessionStore()
+    store.store_dataset(
+        dataset_id=new_id,
+        lazyframe=new_lf,
+        schema=new_schema,
+        metadata={
+            'row_count': len(df),
+            'parent_dataset_id': dataset_id,
+            'preprocessing_config': cfg,
+        },
+        parent_id=dataset_id,
+    )
+
+    return {
+        'preprocessed_dataset_id': new_id,
+        'column_mapping': column_mapping,
+        'row_count': len(df),
+        'column_count': len(new_schema),
+    }
@@ -0,0 +1,22 @@
+"""profile_data handler — compute per-column statistics and correlations."""
+from typing import Optional
+
+import polars as pl
+
+from ..data_profiler import profile_dataset as _profile_dataset
+from ._helpers import _resolve_dataset
+
+
+async def _handle_profile_data(
+    dataset_id: str,
+    sample_size: int = 1000,
+    columns: Optional[list[str]] = None,
+) -> dict:
+    """Profile dataset columns and correlations."""
+    entry, err = _resolve_dataset(dataset_id)
+    if err:
+        return err
+
+    lf: pl.LazyFrame = entry['lazyframe']
+    profile = _profile_dataset(lf, sample_size=sample_size, columns=columns)
+    return profile
@@ -15,7 +15,7 @@ urlpatterns = [
    path('runs/<int:display_id>/', views.run_detail, name='run_detail'),
    path('runs/<int:display_id>/status/', views.run_status_api, name='run_status'),
    path('clusters/<int:display_id>/', views.cluster_overview, name='cluster_overview'),
-    path('clusters/<int:display_id>/<int:cluster_label>/', views.cluster_detail, name='cluster_detail'),
+    path('clusters/<int:display_id>/<path:cluster_label>/', views.cluster_detail, name='cluster_detail'),
    path('entities/<int:entity_id>/', views.entity_profile, name='entity_profile'),
    path('config/', views.config_view, name='config'),
    path('api/llm/test/', views.llm_test, name='llm_test'),
@@ -25,6 +25,7 @@ urlpatterns = [
    path('logs/', views.log_viewer, name='log_viewer'),
    path('globe/', views.globe_view, name='globe'),
    path('tools/run/', views.tool_lab_run, name='tool_lab_run'),
+    path('tools/reload-run/', views.reload_run_data, name='reload_run_data'),
    path('tools/plan/', views.tool_plan, name='tool_plan'),
    path('tools/filter/', views.apply_filter, name='apply_filter'),
 ]
@@ -0,0 +1,13 @@
+"""Analysis views package — re-exports all public view functions."""
+from .helpers import _extract_lat, _extract_lon
+from .dashboard import dashboard, run_list, run_detail, run_status_api, start_analysis
+from .pipeline import _run_pipeline_worker, _background_process
+from .clustering import cluster_overview, cluster_detail, _run_clustering_pipeline, _get_globe_flows
+from .entity import entity_profile
+from .upload import upload_page, upload_csv, upload_csv_batch, finalize_upload, delete_upload
+from .manual import manual_page, manual_run_analysis
+from .auto import auto_page, run_llm_analysis_view
+from .globe import globe_view, _extract_flows_from_df
+from .config import config_view, llm_test
+from .log_viewer import log_viewer
+from .tools import tool_lab, tool_lab_run, tool_plan, apply_filter, reload_run_data, retry_run
@@ -0,0 +1,346 @@
+"""LLM auto analysis views."""
+import json
+import asyncio
+import threading
+import traceback
+import logging
+
+from django.shortcuts import render, get_object_or_404
+from django.http import JsonResponse
+from django.views.decorators.csrf import csrf_exempt
+
+from config import get_config
+from analysis.models import AnalysisRun
+from .pipeline import _run_pipeline_worker
+from .clustering import _run_clustering_pipeline
+from .helpers import _PLANS_DIR, _add_to_auto_index
+
+logger = logging.getLogger(__name__)
+
+
+def auto_page(request):
+    """LLM-driven auto analysis page."""
+    from analysis.session_store import SessionStore
+    import json as _json
+
+    cfg = get_config()
+
+    # Get datasets from SessionStore + ALL runs from DB
+    store = SessionStore()
+    session_datasets = store.list_datasets() if hasattr(store, 'list_datasets') else []
+    session_ds_map = {ds['dataset_id']: ds for ds in session_datasets}
+
+    # Build dataset entries from ALL AnalysisRun records
+    all_runs = AnalysisRun.objects.all().order_by('-created_at')[:100]
+    datasets_with_status = []
+
+    for run in all_runs:
+        if run.run_type == 'upload':
+            ds_id = f'upload_{run.display_id}'
+        else:
+            ds_id = f'run_{run.display_id}'
+
+        session_entry = session_ds_map.get(ds_id)
+
+        if session_entry:
+            datasets_with_status.append({
+                **session_entry,
+                'dataset_id': ds_id,
+                'run_status': run.status,
+                'total_flows': run.total_flows,
+                'display_id': run.display_id,
+                'needs_reload': False,
+                'csv_glob': run.csv_glob or '',
+                'sqlite_table': run.sqlite_table or '',
+            })
+        elif run.csv_glob or run.sqlite_table:
+            datasets_with_status.append({
+                'dataset_id': ds_id,
+                'run_status': run.status,
+                'display_id': run.display_id,
+                'total_flows': run.total_flows,
+                'row_count': run.total_flows,
+                'file_count': None,
+                'columns': [],
+                'column_count': 0,
+                'svd_components': 0,
+                'needs_reload': True,
+                'csv_glob': run.csv_glob or '',
+                'sqlite_table': run.sqlite_table or '',
+            })
+
+    for ds_info in datasets_with_status:
+        s = ds_info['run_status']
+        if s in ('completed',):
+            ds_info['status_label'] = '已分析'
+            ds_info['badge_class'] = 'completed'
+        elif s in ('failed',):
+            ds_info['status_label'] = '错误'
+            ds_info['badge_class'] = 'failed'
+        elif s in ('pending', 'loading', 'profiling', 'aggregating', 'clustering', 'extracting'):
+            ds_info['status_label'] = '分析中'
+            ds_info['badge_class'] = 'analyzing'
+        else:
+            ds_info['status_label'] = '待分析'
+            ds_info['badge_class'] = 'ready'
+
+    return render(request, 'tianxuan/auto.html', {
+        'llm_configured': bool(cfg.llm.base_url and cfg.llm.api_key),
+        'datasets_with_status_json': _json.dumps(datasets_with_status, ensure_ascii=False),
+    })
+
+
+@csrf_exempt
+def run_llm_analysis_view(request):
+    """AJAX POST: run LLM-driven analysis pipeline."""
+    if request.method != 'POST':
+        return JsonResponse({'error': 'POST required'}, status=405)
+    body = json.loads(request.body)
+    display_id = body.get('run_id')
+    run = get_object_or_404(AnalysisRun, display_id=display_id)
+    pk = run.id
+    run.run_type = 'auto'
+    run.save(update_fields=['run_type'])
+    head = body.get('head')
+
+    from analysis.session_store import SessionStore
+    store = SessionStore()
+
+    # Determine dataset key format (consistent with manual_page)
+    upload_ds_id = f'upload_{display_id}'
+    run_ds_id = f'run_{display_id}'
+    entry = store.get_dataset(upload_ds_id)
+    if entry is None:
+        entry = store.get_dataset(run_ds_id)
+
+    if entry is None:
+        # Try primary key fallback
+        upload_pk_id = f'upload_{pk}'
+        entity_pk_id = f'entity_{pk}'
+        entry = store.get_dataset(upload_pk_id)
+        if entry is None:
+            entry = store.get_dataset(entity_pk_id)
+
+    if entry is None:
+        # Auto-reload from sqlite_table or csv_glob
+        from analysis.data_loader import load_csv_directory, load_from_db
+        lf = None
+        schema = {}
+        if run.sqlite_table:
+            try:
+                lf = load_from_db(run.sqlite_table)
+            except Exception:
+                logger.error("unknown failed: {}".format(traceback.format_exc()))
+                pass
+        if lf is None and run.csv_glob:
+            try:
+                lf, schema, _, _, _ = load_csv_directory(run.csv_glob)
+            except Exception:
+                logger.error("unknown failed: {}".format(traceback.format_exc()))
+                pass
+        if lf is None:
+            return JsonResponse({'error': '数据集不存在，请先上传并等待预处理完成'})
+        entry_key = upload_ds_id if run.run_type == 'upload' else run_ds_id
+        store.store_dataset(entry_key, lf, schema=schema, metadata={
+            'row_count': run.total_flows,
+            'csv_glob': run.csv_glob or '',
+            'sqlite_table': run.sqlite_table or '',
+        })
+        # Re-fetch entry
+        entry = store.get_dataset(entry_key)
+
+    # Determine the active dataset_id
+    for candidate in [upload_ds_id, run_ds_id, f'upload_{pk}', f'entity_{pk}']:
+        if store.get_dataset(candidate):
+            dataset_id = candidate
+            break
+    else:
+        return JsonResponse({'error': '数据集不存在，请先上传并等待预处理完成'})
+
+    # ── Optional: apply filter before LLM pipeline ──
+    filters = body.get('filters', [])
+    if filters and len(filters) > 0:
+        logic = body.get('logic', 'and')
+        try:
+            from analysis.tool_registry import _handle_filter_data
+            filter_result = asyncio.run(_handle_filter_data(
+                dataset_id=dataset_id,
+                filters=filters,
+                logic=logic,
+            ))
+            if 'error' not in filter_result:
+                filtered_id = filter_result.get('filtered_dataset_id', '')
+                if filtered_id and store.get_dataset(filtered_id):
+                    dataset_id = filtered_id
+        except Exception:
+            logger.warning(f'[LLM auto] Filter failed (non-fatal): {traceback.format_exc()}')
+
+    def _analysis_fn(run, ctx):
+        from tianxuan.llm_orchestrator import run_llm_pipeline, LLMConfig
+        dataset_id = ctx['dataset_id']
+        pk = ctx['pk']
+
+        def progress_callback(step, tool_name, result=None, meta=None):
+            nonlocal run
+            try:
+                run = AnalysisRun.objects.get(id=pk)
+                pct = min(10 + step * 6, 95)
+                run.progress_pct = pct
+                run.progress_msg = f'步骤 {step}: {tool_name}'
+
+                save_fields = ['progress_pct', 'progress_msg']
+                if tool_name == 'thinking':
+                    # Accumulate LLM thinking text (appended to llm_thinking)
+                    if result:
+                        run.llm_thinking = (run.llm_thinking or '') + f'\n[{step}] {result}'
+                    save_fields = ['progress_pct', 'progress_msg', 'llm_thinking']
+                elif tool_name == 'complete':
+                    pass  # No extra tracking needed
+                else:
+                    # Single tool call — always has result
+                    if isinstance(result, dict):
+                        summary = json.dumps(result, default=str, ensure_ascii=False)[:500]
+                    else:
+                        summary = str(result)[:500]
+                    run.run_log += f'[{step}] {tool_name}: {summary}...\n'
+
+                    # One entry per tool call
+                    if meta and isinstance(meta, dict) and 'args' in meta:
+                        tool_calls = list(run.tool_calls_json or [])
+                        tool_calls.append({
+                            'step': step,
+                            'name': tool_name,
+                            'input': meta['args'],
+                            'output': result,
+                        })
+                        run.tool_calls_json = tool_calls
+                    save_fields = ['progress_pct', 'progress_msg', 'run_log', 'tool_calls_json']
+
+                run.save(update_fields=save_fields)
+            except Exception as e:
+                logger.warning('progress_callback error: %s', e)
+
+        try:
+            from config import get_config
+            cfg = get_config()
+            llm_cfg = LLMConfig(cfg.llm.base_url, cfg.llm.api_key, cfg.llm.model)
+            run.run_log = '启动 LLM 分析...\n'
+            run.status = 'loading'
+            run.progress_pct = 5
+            run.progress_msg = 'LLM 初始化中...'
+            run.save(update_fields=['status', 'progress_pct', 'progress_msg', 'run_log'])
+
+            result = run_llm_pipeline(dataset_id, config=llm_cfg, callback=progress_callback)
+
+            if 'error' in result:
+                run.status = 'failed'
+                run.progress_msg = f'失败: {result["error"]}'
+                run.run_log += f'\n[ERROR] {result["error"]}'
+                run.save(update_fields=['status', 'progress_pct', 'progress_msg', 'run_log'])
+            else:
+                run.progress_pct = 55
+                run.progress_msg = f'LLM分析完成: {result.get("result", "")[:200]}'
+                run.run_log += f'\n[完成] {result.get("result", "")}\n[INFO] 开始自动聚类流程...'
+                run.save(update_fields=['progress_pct', 'progress_msg', 'run_log'])
+
+                # ── Auto-run clustering pipeline after LLM analysis ──
+                entity_ds_id = None
+                tool_calls = run.tool_calls_json or []
+                for tc in tool_calls:
+                    tool_name = tc.get('name', '')
+                    tool_input = tc.get('input', {})
+                    if tool_name in ('build_entity_profiles', 'run_clustering'):
+                        entity_ds_id = tool_input.get('dataset_id', '')
+                        if entity_ds_id and store.get_dataset(entity_ds_id):
+                            break
+                        entity_ds_id = None
+
+                # Fallback: try common dataset ID patterns
+                if not entity_ds_id:
+                    candidates = [
+                        ctx.get('dataset_id', ''),
+                        f'entity_{pk}',
+                        f'entity_{run.display_id}',
+                        f'upload_{run.display_id}',
+                    ]
+                    for cid in candidates:
+                        if cid and store.get_dataset(cid):
+                            entity_ds_id = cid
+                            break
+
+                if entity_ds_id:
+                    _run_clustering_pipeline(
+                        run=run,
+                        store=store,
+                        entity_ds_id=entity_ds_id,
+                        feature_columns=None,
+                        algorithm='agglomerative',
+                        min_cluster_size=5,
+                        run_umap=True,
+                        head=ctx.get('head'),
+                    )
+                else:
+                    logger.warning(
+                        f'[LLM auto] No entity dataset found for clustering. '
+                        f'Tool calls: {[tc.get("name") for tc in tool_calls]}. '
+                        f'Marking run as completed without clustering.'
+                    )
+                    run.status = 'completed'
+                    run.progress_pct = 100
+                    run.progress_msg = 'LLM分析完成（无实体数据，跳过聚类）'
+                    run.save(update_fields=['status', 'progress_pct', 'progress_msg'])
+
+                # ── Auto-save workflow as loadable plan ──
+                try:
+                    tool_calls_data = run.tool_calls_json or []
+                    steps = []
+                    for tc in tool_calls_data:
+                        tc_name = tc.get('name', '')
+                        if tc_name == 'thinking':
+                            continue
+                        steps.append({
+                            'tool_name': tc_name,
+                            'params': tc.get('input', {}),
+                        })
+                    if steps:
+                        plan_data = {
+                            'name': f'_auto_{run.display_id}',
+                            'steps': steps,
+                            'auto': True,
+                            'display_id': run.display_id,
+                            'created_at': str(run.created_at) if run.created_at else '',
+                        }
+                        plan_path = _PLANS_DIR / f'_auto_{run.display_id}.json'
+                        plan_path.write_text(
+                            json.dumps(plan_data, ensure_ascii=False, indent=2),
+                            encoding='utf-8',
+                        )
+                        _add_to_auto_index(f'_auto_{run.display_id}.json')
+                        logger.info(
+                            '[LLM auto] Saved auto plan %s (%d steps)',
+                            plan_path.name, len(steps),
+                        )
+                except Exception as save_err:
+                    logger.warning('[LLM auto] Failed to save auto plan: %s', save_err)
+
+        except Exception:
+            tb = traceback.format_exc()
+            logger.error(tb)
+            try:
+                run = AnalysisRun.objects.get(id=pk)
+                run.status = 'failed'
+                run.progress_msg = f'失败: {tb}'
+                run.run_log += f'\n[ERROR] {tb}'
+                run.error_message = tb
+                run.save(update_fields=['status', 'progress_msg', 'run_log', 'error_message'])
+            except Exception as e:
+                logger.warning('save failed after pipeline error: %s', e)
+
+    t = threading.Thread(
+        target=_run_pipeline_worker,
+        args=(pk, _analysis_fn),
+        kwargs=dict(dataset_id=dataset_id, head=head, pk=pk),
+        daemon=True,
+    )
+    t.start()
+    return JsonResponse({'status': 'started', 'run_id': run.display_id})
@@ -0,0 +1,493 @@
+"""Cluster views: overview, detail, globe flows, and the clustering pipeline."""
+import asyncio
+import json
+import traceback
+import logging
+
+from django.shortcuts import render, get_object_or_404
+from django.http import Http404
+
+from analysis.models import AnalysisRun, ClusterResult, EntityProfile
+from analysis import nl_describe
+from analysis.constants import RANDOM_SEED, UMAP_BATCH_SIZE, UMAP_TRAIN_SAMPLE
+from analysis.profile_util import profile
+
+from .helpers import _extract_lat, _extract_lon
+
+logger = logging.getLogger(__name__)
+
+
+def cluster_overview(request, display_id):
+    """Overview of all clusters for a run, with UMAP scatter data and geo scatter data."""
+    run = get_object_or_404(AnalysisRun, display_id=display_id)
+    clusters = run.clusters.exclude(cluster_label=-1).order_by('-size').prefetch_related('features')
+    noise = run.clusters.filter(cluster_label=-1).first()
+
+    # Build UMAP scatter data for Canvas / Three.js 3D
+    entities = run.entities.exclude(embedding_x=None).exclude(embedding_y=None).only(
+        'embedding_x', 'embedding_y', 'embedding_z', 'cluster_label', 'entity_value'
+    )
+    has_z = any(e.embedding_z is not None for e in entities)
+    scatter_data = [
+        {'x': e.embedding_x, 'y': e.embedding_y, 'z': e.embedding_z if has_z else 0,
+         'label': e.cluster_label if e.cluster_label is not None else -1, 'entity': e.entity_value}
+        for e in entities
+    ]
+
+    # Noise cluster: features + entity detail
+    noise_features = []
+    noise_summary = ''
+    noise_entity_count = 0
+    if noise:
+        noise_features_qs = noise.features.all().order_by('-distinguishing_score')[:15]
+        noise_features = [
+            {'feature_name': f.feature_name, 'score': f.distinguishing_score, 'mean': f.mean,
+             'std': f.std, 'p25': f.p25, 'p75': f.p75}
+            for f in noise_features_qs
+        ]
+        if noise_features:
+            noise_summary = nl_describe.describe_cluster(noise_features)
+        noise_entity_count = noise.size
+
+    # LLM workflow timeline for auto runs
+    llm_timeline_json = ''
+    if run.run_type == 'auto':
+        llm_thinking = run.llm_thinking or ''
+        tool_calls = run.tool_calls_json or []
+        if tool_calls or llm_thinking:
+            llm_timeline_json = json.dumps({
+                'thinking': llm_thinking,
+                'tool_calls': tool_calls,
+            })
+
+    # Build geo scatter data from feature_json (lat/lon)
+    geo_data = []
+    geo_skipped = 0
+    for e in run.entities.all().only('feature_json', 'cluster_label', 'entity_value'):
+        fj = e.feature_json
+        if not fj:
+            continue
+        lat = _extract_lat(fj.get('avg_latitude'))
+        lon = _extract_lon(fj.get('avg_longitude'))
+        if lat is not None and lon is not None:
+            geo_data.append({
+                'x': lon,
+                'y': lat,
+                'label': e.cluster_label if e.cluster_label is not None else -1,
+                'entity': e.entity_value,
+            })
+        else:
+            geo_skipped += 1
+
+    # Cluster sizes for bar chart
+    cluster_sizes = [{'label': c.cluster_label, 'size': c.size, 'silhouette': c.silhouette_score}
+                     for c in clusters]
+
+    # Top features per cluster from ClusterFeature + NL summaries
+    top_features = {}
+    for c in clusters:
+        features_qs = c.features.all().order_by('-distinguishing_score')[:10]
+        feats = [
+            {'feature_name': f.feature_name, 'score': f.distinguishing_score, 'mean': f.mean,
+             'std': f.std}
+            for f in features_qs
+        ]
+        top_features[str(c.cluster_label)] = feats
+        # Attach features + NL summary directly to cluster object for template
+        c._top_features = feats
+        c.nl_summary = nl_describe.describe_cluster(feats) if feats else ''
+
+    return render(request, 'analysis/cluster_overview.html', {
+        'run': run,
+        'clusters': clusters,
+        'noise': noise,
+        'scatter_data_json': json.dumps(scatter_data),
+        'has_z': has_z,
+        'geo_data_json': json.dumps(geo_data),
+        'geo_skipped': geo_skipped,
+        'geo_count': len(geo_data),
+        'cluster_sizes_json': json.dumps(cluster_sizes),
+        'top_features_json': json.dumps(top_features),
+        'noise_features': noise_features,
+        'noise_summary': noise_summary,
+        'noise_entity_count': noise_entity_count,
+        'llm_timeline_json': llm_timeline_json,
+        # Globe flow data for the sidebar
+        'globe_flows_json': json.dumps(_get_globe_flows(run)),
+    })
+
+
+def _get_globe_flows(run, max_rows=500):
+    """Extract flow data for 3D globe visualization from a run."""
+    from analysis.data_loader import load_csv_directory, load_from_db
+    try:
+        lf = None
+        if run.sqlite_table:
+            lf = load_from_db(run.sqlite_table)
+        if lf is None and run.csv_glob:
+            lf, _, _, _, _ = load_csv_directory(run.csv_glob)
+        if lf is None:
+            return []
+        df = lf.head(max_rows).collect()
+        from analysis.type_classifier import TLS_HEX_MAP
+        from analysis.geoip import lookup as geo_lookup
+        flows = []
+        src_ip_col = next((c for c in df.columns if 'src_ip' in c.lower() or ':ips' in c.lower()), None)
+        dst_ip_col = next((c for c in df.columns if 'dst_ip' in c.lower() or ':ipd' in c.lower()), None)
+        tls_col = next((c for c in df.columns if c.lower() in ('0ver', 'tls_version', '_tlsver', 'version')), None)
+        bytes_col = next((c for c in df.columns if c.lower() in ('8ack', '8pak', '8byt', 'bytes_sent', 'bytes')), None)
+        for row in df.iter_rows(named=True):
+            sip = str(row.get(src_ip_col, '')) if src_ip_col else ''
+            dip = str(row.get(dst_ip_col, '')) if dst_ip_col else ''
+            if not sip or not dip:
+                continue
+            sg = geo_lookup(sip)
+            dg = geo_lookup(dip)
+            if not sg or not dg:
+                continue
+            tls_val = str(row.get(tls_col, '') or '') if tls_col else ''
+            tls_ver = TLS_HEX_MAP.get(tls_val.replace(' ', ''), tls_val)
+            flows.append({
+                'slat': sg['lat'], 'slon': sg['lon'],
+                'dlat': dg['lat'], 'dlon': dg['lon'],
+                'tls': tls_ver,
+                'bytes': float(row.get(bytes_col, 0) or 0) if bytes_col else 0,
+            })
+        return flows
+    except Exception:
+        logger.error("unknown failed: {}".format(traceback.format_exc()))
+        return []
+
+
+def cluster_detail(request, display_id, cluster_label):
+    """Detailed view of a single cluster with entity list, SVD features, and NL summary."""
+    run = get_object_or_404(AnalysisRun, display_id=display_id)
+    try:
+        cluster_label = int(cluster_label)
+    except (ValueError, TypeError):
+        raise Http404("Invalid cluster label")
+    cluster = get_object_or_404(ClusterResult, run=run, cluster_label=cluster_label)
+    features = cluster.features.all().order_by('-distinguishing_score')[:50]
+    entities = cluster.entities.all()[:100]
+    nl_summary = ''
+    feats_list = [{'feature_name': f.feature_name, 'score': f.distinguishing_score,
+                   'mean': f.mean, 'std': f.std} for f in features[:10]]
+    if feats_list:
+        nl_summary = nl_describe.describe_cluster(feats_list)
+
+    # Build entity data with feature_json values
+    entity_data = []
+    for e in entities:
+        entry = {'id': e.id, 'value': e.entity_value,
+                 'embedding_x': e.embedding_x, 'embedding_y': e.embedding_y}
+        if e.feature_json:
+            entry['features'] = {k: round(float(v), 4) if isinstance(v, (int, float)) else str(v)
+                                 for k, v in e.feature_json.items()}
+        entity_data.append(entry)
+
+    return render(request, 'analysis/cluster_detail.html', {
+        'run': run,
+        'cluster': cluster,
+        'features': features,
+        'entities': entities,
+        'entity_data_json': json.dumps(entity_data),
+        'nl_summary': nl_summary,
+    })
+
+
+@profile
+def _run_clustering_pipeline(run, store, entity_ds_id, feature_columns, algorithm,
+                              min_cluster_size, run_umap=True, head=None):
+    """Unified clustering pipeline: clustering → feature extraction → UMAP → ORM save.
+
+    Args:
+        run: AnalysisRun ORM object.
+        store: SessionStore instance.
+        entity_ds_id: dataset ID in SessionStore pointing to entity-aggregated data.
+        feature_columns: list of feature column names (None = auto-detect).
+        algorithm: 'hdbscan' or 'kmeans'.
+        min_cluster_size: int for HDBSCAN.
+        run_umap: bool, whether to compute and save UMAP-2D embeddings.
+        head: optional int, downsample to at most this many rows (min 100).
+    """
+    import warnings
+    warnings.filterwarnings('ignore', category=RuntimeWarning, module='sklearn')
+
+    try:
+        from analysis.tool_registry import _handle_run_clustering, _handle_extract_features
+        import polars as pl
+
+        entry = store.get_dataset(entity_ds_id)
+        if entry is None:
+            run.status = 'failed'
+            run.error_message = 'Entity dataset not found in session store'
+            run.save(update_fields=['status', 'error_message'])
+            return
+
+        schema = entry.get('schema', {})
+
+        # ── Downsampling ──
+        if head is not None and head > 0:
+            lf = entry['lazyframe']
+            try:
+                row_count = lf.select(pl.len()).collect(streaming=True).item()
+            except Exception:
+                logger.error("unknown failed: {}".format(traceback.format_exc()))
+                row_count = 0
+
+            if row_count > head:
+                head = max(head, 100)  # ensure minimum rows for clustering
+                run.progress_msg = f'正在采样 {head}/{row_count} 行...'
+                run.save(update_fields=['progress_msg'])
+                lf = lf.head(head)
+                store.store_dataset(
+                    entity_ds_id, lf,
+                    schema=schema,
+                    metadata=entry.get('metadata', {}),
+                )
+                entry = store.get_dataset(entity_ds_id)  # refresh
+                row_count = head
+
+        # Resolve feature columns — use actual LazyFrame dtypes (not stored schema dict
+        # which may be stale after Utf8 coercion in _background_process).
+        numeric_types = (pl.Int8, pl.Int16, pl.Int32, pl.Int64,
+                         pl.UInt8, pl.UInt16, pl.UInt32, pl.UInt64,
+                         pl.Float32, pl.Float64)
+
+        lf = entry['lazyframe']
+        live_schema = lf.collect_schema()
+        live_names = live_schema.names()
+        live_dtypes = list(live_schema.dtypes())
+
+        if feature_columns:
+            feature_cols = [
+                c for c in feature_columns
+                if c in live_names and live_dtypes[live_names.index(c)] in numeric_types
+            ]
+        else:
+            feature_cols = [
+                live_names[i] for i, dt in enumerate(live_dtypes)
+                if dt in numeric_types and not live_names[i].startswith('_')
+            ][:10]
+
+        # ── Step 1: Clustering ────────────────────────────────────────────────
+        # If no numeric columns detected, pass empty list to let
+        # _handle_run_clustering apply its Utf8→Float64 coercion fallback.
+        run.status = 'clustering'
+        run.progress_pct = 70
+        run.progress_msg = '正在进行聚类分析...'
+        run.save(update_fields=['status', 'progress_pct', 'progress_msg'])
+
+        result = asyncio.run(_handle_run_clustering(
+            dataset_id=entity_ds_id,
+            cluster_columns=feature_cols,
+            algorithm=algorithm,
+            params={'min_cluster_size': min_cluster_size},
+            random_state=RANDOM_SEED,
+        ))
+
+        if 'error' in result:
+            err = result['error']
+            if 'numeric' in err.lower():
+                available = [
+                    f"{k}({v})" for k, v in schema.items()
+                    if v.split('(')[0].strip() in numeric_types and not k.startswith('_')
+                ]
+                err += f"。可用数值列: {available}"
+            raise Exception(err)
+
+        cluster_id = result.get('cluster_result_id', '')
+        run.cluster_count = result.get('n_clusters', 0)
+        run.save(update_fields=['cluster_count'])
+
+        # ── Step 2: Feature extraction ──────────────────────────────────────
+        run.status = 'extracting'
+        run.progress_pct = 90
+        run.progress_msg = '正在提取聚类特征...'
+        run.save(update_fields=['status', 'progress_pct', 'progress_msg'])
+
+        feat_result = asyncio.run(_handle_extract_features(
+            dataset_id=entity_ds_id,
+            cluster_result_id=cluster_id,
+            top_k=10, method='zscore', save_to_db=True,
+            run_id=run.id,
+        ))
+
+        if 'error' in feat_result:
+            raise Exception(feat_result['error'])
+
+        # ── Step 2b: Cluster SVD feature extraction ──────────────────────
+        run.progress_msg = '正在进行聚类SVD特征提取...'
+        run.save(update_fields=['progress_msg'])
+        try:
+            from analysis.distance import cluster_svd_extract
+            lf = entry['lazyframe']
+            cluster_entry = store.get_cluster_result(cluster_id)
+            labels_list = cluster_entry.get('labels', []) if cluster_entry else []
+
+            if labels_list and feature_cols:
+                svd_results = cluster_svd_extract(lf, labels_list, feature_cols)
+
+                # Persist SVD features to ClusterFeature (method='cluster_svd')
+                from analysis.models import ClusterFeature as CF
+                from analysis.models import ClusterResult as CR
+                for label, svd_info in svd_results.items():
+                    cr = CR.objects.filter(run=run, cluster_label=label).first()
+                    if cr is None:
+                        continue
+                    # Delete old zscore entries for this cluster, replace with SVD
+                    CF.objects.filter(cluster=cr, distinguishing_method='zscore').delete()
+                    for feat_name in svd_info.get('features', [])[:10]:
+                        idx = svd_info['features'].index(feat_name) if feat_name in svd_info['features'] else -1
+                        strength = (svd_info.get('feature_strength', [])[idx]
+                                    if idx >= 0 and idx < len(svd_info.get('feature_strength', []))
+                                    else None)
+                        CF.objects.get_or_create(
+                            cluster=cr,
+                            feature_name=feat_name,
+                            defaults={
+                                'mean': None, 'std': None, 'median': None,
+                                'p25': None, 'p75': None, 'missing_rate': None,
+                                'distinguishing_score': float(strength) if strength is not None else None,
+                                'distinguishing_method': 'cluster_svd',
+                            },
+                        )
+        except Exception as svd_err:
+            logger.warning(f'Cluster SVD extraction skipped: {svd_err}')
+
+        # ── Step 3: UMAP-3D embedding (2D fallback) ───────────────────────
+        if run_umap:
+            try:
+                lf = entry['lazyframe']
+                # UMAP: sample max 10K for training, batch-transform rest in 1K batches
+                MAX_UMAP_TRAIN = UMAP_TRAIN_SAMPLE
+                BATCH_SIZE = UMAP_BATCH_SIZE
+                try:
+                    n_total = lf.select(pl.len()).collect(streaming=True).item()
+                except Exception:
+                    logger.error("unknown failed: {}".format(traceback.format_exc()))
+                    n_total = 0
+                from sklearn.preprocessing import StandardScaler
+                import umap
+                import numpy as np
+
+                live_schema = lf.collect_schema()
+                num_cols = [name for name, dt in zip(live_schema.names(), live_schema.dtypes())
+                            if dt in (pl.Int8, pl.Int16, pl.Int32, pl.Int64,
+                                      pl.UInt8, pl.UInt16, pl.UInt32, pl.UInt64,
+                                      pl.Float32, pl.Float64) and not name.startswith('_')]
+
+                if len(num_cols) >= 2:
+                    if n_total > MAX_UMAP_TRAIN:
+                        run.progress_msg = f'UMAP 采样 {MAX_UMAP_TRAIN}/{n_total} 实体训练...'
+                        run.save(update_fields=['progress_msg'])
+                        df_sample = lf.sample(n=MAX_UMAP_TRAIN, seed=RANDOM_SEED).collect(streaming=True)
+                        df_umap = lf.collect(streaming=True)
+                    else:
+                        df_umap = lf.collect(streaming=True)
+
+                    umap_components = 3  # try 3D first
+                    coords = None
+                    reducer = None
+
+                    if len(df_umap) > 2:
+                        for attempt_n in (3, 2):
+                            try:
+                                if n_total > MAX_UMAP_TRAIN:
+                                    # Train UMAP on 10K sample, batch-transform full dataset
+                                    mat_sample = df_sample.select(num_cols).to_numpy()
+                                    scaler = StandardScaler().fit(mat_sample)
+                                    mat_sample_scaled = np.nan_to_num(
+                                        scaler.transform(mat_sample), nan=0.0)
+                                    reducer = umap.UMAP(n_components=attempt_n,
+                                                        random_state=RANDOM_SEED,
+                                                        n_neighbors=15, min_dist=0.1,
+                                                        metric='euclidean')
+                                    reducer.fit(mat_sample_scaled)
+                                    # Batch transform in 1K-row batches
+                                    coords_list = []
+                                    for start in range(0, len(df_umap), BATCH_SIZE):
+                                        end = min(start + BATCH_SIZE, len(df_umap))
+                                        mat_batch = scaler.transform(
+                                            df_umap[start:end].select(num_cols).to_numpy())
+                                        mat_batch = np.nan_to_num(mat_batch, nan=0.0)
+                                        coords_list.append(reducer.transform(mat_batch))
+                                    coords = np.vstack(coords_list)
+                                else:
+                                    mat = np.nan_to_num(StandardScaler().fit_transform(
+                                        df_umap.select(num_cols).to_numpy()), nan=0.0)
+                                    reducer = umap.UMAP(n_components=attempt_n,
+                                                        random_state=RANDOM_SEED,
+                                                        n_neighbors=15, min_dist=0.1,
+                                                        metric='euclidean')
+                                    coords = reducer.fit_transform(mat)
+                                umap_components = attempt_n
+                                break
+                            except Exception as e_umap:
+                                if attempt_n == 2:
+                                    raise
+                                logger.warning(f'UMAP 3D failed ({e_umap}), falling back to 2D')
+
+                        if coords is None:
+                            raise Exception('UMAP produced no output')
+
+                        non_num = [c for c in df_umap.columns
+                                   if c not in num_cols and not c.startswith('_')]
+                        ent_col = non_num[0] if non_num else df_umap.columns[0]
+
+                        cluster_entry = store.get_cluster_result(cluster_id)
+                        labels_list = cluster_entry.get('labels', []) if cluster_entry else []
+
+                        from analysis.models import EntityProfile as EP
+                        from analysis.models import ClusterResult as CR
+
+                        profiles = []
+                        cr_cache = {}
+                        for i, row in enumerate(df_umap.iter_rows(named=True)):
+                            ev = str(row.get(ent_col, '')) or 'entity'
+                            ev = f'{ev}_{i}'
+                            lbl = labels_list[i] if i < len(labels_list) else -1
+                            cache_key = f'{run.id}_{lbl}'
+                            if cache_key not in cr_cache:
+                                cr_cache[cache_key] = CR.objects.filter(
+                                    run=run, cluster_label=lbl).first()
+                            has_z = umap_components >= 3 and coords.shape[1] >= 3
+                            profiles.append(EP(
+                                entity_value=ev, run=run, cluster_label=lbl,
+                                cluster=cr_cache[cache_key],
+                                embedding_x=float(coords[i, 0]) if i < len(coords) else None,
+                                embedding_y=float(coords[i, 1]) if i < len(coords) else None,
+                                embedding_z=float(coords[i, 2]) if has_z
+                                    and i < len(coords) else None,
+                            ))
+                        if profiles:
+                            EP.objects.bulk_create(profiles, ignore_conflicts=True, batch_size=1000)
+                            run.entity_count = len(profiles)
+                            run.save(update_fields=['entity_count'])
+            except Exception as umap_err:
+                logger.warning(f'UMAP embedding skipped: {umap_err}')
+
+        # Fallback: if entity_count not set (UMAP skipped), derive from cluster sizes
+        if run.entity_count is None:
+            from django.db.models import Sum
+            total = run.clusters.aggregate(total=Sum('size'))['total'] or 0
+            run.entity_count = total
+            run.save(update_fields=['entity_count'])
+
+        # ── Done ──
+        run.status = 'completed'
+        run.progress_pct = 100
+        run.progress_msg = '分析完成'
+        run.save(update_fields=['status', 'progress_pct', 'progress_msg', 'entity_count'])
+
+    except Exception:
+        tb = traceback.format_exc()
+        logger.error(tb)
+        try:
+            run.status = 'failed'
+            run.error_message = tb
+            run.progress_msg = f'失败: {tb}'
+            run.run_log += f'\n[ERROR] {tb}'
+            run.save(update_fields=['status', 'error_message', 'progress_msg', 'run_log'])
+        except Exception as e:
+            logger.warning('save failed after pipeline error: %s', e)
@@ -0,0 +1,132 @@
+"""Configuration views: config editor and LLM connectivity test."""
+import traceback
+import json
+import time
+import urllib.request
+import urllib.error
+import logging
+
+from django.shortcuts import render
+from django.http import JsonResponse
+from django.views.decorators.csrf import csrf_exempt
+
+from config import get_config, save_config, Config
+
+logger = logging.getLogger(__name__)
+
+
+def config_view(request):
+    """Config page: GET renders current config, POST saves updates."""
+    cfg = get_config()
+
+    if request.method == 'POST':
+        # Build a new Config from POST data (fall back to current values)
+        server = Config.Server(
+            host=request.POST.get('server_host', cfg.server.host),
+            port=int(request.POST.get('server_port', cfg.server.port)),
+            debug=request.POST.get('server_debug') == 'true',
+        )
+        data = Config.Data(
+            schema_strict=request.POST.get('data_schema_strict') == 'true',
+            recursive=request.POST.get('data_recursive') == 'true',
+        )
+        clustering = Config.Clustering(
+            algorithm=request.POST.get('clustering_algorithm', cfg.clustering.algorithm),
+            min_cluster_size=int(request.POST.get('clustering_min_cluster_size', cfg.clustering.min_cluster_size)),
+            random_state=cfg.clustering.random_state,
+        )
+        llm = Config.LLM(
+            enabled=request.POST.get('llm_enabled') == 'true',
+            base_url=request.POST.get('llm_base_url', cfg.llm.base_url),
+            api_key=request.POST.get('llm_api_key', cfg.llm.api_key),
+            model=request.POST.get('llm_model', cfg.llm.model),
+        )
+        cfg = Config(server=server, data=data, clustering=clustering, llm=llm)
+        save_config(cfg)
+        # Re-render with saved flag
+        return render(request, 'tianxuan/config.html', {'config': cfg, 'saved': True})
+
+    return render(request, 'tianxuan/config.html', {'config': cfg})
+
+
+@csrf_exempt
+def llm_test(request):
+    """LLM connectivity test: POST JSON {base_url, api_key, model} → chat/completions."""
+    if request.method != 'POST':
+        return JsonResponse({'success': False, 'message': '仅支持 POST 请求', 'latency_ms': 0}, status=405)
+
+    try:
+        body = json.loads(request.body)
+    except json.JSONDecodeError:
+        return JsonResponse({'success': False, 'message': '无效的 JSON 请求体', 'latency_ms': 0})
+
+    base_url = (body.get('base_url') or '').rstrip('/')
+    api_key = body.get('api_key') or ''
+    model = body.get('model') or ''
+
+    if not base_url:
+        return JsonResponse({'success': False, 'message': 'Base URL 不能为空', 'latency_ms': 0})
+    if not api_key:
+        return JsonResponse({'success': False, 'message': 'API Key 不能为空', 'latency_ms': 0})
+    if not model:
+        return JsonResponse({'success': False, 'message': 'Model 不能为空', 'latency_ms': 0})
+
+    url = f'{base_url}/chat/completions'
+    payload = json.dumps({
+        'model': model,
+        'messages': [{'role': 'user', 'content': 'respond with ok'}],
+        'max_tokens': 5,
+    }).encode('utf-8')
+
+    req = urllib.request.Request(
+        url,
+        data=payload,
+        headers={
+            'Content-Type': 'application/json',
+            'Authorization': f'Bearer {api_key}',
+        },
+        method='POST',
+    )
+
+    start = time.monotonic()
+    try:
+        with urllib.request.urlopen(req, timeout=5) as resp:
+            elapsed = int((time.monotonic() - start) * 1000)
+            data = json.loads(resp.read().decode('utf-8'))
+            if 'choices' in data and len(data['choices']) > 0:
+                logger.info(f'[LLM_TEST] base_url={base_url} model={model} success=True latency={elapsed}ms')
+                return JsonResponse({
+                    'success': True,
+                    'message': f'连接成功，模型 {model} 返回正常',
+                    'latency_ms': elapsed,
+                })
+            else:
+                logger.info(f'[LLM_TEST] base_url={base_url} model={model} success=True latency={elapsed}ms')
+                return JsonResponse({
+                    'success': True,
+                    'message': f'连接成功，但响应中无 choices（原始响应已记录）',
+                    'latency_ms': elapsed,
+                })
+    except urllib.error.HTTPError as e:
+        elapsed = int((time.monotonic() - start) * 1000)
+        try:
+            detail = json.loads(e.read().decode('utf-8', errors='replace'))
+            msg = detail.get('error', {}).get('message', str(e))
+        except Exception:
+            logger.error("unknown failed: {}".format(traceback.format_exc()))
+            msg = str(e)
+        logger.info(f'[LLM_TEST] base_url={base_url} model={model} success=False latency={elapsed}ms')
+        return JsonResponse({'success': False, 'message': f'HTTP {e.code}: {msg}', 'latency_ms': elapsed})
+    except urllib.error.URLError as e:
+        elapsed = int((time.monotonic() - start) * 1000)
+        reason = str(e.reason) if e.reason else '连接失败'
+        logger.info(f'[LLM_TEST] base_url={base_url} model={model} success=False latency={elapsed}ms')
+        return JsonResponse({'success': False, 'message': f'连接失败: {reason}', 'latency_ms': elapsed})
+    except TimeoutError:
+        elapsed = int((time.monotonic() - start) * 1000)
+        logger.info(f'[LLM_TEST] base_url={base_url} model={model} success=False latency={elapsed}ms')
+        return JsonResponse({'success': False, 'message': '请求超时（5秒）', 'latency_ms': elapsed})
+    except Exception as e:
+        elapsed = int((time.monotonic() - start) * 1000)
+        logger.info(f'[LLM_TEST] base_url={base_url} model={model} success=False latency={elapsed}ms')
+        return JsonResponse({'success': False, 'message': f'未知错误: {str(e)}', 'latency_ms': elapsed})
@@ -0,0 +1,63 @@
+"""Dashboard views: home, run list, run detail, run status."""
+from django.shortcuts import render, get_object_or_404
+from django.http import JsonResponse
+
+from analysis.models import AnalysisRun
+
+
+def dashboard(request):
+    """Home page: recent runs overview."""
+    runs = AnalysisRun.objects.all()[:20]
+    return render(request, 'analysis/dashboard.html', {'runs': runs})
+
+
+def run_list(request):
+    """List all analysis runs with pagination and optional ?type= filter."""
+    run_type_filter = request.GET.get('type')
+    page = int(request.GET.get('page', 1))
+    per_page = 20
+
+    qs = AnalysisRun.objects.all()
+    if run_type_filter in dict(AnalysisRun.RUN_TYPE_CHOICES):
+        qs = qs.filter(run_type=run_type_filter)
+
+    total = qs.count()
+    runs = qs[(page - 1) * per_page: page * per_page]
+    return render(request, 'analysis/run_list.html', {
+        'runs': runs,
+        'page': page,
+        'total_pages': (total + per_page - 1) // per_page if total else 1,
+        'current_type': run_type_filter or '',
+    })
+
+
+def run_detail(request, display_id):
+    """Details for a single analysis run."""
+    run = get_object_or_404(AnalysisRun, display_id=display_id)
+    clusters = run.clusters.all().order_by('-size')
+    return render(request, 'analysis/run_detail.html', {
+        'run': run,
+        'clusters': clusters,
+    })
+
+
+def run_status_api(request, display_id):
+    """Return current run status as JSON."""
+    run = get_object_or_404(AnalysisRun, display_id=display_id)
+    return JsonResponse({
+        'id': run.display_id,
+        'status': run.status,
+        'entity_count': run.entity_count,
+        'cluster_count': run.cluster_count,
+        'error_message': run.error_message,
+        'progress_pct': run.progress_pct,
+        'progress_msg': run.progress_msg,
+        'run_log': run.run_log[-2000:],  # last 2000 chars
+        'llm_thinking': run.llm_thinking[-5000:],  # last 5K chars
+        'tool_calls': run.tool_calls_json,
+    })
+
+
+def start_analysis(request):
+    """Placeholder: API endpoint to trigger analysis via MCP."""
+    return JsonResponse({'status': 'not_implemented', 'message': 'Use MCP tools to trigger analysis'})
@@ -0,0 +1,33 @@
+"""Entity profile view."""
+import json
+
+from django.shortcuts import render, get_object_or_404
+
+from analysis.models import EntityProfile
+
+
+def entity_profile(request, entity_id):
+    """Detailed profile for a single entity."""
+    entity = get_object_or_404(EntityProfile, id=entity_id)
+    run = entity.run
+
+    # Compute deviation from cluster mean
+    deviations = {}
+    if entity.feature_json and entity.cluster:
+        cluster_features = {
+            f.feature_name: {'mean': f.mean, 'std': f.std}
+            for f in entity.cluster.features.all()
+        }
+        for col, val in entity.feature_json.items():
+            if col in cluster_features and cluster_features[col].get('std') and cluster_features[col]['std'] > 1e-10:
+                deviations[col] = {
+                    'value': val,
+                    'cluster_mean': cluster_features[col]['mean'],
+                    'zscore': (val - cluster_features[col]['mean']) / cluster_features[col]['std'],
+                }
+
+    return render(request, 'analysis/entity_profile.html', {
+        'entity': entity,
+        'run': run,
+        'deviations': deviations,
+    })
@@ -0,0 +1,212 @@
+"""3D Globe view and flow extraction."""
+import json
+import logging
+import traceback
+
+from django.shortcuts import render
+
+from analysis.constants import GLOBE_MAX_ROWS
+from analysis.models import AnalysisRun
+from .helpers import _extract_lat, _extract_lon
+
+logger = logging.getLogger(__name__)
+
+
+def _extract_flows_from_df(df, MAX_ROWS):
+    """Extract flow data (lat/lon/TLS/bytes) from a Polars DataFrame."""
+    from analysis.geoip import lookup as geo_lookup
+    from analysis.type_classifier import TLS_HEX_MAP
+
+    src_lat_col = next((c for c in df.columns if c.lower() in (
+        ':ips.latd', ':ips_latd', ':ips_lat', 'src_latitude', 'src_lat', 'source_latitude', 'source_lat')), None)
+    src_lon_col = next((c for c in df.columns if c.lower() in (
+        ':ips.lond', ':ips_lond', ':ips_lon', 'src_longitude', 'src_lon', 'source_longitude', 'source_lon')), None)
+    dst_lat_col = next((c for c in df.columns if c.lower() in (
+        ':ipd.latd', ':ipd_latd', ':ipd_lat', 'dst_latitude', 'dst_lat', 'dest_latitude', 'dest_lat')), None)
+    dst_lon_col = next((c for c in df.columns if c.lower() in (
+        ':ipd.lond', ':ipd_lond', ':ipd_lon', 'dst_longitude', 'dst_lon', 'dest_longitude', 'dest_lon')), None)
+    src_ip_col = next((c for c in df.columns if 'src_ip' in c.lower() or ':ips' in c.lower()), None)
+    dst_ip_col = next((c for c in df.columns if 'dst_ip' in c.lower() or ':ipd' in c.lower()), None)
+    tls_col = next((c for c in df.columns if c.lower() in ('0ver', 'tls_version', '_tlsver', 'version')), None)
+    bytes_col = next((c for c in df.columns if c.lower() in ('8ack', '8pak', '8byt', 'bytes_sent', 'bytes')), None)
+    time_col = next((c for c in df.columns if c.lower() in (
+        'timestamp', 'time', '_time', 'ts', 'datetime', 'epoch',
+        'created_at', 'created', 'event_time')), None)
+
+    flows = []
+    for row in df.iter_rows(named=True):
+        slat = slon = dlat = dlon = None
+
+        if src_lat_col and src_lon_col:
+            slat = _extract_lat(row.get(src_lat_col))
+            slon = _extract_lon(row.get(src_lon_col))
+        if dst_lat_col and dst_lon_col:
+            dlat = _extract_lat(row.get(dst_lat_col))
+            dlon = _extract_lon(row.get(dst_lon_col))
+
+        if (slat is None or slon is None) and src_ip_col:
+            sg = geo_lookup(str(row.get(src_ip_col, '')))
+            if sg:
+                slat, slon = sg['lat'], sg['lon']
+        if (dlat is None or dlon is None) and dst_ip_col:
+            dg = geo_lookup(str(row.get(dst_ip_col, '')))
+            if dg:
+                dlat, dlon = dg['lat'], dg['lon']
+
+        if slat is None or slon is None or dlat is None or dlon is None:
+            continue
+
+        tls_val = str(row.get(tls_col or '', ''))
+        # Check hex format first (e.g. "03 03" or "0303")
+        hex_key = tls_val.replace(' ', '').strip()
+        if hex_key in TLS_HEX_MAP:
+            tls_label = TLS_HEX_MAP[hex_key]
+        elif '1.3' in tls_val:
+            tls_label = 'TLSv1.3'
+        elif '1.2' in tls_val:
+            tls_label = 'TLSv1.2'
+        else:
+            tls_label = 'other'
+
+        bs = float(row.get(bytes_col or '', 0) or 0)
+
+        # Extract timestamp for animation ordering
+        t_raw = row.get(time_col) if time_col else None
+        t_val = None
+        if t_raw is not None:
+            if isinstance(t_raw, (int, float)):
+                t_val = float(t_raw)
+            elif hasattr(t_raw, 'timestamp'):
+                t_val = t_raw.timestamp()
+            else:
+                try:
+                    t_val = float(t_raw)
+                except (ValueError, TypeError):
+                    pass
+
+        flows.append({
+            'slat': slat, 'slon': slon,
+            'dlat': dlat, 'dlon': dlon,
+            'tls': tls_label,
+            'bytes': bs,
+            'time': t_val,
+        })
+
+    # Normalize timestamps to 0-1 and sort flows by time
+    timed = [f for f in flows if f['time'] is not None]
+    if len(timed) > 1:
+        timed.sort(key=lambda f: f['time'])
+        t_min, t_max = timed[0]['time'], timed[-1]['time']
+        t_range = max(t_max - t_min, 1)
+        for f in timed:
+            f['time'] = (f['time'] - t_min) / t_range
+        # Untimed flows get evenly spaced after timed ones
+        untimed = [f for f in flows if f['time'] is None]
+        for i, f in enumerate(untimed):
+            f['time'] = min(1.0, (len(timed) + i) / max(len(flows) - 1, 1))
+        flows = timed + untimed
+    else:
+        for i, f in enumerate(flows):
+            f['time'] = i / max(len(flows) - 1, 1)
+
+    return flows
+
+
+def globe_view(request):
+    from analysis.geoip import lookup as geo_lookup
+    from analysis.data_loader import load_csv_directory, load_from_db
+    from analysis.models import AnalysisRun
+    from analysis.session_store import SessionStore
+
+    MAX_ROWS_PER_RUN = GLOBE_MAX_ROWS  # Limit per run to prevent browser overload
+
+    # Check for ?data=dataset_id parameter (direct data loading from session store)
+    data_param = request.GET.get('data')
+    if data_param:
+        store = SessionStore()
+        flows = []
+        all_runs = list(AnalysisRun.objects.filter(status='completed').order_by('-id').values('display_id', 'csv_glob'))
+        try:
+            entry = store.get_dataset(data_param)
+            if entry is not None:
+                lf = entry['lazyframe']
+                df = lf.head(MAX_ROWS_PER_RUN).collect()
+                flows = _extract_flows_from_df(df, MAX_ROWS_PER_RUN)
+        except Exception as exc:
+            logging.getLogger(__name__).warning(f'globe ?data= param failed: {exc}')
+        return render(request, 'tianxuan/globe.html', {
+            'geo_flows': json.dumps(flows),
+            'all_runs': all_runs,
+            'selected_ids': [],
+        })
+
+    # Get all completed runs for the selector
+    all_runs = list(AnalysisRun.objects.filter(status='completed').order_by('-id').values('display_id', 'csv_glob'))
+
+    # Get selected run IDs from query params (multiple checkboxes)
+    selected_ids_list = request.GET.getlist('runs')
+    selected_runs = []
+    if selected_ids_list:
+        for sid in selected_ids_list:
+            sid = sid.strip()
+            if sid.isdigit():
+                try:
+                    r = AnalysisRun.objects.get(display_id=int(sid), status='completed')
+                    selected_runs.append(r)
+                except AnalysisRun.DoesNotExist:
+                    pass
+
+    # Check if user-selected runs have valid files; if not, fall through
+    all_selected_failed = bool(selected_ids_list)
+    fallback_run = None
+
+    if not selected_runs:
+        all_selected_failed = False
+
+    flows = []
+
+    for run in selected_runs:
+        try:
+            lf = None
+            if run.sqlite_table:
+                lf = load_from_db(run.sqlite_table)
+            if lf is None:
+                lf, schema, rc, fc, mem = load_csv_directory(run.csv_glob)
+            df = lf.head(MAX_ROWS_PER_RUN).collect()
+            flows.extend(_extract_flows_from_df(df, MAX_ROWS_PER_RUN))
+            all_selected_failed = False
+            break  # stop after first successful load
+        except Exception as exc:
+            logger.warning('[GLOBE] Failed to load run %s (%s): %s', run.display_id, run.csv_glob, exc)
+            continue
+
+    if all_selected_failed or not flows:
+        # Fallback: latest completed run with valid data (SQLite or CSV)
+        for latest in AnalysisRun.objects.filter(status='completed').order_by('-id'):
+            if not latest.csv_glob and not latest.sqlite_table:
+                continue
+            try:
+                lf = None
+                if latest.sqlite_table:
+                    lf = load_from_db(latest.sqlite_table)
+                if lf is None:
+                    if not latest.csv_glob:
+                        continue
+                    lf, schema, rc, fc, mem = load_csv_directory(latest.csv_glob)
+                df = lf.head(MAX_ROWS_PER_RUN).collect()
+                flows.extend(_extract_flows_from_df(df, MAX_ROWS_PER_RUN))
+                selected_runs = [latest]
+                break
+            except Exception:
+                logger.error("unknown failed: {}".format(traceback.format_exc()))
+                continue
+
+    resp = render(request, 'tianxuan/globe.html', {
+        'geo_flows': json.dumps(flows),
+        'all_runs': all_runs,
+        'selected_ids': [r.display_id for r in selected_runs],
+    })
+    resp['Cache-Control'] = 'no-cache, no-store, must-revalidate'
+    resp['Pragma'] = 'no-cache'
+    resp['Expires'] = '0'
+    return resp
@@ -0,0 +1,75 @@
+"""Geo extraction helpers and plan utilities."""
+import traceback
+from typing import Optional
+from pathlib import Path
+import json as _json
+import logging
+logger = logging.getLogger(__name__)
+
+
+def _extract_lat(val):
+    """Try to extract a latitude from *val* (number or str)."""
+    if val is None:
+        return None
+    try:
+        v = float(val)
+        return None if v < -90 or v > 90 else v
+    except (ValueError, TypeError):
+        return None
+
+
+def _extract_lon(val):
+    """Try to extract a longitude from *val* (number or str)."""
+    if val is None:
+        return None
+    try:
+        v = float(val)
+        return None if v < -180 or v > 180 else v
+    except (ValueError, TypeError):
+        return None
+
+
+# ── Plan index utilities (shared by tools.py and auto.py) ─────────────
+
+_PLANS_DIR = Path(__file__).resolve().parent.parent.parent / '.omo' / 'plans'
+_PLANS_DIR.mkdir(parents=True, exist_ok=True)
+
+_AUTO_INDEX_PATH = _PLANS_DIR / '_auto_index.json'
+_MAX_AUTO_PLANS = 10
+
+
+def _read_auto_index():
+    """Read the auto index file, returning list of {file, pinned}."""
+    if not _AUTO_INDEX_PATH.exists():
+        return []
+    try:
+        data = _json.loads(_AUTO_INDEX_PATH.read_text(encoding='utf-8'))
+        if not isinstance(data, list):
+            return []
+        return data
+    except Exception:
+        logger.error("_read_auto_index failed: {}".format(traceback.format_exc()))
+        return []
+
+
+def _write_auto_index(index):
+    """Write the auto index list."""
+    _AUTO_INDEX_PATH.write_text(
+        _json.dumps(index, ensure_ascii=False, indent=2),
+        encoding='utf-8',
+    )
+
+
+def _add_to_auto_index(filename):
+    """Prepend filename to auto index, evict unpinned entries when >10."""
+    index = _read_auto_index()
+    # Remove if already present
+    index = [entry for entry in index if entry.get('file') != filename]
+    # Prepend
+    index.insert(0, {'file': filename, 'pinned': False})
+    # Evict unpinned entries beyond max
+    pinned_entries = [e for e in index if e.get('pinned')]
+    unpinned_entries = [e for e in index if not e.get('pinned')]
+    # Keep all pinned + fill remaining slots with unpinned
+    result = pinned_entries + unpinned_entries[:_MAX_AUTO_PLANS - len(pinned_entries)]
+    _write_auto_index(result)
@@ -0,0 +1,36 @@
+"""Log viewer view."""
+from django.shortcuts import render
+from django.http import HttpResponse
+
+
+def log_viewer(request):
+    """Display last 200 lines of the app log (DEBUG only).
+    With ?pos=N, returns text: line_count\\nnew_content for AJAX polling.
+    """
+    from django.conf import settings
+    if not settings.DEBUG:
+        return HttpResponse('Log viewer only available in DEBUG mode', status=403)
+    log_path = settings.BASE_DIR / 'logs' / 'tianxuan.log'
+    pos = request.GET.get('pos')
+    if pos is not None:
+        # AJAX polling mode: return new content since byte position
+        try:
+            pos = int(pos)
+        except ValueError:
+            pos = 0
+        if log_path.exists():
+            with open(log_path, 'r', encoding='utf-8') as f:
+                total = f.seek(0, 2)  # seek to end
+                if pos >= total:
+                    return HttpResponse(f'{total}\n', content_type='text/plain')
+                f.seek(pos)
+                new_content = f.read()
+                return HttpResponse(f'{total}\n{new_content}', content_type='text/plain')
+        return HttpResponse('0\n', content_type='text/plain')
+
+    # HTML page mode
+    lines = []
+    if log_path.exists():
+        with open(log_path, 'r', encoding='utf-8') as f:
+            lines = f.readlines()[-200:]
+    return render(request, 'tianxuan/log_viewer.html', {'lines': lines})
@@ -0,0 +1,191 @@
+"""Manual analysis page: workflow builder and run execution."""
+import json
+import threading
+import traceback
+import logging
+
+from django.shortcuts import render, get_object_or_404
+from django.http import JsonResponse
+from django.views.decorators.csrf import csrf_exempt
+
+from analysis.models import AnalysisRun
+from .pipeline import _run_pipeline_worker
+from .clustering import _run_clustering_pipeline
+
+logger = logging.getLogger(__name__)
+
+
+def manual_page(request):
+    """Workflow builder: select dataset, add MCP tool steps, execute."""
+    from analysis.session_store import SessionStore
+    from analysis.tool_registry import get_tools_meta
+    import json as _json
+
+    # Get all tools metadata
+    tools = get_tools_meta()
+    core_names = {'profile_data', 'build_entity_profiles', 'compute_scores',
+                  'run_clustering', 'extract_features', 'detect_anomalies',
+                  'visualize_anomalies'}
+    diag_names = {'validate_data', 'explore_distributions', 'find_outliers',
+                  'diagnose_clustering', 'compare_datasets', 'export_debug_sample',
+                  'repair_schema'}
+    analysis_names = {'analyze_patterns', 'analyze_temporal', 'analyze_tls_health',
+                      'analyze_geo_distribution', 'analyze_entity_detail'}
+    core_tools, diag_tools, analysis_tools, other_tools = [], [], [], []
+    for t in tools:
+        if t.name in core_names: core_tools.append(t)
+        elif t.name in diag_names: diag_tools.append(t)
+        elif t.name in analysis_names: analysis_tools.append(t)
+        else: other_tools.append(t)
+
+    meta_list = [{'name': t.name, 'description': t.description, 'inputSchema': t.inputSchema} for t in tools]
+
+    # Get datasets from SessionStore
+    store = SessionStore()
+    session_datasets = store.list_datasets() if hasattr(store, 'list_datasets') else []
+    session_ds_map = {ds['dataset_id']: ds for ds in session_datasets}
+
+    # Build dataset entries from ALL AnalysisRun records (upload/manual/auto, any status)
+    from analysis.models import AnalysisRun
+    all_runs = AnalysisRun.objects.all().order_by('-created_at')[:100]
+    datasets_with_status = []
+
+    for run in all_runs:
+        # Determine how the dataset was keyed in SessionStore
+        if run.run_type == 'upload':
+            ds_id = f'upload_{run.display_id}'
+        else:
+            ds_id = f'run_{run.display_id}'
+
+        session_entry = session_ds_map.get(ds_id)
+
+        if session_entry:
+            # Full data available in SessionStore — merge with run status
+            datasets_with_status.append({
+                **session_entry,
+                'dataset_id': ds_id,
+                'run_status': run.status,
+                'total_flows': run.total_flows,
+                'display_id': run.display_id,
+                'needs_reload': False,
+            })
+        elif run.csv_glob or run.sqlite_table:
+            # Data not in SessionStore but recoverable from disk/DB
+            datasets_with_status.append({
+                'dataset_id': ds_id,
+                'run_status': run.status,
+                'display_id': run.display_id,
+                'total_flows': run.total_flows,
+                'row_count': run.total_flows,
+                'file_count': None,
+                'column_count': 0,
+                'columns': [],
+                'svd_components': 0,
+                'needs_reload': True,
+                'csv_glob': run.csv_glob,
+                'sqlite_table': run.sqlite_table or '',
+            })
+
+    # Map statuses to user-facing labels for the filter tabs
+    # ready → 待分析, completed → 已分析, failed → 错误
+    # Everything else (pending/loading/profiling/aggregating/clustering/extracting) → analyzing → 分析中
+    for ds_info in datasets_with_status:
+        s = ds_info['run_status']
+        if s in ('completed',):
+            ds_info['status_label'] = '已分析'
+            ds_info['badge_class'] = 'completed'
+        elif s in ('failed',):
+            ds_info['status_label'] = '错误'
+            ds_info['badge_class'] = 'failed'
+        elif s in ('pending', 'loading', 'profiling', 'aggregating', 'clustering', 'extracting'):
+            ds_info['status_label'] = '分析中'
+            ds_info['badge_class'] = 'analyzing'
+        else:
+            ds_info['status_label'] = '待分析'
+            ds_info['badge_class'] = 'ready'
+
+    return render(request, 'tianxuan/manual.html', {
+        'core_tools': core_tools,
+        'diag_tools': diag_tools,
+        'analysis_tools': analysis_tools,
+        'other_tools': other_tools,
+        'tools_meta': _json.dumps(meta_list, ensure_ascii=False),
+        'datasets': datasets_with_status,
+        'datasets_with_status_json': _json.dumps(datasets_with_status, ensure_ascii=False),
+    })
+
+
+@csrf_exempt
+def manual_run_analysis(request):
+    """AJAX POST: run clustering pipeline on the upload dataset."""
+    if request.method != 'POST':
+        return JsonResponse({'error': 'POST required'}, status=405)
+    import json as _json
+    body = _json.loads(request.body)
+    display_id = body.get('run_id')
+    run = get_object_or_404(AnalysisRun, display_id=display_id)
+    pk = run.id
+    run.run_type = 'manual'
+    run.save(update_fields=['run_type'])
+    feature_columns = body.get('feature_columns')
+    algorithm = body.get('algorithm', 'agglomerative')
+    min_cluster_size = int(body.get('min_cluster_size', 5))
+    head = body.get('head')
+    cluster_mode = body.get('cluster_mode', 'raw')
+
+    def _analysis_fn(run, ctx):
+        from analysis.session_store import SessionStore
+
+        feature_columns = ctx['feature_columns']
+        algorithm = ctx['algorithm']
+        min_cluster_size = ctx['min_cluster_size']
+        head = ctx.get('head')
+        pk = ctx['pk']
+        cluster_mode = ctx.get('cluster_mode', 'raw')
+
+        store = SessionStore()
+        try:
+            # Use display_id for dataset key (consistent with _background_process)
+            display_id = ctx.get('display_id', pk)
+            upload_ds_id = f'upload_{display_id}'
+            entry = store.get_dataset(upload_ds_id)
+            if entry is None:
+                # Fallback: try PK-based key (old pipeline)
+                entry = store.get_dataset(f'upload_{pk}')
+            if entry is None:
+                # Fallback: try entity dataset (from older pipeline)
+                entry = store.get_dataset(f'entity_{pk}')
+            if entry is None:
+                run.error_message = '请先上传并等待预处理完成'
+                run.status = 'failed'
+                run.save(update_fields=['status', 'error_message'])
+                return
+
+            ds_id = upload_ds_id if store.get_dataset(upload_ds_id) else f'upload_{pk}'
+
+            # Use the unified clustering pipeline (clustering → extraction → UMAP)
+            _run_clustering_pipeline(
+                run=run, store=store, entity_ds_id=ds_id,
+                feature_columns=feature_columns,
+                algorithm=algorithm, min_cluster_size=min_cluster_size,
+                run_umap=True, head=head,
+            )
+        except Exception:
+            tb = traceback.format_exc()
+            logger.error(tb)
+            run.status = 'failed'
+            run.error_message = tb
+            run.progress_msg = f'失败: {tb}'
+            run.run_log += f'\n[ERROR] {tb}'
+            run.save(update_fields=['status', 'error_message', 'progress_msg', 'run_log'])
+
+    t = threading.Thread(
+        target=_run_pipeline_worker,
+        args=(pk, _analysis_fn),
+        kwargs=dict(feature_columns=feature_columns, algorithm=algorithm,
+                     min_cluster_size=min_cluster_size, head=head, pk=pk,
+                     display_id=run.display_id, cluster_mode=cluster_mode),
+        daemon=True,
+    )
+    t.start()
+    return JsonResponse({'status': 'started', 'redirect': f'/runs/{run.display_id}/'})
@@ -0,0 +1,300 @@
+"""Background pipeline: CSV loading, SVD, and pipeline worker."""
+import os
+import threading
+import traceback
+import logging
+
+from analysis.appdata import ensure_appdata_dir
+from analysis.constants import MAX_CLUSTER_FEATURES, RANDOM_SEED
+from analysis.models import AnalysisRun
+from analysis.profile_util import profile
+
+logger = logging.getLogger(__name__)
+
+
+def _run_pipeline_worker(run_id, analysis_fn, **ctx):
+    """Unified background worker: setup / ORM / progress / error handling."""
+    import django
+    os.environ['DJANGO_SETTINGS_MODULE'] = 'tianxuan.settings'
+    os.environ['DJANGO_ALLOW_ASYNC_UNSAFE'] = 'true'
+    django.setup()
+
+    from analysis.db_utils import retry_on_lock
+    from django.db import close_old_connections
+
+    close_old_connections()
+
+    @retry_on_lock()
+    def _get_run():
+        return AnalysisRun.objects.get(id=run_id)
+
+    run = _get_run()
+    try:
+        analysis_fn(run, ctx)
+    except Exception:
+        tb = traceback.format_exc()
+        logger.error(tb)
+        run.status = 'failed'
+        run.error_message = tb
+        run.run_log += f'\n[FATAL] {tb}'
+        retry_on_lock()(lambda: run.save(update_fields=['status', 'error_message', 'run_log']))()
+
+
+@profile
+def _background_process(run_id, upload_dir):
+    """Background: stream-load CSVs → save to SQLite → ready for analysis."""
+    import django
+    os.environ['DJANGO_SETTINGS_MODULE'] = 'tianxuan.settings'
+    django.setup()
+    from analysis.models import AnalysisRun
+    from analysis.session_store import SessionStore
+
+    run = AnalysisRun.objects.get(id=run_id)
+    store = SessionStore()
+    try:
+        # Check upload directory still exists (may have been temp-cleaned)
+        if not upload_dir.is_dir():
+            msg = '上传数据目录已不存在，请重新上传'
+            logger.error('[BACKGROUND] %s (run_id=%s, upload_dir=%s)', msg, run_id, upload_dir)
+            run.status = 'failed'
+            run.error_message = msg
+            run.save(update_fields=['status', 'error_message'])
+            return
+
+        import glob
+        import polars as pl
+        csv_paths = sorted(glob.glob(str(upload_dir / '*.csv')))
+        if not csv_paths:
+            msg = '没有找到 CSV 文件'
+            logger.error('[BACKGROUND] %s (run_id=%s)', msg, run_id)
+            run.status = 'failed'
+            run.error_message = msg
+            run.save(update_fields=['status', 'error_message'])
+            return
+
+        # ── Detect lat/lon columns for schema override ──
+        import csv as _csv
+        _LAT_LON_KW = ('latd', 'lond', 'latitude', 'longitude')
+        lat_lon_overrides = {}
+        try:
+            with open(csv_paths[0], 'r', encoding='utf-8') as _f:
+                _reader = _csv.reader(_f)
+                _headers = next(_reader, [])
+            for _col in _headers:
+                _norm = _col.lower().replace('-', '_').replace('.', '_').replace(' ', '_')
+                if any(kw in _norm for kw in _LAT_LON_KW):
+                    lat_lon_overrides[_col] = pl.Utf8
+                    logger.info('[BACKGROUND] Forcing lat/lon column %r to Utf8 for safe parsing', _col)
+        except Exception:
+            logger.warning('[BACKGROUND] Failed to detect lat/lon columns', exc_info=True)
+
+        total_files = len(csv_paths)
+        logger.info('[BACKGROUND] Found %d CSV files in %s', total_files, upload_dir)
+        # Adaptive batch size: aim for ~20 batches total, cap at 500 files per batch
+        batch_size = max(1, min(500, total_files // 20 or 1))
+        total_batches = (total_files + batch_size - 1) // batch_size
+
+        run.status = 'loading'; run.progress_pct = 10
+        run.progress_msg = f'正在流式加载 {total_files} 个文件（共 {total_batches} 批）...'
+        run.save(update_fields=['status', 'progress_pct', 'progress_msg'])
+
+        from analysis.data_loader import save_to_db, drop_sqlite_table, load_from_db, _dtype_to_sqlite
+
+        table_name = f'_data_{run_id}'
+
+        # ── Phase 1: Pre-scan all files for full union column set ──
+        all_columns = []
+        for _path in csv_paths:
+            with open(_path, 'r', encoding='utf-8') as _f:
+                _reader = _csv.reader(_f)
+                _headers = next(_reader, [])
+                for _h in _headers:
+                    if _h not in all_columns:
+                        all_columns.append(_h)
+        reference_columns = sorted(all_columns, key=str.lower)
+        schema = {col: 'Utf8' for col in all_columns}
+
+        # ── Phase 2: Create SQLite table once with all columns as TEXT ──
+        import sqlite3 as _sqlite3
+        from django.conf import settings as _dj_settings
+        _db_path = _dj_settings.DATABASES['default']['NAME']
+        _conn = _sqlite3.connect(_db_path)
+        try:
+            _col_defs = [f'"{_col}" TEXT' for _col in reference_columns]
+            _conn.execute(f'DROP TABLE IF EXISTS "{table_name}"')
+            _conn.execute(f'CREATE TABLE "{table_name}" ({", ".join(_col_defs)})')
+            _conn.commit()
+        finally:
+            _conn.close()
+
+        # ── Phase 3: Batched collection → SQLite append ──
+        schema_overrides = {col: pl.Utf8 for col in reference_columns}
+        completed_batches = 0
+        total_rows = 0
+        for i in range(0, total_files, batch_size):
+            batch = csv_paths[i:i+batch_size]
+            # Scan each file INDIVIDUALLY — batch scan fails when files have different schemas
+            dfs = []
+            for fp in batch:
+                lf = pl.scan_csv(fp, infer_schema_length=0,
+                                 schema_overrides=schema_overrides)
+                df = lf.collect(streaming=True)
+                # Fill missing columns with NULL
+                for col in reference_columns:
+                    if col not in df.columns:
+                        df = df.with_columns(pl.lit(None).alias(col))
+                # Select all reference_columns in canonical order
+                df = df.select(reference_columns)
+                dfs.append(df)
+            df_batch = pl.concat(dfs, how='diagonal_relaxed')
+
+            # Append to pre-created SQLite table
+            save_to_db(run.id, df_batch.lazy(), mode='append')
+
+            # Delete temp files after successful SQLite write
+            for f in batch:
+                try:
+                    os.unlink(f)
+                except OSError:
+                    pass
+
+            completed_batches += 1
+            run.progress_pct = int((completed_batches / total_batches) * 80 + 10)
+            run.progress_msg = f'正在加载数据... ({completed_batches}/{total_batches} 批)'
+            run.save(update_fields=['progress_pct', 'progress_msg'])
+
+        # ── Phase 3: Reload from SQLite with correct types ──
+        lf = load_from_db(table_name, schema_overrides=schema)
+        if lf is None:
+            raise RuntimeError(f'SQLite 表 {table_name} 加载失败')
+
+        # Apply _coerce_to_float on lat/lon columns (restore Float64 from Utf8 storage)
+        from analysis.data_loader import _coerce_to_float as _coerce_to_float_fn
+        for lat_lon_col in lat_lon_overrides:
+            if lat_lon_col in lf.columns:
+                lf = lf.with_columns(
+                    pl.col(lat_lon_col).map_batches(_coerce_to_float_fn, return_dtype=pl.Float64).alias(lat_lon_col)
+                )
+
+        # ── Restore numeric types for Utf8 columns coerced during CSV load ──
+        # All columns were forced to Utf8 for safe SQLite storage.  Now try to
+        # recover the original numeric types (Int64 / Float64) from the data.
+        numeric_types = (pl.Int8, pl.Int16, pl.Int32, pl.Int64,
+                         pl.UInt8, pl.UInt16, pl.UInt32, pl.UInt64,
+                         pl.Float32, pl.Float64)
+        # Get current schema to know which columns are still Utf8
+        live_schema = lf.collect_schema()
+        live_names = live_schema.names()
+        live_dtypes = list(live_schema.dtypes())
+        utf8_cols = [
+            live_names[i] for i, dt in enumerate(live_dtypes)
+            if dt == pl.Utf8 and not live_names[i].startswith('_')
+        ]
+        if utf8_cols:
+            # Infer types from a small sample to decide Float64 vs Int64
+            sample_df = lf.select(utf8_cols).head(1000).collect(streaming=True)
+            type_map: dict[str, pl.DataType] = {}
+            for col in utf8_cols:
+                try:
+                    # Try Float64 first (most general numeric type)
+                    casted = sample_df[col].cast(pl.Float64, strict=False)
+                    non_null = casted.is_not_null().sum()
+                    if non_null > len(casted) * 0.5:
+                        # Check if all non-null values are integers → use Int64
+                        all_int = True
+                        for v in casted.head(100).to_list():
+                            if v is not None and v != int(v):
+                                all_int = False
+                                break
+                        type_map[col] = pl.Int64 if all_int else pl.Float64
+                except Exception:
+                    logger.error("unknown failed: {}".format(traceback.format_exc()))
+                    pass
+
+            if type_map:
+                # Update schema dict to reflect restored types
+                for col, dtype in type_map.items():
+                    schema[col] = str(dtype)
+                # Apply casts to the LazyFrame
+                casts = [pl.col(col).cast(dtype, strict=False) for col, dtype in type_map.items()]
+                lf = lf.with_columns(casts)
+                logger.info('[BACKGROUND] Restored numeric types for %d columns: %s',
+                            len(type_map), list(type_map.keys())[:10])
+
+        # ── SVD dimensionality reduction ──
+        svd_components = 0
+        # Only operate on columns with known numeric types in the schema
+        numeric_dtypes = {'Int64', 'Float64', 'Int32', 'Float32',
+                          'Int8', 'Int16', 'UInt8', 'UInt16', 'UInt32', 'UInt64'}
+        numeric_cols = [c for c, dt in schema.items() if dt in numeric_dtypes]
+        n_numeric = len(numeric_cols)
+
+        if n_numeric >= 2:
+            n_components = min(MAX_CLUSTER_FEATURES, n_numeric - 1)
+            try:
+                from sklearn.decomposition import TruncatedSVD  # fmt: skip
+                import numpy as np  # fmt: skip
+
+                # Collect numeric data for SVD (full materialise → transform → add back)
+                df_full = lf.select(numeric_cols).collect(streaming=True)
+                X = df_full.to_numpy()
+                X = np.nan_to_num(X, nan=0.0, posinf=0.0, neginf=0.0)
+
+                svd = TruncatedSVD(n_components=n_components, random_state=RANDOM_SEED)
+                X_svd = svd.fit_transform(X)
+                svd_components = n_components
+
+                # Build a new LazyFrame that includes _svd_* columns
+                # Strategy: collect full LazyFrame once, add SVD columns, re-wrap as lazy
+                df_all = lf.collect(streaming=True)
+                for i in range(n_components):
+                    col_name = f'_svd_{i}'
+                    schema[col_name] = 'Float64'
+                    df_all = df_all.with_columns(
+                        pl.Series(col_name, X_svd[:, i])
+                    )
+                lf = df_all.lazy()
+
+                logger.info('[BACKGROUND] SVD: %d components from %d numeric columns '
+                            '(explained variance ratio sum=%.4f)',
+                            svd_components, n_numeric,
+                            float(svd.explained_variance_ratio_.sum()))
+            except Exception as e:
+                logger.warning('[BACKGROUND] SVD failed (non-fatal): %s', e)
+        elif n_numeric == 1:
+            logger.info('[BACKGROUND] SVD skipped: only 1 numeric column (%s)',
+                        numeric_cols[0])
+
+        # Get row count
+        df_count = lf.select(pl.len()).collect(streaming=True)
+        total_rows = df_count[0, 0] if df_count.height > 0 else 0
+
+        run.sqlite_table = table_name
+        run.total_flows = total_rows
+        run.save(update_fields=['sqlite_table', 'total_flows'])
+
+        ds_id = f'upload_{run.display_id}'
+        store.store_dataset(ds_id, lf, schema=schema, metadata={
+            'row_count': total_rows, 'file_count': total_files, 'upload_dir': str(upload_dir),
+            'csv_glob': str(upload_dir / '*.csv'),
+            'sqlite_table': table_name,
+            'svd_components': svd_components,
+        })
+
+        # ── Phase 3: Data ready for filtering/clustering via MCP tools ──
+        run.status = 'ready'; run.progress_pct = 60; run.progress_msg = '数据准备就绪'
+        run.save(update_fields=['status', 'progress_pct', 'progress_msg'])
+    except Exception:
+        tb = traceback.format_exc()
+        logger.error(tb)
+        # Drop partial SQLite table on failure (idempotent)
+        try:
+            from analysis.data_loader import drop_sqlite_table  # fmt: skip
+            drop_sqlite_table(f'_data_{run_id}')
+        except Exception:
+            logger.error("unknown failed: {}".format(traceback.format_exc()))
+            pass  # Expected: drop_sqlite_table may fail if table already gone
+        run.status = 'failed'
+        run.error_message = tb
+        run.run_log += f'\n[ERROR] {tb}'
+        run.save(update_fields=['status', 'error_message', 'run_log'])
@@ -0,0 +1,381 @@
+"""MCP Tool Lab, plan management, and utility views."""
+import json
+import asyncio
+import threading
+import traceback
+import logging
+from pathlib import Path
+
+from django.shortcuts import render, get_object_or_404, redirect
+from django.http import JsonResponse
+from django.views.decorators.csrf import csrf_exempt
+
+from analysis.appdata import ensure_appdata_dir
+from analysis.models import AnalysisRun
+from .pipeline import _background_process
+from .helpers import (
+    _PLANS_DIR, _AUTO_INDEX_PATH, _MAX_AUTO_PLANS,
+    _read_auto_index, _write_auto_index, _add_to_auto_index,
+)
+
+logger = logging.getLogger(__name__)
+
+
+def tool_lab(request):
+    """List all available MCP tools with descriptions and input schemas."""
+    from analysis.tool_registry import get_tools_meta
+    tools = get_tools_meta()
+    # Group tools by category
+    core_names = {'profile_data', 'build_entity_profiles', 'compute_scores',
+                  'run_clustering', 'extract_features', 'detect_anomalies',
+                  'visualize_anomalies'}
+    diag_names = {'validate_data', 'explore_distributions', 'find_outliers',
+                  'diagnose_clustering', 'compare_datasets', 'export_debug_sample',
+                  'repair_schema'}
+    analysis_names = {'analyze_patterns', 'analyze_temporal', 'analyze_tls_health',
+                      'analyze_geo_distribution', 'analyze_entity_detail'}
+
+    core_tools, diag_tools, analysis_tools, other_tools = [], [], [], []
+    for t in tools:
+        if t.name in core_names:
+            core_tools.append(t)
+        elif t.name in diag_names:
+            diag_tools.append(t)
+        elif t.name in analysis_names:
+            analysis_tools.append(t)
+        else:
+            other_tools.append(t)
+
+    # Get list of datasets for parameter autofill
+    from analysis.session_store import SessionStore
+    store = SessionStore()
+    ds_list = store.list_datasets() if hasattr(store, 'list_datasets') else []
+
+    import json as _json
+    # Serialize tools metadata for JS
+    meta_list = []
+    for t in tools:
+        meta_list.append({
+            'name': t.name,
+            'description': t.description,
+            'inputSchema': t.inputSchema,
+        })
+    return render(request, 'analysis/tool_lab.html', {
+        'core_tools': core_tools,
+        'diag_tools': diag_tools,
+        'analysis_tools': analysis_tools,
+        'other_tools': other_tools,
+        'datasets': ds_list,
+        'tools_meta': _json.dumps(meta_list, ensure_ascii=False),
+    })
+
+
+@csrf_exempt
+def tool_lab_run(request):
+    """Execute a single MCP tool via AJAX POST."""
+    if request.method != 'POST':
+        return JsonResponse({'error': 'POST required'}, status=405)
+    import json as _json
+    try:
+        body = _json.loads(request.body)
+    except Exception:
+        logger.error("tool_lab_run failed: {}".format(traceback.format_exc()))
+        return JsonResponse({'error': 'Invalid JSON body'}, status=400)
+
+    tool_name = body.get('tool')
+    params = body.get('params', {})
+    if not tool_name:
+        return JsonResponse({'error': 'Missing tool name'}, status=400)
+
+    # ── Backend validation: check required params + dataset existence ──
+    from analysis.tool_registry import handle_call, get_tools_meta
+    from analysis.session_store import SessionStore
+    tools_meta = get_tools_meta()
+    tool_schema = None
+    for t in tools_meta:
+        if t.name == tool_name:
+            tool_schema = t.inputSchema
+            break
+    if tool_schema:
+        required_fields = tool_schema.get('required', [])
+        for field in required_fields:
+            val = params.get(field)
+            if val is None or (isinstance(val, str) and val.strip() == ''):
+                return JsonResponse({'error': f'Missing required parameter: {field}'}, status=400)
+            # If field is a dataset_id, verify it exists in the store
+            if 'dataset' in field.lower():
+                store = SessionStore()
+                if not store.get_dataset(str(val)):
+                    return JsonResponse({'error': f'Dataset not found: {val}'}, status=400)
+
+    result = asyncio.run(handle_call(tool_name, params))
+    return JsonResponse({'status': 'ok', 'result': result})
+
+
+@csrf_exempt
+def reload_run_data(request):
+    """Reload a run's data from csv_glob or sqlite_table into SessionStore."""
+    if request.method != 'POST':
+        return JsonResponse({'error': 'POST required'}, status=405)
+    import json as _json
+    try:
+        body = _json.loads(request.body)
+    except Exception:
+        logger.error("reload_run_data failed: {}".format(traceback.format_exc()))
+        return JsonResponse({'error': 'Invalid JSON body'}, status=400)
+
+    display_id = body.get('display_id')
+    if not display_id:
+        return JsonResponse({'error': 'Missing display_id'}, status=400)
+
+    from analysis.models import AnalysisRun
+    from analysis.data_loader import load_csv_directory, load_from_db
+
+    run = get_object_or_404(AnalysisRun, display_id=display_id)
+
+    # Determine dataset_id (same keying as manual_page)
+    ds_id = f'upload_{display_id}' if run.run_type == 'upload' else f'run_{display_id}'
+
+    from analysis.session_store import SessionStore
+    store = SessionStore()
+    # Check if already loaded
+    existing = store.get_dataset(ds_id)
+    if existing is not None:
+        return JsonResponse({'status': 'ok', 'dataset_id': ds_id, 'reloaded': False})
+
+    # Try loading from sqlite_table first, then csv_glob
+    lf = None
+    schema = {}
+    row_count = 0
+    file_count = 0
+
+    if run.sqlite_table:
+        try:
+            lf = load_from_db(run.sqlite_table)
+            if lf is not None:
+                import polars as pl
+                row_count = lf.select(pl.len()).collect(streaming=True)[0, 0]
+        except Exception as exc:
+            logging.getLogger(__name__).warning(f'Failed to load from sqlite_table: {exc}')
+
+    if lf is None and run.csv_glob:
+        try:
+            from pathlib import Path
+            csv_path = Path(run.csv_glob)
+            if csv_path.is_file() or csv_path.is_dir() or csv_path.parent.is_dir():
+                lf, schema, row_count, file_count, _ = load_csv_directory(run.csv_glob)
+        except Exception as exc:
+            logging.getLogger(__name__).warning(f'Failed to load from csv_glob: {exc}')
+
+    if lf is None:
+        return JsonResponse({'error': 'No data source available (upload dir or sqlite table gone)'}, status=404)
+
+    store.store_dataset(ds_id, lf, schema=schema, metadata={
+        'row_count': row_count,
+        'file_count': file_count,
+        'csv_glob': run.csv_glob or '',
+        'sqlite_table': run.sqlite_table or '',
+    })
+
+    return JsonResponse({
+        'status': 'ok',
+        'dataset_id': ds_id,
+        'reloaded': True,
+        'row_count': row_count,
+        'columns': list(schema.keys()),
+    })
+
+
+@csrf_exempt
+def tool_plan(request):
+    """Save, list, load, delete, or pin/unpin an analysis plan."""
+    import json as _j
+    if request.method == 'GET':
+        name = request.GET.get('name')
+        if name:
+            # Auto plans stored with _auto_ prefix
+            if name.startswith('_auto_'):
+                path = _PLANS_DIR / f'{name}.json'
+            else:
+                path = _PLANS_DIR / f'{name}.json'
+            if not path.exists():
+                return JsonResponse({'error': f'Plan "{name}" not found'}, status=404)
+            try:
+                data = _j.loads(path.read_text(encoding='utf-8'))
+                return JsonResponse(data)
+            except Exception as e:
+                return JsonResponse({'error': str(e)}, status=500)
+
+        # List all plans (regular + auto)
+        auto_index = _read_auto_index()
+        auto_index_map = {e.get('file', ''): e.get('pinned', False) for e in auto_index}
+        auto_names = set()
+        plans = []
+        for p in sorted(_PLANS_DIR.glob('*.json')):
+            # Skip index file and auto plans (handled separately)
+            if p.name == '_auto_index.json':
+                continue
+            if p.name.startswith('_auto_'):
+                auto_names.add(p.name)
+                continue
+            try:
+                content = _j.loads(p.read_text(encoding='utf-8'))
+                steps = len(content.get('steps', []))
+                plans.append({'name': p.stem, 'steps': steps})
+            except Exception:
+                logger.error("unknown failed: {}".format(traceback.format_exc()))
+                plans.append({'name': p.stem, 'steps': 0})
+
+        # Add auto plans with markers
+        auto_plans = []
+        for auto_name in sorted(auto_names):
+            try:
+                path = _PLANS_DIR / auto_name
+                content = _j.loads(path.read_text(encoding='utf-8'))
+                steps = len(content.get('steps', []))
+                auto_plans.append({
+                    'name': auto_name.replace('.json', ''),
+                    'steps': steps,
+                    'auto': True,
+                    'pinned': auto_index_map.get(auto_name, False),
+                })
+            except Exception:
+                logger.error("unknown failed: {}".format(traceback.format_exc()))
+                auto_plans.append({
+                    'name': auto_name.replace('.json', ''),
+                    'steps': 0,
+                    'auto': True,
+                    'pinned': auto_index_map.get(auto_name, False),
+                })
+        # Interleave: auto plans first, then regular
+        return JsonResponse({'plans': auto_plans + plans})
+
+    elif request.method == 'POST':
+        try:
+            body = _j.loads(request.body)
+            # Pin/unpin toggle for auto plans
+            if 'pin' in body and body.get('name', '').startswith('_auto_'):
+                filename = body['name'].replace('.json', '') + '.json' \
+                    if not body['name'].endswith('.json') else body['name']
+                index = _read_auto_index()
+                new_pinned = bool(body['pin'])
+                found = False
+                for entry in index:
+                    if entry.get('file') == filename:
+                        entry['pinned'] = new_pinned
+                        found = True
+                        break
+                if not found and new_pinned:
+                    index.append({'file': filename, 'pinned': True})
+                _write_auto_index(index)
+                return JsonResponse({
+                    'status': 'pinned' if new_pinned else 'unpinned',
+                    'name': body['name'],
+                    'pinned': new_pinned,
+                })
+
+            # Regular save
+            name = body.get('name', '').strip()
+            steps = body.get('steps', [])
+            if not name:
+                return JsonResponse({'error': 'Missing plan name'}, status=400)
+            path = _PLANS_DIR / f'{name}.json'
+            path.write_text(_j.dumps({'name': name, 'steps': steps}, ensure_ascii=False, indent=2), encoding='utf-8')
+            return JsonResponse({'status': 'saved', 'name': name, 'steps': len(steps)})
+        except Exception as e:
+            return JsonResponse({'error': str(e)}, status=500)
+
+    elif request.method == 'DELETE':
+        name = request.GET.get('name', '').strip()
+        if not name:
+            return JsonResponse({'error': 'Missing plan name'}, status=400)
+        path = _PLANS_DIR / f'{name}.json'
+        if path.exists():
+            path.unlink()
+        # Also remove from auto index if applicable
+        if name.startswith('_auto_'):
+            filename = name + '.json' if not name.endswith('.json') else name
+            index = _read_auto_index()
+            index = [e for e in index if e.get('file') != filename]
+            _write_auto_index(index)
+        return JsonResponse({'status': 'deleted', 'name': name})
+
+    return JsonResponse({'error': 'Method not allowed'}, status=405)
+
+
+# ── Filter endpoint for manual analysis page ──────────────────────────
+
+@csrf_exempt
+def apply_filter(request):
+    """POST {dataset_id, filters, logic} → apply filter_data tool → return filtered_X id."""
+    if request.method != 'POST':
+        return JsonResponse({'error': 'POST required'}, status=405)
+    import json as _json
+    body = _json.loads(request.body)
+    dataset_id = body.get('dataset_id')
+    filters = body.get('filters', [])
+    logic = body.get('logic', 'and')
+
+    if not dataset_id:
+        return JsonResponse({'error': 'dataset_id is required'})
+    if not filters:
+        return JsonResponse({'error': 'filters is required'})
+
+    try:
+        from analysis.tool_registry import _handle_filter_data
+        result = asyncio.run(_handle_filter_data(
+            dataset_id=dataset_id,
+            filters=filters,
+            logic=logic,
+        ))
+        if 'error' in result:
+            return JsonResponse({'error': result['error']})
+
+        filtered_id = result.get('filtered_dataset_id', '')
+        row_count = result.get('row_count', 0)
+
+        # Use the unique flt_xxx ID from _handle_filter_data — never rename
+        # to filtered_{num} which would collide on repeated filters of the same source.
+        return JsonResponse({
+            'dataset_id': filtered_id,
+            'row_count': row_count,
+        })
+    except Exception as e:
+        logger.error(f'[apply_filter] {traceback.format_exc()}')
+        return JsonResponse({'error': str(e)})
+
+
+@csrf_exempt
+def retry_run(request, display_id):
+    """Reset a failed run and re-trigger based on run_type."""
+    if request.method != 'POST':
+        return JsonResponse({'error': 'POST required'}, status=405)
+
+    run = get_object_or_404(AnalysisRun, display_id=display_id)
+    if run.status != 'failed':
+        return JsonResponse({'error': '只能重试失败的分析运行'}, status=400)
+
+    run.status = 'pending'
+    run.error_message = ''
+    run.progress_pct = 0
+    run.progress_msg = ''
+    run.save(update_fields=['status', 'error_message', 'progress_pct', 'progress_msg'])
+
+    if run.run_type == 'upload' and run.csv_glob:
+        upload_dir = Path(run.csv_glob).parent
+        expected_base = ensure_appdata_dir() / 'data' / 'uploads'
+        if not str(upload_dir.resolve()).startswith(str(expected_base.resolve())):
+            return JsonResponse({'error': 'Invalid upload path'}, status=400)
+        if upload_dir.is_dir():
+            t = threading.Thread(
+                target=_background_process,
+                args=(run.id, upload_dir),
+                daemon=True,
+            )
+            t.start()
+        else:
+            run.error_message = '上传数据目录已不存在，请重新上传'
+            run.status = 'failed'
+            run.save(update_fields=['status', 'error_message'])
+
+    return redirect('analysis:run_detail', display_id=run.display_id)
@@ -0,0 +1,133 @@
+"""Upload views: CSV upload, batch, finalize, delete."""
+import threading
+import logging
+import shutil
+from pathlib import Path
+
+from django.shortcuts import render, get_object_or_404
+from django.http import JsonResponse
+from django.views.decorators.csrf import csrf_exempt
+
+from analysis.appdata import ensure_appdata_dir
+from analysis.models import AnalysisRun
+from .pipeline import _background_process
+
+logger = logging.getLogger(__name__)
+
+
+def upload_page(request):
+    """CSV upload page."""
+    runs = AnalysisRun.objects.all().order_by('-created_at')[:20]
+    return render(request, 'tianxuan/upload.html', {'runs': runs})
+
+
+@csrf_exempt
+def upload_csv(request):
+    """Handle CSV upload AJAX POST."""
+    if request.method != 'POST':
+        return JsonResponse({'error': 'POST required'}, status=405)
+    files = request.FILES.getlist('files')
+    if not files:
+        return JsonResponse({'error': 'No files uploaded'})
+    logger.info('[UPLOAD] Received %d files', len(files))
+    for f in files:
+        if f.size > 500 * 1024 * 1024:
+            return JsonResponse({'warning': f'文件 {f.name} 超过 500MB，建议分割后上传'})
+    total_size = sum(f.size for f in files)
+    if total_size > 50 * 1024 * 1024 * 1024:
+        return JsonResponse({'error': '总大小超过 50GB 限制'})
+    from django.utils import timezone
+    ts = timezone.now().strftime('%Y%m%d_%H%M%S')
+    from analysis.appdata import ensure_appdata_dir
+    base = ensure_appdata_dir() / 'data' / 'uploads'
+    upload_dir = base / ts
+    upload_dir.mkdir(parents=True, exist_ok=True)
+    saved = []
+    for f in files:
+        path = upload_dir / f.name
+        with open(path, 'wb+') as dst:
+            for chunk in f.chunks():
+                dst.write(chunk)
+        saved.append(str(path))
+    # Create run with status='pending' — background processing deferred to finalize endpoint
+    run = AnalysisRun.objects.create(
+        csv_glob=str(upload_dir / '*.csv'),
+        status='pending',
+        run_type='upload',
+    )
+    # Deferred: _background_process will be started by finalize_upload endpoint
+    return JsonResponse({'run_id': run.display_id, 'files': saved, 'status': 'pending'})
+
+
+@csrf_exempt
+def upload_csv_batch(request, display_id):
+    """Append files to existing upload run. Does NOT re-trigger _background_process."""
+    run = get_object_or_404(AnalysisRun, display_id=display_id)
+    files = request.FILES.getlist('files')
+    if not files:
+        return JsonResponse({'error': 'No files uploaded'})
+    upload_dir = Path(run.csv_glob).parent if run.csv_glob else None
+    if upload_dir:
+        expected_base = ensure_appdata_dir() / 'data' / 'uploads'
+        if not str(upload_dir.resolve()).startswith(str(expected_base.resolve())):
+            return JsonResponse({'error': 'Invalid upload path'}, status=400)
+    if not upload_dir or not upload_dir.exists():
+        return JsonResponse({'error': 'Upload directory not found'})
+    saved = []
+    for f in files:
+        path = upload_dir / f.name
+        with open(path, 'wb+') as dst:
+            for chunk in f.chunks():
+                dst.write(chunk)
+        saved.append(str(path))
+    # NO background process re-trigger — first batch handles it
+    return JsonResponse({'run_id': run.display_id, 'files': len(saved), 'status': 'ok'})
+
+
+@csrf_exempt
+def finalize_upload(request, display_id):
+    """Finalize batch upload: validate upload_dir, set status='loading', start background processing."""
+    if request.method != 'POST':
+        return JsonResponse({'error': 'POST required'}, status=405)
+    run = get_object_or_404(AnalysisRun, display_id=display_id)
+    if run.status != 'pending':
+        return JsonResponse({'error': f'Run status is {run.status}, expected pending'}, status=400)
+    upload_dir = Path(run.csv_glob).parent if run.csv_glob else None
+    if not upload_dir or not upload_dir.exists():
+        return JsonResponse({'error': 'Upload directory not found'}, status=400)
+    run.status = 'loading'
+    run.save(update_fields=['status'])
+    t = threading.Thread(target=_background_process, args=(run.id, upload_dir), daemon=True)
+    t.start()
+    return JsonResponse({'run_id': run.display_id, 'status': 'loading'})
+
+
+@csrf_exempt
+def delete_upload(request, display_id):
+    """Delete uploaded files, SQLite data table, and AnalysisRun.
+
+    Handles all run statuses (loading, profiling, clustering, completed, failed).
+    Gracefully tolerates null csv_glob and missing/errored SQLite tables.
+    """
+    run = get_object_or_404(AnalysisRun, display_id=display_id)
+
+    # 1. Drop the SQLite data table if one exists
+    if run.sqlite_table:
+        try:
+            from analysis.data_loader import drop_sqlite_table  # fmt: skip
+            drop_sqlite_table(run.sqlite_table)
+        except Exception as exc:
+            logger.warning('[DELETE] failed to drop SQLite table %s: %s', run.sqlite_table, exc)
+
+    # 2. Remove the upload directory if csv_glob is set
+    if run.csv_glob:
+        p = Path(run.csv_glob).parent
+        expected_base = ensure_appdata_dir() / 'data' / 'uploads'
+        if not str(p.resolve()).startswith(str(expected_base.resolve())):
+            return JsonResponse({'error': 'Invalid upload path'}, status=400)
+        if p.exists():
+            shutil.rmtree(p, ignore_errors=True)
+
+    # 3. Delete the ORM record
+    run.delete()
+    return JsonResponse({'deleted': True})
@@ -15,12 +15,7 @@ from django.views.decorators.csrf import csrf_exempt
 from config import get_config, save_config, Config
 from .models import AnalysisRun, ClusterResult, EntityProfile
 from . import nl_describe
-
-# Conditional line_profiler support (no-op when not installed)
-try:
-    profile  # type: ignore[name-defined]
-except NameError:
-    profile = lambda f: f
+from .profile_util import profile


 def dashboard(request):
@@ -91,23 +86,38 @@ def cluster_overview(request, display_id):
    entities = run.entities.exclude(embedding_x=None).exclude(embedding_y=None).only(
        'embedding_x', 'embedding_y', 'embedding_z', 'cluster_label', 'entity_value'
    )
+    has_z = any(e.embedding_z is not None for e in entities)
    scatter_data = [
-        {'x': e.embedding_x, 'y': e.embedding_y, 'z': e.embedding_z,
-         'label': e.cluster_label, 'entity': e.entity_value}
+        {'x': e.embedding_x, 'y': e.embedding_y, 'z': e.embedding_z if has_z else 0,
+         'label': e.cluster_label if e.cluster_label is not None else -1, 'entity': e.entity_value}
        for e in entities
    ]

-    # Noise cluster features for enhanced noise card
+    # Noise cluster: features + entity detail
    noise_features = []
    noise_summary = ''
+    noise_entity_count = 0
    if noise:
-        noise_features_qs = noise.features.all().order_by('-distinguishing_score')[:10]
+        noise_features_qs = noise.features.all().order_by('-distinguishing_score')[:15]
        noise_features = [
-            {'feature_name': f.feature_name, 'score': f.distinguishing_score, 'mean': f.mean}
+            {'feature_name': f.feature_name, 'score': f.distinguishing_score, 'mean': f.mean,
+             'std': f.std, 'p25': f.p25, 'p75': f.p75}
            for f in noise_features_qs
        ]
        if noise_features:
            noise_summary = nl_describe.describe_cluster(noise_features)
+        noise_entity_count = noise.size
+
+    # LLM workflow timeline for auto runs
+    llm_timeline_json = ''
+    if run.run_type == 'auto':
+        llm_thinking = run.llm_thinking or ''
+        tool_calls = run.tool_calls_json or []
+        if tool_calls or llm_thinking:
+            llm_timeline_json = json.dumps({
+                'thinking': llm_thinking,
+                'tool_calls': tool_calls,
+            })

    # Build geo scatter data from feature_json (lat/lon)
    geo_data = []
@@ -136,12 +146,14 @@ def cluster_overview(request, display_id):
    top_features = {}
    for c in clusters:
        features_qs = c.features.all().order_by('-distinguishing_score')[:10]
-        top_features[str(c.cluster_label)] = [
-            {'feature_name': f.feature_name, 'score': f.distinguishing_score, 'mean': f.mean}
+        feats = [
+            {'feature_name': f.feature_name, 'score': f.distinguishing_score, 'mean': f.mean,
+             'std': f.std}
            for f in features_qs
        ]
-        # Generate NL summary — attach to cluster object for template access
-        feats = top_features[str(c.cluster_label)]
+        top_features[str(c.cluster_label)] = feats
+        # Attach features + NL summary directly to cluster object for template
+        c._top_features = feats
        c.nl_summary = nl_describe.describe_cluster(feats) if feats else ''

    return render(request, 'analysis/cluster_overview.html', {
@@ -149,6 +161,7 @@ def cluster_overview(request, display_id):
        'clusters': clusters,
        'noise': noise,
        'scatter_data_json': json.dumps(scatter_data),
+        'has_z': has_z,
        'geo_data_json': json.dumps(geo_data),
        'geo_skipped': geo_skipped,
        'geo_count': len(geo_data),
@@ -156,21 +169,88 @@ def cluster_overview(request, display_id):
        'top_features_json': json.dumps(top_features),
        'noise_features': noise_features,
        'noise_summary': noise_summary,
+        'noise_entity_count': noise_entity_count,
+        'llm_timeline_json': llm_timeline_json,
+        # Globe flow data for the sidebar
+        'globe_flows_json': json.dumps(_get_globe_flows(run)),
    })


+def _get_globe_flows(run, max_rows=500):
+    """Extract flow data for 3D globe visualization from a run."""
+    import json
+    from analysis.data_loader import load_csv_directory, load_from_db
+    try:
+        lf = None
+        if run.sqlite_table:
+            lf = load_from_db(run.sqlite_table)
+        if lf is None and run.csv_glob:
+            lf, _, _, _, _ = load_csv_directory(run.csv_glob)
+        if lf is None:
+            return []
+        df = lf.head(max_rows).collect()
+        from analysis.geoip import lookup as geo_lookup
+        flows = []
+        TLS_HEX_MAP = {'0303': 'TLSv1.2', '0304': 'TLSv1.3', '0302': 'TLSv1.1', '0301': 'TLSv1.0'}
+        src_ip_col = next((c for c in df.columns if 'src_ip' in c.lower() or ':ips' in c.lower()), None)
+        dst_ip_col = next((c for c in df.columns if 'dst_ip' in c.lower() or ':ipd' in c.lower()), None)
+        tls_col = next((c for c in df.columns if c.lower() in ('0ver', 'tls_version', '_tlsver', 'version')), None)
+        bytes_col = next((c for c in df.columns if c.lower() in ('8ack', '8pak', '8byt', 'bytes_sent', 'bytes')), None)
+        for row in df.iter_rows(named=True):
+            sip = str(row.get(src_ip_col, '')) if src_ip_col else ''
+            dip = str(row.get(dst_ip_col, '')) if dst_ip_col else ''
+            if not sip or not dip:
+                continue
+            sg = geo_lookup(sip)
+            dg = geo_lookup(dip)
+            if not sg or not dg:
+                continue
+            tls_val = str(row.get(tls_col, '') or '') if tls_col else ''
+            tls_ver = TLS_HEX_MAP.get(tls_val.replace(' ', ''), tls_val)
+            flows.append({
+                'slat': sg['lat'], 'slon': sg['lon'],
+                'dlat': dg['lat'], 'dlon': dg['lon'],
+                'tls': tls_ver,
+                'bytes': float(row.get(bytes_col, 0) or 0) if bytes_col else 0,
+            })
+        return flows
+    except Exception:
+        return []
+
+
 def cluster_detail(request, display_id, cluster_label):
-    """Detailed view of a single cluster with entity list."""
+    """Detailed view of a single cluster with entity list, SVD features, and NL summary."""
    run = get_object_or_404(AnalysisRun, display_id=display_id)
+    try:
+        cluster_label = int(cluster_label)
+    except (ValueError, TypeError):
+        raise Http404("Invalid cluster label")
    cluster = get_object_or_404(ClusterResult, run=run, cluster_label=cluster_label)
    features = cluster.features.all().order_by('-distinguishing_score')[:50]
-    entities = cluster.entities.all()[:50]
+    entities = cluster.entities.all()[:100]
+    nl_summary = ''
+    feats_list = [{'feature_name': f.feature_name, 'score': f.distinguishing_score,
+                   'mean': f.mean, 'std': f.std} for f in features[:10]]
+    if feats_list:
+        nl_summary = nl_describe.describe_cluster(feats_list)
+
+    # Build entity data with feature_json values
+    entity_data = []
+    for e in entities:
+        entry = {'id': e.id, 'value': e.entity_value,
+                 'embedding_x': e.embedding_x, 'embedding_y': e.embedding_y}
+        if e.feature_json:
+            entry['features'] = {k: round(float(v), 4) if isinstance(v, (int, float)) else str(v)
+                                 for k, v in e.feature_json.items()}
+        entity_data.append(entry)

    return render(request, 'analysis/cluster_detail.html', {
        'run': run,
        'cluster': cluster,
        'features': features,
        'entities': entities,
+        'entity_data_json': json.dumps(entity_data),
+        'nl_summary': nl_summary,
    })


@@ -737,33 +817,49 @@ def manual_page(request):

    # Get datasets from SessionStore
    store = SessionStore()
-    datasets = store.list_datasets() if hasattr(store, 'list_datasets') else []
+    session_datasets = store.list_datasets() if hasattr(store, 'list_datasets') else []
+    session_ds_map = {ds['dataset_id']: ds for ds in session_datasets}

-    # Get upload AnalysisRun statuses for dataset badges
+    # Build dataset entries from ALL AnalysisRun records (upload/manual/auto, any status)
    from analysis.models import AnalysisRun
-    upload_runs = AnalysisRun.objects.filter(run_type='upload').order_by('-created_at')[:50]
-    run_status_map = {
-        f'upload_{r.display_id}': {
-            'status': r.status,
-            'total_flows': r.total_flows,
-            'display_id': r.display_id,
-        }
-        for r in upload_runs
-    }
-
-    # Build datasets_with_status: each dataset annotated with run status
+    all_runs = AnalysisRun.objects.all().order_by('-created_at')[:100]
    datasets_with_status = []
-    for ds in datasets:
-        run_info = run_status_map.get(ds['dataset_id'])
-        if run_info:
-            datasets_with_status.append({
-                **ds,
-                'run_status': run_info['status'],
-                'total_flows': run_info['total_flows'],
-                'display_id': run_info['display_id'],
-            })
+
+    for run in all_runs:
+        # Determine how the dataset was keyed in SessionStore
+        if run.run_type == 'upload':
+            ds_id = f'upload_{run.display_id}'
        else:
-            datasets_with_status.append({**ds, 'run_status': 'ready', 'total_flows': None, 'display_id': None})
+            ds_id = f'run_{run.display_id}'
+
+        session_entry = session_ds_map.get(ds_id)
+
+        if session_entry:
+            # Full data available in SessionStore — merge with run status
+            datasets_with_status.append({
+                **session_entry,
+                'dataset_id': ds_id,
+                'run_status': run.status,
+                'total_flows': run.total_flows,
+                'display_id': run.display_id,
+                'needs_reload': False,
+            })
+        elif run.csv_glob or run.sqlite_table:
+            # Data not in SessionStore but recoverable from disk/DB
+            datasets_with_status.append({
+                'dataset_id': ds_id,
+                'run_status': run.status,
+                'display_id': run.display_id,
+                'total_flows': run.total_flows,
+                'row_count': run.total_flows,
+                'file_count': None,
+                'column_count': 0,
+                'columns': [],
+                'svd_components': 0,
+                'needs_reload': True,
+                'csv_glob': run.csv_glob,
+                'sqlite_table': run.sqlite_table or '',
+            })

    # Map statuses to user-facing labels for the filter tabs
    # ready → 待分析, completed → 已分析, failed → 错误
@@ -772,12 +868,16 @@ def manual_page(request):
        s = ds_info['run_status']
        if s in ('completed',):
            ds_info['status_label'] = '已分析'
+            ds_info['badge_class'] = 'completed'
        elif s in ('failed',):
            ds_info['status_label'] = '错误'
+            ds_info['badge_class'] = 'failed'
        elif s in ('pending', 'loading', 'profiling', 'aggregating', 'clustering', 'extracting'):
            ds_info['status_label'] = '分析中'
+            ds_info['badge_class'] = 'analyzing'
        else:
            ds_info['status_label'] = '待分析'
+            ds_info['badge_class'] = 'ready'

    return render(request, 'tianxuan/manual.html', {
        'core_tools': core_tools,
@@ -785,7 +885,7 @@ def manual_page(request):
        'analysis_tools': analysis_tools,
        'other_tools': other_tools,
        'tools_meta': _json.dumps(meta_list, ensure_ascii=False),
-        'datasets': datasets,
+        'datasets': datasets_with_status,
        'datasets_with_status_json': _json.dumps(datasets_with_status, ensure_ascii=False),
    })

@@ -1135,7 +1235,7 @@ def manual_run_analysis(request):
    run.run_type = 'manual'
    run.save(update_fields=['run_type'])
    feature_columns = body.get('feature_columns')
-    algorithm = body.get('algorithm', 'hdbscan')
+    algorithm = body.get('algorithm', 'agglomerative')
    min_cluster_size = int(body.get('min_cluster_size', 5))
    head = body.get('head')
    cluster_mode = body.get('cluster_mode', 'raw')
@@ -1236,30 +1336,10 @@ def apply_filter(request):
        filtered_id = result.get('filtered_dataset_id', '')
        row_count = result.get('row_count', 0)

-        # Re-store with filtered_X ID pattern
-        if dataset_id.startswith('upload_'):
-            display_suffix = dataset_id.split('_', 1)[1]
-            new_id = f'filtered_{display_suffix}'
-        else:
-            new_id = filtered_id
-
-        from analysis.session_store import SessionStore
-        store = SessionStore()
-        entry = store.get_dataset(filtered_id)
-        if entry and new_id != filtered_id:
-            store.store_dataset(
-                dataset_id=new_id,
-                lazyframe=entry['lazyframe'],
-                schema=entry.get('schema', {}),
-                metadata=dict(entry.get('metadata', {}), parent_dataset_id=dataset_id),
-                parent_id=dataset_id,
-            )
-            new_id_to_use = new_id
-        else:
-            new_id_to_use = filtered_id
-
+        # Use the unique flt_xxx ID from _handle_filter_data — never rename
+        # to filtered_{num} which would collide on repeated filters of the same source.
        return JsonResponse({
-            'dataset_id': new_id_to_use,
+            'dataset_id': filtered_id,
            'row_count': row_count,
        })
    except Exception as e:
@@ -1287,22 +1367,74 @@ def run_llm_analysis_view(request):

    from analysis.session_store import SessionStore
    store = SessionStore()
-    # Use display_id for dataset key (consistent with _background_process)
+
+    # Determine dataset key format (consistent with manual_page)
    upload_ds_id = f'upload_{display_id}'
+    run_ds_id = f'run_{display_id}'
    entry = store.get_dataset(upload_ds_id)
    if entry is None:
-        # Fallback: try PK-based key (old pipeline)
-        upload_ds_id = f'upload_{pk}'
-        entry = store.get_dataset(upload_ds_id)
+        entry = store.get_dataset(run_ds_id)

    if entry is None:
-        # Try entity dataset as fallback
-        entity_ds_id = f'entity_{pk}'
-        entry = store.get_dataset(entity_ds_id)
+        # Try primary key fallback
+        upload_pk_id = f'upload_{pk}'
+        entity_pk_id = f'entity_{pk}'
+        entry = store.get_dataset(upload_pk_id)
        if entry is None:
-            return JsonResponse({'error': '数据集不存在，请先上传并等待预处理完成'})
+            entry = store.get_dataset(entity_pk_id)

-    dataset_id = upload_ds_id if store.get_dataset(upload_ds_id) else f'entity_{pk}'
+    if entry is None:
+        # Auto-reload from sqlite_table or csv_glob
+        from analysis.data_loader import load_csv_directory, load_from_db
+        lf = None
+        schema = {}
+        if run.sqlite_table:
+            try:
+                lf = load_from_db(run.sqlite_table)
+            except Exception:
+                pass
+        if lf is None and run.csv_glob:
+            try:
+                lf, schema, _, _, _ = load_csv_directory(run.csv_glob)
+            except Exception:
+                pass
+        if lf is None:
+            return JsonResponse({'error': '数据集不存在，请先上传并等待预处理完成'})
+        entry_key = upload_ds_id if run.run_type == 'upload' else run_ds_id
+        store.store_dataset(entry_key, lf, schema=schema, metadata={
+            'row_count': run.total_flows,
+            'csv_glob': run.csv_glob or '',
+            'sqlite_table': run.sqlite_table or '',
+        })
+        # Re-fetch entry
+        entry = store.get_dataset(entry_key)
+
+    # Determine the active dataset_id
+    for candidate in [upload_ds_id, run_ds_id, f'upload_{pk}', f'entity_{pk}']:
+        if store.get_dataset(candidate):
+            dataset_id = candidate
+            break
+    else:
+        return JsonResponse({'error': '数据集不存在，请先上传并等待预处理完成'})
+
+    # ── Optional: apply filter before LLM pipeline ──
+    filters = body.get('filters', [])
+    if filters and len(filters) > 0:
+        logic = body.get('logic', 'and')
+        try:
+            from analysis.tool_registry import _handle_filter_data
+            filter_result = asyncio.run(_handle_filter_data(
+                dataset_id=dataset_id,
+                filters=filters,
+                logic=logic,
+            ))
+            if 'error' not in filter_result:
+                filtered_id = filter_result.get('filtered_dataset_id', '')
+                if filtered_id and store.get_dataset(filtered_id):
+                    dataset_id = filtered_id
+        except Exception:
+            import traceback
+            logger.warning(f'[LLM auto] Filter failed (non-fatal): {traceback.format_exc()}')

    def _analysis_fn(run, ctx):
        from tianxuan.llm_orchestrator import run_llm_pipeline, LLMConfig
@@ -1319,26 +1451,22 @@ def run_llm_analysis_view(request):

                save_fields = ['progress_pct', 'progress_msg']
                if tool_name == 'thinking':
-                    # Accumulate LLM thinking text
+                    # Accumulate LLM thinking text (appended to llm_thinking)
                    if result:
                        run.llm_thinking = (run.llm_thinking or '') + f'\n[{step}] {result}'
                    save_fields = ['progress_pct', 'progress_msg', 'llm_thinking']
                elif tool_name == 'complete':
                    pass  # No extra tracking needed
                else:
-                    # Real tool call — log it
-                    if result:
-                        if isinstance(result, dict):
-                            # Format dict as JSON, truncate large values
-                            import json as _json
-                            summary = _json.dumps(result, default=str, ensure_ascii=False)[:500]
-                        else:
-                            summary = str(result)[:500]
-                        run.run_log += f'[{step}] {tool_name}: {summary}...\n'
+                    # Single tool call — always has result
+                    if isinstance(result, dict):
+                        import json as _json
+                        summary = _json.dumps(result, default=str, ensure_ascii=False)[:500]
                    else:
-                        run.run_log += f'[{step}] {tool_name}\n'
+                        summary = str(result)[:500]
+                    run.run_log += f'[{step}] {tool_name}: {summary}...\n'

-                    # Capture structured tool call data
+                    # One entry per tool call
                    if meta and isinstance(meta, dict) and 'args' in meta:
                        tool_calls = list(run.tool_calls_json or [])
                        tool_calls.append({
@@ -1408,7 +1536,7 @@ def run_llm_analysis_view(request):
                        store=store,
                        entity_ds_id=entity_ds_id,
                        feature_columns=None,
-                        algorithm='hdbscan',
+                        algorithm='agglomerative',
                        min_cluster_size=5,
                        run_umap=True,
                        head=ctx.get('head'),
@@ -1488,37 +1616,67 @@ def auto_page(request):

    cfg = get_config()

-    # Get upload runs for dataset selector (with status badges)
-    runs = AnalysisRun.objects.filter(run_type='upload').order_by('-created_at')[:50]
+    # Get datasets from SessionStore + ALL runs from DB
+    store = SessionStore()
+    session_datasets = store.list_datasets() if hasattr(store, 'list_datasets') else []
+    session_ds_map = {ds['dataset_id']: ds for ds in session_datasets}

-    # Build datasets_with_status for filter tabs
-    from analysis.models import AnalysisRun as _AR
-    upload_runs = _AR.objects.filter(run_type='upload').order_by('-created_at')[:50]
+    # Build dataset entries from ALL AnalysisRun records
+    all_runs = AnalysisRun.objects.all().order_by('-created_at')[:100]
    datasets_with_status = []
-    for r in upload_runs:
-        s = r.status
-        if s in ('completed',):
-            label = '已分析'
-        elif s in ('failed',):
-            label = '错误'
-        elif s in ('pending', 'loading', 'profiling', 'aggregating', 'clustering', 'extracting'):
-            label = '分析中'
+
+    for run in all_runs:
+        if run.run_type == 'upload':
+            ds_id = f'upload_{run.display_id}'
        else:
-            label = '待分析'
-        datasets_with_status.append({
-            'dataset_id': r.display_id,
-            'display_id': r.display_id,
-            'run_status': s,
-            'status_label': label,
-            'row_count': r.total_flows,
-            'file_count': getattr(r, 'file_count', None),
-            'csv_glob': r.csv_glob or '',
-            'sqlite_table': r.sqlite_table or '',
-        })
+            ds_id = f'run_{run.display_id}'
+
+        session_entry = session_ds_map.get(ds_id)
+
+        if session_entry:
+            datasets_with_status.append({
+                **session_entry,
+                'dataset_id': ds_id,
+                'run_status': run.status,
+                'total_flows': run.total_flows,
+                'display_id': run.display_id,
+                'needs_reload': False,
+                'csv_glob': run.csv_glob or '',
+                'sqlite_table': run.sqlite_table or '',
+            })
+        elif run.csv_glob or run.sqlite_table:
+            datasets_with_status.append({
+                'dataset_id': ds_id,
+                'run_status': run.status,
+                'display_id': run.display_id,
+                'total_flows': run.total_flows,
+                'row_count': run.total_flows,
+                'file_count': None,
+                'columns': [],
+                'column_count': 0,
+                'svd_components': 0,
+                'needs_reload': True,
+                'csv_glob': run.csv_glob or '',
+                'sqlite_table': run.sqlite_table or '',
+            })
+
+    for ds_info in datasets_with_status:
+        s = ds_info['run_status']
+        if s in ('completed',):
+            ds_info['status_label'] = '已分析'
+            ds_info['badge_class'] = 'completed'
+        elif s in ('failed',):
+            ds_info['status_label'] = '错误'
+            ds_info['badge_class'] = 'failed'
+        elif s in ('pending', 'loading', 'profiling', 'aggregating', 'clustering', 'extracting'):
+            ds_info['status_label'] = '分析中'
+            ds_info['badge_class'] = 'analyzing'
+        else:
+            ds_info['status_label'] = '待分析'
+            ds_info['badge_class'] = 'ready'

    return render(request, 'tianxuan/auto.html', {
        'llm_configured': bool(cfg.llm.base_url and cfg.llm.api_key),
-        'runs': runs,
        'datasets_with_status_json': _json.dumps(datasets_with_status, ensure_ascii=False),
    })

@@ -1872,6 +2030,79 @@ def tool_lab_run(request):
    return JsonResponse({'status': 'ok', 'result': result})


+@csrf_exempt
+def reload_run_data(request):
+    """Reload a run's data from csv_glob or sqlite_table into SessionStore."""
+    if request.method != 'POST':
+        return JsonResponse({'error': 'POST required'}, status=405)
+    import json as _json
+    try:
+        body = _json.loads(request.body)
+    except Exception:
+        return JsonResponse({'error': 'Invalid JSON body'}, status=400)
+
+    display_id = body.get('display_id')
+    if not display_id:
+        return JsonResponse({'error': 'Missing display_id'}, status=400)
+
+    from analysis.models import AnalysisRun
+    from analysis.data_loader import load_csv_directory, load_from_db
+
+    run = get_object_or_404(AnalysisRun, display_id=display_id)
+
+    # Determine dataset_id (same keying as manual_page)
+    ds_id = f'upload_{display_id}' if run.run_type == 'upload' else f'run_{display_id}'
+
+    store = SessionStore()
+    # Check if already loaded
+    existing = store.get_dataset(ds_id)
+    if existing is not None:
+        return JsonResponse({'status': 'ok', 'dataset_id': ds_id, 'reloaded': False})
+
+    # Try loading from sqlite_table first, then csv_glob
+    lf = None
+    schema = {}
+    row_count = 0
+    file_count = 0
+
+    if run.sqlite_table:
+        try:
+            lf = load_from_db(run.sqlite_table)
+            if lf is not None:
+                row_count = lf.select(pl.len()).collect(streaming=True)[0, 0]
+        except Exception as exc:
+            import logging
+            logging.getLogger(__name__).warning(f'Failed to load from sqlite_table: {exc}')
+
+    if lf is None and run.csv_glob:
+        try:
+            from pathlib import Path
+            csv_path = Path(run.csv_glob)
+            if csv_path.is_file() or csv_path.is_dir() or csv_path.parent.is_dir():
+                lf, schema, row_count, file_count, _ = load_csv_directory(run.csv_glob)
+        except Exception as exc:
+            import logging
+            logging.getLogger(__name__).warning(f'Failed to load from csv_glob: {exc}')
+
+    if lf is None:
+        return JsonResponse({'error': 'No data source available (upload dir or sqlite table gone)'}, status=404)
+
+    store.store_dataset(ds_id, lf, schema=schema, metadata={
+        'row_count': row_count,
+        'file_count': file_count,
+        'csv_glob': run.csv_glob or '',
+        'sqlite_table': run.sqlite_table or '',
+    })
+
+    return JsonResponse({
+        'status': 'ok',
+        'dataset_id': ds_id,
+        'reloaded': True,
+        'row_count': row_count,
+        'columns': list(schema.keys()),
+    })
+
+
 # ── Save / Load / Delete analysis plans ─────────────────────────────

 from pathlib import Path
@@ -2070,3 +2301,8 @@ def retry_run(request, display_id):

    return redirect('analysis:run_detail', display_id=run.display_id)

+
+# Auto-profile all public functions in this module
+from analysis.profile_util import auto_profile_module  # noqa: E402
+auto_profile_module(__name__)
+
@@ -25,7 +25,7 @@ anomaly_detection:
 llm:
  enabled: true
  base_url: 'https://api.deepseek.com'
-  api_key: 'sk-ecd33e5341ff4ec9a2c5a5c97a97699b'
+  api_key: 'sk-your-api-key-here'
  model: deepseek-v4-flash
 entity:
  subnet_masks: [24, 28]
@@ -19,7 +19,7 @@ class Config(BaseModel):
        recursive: bool = False

    class Clustering(BaseModel):
-        algorithm: str = 'hdbscan'
+        algorithm: str = 'agglomerative'
        min_cluster_size: int = 5
        random_state: int = 42

@@ -0,0 +1,49 @@
+{
+  "8.8.8.8": {
+    "lat": 37.422,
+    "lon": -122.084,
+    "city": "Sunnyvale",
+    "country": "US"
+  },
+  "1.1.1.1": {
+    "lat": 37.7749,
+    "lon": -122.4194,
+    "city": "San Francisco",
+    "country": "US"
+  },
+  "52.0.0.1": {
+    "lat": 39.0438,
+    "lon": -77.4874,
+    "city": "Ashburn",
+    "country": "US"
+  },
+  "104.16.1.1": {
+    "lat": 37.7749,
+    "lon": -122.4194,
+    "city": "San Francisco",
+    "country": "US"
+  },
+  "47.96.1.1": {
+    "lat": 31.2304,
+    "lon": 121.4737,
+    "city": "Shanghai",
+    "country": "CN"
+  },
+  "218.1.1.1": null,
+  "157.1.1.1": {
+    "lat": 34.6937,
+    "lon": 135.5023,
+    "city": "Japan",
+    "country": "JP"
+  },
+  "10.0.0.1": null,
+  "192.168.1.1": null,
+  "13.250.0.1": null,
+  "185.234.0.1": null,
+  "103.235.0.1": {
+    "lat": 20.0,
+    "lon": 77.0,
+    "city": "",
+    "country": "印度"
+  }
+}
@@ -1,808 +0,0 @@
-# ===== Offline GeoIP Database — 天璇 (TianXuan) =====
-# Format: start_ip,end_ip,lat,lon,city,country
-# Layers: Private, DNS, AWS, Azure, GCP, Alibaba, Cloudflare, Akamai, Fastly, CN_ISP, US_ISP, EU_ISP, JP_ISP
-# Strategy: priority-ordered, overlap-resolved (first wins), gaps filled with city blocks
-
-1.0.0.0,1.0.0.0,31.2304,121.4737,Shanghai,CN
-1.0.0.1,1.0.0.1,37.7749,-122.4194,San Francisco,US
-1.0.0.2,1.1.1.0,31.2304,121.4737,Shanghai,CN
-1.1.1.1,1.1.1.1,37.7749,-122.4194,San Francisco,US
-1.1.1.2,1.255.255.255,31.2304,121.4737,Shanghai,CN
-2.0.0.0,2.15.255.255,51.5074,-0.1278,London,GB
-2.16.0.0,2.23.255.255,42.3736,-71.1097,Cambridge,US
-2.24.0.0,2.255.255.255,51.5074,-0.1278,London,GB
-3.0.0.0,3.1.255.255,1.3521,103.8198,Singapore,SG
-3.2.0.0,3.5.255.255,31.2304,121.4737,Shanghai,CN
-3.6.0.0,3.7.255.255,19.076,72.8777,Mumbai,IN
-3.8.0.0,3.15.255.255,51.5074,-0.1278,London,GB
-3.16.0.0,3.23.255.255,39.9042,116.4074,Beijing,CN
-3.24.0.0,3.27.255.255,-33.8688,151.2093,Sydney,AU
-3.28.0.0,3.33.255.255,35.6762,139.6503,Tokyo,JP
-3.34.0.0,3.35.255.255,37.5665,126.978,Seoul,KR
-3.36.0.0,3.51.255.255,37.5665,126.978,Seoul,KR
-3.52.0.0,3.63.255.255,1.3521,103.8198,Singapore,SG
-3.64.0.0,3.79.255.255,50.1109,8.6821,Frankfurt,DE
-3.80.0.0,3.95.255.255,51.5074,-0.1278,London,GB
-3.96.0.0,3.99.255.255,45.5017,-73.5673,Montreal,CA
-3.100.0.0,3.111.255.255,50.1109,8.6821,Frankfurt,DE
-3.112.0.0,3.115.255.255,35.6762,139.6503,Tokyo,JP
-3.116.0.0,3.131.255.255,40.7128,-74.006,New York,US
-3.132.0.0,3.147.255.255,37.7749,-122.4194,San Francisco,US
-3.148.0.0,3.163.255.255,-33.8688,151.2093,Sydney,AU
-3.164.0.0,3.179.255.255,19.076,72.8777,Mumbai,IN
-3.180.0.0,3.195.255.255,48.8566,2.3522,Paris,FR
-3.196.0.0,3.207.255.255,52.3676,4.9041,Amsterdam,NL
-3.208.0.0,3.215.255.255,39.0438,-77.4874,Ashburn,US
-3.216.0.0,3.223.255.255,-23.5505,-46.6333,Sao Paulo,BR
-3.224.0.0,3.239.255.255,39.0438,-77.4874,Ashburn,US
-3.240.0.0,4.111.255.255,55.7558,37.6173,Moscow,RU
-4.112.0.0,4.239.255.255,41.8781,-87.6298,Chicago,US
-4.240.0.0,4.255.255.255,32.7767,-96.797,Dallas,US
-5.0.0.0,5.255.255.255,51.5074,-0.1278,London,GB
-6.0.0.0,6.127.255.255,33.749,-84.388,Atlanta,US
-6.128.0.0,6.255.255.255,43.6532,-79.3832,Toronto,CA
-7.0.0.0,7.127.255.255,22.3193,114.1694,Hong Kong,HK
-7.128.0.0,7.255.255.255,25.033,121.5654,Taipei,TW
-8.0.0.0,8.8.4.3,40.4168,-3.7038,Madrid,ES
-8.8.4.4,8.8.4.4,37.3861,-122.0839,Mountain View,US
-8.8.4.5,8.8.8.7,45.4642,9.19,Milan,IT
-8.8.8.8,8.8.8.8,37.3861,-122.0839,Mountain View,US
-8.8.8.9,8.136.8.8,59.3293,18.0686,Stockholm,SE
-8.136.8.9,9.8.8.8,34.6937,135.5022,Osaka,JP
-9.8.8.9,9.136.8.8,23.1291,113.2644,Guangzhou,CN
-9.136.8.9,9.255.255.255,22.5431,114.0579,Shenzhen,CN
-10.0.0.0,10.255.255.255,39.9042,116.4074,Beijing,CN
-11.0.0.0,11.127.255.255,52.52,13.405,Berlin,DE
-11.128.0.0,11.255.255.255,25.2048,55.2708,Dubai,AE
-12.0.0.0,12.255.255.255,32.7767,-96.797,Dallas,US
-13.0.0.0,13.15.255.255,13.7563,100.5018,Bangkok,TH
-13.16.0.0,13.31.255.255,31.2304,121.4737,Shanghai,CN
-13.32.0.0,13.47.255.255,39.9042,116.4074,Beijing,CN
-13.48.0.0,13.51.255.255,59.3293,18.0686,Stockholm,SE
-13.52.0.0,13.55.255.255,37.7749,-122.4194,San Francisco,US
-13.56.0.0,13.59.255.255,37.7749,-122.4194,San Francisco,US
-13.60.0.0,13.63.255.255,35.6762,139.6503,Tokyo,JP
-13.64.0.0,13.95.255.255,37.7749,-122.4194,San Francisco,US
-13.96.0.0,13.111.255.255,37.5665,126.978,Seoul,KR
-13.112.0.0,13.115.255.255,35.6762,139.6503,Tokyo,JP
-13.116.0.0,13.123.255.255,1.3521,103.8198,Singapore,SG
-13.124.0.0,13.124.255.255,37.5665,126.978,Seoul,KR
-13.125.0.0,13.125.255.255,51.5074,-0.1278,London,GB
-13.126.0.0,13.127.255.255,19.076,72.8777,Mumbai,IN
-13.128.0.0,13.143.255.255,50.1109,8.6821,Frankfurt,DE
-13.144.0.0,13.159.255.255,40.7128,-74.006,New York,US
-13.160.0.0,13.175.255.255,37.7749,-122.4194,San Francisco,US
-13.176.0.0,13.191.255.255,-33.8688,151.2093,Sydney,AU
-13.192.0.0,13.207.255.255,19.076,72.8777,Mumbai,IN
-13.208.0.0,13.208.255.255,34.6937,135.5022,Osaka,JP
-13.209.0.0,13.209.255.255,37.5665,126.978,Seoul,KR
-13.210.0.0,13.211.255.255,-33.8688,151.2093,Sydney,AU
-13.212.0.0,13.215.255.255,1.3521,103.8198,Singapore,SG
-13.216.0.0,13.227.255.255,48.8566,2.3522,Paris,FR
-13.228.0.0,13.229.255.255,1.3521,103.8198,Singapore,SG
-13.230.0.0,13.231.255.255,35.6762,139.6503,Tokyo,JP
-13.232.0.0,13.243.255.255,52.3676,4.9041,Amsterdam,NL
-13.244.0.0,13.245.255.255,-33.9249,18.4241,Cape Town,ZA
-13.246.0.0,13.255.255.255,-23.5505,-46.6333,Sao Paulo,BR
-14.0.0.0,14.255.255.255,39.9042,116.4074,Beijing,CN
-15.0.0.0,15.127.255.255,55.7558,37.6173,Moscow,RU
-15.128.0.0,15.151.255.255,41.8781,-87.6298,Chicago,US
-15.152.0.0,15.152.255.255,34.6937,135.5022,Osaka,JP
-15.153.0.0,15.159.255.255,32.7767,-96.797,Dallas,US
-15.160.0.0,15.160.255.255,45.4642,9.19,Milan,IT
-15.161.0.0,15.163.255.255,33.749,-84.388,Atlanta,US
-15.164.0.0,15.164.255.255,37.5665,126.978,Seoul,KR
-15.165.0.0,15.180.255.255,43.6532,-79.3832,Toronto,CA
-15.181.0.0,15.183.255.255,22.3193,114.1694,Hong Kong,HK
-15.184.0.0,15.185.255.255,26.0667,50.5577,Bahrain,BH
-15.186.0.0,15.187.255.255,25.033,121.5654,Taipei,TW
-15.188.0.0,15.188.255.255,48.8566,2.3522,Paris,FR
-15.189.0.0,15.204.255.255,40.4168,-3.7038,Madrid,ES
-15.205.0.0,15.205.255.255,45.4642,9.19,Milan,IT
-15.206.0.0,15.206.255.255,19.076,72.8777,Mumbai,IN
-15.207.0.0,15.221.255.255,59.3293,18.0686,Stockholm,SE
-15.222.0.0,15.223.255.255,45.5017,-73.5673,Montreal,CA
-15.224.0.0,15.227.255.255,34.6937,135.5022,Osaka,JP
-15.228.0.0,15.229.255.255,-23.5505,-46.6333,Sao Paulo,BR
-15.230.0.0,15.245.255.255,23.1291,113.2644,Guangzhou,CN
-15.246.0.0,16.5.255.255,22.5431,114.0579,Shenzhen,CN
-16.6.0.0,16.15.255.255,52.52,13.405,Berlin,DE
-16.16.0.0,16.16.255.255,59.3293,18.0686,Stockholm,SE
-16.17.0.0,16.144.255.255,25.2048,55.2708,Dubai,AE
-16.145.0.0,16.169.255.255,13.7563,100.5018,Bangkok,TH
-16.170.0.0,16.171.255.255,59.3293,18.0686,Stockholm,SE
-16.172.0.0,17.43.255.255,31.2304,121.4737,Shanghai,CN
-17.44.0.0,17.171.255.255,39.9042,116.4074,Beijing,CN
-17.172.0.0,18.43.255.255,35.6762,139.6503,Tokyo,JP
-18.44.0.0,18.135.255.255,37.5665,126.978,Seoul,KR
-18.136.0.0,18.139.255.255,1.3521,103.8198,Singapore,SG
-18.140.0.0,18.155.255.255,1.3521,103.8198,Singapore,SG
-18.156.0.0,18.161.255.255,51.5074,-0.1278,London,GB
-18.162.0.0,18.163.255.255,22.3193,114.1694,Hong Kong,HK
-18.164.0.0,18.167.255.255,50.1109,8.6821,Frankfurt,DE
-18.168.0.0,18.171.255.255,51.5074,-0.1278,London,GB
-18.172.0.0,18.175.255.255,40.7128,-74.006,New York,US
-18.176.0.0,18.177.255.255,35.6762,139.6503,Tokyo,JP
-18.178.0.0,18.183.255.255,37.7749,-122.4194,San Francisco,US
-18.184.0.0,18.187.255.255,50.1109,8.6821,Frankfurt,DE
-18.188.0.0,18.193.255.255,-33.8688,151.2093,Sydney,AU
-18.194.0.0,18.195.255.255,50.1109,8.6821,Frankfurt,DE
-18.196.0.0,18.211.255.255,19.076,72.8777,Mumbai,IN
-18.212.0.0,18.227.255.255,48.8566,2.3522,Paris,FR
-18.228.0.0,18.229.255.255,-23.5505,-46.6333,Sao Paulo,BR
-18.230.0.0,18.231.255.255,52.3676,4.9041,Amsterdam,NL
-18.232.0.0,18.235.255.255,39.0438,-77.4874,Ashburn,US
-18.236.0.0,19.107.255.255,-23.5505,-46.6333,Sao Paulo,BR
-19.108.0.0,19.235.255.255,55.7558,37.6173,Moscow,RU
-19.236.0.0,19.255.255.255,41.8781,-87.6298,Chicago,US
-20.0.0.0,20.31.255.255,39.0438,-77.4874,Ashburn,US
-20.32.0.0,20.35.255.255,32.7767,-96.797,Dallas,US
-20.36.0.0,20.39.255.255,-33.8688,151.2093,Sydney,AU
-20.40.0.0,20.40.255.255,33.749,-84.388,Atlanta,US
-20.41.0.0,20.41.255.255,37.5665,126.978,Seoul,KR
-20.42.0.0,20.42.255.255,43.6532,-79.3832,Toronto,CA
-20.43.0.0,20.43.255.255,48.8566,2.3522,Paris,FR
-20.44.0.0,20.45.255.255,35.6762,139.6503,Tokyo,JP
-20.46.0.0,20.48.255.255,22.3193,114.1694,Hong Kong,HK
-20.49.0.0,20.49.255.255,51.5074,-0.1278,London,GB
-20.50.0.0,20.51.255.255,25.033,121.5654,Taipei,TW
-20.52.0.0,20.52.255.255,50.1109,8.6821,Frankfurt,DE
-20.53.0.0,20.55.255.255,40.4168,-3.7038,Madrid,ES
-20.56.0.0,20.56.255.255,52.3676,4.9041,Amsterdam,NL
-20.57.0.0,20.72.255.255,45.4642,9.19,Milan,IT
-20.73.0.0,20.88.255.255,59.3293,18.0686,Stockholm,SE
-20.89.0.0,20.104.255.255,34.6937,135.5022,Osaka,JP
-20.105.0.0,20.120.255.255,23.1291,113.2644,Guangzhou,CN
-20.121.0.0,20.136.255.255,22.5431,114.0579,Shenzhen,CN
-20.137.0.0,20.152.255.255,52.52,13.405,Berlin,DE
-20.153.0.0,20.168.255.255,25.2048,55.2708,Dubai,AE
-20.169.0.0,20.183.255.255,13.7563,100.5018,Bangkok,TH
-20.184.0.0,20.185.255.255,1.3521,103.8198,Singapore,SG
-20.186.0.0,20.191.255.255,31.2304,121.4737,Shanghai,CN
-20.192.0.0,20.192.255.255,19.076,72.8777,Mumbai,IN
-20.193.0.0,20.195.255.255,39.9042,116.4074,Beijing,CN
-20.196.0.0,20.196.255.255,22.3193,114.1694,Hong Kong,HK
-20.197.0.0,20.197.255.255,-23.5505,-46.6333,Sao Paulo,BR
-20.198.0.0,21.69.255.255,35.6762,139.6503,Tokyo,JP
-21.70.0.0,21.197.255.255,37.5665,126.978,Seoul,KR
-21.198.0.0,22.69.255.255,1.3521,103.8198,Singapore,SG
-22.70.0.0,22.197.255.255,51.5074,-0.1278,London,GB
-22.198.0.0,22.255.255.255,50.1109,8.6821,Frankfurt,DE
-23.0.0.0,23.15.255.255,42.3736,-71.1097,Cambridge,US
-23.16.0.0,23.31.255.255,40.7128,-74.006,New York,US
-23.32.0.0,23.63.255.255,42.3736,-71.1097,Cambridge,US
-23.64.0.0,23.67.255.255,42.3736,-71.1097,Cambridge,US
-23.68.0.0,23.71.255.255,37.7749,-122.4194,San Francisco,US
-23.72.0.0,23.79.255.255,42.3736,-71.1097,Cambridge,US
-23.80.0.0,23.95.255.255,-33.8688,151.2093,Sydney,AU
-23.96.0.0,23.103.255.255,41.8781,-87.6298,Chicago,US
-23.104.0.0,23.119.255.255,19.076,72.8777,Mumbai,IN
-23.120.0.0,23.135.255.255,48.8566,2.3522,Paris,FR
-23.136.0.0,23.151.255.255,52.3676,4.9041,Amsterdam,NL
-23.152.0.0,23.167.255.255,-23.5505,-46.6333,Sao Paulo,BR
-23.168.0.0,23.183.255.255,55.7558,37.6173,Moscow,RU
-23.184.0.0,23.191.255.255,41.8781,-87.6298,Chicago,US
-23.192.0.0,23.223.255.255,42.3736,-71.1097,Cambridge,US
-23.224.0.0,23.239.255.255,32.7767,-96.797,Dallas,US
-23.240.0.0,23.255.255.255,33.749,-84.388,Atlanta,US
-24.0.0.0,24.255.255.255,40.7128,-74.006,New York,US
-25.0.0.0,25.255.255.255,51.5074,-0.1278,London,GB
-26.0.0.0,26.127.255.255,43.6532,-79.3832,Toronto,CA
-26.128.0.0,26.255.255.255,22.3193,114.1694,Hong Kong,HK
-27.0.0.0,27.255.255.255,31.2304,121.4737,Shanghai,CN
-28.0.0.0,28.127.255.255,25.033,121.5654,Taipei,TW
-28.128.0.0,28.255.255.255,40.4168,-3.7038,Madrid,ES
-29.0.0.0,29.127.255.255,45.4642,9.19,Milan,IT
-29.128.0.0,29.255.255.255,59.3293,18.0686,Stockholm,SE
-30.0.0.0,30.127.255.255,34.6937,135.5022,Osaka,JP
-30.128.0.0,30.255.255.255,23.1291,113.2644,Guangzhou,CN
-31.0.0.0,31.255.255.255,51.5074,-0.1278,London,GB
-32.0.0.0,32.255.255.255,33.749,-84.388,Atlanta,US
-33.0.0.0,33.127.255.255,22.5431,114.0579,Shenzhen,CN
-33.128.0.0,33.255.255.255,52.52,13.405,Berlin,DE
-34.0.0.0,34.207.255.255,39.0438,-77.4874,Ashburn,US
-34.208.0.0,34.223.255.255,45.5152,-122.6784,Portland,US
-34.224.0.0,34.239.255.255,39.0438,-77.4874,Ashburn,US
-34.240.0.0,34.247.255.255,53.3498,-6.2603,Dublin,IE
-34.248.0.0,34.255.255.255,39.0438,-77.4874,Ashburn,US
-35.0.0.0,35.127.255.255,25.2048,55.2708,Dubai,AE
-35.128.0.0,35.151.255.255,13.7563,100.5018,Bangkok,TH
-35.152.0.0,35.152.255.255,45.4642,9.19,Milan,IT
-35.153.0.0,35.153.255.255,31.2304,121.4737,Shanghai,CN
-35.154.0.0,35.155.255.255,19.076,72.8777,Mumbai,IN
-35.156.0.0,35.159.255.255,50.1109,8.6821,Frankfurt,DE
-35.160.0.0,35.175.255.255,39.9042,116.4074,Beijing,CN
-35.176.0.0,35.177.255.255,51.5074,-0.1278,London,GB
-35.178.0.0,35.179.255.255,51.5074,-0.1278,London,GB
-35.180.0.0,35.183.255.255,48.8566,2.3522,Paris,FR
-35.184.0.0,35.191.255.255,35.6762,139.6503,Tokyo,JP
-35.192.0.0,35.195.255.255,25.033,121.5654,Taipei,TW
-35.196.0.0,35.198.255.255,37.5665,126.978,Seoul,KR
-35.199.0.0,35.199.255.255,-23.5505,-46.6333,Sao Paulo,BR
-35.200.0.0,35.203.255.255,35.6762,139.6503,Tokyo,JP
-35.204.0.0,35.207.255.255,52.3676,4.9041,Amsterdam,NL
-35.208.0.0,35.215.255.255,1.3521,103.8198,Singapore,SG
-35.216.0.0,35.219.255.255,37.5665,126.978,Seoul,KR
-35.220.0.0,35.221.255.255,32.7767,-96.797,Dallas,US
-35.222.0.0,35.223.255.255,41.8781,-87.6298,Chicago,US
-35.224.0.0,35.239.255.255,39.0438,-77.4874,Ashburn,US
-35.240.0.0,35.247.255.255,51.5074,-0.1278,London,GB
-35.248.0.0,35.255.255.255,51.5074,-0.1278,London,GB
-36.0.0.0,36.255.255.255,23.1291,113.2644,Guangzhou,CN
-37.0.0.0,37.255.255.255,48.8566,2.3522,Paris,FR
-38.0.0.0,38.127.255.255,50.1109,8.6821,Frankfurt,DE
-38.128.0.0,38.255.255.255,40.7128,-74.006,New York,US
-39.0.0.0,39.95.255.255,37.7749,-122.4194,San Francisco,US
-39.96.0.0,39.103.255.255,39.9042,116.4074,Beijing,CN
-39.104.0.0,39.231.255.255,-33.8688,151.2093,Sydney,AU
-39.232.0.0,40.63.255.255,19.076,72.8777,Mumbai,IN
-40.64.0.0,40.127.255.255,39.0438,-77.4874,Ashburn,US
-40.128.0.0,40.255.255.255,48.8566,2.3522,Paris,FR
-41.0.0.0,41.127.255.255,52.3676,4.9041,Amsterdam,NL
-41.128.0.0,41.255.255.255,-23.5505,-46.6333,Sao Paulo,BR
-42.0.0.0,42.255.255.255,39.9042,116.4074,Beijing,CN
-43.0.0.0,43.127.255.255,55.7558,37.6173,Moscow,RU
-43.128.0.0,43.255.255.255,41.8781,-87.6298,Chicago,US
-44.0.0.0,44.127.255.255,32.7767,-96.797,Dallas,US
-44.128.0.0,44.191.255.255,33.749,-84.388,Atlanta,US
-44.192.0.0,44.223.255.255,39.0438,-77.4874,Ashburn,US
-44.224.0.0,44.255.255.255,45.5152,-122.6784,Portland,US
-45.0.0.0,45.255.255.255,40.7128,-74.006,New York,US
-46.0.0.0,46.136.255.255,52.3676,4.9041,Amsterdam,NL
-46.137.0.0,46.137.127.255,1.3521,103.8198,Singapore,SG
-46.137.128.0,46.255.255.255,52.3676,4.9041,Amsterdam,NL
-47.0.0.0,47.15.255.255,43.6532,-79.3832,Toronto,CA
-47.16.0.0,47.31.255.255,22.3193,114.1694,Hong Kong,HK
-47.32.0.0,47.47.255.255,25.033,121.5654,Taipei,TW
-47.48.0.0,47.51.255.255,40.4168,-3.7038,Madrid,ES
-47.52.0.0,47.55.255.255,22.3193,114.1694,Hong Kong,HK
-47.56.0.0,47.57.255.255,22.3193,114.1694,Hong Kong,HK
-47.58.0.0,47.73.255.255,45.4642,9.19,Milan,IT
-47.74.0.0,47.75.255.255,1.3521,103.8198,Singapore,SG
-47.76.0.0,47.77.255.255,37.7749,-122.4194,San Francisco,US
-47.78.0.0,47.87.255.255,59.3293,18.0686,Stockholm,SE
-47.88.0.0,47.88.255.255,1.3521,103.8198,Singapore,SG
-47.89.0.0,47.89.255.255,22.3193,114.1694,Hong Kong,HK
-47.90.0.0,47.90.255.255,34.6937,135.5022,Osaka,JP
-47.91.0.0,47.91.255.255,35.6762,139.6503,Tokyo,JP
-47.92.0.0,47.95.255.255,39.9042,116.4074,Beijing,CN
-47.96.0.0,47.97.255.255,30.2741,120.1551,Hangzhou,CN
-47.98.0.0,47.99.255.255,30.2741,120.1551,Hangzhou,CN
-47.100.0.0,47.103.255.255,31.2304,121.4737,Shanghai,CN
-47.104.0.0,47.105.255.255,23.1291,113.2644,Guangzhou,CN
-47.106.0.0,47.106.255.255,22.5431,114.0579,Shenzhen,CN
-47.107.0.0,47.107.255.255,22.5431,114.0579,Shenzhen,CN
-47.108.0.0,47.109.255.255,30.5728,104.0668,Chengdu,CN
-47.110.0.0,47.111.255.255,52.52,13.405,Berlin,DE
-47.112.0.0,47.112.255.255,19.076,72.8777,Mumbai,IN
-47.113.0.0,47.113.255.255,25.2048,55.2708,Dubai,AE
-47.114.0.0,47.114.255.255,-33.8688,151.2093,Sydney,AU
-47.115.0.0,47.115.255.255,22.5431,114.0579,Shenzhen,CN
-47.116.0.0,47.116.255.255,50.1109,8.6821,Frankfurt,DE
-47.117.0.0,47.117.255.255,35.6762,139.6503,Tokyo,JP
-47.118.0.0,47.118.255.255,51.5074,-0.1278,London,GB
-47.119.0.0,47.119.255.255,25.2048,55.2708,Dubai,AE
-47.120.0.0,47.120.255.255,3.139,101.6869,Kuala Lumpur,MY
-47.121.0.0,47.121.255.255,13.7563,100.5018,Bangkok,TH
-47.122.0.0,47.122.255.255,-6.2088,106.8456,Jakarta,ID
-47.123.0.0,47.250.255.255,31.2304,121.4737,Shanghai,CN
-47.251.0.0,47.252.255.255,39.9042,116.4074,Beijing,CN
-47.253.0.0,47.253.255.255,37.7749,-122.4194,San Francisco,US
-47.254.0.0,47.254.255.255,50.1109,8.6821,Frankfurt,DE
-47.255.0.0,48.126.255.255,35.6762,139.6503,Tokyo,JP
-48.127.0.0,48.254.255.255,37.5665,126.978,Seoul,KR
-48.255.0.0,49.126.255.255,1.3521,103.8198,Singapore,SG
-49.127.0.0,49.254.255.255,51.5074,-0.1278,London,GB
-49.255.0.0,50.17.255.255,50.1109,8.6821,Frankfurt,DE
-50.18.0.0,50.19.255.255,37.7749,-122.4194,San Francisco,US
-50.20.0.0,50.35.255.255,40.7128,-74.006,New York,US
-50.36.0.0,50.51.255.255,37.7749,-122.4194,San Francisco,US
-50.52.0.0,50.67.255.255,-33.8688,151.2093,Sydney,AU
-50.68.0.0,50.83.255.255,19.076,72.8777,Mumbai,IN
-50.84.0.0,50.99.255.255,48.8566,2.3522,Paris,FR
-50.100.0.0,50.111.255.255,52.3676,4.9041,Amsterdam,NL
-50.112.0.0,50.112.255.255,45.5152,-122.6784,Portland,US
-50.113.0.0,50.127.255.255,-23.5505,-46.6333,Sao Paulo,BR
-50.128.0.0,50.255.255.255,41.8781,-87.6298,Chicago,US
-51.0.0.0,51.102.255.255,51.5074,-0.1278,London,GB
-51.103.0.0,51.103.255.255,50.1109,8.6821,Frankfurt,DE
-51.104.0.0,51.104.255.255,51.5074,-0.1278,London,GB
-51.105.0.0,51.105.255.255,51.5074,-0.1278,London,GB
-51.106.0.0,51.139.255.255,51.5074,-0.1278,London,GB
-51.140.0.0,51.143.255.255,51.5074,-0.1278,London,GB
-51.144.0.0,51.255.255.255,51.5074,-0.1278,London,GB
-52.0.0.0,52.1.255.255,39.0438,-77.4874,Ashburn,US
-52.2.0.0,52.3.255.255,39.0438,-77.4874,Ashburn,US
-52.4.0.0,52.7.255.255,39.0438,-77.4874,Ashburn,US
-52.8.0.0,52.15.255.255,55.7558,37.6173,Moscow,RU
-52.16.0.0,52.19.255.255,53.3498,-6.2603,Dublin,IE
-52.20.0.0,52.23.255.255,39.0438,-77.4874,Ashburn,US
-52.24.0.0,52.27.255.255,41.8781,-87.6298,Chicago,US
-52.28.0.0,52.31.255.255,50.1109,8.6821,Frankfurt,DE
-52.32.0.0,52.46.255.255,32.7767,-96.797,Dallas,US
-52.47.0.0,52.47.255.255,48.8566,2.3522,Paris,FR
-52.48.0.0,52.51.255.255,53.3498,-6.2603,Dublin,IE
-52.52.0.0,52.56.255.255,33.749,-84.388,Atlanta,US
-52.57.0.0,52.57.255.255,50.1109,8.6821,Frankfurt,DE
-52.58.0.0,52.59.255.255,43.6532,-79.3832,Toronto,CA
-52.60.0.0,52.60.255.255,45.5017,-73.5673,Montreal,CA
-52.61.0.0,52.61.255.255,22.3193,114.1694,Hong Kong,HK
-52.62.0.0,52.62.255.255,-33.8688,151.2093,Sydney,AU
-52.63.0.0,52.65.255.255,25.033,121.5654,Taipei,TW
-52.66.0.0,52.66.255.255,19.076,72.8777,Mumbai,IN
-52.67.0.0,52.73.255.255,40.4168,-3.7038,Madrid,ES
-52.74.0.0,52.74.255.255,1.3521,103.8198,Singapore,SG
-52.75.0.0,52.77.255.255,45.4642,9.19,Milan,IT
-52.78.0.0,52.78.255.255,37.5665,126.978,Seoul,KR
-52.79.0.0,52.94.255.255,59.3293,18.0686,Stockholm,SE
-52.95.0.0,52.110.255.255,34.6937,135.5022,Osaka,JP
-52.111.0.0,52.126.255.255,23.1291,113.2644,Guangzhou,CN
-52.127.0.0,52.138.255.255,22.5431,114.0579,Shenzhen,CN
-52.139.0.0,52.139.255.255,1.3521,103.8198,Singapore,SG
-52.140.0.0,52.140.255.255,52.52,13.405,Berlin,DE
-52.141.0.0,52.141.255.255,37.5665,126.978,Seoul,KR
-52.142.0.0,52.144.255.255,25.2048,55.2708,Dubai,AE
-52.145.0.0,52.145.255.255,39.0438,-77.4874,Ashburn,US
-52.146.0.0,52.146.255.255,13.7563,100.5018,Bangkok,TH
-52.147.0.0,52.147.255.255,-33.8688,151.2093,Sydney,AU
-52.148.0.0,52.163.255.255,31.2304,121.4737,Shanghai,CN
-52.164.0.0,52.179.255.255,39.9042,116.4074,Beijing,CN
-52.180.0.0,52.184.255.255,35.6762,139.6503,Tokyo,JP
-52.185.0.0,52.185.255.255,35.6762,139.6503,Tokyo,JP
-52.186.0.0,52.201.255.255,37.5665,126.978,Seoul,KR
-52.202.0.0,52.217.255.255,1.3521,103.8198,Singapore,SG
-52.218.0.0,52.223.255.255,51.5074,-0.1278,London,GB
-52.224.0.0,52.255.255.255,39.0438,-77.4874,Ashburn,US
-53.0.0.0,53.127.255.255,50.1109,8.6821,Frankfurt,DE
-53.128.0.0,53.255.255.255,40.7128,-74.006,New York,US
-54.0.0.0,54.63.255.255,37.7749,-122.4194,San Francisco,US
-54.64.0.0,54.65.255.255,35.6762,139.6503,Tokyo,JP
-54.66.0.0,54.66.255.255,-33.8688,151.2093,Sydney,AU
-54.67.0.0,54.67.255.255,-33.8688,151.2093,Sydney,AU
-54.68.0.0,54.71.255.255,45.5152,-122.6784,Portland,US
-54.72.0.0,54.79.255.255,53.3498,-6.2603,Dublin,IE
-54.80.0.0,54.95.255.255,39.0438,-77.4874,Ashburn,US
-54.96.0.0,54.111.255.255,19.076,72.8777,Mumbai,IN
-54.112.0.0,54.127.255.255,48.8566,2.3522,Paris,FR
-54.128.0.0,54.143.255.255,52.3676,4.9041,Amsterdam,NL
-54.144.0.0,54.151.255.255,-23.5505,-46.6333,Sao Paulo,BR
-54.152.0.0,54.159.255.255,39.0438,-77.4874,Ashburn,US
-54.160.0.0,54.191.255.255,39.0438,-77.4874,Ashburn,US
-54.192.0.0,54.205.255.255,55.7558,37.6173,Moscow,RU
-54.206.0.0,54.206.255.255,-33.8688,151.2093,Sydney,AU
-54.207.0.0,54.207.255.255,41.8781,-87.6298,Chicago,US
-54.208.0.0,54.209.255.255,39.0438,-77.4874,Ashburn,US
-54.210.0.0,54.223.255.255,32.7767,-96.797,Dallas,US
-54.224.0.0,54.239.255.255,39.0438,-77.4874,Ashburn,US
-54.240.0.0,54.240.255.255,33.749,-84.388,Atlanta,US
-54.241.0.0,54.241.255.255,37.7749,-122.4194,San Francisco,US
-54.242.0.0,54.243.255.255,43.6532,-79.3832,Toronto,CA
-54.244.0.0,54.245.255.255,45.5152,-122.6784,Portland,US
-54.246.0.0,55.117.255.255,22.3193,114.1694,Hong Kong,HK
-55.118.0.0,55.245.255.255,25.033,121.5654,Taipei,TW
-55.246.0.0,56.117.255.255,40.4168,-3.7038,Madrid,ES
-56.118.0.0,56.245.255.255,45.4642,9.19,Milan,IT
-56.246.0.0,57.117.255.255,59.3293,18.0686,Stockholm,SE
-57.118.0.0,57.245.255.255,34.6937,135.5022,Osaka,JP
-57.246.0.0,57.255.255.255,23.1291,113.2644,Guangzhou,CN
-58.0.0.0,58.255.255.255,22.5431,114.0579,Shenzhen,CN
-59.0.0.0,59.109.255.255,31.2304,121.4737,Shanghai,CN
-59.110.0.0,59.110.255.255,39.9042,116.4074,Beijing,CN
-59.111.0.0,59.255.255.255,31.2304,121.4737,Shanghai,CN
-60.0.0.0,60.255.255.255,39.9042,116.4074,Beijing,CN
-61.0.0.0,61.255.255.255,31.2304,121.4737,Shanghai,CN
-62.0.0.0,62.255.255.255,52.3676,4.9041,Amsterdam,NL
-63.0.0.0,63.15.255.255,22.5431,114.0579,Shenzhen,CN
-63.16.0.0,63.31.255.255,52.52,13.405,Berlin,DE
-63.32.0.0,63.35.255.255,53.3498,-6.2603,Dublin,IE
-63.36.0.0,63.163.255.255,25.2048,55.2708,Dubai,AE
-63.164.0.0,63.255.255.255,13.7563,100.5018,Bangkok,TH
-64.0.0.0,64.255.255.255,41.8781,-87.6298,Chicago,US
-65.0.0.0,65.255.255.255,37.7749,-122.4194,San Francisco,US
-66.0.0.0,66.255.255.255,32.7767,-96.797,Dallas,US
-67.0.0.0,67.159.255.255,25.7617,-80.1918,Miami,US
-67.160.0.0,67.191.255.255,39.9526,-75.1652,Philadelphia,US
-67.192.0.0,67.255.255.255,25.7617,-80.1918,Miami,US
-68.0.0.0,68.127.255.255,31.2304,121.4737,Shanghai,CN
-68.128.0.0,68.255.255.255,39.9042,116.4074,Beijing,CN
-69.0.0.0,69.135.255.255,40.7128,-74.006,New York,US
-69.136.0.0,69.143.255.255,42.3601,-71.0589,Boston,US
-69.144.0.0,69.239.255.255,40.7128,-74.006,New York,US
-69.240.0.0,69.255.255.255,47.6062,-122.3321,Seattle,US
-70.0.0.0,70.255.255.255,41.8781,-87.6298,Chicago,US
-71.0.0.0,71.127.255.255,35.6762,139.6503,Tokyo,JP
-71.128.0.0,71.159.255.255,37.5665,126.978,Seoul,KR
-71.160.0.0,71.175.255.255,40.7128,-74.006,New York,US
-71.176.0.0,71.191.255.255,1.3521,103.8198,Singapore,SG
-71.192.0.0,71.255.255.255,37.7749,-122.4194,San Francisco,US
-72.0.0.0,72.15.255.255,51.5074,-0.1278,London,GB
-72.16.0.0,72.31.255.255,50.1109,8.6821,Frankfurt,DE
-72.32.0.0,72.47.255.255,40.7128,-74.006,New York,US
-72.48.0.0,72.63.255.255,37.7749,-122.4194,San Francisco,US
-72.64.0.0,72.127.255.255,32.7767,-96.797,Dallas,US
-72.128.0.0,72.143.255.255,-33.8688,151.2093,Sydney,AU
-72.144.0.0,72.159.255.255,19.076,72.8777,Mumbai,IN
-72.160.0.0,72.175.255.255,48.8566,2.3522,Paris,FR
-72.176.0.0,72.191.255.255,52.3676,4.9041,Amsterdam,NL
-72.192.0.0,72.207.255.255,-23.5505,-46.6333,Sao Paulo,BR
-72.208.0.0,72.223.255.255,55.7558,37.6173,Moscow,RU
-72.224.0.0,72.239.255.255,41.8781,-87.6298,Chicago,US
-72.240.0.0,72.245.255.255,32.7767,-96.797,Dallas,US
-72.246.0.0,72.247.255.255,42.3736,-71.1097,Cambridge,US
-72.248.0.0,72.255.255.255,33.749,-84.388,Atlanta,US
-73.0.0.0,73.255.255.255,33.749,-84.388,Atlanta,US
-74.0.0.0,74.15.255.255,43.6532,-79.3832,Toronto,CA
-74.16.0.0,74.31.255.255,22.3193,114.1694,Hong Kong,HK
-74.32.0.0,74.47.255.255,25.033,121.5654,Taipei,TW
-74.48.0.0,74.63.255.255,40.4168,-3.7038,Madrid,ES
-74.64.0.0,74.79.255.255,45.4642,9.19,Milan,IT
-74.80.0.0,74.95.255.255,59.3293,18.0686,Stockholm,SE
-74.96.0.0,74.127.255.255,33.749,-84.388,Atlanta,US
-74.128.0.0,74.143.255.255,34.6937,135.5022,Osaka,JP
-74.144.0.0,74.159.255.255,23.1291,113.2644,Guangzhou,CN
-74.160.0.0,74.175.255.255,22.5431,114.0579,Shenzhen,CN
-74.176.0.0,74.191.255.255,52.52,13.405,Berlin,DE
-74.192.0.0,74.207.255.255,25.2048,55.2708,Dubai,AE
-74.208.0.0,74.223.255.255,13.7563,100.5018,Bangkok,TH
-74.224.0.0,74.239.255.255,31.2304,121.4737,Shanghai,CN
-74.240.0.0,74.255.255.255,39.9042,116.4074,Beijing,CN
-75.0.0.0,75.255.255.255,37.7749,-122.4194,San Francisco,US
-76.0.0.0,76.255.255.255,29.7604,-95.3698,Houston,US
-77.0.0.0,77.255.255.255,52.3676,4.9041,Amsterdam,NL
-78.0.0.0,78.255.255.255,52.52,13.405,Berlin,DE
-79.0.0.0,79.191.255.255,52.52,13.405,Berlin,DE
-79.192.0.0,79.255.255.255,50.1109,8.6821,Frankfurt,DE
-80.0.0.0,80.127.255.255,52.3676,4.9041,Amsterdam,NL
-80.128.0.0,80.159.255.255,52.52,13.405,Berlin,DE
-80.160.0.0,80.255.255.255,52.3676,4.9041,Amsterdam,NL
-81.0.0.0,81.255.255.255,51.5074,-0.1278,London,GB
-82.0.0.0,82.255.255.255,52.3676,4.9041,Amsterdam,NL
-83.0.0.0,83.255.255.255,52.3676,4.9041,Amsterdam,NL
-84.0.0.0,84.255.255.255,52.3676,4.9041,Amsterdam,NL
-85.0.0.0,85.255.255.255,52.3676,4.9041,Amsterdam,NL
-86.0.0.0,86.255.255.255,52.3676,4.9041,Amsterdam,NL
-87.0.0.0,87.127.255.255,52.52,13.405,Berlin,DE
-87.128.0.0,87.191.255.255,48.1351,11.582,Munich,DE
-87.192.0.0,87.255.255.255,52.52,13.405,Berlin,DE
-88.0.0.0,88.255.255.255,52.3676,4.9041,Amsterdam,NL
-89.0.0.0,89.255.255.255,52.3676,4.9041,Amsterdam,NL
-90.0.0.0,90.255.255.255,52.3676,4.9041,Amsterdam,NL
-91.0.0.0,91.63.255.255,50.1109,8.6821,Frankfurt,DE
-91.64.0.0,91.191.255.255,35.6762,139.6503,Tokyo,JP
-91.192.0.0,91.255.255.255,37.5665,126.978,Seoul,KR
-92.0.0.0,92.121.255.255,55.7558,37.6173,Moscow,RU
-92.122.0.0,92.123.255.255,42.3736,-71.1097,Cambridge,US
-92.124.0.0,92.255.255.255,55.7558,37.6173,Moscow,RU
-93.0.0.0,93.127.255.255,1.3521,103.8198,Singapore,SG
-93.128.0.0,93.191.255.255,51.5074,-0.1278,London,GB
-93.192.0.0,93.255.255.255,52.52,13.405,Berlin,DE
-94.0.0.0,94.255.255.255,55.7558,37.6173,Moscow,RU
-95.0.0.0,95.99.255.255,52.52,13.405,Berlin,DE
-95.100.0.0,95.101.255.255,42.3736,-71.1097,Cambridge,US
-95.102.0.0,95.255.255.255,52.52,13.405,Berlin,DE
-96.0.0.0,96.5.255.255,50.1109,8.6821,Frankfurt,DE
-96.6.0.0,96.7.255.255,42.3736,-71.1097,Cambridge,US
-96.8.0.0,96.15.255.255,40.7128,-74.006,New York,US
-96.16.0.0,96.17.255.255,42.3736,-71.1097,Cambridge,US
-96.18.0.0,96.145.255.255,37.7749,-122.4194,San Francisco,US
-96.146.0.0,96.223.255.255,-33.8688,151.2093,Sydney,AU
-96.224.0.0,96.255.255.255,40.7128,-74.006,New York,US
-97.0.0.0,97.127.255.255,19.076,72.8777,Mumbai,IN
-97.128.0.0,97.255.255.255,48.8566,2.3522,Paris,FR
-98.0.0.0,98.127.255.255,52.3676,4.9041,Amsterdam,NL
-98.128.0.0,98.191.255.255,-23.5505,-46.6333,Sao Paulo,BR
-98.192.0.0,98.255.255.255,41.8781,-87.6298,Chicago,US
-99.0.0.0,99.78.255.255,40.7128,-74.006,New York,US
-99.79.0.0,99.79.255.255,45.5017,-73.5673,Montreal,CA
-99.80.0.0,99.255.255.255,40.7128,-74.006,New York,US
-100.0.0.0,100.63.255.255,41.8781,-87.6298,Chicago,US
-100.64.0.0,100.191.255.255,55.7558,37.6173,Moscow,RU
-100.192.0.0,100.255.255.255,41.8781,-87.6298,Chicago,US
-101.0.0.0,101.131.255.255,39.9042,116.4074,Beijing,CN
-101.132.0.0,101.132.255.255,31.2304,121.4737,Shanghai,CN
-101.133.0.0,101.199.255.255,39.9042,116.4074,Beijing,CN
-101.200.0.0,101.201.255.255,39.9042,116.4074,Beijing,CN
-101.202.0.0,101.255.255.255,39.9042,116.4074,Beijing,CN
-102.0.0.0,102.127.255.255,32.7767,-96.797,Dallas,US
-102.128.0.0,102.255.255.255,33.749,-84.388,Atlanta,US
-103.0.0.0,103.21.243.255,43.6532,-79.3832,Toronto,CA
-103.21.244.0,103.21.247.255,37.7749,-122.4194,San Francisco,US
-103.21.248.0,103.22.199.255,22.3193,114.1694,Hong Kong,HK
-103.22.200.0,103.22.203.255,37.7749,-122.4194,San Francisco,US
-103.22.204.0,103.31.3.255,25.033,121.5654,Taipei,TW
-103.31.4.0,103.31.7.255,37.7749,-122.4194,San Francisco,US
-103.31.8.0,103.159.7.255,40.4168,-3.7038,Madrid,ES
-103.159.8.0,103.255.255.255,45.4642,9.19,Milan,IT
-104.0.0.0,104.15.255.255,32.7767,-96.797,Dallas,US
-104.16.0.0,104.23.255.255,37.7749,-122.4194,San Francisco,US
-104.24.0.0,104.27.255.255,37.7749,-122.4194,San Francisco,US
-104.28.0.0,104.39.255.255,32.7767,-96.797,Dallas,US
-104.40.0.0,104.47.255.255,39.0438,-77.4874,Ashburn,US
-104.48.0.0,104.63.255.255,32.7767,-96.797,Dallas,US
-104.64.0.0,104.127.255.255,42.3736,-71.1097,Cambridge,US
-104.128.0.0,104.151.255.255,32.7767,-96.797,Dallas,US
-104.152.0.0,104.153.255.255,50.1109,8.6821,Frankfurt,DE
-104.154.0.0,104.155.255.255,41.8781,-87.6298,Chicago,US
-104.156.0.0,104.195.255.255,32.7767,-96.797,Dallas,US
-104.196.0.0,104.199.255.255,39.0438,-77.4874,Ashburn,US
-104.200.0.0,104.210.255.255,32.7767,-96.797,Dallas,US
-104.211.0.0,104.211.255.255,19.076,72.8777,Mumbai,IN
-104.212.0.0,104.255.255.255,32.7767,-96.797,Dallas,US
-105.0.0.0,105.127.255.255,59.3293,18.0686,Stockholm,SE
-105.128.0.0,105.255.255.255,34.6937,135.5022,Osaka,JP
-106.0.0.0,106.13.255.255,23.1291,113.2644,Guangzhou,CN
-106.14.0.0,106.15.255.255,31.2304,121.4737,Shanghai,CN
-106.16.0.0,106.143.255.255,22.5431,114.0579,Shenzhen,CN
-106.144.0.0,106.255.255.255,52.52,13.405,Berlin,DE
-107.0.0.0,107.255.255.255,33.749,-84.388,Atlanta,US
-108.0.0.0,108.162.191.255,41.8781,-87.6298,Chicago,US
-108.162.192.0,108.162.255.255,37.7749,-122.4194,San Francisco,US
-108.163.0.0,108.255.255.255,41.8781,-87.6298,Chicago,US
-109.0.0.0,109.255.255.255,51.5074,-0.1278,London,GB
-110.0.0.0,110.255.255.255,23.1291,113.2644,Guangzhou,CN
-111.0.0.0,111.255.255.255,31.2304,121.4737,Shanghai,CN
-112.0.0.0,112.255.255.255,39.9042,116.4074,Beijing,CN
-113.0.0.0,113.255.255.255,31.2304,121.4737,Shanghai,CN
-114.0.0.0,114.255.255.255,39.9042,116.4074,Beijing,CN
-115.0.0.0,115.255.255.255,31.2304,121.4737,Shanghai,CN
-116.0.0.0,116.255.255.255,39.9042,116.4074,Beijing,CN
-117.0.0.0,117.255.255.255,31.2304,121.4737,Shanghai,CN
-118.0.0.0,118.255.255.255,39.9042,116.4074,Beijing,CN
-119.0.0.0,119.255.255.255,31.2304,121.4737,Shanghai,CN
-120.0.0.0,120.23.255.255,39.9042,116.4074,Beijing,CN
-120.24.0.0,120.27.255.255,39.9042,116.4074,Beijing,CN
-120.28.0.0,120.47.255.255,39.9042,116.4074,Beijing,CN
-120.48.0.0,120.49.255.255,39.9042,116.4074,Beijing,CN
-120.50.0.0,120.54.255.255,39.9042,116.4074,Beijing,CN
-120.55.0.0,120.55.255.255,31.2304,121.4737,Shanghai,CN
-120.56.0.0,120.75.255.255,39.9042,116.4074,Beijing,CN
-120.76.0.0,120.77.255.255,30.2741,120.1551,Hangzhou,CN
-120.78.0.0,120.79.255.255,22.5431,114.0579,Shenzhen,CN
-120.80.0.0,120.81.255.255,23.1291,113.2644,Guangzhou,CN
-120.82.0.0,120.255.255.255,39.9042,116.4074,Beijing,CN
-121.0.0.0,121.39.255.255,31.2304,121.4737,Shanghai,CN
-121.40.0.0,121.43.255.255,30.2741,120.1551,Hangzhou,CN
-121.44.0.0,121.195.255.255,31.2304,121.4737,Shanghai,CN
-121.196.0.0,121.199.255.255,30.2741,120.1551,Hangzhou,CN
-121.200.0.0,121.255.255.255,31.2304,121.4737,Shanghai,CN
-122.0.0.0,122.255.255.255,39.9042,116.4074,Beijing,CN
-123.0.0.0,123.255.255.255,31.2304,121.4737,Shanghai,CN
-124.0.0.0,124.255.255.255,39.9042,116.4074,Beijing,CN
-125.0.0.0,125.255.255.255,31.2304,121.4737,Shanghai,CN
-126.0.0.0,126.255.255.255,35.6762,139.6503,Tokyo,JP
-127.0.0.0,127.127.255.255,25.2048,55.2708,Dubai,AE
-127.128.0.0,127.255.255.255,13.7563,100.5018,Bangkok,TH
-128.0.0.0,128.255.255.255,48.8566,2.3522,Paris,FR
-129.0.0.0,129.255.255.255,48.8566,2.3522,Paris,FR
-130.0.0.0,130.255.255.255,52.52,13.405,Berlin,DE
-131.0.0.0,131.0.71.255,59.3293,18.0686,Stockholm,SE
-131.0.72.0,131.0.75.255,37.7749,-122.4194,San Francisco,US
-131.0.76.0,131.255.255.255,59.3293,18.0686,Stockholm,SE
-132.0.0.0,132.255.255.255,45.4642,9.19,Milan,IT
-133.0.0.0,133.255.255.255,35.6762,139.6503,Tokyo,JP
-134.0.0.0,134.255.255.255,52.52,13.405,Berlin,DE
-135.0.0.0,135.127.255.255,31.2304,121.4737,Shanghai,CN
-135.128.0.0,135.255.255.255,39.9042,116.4074,Beijing,CN
-136.0.0.0,136.127.255.255,35.6762,139.6503,Tokyo,JP
-136.128.0.0,136.255.255.255,37.5665,126.978,Seoul,KR
-137.0.0.0,137.255.255.255,48.8566,2.3522,Paris,FR
-138.0.0.0,138.90.255.255,45.4642,9.19,Milan,IT
-138.91.0.0,138.91.255.255,37.7749,-122.4194,San Francisco,US
-138.92.0.0,138.255.255.255,45.4642,9.19,Milan,IT
-139.0.0.0,139.195.255.255,23.1291,113.2644,Guangzhou,CN
-139.196.0.0,139.199.255.255,31.2304,121.4737,Shanghai,CN
-139.200.0.0,139.255.255.255,23.1291,113.2644,Guangzhou,CN
-140.0.0.0,140.127.255.255,1.3521,103.8198,Singapore,SG
-140.128.0.0,140.255.255.255,51.5074,-0.1278,London,GB
-141.0.0.0,141.101.63.255,52.52,13.405,Berlin,DE
-141.101.64.0,141.101.127.255,37.7749,-122.4194,San Francisco,US
-141.101.128.0,141.143.255.255,52.52,13.405,Berlin,DE
-141.144.0.0,141.159.255.255,32.7767,-96.797,Dallas,US
-141.160.0.0,141.255.255.255,52.52,13.405,Berlin,DE
-142.0.0.0,142.127.255.255,50.1109,8.6821,Frankfurt,DE
-142.128.0.0,142.255.255.255,40.7128,-74.006,New York,US
-143.0.0.0,143.255.255.255,45.4642,9.19,Milan,IT
-144.0.0.0,144.127.255.255,37.7749,-122.4194,San Francisco,US
-144.128.0.0,144.255.255.255,-33.8688,151.2093,Sydney,AU
-145.0.0.0,145.255.255.255,48.8566,2.3522,Paris,FR
-146.0.0.0,146.255.255.255,51.5074,-0.1278,London,GB
-147.0.0.0,147.255.255.255,48.8566,2.3522,Paris,FR
-148.0.0.0,148.127.255.255,19.076,72.8777,Mumbai,IN
-148.128.0.0,148.255.255.255,48.8566,2.3522,Paris,FR
-149.0.0.0,149.128.255.255,52.52,13.405,Berlin,DE
-149.129.0.0,149.129.255.255,1.3521,103.8198,Singapore,SG
-149.130.0.0,149.255.255.255,52.52,13.405,Berlin,DE
-150.0.0.0,150.191.255.255,35.6762,139.6503,Tokyo,JP
-150.192.0.0,150.255.255.255,40.4168,-3.7038,Madrid,ES
-151.0.0.0,151.100.255.255,45.4642,9.19,Milan,IT
-151.101.0.0,151.101.255.255,37.7749,-122.4194,San Francisco,US
-151.102.0.0,151.255.255.255,45.4642,9.19,Milan,IT
-152.0.0.0,152.127.255.255,52.3676,4.9041,Amsterdam,NL
-152.128.0.0,152.255.255.255,-23.5505,-46.6333,Sao Paulo,BR
-153.0.0.0,153.127.255.255,55.7558,37.6173,Moscow,RU
-153.128.0.0,153.191.255.255,34.6937,135.5022,Osaka,JP
-153.192.0.0,154.63.255.255,41.8781,-87.6298,Chicago,US
-154.64.0.0,154.191.255.255,32.7767,-96.797,Dallas,US
-154.192.0.0,154.255.255.255,33.749,-84.388,Atlanta,US
-155.0.0.0,155.255.255.255,59.3293,18.0686,Stockholm,SE
-156.0.0.0,156.255.255.255,48.8566,2.3522,Paris,FR
-157.0.0.0,157.52.63.255,35.6762,139.6503,Tokyo,JP
-157.52.64.0,157.52.127.255,37.7749,-122.4194,San Francisco,US
-157.52.128.0,157.52.255.255,37.7749,-122.4194,San Francisco,US
-157.53.0.0,157.174.255.255,35.6762,139.6503,Tokyo,JP
-157.175.0.0,157.175.255.255,26.0667,50.5577,Bahrain,BH
-157.176.0.0,157.255.255.255,35.6762,139.6503,Tokyo,JP
-158.0.0.0,158.255.255.255,59.3293,18.0686,Stockholm,SE
-159.0.0.0,159.255.255.255,48.8566,2.3522,Paris,FR
-160.0.0.0,160.255.255.255,34.6937,135.5022,Osaka,JP
-161.0.0.0,161.255.255.255,48.8566,2.3522,Paris,FR
-162.0.0.0,162.127.255.255,43.6532,-79.3832,Toronto,CA
-162.128.0.0,162.157.255.255,22.3193,114.1694,Hong Kong,HK
-162.158.0.0,162.159.255.255,37.7749,-122.4194,San Francisco,US
-162.160.0.0,162.175.255.255,25.033,121.5654,Taipei,TW
-162.176.0.0,162.191.255.255,40.4168,-3.7038,Madrid,ES
-162.192.0.0,162.207.255.255,45.4642,9.19,Milan,IT
-162.208.0.0,162.223.255.255,59.3293,18.0686,Stockholm,SE
-162.224.0.0,162.239.255.255,34.6937,135.5022,Osaka,JP
-162.240.0.0,162.255.255.255,23.1291,113.2644,Guangzhou,CN
-163.0.0.0,163.255.255.255,35.6762,139.6503,Tokyo,JP
-164.0.0.0,164.255.255.255,34.6937,135.5022,Osaka,JP
-165.0.0.0,165.127.255.255,22.5431,114.0579,Shenzhen,CN
-165.128.0.0,165.255.255.255,52.52,13.405,Berlin,DE
-166.0.0.0,166.127.255.255,25.2048,55.2708,Dubai,AE
-166.128.0.0,166.255.255.255,13.7563,100.5018,Bangkok,TH
-167.0.0.0,167.81.255.255,31.2304,121.4737,Shanghai,CN
-167.82.0.0,167.82.127.255,37.7749,-122.4194,San Francisco,US
-167.82.128.0,167.82.191.255,37.7749,-122.4194,San Francisco,US
-167.82.192.0,167.210.191.255,39.9042,116.4074,Beijing,CN
-167.210.192.0,168.82.191.255,35.6762,139.6503,Tokyo,JP
-168.82.192.0,168.210.191.255,37.5665,126.978,Seoul,KR
-168.210.192.0,169.82.191.255,1.3521,103.8198,Singapore,SG
-169.82.192.0,169.210.191.255,51.5074,-0.1278,London,GB
-169.210.192.0,170.82.191.255,50.1109,8.6821,Frankfurt,DE
-170.82.192.0,170.210.191.255,40.7128,-74.006,New York,US
-170.210.192.0,170.255.255.255,37.7749,-122.4194,San Francisco,US
-171.0.0.0,171.255.255.255,59.3293,18.0686,Stockholm,SE
-172.0.0.0,172.15.255.255,-33.8688,151.2093,Sydney,AU
-172.16.0.0,172.31.255.255,31.2304,121.4737,Shanghai,CN
-172.32.0.0,172.47.255.255,19.076,72.8777,Mumbai,IN
-172.48.0.0,172.63.255.255,48.8566,2.3522,Paris,FR
-172.64.0.0,172.71.255.255,37.7749,-122.4194,San Francisco,US
-172.72.0.0,172.87.255.255,52.3676,4.9041,Amsterdam,NL
-172.88.0.0,172.103.255.255,-23.5505,-46.6333,Sao Paulo,BR
-172.104.0.0,172.111.63.255,55.7558,37.6173,Moscow,RU
-172.111.64.0,172.111.127.255,37.7749,-122.4194,San Francisco,US
-172.111.128.0,172.239.127.255,41.8781,-87.6298,Chicago,US
-172.239.128.0,173.47.255.255,32.7767,-96.797,Dallas,US
-173.48.0.0,173.63.255.255,25.7617,-80.1918,Miami,US
-173.64.0.0,173.191.255.255,33.749,-84.388,Atlanta,US
-173.192.0.0,173.245.47.255,43.6532,-79.3832,Toronto,CA
-173.245.48.0,173.245.63.255,37.7749,-122.4194,San Francisco,US
-173.245.64.0,174.117.63.255,22.3193,114.1694,Hong Kong,HK
-174.117.64.0,174.245.63.255,25.033,121.5654,Taipei,TW
-174.245.64.0,175.117.63.255,40.4168,-3.7038,Madrid,ES
-175.117.64.0,175.245.63.255,45.4642,9.19,Milan,IT
-175.245.64.0,175.255.255.255,59.3293,18.0686,Stockholm,SE
-176.0.0.0,176.255.255.255,51.5074,-0.1278,London,GB
-177.0.0.0,177.127.255.255,34.6937,135.5022,Osaka,JP
-177.128.0.0,177.255.255.255,23.1291,113.2644,Guangzhou,CN
-178.0.0.0,178.127.255.255,22.5431,114.0579,Shenzhen,CN
-178.128.0.0,178.255.255.255,52.52,13.405,Berlin,DE
-179.0.0.0,179.127.255.255,25.2048,55.2708,Dubai,AE
-179.128.0.0,179.255.255.255,13.7563,100.5018,Bangkok,TH
-180.0.0.0,180.255.255.255,31.2304,121.4737,Shanghai,CN
-181.0.0.0,181.127.255.255,31.2304,121.4737,Shanghai,CN
-181.128.0.0,181.255.255.255,39.9042,116.4074,Beijing,CN
-182.0.0.0,182.91.255.255,39.9042,116.4074,Beijing,CN
-182.92.0.0,182.92.255.255,39.9042,116.4074,Beijing,CN
-182.93.0.0,182.255.255.255,39.9042,116.4074,Beijing,CN
-183.0.0.0,183.255.255.255,31.2304,121.4737,Shanghai,CN
-184.0.0.0,184.15.255.255,35.6762,139.6503,Tokyo,JP
-184.16.0.0,184.23.255.255,37.5665,126.978,Seoul,KR
-184.24.0.0,184.31.255.255,42.3736,-71.1097,Cambridge,US
-184.32.0.0,184.47.255.255,1.3521,103.8198,Singapore,SG
-184.48.0.0,184.63.255.255,51.5074,-0.1278,London,GB
-184.64.0.0,184.71.255.255,50.1109,8.6821,Frankfurt,DE
-184.72.0.0,184.73.255.255,37.7749,-122.4194,San Francisco,US
-184.74.0.0,184.83.255.255,40.7128,-74.006,New York,US
-184.84.0.0,184.87.255.255,42.3736,-71.1097,Cambridge,US
-184.88.0.0,184.215.255.255,37.7749,-122.4194,San Francisco,US
-184.216.0.0,184.255.255.255,-33.8688,151.2093,Sydney,AU
-185.0.0.0,185.31.15.255,51.5074,-0.1278,London,GB
-185.31.16.0,185.31.19.255,37.7749,-122.4194,San Francisco,US
-185.31.20.0,185.255.255.255,51.5074,-0.1278,London,GB
-186.0.0.0,186.127.255.255,19.076,72.8777,Mumbai,IN
-186.128.0.0,186.255.255.255,48.8566,2.3522,Paris,FR
-187.0.0.0,187.127.255.255,52.3676,4.9041,Amsterdam,NL
-187.128.0.0,187.255.255.255,-23.5505,-46.6333,Sao Paulo,BR
-188.0.0.0,188.114.95.255,51.5074,-0.1278,London,GB
-188.114.96.0,188.114.111.255,37.7749,-122.4194,San Francisco,US
-188.114.112.0,188.255.255.255,51.5074,-0.1278,London,GB
-189.0.0.0,189.127.255.255,55.7558,37.6173,Moscow,RU
-189.128.0.0,189.255.255.255,41.8781,-87.6298,Chicago,US
-190.0.0.0,190.93.239.255,32.7767,-96.797,Dallas,US
-190.93.240.0,190.93.255.255,37.7749,-122.4194,San Francisco,US
-190.94.0.0,190.221.255.255,33.749,-84.388,Atlanta,US
-190.222.0.0,191.93.255.255,43.6532,-79.3832,Toronto,CA
-191.94.0.0,191.221.255.255,22.3193,114.1694,Hong Kong,HK
-191.222.0.0,191.231.255.255,25.033,121.5654,Taipei,TW
-191.232.0.0,191.233.255.255,-23.5505,-46.6333,Sao Paulo,BR
-191.234.0.0,191.249.255.255,40.4168,-3.7038,Madrid,ES
-191.250.0.0,192.9.255.255,45.4642,9.19,Milan,IT
-192.10.0.0,192.25.255.255,59.3293,18.0686,Stockholm,SE
-192.26.0.0,192.41.255.255,34.6937,135.5022,Osaka,JP
-192.42.0.0,192.49.255.255,23.1291,113.2644,Guangzhou,CN
-192.50.0.0,192.50.255.255,35.1815,136.9066,Nagoya,JP
-192.51.0.0,192.66.255.255,22.5431,114.0579,Shenzhen,CN
-192.67.0.0,192.82.255.255,52.52,13.405,Berlin,DE
-192.83.0.0,192.98.255.255,25.2048,55.2708,Dubai,AE
-192.99.0.0,192.114.255.255,13.7563,100.5018,Bangkok,TH
-192.115.0.0,192.130.255.255,31.2304,121.4737,Shanghai,CN
-192.131.0.0,192.146.255.255,39.9042,116.4074,Beijing,CN
-192.147.0.0,192.162.255.255,35.6762,139.6503,Tokyo,JP
-192.163.0.0,192.167.255.255,37.5665,126.978,Seoul,KR
-192.168.0.0,192.168.255.255,23.1291,113.2644,Guangzhou,CN
-192.169.0.0,192.184.255.255,1.3521,103.8198,Singapore,SG
-192.185.0.0,192.200.255.255,51.5074,-0.1278,London,GB
-192.201.0.0,192.216.255.255,50.1109,8.6821,Frankfurt,DE
-192.217.0.0,192.232.255.255,40.7128,-74.006,New York,US
-192.233.0.0,192.248.255.255,37.7749,-122.4194,San Francisco,US
-192.249.0.0,192.255.255.255,-33.8688,151.2093,Sydney,AU
-193.0.0.0,193.255.255.255,52.52,13.405,Berlin,DE
-194.0.0.0,194.255.255.255,48.8566,2.3522,Paris,FR
-195.0.0.0,195.255.255.255,52.3676,4.9041,Amsterdam,NL
-196.0.0.0,196.127.255.255,19.076,72.8777,Mumbai,IN
-196.128.0.0,196.255.255.255,48.8566,2.3522,Paris,FR
-197.0.0.0,197.127.255.255,52.3676,4.9041,Amsterdam,NL
-197.128.0.0,197.234.239.255,-23.5505,-46.6333,Sao Paulo,BR
-197.234.240.0,197.234.243.255,37.7749,-122.4194,San Francisco,US
-197.234.244.0,197.250.243.255,55.7558,37.6173,Moscow,RU
-197.250.244.0,198.10.243.255,41.8781,-87.6298,Chicago,US
-198.10.244.0,198.26.243.255,32.7767,-96.797,Dallas,US
-198.26.244.0,198.41.127.255,33.749,-84.388,Atlanta,US
-198.41.128.0,198.41.255.255,37.7749,-122.4194,San Francisco,US
-198.42.0.0,198.169.255.255,43.6532,-79.3832,Toronto,CA
-198.170.0.0,199.27.71.255,22.3193,114.1694,Hong Kong,HK
-199.27.72.0,199.27.79.255,37.7749,-122.4194,San Francisco,US
-199.27.80.0,199.155.79.255,25.033,121.5654,Taipei,TW
-199.155.80.0,199.231.255.255,40.4168,-3.7038,Madrid,ES
-199.232.0.0,199.232.255.255,37.7749,-122.4194,San Francisco,US
-199.233.0.0,200.104.255.255,45.4642,9.19,Milan,IT
-200.105.0.0,200.232.255.255,59.3293,18.0686,Stockholm,SE
-200.233.0.0,201.104.255.255,34.6937,135.5022,Osaka,JP
-201.105.0.0,201.232.255.255,23.1291,113.2644,Guangzhou,CN
-201.233.0.0,201.255.255.255,22.5431,114.0579,Shenzhen,CN
-202.0.0.0,202.255.255.255,39.9042,116.4074,Beijing,CN
-203.0.0.0,203.255.255.255,39.9042,116.4074,Beijing,CN
-204.0.0.0,204.127.255.255,52.52,13.405,Berlin,DE
-204.128.0.0,204.255.255.255,25.2048,55.2708,Dubai,AE
-205.0.0.0,205.127.255.255,13.7563,100.5018,Bangkok,TH
-205.128.0.0,205.255.255.255,31.2304,121.4737,Shanghai,CN
-206.0.0.0,206.127.255.255,39.9042,116.4074,Beijing,CN
-206.128.0.0,206.255.255.255,35.6762,139.6503,Tokyo,JP
-207.0.0.0,207.127.255.255,37.5665,126.978,Seoul,KR
-207.128.0.0,207.255.255.255,1.3521,103.8198,Singapore,SG
-208.0.0.0,208.127.255.255,51.5074,-0.1278,London,GB
-208.128.0.0,208.255.255.255,50.1109,8.6821,Frankfurt,DE
-209.0.0.0,209.127.255.255,40.7128,-74.006,New York,US
-209.128.0.0,209.255.255.255,37.7749,-122.4194,San Francisco,US
-210.0.0.0,210.255.255.255,31.2304,121.4737,Shanghai,CN
-211.0.0.0,211.255.255.255,39.9042,116.4074,Beijing,CN
-212.0.0.0,212.255.255.255,51.5074,-0.1278,London,GB
-213.0.0.0,213.255.255.255,52.3676,4.9041,Amsterdam,NL
-214.0.0.0,214.127.255.255,-33.8688,151.2093,Sydney,AU
-214.128.0.0,214.255.255.255,19.076,72.8777,Mumbai,IN
-215.0.0.0,215.127.255.255,48.8566,2.3522,Paris,FR
-215.128.0.0,215.255.255.255,52.3676,4.9041,Amsterdam,NL
-216.0.0.0,216.127.255.255,-23.5505,-46.6333,Sao Paulo,BR
-216.128.0.0,216.255.255.255,55.7558,37.6173,Moscow,RU
-217.0.0.0,217.79.255.255,52.52,13.405,Berlin,DE
-217.80.0.0,217.95.255.255,48.1351,11.582,Munich,DE
-217.96.0.0,217.223.255.255,52.52,13.405,Berlin,DE
-217.224.0.0,217.255.255.255,50.1109,8.6821,Frankfurt,DE
-218.0.0.0,218.255.255.255,31.2304,121.4737,Shanghai,CN
-219.0.0.0,219.255.255.255,39.9042,116.4074,Beijing,CN
-220.0.0.0,220.255.255.255,23.1291,113.2644,Guangzhou,CN
-221.0.0.0,221.255.255.255,31.2304,121.4737,Shanghai,CN
-222.0.0.0,222.255.255.255,39.9042,116.4074,Beijing,CN
-223.0.0.0,223.255.255.255,39.9042,116.4074,Beijing,CN
@@ -1,6 +1,6 @@
 # 天璇 (TianXuan) 分析工作流

-> 从原始 TLS 流 CSV 到实体画像、聚类结果、UDT 区分特征、UMAP 可视化的完整流水线文档。当前版本 v2.0。
+> 从原始 TLS 流 CSV 到实体画像、聚类结果、区分特征、UMAP 可视化的完整流水线文档。当前版本 v3.0。

 ---

@@ -9,34 +9,34 @@
 1. [工具总览](#1-工具总览)
 2. [标准分析流程](#2-标准分析流程)
 3. [数据准备与加载](#3-数据准备与加载)
-4. [数据校验与清洗](#4-数据校验与清洗)
+4. [类型检测与数据校验](#4-类型检测与数据校验)
 5. [实体识别与聚合](#5-实体识别与聚合)
 6. [自适应评分](#6-自适应评分)
 7. [聚类分析](#7-聚类分析)
 8. [特征提取与 UMAP 嵌入](#8-特征提取与-umap-嵌入)
 9. [可视化](#9-可视化)
 10. [LLM 集成](#10-llm-集成)
-11. [简易分析模块](#11-简易分析模块)
+11. [手动分析工作流](#11-手动分析工作流)
 12. [跨机部署](#12-跨机部署)

 ---

 ## 1. 工具总览

-系统现有 **30 个 MCP 工具**, 分四层。完整列表见 [AGENTS.md](../AGENTS.md) 中的「30 MCP Tools」表。
+系统现有 **30 个 MCP 工具**, 分四层, 实现于 `analysis/tools/` 包的 14 个 handler 模块。完整列表见 [AGENTS.md](../AGENTS.md) 中的「30 MCP Tools」表。

-**标准分析链 (`load_data → profile_data → build_entity_profiles → compute_scores → run_clustering → extract_features`)** 覆盖从 CSV 加载到 UMAP 可视化的全流程。LLM 自动模式由 `tianxuan/llm_orchestrator.py` 驱动，策略式提示词: 先 profile 了解数据 → 根据数据决策下一步 → 失败自动调诊断工具 → 最终中文总结。
+**标准分析链 (`load_data → filter_data → profile_data → build_entity_profiles → compute_scores → run_clustering → extract_features`)** 覆盖从 CSV 加载到 UMAP 可视化的全流程。LLM 自动模式由 `tianxuan/llm_orchestrator.py` 驱动, 策略式提示词: 先 profile 了解数据 → 根据数据决策下一步 → 失败自动调诊断工具 → 最终中文总结。

 ### 降维方案: UMAP + TruncatedSVD

 | 方法 | 用途 | 触发条件 | 代码位置 |
-|------|------|---------|---------|
-| **UMAP** (n_components=2) | 实体 2D 嵌入可视化, 存入 EntityProfile 表 (embedding_x/y) | `extract_features` 执行时自动计算 | `tool_registry.py::_handle_extract_features` |
-| **TruncatedSVD** (n_components=50) | 聚类前高维特征降维 (>50 维时) | `run_clustering` / `filter_and_cluster` 执行时 | `tool_registry.py::_handle_run_clustering` |
-| **TruncatedSVD** (n_components=1) | 自适应评分权重学习 (SVD 第一分量 loading) | `compute_scores` 被调用时 | `tool_registry.py::_add_adaptive_scores` |
-| **PCA-2D** (legacy) | 前端 cluster_overview 散点图 (Canvas 绘制) | CLI pipeline 步骤 6 | `views.py` / `run_pipeline.py` |
+|------|------|---------|------------|
+| **UMAP** (n_components=2) | 实体 2D 嵌入可视化, 存入 EntityProfile 表 (embedding_x/y) | `extract_features` 执行时自动计算 | `tools/features.py::_handle_extract_features` |
+| **TruncatedSVD** (n_components=50) | 聚类前高维特征降维 (>50 维时) | `run_clustering` / `filter_and_cluster` 执行时 | `tools/clustering.py::_handle_run_clustering` |
+| **TruncatedSVD** (n_components=1) | 自适应评分权重学习 (SVD 第一分量 loading) | `compute_scores` 被调用时 | `tools/entities.py::_add_adaptive_scores` |
+| **PCA-2D** (legacy) | 前端 cluster_overview 散点图 (Canvas 绘制) | CLI pipeline 步骤 6 | `views/pipeline.py` / `run_pipeline.py` |

-**距离计算**: `compute_distance_matrix` 工具允许 LLM 编写 Python 距离函数体 (签名 `def distance_fn(row: dict) -> float:`)，在受限命名空间 (math, numpy) 中逐行执行，返回 min/max/mean/std + 评分行样本。用于现有工具无法表达的一次性度量。
+**距离计算**: `analysis/distance.py` 提供 IP 子网距离、地理距离 (Haversine)、Levenshtein、Hamming、FFT 相位距离。`compute_distance_matrix` 工具额外允许 LLM 编写 Python 距离函数体 (签名 `def distance_fn(row: dict) -> float:`), 在受限命名空间 (math, numpy) 中逐行执行, 用于现有工具无法表达的一次性度量。

 ---

@@ -58,6 +58,8 @@ load_data(
 # 返回: dataset_id, schema, row_count, file_count, memory_mb
 ```

+实现: `tools/load_data.py::_handle_load_data` → `data_loader.py::load_csv_directory` (BOM 检测、编码回退、schema 容错合并)。
+
 ### 2.2 了解数据 (`profile_data`)

 ```python
@@ -65,6 +67,8 @@ profile_data(dataset_id="ds_...", sample_size=1000)
 # 返回: 列类型/统计/空值率/相关性矩阵 (自动截断 64KB)
 ```

+实现: `tools/profile.py::_handle_profile_data` → `data_profiler.py` 列统计 + 相关性矩阵计算。
+
 ### 2.3 构建实体画像 (`build_entity_profiles`)

 ```python
@@ -77,6 +81,8 @@ build_entity_profiles(
 # 返回: dataset_id (entity_前缀), entity_columns, n_entities, n_features
 ```

+实现: `tools/entities.py::_handle_build_entity_profiles`, 融合了原 entity_detector (列检测) 和 entity_aggregator (聚合) 的逻辑。
+
 ### 2.4 计算异常分数 (`compute_scores`)

 ```python
@@ -88,6 +94,8 @@ compute_scores(dataset_id="entity_ds_...")
 # 4. 复合 → threat_score
 ```

+实现: `tools/entities.py::_handle_compute_scores` + `_add_adaptive_scores`。
+
 ### 2.5 聚类分析 (`run_clustering`)

 ```python
@@ -101,6 +109,8 @@ run_clustering(

 预处理链: 列过滤 → NaN 填充 (列均值) → StandardScaler → 方差过滤 (< 1e-10) → 相关过滤 (> 0.95 Pearson) → **TruncatedSVD (>50 特征时降维至 50)** → 聚类

+实现: `tools/clustering.py::_handle_run_clustering`。
+
 ### 2.6 提取特征 (`extract_features`)

 ```python
@@ -114,6 +124,8 @@ extract_features(

 执行: Z-Score/ANOVA 特征重要性 → **UMAP-2D 嵌入** (训练 10K 样本, 批量变换 1K 批次) → EntityProfile 表存储坐标 → ClusterFeature 表存储区分特征。

+实现: `tools/features.py::_handle_extract_features` + `_save_features_to_db`。
+
 ---

 ## 3. 数据准备与加载
@@ -139,46 +151,79 @@ extract_features(

 > **生产建议**: 始终使用 UTF-8 无 BOM 编码。`run.bat` 已内置 `PYTHONUTF8=1`。

-### 3.3 推荐列名 (关键词匹配)
-
-系统通过 `entity_aggregator.py` 中的 `_find_column()` 下划线分词匹配关键词。完整列名要求见配置文档。关键列名: `src_ip`, `dst_ip`, `bytes_sent`, `bytes_rev`, `duration`, `tls_version`, `cipher_suite`, `sni`, `dst_port`, `lat`, `lon` 等。
-
-### 3.4 Schema 容错
+### 3.3 Schema 容错

 | 模式 | 行为 |
 |------|------|
 | `schema_strict: false` (默认) | 自动并集合并, 缺失列填 null |
 | `schema_strict: true` | 列名不一致时报 ValueError |

+实现: `data_loader.py::load_csv_directory` 中的 `diagonal_relaxed` 合并策略。
+
 ---

-## 4. 数据校验与清洗
+## 4. 类型检测与数据校验

-校验入口: `analysis/data_validator.py::validate()`。检查项: 列类型矛盾、缺失率 (>50% 高风险)、异常值 (Z-score > 5 比例)、枚举分布、IPv4 有效性。
+### 4.1 值优先类型检测

-清洗在 `load_csv_directory()` 中自动执行: BOOL_ENUM 标准化 (`"+"` → True, `""` → null)、HEX 列添加 `hex_pairs_count` 辅助列、通用数值清洗 (`_coerce_to_float` 处理伪值)。
+`analysis/type_classifier.py` 实现六阶段类型推断:
+
+```
+配置指定 (config.yaml columns section) → 值采样检测 (85%+ 合规即判定)
+    → 名称推断 (下划线分词关键词匹配) → STRING 回退
+```
+
+支持类型: MAC, PORT, IPv4, URL, HEX, ENUM, LAT_LON, BOOL_ENUM, TIMESTAMP, INT, FLOAT, STRING。
+
+### 4.2 数据校验
+
+`analysis/data_validator.py::validate()` 检查: 列类型矛盾、缺失率 (>50% 高风险)、异常值 (Z-score > 5 比例)、枚举分布、IPv4 有效性。
+
+### 4.3 自动清洗
+
+在 `load_csv_directory()` 中自动执行: BOOL_ENUM 标准化 (`"+"` → True, `""` → null)、HEX 列添加 `hex_pairs_count` 辅助列、通用数值清洗 (`_coerce_to_float` 处理 `+`/`-`/空白/N/A 等伪值)。

 ---

 ## 5. 实体识别与聚合

-### 5.1 自动检测
+### 5.1 实体列检测

-`entity_detector.py::detect_entity_column()` 三维打分: 名称匹配 (+3.0)、唯一值比率 (0~10)、字符串类型 (+2.0)。高唯一性 (>50% unique_ratio) 罚 -5.0, 高空值率 (>50% null) ×0.1 惩罚。
+`tools/entities.py::_handle_build_entity_profiles` 中的自动检测逻辑 (合并了原 `entity_detector.py` 的功能):
+
+- **关键词匹配**: 列名下划线分词后匹配 `_ENTITY_KEYWORDS` 集合 (IP, SNI, host, MAC, domain, user, id, name, ...)
+- **数据类型判断**: IPv4 列、ENUM 列自动纳入候选
+- **唯一值比率**: 高唯一率 (>50%) 加权, 极高唯一率 (>90%) 惩罚
+- **空值惩罚**: 高空值率 (>50%) 大幅降权
+- **多列复合键**: 多个 IP 列 (src_ip + dst_ip) 自动组合为复合实体键

 ### 5.2 IP 子网聚合

-对 `DataType.IPv4` 列自动生成 `/24` 子网 (如 `192.168.1.0/24`), `unique_24_networks` 反映网络分布广度。`config.yaml` 可配置子网掩码列表。
+对 `DataType.IPv4` 列自动生成子网 (如 `192.168.1.0/24`), `unique_24_networks` 反映网络分布广度。`config.yaml` 可配置子网掩码列表。
+
+实现: `ip_clustering.py::ip_to_subnet` 和 `apply_subnet_aggregation`。

 ### 5.3 类型感知聚合

-数据类型到聚合函数的映射: INT → mean/sum, FLOAT → mean, ENUM → mode, HEX → hex 对均值, URL → 唯一域名数, IPv4 → 子网多样性, LAT_LON → mean, TIMESTAMP → min/max。
+数据类型到聚合函数的映射位于 `tools/entities.py`:
+
+| 数据类型 | 聚合函数 |
+|---------|---------|
+| INT | mean, sum |
+| FLOAT | mean |
+| ENUM | mode |
+| HEX | hex 对均值 |
+| URL | 唯一域名数 |
+| IPv4 | 子网多样性 |
+| LAT_LON | mean |
+| TIMESTAMP | min, max |

 ---

 ## 6. 自适应评分

-`_add_adaptive_scores()` (tool_registry.py):
+`tools/entities.py::_add_adaptive_scores()`:
+
 1. 采样 ≤10K 行
 2. **TruncatedSVD(n_components=1)** 学习特征权重 (第一分量 loading 的绝对值归一化)
 3. 全量加权求和 → `proxy_score` (0-1), `modern_tls_rate` 取反
@@ -200,6 +245,8 @@ extract_features(

 1. 列过滤 → 2. 全 NaN 列丢弃 → 3. 列均值填充 NaN → 4. StandardScaler → 5. 方差过滤 (< 1e-10) → 6. 相关过滤 (> 0.95) → 7. **TruncatedSVD (>50 维时)** → 8. min_cluster_size 自适应 (n//20) → 9. 聚类

+实现: `tools/clustering.py::_handle_run_clustering`。
+
 ### 7.3 低内存保护

 可用内存 < 2GB 时自动降采样到 50K 行。样本数 > 100K 时采样 100K。样本数 > 50K 时采样 50K。
@@ -213,6 +260,8 @@ extract_features(
 | Calinski-Harabasz Score | [0, +∞) | 越高越好 |
 | Noise Ratio | [0, 1] | HDBSCAN 噪声点比例 |

+实现: `tools/evaluate.py::_handle_evaluate_clustering`。
+
 ---

 ## 8. 特征提取与 UMAP 嵌入
@@ -223,7 +272,7 @@ extract_features(

 ### 8.2 UMAP-2D 嵌入

-在 `extract_features` 执行时自动计算:
+在 `extract_features` 执行时自动计算 (`tools/features.py`):
 - **训练**: 采样 ≤10K 行 → StandardScaler → UMAP(n_components=2, n_neighbors=15, min_dist=0.1)
 - **批量变换**: 1K 行批次 → 全量数据坐标
 - **持久化**: `EntityProfile` 表的 `embedding_x` / `embedding_y` 字段
@@ -233,9 +282,9 @@ extract_features(

 | 分数范围 | 含义 |
 |---------|------|
-| |Z| < 1.0 | 无明显区分力 |
-| 1.0 ≤ |Z| < 2.0 | 中等区分力 |
-| |Z| ≥ 2.0 | **强区分特征** (Web 界面颜色高亮) |
+| \|Z\| < 1.0 | 无明显区分力 |
+| 1.0 ≤ \|Z\| < 2.0 | 中等区分力 |
+| \|Z\| ≥ 2.0 | **强区分特征** (Web 界面颜色高亮) |

 ---

@@ -294,12 +343,17 @@ llm:

 ---

-## 11. 简易分析模块
+## 11. 手动分析工作流

-`simple_analysis/` 为独立 Django app (路由 `/simple/`), 提供简化的分析入口:
- 上传 CSV → 选择目标 IP 列 → 按目标筛选 → 地理聚类 / IP 子网聚类 → Leaflet 地图展示
- 零依赖外部地图服务 (OSM 瓦片可离线缓存)
- 与主分析模块共享 SessionStore 和配置
+手动分析页面 (`templates/tianxuan/manual.html`) 为**工作流构建器**:
+
+1. 用户添加/移除/重排分析步骤
+2. 每步选择 MCP 工具并填写参数
+3. 方案可保存/加载/删除 (JSON 文件, 存储于 `.omo/plans/`)
+4. 支持单步执行或全部自动串行执行
+5. 每步执行结果实时显示在步骤下方
+
+后端实现: `views/manual.py::manual_page` (页面渲染) + `views/manual.py::manual_run_analysis` (执行引擎)。

 ---

@@ -350,4 +404,10 @@ runtime\python\python.exe -c "import numpy, sklearn; print(numpy.__version__, sk

 ---

+## 附录: 模块历史
+
+> **simple_analysis 模块**: 原独立 Django app (`/simple/` 路由) 提供简化的上传→筛选→Leaflet 地图工作流。在 beta-clean 分支重构中已删除, 独立维护于 master 分支。当前版本不再包含此模块。
+
+---
+
 > **更多技术细节**: 请参阅 `AGENTS.md` 了解完整配置、架构决策和操作流程。
@@ -44,7 +44,7 @@ start /B runtime\python\python.exe manage.py runserver
 runtime\python\python.exe manage.py migrate
 ```

-`wsgi.py` 已包含 SQLite 启动自修复逻辑，自动创建缺失的数据库文件。
+`wsgi.py` 已包含 SQLite 启动自修复逻辑, 自动创建缺失的数据库文件。

 ### 1.3 端口被占用

@@ -59,6 +59,19 @@ server:
  port: 8080
 ```

+### 1.4 残留进程未清理
+
+**症状**: 前一次启动后进程未终止, 再次启动端口占用
+
+**解决**:
+```bat
+:: 检查 .server_pid 文件
+type .server_pid
+:: 强制终止
+taskkill /PID <pid> /F
+del .server_pid
+```
+
 ---

 ## 2. 数据加载问题
@@ -115,6 +128,12 @@ validate_data(dataset_id="ds_...")
 - 在 `config.yaml` 中手动指定: `columns: { src_ip: ipv4, dst_ip: ipv4 }`
 - 配置优先级: config → 值优先检测 → 名称推断 → STRING 回退

+### 2.5 ZIP 文件解压失败
+
+**症状**: 上传 ZIP 文件后无数据
+
+**解决**: 检查 ZIP 内文件是否为 `.csv` 格式。系统仅自动解压 ZIP 中的 CSV 文件。
+
 ---

 ## 3. 聚类问题
@@ -145,7 +164,7 @@ diagnose_clustering(dataset_id="...", cluster_result_id="...")

 **原因**: 跨机 dtype 推断不一致 (一台推断为 Float, 另一台为 String)

-**解决**: 已在 `build_aggregation_params()` 中修复 — 聚合前检查 dtype, 非数值列自动 cast。`_coerce_to_float()` 兜底。
+**解决**: 已在 `tools/entities.py::_handle_build_entity_profiles` 的聚合参数构建中修复 — 聚合前检查 dtype, 非数值列自动 cast。`_coerce_to_float()` 兜底。

 ### 3.3 KMeans 簇数与预期不符

@@ -226,7 +245,7 @@ runtime\python\python.exe scripts\debug_db.py
 ```

 **常见原因**:
- Django ORM sync_to_async 包装缺失 (已在 tool_registry.py 中修复)
+- Django ORM sync_to_async 包装缺失 (已在 `tools/features.py` 中修复)
 - AnalysisRun 记录未找到 (csv_glob 匹配失败)
 - 权限问题 (SQLite 文件只读)

@@ -282,7 +301,7 @@ type logs\tianxuan.log | findstr "[TOOL] [LLM]"
 **症状**: LLM 完成但无聚类结果

 **常见原因**:
- 实体检测失败 → 手动指定 `entity_column`
+- 实体检测失败 → 手动指定 `entity_columns`
 - 数据量太小 → 增大样本
 - 实体列检测到非 IP 列 → 通过 `entity_columns` 强制指定

@@ -310,22 +329,23 @@ type logs\tianxuan.log | findstr "[TOOL] [LLM]"
 **症状**: `/globe/` 页面渲染为空白

 **原因**:
+- WebGL2 不可用 (核显驱动过旧或浏览器不支持)
 - Three.js 未加载 (检查 Network tab, 603KB 的 three.min.js)
 - 地球贴图未加载 (1.4MB earth_atmos_2048.jpg)
- 数据源中没有经纬度信息

 **解决**:
+- 确认浏览器支持 WebGL2: 访问 `chrome://gpu` 检查 WebGL2 状态
 - 确认 `static/tianxuan/three.min.js` 和 `earth_atmos_2048.jpg` 存在
 - 数据中优先使用 `src_latitude/src_longitude` 列, 无则 GeoIP 回退
 - 确保 GeoIP 数据文件 `data/geoip_data.txt` 存在

-### 7.3 Leaflet 地图不加载 (简易分析模块)
+### 7.3 地理分布图空白

-**症状**: `/simple/` 页面地图灰色
+**症状**: 聚类概览页的地理分布 Canvas 不显示

-**原因**: OSM 瓦片服务器不可达 (离线环境)
+**原因**: 数据中无经纬度信息且 GeoIP 查询失败

-**解决**: 简易分析模块的 Leaflet 使用 OSM 瓦片, 离线时需预先缓存或切换到本地瓦片源。
+**解决**: 确保 GeoIP 数据库 `data/geoip_data.txt` 存在且格式正确。GeoIP 仅支持离线查询, 无网络依赖。

 ---

@@ -376,11 +396,11 @@ print(f'hashseed: {sys.flags.hash_randomization}')

 | 问题 | 修复方案 | 代码位置 |
 |------|---------|---------|
-| "mean on str" | frozenset → tuple | entity_aggregator.py |
+| "mean on str" | frozenset → tuple + 类型安全聚合 | tools/entities.py, ip_clustering.py |
 | 编码漂移 | PYTHONUTF8=1 | run.bat |
-| DB 特征不保存 | sync_to_async 包装 | tool_registry.py |
-| Chart.js 缺失 | 纯 Canvas 2D | 全部图表 |
-| 聚类结果随机 | random_state=42 | run_clustering/run_pipeline |
+| DB 特征不保存 | sync_to_async 包装 | tools/features.py |
+| 图表依赖缺失 | 纯 Canvas 2D | 全部图表 |
+| 聚类结果随机 | random_state=42 | tools/clustering.py, run_pipeline.py |

 ---

@@ -423,7 +443,7 @@ load_data(csv_glob="data/*.csv")

 **说明**: UMAP 计算分为训练 (≤10K 样本) + 批量变换 (1K 批次), 已是最优方案。

-**如需加速**: 减小 `MAX_UMAP_TRAIN` (tool_registry.py 中的硬编码常量, 默认 10000)。
+**如需加速**: 减小 `MAX_UMAP_TRAIN` (`tools/features.py` 中的硬编码常量, 默认 10000)。

 ---

@@ -436,6 +456,8 @@ load_data(csv_glob="data/*.csv")
 → diagnose_clustering (聚类诊断) → compare_datasets (处理前后对比) → export_debug_sample (导出样本)
 ```

+所有诊断工具位于 `tools/diagnostics.py`。
+
 ### 10.2 列结构调查

 ```bash
@@ -465,12 +487,12 @@ runtime\python\python.exe scripts\diagnose_compare.py log_machine_a.txt log_mach
 | `[LOAD_SCHEMA]` | data_loader.py | 每列推断类型 (最重要!) |
 | `[CLEAN]` | data_loader.py | 清洗操作 |
 | `[VALIDATE]` | data_validator.py | 数据校验 |
-| `[FIND_COL]` | entity_aggregator.py | 关键词匹配 |
-| `[CLUSTER_INPUT]` | tool_registry.py | 聚类输入形状 |
-| `[CLUSTER_SCALE]` | tool_registry.py | StandardScaler |
-| `[SCORE]` | tool_registry.py | 自适应评分权重 |
-| `[UMAP_ERR]` | tool_registry.py | UMAP 异常 |
-| `[DB_SAVE_ERROR]` | tool_registry.py | DB 持久化失败 |
+| `[FIND_COL]` | tools/entities.py | 实体列关键词匹配 |
+| `[CLUSTER_INPUT]` | tools/clustering.py | 聚类输入形状 |
+| `[CLUSTER_SCALE]` | tools/clustering.py | StandardScaler |
+| `[SCORE]` | tools/entities.py | 自适应评分权重 |
+| `[UMAP_ERR]` | tools/features.py | UMAP 异常 |
+| `[DB_SAVE_ERROR]` | tools/features.py | DB 持久化失败 |
 | `[DB_LOCKED]` | db_utils.py | SQLite 锁重试 |
 | `[LLM]` | llm_orchestrator.py | LLM 调用 |
 | `[TOOL]` | llm_orchestrator.py | LLM 调用工具 |
@@ -1,7 +1,30 @@
@echo off
 set PYTHONUTF8=1
 cd /d "%~dp0"
-echo Starting TianXuan...
-start /B runtime\python\python.exe scripts/start_server.py
-timeout /t 3 >nul
-timeout /t 5 >nul
+
+:: Start server in a separate window (detached, survives parent exit)
+:: /MIN runs minimized so it doesn't pop up a console window
+start "" /MIN runtime\python\python.exe scripts\start_server.py
+
+:: Wait for PID file (up to 10 seconds)
+set WAIT=0
+:waitloop
+if exist .server_pid goto gotpid
+ping -n 2 127.0.0.1 >nul
+set /a WAIT+=1
+if %WAIT% lss 5 goto waitloop
+
+:gotpid
+if exist .server_pid (
+    set /p PID=<.server_pid
+)
+if defined PID (
+    echo TianXuan started. PID: %PID%
+    echo Visit: http://127.0.0.1:8000/
+    echo.
+    echo To stop: taskkill /F /PID %PID%
+) else (
+    echo TianXuan started.
+    echo Visit: http://127.0.0.1:8000/
+    echo To stop: taskkill /F /IM python.exe
+)
@@ -0,0 +1,38 @@
+"""Download latest GeoLite2-City.mmdb for offline GeoIP resolution.
+
+Usage: runtime\python\python.exe scripts\download_geoip.py
+
+Downloads the latest GeoLite2-City.mmdb from MaxMind's mirror (via
+python-geoip-geolite2) and copies it to a non-Chinese path so the
+maxminddb C extension can read it on Windows.
+
+Requirements: python-geoip-geolite2, maxminddb
+"""
+
+from pathlib import Path
+import os
+import shutil
+import subprocess
+import sys
+
+
+def main():
+    print('Installing/updating python-geoip-geolite2...')
+    subprocess.check_call([
+        sys.executable, '-m', 'pip', 'install', '--upgrade',
+        'python-geoip-geolite2', 'maxminddb',
+    ])
+
+    import _geoip_geolite2
+    src = Path(_geoip_geolite2.__file__).parent / 'GeoLite2-City.mmdb'
+    dst = Path(os.environ.get('TEMP', 'C:/temp')) / 'opencode' / 'GeoLite2-City.mmdb'
+
+    dst.parent.mkdir(parents=True, exist_ok=True)
+    shutil.copy2(str(src), str(dst))
+    print(f'Copied {src.name} ({src.stat().st_size / 1e6:.1f} MB)')
+    print(f'  to {dst}')
+    print('GeoIP database ready.')
+
+
+if __name__ == '__main__':
+    main()
@@ -1,13 +1,10 @@
 """Start the development server with host and port from config.yaml.

-Reads server.host and server.port from config/config.yaml via the project's
-config loader, runs ``manage.py runserver <host>:<port>``, and opens a browser
-to http://127.0.0.1:<port> .
-
-Usage (by run.bat):
-    start /B runtime\\python\\python.exe scripts\\start_server.py
+All output (stdout + stderr) is redirected to log files so the process
+can run in the background without blocking any terminal.
 """

+import os
 import subprocess
 import sys
 import webbrowser
@@ -22,21 +19,38 @@ from config import get_config

 def main() -> None:
    cfg = get_config()
-
    host = cfg.server.host or '0.0.0.0'
-    port = cfg.server.port or 80
-
+    port = cfg.server.port or 8000
    addr = f'{host}:{port}'
-    manage_py = Path(__file__).resolve().parent.parent / 'manage.py'
+    manage_py = _project_root / 'manage.py'

-    # Open browser — use 127.0.0.1 regardless of bind address so the user's
-    # browser always reaches the local machine.
+    # Log files — capture ALL output so nothing goes to console
+    stdout_log = _project_root / 'server_stdout.txt'
+    stderr_log = _project_root / 'server_stderr.txt'
+    pid_file = _project_root / '.server_pid'
+
+    # Write PID so run.bat can print it
+    pid_file.write_text(str(os.getpid()), encoding='utf-8')
+
+    # Open browser
    webbrowser.open(f'http://127.0.0.1:{port}/')

-    # Run the server.  subprocess.call is blocking, so this script stays alive
-    # as long as the dev server does — the same effect as calling manage.py
-    # directly under start /B.
-    sys.exit(subprocess.call([sys.executable, str(manage_py), 'runserver', addr, '--noreload']))
+    # Run server with ALL output redirected to files
+    with open(stdout_log, 'a', encoding='utf-8') as out_fh, \
+         open(stderr_log, 'a', encoding='utf-8') as err_fh:
+        rc = subprocess.call(
+            [sys.executable, str(manage_py), 'runserver', addr, '--noreload'],
+            stdout=out_fh,
+            stderr=err_fh,
+        )
+
+    # Clean up PID file on exit
+    try:
+        pid_file.unlink(missing_ok=True)
+    except Exception:
+        pass
+
+    sys.exit(rc)


 if __name__ == '__main__':
@@ -0,0 +1,178 @@
+Restored 2 datasets from disk
+[2026-07-23 13:06:51,899] INFO django: Restored 2 datasets from disk
+[2026-07-23 13:06:52,038] INFO analysis.data_loader: [LOAD_FROM_DB_LAZY] table=_data_64 rows=9990 cols=57
+[2026-07-23 13:06:52,043] INFO analysis.data_loader: [LOAD_FROM_DB_LAZY] table=_data_66 rows=2000 cols=9
+[23/Jul/2026 13:21:24] "GET / HTTP/1.1" 200 10790
+[23/Jul/2026 13:21:24,471] - Broken pipe from ('127.0.0.1', 64227)
+[2026-07-23 13:27:21,369] INFO analysis.views: [UPLOAD] Received 1998 files
+[23/Jul/2026 13:27:22] "POST /upload/csv/ HTTP/1.1" 200 178755
+[23/Jul/2026 13:27:22] "POST /runs/5/finalize/ HTTP/1.1" 200 34
+[23/Jul/2026 13:27:22] "GET /runs/5/status/ HTTP/1.1" 200 188
+[2026-07-23 13:27:22,672] INFO analysis.views: [BACKGROUND] Forcing lat/lon column ':ips.latd' to Utf8 for safe parsing
+[2026-07-23 13:27:22,672] INFO analysis.views: [BACKGROUND] Forcing lat/lon column ':ips.lond' to Utf8 for safe parsing
+[2026-07-23 13:27:22,672] INFO analysis.views: [BACKGROUND] Forcing lat/lon column ':ipd.latd' to Utf8 for safe parsing
+[2026-07-23 13:27:22,672] INFO analysis.views: [BACKGROUND] Forcing lat/lon column ':ipd.lond' to Utf8 for safe parsing
+[2026-07-23 13:27:22,672] INFO analysis.views: [BACKGROUND] Found 1998 CSV files in C:\Users\25044\AppData\Roaming\TianXuan\data\uploads\20260723_052721
+[2026-07-23 13:27:23,325] INFO analysis.data_loader: [SAVE_TO_DB] table=_data_68 rows=495 mode=append
+[2026-07-23 13:27:23,609] INFO analysis.data_loader: [SAVE_TO_DB] table=_data_68 rows=495 mode=append
+[2026-07-23 13:27:23,895] INFO analysis.data_loader: [SAVE_TO_DB] table=_data_68 rows=495 mode=append
+[2026-07-23 13:27:24,170] INFO analysis.data_loader: [SAVE_TO_DB] table=_data_68 rows=495 mode=append
+[2026-07-23 13:27:24,447] INFO analysis.data_loader: [SAVE_TO_DB] table=_data_68 rows=495 mode=append
+[2026-07-23 13:27:24,733] INFO analysis.data_loader: [SAVE_TO_DB] table=_data_68 rows=495 mode=append
+[2026-07-23 13:27:25,000] INFO analysis.data_loader: [SAVE_TO_DB] table=_data_68 rows=495 mode=append
+[2026-07-23 13:27:25,268] INFO analysis.data_loader: [SAVE_TO_DB] table=_data_68 rows=495 mode=append
+[2026-07-23 13:27:25,553] INFO analysis.data_loader: [SAVE_TO_DB] table=_data_68 rows=495 mode=append
+[23/Jul/2026 13:27:25] "GET /runs/5/status/ HTTP/1.1" 200 242
+[2026-07-23 13:27:25,838] INFO analysis.data_loader: [SAVE_TO_DB] table=_data_68 rows=495 mode=append
+[2026-07-23 13:27:26,116] INFO analysis.data_loader: [SAVE_TO_DB] table=_data_68 rows=495 mode=append
+[2026-07-23 13:27:26,392] INFO analysis.data_loader: [SAVE_TO_DB] table=_data_68 rows=495 mode=append
+[2026-07-23 13:27:26,668] INFO analysis.data_loader: [SAVE_TO_DB] table=_data_68 rows=495 mode=append
+[2026-07-23 13:27:26,946] INFO analysis.data_loader: [SAVE_TO_DB] table=_data_68 rows=495 mode=append
+[2026-07-23 13:27:27,231] INFO analysis.data_loader: [SAVE_TO_DB] table=_data_68 rows=495 mode=append
+[2026-07-23 13:27:27,497] INFO analysis.data_loader: [SAVE_TO_DB] table=_data_68 rows=495 mode=append
+[2026-07-23 13:27:27,782] INFO analysis.data_loader: [SAVE_TO_DB] table=_data_68 rows=495 mode=append
+[2026-07-23 13:27:28,071] INFO analysis.data_loader: [SAVE_TO_DB] table=_data_68 rows=495 mode=append
+[2026-07-23 13:27:28,349] INFO analysis.data_loader: [SAVE_TO_DB] table=_data_68 rows=495 mode=append
+[2026-07-23 13:27:28,621] INFO analysis.data_loader: [SAVE_TO_DB] table=_data_68 rows=495 mode=append
+[23/Jul/2026 13:27:28] "GET /runs/5/status/ HTTP/1.1" 200 243
+[2026-07-23 13:27:28,713] INFO analysis.data_loader: [SAVE_TO_DB] table=_data_68 rows=90 mode=append
+E:\hjq\澶╃拠\analysis\views.py:584: PerformanceWarning: Determining the column names of a LazyFrame requires resolving its schema, which is a potentially expensive operation. Use `LazyFrame.collect_schema().names()` to get the column names without this warning.
+  if lat_lon_col in lf.columns:
+[2026-07-23 13:27:29,071] INFO analysis.views: [BACKGROUND] Restored numeric types for 16 columns: ['1ipp', '2tmo', '4dbn', '4dur', '4srs', '8ack', '8byt', '8dbd', '8did', '8pak']
+[2026-07-23 13:27:29,150] INFO analysis.views: [BACKGROUND] SVD: 15 components from 16 numeric columns (explained variance ratio sum=1.0000)
+[23/Jul/2026 13:27:31] "GET /runs/5/status/ HTTP/1.1" 200 223
+[23/Jul/2026 13:27:56] "POST /analyze/run/ HTTP/1.1" 200 45
+[23/Jul/2026 13:27:56] "GET /runs/5/status/ HTTP/1.1" 200 223
+[2026-07-23 13:27:57,344] INFO analysis.tool_registry: [CLUSTER_INPUT] rows=9990 cols=10 feature_cols=['1ipp', '2tmo', '4dbn', '4dur', '4srs', '8ack', '8byt', '8dbd', '8did', '8pak']
+[2026-07-23 13:27:57,344] INFO analysis.tool_registry: [CLUSTER_SCALE] Starting StandardScaler...
+[2026-07-23 13:27:57,344] INFO analysis.tool_registry: [CLUSTER_SCALE] Done. shape=(9990, 10)
+E:\hjq\澶╃拠\runtime\python\Lib\site-packages\sklearn\cluster\_hdbscan\hdbscan.py:722: FutureWarning: The default value of `copy` will change from False to True in 1.10. Explicitly set a value for `copy` to silence this warning.
+  warn(
+[23/Jul/2026 13:27:59] "GET /runs/5/status/ HTTP/1.1" 200 240
+[23/Jul/2026 13:28:02] "GET /runs/5/status/ HTTP/1.1" 200 240
+[23/Jul/2026 13:28:06] "GET /runs/5/status/ HTTP/1.1" 200 240
+E:\hjq\澶╃拠\runtime\python\Lib\site-packages\umap\umap_.py:1952: UserWarning: n_jobs value 1 overridden to 1 by setting random_state. Use no seed for parallelism.
+  warn(
+[23/Jul/2026 13:28:09] "GET /runs/5/status/ HTTP/1.1" 200 240
+[23/Jul/2026 13:28:12] "GET /runs/5/status/ HTTP/1.1" 200 240
+[23/Jul/2026 13:28:15] "GET /runs/5/status/ HTTP/1.1" 200 240
+[23/Jul/2026 13:28:18] "GET /runs/5/status/ HTTP/1.1" 200 240
+[23/Jul/2026 13:28:21] "GET /runs/5/status/ HTTP/1.1" 200 240
+[23/Jul/2026 13:28:24] "GET /runs/5/status/ HTTP/1.1" 200 240
+[23/Jul/2026 13:28:27] "GET /runs/5/status/ HTTP/1.1" 200 240
+[23/Jul/2026 13:28:30] "GET /runs/5/status/ HTTP/1.1" 200 240
+[23/Jul/2026 13:28:33] "GET /runs/5/status/ HTTP/1.1" 200 240
+[23/Jul/2026 13:28:37] "GET /runs/5/status/ HTTP/1.1" 200 240
+[2026-07-23 13:28:38,183] INFO analysis.geoip: [GEOIP] loaded 803 ranges from E:\hjq\澶╃拠\data\geoip_data.txt (0 skipped)
+E:\hjq\澶╃拠\runtime\python\Lib\site-packages\umap\umap_.py:1952: UserWarning: n_jobs value 1 overridden to 1 by setting random_state. Use no seed for parallelism.
+  warn(
+[23/Jul/2026 13:28:40] "GET /runs/5/status/ HTTP/1.1" 200 254
+Restored 3 datasets from disk
+[2026-07-23 21:16:22,571] INFO django: Restored 3 datasets from disk
+[2026-07-23 21:16:22,802] INFO analysis.data_loader: [LOAD_FROM_DB_LAZY] table=_data_64 rows=9990 cols=57
+[2026-07-23 21:16:22,804] INFO analysis.data_loader: [LOAD_FROM_DB_LAZY] table=_data_66 rows=2000 cols=9
+[2026-07-23 21:16:22,992] INFO analysis.data_loader: [LOAD_FROM_DB_LAZY] table=_data_68 rows=9990 cols=57
+[23/Jul/2026 21:16:44] "GET / HTTP/1.1" 200 12728
+[23/Jul/2026 21:16:44] "GET /static/tianxuan/favicon.svg HTTP/1.1" 304 0
+[23/Jul/2026 21:17:14] "HEAD / HTTP/1.1" 200 0
+[23/Jul/2026 21:17:20] "HEAD / HTTP/1.1" 200 0
+Restored 3 datasets from disk
+[2026-07-23 21:22:58,596] INFO django: Restored 3 datasets from disk
+[2026-07-23 21:22:58,783] INFO analysis.data_loader: [LOAD_FROM_DB_LAZY] table=_data_64 rows=9990 cols=57
+[2026-07-23 21:22:58,801] INFO analysis.data_loader: [LOAD_FROM_DB_LAZY] table=_data_66 rows=2000 cols=9
+[2026-07-23 21:22:59,009] INFO analysis.data_loader: [LOAD_FROM_DB_LAZY] table=_data_68 rows=9990 cols=57
+[23/Jul/2026 21:22:59] "GET / HTTP/1.1" 200 12728
+Restored 3 datasets from disk
+[2026-07-23 21:37:07[23/Jul/2026 21:37:07] "GET / HTTP/1.1" 200 12728
[[23/Jul/2026 21:37:38] "HEAD / HTTP/1.1" 200 0
+[23/Jul/2026 21:37:49] "HEAD / HTTP/1.1" 200 0
+[23/Jul/2026 21:38:08] "HEAD / HTTP/1.1" 200 0
+[23/Jul/2026 21:38:38] "HEAD / HTTP/1.1" 200 0
+[23/Jul/2026 21:39:08] "HEAD / HTTP/1.1" 200 0
+[23/Jul/2026 21:39:38] "HEAD / HTTP/1.1" 200 0
+[23/Jul/2026 21:39:43] "HEAD / HTTP/1.1" 200 0
+[23/Jul/2026 21:39:45] "HEAD / HTTP/1.1" 200 0
+[23/Jul/2026 21:39:49] "HEAD / HTTP/1.1" 200 0
+[23/Jul/2026 21:40:08] "HEAD / HTTP/1.1" 200 0
+[23/Jul/2026 21:40:38] "HEAD / HTTP/1.1" 200 0
+[23/Jul/2026 21:41:35] "HEAD / HTTP/1.1" 200 0
+Restored 3 datasets from disk
+[2026-07-23 21:42:01,918] INFO django: Restored 3 datasets from disk
+[2026-07-23 21:42:02,145] INFO analysis.data_loader: [LOAD_FROM_DB_LAZY] table=_data_64 rows=9990 cols=57
+[2026-07-23 21:42:02,153] INFO analysis.data_loader: [LOAD_FROM_DB_LAZY] table=_data_66 rows=2000 cols=9
+[2026-07-23 21:42:02,295] INFO analysis.data_loader: [LOAD_FROM_DB_LAZY] table=_data_68 rows=9990 cols=57
+esponse = wrapped_callback(request, *callback_args, **callback_kwargs)
+               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
+  File "E:\hjq\天璇\simple_analysis\views.py", line 796, in index
+  File "E:\hjq\天璇\runtime\python\Lib\site-packages\django\shortcuts.py", line 24, in render
+    content = loader.render_to_string(template_name, context, request, using=using)
+              ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
+  File "E:\hjq\天璇\runtime\python\Lib\site-packages\django\template\loader.py", line 61, in render_to_string
+    template = get_template(template_name, using=using)
+               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
+  File "E:\hjq\天璇\runtime\python\Lib\site-packages\django\template\loader.py", line 19, in get_template
+    raise TemplateDoesNotExist(template_name, chain=chain)
+django.template.exceptions.TemplateDoesNotExist: simple_analysis/simple_analysis.html
+[2026-07-23 21:41:56,536] ERROR django.request: Internal Server Error: /simple/
+Traceback (most recent call last):
+  File "E:\hjq\天璇\runtime\python\Lib\site-packages\django\core\handlers\exception.py", line 55, in inner
+    response = get_response(request)
+               ^^^^^^^^^^^^^^^^^^^^^
+  File "E:\hjq\天璇\runtime\python\Lib\site-packages\django\core\handlers\base.py", line 197, in _get_response
+    response = wrapped_callback(request, *callback_args, **callback_kwargs)
+               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
+  File "E:\hjq\天璇\simple_analysis\views.py", line 796, in index
+  File "E:\hjq\天璇\runtime\python\Lib\site-packages\django\shortcuts.py", line 24, in render
+    content = loader.render_to_string(template_name, context, request, using=using)
+              ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
+  File "E:\hjq\天璇\runtime\python\Lib\site-packages\django\template\loader.py", line 61, in render_to_string
+    template = get_template(template_name, using=using)
+               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
+  File "E:\hjq\天璇\runtime\python\Lib\site-packages\django\template\loader.py", line 19, in get_template
+    raise TemplateDoesNotExist(template_name, chain=chain)
+django.template.exceptions.TemplateDoesNotExist: simple_analysis/simple_analysis.html
+[23/Jul/2026 21:41:56] "GET /simple/ HTTP/1.1" 500 80559
+Not Found: /favicon.ico
+[2026-07-23 21:41:56,814] WARNING django.request: Not Found: /favicon.ico
+[23/Jul/2026 21:41:56] "GET /favicon.ico HTTP/1.1" 404 6872
+Restored 3 datasets from disk
+[2026-07-23 21:45:23,039] INFO django: Restored 3 datasets from disk
+[2026-07-23 21:45:23,182] INFO analysis.data_loader: [LOAD_FROM_DB_LAZY] table=_data_64 rows=9990 cols=57
+[2026-07-23 21:45:23,190] INFO analysis.data_loader: [LOAD_FROM_DB_LAZY] table=_data_66 rows=2000 cols=9
+[2026-07-23 21:45:23,314] INFO analysis.data_loader: [LOAD_FROM_DB_LAZY] table=_data_68 rows=9990 cols=57
+[23/Jul/2026 21:45:28] "GET / HTTP/1.1" 200 12645
+[23/Jul/2026 21:45:31] "GET /upload/ HTTP/1.1" 200 19078
+[23/Jul/2026 21:45:44] "POST /runs/5/delete/ HTTP/1.1" 200 17
+[23/Jul/2026 21:45:44] "GET /upload/ HTTP/1.1" 200 18534
+[23/Jul/2026 21:45:46] "POST /runs/4/delete/ HTTP/1.1" 200 17
+[23/Jul/2026 21:45:46] "GET /upload/ HTTP/1.1" 200 17939
+[23/Jul/2026 21:45:48] "POST /runs/3/delete/ HTTP/1.1" 200 17
+[23/Jul/2026 21:45:48] "GET /upload/ HTTP/1.1" 200 17395
+[23/Jul/2026 21:45:51] "POST /runs/2/delete/ HTTP/1.1" 200 17
+[23/Jul/2026 21:45:51] "GET /upload/ HTTP/1.1" 200 16853
+[23/Jul/2026 21:45:52] "POST /runs/1/delete/ HTTP/1.1" 200 17
+[23/Jul/2026 21:45:52] "GET /upload/ HTTP/1.1" 200 16136
+[23/Jul/2026 21:46:23] "HEAD / HTTP/1.1" 200 0
+[23/Jul/2026 21:46:53] "HEAD / HTTP/1.1" 200 0
+[23/Jul/2026 21:47:23] "HEAD / HTTP/1.1" 200 0
+[23/Jul/2026 21:47:53] "HEAD / HTTP/1.1" 200 0
+[23/Jul/2026 21:48:23] "HEAD / HTTP/1.1" 200 0
+[23/Jul/2026 21:48:53] "HEAD / HTTP/1.1" 200 0
+[23/Jul/2026 21:49:23] "HEAD / HTTP/1.1" 200 0
+[23/Jul/2026 21:49:53] "HEAD / HTTP/1.1" 200 0
+[23/Jul/2026 21:50:23] "HEAD / HTTP/1.1" 200 0
+Restored 3 datasets from disk
+[2026-07-23 21:50:50,464] INFO django: Restored 3 datasets from disk
+[23/Jul/2026 21:50:54] "GET /analyze/manual/ HTTP/1.1" 200 65847
+[23/Jul/2026 21:50:54] "GET /tools/plan/ HTTP/1.1" 200 13
+[23/Jul/2026 21:50:57] "GET /analyze/auto/ HTTP/1.1" 200 39486
+[23/Jul/2026 21:51:09] "GET /runs/ HTTP/1.1" 200 9674
+[2026-07-23 21:51:10,324] INFO analysis.geoip: [GEOIP] loaded 0 ranges (0 skipped)
+[2026-07-23 21:51:10,325] INFO analysis.geoip: [GEOIP] loaded 12 cached entries
+[23/Jul/2026 21:51:10] "GET /globe/ HTTP/1.1" 200 18520
+[23/Jul/2026 21:51:10] "GET /static/tianxuan/three.min.js HTTP/1.1" 304 0
+[23/Jul/2026 21:51:10] "GET /static/tianxuan/world_borders.js HTTP/1.1" 304 0
+[23/Jul/2026 21:51:10] "GET /static/tianxuan/three.min.js HTTP/1.1" 304 0
+[23/Jul/2026 21:51:10] "GET /static/tianxuan/world_borders.js HTTP/1.1" 304 0
+[23/Jul/2026 21:51:10] "GET /static/tianxuan/earth_atmos_2048.jpg HTTP/1.1" 304 0
+[23/Jul/2026 21:51:27] "GET /config/ HTTP/1.1" 200 14372
+[23/Jul/2026 21:51:31] "GET /analyze/auto/ HTTP/1.1" 200 39486
@@ -0,0 +1,65 @@
+Performing system checks...
+
+System check identified no issues (0 silenced).
+July 23, 2026 - 13:06:51
+Django version 4.2.30, using settings 'tianxuan.settings'
+Starting development server at http://127.0.0.1:18766/
+Quit the server with CTRL-BREAK.
+
+Performing system checks...
+
+System check identified no issues (0 silenced).
+Performing system checks...
+
+System check identified no issues (0 silenced).
+Performing system checks...
+
+System check identified no issues (0 silenced).
+July 23, 2026 - 21:16:22
+Django version 4.2.30, using settings 'tianxuan.settings'
+Starting development server at http://0.0.0.0:8000/
+Quit the server with CTRL-BREAK.
+
+Performing system checks...
+
+System check identified no issues (0 silenced).
+Performing system checks...
+
+System check identified no issues (0 silenced).
+July 23, 2026 - 21:22:58
+Django version 4.2.30, using settings 'tianxuan.settings'
+Starting development server at http://0.0.0.0:8000/
+Quit the server with CTRL-BREAK.
+
+Performing system checks...
+
+System check identified no issues (0 silenced).
+July 23, 2026 - 21:37:07
+Django version 4.2.30, using settings 'tianxuan.settings'
+Starting development server at http://0.0.0.0:8000/
+Quit the server with CTRL-BREAK.
+
+Performing system checks...
+
+System check identified no issues (0 silenced).
+July 23, 2026 - 21:42:01
+Django version 4.2.30, using settings 'tianxuan.settings'
+Starting development server at http://0.0.0.0:8000/
+Quit the server with CTRL-BREAK.
+
+Performing system checks...
+
+System check identified no issues (0 silenced).
+July 23, 2026 - 21:45:23
+Django version 4.2.30, using settings 'tianxuan.settings'
+Starting development server at http://0.0.0.0:8000/
+Quit the server with CTRL-BREAK.
+
+Performing system checks...
+
+System check identified no issues (0 silenced).
+July 23, 2026 - 21:50:50
+Django version 4.2.30, using settings 'tianxuan.settings'
+Starting development server at http://0.0.0.0:8000/
+Quit the server with CTRL-BREAK.
+
@@ -1,7 +0,0 @@
-from django.apps import AppConfig
-
-
-class SimpleAnalysisConfig(AppConfig):
-    default_auto_field = 'django.db.models.BigAutoField'
-    name = 'simple_analysis'
-    label = 'simple_analysis'
@@ -1,8 +0,0 @@
-/* 天璇 简单分析模块 — 前端辅助逻辑
- * 主要逻辑内嵌在 simple_analysis.html 中，
- * 此文件供未来扩展或拆分使用。
- */
-(function() {
-    'use strict';
-    console.log('简单分析模块已加载');
-})();
@@ -1,661 +0,0 @@
-/* required styles */
-
-.leaflet-pane,
-.leaflet-tile,
-.leaflet-marker-icon,
-.leaflet-marker-shadow,
-.leaflet-tile-container,
-.leaflet-pane > svg,
-.leaflet-pane > canvas,
-.leaflet-zoom-box,
-.leaflet-image-layer,
-.leaflet-layer {
-	position: absolute;
-	left: 0;
-	top: 0;
-	}
-.leaflet-container {
-	overflow: hidden;
-	}
-.leaflet-tile,
-.leaflet-marker-icon,
-.leaflet-marker-shadow {
-	-webkit-user-select: none;
-	   -moz-user-select: none;
-	        user-select: none;
-	  -webkit-user-drag: none;
-	}
-/* Prevents IE11 from highlighting tiles in blue */
-.leaflet-tile::selection {
-	background: transparent;
-}
-/* Safari renders non-retina tile on retina better with this, but Chrome is worse */
-.leaflet-safari .leaflet-tile {
-	image-rendering: -webkit-optimize-contrast;
-	}
-/* hack that prevents hw layers "stretching" when loading new tiles */
-.leaflet-safari .leaflet-tile-container {
-	width: 1600px;
-	height: 1600px;
-	-webkit-transform-origin: 0 0;
-	}
-.leaflet-marker-icon,
-.leaflet-marker-shadow {
-	display: block;
-	}
-/* .leaflet-container svg: reset svg max-width decleration shipped in Joomla! (joomla.org) 3.x */
-/* .leaflet-container img: map is broken in FF if you have max-width: 100% on tiles */
-.leaflet-container .leaflet-overlay-pane svg {
-	max-width: none !important;
-	max-height: none !important;
-	}
-.leaflet-container .leaflet-marker-pane img,
-.leaflet-container .leaflet-shadow-pane img,
-.leaflet-container .leaflet-tile-pane img,
-.leaflet-container img.leaflet-image-layer,
-.leaflet-container .leaflet-tile {
-	max-width: none !important;
-	max-height: none !important;
-	width: auto;
-	padding: 0;
-	}
-
-.leaflet-container img.leaflet-tile {
-	/* See: https://bugs.chromium.org/p/chromium/issues/detail?id=600120 */
-	mix-blend-mode: plus-lighter;
-}
-
-.leaflet-container.leaflet-touch-zoom {
-	-ms-touch-action: pan-x pan-y;
-	touch-action: pan-x pan-y;
-	}
-.leaflet-container.leaflet-touch-drag {
-	-ms-touch-action: pinch-zoom;
-	/* Fallback for FF which doesn't support pinch-zoom */
-	touch-action: none;
-	touch-action: pinch-zoom;
-}
-.leaflet-container.leaflet-touch-drag.leaflet-touch-zoom {
-	-ms-touch-action: none;
-	touch-action: none;
-}
-.leaflet-container {
-	-webkit-tap-highlight-color: transparent;
-}
-.leaflet-container a {
-	-webkit-tap-highlight-color: rgba(51, 181, 229, 0.4);
-}
-.leaflet-tile {
-	filter: inherit;
-	visibility: hidden;
-	}
-.leaflet-tile-loaded {
-	visibility: inherit;
-	}
-.leaflet-zoom-box {
-	width: 0;
-	height: 0;
-	-moz-box-sizing: border-box;
-	     box-sizing: border-box;
-	z-index: 800;
-	}
-/* workaround for https://bugzilla.mozilla.org/show_bug.cgi?id=888319 */
-.leaflet-overlay-pane svg {
-	-moz-user-select: none;
-	}
-
-.leaflet-pane         { z-index: 400; }
-
-.leaflet-tile-pane    { z-index: 200; }
-.leaflet-overlay-pane { z-index: 400; }
-.leaflet-shadow-pane  { z-index: 500; }
-.leaflet-marker-pane  { z-index: 600; }
-.leaflet-tooltip-pane   { z-index: 650; }
-.leaflet-popup-pane   { z-index: 700; }
-
-.leaflet-map-pane canvas { z-index: 100; }
-.leaflet-map-pane svg    { z-index: 200; }
-
-.leaflet-vml-shape {
-	width: 1px;
-	height: 1px;
-	}
-.lvml {
-	behavior: url(#default#VML);
-	display: inline-block;
-	position: absolute;
-	}
-
-
-/* control positioning */
-
-.leaflet-control {
-	position: relative;
-	z-index: 800;
-	pointer-events: visiblePainted; /* IE 9-10 doesn't have auto */
-	pointer-events: auto;
-	}
-.leaflet-top,
-.leaflet-bottom {
-	position: absolute;
-	z-index: 1000;
-	pointer-events: none;
-	}
-.leaflet-top {
-	top: 0;
-	}
-.leaflet-right {
-	right: 0;
-	}
-.leaflet-bottom {
-	bottom: 0;
-	}
-.leaflet-left {
-	left: 0;
-	}
-.leaflet-control {
-	float: left;
-	clear: both;
-	}
-.leaflet-right .leaflet-control {
-	float: right;
-	}
-.leaflet-top .leaflet-control {
-	margin-top: 10px;
-	}
-.leaflet-bottom .leaflet-control {
-	margin-bottom: 10px;
-	}
-.leaflet-left .leaflet-control {
-	margin-left: 10px;
-	}
-.leaflet-right .leaflet-control {
-	margin-right: 10px;
-	}
-
-
-/* zoom and fade animations */
-
-.leaflet-fade-anim .leaflet-popup {
-	opacity: 0;
-	-webkit-transition: opacity 0.2s linear;
-	   -moz-transition: opacity 0.2s linear;
-	        transition: opacity 0.2s linear;
-	}
-.leaflet-fade-anim .leaflet-map-pane .leaflet-popup {
-	opacity: 1;
-	}
-.leaflet-zoom-animated {
-	-webkit-transform-origin: 0 0;
-	    -ms-transform-origin: 0 0;
-	        transform-origin: 0 0;
-	}
-svg.leaflet-zoom-animated {
-	will-change: transform;
-}
-
-.leaflet-zoom-anim .leaflet-zoom-animated {
-	-webkit-transition: -webkit-transform 0.25s cubic-bezier(0,0,0.25,1);
-	   -moz-transition:    -moz-transform 0.25s cubic-bezier(0,0,0.25,1);
-	        transition:         transform 0.25s cubic-bezier(0,0,0.25,1);
-	}
-.leaflet-zoom-anim .leaflet-tile,
-.leaflet-pan-anim .leaflet-tile {
-	-webkit-transition: none;
-	   -moz-transition: none;
-	        transition: none;
-	}
-
-.leaflet-zoom-anim .leaflet-zoom-hide {
-	visibility: hidden;
-	}
-
-
-/* cursors */
-
-.leaflet-interactive {
-	cursor: pointer;
-	}
-.leaflet-grab {
-	cursor: -webkit-grab;
-	cursor:    -moz-grab;
-	cursor:         grab;
-	}
-.leaflet-crosshair,
-.leaflet-crosshair .leaflet-interactive {
-	cursor: crosshair;
-	}
-.leaflet-popup-pane,
-.leaflet-control {
-	cursor: auto;
-	}
-.leaflet-dragging .leaflet-grab,
-.leaflet-dragging .leaflet-grab .leaflet-interactive,
-.leaflet-dragging .leaflet-marker-draggable {
-	cursor: move;
-	cursor: -webkit-grabbing;
-	cursor:    -moz-grabbing;
-	cursor:         grabbing;
-	}
-
-/* marker & overlays interactivity */
-.leaflet-marker-icon,
-.leaflet-marker-shadow,
-.leaflet-image-layer,
-.leaflet-pane > svg path,
-.leaflet-tile-container {
-	pointer-events: none;
-	}
-
-.leaflet-marker-icon.leaflet-interactive,
-.leaflet-image-layer.leaflet-interactive,
-.leaflet-pane > svg path.leaflet-interactive,
-svg.leaflet-image-layer.leaflet-interactive path {
-	pointer-events: visiblePainted; /* IE 9-10 doesn't have auto */
-	pointer-events: auto;
-	}
-
-/* visual tweaks */
-
-.leaflet-container {
-	background: #ddd;
-	outline-offset: 1px;
-	}
-.leaflet-container a {
-	color: #0078A8;
-	}
-.leaflet-zoom-box {
-	border: 2px dotted #38f;
-	background: rgba(255,255,255,0.5);
-	}
-
-
-/* general typography */
-.leaflet-container {
-	font-family: "Helvetica Neue", Arial, Helvetica, sans-serif;
-	font-size: 12px;
-	font-size: 0.75rem;
-	line-height: 1.5;
-	}
-
-
-/* general toolbar styles */
-
-.leaflet-bar {
-	box-shadow: 0 1px 5px rgba(0,0,0,0.65);
-	border-radius: 4px;
-	}
-.leaflet-bar a {
-	background-color: #fff;
-	border-bottom: 1px solid #ccc;
-	width: 26px;
-	height: 26px;
-	line-height: 26px;
-	display: block;
-	text-align: center;
-	text-decoration: none;
-	color: black;
-	}
-.leaflet-bar a,
-.leaflet-control-layers-toggle {
-	background-position: 50% 50%;
-	background-repeat: no-repeat;
-	display: block;
-	}
-.leaflet-bar a:hover,
-.leaflet-bar a:focus {
-	background-color: #f4f4f4;
-	}
-.leaflet-bar a:first-child {
-	border-top-left-radius: 4px;
-	border-top-right-radius: 4px;
-	}
-.leaflet-bar a:last-child {
-	border-bottom-left-radius: 4px;
-	border-bottom-right-radius: 4px;
-	border-bottom: none;
-	}
-.leaflet-bar a.leaflet-disabled {
-	cursor: default;
-	background-color: #f4f4f4;
-	color: #bbb;
-	}
-
-.leaflet-touch .leaflet-bar a {
-	width: 30px;
-	height: 30px;
-	line-height: 30px;
-	}
-.leaflet-touch .leaflet-bar a:first-child {
-	border-top-left-radius: 2px;
-	border-top-right-radius: 2px;
-	}
-.leaflet-touch .leaflet-bar a:last-child {
-	border-bottom-left-radius: 2px;
-	border-bottom-right-radius: 2px;
-	}
-
-/* zoom control */
-
-.leaflet-control-zoom-in,
-.leaflet-control-zoom-out {
-	font: bold 18px 'Lucida Console', Monaco, monospace;
-	text-indent: 1px;
-	}
-
-.leaflet-touch .leaflet-control-zoom-in, .leaflet-touch .leaflet-control-zoom-out  {
-	font-size: 22px;
-	}
-
-
-/* layers control */
-
-.leaflet-control-layers {
-	box-shadow: 0 1px 5px rgba(0,0,0,0.4);
-	background: #fff;
-	border-radius: 5px;
-	}
-.leaflet-control-layers-toggle {
-	background-image: url(images/layers.png);
-	width: 36px;
-	height: 36px;
-	}
-.leaflet-retina .leaflet-control-layers-toggle {
-	background-image: url(images/layers-2x.png);
-	background-size: 26px 26px;
-	}
-.leaflet-touch .leaflet-control-layers-toggle {
-	width: 44px;
-	height: 44px;
-	}
-.leaflet-control-layers .leaflet-control-layers-list,
-.leaflet-control-layers-expanded .leaflet-control-layers-toggle {
-	display: none;
-	}
-.leaflet-control-layers-expanded .leaflet-control-layers-list {
-	display: block;
-	position: relative;
-	}
-.leaflet-control-layers-expanded {
-	padding: 6px 10px 6px 6px;
-	color: #333;
-	background: #fff;
-	}
-.leaflet-control-layers-scrollbar {
-	overflow-y: scroll;
-	overflow-x: hidden;
-	padding-right: 5px;
-	}
-.leaflet-control-layers-selector {
-	margin-top: 2px;
-	position: relative;
-	top: 1px;
-	}
-.leaflet-control-layers label {
-	display: block;
-	font-size: 13px;
-	font-size: 1.08333em;
-	}
-.leaflet-control-layers-separator {
-	height: 0;
-	border-top: 1px solid #ddd;
-	margin: 5px -10px 5px -6px;
-	}
-
-/* Default icon URLs */
-.leaflet-default-icon-path { /* used only in path-guessing heuristic, see L.Icon.Default */
-	background-image: url(images/marker-icon.png);
-	}
-
-
-/* attribution and scale controls */
-
-.leaflet-container .leaflet-control-attribution {
-	background: #fff;
-	background: rgba(255, 255, 255, 0.8);
-	margin: 0;
-	}
-.leaflet-control-attribution,
-.leaflet-control-scale-line {
-	padding: 0 5px;
-	color: #333;
-	line-height: 1.4;
-	}
-.leaflet-control-attribution a {
-	text-decoration: none;
-	}
-.leaflet-control-attribution a:hover,
-.leaflet-control-attribution a:focus {
-	text-decoration: underline;
-	}
-.leaflet-attribution-flag {
-	display: inline !important;
-	vertical-align: baseline !important;
-	width: 1em;
-	height: 0.6669em;
-	}
-.leaflet-left .leaflet-control-scale {
-	margin-left: 5px;
-	}
-.leaflet-bottom .leaflet-control-scale {
-	margin-bottom: 5px;
-	}
-.leaflet-control-scale-line {
-	border: 2px solid #777;
-	border-top: none;
-	line-height: 1.1;
-	padding: 2px 5px 1px;
-	white-space: nowrap;
-	-moz-box-sizing: border-box;
-	     box-sizing: border-box;
-	background: rgba(255, 255, 255, 0.8);
-	text-shadow: 1px 1px #fff;
-	}
-.leaflet-control-scale-line:not(:first-child) {
-	border-top: 2px solid #777;
-	border-bottom: none;
-	margin-top: -2px;
-	}
-.leaflet-control-scale-line:not(:first-child):not(:last-child) {
-	border-bottom: 2px solid #777;
-	}
-
-.leaflet-touch .leaflet-control-attribution,
-.leaflet-touch .leaflet-control-layers,
-.leaflet-touch .leaflet-bar {
-	box-shadow: none;
-	}
-.leaflet-touch .leaflet-control-layers,
-.leaflet-touch .leaflet-bar {
-	border: 2px solid rgba(0,0,0,0.2);
-	background-clip: padding-box;
-	}
-
-
-/* popup */
-
-.leaflet-popup {
-	position: absolute;
-	text-align: center;
-	margin-bottom: 20px;
-	}
-.leaflet-popup-content-wrapper {
-	padding: 1px;
-	text-align: left;
-	border-radius: 12px;
-	}
-.leaflet-popup-content {
-	margin: 13px 24px 13px 20px;
-	line-height: 1.3;
-	font-size: 13px;
-	font-size: 1.08333em;
-	min-height: 1px;
-	}
-.leaflet-popup-content p {
-	margin: 17px 0;
-	margin: 1.3em 0;
-	}
-.leaflet-popup-tip-container {
-	width: 40px;
-	height: 20px;
-	position: absolute;
-	left: 50%;
-	margin-top: -1px;
-	margin-left: -20px;
-	overflow: hidden;
-	pointer-events: none;
-	}
-.leaflet-popup-tip {
-	width: 17px;
-	height: 17px;
-	padding: 1px;
-
-	margin: -10px auto 0;
-	pointer-events: auto;
-
-	-webkit-transform: rotate(45deg);
-	   -moz-transform: rotate(45deg);
-	    -ms-transform: rotate(45deg);
-	        transform: rotate(45deg);
-	}
-.leaflet-popup-content-wrapper,
-.leaflet-popup-tip {
-	background: white;
-	color: #333;
-	box-shadow: 0 3px 14px rgba(0,0,0,0.4);
-	}
-.leaflet-container a.leaflet-popup-close-button {
-	position: absolute;
-	top: 0;
-	right: 0;
-	border: none;
-	text-align: center;
-	width: 24px;
-	height: 24px;
-	font: 16px/24px Tahoma, Verdana, sans-serif;
-	color: #757575;
-	text-decoration: none;
-	background: transparent;
-	}
-.leaflet-container a.leaflet-popup-close-button:hover,
-.leaflet-container a.leaflet-popup-close-button:focus {
-	color: #585858;
-	}
-.leaflet-popup-scrolled {
-	overflow: auto;
-	}
-
-.leaflet-oldie .leaflet-popup-content-wrapper {
-	-ms-zoom: 1;
-	}
-.leaflet-oldie .leaflet-popup-tip {
-	width: 24px;
-	margin: 0 auto;
-
-	-ms-filter: "progid:DXImageTransform.Microsoft.Matrix(M11=0.70710678, M12=0.70710678, M21=-0.70710678, M22=0.70710678)";
-	filter: progid:DXImageTransform.Microsoft.Matrix(M11=0.70710678, M12=0.70710678, M21=-0.70710678, M22=0.70710678);
-	}
-
-.leaflet-oldie .leaflet-control-zoom,
-.leaflet-oldie .leaflet-control-layers,
-.leaflet-oldie .leaflet-popup-content-wrapper,
-.leaflet-oldie .leaflet-popup-tip {
-	border: 1px solid #999;
-	}
-
-
-/* div icon */
-
-.leaflet-div-icon {
-	background: #fff;
-	border: 1px solid #666;
-	}
-
-
-/* Tooltip */
-/* Base styles for the element that has a tooltip */
-.leaflet-tooltip {
-	position: absolute;
-	padding: 6px;
-	background-color: #fff;
-	border: 1px solid #fff;
-	border-radius: 3px;
-	color: #222;
-	white-space: nowrap;
-	-webkit-user-select: none;
-	-moz-user-select: none;
-	-ms-user-select: none;
-	user-select: none;
-	pointer-events: none;
-	box-shadow: 0 1px 3px rgba(0,0,0,0.4);
-	}
-.leaflet-tooltip.leaflet-interactive {
-	cursor: pointer;
-	pointer-events: auto;
-	}
-.leaflet-tooltip-top:before,
-.leaflet-tooltip-bottom:before,
-.leaflet-tooltip-left:before,
-.leaflet-tooltip-right:before {
-	position: absolute;
-	pointer-events: none;
-	border: 6px solid transparent;
-	background: transparent;
-	content: "";
-	}
-
-/* Directions */
-
-.leaflet-tooltip-bottom {
-	margin-top: 6px;
-}
-.leaflet-tooltip-top {
-	margin-top: -6px;
-}
-.leaflet-tooltip-bottom:before,
-.leaflet-tooltip-top:before {
-	left: 50%;
-	margin-left: -6px;
-	}
-.leaflet-tooltip-top:before {
-	bottom: 0;
-	margin-bottom: -12px;
-	border-top-color: #fff;
-	}
-.leaflet-tooltip-bottom:before {
-	top: 0;
-	margin-top: -12px;
-	margin-left: -6px;
-	border-bottom-color: #fff;
-	}
-.leaflet-tooltip-left {
-	margin-left: -6px;
-}
-.leaflet-tooltip-right {
-	margin-left: 6px;
-}
-.leaflet-tooltip-left:before,
-.leaflet-tooltip-right:before {
-	top: 50%;
-	margin-top: -6px;
-	}
-.leaflet-tooltip-left:before {
-	right: 0;
-	margin-right: -12px;
-	border-left-color: #fff;
-	}
-.leaflet-tooltip-right:before {
-	left: 0;
-	margin-left: -12px;
-	border-right-color: #fff;
-	}
-
-/* Printing */
-
-@media print {
-	/* Prevent printers from removing background-images of controls. */
-	.leaflet-control {
-		-webkit-print-color-adjust: exact;
-		print-color-adjust: exact;
-		}
-	}
@@ -1,11 +0,0 @@
-from django.urls import path
-from . import views
-
-app_name = 'simple_analysis'
-urlpatterns = [
-    path('', views.index, name='index'),
-    path('upload/', views.upload_csv, name='upload'),
-    path('filter/', views.filter_data, name='filter'),
-    path('cluster/', views.run_clustering, name='cluster'),
-    path('cluster/<int:label>/', views.cluster_detail, name='cluster_detail'),
-]
@@ -1,796 +0,0 @@
-"""简单分析模块 — 独立的上传→筛选→聚类→地图可视化工作流。
-
-不与天璇原有分析模块共享数据库或 session store，所有中间数据
-存储在模块级内存缓存（SimpleAnalysisCache）或临时目录中。
-"""
-
-import json
-import logging
-import os
-import re
-import shutil
-import tempfile
-import uuid
-from pathlib import Path
-from collections import Counter
-
-import numpy as np
-import polars as pl
-from django.conf import settings
-from django.http import JsonResponse
-from django.shortcuts import render
-from django.views.decorators.csrf import csrf_exempt
-from django.views.decorators.http import require_POST, require_GET
-from sklearn.cluster import HDBSCAN
-from sklearn.preprocessing import StandardScaler
-from sklearn.cluster import DBSCAN
-
-logger = logging.getLogger(__name__)
-
-# ── In-memory cache ──────────────────────────────────────────────────────────
-# Thread-safe for single-user desktop usage; each user session gets a unique key.
-_cache: dict[str, dict] = {}
-_TEMP_DIR = Path(settings.FILE_UPLOAD_TEMP_DIR) / 'simple_analysis'
-_TEMP_DIR.mkdir(parents=True, exist_ok=True)
-
-
-def _new_id() -> str:
-    return uuid.uuid4().hex[:12]
-
-
-def _cached(key: str) -> dict | None:
-    return _cache.get(key)
-
-
-def _store(key: str, data: dict) -> None:
-    _cache[key] = data
-
-
-# ── Column-name helpers ──────────────────────────────────────────────────────
-
-def _norm(name: str) -> str:
-    """Normalise column name: lower-case, strip whitespace, replace hyphens."""
-    return name.strip().lower().replace('-', '_').replace(' ', '_')
-
-
-def _find_col(col_names: list[str], *patterns: str) -> str | None:
-    """Find first column matching any pattern (exact or suffix after stripping)."""
-    normed = {c: _norm(c) for c in col_names}
-    for p in patterns:
-        np_norm = _norm(p)
-        for orig, n in normed.items():
-            if n == np_norm or n.endswith('_' + np_norm) or n.startswith(np_norm + '_'):
-                return orig
-    return None
-
-
-# ── Step 1: Upload ───────────────────────────────────────────────────────────
-
-@csrf_exempt
-@require_POST
-def upload_csv(request):
-    """Upload one or more CSV files, merge, return column info + cnam frequency."""
-    files = request.FILES.getlist('files')
-    if not files:
-        return JsonResponse({'error': '未上传任何文件'}, status=400)
-
-    session_id = _new_id()
-    upload_dir = _TEMP_DIR / session_id
-    upload_dir.mkdir(parents=True, exist_ok=True)
-
-    saved_paths = []
-    for f in files:
-        dest = upload_dir / f.name
-        with open(dest, 'wb') as out:
-            for chunk in f.chunks():
-                out.write(chunk)
-        saved_paths.append(str(dest))
-
-    # Load & merge CSV files using Polars
-    try:
-        lf = pl.scan_csv(
-            saved_paths[0] if len(saved_paths) == 1 else str(upload_dir / '*.csv'),
-            infer_schema_length=10000,
-            try_parse_dates=True,
-        )
-        if len(saved_paths) > 1:
-            frames = [pl.scan_csv(p, infer_schema_length=10000) for p in saved_paths]
-            lf = pl.concat(frames, how='diagonal_relaxed')
-
-        df = lf.collect(streaming=True)
-    except Exception as e:
-        shutil.rmtree(upload_dir, ignore_errors=True)
-        return JsonResponse({'error': f'CSV 解析失败: {e}'}, status=400)
-
-    col_names = df.columns
-    total_rows = len(df)
-
-    # Detect key columns — expanded patterns to match actual CSV column names
-    cnam_col = _find_col(col_names, 'cnam', 'snam', 'class_name', 'category', 'target', 'name')
-    cnrs_col = _find_col(col_names, 'cnrs', 'conn_success', 'result', '4srs')
-    isrs_col = _find_col(col_names, 'isrs', 'is_resolved', 'resolved')
-    src_lat_col = _find_col(col_names, ':ips.latd', 'src_latitude', 'src_lat', 'source_lat')
-    src_lon_col = _find_col(col_names, ':ips.lond', 'src_longitude', 'src_lon', 'source_lon')
-    dst_lat_col = _find_col(col_names, ':ipd.latd', 'dst_latitude', 'dst_lat', 'dest_lat')
-    dst_lon_col = _find_col(col_names, ':ipd.lond', 'dst_longitude', 'dst_lon', 'dest_lon')
-    ips_col = _find_col(col_names, ':ips', 'src_ip', 'source_ip', 'ip_src')
-    ipd_col = _find_col(col_names, ':ipd', 'dst_ip', 'dest_ip', 'destination_ip', 'ip_dst')
-
-    # CNAM frequency
-    cnam_freq = []
-    top_cnam = None
-    if cnam_col:
-        try:
-            cnam_series = df[cnam_col].cast(pl.Utf8).fill_null('')
-            counts = Counter(cnam_series.to_list())
-            total_cnam = sum(counts.values())
-            for val, cnt in counts.most_common():
-                cnam_freq.append({
-                    'value': val if val else '(空)',
-                    'count': cnt,
-                    'pct': round(cnt / total_cnam * 100, 1),
-                })
-            if cnam_freq:
-                top_cnam = cnam_freq[0]
-        except Exception as e:
-            logger.warning('cnam frequency failed: %s', e)
-
-    # Store metadata in cache
-    meta = {
-        'session_id': session_id,
-        'upload_dir': str(upload_dir),
-        'total_rows': total_rows,
-        'col_names': col_names,
-        'cnam_col': cnam_col,
-        'cnrs_col': cnrs_col,
-        'isrs_col': isrs_col,
-        'src_lat_col': src_lat_col,
-        'src_lon_col': src_lon_col,
-        'dst_lat_col': dst_lat_col,
-        'dst_lon_col': dst_lon_col,
-        'ips_col': ips_col,
-        'ipd_col': ipd_col,
-    }
-    _store(session_id, {'meta': meta, 'df': df})
-
-    return JsonResponse({
-        'session_id': session_id,
-        'total_rows': total_rows,
-        'columns': col_names[:50],
-        'detected_columns': {
-            'cnam': cnam_col,
-            'cnrs': cnrs_col,
-            'isrs': isrs_col,
-            'src_lat': src_lat_col,
-            'src_lon': src_lon_col,
-            'dst_lat': dst_lat_col,
-            'dst_lon': dst_lon_col,
-            'ips': ips_col,
-            'ipd': ipd_col,
-        },
-        'cnam_frequency': cnam_freq[:30],
-        'top_cnam': top_cnam,
-    })
-
-
-# ── Step 2: Filter ───────────────────────────────────────────────────────────
-
-@csrf_exempt
-@require_POST
-def filter_data(request):
-    """Apply cnam + cnrs + isrs filters; return preview rows."""
-    body = json.loads(request.body)
-    session_id = body.get('session_id')
-    selected_cnam = body.get('selected_cnam', '')
-
-    entry = _cached(session_id)
-    if entry is None:
-        return JsonResponse({'error': '会话已过期，请重新上传'}, status=400)
-
-    df: pl.DataFrame = entry['df']
-    meta = entry['meta']
-    cnam_col = meta['cnam_col']
-    cnrs_col = meta['cnrs_col']
-    isrs_col = meta['isrs_col']
-
-    # Validate required columns
-    missing = []
-    if not cnam_col:
-        missing.append('cnam/snam (目标类别)')
-
-    if missing:
-        return JsonResponse({
-            'error': f'缺少关键列: {", ".join(missing)}。请检查CSV文件格式。',
-            'missing_columns': missing,
-        }, status=400)
-
-    # Apply filter — cnrs and isrs are optional; only apply if column exists
-    try:
-        cond = (pl.col(cnam_col).cast(pl.Utf8) == selected_cnam)
-        if cnrs_col:
-            cond = cond & (pl.col(cnrs_col).cast(pl.Utf8).str.strip_chars() == '+')
-        if isrs_col:
-            cond = cond & (pl.col(isrs_col).cast(pl.Utf8).str.strip_chars() == '+')
-        filtered = df.filter(cond)
-    except Exception as e:
-        return JsonResponse({'error': f'筛选失败: {e}'}, status=400)
-
-    filtered_rows = len(filtered)
-    meta['filtered_cnam'] = selected_cnam
-    meta['filtered_rows'] = filtered_rows
-    entry['filtered_df'] = filtered
-
-    # Preview data (first 20 rows)
-    preview_cols = [c for c in filtered.columns if not c.startswith('_')][:30]
-    preview_rows = []
-    for row in filtered.head(20).select(preview_cols).iter_rows(named=True):
-        clean = {}
-        for k, v in row.items():
-            if v is None or (isinstance(v, float) and np.isnan(v)):
-                clean[k] = None
-            else:
-                clean[k] = str(v)[:120]
-        preview_rows.append(clean)
-
-    return JsonResponse({
-        'filtered_rows': filtered_rows,
-        'preview_columns': preview_cols,
-        'preview_rows': preview_rows,
-        'total_before_filter': len(df),
-    })
-
-
-# ── Step 3: Cluster ──────────────────────────────────────────────────────────
-
-@csrf_exempt
-@require_POST
-def run_clustering(request):
-    """Run HDBSCAN geo-clustering or IP-subnet soft-clustering."""
-    body = json.loads(request.body)
-    session_id = body.get('session_id')
-    mode = body.get('mode', 'geo')  # 'geo' or 'subnet'
-    min_cluster_size = int(body.get('min_cluster_size', 5))
-    cluster_selection_epsilon = float(body.get('cluster_selection_epsilon', 0.005))
-    min_records_per_subnet = int(body.get('min_records_per_subnet', 3))
-    max_cluster_size = int(body.get('max_cluster_size', 100))
-
-    entry = _cached(session_id)
-    if entry is None:
-        return JsonResponse({'error': '会话已过期，请重新上传'}, status=400)
-
-    filtered = entry.get('filtered_df')
-    if filtered is None:
-        return JsonResponse({'error': '请先执行筛选（步骤二）'}, status=400)
-
-    meta = entry['meta']
-    src_lat_col = meta['src_lat_col']
-    src_lon_col = meta['src_lon_col']
-    ips_col = meta['ips_col']
-
-    if mode == 'geo':
-        result = _cluster_geo(entry, filtered, meta, min_cluster_size, cluster_selection_epsilon)
-    else:
-        result = _cluster_subnet(entry, filtered, meta, min_records_per_subnet, max_cluster_size)
-    # Update cache with any modifications made by cluster functions
-    _store(session_id, entry)
-    return result
-
-
-def _cluster_geo(entry: dict, df: pl.DataFrame, meta: dict,
-                 min_cluster_size: int, cluster_selection_epsilon: float = 0.005):
-    """HDBSCAN clustering on source IP latitude/longitude using Haversine distance."""
-    src_lat_col = meta['src_lat_col']
-    src_lon_col = meta['src_lon_col']
-    ips_col = meta['ips_col']
-
-    if not src_lat_col or not src_lon_col:
-        return JsonResponse({'error': '缺少源经纬度列（:ips.latd / :ips.lond），无法执行地理聚类'}, status=400)
-
-    # Drop rows with missing lat/lon
-    clean = df.filter(
-        pl.col(src_lat_col).is_not_null()
-        & pl.col(src_lon_col).is_not_null()
-    )
-
-    if len(clean) == 0:
-        return JsonResponse({'error': '所有记录均缺失经纬度，无法聚类'}, status=400)
-
-    # Clean lat/lon: cast to string, filter out '+'/'-' artifacts
-    try:
-        lat_raw = clean[src_lat_col].cast(pl.Utf8)
-        lon_raw = clean[src_lon_col].cast(pl.Utf8)
-        # Build validity mask: exclude '+', '-', empty, and None
-        valid_mask = (
-            lat_raw.is_not_null() & lon_raw.is_not_null()
-            & (lat_raw != '+') & (lat_raw != '-')
-            & (lon_raw != '+') & (lon_raw != '-')
-            & (lat_raw != '') & (lon_raw != '')
-        )
-        clean = clean.filter(valid_mask)
-        if len(clean) == 0:
-            return JsonResponse({'error': '无有效经纬度记录（排除符号值后）'}, status=400)
-        lat_vals = clean[src_lat_col].cast(pl.Float64, strict=False).to_numpy()
-        lon_vals = clean[src_lon_col].cast(pl.Float64, strict=False).to_numpy()
-        # Final NaN check
-        lat_vals = np.where(np.isnan(lat_vals), 0.0, lat_vals)
-        lon_vals = np.where(np.isnan(lon_vals), 0.0, lon_vals)
-    except Exception as e:
-        return JsonResponse({'error': f'经纬度列转换失败: {e}'}, status=400)
-
-    # Filter to valid lat/lon ranges
-    valid_range_mask = (
-        (lat_vals >= -90) & (lat_vals <= 90)
-        & (lon_vals >= -180) & (lon_vals <= 180)
-        & ~np.isnan(lat_vals) & ~np.isnan(lon_vals)
-    )
-    lat_vals = lat_vals[valid_range_mask]
-    lon_vals = lon_vals[valid_range_mask]
-    clean = clean.filter(
-        pl.Series(valid_range_mask)
-    )
-
-    if len(clean) < min_cluster_size:
-        return JsonResponse({
-            'error': f'有效经纬度记录数（{len(clean)}）少于最小聚类规模（{min_cluster_size}）',
-        }, status=400)
-
-    # ── HDBSCAN with Haversine distance ────────────────────────────────────
-    # Convert degrees to radians for haversine metric
-    lat_rad = np.radians(lat_vals)
-    lon_rad = np.radians(lon_vals)
-    coords = np.column_stack([lat_rad, lon_rad])
-
-    clusterer = HDBSCAN(
-        min_cluster_size=min_cluster_size,
-        min_samples=min(3, min_cluster_size),
-        metric='haversine',
-        cluster_selection_epsilon=cluster_selection_epsilon,
-    )
-    labels = clusterer.fit_predict(coords)
-
-    n_clusters = int(len(set(labels)) - (1 if -1 in labels else 0))
-    n_noise = int((labels == -1).sum())
-    noise_ratio = n_noise / len(labels) if len(labels) > 0 else 0
-
-    # Check for degenerate clustering (fallback suggestion)
-    fallback_suggested = False
-    if noise_ratio > 0.8 or n_clusters <= 1:
-        fallback_suggested = True
-
-    # Build cluster overview
-    cluster_map = {}
-    unique_labels = sorted(set(labels))
-    data_points_by_cluster = {}
-    for lbl in unique_labels:
-        mask = labels == lbl
-        idxs = np.where(mask)[0]
-        cluster_lats = lat_vals[idxs]
-        cluster_lons = lon_vals[idxs]
-        center_lat = float(np.median(cluster_lats))
-        center_lon = float(np.median(cluster_lons))
-
-        # Unique IPs
-        if ips_col:
-            ips_in_cluster = clean.filter(pl.Series(mask))[ips_col].cast(pl.Utf8).unique().to_list()
-            n_unique_ips = len(ips_in_cluster)
-        else:
-            ips_in_cluster = []
-            n_unique_ips = 0
-
-        cluster_map[int(lbl)] = {
-            'label': int(lbl),
-            'size': int(mask.sum()),
-            'center_lat': round(float(center_lat), 4),
-            'center_lon': round(float(center_lon), 4),
-            'n_unique_ips': int(n_unique_ips),
-            'is_noise': bool(lbl == -1),
-        }
-
-        # Collect individual data points for map rendering (lat, lon, index)
-        pts = []
-        for pt_idx, i in enumerate(idxs):
-            pts.append({
-                'lat': round(float(cluster_lats[pt_idx]), 4),
-                'lon': round(float(cluster_lons[pt_idx]), 4),
-                'idx': int(i),
-            })
-        data_points_by_cluster[int(lbl)] = pts
-
-    # Annotate original filtered DataFrame with cluster labels
-    label_series = np.full(len(df), -1, dtype=int)
-    # Map back: only the valid rows got labels
-    valid_indices = np.where(valid_range_mask)[0]
-    for i, idx in enumerate(valid_indices):
-        label_series[idx] = int(labels[i])
-
-    entry['cluster_labels'] = label_series.tolist()
-    entry['cluster_map'] = cluster_map
-    entry['cluster_mode'] = 'geo'
-    entry['n_clusters'] = n_clusters
-    entry['n_noise'] = n_noise
-
-    # Update filtered_df with cluster label column
-    df_with_labels = df.with_columns(pl.Series('_cluster_label', label_series))
-    entry['filtered_df'] = df_with_labels
-
-    return JsonResponse({
-        'n_clusters': n_clusters,
-        'n_noise': n_noise,
-        'n_valid_points': len(clean),
-        'clusters': cluster_map,
-        'labels': label_series.tolist(),
-        'data_points': data_points_by_cluster,
-        'fallback_suggested': fallback_suggested,
-    })
-
-
-def _cluster_subnet(entry: dict, df: pl.DataFrame, meta: dict,
-                     min_records_per_subnet: int = 3, max_cluster_size: int = 100):
-    """Soft clustering by /24 subnet prefix.
-
-    Subnets with fewer than *min_records_per_subnet* records are marked noise (-1).
-    Subnets with more than *max_cluster_size* records are auto-split using geo
-    sub-clustering (HDBSCAN + haversine) on the available lat/lon within the subnet.
-    """
-    ips_col = meta['ips_col']
-    src_lat_col = meta['src_lat_col']
-    src_lon_col = meta['src_lon_col']
-    if not ips_col:
-        return JsonResponse({'error': '缺少源IP列（:ips），无法执行子网聚类'}, status=400)
-
-    try:
-        ip_series = df[ips_col].cast(pl.Utf8)
-    except Exception as e:
-        return JsonResponse({'error': f'IP列转换失败: {e}'}, status=400)
-
-    # Extract /24 subnet
-    def _subnet24(ip: str) -> str | None:
-        if not ip or ip == 'None' or ip == '':
-            return None
-        parts = ip.split('.')
-        if len(parts) < 4:
-            return None
-        return '.'.join(parts[:3]) + '.0/24'
-
-    subnets = [None if v is None else _subnet24(str(v)) for v in ip_series.to_list()]
-    subnet_count = Counter(s for s in subnets if s is not None)
-
-    # Assign cluster labels based on subnet frequency
-    labels = []
-    cluster_map: dict[int, dict] = {}
-    next_label = 0
-    subnet_to_label: dict[str, int] = {}
-    # Track which records belong to large subnets for geo splitting
-    large_subnet_indices: dict[str, list[int]] = {}
-
-    for row_idx, s in enumerate(subnets):
-        if s is None:
-            labels.append(-1)
-        else:
-            cnt = subnet_count.get(s, 0)
-            if cnt < min_records_per_subnet:
-                labels.append(-1)  # noise (too few records)
-            elif cnt > max_cluster_size:
-                # Large subnet → will apply geo sub-clustering later
-                if s not in subnet_to_label:
-                    subnet_to_label[s] = next_label
-                    cluster_map[next_label] = {
-                        'label': next_label,
-                        'size': 0,
-                        'subnet': s,
-                        'is_noise': False,
-                        '_is_large': True,
-                        '_large_subnet_name': s,
-                    }
-                    next_label += 1
-                    large_subnet_indices[s] = []
-                lbl = subnet_to_label[s]
-                labels.append(lbl)
-                cluster_map[lbl]['size'] = cluster_map[lbl].get('size', 0) + 1
-                large_subnet_indices[s].append(row_idx)
-            else:
-                if s not in subnet_to_label:
-                    subnet_to_label[s] = next_label
-                    cluster_map[next_label] = {
-                        'label': next_label,
-                        'size': 0,
-                        'subnet': s,
-                        'is_noise': False,
-                    }
-                    next_label += 1
-                lbl = subnet_to_label[s]
-                labels.append(lbl)
-                cluster_map[lbl]['size'] = cluster_map[lbl].get('size', 0) + 1
-
-    # ── Geo sub-clustering for large subnets ────────────────────────────────
-    if large_subnet_indices and src_lat_col and src_lon_col:
-        for subnet_name, indices in large_subnet_indices.items():
-            old_label = subnet_to_label[subnet_name]
-            # Remove the temporary large-subnet cluster entry
-            cluster_map.pop(old_label, None)
-
-            # Get lat/lon for records in this subnet
-            subnet_df = df[pl.Series(indices)]
-            try:
-                sub_lats = subnet_df[src_lat_col].cast(pl.Float64).to_numpy()
-                sub_lons = subnet_df[src_lon_col].cast(pl.Float64).to_numpy()
-                valid = (
-                    ~np.isnan(sub_lats) & ~np.isnan(sub_lons)
-                    & (sub_lats >= -90) & (sub_lats <= 90)
-                    & (sub_lons >= -180) & (sub_lons <= 180)
-                )
-                sub_lats = sub_lats[valid]
-                sub_lons = sub_lons[valid]
-                if len(sub_lats) >= 3:
-                    # Run HDBSCAN with haversine on this subnet's points
-                    coords = np.column_stack([np.radians(sub_lats), np.radians(sub_lons)])
-                    sub_cluster = HDBSCAN(
-                        min_cluster_size=min(3, len(sub_lats) // 5),
-                        min_samples=2,
-                        metric='haversine',
-                        cluster_selection_epsilon=0.005,
-                    )
-                    sub_labels = sub_cluster.fit_predict(coords)
-
-                    # Create sub-clusters
-                    sub_label_map: dict[int, int] = {}
-                    valid_idx_iter = 0
-                    for orig_rank, orig_idx in enumerate(indices):
-                        if not valid[orig_rank - valid_idx_iter]:
-                            # Was invalid, check if we skipped
-                            pass
-
-                    # Re-map: skip invalid points (mark noise)
-                    valid_counter = 0
-                    for orig_rank, orig_idx in enumerate(indices):
-                        if not valid[orig_rank]:
-                            labels[orig_idx] = -1  # no lat/lon → noise
-                        else:
-                            sl = int(sub_labels[valid_counter])
-                            valid_counter += 1
-                            if sl == -1:
-                                labels[orig_idx] = -1
-                            else:
-                                sub_cluster_name = f'{subnet_name}/sub{sl}'
-                                if sub_cluster_name not in subnet_to_label:
-                                    subnet_to_label[sub_cluster_name] = next_label
-                                    cluster_map[next_label] = {
-                                        'label': next_label,
-                                        'size': 0,
-                                        'subnet': sub_cluster_name,
-                                        'is_noise': False,
-                                    }
-                                    next_label += 1
-                                new_lbl = subnet_to_label[sub_cluster_name]
-                                labels[orig_idx] = new_lbl
-                                cluster_map[new_lbl]['size'] = cluster_map[new_lbl].get('size', 0) + 1
-                else:
-                    # Not enough geo data → keep as single cluster
-                    cluster_map[old_label] = {
-                        'label': old_label,
-                        'size': len(indices),
-                        'subnet': subnet_name,
-                        'is_noise': False,
-                    }
-                    subnet_to_label[subnet_name] = old_label
-            except Exception as e:
-                logger.warning('Subnet geo-splitting failed for %s: %s', subnet_name, e)
-                # Keep the original cluster
-                cluster_map[old_label] = {
-                    'label': old_label,
-                    'size': len(indices),
-                    'subnet': subnet_name,
-                    'is_noise': False,
-                }
-                subnet_to_label[subnet_name] = old_label
-
-    # Update cluster maps with centers and unique IP counts
-    for lbl in list(cluster_map.keys()):
-        mask = np.array(labels) == lbl
-        ips_in_cluster = df.filter(pl.Series(mask))[ips_col].cast(pl.Utf8).unique().to_list()
-        cluster_map[lbl]['n_unique_ips'] = len(ips_in_cluster)
-        # Try to get geo center if lat/lon available
-        if src_lat_col and src_lon_col:
-            try:
-                clat = df.filter(pl.Series(mask))[src_lat_col].cast(pl.Float64).drop_nulls()
-                clon = df.filter(pl.Series(mask))[src_lon_col].cast(pl.Float64).drop_nulls()
-                if len(clat) > 0 and len(clon) > 0:
-                    cluster_map[lbl]['center_lat'] = round(float(clat.median()), 4)
-                    cluster_map[lbl]['center_lon'] = round(float(clon.median()), 4)
-            except Exception:
-                cluster_map[lbl]['center_lat'] = None
-                cluster_map[lbl]['center_lon'] = None
-        else:
-            cluster_map[lbl]['center_lat'] = None
-            cluster_map[lbl]['center_lon'] = None
-
-    n_noise = labels.count(-1)
-    n_clusters = len(cluster_map)
-
-    # Build data_points for map
-    data_points_by_cluster: dict[int, list] = {}
-    for lbl in list(cluster_map.keys()):
-        mask = np.array(labels) == lbl
-        pts = []
-        if src_lat_col and src_lon_col:
-            try:
-                sub = df.filter(pl.Series(mask))
-                lats = sub[src_lat_col].cast(pl.Float64).to_numpy()
-                lons = sub[src_lon_col].cast(pl.Float64).to_numpy()
-                for i in range(len(sub)):
-                    if not np.isnan(lats[i]) and not np.isnan(lons[i]):
-                        pts.append({'lat': round(float(lats[i]), 4),
-                                    'lon': round(float(lons[i]), 4),
-                                    'idx': int(i)})
-            except Exception:
-                pass
-        data_points_by_cluster[lbl] = pts
-
-    # Also collect noisy points' coordinates
-    noise_mask = np.array(labels) == -1
-    noise_points = []
-    if src_lat_col and src_lon_col:
-        try:
-            noise_sub = df.filter(pl.Series(noise_mask))
-            nlats = noise_sub[src_lat_col].cast(pl.Float64).to_numpy()
-            nlons = noise_sub[src_lon_col].cast(pl.Float64).to_numpy()
-            for i in range(len(noise_sub)):
-                if not np.isnan(nlats[i]) and not np.isnan(nlons[i]):
-                    noise_points.append({'lat': round(float(nlats[i]), 4),
-                                         'lon': round(float(nlons[i]), 4),
-                                         'idx': int(i)})
-        except Exception:
-            pass
-    if noise_points:
-        data_points_by_cluster[-1] = noise_points
-
-    # Annotate DataFrame
-    df_with_labels = df.with_columns(pl.Series('_cluster_label', labels))
-    entry['cluster_labels'] = labels
-    entry['cluster_map'] = cluster_map
-    entry['cluster_mode'] = 'subnet'
-    entry['filtered_df'] = df_with_labels
-    entry['n_clusters'] = n_clusters
-    entry['n_noise'] = n_noise
-
-    return JsonResponse({
-        'n_clusters': n_clusters,
-        'n_noise': n_noise,
-        'n_valid_points': len(df),
-        'clusters': cluster_map,
-        'labels': labels,
-        'data_points': data_points_by_cluster,
-    })
-
-
-# ── Step 4: Cluster detail ───────────────────────────────────────────────────
-
-@require_GET
-def cluster_detail(request, label: int):
-    """Return detailed profile for a specific cluster."""
-    session_id = request.GET.get('session_id')
-    if not session_id:
-        return JsonResponse({'error': '缺少 session_id'}, status=400)
-
-    entry = _cached(session_id)
-    if entry is None:
-        return JsonResponse({'error': '会话已过期'}, status=400)
-
-    df = entry.get('filtered_df')
-    meta = entry.get('meta', {})
-    cluster_map = entry.get('cluster_map', {})
-
-    if df is None:
-        return JsonResponse({'error': '数据未就绪'}, status=400)
-
-    if '_cluster_label' not in df.columns:
-        return JsonResponse({'error': '未执行聚类'}, status=400)
-
-    cluster_df = df.filter(pl.col('_cluster_label') == label)
-    rows = len(cluster_df)
-    if rows == 0:
-        return JsonResponse({'error': f'簇 {label} 无数据'}, status=404)
-
-    cluster_info = cluster_map.get(label, {})
-
-    # Helper: get unique values for a column
-    def _top_values(col_name: str, top_n: int = 5) -> list:
-        if col_name not in df.columns:
-            return []
-        try:
-            vals = (
-                cluster_df[col_name]
-                .cast(pl.Utf8)
-                .fill_null('(空)')
-                .value_counts()
-                .sort('count', descending=True)
-                .head(top_n)
-            )
-            total = vals['count'].sum()
-            return [
-                {'value': v['counts'] if isinstance(v, dict) else str(row[0]),
-                 'count': int(row[1]),
-                 'pct': round(int(row[1]) / total * 100, 1)}
-                for row in vals.iter_rows()
-            ]
-        except Exception:
-            return []
-
-    # Source IPs
-    ips_col = meta.get('ips_col')
-    unique_ips = []
-    if ips_col and ips_col in cluster_df.columns:
-        try:
-            ip_vals = cluster_df[ips_col].cast(pl.Utf8).unique().to_list()
-            unique_ips = sorted(str(v) for v in ip_vals if v not in (None, '', 'None'))
-        except Exception:
-            pass
-
-    # Detect useful columns for profiling
-    col_names = df.columns
-    dst_ip_col = _find_col(col_names, ':ipd', 'dst_ip', 'dest_ip', 'destination_ip')
-    dst_port_col = _find_col(col_names, 'dst_port', 'dest_port', 'port', 'dport')
-    tls_ver_col = _find_col(col_names, '0ver', 'tls_version', 'version', 'tlsver')
-    cipher_col = _find_col(col_names, '0cph', 'cipher_suite', 'cipher', 'tls_cipher')
-    sni_col = _find_col(col_names, 'sni', 'server_name', 'tls_sni')
-    bytes_col = _find_col(col_names, '8byt', 'bytes_sent', 'bytes', 'src_bytes')
-    dur_col = _find_col(col_names, '4dur', 'duration', 'dur', 'flow_duration')
-    proto_col = _find_col(col_names, 'proto', 'protocol', 'l4_proto')
-
-    profile = {
-        'label': label,
-        'size': rows,
-        'cluster_info': cluster_info,
-        'unique_ips': unique_ips[:100],
-        'unique_ips_count': len(unique_ips),
-        'top_dst_ips': _top_values(dst_ip_col) if dst_ip_col else [],
-        'top_dst_ports': _top_values(dst_port_col) if dst_port_col else [],
-        'top_tls_versions': _top_values(tls_ver_col) if tls_ver_col else [],
-        'top_ciphers': _top_values(cipher_col) if cipher_col else [],
-        'top_sni': _top_values(sni_col) if sni_col else [],
-        'top_protocols': _top_values(proto_col) if proto_col else [],
-        # Traffic stats
-        'traffic': {},
-    }
-
-    # Traffic stats
-    if bytes_col and bytes_col in cluster_df.columns:
-        try:
-            total_bytes = cluster_df[bytes_col].cast(pl.Float64).sum()
-            profile['traffic']['total_bytes'] = round(total_bytes, 2) if total_bytes else 0
-        except Exception:
-            pass
-    if dur_col and dur_col in cluster_df.columns:
-        try:
-            total_dur = cluster_df[dur_col].cast(pl.Float64).sum()
-            avg_dur = cluster_df[dur_col].cast(pl.Float64).mean()
-            profile['traffic']['total_duration'] = round(total_dur, 2) if total_dur else 0
-            profile['traffic']['avg_duration'] = round(avg_dur, 2) if avg_dur else 0
-        except Exception:
-            pass
-
-    # Geo bounding box (if lat/lon available)
-    src_lat_col = meta.get('src_lat_col')
-    src_lon_col = meta.get('src_lon_col')
-    if src_lat_col and src_lon_col and src_lat_col in cluster_df.columns:
-        try:
-            lats = cluster_df[src_lat_col].cast(pl.Float64).drop_nulls()
-            lons = cluster_df[src_lon_col].cast(pl.Float64).drop_nulls()
-            if len(lats) > 0 and len(lons) > 0:
-                profile['geo_bounds'] = {
-                    'lat_min': round(float(lats.min()), 4),
-                    'lat_max': round(float(lats.max()), 4),
-                    'lon_min': round(float(lons.min()), 4),
-                    'lon_max': round(float(lons.max()), 4),
-                }
-        except Exception:
-            pass
-
-    return JsonResponse(profile)
-
-
-# ── Main page ────────────────────────────────────────────────────────────────
-
-def index(request):
-    """Render the simple analysis workbench page."""
-    return render(request, 'simple_analysis/simple_analysis.html')
@@ -1,62 +1,120 @@
 {% extends 'base.html' %}
-{% block title %}Cluster #{{ cluster.cluster_label }} - Run #{{ run.display_id }}{% endblock %}
+{% load static %}
+{% block title %}Cluster #{{ cluster.cluster_label }} — Run #{{ run.display_id }}{% endblock %}
 {% block content %}
+<style>
+.entity-card { border:1px solid #e0e0e0; border-radius:6px; padding:0.6rem; margin-bottom:0.4rem; background:#fafafa; }
+.entity-card .val { font-weight:600; font-size:0.85rem; }
+.entity-card .feat { font-size:0.75rem; color:#666; display:inline-block; margin-right:0.5rem; }
+.feat-positive { color:#2e7d32; }
+.feat-negative { color:#c62828; }
+.feat-table { width:100%; border-collapse:collapse; font-size:0.8rem; }
+.feat-table th { background:#f5f5f5; padding:0.3rem 0.5rem; text-align:left; font-weight:600; border-bottom:2px solid #ddd; }
+.feat-table td { padding:0.25rem 0.5rem; border-bottom:1px solid #eee; }
+</style>
+
 <div class="card">
-    <a href="{% url 'analysis:cluster_overview' run.display_id %}" style="color:#4361ee;text-decoration:none;font-size:0.9rem;">← Back to overview</a>
-    <h2 style="margin-top:0.5rem;">Cluster #{{ cluster.cluster_label }}</h2>
-    <p>{{ cluster.size }} entities ({{ cluster.proportion|floatformat:2 }}% of total)</p>
-    <p>Silhouette Score: {{ cluster.silhouette_score|floatformat:4|default:"-" }}</p>
+    <a href="{% url 'analysis:cluster_overview' run.display_id %}" style="color:#4361ee;text-decoration:none;font-size:0.9rem;">← Cluster Overview</a>
+    <div style="display:flex;justify-content:space-between;align-items:center;flex-wrap:wrap;gap:0.5rem;margin-top:0.5rem;">
+        <div>
+            <h2 style="margin:0;">Cluster #{{ cluster.cluster_label }}
+                {% if cluster.cluster_label == -1 %}<span class="badge badge-warning">Noise</span>{% endif %}
+            </h2>
+            <p style="margin:0.25rem 0 0;color:#666;font-size:0.9rem;">
+                {{ cluster.size }} entities
+                {% if cluster.proportion %} — {{ cluster.proportion|floatformat:1 }}% of total{% endif %}
+                {% if cluster.silhouette_score is not None %} — Silhouette: {{ cluster.silhouette_score|floatformat:4 }}{% endif %}
+            </p>
+        </div>
+    </div>
+    {% if nl_summary %}
+    <div style="background:#f8f9fa;border-left:3px solid #4361ee;padding:0.6rem;margin:0.75rem 0;border-radius:0 4px 4px 0;font-size:0.85rem;line-height:1.6;color:#333;">{{ nl_summary }}</div>
+    {% endif %}
 </div>

-<div class="grid-2">
-    <div class="card">
-        <h2>Feature Summary (top 50)</h2>
-        <table>
-            <thead>
-                <tr>
-                    <th>Feature</th>
-                    <th>Mean</th>
-                    <th>Std</th>
-                    <th>Median</th>
-                    <th>Dist. Score</th>
-                </tr>
-            </thead>
-            <tbody>
-                {% for f in features %}
-                <tr>
-                    <td><code>{{ f.feature_name }}</code></td>
-                    <td>{{ f.mean|floatformat:3|default:"-" }}</td>
-                    <td>{{ f.std|floatformat:3|default:"-" }}</td>
-                    <td>{{ f.median|floatformat:3|default:"-" }}</td>
-                    <td>{{ f.distinguishing_score|floatformat:3|default:"-" }}</td>
-                </tr>
-                {% empty %}
-                <tr><td colspan="5">No features</td></tr>
-                {% endfor %}
-            </tbody>
-        </table>
-    </div>
+<!-- ── Feature Analysis ── -->
+<div class="card">
+    <h3>📊 Feature Analysis <span style="font-weight:400;font-size:0.8rem;color:#888;">{{ features|length }} features</span></h3>
+    {% if features %}
+    <table class="feat-table">
+        <thead>
+            <tr>
+                <th>#</th>
+                <th>Feature</th>
+                <th>Dist. Score</th>
+                <th>Mean</th>
+                <th>Std</th>
+                <th>Median</th>
+                <th>P25</th>
+                <th>P75</th>
+            </tr>
+        </thead>
+        <tbody>
+            {% for f in features %}
+            <tr>
+                <td>{{ forloop.counter }}</td>
+                <td><code>{{ f.feature_name }}</code></td>
+                <td><span class="{% if f.distinguishing_score > 0 %}feat-positive{% else %}feat-negative{% endif %}">{{ f.distinguishing_score|floatformat:3 }}</span></td>
+                <td>{{ f.mean|floatformat:3|default:"-" }}</td>
+                <td>{{ f.std|floatformat:3|default:"-" }}</td>
+                <td>{{ f.median|floatformat:3|default:"-" }}</td>
+                <td>{{ f.p25|floatformat:3|default:"-" }}</td>
+                <td>{{ f.p75|floatformat:3|default:"-" }}</td>
+            </tr>
+            {% endfor %}
+        </tbody>
+    </table>
+    {% else %}
+    <div class="empty-state"><p>No feature data for this cluster.</p></div>
+    {% endif %}
+</div>

-    <div class="card">
-        <h2>Entities (top 50)</h2>
-        <table>
-            <thead>
-                <tr>
-                    <th>Entity</th>
-                    <th></th>
-                </tr>
-            </thead>
-            <tbody>
-                {% for e in entities %}
-                <tr>
-                    <td><code>{{ e.entity_value }}</code></td>
-                    <td><a href="{% url 'analysis:entity_profile' e.id %}" class="btn btn-primary">Profile</a></td>
-                </tr>
-                {% empty %}
-                <tr><td colspan="2">No entities</td></tr>
-                {% endfor %}
-            </tbody>
-        </table>
+<!-- ── SVD Singular Values ── -->
+{% if cluster.cluster_label != -1 %}
+{# SVD-extracted features (from _run_clustering_pipeline cluster_svd_extract) have higher distinguishing scores #}
+{% with svd_features=features|slice:":5" %}
+{% if svd_features %}
+<div class="card">
+    <h3>🔬 SVD Key Features <span style="font-weight:400;font-size:0.8rem;color:#888;">Top 5 by distinguishing score</span></h3>
+    <div style="display:grid;grid-template-columns:1fr 1fr;gap:0.5rem;">
+    {% for f in svd_features %}
+        <div style="background:#f8f9fa;border-radius:6px;padding:0.5rem;border:1px solid #e0e0e0;">
+            <div style="font-weight:600;font-size:0.8rem;">{{ f.feature_name }}</div>
+            <div style="font-size:0.75rem;color:#666;margin-top:0.2rem;">
+                Score: <span class="{% if f.distinguishing_score > 0 %}feat-positive{% else %}feat-negative{% endif %}">{{ f.distinguishing_score|floatformat:2 }}</span>
+                | Mean: {{ f.mean|floatformat:2 }}
+                | Std: {{ f.std|floatformat:2 }}
+            </div>
+        </div>
+    {% endfor %}
    </div>
 </div>
-{% endblock %}
+{% endif %}
+{% endwith %}
+{% endif %}
+
+<!-- ── Entity List ── -->
+<div class="card">
+    <h3>👤 Entities <span style="font-weight:400;font-size:0.8rem;color:#888;">{{ entities|length }} shown</span></h3>
+    <div id="entityList">
+    {% for e in entities %}
+        <div class="entity-card">
+            <div style="display:flex;justify-content:space-between;align-items:center;">
+                <span class="val">{{ e.entity_value }}</span>
+                <a href="{% url 'analysis:entity_profile' e.id %}" class="btn btn-sm" style="font-size:0.75rem;padding:0.2rem 0.6rem;background:#4361ee;color:#fff;text-decoration:none;border-radius:4px;">Profile</a>
+            </div>
+            {% if e.feature_json %}
+            <div style="margin-top:0.3rem;">
+                {% for k, v in e.feature_json.items|slice:":8" %}
+                <span class="feat"><strong>{{ k }}</strong>: {% if v is None %}-{% else %}{{ v|floatformat:2 }}{% endif %}</span>
+                {% endfor %}
+                {% if e.feature_json.items|length > 8 %}<span class="feat" style="color:#999;">+{{ e.feature_json.items|length|add:"-8" }} more</span>{% endif %}
+            </div>
+            {% endif %}
+        </div>
+    {% empty %}
+        <div class="empty-state"><p>No entities in this cluster.</p></div>
+    {% endfor %}
+    </div>
+</div>
+{% endblock %}
@@ -81,7 +81,6 @@
        <a href="{% url 'analysis:run_list' %}">运行记录</a>
        <a href="{% url 'analysis:globe' %}">态势</a>
        <a href="{% url 'analysis:config' %}">配置</a>
-        <a href="{% url 'simple_analysis:index' %}" style="color:#f4a261;font-weight:600;">简单分析</a>
    </nav>
    <div class="container">
        {% block content %}{% endblock %}
@@ -123,6 +122,36 @@
        const msg = event.reason && event.reason.message ? event.reason.message : String(event.reason || 'Unknown error');
        showError('未处理的错误: ' + msg);
    });
+
+    // ── Server heartbeat: detect when Django crashes ──
+    (function() {
+        var heartbeatBanner = null;
+
+        function checkServer() {
+            fetch('/', { method: 'HEAD', cache: 'no-store' })
+                .then(function() {
+                    if (heartbeatBanner) {
+                        heartbeatBanner.remove();
+                        heartbeatBanner = null;
+                    }
+                })
+                .catch(function() {
+                    if (!heartbeatBanner) {
+                        heartbeatBanner = document.createElement('div');
+                        heartbeatBanner.style.cssText = 'position:fixed;top:0;left:0;right:0;z-index:10000;background:#dc3545;color:#fff;text-align:center;padding:0.5rem;font-size:0.85rem;font-weight:600;';
+                        heartbeatBanner.textContent = '⚠️ 服务器连接断开 — Django 进程可能已崩溃。请重启应用。';
+                        document.body.prepend(heartbeatBanner);
+                    }
+                });
+        }
+
+        // Check every 30 seconds
+        setInterval(checkServer, 30000);
+        // Also check when the page becomes visible again
+        document.addEventListener('visibilitychange', function() {
+            if (!document.hidden) checkServer();
+        });
+    })();
    </script>
 </body>
 </html>
@@ -148,6 +148,39 @@
    <button class="btn btn-primary" style="margin-top:1rem;" id="startAutoBtn" onclick="startAuto()" {% if not llm_configured %}disabled{% endif %}>开始自动分析</button>
 </div>

+<!-- Filter builder card -->
+<div class="card" id="filterCard" style="background:#f0f4ff;border-color:#c8d6e5;">
+    <h3>🔍 数据筛选（可选）</h3>
+    <div id="filterNoDataset" style="text-align:center;color:#999;padding:1rem;font-size:0.9rem;">选择数据集后可用</div>
+    <div id="filterRows" style="display:none;">
+        <div class="filter-row" data-idx="0" style="display:flex;gap:0.5rem;align-items:center;margin-bottom:0.4rem;">
+            <select class="filter-col" style="flex:1;padding:0.35rem;border:1px solid #ccc;border-radius:4px;"><option value="">-- 列 --</option></select>
+            <select class="filter-op" style="width:130px;padding:0.35rem;border:1px solid #ccc;border-radius:4px;">
+                <option value="eq">==</option><option value="neq">!=</option>
+                <option value="gt">&gt;</option><option value="lt">&lt;</option>
+                <option value="gte">&gt;=</option><option value="lte">&lt;=</option>
+                <option value="contains">contains</option>
+                <option value="is_null">is_null</option>
+                <option value="not_null">not_null</option>
+                <option value="in">in</option>
+            </select>
+            <input class="filter-val" placeholder="值" style="flex:1;padding:0.35rem;border:1px solid #ccc;border-radius:4px;">
+            <button class="btn" onclick="removeFilterRow(this)" style="background:#dc3545;color:#fff;font-size:0.8rem;padding:0.2rem 0.5rem;display:none;">✕</button>
+        </div>
+    </div>
+    <div id="filterControls" style="display:none;margin:0.5rem 0;gap:0.5rem;align-items:center;">
+        <label style="font-size:0.85rem;white-space:nowrap;">逻辑:</label>
+        <select id="filterLogic" style="padding:0.35rem;border:1px solid #ccc;border-radius:4px;">
+            <option value="and">AND</option><option value="or">OR</option>
+        </select>
+        <button class="btn" onclick="addFilterRow()" style="font-size:0.8rem;background:#e8f0fe;color:#4361ee;">+ 添加条件</button>
+        <button class="btn" onclick="resetFilter()" style="font-size:0.8rem;background:#fce4ec;color:#c62828;">重置</button>
+        <button class="btn btn-primary" onclick="applyFilter()" style="margin-left:auto;">应用筛选</button>
+    </div>
+    <div id="filterResult" style="display:none;margin-top:0.5rem;padding:0.5rem;background:#e8f5e9;border-radius:4px;font-size:0.85rem;"></div>
+    <div id="activeFilterInfo" style="display:none;margin-top:0.5rem;padding:0.35rem 0.5rem;background:#fff3cd;border-radius:4px;font-size:0.8rem;color:#856404;"></div>
+</div>
+
 <div id="autoProgress" style="display:none;margin-top:1rem;">
    <p>LLM 编排中... <span id="autoStatus"></span></p>
    <div style="background:#eee;height:8px;border-radius:4px;overflow:hidden;">
@@ -171,16 +204,27 @@
 const DATASETS_WITH_STATUS = {{ datasets_with_status_json|safe }};
 let activeFilter = 'all';
 let selectedRunId = null;
+let activeFilteredDatasetId = null;
+let currentFilters = [];
+let currentFilterLogic = 'and';
+
+// ── Build column map from session datasets ──
+const DATASET_COLUMNS = {};
+DATASETS_WITH_STATUS.forEach(ds => {
+    if (ds.columns && ds.columns.length > 0) {
+        DATASET_COLUMNS[ds.dataset_id] = ds.columns;
+    }
+});

 // ── Dataset table rendering ──
 function buildDatasetTable(filter) {
    const container = document.getElementById('datasetTable');
    let rows = DATASETS_WITH_STATUS.filter(ds => {
        if (filter === 'all') return true;
-        if (filter === 'ready') return ds.run_status === 'ready' || (!ds.run_status || ds.run_status === 'ready');
-        if (filter === 'analyzing') return ['pending', 'loading', 'profiling', 'aggregating', 'clustering', 'extracting'].includes(ds.run_status);
-        if (filter === 'completed') return ds.run_status === 'completed';
-        if (filter === 'failed') return ds.run_status === 'failed';
+        if (filter === 'ready') return ds.badge_class === 'ready';
+        if (filter === 'analyzing') return ds.badge_class === 'analyzing';
+        if (filter === 'completed') return ds.badge_class === 'completed';
+        if (filter === 'failed') return ds.badge_class === 'failed';
        return true;
    });

@@ -192,9 +236,219 @@ function buildDatasetTable(filter) {
    let html = '<table class="ds-table"><thead><tr><th></th><th>ID</th><th>状态</th><th>行数</th><th>文件数</th><th>来源</th></tr></thead><tbody>';
    rows.forEach(ds => {
        const label = ds.status_label || '待分析';
-        const badgeClass = (ds.run_status === 'completed') ? 'completed' :
-                          (ds.run_status === 'failed') ? 'failed' :
-                          (ds.run_status && ds.run_status !== 'ready') ? 'analyzing' : 'ready';
+        const badgeClass = ds.badge_class || 'ready';
+        const dispId = ds.display_id || ds.dataset_id;
+        const rowCount = ds.row_count != null ? ds.row_count.toLocaleString() : '?';
+        const fileCount = ds.file_count != null ? ds.file_count : '?';
+        const source = ds.sqlite_table ? ('SQLite: ' + ds.sqlite_table) : (ds.csv_glob || '?');
+        const checked = (selectedRunId === String(ds.dataset_id)) ? 'checked' : '';
+        const reloadIndicator = ds.needs_reload ? ' <span style="color:#e65100;font-size:0.7rem;">(需加载)</span>' : '';
+        html += '<tr data-ds-id="' + ds.dataset_id + '" onclick="selectRow(\'' + ds.dataset_id + '\')" class="' + (selectedRunId === String(ds.dataset_id) ? 'selected' : '') + '">' +
+            '<td><input type="radio" name="auto_run_id" value="' + ds.dataset_id + '" ' + checked + ' onclick="event.stopPropagation();selectRow(\'' + ds.dataset_id + '\')"></td>' +
+            '<td style="font-weight:600;">#' + dispId + '</td>' +
+            '<td><span class="status-badge ' + badgeClass + '">' + label + reloadIndicator + '</span></td>' +
+            '<td>' + rowCount + '</td>' +
+            '<td>' + fileCount + '</td>' +
+            '<td style="max-width:180px;overflow:hidden;text-overflow:ellipsis;white-space:nowrap;">' + escapeHtml(source) + '</td>' +
+        '</tr>';
+    });
+    html += '</tbody></table>';
+    container.innerHTML = html;
+}
+
+function selectRow(dsId) {
+    selectedRunId = String(dsId);
+    // Update radio buttons and row highlight
+    document.querySelectorAll('#datasetTable input[name="auto_run_id"]').forEach(r => {
+        r.checked = (r.value === String(dsId));
+    });
+    document.querySelectorAll('#datasetTable tr').forEach(tr => tr.classList.remove('selected'));
+    const row = document.querySelector('#datasetTable tr[data-ds-id="' + dsId + '"]');
+    if (row) row.classList.add('selected');
+
+    // Check if data needs reload
+    const dsInfo = DATASETS_WITH_STATUS.find(ds => ds.dataset_id === dsId);
+    if (dsInfo && dsInfo.needs_reload) {
+        // Auto-reload data
+        if (row) row.style.opacity = '0.6';
+        fetch('/tools/reload-run/', {
+            method: 'POST',
+            headers: { 'Content-Type':'application/json', 'X-CSRFToken':'{{ csrf_token }}' },
+            body: JSON.stringify({ display_id: dsInfo.display_id || dsInfo.dataset_id }),
+        })
+        .then(r => r.json())
+        .then(data => {
+            if (data.error) {
+                showError('加载数据集失败: ' + data.error);
+                if (row) row.style.opacity = '1';
+                return;
+            }
+            dsInfo.needs_reload = false;
+            if (data.columns) {
+                dsInfo.columns = data.columns;
+                DATASET_COLUMNS[dsId] = data.columns;
+            }
+            if (row) row.style.opacity = '1';
+            // Rebuild table to remove reload indicator
+            buildDatasetTable(activeFilter);
+            selectRow(dsId);
+            showSuccess('数据集已加载: ' + dsId);
+        })
+        .catch(err => {
+            showError('加载数据集失败: ' + err.message);
+            if (row) row.style.opacity = '1';
+        });
+        return;
+    }
+
+    // Populate filter column dropdowns
+    onSelectDataset(dsId);
+}
+
+function onSelectDataset(dsId) {
+    const noDatasetMsg = document.getElementById('filterNoDataset');
+    const filterRows = document.getElementById('filterRows');
+    const filterControls = document.getElementById('filterControls');
+    const filterResult = document.getElementById('filterResult');
+
+    if (!dsId || !DATASET_COLUMNS[dsId] || DATASET_COLUMNS[dsId].length === 0) {
+        noDatasetMsg.style.display = 'block';
+        filterRows.style.display = 'none';
+        filterControls.style.display = 'none';
+        filterResult.style.display = 'none';
+        return;
+    }
+
+    noDatasetMsg.style.display = 'none';
+    filterRows.style.display = 'block';
+    filterControls.style.display = 'flex';
+
+    const cols = DATASET_COLUMNS[dsId] || [];
+    document.querySelectorAll('.filter-col').forEach(sel => {
+        const cur = sel.value;
+        sel.innerHTML = '<option value="">-- 列 --</option>';
+        cols.forEach(c => { sel.innerHTML += '<option value="' + c + '" ' + (c===cur?'selected':'') + '>' + c + '</option>'; });
+    });
+}
+
+function setDatasetFilter(filter) {
+    activeFilter = filter;
+    document.querySelectorAll('#statusFilter .filter-btn').forEach(btn => {
+        btn.classList.toggle('active', btn.dataset.filter === filter);
+    });
+    buildDatasetTable(filter);
+}
+
+// Init filter tabs
+document.querySelectorAll('#statusFilter .filter-btn').forEach(btn => {
+    btn.addEventListener('click', function() {
+        setDatasetFilter(this.dataset.filter);
+    });
+});
+buildDatasetTable('all');
+
+// ── Filter builder functions ──
+function addFilterRow() {
+    const rows = document.querySelectorAll('#filterRows .filter-row');
+    if (rows.length >= 5) return;
+    const template = rows[0].cloneNode(true);
+    template.querySelector('.filter-val').value = '';
+    template.querySelector('.filter-col').selectedIndex = 0;
+    template.querySelector('.filter-op').selectedIndex = 0;
+    template.querySelector('button').style.display = 'inline-block';
+    document.getElementById('filterRows').appendChild(template);
+    if (selectedRunId) onSelectDataset(selectedRunId);
+}
+
+function removeFilterRow(btn) {
+    const rows = document.querySelectorAll('#filterRows .filter-row');
+    if (rows.length <= 1) return;
+    btn.closest('.filter-row').remove();
+}
+
+function resetFilter() {
+    const rows = document.querySelectorAll('#filterRows .filter-row');
+    for (let i = rows.length - 1; i >= 1; i--) rows[i].remove();
+    const firstRow = rows[0];
+    firstRow.querySelector('.filter-val').value = '';
+    firstRow.querySelector('.filter-col').selectedIndex = 0;
+    firstRow.querySelector('.filter-op').selectedIndex = 0;
+    firstRow.querySelector('button').style.display = 'none';
+    document.getElementById('filterResult').style.display = 'none';
+    document.getElementById('activeFilterInfo').style.display = 'none';
+    currentFilters = [];
+    activeFilteredDatasetId = null;
+}
+
+function collectFilters() {
+    const filters = [];
+    document.querySelectorAll('#filterRows .filter-row').forEach(row => {
+        const col = row.querySelector('.filter-col').value;
+        const op = row.querySelector('.filter-op').value;
+        let val = row.querySelector('.filter-val').value;
+        if (!col || !op) return;
+        if ((op === 'in') && val) {
+            val = val.split(',').map(s => s.trim()).filter(s => s);
+        }
+        filters.push({ column: col, op: op, value: val });
+    });
+    return filters;
+}
+
+async function applyFilter() {
+    if (!selectedRunId) { showError('请先选择数据集'); return; }
+    const filters = collectFilters();
+    if (filters.length === 0) { showError('请添加至少一个筛选条件'); return; }
+    const logic = document.getElementById('filterLogic').value;
+
+    try {
+        const resp = await fetch('/tools/filter/', {
+            method: 'POST',
+            headers: { 'Content-Type':'application/json', 'X-CSRFToken':'{{ csrf_token }}' },
+            body: JSON.stringify({ dataset_id: selectedRunId, filters, logic }),
+        });
+        const data = await resp.json();
+        if (data.error) { showError('筛选失败: ' + data.error); return; }
+
+        const resultDiv = document.getElementById('filterResult');
+        resultDiv.style.display = 'block';
+        resultDiv.innerHTML = '<strong>✅ 筛选完成</strong>: 数据集 <code>' + data.dataset_id + '</code>，共 <strong>' + data.row_count + '</strong> 行';
+        showSuccess('筛选完成: ' + data.row_count + ' 行');
+
+        currentFilters = filters;
+        currentFilterLogic = logic;
+        activeFilteredDatasetId = data.dataset_id;
+
+        const infoDiv = document.getElementById('activeFilterInfo');
+        infoDiv.style.display = 'block';
+        infoDiv.textContent = '已应用筛选 (' + filters.length + ' 条件) 到 ' + data.dataset_id;
+    } catch(e) {
+        showError('筛选请求失败: ' + e.message);
+    }
+}
+
+
+// ── Dataset table rendering ──
+function buildDatasetTable(filter) {
+    const container = document.getElementById('datasetTable');
+    let rows = DATASETS_WITH_STATUS.filter(ds => {
+        if (filter === 'all') return true;
+        if (filter === 'ready') return ds.badge_class === 'ready';
+        if (filter === 'analyzing') return ds.badge_class === 'analyzing';
+        if (filter === 'completed') return ds.badge_class === 'completed';
+        if (filter === 'failed') return ds.badge_class === 'failed';
+        return true;
+    });
+
+    if (!rows.length) {
+        container.innerHTML = '<p style="text-align:center;color:#999;padding:1rem;">无匹配的数据集</p>';
+        return;
+    }
+
+    let html = '<table class="ds-table"><thead><tr><th></th><th>ID</th><th>状态</th><th>行数</th><th>文件数</th><th>来源</th></tr></thead><tbody>';
+    rows.forEach(ds => {
+        const label = ds.status_label || '待分析';
+        const badgeClass = ds.badge_class || 'ready';
        const dispId = ds.display_id || ds.dataset_id;
        const rowCount = ds.row_count != null ? ds.row_count.toLocaleString() : '?';
        const fileCount = ds.file_count != null ? ds.file_count : '?';
@@ -260,33 +514,64 @@ function toggleStep(el) {
 }

 function buildTimeline(llmThinking, toolCalls) {
-    // Build tool entries from tool_calls_json (no separate thought entries)
-    const entries = (toolCalls || []).map(tc => ({
-        step: tc.step,
-        type: 'tool',
-        name: tc.name,
-        input: tc.input,
-        output: tc.output,
-    }));
-
-    // Match llm_thinking text to tool entries by step number
+    // Parse llm_thinking into per-step segments
+    const thinkingByStep = {};
+    const thinkOrder = []; // track step order for interleaving
    if (llmThinking) {
-        const thinkingByStep = {};
        const parts = llmThinking.split(/\n(?=\[\d+\])/);
        for (const part of parts) {
            const m = part.match(/^\[(\d+)\]\s*(.*)/s);
            if (m) {
-                thinkingByStep[parseInt(m[1])] = m[2].trim();
-            }
-        }
-        for (const entry of entries) {
-            if (thinkingByStep[entry.step] !== undefined) {
-                entry.thinking = thinkingByStep[entry.step];
+                const step = parseInt(m[1]);
+                if (!(step in thinkingByStep)) {
+                    thinkOrder.push(step);
+                }
+                thinkingByStep[step] = (thinkingByStep[step] || '') + (thinkingByStep[step] ? '\n' : '') + m[2].trim();
+            }
+        }
+    }
+
+    // Build entries: interleave thinking + tool per step
+    const toolMap = {}; // step -> tool entries
+    (toolCalls || []).forEach(tc => {
+        const s = tc.step;
+        if (!toolMap[s]) toolMap[s] = [];
+        toolMap[s].push({
+            type: 'tool',
+            name: tc.name,
+            input: tc.input,
+            output: tc.output,
+        });
+    });
+
+    const entries = [];
+    // Collect all step numbers that appear in either thinking or tool_calls
+    const allSteps = new Set([...thinkOrder, ...Object.keys(toolMap).map(Number)]);
+    const sortedSteps = Array.from(allSteps).sort((a, b) => a - b);
+
+    for (const step of sortedSteps) {
+        // Thinking first (独立条目)
+        if (thinkingByStep[step] !== undefined) {
+            // Last step with only thinking and no tool call = 回答
+            const isLastStep = step === sortedSteps[sortedSteps.length - 1];
+            const hasTool = toolMap[step] && toolMap[step].length > 0;
+            if (isLastStep && !hasTool) {
+                entries.push({ step, type: 'answer', text: thinkingByStep[step] });
+            } else {
+                entries.push({ step, type: 'thinking', text: thinkingByStep[step] });
+            }
+        }
+
+        // Then tool entries for this step
+        if (toolMap[step]) {
+            for (const te of toolMap[step]) {
+                // Attach thinking to first tool entry only if not already shown separately
+                // (No, we already show thinking separately above)
+                entries.push({ step, ...te, thinking: '' });
            }
        }
    }

-    entries.sort((a, b) => a.step - b.step);
    return entries;
 }

@@ -308,25 +593,33 @@ function renderTimeline(entries) {
            lastStep = entry.step;
        }

-        if (entry.type === 'tool') {
+        if (entry.type === 'thinking') {
+            // 思考条目 — 浅蓝背景文本块
+            const div = document.createElement('div');
+            div.style.cssText = 'background:#eef2ff;padding:0.6rem;border-radius:6px;font-size:0.78rem;line-height:1.6;color:#333;max-height:200px;overflow:auto;margin-bottom:0.3rem;white-space:pre-wrap;border:1px solid #d0d8f0;';
+            div.innerHTML =
+                '<span style="font-weight:600;color:#4361ee;font-size:0.75rem;display:block;margin-bottom:0.3rem;">💭 思考</span>' +
+                escapeHtml(entry.text);
+            container.appendChild(div);
+        } else if (entry.type === 'answer') {
+            // 回答条目 — 绿色背景
+            const div = document.createElement('div');
+            div.style.cssText = 'background:#e8f5e9;padding:0.6rem;border-radius:6px;font-size:0.78rem;line-height:1.6;color:#333;max-height:300px;overflow:auto;margin-bottom:0.3rem;white-space:pre-wrap;border:1px solid #c8e6c9;';
+            div.innerHTML =
+                '<span style="font-weight:600;color:#2e7d32;font-size:0.75rem;display:block;margin-bottom:0.3rem;">✅ 回答</span>' +
+                escapeHtml(entry.text);
+            container.appendChild(div);
+        } else if (entry.type === 'tool') {
            const card = document.createElement('div');
            card.style.cssText = 'border:1px solid #e0e0e0;border-radius:6px;margin-bottom:0.3rem;overflow:hidden;background:#fff;';
            const inputStr = prettyJson(entry.input);
-            const hasThinking = entry.thinking && (entry.output == null || entry.output === '' || entry.output === 'null' || entry.output === 'None');
            const outputStr = prettyJson(entry.output);
            let bodyInner = '';
            bodyInner +=
                '<div style="font-weight:600;font-size:0.72rem;color:#555;margin-bottom:0.2rem;">输入 (INPUT):</div>' +
-                '<pre style="background:#1a1a2e;color:#e0e0e0;padding:0.5rem;border-radius:4px;font-size:0.7rem;max-height:200px;overflow:auto;margin:0 0 0.3rem 0;"><code>' + escapeHtml(inputStr) + '</code></pre>';
-            if (hasThinking) {
-                bodyInner +=
-                    '<div style="font-weight:600;font-size:0.72rem;color:#555;margin-bottom:0.2rem;">思考 (THINKING):</div>' +
-                    '<div style="background:#eef2ff;padding:0.6rem;border-radius:4px;font-size:0.78rem;line-height:1.6;color:#333;max-height:200px;overflow:auto;margin:0;white-space:pre-wrap;">' + escapeHtml(entry.thinking) + '</div>';
-            } else {
-                bodyInner +=
-                    '<div style="font-weight:600;font-size:0.72rem;color:#555;margin-bottom:0.2rem;">输出 (OUTPUT):</div>' +
-                    '<pre style="background:#1a1a2e;color:#e0e0e0;padding:0.5rem;border-radius:4px;font-size:0.7rem;max-height:200px;overflow:auto;margin:0;"><code>' + escapeHtml(outputStr) + '</code></pre>';
-            }
+                '<pre style="background:#1a1a2e;color:#e0e0e0;padding:0.5rem;border-radius:4px;font-size:0.7rem;max-height:200px;overflow:auto;margin:0 0 0.3rem 0;"><code>' + escapeHtml(inputStr) + '</code></pre>' +
+                '<div style="font-weight:600;font-size:0.72rem;color:#555;margin-bottom:0.2rem;">输出 (OUTPUT):</div>' +
+                '<pre style="background:#1a1a2e;color:#e0e0e0;padding:0.5rem;border-radius:4px;font-size:0.7rem;max-height:200px;overflow:auto;margin:0;"><code>' + escapeHtml(outputStr) + '</code></pre>';
            card.innerHTML =
                '<div class="tool-call-header" onclick="toggleStep(this)" style="padding:0.4rem 0.8rem;">' +
                    '<span><span style="background:#43aa8b;color:#fff;border-radius:3px;padding:0 5px;font-size:0.7rem;font-weight:600;margin-right:6px;">🔧</span>' +
@@ -358,6 +651,16 @@ async function startAuto() {
        return;
    }

+    // Extract numeric display_id from dataset key (upload_XX or run_XX)
+    let displayId;
+    const uploadMatch = runId.match(/^(?:upload_|run_)(\d+)$/);
+    if (uploadMatch) {
+        displayId = parseInt(uploadMatch[1]);
+    } else {
+        displayId = parseInt(runId);
+    }
+    if (isNaN(displayId)) { showError('无效的数据集ID: ' + runId); return; }
+
    const timelineContainer = document.getElementById('timeline');
    const timelinePanel = document.getElementById('timelinePanel');

@@ -369,12 +672,20 @@ async function startAuto() {
    timelinePanel.style.display = 'none';
    timelineContainer.innerHTML = '';

+    // Build request body
+    const body = { run_id: displayId };
+    if (activeFilteredDatasetId) {
+        // Use filtered dataset: re-apply filter on the backend
+        body.filters = currentFilters;
+        body.logic = currentFilterLogic;
+    }
+
    let resp;
    try {
        resp = await fetch('/analyze/llm/', {
            method: 'POST',
            headers: { 'Content-Type': 'application/json', 'X-CSRFToken': '{{ csrf_token }}' },
-            body: JSON.stringify({ run_id: parseInt(runId) }),
+            body: JSON.stringify(body),
        });
    } catch (e) { showError('启动 LLM 分析失败: ' + e.message); return; }
    const data = await resp.json();
@@ -383,7 +694,7 @@ async function startAuto() {

    statusPoll = setInterval(async () => {
        try {
-            const r = await fetch('/runs/' + runId + '/status/');
+            const r = await fetch('/runs/' + displayId + '/status/');
            const s = await r.json();
            const pct = s.progress_pct || 0;
            const msg = s.progress_msg || s.status;
@@ -403,7 +714,7 @@ async function startAuto() {
                document.getElementById('autoBar').style.width = '100%';
                document.getElementById('cancelBtn').style.display = 'none';
                document.getElementById('autoStatus').textContent = '分析完成！正在跳转...';
-                setTimeout(() => { window.location.href = '/clusters/' + runId + '/'; }, 1500);
+                setTimeout(() => { window.location.href = '/clusters/' + displayId + '/'; }, 1500);
            }
            if (s.status === 'failed') {
                clearInterval(statusPoll);
@@ -183,9 +183,9 @@ function buildDatasetTable(filter) {
    const container = document.getElementById('datasetTable');
    let rows = DATASETS_WITH_STATUS.filter(ds => {
        if (filter === 'all') return true;
-        if (filter === 'ready') return ds.run_status === 'ready' || (!ds.run_status || ds.run_status === 'ready');
-        if (filter === 'completed') return ds.run_status === 'completed';
-        if (filter === 'failed') return ds.run_status === 'failed';
+        if (filter === 'ready') return ds.badge_class === 'ready';
+        if (filter === 'completed') return ds.badge_class === 'completed';
+        if (filter === 'failed') return ds.badge_class === 'failed';
        return true;
    });

@@ -197,9 +197,7 @@ function buildDatasetTable(filter) {
    let html = '<table class="ds-table"><thead><tr><th>ID</th><th>状态</th><th>行数</th><th>文件数</th><th>SVD</th><th></th></tr></thead><tbody>';
    rows.forEach(ds => {
        const label = ds.status_label || '待分析';
-        const badgeClass = (ds.run_status === 'completed') ? 'completed' :
-                          (ds.run_status === 'failed') ? 'failed' :
-                          (ds.run_status && ds.run_status !== 'ready') ? 'analyzing' : 'ready';
+        const badgeClass = ds.badge_class || 'ready';
        const dispId = ds.display_id || ds.dataset_id;
        const rowCount = ds.row_count != null ? ds.row_count.toLocaleString() : '?';
        const fileCount = ds.file_count != null ? ds.file_count : '?';
@@ -218,6 +216,58 @@ function buildDatasetTable(filter) {
 }

 function selectDataset(dsId) {
+    // Check if this dataset needs data reload
+    const dsInfo = DATASETS_WITH_STATUS.find(ds => ds.dataset_id === dsId);
+    if (dsInfo && dsInfo.needs_reload) {
+        // Trigger background reload
+        const row = document.querySelector(`#datasetTable tr[data-ds-id="${dsId}"]`);
+        if (row) {
+            row.style.opacity = '0.6';
+            const statusCell = row.querySelector('.status-badge');
+            if (statusCell) statusCell.textContent = '加载中...';
+        }
+        fetch('/tools/reload-run/', {
+            method: 'POST',
+            headers: { 'Content-Type':'application/json', 'X-CSRFToken':'{{ csrf_token }}' },
+            body: JSON.stringify({ display_id: dsInfo.display_id }),
+        })
+        .then(r => r.json())
+        .then(data => {
+            if (data.error) {
+                showError('加载数据集失败: ' + data.error);
+                if (row) row.style.opacity = '1';
+                return;
+            }
+            // Mark as no longer needing reload
+            dsInfo.needs_reload = false;
+            // Add to dropdown if not present
+            const sel = document.getElementById('datasetSelect');
+            if (!Array.from(sel.options).some(o => o.value === dsId)) {
+                const opt = document.createElement('option');
+                opt.value = dsId;
+                opt.text = dsId + ' (' + data.row_count + ' rows)';
+                sel.add(opt);
+            }
+            sel.value = dsId;
+            onDatasetChange();
+            // Update status badge
+            if (row) {
+                row.style.opacity = '1';
+                const badge = row.querySelector('.status-badge');
+                if (badge) {
+                    badge.textContent = '待分析';
+                    badge.className = 'status-badge ready';
+                }
+            }
+            showSuccess('数据集已加载: ' + dsId);
+        })
+        .catch(err => {
+            showError('加载数据集失败: ' + err.message);
+            if (row) row.style.opacity = '1';
+        });
+        return;
+    }
+
    const sel = document.getElementById('datasetSelect');
    let opt = Array.from(sel.options).find(o => o.value === dsId);
    if (!opt) {
@@ -394,9 +444,13 @@ async function executeStep(idx) {
    // Raw row-level clustering: skip entity aggregation entirely
    if (step.tool.name === 'build_entity_profiles') {
        step.status = 'done';
-        resultDiv.textContent = '[已跳过] 逐行直接聚类模式，无需实体聚合';
-        resultDiv.style.display = 'block';
+        step._resultText = '[已跳过] 逐行直接聚类模式，无需实体聚合';
        renderSteps();
+        const skipResultDiv = document.getElementById(`result-${idx}`);
+        if (skipResultDiv) {
+            skipResultDiv.textContent = step._resultText;
+            skipResultDiv.style.display = 'block';
+        }
        return;
    }
    // Always cluster raw rows directly
@@ -411,8 +465,7 @@ async function executeStep(idx) {
            body: JSON.stringify({ tool: step.tool.name, params }),
        });
        const data = await resp.json();
-        resultDiv.textContent = JSON.stringify(data, null, 2);
-        resultDiv.style.display = 'block';
+        step._resultText = JSON.stringify(data, null, 2);
        step.status = data.error ? 'failed' : 'done';
        // update dataset select from result
        if (data && data.result && data.result.dataset_id) {
@@ -425,12 +478,17 @@ async function executeStep(idx) {
            }
        }
    } catch(err) {
-        resultDiv.textContent = 'Error: ' + err.message;
-        resultDiv.style.display = 'block';
+        step._resultText = 'Error: ' + err.message;
        step.status = 'failed';
        showError('步骤 ' + step.tool.name + ' 执行失败: ' + err.message);
    }
    renderSteps();
+    // Re-populate result content after renderSteps() recreates the DOM
+    const newResultDiv = document.getElementById(`result-${idx}`);
+    if (newResultDiv && step._resultText) {
+        newResultDiv.textContent = step._resultText;
+        newResultDiv.style.display = 'block';
+    }
 }

 // ── Run All (sequential) ──
@@ -22,8 +22,8 @@ TOOLS = [
        "description": "使用自适应权重学习在实体数据上计算自适应代理/异常/威胁评分。在build_entity_profiles之后调用。添加proxy_score/risk_level/threat_score列。",
        "parameters": {"type": "object", "properties": {"dataset_id": {"type": "string"}}, "required": ["dataset_id"]}}},
    {"type": "function", "function": {"name": "run_clustering",
-        "description": "将实体数据聚类为行为组。需要实体数据集ID。探索性分析选择hdbscan（自动检测聚类数）；固定簇数选择kmeans。",
-        "parameters": {"type": "object", "properties": {"dataset_id": {"type": "string"}, "cluster_columns": {"type": "array", "items": {"type": "string"}}, "algorithm": {"type": "string", "enum": ["hdbscan", "kmeans"]}}, "required": ["dataset_id", "cluster_columns"]}}},
+"description": "将实体数据聚类为行为组。需要实体数据集ID。agglomerative（默认/凝聚层次聚类）适合预知簇数；hdbscan自动检测聚类数；kmeans固定簇数。",
+"parameters": {"type": "object", "properties": {"dataset_id": {"type": "string"}, "cluster_columns": {"type": "array", "items": {"type": "string"}}, "algorithm": {"type": "string", "enum": ["agglomerative", "hdbscan", "kmeans"]}}, "required": ["dataset_id", "cluster_columns"]}}},
    {"type": "function", "function": {"name": "extract_features",
        "description": "识别每个聚类的区分特征（z-score或anova）。在run_clustering之后调用。",
        "parameters": {"type": "object", "properties": {"dataset_id": {"type": "string"}, "cluster_result_id": {"type": "string"}, "method": {"type": "string", "enum": ["zscore", "anova"]}}, "required": ["dataset_id", "cluster_result_id"]}}},
@@ -363,13 +363,20 @@ def run_llm_pipeline(dataset_id: str, entity_col: str = "",
            called_tools.add(tool_key)

            logger.info(f'[TOOL] step={step} name={name} args={json.dumps(args)[:200]}')
-            if callback:
-                callback(step, name, None, meta={"args": args})

            try:
-                result = asyncio.run(handle_call(name, args))
+                async def _call_with_timeout():
+                    return await asyncio.wait_for(
+                        handle_call(name, args),
+                        timeout=120,
+                    )
+                result = asyncio.run(_call_with_timeout())
+            except asyncio.TimeoutError:
+                result = {'error': f'Tool {name} timed out after 120s'}
+                logger.warning(f'[TOOL] {name} timed out')
            except Exception as e:
-                raise SystemExit(f"MCP tool error: {e}") from e
+                result = {'error': f'MCP tool error: {e}'}
+                logger.error(f'[TOOL] {name} error: {e}')

            if callback:
                callback(step, name, result, meta={"args": args})
@@ -32,7 +32,6 @@ INSTALLED_APPS = [
    "django.contrib.messages",
    "django.contrib.staticfiles",
    "analysis.apps.AnalysisConfig",
-    "simple_analysis.apps.SimpleAnalysisConfig",
 ]

 MIDDLEWARE = [
@@ -4,5 +4,5 @@ from django.urls import path, include
 urlpatterns = [
    path('admin/', admin.site.urls),
    path('', include('analysis.urls')),
-    path('simple/', include('simple_analysis.urls')),
 ]
+
Author	SHA1	Message	Date
PM-pinou	88b26f16fe	docs: rewrite all markdown documentation for views/tools package refactoring - Update README.md: new architecture (views/ 12 modules, tools/ 17 modules), remove simple_analysis refs, 30 MCP tools, 8GB RAM spec - Update AGENTS.md: project structure tree, 30 tools organized by implementation module, remove entity_detector/entity_aggregator refs - Update docs/工作流.md: full pipeline v3.0, code references to tools/ package, remove simple_analysis section (note moved to master) - Update docs/故障诊断手册.md: fix code location references, add WebGL2 tips, remove Leaflet/simple_analysis refs Branch: beta-clean	2026-07-23 22:51:36 +08:00
PM-pinou	ef60e3e489	cleanup: consolidate TLS_HEX_MAP, extract constants, remove dead code, add traceback logging 3.1 Unified TLS_HEX_MAP: defined once in type_classifier.py, imported in globe.py and clustering.py 3.2 Created analysis/constants.py: centralized RANDOM_SEED, MAX_SAMPLE_ROWS, UMAP constants, GLOBE_MAX_ROWS, etc. 3.3 Removed 14 pointless try/except:raise dead code blocks across tools/ and views/ 3.4 Added logger.error(traceback) to all bare except Exception: blocks in tools/ and views/ 3.5 Moved inline import traceback/asyncio to top-level imports in features.py, clustering.py, pipeline.py 3.6 Removed auto_profile_module side-effect imports from 6 modules	2026-07-23 22:44:42 +08:00
PM-pinou	885da89b6f	refactor: split tool_registry.py (3368 lines) into analysis/tools/ package - 15 sub-modules: _helpers, _registry, _dispatch, load_data, profile, filter, preprocess, clustering, evaluate, features, export, entities, anomalies, diagnostics, analysis, data_mgmt, distance_matrix - Extracted shared helpers: _resolve_dataset(), _count_rows(), _get_numeric_columns(), NUMERIC_DTYPES, NUMERIC_TYPE_NAMES - Extracted _cluster_core() for shared clustering pipeline - Thin tool_registry.py wrapper preserves backward compatibility - Old file preserved as tool_registry_backup.py - manage.py check passes (0 issues)	2026-07-23 22:31:17 +08:00
PM-pinou	d4c82768a8	refactor: split analysis/views.py into views/ package (12 sub-modules) - helpers.py: _extract_lat, _extract_lon, plan index utilities - dashboard.py: dashboard, run_list, run_detail, run_status_api, start_analysis - pipeline.py: _run_pipeline_worker, _background_process - clustering.py: cluster_overview, cluster_detail, _run_clustering_pipeline, _get_globe_flows - entity.py: entity_profile - upload.py: upload_page, upload_csv, upload_csv_batch, finalize_upload, delete_upload - auto.py: auto_page, run_llm_analysis_view - manual.py: manual_page, manual_run_analysis - globe.py: globe_view, _extract_flows_from_df - config.py: config_view, llm_test - log_viewer.py: log_viewer - tools.py: tool_lab, tool_lab_run, tool_plan, apply_filter, reload_run_data, retry_run All function logic preserved exactly. views/__init__.py re-exports all public functions for backward compatibility with urls.py and run_pipeline.py. Original views.py archived as views_backup.py.	2026-07-23 22:22:28 +08:00
PM-pinou	123db6b08f	chore: consolidate all session fixes — migrations squash, views/tools refactor, cluster UI, geoip upgrade, simple_analysis removal, non-blocking startup, chinese UI, API key removal	2026-07-23 22:10:26 +08:00
PM-pinou	02176cf6f4	chore: remove config.yaml from tracking (contains API keys), add example config	2026-07-23 20:51:32 +08:00