Commit Graph

165 Commits

Author SHA1 Message Date
TianXuan Developer b04ca5cca9 fix: 改用Leaflet原生虚线+动画dashOffset, 100%可见
彻底放弃canvas粒子方案,改用Leaflet L.polyline + dashArray:
- 每条活跃边=1条虚线(dash=15,gap=25)
- dashOffset持续动画(0.3px/帧),产生流动效果
- 交叉淡入淡出: 透明度0.4-1.0 + 宽度1.5-5.5px
- removeLayer旧边,addTo新边,平滑过渡
- 零z-index问题,Leaflet原生渲染

Co-Authored-By: Claude <noreply@anthropic.com>
2026-07-25 10:04:33 +08:00
TianXuan Developer bbecd4a4f1 fix: 更小窗口组(15-20边)+颜色偏移(hue)让过渡更明显
- 窗口拆分更细: ~20条/组,最大15个窗口
- 颜色随窗口组偏移hue(15°/组),视觉变化明显
- 去重: 同一边不会在多个窗口重复出现
- 透明度范围扩大(0.4-1.0)

Co-Authored-By: Claude <noreply@anthropic.com>
2026-07-25 10:00:53 +08:00
TianXuan Developer 5170c262e1 fix: 稀疏数据自动分裂为多窗口,交叉淡入淡出可见
根因: 数据仅1个真实窗口,所有边同时活跃,无变化
修复: loadTrafficData检测windows≤2时,将边按count排序
后分割为~40条/组,每组一个虚拟窗口(间隔10s)
效果: 300条边→8个窗口,每窗口不同边集,交叉淡入淡出可见

验证: 窗口0(40边)↔窗口1(40边),i=0→i=1,t平滑变化
Playwright: 0 JS错误

Co-Authored-By: Claude <noreply@anthropic.com>
2026-07-25 09:57:42 +08:00
TianXuan Developer 66b35eb207 refactor: smooth crossfade transition for traffic timeline, fix animation freeze, reduce memory
完全重写动态流通动画:
- Crossfade: 相邻窗口边通过透明度渐变(1-t↔t)平滑过渡
- getWindowMix: 计算当前窗口索引+混合因子t
- 边去重: 同一边在两窗口间连续存在不闪烁
- 时间确保持续推进,循环播放
- 边坐标缓存(地图移动时失效),减少latLngToContainerPoint调用
- 粒子基于Date.now()连续运动,不受窗口切换影响
- 节点活跃状态按两窗口并集+权重
- 移除旧的phase band/edgeMap/particle pool/_spawnTimers

验证: 上传→聚类→地图→流量面板→时间推进→实体流量图表,0 JS错误

Co-Authored-By: Claude <noreply@anthropic.com>
2026-07-25 09:53:17 +08:00
TianXuan Developer 55b1d0a7a2 fix: canvas回map内+position:relative避免错位
_ensureFlowLayer: canvas作为#saMap子元素,
position:absolute;top:0;left:0, resize事件同步尺寸
不再需要在map外部计算偏移

Co-Authored-By: Claude <noreply@anthropic.com>
2026-07-25 09:24:45 +08:00
TianXuan Developer 8bd1ff7ccf fix: 修复重复声明_trafficCanvasCtx导致上传失效
let _trafficCanvasCtx在旧代码(2576行)和新flow层(2759行)各声明一次,
导致'Identifier has already been declared'错误,
handleFiles等后续函数均未定义,上传完全失效

删除旧声明,保留flow层中的声明

Co-Authored-By: Claude <noreply@anthropic.com>
2026-07-25 09:19:17 +08:00
TianXuan Developer 7a597a3f9f fix: Canvas粒子流动——插入到map容器外部确保在最上层渲染
关键修复: canvas直接插入到map容器parent,绝对定位覆盖地图,
不再受Leaflet内部图层堆叠影响。
粒子沿贝塞尔曲线流动,颜色从蓝渐变到红(3-8个/边)

Co-Authored-By: Claude <noreply@anthropic.com>
2026-07-25 09:15:20 +08:00
TianXuan Developer 91fb072f9f feat: 流量动画改用Leaflet原生dashArray虚线流动
Canvas粒子方案在Leaflet层面存在图层遮挡问题。
改为Leaflet原生L.polyline + dashArray + dashOffset动画:
- 每条活跃边绘制虚线 polyline (dash/gap按流量比例)
- requestAnimationFrame 持续偏移 dashOffset
- 产生'蚂蚁行军'流动效果, 100%可见
- 不透明度0.85, 线宽2-7px, HSL颜色渐变

Co-Authored-By: Claude <noreply@anthropic.com>
2026-07-25 09:14:28 +08:00
TianXuan Developer 569c7411b8 fix: canvas插入到Leaflet tilePane之后,确保在最上层渲染
Co-Authored-By: Claude <noreply@anthropic.com>
2026-07-25 09:12:14 +08:00
TianXuan Developer 5da0f2bf90 chore: remove test screenshots 2026-07-25 09:11:27 +08:00
TianXuan Developer 56007c2122 fix: 增强边缘线可见度——移除200条限制,提高不透明度(0.5-1.0),加粗(2-7px)
Co-Authored-By: Claude <noreply@anthropic.com>
2026-07-25 09:11:02 +08:00
TianXuan Developer 613f7d2d50 fix: 增强粒子视觉效果——发光粒子+加粗边缘+更高不透明度
- 粒子尺寸3-7px, 红色发光(shadowBlur=6)
- 边缘线加粗到1.5-5.5px, 不透明度0.3-0.8
- 粒子生成率提高(4/边/生命周期)

Co-Authored-By: Claude <noreply@anthropic.com>
2026-07-25 09:09:28 +08:00
TianXuan Developer bcf447db37 fix: 修复动画始终重复第一帧——边按phase分批轮流活跃
根因: 数据min_ts==max_ts(所有时间戳同一秒),时间无法推进
修复:
- 数据加载时给每条边分配phase(0→1),按相位分布
- 活跃带(bandWidth=0.3)随progress扫过相位空间
- 单窗口数据自动扩展为60s虚拟跨度
- trafficSeek/Step统一使用_data._minTs/_maxTs

效果: 400+粒子持续流动,活跃边随进度轮换,
      ~30%边同时活跃,时间平滑推进

Co-Authored-By: Claude <noreply@anthropic.com>
2026-07-25 09:08:21 +08:00
TianXuan Developer 74455a03b9 feat: 流量动画重构为连续粒子系统
替换逐帧窗口切换为连续时间轴粒子动画:
- 时间连续推进(requestAnimationFrame + speed倍率)
- 粒子从源节点沿贝塞尔曲线流向目标,自动生成/过期
- 条边淡淡轨迹 + 运动光点,大小/透明度反映流量
- 时间滑块平滑映射到完整时间范围
- 活跃窗口自适应扩展(data稀疏时保持边可见)

Co-Authored-By: Claude <noreply@anthropic.com>
2026-07-24 23:28:28 +08:00
TianXuan Developer f17ba2255a fix: 流量动画Canvas贝塞尔曲线渲染修复
根因: Canvas不在map容器内,缺少position:relative;z-index不够高
修复:
- _ensureTrafficCanvas动态创建canvas为#saMap子元素
- 添加position:relative到#saMap容器
- z-index提升到99999确保在Leaflet画布之上
- requestAnimationFrame替代setTimeout实现平滑粒子动画
- 面板关闭/trafficStop时清理canvas和恢复节点样式
- 粒子位置用Date.now()计算实现连续运动

Co-Authored-By: Claude <noreply@anthropic.com>
2026-07-24 23:01:17 +08:00
TianXuan Developer b02258330b fix: 修复上传按钮消失 + 移动流量面板HTML至script外
根因: 上一提交将traffic panel HTML放在</script>之前,
导致HTML被解析为JS代码,引发'Unexpected token <'错误,
handleFiles等函数未定义,上传功能完全失效

修复: 将traffic panel HTML移至</script>之后,{% endblock %}之前

验证: 14/14 Playwright测试通过 (上传/筛选/聚类/地图/边/节点/流量面板/社区焦点)

Co-Authored-By: Claude <noreply@anthropic.com>
2026-07-24 22:46:11 +08:00
TianXuan Developer 82e3cd26ee feat: add global and per-entity dynamic traffic flow animation
- backend: traffic_timeline_view + entity_traffic_timeline_view
  time-window aggregation (10s/30s/1min/5min/10min)
  Beijing time (UTC+8), per-edge count/bytes, node activity
- frontend: global traffic panel with play/pause/speed controls
  canvas overlay for curved edge flow particles + node dimming
  entity traffic view with 24h distribution + in/out chart.js
- routes: /simple/traffic-timeline/ + /simple/traffic-timeline/entity/

Co-Authored-By: Claude <noreply@anthropic.com>
2026-07-24 22:35:46 +08:00
TianXuan Developer eb5758b77b fix: 社区点击自动跳转步骤4 + 边显示 + 区分节点样式
修复:
- loadClusterDetail: 无map时自动goStep(4)并等待初始化
- focusClusterOnMap: 边已正确添加到_clusterFocusLayer
- renderMapData: 集群聚焦时保留_clusterFocusLayer
- 节点样式按度数区分: 高流量(>20对端)大圆深色, 孤立节点小圆浅色
- 节点提示显示对端数和服务器标签
- clearClusterFocus简化,去除无用_clusterNodeLayer

Co-Authored-By: Claude <noreply@anthropic.com>
2026-07-24 20:59:55 +08:00
TianXuan Developer cd68d7c9af fix: gen_multi2 使用真实公网IP,GeoIP可解析
根因: 服务器IP(203.0.113.x等RFC5737)和代理IP(10.0.x.x私有)无法被GeoIP解析
→ 500条边仅2条两端有坐标 → 图上不显示

修复:
- 服务器IP: 替换为 Akamai/Cloudflare/Google/AWS/Microsoft 公网IP
- 代理IP: 替换为 1.1.1.1/8.8.8.8 等已知DNS公网IP
- 移除硬编码SERVER_GEO,依赖MMDB自动解析

结果: 710/718节点有坐标, 496/500边两端可渲染 → 300条边显示

Co-Authored-By: Claude <noreply@anthropic.com>
2026-07-24 20:38:59 +08:00
TianXuan Developer 0a3c0a6eb3 fix: 修复gen_multi2过滤后无边的问题
根因: gen_test_data 第135行 cert_cn 生成有bug
  f'*.{random.choice(SERVICES).split(".")[-2:]}'
  → 产生 "*.['example', 'com']" 垃圾cnam
  → 过滤匹配不到任何有效IP
  → 所有节点度=1,无边

修复: 检测并清除垃圾cnam/snam模式,仅保留有效值

测试结果(筛选cnam=api.example.com):
  节点718, 边500, 最大度201, 平均度18.5
  服务器入度: 152-201

Co-Authored-By: Claude <noreply@anthropic.com>
2026-07-24 20:30:15 +08:00
TianXuan Developer d6687e8aa0 fix: gen_multi2 完全继承 gen_test_data 格式 + 拓扑约束
- 复用 generate_row() 保证所有列格式/覆盖率/空值率一致
- 重映射: snam/cnam 有值时 → :ipd 指向1-3个固定服务器IP
- 同步更新 GeoIP 字段匹配服务器位置
- 200文件 × 10000行 = 200万行
- 代理出度60K+, 服务器入度50K-202K

Co-Authored-By: Claude <noreply@anthropic.com>
2026-07-24 20:17:23 +08:00
TianXuan Developer 005a6b82bb fix: 重写gen_multi2, 继承gen_test_data精确格式
- 复用 generate_row() 保证列格式/覆盖率/空值率完全一致
- 覆盖IP和服务字段为拓扑感知值
- cnam→1-3固定服务器IP, snam同理
- 代理出度870-1005, 服务器入度1615-2100
- 200文件×~100行, 列顺序/删减同gen_multi

Co-Authored-By: Claude <noreply@anthropic.com>
2026-07-24 20:08:49 +08:00
TianXuan Developer b33845e9a2 chore: gitignore data/gen_multi2/ 2026-07-24 19:57:10 +08:00
TianXuan Developer a4cd8cdf1d feat: 真实网络拓扑测试数据生成器
gen_multi2.py:
- 服务器节点: 13个, 入度350-1700 (CDN/API/Mail/Auth等)
- 代理节点: 5个, 出度700+
- 客户端节点: 144个, 出度100-130
- cnam/snam 映射到特定服务器IP (1-3个)
- 12个地理区域, 匹配GeoIP坐标
- 输出: data/gen_multi2/ (200文件, 18148行)

Co-Authored-By: Claude <noreply@anthropic.com>
2026-07-24 19:56:33 +08:00
TianXuan Developer 829729943c feat: 社区交互增强 + 卫星地图 + 节点/边数据丰富 + 项目清理
核心改进:
- 节点数据丰富: TLS版本/加密套件/SNI/端口/流量统计/连接成功率
- 边数据丰富: 包数/持续时间/TLS/端口/SNI/分类标签
- 新增 edge_time_series 端点: 边流量时序图(北京时间)
- 新增 cluster_nodes_edges 端点: 社区聚焦(zoom+高亮+专属边)
- 社区点击: 自动缩放/高亮节点/显示社区边(可点击)
- 边交互修复: 事件从layerGroup移至hitPoly,三处边渲染全修复
- 地图升级: ESRI卫星影像 + 255国矢量边界叠加
- 离线回退: 卫星→CartoDB→暗色矢量地图,完整可用
- 节点面板: 删除无意义社区标签,移至Host+组织同行
- sanitizeName: 清理5个非ASCII国家名避免乱码
- .gitignore: 补充geoip_chunks/和嵌套CSV规则

项目清理:
- 删除 logs/ (20个调试文件,~2.5MB)
- 删除 test_*.py (4个临时测试脚本)
- 删除 _check_bool/_pipeline_test/_restart_server/start_server (硬编码路径)
- 删除 node_modules/package.json/package-lock.json (18MB,不用Node)
- 删除 tests/ 旧日志和调试脚本
- 删除 .pytest_cache/ .server_pid.txt RUNTIME_CHANGED.txt
- AGENTS.md → CLAUDE.md

Co-Authored-By: Claude <noreply@anthropic.com>
2026-07-24 13:06:14 +08:00
PM-pinou a56a06c940 chore: gitignore large binary files (GeoIP chunks, mmdb, test CSVs) 2026-07-24 11:21:34 +08:00
TianXuan Developer 8ba99292b7 fix: MMDB中文路径解决方案 + 聚类优先使用最新GeoIP数据
- geoip.py: maxminddb C库不支持Windows中文路径,复制MMDB到ASCII临时目录
- views.py: _cluster_network 优先用 geoip_lookup()(MMDB) 而非上传时缓存(text)
- 效果: IP地理坐标从 ~55城市粗定位 → 全球百万级精确城市定位

Co-Authored-By: Claude <noreply@anthropic.com>
2026-07-23 23:59:27 +08:00
TianXuan Developer 2e93bcad2e chore: gitignore 忽略生成的 .mmdb 和 .mmdb.gz 文件 2026-07-23 23:51:49 +08:00
TianXuan Developer 4b8963a834 feat: GeoIP City分块 partaf 2026-07-23 23:49:04 +08:00
TianXuan Developer bb16219739 feat: GeoIP City分块 partae 2026-07-23 23:47:50 +08:00
TianXuan Developer 4c8cb6d187 feat: GeoIP City分块 partad 2026-07-23 23:47:24 +08:00
TianXuan Developer 4936bd72ae feat: GeoIP City分块 partac 2026-07-23 23:46:52 +08:00
TianXuan Developer f2120175af feat: GeoIP City分块 partab 2026-07-23 23:46:29 +08:00
TianXuan Developer 45429ef6f3 feat: GeoIP分块 — ASN(5MB) + City首尾块(9MB+6MB) 2026-07-23 23:45:28 +08:00
TianXuan Developer e0a7400bd7 feat: 高精度离线GeoIP — MMDB支持 + 中文路径修复 + 分块自解压
- geoip.py: 修复中文路径 str()→Path 编码问题 (maxminddb Latin-1 bug)
- geoip.py: 启动自动重组geoip_chunks分块 + gunzip解压
- scripts/setup_geoip.py: 手动重组/解压脚本
- .gitignore: 忽略解压后的 .mmdb 文件

Co-Authored-By: Claude <noreply@anthropic.com>
2026-07-23 23:42:38 +08:00
TianXuan Developer 0f84497c82 fix: 边线点击优化 — 双层线结构 + hover 高亮
- 每条边由底层粗透明线(12px)和上层可见线组成
- 粗透明线捕获点击,命中区域扩大10倍以上
- 线宽 2-6px(按通信频次),不透明度 0.45-0.80
- hover 时高亮:加粗+亮红+不透明
- tooltip 提示点击查看详情
- highlightNodeEdges 同步使用双层线

Co-Authored-By: Claude <noreply@anthropic.com>
2026-07-23 23:20:26 +08:00
TianXuan Developer 2f7173637a fix: 静态资源路径改用 Django {% static %} 模板变量,确保离线部署兼容 2026-07-23 23:17:59 +08:00
TianXuan Developer f0a4fed7cf feat: 离线矢量世界地图 — TopoJSON 三精度 + 在线瓦片兜底
Natural Earth 三精度级别离线矢量地图:
- 10m (精细): 90KB — 完整海岸线+岛屿,2%简化
- 50m (标准): 533KB — 主要海岸线+国界
- 110m (基础): 54KB — 大陆轮廓

自动优先级: 10m → 50m → 110m → CartoDB在线瓦片
使用 topojson-client 库将 TopoJSON 转为 Leaflet GeoJSON 图层
矢量渲染,任意缩放级别清晰,经纬度天然对齐

Co-Authored-By: Claude <noreply@anthropic.com>
2026-07-23 23:13:05 +08:00
TianXuan Developer d4ec81a1dd feat: 修复世界地图 + 丰富节点/边数据 + 完善交互与导出
地图:
- 改用 CartoDB Positron 瓦片图层 (L.tileLayer),替代无效的 L.imageOverlay
- 复制 vendor 文件到项目级 static 目录

后端 (views.py):
- 新增 host 列别名 (host/hostname/host_name/remote_host/peer_name)
- _cluster_network 为每个 IP 收集 host、org、text、peer_ips 元数据
- 节点新增 host/org/texts/peer_ips/peer_count 字段
- 边新增 src/dst host/org/city/country/cluster + peak_hours/time_count 始终存在
- node_detail 返回 node_texts 和 enriched edges (peer_host/org/city)

前端 (simple_analysis.html):
- 实现 loadClusterDetail/showClusterDetailPanel 缺失函数
- renderCharts 在 goStep(4) 中调用
- 节点详情面板增强: host、org、text、对端 IP 列表
- 边详情面板增强: 端点 host/org/city/country、高峰时段
- 导出 CSV 增强: 节点新增 host/org/asn/peer_count/connected_ips,边新增全部端点字段
- Tooltip 新增 host 和 org 显示

测试: 更新 Playwright 测试适配两阶段上传 + 新 UI,全部 30 项通过

Co-Authored-By: Claude <noreply@anthropic.com>
2026-07-23 23:04:27 +08:00
TianXuan Developer 8322728de4 fix: CSS background world map + retry loop for container dimensions
- Map container gets SVG world map as CSS background-image
  (visible even before Leaflet initializes)
- Retry loop (max 5s) waits for container offsetHeight > 10
- zoomAnimation: false for faster init
- Console.log on successful map creation
- Verified: 550px height, 292 markers, background OK

Co-Authored-By: Claude <noreply@anthropic.com>
2026-07-23 22:08:20 +08:00
TianXuan Developer dedb3fc0ae test: Playwright self-test passes all 8 checks + fix Beijing time
- Self-test: page load -> upload -> cluster -> map -> node click -> edge -> export
- All pass: 600 rows, 542 nodes, 535 markers, 300 edges, 542/500 CSV rows
- Fix toBeijing to handle .00000 microsecond timestamps

Co-Authored-By: Claude <noreply@anthropic.com>
2026-07-23 22:02:11 +08:00
TianXuan Developer 66e3649152 feat: static SVG world map + large clickable points + edge detail
- Generate high-quality SVG world map (static, no tiles)
- Points: radius 7, white border, high-contrast subnet colors
- Node click: show detail panel + highlight edges on map
- Highlighted edges: clickable, show edge detail panel
- Edge detail: Beijing time, avg bytes, src/dst IP clickable
- Map init: requestAnimationFrame ensures container has dimensions
- Cleanup: remove polygon-based continent drawing code

Co-Authored-By: Claude <noreply@anthropic.com>
2026-07-23 21:58:02 +08:00
TianXuan Developer 04d49101d7 fix: robust map init via requestAnimationFrame + edge click detail panel
- Map init: requestAnimationFrame loop until container has dimensions
- Visible continent colors: darker fill (#d5dce3) and outline (#9aa8b8)
- Container background: #e8ecf2 for visibility even without data
- Edge click: L.DomEvent.stopPropagation, sticky tooltips
- Edge detail panel: Beijing time conversion, avg bytes per communication
- Save/restore edge layers when node highlighting
- Steps back to display:none (reliable, map inits via rAF)

Co-Authored-By: Claude <noreply@anthropic.com>
2026-07-23 21:47:35 +08:00
TianXuan Developer 0e31328598 fix: visible map + node-click edge highlighting
- Step visibility: use visibility:hidden instead of display:none
  so map container always has dimensions for Leaflet init
- Simplify initMap: no more retry loop, always succeeds
- Node click: show related edges in panel, highlight on map
- Edge click: click edges in panel to show edge detail
- closeDetail: remove highlighted edges, restore edge toggle
- Edge rows in panel: hover effect, click to highlight+detail

Co-Authored-By: Claude <noreply@anthropic.com>
2026-07-23 21:42:56 +08:00
TianXuan Developer e3a7cc20d0 test: Playwright E2E test for V3 simple analysis + fix initFilterBuilder
- Add comprehensive E2E test: upload → cluster → map → node/edge click → export
- Fix goStep initFilterBuilder guard (was !SA.colNames.length, now always called)
- All 9 test sections pass: upload, cluster, map, node click, edge display, edge click, dual export, charts

Co-Authored-By: Claude <noreply@anthropic.com>
2026-07-23 21:31:22 +08:00
TianXuan Developer 427890d6fb fix: robust map init — wait for container to have dimensions
- Check container.offsetHeight before initializing Leaflet map
- Retry up to 10 times (150ms intervals) until container is visible
- Fix grid line coordinate order (lat,lon)
- Add try-catch around continent drawing
- Call invalidateSize() after initialization

Co-Authored-By: Claude <noreply@anthropic.com>
2026-07-23 20:59:53 +08:00
TianXuan Developer be1150e67c fix: real continent outlines, subnet-colored markers, click interaction
- Replace rough SVG with Leaflet polygon continents (recognizable shapes)
- Color markers by /24 subnet hash instead of cluster IDs
- Remove count bubbles and cluster numbers from map
- Fix click handlers with stopPropagation + bubblingMouseEvents:false
- Tooltips show IP, subnet, city, country, ISP
- Legend shows community size + subnet info

Co-Authored-By: Claude <noreply@anthropic.com>
2026-07-23 20:41:31 +08:00
TianXuan Developer f065efab26 perf: static world map overlay + grid clustering + GeoIP status
Major performance improvements for map visualization:
- Replace tile-based map with inline SVG world map (zero network requests)
- Grid-based marker clustering: merge nearby markers into count bubbles
- Adaptive cell size by zoom level, max 300 edge lines
- Remove all tile caching code (Cache API, CachedTileLayer, etc.)
- Add /simple/geoip-status/ API endpoint
- Debug panel shows GeoIP source and MMDB hint
- min_freq slider on edge control auto-refreshes

Co-Authored-By: Claude <noreply@anthropic.com>
2026-07-23 20:32:21 +08:00
TianXuan Developer 15e4c5e699 refactor: remove noise concept — all nodes assigned to clusters
- Backend: unclusterable nodes get their own single-node cluster (never -1)
- min_cluster_size slider starts at 1 instead of 3
- Remove n_noise, all_noise, is_noise from responses and UI
- Solo clusters shown as '独立节点' with lighter color
- Remove noise toggle from legend, simplify map rendering

Co-Authored-By: Claude <noreply@anthropic.com>
2026-07-23 20:21:27 +08:00
TianXuan Developer 8ba00b1240 fix: auto-scroll to results after upload/filter/cluster
- Upload result: scrollIntoView after success
- Filter preview: scrollIntoView after apply
- Cluster result: scrollIntoView after analysis
- All action buttons stay above scrollable content areas

Co-Authored-By: Claude <noreply@anthropic.com>
2026-07-23 20:12:02 +08:00