d20b1c3b28
v1.1.3: default server 0.0.0.0:80 + row-level clustering + filter UI + dataset selector
2026-07-21 14:15:56 +08:00
ec83d5df85
v1.1.3: row-level clustering + dataset selector with status filter
...
- Add cluster_mode toggle (raw row-level vs entity-aggregated) in manual.html
- manual_run_analysis supports cluster_mode='raw' skipping entity aggregation
- Replace simple dataset dropdown with selector showing status badges and filter tabs
- Filter tabs: 全部/待分析/已分析/错误, default 待分析
- E2E verified: raw mode entity_count=total_flows, entity mode regression OK
2026-07-21 14:00:18 +08:00
9de1d93146
fix: entity_count fallback + globe SQLite loading
...
- entity_count: fallback from ClusterResult sizes when PCA is skipped
- globe_view: load from SQLite table (_data_N) instead of csv_glob path
2026-07-21 13:26:50 +08:00
745b1481cd
v1.1.3: entity_count fix + geoip database + multi-file test data + csv_glob display fix
...
- Fix entity_count not populated after clustering pipeline
- Create comprehensive geoip_data.txt (803 IP ranges, 49 cities)
- Generate 100 multi_upload test files with missing columns
- Replace csv_glob path with SQLite table name in frontend
- Fix globe zoom-drag point-fixing (camera.z/12.0 scaling)
- Fix orphaned manual_run_analysis function
2026-07-21 13:21:20 +08:00
917e065609
fix: globe zoom-drag point fixing + pipeline manual_run_analysis bug
...
- Globe drag now scales rotation by camera.z/12.0 so the point under cursor stays fixed at all zoom levels (drag AND inertia)
- Fixed orphaned manual_run_analysis function that was trapped inside _run_clustering_pipeline's except handler, causing NameError
- Updated urls.py route to correct function name
2026-07-21 12:52:44 +08:00
3617b7aabc
release: v1.1.3 — 13 fixes across build, frontend, pipeline, and data handling
...
Build: fix incremental package (update_info.json SHA256, __delete__, test exclude, files/ prefix), fix bump_version for 3-segment, PYTHONUTF8 in update.bat
Frontend: upload→manual redirect, LLM→cluster_overview redirect, filter UI before clustering, 地球→态势 rename, fix globe drag direction+inertia
Pipeline: LLM→auto clustering+PCA, column union handling (keep extra cols, no dropping), FFT spectral analysis tool
Quality: fix 24 bare except:pass patterns, enrich LLM context with TlsDB metadata, auto-save workflows with pinning, TlsDB-based column types
2026-07-21 12:01:52 +08:00
9314ac6942
chore: bump version to v1.1.2
...
- New timeline UI: interleaved LLM reasoning + tool call display
- Fix: run_log truncation for large tool outputs
- Fix: lat/lon + values reduced to ~1% (was 99%)
- Fix: @csrf_exempt on upload/delete endpoints
- Fix: DATA_UPLOAD_MAX_NUMBER_FILES for 1998-file upload
- Fix: dtype alignment for multi-batch CSV processing
- New: tests/test_e2e_full.py with real DeepSeek LLM
- New: 1998-file test data generator
2026-07-20 23:32:32 +08:00
406bf81600
AGENTS.md: add v8 fix table + final verification
2026-07-20 19:41:12 +08:00
efc6e3ab9d
v8: fix server startup, CSV parse, upload flow, load_from_db types, integration test
2026-07-20 19:40:49 +08:00
7fe14a3f05
chore: bump version to v1.1.1
...
Ultraworked with [Sisyphus](https://github.com/code-yeongyu/oh-my-openagent )
Co-authored-by: Sisyphus <clio-agent@sisyphuslabs.ai >
2026-07-20 18:46:03 +08:00
7a670fd5e2
feat(scripts): add update.bat and rollback.bat launchers
...
Add user-facing launchers for the incremental update system:
- update.bat: one-click update launcher
- rollback.bat: rollback to previous version
- tests/test_update_integration.py: integration tests for the update system
- README.md: add incremental update guide (section 10)
- .gitignore: add .update_cache/ entry
Ultraworked with [Sisyphus](https://github.com/code-yeongyu/oh-my-openagent )
Co-authored-by: Sisyphus <clio-agent@sisyphuslabs.ai >
2026-07-20 14:14:17 +08:00
00774ea52b
feat(scripts): add incremental update system with version tracking and diff utilities
...
Add complete incremental update infrastructure:
- VERSION: project version tracking file
- scripts/version_utils.py: shared version parsing and comparison utilities
- scripts/build_update.py: build incremental update package from git diff
- scripts/apply_update.py: apply offline incremental update
- scripts/merge_config.py: safe config merge logic for update safety
- scripts/migrate_db_to_appdata.py: migrate DB from project dir to APPDATA
Ultraworked with [Sisyphus](https://github.com/code-yeongyu/oh-my-openagent )
Co-authored-by: Sisyphus <clio-agent@sisyphuslabs.ai >
2026-07-20 14:14:12 +08:00
0b6672781f
feat(settings): migrate db.sqlite3 to APPDATA dir, extract shared appdata.py
...
Migrate database path from project-local db.sqlite3 to %%APPDATA%%/TianXuan/ so data persists across updates.
- tianxuan/settings.py: DB path to APPDATA
- tianxuan/wsgi.py: use settings.DATABASES path instead of hardcoded db.sqlite3
- analysis/appdata.py: extract shared APPDATA path resolution
- analysis/session_store.py: use appdata.py for session store path
- analysis/views.py: use appdata.py for view paths
Ultraworked with [Sisyphus](https://github.com/code-yeongyu/oh-my-openagent )
Co-authored-by: Sisyphus <clio-agent@sisyphuslabs.ai >
2026-07-20 14:14:07 +08:00
8fa36b774e
v7: 19 issues fixed — SQLite storage, new clustering pipeline, display_id, globe rewrite, Chinese tools
...
Key changes:
- New: data_loader.py SQLite persistence with drop_sqlite_table
- New: db_utils.py retry_on_lock decorator (3 retries, exponential backoff)
- New: tool_registry.py with 27 MCP tools (filter_and_cluster, compute_scores, etc.)
- New: tls_ref.py for TLS cipher/reference data
- New: import_tlsdb.py management command
- New: scripts/start_server.py for portable runtime
- New: migrations 0003-0007 for SQLite table, display_id, llm fields
- Changed: views.py unified pipeline worker, retry_run, display_id everywhere
- Changed: models.py with display_id auto-assignment, run_type, sqlite_table, llm_thinking, tool_calls_json
- Changed: urls.py added retry_run route
- Changed: session_store.py robust JSON persistence
- Changed: AGENTS.md v7 fix summary added
- Changed: templates — globe rewrite (inertia/polar flip), auto.html (thinking/tool accordions), base.html (toast/config), all pages use display_id
- Changed: run.bat PYTHONUTF8=1
- Deleted: entity_detector.py, entity_aggregator.py (replaced by filter_and_cluster clustering pipeline)
- Test: 92/92 unit tests passing
2026-07-20 13:33:13 +08:00
02f18944d5
fix: manual page dataset selector uses dataset_id not ds.id
2026-07-19 23:33:06 +08:00
7af05ab97a
docs: update AGENTS.md with MCP tool system + cleanup
2026-07-19 23:20:29 +08:00
763bc144ea
test: full-link E2E test - 120 unit + 8 pipeline + 10 tool integration tests pass
2026-07-19 23:19:51 +08:00
6e89107a30
feat: save/load workflow plans + run all sequentially + step reorder (manual workflow builder)
2026-07-19 23:15:50 +08:00
4fb1e4fdda
refactor: integrate 27 MCP tools into manual analysis workflow builder (remove separate tool_lab)
2026-07-19 23:12:16 +08:00
06db287c0e
feat: MCP tool lab page + workflow docs + nav link
2026-07-19 23:09:13 +08:00
7fbf5e2720
feat: add 5 read-only analysis tools for LLM deep-dive (patterns/temporal/tls-health/geo/entity-detail)
2026-07-19 23:06:53 +08:00
8781e8b135
feat: rewrite LLM orchestrator with strategy prompt, auto error recovery, duplicate call detection, and result truncation
2026-07-19 23:02:40 +08:00
aa238b9439
feat: add 7 diagnostic MCP tools for edge-case data analysis (validate/explore/outliers/diagnose/compare/export/repair)
2026-07-19 22:54:19 +08:00
27dd736443
feat: add 3 composable MCP tools (compute_scores/detect_anomalies/visualize_anomalies) for LLM-driven workflow
2026-07-19 22:51:16 +08:00
2caa09ed21
feat: adaptive anomaly scoring via PCA weight learning + IQR thresholds (no hardcoded params)
2026-07-19 22:44:20 +08:00
62ce147a05
perf: lazy filter _ds_idx before collect + pre-collect downsampling in _handle_run_clustering
2026-07-19 21:51:42 +08:00
6ed109dbfe
perf: PCA downsample to 50K for 10M+ entities + directory upload via webkitdirectory
2026-07-19 21:46:34 +08:00
b90dcbf11c
fix: cluster charts cap at 50 bars + silhouette label fix + manual page progress bar
2026-07-19 21:36:51 +08:00
3e98acecf9
fix: lower arc to R=5.06 e=0.65 (clearance 0.06, not floating)
2026-07-19 21:24:53 +08:00
3c63fe9d23
fix: globe run selection fallback + arc start height 5.2 / ecc 0.85 to avoid earth intersection
2026-07-18 21:51:26 +08:00
3664d2d8ee
fix: use direct parametric ellipse equation for trail sphere animation
2026-07-18 21:44:57 +08:00
b1bd7ad5bb
fix: replace QuadraticBezierCurve with centered elliptical arc (no earth intersection)
2026-07-18 21:43:56 +08:00
7fd081d991
fix: add 11 missing TlsDB columns to type_classifier + entity_aggregator
2026-07-18 21:05:44 +08:00
f4c45e9ec8
chore: add WAL files to gitignore, clean up temp test files
2026-07-18 20:57:35 +08:00
e2e09b1efd
fix: globe view skip missing files + TLS/bytes column matching for 0ver/8ack
2026-07-18 20:57:21 +08:00
aab354cc51
v4: progress bar position + sklearn warning + N+1 fix + PRAGMA opt + zero-var fallback
2026-07-17 19:07:57 +08:00
fa305515c1
fix: D5 downsampling index mismatch + add 1M-row medium-scale test to docs
2026-07-17 14:02:30 +08:00
70e2167fea
docs: update AGENTS.md with v3 fix status for all 7 issues
2026-07-17 13:33:12 +08:00
45d9f508ab
feat: add proxy/anomaly detection scoring + retry analysis for failed runs
2026-07-17 13:33:02 +08:00
548ff8ca1b
perf: replace per-row EntityProfile save with bulk_create (views.py + tool_registry.py)
2026-07-17 13:28:27 +08:00
4e6339253d
fix(globe): arc sphere size independent of camera zoom using inverse distance scaling
2026-07-17 13:27:06 +08:00
1af5704796
fix(cluster_overview): responsive charts with adaptive bandwidth and overflow-safe legend
2026-07-17 13:25:08 +08:00
03be495593
fix: lat/lon column name matching (:ips.latd) + cnrs/isrs boolean conversion
2026-07-17 13:22:34 +08:00
2bb3b7273a
xiufu: fix globe ?data=, add pipeline tests, update settings, rewrite AGENTS.md
...
- Fix globe_view ?data= 500 error (try/except safety + DATA_UPLOAD_MAX_NUMBER_FIELDS=1000000)
- Add test_pipeline.py (8 tests: 3 datasets × 2 algorithms + edge cases)
- Update AGENTS.md with all 18 issue statuses
- Clean up server processes
2026-07-16 23:54:19 +08:00
b271626abc
feat: add downsampling support (head parameter for manual + LLM)
2026-07-16 23:35:10 +08:00
50f4d025ce
feat: SessionStore persistence to JSON file + startup auto recovery
2026-07-16 23:33:19 +08:00
29e21a6a23
fix: remove tb[:200] truncation, add TLS_HEX_MAP to type_classifier, fix + FLOAT detection
2026-07-16 23:31:35 +08:00
b1d7393921
deep merge: unify 3 pipeline flows + fix DB save race + add PCA to manual flow
...
- Create shared _run_clustering_pipeline() in views.py
- Manual POST, CLI, and LLM flows all call the same function
- Consistent asyncio.run() tool calls throughout (_handle_run_clustering,
_handle_extract_features) - removed _cluster_sync
- Add PCA-2D embedding to manual flow (was CLI-only)
- Fix MultipleObjectsReturned in _handle_extract_features DB save:
pass run_id from pipeline → use get() instead of get_or_create()
- 120/120 tests passing, CLI pipeline clean
2026-07-16 23:08:26 +08:00
75af722f24
fix gen_test_data column count, cleanup temp data, add .gitignore for runtime/
2026-07-16 22:56:23 +08:00
b58d01c7f2
cleanup: remove v6 garbage, restore original files, unify views.py workers
...
- Delete 15 garbage scripts (wnl_to_user_csv, eval_clustering, debug_db2/3,
debug_persistence, test_orch/orch2, test_pca, test_profile, test_tool_*,
test_e2e_full, test_32b, test_chinese_path, verify_runtime)
- Delete data/wnl_converted.csv
- Delete analysis/value_normalizer.py + tests/test_value_normalizer.py
- Remove normalize_lf references from data_loader.py
- Remove _norm column lookups from entity_aggregator.py
- Fix settings.py: FILE_UPLOAD_MAX_MEMORY_SIZE 10MB -> 100MB
- Fix upload.html: add per-file X delete button + removeFile()
- Refactor views.py: unify _run_analysis_worker/_run_llm_analysis into
_run_pipeline_worker with callback pattern
- Wrap sync ORM call in tool_registry.py with sync_to_async
- Update run_pipeline.py to use _run_pipeline_worker
- Rebuild gen_test_data.py as simple version (replace 3-profile v6 version)
- Add runtime/ db.sqlite3 logs/ .omo/ to .gitignore
- Delete old .omo/plans/ plan files (tianxiu-v4/v5/v6, globe-v6)
- 120/120 tests passing
2026-07-16 22:53:07 +08:00
PM-pinou