HJQ/tianxuan
2
1
Fork 0
Code Issues Pull Requests 2 Actions Packages Projects Releases Wiki Activity
178 Commits 5 Branches 0 Tags
master
Commit Graph

178 Commits

This Branch
This Branch
All Branches
Author SHA1 Message Date
PM-pinou 2a64f554c4 fix: Wave 1 - 3 MCP crashes + delete whitelist + exception propagation 2026-07-22 20:41:25 +08:00
PM-pinou d9b79c74b4 chore: freeze requirements from runtime Python 3.12.2 (59 packages) 2026-07-22 20:31:37 +08:00
PM-pinou 763fdbf605 chore: freeze requirements from runtime Python 3.12.2 (59 packages) 2026-07-22 20:31:09 +08:00
PM-pinou 0baffe4d9e update AGENTS.md 2026-07-22 19:58:50 +08:00
PM-pinou 7aa847ba3e v1.1.6: cleanup — schema sort fix, batch upload, temp dir to D:, update.bat wildcard fix 2026-07-22 19:10:39 +08:00
PM-pinou e4689ed0a9 fix: batch upload no longer re-triggers _background_process to avoid DROP TABLE race 2026-07-22 14:01:56 +08:00
PM-pinou a59c153f4c fix: reset VERSION to v1.1.6 2026-07-22 13:59:29 +08:00
PM-pinou db0e473119 chore: bump to v1.1.7 2026-07-22 13:58:33 +08:00
PM-pinou edd942695b Revert "chore: bump to v1.1.7" 
This reverts commit 1ab655d17940aa0905c031753b89d67fb51a61e2.
 2026-07-22 13:58:29 +08:00
PM-pinou ef27ddde4e chore: bump to v1.1.7 2026-07-22 13:58:22 +08:00
PM-pinou 8c90bad9f0 v1.1.6: batch upload BATCH=5 + temp dir to D: + schema sort + update.bat fix 2026-07-22 13:58:19 +08:00
PM-pinou 2b3f10a03c fix: set FILE_UPLOAD_TEMP_DIR to D: drive to avoid ENOSPC on C: 2026-07-22 13:50:18 +08:00
PM-pinou f18f0d84cf fix: reduce upload batch size to 5 files 2026-07-22 13:48:23 +08:00
PM-pinou 9ca5da1237 release: v1.1.6 — update.bat wildcard fix + schema column-order fix + gen_test format alignment + pipeline verify 2026-07-22 11:39:35 +08:00
PM-pinou db82f58ff2 fix: reset VERSION to v1.1.4 2026-07-21 22:33:17 +08:00
PM-pinou 4a0296761b chore: bump to v1.1.5 2026-07-21 22:33:03 +08:00
PM-pinou e44e4ead08 fix: reset VERSION to v1.1.4 (post-build) 2026-07-21 22:15:15 +08:00
PM-pinou c75a423261 chore: bump to v1.1.5 2026-07-21 22:15:02 +08:00
PM-pinou d8eadafc83 auto: update for vv1.1.4 2026-07-21 22:14:55 +08:00
PM-pinou 4629416413 fix: gitignore data CSV files and untrack test data 2026-07-21 22:09:03 +08:00
PM-pinou 61f0174781 chore: bump to v1.1.5 2026-07-21 21:48:41 +08:00
PM-pinou 49f825175c auto: update for vv1.1.4 2026-07-21 21:47:45 +08:00
PM-pinou 8ee538492d release: v1.1.4 2026-07-21 21:40:56 +08:00
PM-pinou 6a633016d2 v1.1.4: gen_test reads TlsDB coverage + schema fixes 
- gen_test_data.py: load_coverage() from TlsDB.csv, replaced all hardcoded _blank_or rates with coverage lookups
- gen_multi.py: coverage-based per-file column dropping
- collect_schema: force Utf8 for cross-file compatibility
- _background_process: pre-scan union columns, single CREATE TABLE, no ALTER TABLE
 2026-07-21 19:34:14 +08:00
PM-pinou 21b28823e3 fix: reset VERSION to v1.1.3 2026-07-21 14:17:49 +08:00
PM-pinou 929b6aca2f chore: bump to v1.1.5 2026-07-21 14:16:14 +08:00
PM-pinou e8e93a0fc8 chore: bump to v1.1.4 2026-07-21 14:16:08 +08:00
PM-pinou d20b1c3b28 v1.1.3: default server 0.0.0.0:80 + row-level clustering + filter UI + dataset selector 2026-07-21 14:15:56 +08:00
PM-pinou ec83d5df85 v1.1.3: row-level clustering + dataset selector with status filter 
- Add cluster_mode toggle (raw row-level vs entity-aggregated) in manual.html
- manual_run_analysis supports cluster_mode='raw' skipping entity aggregation
- Replace simple dataset dropdown with selector showing status badges and filter tabs
- Filter tabs: 全部/待分析/已分析/错误, default 待分析
- E2E verified: raw mode entity_count=total_flows, entity mode regression OK
 2026-07-21 14:00:18 +08:00
PM-pinou 9de1d93146 fix: entity_count fallback + globe SQLite loading 
- entity_count: fallback from ClusterResult sizes when PCA is skipped
- globe_view: load from SQLite table (_data_N) instead of csv_glob path
 2026-07-21 13:26:50 +08:00
PM-pinou 745b1481cd v1.1.3: entity_count fix + geoip database + multi-file test data + csv_glob display fix 
- Fix entity_count not populated after clustering pipeline
- Create comprehensive geoip_data.txt (803 IP ranges, 49 cities)
- Generate 100 multi_upload test files with missing columns
- Replace csv_glob path with SQLite table name in frontend
- Fix globe zoom-drag point-fixing (camera.z/12.0 scaling)
- Fix orphaned manual_run_analysis function
 2026-07-21 13:21:20 +08:00
PM-pinou 917e065609 fix: globe zoom-drag point fixing + pipeline manual_run_analysis bug 
- Globe drag now scales rotation by camera.z/12.0 so the point under cursor stays fixed at all zoom levels (drag AND inertia)
- Fixed orphaned manual_run_analysis function that was trapped inside _run_clustering_pipeline's except handler, causing NameError
- Updated urls.py route to correct function name
 2026-07-21 12:52:44 +08:00
PM-pinou 3617b7aabc release: v1.1.3 — 13 fixes across build, frontend, pipeline, and data handling 
Build: fix incremental package (update_info.json SHA256, __delete__, test exclude, files/ prefix), fix bump_version for 3-segment, PYTHONUTF8 in update.bat
Frontend: upload→manual redirect, LLM→cluster_overview redirect, filter UI before clustering, 地球→态势 rename, fix globe drag direction+inertia
Pipeline: LLM→auto clustering+PCA, column union handling (keep extra cols, no dropping), FFT spectral analysis tool
Quality: fix 24 bare except:pass patterns, enrich LLM context with TlsDB metadata, auto-save workflows with pinning, TlsDB-based column types
 2026-07-21 12:01:52 +08:00
PM-pinou 9314ac6942 chore: bump version to v1.1.2 
- New timeline UI: interleaved LLM reasoning + tool call display
- Fix: run_log truncation for large tool outputs
- Fix: lat/lon + values reduced to ~1% (was 99%)
- Fix: @csrf_exempt on upload/delete endpoints
- Fix: DATA_UPLOAD_MAX_NUMBER_FILES for 1998-file upload
- Fix: dtype alignment for multi-batch CSV processing
- New: tests/test_e2e_full.py with real DeepSeek LLM
- New: 1998-file test data generator
 2026-07-20 23:32:32 +08:00
PM-pinou 406bf81600 AGENTS.md: add v8 fix table + final verification 2026-07-20 19:41:12 +08:00
PM-pinou efc6e3ab9d v8: fix server startup, CSV parse, upload flow, load_from_db types, integration test 2026-07-20 19:40:49 +08:00
PM-pinou 7fe14a3f05 chore: bump version to v1.1.1 
Ultraworked with [Sisyphus](https://github.com/code-yeongyu/oh-my-openagent)

Co-authored-by: Sisyphus <clio-agent@sisyphuslabs.ai>
 2026-07-20 18:46:03 +08:00
PM-pinou 7a670fd5e2 feat(scripts): add update.bat and rollback.bat launchers 
Add user-facing launchers for the incremental update system:
- update.bat: one-click update launcher
- rollback.bat: rollback to previous version
- tests/test_update_integration.py: integration tests for the update system
- README.md: add incremental update guide (section 10)
- .gitignore: add .update_cache/ entry

Ultraworked with [Sisyphus](https://github.com/code-yeongyu/oh-my-openagent)

Co-authored-by: Sisyphus <clio-agent@sisyphuslabs.ai>
 2026-07-20 14:14:17 +08:00
PM-pinou 00774ea52b feat(scripts): add incremental update system with version tracking and diff utilities 
Add complete incremental update infrastructure:
- VERSION: project version tracking file
- scripts/version_utils.py: shared version parsing and comparison utilities
- scripts/build_update.py: build incremental update package from git diff
- scripts/apply_update.py: apply offline incremental update
- scripts/merge_config.py: safe config merge logic for update safety
- scripts/migrate_db_to_appdata.py: migrate DB from project dir to APPDATA

Ultraworked with [Sisyphus](https://github.com/code-yeongyu/oh-my-openagent)

Co-authored-by: Sisyphus <clio-agent@sisyphuslabs.ai>
 2026-07-20 14:14:12 +08:00
PM-pinou 0b6672781f feat(settings): migrate db.sqlite3 to APPDATA dir, extract shared appdata.py 
Migrate database path from project-local db.sqlite3 to %%APPDATA%%/TianXuan/ so data persists across updates.
- tianxuan/settings.py: DB path to APPDATA
- tianxuan/wsgi.py: use settings.DATABASES path instead of hardcoded db.sqlite3
- analysis/appdata.py: extract shared APPDATA path resolution
- analysis/session_store.py: use appdata.py for session store path
- analysis/views.py: use appdata.py for view paths

Ultraworked with [Sisyphus](https://github.com/code-yeongyu/oh-my-openagent)

Co-authored-by: Sisyphus <clio-agent@sisyphuslabs.ai>
 2026-07-20 14:14:07 +08:00
PM-pinou 8fa36b774e v7: 19 issues fixed — SQLite storage, new clustering pipeline, display_id, globe rewrite, Chinese tools 
Key changes:
- New: data_loader.py SQLite persistence with drop_sqlite_table
- New: db_utils.py retry_on_lock decorator (3 retries, exponential backoff)
- New: tool_registry.py with 27 MCP tools (filter_and_cluster, compute_scores, etc.)
- New: tls_ref.py for TLS cipher/reference data
- New: import_tlsdb.py management command
- New: scripts/start_server.py for portable runtime
- New: migrations 0003-0007 for SQLite table, display_id, llm fields
- Changed: views.py unified pipeline worker, retry_run, display_id everywhere
- Changed: models.py with display_id auto-assignment, run_type, sqlite_table, llm_thinking, tool_calls_json
- Changed: urls.py added retry_run route
- Changed: session_store.py robust JSON persistence
- Changed: AGENTS.md v7 fix summary added
- Changed: templates — globe rewrite (inertia/polar flip), auto.html (thinking/tool accordions), base.html (toast/config), all pages use display_id
- Changed: run.bat PYTHONUTF8=1
- Deleted: entity_detector.py, entity_aggregator.py (replaced by filter_and_cluster clustering pipeline)
- Test: 92/92 unit tests passing
 2026-07-20 13:33:13 +08:00
PM-pinou 02f18944d5 fix: manual page dataset selector uses dataset_id not ds.id 2026-07-19 23:33:06 +08:00
PM-pinou 7af05ab97a docs: update AGENTS.md with MCP tool system + cleanup 2026-07-19 23:20:29 +08:00
PM-pinou 763bc144ea test: full-link E2E test - 120 unit + 8 pipeline + 10 tool integration tests pass 2026-07-19 23:19:51 +08:00
PM-pinou 6e89107a30 feat: save/load workflow plans + run all sequentially + step reorder (manual workflow builder) 2026-07-19 23:15:50 +08:00
PM-pinou 4fb1e4fdda refactor: integrate 27 MCP tools into manual analysis workflow builder (remove separate tool_lab) 2026-07-19 23:12:16 +08:00
PM-pinou 06db287c0e feat: MCP tool lab page + workflow docs + nav link 2026-07-19 23:09:13 +08:00
PM-pinou 7fbf5e2720 feat: add 5 read-only analysis tools for LLM deep-dive (patterns/temporal/tls-health/geo/entity-detail) 2026-07-19 23:06:53 +08:00
PM-pinou 8781e8b135 feat: rewrite LLM orchestrator with strategy prompt, auto error recovery, duplicate call detection, and result truncation 2026-07-19 23:02:40 +08:00
PM-pinou aa238b9439 feat: add 7 diagnostic MCP tools for edge-case data analysis (validate/explore/outliers/diagnose/compare/export/repair) 2026-07-19 22:54:19 +08:00