8fa36b774e
v7: 19 issues fixed — SQLite storage, new clustering pipeline, display_id, globe rewrite, Chinese tools
...
Key changes:
- New: data_loader.py SQLite persistence with drop_sqlite_table
- New: db_utils.py retry_on_lock decorator (3 retries, exponential backoff)
- New: tool_registry.py with 27 MCP tools (filter_and_cluster, compute_scores, etc.)
- New: tls_ref.py for TLS cipher/reference data
- New: import_tlsdb.py management command
- New: scripts/start_server.py for portable runtime
- New: migrations 0003-0007 for SQLite table, display_id, llm fields
- Changed: views.py unified pipeline worker, retry_run, display_id everywhere
- Changed: models.py with display_id auto-assignment, run_type, sqlite_table, llm_thinking, tool_calls_json
- Changed: urls.py added retry_run route
- Changed: session_store.py robust JSON persistence
- Changed: AGENTS.md v7 fix summary added
- Changed: templates — globe rewrite (inertia/polar flip), auto.html (thinking/tool accordions), base.html (toast/config), all pages use display_id
- Changed: run.bat PYTHONUTF8=1
- Deleted: entity_detector.py, entity_aggregator.py (replaced by filter_and_cluster clustering pipeline)
- Test: 92/92 unit tests passing
2026-07-20 13:33:13 +08:00
763bc144ea
test: full-link E2E test - 120 unit + 8 pipeline + 10 tool integration tests pass
2026-07-19 23:19:51 +08:00
7fbf5e2720
feat: add 5 read-only analysis tools for LLM deep-dive (patterns/temporal/tls-health/geo/entity-detail)
2026-07-19 23:06:53 +08:00
aa238b9439
feat: add 7 diagnostic MCP tools for edge-case data analysis (validate/explore/outliers/diagnose/compare/export/repair)
2026-07-19 22:54:19 +08:00
27dd736443
feat: add 3 composable MCP tools (compute_scores/detect_anomalies/visualize_anomalies) for LLM-driven workflow
2026-07-19 22:51:16 +08:00
62ce147a05
perf: lazy filter _ds_idx before collect + pre-collect downsampling in _handle_run_clustering
2026-07-19 21:51:42 +08:00
aab354cc51
v4: progress bar position + sklearn warning + N+1 fix + PRAGMA opt + zero-var fallback
2026-07-17 19:07:57 +08:00
fa305515c1
fix: D5 downsampling index mismatch + add 1M-row medium-scale test to docs
2026-07-17 14:02:30 +08:00
548ff8ca1b
perf: replace per-row EntityProfile save with bulk_create (views.py + tool_registry.py)
2026-07-17 13:28:27 +08:00
b1d7393921
deep merge: unify 3 pipeline flows + fix DB save race + add PCA to manual flow
...
- Create shared _run_clustering_pipeline() in views.py
- Manual POST, CLI, and LLM flows all call the same function
- Consistent asyncio.run() tool calls throughout (_handle_run_clustering,
_handle_extract_features) - removed _cluster_sync
- Add PCA-2D embedding to manual flow (was CLI-only)
- Fix MultipleObjectsReturned in _handle_extract_features DB save:
pass run_id from pipeline → use get() instead of get_or_create()
- 120/120 tests passing, CLI pipeline clean
2026-07-16 23:08:26 +08:00
b58d01c7f2
cleanup: remove v6 garbage, restore original files, unify views.py workers
...
- Delete 15 garbage scripts (wnl_to_user_csv, eval_clustering, debug_db2/3,
debug_persistence, test_orch/orch2, test_pca, test_profile, test_tool_*,
test_e2e_full, test_32b, test_chinese_path, verify_runtime)
- Delete data/wnl_converted.csv
- Delete analysis/value_normalizer.py + tests/test_value_normalizer.py
- Remove normalize_lf references from data_loader.py
- Remove _norm column lookups from entity_aggregator.py
- Fix settings.py: FILE_UPLOAD_MAX_MEMORY_SIZE 10MB -> 100MB
- Fix upload.html: add per-file X delete button + removeFile()
- Refactor views.py: unify _run_analysis_worker/_run_llm_analysis into
_run_pipeline_worker with callback pattern
- Wrap sync ORM call in tool_registry.py with sync_to_async
- Update run_pipeline.py to use _run_pipeline_worker
- Rebuild gen_test_data.py as simple version (replace 3-profile v6 version)
- Add runtime/ db.sqlite3 logs/ .omo/ to .gitignore
- Delete old .omo/plans/ plan files (tianxiu-v4/v5/v6, globe-v6)
- 120/120 tests passing
2026-07-16 22:53:07 +08:00
814f29645a
chore: initial commit of 天璇 project state before cleanup
2026-07-16 22:41:26 +08:00
PM-pinou