9f06e40a41
refactor: clean views/clustering.py and split data_loader.py into package
...
Task A: Replace _run_clustering_pipeline (533L->228L) — remove dead
docstring code, delegate to analysis.services.clustering.
Task B: Split analysis/data_loader.py (746L) into package:
analysis/data_loader/
__init__.py — re-exports all public symbols
_csv.py — load_csv_directory, BOM/encoding detection
_schema.py — schema merging and validation
_clean.py — numeric coercion helper
_sqlite.py — save_to_db, load_from_db, load_from_db_lazy, etc.
Keep old data_loader.py as thin re-export wrapper.
Verified: manage.py check (0 issues), run_pipeline (3 clusters, OK)
2026-07-24 13:52:07 +08:00
c54917791e
cleanup: remove dead backup files, orphaned static files, fix merge conflict markers
2026-07-24 13:43:49 +08:00
3fd3896aba
merge: service layer + frontend static extraction — resolve clustering.py conflict
2026-07-24 13:30:59 +08:00
66f68a2062
refactor(services): extract clustering pipeline from views/ to analysis/services/
...
Move _run_clustering_pipeline (~295 lines) from analysis/views/clustering.py
into analysis/services/clustering.py as three clean functions:
- run_clustering_pipeline: main pipeline orchestration
- compute_umap_embedding: UMAP 3D→2D fallback computation
- save_entity_profiles: persist cluster labels + UMAP coords to ORM
Views/clustering.py now delegates via thin wrapper. Callers (auto.py,
manual.py, run_pipeline.py) import directly from services.clustering.
2026-07-24 13:27:11 +08:00
244967bad5
fix(cluster_detail): show raw data rows instead of entity profiles
2026-07-24 11:58:31 +08:00
d2e34ca43d
fix(cluster_detail): always persist per-row cluster labels to EntityProfile for data row display
2026-07-24 11:56:02 +08:00
2c1aad6bd1
feat(run_detail): global SVD feature analysis + LLM workflow timeline
2026-07-24 11:51:23 +08:00
5280ce5819
fix(umap): LazyFrame.sample not available in Polars 1.42, use DataFrame.sample after collect
2026-07-24 11:39:00 +08:00
e2c64fd068
fix: raw row clustering — remove entity concept, use row index, all numeric features
2026-07-24 11:35:18 +08:00
ef60e3e489
cleanup: consolidate TLS_HEX_MAP, extract constants, remove dead code, add traceback logging
...
3.1 Unified TLS_HEX_MAP: defined once in type_classifier.py, imported in globe.py and clustering.py
3.2 Created analysis/constants.py: centralized RANDOM_SEED, MAX_SAMPLE_ROWS, UMAP constants, GLOBE_MAX_ROWS, etc.
3.3 Removed 14 pointless try/except:raise dead code blocks across tools/ and views/
3.4 Added logger.error(traceback) to all bare except Exception: blocks in tools/ and views/
3.5 Moved inline import traceback/asyncio to top-level imports in features.py, clustering.py, pipeline.py
3.6 Removed auto_profile_module side-effect imports from 6 modules
2026-07-23 22:44:42 +08:00
d4c82768a8
refactor: split analysis/views.py into views/ package (12 sub-modules)
...
- helpers.py: _extract_lat, _extract_lon, plan index utilities
- dashboard.py: dashboard, run_list, run_detail, run_status_api, start_analysis
- pipeline.py: _run_pipeline_worker, _background_process
- clustering.py: cluster_overview, cluster_detail, _run_clustering_pipeline, _get_globe_flows
- entity.py: entity_profile
- upload.py: upload_page, upload_csv, upload_csv_batch, finalize_upload, delete_upload
- auto.py: auto_page, run_llm_analysis_view
- manual.py: manual_page, manual_run_analysis
- globe.py: globe_view, _extract_flows_from_df
- config.py: config_view, llm_test
- log_viewer.py: log_viewer
- tools.py: tool_lab, tool_lab_run, tool_plan, apply_filter, reload_run_data, retry_run
All function logic preserved exactly. views/__init__.py re-exports all public
functions for backward compatibility with urls.py and run_pipeline.py.
Original views.py archived as views_backup.py.
2026-07-23 22:22:28 +08:00
PM-pinou