Catty Steve
26809df720
- Add workshop-schedule crate (petgraph-based DAG scheduling) - Implement all bake decorators with combinator chain execution - Refactor notify module: extract handler, rule, template, util, queue - Add NotificationQueue with priority/drain semantics - Enhance finalize plugin system with internal plugin registration - Update ExecutionContext and event types - Add per-crate AGENTS.md knowledge base files - Remove inline AGENTS.md files (consolidated into per-crate docs)
81 lines
5.7 KiB
Markdown
81 lines
5.7 KiB
Markdown
# WORKSHOP-BAKER CRATE KNOWLEDGE BASE
|
|
|
|
## OVERVIEW
|
|
Primary crate — the only crate with a `[[bin]]` target. Contains the `workshop-baker` binary and `workshop_baker` library. Orchestrates the full CI/CD pipeline: prebake (env setup) → bake (script execution) → finalize (plugins, cleanup) → drain notification queue. ~69 .rs source files.
|
|
|
|
## STRUCTURE
|
|
```
|
|
src/
|
|
├── main.rs # Entry: default(full pipeline) | bare(dev-only) | daemon(unimplemented)
|
|
├── lib.rs # Clap Cli/Commands/BareCommands — CLI structs in lib, not bin
|
|
├── error.rs # CliError enum (wraps stage errors, delegates exit_code)
|
|
├── bare.rs # Dev-only single-stage execution, ctx.privileged always false
|
|
├── utils.rs # Cross-cutting helpers
|
|
├── bake/ # Script parsing + DAG scheduling + execution
|
|
│ ├── decorator.rs # @decorator(args) parsing, 81 tests — heaviest coverage in crate
|
|
│ ├── parser.rs # Step/function extraction
|
|
│ ├── schedule.rs # DAG build → topological sort
|
|
│ ├── execute.rs # Per-step timeout/retry
|
|
│ ├── builder.rs # Script assembly
|
|
│ ├── trivial.rs # Trivial step detection (skip in dry-run)
|
|
│ ├── constant.rs / error.rs / util.rs
|
|
├── prebake/ # Environment provisioning
|
|
│ ├── config.rs # PrebakeConfig deserialization
|
|
│ ├── stage.rs + stage/{bootstrap,depssystem,depsuser,environment,hook}.rs
|
|
│ ├── env.rs + env/{container,firecracker,baremetal,custom}.rs # Builder dispatch
|
|
│ ├── security.rs # Privilege/sandbox checks
|
|
│ ├── event.rs # Event sender/receiver channel
|
|
│ ├── types.rs + types/{memsize,cache,builderconfig,architecture}.rs
|
|
│ ├── constant.rs / error.rs
|
|
├── finalize/ # Post-build hooks + plugins
|
|
│ ├── config.rs # FinalizeConfig
|
|
│ ├── stage.rs + stage/hook.rs
|
|
│ ├── template.rs # Output templating (minijinja)
|
|
│ ├── plugin.rs + plugin/{shell,dylib,rhai,internal}.rs
|
|
│ │ └── plugin/internal/{dummy,mail,satori,webhook,insitenotify}.rs
|
|
│ ├── types.rs + types/compression.rs
|
|
│ ├── constant.rs / error.rs / event.rs
|
|
├── notify/ # Async notification dispatch
|
|
│ ├── queue.rs # NotificationQueue — drain on pipeline complete
|
|
│ ├── handler.rs / rule.rs / template.rs / types.rs / util.rs / error.rs
|
|
└── types/ # Shared types — ZERO deps on stage modules
|
|
├── resource.rs # ResourceRegistry (populated pre-pipeline, consumed read-only)
|
|
└── buildstatus.rs
|
|
```
|
|
|
|
## WHERE TO LOOK
|
|
|
|
| Task | Location | Notes |
|
|
|------|----------|-------|
|
|
| CLI args + subcommands | `src/lib.rs` | Cli, Commands(Bare/Daemon), BareCommands |
|
|
| Pipeline orchestration | `src/main.rs` | worker_main(): prebake→bake→finalize→queue.drain() |
|
|
| Decorator syntax | `src/bake/decorator.rs` | `# @decorator` or `# @decorator(args)`, line-by-line regex |
|
|
| Build execution | `src/bake/execute.rs` | timeout, retry, exit code capture |
|
|
| DAG scheduling | `src/bake/schedule.rs` | Uses workshop-schedule crate |
|
|
| Docker builder | `src/prebake/env/container.rs` | Bollard-based, buildkit support |
|
|
| Firecracker / Bare metal | `.../firecracker.rs`, `.../baremetal.rs` | MicroVM;
|
|
needs PIPELINE_BAREMETAL_ELEVATE |
|
|
| Plugin dispatch | `src/finalize/plugin.rs` | Dispatches shell / dylib(stub) / rhai(stub) / internal |
|
|
| Notification queue | `src/notify/queue.rs` | Spawned early, drained after pipeline, retry support |
|
|
| Error type | `src/error.rs` | CliError wraps PrebakeError/BakeError/FinalizeError/anyhow |
|
|
| Dev bare mode | `src/bare.rs` | ctx.privileged=false always; do not extrapolate to production |
|
|
|
|
## DEPENDENCIES
|
|
**Internal:** workshop-engine, workshop-schedule, workshop-getterurl, workshop-baker-params. **External:** bollard (Docker), clap (CLI), axum 0.7 (HTTP), lettre 0.11 (email), minijinja (templates), cgroups-rs (limits), privdrop.
|
|
|
|
## CONVENTIONS (CRATE-SPECIFIC)
|
|
- **Lint overrides** (this crate only): `dead_code=allow`, `unreachable_code=allow`, `inherent_to_string=allow`, `non_canonical_partial_ord_impl=allow`
|
|
- **CLI structs in lib.rs, not main.rs** — enables integration tests to import and construct Cli directly.
|
|
- **Pipeline stages return `anyhow::Result<()>`** — stages propagate with `?`; only main.rs converts to CliError.
|
|
- **ResourceRegistry pattern** — populated before pipeline stages, consumed read-only by stages. New resource types go in `types/resource.rs`.
|
|
- **Notification queue fire-and-forget + drain** — notify tasks spawned early, send during pipeline, drain on completion. `NotificationQueue::drain()` must be called.
|
|
- **Trivial step detection** — steps with no external deps can be marked trivial (`trivial.rs`), skipped in dry-run.
|
|
## ANTI-PATTERNS (THIS CRATE)
|
|
- **Do NOT derive production behavior from bare mode** — ctx.privileged is hardcoded false.
|
|
- **Do NOT add CLI args bypassing prebake→bake→finalize ordering** — the pipeline is linear. Parallelism lives in the DAG scheduler within bake.
|
|
- **Do NOT import types/ from bake/, prebake/, or finalize/** — `types/` must have zero crate-internal deps.
|
|
- **Do NOT add lint allows without justification** — the four existing overrides are intentional.
|
|
- **Do NOT spawn blocking work on tokio runtime** — use `spawn_blocking` for CPU-heavy tasks.
|
|
- **Decorator regex is line-by-line** — multi-line decorators need a parser rewrite, not a regex hack.
|
|
- **Careful with `unimplemented!()`** — Daemon, Rhai/Dylib plugins, Custom builder are stubs. Prefer error over crash.
|