Compare commits

...

60 Commits

Author SHA1 Message Date
Catty Steve fdc00f0635 feat(baker): add debug feature flags and resource prefetching
Add DebugFeature bitflags for controlling notify.dummy, events,
scheduler,
and script debug behaviors. Add prefetch module to fetch plugins and
templates before pipeline runs. Add cleanup and artifact handling to
finalize stage. Add build_id to ExecutionContext.
2026-06-17 20:43:17 +08:00
Catty Steve 26809df720 feat(bake): DAG scheduler, decorator execution, notify refactor, workshop-schedule crate
- Add workshop-schedule crate (petgraph-based DAG scheduling)
- Implement all bake decorators with combinator chain execution
- Refactor notify module: extract handler, rule, template, util, queue
- Add NotificationQueue with priority/drain semantics
- Enhance finalize plugin system with internal plugin registration
- Update ExecutionContext and event types
- Add per-crate AGENTS.md knowledge base files
- Remove inline AGENTS.md files (consolidated into per-crate docs)
2026-06-15 20:45:06 +08:00
Catty Steve 3629c33fc4 feat(bake): support daemon, health and async decorator
Add `@async` decorator for fire-and-forget tasks, `@pipe` for stdout
routing between functions, and `@daemon` with configurable health
probes. Functions now resolve execution mode and track stdout cache for
inter-function communication.
2026-06-11 16:26:44 +08:00
Catty Steve 6d9ddbca56 refactor(bake/baker): tidy up bake.rs 2026-06-09 23:36:58 +08:00
Catty Steve 9ff9073fce feat(baker/bake): implement all available decorators 2026-06-09 23:09:57 +08:00
Catty Steve df5026cdbe refactor(notify): extract handler, rule, util
CI / ${{ matrix.crate }} (workshop-engine) (push) Failing after 3m54s
CI / ${{ matrix.crate }} (workshop-baker) (push) Failing after 27m50s
Split the monolithic notify module into three submodules:
- `handler`: async bucket dispatch with cooldown, retry, escalation
- `rule`: rule parsing from YAML config, template merging, overrides
- `util`: validation, event timing, helper functions
  `NotificationTemplateDef` replaces `AddAssign` with `apply()` for
  non-destructive merging (self wins, other fills gaps).
  `insitenotify` logs instead of panicking.
2026-05-22 00:16:28 +08:00
Catty Steve d1ad08ef3a feat: add workshop-baker-params for config layering
Implement a priority-based merge system for pipeline configuration.
Parameters carry their source (Base, Courier, Bakerd) and only
higher-priority writers override. Migrate notification, prebake, bake,
and finalize config to use ParamVal wrappers. Remove hardcoded
constants and the InteractiveServer component.

BREAKING CHANGE: Remove NotAvailable variant from MethodStatus enum
2026-05-21 20:04:47 +08:00
Catty Steve cf2968e720 refactor: update AGENTS.md and remove osbolete modules
Archive 7 deprecated crates. Add Resource model documentation.
Remove internal fetch module from workshop-baker.
2026-05-20 16:30:50 +08:00
Catty Steve 28abc5d207 refactor: replace internal fetch utilities with ResourceRegistry
Remove the monolithic utils::fetch module (HTTP download, git clone,
checksum, extraction) and utils::fetch_plugin module from the baker.
Introduce a ResourceRegistry type to centrally track pipeline
resources by name and path.

The actual fetching logic is now delegated to the workshop-getterurl
crate, which gains optional md5 and sha1 hash support behind default
features. Also flatten the builtin module in workshop-getterurl.
2026-05-19 23:22:59 +08:00
Catty Steve 6e7b96cd66 refactor: remove unused crates, refactor fetch
- Move fetch_plugin from finalize::plugin::fetch to utils::fetch_plugin
- Delete unused crates: workshop-cert, workshop-deviceid, workshop-vault
- Remove parser benchmarks and empty daemon/socket modules
- Add new workshop-getterurl crate for resource fetching
2026-05-19 21:53:41 +08:00
Catty Steve 3e80825d58 refactor(plugin)!: remove dedicated plugin fetch subsystem
Drop the entire plugin fetch module (HTTP, git, checksums, extraction).
Simplify mail notification plugin to a bare stub.
Remove outdated metadata tests.

BREAKING CHANGE: Not a buildable version!
2026-05-19 16:24:19 +08:00
Catty Steve c97eafab29 refactor(baker): Restructure CLI, drop legacy tests
- Remove Python-based test framework, Dockerfiles, and runner scripts
- Add bare subcommand for standalone pipeline stage execution
- Implement unified worker mode and error module
- Refactor NotificationConfig to support YAML string keys
- Add dummy internal plugin and clean up mail template
2026-05-19 16:19:28 +08:00
Catty Steve f737b6a24d feat(baker): add axum HTTP server and refactor plugin fetching
Refactor plugin imports to use workshop_engine types, consolidate
resource fetching into utils/fetch module with checksum verification,
and add interactive notification server using axum.
2026-05-18 10:01:55 +08:00
Catty Steve 681b7d1333 style: format all code to comply with cargo fmt check
CI / ${{ matrix.crate }} (workshop-engine) (push) Failing after 36m14s
CI / ${{ matrix.crate }} (workshop-baker) (push) Successful in 1h35m51s
2026-04-28 19:17:56 +08:00
Catty Steve f07588a716 test: add Go UPM system tests for bare prebake 2026-04-28 19:12:53 +08:00
Catty Steve b0530fb858 feat(upm): add Python and Go package manager support 2026-04-28 16:56:53 +08:00
Catty Steve fd18b547e5 feat(upm): implement Rust user package manager
Adds the `user.rust` UPM to workshop-engine for managing Rust
toolchains,
components, cross-compilation targets, and cargo operations:

- `rustup default <toolchain>` / `rustup component add` / `rustup target
  add`
- `cargo fetch` for dependency pre-fetch with optional `--locked`
- `cargo install` for global crate tools with registry support

Includes 275-line test suite (`tests/upm/test_rust.py`) covering the
full
lifecycle with optional network tests.
2026-04-28 12:20:05 +08:00
Catty Steve cf317b55c6 feat(pm): add apk/dnf implementations and multi-distro test
CI / ${{ matrix.crate }} (workshop-engine) (push) Failing after 28m8s
CI / ${{ matrix.crate }} (workshop-baker) (push) Failing after 33m39s
infrastructure
2026-04-27 23:01:43 +08:00
Catty Steve 20e2ab4224 refactor: reorganize imports and add lint allows
CI / ${{ matrix.crate }} (workshop-engine) (push) Failing after 13m13s
CI / ${{ matrix.crate }} (workshop-baker) (push) Failing after 28m54s
2026-04-27 14:31:11 +08:00
Catty Steve 50b42e654c refactor(workshop-engine): replace tokio Child with ManagedChild trait
for testability

- Add `ManagedChild` trait abstracting process lifecycle (real
  `TokioChild` + `MockChild` for tests)
- Use Unix process groups (`process_group(0)` + `libc::kill(-pgid,
  SIGKILL)`) for tree-wide termination
- Remove cgroup-related code (`CgroupManager`, `ResourceUsage`,
  `cgroup_path` field)
- Update imports across test files to use `workshop-engine` crate
  directly
2026-04-27 12:02:17 +08:00
Catty Steve a64e47d772 ci(github-actions): add workshop-engine to CI matrix strategy
CI / ${{ matrix.crate }} (workshop-baker) (push) Failing after 28m8s
CI / ${{ matrix.crate }} (workshop-engine) (push) Failing after 1m27s
Expand CI to run on workshop-engine crate alongside workshop-baker using
a matrix strategy to avoid job duplication.
2026-04-26 23:27:39 +08:00
Catty Steve 8fd807f174 refactor!: extract execution engine into standalone workshop-engine
crate

BREAKING CHANGE: Moved engine/ module (cgroups, executor, pm, upm,
types) and shared types (command, repology, time) from workshop-baker to
new workshop-engine crate. All internal error types consolidated into
workshop-engine::error. Import paths changed throughout codebase.
2026-04-26 23:07:08 +08:00
Catty Steve e8ce2eec9e docs: add comprehensive documentation and tests to workshop-baker 2026-04-25 11:30:59 +08:00
Catty Steve 9e45bdbbfb refactor(notify): rename NotifyEvent to NotificationEvent
CI / Check & Lint (push) Failing after 19m30s
2026-04-24 23:10:04 +08:00
Catty Steve 5ce8b72900 refactor(notify): extract types to separate module and redesign
NotificationRule

- Move NotifyEvent, NotifyPluginMap, and related types to
  notify/types.rs
- Introduce NotificationRule struct replacing NotificationMethod fields
- Add NotificationRulesMap for priority-based notification configuration
- Comment out main event loop and add todo!() placeholders for WIP
2026-04-24 23:01:40 +08:00
Catty Steve 2c9d32e954 refactor(error): extract exit code logic into HasExitCode trait
Introduce a common HasExitCode trait for PrebakeError, BakeError,
ExecutionError, and FinalizeError. Refactor main.rs to consolidate
CLI command handling and use the trait for unified exit code resolution.
2026-04-24 21:45:57 +08:00
Catty Steve 901005639b feat(cli): add environment variable support and restructure commands
- Add HBW_USERNAME, HBW_PIPELINE, HBW_BUILD_ID, HBW_DRY_RUN env vars
- Wrap prebake/bake/finalize in Bare subcommand for cleaner hierarchy
- Remove unimplemented Monitor command
2026-04-24 21:12:10 +08:00
Catty Steve 84b3b7d8e8 test: add Docker-based system test infrastructure
Adds a complete pytest system test suite for the HoneyBiscuitWorkshop
CI/CD system:

- runner.sh: orchestrates build, Docker image creation, and test
  execution
- Dockerfile: Arch Linux-based test image with Python venv
- conftest.py: shared fixtures (baker_bin, run_baker, work_dir, llm_env)
- probes/: environment probes for checking users, files, packages,
  processes
- llm/: LLM-based log judgment system with caching for semantic test
  evaluation
- fixtures/: test configs and scripts for prebake, bake, and finalize
  stages
- test_prebake.py, test_bake.py, test_finalize.py: test suites for each
  pipeline stage

Also refactors finalize.yml.tmpl notification config to use a templates
system.
2026-04-24 15:54:56 +08:00
Catty Steve 36112c671d style: apply rustfmt and clippy fixes across codebase
CI / Check & Lint (push) Failing after 23m46s
2026-04-23 18:34:26 +08:00
Catty Steve 9d3d61ba66 ci: enable GitHub/Gitea worker
CI / Check & Lint (push) Failing after 11m51s
2026-04-23 16:30:51 +08:00
Catty Steve 117660d5c4 feat(baker/finalize): notification system and consolidate status types
- Rename types/build_status.rs to types/buildstatus.rs
- Rename StageInfo.sub_stage field to substage for consistency
- Move notification logic from finalize to dedicated notify module
- Remove notification module from finalize (moved to notify)
- Add notify module with event queuing, retry, and priority scheduling
- Add exponential backoff for temporary notification failures
- Implement plugin filtering by name list during registration
- Add fetch_plugins helper for batch plugin download
- Update finalize plugin registration to accept optional plugin filter
- Fix finalize config template indentation for notification policy
- Move all YAML templates to templates/ directory
2026-04-23 16:18:20 +08:00
Catty Steve 07d76d612d refactor!: move notification module to src/notify.rs
BREAKING CHANGE: notification system is now separate module
2026-04-22 21:21:26 +08:00
Catty Steve 2658d0ebfa feat(baker): add build status tracking and notification dispatch system
Add new `build_status` types module for tracking pipeline stage
execution
status across prebake, bake, and finalize phases. Introduce
`NotificationCore`
dispatcher that processes notification methods by priority groups with
fallback
on failure. Status files written to `/tmp/.hbwstatus` in JSON lines
format.
2026-04-22 18:45:22 +08:00
Catty Steve 40f307b471 chore: remove unused crates and add documentation
- Delete workshop-agent (HTTP server stub)
- Delete workshop-builder-native (empty stub)
- Delete workshop-helper-mac (unused)
- Delete workshop-llm-detector (moved to separate repo)
- Add AGENTS.md to bake/, engine/, finalize/, prebake/, cert/, vault/
- Fix finalize plugin validation and logging improvements
2026-04-22 12:47:48 +08:00
Catty Steve 7b3b71a4b3 feat(finalize): support multiple manifest extensions and add hook tests
- Remove underscore suffix from FINALIZE_SHELL_ENVPREFIX constant
- Split FINALIZE_PLUGIN_MANIFEST into name + extensions for flexible
  manifest discovery
- Add ManifestNotFound error variant for clearer plugin loading errors
- Add debug logging for plugin download operations
- Add integration tests for finalize hook execution
- Remove obsolete clarification.md
2026-04-21 21:40:19 +08:00
Catty Steve 0dd4a157ef feat(finalize): add standalone mode with automatic plugin fetching and
hook execution

- Add FinalizeHooks config for early/late stage hook execution
- Support plugin: prefixed commands in hooks to invoke registered
  plugins
- Implement standalone mode that auto-downloads plugins before
  registration
- Add shallow clone and checksum options to plugin config
- Normalize retention format to lowercase in template
- Refactor PathBuf to Path in plugin fetch modules for consistency
2026-04-21 18:13:24 +08:00
Catty Steve d8ba166a3b refactor: centralize ExecutionContext and event handling in main
- Lift ctx creation and event_receiver spawning to main
- Inject ctx and event_tx into prebake/bake/finalize
- Add standalone flag to ExecutionContext
- Wire up finalize command in main
2026-04-21 11:38:02 +08:00
Catty Steve e9128adfa0 feat(baker/finalize): implement plugin system with fetch and mail
support

- Add finalize plugin system with fetch logic for git/https/file sources
- Add plugin checksum verification (SHA256/SHA512, reject MD5/SHA1)
- Add tar archive extraction with gzip/zstd support
- Add git clone and checkout with tag/commit fallback
- Add internal mail plugin with SMTP support and template rendering
- Add shell plugin with YAML-to-envvar argument conversion
- Add retry and timeout configuration for plugins
- Add duration deserialization to milliseconds with human-readable
  formats
- Replace timeout fields with timeout_ms throughout codebase
- Update CustomCommand timeout_ms and hook timeout handling
- Add HBW_ARG_ prefix for shell plugin environment variables
2026-04-21 10:33:21 +08:00
Catty Steve 0c089fd634 feat(baker/finalize): add YAML to shell env var conversion 2026-04-19 09:40:05 +08:00
Catty Steve a1da093176 feat(baker/finalize): Implement plugin manifest support and typed
metadata

- Rename env vars WS_* to HBW_* for consistency
- Add PluginMetadata struct and PluginType enum for typed plugin info
- Support manifest.yml files for external plugin discovery
- Add shell/rhai/dylib plugin stubs with proper type signatures
- Implement shallow argument merge for plugin runtime arguments
2026-04-18 23:58:33 +08:00
Catty Steve a8ed935f6a feat(baker/finalize): Add finalize stage framework with plugin system
and template

Implements the finalize stage infrastructure for HoneyBiscuitWorkshop:
- New finalize module with config, error, event, plugin, stage, and
  template submodules
- Plugin system supporting internal (webhook, mail, satori, in-site) and
  external plugins
- FinalizeConfig schema with version validation for finalize.yml
- Add glob, duration-str, lettre dependencies to workshop-baker
- Expand project documentation (clarification.md, description.md)
- Update .gitignore and fix bake.rs pipeline_functions access

BREAKING CHANGE: Not a buildable version!
2026-04-16 23:33:03 +08:00
Catty Steve 73e1238e20 refactor(bake): redesign parser with state machine and improve builder
- Replace flat parser with state machine (Normal/Decorator/InFunction)
- Change parse_script to accept reference and add proper error handling
- Add Function::find_decorator helper method
- Refactor build_script to accept &Function and support @if/@export
  decorators
- Add condition code generation for @if decorator
- Add export/preexport code for environment variable capture
- Handle trivial scripts by creating Function directly
- Add constants: PIPELINE_SKIP_ERRORCODE and TRIVIAL_SCRIPT_NAME
- Temporarily disable order-based sorting assertion (see TODO)
- Add empty finalize.rs module skeleton
- Update integration tests to expect errors appropriately
2026-04-06 00:07:02 +08:00
Catty Steve 246c2b22f1 feat(bake): Add streaming execution output and Repology endpoint config
- Implement async stdout/stderr streaming with event channel
- Add Repology endpoint configuration in dependencies config
- Refactor PackageManager trait to return Vec<Command>
- Add bootstrap configuration example
- Remove temp/privdrop.rs
2026-04-04 16:59:39 +08:00
Catty Steve 418bdfb2f0 refactor!: remove workshop-pipeline crate and add repology support
BREAKING CHANGE: workshop-pipeline crate removed entirely

- Remove workshop-pipeline/ (config structures, template resolution)
- Add Repology package name resolution to workshop-baker
- Add UPM system dependency collection for cross-pm resolution
- Replace AGENTS.md with concise knowledge base format
- Add crate-specific AGENTS.md files
- Fix various clippy warnings and derive Default where applicable
2026-04-04 13:24:26 +08:00
Catty Steve 2133f8b9b3 feat(bake): implement full bake functionality with template-based script
generation and pipeline scheduling

- Add benchmark infrastructure (Criterion) for parser performance
  testing
- Enhance decorator parsing with new types: Pipe, Parallel, Daemon,
  Health, and Unknown
- Improve parser to track line numbers and handle nested braces
  correctly
- Add implicit `@after` dependencies for pipeline functions in
  topological sort
- Introduce `use_template` flag in builder to support trivial script
  execution
- Update bake function signature to accept Cli parameters and privilege
  detection
- Refactor error handling with structured `ScriptGenerationFailed`
  variant
- Extend integration tests for privilege dropping, engine execution, and
  bake phase
- Comment out daemon and apt modules temporarily for refactoring
2026-04-04 10:12:57 +08:00
Catty Steve b59a702925 feat(bake): add bake module for script execution and refactor error
handling

- Add new bake module with parser, scheduler, decorator, and builder
- Implement topological sorting for functions with @after dependencies
- Add error handling for circular dependencies and unknown dependencies
- Introduce constants module for shared path definitions
- Add BakeError type (renamed from BuilderError) with new variants
- Add PrebakeStage::Never variant for "never drop privileges"
- Simplify decorator parsing and clean up comments
- Remove RustyVault and rust-tongsuo submodules as they are unused
  (temporarily)
- Update .gitignore to include .ruff_cache
2026-04-02 19:34:21 +08:00
Catty Steve b02407a02d chore(baker/prebake): Add comprehensive documentation to prebake module
- Document prebake.rs, bootstrap.rs, and hook.rs with module-level and
  function-level docs
- Rename parse_prebake_config to parse for conciseness
- Change bootstrap() return type from ExecutionResult to ()
- Add dry-run support to bootstrap stage
- Improve test robustness with Docker availability check and unique
  container names
2026-04-01 10:54:51 +08:00
Catty Steve 15aaa6f310 feat(baker/prebake): Add configurable bootstrap stage with dynamic
script generation

The bootstrap stage now reads configuration from prebake.yml to
dynamically generate bootstrap scripts for user creation, workspace
setup, and privilege executor configuration.
Adds the `which` crate for binary detection.
2026-03-31 23:46:11 +08:00
Catty Steve 9174c4dd2a feat: rename to workshop-baker and implement stage-based prebake
workflow

- Rename crate/library/binary from workshop-executor to workshop-baker
- Remove unused modules (decorator, parser, schedule, variable) from lib
- Implement stage-based privilege dropping with prebake_drop_privilege()
- Add dry-run support in executor and CLI
- Add bootstrap.sh template script for user/workspace setup
- Add PrebakeStage::from_str() and Default impl
- Add get_drop_after() function for security config
- Fix version matching to handle "1.0" format variants
- Fix UPM all_managers() to return custom::Custom
- Add Python integration tests with Docker
- Simplify examples/prebake.yml to match template
2026-03-31 20:54:48 +08:00
Catty Steve 20919d53ad refactor(baker/bake): move relating part into separate directory 2026-03-31 13:16:14 +08:00
Catty Steve 3e8b256889 feat(baker/engine): add depsuser stage 2026-03-31 11:53:55 +08:00
Catty Steve 0f6aed6f94 feat(engine): implement UPM/UserPackageManager interface 2026-03-31 11:39:47 +08:00
Catty Steve 960f03269c feat: rename workshop-executor to workshop-baker and add comprehensive
documentation

- Rename crate from workshop-executor to workshop-baker
- Add MDBook documentation infrastructure with Chinese docs (Vol1/2/3)
- Implement engine module with cgroups resource management
- Add package manager support (apt, pacman)
- Add daemon module with Unix socket IPC and heartbeat
- Add privdrop dependency for privilege dropping
- Implement prebake stages (bootstrap, depssystem, depsuser,
  environment, hooks)
- Add new bake.sh.tmpl pipeline template for osu! project
- Add pipeline implicit parameter documentation (PIPELINE_BAREMETAL_*,
  PIPELINE_CONTAINER_*, PIPELINE_UNCOVER_SECRET)
2026-03-30 20:24:54 +08:00
Catty Steve 69056a3a4a feat: add Package Manager 2026-03-11 14:49:10 +08:00
Catty Steve c80500f6ac feat: implement docker env creation for prebake
Also we introduced opencode for code production.
2026-03-10 11:59:58 +08:00
Catty Steve d7ba65fbf4 feat: finish monitor in executor, daemonize executor
Signed-off-by: Catty Steve <4795515+Catty2014@user.noreply.gitee.com>
2026-01-15 11:54:42 +08:00
Catty Steve ab7cbfd730 feat: add engine to ws-executor
Signed-off-by: Catty Steve <4795515+Catty2014@user.noreply.gitee.com>
To be done.
2025-12-28 23:05:02 +08:00
Catty Steve ecf3c66905 feat: add workshop-{executor, helper-mac}
Signed-off-by: Catty Steve <4795515+Catty2014@user.noreply.gitee.com>
2025-12-27 21:46:16 +08:00
Catty Steve f7ea4dfb3e feat: add workshop-pipeline
Signed-off-by: Catty Steve <4795515+Catty2014@user.noreply.gitee.com>
Workshop-pipeline is a crate for parsing prebake/bake/finalize config.
Not finished yet, further testing required.
2025-11-14 11:05:05 +08:00
Catty Steve f66a36d0f4 feat: add workshop-agent and workshop-cert
Signed-off-by: Catty Steve <4795515+Catty2014@user.noreply.gitee.com>
2025-11-13 23:49:16 +08:00
213 changed files with 25399 additions and 4489 deletions
+62
View File
@@ -0,0 +1,62 @@
name: CI
on:
push:
branches: [ master, main ]
paths:
- 'workshop-baker/**'
- 'workshop-engine/**'
- '.github/workflows/ci.yml'
pull_request:
branches: [ master, main ]
paths:
- 'workshop-baker/**'
- 'workshop-engine/**'
- '.github/workflows/ci.yml'
env:
CARGO_TERM_COLOR: always
jobs:
ci:
name: ${{ matrix.crate }}
runs-on: ubuntu-latest
strategy:
fail-fast: false
matrix:
crate: [workshop-baker, workshop-engine]
defaults:
run:
working-directory: ./${{ matrix.crate }}
steps:
- uses: actions/checkout@v4
- name: Install Rust toolchain
uses: dtolnay/rust-toolchain@stable
with:
components: clippy, rustfmt
- name: Cache cargo dependencies
uses: actions/cache@v4
with:
path: |
~/.cargo/registry
~/.cargo/git
${{ matrix.crate }}/target
key: ${{ runner.os }}-cargo-${{ matrix.crate }}-${{ hashFiles(format('{0}/Cargo.lock', matrix.crate)) }}
restore-keys: |
${{ runner.os }}-cargo-${{ matrix.crate }}-
- name: Check compilation
run: cargo check --all-features
- name: Run tests
run: cargo test --all-features
- name: Check formatting
run: cargo fmt -- --check
- name: Run Clippy lints
run: cargo clippy --all-features -- -D warnings
+20
View File
@@ -1,2 +1,22 @@
target
playground
*.old
.ruff_cache
.codeartsdoer
archived
.sisyphus
*.bak
*.clean
session*
.secret
# System test artifacts
tests/results/
tests/__pycache__/
tests/**/__pycache__/
tests/.pytest_cache/
tests/llm/.cache/
*.pyc
.staging
.pytest_cache
+127
View File
@@ -0,0 +1,127 @@
# PROJECT KNOWLEDGE BASE
**Generated:** 2026-06-12T16:08:15Z
**Commit:** 3629c33
**Branch:** feat/bake-dag-tester
## OVERVIEW
HoneyBiscuitWorkshop (蜜饼工坊) — Rust-based CI/CD system inspired by Concourse CI. Pipeline: prebake.yml → bake.sh → finalize.yml. Multi-builder (Docker, Firecracker, bare metal). Plugin system.
## STRUCTURE
```
./
├── workshop-baker/ # Main binary + lib — orchestrator, CLI, plugins
├── workshop-engine/ # Core execution engine — process mgmt, package managers
├── workshop-schedule/ # DAG scheduling library (petgraph-based)
├── workshop-getterurl/ # URL fetching with hash verification
├── workshop-baker-params/ # Config/parameter type definitions
├── docs/ # mdBook documentation (Chinese, 3 volumes)
├── templates/ # Pipeline config templates (prebake/bake/finalize)
└── .github/workflows/ # CI (tests workshop-baker + workshop-engine)
```
**No root Cargo workspace** — crates linked by path deps. Build per-crate.
## WHERE TO LOOK
| Task | Location | Notes |
|------|----------|-------|
| CLI definition | `workshop-baker/src/lib.rs` | Clap Cli/Commands/BareCommands |
| Pipeline orchestration | `workshop-baker/src/main.rs` | prebake→bake→finalize→notify flow |
| Decorator parsing | `workshop-baker/src/bake/decorator.rs` | `# @decorator(args)` syntax |
| Build execution | `workshop-baker/src/bake/execute.rs` | Script execution with timeout/retry |
| Docker integration | `workshop-baker/src/prebake/env/container.rs` | Bollard-based |
| Package managers | `workshop-engine/src/pm/` | apt, pacman, dnf, apk |
| User package managers | `workshop-engine/src/upm/` | rustup, uv, go |
| Notification system | `workshop-baker/src/notify/` | Email, webhook, IM |
| Plugin system | `workshop-baker/src/finalize/plugin/` | shell, dylib, rhai, internal |
| Error types | `workshop-baker/src/error.rs` | CliError + HasExitCode |
| Exit codes | `workshop-engine/src/error.rs` | EXITCODE_* constants |
| DAG scheduling | `workshop-schedule/src/dag.rs` | Topological sort, batch picking |
| Pipeline templates | `templates/` | Reference config DSL |
## ENTRY POINTS
| Entry | Type | Path | Role |
|-------|------|------|------|
| `workshop-baker` | Rust bin | `workshop-baker/src/main.rs` | Primary CLI — dispatch: default (full pipeline), bare (single stage), daemon (unimplemented) |
| `workshop_baker` | Rust lib | `workshop-baker/src/lib.rs` | Baker library + CLI structs |
| `workshop_engine` | Rust lib | `workshop-engine/src/lib.rs` | Core engine — ExecutionContext, EventSender, Executor |
| `workshop_schedule` | Rust lib | `workshop-schedule/src/lib.rs` | Dag, DagNode, EdgeKind |
| `workshop_baker_params` | Rust lib | `workshop-baker-params/src/lib.rs` | Layered config: defaults → base → courier |
| `workshop_getterurl` | Rust lib | `workshop-getterurl/src/lib.rs` | GetterUrl, fetch() — http/git/file |
| docs | mdBook | `docs/book.toml` | Project documentation site |
## CODE MAP
### Crate Dependency Graph
```
workshop-baker (bin)
├── workshop-engine
├── workshop-schedule
├── workshop-baker-params
└── workshop-getterurl
```
### workshop-baker Module Map
| Module | Submodules | Role |
|--------|-----------|------|
| `bake` | decorator, execute, parser, schedule, builder, trivial, util, constant, error | Script parsing + execution |
| `prebake` | config, stage (bootstrap, depssystem, depsuser, environment, hook), env (container, firecracker, baremetal, custom), security, event, types, constant, error | Environment setup |
| `finalize` | plugin (shell, dylib, rhai, internal), stage, template, types, config, constant, error, event | Post-build hooks |
| `notify` | handler, queue, rule, template, types, util, error | Async notification dispatch |
| `bare` | — | Standalone single-stage CLI mode |
| `types` | resource, buildstatus | Shared type definitions |
| `utils` | — | Cross-cutting utilities |
## CONVENTIONS
- **Edition**: Rust 2024 (requires Rust 1.85+)
- **Formatting**: Default `rustfmt` — no `rustfmt.toml` / `.editorconfig`
- **Lint overrides** (workshop-baker only): `dead_code = "allow"`, `unreachable_code = "allow"`, `inherent_to_string = "allow"`, `non_canonical_partial_ord_impl = "allow"`
- **Import order**: `std` → external crates → `crate::` (blank line between groups)
- **Error handling**: `thiserror` for libraries, `anyhow` for binaries. Implement `HasExitCode` for exit code mapping.
- **Type system**: Newtype pattern preferred (e.g., `MemSize(u64)`). Shared types in `types/` module. `types/` must NOT depend on `bake/`, `engine/`, `prebake/`, `finalize/`.
- **Tests**: Inline `#[cfg(test)] mod tests` at bottom of source files. Async: `#[tokio::test]`. CI runs `cargo test --all-features`.
- **Commits**: `type(scope): description``feat|fix|docs|test|refactor|style`
- **AI annotation**: `// Code in this module PARTIALLY or FULLY utilized AI Coding Agent` — never delete if present
- **Docs language**: Chinese (zh-CN), mdBook
- **No `no_std`**: Explicitly unsupported
## ANTI-PATTERNS (THIS PROJECT)
- **Do NOT derive production behavior from bare mode** — bare mode is dev-only, ctx.privileged is fake
- **Do NOT use `detect()`/`adopt()` PM methods in production** — PM selection must come from env config
- **`drop_after < DepsUser` is FORBIDDEN** — will break system dependency installation (needs root)
- **Do NOT add magic URL inference** — URL is URL, behavior bound explicitly via `getter::` prefix
- **Do NOT use `@` for version locking in URLs** — it's an auth delimiter; use `?ref=`
- **Do NOT duplicate type definitions across modules** — single source of truth in `types/`
- **Careful with `unimplemented!()`** — Daemon mode, Rhai/Dylib plugins, and Custom builder are stubs
- **Security**: PIPELINE_CONTAINER_PRIVILEGED, PIPELINE_BAREMETAL_ELEVATE, PIPELINE_UNCOVER_SECRET require explicit approval + audit
## COMMANDS
```bash
# Build (per crate, no workspace)
cd workshop-baker && cargo build --all-features
# Test (per crate)
cd workshop-baker && cargo test --all-features
cd workshop-engine && cargo test --all-features
# Lint (CI gate)
cargo fmt -- --check
cargo clippy --all-features -- -D warnings
# Docs
cd docs && mdbook build # output in docs/book/
cd docs && mdbook serve # local preview at localhost:3000
```
## NOTES
- **No workspace Cargo.toml** at root — each crate has its own `Cargo.lock`. Build individually.
- **CI only tests** `workshop-baker` and `workshop-engine`; smaller crates tested transitively.
- **`quicktest.sh`** in workshop-baker/ is a dev smoke-test script (Docker-based, not CI).
- **Git submodules**: `rust-tongsuo`, `RustyVault` (crypto — not yet integrated into build).
- **`.secret/`** contains real credentials — never commit, never read into context.
- See `docs/src/zh-CN/vol3_dev/` for full developer documentation (architecture, design decisions, code style).
Submodule RustyVault deleted from 4efe033ce7
+62
View File
@@ -0,0 +1,62 @@
# HBW(HoneyBiscuitWorkshop) 项目规划与介绍
本项目为一个基于Rust的CI/CD系统,名为“蜜饼工坊”(HoneyBiscuitWorkshop)。
该名称取材于明日方舟世界观中小刻爱吃的蜜饼和她的管理者火神大姐的工坊。
## 目标愿景
这个CI/CD的最终用途如下:
0. 首先是一个合格的CI/CD。其参考了concourseconcourse也是本项目的起始点与初衷,见Vol3/Story.md),意图实现其大部分功能。
1. 软件包自动化
接入AOSC的autobuild+abbs树,或者Arch Linux的ABS,实现软件包的自动构建与更新
在Web/CLI(./workshop)中允许使用流水线模板快捷推送任务。示例:
./workshop push aosc:linux-kernel-6.19
./workshop push arch:python-pytorch-rocm
2. 机器学习支持,支持CUDA/ROCm/Others方案运行机器学习/深度学习/大模型训练程序
3. 对于部分工业软件/行业软件的支持,例如允许通过Vivado实现硬件设计远程综合,或者通过Blender渲染一个项目
一定程度是2的延伸
## 特性设计
1. LLM Skill/MCP 适配
允许通过Skill调用API/CLI(prefer!)与服务器通信,实现流水线自愈,流水线配置快速生成等
2. 流水线进度感知
具体说来程序可以通过读取的流水线日志来判断进行的进度,以估计剩余时间,或给出超时警告
实现上尽可能避开大模型,使用纯粹机器学习/深度学习方法,TTR=1s,单流水线持久化数据不大于16KB
3. 与koishi(Koishi.js,一个聊天机器人管理项目,https://koishi.chat/zh-CN/)等结合,实现消息推送与流水线简单管理(如快捷重启,定时启动等)
4. 基于cgroups/JobObject(计划中)的任务资源管理与计数
资源管理可以对任务资源进行限制
资源计数可以对不同权限的账号计算配额
5. prebake.yml, bake.sh, finalize.yml的最简流水线配置
prebake.yml的模板于本目录给出,尽可能完善
prebake为纯粹声明式配置,具有固定的用户自定义锚点,具有权限配置
bake.sh的模板于本目录给出,仅供参考
bake使用@decorator(私有)装饰器语法增强语义,主要是命令式,兼容纯shell
finalize.yml的模板正在设计,计划上支持shell/rhai/dylib(待Rust ABI稳定)
6. 静态二进制发布,支持各种架构,系统与C库
我们尽可能为Rust Tier1架构做完整支持(Windows暂缓)
会尤其注重新兴架构: RISC-V和LoongArch的支持
理论上不支持32位,但不会限制
!!!不支持没有操作系统的裸机(nostd)!!!
7. 构建机支持裸机,容器,FireCracker虚拟机与自定义环境
8. 可配置的构建缓存与环境缓存
9. 本地流水线快捷验证
可以在不借助服务器的情况下快速验证流水线语法的正确性,在借助本机容器/虚拟机的情况下进行试构建
10. 本地文件夹推送构建
可以推送本地的任何一个文件夹到服务器,其会根据.workshop中的规则文件/LLM自生成进行构建,支持基于哈希+分块的增量传输
11. Vol1/2/3 用户/管理员/开发者文档
CI/CD采用Web+CLI+Server+Multi (Agent+Baker.bin(Bakerd+Baker))设计。
12. 最小化外部依赖
可能且推荐的外部扩展包括:
- RustyVault/Vault: 凭据存储
- Koishi: IM集成与通知
- Repology: 跨发行版包名映射
可能的外部扩展包括:
- SQL DB
- NoSQL DB
- MQ
13. 分发
事实上单二进制是与有系统依赖相对的。分发时,server/agent/baker需要动态下载。
除去官方分发(软件包或者统一软件包(如flatpak))外,通过curl+sh下载验证并安装server。
server自动下载并验证正确的agent与bakeragent/baker可缓存。
提供agent.tar与baker.tar便于离线部署
+1
View File
@@ -0,0 +1 @@
book
+48
View File
@@ -0,0 +1,48 @@
# DOCS KNOWLEDGE BASE
## OVERVIEW
mdBook project in zh-CN. Authoritative source for project conventions, coding style, and architecture docs. Build output at `docs/book/` (gitignored).
## STRUCTURE
```
docs/
├── book.toml # mdBook config (title, language=zh-CN)
├── src/SUMMARY.md # Sidebar nav — source of truth for page listing
└── src/zh-CN/
├── index.md # Landing
├── vol1_user/ # End-user documentation
├── vol2_admin/ # Admin reference: decorator cheatsheet, deployment,
│ # env vars, exit codes, plugin ref, security, troubleshooting
└── vol3_dev/ # Developer docs: architecture, code style, commit
# convention, testing, error handling, type system,
# design decisions, PR process
```
## WHERE TO LOOK
| Task | Location | Notes |
|------|----------|-------|
| Sidebar/nav editing | `src/SUMMARY.md` | Add entry here to surface a new page |
| Coding conventions | `vol3_dev/code_style.md` | Naming, imports, formatting |
| Commit rules | `vol3_dev/commit_convention.md` | feat/fix/docs/test/refactor/style |
| Error handling | `vol3_dev/error_handling_patterns.md` | thiserror+anyhow+HasExitCode |
| Type system | `vol3_dev/type_system_guide.md` | Newtypes, types/ module isolation |
| Testing strategy | `vol3_dev/testing_strategy.md` | 3 tiers: unit, integration, system |
| Anti-patterns | `vol3_dev/design_decisions.md` | Documented design rejections |
| Architecture | `vol3_dev/architecture.md` | Crate dependency graph, module map |
## CONVENTIONS
- **Language**: All prose in zh-CN. Code blocks in Rust (default) or bash.
- **Adding a page**: Create `.md` under `volN_*/`, then add to `SUMMARY.md`.
- **Formatting**: Standard Markdown. mdBook preprocessors handle TOC and links.
- **Cross-references**: Relative paths from source root (e.g., `../vol3_dev/code_style.md`).
- **Code blocks**: Fenced with language tag. No line numbers in docs.
- **File naming**: `snake_case.md`.
## COMMANDS
```bash
mdbook build # static site → docs/book/
mdbook serve # live preview at localhost:3000
```
+5
View File
@@ -0,0 +1,5 @@
[book]
title = "HoneyBiscuitWorkshop Documentation"
authors = ["Catty Steve"]
language = "zh-CN"
+38
View File
@@ -0,0 +1,38 @@
# HoneyBiscuitWorkshop Documentation
- [中文](zh-CN/index.md)
- [卷1 用户手册](zh-CN/vol1_user/index.md)
- [卷2 管理员手册](zh-CN/vol2_admin/index.md)
- [流水线隐参数](zh-CN/vol2_admin/pipeline_implicit_parameter.md)
- [PIPELINE_UNCOVER_SECRET](zh-CN/vol2_admin/pipeline_uncover_secret.md)
- [PIPELINE_BAREMETAL_PKGMAN](zh-CN/vol2_admin/pipeline_baremetal_pkgman.md)
- [PIPELINE_BAREMETAL_ELEVATE](zh-CN/vol2_admin/pipeline_baremetal_elevate.md)
- [PIPELINE_CONTAINER_PRIVILEGED](zh-CN/vol2_admin/pipeline_container_privileged.md)
- [环境变量](zh-CN/vol2_admin/environment_variables.md)
- [Socket 配置](zh-CN/vol2_admin/socket_config.md)
- [工作空间](zh-CN/vol2_admin/workspace.md)
- [部署](zh-CN/vol2_admin/deployment.md)
- [故障排除](zh-CN/vol2_admin/troubleshooting.md)
- [Decorator 速查表](zh-CN/vol2_admin/decorator_cheatsheet.md)
- [Prebake 配置参考](zh-CN/vol2_admin/prebake_config_reference.md)
- [Finalize 插件参考](zh-CN/vol2_admin/finalize_plugin_reference.md)
- [退出码速查](zh-CN/vol2_admin/exit_code_guide.md)
- [构建状态文件格式](zh-CN/vol2_admin/status_file_format.md)
- [卷3 开发者手册](zh-CN/vol3_dev/index.md)
- [架构](zh-CN/vol3_dev/architecture.md)
- [开发环境](zh-CN/vol3_dev/development_environment.md)
- [代码规范](zh-CN/vol3_dev/code_style.md)
- [提交规范](zh-CN/vol3_dev/commit_convention.md)
- [PR 流程](zh-CN/vol3_dev/pr_process.md)
- [错误处理模式](zh-CN/vol3_dev/error_handling_patterns.md)
- [类型系统指南](zh-CN/vol3_dev/type_system_guide.md)
- [测试策略](zh-CN/vol3_dev/testing_strategy.md)
- [插件系统指南](zh-CN/vol3_dev/plugin_system_guide.md)
- [Engine 模块指南](zh-CN/vol3_dev/engine_module_guide.md)
- [设计决策](zh-CN/vol3_dev/design_decisions.md)
- [添加包管理器](zh-CN/vol3_dev/add_package_manager.md)
- [自定义插件](zh-CN/vol3_dev/custom_plugin.md)
- [自定义 Decorator](zh-CN/vol3_dev/custom_decorator.md)
- [模板变量](zh-CN/vol3_dev/template_variables.md)
- [邮件模板](zh-CN/vol3_dev/mail_template.md)
- [Resource 模型](zh-CN/vol3_dev/resource_model.md)
View File
+1
View File
@@ -0,0 +1 @@
some
@@ -0,0 +1,44 @@
# PIPELINE_<PERMISSION_NAME>: 名称的字面意思描述
[一句话简介]
0. 预警/FOREWARN
[警告读者可能的风险]
1. 概要/Synopsis
[给出其定义与作用]
2. 场景/Scenario
[给出其常见配置场景]
3. 描述/Description
[给出其细节描述]
4. 授权/Authorize
[给出权限的获得方法]
5. 撤回/Revoke
[给出权限的撤回方法]
6. 生命周期/Lifecycle
[给出权限的生命周期描述]
7. 审计/Audit
[描述其对审计系统的影响,若无请忽略]
8. 参数/Options
[给出可配置的参数,若无请忽略]
9. 建议/Suggestions
[字面意思]
10. 常见问题/FAQ
[给出用户提出的问题。此时不应该填充任何内容。]
11~90. 自定义字段
98. 作者/Author
[字面意思,LLM的使用应该也在这里标注]
99. 参见/See Also
[给出可能有关的页面]
@@ -0,0 +1,36 @@
# Decorator 速查表
bake.sh 中所有可用的 `# @decorator` 注解:
| Decorator | 语法 | 作用 | 示例 |
|-----------|------|------|------|
| `@pipeline` | `# @pipeline` | 标记为主构建步骤 | 无参数 |
| `@after` | `# @after(func_name)` | 指定依赖的前置步骤 | `# @after(build)` |
| `@timeout` | `# @timeout(N)` | 超时时间(秒) | `# @timeout(300)` |
| `@retry` | `# @retry(count, delay)` | 失败重试次数与间隔(秒) | `# @retry(3, 10)` |
| `@if` | `# @if(condition)` | 条件执行(shell 条件表达式) | `# @if([ -f Makefile ])` |
| `@fallible` | `# @fallible` | 允许失败,不中断流水线 | 无参数 |
| `@parallel` | `# @parallel` | 与其他 `@parallel` 步骤并发执行 | 无参数 |
| `@loop` | `# @loop(N)` | 重复执行 N 次 | `# @loop(5)` |
| `@export` | `# @export` | 导出环境变量供后续步骤使用 | 无参数 |
| `@pipe` | `# @pipe(f1, f2, ...)` | 管道串联多个函数 | `# @pipe(lint, build)` |
| `@daemon` | `# @daemon` | 作为后台守护进程运行 | 无参数 |
| `@health` | `# @health(endpoint)` | 指定健康检查端点 | `# @health(/health)` |
## 语法规则
- `#``@` 之间必须有空格:`# @pipeline` 合法,`#@pipeline` 不合法
- `@name``(args)` 之间不能有空格:`@timeout(60)` 合法,`@timeout (60)` 不合法
- 每个 decorator 独占一行,位于目标函数定义的上方
## 常见组合
```bash
# @pipeline
# @after(setup)
# @timeout(600)
# @retry(2, 30)
build() {
cargo build --release
}
```
+129
View File
@@ -0,0 +1,129 @@
# 部署
## 安装
### 从源码构建
```bash
cargo build --release -p workshop-baker
```
编译输出位于 `target/release/workshop-baker`
### 下载预编译二进制
从项目 Release 页面下载对应平台的预编译二进制文件,放置到 `$PATH` 可达路径即可。
## 运行模式
Baker 支持两种运行模式:
### CLI 模式
单次执行模式,每次运行完成一个构建阶段后退出:
```bash
# 预烘焙阶段
workshop-baker -u <username> -p <pipeline> -b <build-id> bare prebake config.yml
# 烘焙阶段
workshop-baker -u <username> -p <pipeline> -b <build-id> bare bake script.sh \
--bake-base ./bake_base.sh --prebake ./prebake.yml
# 终结阶段
workshop-baker -u <username> -p <pipeline> -b <build-id> bare finalize config.yml
```
### Daemon 模式
常驻后台模式通过 Socket 监听请求,支持多构建任务管理。当前处于重构中,暂未实现。Daemon 模式下不指定 `bare` 子命令即可进入:
```bash
workshop-baker -u <username> -p <pipeline> -b <build-id>
```
## 预烘焙环境
prebake 阶段负责准备构建运行环境,支持三种方式:
### Docker
在容器内执行构建,通过配置文件指定容器参数:
```yaml
prebake:
type: docker
image: "rust:1.85"
dockerfile: "Dockerfile.build"
build_args:
- "CARGO_PROFILE=release"
```
- `image` — 指定预构建的 Docker 镜像
- `dockerfile` — 指定自定义 Dockerfile 路径(与 `image` 二选一)
- `build_args` — Docker 构建参数列表
### Firecracker
在 microVM 内执行构建,提供更强的隔离性:
```yaml
prebake:
type: firecracker
kernel: "/path/to/vmlinux"
rootfs: "/path/to/rootfs.ext4"
vcpu_count: 2
mem_size_mb: 512
```
### Baremetal
直接在宿主机运行构建,不使用容器或虚拟机隔离:
```yaml
prebake:
type: baremetal
pkgman: "apt"
elevate: true
```
## Systemd 服务
以下为 Baker Daemon 的 systemd unit 文件示例:
```ini
[Unit]
Description=HoneyBiscuitWorkshop Baker Daemon
After=network.target
[Service]
Type=simple
ExecStart=/usr/local/bin/workshop-baker -u vulcan -p default -b 0 --socket unix:///run/hbw.sock
Restart=on-failure
RestartSec=5
Environment=RUST_LOG=info
Environment=HBW_WORKSPACE=/workspace
[Install]
WantedBy=multi-user.target
```
部署步骤:
```bash
# 安装 unit 文件
sudo cp workshop-baker.service /etc/systemd/system/
sudo systemctl daemon-reload
sudo systemctl enable --now workshop-baker
```
## 权限模型
Baker 采用"先提权后降权"的权限模型:
| 阶段 | 执行用户 | 原因 |
|------|----------|------|
| prebake | root | 需要创建容器/虚拟机、挂载文件系统等特权操作 |
| bake | vulcan | 降权执行,限制构建过程权限 |
prebake 阶段以 root 身份运行,完成环境准备工作后,Baker 会将执行权限降级为 `vulcan` 用户再进入 bake 阶段,确保构建过程不会以 root 权限运行。
@@ -0,0 +1,44 @@
# 环境变量
HoneyBiscuitWorkshop 通过以下环境变量控制构建行为:
| 变量 | 说明 | 默认值 |
|------|------|--------|
| `HBW_USERNAME` | 提交者用户名(来自 base / server | — |
| `HBW_PIPELINE` | 流水线标识符 | — |
| `HBW_BUILD_ID` | 构建唯一 ID | — |
| `HBW_DRY_RUN` | 干运行模式,不执行实际构建 | — |
| `HBW_WORKSPACE` | 工作空间路径 | `/workspace` |
| `RUST_LOG` | 日志级别 (error/warn/info/debug/trace) | `warn` |
### HBW_USERNAME
提交者用户名,由上游 server / base 系统传入,标识触发本次构建的用户。与构建系统内的 Unix/Windows 用户名(如 `vulcan`)无关,仅用于日志、通知与审计追踪。
### HBW_PIPELINE
标识当前运行的流水线。在 prebake 阶段自动从配置文件获取,贯穿整个构建生命周期。
### HBW_BUILD_ID
每次构建的唯一标识符,用于日志追踪和状态标识。
### HBW_DRY_RUN
设置为任意非空值时启用干运行模式。该模式下 Baker 会解析流水线配置并生成执行计划,但不会执行实际构建操作。适用于配置验证和调试。
### HBW_WORKSPACE
构建工作空间的绝对路径。所有构建输出和中间文件都将输出到此目录下。
### RUST_LOG
控制日志输出级别,遵循 `tracing` crate 的规范:
- `error` — 仅输出错误
- `warn` — 输出警告和错误
- `info` — 输出一般信息
- `debug` — 输出调试信息
- `trace` — 输出全部跟踪信息
支持模块级别过滤,例如 `RUST_LOG=workshop_baker=trace` 仅跟踪 Baker 模块。
@@ -0,0 +1,20 @@
# 退出码速查
| 退出码 | 常量名 | 含义 | 典型触发场景 |
|--------|--------|------|-------------|
| 0 | `EXITCODE_OK` | 成功 | 流水线正常完成 |
| 1 | `EXITCODE_GENERAL_ERROR` | 通用错误 | 未分类的运行时错误 |
| 2 | `EXITCODE_INVALID_ARGS` | 参数错误 | 命令行参数或配置无效 |
| 3 | `EXITCODE_IO_ERROR` | IO 错误 | 文件读写、目录创建失败 |
| 4 | `EXITCODE_PARSE_ERROR` | 解析错误 | YAML 配置或 Decorator 语法错误 |
| 5 | `EXITCODE_EXECUTION_ERROR` | 执行错误 | 构建脚本执行失败 |
| 124 | `EXITCODE_TIMEOUT` | 超时 | 构建超过 `@timeout()` 限制 |
| 201 | `EXITCODE_PRIV_DROP_FAILED` | 权限降级失败 | 无法从 root 切换到 vulcan 用户 |
| 233 | `PIPELINE_SKIP_ERRORCODE` | 流水线跳过 | `@if` 条件不满足,步骤跳过 |
## 排查命令
```bash
# 结合日志定位具体错误
RUST_LOG=debug workshop-baker -u testuser -p my-pipeline -b 1 bare bake script.sh 2>&1 | grep -i "error\|fail"
```
@@ -0,0 +1,91 @@
# Finalize 插件参考
Finalize 阶段通过插件系统完成打包、部署与通知。
## 插件类型
| 类型 | 说明 | 配置位置 |
|------|------|---------|
| `internal` | 内置插件(随 baker 编译) | `finalize.yml``notification` 下直接配置 |
| `shell` | 可执行脚本插件 | 提供 `manifest.yml` 描述文件 |
| `dylib` | 动态链接库插件 | 通过 `dlopen` 加载 |
| `rhai` | Rhai 脚本引擎插件 | `.rhai` 脚本文件(暂未实现) |
## 内置插件清单
### mail — SMTP 邮件通知
发送构建结果邮件,支持 HTML 模板与多收件人。
| 字段 | 必需 | 类型 | 说明 |
|------|------|------|------|
| `to` | 是 | String / List | 收件人地址,支持模板变量 |
| `from` | 是 | String / Object | 发件人,单字符串为 emailObject 可含 `name``email` |
| `cc` | 否 | List | 抄送 |
| `bcc` | 否 | List | 密送 |
| `reply_to` | 否 | String | 回复地址 |
| `subject` | 否 | String | 邮件主题,支持模板语法 |
| `body` | 否 | String | 邮件正文,支持模板语法与 HTML |
| `schema` | 否 | String | `"default"``"custom"`,控制是否使用内置模板 |
| `smtp` | 是 | Object | SMTP 服务器配置 |
| `timeout` | 否 | String | 发送超时,如 `"90s"` |
| `retry` | 否 | Integer | 失败重试次数 |
| `fallible` | 否 | Boolean | 允许失败不中断流水线 |
#### smtp 子字段
| 字段 | 必需 | 说明 |
|------|------|------|
| `host` | 是 | SMTP 服务器地址 |
| `port` | 是 | 端口号 |
| `auth.type` | 否 | `"password"` / `"oauth2"` / `"none"` |
| `auth.username` | 条件 | `type=password` 时必需 |
| `auth.password` | 条件 | `type=password` 时必需 |
| `encryption` | 否 | `"tls"` / `"starttls"` / `"none"` |
| `connect_timeout` | 否 | 连接超时 |
#### 模板变量
| 变量 | 示例值 | 说明 |
|------|--------|------|
| `{{ build.status }}` | `success` | 构建状态 |
| `{{ build.id }}` | `42` | 构建编号 |
| `{{ build.duration }}` | `125s` | 构建耗时 |
| `{{ build.url }}` | `https://ci.example.com/42` | 构建详情链接 |
| `{{ pipeline.name }}` | `my-project` | 流水线名称 |
| `{{ commit.hash }}` | `abc1234` | 提交哈希 |
| `{{ commit.author }}` | `Alice` | 提交者 |
| `{{ commit.message }}` | `fix: typo` | 提交信息 |
| `{{ secret.xxx }}` | — | Vault 密钥引用 |
### webhook — HTTP 回调
向外部服务发送 HTTP POST 请求。
| 字段 | 必需 | 说明 |
|------|------|------|
| `url` | 是 | 回调地址 |
| `method` | 否 | HTTP 方法,默认 `POST` |
| `headers` | 否 | 请求头键值对 |
| `body` | 否 | 请求体,支持模板变量 |
| `timeout` | 否 | 请求超时 |
| `retry` | 否 | 失败重试次数 |
| `fallible` | 否 | 允许失败 |
## 最小配置示例
```yaml
notification:
1:
mail:
to: "admin@example.com"
from: "ci@example.com"
smtp:
host: "smtp.gmail.com"
port: 587
auth:
type: "password"
username: "{{ secret.smtp_user }}"
password: "{{ secret.smtp_pass }}"
encryption: "starttls"
```
+1
View File
@@ -0,0 +1 @@
meow?
@@ -0,0 +1,184 @@
# PIPELINE_BAREMETAL_ELEVATE:裸机环境完整提权
> [!WARNING]
> 此权限允许裸机流水线以完整 root 权限运行,可执行任意系统级操作。
> 滥用可能导致系统崩溃、数据永久丢失、安全入侵或合规违规。
> **仅在绝对必要时使用,必须经过严格审批、2FA 验证并记录完整审计。**
## 1. 概要
PIPELINE_BAREMETAL_ELEVATE 是一个流水线隐参数,用于**在裸机环境下临时授予流水线完整的 root 权限**。当该参数生效时,流水线可以执行任何系统级操作,包括修改系统配置、安装软件、操作内核模块等。
**此参数默认不开启,需要管理员通过严格流程审批并授权。**
## 2. 场景
| 适用场景 | 说明 |
|----------|------|
| 内核模块编译与加载 | 需要安装 kernel headers 并动态加载模块 |
| 系统级性能调优 | 修改 sysctl 参数、CPU 调频策略、IRQ 亲和性 |
| 硬件设备配置 | 配置 PCIe 直通、GPU 驱动安装、FPGA 编程 |
| 系统镜像定制 | 修改根文件系统、引导配置 |
| 遗留软件兼容 | 某些老旧软件硬编码需要 root 运行 |
| 不适用场景 | 说明 |
|------------|------|
| 普通软件包安装 | 请用 PIPELINE_BAREMETAL_PKGMAN |
| 语言级依赖 | 请用 DepsUser 阶段的包管理器 |
| 容器化构建 | 不需要裸机提权 |
| 日常构建 | 正常流水线不应使用 root |
## 3. 描述
### 技术本质
PIPELINE_BAREMETAL_ELEVATE 通过临时切换进程凭据实现完整提权:
1. 在 DepsSystem 阶段开始前,baker 进程的 uid 临时切换为 0 (root)
2. 所有子进程继承 root 权限
3. 可执行任何系统调用、访问任何文件、修改任何配置
4. DepsSystem 阶段结束后自动降权回普通用户
5. 提权期间的所有操作都被强制审计
### 边界
| 允许的操作 | 不允许的操作 |
|------------|--------------|
| 安装/卸载系统软件包 | 修改系统引导项(超出流水线生命周期) |
| 修改系统配置文件 | 永久关闭 SELinux/AppArmor |
| 加载/卸载内核模块 | 安装持久化后门或定时任务 |
| 操作其他用户的文件 | 删除其他用户的流水线数据 |
| 修改系统服务状态 | 修改审计系统配置 |
| 访问硬件设备 | 覆盖系统关键二进制 |
### 与其他参数的关系
| 参数 | 关系 |
|------|------|
| PIPELINE_BAREMETAL_PKGMAN | ELEVATE 包含 PKGMAN 的所有能力,两者互斥使用 |
| PIPELINE_UNCOVER_SECRET | 可同时使用,但风险叠加,需特别审批 |
| PIPELINE_CONTAINER_PRIVILEGED | 适用于不同环境,无直接关联 |
## 4. 授权
| 方式 | 适用场景 | 操作示例 |
|------|----------|----------|
| 管理员 CLI + 2FA | 紧急生产问题 | |
| 审批系统集成 | 企业级合规流程 | |
| 双人授权 | 高安全环境 | |
**授权流程**
1. 用户或管理员发起授权请求,必须填写详细理由
2. 系统要求发起人完成 2FA 验证
3. 需要至少一名其他管理员二次审批(可选,可配置)
4. 授权确认后,系统生成短期令牌随流水线下发
**授权确认示例**
```
🚨 高危操作授权请求 🚨
正在申请 PIPELINE_BAREMETAL_ELEVATE 权限
流水线: pipeline-12345
申请人: user@example.com
理由: 需要安装自定义内核模块并调优 NUMA 参数
风险确认:
- 此权限允许完整 root 访问
- 可能造成系统不稳定或数据丢失
- 所有操作将被强制审计
2FA 验证: ******
请输入 "I UNDERSTAND THE RISKS" 确认:
>
```
## 5. 撤回
| 撤回方式 | 说明 |
|----------|------|
| 手动撤回 | |
| 自动失效 | DepsSystem 阶段结束后自动降权 |
| 流水线结束 | 流水线结束时强制回收权限 |
| 强制回收 | 审计系统检测到异常行为时可立即终止流水线 |
| 超时回收 | 超过配置的最大提权时长后自动降权 |
## 6. 生命周期
| 状态 | 说明 |
|------|------|
| PENDING | 等待审批 |
| APPROVED | 已审批通过 |
| ACTIVE | 提权生效中 |
| EXPIRED | 超过最大时长或阶段结束 |
| REVOKED | 被管理员强制撤回 |
| AUDITED | 已完成审计归档 |
## 7. 审计
所有提权期间的操作都会被强制记录,且日志不可篡改:
| 字段 | 说明 |
|------|------|
| `event` | GRANTED / COMMAND_EXECUTED / FILE_ACCESS / EXPIRE |
| `user` | 申请人 |
| `approvers` | 审批人列表 |
| `pipeline_id` | 关联流水线 ID |
| `timestamp` | 操作时间 |
| `command` | 执行的完整命令行 |
| `working_dir` | 执行路径 |
| `exit_code` | 命令执行结果 |
| `accessed_files` | 访问的关键文件列表 |
| `syscalls` | 关键系统调用记录 |
**日志示例**
```json
{
"timestamp": "",
"event": "",
"pipeline_id": "",
"user": "",
"command": ""
}
```
## 8. 参数
### 全局配置
```toml
[security.baremetal_elevate]
enabled = false # 默认关闭
require_2fa = true # 必须 2FA
require_dual_approval = true # 需要双人审批
max_duration = "30m" # 最长提权时间
allowed_stages = ["DepsSystem"] # 允许提权的阶段
audit_log = true # 强制审计
notify_channels = ["email", "slack"] # 通知渠道
```
### 运行时参数
## 9. 建议
### 对用户
### 对管理员
## 10. 常见问题
## 98. 作者
## 99. 参见
@@ -0,0 +1,161 @@
# PIPELINE_BAREMETAL_PKGMAN:裸机环境包管理器权限
> [!WARNING]
> 此权限允许裸机流水线在 DepsSystem 阶段临时使用系统包管理器(需要 root 权限的操作)。
> 滥用可能导致系统环境被破坏或意外安装恶意软件包。
> **非必要不使用,使用时必须经过管理员审批并记录审计。**
## 1. 概要
PIPELINE_BAREMETAL_PKGMAN 是一个流水线隐参数,用于**在裸机环境下临时授予 DepsSystem 阶段使用系统包管理器的权限**。当该参数生效时,流水线可以执行 apt-get、pacman、yum 等包管理器命令安装构建依赖,但**不授予完整的 root 权限**。
**此参数默认不开启,需要管理员显式授予。**
## 2. 场景
| 适用场景 | 说明 |
|----------|------|
| 系统依赖安装 | 构建过程需要 gcc、make、openssl-dev 等系统包 |
| 基础环境准备 | 裸机环境缺少必要的构建工具链 |
| 内核模块编译 | 需要安装 kernel headers 等特权操作 |
| 性能调优工具 | 某些性能分析工具需要系统级安装 |
| 不适用场景 | 说明 |
|------------|------|
| 任意命令执行 | 请用 PIPELINE_BAREMETAL_ELEVATE |
| 容器化构建 | 不需要裸机包管理器 |
| 语言级依赖 | 请用 DepsUser 阶段的 cargo/pip/npm |
## 3. 描述
### 技术本质
PIPELINE_BAREMETAL_PKGMAN 通过 sudoers 白名单实现受限提权:
1. 系统在 DepsSystem 阶段开始前,为 baker 配置临时 sudo 规则
2. 仅允许执行预设的包管理器命令列表
3. 所有命令执行都被审计记录
4. DepsSystem 阶段结束后自动移除 sudo 权限
### 边界
| 允许的操作 | 不允许的操作 |
|------------|--------------|
| apt-get install | 任意 shell 命令 |
| pacman -S | 修改系统配置文件 |
| yum install | 操作其他用户的文件 |
| dnf install | 加载内核模块 |
| zypper install | 安装持久化服务 |
| 包管理器缓存更新 | 修改系统引导项 |
### 依赖关系
- 需要系统启用 `security.baremetal_pkgman.enabled`(默认开启)
- 与其他隐参数(如 UNCOVER_SECRET)可同时使用,但需评估叠加风险
- 与 BAREMETAL_ELEVATE 互斥(后者已包含包管理器权限)
## 4. 授权
| 方式 | 适用场景 | 操作示例 |
|------|----------|----------|
| 管理员 CLI | 临时授权 | |
| 审批系统集成 | 企业流程 | |
| 用户申请 | 常规需求 | |
**授权流程**
1. 管理员或用户发起授权请求
2. 系统验证请求者权限
3. 授权确认后,参数随流水线下发
**授权确认示例**
```
⚠️ 正在授予 PIPELINE_BAREMETAL_PKGMAN 权限
流水线: pipeline-12345
申请人: user@example.com
理由: 需要安装 gcc、make 等构建依赖
此权限允许:
- 在 DepsSystem 阶段使用包管理器
- 仅限预设命令列表
- 不授予其他 root 权限
确认授予?(y/N)
```
## 5. 撤回
| 撤回方式 | 说明 |
|----------|------|
| 手动撤回 | |
| 自动失效 | DepsSystem 阶段结束后自动回收 |
| 流水线结束 | 流水线结束时权限自动回收 |
| 强制回收 | 审计系统检测到异常时可终止流水线 |
## 6. 生命周期
| 状态 | 说明 |
|------|------|
| PENDING | 等待审批 |
| ACTIVE | 已授权,DepsSystem 阶段可用 |
| EXPIRED | 阶段结束或流水线结束 |
| REVOKED | 被管理员强制撤回 |
## 7. 审计
| 字段 | 说明 |
|------|------|
| `event` | GRANTED / COMMAND_EXECUTED / EXPIRED / REVOKED |
| `user` | 申请人 |
| `pipeline_id` | 关联流水线 ID |
| `timestamp` | 操作时间 |
| `command` | 执行的包管理器命令 |
| `exit_code` | 命令执行结果 |
**日志示例**
```json
{
"timestamp": "",
"event": "",
"pipeline_id": "",
"user": ""
}
```
## 8. 参数
### 全局配置
```toml
[security.baremetal_pkgman]
enabled = true
require_approval = true
default_timeout = "30m"
allowed_commands = ["apt-get", "pacman", "yum", "dnf", "zypper"]
audit_log = true
```
### 运行时参数
## 9. 建议
### 对用户
### 对管理员
## 10. 常见问题
## 98. 作者
## 99. 参见
@@ -0,0 +1,187 @@
# PIPELINE_CONTAINER_PRIVILEGED:容器环境特权模式
> [!WARNING]
> 此权限允许容器以 privileged 模式运行,使容器内的进程拥有接近宿主机的权限。
> 滥用可能导致容器逃逸、宿主机被入侵或影响其他容器。
> **仅在绝对必要时使用,必须经过严格审批并记录完整审计。**
## 1. 概要
PIPELINE_CONTAINER_PRIVILEGED 是一个流水线隐参数,用于**在容器环境下授予流水线 privileged 权限**。当该参数生效时,容器将获得所有 capabilities,并可以访问宿主机的所有设备,能够执行需要特权的操作如挂载文件系统、加载内核模块等。
**此参数默认不开启,需要管理员通过严格流程审批并授权。**
## 2. 场景
| 适用场景 | 说明 |
|----------|------|
| Docker in Docker | 需要在容器内运行 Docker 命令构建镜像 |
| FUSE 文件系统 | 需要在容器内挂载用户态文件系统 |
| 内核模块测试 | 需要在容器内加载/测试内核模块 |
| 系统调用调试 | 需要 ptrace 或其他特权系统调用 |
| 性能分析工具 | 需要访问 perf、systemtap 等性能工具 |
| 硬件设备访问 | 需要直接访问 GPU、FPGA 或其他硬件设备 |
| 不适用场景 | 说明 |
|------------|------|
| 普通软件包安装 | 请用容器镜像预装或 DepsUser 阶段 |
| 构建普通应用 | 不需要特权模式 |
| 网络抓包调试 | 请用容器网络配置而非特权模式 |
## 3. 描述
### 技术本质
PIPELINE_CONTAINER_PRIVILEGED 通过修改容器运行时配置实现提权:
1. 容器启动时添加 `--privileged` 标志
2. 容器内的 root 用户拥有宿主机 root 的完整 capabilities
3. 容器可以访问宿主机所有设备(`/dev/*`
4. 可以执行通常被容器运行时限制的操作(如 mount)
5. 权限仅在该容器生命周期内有效,容器销毁后自动回收
### 与 CAP_ADD 的区别
| 模式 | 说明 | 适用场景 |
|------|------|----------|
| PRIVILEGED | 全部 capabilities + 设备访问 | 需要完整系统级访问 |
| CAP_ADD | 添加特定 capabilities | 仅需个别特权操作 |
### 边界
| 允许的操作 | 不允许的操作 |
|------------|--------------|
| 挂载文件系统 | 修改宿主机关键配置 |
| 运行 Docker 命令 | 关闭宿主机审计系统 |
| 加载内核模块 | 删除其他容器数据 |
| 访问宿主机设备 | 修改宿主机网络配置 |
| 执行 ptrace 调试 | 安装持久化后门 |
| 配置网络命名空间 | 覆盖宿主机二进制 |
### 与其他参数的关系
| 参数 | 关系 |
|------|------|
| PIPELINE_CONTAINER_CAP_ADD | 互补关系,PRIVILEGED 是更粗粒度的权限 |
| PIPELINE_UNCOVER_SECRET | 可同时使用,需评估叠加风险 |
| PIPELINE_BAREMETAL_ELEVATE | 适用于不同环境,无直接关联 |
## 4. 授权
| 方式 | 适用场景 | 操作示例 |
|------|----------|----------|
| 管理员 CLI | 临时授权 | |
| 审批系统集成 | 企业流程 | |
| 用户申请 | 常规需求 | |
**授权流程**
1. 管理员或用户发起授权请求
2. 系统验证请求者权限并记录理由
3. 授权确认后,参数随容器启动下发给运行时
4. 容器销毁后权限自动回收
**授权确认示例**
```
⚠️ 正在授予 PIPELINE_CONTAINER_PRIVILEGED 权限
容器: pipeline-12345-container
申请人: user@example.com
理由: 需要在 CI 中构建 Docker 镜像并运行容器测试
此权限允许:
- 容器以 privileged 模式运行
- 可访问宿主机所有设备
- 可执行 mount 等特权操作
- 仅在该容器生命周期内有效
风险提示:
- 可能造成容器逃逸
- 可能影响宿主机稳定性
- 所有操作将被强制审计
确认授予?(y/N)
```
## 5. 撤回
| 撤回方式 | 说明 |
|----------|------|
| 手动撤回 | |
| 自动失效 | 容器停止时权限自动回收 |
| 流水线结束 | 流水线结束时强制销毁容器 |
| 强制回收 | 审计系统检测到异常时可终止容器 |
| 超时回收 | 超过配置的最大时长后自动终止容器 |
## 6. 生命周期
| 状态 | 说明 |
|------|------|
| PENDING | 等待审批 |
| ACTIVE | 容器以特权模式运行中 |
| EXPIRED | 容器已停止 |
| REVOKED | 被管理员强制终止 |
## 7. 审计
所有特权容器内的关键操作都会被记录:
| 字段 | 说明 |
|------|------|
| `event` | GRANTED / CONTAINER_START / CONTAINER_STOP / COMMAND_EXECUTED |
| `user` | 申请人 |
| `pipeline_id` | 关联流水线 ID |
| `container_id` | 容器 ID |
| `timestamp` | 操作时间 |
| `command` | 执行的命令 |
| `mounts` | 挂载的操作 |
| `devices_accessed` | 访问的设备 |
**日志示例**
```json
{
"timestamp": "",
"event": "",
"pipeline_id": "",
"container_id": "",
"user": ""
}
```
## 8. 参数
### 全局配置
```toml
[security.container_privileged]
enabled = false # 默认关闭
require_approval = true # 需要审批
max_duration = "60m" # 最大运行时间
allowed_images = [] # 允许特权运行的镜像白名单
audit_log = true # 强制审计
notify_on_start = true # 启动时通知管理员
```
### 运行时参数
## 9. 建议
### 对用户
### 对管理员
## 10. 常见问题
## 98. 作者
## 99. 参见
@@ -0,0 +1,12 @@
# 流水线隐参数
流水线隐参数是 server 在运行时根据权限、环境、策略动态生成的参数,
随流水线定义一起下发给执行组件。
常见隐参数:
- `PIPELINE_BAREMETAL_PKGMAN`: 允许裸机环境使用包管理器(需 root)
- `PIPELINE_CONTAINER_PRIVILEGED`: 允许容器以 privileged 模式运行
- `PIPELINE_UNCOVER_SECRET`: 允许输出 secret 明文
这些参数不在 prebake.yml 中配置,由 server 管理,
并随流水线生命周期自动生效和废弃。
@@ -0,0 +1,198 @@
# PIPELINE_UNCOVER_SECRET:临时暴露敏感信息
> [!WARNING]
> 此权限允许流水线临时输出所有 secret 的明文,包括环境变量、配置文件、构建日志中的敏感信息。
> 滥用可能导致凭据泄露、数据泄露或安全入侵。
> **非必要不使用,使用时必须经过 2FA 验证并记录审计。**
## 1. 概要
PIPELINE_UNCOVER_SECRET 是一个流水线隐参数,用于**临时解除系统对敏感信息的默认保护**。当该参数生效时,原本会被遮蔽的 secret 值将以明文形式出现在日志、标准输出和构建产物中,便于调试、审计或教学。
**此参数默认不开启,需要用户通过 2FA 显式申请并获得临时授权。**
## 2. 场景
| 适用场景 | 说明 |
|----------|------|
| 调试构建失败 | 怀疑 secret 传递错误,需要确认环境变量实际值 |
| 合规性审计 | 需要验证 secret 是否正确配置和使用 |
| 教学演示 | 展示 CI/CD 流程中 secret 的完整生命周期 |
| 遗留系统兼容 | 某些老旧脚本硬编码依赖 echo secret 的方式工作 |
| 不适用场景 | 说明 |
|------------|------|
| 日常构建 | 正常流水线不应暴露 secret |
| 生产环境调试 | 请先在预发环境复现问题 |
| 权限测试 | 请用专门的测试账号 |
## 3. 描述
### 技术本质
PIPELINE_UNCOVER_SECRET 通过以下机制实现:
1. 在流水线启动时,系统生成一个临时令牌并注入 baker 环境
2. baker 在运行期间检测到该令牌,**暂停所有输出遮蔽逻辑**
3. 所有敏感变量(包括环境变量、文件内容、构建日志)都以原始形式输出
4. 权限到期后,系统自动恢复遮蔽保护
### 边界
| 允许的操作 | 不允许的操作 |
|------------|--------------|
| 查看 secret 明文 | 将 secret 持久化到外部系统 |
| 输出 secret 到日志 | 关闭审计日志 |
| 调试脚本中的 secret 使用 | 延长权限有效期 |
### 依赖关系
- 需要用户已配置 **2FA**
- 需要系统启用 `security.uncover_secret.enabled`(默认开启)
- 与其他隐参数(如 BAREMETAL_ELEVATE)互斥使用时需注意叠加风险
## 4. 授权
| 方式 | 适用场景 | 操作示例 |
|------|----------|----------|
| 用户自申请 | 调试/审计 | `hbw pipeline run --uncover-secret` |
| 紧急授权 | 生产问题 | 需 2FA + 管理员二次确认 |
**授权流程**
1. 用户发起请求时附加 `--uncover-secret` 标志
2. 系统要求输入 2FA 验证码
3. 验证通过后显示警告并要求确认
4. 用户确认后,系统生成有效期 5 分钟的临时令牌
**授权确认示例**
```
Request for PIPELINE_UNCOVER_SECRET
2FA Code: ******
WARNING: You are about to expose ALL secrets in this pipeline.
Sensitive data may appear in logs, outputs, and artifacts.
This operation is audited. Continue? (y/N)
```
## 5. 撤回
| 撤回方式 | 说明 |
|----------|------|
| 手动撤回 | 管理员可执行 `hbw admin revoke-uncover pipeline-12345` 强制终止 |
| 自动失效 | 5 分钟有效期到达后自动恢复遮蔽 |
| 流水线结束 | 无论是否到期,流水线结束时权限自动回收 |
| 强制回收 | 审计系统检测到异常行为时可自动终止流水线并回收权限 |
## 6. 生命周期
```
申请 → 2FA验证 → 确认 → 授权 → 使用 → 失效
↑ ↑ ↑ ↑ ↑ ↑
用户 系统 用户 系统 流水线 时间/事件
```
| 状态 | 说明 |
|------|------|
| PENDING | 等待 2FA 验证 |
| ACTIVE | 已授权,secret 可明文输出 |
| EXPIRED | 超过 5 分钟有效期 |
| REVOKED | 被管理员或系统强制撤回 |
## 7. 审计
所有涉及 PIPELINE_UNCOVER_SECRET 的操作都会被详细记录:
| 字段 | 说明 |
|------|------|
| `event` | GRANTED / USED / EXPIRED / REVOKED |
| `user` | 操作人邮箱 |
| `pipeline_id` | 关联流水线 ID |
| `timestamp` | 操作时间 |
| `auth_method` | 2FA |
| `expires_at` | 授权到期时间 |
| `accessed_secrets` | 授权期间访问过的 secret 列表 |
**日志示例**
```json
{
"timestamp": "2026-03-19T10:23:45Z",
"event": "PIPELINE_UNCOVER_SECRET_GRANTED",
"user": "developer@example.com",
"pipeline_id": "pl-12345",
"auth_method": "2FA",
"expires_at": "2026-03-19T10:28:45Z"
}
{
"timestamp": "2026-03-19T10:25:12Z",
"event": "SECRET_ACCESSED",
"pipeline_id": "pl-12345",
"variable": "DOCKER_PASSWORD",
"access_type": "ENV_VAR_READ"
}
```
管理员可通过审计日志追溯:
- 谁在什么时候申请了该权限
- 授权给了哪个流水线
- 哪些 secret 在授权期间被访问
- 权限是否在有效期内
## 8. 参数
### 全局配置 (bakerd.toml)
```toml
[security.uncover_secret]
enabled = true # 是否允许使用该功能
default_timeout = "5m" # 默认授权时长
require_2fa = true # 是否强制 2FA
audit_log = true # 是否记录审计日志
max_concurrent_per_user = 1 # 每个用户同时最多可授权几个流水线
```
### 运行时参数
```bash
# 申请权限
hbw pipeline run --uncover-secret
# 指定超时(需管理员权限)
hbw pipeline run --uncover-secret --uncover-timeout 10m
# 带理由(便于审计)
hbw pipeline run --uncover-secret --reason "调试数据库连接失败"
```
## 9. 建议
### 对用户
- **非必要不使用**:能用常规调试手段解决的问题,不要依赖暴露 secret
- **用完后立即结束流水线**:避免权限窗口被意外延长
- **检查日志**:使用结束后检查是否有 secret 被意外记录到持久化存储中
- **轮换凭据**:如果确认 secret 在授权期间被暴露,建议立即轮换相关凭据
### 对管理员
- **定期审计**:每周检查 UNCOVER_SECRET 使用记录,发现异常及时处理
- **设置合理超时**:根据团队需求调整默认超时时间(建议不超过 15 分钟)
- **培训用户**:确保团队成员理解该权限的风险和使用场景
- **与审批系统集成**:对高风险操作增加二次审批流程
## 10. 常见问题
## 98. 作者
```yaml
---
authors:
---
```
## 99. 参见
@@ -0,0 +1,119 @@
# Prebake 配置参考
`prebake.yml` 定义构建环境的准备阶段。
## 顶层字段
| 字段 | 必需 | 类型 | 默认值 | 说明 |
|------|------|------|--------|------|
| `version` | 是 | String | — | 语义化版本,须满足 `^1.0` |
| `environment` | 是 | Object | — | 构建环境类型与资源配置 |
| `bootstrap` | 否 | Object | 见下文 | 用户、工作目录初始化 |
| `dependencies` | 否 | Object | — | 系统包与用户包依赖 |
| `envvars` | 否 | Object | — | 环境变量注入 |
| `cache` | 否 | Object | — | 缓存目录与策略 |
| `security` | 否 | Object | — | 权限降级配置 |
| `hooks` | 否 | Object | — | 前置/后置钩子 |
| `metadata` | 否 | Object | — | 流水线元信息 |
## environment
| 字段 | 必需 | 类型 | 默认值 | 说明 |
|------|------|------|--------|------|
| `builder` | 是 | String | — | `baremetal` / `docker` / `firecracker` / `custom` |
| `resources` | 否 | Object | 见下文 | CPU、内存、磁盘、架构约束 |
| `network` | 否 | Object | 见下文 | 网络开关与代理 |
### resources 默认值
| 字段 | 默认值 | 说明 |
|------|--------|------|
| `cpu` | `1` | vCPU 数量 |
| `memory` | `1 GiB` | 内存限制 |
| `disk` | `1 GiB` | 磁盘限制 |
| `architecture` | `[]` | 允许的目标架构,支持字符串或数组 |
| `tags` | `[]` | 节点标签筛选 |
### network
| 字段 | 默认值 | 说明 |
|------|--------|------|
| `enabled` | `true` | 是否启用网络 |
| `outbound` | `true` | 是否允许出站连接 |
| `dns_servers` | `[]` | 自定义 DNS 服务器列表 |
| `proxies` | `[]` | HTTP/HTTPS 代理配置 |
## bootstrap
| 字段 | 默认值 | 说明 |
|------|--------|------|
| `user` | `"vulcan"` | 构建用户 |
| `workspace.path` | `"/home/vulcan/workspace"` | 工作目录 |
| `workspace.fallback` | `true` | 目录不存在时自动创建 |
| `sudoers` | `null` | sudoers 文件内容 |
| `doas` | `null` | doas.conf 文件内容 |
| `custom` | `null` | 自定义初始化脚本 |
## dependencies
### system
以包管理器名为键,每个值包含:
| 字段 | 必需 | 说明 |
|------|------|------|
| `packages` | 是 | 包名列表 |
| `repositories` | 否 | 额外软件源配置 |
| `mirror` | 否 | 镜像地址 |
```yaml
dependencies:
system:
pacman:
packages: ["curl", "wget"]
```
### user
以包管理器名为键,值为自由格式(由对应 UPM 解析)。
## envvars
| 字段 | 说明 |
|------|------|
| `prebake` | 在 prebake 阶段注入的环境变量 |
| `bake` | 在 bake 阶段注入的环境变量 |
## cache
| 字段 | 必需 | 说明 |
|------|------|------|
| `directory` | 是 | 缓存目录列表,每项含 `path` 与可选 `strategy` |
| `strategy` | 否 | 缓存策略配置 |
## 完整示例
```yaml
version: "1.0"
environment:
builder: "baremetal"
resources:
cpu: 2
memory: "4 GiB"
architecture: ["x86_64", "aarch64"]
bootstrap:
user: "vulcan"
workspace:
path: "/home/vulcan/workspace"
fallback: true
dependencies:
system:
pacman:
packages: ["rust", "git"]
envvars:
prebake:
RUST_BACKTRACE: "1"
cache:
directory:
- path: "/var/cache/pacman"
```
@@ -0,0 +1,18 @@
# Socket 配置
Baker Daemon 通过 Socket 与 CLI 通信,支持以下连接方式:
- **Unix Domain Socket**: `unix://path` — 使用本地 Unix 域套接字,适用于同机通信
```bash
workshop-baker -u vulcan -p default -b 0 --socket unix:///run/hbw.sock
```
- **TCP Socket**: `tcp://host:port` — 使用 TCP 连接,适用于远程管理
```bash
workshop-baker -u vulcan -p default -b 0 --socket tcp://127.0.0.1:9090
```
- **Dryrun 模式**: `dryrun` — 测试模式,不创建实际连接
```bash
workshop-baker -u vulcan -p default -b 0 --socket dryrun
```
默认使用 `unix:///run/hbw.sock`。
@@ -0,0 +1,33 @@
# 构建状态文件格式
Baker 将构建状态写入 `/tmp/.hbwstatus`,格式为 **JSON Lines**(每行一条独立 JSON)。
## 字段说明
| 字段 | 类型 | 说明 |
|------|------|------|
| `name` | String | 阶段名称 |
| `phase` | String | 执行阶段:`prebake` / `bake` / `finalize` |
| `substage` | String | 子阶段标识 |
| `result` | String | 结果:`success` / `failure` / `canceled` / `skipped` |
| `duration_ms` | u64 | 执行耗时(毫秒) |
| `started_at` | ISO 8601 | 开始时间 |
| `finished_at` | ISO 8601 | 结束时间 |
| `error_message` | String/Null | 失败时的错误信息 |
## 示例
```json
{"name":"bootstrap","phase":"Prebake","substage":"bootstrap","result":"success","duration_ms":1500,"started_at":"2026-04-25T10:00:00Z","finished_at":"2026-04-25T10:00:01.5Z","error_message":null}
{"name":"build","phase":"Bake","substage":"build","result":"failure","duration_ms":30000,"started_at":"2026-04-25T10:01:00Z","finished_at":"2026-04-25T10:01:30Z","error_message":"cargo build failed with exit code 101"}
```
## 读取状态
```bash
# 查看最新阶段
tail -1 /tmp/.hbwstatus | jq .
# 统计失败阶段数
cat /tmp/.hbwstatus | jq -s 'map(select(.result == "failure")) | length'
```
@@ -0,0 +1,102 @@
# 故障排除
## 日志调试
启用详细日志输出以定位问题:
```bash
# 查看 Baker 详细日志
RUST_LOG=debug workshop-baker -u testuser -p my-pipeline -b 1 bare bake script.sh
# 查看特定模块日志
RUST_LOG=workshop_baker=trace workshop-baker -u testuser -p my-pipeline -b 1 bare bake script.sh
# 组合过滤
RUST_LOG=workshop_baker=debug,workshop_pipeline=trace workshop-baker -u testuser -p my-pipeline -b 1 bare bake script.sh
```
常用日志级别组合:
- `RUST_LOG=info` — 生产环境推荐
- `RUST_LOG=debug` — 一般调试
- `RUST_LOG=workshop_baker=trace` — 深度调试 Baker 内部逻辑
## 退出码对照表
| 退出码 | 含义 | 说明 |
|--------|------|------|
| 0 | OK | 构建成功完成 |
| 1 | 通用错误 | 未分类的运行时错误 |
| 2 | 参数错误 | 命令行参数或配置无效 |
| 3 | IO 错误 | 文件读写失败 |
| 4 | 解析错误 | YAML 配置或 Decorator 语法解析失败 |
| 5 | 执行错误 | 构建脚本执行失败 |
| 124 | 超时 | 构建超过 `@timeout()` 指定的时间限制 |
| 201 | 权限降级失败 | 无法从 root 切换到 vulcan 用户 |
| 233 | 流水线跳过 | 流水线条件不满足,跳过执行 |
## 常见问题与解决
### Socket 连接失败
**现象**: CLI 无法连接 Daemon,报 Connection refused 或 Permission denied。
**解决**: 检查 Unix socket 路径是否存在以及文件权限:
```bash
# 检查 socket 文件
ls -la /run/hbw.sock
# 确认 Daemon 正在运行
pgrep -a workshop-baker
# 检查 socket 权限
# 确保当前用户有读写权限,或使用 sudo
```
### Decorator 解析错误
**现象**: 构建脚本的 Decorator 注解无法识别。
**解决**: Decorator 语法必须严格遵循 `# @name(args)` 格式。常见错误:
```bash
# 错误 — @ 符号前缺少空格
#@timeout(60)
# 错误 — 名称后有空格
# @ timeout(60)
# 正确
# @timeout(60)
```
注意:`#``@` 之间必须有空格,`@name``(args)` 之间不能有空格。
### Hook 执行权限拒绝
**现象**: prebake 或 finalize 阶段的 Hook 脚本报 Permission denied。
**解决**: 检查 `vulcan` 用户的 sudoers 配置,确保其有执行必要操作的权限:
```bash
# 检查 sudoers 配置
sudo visudo -c
# 确认 vulcan 用户权限
sudo -lU vulcan
```
### 包管理器检测失败
**现象**: baremetal 模式下 Baker 无法检测系统包管理器。
**解决**: 检查 `/etc/os-release` 文件是否存在且内容完整:
```bash
# 检查文件
cat /etc/os-release
# 常见缺失情况:Docker 最小镜像或 musl 环境
# 需手动安装 base-files 或创建该文件
```
+13
View File
@@ -0,0 +1,13 @@
# 工作空间
工作空间是 Baker 执行构建的根目录,默认路径为 `/workspace`。目录结构如下:
```
/workspace/
├── src/ # 源代码检出目录
├── build/ # 编译中间输出
├── output/ # 构建最终输出
└── cache/ # 构建缓存
```
可通过 `HBW_WORKSPACE` 环境变量或配置文件覆盖默认路径。
@@ -0,0 +1,41 @@
# 添加包管理器
要支持新的系统包管理器,需要实现 `PackageManager` trait。
```rust
trait PackageManager {
fn name(&self) -> &str;
fn install(&self, packages: &[&str]) -> Result<()>;
fn update(&self) -> Result<()>;
}
```
参考 `workshop-baker/src/engine/pm/pacman.rs` 中的实现。步骤如下:
1.`workshop-baker/src/engine/pm/` 下创建新文件(如 `yum.rs`)。
2. 实现 `PackageManager` trait 的所有方法。
3.`pm/mod.rs` 中注册新变体到 `PackageManagerEnum`
4. 在配置解析中增加对应的匹配分支。
## 示例:pacman 实现
```rust
pub struct PacmanManager;
impl PackageManager for PacmanManager {
fn name(&self) -> &str { "pacman" }
fn install(&self, packages: &[&str]) -> Result<()> {
let args = vec!["-S", "--noconfirm"].iter()
.chain(packages.iter())
.collect::<Vec<_>>();
Command::new("pacman").args(&args).status()?;
Ok(())
}
fn update(&self) -> Result<()> {
Command::new("pacman").args(["-Syu", "--noconfirm"]).status()?;
Ok(())
}
}
```
+87
View File
@@ -0,0 +1,87 @@
# 架构
## 整体架构
当前活跃维护的 crate
- **workshop-baker** 是核心编排器,负责流水线的调度与执行,提供 CLI 和 Daemon 两种运行模式。
- **workshop-vault** 封装 HashiCorp Vault 客户端,提供密钥的读写与引用解析。
- **workshop-cert** 负责生成自签名 TLS 证书。
- **workshop-deviceid** 提供设备唯一标识,用于多节点环境下的节点识别。
其余模块(如 workshop-agent、workshop-llm-detector、workshop-pipeline 等)已归档至 `archived/` 目录,不再活跃维护。
## 数据流
流水线分为三个阶段,顺序执行:
1. **Prebake(环境准备)**:根据 prebake.yml 配置创建构建环境,支持 Docker、Firecracker 和 Bare Metal 三种模式。包括包安装、用户创建、安全降权等。
2. **Bake(脚本执行)**:加载 bake.sh 脚本,解析 `# @decorator` 注解,进行拓扑排序后按依赖顺序执行构建步骤。
3. **Finalize(后处理)**:执行打包、部署、通知等收尾工作,通过插件系统完成邮件发送、Webhook 回调等操作。
三个阶段通过 `ExecutionContext` 串联,上下文信息在阶段间传递。
## 核心模块详解
### Engine
Engine 是 Bake 阶段的核心执行引擎,主要包含:
- **Executor**:负责脚本的执行,管理子进程的生命周期和输出捕获。
- **PackageManager**:包管理抽象 trait,支持 pacman、apt 等多种包管理器的统一调用。
### Bake
Bake 模块处理构建脚本解析和调度:
- **Parser**:解析 bake.sh 中的 `# @decorator` 注解,提取超时、重试、并行等指令。
- **Schedule**:基于依赖关系对构建步骤进行拓扑排序,支持并行执行无依赖的步骤。
- **Builder**:使用模板引擎渲染构建脚本,支持变量替换和条件逻辑。
### Prebake
Prebake 模块负责构建环境准备:
- **Config**:解析 prebake.yml,提取环境类型(Docker/Firecracker/Bare Metal)和配置项。
- **Bootstrap**:生成环境启动脚本,包括系统初始化和工具安装。
- **Security**:处理权限降级,创建构建用户(默认 vulcan)并设置目录权限。
### Finalize
Finalize 模块完成后处理工作:
- **Plugin**:插件系统,支持 Shell、Dylib、Rhai 和 Internal 四种插件类型。
- **Hook**:在流水线关键节点执行钩子函数,如构建成功后的通知发送。
## 类型系统
核心类型贯穿三个阶段:
- **ExecutionContext**:在 Prebake → Bake → Finalize 间传递的上下文对象,包含环境信息、变量和构建状态。
- **ResourceLimits**:定义 CPU、内存、磁盘等资源限制,用于 CgroupManager 配置。
- **BuildStatus**:表示构建状态(Pending/Running/Success/Failed/Cancelled),贯穿整个生命周期。
## 插件架构
插件系统基于 `Plugin` trait 和 `AsyncPluginFn` 类型别名:
```rust
trait Plugin {
async fn execute(&self, ctx: &ExecutionContext) -> Result<()>;
}
```
### 内部插件
内部插件随 baker 编译,直接调用 Rust 代码:
- **Mail**:基于 SMTP 发送构建通知邮件,支持 minijinja 模板。
- **Webhook**:向外部服务发送 HTTP 回调。
### 外部插件
外部插件运行在 baker 进程之外:
- **Shell**:可执行脚本,通过子进程调用,需提供 `manifest.yml` 描述文件。
- **Dylib**:动态链接库,通过 `dlopen` 加载并调用导出函数。
- **Rhai**:使用 Rhai 脚本引擎执行 `.rhai` 脚本文件。
+30
View File
@@ -0,0 +1,30 @@
# 代码规范
## 命名约定
| 类型 | 风格 | 示例 |
|------|------|------|
| 文件/模块 | snake_case | `engine.rs`, `pm/` |
| 类型/结构体/枚举 | PascalCase | `PipelineConfig`, `BuildStatus` |
| 常量 | SCREAMING_SNAKE_CASE | `MAX_RETRIES`, `DEFAULT_TIMEOUT` |
| 函数/变量 | snake_case | `parse_decorator`, `build_id` |
## 导入顺序
按以下顺序组织 `use` 语句,每组之间空一行:
1. `std` 标准库
2. 外部 crate
3. `crate::` 内部模块
```rust
use std::path::PathBuf;
use std::process::Command;
use anyhow::Result;
use serde::Deserialize;
use tokio::time::sleep;
use crate::bake::Parser;
use crate::engine::Executor;
```
@@ -0,0 +1,20 @@
# 提交规范
提交信息使用类型前缀:
| 前缀 | 说明 |
|------|------|
| `feat` | 新功能 |
| `fix` | Bug 修复 |
| `docs` | 文档变更 |
| `test` | 测试相关 |
| `refactor` | 重构 |
| `style` | 代码风格(不影响功能) |
示例:
```
feat(baker): add timeout support for build steps
fix(agent): fix TLS certificate reload on SIGHUP
docs: update contributing guide
```
@@ -0,0 +1,32 @@
# 自定义 Decorator
在 bake.sh 脚本中,`# @decorator` 注解用于控制构建行为。添加自定义 Decorator 的步骤:
1.`workshop-baker/src/bake/decorator.rs` 中的 `Decorator` enum 添加新变体:
```rust
pub enum Decorator {
Timeout { ms: u64 },
Retry { count: u32 },
Parallel,
Fallible,
MyCustom { value: String }, // 新增
}
```
2.`parse_decorator` 函数的 match 中添加解析逻辑:
```rust
fn parse_decorator(line: &str) -> Option<Decorator> {
match decorator_name {
"timeout" => Some(Decorator::Timeout { ms: ... }),
"retry" => Some(Decorator::Retry { count: ... }),
"parallel" => Some(Decorator::Parallel),
"fallible" => Some(Decorator::Fallible),
"my_custom" => Some(Decorator::MyCustom { value: ... }), // 新增
_ => None,
}
}
```
3. 在调度器中处理新 Decorator 的执行逻辑。
+43
View File
@@ -0,0 +1,43 @@
# 自定义插件
## Shell 插件
Shell 插件是最简单的外部插件形式,只需提供一个可执行脚本和 `manifest.yml`
**可执行脚本** (`my-plugin.sh`)
```bash
#!/bin/bash
echo "Running custom plugin"
```
**manifest.yml 格式**
```yaml
name: my-plugin
version: "1.0"
type: shell
command: ./my-plugin.sh
timeout: 300
env:
MY_VAR: "hello"
```
字段说明:
| 字段 | 说明 |
|------|------|
| name | 插件名称 |
| version | 插件版本 |
| type | 插件类型:`shell` / `dylib` / `rhai` |
| command | 可执行文件路径(shell 类型) |
| timeout | 超时时间(秒) |
| env | 环境变量映射 |
## Dylib 插件
动态链接库,通过 `dlopen` 加载并调用约定符号的导出函数。需确保 ABI 兼容。
## Rhai 插件
使用 Rhai 脚本引擎执行 `.rhai` 文件。当前为 `todo!()` 占位,尚未实现。
+412
View File
@@ -0,0 +1,412 @@
# 设计决策
本文档记录了 HoneyBiscuitWorkshop 架构演进过程中的关键设计决策及其背后的思考。
> **注意**:本文档面向开发者,记录的是**目标架构**的设计方向,而非当前代码的实现现状。
> 当前 bakerd 尚未完全实现(`daemon.rs` 为 `todo!()`),bare mode 仅用于开发调试。
---
## 1. Bare Mode 与 Daemon Mode
### 决策
**Bare mode 仅用于开发调试,不承载生产语义。**
### 背景
当前代码提供两种运行入口:
| 模式 | 入口 | 状态 | 用途 |
|------|------|------|------|
| **Bare mode** | `cargo run -- prebake/bake/finalize <args>` | 可用 | 开发调试 |
| **Daemon mode** | `cargo run --` (无子命令) | `todo!()` | 生产 |
### 推论
- Bare mode 中 `ctx.privileged` 是写死的 fake 值(`Prebake: true`, `Bake: false`),不代表生产行为
- Bare mode 中 `os_info::get()` 读取的是 HOST 的 OS 信息,不代表环境内的 OS
- 不要从 bare mode 的行为推导生产路径的问题
- 不要为了"修复" bare mode 的问题而给生产设计增加不必要的抽象
### 代码残留
当前 `bake.rs` 中存在以下已在设计层面淘汰、但因 bare mode 未被清理的代码:
- `fn privileged()` — 从未被调用
- `type TaskResult` — 从未被使用
- `type TaskFn` — 从未被使用
这些是 `bake.rs` 废弃功能的残留,不涉及生产设计。清理它们不改变任何行为。
---
## 2. 环境模型
### 决策
**bakerd 先创建环境(容器/VM/裸机),然后 baker 在环境内执行全部三个阶段(prebake → bake → finalize)。**
### 架构
```
bakerd
├── prepare_environment (容器/VM/裸机) ← 先建隔离环境
├── prebake (全在环境内执行) ← 包括 depssystem 的包安装
├── bake (全在环境内执行) ← 构建脚本
├── finalize (全在环境内执行) ← 部署通知
└── cleanup
```
### 推论
- `prepare_environment` 不是 prebake 的一个 stage——它是 prebake 的前置条件
- `depssystem.rs``os_info::get()` 读取的应当是**环境内**的 `/etc/os-release`,逻辑正确
- 容器镜像只需拉取(`prepare_container`),不需要在代码层面管理容器生命周期——那是 bakerd 的职责
- 当前 `prepare_container` 返回的 `_container` 被丢弃,是因为 bakerd 尚未实现,不是设计缺陷
### 环境类型
```yaml
env:
type: container # Docker/Podman 容器
# type: firecracker # Firecracker 微 VM
# type: baremetal # 裸机直接执行
image: ubuntu:24.04
```
环境声明决定了:
1. 隔离级别(进程级 / VM级 / 无隔离)
2. 操作系统和工具链(通过 `image`
3. 包管理器(从 OS 推导)
---
## 3. PM 选择链
### 决策
**包管理器(PM)应从环境声明中推导,而不是通过运行时探测。**
### 选择链
```
用户声明 bakerd 确定 执行时确定
env.image: ubuntu:24.04
容器运行时 (Docker/VM)
│ os_info::get() 在环境内
OS: Ubuntu 24.04
│ 从已知映射推导
PM: apt
│ depssystem 使用 PM trait 操作
apt install -y {packages}
```
### 为什么淘汰 detect()
`pm::detect()` 的当前逻辑:
```rust
pub fn detect(distro: &os_info::Info) -> Option<Box<dyn PackageManager>> {
all_managers().into_iter().find(|pm| pm.adopt(distro))
}
```
它在解决一个**已经知道答案的问题**:
- 用户声明了 `image: ubuntu:24.04`
- 容器内 `/etc/os-release` 确认了 Ubuntu
- `detect()` 遍历所有 PM 并匹配,得出 apt
- **结果和环境声明一致,但绕了一大圈**
`detect()` 的唯一价值场景:
- **Bare mode** 用户没配环境时,省一个配置项
- 这在生产路径中不应出现(bakerd 总是提供环境)
### 推论
- PM trait 保留(见下节),但 `adopt()` 方法在生产路径中不需要
- `detect()` / `adopt()` / `os_info::get()` 这条路径应当是 dev-only 的便利设施
- Daemon 模式中 PM 的选择依据是环境声明,不是运行时探测
- 当前 `os_info::Type` 的 58 个变体覆盖不全(仅 Pacman 可用)——这是 bare mode 的问题,不影响生产设计
---
## 4. PM Trait 的职责边界
### 决策
**PM trait 保留核心操作能力,淘汰探测分类能力。**
### 核心方法
```rust
pub trait PackageManager {
fn name(&self) -> &'static str;
fn update(&self) -> Vec<Command>;
fn install(&self, package: &[String]) -> Vec<Command>;
fn change_mirror(&self, repo: &str, osinfo: Option<&Info>) -> Vec<Command>;
fn add_repository(&self, repo: &Value, osinfo: Option<&Info>) -> Vec<Command>;
}
```
这四个方法是 PM-specific 的操作,具有真实复杂度:
| 操作 | PM 间差异 | 原因 |
|------|-----------|------|
| `install` | 参数完全不同 | `apt install -y` vs `pacman -S --noconfirm` |
| `update` | 命令不同 | `apt update` vs `pacman -Sy` |
| `change_mirror` | 配置格式不同 | sources.list vs mirrorlist vs .repo |
| `add_repository` | 仓库机制不同 | PPA vs AUR vs COPR |
### 为什么镜像源和仓库在容器中仍然需要
容器不解决源管理问题:
```
docker pull ubuntu:24.04
# 容器内 sources.list → archive.ubuntu.com
# 但用户在北京 → 需要 mirror.tuna.tsinghua.edu.cn
# 这不是预置在镜像中的——源是运行态配置
```
`change_mirror``add_repository` 都是**容器运行时需要**的操作,不能预置在基础镜像中。PM trait 的存在价值就在于这些操作。
### 淘汰的方法
```rust
fn adopt(&self, distro: &Info) -> bool; // 生产路径不需要
fn binary(&self) -> &'static str; // 从未被使用
```
### 实现策略
在 daemon 模式下,PM 的选型由环境声明确定(`image: ubuntu:24.04``apt`),PM 实现仅需按需注册,不需要自述适配范围:
```rust
// 不再需要 detect() + adopt()
// PM 实例由工厂方法根据 PM 名称字符串创建
pub fn from_name(name: &str) -> Option<Box<dyn PackageManager>> {
match name {
"apt" => Some(Box::new(Apt)),
"pacman" => Some(Box::new(Pacman)),
"dnf" => Some(Box::new(Dnf)),
_ => None,
}
}
```
---
## 5. 发行版多态
### 决策
**流水线不应是发行版多态的。发行版差异由容器化解决。**
### 原因
流水线的执行体(bake 脚本)天然发行版绑定:
```bash
# @pipeline
function build() {
cmake -B build -DCMAKE_INSTALL_PREFIX=/usr # Linux 路径
make -j$(nproc)
}
```
- 换到 macOS`/usr` 只读,路径不同
- 换到 WindowsMSVC 而不是 GCC
- 换到 Alpinemusl 而不是 glibc
这与 PM 的差异是同一类问题,但 PM 抽象不能解决它。**容器才是解决方案:**
```yaml
# 在 Ubuntu 上构建
env:
type: container
image: ubuntu:24.04
```
```yaml
# 在 Alpine 上构建
env:
type: container
image: alpine:3.19
```
### 各层面的发行版相关性
| 层面 | 发行版相关 | 解决方案 |
|------|-----------|----------|
| DepsSystem(系统包) | ✅ | 容器内 PM |
| DepsUser(用户包管理器) | ❌ | 天然无关(cargo/npm/pip 跨发行版一致) |
| Bake(构建脚本) | ✅ | 容器内执行 |
| Finalize(后处理) | ❌ | 纯产物操作,无关发行版 |
### UPM 的特殊位置
用户包管理器(UPM,如 cargo、npm、pip、nix)天然跨发行版:
```bash
cargo install --locked bat # Ubuntu 和 Arch 上都一样
npm install -g typescript # Ubuntu 和 Arch 上都一样
```
`depsuser.rs` 处理的正是这个层面。但 `collect_upm_sysdeps()` 需要知道系统 PM——这是 UPM 和系统 PM 之间的 bridge。在设计上,这个 bridge 的 PM 信息应来自环境声明,而非 `detect()`
---
## 6. drop_after 安全模型
### 决策
**`drop_after` 指定流水线在哪个 stage 之后从 root 降权到 vulcan 用户。合理的默认值是 `DepsUser`。**
### Stage 顺序
```rust
pub enum PrebakeStage {
Init,
Bootstrap,
EarlyHook,
DepsSystem, // ← 默认
DepsUser,
LateHook,
Ready,
Never, // ← 永远不降权
}
```
### 各 stage 的特权需求
| Stage | 需要 root | 原因 |
|-------|-----------|------|
| Bootstrap | ✅ | 创建系统用户(vulcan)、配置 sudo/doas |
| EarlyHook | ❓ | 用户自定义 |
| DepsSystem | ✅ | `apt install` 需要 root |
| DepsUser | ❌ | `cargo install` 不需要 root |
| LateHook | ❓ | 用户自定义 |
| Ready / Bake | ❌ | 构建应以非特权用户运行 |
### 正确的配置
```yaml
security:
drop_after: "DepsUser"
```
执行效果:
```
Bootstrap root ← 创建用户
EarlyHook root ← hook 以 root 运行
DepsSystem root ← apt install -y {packages}
DepsUser root ← cargo install {packages}
LateHook vulcan ← 降权,用户脚本不能 rm -rf /
Ready vulcan
Bake vulcan ← 构建以 vulcan 运行
```
### 禁止的配置
`drop_after` 设置为 `Bootstrap``EarlyHook` 在当前 stage 顺序下会损坏 DepsStage(需要 root 权限的 `apt install` 在降权后无法执行)。
**配置验证应拒绝 `drop_after < DepsUser`**(以 root 运行的 user 级包安装没有意义,但 DepsSystem 必须 root)。
### 特殊值 Never
```yaml
security:
drop_after: "Never" # 整条流水线以 root 运行
```
适用场景:Docker 构建、内核模块编译等需要 root 权限的操作。Bootstrap 的 `user: root` 配置也会产生相同效果。
---
## 7. 当前代码中的真问题
以下问题是当前代码中确实存在的,与 bare/daemon 模式无关:
### 7.1 Apt::adopt 永远返回 false
```rust
// workshop-engine/src/pm/apt.rs:20-23
fn adopt(&self, distro: &Info) -> bool {
return false; // ← 永远 false
// todo!(); // ← 死代码
}
```
- `adopt()``return false` 后无法执行 `todo!()`
- Apt 永远不会被 `detect()` 选中
- 后果:Debian/Ubuntu 用户在 bare mode 下永远检测不到 PM
- 修复方向:移除 return false,实现真正的 adopt 逻辑(Debian/Ubuntu/Mint/Pop 等发行版的匹配)
### 7.2 PM 覆盖严重不全
当前 `all_managers()` 仅注册了 Pacman
```rust
pub fn all_managers() -> Vec<Box<dyn PackageManager>> {
vec![
// Box::new(apt::Apt), // adopt 坏了
Box::new(pacman::Pacman), // 唯一可用
// Box::new(dnf::Dnf), // 未实现
]
}
```
os_info 3.14.0 定义了 58 种发行版类型,Pacman 仅覆盖其中 9 种(Arch 系)。其余 49 种无 PM 覆盖。
### 7.3 死代码
| 位置 | 内容 | 状态 |
|------|------|------|
| `bake.rs:98` | `fn privileged()` | 定义但未调用 |
| `bake.rs:108` | `type TaskResult` | 定义但未使用 |
| `bake.rs:110` | `type TaskFn` | 定义但未使用 |
| `workshop-baker/src/prebake/stage/environment.rs` | `prepare_environment()` | 定义但未接入执行流 |
---
## 8. Daemon 设计原则(汇总)
以上讨论最终可以提炼为以下设计原则,供 bakerd 重构时参考:
### P1. 环境声明驱动
```
用户声明 env.image → bakerd 创建环境 → 环境内确定 PM → depssystem 操作 PM
```
PM 的选择权在环境声明,不在运行时探测。`detect()` 不出现在生产路径中。
### P2. 容器是发行版隔离的边界
发行版差异在容器边界被隔断。容器内是已知、确定的环境。PM trait 只负责"已知 PM 的操作"install、update、mirror、repo),不负责"猜是什么 PM"。
### P3. 降权边界可配置但需校验
`drop_after` 从 root 降权到 vulcan,默认在 DepsUser 之后。配置验证需确保:降权不会损坏需要 root 的 stage。
### P4. 最小化 trait 抽象
PM trait 保留操作系统所需的核心方法。不在 trait 中嵌入分类功能(adopt)。分类是一个数据映射问题,不是多态问题。
### P5. Bare mode 是开发工具,不承载生产行为
Bare mode 的行为(host 上直接执行、write static privileged 等)不代表生产设计。不要为了优化 bare mode 而增加生产路径的抽象负担。
### P6. 天然跨发行版的部分不需要抽象
UPMcargo/npm/pip/nix)跨发行版一致,不需要 PM 级别的抽象,不需要 detect,不需要 distro 感知。trait 只应用于确实有 PM-specific 差异的部分。
@@ -0,0 +1,15 @@
# 开发环境
本项目要求:
- **Rust 1.85+**(使用 edition 2024
- **Docker**(集成测试需要)
- **mdBook**(文档构建,可选)
初始化开发环境:
```bash
git clone https://github.com/user/HoneyBiscuitWorkshop.git
cd HoneyBiscuitWorkshop
cargo build
```
@@ -0,0 +1,57 @@
# Engine 模块指南
`engine/` 是 Baker 的运行时层,负责构建脚本执行、资源隔离与包管理。
## 模块划分
| 文件 | 职责 |
|------|------|
| `executor.rs` | 脚本执行,管理子进程生命周期与输出捕获 |
| `cgroups.rs` | Linux cgroup v2 资源限制配置(CPU、内存、IO) |
| `pm.rs` | 系统包管理器检测与抽象 trait |
| `pm/pacman.rs` | Pacman 包管理器实现 |
| `upm.rs` | 用户级包管理器(Nix、Guix、Cargo、NPM 等) |
| `repology.rs` | Repology 包名查询与版本解析 |
| `repology/local.rs` | 本地包数据库缓存 |
| `socket.rs` | Unix Domain Socket 通信(daemon 模式) |
## PackageManager Trait
系统包管理器的统一接口:
```rust
trait PackageManager {
fn name(&self) -> &str;
fn install(&self, packages: &[&str]) -> Result<()>;
fn update(&self) -> Result<()>;
}
```
新增包管理器时:
1.`pm/` 下新建文件实现 `PackageManager`
2.`pm.rs``PackageManagerEnum` 中注册变体
3.`PrebakeConfig` 解析中增加匹配分支
## 资源限制
`cgroups.rs` 通过 Linux cgroup v2 限制构建资源:
| 限制项 | cgroup 文件 | 配置来源 |
|--------|-------------|---------|
| CPU 配额 | `cpu.max` | `resources.cpu` |
| 内存上限 | `memory.max` | `resources.memory` |
| 磁盘 IO | `io.max` | 预留,暂未绑定 |
## 执行流程
1. `Executor` 接收渲染后的脚本路径
2. 根据 `PrebakeConfig` 创建 cgroup 限制(如启用)
3. `std::process::Command` 启动子进程
4. 实时捕获 stdout/stderr
5. 超时或完成后,收集退出码与输出
## 注意事项
- `cgroups.rs` 仅在 Linux 上生效,非 Linux 目标编译时相关代码被条件编译排除
- `upm.rs` 将用户包管理器调用委托给子进程,不直接管理包状态
- `repology.rs` 通过 HTTP 查询 Repology API 将通用包名映射到发行版特定名称
@@ -0,0 +1,60 @@
# 错误处理模式
Baker 采用 `thiserror` + `anyhow` 的组合策略处理错误。
## 选取原则
| 场景 | 使用 | 原因 |
|------|------|------|
| 库代码(types/、bake/、engine/ | `thiserror` | 结构化错误,调用方需要 match 处理 |
| 二进制入口(main.rs | `anyhow` | 统一包装,快速传播 |
| 测试代码 | `anyhow::Result` | 减少样板,失败即 panic |
## thiserror 示例
```rust
use thiserror::Error;
#[derive(Error, Debug)]
pub enum BakeError {
#[error("Decorator parse error @{decorator} (line {line_num}): {reason}")]
DecoratorParseError {
decorator: String,
line_num: usize,
reason: String,
},
#[error("Circular dependency: {0:?}")]
CircularDependency(Vec<String>),
#[error("IO error: {0}")]
IoError(#[from] std::io::Error),
}
```
## anyhow 示例
```rust
use anyhow::{Context, Result};
fn read_config(path: &str) -> Result<String> {
std::fs::read_to_string(path)
.with_context(|| format!("Failed to read config: {}", path))
}
```
## 退出码映射
实现 `HasExitCode` trait 将错误映射到标准退出码:
```rust
impl HasExitCode for BakeError {
fn exit_code(&self) -> i32 {
match self {
BakeError::DecoratorParseError { .. } => EXITCODE_PARSE_ERROR,
BakeError::IoError(_) => EXITCODE_IO_ERROR,
_ => EXITCODE_GENERAL_ERROR,
}
}
}
```
+26
View File
@@ -0,0 +1,26 @@
# 卷3 开发者手册
## 项目简介
蜜饼工坊 (HoneyBiscuitWorkshop) 是一个 Rust 编写的 CI/CD 系统,支持 LLM 辅助的流水线自动配置。
## 代码结构
当前活跃维护的 crate
| Crate | 说明 |
|-------|------|
| workshop-baker | 主编排器 (CLI + Daemon) |
| workshop-vault | 密钥管理客户端 |
| workshop-cert | TLS 证书生成器 |
| workshop-deviceid | 设备 ID 库 |
其余模块(如 workshop-agent、workshop-llm-detector 等)已归档至 `archived/` 目录,不再活跃维护。
## 阅读指南
本卷面向开发者,涵盖架构、扩展和贡献指南。各章节内容如下:
- **架构**:整体架构、数据流、核心模块与插件系统
- **扩展**:添加包管理器、自定义插件与 Decorator、模板语法
- **贡献指南**:开发环境、代码规范、测试与提交流程
+26
View File
@@ -0,0 +1,26 @@
# 邮件模板
`MailConfig` 中的 `template` 字段支持 minijinja 语法,用于自定义邮件内容:
```yaml
mail:
smtp_host: "smtp.example.com"
smtp_port: 587
from: "ci@example.com"
to: ["dev@example.com"]
template: |
Subject: [CI] {{PROJECT_NAME}} - {{BUILD_STATUS}}
项目: {{PROJECT_NAME}}
分支: {{BRANCH}}
提交: {{COMMIT_SHA}}
状态: {{BUILD_STATUS}}
{% if BUILD_STATUS == "success" %}
构建成功!
{% else %}
构建失败,请检查日志。
{% endif %}
```
邮件模板中可使用所有 ExecutionContext 提供的模板变量。
@@ -0,0 +1,74 @@
# 插件系统指南
Baker 的插件系统基于 `Plugin` trait,支持四种运行时形态。
## Plugin Trait
```rust
trait Plugin {
async fn execute(&self, ctx: &ExecutionContext) -> Result<()>;
}
```
所有插件接收统一的 `ExecutionContext`,包含任务 ID、构建状态、环境变量与模板变量表。
## 内部插件
随 baker 二进制编译,直接调用 Rust 代码,无额外启动开销。
| 插件 | 文件 | 功能 |
|------|------|------|
| `mail` | `finalize/plugin/internal/mail.rs` | SMTP 邮件通知,支持 minijinja 模板 |
| `webhook` | `finalize/plugin/internal/webhook.rs` | HTTP POST 回调 |
| `satori` | `finalize/plugin/internal/satori.rs` | Satori 集成(占位) |
| `insitenotify` | `finalize/plugin/internal/insitenotify.rs` | 站内通知(占位) |
内部插件配置位于 `finalize.yml``notification` 节点下,按数字键分组:
```yaml
notification:
1:
mail:
to: "admin@example.com"
2:
webhook:
url: "https://hooks.example.com/ci"
```
## 外部插件
### Shell 插件
可执行脚本,通过子进程调用。需在同级目录提供 `manifest.yml`
```yaml
name: "my-plugin"
type: "shell"
entry: "run.sh"
```
### Dylib 插件
动态链接库,通过 `dlopen` 加载并调用约定符号的导出函数。
### Rhai 插件
使用 Rhai 脚本引擎执行 `.rhai` 文件(当前为 `todo!()` 占位,尚未实现)。
## 插件注册流程
1. `finalize.yml` 中声明插件配置
2. `FinalizeStage::Plugin` 阶段遍历配置
3. 根据 `type` 字段实例化对应插件类型
4. 调用 `Plugin::execute()`,传入当前 `ExecutionContext`
## 模板上下文
所有插件共享同一模板变量表,由 `ExecutionContext` 提供:
| 命名空间 | 可用变量 |
|----------|---------|
| `build` | `status`, `id`, `duration`, `url`, `status_color` |
| `pipeline` | `name` |
| `commit` | `hash`, `author`, `author_email`, `message` |
| `secret` | Vault 中存储的任意密钥(通过 `{{ secret.key }}` 引用) |
+11
View File
@@ -0,0 +1,11 @@
# PR 流程
提交 PR 前请确保通过所有检查:
```bash
cargo fmt --all -- --check
cargo clippy -- -D warnings
cargo test
```
然后提交并创建 Pull Request。CI 会自动运行完整测试套件。
+88
View File
@@ -0,0 +1,88 @@
# Resource 模型
## 核心思想
整个系统只关心两件事:东西从哪来(fetch),东西到哪去(publish)。
fetch 方向:URL → 本地路径 → 使用。
publish 方向:本地路径 → 变换(插件)→ URL。
两个方向共用同一套寻址方式,但 Resource 中间态只存在于 fetch 方向。
## URL 格式
```
getter::url?param1=value1&param2=value2
```
getter 显式声明意图。不是"我猜这是个 git 仓库",而是"使用者告诉我这是什么"。
只有三个 getter
- `git` — 克隆仓库,必要时 checkout 指定版本
- `http` / `https` — 下载文件,必要时校验 checksum
- `file` — 复制本地文件或目录
版本锁定不用 `@v1.0``@` 在 URL 里是认证分隔符),用 `?ref=v1.0`
没有 `github.com/user/repo` 看起来应该 clone 它这种黑魔法。
没有 `.tar.gz` 看起来应该解压它的惊喜。
URL 是 URL,行为是行为,两者用 getter:: 显式绑定。
## Resource 不是网络地址
流水线阶段(prebake / finalize / notify)只接触 Resource:一个名字 + 一个本地路径,已就绪。它们不经由网络、不解析 URL、不碰 getter。
所有 fetch 行为在阶段启动前完成:
- worker 模式:bakerd fetch → 填入 ResourceRegistry → 启动 baker
- standalone 模式:baker 自己 fetch → 填入 ResourceRegistry → 执行阶段
- bare 模式:不 fetch,传入的必须是 Resource
ResourceRegistry 是名字到路径的映射。填进去就只读了。
## getterurl 是独立的 crate
URL 格式解析 + 基本 fetch 放在 workshop-getterurl,和 baker 本身没有耦合。任何人都可以用。
crate 分两层:
```
结构层(parser):
"git::https://host/repo.git?ref=v1.0" → { getter: "git", url: Url, params: Params }
获取层(builtin getters):
getter.fetch(url, params, dst) → local_path
git: git clone + optional checkout
http: HTTP GET + optional checksum
file: std::fs::copy
```
获取层是字面意义的——不解压、不推断、不猜意图。
## Checksum
checksum 作为 URL query 参数传递。三态语义:
- `?checksum=sha256:abc` — 必须匹配
- `?checksum=` — 显式禁用校验
- 无 checksum — 如果服务端返回 Content-Digest 头,校验它
getter 不主动校验,它只执行 URL 里指定的操作。
## Artifact 不经过 Resource
artifact 是 bake 的产物,自产生就开始被使用,不存在中间态。它的路径就是 bake 产出的本地路径,不走 ResourceRegistry。
artifact 的 publish 方向由插件定义行为——可能上传、可能扫描、可能只记录。插件定义最终的行为,只要是自洽的。
## 三种运行模式
```
fetch 执行者 fetch 在阶段内
daemon 模式 bakerd 否
standalone baker 自身 否(阶段启动前已完成)
bare 模式 无 否(必须传入 Resource
```
bare 模式下传入的必须是已经就绪的 Resource。外部 URL 是 daemon 或 standalone 的事,bare 不负责。
@@ -0,0 +1,23 @@
# 模板变量
Finalize 阶段使用 `{{VARIABLE}}` 语法进行模板渲染,可用变量来自 `ExecutionContext`
| 变量 | 说明 |
|------|------|
| `{{PROJECT_NAME}}` | 项目名称 |
| `{{BUILD_ID}}` | 构建ID |
| `{{BUILD_STATUS}}` | 构建状态 |
| `{{COMMIT_SHA}}` | 提交哈希 |
| `{{BRANCH}}` | 分支名 |
| `{{START_TIME}}` | 构建开始时间 |
| `{{END_TIME}}` | 构建结束时间 |
模板引擎基于 minijinja,支持条件、循环等高级语法:
```yaml
{% if BUILD_STATUS == "success" %}
Build {{PROJECT_NAME}}/{{BRANCH}} succeeded!
{% else %}
Build {{PROJECT_NAME}}/{{BRANCH}} failed.
{% endif %}
```
@@ -0,0 +1,61 @@
# 测试策略
## 测试层级
| 层级 | 位置 | 适用场景 | 工具 |
|------|------|---------|------|
| 单元测试 | `src/*/mod.rs``#[cfg(test)]` | 纯逻辑、无 IO、无副作用 | `#[test]` / `#[tokio::test]` |
| 集成测试 | `tests/*.rs` | 跨模块协作、crate 公共 API | `cargo test --test name` |
| 系统测试 | `tests/` (pytest) | Docker 环境、端到端 | `pytest` |
## 单元测试规范
```rust
#[cfg(test)]
mod tests {
use super::*;
#[test]
fn test_parse_valid_duration() {
assert_eq!(parse_duration_to_ms("30s").unwrap(), 30_000);
}
#[test]
fn test_parse_duration_overflow_fails() {
let result = parse_duration_to_ms("9999999999y");
assert!(result.is_err());
}
}
```
## 异步测试
```rust
#[tokio::test]
async fn test_async_execution() {
let result = execute_command("echo hello").await.unwrap();
assert_eq!(result.stdout, "hello\n");
}
```
## 集成测试规范
- 使用 `use workshop_baker::*;` 导入 crate 公共 API
- 每个测试独立,不依赖执行顺序
- 临时文件使用 `tempfile` crate,测试后自动清理
## 运行命令
```bash
# 全部测试
cargo test -p workshop-baker
# 仅单元测试
cargo test --lib -p workshop-baker
# 仅指定集成测试
cargo test --test types_config_integration
# 系统测试(需 Docker
pytest tests/integration/test_prebake.py -v
```
@@ -0,0 +1,53 @@
# 类型系统指南
`types/` 模块集中定义跨模块共享的核心类型,避免循环依赖与重复定义。
## 设计原则
| 原则 | 说明 |
|------|------|
| 单一事实来源 | 同一概念只在 `types/` 定义一次,各模块通过 `use crate::types::` 引用 |
| 自包含序列化 | 复杂类型自带 serde 自定义序列化/反序列化逻辑,调用方无感知 |
| 零成本抽象 | 偏好新类型模式(如 `MemSize(u64)`)而非裸原生类型 |
## 核心类型速查
| 类型 | 文件 | 用途 |
|------|------|------|
| `Architecture` | `architecture.rs` | 目标架构枚举,支持 serde 单字符串或数组 |
| `MemSize` | `memsize.rs` | 内存/磁盘大小,支持 `"1 GiB"` / `"512 MiB"` 等人类可读格式 |
| `BuilderConfig` | `builderconfig.rs` | 构建环境配置(Docker/Firecracker/Baremetal/Custom |
| `CompressionMethod` | `compression.rs` | 压缩算法枚举 |
| `CacheStrategy` / `CacheDirectory` | `cache.rs` | 缓存策略与目录配置 |
| `CustomCommand` | `command.rs` | 自定义命令及其超时配置 |
| `GitVersion` | `repology.rs` | Git 引用版本解析(branch/tag/commit |
| `RepologyEndpoint` | `repology.rs` | Repology 包查询端点配置 |
| `BuildStatus` | `buildstatus.rs` | 流水线整体状态 |
| `StagePhase` / `StageResult` | `buildstatus.rs` | 阶段执行阶段与结果 |
## Serde 定制示例
### 单值或数组兼容反序列化
`Architecture` 支持 YAML 中写 `"x86_64"``["x86_64", "aarch64"]`
```rust
fn parse_architecture<'de, D>(deserializer: D) -> Result<HashSet<Architecture>, D::Error>
where D: serde::Deserializer<'de>
{
// 实现 Visitor,同时处理 visit_str 与 visit_seq
}
```
### 人类可读大小解析
`MemSize``"4 GiB"` 解析为底层字节数:
```rust
let mem: MemSize = "4 GiB".parse()?; // MemSize(4294967296)
```
## 使用约束
- `types/` 不依赖 `bake/``engine/``prebake/``finalize/` 中的任何模块
- 新增跨模块共享类型时,应优先放入 `types/` 而非散落在业务模块中
Submodule rust-tongsuo deleted from 6248690a45
+85
View File
@@ -0,0 +1,85 @@
#!/bin/bash
#Implicit set -euo pipefail
# function to be used by any pipeline stage
function get_custom_library() {
curl -LO https://archive.ppy.sh/$2/$1.tar.gz
}
# @pipeline
prepare_dotnet() {
curl -L https://dot.net/v1/dotnet-install.sh | bash
bake_info "Successfully installed dotnet"
}
# @pipeline
fetch_source() {
git clone https://github.com/ppy/osu.git --depth 1
}
# @pipeline
# @if(WS_PIPELINE_ARCH == amd64)
# @timeout(720)
# @retry(3, 5)
# @parallel
fetch_library() {
get_custom_library ffmpeg amd64
get_custom_library libesqlite3 amd64
get_custom_library libbass amd64
}
# @pipeline
# @fallible
# @parallel
fetch_osu_framework() {
cd ..
git clone https://github.com/ppy/osu-framework.git --depth 1
}
# @pipeline
# @timeout(600)
restore_workload() {
cd osu
dotnet workload restore
}
# @pipeline
# @export
set_environment() {
export DOTNET_ENVIRONMENT=Production
}
# @pipeline
# @export
build() {
dotnet build osu.Desktop
dotnet publish osu.Desktop -o osu-build --sc
}
# @pipeline
# @loop(3)
test() {
dotnet test
}
# @pipeline
# @parallel
# @after(fetch_library)
compress_library() {
mkdir -p osu-native
cp osu-build/*.so ../osu-native
tar -cf osu-native osu-native.tar
zstd osu.tar
}
# @pipeline
# @after(test)
compress_artifact() {
tar -cf osu osu.tar
zstd osu.tar
}
+349
View File
@@ -0,0 +1,349 @@
# finalize.yml.tmpl
#
# 蜜饼工坊 (HoneyBiscuitWorkshop) 构建流水线 - 完成阶段配置
#
# 版本号,用于格式兼容性检查
# 版本迁移系统内置,预期 1.0 后不再变更 schema
version: "0.0.1"
# =============================================================================
# 插件定义
# =============================================================================
# 插件在 finalize 阶段执行特定任务(如推送镜像、上传制品等)
# 插件可以是 Shell 脚本或 Rhai 脚本,由文件扩展名自动检测
#
# 插件来源格式:
# git://<git-url>@<tag>/<commit> - Git 仓库(在 prebake 阶段拉取)
# https://<url> - HTTPS 下载(支持脚本直接下载)
# file://<path> - 本地文件(在 finalize.begin 阶段加载)
# http://<url> - HTTP 下载(计划支持)
# hg://<url> - Mercurial 仓库(计划支持)
#
# 插件隔离:
# - Docker/VM 环境:允许任意插件,由环境提供隔离
# - Bare-metal 环境:需要额外权限配置(PIP: Pipeline Implicit Parameters
#
# 插件通信:
# - Shell 脚本:通过命令行参数和环境变量接收配置
# - Rhai 脚本:通过注入的对象/函数接收配置(WIP)
# - 返回值:通过退出码表示成功/失败
# - 日志:会被捕获但不作为执行证据
plugin:
# 插件逻辑名称,用于在 artifact.publish 和 notification 中引用
# 名称必须唯一,不可重复定义
docker-push:
# 插件来源 URI,包含版本锁定
# @main/1234abcd 表示 main 分支的 1234abcd 提交
# 如果该提交不在指定分支中,则执行失败
"git://github.com/user/hbw-finalize-docker-push@main/1234abcd":
# 认证主机,仅用于认证目的
# 如果 publish 中指定了 image 且包含主机部分,此处可省略
host: "docker.io"
# 用户凭证,支持模板变量
# {{ variable.xxx }} 为用户配置值
# {{ secret.xxx }} 为从 Vault 获取的密钥
user: {{ variable.dockerhub.user }}
token: {{ secret.dockerhub.token }}
# fallible: 是否允许失败
# true - 插件失败不会导致构建失败,错误不会向上传播
# false - 插件失败会导致构建失败
# 语义与 bake.sh 的 @fallible 装饰器相同
fallible: false
# 镜像安全检查插件示例(HTTPS 下载)
image-check:
"https://example.com/trivy/trivy.sh":
user: {{ variable.trivy.user }}
password: {{ secret.trivy.password }}
fallible: true
# 短信通知插件示例(本地 Rhai 脚本)
sms-notify:
"file://.workshop/plugins/sms-notify.rhai":
provider: "aliyun"
access_key: {{ secret.aliyun.sms_key }}
access_secret: {{ secret.aliyun.sms_secret }}
# =============================================================================
# 通知配置
# =============================================================================
# 通知采用优先级组机制:
# - 优先级为整数,范围 0 ~ INT_MAX
# - 数值越小优先级越高
# - 同一优先级组内的通知方式并行执行
# - 当组内任一方式失败时,触发下一优先级组(fallback)
# - fallible: true 可阻止错误传播,不触发 fallback
#
# 系统级 fallback(在用户配置之外):
# 1. 站内通知(存储在数据库,显示在 Baker Dashboard
# 2. 系统邮件(用户创建时配置)
# 3. 放弃通知,管理员接收告警(如已配置)
#
# 优先级组为空时会产生警告,但不会自动修正
# 建议将优先级设为 0、1、2... 等连续值
notification:
templates:
sms-tmpl1:
on_success: "Build success!"
on_failure: "Build failed!"
on_finish_of: "Build stage {{ build.status }} finished."
sms-tmpl2:
use: "git://https://gitee.com/example_user/example_template_for_sms.git@a1b2c3d4"
# 最高优先级组 - 首选通知方式
0:
# Webhook 通知
webhook:
url: "https://example.com/hook"
template:
# schema 定义负载格式:
# - "default": 使用系统(ws-base)指定的默认格式,standalone模式不可用,client模式的默认
# - "fallback": 使用插件指定的默认格式,standalone模式的默认
# - "custom": 自定义格式,需要提供全部 on_* 模板
# - "<name>": 使用templates节中定义的名字
# 当任何on_*被设置,则等效于replace模式:schema做基底,on_*做替换
schema: "default"
# Satori 协议通知(如 Koishi
satori:
# TODO: 需要大量重构
# 次优先级组 - 备用通知方式(当优先级 0 中任一方式失败时触发)
1:
# 邮件通知(内置插件)
mail:
to: "user@gmail.com"
# TODO: 替换为实际mail里的字段
schema: "default"
# 插件也可以直接作为通知方式使用
# 需要插件支持通知触发器(on_success/on_failure/on_finish_of
sms-notify:
to: "+8610012345678"
policy:
- trigger:
- on_success
template:
schema: "custom"
on_success: "[HBW Notify] Build Succeeded: {{ build.status }} ({{ build.duration }})"
to: "+8610012345678"
- trigger:
- on_success
- on_failure
template:
schema: "sms-tmpl1"
# on_success: "[HBW Notify] Build Succeeded: {{ build.status }} ({{ build.duration }})"
# on_failure: "[HBW Notify] Build Failed: {{ build.status }} ({{ build.duration }})"
to: "+8610012345679"
- trigger:
- on_finish_of:
- "bake.build"
- "bake.clean.*"
- 're:bake\.prepare\.*'
template:
schema: "custom"
on_finish_of: "[HBW Notify] Build Progress: {{ build.status }} ({{ build.duration }})"
to: "+8610012345677"
fallible: true
# =============================================================================
# 制品定义
# =============================================================================
# 制品是构建产生的输出文件,可被存储、发布或清理
#
# 制品 ID
# - 可选,如未指定则由系统自动生成
# - 格式:MD5(PUID + PATH)
# - PUID: Pipeline Unique Identifier
#
# 制品路径:
# - 支持通配符,会被编译为正则表达式
# - 在配置加载时检查语法,在执行时编译和匹配
# - 总限制时间 1 秒(编译 + 匹配)
# - 一个路径匹配 = 一个制品元素
# - 空路径表示非文件系统制品(如 Docker 镜像)
#
# 制品传输:
# - 通过 hard-link/rsync 传输到插件工作空间
# - 插件可自定义传输方式
artifact:
# 示例制品 1:普通文件制品
- # 制品 ID,可选
# 如不指定,系统自动生成 MD5(PUID + PATH)
id: "<SOME_ARTIFACT_ID>"
# 制品路径,支持 glob 模式
# 匹配的每个文件/目录作为独立制品
path: "/workspace/target/release/*"
# 保留时长
# TODO
retention: "7d"
# 压缩方式:
# - "zstd": Zstandard 压缩(推荐)
# - "gzip": Gzip 压缩
# - "none": 不压缩
# - "dir": 保留为目录结构(用于需要完整目录树的场景)
compression: "zstd"
# 触发条件:
# - "success": 构建成功时处理
# - "failure": 构建失败时处理(WIP
# - "always": 总是处理(WIP
# 支持多个条件
on:
- success
# 示例制品 2:Docker 镜像制品
- id: "<ANOTHER_ARTIFACT_ID>"
# 空路径表示非文件系统制品
# 用于 Docker 镜像、远程制品等
# 此类制品的"内容"由 publish 定义
path: ""
retention: "5d"
compression: "none"
on:
- success
# 发布配置
# 制品可以发布到一个或多个目标
# 多个发布目标并行执行(除非受 Token 约束)
publish:
# 引用 plugin 部分定义的插件名称
docker-push:
# 覆盖/补充插件配置
# 镜像名称,包含 registry 地址
# 如 plugin 中已定义 host,此处可省略 registry 部分
image: "docker.io/user/image:tag"
# 同一制品可以发布到多个插件
# 通过 Token/Lock 机制协调访问
image-check:
image: "docker.io/user/image:tag"
# =============================================================================
# 钩子定义
# =============================================================================
# 在 finalize 阶段执行的前置/后置钩子
# 可用于清理、预处理、后处理等操作
#
# 执行时机:
# - early: 制品收集之前执行
# - late: 所有操作完成后执行
#
# 命令类型:
# - 普通命令:直接执行 shell 命令
# - 插件命令:以 "plugin:" 开头,调用已定义的插件
hooks:
# 早期钩子:在制品收集之前执行
early:
- # 钩子名称,用于日志和调试
name: "cleanup temp files before artifact collection"
# 执行命令
command: "rm **/.tmp"
# 工作目录
working_dir: "/workspace"
# 插件类型钩子示例
- name: "Early hook internal plugin"
command: "plugin:early-hook-internal"
# 超时时间(秒)
timeout: 600
# 后期钩子:在所有操作完成后执行
late:
- name: "Build completion notification"
command: "echo 'Build completed!'"
# =============================================================================
# 清理配置
# =============================================================================
# 清理策略控制构建完成后的资源回收
# WIP: 更多选项待设计
cleanup:
# 清理策略:
# - "auto": 自动清理(默认)
# - "manual": 手动清理
# - "on_failure": 仅失败时清理(WIP
policy: "manual"
# =============================================================================
# 模板变量参考
# =============================================================================
# finalize.yml 支持以下模板变量
#
# 用户配置:
# {{ variable.xxx }} - 用户在 Baker 中配置的值
#
# 密钥(来自 Vault):
# {{ secret.xxx }} - 从 Vault 获取的密钥值
# - 仅存在于内存中,不写入磁盘
# - 日志中自动遮蔽
# - 需要 PIPELINE_UNCOVER_SECRET 权限才能查看(需 2FA
#
# 构建信息:
# {{ build.status }} - 构建状态(动态,表示最后确认的状态)
# {{ build.duration }} - 构建耗时
# {{ build.stage }} - 当前/最后阶段
#
# 目标信息:
# {{ target.xxx }} - 目标相关变量
#
# 注意:
# - 变量可用性取决于执行阶段
# - 在 prebake.ready 阶段,build.duration 可能不可用
# - 使用不可用变量时行为 TBD
# =============================================================================
# 与 prebake.yml 和 bake.sh 的关系
# =============================================================================
# 完整的 HBW 流水线包含三个阶段:
#
# 1. prebake (prebake.yml)
# - 环境准备:Docker/Firecracker/Bare-metal
# - 依赖安装:系统包、语言包
# - 环境变量设置
# - 缓存配置
# - 输出:准备好的构建环境
#
# 2. bake (bake.sh)
# - 构建执行:编译、测试、打包
# - 通过 @decorator 控制执行流程
# - 输出:构建产物
#
# 3. finalize (finalize.yml)
# - 制品收集:收集构建产物
# - 制品发布:推送到目标位置
# - 通知发送:通知构建结果
# - 资源清理:回收临时资源
# - 输出:发布的制品、通知记录
#
# 数据流:
# prebake → bake → finalize
# 环境变量从 prebake 传递到 bake
# 制品从 bake 传递到 finalize
# Secret 在所有阶段可用(通过 {{ secret.xxx }}
# =============================================================================
# 示例:完整流水线配置
# =============================================================================
# 项目 .workshop 目录通常包含:
# - prebake.yml 或 prebake.yml.tmpl
# - bake.sh 或 bake.sh.tmpl
# - finalize.yml 或 finalize.yml.tmpl
# - .env 环境变量(不应包含 secret)
#
# 模板文件(.tmpl 后缀)包含占位符,会在首次运行时生成实际配置
# 系统会通过 LLM 或模板引擎填充这些占位符
# vim: set ft=yaml ts=2 sw=2 et:
+243
View File
@@ -0,0 +1,243 @@
# prebake.yaml.tmpl
#
# 术语说明:
# 可选:该字段可以不给出
# 默认为...:隐含“可选”
# 留空:尚未定义,留作后续
# 版本号,用于格式兼容性检查
# 到目前为止,"1.0"的版本号并不意味着schema就这么确定下来
version: "1.0"
# 构建环境定义
environment:
# 构建机类型:docker | firecracker | custom | baremetal
builder: "docker"
# 构建机配置,与builder一致的被启用。
baremetal: {}
docker:
# 镜像名,与dockerfile互斥
image: "rust:1.70-slim"
# Dockerfile路径,相对于项目根目录,与image互斥
dockerfile: "/Dockerfile"
# Docker构建参数,可选
build_args:
RUST_VERSION: "1.70"
firecracker: {}
# TODO: 定义firecracker结构
custom:
# 自定义构建机名称
name: "my-custom-builder"
# 设置脚本,相对于项目根目录
setup_script: "setup-builder.sh"
# 清洁脚本,相对于项目根目录
cleanup_script: "cleanup-builder.sh"
# 硬件资源要求,满足最小要求的构建机可以被选中
resources:
cpu: 2 # CPU核心数,默认为1
memory: "2G" # 内存大小,也可以写作2048MB,默认为"1G"
disk: "10G" # 磁盘空间,也可以写作10240MB,默认为"1G"
architecture: "x86_64" # 架构要求,只能是"noarch"或{"x86_64"/"amd64", "arm64"/"aarch64", "riscv64", "loongarch64"/"loong64"}中的一个或几个
# TODO: 重新设计GPU字段
# gpu: false # 是否需要GPU,默认为false
# gpu_type: "cuda" # GPU类型(如果需要),可以是"cuda", "rocm", "vulkan", "oneapi"中的一个或几个
tags: # 自定义标签,只有满足标签的构建机可以被选中
- "custom_tag"
# 网络配置
network:
enabled: true # 启用网络,对baremetal不适用,默认为true
outbound: true # 允许出站连接,对baremetal不适用,默认为true
dns_servers: # 指定构建机的DNS服务器,对baremetal不适用,custom应该自行处理DNS问题,可选
- "8.8.8.8"
- "1.1.1.1"
proxies: # 代理配置,暂时留空,可选
# 构建环境初始化
bootstrap:
# 构建用户,默认为 vulcan(火神)
# 设定为 "root" 表示忽略用户切换并禁用安全特性,需要 PIPELINE_PRIVILEGED 权限
user: "vulcan"
# 工作空间
workspace:
# 工作目录路径
path: "/home/vulcan/workspace"
# 路径不可用时回退到 /workspace
# 即使路径可用,也会在 /workspace 建立软链接
fallback: true
# 自定义 sudoers 文件内容
# 这会覆盖默认行为,可能绕过安全机制
# 需要 PIPELINE_CUSTOM_SUDOERS 权限
sudoers: "vulcan ALL=(ALL:ALL) NOPASSWD: ALL"
# 自定义 doas 文件内容
# 这会覆盖默认行为,可能绕过安全机制
# 与 sudoers 共享 PIPELINE_CUSTOM_SUDOERS 权限
doas: "permit nopass vulcan as root"
# 自定义 bootstrap 脚本(与上述所有字段互斥)
# 支持多种来源:
# workspace:///.workshop/bootstrap.sh - 项目内文件
# file:///bin/bootstrap.sh - 环境内文件
# server://bootstrap.sh - 由服务器提供
# https://example.com/bootstrap.sh - http/https网络下载,请确保构建环境可以连接到该URL
# (content) - 直接给出内容,当文本开头无法匹配任何scheme时采用
# 需要 PIPELINE_CUSTOM_BOOTSTRAP 权限
custom: "server://bootstrap-alpine.sh"
# 依赖定义,安装依赖时按system-user顺序进行
dependencies:
# Repology 端点配置,用于将 UPM 包名解析为系统包名
# 可选值:disabled, none, local, remote, server, default
# - disabled: 完全禁用 Repology
# - none: 不解析,直接使用原始包名
# - local: 使用本地 Repology 服务器
# - remote: 使用远程 Repology API
# - server: 由服务器提供 Repology 服务
# - default: 默认行为(通常等同于 local)
config:
repology_endpoint: "default"
# 系统包依赖,按包管理器枚举给出,只应该给出与环境契合的字段。
# 根据security.drop-after的配置,在此阶段一般具有root或免密root权限
# 不适用于裸机
system:
apt:
packages: # 安装的软件包
- "git"
- "curl"
- "build-essential"
- "pkg-config"
- "libssl-dev"
repositories: # 该字段按serde Value原样传递给插件,config.rs不做解析
- url: "ppa:custom/ppa" # 自定义PPAUbuntu
- url: "https://download.docker.com/linux/ubuntu" # DEB822
key: "https://download.docker.com/linux/ubuntu/gpg"
mirror: "https://mirrors.tuna.tsinghua.edu.cn" # 指定软件包镜像源,可选
pacman:
packages:
- "git"
- "curl"
- "base-devel"
- "pkg-config"
- "openssl"
repositories:
- server: "https://mirrors.tuna.tsinghua.edu.cn/arch4edu/$arch" # 自定义仓库(Arch
name: "arch4edu"
keypackage: "arch4edu-keyring"
mirror: "https://mirrors.tuna.tsinghua.edu.cn" # 指定软件包镜像源,可选
#dnf:
# 略
# 用户依赖
# 根据security.drop-after的配置,在此阶段一般不具有root权限
# 例外:语言依赖会在system中产生隐式依赖
user:
# 以下字段按serde Value原样传递给插件,config.rs不做解析
# rust:
# type: "rustup" # 此时会向系统引入隐式的rustup依赖
# toolchain: "nightly-x86_64-unknown-linux-gnu"
# manifest: "Cargo.toml"
# lock: "Cargo.lock"
# python:
# type: "uv"
# python: "python3.14"
# index: "http://mirrors.tuna.tsinghua.edu.cn/pypi/web/simple"
# directory: ".venv"
# env: # 优先,高级选项
# UV_PYTHON: "python3.14"
# UV_INDEX: "http://mirrors.tuna.tsinghua.edu.cn/pypi/web/simple"
# UV_DIRECTORY: ".venv"
# manifest: "requirements.txt" # 或pyproject.toml
# nix: {}
# # TBD: 没用过lol
# # PR welcome
custom: # 绝对的自定义,与上述全部配置互斥(custom独占)
- command: "cargo fetch"
working_dir: "/tmp"
# 环境变量配置
envvars:
# 构建环境变量,只在 prebake 阶段有效
# 用于控制 prebake 阶段的行为(apt/cargo/rustc 等)
prebake:
RUST_BACKTRACE: "full" # Rust 调试
CARGO_NET_GIT_FETCH_WITH_CLI: "true" # Cargo 配置
NODE_ENV: "production" # Node 环境
# 运行时环境变量(会传递给 bake 阶段)
# 这些变量会与项目中的 .env 文件合并后,作为最终环境
bake:
# 支持 secret 引用,格式: {{ secret.<key> }}
DATABASE_URL: "postgresql://user:{{ secret.dbpassword }}@localhost/db"
LOG_LEVEL: "info"
# 特殊变量:指定项目中 .env 文件的位置
# 系统会读取该文件,并将其中的变量与上面定义的变量合并
# 合并规则:prebake.yml 中定义的变量优先级更高,会覆盖 .env 中的同名变量
# 此变量本身不会传递给 bake 阶段
# .env不应该包含任何 secret
__ENV_FILE: ".env"
# 注意:
# - secret 只存在于内存中,不会写入磁盘
# - 日志中会自动遮蔽 secret 的值
# - 如需临时查看 secret,可使用 PIPELINE_UNCOVER_SECRET 权限(需 2FA
# 缓存配置
cache:
# 缓存目录
directory:
- path: "/usr/local/cargo"
strategy: "always" # 在"always","readonly","clean"和"never"中选择一个,可以被流水线启动时配置覆盖,默认为"always"
mode: "zstd" # 在"gzip","zstd","none"和"dir"中选择一个
- path: "/root/.npm"
strategy: "clean"
mode: "none"
- path: "/app/target"
strategy: "always"
mode: "dir"
# 缓存策略
strategy:
ttl_days: 30 # 缓存生存时间
max_size_gb: 20 # 最大缓存大小
cleanup_policy: "lru" # 清理策略:lru, fifo, size
# 安全配置
security:
#
drop-after: ""
# 留空
# 钩子脚本
hooks:
# 依赖安装前
early:
- command: "echo 'Starting dependency installation'"
name: ""
# 依赖安装后
late:
- command: "cargo fetch"
working_dir: "/tmp"
# 元数据
metadata:
description: "Rust project build environment"
maintainer: "team@example.com"
# vim: set ft=yaml ts=2 sw=2 et:
+107
View File
@@ -0,0 +1,107 @@
# This file is automatically @generated by Cargo.
# It is not intended for manual editing.
version = 4
[[package]]
name = "itoa"
version = "1.0.18"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "8f42a60cbdf9a97f5d2305f08a87dc4e09308d1276d28c869c684d7777685682"
[[package]]
name = "memchr"
version = "2.8.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "f8ca58f447f06ed17d5fc4043ce1b10dd205e060fb3ce5b979b8ed8e59ff3f79"
[[package]]
name = "proc-macro2"
version = "1.0.106"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "8fd00f0bb2e90d81d1044c2b32617f68fcb9fa3bb7640c23e9c748e53fb30934"
dependencies = [
"unicode-ident",
]
[[package]]
name = "quote"
version = "1.0.45"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "41f2619966050689382d2b44f664f4bc593e129785a36d6ee376ddf37259b924"
dependencies = [
"proc-macro2",
]
[[package]]
name = "serde"
version = "1.0.228"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "9a8e94ea7f378bd32cbbd37198a4a91436180c5bb472411e48b5ec2e2124ae9e"
dependencies = [
"serde_core",
"serde_derive",
]
[[package]]
name = "serde_core"
version = "1.0.228"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "41d385c7d4ca58e59fc732af25c3983b67ac852c1a25000afe1175de458b67ad"
dependencies = [
"serde_derive",
]
[[package]]
name = "serde_derive"
version = "1.0.228"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "d540f220d3187173da220f885ab66608367b6574e925011a9353e4badda91d79"
dependencies = [
"proc-macro2",
"quote",
"syn",
]
[[package]]
name = "serde_json"
version = "1.0.149"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "83fc039473c5595ace860d8c4fafa220ff474b3fc6bfdb4293327f1a37e94d86"
dependencies = [
"itoa",
"memchr",
"serde",
"serde_core",
"zmij",
]
[[package]]
name = "syn"
version = "2.0.117"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "e665b8803e7b1d2a727f4023456bbbbe74da67099c585258af0ad9c5013b9b99"
dependencies = [
"proc-macro2",
"quote",
"unicode-ident",
]
[[package]]
name = "unicode-ident"
version = "1.0.24"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "e6e4313cd5fcd3dad5cafa179702e2b244f760991f45397d14d4ebf38247da75"
[[package]]
name = "workshop-baker-params"
version = "0.1.0"
dependencies = [
"serde",
"serde_json",
]
[[package]]
name = "zmij"
version = "1.0.21"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "b8848ee67ecc8aedbaf3e4122217aff892639231befc6a1b58d29fff4c2cabaa"
+8
View File
@@ -0,0 +1,8 @@
[package]
name = "workshop-baker-params"
version = "0.1.0"
edition = "2024"
[dependencies]
serde = { version = "1.0.228", features = ["derive"] }
serde_json = "1.0.149"
+54
View File
@@ -0,0 +1,54 @@
use crate::apply_field;
use serde::{Deserialize, Serialize};
use crate::{ParamVal, Writer, Overridden, traits::MergeParams};
#[derive(Debug, Clone, Serialize, Deserialize)]
#[serde(deny_unknown_fields)]
pub struct BakeParams {
#[serde(default)] pub workspace: ParamVal<String>,
/// Health probe interval in seconds (default 5).
#[serde(default)] pub health_period: ParamVal<u64>,
/// Health probe per-attempt timeout in seconds (default 10).
#[serde(default)] pub health_timeout: ParamVal<u64>,
/// Max health probe retries before giving up (default 12).
#[serde(default)] pub health_max_retries: ParamVal<u32>,
}
impl MergeParams for BakeParams {
fn defaults() -> Self {
Self {
workspace: ParamVal::new("/workspace".into()),
health_period: ParamVal::new(5),
health_timeout: ParamVal::new(10),
health_max_retries: ParamVal::new(12),
}
}
fn apply(&mut self, i: Self, w: Writer, ov: &mut Overridden, p: &str) {
apply_field!(self.workspace, i.workspace, w, ov, p, "workspace");
apply_field!(self.health_period, i.health_period, w, ov, p, "health_period");
apply_field!(self.health_timeout, i.health_timeout, w, ov, p, "health_timeout");
apply_field!(self.health_max_retries, i.health_max_retries, w, ov, p, "health_max_retries");
}
}
impl Default for BakeParams { fn default() -> Self { Self::defaults() } }
#[derive(Debug, Clone)]
pub struct BakeSettings {
pub workspace: String,
pub health_period: u64,
pub health_timeout: u64,
pub health_max_retries: u32,
}
impl Default for BakeSettings { fn default() -> Self { BakeParams::defaults().into() } }
impl From<&BakeParams> for BakeSettings {
fn from(p: &BakeParams) -> Self {
Self {
workspace: p.workspace.value.clone(),
health_period: p.health_period.value,
health_timeout: p.health_timeout.value,
health_max_retries: p.health_max_retries.value,
}
}
}
impl From<BakeParams> for BakeSettings { fn from(p: BakeParams) -> Self { (&p).into() } }
+23
View File
@@ -0,0 +1,23 @@
use crate::apply_field;
use serde::{Deserialize, Serialize};
use crate::{ParamVal, Writer, Overridden, traits::MergeParams};
#[derive(Debug, Clone, Serialize, Deserialize)]
#[serde(deny_unknown_fields)]
pub struct FinalizeParams {
#[serde(default)] pub artifact_retention_days: ParamVal<u32>,
}
impl MergeParams for FinalizeParams {
fn defaults() -> Self { Self { artifact_retention_days: ParamVal::new(7) } }
fn apply(&mut self, i: Self, w: Writer, ov: &mut Overridden, p: &str) {
apply_field!(self.artifact_retention_days, i.artifact_retention_days, w, ov, p, "artifact_retention_days");
}
}
impl Default for FinalizeParams { fn default() -> Self { Self::defaults() } }
#[derive(Debug, Clone)]
pub struct FinalizeSettings { pub artifact_retention_days: u32 }
impl Default for FinalizeSettings { fn default() -> Self { FinalizeParams::defaults().into() } }
impl From<&FinalizeParams> for FinalizeSettings { fn from(p: &FinalizeParams) -> Self { Self { artifact_retention_days: p.artifact_retention_days.value } } }
impl From<FinalizeParams> for FinalizeSettings { fn from(p: FinalizeParams) -> Self { (&p).into() } }
+168
View File
@@ -0,0 +1,168 @@
pub mod notification;
pub mod prebake;
pub mod bake;
pub mod finalize;
pub mod writer;
pub mod param;
pub mod traits;
pub use notification::{NotificationParams, NotificationSettings};
pub use prebake::{PrebakeParams, PrebakeSettings};
pub use bake::{BakeParams, BakeSettings};
pub use finalize::{FinalizeParams, FinalizeSettings};
pub use writer::Writer;
pub use param::ParamVal;
pub use traits::MergeParams;
use serde::{Deserialize, Serialize};
use std::collections::HashMap;
/// Root parameter tree. Each component gets its own subtree.
/// Used for both IPC (Serialized to JSON) and in-process merging.
#[derive(Debug, Clone, Serialize, Deserialize, Default)]
pub struct Params {
#[serde(default, skip_serializing_if = "Option::is_none")]
pub notification: Option<NotificationParams>,
#[serde(default, skip_serializing_if = "Option::is_none")]
pub prebake: Option<PrebakeParams>,
#[serde(default, skip_serializing_if = "Option::is_none")]
pub bake: Option<BakeParams>,
#[serde(default, skip_serializing_if = "Option::is_none")]
pub finalize: Option<FinalizeParams>,
}
/// Apply and merge helpers.
impl Params {
/// Merge a layer from a specific writer into the current params.
/// Returns entries that were overridden (for debug).
pub fn apply(&mut self, incoming: Params, writer: Writer) -> Overridden {
let mut ov = Overridden::new();
if let Some(l) = incoming.notification {
if let Some(ref mut b) = self.notification {
b.apply(l, writer, &mut ov, "notification.");
} else {
self.notification = Some(l);
}
}
if let Some(l) = incoming.prebake {
if let Some(ref mut b) = self.prebake {
b.apply(l, writer, &mut ov, "prebake.");
} else {
self.prebake = Some(l);
}
}
if let Some(l) = incoming.bake {
if let Some(ref mut b) = self.bake {
b.apply(l, writer, &mut ov, "bake.");
} else {
self.bake = Some(l);
}
}
if let Some(l) = incoming.finalize {
if let Some(ref mut b) = self.finalize {
b.apply(l, writer, &mut ov, "finalize.");
} else {
self.finalize = Some(l);
}
}
ov
}
/// Convert to baker-consumable settings. All params guaranteed to have values.
pub fn to_settings(&self) -> Result<AllSettings, String> {
Ok(AllSettings {
notification: self.notification.as_ref().map(NotificationSettings::from).unwrap_or_default(),
prebake: self.prebake.as_ref().map(PrebakeSettings::from).unwrap_or_default(),
bake: self.bake.as_ref().map(BakeSettings::from).unwrap_or_default(),
finalize: self.finalize.as_ref().map(FinalizeSettings::from).unwrap_or_default(),
})
}
}
/// Plain-value settings for baker consumption. No metadata, no Option.
#[derive(Debug, Clone)]
pub struct AllSettings {
pub notification: NotificationSettings,
pub prebake: PrebakeSettings,
pub bake: BakeSettings,
pub finalize: FinalizeSettings,
}
/// Debug — values that were overridden during merge, keyed by dot-path.
#[derive(Debug, Clone, Default)]
pub struct Overridden {
pub entries: std::collections::HashMap<String, serde_json::Value>,
}
impl Overridden {
pub fn new() -> Self { Self { entries: HashMap::new() } }
pub fn is_empty(&self) -> bool { self.entries.is_empty() }
}
#[cfg(test)]
mod tests {
use super::*;
#[test]
fn test_empty_params_serializes_empty() {
let p = Params::default();
let json = serde_json::to_string(&p).unwrap();
assert_eq!(json, "{}");
}
#[test]
fn test_notification_params_serialize() {
let mut p = Params::default();
p.notification = Some(NotificationParams::defaults());
let json = serde_json::to_string_pretty(&p).unwrap();
assert!(json.contains("max_lives"));
assert!(json.contains("3"));
}
#[test]
fn test_apply_courier_overrides_default() {
let mut base = Params::default();
base.notification = Some(NotificationParams::defaults());
let incoming = Params {
notification: Some(NotificationParams {
max_lives: ParamVal::with(5, Writer::Courier),
..Default::default()
}),
..Default::default()
};
let ov = base.apply(incoming, Writer::Courier);
let s = base.to_settings().unwrap();
assert_eq!(s.notification.max_lives, 5);
assert!(ov.is_empty()); // default had no writer, not an "override"
}
#[test]
fn test_apply_upstream_wins() {
let mut base = Params::default();
base.notification = Some(NotificationParams {
max_lives: ParamVal::with(3, Writer::Base),
..Default::default()
});
let incoming = Params {
notification: Some(NotificationParams {
max_lives: ParamVal::with(5, Writer::Courier),
..Default::default()
}),
..Default::default()
};
let ov = base.apply(incoming, Writer::Courier);
let s = base.to_settings().unwrap();
// base(3) > courier(2) — rejected, base stays
assert_eq!(s.notification.max_lives, 3);
assert!(ov.is_empty()); // nothing overridden
}
#[test]
fn test_deny_unknown_fields() {
let bad = r#"{"max_lives":{"value":5},"ghost_param":{"value":1}}"#;
let result: Result<NotificationParams, _> = serde_json::from_str(bad);
assert!(result.is_err()); // unknown field should be denied
}
}
+65
View File
@@ -0,0 +1,65 @@
use crate::apply_field;
use serde::{Deserialize, Serialize};
use crate::{ParamVal, Writer, Overridden, traits::MergeParams};
#[derive(Debug, Clone, Serialize, Deserialize)]
#[serde(deny_unknown_fields)]
pub struct NotificationParams {
#[serde(default)] pub max_lives: ParamVal<u32>,
#[serde(default)] pub timeout_ms: ParamVal<u64>,
#[serde(default)] pub retry_backoff: ParamVal<f64>,
#[serde(default)] pub retry_delay_seconds: ParamVal<f64>,
#[serde(default)] pub max_retry_delay_seconds: ParamVal<f64>,
#[serde(default)] pub life_impact_factor: ParamVal<f64>,
#[serde(default)] pub batch_period: ParamVal<u32>,
#[serde(default)] pub batch_watermark: ParamVal<u32>,
#[serde(default)] pub template_ref_depth: ParamVal<u32>,
}
impl MergeParams for NotificationParams {
fn defaults() -> Self {
Self {
max_lives: ParamVal::new(3), timeout_ms: ParamVal::new(30000),
retry_backoff: ParamVal::new(2.0), retry_delay_seconds: ParamVal::new(5.0),
max_retry_delay_seconds: ParamVal::new(300.0),
life_impact_factor: ParamVal::new(0.0),
batch_period: ParamVal::new(5), batch_watermark: ParamVal::new(1),
template_ref_depth: ParamVal::new(10),
}
}
fn apply(&mut self, incoming: Self, writer: Writer, ov: &mut Overridden, prefix: &str) {
apply_field!(self.max_lives, incoming.max_lives, writer, ov, prefix, "max_lives");
apply_field!(self.timeout_ms, incoming.timeout_ms, writer, ov, prefix, "timeout_ms");
apply_field!(self.retry_backoff, incoming.retry_backoff, writer, ov, prefix, "retry_backoff");
apply_field!(self.retry_delay_seconds, incoming.retry_delay_seconds, writer, ov, prefix, "retry_delay_seconds");
apply_field!(self.max_retry_delay_seconds, incoming.max_retry_delay_seconds, writer, ov, prefix, "max_retry_delay_seconds");
apply_field!(self.life_impact_factor, incoming.life_impact_factor, writer, ov, prefix, "life_impact_factor");
apply_field!(self.batch_period, incoming.batch_period, writer, ov, prefix, "batch_period");
apply_field!(self.batch_watermark, incoming.batch_watermark, writer, ov, prefix, "batch_watermark");
apply_field!(self.template_ref_depth, incoming.template_ref_depth, writer, ov, prefix, "template_ref_depth");
}
}
impl Default for NotificationParams { fn default() -> Self { Self::defaults() } }
#[derive(Debug, Clone)]
pub struct NotificationSettings {
pub max_lives: u32, pub timeout_ms: u64, pub retry_backoff: f64,
pub retry_delay_seconds: f64, pub max_retry_delay_seconds: f64,
pub life_impact_factor: f64,
pub batch_period: u32, pub batch_watermark: u32, pub template_ref_depth: u32,
}
impl Default for NotificationSettings { fn default() -> Self { NotificationParams::defaults().into() } }
impl From<&NotificationParams> for NotificationSettings {
fn from(p: &NotificationParams) -> Self { Self {
max_lives: p.max_lives.value, timeout_ms: p.timeout_ms.value,
retry_backoff: p.retry_backoff.value, retry_delay_seconds: p.retry_delay_seconds.value,
max_retry_delay_seconds: p.max_retry_delay_seconds.value,
life_impact_factor: p.life_impact_factor.value,
batch_period: p.batch_period.value, batch_watermark: p.batch_watermark.value,
template_ref_depth: p.template_ref_depth.value,
}}
}
impl From<NotificationParams> for NotificationSettings { fn from(p: NotificationParams) -> Self { (&p).into() } }
+32
View File
@@ -0,0 +1,32 @@
use serde::{Deserialize, Serialize};
use crate::Writer;
/// A strongly-typed parameter value with writer tracking.
///
/// Serializes as `{ "value": 3, "writer": "base" }`.
#[derive(Debug, Clone, Serialize, Deserialize)]
pub struct ParamVal<T> {
pub value: T,
#[serde(skip_serializing_if = "Option::is_none")]
pub writer: Option<Writer>,
}
impl<T: Default> Default for ParamVal<T> {
fn default() -> Self {
ParamVal { value: T::default(), writer: None }
}
}
impl<T> ParamVal<T> {
pub fn new(value: T) -> Self {
ParamVal { value, writer: None }
}
pub fn with(value: T, writer: Writer) -> Self {
ParamVal { value, writer: Some(writer) }
}
/// Whether an incoming value from `incoming_writer` should override this one.
pub fn should_override(&self, incoming: Writer) -> bool {
let current = self.writer.map_or(0, |w| w.priority());
incoming.priority() > current
}
}
+54
View File
@@ -0,0 +1,54 @@
pub mod network;
pub mod cache;
pub mod engine;
use crate::apply_field;
use crate::{ParamVal, Writer, Overridden, traits::MergeParams};
use serde::{Deserialize, Serialize};
#[derive(Debug, Clone, Serialize, Deserialize)]
#[serde(deny_unknown_fields)]
pub struct PrebakeParams {
#[serde(default)] pub bootstrap_user: ParamVal<String>,
#[serde(default)] pub workspace: ParamVal<String>,
#[serde(default)] pub network: network::NetworkParams,
#[serde(default)] pub cache: cache::CacheParams,
#[serde(default)] pub engine: engine::EngineParams,
#[serde(default)] pub repology_endpoint: ParamVal<String>,
}
impl MergeParams for PrebakeParams {
fn defaults() -> Self {
Self {
bootstrap_user: ParamVal::new("vulcan".into()), workspace: ParamVal::new("/home/vulcan/workspace".into()),
network: network::NetworkParams::defaults(), cache: cache::CacheParams::defaults(),
engine: engine::EngineParams::defaults(),
repology_endpoint: ParamVal::new("default".into()),
}
}
fn apply(&mut self, i: Self, w: Writer, ov: &mut Overridden, p: &str) {
apply_field!(self.bootstrap_user, i.bootstrap_user, w, ov, p, "bootstrap_user");
apply_field!(self.workspace, i.workspace, w, ov, p, "workspace");
apply_field!(self.repology_endpoint, i.repology_endpoint, w, ov, p, "repology_endpoint");
self.network.apply(i.network, w, ov, &format!("{}network.", p));
self.cache.apply(i.cache, w, ov, &format!("{}cache.", p));
self.engine.apply(i.engine, w, ov, &format!("{}engine.", p));
}
}
impl Default for PrebakeParams { fn default() -> Self { Self::defaults() } }
#[derive(Debug, Clone)]
pub struct PrebakeSettings {
pub bootstrap_user: String, pub workspace: String,
pub network: network::NetworkSettings, pub cache: cache::CacheSettings,
pub engine: engine::EngineSettings, pub repology_endpoint: String,
}
impl Default for PrebakeSettings { fn default() -> Self { PrebakeParams::defaults().into() } }
impl From<&PrebakeParams> for PrebakeSettings {
fn from(p: &PrebakeParams) -> Self { Self {
bootstrap_user: p.bootstrap_user.value.clone(), workspace: p.workspace.value.clone(),
network: (&p.network).into(), cache: (&p.cache).into(), engine: (&p.engine).into(),
repology_endpoint: p.repology_endpoint.value.clone(),
}}
}
impl From<PrebakeParams> for PrebakeSettings { fn from(p: PrebakeParams) -> Self { (&p).into() } }
@@ -0,0 +1,37 @@
use crate::apply_field;
use crate::{ParamVal, Writer, Overridden, traits::MergeParams};
use serde::{Deserialize, Serialize};
#[derive(Debug, Clone, Serialize, Deserialize)]
pub struct CacheParams {
#[serde(default)] pub strategy: ParamVal<String>,
#[serde(default)] pub mode: ParamVal<String>,
#[serde(default)] pub ttl_days: ParamVal<u32>,
#[serde(default)] pub max_size_gb: ParamVal<u32>,
#[serde(default)] pub cleanup_policy: ParamVal<String>,
}
impl MergeParams for CacheParams {
fn defaults() -> Self {
Self {
strategy: ParamVal::new("always".into()), mode: ParamVal::new("zstd".into()),
ttl_days: ParamVal::new(30), max_size_gb: ParamVal::new(20),
cleanup_policy: ParamVal::new("lru".into()),
}
}
fn apply(&mut self, i: Self, w: Writer, ov: &mut Overridden, p: &str) {
apply_field!(self.strategy, i.strategy, w, ov, p, "strategy");
apply_field!(self.mode, i.mode, w, ov, p, "mode");
apply_field!(self.ttl_days, i.ttl_days, w, ov, p, "ttl_days");
apply_field!(self.max_size_gb, i.max_size_gb, w, ov, p, "max_size_gb");
apply_field!(self.cleanup_policy, i.cleanup_policy, w, ov, p, "cleanup_policy");
}
}
impl Default for CacheParams { fn default() -> Self { Self::defaults() } }
#[derive(Debug, Clone)]
pub struct CacheSettings { pub strategy: String, pub mode: String, pub ttl_days: u32, pub max_size_gb: u32, pub cleanup_policy: String }
impl Default for CacheSettings { fn default() -> Self { CacheParams::defaults().into() } }
impl From<&CacheParams> for CacheSettings { fn from(p: &CacheParams) -> Self {
Self { strategy: p.strategy.value.clone(), mode: p.mode.value.clone(), ttl_days: p.ttl_days.value, max_size_gb: p.max_size_gb.value, cleanup_policy: p.cleanup_policy.value.clone() }
} }
impl From<CacheParams> for CacheSettings { fn from(p: CacheParams) -> Self { (&p).into() } }
@@ -0,0 +1,21 @@
use crate::apply_field;
use crate::{ParamVal, Writer, Overridden, traits::MergeParams};
use serde::{Deserialize, Serialize};
#[derive(Debug, Clone, Serialize, Deserialize)]
pub struct EngineParams {
#[serde(default)] pub timeout_ms: ParamVal<u64>,
}
impl MergeParams for EngineParams {
fn defaults() -> Self { Self { timeout_ms: ParamVal::new(300000) } }
fn apply(&mut self, i: Self, w: Writer, ov: &mut Overridden, p: &str) {
apply_field!(self.timeout_ms, i.timeout_ms, w, ov, p, "timeout_ms");
}
}
impl Default for EngineParams { fn default() -> Self { Self::defaults() } }
#[derive(Debug, Clone)]
pub struct EngineSettings { pub timeout_ms: u64 }
impl Default for EngineSettings { fn default() -> Self { EngineParams::defaults().into() } }
impl From<&EngineParams> for EngineSettings { fn from(p: &EngineParams) -> Self { Self { timeout_ms: p.timeout_ms.value } } }
impl From<EngineParams> for EngineSettings { fn from(p: EngineParams) -> Self { (&p).into() } }
@@ -0,0 +1,23 @@
use crate::apply_field;
use crate::{ParamVal, Writer, Overridden, traits::MergeParams};
use serde::{Deserialize, Serialize};
#[derive(Debug, Clone, Serialize, Deserialize)]
pub struct NetworkParams {
#[serde(default)] pub enabled: ParamVal<bool>,
#[serde(default)] pub outbound: ParamVal<bool>,
}
impl MergeParams for NetworkParams {
fn defaults() -> Self { Self { enabled: ParamVal::new(true), outbound: ParamVal::new(true) } }
fn apply(&mut self, i: Self, w: Writer, ov: &mut Overridden, p: &str) {
apply_field!(self.enabled, i.enabled, w, ov, p, "enabled");
apply_field!(self.outbound, i.outbound, w, ov, p, "outbound");
}
}
impl Default for NetworkParams { fn default() -> Self { Self::defaults() } }
#[derive(Debug, Clone)]
pub struct NetworkSettings { pub enabled: bool, pub outbound: bool }
impl Default for NetworkSettings { fn default() -> Self { NetworkParams::defaults().into() } }
impl From<&NetworkParams> for NetworkSettings { fn from(p: &NetworkParams) -> Self { Self { enabled: p.enabled.value, outbound: p.outbound.value } } }
impl From<NetworkParams> for NetworkSettings { fn from(p: NetworkParams) -> Self { (&p).into() } }
+28
View File
@@ -0,0 +1,28 @@
use crate::{Writer, Overridden};
/// A parameter tree node that can merge itself with an incoming layer.
pub trait MergeParams: Sized {
fn apply(&mut self, incoming: Self, writer: Writer, ov: &mut Overridden, prefix: &str);
fn defaults() -> Self;
}
/// Apply one field's merge logic.
/// ```ignore
/// apply_field!(self.foo, incoming.foo, writer, ov, prefix, "foo");
/// ```
#[macro_export]
macro_rules! apply_field {
($self:ident.$field:ident, $incoming:ident.$field2:ident, $writer:expr, $ov:ident, $prefix:expr, $name:expr) => {
let old_w = $self.$field.writer;
if $self.$field.should_override($writer) {
let old_v = ::std::mem::replace(&mut $self.$field.value, $incoming.$field2.value);
$self.$field.writer = ::std::option::Option::Some($writer);
if let ::std::option::Option::Some(w) = old_w {
$ov.entries.insert(
format!("{}{}", $prefix, $name),
::serde_json::json!({"value": old_v, "writer": w.as_str()}),
);
}
}
};
}
+26
View File
@@ -0,0 +1,26 @@
use serde::{Deserialize, Serialize};
#[derive(Debug, Clone, Copy, PartialEq, Eq, Serialize, Deserialize)]
#[serde(rename_all = "lowercase")]
pub enum Writer {
Base,
Courier,
Bakerd,
}
impl Writer {
/// Priority: Base=3 > Courier=2 > Bakerd=1
pub fn priority(self) -> u8 {
match self {
Writer::Base => 3,
Writer::Courier => 2,
Writer::Bakerd => 1,
}
}
pub fn as_str(self) -> &'static str {
match self {
Writer::Base => "base", Writer::Courier => "courier", Writer::Bakerd => "bakerd",
}
}
}
impl Default for Writer { fn default() -> Self { Writer::Base } }
+4
View File
@@ -0,0 +1,4 @@
temp
examples
quicktest.sh
_env.sh
+80
View File
@@ -0,0 +1,80 @@
# WORKSHOP-BAKER CRATE KNOWLEDGE BASE
## OVERVIEW
Primary crate — the only crate with a `[[bin]]` target. Contains the `workshop-baker` binary and `workshop_baker` library. Orchestrates the full CI/CD pipeline: prebake (env setup) → bake (script execution) → finalize (plugins, cleanup) → drain notification queue. ~69 .rs source files.
## STRUCTURE
```
src/
├── main.rs # Entry: default(full pipeline) | bare(dev-only) | daemon(unimplemented)
├── lib.rs # Clap Cli/Commands/BareCommands — CLI structs in lib, not bin
├── error.rs # CliError enum (wraps stage errors, delegates exit_code)
├── bare.rs # Dev-only single-stage execution, ctx.privileged always false
├── utils.rs # Cross-cutting helpers
├── bake/ # Script parsing + DAG scheduling + execution
│ ├── decorator.rs # @decorator(args) parsing, 81 tests — heaviest coverage in crate
│ ├── parser.rs # Step/function extraction
│ ├── schedule.rs # DAG build → topological sort
│ ├── execute.rs # Per-step timeout/retry
│ ├── builder.rs # Script assembly
│ ├── trivial.rs # Trivial step detection (skip in dry-run)
│ ├── constant.rs / error.rs / util.rs
├── prebake/ # Environment provisioning
│ ├── config.rs # PrebakeConfig deserialization
│ ├── stage.rs + stage/{bootstrap,depssystem,depsuser,environment,hook}.rs
│ ├── env.rs + env/{container,firecracker,baremetal,custom}.rs # Builder dispatch
│ ├── security.rs # Privilege/sandbox checks
│ ├── event.rs # Event sender/receiver channel
│ ├── types.rs + types/{memsize,cache,builderconfig,architecture}.rs
│ ├── constant.rs / error.rs
├── finalize/ # Post-build hooks + plugins
│ ├── config.rs # FinalizeConfig
│ ├── stage.rs + stage/hook.rs
│ ├── template.rs # Output templating (minijinja)
│ ├── plugin.rs + plugin/{shell,dylib,rhai,internal}.rs
│ │ └── plugin/internal/{dummy,mail,satori,webhook,insitenotify}.rs
│ ├── types.rs + types/compression.rs
│ ├── constant.rs / error.rs / event.rs
├── notify/ # Async notification dispatch
│ ├── queue.rs # NotificationQueue — drain on pipeline complete
│ ├── handler.rs / rule.rs / template.rs / types.rs / util.rs / error.rs
└── types/ # Shared types — ZERO deps on stage modules
├── resource.rs # ResourceRegistry (populated pre-pipeline, consumed read-only)
└── buildstatus.rs
```
## WHERE TO LOOK
| Task | Location | Notes |
|------|----------|-------|
| CLI args + subcommands | `src/lib.rs` | Cli, Commands(Bare/Daemon), BareCommands |
| Pipeline orchestration | `src/main.rs` | worker_main(): prebake→bake→finalize→queue.drain() |
| Decorator syntax | `src/bake/decorator.rs` | `# @decorator` or `# @decorator(args)`, line-by-line regex |
| Build execution | `src/bake/execute.rs` | timeout, retry, exit code capture |
| DAG scheduling | `src/bake/schedule.rs` | Uses workshop-schedule crate |
| Docker builder | `src/prebake/env/container.rs` | Bollard-based, buildkit support |
| Firecracker / Bare metal | `.../firecracker.rs`, `.../baremetal.rs` | MicroVM;
needs PIPELINE_BAREMETAL_ELEVATE |
| Plugin dispatch | `src/finalize/plugin.rs` | Dispatches shell / dylib(stub) / rhai(stub) / internal |
| Notification queue | `src/notify/queue.rs` | Spawned early, drained after pipeline, retry support |
| Error type | `src/error.rs` | CliError wraps PrebakeError/BakeError/FinalizeError/anyhow |
| Dev bare mode | `src/bare.rs` | ctx.privileged=false always; do not extrapolate to production |
## DEPENDENCIES
**Internal:** workshop-engine, workshop-schedule, workshop-getterurl, workshop-baker-params. **External:** bollard (Docker), clap (CLI), axum 0.7 (HTTP), lettre 0.11 (email), minijinja (templates), cgroups-rs (limits), privdrop.
## CONVENTIONS (CRATE-SPECIFIC)
- **Lint overrides** (this crate only): `dead_code=allow`, `unreachable_code=allow`, `inherent_to_string=allow`, `non_canonical_partial_ord_impl=allow`
- **CLI structs in lib.rs, not main.rs** — enables integration tests to import and construct Cli directly.
- **Pipeline stages return `anyhow::Result<()>`** — stages propagate with `?`; only main.rs converts to CliError.
- **ResourceRegistry pattern** — populated before pipeline stages, consumed read-only by stages. New resource types go in `types/resource.rs`.
- **Notification queue fire-and-forget + drain** — notify tasks spawned early, send during pipeline, drain on completion. `NotificationQueue::drain()` must be called.
- **Trivial step detection** — steps with no external deps can be marked trivial (`trivial.rs`), skipped in dry-run.
## ANTI-PATTERNS (THIS CRATE)
- **Do NOT derive production behavior from bare mode** — ctx.privileged is hardcoded false.
- **Do NOT add CLI args bypassing prebake→bake→finalize ordering** — the pipeline is linear. Parallelism lives in the DAG scheduler within bake.
- **Do NOT import types/ from bake/, prebake/, or finalize/** — `types/` must have zero crate-internal deps.
- **Do NOT add lint allows without justification** — the four existing overrides are intentional.
- **Do NOT spawn blocking work on tokio runtime** — use `spawn_blocking` for CPU-heavy tasks.
- **Decorator regex is line-by-line** — multi-line decorators need a parser rewrite, not a regex hack.
- **Careful with `unimplemented!()`** — Daemon, Rhai/Dylib plugins, Custom builder are stubs. Prefer error over crash.
File diff suppressed because it is too large Load Diff
+71
View File
@@ -0,0 +1,71 @@
[package]
name = "workshop-baker"
version = "0.1.0"
edition = "2024"
[lib]
name = "workshop_baker"
path = "src/lib.rs"
[[bin]]
name = "workshop-baker"
path = "src/main.rs"
[features]
default = []
integration-tests = []
[dependencies]
workshop-engine = { path = "../workshop-engine" }
anyhow = "1.0.100"
bollard = { version = "0.20.0", features = ["buildkit", "chrono", "ssh"] }
cgroups-rs = "0.5.0"
chrono = { version = "0.4.42", features = ["serde"] }
clap = { version = "4.5.53", features = ["derive", "env"] }
config = "0.15.19"
duration-str = "0.21.0"
env_logger = "0.11.8"
glob = "0.3.2"
futures-util = "0.3.31"
lazy_static = "1.5.0"
lettre = { version = "0.11", default-features = false, features = ["tokio1-rustls", "rustls-tls", "builder", "smtp-transport"] }
log = "0.4.28"
minijinja = "2.14.0"
regex = "1.12.2"
semver = "1.0.27"
serde = { version = "1.0.228", features = ["derive"] }
serde_json = "1.0.148"
serde_yaml = "0.9.34"
shlex = "1.3.0"
tar = "0.4.44"
tempfile = "3.24.0"
thiserror = "2.0.17"
tokio = { version = "1.48.0", features = ["full"] }
workshop-schedule = { path = "../workshop-schedule" }
async-trait = "0.1.87"
uuid = { version = "1.19.0", features = ["v4"] }
libc = "0.2"
os_info = "3.14.0"
privdrop = "0.5.6"
which = "8.0.2"
globset = "0.4.18"
axum = "0.7"
workshop-getterurl = { path = "../workshop-getterurl" }
workshop-baker-params = { version = "0.1.0", path = "../workshop-baker-params" }
bitflags = "2"
url = "2.5.8"
zstd = "0.13.3"
flate2 = "1.1.9"
capctl = "0.2.4"
[dev-dependencies]
criterion = "0.5"
[lints.rust]
dead_code = "allow"
unreachable_code = "allow"
[lints.clippy]
inherent_to_string = "allow"
non_canonical_partial_ord_impl = "allow"
+5
View File
@@ -0,0 +1,5 @@
FROM alpine:latest
RUN echo "Hello from Docker!" && \
aps add --no-cache curl && \
curl --version
CMD ["sh", "-c", "echo 'Container started' && sleep 3600"]
+240
View File
@@ -0,0 +1,240 @@
# prebake.yaml.tmpl
#
# 术语说明:
# 可选:该字段可以不给出
# 默认为...:隐含“可选”
# 留空:尚未定义,留作后续
# 版本号,用于格式兼容性检查
version: "1.0"
# 构建环境定义
environment:
# 构建机类型:docker | firecracker | custom | baremetal
builder: "docker"
# 构建机配置,与builder一致的被启用。
baremetal: {}
docker:
# 镜像名,与dockerfile互斥
image: "rust:1.70-slim"
# Dockerfile路径,相对于项目根目录,与image互斥
dockerfile: "/Dockerfile"
# Docker构建参数,可选
build_args:
RUST_VERSION: "1.70"
firecracker: {}
# TODO: 定义firecracker结构
custom:
# 自定义构建机名称
name: "my-custom-builder"
# 设置脚本,相对于项目根目录
setup_script: "setup-builder.sh"
# 清洁脚本,相对于项目根目录
cleanup_script: "cleanup-builder.sh"
# 硬件资源要求,满足最小要求的构建机可以被选中
resources:
cpu: 2 # CPU核心数,默认为1
memory: "2G" # 内存大小,也可以写作2048MB,默认为"1G"
disk: "10G" # 磁盘空间,也可以写作10240MB,默认为"1G"
architecture: "x86_64" # 架构要求,只能是"noarch"或{"x86_64"/"amd64", "arm64"/"aarch64", "riscv64", "loongarch64"/"loong64"}中的一个或几个
# TODO: 重新设计GPU字段
# gpu: false # 是否需要GPU,默认为false
# gpu_type: "cuda" # GPU类型(如果需要),可以是"cuda", "rocm", "vulkan", "oneapi"中的一个或几个
tags: # 自定义标签,只有满足标签的构建机可以被选中
- "custom_tag"
# 网络配置
network:
enabled: true # 启用网络,对baremetal不适用,默认为true
outbound: true # 允许出站连接,对baremetal不适用,默认为true
dns_servers: # 指定构建机的DNS服务器,对baremetal不适用,custom应该自行处理DNS问题,可选
- "8.8.8.8"
- "1.1.1.1"
proxies: # 代理配置,暂时留空,可选
# 构建环境初始化
bootstrap:
# 构建用户,默认为 vulcan(火神)
# 设定为 "root" 表示忽略用户切换并禁用安全特性,需要 PIPELINE_PRIVILEGED 权限
user: "vulcan"
# 工作空间
workspace:
# 工作目录路径
path: "/home/vulcan/workspace"
# 路径不可用时回退到 /workspace
# 即使路径可用,也会在 /workspace 建立软链接
fallback: true
# 自定义 sudoers 文件内容
# 这会覆盖默认行为,可能绕过安全机制
# 需要 PIPELINE_CUSTOM_SUDOERS 权限
sudoers: "vulcan ALL=(ALL:ALL) NOPASSWD: ALL"
# 自定义 doas 文件内容
# 这会覆盖默认行为,可能绕过安全机制
# 与 sudoers 共享 PIPELINE_CUSTOM_SUDOERS 权限
doas: "permit nopass vulcan as root"
# 自定义 bootstrap 脚本(与上述所有字段互斥)
# 支持多种来源:
# workspace:///.workshop/bootstrap.sh - 项目内文件
# file:///bin/bootstrap.sh - 环境内文件
# server://bootstrap.sh - 由服务器提供
# https://example.com/bootstrap.sh - http/https网络下载,请确保构建环境可以连接到该URL
# (content) - 直接给出内容,当文本开头无法匹配任何scheme时采用
# 需要 PIPELINE_CUSTOM_BOOTSTRAP 权限
custom: "server://bootstrap-alpine.sh"
# 依赖定义,安装依赖时按system-user顺序进行
dependencies:
# Repology 端点配置,用于将 UPM 包名解析为系统包名
# 可选值:disabled, none, local, remote, server, default
# - disabled: 完全禁用 Repology
# - none: 不解析,直接使用原始包名
# - local: 使用本地 Repology 服务器
# - remote: 使用远程 Repology API
# - server: 由服务器提供 Repology 服务
# - default: 默认行为(通常等同于 local)
config:
repology_endpoint: "default"
# 系统包依赖,按包管理器枚举给出,只应该给出与环境契合的字段。
# 根据security.drop-after的配置,在此阶段一般具有root或免密root权限
# 不适用于裸机
system:
apt:
packages: # 安装的软件包
- "git"
- "curl"
- "build-essential"
- "pkg-config"
- "libssl-dev"
repositories: # 该字段按serde Value原样传递给插件,config.rs不做解析
- url: "ppa:custom/ppa" # 自定义PPAUbuntu
- url: "https://download.docker.com/linux/ubuntu" # DEB822
key: "https://download.docker.com/linux/ubuntu/gpg"
mirror: "https://mirrors.tuna.tsinghua.edu.cn" # 指定软件包镜像源,可选
pacman:
packages:
- "git"
- "curl"
- "base-devel"
- "pkg-config"
- "openssl"
repositories:
- server: "https://mirrors.tuna.tsinghua.edu.cn/arch4edu/$arch" # 自定义仓库(Arch
name: "arch4edu"
keypackage: "arch4edu-keyring"
mirror: "https://mirrors.tuna.tsinghua.edu.cn" # 指定软件包镜像源,可选
#dnf:
# 略
# 用户依赖
# 根据security.drop-after的配置,在此阶段一般不具有root权限
# 例外:语言依赖会在system中产生隐式依赖
user:
# 以下字段按serde Value原样传递给插件,config.rs不做解析
# rust:
# type: "rustup" # 此时会向系统引入隐式的rustup依赖
# toolchain: "nightly-x86_64-unknown-linux-gnu"
# manifest: "Cargo.toml"
# lock: "Cargo.lock"
# python:
# type: "uv"
# python: "python3.14"
# index: "http://mirrors.tuna.tsinghua.edu.cn/pypi/web/simple"
# directory: ".venv"
# env: # 优先,高级选项
# UV_PYTHON: "python3.14"
# UV_INDEX: "http://mirrors.tuna.tsinghua.edu.cn/pypi/web/simple"
# UV_DIRECTORY: ".venv"
# manifest: "requirements.txt" # 或pyproject.toml
# nix: {}
# # TBD: 没用过lol
# # PR welcome
custom: # 绝对的自定义,与上述全部配置互斥(custom独占)
- command: "cargo fetch"
working_dir: "/tmp"
# 环境变量配置
envvars:
# 构建环境变量,只在 prebake 阶段有效
# 用于控制 prebake 阶段的行为(apt/cargo/rustc 等)
prebake:
RUST_BACKTRACE: "full" # Rust 调试
CARGO_NET_GIT_FETCH_WITH_CLI: "true" # Cargo 配置
NODE_ENV: "production" # Node 环境
# 运行时环境变量(会传递给 bake 阶段)
# 这些变量会与项目中的 .env 文件合并后,作为最终环境
bake:
# 支持 secret 引用,格式: {{ secret.<key> }}
DATABASE_URL: "postgresql://user:{{ secret.dbpassword }}@localhost/db"
LOG_LEVEL: "info"
# 特殊变量:指定项目中 .env 文件的位置
# 系统会读取该文件,并将其中的变量与上面定义的变量合并
# 合并规则:prebake.yml 中定义的变量优先级更高,会覆盖 .env 中的同名变量
# 此变量本身不会传递给 bake 阶段
# .env不应该包含任何 secret
__ENV_FILE: ".env"
# 注意:
# - secret 只存在于内存中,不会写入磁盘
# - 日志中会自动遮蔽 secret 的值
# - 如需临时查看 secret,可使用 PIPELINE_UNCOVER_SECRET 权限(需 2FA
# 缓存配置
cache:
# 缓存目录
directory:
- path: "/usr/local/cargo"
strategy: "always" # 在"always","readonly","clean"和"never"中选择一个,可以被流水线启动时配置覆盖,默认为"always"
mode: "zstd" # 在"gzip","zstd","none"和"dir"中选择一个
- path: "/root/.npm"
strategy: "clean"
mode: "none"
- path: "/app/target"
strategy: "always"
mode: "dir"
# 缓存策略
strategy:
ttl_days: 30 # 缓存生存时间
max_size_gb: 20 # 最大缓存大小
cleanup_policy: "lru" # 清理策略:lru, fifo, size
# 安全配置
security:
#
drop-after: ""
# 留空
# 钩子脚本
hooks:
# 依赖安装前
early:
- command: "echo 'Starting dependency installation'"
name: ""
# 依赖安装后
late:
- command: "cargo fetch"
working_dir: "/tmp"
# 元数据
metadata:
description: "Rust project build environment"
maintainer: "team@example.com"
+242
View File
@@ -0,0 +1,242 @@
use crate::types::debug::DebugFeature;
use crate::bake::builder::RenderMode;
use crate::bake::constant::DEFAULT_WORKSPACE;
use crate::bake::decorator::Decorator;
use crate::bake::error::BakeError;
use crate::bake::util::{resolve_mode, FuncMode};
use crate::cli::Cli;
use crate::prebake;
use crate::prebake::PrebakeConfig;
use std::collections::{HashMap, HashSet};
use std::path::{Path, PathBuf};
use std::time::Duration;
use workshop_baker_params::BakeSettings;
use workshop_engine::{Engine, EventSender, ExecutionContext};
mod builder;
pub mod constant;
mod decorator;
pub mod error;
mod execute;
pub mod parser;
mod schedule;
pub mod trivial;
pub mod util;
pub async fn bake(
script_path: &Path,
bake_base_path: &Path,
prebake_path: &Path,
_cli: &Cli,
ctx: &mut ExecutionContext,
event_tx: EventSender,
nevent_tx: Option<crate::notify::types::NotificationEventSender>,
) -> Result<(), BakeError> {
log::debug!("Reading script: {:?}", script_path);
let script_content = std::fs::read_to_string(script_path).map_err(|e| {
BakeError::IoError { path: script_path.display().to_string(), source: e }
})?;
log::debug!("Reading prebake.yml: {:?}", prebake_path);
let prebake_content = std::fs::read_to_string(prebake_path).map_err(|e| {
BakeError::IoError { path: prebake_path.display().to_string(), source: e }
})?;
let prebake = prebake::parse(prebake_content.as_str()).map_err(|e| {
log::error!("Failed to parse prebake.yml: {}", e);
BakeError::YamlParseError(e)
})?;
let workspace = get_workspace(&prebake);
let functions = parser::parse_script(&script_content)?;
let has_pipeline = functions.pipeline_functions.iter().any(|f| {
f.decorators
.iter()
.any(|d| matches!(d, decorator::Decorator::Pipeline))
});
// TODO: if PIP(pipeline implicit parameter).trivial is set, always use RenderMode::Off
let render_mode = RenderMode::Full;
let engine = Engine::new();
// TODO: resolve from merged param layers (PIP integration)
let bake_settings = BakeSettings::default();
if let Some(env) = &prebake.envvars
&& let Some(prebake_env) = &env.bake
{
ctx.env_vars.extend(prebake_env.clone());
}
if !has_pipeline {
log::trace!("Running trivial script");
trivial::run_trivial(
&script_content,
bake_base_path,
&workspace,
&engine,
ctx,
&event_tx,
render_mode,
)
.await?;
} else {
log::trace!("Running pipeline script");
let remaining = functions.remaining_code;
let (mut dag, func_map) = schedule::build_dag(functions.pipeline_functions)?;
// Pre-scan: pipe sources + decorator conflict checks.
let mut pipe_sources: HashSet<String> = HashSet::new();
for func in func_map.values() {
decorator::check_conflicts(func)?;
for decorator in &func.decorators {
if let Decorator::Pipe(names) = decorator {
for source in names {
if *source != func.name {
pipe_sources.insert(source.clone());
}
}
}
}
}
let mut stdout_cache: HashMap<String, String> = HashMap::new();
let mut async_handles: Vec<tokio::task::JoinHandle<()>> = Vec::new();
while !dag.is_empty() {
let scheduler_debug = DebugFeature::from_bits_retain(ctx.debug_flags).contains(DebugFeature::Scheduler);
if scheduler_debug {
log::info!("[BAKE_SCHED] ready={} seq={} wild={} groups={:?}",
dag.ready_count(),
dag.ready_sequential().len(),
dag.ready_wildcards().len(),
dag.ready_groups(),
);
} else {
log::debug!("[BAKE_SCHED] ready={} seq={} wild={} groups={:?}",
dag.ready_count(),
dag.ready_sequential().len(),
dag.ready_wildcards().len(),
dag.ready_groups(),
);
}
// Rule 1: sequential always goes first
let seq_candidates = dag.ready_sequential();
if let Some(seq_name) = seq_candidates.first() {
let seq_name = seq_name.to_string();
log::info!("[BAKE_SCHED] seq {}", seq_name);
let func = &func_map[&seq_name];
let mode = resolve_mode(func);
match mode {
FuncMode::Daemon => {
execute::execute_daemon(
&engine, func, ctx, &event_tx,
&workspace, &remaining, bake_base_path,
Duration::from_secs(bake_settings.health_period),
Duration::from_secs(bake_settings.health_timeout),
bake_settings.health_max_retries,
).await?;
}
FuncMode::Async => {
let handle = execute::execute_async(
func, ctx, &event_tx,
&workspace, &remaining, bake_base_path,
);
async_handles.push(handle);
}
FuncMode::Normal => {
let result = execute::execute_normal(
&engine, func, ctx, &event_tx,
&workspace, &remaining, bake_base_path, &stdout_cache,
).await;
match result {
Ok(stdout) => {
if pipe_sources.contains(&seq_name) {
stdout_cache.insert(seq_name.clone(), stdout);
}
}
Err(e) if dag.is_fallible(&seq_name) => {
log::warn!("'{}' failed (fallible): {}", seq_name, e);
}
Err(e) => return Err(e),
}
}
}
// Pop AFTER execution (Daemon/Async pop at start)
if !matches!(mode, FuncMode::Daemon | FuncMode::Async) {
dag.pop(&seq_name).map_err(|e| BakeError::IncompatibleDecorators {
function: seq_name.clone(),
reason: e.to_string(),
})?;
} else {
dag.pop(&seq_name).map_err(|e| BakeError::IncompatibleDecorators {
function: seq_name.clone(),
reason: e.to_string(),
})?;
}
continue;
}
// All remaining are parallel — pick a batch
if let Some(batch) = dag.pick_batch() {
log::info!("[BAKE_SCHED] par {:?}", batch);
let results = execute::fire_parallel_batch(
&batch, &engine, &func_map, ctx, &event_tx,
&workspace, &remaining, bake_base_path, &stdout_cache,
).await?;
for (name, stdout) in results {
if pipe_sources.contains(&name) {
stdout_cache.insert(name.clone(), stdout);
}
if let Err(e) = dag.pop(&name) {
log::warn!("Failed to pop '{}' from DAG: {}", name, e);
}
}
} else if dag.is_stalled() {
// Deadlock: named parallel groups exist but can't complete
let stalled = dag.stalled_groups();
let remaining = dag.remaining();
return Err(BakeError::IncompatibleDecorators {
function: remaining.join(", "),
reason: format!(
"parallel group(s) {:?} can never complete — blocked by @after chain",
stalled
),
});
} else {
// Ready set is empty — cycle
let remaining = dag.remaining();
if !remaining.is_empty() {
return Err(BakeError::CircularDependency(
remaining.into_iter().map(|s| s.to_string()).collect(),
));
}
break;
}
}
// Wait for all @async tasks to finish
for handle in async_handles {
let _ = handle.await;
}
}
crate::notify::types::send_stage_event(&nevent_tx, crate::types::buildstatus::StagePhase::Bake, "bake", crate::types::buildstatus::StageOutcome::Success).await;
Ok(())
}
fn get_workspace(prebake_config: &PrebakeConfig) -> PathBuf {
prebake_config
.bootstrap
.as_ref()
.and_then(|b| b.workspace.as_ref())
.map(|ws| {
if ws.fallback {
PathBuf::from(DEFAULT_WORKSPACE)
} else {
PathBuf::from(ws.path.clone())
}
})
.unwrap_or_else(|| PathBuf::from(DEFAULT_WORKSPACE))
}
// ── AI Coding Agent marker ──
// Code in this module PARTIALLY or FULLY utilized AI Coding Agent
+187
View File
@@ -0,0 +1,187 @@
use crate::types::debug::DebugFeature;
use crate::bake::constant::PIPELINE_SKIP_ERRORCODE;
use crate::bake::decorator::Decorator;
use crate::bake::error::BakeError;
use crate::bake::parser::Function;
use minijinja::{Environment, context};
use std::fs::Permissions;
use std::os::unix::fs::PermissionsExt;
use std::path::Path;
use std::path::PathBuf;
use tempfile::NamedTempFile;
/// Three-level rendering control.
#[derive(Debug, Clone, Copy, PartialEq, Eq)]
pub enum RenderMode {
/// No template, hardcoded "{{ main }}". Debug escape hatch.
Off,
/// Uses bake_base.sh with essential execution skeleton only.
Minimal,
/// Uses bake_base.sh with helpers (logs, colors, etc.).
Full,
}
impl RenderMode {
pub fn as_str(&self) -> &'static str {
match self {
RenderMode::Off => "off",
RenderMode::Minimal => "minimal",
RenderMode::Full => "full",
}
}
}
/// Hardcoded template for RenderMode::Off.
const TRIVIAL_TEMPLATE: &str = "{{ main }}";
/// Renders a `Function` into an executable shell script using a minijinja template.
///
/// `render_mode` controls the template source:
/// - `Off`: hardcoded `{{ main }}`, no bake_base file needed.
/// - `Minimal` / `Full`: reads bake_base.sh; the template distinguishes the two
/// via the `render_mode` context variable.
pub fn build_script(
function: &Function,
bake_base: &Path,
target_dir: &Path,
render_mode: RenderMode,
debug_flags: u32,
) -> Result<PathBuf, BakeError> {
let name = &function.name;
let body = &function.body;
let condition = function.find_decorator(|d| {
if let Decorator::If(content) = d {
Some(content.clone())
} else {
None
}
});
// let temp_path = temp_path.unwrap_or_else(|| PathBuf::from("/tmp"));
let temp_path = target_dir;
let filename = format!("bakefn_{}.sh", name);
let target = target_dir.join(filename);
let template_content = match render_mode {
RenderMode::Off => TRIVIAL_TEMPLATE.to_string(),
RenderMode::Minimal | RenderMode::Full => {
let script_debug = DebugFeature::from_bits_retain(debug_flags).contains(DebugFeature::Script);
if script_debug {
log::info!("Reading bake_base: {:?}", bake_base);
} else {
log::debug!("Reading bake_base: {:?}", bake_base);
}
std::fs::read_to_string(bake_base).map_err(|e| BakeError::IoError {
path: bake_base.display().to_string(),
source: e,
})?
}
};
let condition_code = match condition {
Some(content) => {
// with some bash workaround
format!(
"if [[ ! {} ]]; then\n exit {}\nfi\n",
content, PIPELINE_SKIP_ERRORCODE
)
}
None => String::new(),
};
// preexport/export: stubs for the future IPC-based @export.
let preexport_code = String::new();
let export_code = String::new();
let mut env = Environment::new();
env.add_template("script", &template_content)?;
let rendered = env.get_template("script")?.render(
context! {
main => body,
condition => condition_code,
preexport => preexport_code,
name => name,
export => export_code,
render_mode => render_mode.as_str(),
},
)?;
let mut temp_file =
NamedTempFile::new_in(&temp_path).map_err(|e| BakeError::ScriptGenerationFailed {
script: temp_path.display().to_string(),
action: "creating temp file".to_string(),
reason: e.to_string(),
})?;
temp_file
.as_file_mut()
.set_permissions(Permissions::from_mode(0o700))
.map_err(|e| BakeError::ScriptGenerationFailed {
script: temp_path.display().to_string(),
action: "setting permissions".to_string(),
reason: e.to_string(),
})?;
std::fs::write(&temp_file, rendered).map_err(|e| BakeError::ScriptGenerationFailed {
script: temp_path.display().to_string(),
action: "writing to temp file".to_string(),
reason: e.to_string(),
})?;
temp_file
.persist(&target)
.map_err(|e| BakeError::ScriptGenerationFailed {
script: target.display().to_string(),
action: "persisting temp file".to_string(),
reason: e.error.to_string(),
})?;
Ok(target)
}
#[cfg(test)]
mod tests {
use super::*;
#[test]
fn test_build_script_full_mode() {
let temp_dir = tempfile::tempdir().unwrap();
let bake_base = temp_dir.path().join("bake_base.sh");
std::fs::write(&bake_base, "{{ main }}\n").unwrap();
let function = Function {
name: "test".to_string(),
decorators: vec![],
body: "echo 'hello world'".to_string(),
};
let result = build_script(
&function, &bake_base, temp_dir.path(), RenderMode::Full, 0,
);
assert!(result.is_ok());
let script_path = result.unwrap();
let content = std::fs::read_to_string(&script_path).unwrap();
assert!(content.contains("echo 'hello world'"));
}
#[test]
fn test_build_script_off_mode() {
let temp_dir = tempfile::tempdir().unwrap();
let function = Function {
name: "test".to_string(),
decorators: vec![],
body: "echo 'direct output'".to_string(),
};
let result = build_script(
&function,
Path::new("/nonexistent"),
temp_dir.path(),
RenderMode::Off,
0,
);
assert!(result.is_ok());
let script_path = result.unwrap();
let content = std::fs::read_to_string(&script_path).unwrap();
assert_eq!(content.trim(), "echo 'direct output'");
}
}
+4
View File
@@ -0,0 +1,4 @@
pub const DEFAULT_WORKSPACE: &str = "/workspace";
pub const BAKE_SCRIPT_PATH: &str = "bake.sh";
pub const TRIVIAL_SCRIPT_NAME: &str = "BAKE_TRIVIAL";
pub const PIPELINE_SKIP_ERRORCODE: i32 = 233;
+999
View File
@@ -0,0 +1,999 @@
use lazy_static::lazy_static;
use regex::Regex;
use std::fmt::Display;
use std::fmt::Formatter;
use crate::bake::error::BakeError;
use crate::bake::parser::Function;
lazy_static! {
// "# @decorator" or "# @decorator(parameters)"
static ref DECORATOR_REGEX: Regex = Regex::new(r"^\s*#\s+@(\w+)\s?(\((.*)\))?").unwrap();
static ref UNEXPECTED_ARGUMENT: &'static str = "decorator does not accept arguments";
static ref MISSING_ARGUMENT: &'static str = "decorator requires an argument";
static ref UNKNOWN_DECORATOR: &'static str = "unknown decorator";
static ref INVALID_ARGUMENT: &'static str = "invalid argument";
}
/// Pipeline decorator representing shell script annotations in `# @name(args)` syntax.
#[derive(Debug, Clone, PartialEq)]
pub enum Decorator {
/// Unrecognized decorator name (used for error reporting).
Unknown,
/// Marks a function as a main pipeline step.
Pipeline,
/// Conditional execution with shell condition.
If(String),
/// Allows the step to fail without halting the pipeline.
Fallible,
/// Repeat the function body N times.
Loop(usize),
/// Run after the specified function completes.
After(String),
/// Execution timeout in seconds.
Timeout(u32),
/// Retry N times with M second delay between attempts.
Retry(usize, u32),
/// Export environment variables to subsequent steps.
Export,
/// Chain functions via pipe with comma-separated names.
Pipe(Vec<String>),
/// Run concurrently with other parallel functions.
/// `None` = wildcard ("*" group, compatible with any group).
/// `Some(name)` = restricted to the named group.
Parallel(Option<String>),
/// Run as a background daemon process.
Daemon,
/// Health check command for daemon verification (shell statement).
Health(String),
/// Fire-and-forget execution; DAG pop happens at start, not end.
Async,
}
/// Returns true if the decorator is a flag (position-independent, consumed after execution).
pub fn is_flag(d: &Decorator) -> bool {
matches!(
d,
Decorator::Pipeline
| Decorator::Fallible
| Decorator::Export
| Decorator::If(_)
| Decorator::After(_)
| Decorator::Parallel(_)
| Decorator::Health(_)
| Decorator::Async
| Decorator::Unknown
)
}
/// Returns true if the decorator is a combinator (position-dependent, nestable).
pub fn is_combinator(d: &Decorator) -> bool {
matches!(d, Decorator::Loop(_) | Decorator::Timeout(_) | Decorator::Retry(..))
}
/// Returns true if the decorator is a mode (at most one, alters what gets executed).
pub fn is_mode(d: &Decorator) -> bool {
matches!(d, Decorator::Pipe(_) | Decorator::Daemon | Decorator::Async)
}
/// Extract the parallel group name. Returns "*" for wildcard (no group specified).
pub fn parallel_group(d: &Decorator) -> &str {
match d {
Decorator::Parallel(Some(g)) => g.as_str(),
_ => "*",
}
}
/// Check for incompatible decorator combinations on a function.
/// Returns `Err(IncompatibleDecorators)` on conflict, `Ok(())` otherwise.
pub fn check_conflicts(func: &Function) -> Result<(), BakeError> {
let has_daemon = func.decorators.iter().any(|d| matches!(d, Decorator::Daemon));
let has_async = func.decorators.iter().any(|d| matches!(d, Decorator::Async));
let has_pipe = func.decorators.iter().any(|d| matches!(d, Decorator::Pipe(_)));
let has_parallel = func.decorators.iter().any(|d| matches!(d, Decorator::Parallel(_)));
let has_health = func.decorators.iter().any(|d| matches!(d, Decorator::Health(_)));
let has_export = func.decorators.iter().any(|d| matches!(d, Decorator::Export));
let has_combinator = func.decorators.iter().any(|d| is_combinator(d));
let mode_count = [has_daemon, has_async, has_pipe].iter().filter(|&&x| x).count();
if mode_count > 1 {
return Err(BakeError::IncompatibleDecorators {
function: func.name.clone(),
reason: "@daemon, @async, and @pipe are mutually exclusive".into(),
});
}
if has_daemon {
if has_parallel {
return Err(BakeError::IncompatibleDecorators {
function: func.name.clone(),
reason: "@daemon cannot be combined with @parallel".into(),
});
}
if has_combinator {
return Err(BakeError::IncompatibleDecorators {
function: func.name.clone(),
reason: "@daemon cannot be combined with @loop/@retry/@timeout".into(),
});
}
if has_export {
return Err(BakeError::IncompatibleDecorators {
function: func.name.clone(),
reason: "@daemon cannot be combined with @export".into(),
});
}
}
if has_health && !has_daemon {
return Err(BakeError::IncompatibleDecorators {
function: func.name.clone(),
reason: "@health requires @daemon".into(),
});
}
if has_async {
if has_parallel {
return Err(BakeError::IncompatibleDecorators {
function: func.name.clone(),
reason: "@async cannot be combined with @parallel".into(),
});
}
if has_export {
return Err(BakeError::IncompatibleDecorators {
function: func.name.clone(),
reason: "@async cannot be combined with @export".into(),
});
}
}
if has_pipe && has_parallel {
return Err(BakeError::IncompatibleDecorators {
function: func.name.clone(),
reason: "@pipe cannot be combined with @parallel".into(),
});
}
Ok(())
}
impl Display for Decorator {
fn fmt(&self, f: &mut Formatter<'_>) -> std::fmt::Result {
match self {
Decorator::Unknown => write!(f, "@<unknown>"),
Decorator::Pipeline => write!(f, "@pipeline"),
Decorator::If(cond) => write!(f, "@if({})", cond),
Decorator::Fallible => write!(f, "@fallible"),
Decorator::Loop(count) => write!(f, "@loop({})", count),
Decorator::After(func) => write!(f, "@after({})", func),
Decorator::Timeout(timeout) => write!(f, "@timeout({})", timeout),
Decorator::Retry(count, delay) => write!(f, "@retry({}, {})", count, delay),
Decorator::Export => write!(f, "@export"),
Decorator::Pipe(funcs) => write!(f, "@pipe({})", funcs.join(", ")),
Decorator::Parallel(group) => match group {
None => write!(f, "@parallel"),
Some(g) => write!(f, "@parallel({})", g),
},
Decorator::Daemon => write!(f, "@daemon"),
Decorator::Health(check) => write!(f, "@health({})", check),
Decorator::Async => write!(f, "@async"),
}
}
}
/// Parses a line for `# @name(args)` decorator syntax.
/// Returns `Ok(None)` if the line is not a decorator line, `Ok(Some(Decorator))` on match.
pub fn parse_decorator(line: &str, line_num: usize) -> Result<Option<Decorator>, BakeError> {
let caps = match DECORATOR_REGEX.captures(line) {
Some(c) => c,
None => return Ok(None), // Actually not a decorator
};
let name = caps.get(1).unwrap().as_str();
let has_arg = caps.get(2).is_some();
let args = caps.get(3).map(|m| m.as_str()).unwrap_or("");
let decorator_noarg = |d: Decorator| -> Result<Option<Decorator>, BakeError> {
if has_arg {
Err(BakeError::DecoratorParseError {
decorator: name.to_string(),
line_num,
reason: UNEXPECTED_ARGUMENT.to_string(),
})
} else {
Ok(Some(d))
}
};
let required_arg = || -> Result<&str, BakeError> {
if !has_arg || args.is_empty() {
Err(BakeError::DecoratorParseError {
decorator: name.to_string(),
line_num,
reason: MISSING_ARGUMENT.to_string(),
})
} else {
Ok(args)
}
};
match name {
"pipeline" => decorator_noarg(Decorator::Pipeline),
"if" => required_arg().map(|a| Some(Decorator::If(a.to_string()))),
"fallible" => decorator_noarg(Decorator::Fallible),
"loop" => required_arg().and_then(|a| {
let count = a.parse().map_err(|_e| BakeError::DecoratorParseError {
decorator: "loop".to_string(),
line_num,
reason: format!("invalid loop count: {}", a),
})?;
Ok(Some(Decorator::Loop(count)))
}),
"after" => required_arg().map(|a| Some(Decorator::After(a.to_string()))),
"timeout" => required_arg().and_then(|a| {
let timeout = a.parse().map_err(|_e| BakeError::DecoratorParseError {
decorator: "timeout".to_string(),
line_num,
reason: format!("invalid timeout: {}", a),
})?;
Ok(Some(Decorator::Timeout(timeout)))
}),
"retry" => required_arg().and_then(|a| {
let parts: Vec<&str> = a.split(',').collect();
if parts.len() != 2 {
Err(BakeError::DecoratorParseError {
decorator: "retry".to_string(),
line_num,
reason: format!("invalid retry args: {}", a),
})
} else {
let count =
parts[0]
.trim()
.parse()
.map_err(|_e| BakeError::DecoratorParseError {
decorator: "retry".to_string(),
line_num,
reason: format!("invalid retry count: {}", parts[0]),
})?;
let delay_str = parts[1].trim();
if delay_str.is_empty() {
Err(BakeError::DecoratorParseError {
decorator: "retry".to_string(),
line_num,
reason: "no delay specified".to_string(),
})
} else {
let delay = delay_str
.parse()
.map_err(|_e| BakeError::DecoratorParseError {
decorator: "retry".to_string(),
line_num,
reason: format!("invalid retry delay: {}", parts[1]),
})?;
Ok(Some(Decorator::Retry(count, delay)))
}
}
}),
"export" => decorator_noarg(Decorator::Export),
"parallel" => {
if has_arg {
let group = required_arg()?;
Ok(Some(Decorator::Parallel(Some(group.to_string()))))
} else {
Ok(Some(Decorator::Parallel(None)))
}
}
"daemon" => decorator_noarg(Decorator::Daemon),
"health" => required_arg().map(|a| Some(Decorator::Health(a.to_string()))),
"async" => decorator_noarg(Decorator::Async),
"pipe" => required_arg().and_then(|a| {
let functions: Vec<String> = a
.split(',')
.map(|s| s.trim().to_string())
.filter(|s| !s.is_empty())
.collect();
if functions.is_empty() {
Err(BakeError::DecoratorParseError {
decorator: "pipe".to_string(),
line_num,
reason: "no functions specified".to_string(),
})
} else {
Ok(Some(Decorator::Pipe(functions)))
}
}),
d => Err(BakeError::DecoratorParseError {
decorator: d.to_string(),
line_num,
reason: UNKNOWN_DECORATOR.to_string(),
}),
}
}
#[cfg(test)]
mod tests {
use super::*;
// =====================================================================
// Tests for Decorator Display implementation
// =====================================================================
#[test]
fn test_decorator_display_pipeline() {
assert_eq!(format!("{}", Decorator::Pipeline), "@pipeline");
}
#[test]
fn test_decorator_display_if() {
assert_eq!(
format!("{}", Decorator::If("cond".to_string())),
"@if(cond)"
);
assert_eq!(
format!("{}", Decorator::If("x > 0".to_string())),
"@if(x > 0)"
);
}
#[test]
fn test_decorator_display_fallible() {
assert_eq!(format!("{}", Decorator::Fallible), "@fallible");
}
#[test]
fn test_decorator_display_loop() {
assert_eq!(format!("{}", Decorator::Loop(5)), "@loop(5)");
assert_eq!(format!("{}", Decorator::Loop(0)), "@loop(0)");
assert_eq!(
format!("{}", Decorator::Loop(usize::MAX)),
format!("@loop({})", usize::MAX)
);
}
#[test]
fn test_decorator_display_after() {
assert_eq!(
format!("{}", Decorator::After("func1".to_string())),
"@after(func1)"
);
assert_eq!(
format!("{}", Decorator::After("build_step_1".to_string())),
"@after(build_step_1)"
);
}
#[test]
fn test_decorator_display_timeout() {
assert_eq!(format!("{}", Decorator::Timeout(30)), "@timeout(30)");
assert_eq!(format!("{}", Decorator::Timeout(0)), "@timeout(0)");
assert_eq!(
format!("{}", Decorator::Timeout(u32::MAX)),
format!("@timeout({})", u32::MAX)
);
}
#[test]
fn test_decorator_display_retry() {
assert_eq!(format!("{}", Decorator::Retry(3, 100)), "@retry(3, 100)");
assert_eq!(format!("{}", Decorator::Retry(0, 0)), "@retry(0, 0)");
assert_eq!(
format!("{}", Decorator::Retry(10, 5000)),
"@retry(10, 5000)"
);
}
#[test]
fn test_decorator_display_export() {
assert_eq!(format!("{}", Decorator::Export), "@export");
}
#[test]
fn test_decorator_display_pipe() {
assert_eq!(format!("{}", Decorator::Pipe(vec![])), "@pipe()");
assert_eq!(
format!("{}", Decorator::Pipe(vec!["f1".to_string()])),
"@pipe(f1)"
);
assert_eq!(
format!(
"{}",
Decorator::Pipe(vec!["f1".to_string(), "f2".to_string()])
),
"@pipe(f1, f2)"
);
assert_eq!(
format!(
"{}",
Decorator::Pipe(vec!["a".to_string(), "b".to_string(), "c".to_string()])
),
"@pipe(a, b, c)"
);
}
#[test]
fn test_decorator_display_parallel() {
assert_eq!(format!("{}", Decorator::Parallel(None)), "@parallel");
assert_eq!(
format!("{}", Decorator::Parallel(Some("build".to_string()))),
"@parallel(build)"
);
}
#[test]
fn test_decorator_display_daemon() {
assert_eq!(format!("{}", Decorator::Daemon), "@daemon");
}
#[test]
fn test_decorator_display_health() {
assert_eq!(
format!("{}", Decorator::Health("http://localhost:8080".to_string())),
"@health(http://localhost:8080)"
);
assert_eq!(
format!("{}", Decorator::Health("/health".to_string())),
"@health(/health)"
);
}
#[test]
fn test_decorator_display_async() {
assert_eq!(format!("{}", Decorator::Async), "@async");
}
#[test]
fn test_decorator_display_unknown() {
assert_eq!(format!("{}", Decorator::Unknown), "@<unknown>");
}
// =====================================================================
// Tests for parse_decorator - Valid no-argument decorators
// =====================================================================
#[test]
fn test_parse_decorator_pipeline() {
let result = parse_decorator("# @pipeline", 1).unwrap();
assert_eq!(result, Some(Decorator::Pipeline));
}
#[test]
fn test_parse_decorator_fallible() {
let result = parse_decorator("# @fallible", 1).unwrap();
assert_eq!(result, Some(Decorator::Fallible));
}
#[test]
fn test_parse_decorator_export() {
let result = parse_decorator("# @export", 1).unwrap();
assert_eq!(result, Some(Decorator::Export));
}
#[test]
fn test_parse_decorator_parallel() {
let result = parse_decorator("# @parallel", 1).unwrap();
assert_eq!(result, Some(Decorator::Parallel(None)));
}
#[test]
fn test_parse_decorator_parallel_with_group() {
let result = parse_decorator("# @parallel(build)", 1).unwrap();
assert_eq!(result, Some(Decorator::Parallel(Some("build".to_string()))));
}
#[test]
fn test_parse_decorator_daemon() {
let result = parse_decorator("# @daemon", 1).unwrap();
assert_eq!(result, Some(Decorator::Daemon));
}
#[test]
fn test_parse_decorator_async() {
let result = parse_decorator("# @async", 1).unwrap();
assert_eq!(result, Some(Decorator::Async));
}
// =====================================================================
// Tests for parse_decorator - Valid single-argument decorators
// =====================================================================
#[test]
fn test_parse_decorator_if() {
let result = parse_decorator("# @if(condition)", 1).unwrap();
assert_eq!(result, Some(Decorator::If("condition".to_string())));
}
#[test]
fn test_parse_decorator_if_with_complex_condition() {
let result = parse_decorator("# @if(env.STATUS == \"ready\")", 1).unwrap();
assert_eq!(
result,
Some(Decorator::If("env.STATUS == \"ready\"".to_string()))
);
}
#[test]
fn test_parse_decorator_after() {
let result = parse_decorator("# @after(build_step)", 1).unwrap();
assert_eq!(result, Some(Decorator::After("build_step".to_string())));
}
#[test]
fn test_parse_decorator_after_with_underscore() {
let result = parse_decorator("# @after(build_step_1)", 1).unwrap();
assert_eq!(result, Some(Decorator::After("build_step_1".to_string())));
}
#[test]
fn test_parse_decorator_loop() {
let result = parse_decorator("# @loop(5)", 1).unwrap();
assert_eq!(result, Some(Decorator::Loop(5)));
}
#[test]
fn test_parse_decorator_loop_zero() {
let result = parse_decorator("# @loop(0)", 1).unwrap();
assert_eq!(result, Some(Decorator::Loop(0)));
}
#[test]
fn test_parse_decorator_health() {
let result = parse_decorator("# @health(/health)", 1).unwrap();
assert_eq!(result, Some(Decorator::Health("/health".to_string())));
}
#[test]
fn test_parse_decorator_health_url() {
let result = parse_decorator("# @health(http://localhost:8080/health)", 1).unwrap();
assert_eq!(
result,
Some(Decorator::Health(
"http://localhost:8080/health".to_string()
))
);
}
#[test]
fn test_parse_decorator_timeout() {
let result = parse_decorator("# @timeout(30)", 1).unwrap();
assert_eq!(result, Some(Decorator::Timeout(30)));
}
#[test]
fn test_parse_decorator_timeout_zero() {
let result = parse_decorator("# @timeout(0)", 1).unwrap();
assert_eq!(result, Some(Decorator::Timeout(0)));
}
// =====================================================================
// Tests for parse_decorator - Valid multi-argument decorators
// =====================================================================
#[test]
fn test_parse_decorator_retry() {
let result = parse_decorator("# @retry(3, 100)", 1).unwrap();
assert_eq!(result, Some(Decorator::Retry(3, 100)));
}
#[test]
fn test_parse_decorator_retry_with_spaces() {
let result = parse_decorator("# @retry(3,100)", 1).unwrap();
assert_eq!(result, Some(Decorator::Retry(3, 100)));
}
#[test]
fn test_parse_decorator_retry_with_many_spaces() {
let result = parse_decorator("# @retry( 3 , 100 )", 1).unwrap();
assert_eq!(result, Some(Decorator::Retry(3, 100)));
}
#[test]
fn test_parse_decorator_retry_zero_count() {
let result = parse_decorator("# @retry(0, 0)", 1).unwrap();
assert_eq!(result, Some(Decorator::Retry(0, 0)));
}
#[test]
fn test_parse_decorator_pipe() {
let result = parse_decorator("# @pipe(func1)", 1).unwrap();
assert_eq!(result, Some(Decorator::Pipe(vec!["func1".to_string()])));
}
#[test]
fn test_parse_decorator_pipe_multiple() {
let result = parse_decorator("# @pipe(func1, func2, func3)", 1).unwrap();
assert_eq!(
result,
Some(Decorator::Pipe(vec![
"func1".to_string(),
"func2".to_string(),
"func3".to_string()
]))
);
}
#[test]
fn test_parse_decorator_pipe_with_spaces() {
let result = parse_decorator("# @pipe( func1 , func2 )", 1).unwrap();
assert_eq!(
result,
Some(Decorator::Pipe(vec![
"func1".to_string(),
"func2".to_string()
]))
);
}
// =====================================================================
// Tests for parse_decorator - Whitespace handling
// =====================================================================
#[test]
fn test_parse_decorator_with_leading_whitespace() {
let result = parse_decorator(" # @pipeline", 1).unwrap();
assert_eq!(result, Some(Decorator::Pipeline));
}
#[test]
fn test_parse_decorator_with_extra_spaces() {
let result = parse_decorator("# @pipeline", 1).unwrap();
assert_eq!(result, Some(Decorator::Pipeline));
}
#[test]
fn test_parse_decorator_with_tab() {
let result = parse_decorator("#\t@pipeline", 1).unwrap();
assert_eq!(result, Some(Decorator::Pipeline));
}
#[test]
fn test_parse_decorator_no_space_between_hash_and_at() {
// Regex requires at least one whitespace between # and @
// So "#@pipeline" is NOT a valid decorator
let result = parse_decorator("#@pipeline", 1).unwrap();
assert_eq!(result, None);
}
// =====================================================================
// Tests for parse_decorator - Non-decorator lines
// =====================================================================
#[test]
fn test_parse_decorator_no_hash() {
let result = parse_decorator("@pipeline", 1).unwrap();
assert_eq!(result, None);
}
#[test]
fn test_parse_decorator_empty_line() {
let result = parse_decorator("", 1).unwrap();
assert_eq!(result, None);
}
#[test]
fn test_parse_decorator_only_whitespace() {
let result = parse_decorator(" ", 1).unwrap();
assert_eq!(result, None);
}
#[test]
fn test_parse_decorator_regular_comment() {
let result = parse_decorator("# This is a regular comment", 1).unwrap();
assert_eq!(result, None);
}
#[test]
fn test_parse_decorator_code_line() {
let result = parse_decorator("echo 'Hello World'", 1).unwrap();
assert_eq!(result, None);
}
// =====================================================================
// Tests for parse_decorator - Invalid: decorators with wrong arguments
// =====================================================================
#[test]
fn test_parse_decorator_pipeline_with_arg() {
let result = parse_decorator("# @pipeline(something)", 1);
assert!(result.is_err());
let err = result.unwrap_err();
assert!(matches!(err, BakeError::DecoratorParseError { .. }));
if let BakeError::DecoratorParseError {
decorator, reason, ..
} = err
{
assert_eq!(decorator, "pipeline");
assert!(reason.contains("does not accept arguments"));
}
}
#[test]
fn test_parse_decorator_fallible_with_arg() {
let result = parse_decorator("# @fallible(true)", 1);
assert!(result.is_err());
}
#[test]
fn test_parse_decorator_export_with_arg() {
let result = parse_decorator("# @export(mode)", 1);
assert!(result.is_err());
}
#[test]
fn test_parse_decorator_parallel_empty_args() {
let result = parse_decorator("# @parallel()", 1);
assert!(result.is_err());
}
#[test]
fn test_parse_decorator_daemon_with_arg() {
let result = parse_decorator("# @daemon(stop)", 1);
assert!(result.is_err());
}
// =====================================================================
// Tests for parse_decorator - Invalid: decorators missing required arguments
// =====================================================================
#[test]
fn test_parse_decorator_if_no_arg() {
let result = parse_decorator("# @if", 1);
assert!(result.is_err());
let err = result.unwrap_err();
assert!(matches!(err, BakeError::DecoratorParseError { .. }));
if let BakeError::DecoratorParseError {
decorator, reason, ..
} = err
{
assert_eq!(decorator, "if");
assert!(reason.contains("requires an argument"));
}
}
#[test]
fn test_parse_decorator_if_empty_parens() {
let result = parse_decorator("# @if()", 1);
assert!(result.is_err());
}
#[test]
fn test_parse_decorator_after_no_arg() {
let result = parse_decorator("# @after", 1);
assert!(result.is_err());
}
#[test]
fn test_parse_decorator_loop_no_arg() {
let result = parse_decorator("# @loop", 1);
assert!(result.is_err());
}
#[test]
fn test_parse_decorator_loop_empty_parens() {
let result = parse_decorator("# @loop()", 1);
assert!(result.is_err());
}
#[test]
fn test_parse_decorator_timeout_no_arg() {
let result = parse_decorator("# @timeout", 1);
assert!(result.is_err());
}
#[test]
fn test_parse_decorator_retry_no_arg() {
let result = parse_decorator("# @retry", 1);
assert!(result.is_err());
}
#[test]
fn test_parse_decorator_retry_only_count() {
let result = parse_decorator("# @retry(3)", 1);
assert!(result.is_err());
}
#[test]
fn test_parse_decorator_retry_only_comma() {
let result = parse_decorator("# @retry(,)", 1);
assert!(result.is_err());
}
#[test]
fn test_parse_decorator_health_no_arg() {
let result = parse_decorator("# @health", 1);
assert!(result.is_err());
}
#[test]
fn test_parse_decorator_pipe_no_arg() {
let result = parse_decorator("# @pipe", 1);
assert!(result.is_err());
}
#[test]
fn test_parse_decorator_pipe_empty_parens() {
let result = parse_decorator("# @pipe()", 1);
assert!(result.is_err());
}
#[test]
fn test_parse_decorator_pipe_only_commas() {
let result = parse_decorator("# @pipe(,,)", 1);
assert!(result.is_err());
}
// =====================================================================
// Tests for parse_decorator - Invalid: wrong argument types
// =====================================================================
#[test]
fn test_parse_decorator_loop_non_numeric() {
let result = parse_decorator("# @loop(abc)", 1);
assert!(result.is_err());
let err = result.unwrap_err();
if let BakeError::DecoratorParseError { reason, .. } = err {
assert!(reason.contains("invalid loop count"));
}
}
#[test]
fn test_parse_decorator_loop_float() {
let result = parse_decorator("# @loop(3.14)", 1);
assert!(result.is_err());
}
#[test]
fn test_parse_decorator_loop_negative() {
let result = parse_decorator("# @loop(-5)", 1);
// usize parsing will fail for negative numbers
assert!(result.is_err());
}
#[test]
fn test_parse_decorator_timeout_non_numeric() {
let result = parse_decorator("# @timeout(abc)", 1);
assert!(result.is_err());
let err = result.unwrap_err();
if let BakeError::DecoratorParseError { reason, .. } = err {
assert!(reason.contains("invalid timeout"));
}
}
#[test]
fn test_parse_decorator_retry_non_numeric_count() {
let result = parse_decorator("# @retry(abc, 100)", 1);
assert!(result.is_err());
let err = result.unwrap_err();
if let BakeError::DecoratorParseError { reason, .. } = err {
assert!(reason.contains("invalid retry count"));
}
}
#[test]
fn test_parse_decorator_retry_non_numeric_delay() {
let result = parse_decorator("# @retry(3, abc)", 1);
assert!(result.is_err());
let err = result.unwrap_err();
if let BakeError::DecoratorParseError { reason, .. } = err {
assert!(reason.contains("invalid retry delay"));
}
}
#[test]
fn test_parse_decorator_retry_empty_delay() {
let result = parse_decorator("# @retry(3, )", 1);
assert!(result.is_err());
let err = result.unwrap_err();
if let BakeError::DecoratorParseError { reason, .. } = err {
assert!(reason.contains("no delay specified"));
}
}
// =====================================================================
// Tests for parse_decorator - Invalid: unknown decorators
// =====================================================================
#[test]
fn test_parse_decorator_unknown() {
let result = parse_decorator("# @unknown", 1);
assert!(result.is_err());
let err = result.unwrap_err();
assert!(matches!(err, BakeError::DecoratorParseError { .. }));
if let BakeError::DecoratorParseError {
decorator,
reason,
line_num,
} = err
{
assert_eq!(decorator, "unknown");
assert_eq!(line_num, 1);
assert!(reason.contains("unknown decorator"));
}
}
#[test]
fn test_parse_decorator_typo_name() {
let result = parse_decorator("# @pipelne", 1);
assert!(result.is_err());
let err = result.unwrap_err();
if let BakeError::DecoratorParseError { decorator, .. } = err {
assert_eq!(decorator, "pipelne");
}
}
#[test]
fn test_parse_decorator_uppercase_name() {
let result = parse_decorator("# @PIPELINE", 1);
assert!(result.is_err());
// Regex is case-sensitive, so it won't match @PIPELINE
assert!(result.is_err());
}
// =====================================================================
// Tests for parse_decorator - Line number tracking
// =====================================================================
#[test]
fn test_parse_decorator_line_number_preserved() {
let result = parse_decorator("# @pipeline", 42);
assert!(result.is_ok());
// Line number is preserved in error cases
let result = parse_decorator("# @pipeline(invalid)", 42);
assert!(result.is_err());
if let BakeError::DecoratorParseError { line_num, .. } = result.unwrap_err() {
assert_eq!(line_num, 42);
}
}
// =====================================================================
// Tests for parse_decorator - Edge cases
// =====================================================================
#[test]
fn test_parse_decorator_very_large_loop_count() {
let result = parse_decorator(&format!("# @loop({})", usize::MAX), 1);
assert!(result.is_ok());
assert_eq!(result.unwrap(), Some(Decorator::Loop(usize::MAX)));
}
#[test]
fn test_parse_decorator_very_large_timeout() {
let result = parse_decorator(&format!("# @timeout({})", u32::MAX), 1);
assert!(result.is_ok());
assert_eq!(result.unwrap(), Some(Decorator::Timeout(u32::MAX)));
}
#[test]
fn test_parse_decorator_if_with_special_chars() {
// Test condition with special characters that might break parsing
let result = parse_decorator(r#"# @if(x != "" && y > 0)"#, 1).unwrap();
assert_eq!(
result,
Some(Decorator::If(r#"x != "" && y > 0"#.to_string()))
);
}
#[test]
fn test_parse_decorator_pipe_with_underscores() {
let result = parse_decorator("# @pipe(step_1, step_2, step_3)", 1).unwrap();
assert_eq!(
result,
Some(Decorator::Pipe(vec![
"step_1".to_string(),
"step_2".to_string(),
"step_3".to_string()
]))
);
}
#[test]
fn test_parse_decorator_multiple_in_sequence() {
// Simulate multiple decorator lines
let line1 = parse_decorator("# @pipeline", 1).unwrap();
let line2 = parse_decorator("# @if(condition)", 2).unwrap();
let line3 = parse_decorator("# @timeout(60)", 3).unwrap();
assert_eq!(line1, Some(Decorator::Pipeline));
assert_eq!(line2, Some(Decorator::If("condition".to_string())));
assert_eq!(line3, Some(Decorator::Timeout(60)));
}
}
+60
View File
@@ -0,0 +1,60 @@
use thiserror::Error;
use workshop_engine::{
ExecutionError,
error::{EXITCODE_IO_ERROR, EXITCODE_PARSE_ERROR, HasExitCode},
};
#[derive(Error, Debug)]
pub enum BakeError {
#[error("Failed to generate staged script {script} when {action}: {reason}")]
ScriptGenerationFailed { script: String, action: String, reason: String },
#[error("Template error: {0}")]
TemplateError(#[from] minijinja::Error),
#[error("Circular Dependency detected in bake.sh")]
CircularDependency(Vec<String>),
#[error("Unknown @after of function {0}: {1}")]
UnknownDependency(String, String),
#[error("IO error on {path}: {source}")]
IoError { path: String, source: std::io::Error },
#[error("YAML parse error: {0}")]
YamlParseError(#[from] serde_yaml::Error),
#[error("Failed to parse decorator @{decorator} (line #{line_num}): {reason}")]
DecoratorParseError {
decorator: String,
line_num: usize,
reason: String,
},
#[error("Script execution error: {0}")]
ScriptExecutionError(#[from] ExecutionError),
#[error("Incompatible decorators on function '{function}': {reason}")]
IncompatibleDecorators { function: String, reason: String },
#[error("Health probe failed for daemon '{command}': {error}")]
HealthProbeFailed { command: String, error: String },
}
impl HasExitCode for BakeError {
fn exit_code(&self) -> i32 {
match self {
BakeError::ScriptGenerationFailed { .. } => EXITCODE_IO_ERROR,
BakeError::TemplateError(_) => EXITCODE_PARSE_ERROR,
BakeError::CircularDependency(_) => EXITCODE_PARSE_ERROR,
BakeError::UnknownDependency(..) => EXITCODE_PARSE_ERROR,
BakeError::IoError { .. } => EXITCODE_IO_ERROR,
BakeError::YamlParseError(_) => EXITCODE_PARSE_ERROR,
BakeError::DecoratorParseError { .. } => EXITCODE_PARSE_ERROR,
BakeError::ScriptExecutionError(e) => e.exit_code(),
BakeError::IncompatibleDecorators { .. } => EXITCODE_PARSE_ERROR,
BakeError::HealthProbeFailed { .. } => EXITCODE_IO_ERROR,
}
}
}
+383
View File
@@ -0,0 +1,383 @@
use crate::types::debug::DebugFeature;
use crate::bake::builder::{self, RenderMode};
use crate::bake::constant::PIPELINE_SKIP_ERRORCODE;
use crate::bake::decorator::{self, Decorator};
use crate::bake::error::BakeError;
use crate::bake::parser::Function;
use crate::bake::util::{is_fallible, resolve_pipe_stdin};
use std::collections::HashMap;
use std::future::Future;
use std::path::{Path, PathBuf};
use std::pin::Pin;
use std::process::Stdio;
use std::time::Duration;
use workshop_engine::{Engine, EventSender, ExecutionContext, ExecutionError};
/// Execute a single function with combinator wrapping.
/// Returns the captured stdout (for `@pipe` consumers).
pub async fn execute_one(
engine: &Engine,
func: &Function,
ctx: &mut ExecutionContext,
event_tx: &EventSender,
workspace: &Path,
remaining: &str,
bake_base: &Path,
) -> Result<String, BakeError> {
let combos: Vec<&Decorator> = func
.decorators
.iter()
.filter(|d| decorator::is_combinator(d))
.collect();
let mut modified = func.clone();
modified.body = remaining.to_string() + &func.body;
let script = builder::build_script(
&modified, bake_base, workspace, RenderMode::Full, ctx.debug_flags,
)?;
execute_with_combos(engine, &script, ctx, event_tx, &combos).await
}
async fn execute_with_combos(
engine: &Engine,
script: &PathBuf,
ctx: &mut ExecutionContext,
event_tx: &EventSender,
combos: &[&Decorator],
) -> Result<String, BakeError> {
build_combinator_chain(engine, script, ctx, event_tx, combos, 0).await
}
/// Builds a combinator execution chain. Returns the captured stdout.
/// For `@loop` + `@pipe(self)`, each iteration feeds its stdout back as
/// the next iteration's stdin via `ctx.stdin`.
fn build_combinator_chain<'a>(
engine: &'a Engine,
script: &'a PathBuf,
ctx: &'a mut ExecutionContext,
event_tx: &'a EventSender,
combos: &'a [&'a Decorator],
idx: usize,
) -> Pin<Box<dyn Future<Output = Result<String, BakeError>> + Send + 'a>> {
if idx >= combos.len() {
return Box::pin(async move {
let result = engine.execute_script(script, ctx, event_tx).await;
match result {
Ok(r) => Ok(r.stdout),
Err(ExecutionError::ExecutionFailed { exit_code, .. })
if exit_code == PIPELINE_SKIP_ERRORCODE as i32 =>
{
log::debug!("Function skipped (@if condition)");
Ok(String::new())
}
Err(e) => Err(BakeError::ScriptExecutionError(e)),
}
});
}
match combos[idx] {
Decorator::Loop(count) => Box::pin(async move {
let mut last_stdout = String::new();
for i in 0..*count {
let script_debug = DebugFeature::from_bits_retain(ctx.debug_flags).contains(DebugFeature::Script);
if script_debug {
log::info!("Loop {}/{}", i + 1, count);
} else {
log::debug!("Loop {}/{}", i + 1, count);
}
last_stdout = build_combinator_chain(
engine, script, ctx, event_tx, combos, idx + 1,
)
.await?;
// Feed stdout as stdin for the next iteration (@pipe(self))
if i + 1 < *count {
ctx.stdin = Some(last_stdout.clone().into_bytes());
}
}
Ok(last_stdout)
}),
Decorator::Timeout(secs) => Box::pin(async move {
tokio::time::timeout(
Duration::from_secs(*secs as u64),
build_combinator_chain(
engine, script, ctx, event_tx, combos, idx + 1,
),
)
.await
.map_err(|_| BakeError::ScriptExecutionError(ExecutionError::Timeout))?
}),
Decorator::Retry(retries, delay_secs) => Box::pin(async move {
let max_attempts = *retries as usize + 1;
let delay = Duration::from_secs(*delay_secs as u64);
let mut last_err = None;
for attempt in 0..max_attempts {
match build_combinator_chain(
engine, script, ctx, event_tx, combos, idx + 1,
)
.await
{
Ok(stdout) => return Ok(stdout),
Err(e) => {
if matches!(
&e,
BakeError::ScriptExecutionError(ExecutionError::Timeout)
) {
return Err(e);
}
if attempt < max_attempts - 1 {
last_err = Some(e);
log::warn!(
"Retry {}/{} after {}s",
attempt + 1,
retries,
delay_secs
);
tokio::time::sleep(delay).await;
} else {
return Err(e);
}
}
}
}
Err(last_err.unwrap())
}),
_ => build_combinator_chain(
engine, script, ctx, event_tx, combos, idx + 1,
),
}
}
/// Fire a batch of parallel nodes concurrently.
///
/// All nodes in the batch are waited on before returning. A non-`@fallible`
/// failure is surfaced only after all nodes complete (GitHub/GitLab/Concourse
/// convention).
///
/// Returns a `(name, stdout)` pair for each completed node so the caller can
/// populate the stdout cache for `@pipe`.
pub async fn fire_parallel_batch(
batch: &[String],
_engine: &Engine,
func_map: &HashMap<String, Function>,
ctx: &mut ExecutionContext,
event_tx: &EventSender,
workspace: &Path,
remaining: &str,
bake_base: &Path,
stdout_cache: &HashMap<String, String>,
) -> Result<Vec<(String, String)>, BakeError> {
let handles: Vec<_> = batch
.iter()
.map(|name| {
let function = func_map[name.as_str()].clone();
let mut task_ctx = ctx.clone();
// Resolve per-task stdin from @pipe + stdout_cache
task_ctx.stdin = resolve_pipe_stdin(&function, stdout_cache);
let task_events = event_tx.clone();
let task_workspace = workspace.to_path_buf();
let task_remaining = remaining.to_string();
let task_bake_base = bake_base.to_path_buf();
let task_name = name.clone();
tokio::spawn(async move {
let task_engine = Engine::new();
let result = execute_one(
&task_engine,
&function,
&mut task_ctx,
&task_events,
&task_workspace,
&task_remaining,
&task_bake_base,
)
.await;
(task_name, result)
})
})
.collect();
let mut first_fatal: Option<BakeError> = None;
let mut results: Vec<(String, String)> = Vec::with_capacity(handles.len());
for handle in handles {
let (name, result) = handle.await.map_err(|_| {
BakeError::ScriptExecutionError(ExecutionError::ExecutionFailed {
task_id: "parallel".to_string(),
exit_code: -1,
})
})?;
match result {
Ok(stdout) => results.push((name, stdout)),
Err(err) if is_fallible(&func_map[&name]) => {
log::warn!("'{}' failed (fallible, non-fatal): {}", name, err);
}
Err(err) => {
log::error!("'{}' failed (fatal): {}", name, err);
first_fatal.get_or_insert(err);
}
}
}
if let Some(err) = first_fatal {
return Err(err);
}
Ok(results)
}
/// Spawn an `@async` function in the background.
/// Returns a `JoinHandle` that the caller must await before returning.
pub fn execute_async(
func: &Function,
ctx: &ExecutionContext,
event_tx: &EventSender,
workspace: &Path,
remaining: &str,
bake_base: &Path,
) -> tokio::task::JoinHandle<()> {
let mut task_ctx = ctx.clone();
let task_events = event_tx.clone();
let task_workspace = workspace.to_path_buf();
let task_remaining = remaining.to_string();
let task_bake_base = bake_base.to_path_buf();
let task_func = func.clone();
tokio::spawn(async move {
let task_engine = Engine::new();
let _ = execute_one(
&task_engine, &task_func, &mut task_ctx,
&task_events, &task_workspace,
&task_remaining, &task_bake_base,
).await;
})
}
/// Execute a Normal-mode function, resolving pipe stdin first.
pub async fn execute_normal(
engine: &Engine,
func: &Function,
ctx: &mut ExecutionContext,
event_tx: &EventSender,
workspace: &Path,
remaining: &str,
bake_base: &Path,
stdout_cache: &HashMap<String, String>,
) -> Result<String, BakeError> {
ctx.stdin = resolve_pipe_stdin(func, stdout_cache);
execute_one(engine, func, ctx, event_tx, workspace, remaining, bake_base).await
}
// ── Daemon / Health ─────────────────────────────────────────────────────────
/// Execute a `@daemon` function: build script, spawn, periodic health-probe
/// (blocking until healthy), then return.
pub async fn execute_daemon(
engine: &Engine,
func: &Function,
ctx: &ExecutionContext,
event_tx: &EventSender,
workspace: &Path,
remaining: &str,
bake_base: &Path,
health_period: Duration,
health_timeout: Duration,
health_max_retries: u32,
) -> Result<(), BakeError> {
let fallible = is_fallible(func);
let mut modified = func.clone();
modified.body = remaining.to_string() + &func.body;
let script = builder::build_script(
&modified, bake_base, workspace, RenderMode::Full, ctx.debug_flags,
)?;
// Spawn without waiting
let mut command = tokio::process::Command::new(ctx.shell.as_str());
command.arg("-c");
command.arg(script.to_str().unwrap());
command.current_dir(&ctx.working_dir);
command.envs(&ctx.env_vars);
command.stdin(Stdio::null());
command.stdout(Stdio::null());
command.stderr(Stdio::null());
#[cfg(unix)]
command.process_group(0);
//let child = // Necessary?
match command.spawn() {
Ok(c) => c,
Err(e) if fallible => {
log::warn!("Failed to spawn daemon '{}' (fallible): {}", func.name, e);
return Ok(());
}
Err(_) => {
return Err(BakeError::ScriptExecutionError(ExecutionError::ExecutionFailed {
task_id: func.name.clone(),
exit_code: -1,
}));
}
};
// Periodic health probe — block until healthy, retry exhausted, or fatal
if let Some(health_cmd) = func.decorators.iter().find_map(|d| match d {
Decorator::Health(cmd) => Some(cmd.as_str()),
_ => None,
}) {
let mut retries: u32 = 0;
loop {
let probe_result = tokio::time::timeout(
health_timeout,
health_probe_one(engine, health_cmd, remaining, ctx, event_tx),
)
.await;
match probe_result {
Ok(Ok(())) => break, // healthy — done
_ if retries + 1 < health_max_retries => {
retries += 1;
log::warn!(
"Health probe for '{}' failed ({}/{}), retrying in {:?}...",
func.name, retries, health_max_retries, health_period,
);
}
_ if fallible => {
log::warn!("Health probe for '{}' exhausted (fallible), skipping", func.name);
break; // exhausted but fallible → skip health check
}
Ok(Err(e)) => return Err(e),
Err(_) => {
return Err(BakeError::HealthProbeFailed {
command: health_cmd.to_string(),
error: "timeout".to_string(),
});
}
}
tokio::time::sleep(health_period).await;
}
}
// Cleanup is handled by finalize.
Ok(())
}
/// Single health-probe attempt. Exit 0 = healthy.
async fn health_probe_one(
engine: &Engine,
cmd: &str,
remaining: &str,
ctx: &ExecutionContext,
event_tx: &EventSender,
) -> Result<(), BakeError> {
let full_cmd = format!("{}\n{}", remaining, cmd);
let result = engine
.execute_command(&full_cmd, ctx, event_tx)
.await
.map_err(|e| BakeError::HealthProbeFailed {
command: cmd.to_string(),
error: format!("{}", e),
})?;
if !result.success {
return Err(BakeError::HealthProbeFailed {
command: cmd.to_string(),
error: format!("exit code {}", result.exit_code),
});
}
Ok(())
}
+479
View File
@@ -0,0 +1,479 @@
use super::decorator::{Decorator, parse_decorator};
use crate::bake::error::BakeError;
use lazy_static::lazy_static;
use regex::Regex;
use shlex::Shlex;
lazy_static! {
static ref FUNCTION_REGEX: Regex =
Regex::new(r"^\s*(function\s*)([a-zA-Z0-9_-]+)\s*(\(\))?").unwrap();
static ref FUNCTION_REGEX_WITHOUT_KEYWORD: Regex =
Regex::new(r"^\s*([a-zA-Z0-9_-]+)\s*(\(\))").unwrap();
static ref DECORATOR_HEADER_REGEX: Regex = Regex::new(r"\s*#\s+@").unwrap();
}
/// A parsed shell function with decorator annotations and body content.
#[derive(Debug, Clone)]
pub struct Function {
/// Function name (alphanumeric with underscore/hyphen).
pub name: String,
/// Decorators applied to this function.
pub decorators: Vec<Decorator>,
/// Raw function body including `function name() { ... }` syntax.
pub body: String,
}
impl Function {
/// Finds the first decorator matching the given predicate.
pub fn find_decorator<T>(&self, matcher: impl Fn(&Decorator) -> Option<T>) -> Option<T> {
self.decorators.iter().find_map(matcher)
}
}
#[derive(Debug)]
enum ParserState {
Normal,
AfterDecorator,
InFunction {
name: String,
decorators: Vec<Decorator>,
body: String,
brace_depth: i16,
},
}
/// Result of parsing a shell script with decorator annotations.
#[derive(Debug)]
pub struct ParsedScript {
/// Functions marked with `@pipeline` decorator.
pub pipeline_functions: Vec<Function>,
/// Code not belonging to any pipeline function (comments, helpers, etc.).
pub remaining_code: String,
}
/// Parses shell scripts with `# @decorator(args)` annotations into a `Function` list.
/// Lines prefixed with `# @` before a function definition become decorators for that function.
pub fn parse_script(text: &str) -> Result<ParsedScript, BakeError> {
let mut pipeline_functions = Vec::new();
let mut remaining_code = String::new();
let mut state = ParserState::Normal;
let mut pending_decorators: Vec<Decorator> = Vec::new();
for (line_number, line) in text.lines().enumerate() {
let line_number = line_number + 1;
match &mut state {
ParserState::Normal => {
if DECORATOR_HEADER_REGEX.find(line).is_some() {
if let Some(decorator) = parse_decorator(line, line_number)? {
pending_decorators.push(decorator);
}
state = ParserState::AfterDecorator;
continue;
}
if FUNCTION_REGEX.find(line).is_some()
|| FUNCTION_REGEX_WITHOUT_KEYWORD.find(line).is_some()
{
let tokens = Shlex::new(line).collect::<Vec<String>>();
let (name, _has_keyword) = if tokens[0] == "function" {
(FUNCTION_REGEX.captures(line).unwrap()[2].to_string(), true)
} else {
(
FUNCTION_REGEX_WITHOUT_KEYWORD.captures(line).unwrap()[1].to_string(),
false,
)
};
let has_pipeline = pending_decorators
.iter()
.any(|d| matches!(d, Decorator::Pipeline));
let brace_depth = tokens.iter().filter(|&token| token == "{").count() as i16
- tokens.iter().filter(|&token| token == "}").count() as i16;
if brace_depth != 0 {
state = ParserState::InFunction {
name,
decorators: pending_decorators.clone(),
body: line.to_string() + "\n",
brace_depth,
};
} else {
if has_pipeline {
pipeline_functions.push(Function {
name: name.clone(),
decorators: pending_decorators.clone(),
body: line.to_string() + "\n",
});
} else {
for dec in &pending_decorators {
remaining_code.push_str(&format!("# {}\n", dec));
}
remaining_code.push_str(line);
remaining_code.push('\n');
}
pending_decorators.clear();
state = ParserState::Normal;
}
} else {
remaining_code.push_str(line);
remaining_code.push('\n');
}
}
ParserState::AfterDecorator => {
if line.trim().is_empty() {
continue;
}
if DECORATOR_HEADER_REGEX.find(line).is_some() {
if let Some(decorator) = parse_decorator(line, line_number)? {
pending_decorators.push(decorator);
}
continue;
}
if line.trim().starts_with('#') {
continue;
}
if FUNCTION_REGEX.find(line).is_some()
|| FUNCTION_REGEX_WITHOUT_KEYWORD.find(line).is_some()
{
let tokens = Shlex::new(line).collect::<Vec<String>>();
let (name, _has_keyword) = if tokens[0] == "function" {
(FUNCTION_REGEX.captures(line).unwrap()[2].to_string(), true)
} else {
(
FUNCTION_REGEX_WITHOUT_KEYWORD.captures(line).unwrap()[1].to_string(),
false,
)
};
let has_pipeline = pending_decorators
.iter()
.any(|d| matches!(d, Decorator::Pipeline));
let brace_depth = tokens.iter().filter(|&token| token == "{").count() as i16
- tokens.iter().filter(|&token| token == "}").count() as i16;
if brace_depth != 0 {
state = ParserState::InFunction {
name,
decorators: pending_decorators.clone(),
body: line.to_string() + "\n",
brace_depth,
};
} else {
if has_pipeline {
pipeline_functions.push(Function {
name: name.clone(),
decorators: pending_decorators.clone(),
body: line.to_string() + "\n",
});
} else {
for dec in &pending_decorators {
remaining_code.push_str(&format!("# {}\n", dec));
}
remaining_code.push_str(line);
remaining_code.push('\n');
}
pending_decorators.clear();
state = ParserState::Normal;
}
continue;
}
log::warn!(
"Warning: Dangling decorator(s) at line {}. Treating as comments.",
line_number - 1
);
for dec in &pending_decorators {
remaining_code.push_str(&format!("# {}\n", dec));
}
pending_decorators.clear();
state = ParserState::Normal;
remaining_code.push_str(line);
remaining_code.push('\n');
}
ParserState::InFunction {
name,
decorators,
body,
brace_depth,
} => {
let has_pipeline = decorators.iter().any(|d| matches!(d, Decorator::Pipeline));
body.push_str(line);
body.push('\n');
let tokens = Shlex::new(line).collect::<Vec<String>>();
*brace_depth += tokens.iter().filter(|&token| token == "{").count() as i16;
*brace_depth -= tokens.iter().filter(|&token| token == "}").count() as i16;
if *brace_depth == 0 && !body.trim().is_empty() {
if has_pipeline {
pipeline_functions.push(Function {
name: name.clone(),
decorators: decorators.clone(),
body: body.clone(),
});
} else {
for dec in decorators {
remaining_code.push_str(&format!("# {}\n", dec));
}
remaining_code.push_str(body);
}
pending_decorators.clear();
state = ParserState::Normal;
}
}
}
}
if let ParserState::InFunction {
name, brace_depth, ..
} = state
{
return Err(BakeError::ScriptGenerationFailed {
script: "bake.sh".to_string(),
action: "parsing function body".to_string(),
reason: format!(
"Unexpected EOF while parsing function '{}'. Unmatched braces (remaining: {})?",
name, brace_depth
),
});
}
if !pending_decorators.is_empty() {
let last_decorator = pending_decorators.last().unwrap();
return Err(BakeError::ScriptGenerationFailed {
script: "bake.sh".to_string(),
action: "parsing decorators".to_string(),
reason: format!(
"Dangling decorator {} before EOF. Missing function definition?",
last_decorator
),
});
}
Ok(ParsedScript {
pipeline_functions,
remaining_code,
})
}
#[cfg(test)]
mod tests {
use super::*;
#[test]
fn test_empty_script() {
let script = "";
let result = parse_script(&script.to_string()).unwrap();
assert!(result.pipeline_functions.is_empty());
assert!(result.remaining_code.is_empty());
}
#[test]
fn test_only_comments() {
let script = "# comment\n# another";
let result = parse_script(&script.to_string()).unwrap();
assert!(result.pipeline_functions.is_empty());
assert_eq!(result.remaining_code.trim(), "# comment\n# another");
}
#[test]
fn test_function_with_parens_non_pipeline() {
let script = "func() { echo hi; }";
let result = parse_script(&script.to_string()).unwrap();
assert!(result.pipeline_functions.is_empty());
assert!(result.remaining_code.contains("func()"));
}
#[test]
fn test_pipeline_function() {
let script = "# @pipeline\nfunc() { echo hi; }";
let result = parse_script(&script.to_string()).unwrap();
assert_eq!(result.pipeline_functions.len(), 1);
assert_eq!(result.pipeline_functions[0].name, "func");
assert!(result.remaining_code.is_empty());
}
#[test]
fn test_mixed_functions() {
let script = r#"#!/bin/bash
# comment
function helper() { echo "helper"; }
# @pipeline
prepare() { echo "prepare"; }
echo "Done."
"#;
let result = parse_script(&script.to_string()).unwrap();
assert_eq!(result.pipeline_functions.len(), 1);
assert_eq!(result.pipeline_functions[0].name, "prepare");
assert!(result.remaining_code.contains("# comment"));
assert!(result.remaining_code.contains("helper()"));
assert!(result.remaining_code.contains("echo \"Done.\""));
}
#[test]
fn test_only_pipeline_functions_remaining_empty() {
let script = "# @pipeline\nfunc1() { echo 1; }\n# @pipeline\nfunc2() { echo 2; }";
let result = parse_script(&script.to_string()).unwrap();
assert_eq!(result.pipeline_functions.len(), 2);
assert!(result.remaining_code.is_empty());
}
#[test]
fn test_user_example() {
let script = r#"#!/bin/bash
#Implicit set -euo pipefail
# function to be used by any pipeline stage
function get_custom_library() {
curl -LO https://archive.ppy.sh/$2/$1.tar.gz
}
# @pipeline
prepare_dotnet() {
curl -L https://dot.net/v1/dotnet-install.sh | bash
bake_info "Successfully installed dotnet"
}
# @pipeline
fetch_source() {
git clone https://github.com/ppy/osu.git --depth 1
}
echo "Done."
"#;
let result = parse_script(&script.to_string());
match result {
Ok(r) => {
assert_eq!(
r.pipeline_functions.len(),
2,
"Expected 2 pipeline functions, got {}",
r.pipeline_functions.len()
);
assert_eq!(r.pipeline_functions[0].name, "prepare_dotnet");
assert_eq!(r.pipeline_functions[1].name, "fetch_source");
}
Err(e) => {
panic!("Parse error: {:?}", e);
}
}
}
#[test]
fn test_blank_lines_between_decorator_and_function() {
let script = "# @pipeline\n\nfunc() { echo 1; }";
let result = parse_script(&script.to_string()).unwrap();
assert_eq!(result.pipeline_functions.len(), 1);
assert_eq!(result.pipeline_functions[0].name, "func");
assert!(result.remaining_code.is_empty());
}
#[test]
fn test_multiple_blank_lines_between_decorator_and_function() {
let script = "# @pipeline\n\n\nfunc() { echo 1; }";
let result = parse_script(&script.to_string()).unwrap();
assert_eq!(result.pipeline_functions.len(), 1);
assert_eq!(result.pipeline_functions[0].name, "func");
}
#[test]
fn test_comments_between_decorator_and_function() {
let script = "# @pipeline\n# comment\nfunc() { echo 1; }";
let result = parse_script(&script.to_string()).unwrap();
assert_eq!(result.pipeline_functions.len(), 1);
assert_eq!(result.pipeline_functions[0].name, "func");
}
#[test]
fn test_multiple_comments_between_decorator_and_function() {
let script = "# @pipeline\n# comment 1\n# comment 2\nfunc() { echo 1; }";
let result = parse_script(&script.to_string()).unwrap();
assert_eq!(result.pipeline_functions.len(), 1);
assert_eq!(result.pipeline_functions[0].name, "func");
}
#[test]
fn test_mixed_blank_lines_and_comments_between_decorator_and_function() {
let script = "# @pipeline\n# comment\n\n# another comment\n\nfunc() { echo 1; }";
let result = parse_script(&script.to_string()).unwrap();
assert_eq!(result.pipeline_functions.len(), 1);
assert_eq!(result.pipeline_functions[0].name, "func");
}
#[test]
fn test_shell_statement_after_decorator_warns() {
let script = "# @pipeline\necho \"Done.\"";
let result = parse_script(&script.to_string()).unwrap();
assert_eq!(result.pipeline_functions.len(), 0);
assert!(result.remaining_code.contains("# @pipeline"));
assert!(result.remaining_code.contains("echo \"Done.\""));
}
#[test]
fn test_nested_braces_in_string() {
let script = r#"# @pipeline
func() { echo "{"; }"#;
let result = parse_script(&script.to_string()).unwrap();
assert_eq!(result.pipeline_functions.len(), 1);
assert_eq!(result.pipeline_functions[0].name, "func");
}
#[test]
fn test_nested_braces_in_double_string() {
let script = r#"# @pipeline
func() { echo "{}"; }"#;
let result = parse_script(&script.to_string()).unwrap();
assert_eq!(result.pipeline_functions.len(), 1);
assert_eq!(result.pipeline_functions[0].name, "func");
}
#[test]
fn test_nested_braces_in_code() {
let script = r#"# @pipeline
func() {
if [ -z "${VAR}" ]; then
echo "empty"
fi
}"#;
let result = parse_script(&script.to_string()).unwrap();
assert_eq!(result.pipeline_functions.len(), 1);
assert_eq!(result.pipeline_functions[0].name, "func");
}
#[test]
fn test_function_with_curl_braces_url() {
let script = r#"# @pipeline
func() {
curl -LO https://example.com/file.tar.gz
}"#;
let result = parse_script(&script.to_string()).unwrap();
assert_eq!(result.pipeline_functions.len(), 1);
}
#[test]
fn test_function_with_json_like_content() {
let script = r#"# @pipeline
func() { echo '{"key": "value"}'; }"#;
let result = parse_script(&script.to_string()).unwrap();
assert_eq!(result.pipeline_functions.len(), 1);
}
#[test]
fn test_decorator_with_blank_lines_then_shell_statement() {
let script = "# @pipeline\n\n\necho \"Done.\"";
let result = parse_script(&script.to_string()).unwrap();
assert_eq!(result.pipeline_functions.len(), 0);
assert!(result.remaining_code.contains("# @pipeline"));
}
#[test]
fn test_multiple_decorators_then_function() {
let script = "# @pipeline\n# @fallible\n\n\nfunc() { echo 1; }";
let result = parse_script(&script.to_string()).unwrap();
assert_eq!(result.pipeline_functions.len(), 1);
assert_eq!(result.pipeline_functions[0].decorators.len(), 2);
}
#[test]
fn test_multiple_decorators_then_shell_statement() {
let script = "# @pipeline\n# @fallible\necho \"Done.\"";
let result = parse_script(&script.to_string()).unwrap();
assert_eq!(result.pipeline_functions.len(), 0);
assert!(result.remaining_code.contains("# @pipeline"));
assert!(result.remaining_code.contains("# @fallible"));
}
}
+187
View File
@@ -0,0 +1,187 @@
// Code in this module PARTIALLY or FULLY utilized AI Coding Agent
use super::decorator::Decorator;
use super::parser::Function;
use crate::bake::error::BakeError;
use std::collections::HashMap;
use workshop_schedule::{Dag, DagNode, EdgeKind, NodeColor};
/// Thin wrapper so `Function` implements `DagNode`.
#[derive(Clone)]
pub(crate) struct BakeNode {
name: String,
color: NodeColor,
fallible: bool,
/// Original function body + decorators for execution
func: Function,
}
impl DagNode for BakeNode {
fn name(&self) -> &str {
&self.name
}
fn color(&self) -> NodeColor {
self.color.clone()
}
fn fallible(&self) -> bool {
self.fallible
}
}
/// Convert a `Function` into a `BakeNode` by extracting scheduling metadata
/// from decorators.
fn function_to_node(func: &Function) -> BakeNode {
let color = if func
.decorators
.iter()
.any(|d| matches!(d, Decorator::Parallel(_)))
{
let group = func
.decorators
.iter()
.find_map(|d| {
if let Decorator::Parallel(Some(g)) = d {
Some(g.clone())
} else {
None
}
})
.unwrap_or("*".to_string());
if group == "*" {
NodeColor::Wildcard
} else {
NodeColor::Named(group)
}
} else {
NodeColor::Sequential
};
let fallible = func
.decorators
.iter()
.any(|d| matches!(d, Decorator::Fallible));
BakeNode {
name: func.name.clone(),
color,
fallible,
func: func.clone(),
}
}
/// Builds a DAG from functions using `@after` dependencies and implicit
/// ordering for consecutive `@pipeline` functions.
pub fn build_dag(
functions: Vec<Function>,
) -> Result<(Dag<BakeNode>, HashMap<String, Function>), BakeError> {
let function_mapped: HashMap<&str, &Function> = functions
.iter()
.map(|f| (f.name.as_str(), f))
.collect();
let mut dag = Dag::new();
let mut last_pipeline: Option<String> = None;
for function in &functions {
// Determine whether this function is a pipeline node
let is_pipeline = function
.decorators
.iter()
.any(|d| matches!(d, Decorator::Pipeline));
if !is_pipeline {
continue;
}
let has_after = function
.decorators
.iter()
.any(|d| matches!(d, Decorator::After(_)));
let node = function_to_node(function);
dag.add_node(node).map_err(|e| {
BakeError::IncompatibleDecorators {
function: function.name.clone(),
reason: e.to_string(),
}
})?;
// Add implicit @after for consecutive @pipeline functions
if !has_after {
if let Some(ref last) = last_pipeline {
dag.add_edge(last, &function.name, EdgeKind::Implicit)
.map_err(|e| BakeError::IncompatibleDecorators {
function: function.name.clone(),
reason: e.to_string(),
})?;
}
}
last_pipeline = Some(function.name.clone());
// Add explicit @after edges
for decorator in &function.decorators {
if let Decorator::After(dependency) = decorator {
if !function_mapped.contains_key(dependency.as_str()) {
log::error!(
"Unknown dependency of function {}: {}",
function.name,
dependency
);
return Err(BakeError::UnknownDependency(
function.name.clone(),
dependency.clone(),
));
}
dag.add_edge(dependency, &function.name, EdgeKind::Explicit)
.map_err(|e| BakeError::IncompatibleDecorators {
function: function.name.clone(),
reason: e.to_string(),
})?;
}
}
}
dag.freeze();
let owned_map: HashMap<String, Function> = functions
.into_iter()
.map(|f| (f.name.clone(), f))
.collect();
Ok((dag, owned_map))
}
/// Group ready parallel nodes by their declared group.
/// Returns `(star_nodes, named_groups)`.
///
/// Deprecated: use `Dag::ready_wildcards()` and `Dag::ready_groups()` instead.
pub fn partition_by_group<'a>(
ready: &[String],
dag: &Dag<BakeNode>,
) -> (Vec<String>, HashMap<String, Vec<String>>) {
let mut star_nodes = Vec::new();
let mut named_groups: HashMap<String, Vec<String>> = HashMap::new();
for name in ready {
if dag.is_wildcard(name) {
star_nodes.push(name.clone());
} else if let Some(g) = dag.group_of(name) {
named_groups.entry(g).or_default().push(name.clone());
}
// Sequential nodes: not included (caller handles them)
}
(star_nodes, named_groups)
}
/// Pick a complete group to execute.
///
/// Deprecated: use `Dag::pick_batch()` instead.
pub fn pick_batch(
_star_nodes: &[String],
_groups: &HashMap<String, Vec<String>>,
_group_totals: &HashMap<String, usize>,
) -> Option<Vec<String>> {
// This function is replaced by Dag::pick_batch() which has all the
// group metadata internally. Keeping the signature for backward compat
// but redirect callers to use Dag directly.
unimplemented!("use dag.pick_batch() instead")
}
+31
View File
@@ -0,0 +1,31 @@
use crate::bake::builder::{build_script, RenderMode};
use crate::bake::error::BakeError;
use crate::bake::parser::Function;
use std::path::Path;
use workshop_engine::{Engine, EventSender, ExecutionContext};
/// Executes `script_content` as a single function in trivial mode.
///
/// Two entry paths lead here:
/// - **Auto-detection** (`!has_pipeline`): bake.sh has no `@pipeline` functions,
/// the entire content is treated as one trivial function.
/// - **Explicit PIP** (future): `trivial` parameter forces trivial mode even
/// when `@pipeline` functions exist, typically with `RenderMode::Off`.
pub async fn run_trivial(
script_content: &str,
bake_base_path: &Path,
workspace: &Path,
engine: &Engine,
ctx: &mut ExecutionContext,
event_tx: &EventSender,
render_mode: RenderMode,
) -> Result<(), BakeError> {
let function = Function {
name: String::new(),
decorators: vec![],
body: script_content.to_string(),
};
let script = build_script(&function, bake_base_path, workspace, render_mode, ctx.debug_flags)?;
engine.execute_script(&script, ctx, event_tx).await?;
Ok(())
}
+61
View File
@@ -0,0 +1,61 @@
use crate::bake::decorator::Decorator;
use crate::bake::parser::Function;
use std::collections::HashMap;
/// Execution mode for a pipeline function.
#[derive(Debug, Clone, Copy, PartialEq, Eq)]
pub enum FuncMode {
Normal,
Daemon,
Async,
}
/// Determine the execution mode from decorators.
/// `@daemon`, `@async`, `@pipe` are mutually exclusive modes.
pub fn resolve_mode(func: &Function) -> FuncMode {
for d in &func.decorators {
match d {
Decorator::Daemon => return FuncMode::Daemon,
Decorator::Async => return FuncMode::Async,
_ => {}
}
}
FuncMode::Normal
}
/// True if the function is decorated with `@parallel`.
pub fn is_parallel(func: &Function) -> bool {
func.decorators
.iter()
.any(|d| matches!(d, Decorator::Parallel(_)))
}
/// True if the function is decorated with `@fallible`.
pub fn is_fallible(func: &Function) -> bool {
func.decorators
.iter()
.any(|d| matches!(d, Decorator::Fallible))
}
/// Resolve stdin for a `@pipe`-decorated function from the stdout cache.
///
/// Rule 1: self-references are skipped (handled by loop combinator).
/// Rule 3: sources present in the cache are concatenated.
/// Returns `None` if any source is missing (caller decides based on `@fallible`).
pub fn resolve_pipe_stdin(
func: &Function,
cache: &HashMap<String, String>,
) -> Option<Vec<u8>> {
let sources = func.decorators.iter().find_map(|d| match d {
Decorator::Pipe(names) => Some(names.as_slice()),
_ => None,
})?;
let mut combined = String::new();
for source in sources {
if *source == func.name {
continue; // Rule 1: skip self
}
combined.push_str(cache.get(source)?); // Rule 3
}
Some(combined.into_bytes())
}
+44
View File
@@ -0,0 +1,44 @@
use crate::cli::{BareCommands, Cli};
use crate::error::CliError;
use crate::types::resource::ResourceRegistry;
use crate::{bake::bake, finalize::finalize, prebake::prebake};
use workshop_engine::{EventSender, ExecutionContext};
pub async fn run_bare(
cli: &Cli,
command: &BareCommands,
ctx: &mut ExecutionContext,
event_tx: EventSender,
pipeline: &str,
build_id: &str,
registry: &ResourceRegistry,
) -> Result<(), CliError> {
let (stage, config) = match command {
BareCommands::Prebake { .. } => ("prebake", &cli.prebake),
BareCommands::Bake { .. } => ("bake", &cli.bake),
BareCommands::Finalize { .. } => {
("finalize", &cli.finalize)
}
};
log::debug!("Running bare command {} with config {}", stage, config.display());
let prebake_path = &cli.prebake;
let bake_base = &cli.bake_base;
ctx.task_id = format!("{}-{}-{}", pipeline, build_id, stage);
ctx.pipeline_name = pipeline.to_string();
ctx.standalone = true;
match command {
BareCommands::Prebake { .. } => {
prebake(&config, cli, None, ctx, event_tx, None).await?;
}
BareCommands::Bake { .. } => {
bake(&config, &bake_base, &prebake_path, cli, ctx, event_tx, None).await?;
}
BareCommands::Finalize { .. } => {
finalize(&config, cli, ctx, event_tx, registry).await?;
}
}
Ok(())
}
View File
+4
View File
@@ -0,0 +1,4 @@
pub mod baremetal;
pub mod container;
pub mod custom;
pub mod firecracker;
+8
View File
@@ -0,0 +1,8 @@
use crate::prebake::config::PrebakeConfig;
use config::Config;
use log::info;
pub async fn prepare_baremetal(_config: &PrebakeConfig, _settings: &Config) -> anyhow::Result<()> {
info!("Nothing to be done for baremetal environment.");
Ok(())
}
+182
View File
@@ -0,0 +1,182 @@
use crate::prebake::config::PrebakeConfig;
use crate::prebake::types::builderconfig::DockerConfig;
use anyhow::Context;
use bollard::{
API_DEFAULT_VERSION, Docker, body_full,
query_parameters::{BuildImageOptionsBuilder, CreateImageOptionsBuilder},
secret::{BuildInfo, CreateImageInfo},
};
use config::Config;
use futures_util::stream::StreamExt;
use log::{debug, error, info, warn};
use std::fs;
use tar;
use uuid::Uuid;
pub async fn prepare_container(
config: &PrebakeConfig,
settings: &Config,
) -> anyhow::Result<String> {
let container_options = match config.environment.docker.as_ref() {
Some(docker) => docker,
None => {
return Err(anyhow::anyhow!("Missing Docker configuration"));
}
};
info!("Connecting to container engine");
let container_url = settings
.get_string("container.url")
.unwrap_or_else(|_| "__DEFAULT".to_string());
let container_url = container_url.as_str();
let container_connection = if container_url.starts_with("unix://") {
Docker::connect_with_socket(container_url, 20, API_DEFAULT_VERSION)?
} else if container_url.starts_with("http://") {
warn!("It is suggested to run WS-Executor and Docker/Podman on the same machine.");
Docker::connect_with_http(container_url, 20, API_DEFAULT_VERSION)?
} else if container_url.starts_with("ssh://") {
warn!("It is suggested to run WS-Executor and Docker/Podman on the same machine.");
Docker::connect_with_ssh(container_url, 20, API_DEFAULT_VERSION, None)?
} else if container_url.contains("__DEFAULT") {
Docker::connect_with_local_defaults()?
} else {
return Err(anyhow::anyhow!(
"Unsupported container URL scheme: {}",
container_url
));
};
if container_options.image.is_none() {
return container_build(container_options, container_connection, settings).await;
} else {
return container_pull(container_options, container_connection).await;
}
}
fn create_dockerfile_tar(dockerfile: &str) -> anyhow::Result<Vec<u8>> {
let mut tar_buffer = Vec::new();
let mut tar_builder = tar::Builder::new(&mut tar_buffer);
let dockerfile_content = fs::read_to_string(dockerfile)?;
let mut header = tar::Header::new_gnu();
header.set_path("Dockerfile")?;
header.set_size(dockerfile_content.len() as u64);
header.set_mode(0o644);
header.set_entry_type(tar::EntryType::Regular);
header.set_cksum();
tar_builder.append(&header, dockerfile_content.as_bytes())?;
tar_builder.finish()?;
tar_builder.into_inner()?;
Ok(tar_buffer)
}
async fn container_build(
container_options: &DockerConfig,
container_connection: Docker,
settings: &Config,
) -> anyhow::Result<String> {
info!("Building image at Prebake Stage 1...");
let dockerfile = container_options.dockerfile.as_ref().unwrap();
let tag = format!(
"HoneyBiscuitWorkshop/{}_{}",
settings.get_string("pipeline").unwrap().as_str(),
Uuid::new_v4()
);
let tar_buffer = match create_dockerfile_tar(dockerfile) {
Ok(buffer) => buffer,
Err(error) => {
if error.to_string().contains("No such file") {
error!("Couldn't find specified Dockerfile: {}", dockerfile);
} else {
error!("Failed to prepare Dockerfile: {}", error);
}
return Err(error);
}
};
let build_options = BuildImageOptionsBuilder::default()
.dockerfile("Dockerfile")
.t(&tag)
.build();
let mut build_result =
container_connection.build_image(build_options, None, Some(body_full(tar_buffer.into())));
while let Some(stream_item) = build_result.next().await {
let build_info = stream_item.context("Unexpected build result.");
debug!("{:?}", build_info);
match build_info {
Ok(BuildInfo {
id: _,
stream: _,
error_detail: _,
status: _,
progress_detail: _,
aux: _,
}) => {}
Err(error) => {
let error_text = error
.chain()
.map(|err| err.to_string())
.collect::<Vec<_>>()
.join(" ");
error!("Failed to build image: {}", error_text);
return Err(error);
}
}
}
info!("Successfully built the image {}", tag);
Ok(tag)
}
async fn container_pull(
container_options: &DockerConfig,
container_connection: Docker,
) -> anyhow::Result<String> {
let image_name = container_options
.image
.as_ref()
.expect("Unexpected empty image name");
info!("Pulling image: {}", image_name);
let pull_options = CreateImageOptionsBuilder::default()
.from_image(image_name)
.build();
let mut pull_result = container_connection.create_image(Some(pull_options), None, None);
while let Some(stream_item) = pull_result.next().await {
let pull_info = stream_item.context("Unexpected pull result.");
match pull_info {
Ok(CreateImageInfo {
id,
error_detail,
status,
progress_detail: _,
}) => {
if let Some(error_detail) = error_detail {
warn!("Error when pulling: {:?}", error_detail);
}
if let Some(status) = status {
if status.contains("Downloading") || status.contains("Extracting") {
} else if status.contains("Pull complete") {
info!("Layer complete: {}", id.unwrap_or_default());
} else if status.contains("Already exists") {
info!("Layer already exists: {}", id.unwrap_or_default());
} else if status.contains("Image is up to date") {
info!("Up to date image: {}", image_name);
} else if status.contains("Retrying") {
info!("Warning: {}", status);
} else if status.contains("Downloaded newer image") {
info!("Downloaded newer image: {}", image_name);
}
}
}
Err(error) => {
let error_text = error
.chain()
.map(|err| err.to_string())
.collect::<Vec<_>>()
.join(" ");
error!("Failed to pull image: {}", error_text);
return Err(error);
}
}
}
info!("Successfully pulled image: {}", image_name);
Ok(image_name.clone())
}
+6
View File
@@ -0,0 +1,6 @@
use crate::prebake::config::PrebakeConfig;
use config::Config;
pub async fn prepare_custom(_config: &PrebakeConfig, _settings: &Config) -> anyhow::Result<()> {
todo!("Custom builder is not implemented yet")
}
+11
View File
@@ -0,0 +1,11 @@
use crate::prebake::config::PrebakeConfig;
use config::Config;
use log::error;
pub async fn prepare_firecracker(
_config: &PrebakeConfig,
_settings: &Config,
) -> anyhow::Result<()> {
error!("Firecracker is not supported yet.");
Err(anyhow::anyhow!("Firecracker is not supported yet"))
}
+30
View File
@@ -0,0 +1,30 @@
use thiserror::Error;
use workshop_engine::HasExitCode;
#[derive(Error, Debug)]
pub enum CliError {
#[error("{0}")]
Prebake(#[from] crate::prebake::error::PrebakeError),
#[error("{0}")]
Bake(#[from] crate::bake::error::BakeError),
#[error("{0}")]
Finalize(#[from] crate::finalize::FinalizeError),
#[error("{0}")]
General(anyhow::Error),
}
impl From<anyhow::Error> for CliError {
fn from(e: anyhow::Error) -> Self {
CliError::General(e)
}
}
impl CliError {
pub fn exit_code(&self) -> i32 {
match self {
CliError::Prebake(e) => e.exit_code(),
CliError::Bake(e) => e.exit_code(),
CliError::Finalize(e) => e.exit_code(),
CliError::General(_) => 1,
}
}
}
+79
View File
@@ -0,0 +1,79 @@
pub mod config;
pub mod constant;
pub mod error;
pub mod event;
pub mod cleanup;
pub mod artifact;
pub mod plugin;
pub mod stage;
pub mod template;
pub mod types;
use crate::cli::Cli;
use crate::types::resource::ResourceRegistry;
pub use config::*;
pub use error::FinalizeError;
use std::path::Path;
use workshop_engine::EventSender;
pub fn parse(path: &Path) -> Result<FinalizeConfig, FinalizeError> {
let content = std::fs::read_to_string(path).map_err(FinalizeError::IoError)?;
let config: FinalizeConfig = serde_yaml::from_str(&content)?;
Ok(config)
}
pub async fn finalize(
finalize_path: &Path,
_cli: &Cli,
ctx: &mut workshop_engine::ExecutionContext,
event_tx: EventSender,
registry: &ResourceRegistry,
) -> Result<(), FinalizeError> {
let mut finalize = parse(finalize_path)?;
let validate_result = finalize.validate();
if let Err(errors) = validate_result {
for error in &errors {
log::error!("Error: {}", error);
}
return Err(FinalizeError::ValidateError(errors));
};
// Register Plugins (paths resolved from registry)
log::info!("Registering {} plugins", finalize.plugin.len());
let registered_plugins = plugin::register(finalize.plugin, None, &mut finalize.notification, registry)?;
// Early hook
let earlyhook_result = stage::hook::hook(
finalize.hooks.as_ref().and_then(|h| h.early.as_ref()),
ctx,
event_tx.clone(),
&registered_plugins,
)
.await;
// Determine if we should continue with artifacts/latehook
// If prior stages failed, we still send notification but skip artifacts
let _prior_failed = earlyhook_result.is_err();
// Artifact collection and publishing
if !_prior_failed {
log::info!("Processing {} artifacts", finalize.artifact.len());
artifact::collect_and_publish(
&finalize.artifact,
&registered_plugins,
ctx,
event_tx.clone(),
)
.await?;
}
// Late hook
stage::hook::hook(
finalize.hooks.as_ref().and_then(|h| h.late.as_ref()),
ctx,
event_tx.clone(),
&registered_plugins,
)
.await?;
Ok(())
}
+139
View File
@@ -0,0 +1,139 @@
//! Artifact collection and publishing.
//!
//! Collects build artifacts (via glob patterns), optionally compresses them,
//! and dispatches each artifact through its configured publish plugins.
//! The publish plugin receives the artifact path as a system-injected
//! `artifact` field in its argument.
use std::path::{Path, PathBuf};
use crate::finalize::config::ArtifactDef;
use crate::finalize::error::FinalizeError;
use crate::finalize::plugin::PluginMap;
use crate::finalize::types::compression::CompressionMethod;
use workshop_engine::{EventSender, ExecutionContext};
/// Collect artifacts and execute their publish plugins.
///
/// For each artifact:
/// 1. Resolve source files via glob
/// 2. Optionally compress (zstd, gzip, dir)
/// 3. Inject `artifact` key into each publish plugin's config
/// 4. Call the plugin
pub async fn collect_and_publish(
artifacts: &[ArtifactDef],
plugins: &PluginMap,
ctx: &ExecutionContext,
event_tx: EventSender,
) -> Result<(), FinalizeError> {
if artifacts.is_empty() {
return Ok(());
}
for artifact in artifacts {
let artifact_path = process_artifact(artifact).await?;
if let Some(ref path) = artifact_path {
log::info!("Artifact processed: {} -> {}", artifact.path, path.display());
}
for (plugin_name, publish_config) in &artifact.publish {
let plugin = plugins.get(plugin_name).ok_or_else(|| {
FinalizeError::PluginExecutionFailed(format!(
"publish plugin '{}' not found for artifact '{}'",
plugin_name, artifact.path,
))
})?;
// Inject artifact path into the plugin argument
let argument = if let Some(ref path) = artifact_path {
let mut arg = publish_config.clone();
if let Some(map) = arg.as_mapping_mut() {
map.insert(
serde_yaml::Value::String("artifact".into()),
serde_yaml::Value::String(path.to_string_lossy().into()),
);
}
arg
} else {
publish_config.clone()
};
log::info!(
"Publishing artifact '{}' via plugin '{}'",
artifact.path, plugin_name,
);
plugin.call(argument, ctx, &event_tx).await?;
}
}
Ok(())
}
/// Resolve and optionally compress an artifact.
/// Returns the local path to the processed artifact, or `None` if non-file artifact.
async fn process_artifact(artifact: &ArtifactDef) -> Result<Option<PathBuf>, FinalizeError> {
if artifact.path.is_empty() {
return Ok(None); // non-filesystem artifact (e.g. Docker image)
}
let src = PathBuf::from(&artifact.path);
if !src.exists() {
if artifact.path.contains('*') || artifact.path.contains('?') {
return Err(FinalizeError::PluginExecutionFailed(
format!("glob patterns not supported: '{}'; use a directory path", artifact.path),
));
}
return Err(FinalizeError::ArtifactNotFound(src));
}
match artifact.compression {
CompressionMethod::None => Ok(Some(src)),
CompressionMethod::Dir => Ok(Some(src)),
CompressionMethod::Gzip => {
let dst = std::env::temp_dir().join(format!("artifact-{}.tar.gz", artifact.id.as_deref().unwrap_or("unknown")));
compress_tar_zstd_or_gzip(&src, &dst, false).await?;
Ok(Some(dst))
}
CompressionMethod::Zstd => {
let dst = std::env::temp_dir().join(format!("artifact-{}.tar.zst", artifact.id.as_deref().unwrap_or("unknown")));
compress_tar_zstd_or_gzip(&src, &dst, true).await?;
Ok(Some(dst))
}
}
}
async fn compress_tar_zstd_or_gzip(
src: &Path,
dst: &Path,
zstd: bool,
) -> Result<(), FinalizeError> {
let _archive = tokio::task::spawn_blocking({
let src = src.to_path_buf();
let dst = dst.to_path_buf();
move || -> Result<(), String> {
let file = std::fs::File::create(&dst)
.map_err(|e| format!("cannot create {}: {}", dst.display(), e))?;
let writer: Box<dyn std::io::Write> = if zstd {
Box::new(zstd::stream::write::Encoder::new(file, 0).map_err(|e| e.to_string())?)
} else {
Box::new(flate2::write::GzEncoder::new(file, flate2::Compression::default()))
};
let mut tar = tar::Builder::new(writer);
if src.is_dir() {
tar.append_dir_all(".", &src)
.map_err(|e| format!("tar error: {}", e))?;
} else {
tar.append_path(&src)
.map_err(|e| format!("tar error: {}", e))?;
}
tar.into_inner().map_err(|e| e.to_string())?;
Ok(())
}
})
.await
.map_err(|e| FinalizeError::PluginExecutionFailed(format!("spawn_blocking: {}", e)))?
.map_err(FinalizeError::PluginExecutionFailed)?;
Ok(())
}
+67
View File
@@ -0,0 +1,67 @@
//! Pipeline cleanup and resource reclamation.
//!
//! Runs after all pipeline stages and notification dispatch complete.
//! Checks the configured [`CleanupPolicy`] and performs the following:
//!
//! 1. Temp file / cache directory removal
//! 2. Policy-based skip (never, manual, on_success)
use crate::finalize::config::{CleanupConfig, CleanupPolicy};
use crate::finalize::error::FinalizeError;
use workshop_engine::ExecutionContext;
/// Execute cleanup according to the pipeline's [`CleanupPolicy`].
///
/// Returns `Ok(())` when cleanup completes (or is skipped per policy).
/// Returns `Err` only if a cleanup action itself fails (not if the
/// pipeline failed; pipeline outcome is checked via `pipeline_ok`).
pub fn cleanup(
config: &CleanupConfig,
_ctx: &ExecutionContext,
pipeline_ok: bool,
) -> Result<(), FinalizeError> {
match config.policy {
CleanupPolicy::Never => {
log::info!("[cleanup] policy=never, skipping all cleanup");
return Ok(());
}
CleanupPolicy::Manual => {
log::info!("[cleanup] policy=manual, hanging for bakerd signal");
// TODO(daemon): hang on bakerd signal
// For now (standalone/bare) manual == Never
return Ok(());
}
CleanupPolicy::OnSuccess if !pipeline_ok => {
log::info!("[cleanup] policy=on_success, but pipeline failed — skipping");
// TODO(daemon): hang for bakerd
return Ok(());
}
_ => {}
}
log::info!("[cleanup] starting");
cleanup_temp_files();
subprocess_reclaim();
log::info!("[cleanup] complete");
Ok(())
}
/// Reclaim child processes (placeholder for daemon-mode PID tracking).
/// In standalone mode, children are reaped by the kernel on process exit.
fn subprocess_reclaim() {
// TODO(daemon): track spawned PIDs during pipeline and signal them here.
}
/// Remove plugin cache and temporary directories.
fn cleanup_temp_files() {
let cache = std::path::Path::new("/tmp/baker");
if cache.exists() {
match std::fs::remove_dir_all(cache) {
Ok(()) => log::debug!("[cleanup] removed cache: {}", cache.display()),
Err(e) => log::warn!("[cleanup] failed to remove {}: {}", cache.display(), e),
}
}
let workshop_templates = std::env::temp_dir().join("workshop-templates");
if workshop_templates.exists() {
let _ = std::fs::remove_dir_all(&workshop_templates);
}
}
+462
View File
@@ -0,0 +1,462 @@
//! Finalize stage configuration types
//!
//! This module defines the configuration schema for the finalize stage,
//! which handles artifact collection, publishing, and notifications.
use crate::finalize::types::compression::CompressionMethod;
use semver::{Version, VersionReq};
use serde::{Deserialize, Serialize};
use serde_yaml::Value;
use std::collections::HashMap;
use std::path::PathBuf;
use workshop_engine::{CustomCommand, deserialize_duration_ms};
/// Version requirement for finalize.yml schema compatibility
pub const VERSION_REQUIREMENT: &str = "^0.0.1";
pub type RawPluginMap = HashMap<String, HashMap<String, PluginConfig>>;
/// Root configuration for finalize stage
#[derive(Debug, Deserialize, Serialize, Clone)]
pub struct FinalizeConfig {
/// Schema version for compatibility checking
pub version: String,
/// Plugin definitions
/// Map of logical plugin name -> plugin source URI -> plugin config
#[serde(default)]
pub plugin: RawPluginMap,
/// Notification configuration (templates + priority groups)
#[serde(default)]
pub notification: NotificationConfig,
/// Artifact definitions
#[serde(default)]
pub artifact: Vec<ArtifactDef>,
/// Cleanup configuration
#[serde(default)]
pub cleanup: CleanupConfig,
/// Hooks configuration
#[serde(default)]
pub hooks: Option<FinalizeHooks>,
}
/// Plugin configuration
#[derive(Debug, Deserialize, Serialize, Clone)]
pub struct PluginConfig {
/// Whether plugin failure should not fail the build.
/// Same as @fallible decorator
#[serde(default)]
pub fallible: bool,
/// Number of retries on failure (0 = no retry, -1 = infinite retries)
#[serde(default)]
pub retry: i8,
/// Timeout for plugin execution in milliseconds (0 = no timeout)
#[serde(default)]
pub timeout_ms: u64,
/// Whether the plugin should be resolved after bake stage
#[serde(default)]
pub late_binding: bool,
/// (Git) Enable shallow clone
#[serde(default)]
pub shallow: Option<bool>,
/// (Http) Checksum of the plugin package
#[serde(default)]
pub checksum: Option<String>,
/// Baker runtime configuration
#[serde(skip)]
pub runtime: PluginRuntimeConfig,
/// Plugin-specific configuration parameters.
/// These are passed to the plugin via according method.
#[serde(flatten)]
pub config: Value,
}
#[derive(Debug, Clone, Default)]
pub struct PluginRuntimeConfig {
/// Error description if plugin is unavailable at runtime.
/// None means available
pub error: Option<String>,
/// Path of plugin directory in the environment
pub fspath: Option<PathBuf>,
/// manifest.yml
pub manifest: Option<Value>,
}
/// Notification method configuration
#[derive(Debug, Deserialize, Serialize, Clone)]
pub struct NotificationMethod {
/// Template definition
#[serde(default)]
pub template: NotificationTemplate,
/// Trigger definition
#[serde(default)]
pub trigger: Vec<TriggerItemRaw>,
/// Notification-specific configuration (url, token, to, from, etc.)
#[serde(flatten)]
pub config: Value,
/// Policy list for flexible trigger/template/config combinations.
#[serde(default)]
pub policy: Vec<NotifyPolicy>,
}
impl Default for NotificationMethod {
fn default() -> Self {
Self {
template: NotificationTemplate::default(),
trigger: Vec::new(),
config: Value::default(),
policy: Vec::new(),
}
}
}
#[derive(Debug, Deserialize, Serialize, Clone, Default)]
pub struct NotificationTemplate {
/// Payload schema: "default" (plugin-defined) or "custom" (requires on_* templates)
#[serde(default)]
pub schema: Option<String>,
/// Content template for build success notification
#[serde(default)]
pub on_success: Option<String>,
/// Content template for build failure notification
#[serde(default)]
pub on_failure: Option<String>,
/// Stage-triggered notification configuration
#[serde(default)]
pub on_finish_of: Option<OnFinishOf>,
}
impl std::ops::AddAssign for NotificationTemplate {
fn add_assign(&mut self, other: Self) {
if self.schema.is_none() {
self.schema = other.schema;
}
if self.on_success.is_none() {
self.on_success = other.on_success;
}
if self.on_failure.is_none() {
self.on_failure = other.on_failure;
}
if self.on_finish_of.is_none() {
self.on_finish_of = other.on_finish_of;
}
}
}
/// Notification configuration container
#[derive(Debug, Serialize, Clone, Default)]
pub struct NotificationConfig {
/// Template definitions for notification content
#[serde(default)]
pub templates: HashMap<String, NotificationTemplateDef>,
/// Priority groups (YAML keys like `0`, `1` are parsed as u32).
/// Custom Deserialize handles serde_yaml string→u32 conversion.
#[serde(default)]
pub groups: HashMap<u32, HashMap<String, NotificationMethod>>,
}
/// Serde helper: YAML map keys are always strings, convert to u32.
impl<'de> serde::Deserialize<'de> for NotificationConfig {
fn deserialize<D>(deserializer: D) -> Result<Self, D::Error>
where D: serde::Deserializer<'de>
{
#[derive(serde::Deserialize)]
struct Helper {
#[serde(default)]
templates: HashMap<String, NotificationTemplateDef>,
#[serde(flatten)]
groups_raw: HashMap<String, Option<HashMap<String, NotificationMethod>>>,
}
let h = Helper::deserialize(deserializer)?;
let groups: HashMap<u32, _> = h.groups_raw.into_iter()
.filter_map(|(k, v)| {
let parsed = k.parse::<u32>().map(|n| (n, v.unwrap_or_default()));
match parsed {
Ok(pair) => Some(pair),
Err(e) => {
log::warn!("priority key must be integer, got {:?}: {}", k, e);
None
}
}
})
.collect();
Ok(NotificationConfig { templates: h.templates, groups })
}
}
/// Stage-triggered notification configuration
#[derive(Debug, Deserialize, Serialize, Clone)]
pub struct OnFinishOf {
/// Stage name to trigger on
/// Format: "prebake.ready", "bake.build", "finalize.artifact", etc.
pub stage: String,
/// Content template for stage completion notification
pub content: String,
}
/// Notification template definition.
/// Supports inline content (on_success/on_failure/on_finish_of) or external reference (use).
#[derive(Debug, Deserialize, Serialize, Clone, Default)]
pub struct NotificationTemplateDef {
/// External template reference (git URL).
/// Cannot be combined with on_* fields in MVP (validation will reject).
#[serde(default, rename = "use")]
pub use_: Option<String>,
/// Content template for build success notification
#[serde(default)]
pub on_success: Option<String>,
/// Content template for build failure notification
#[serde(default)]
pub on_failure: Option<String>,
/// Content template for stage completion notification
#[serde(default)]
pub on_finish_of: Option<String>,
}
impl NotificationTemplateDef {
/// Merge `other` into `self`, keeping self's values when present.
///
/// `other` fills in gaps where `self` is `None`.
/// Chaining: `a.apply(b).apply(c)` = a wins, then b fills gaps, then c fills gaps.
pub fn apply(mut self, other: Self) -> Self {
if self.on_success.is_none() {
self.on_success = other.on_success;
}
if self.on_failure.is_none() {
self.on_failure = other.on_failure;
}
if self.on_finish_of.is_none() {
self.on_finish_of = other.on_finish_of;
}
self
}
}
/// Builder for [`NotificationTemplateDef`].
pub struct NotificationTemplateDefBuilder(NotificationTemplateDef);
impl NotificationTemplateDefBuilder {
pub fn new() -> Self {
Self(NotificationTemplateDef::default())
}
pub fn with_use(mut self, val: impl Into<String>) -> Self {
self.0.use_ = Some(val.into());
self
}
pub fn with_on_success(mut self, val: impl Into<String>) -> Self {
self.0.on_success = Some(val.into());
self
}
pub fn with_on_failure(mut self, val: impl Into<String>) -> Self {
self.0.on_failure = Some(val.into());
self
}
pub fn with_on_finish_of(mut self, val: impl Into<String>) -> Self {
self.0.on_finish_of = Some(val.into());
self
}
pub fn build(self) -> NotificationTemplateDef {
self.0
}
}
/// Notification trigger type.
#[derive(Debug, Clone, PartialEq, Eq)]
pub enum Trigger {
OnSuccess,
OnFailure,
OnFinishOf(String),
}
/// Raw trigger item for deserialization.
/// Supports `- on_success` (string) and `- on_finish_of: [...]` (map).
#[derive(Debug, Deserialize, Serialize, Clone)]
#[serde(untagged)]
pub enum TriggerItemRaw {
Simple(String),
Map(std::collections::HashMap<String, Vec<String>>),
}
/// Notification policy: a set of triggers with optional template/config overrides.
#[derive(Debug, Deserialize, Serialize, Clone)]
pub struct NotifyPolicy {
/// Trigger conditions for this policy.
/// Empty means inheriting method-level triggers.
#[serde(default)]
pub trigger: Vec<TriggerItemRaw>,
/// Template configuration (schema, on_success, on_failure, on_finish_of, subject, body...).
/// Empty/null means inheriting method-level templates.
#[serde(default)]
pub template: Value,
/// Override fields: to, url, token, fallible, retry, timeout_ms, etc.
#[serde(flatten)]
pub overrides: Value,
}
/// Artifact definition
#[derive(Debug, Deserialize, Serialize, Clone)]
pub struct ArtifactDef {
/// Artifact ID (optional, auto-generated as MD5(PUID + PATH) if not specified)
#[serde(default)]
pub id: Option<String>,
/// Artifact path (supports glob patterns, empty for non-filesystem artifacts like Docker images)
#[serde(default)]
pub path: String,
/// Retention duration in milliseconds
/// Formats: "7d", "168h", "604800" (seconds), 604800 (number=seconds)
/// 0 = permanent retention
#[serde(
default = "default_retention_ms",
deserialize_with = "deserialize_duration_ms"
)]
pub retention_ms: u64,
/// Compression method: "zstd", "gzip", "none", "dir"
#[serde(default)]
pub compression: CompressionMethod,
/// Publish targets: plugin name → configuration (artifact path injected by system)
#[serde(default)]
pub publish: HashMap<String, serde_yaml::Value>
}
impl Default for ArtifactDef {
fn default() -> Self {
Self {
id: None,
path: String::new(),
retention_ms: default_retention_ms(),
compression: CompressionMethod::default(),
publish: HashMap::new(),
}
}
}
/// Artifact
/// TODO: Under heavy refactor
/// Cleanup configuration
#[derive(Debug, Deserialize, Serialize, Clone, Default)]
pub struct CleanupConfig {
/// Cleanup policy: "auto", "manual", "on_failure" (WIP: manual, on_failure)
#[serde(default)]
pub policy: CleanupPolicy,
}
/// Cleanup policy
#[derive(Debug, Deserialize, Serialize, Clone, Default)]
#[serde(rename_all = "lowercase")]
pub enum CleanupPolicy {
#[default]
Auto,
Manual,
OnSuccess,
Never,
}
/// Hooks configuration for finalize stage
#[derive(Debug, Deserialize, Serialize, Clone, Default)]
pub struct FinalizeHooks {
/// Hooks executed before artifact collection
#[serde(default)]
pub early: Option<Vec<CustomCommand>>,
/// Hooks executed after all operations complete
#[serde(default)]
pub late: Option<Vec<CustomCommand>>,
}
// Default value functions
fn default_retention_ms() -> u64 {
7 * 24 * 3600 * 1000
}
impl FinalizeConfig {
/// Validate the finalize configuration
pub fn validate(&self) -> Result<(), Vec<String>> {
let mut errors = Vec::new();
// Validate version
let version_requirement =
VersionReq::parse(VERSION_REQUIREMENT).expect("Invalid version requirement");
let version = &self.version;
let version = match version.matches('.').count() {
0 => format!("{}.0.0", version),
1 => format!("{}.0", version),
_ => version.clone(),
};
match Version::parse(&version) {
Ok(version) if version_requirement.matches(&version) => {}
Ok(version) => errors.push(format!(
"Version {} does not satisfy requirement {}",
version, VERSION_REQUIREMENT
)),
Err(e) => errors.push(format!("Invalid version format: {}", e)),
}
// Validate plugin definitions: each plugin must have exactly one URL
for (plugin_name, urls) in &self.plugin {
match urls.len() {
0 => errors.push(format!("Plugin '{}' has no URL defined", plugin_name)),
1 => {}
n => errors.push(format!(
"Plugin '{}' has {} URL definitions, expected exactly 1",
plugin_name, n
)),
}
}
self.validate_notification(&mut errors);
if errors.is_empty() {
Ok(())
} else {
Err(errors)
}
}
fn validate_notification(&self, errors: &mut Vec<String>) {
// Validate templates
for (name, tmpl) in &self.notification.templates {
if let Some(ref u) = tmpl.use_ {
if u.contains("://") && (
tmpl.on_success.is_some() ||
tmpl.on_failure.is_some() ||
tmpl.on_finish_of.is_some()
) {
errors.push(format!(
"Template '{}' cannot combine external 'use' with 'on_*' fields",
name
));
}
// Local references (no ://) are allowed to have on_* overrides
}
}
}
}
+6
View File
@@ -0,0 +1,6 @@
pub const FINALIZE_PLUGIN_MANIFEST: &str = "manifest";
pub const FINALIZE_PLUGIN_MANIFEST_EXTENSION: &[&str] = &[".yml", ".yaml"];
pub const FINALIZE_SHELL_ENVPREFIX: &str = "HBW_ARG";
/// Standalone mode: directory where fetched plugins are stored
pub const FINALIZE_STANDALONE_PLUGIN_PATH: &str = "/tmp/baker/plugin";
+81
View File
@@ -0,0 +1,81 @@
use std::path::PathBuf;
use thiserror::Error;
use workshop_engine::{
ExecutionError,
error::{
EXITCODE_EXECUTION_ERROR, EXITCODE_GENERAL_ERROR, EXITCODE_IO_ERROR, EXITCODE_PARSE_ERROR,
HasExitCode,
},
};
#[derive(Error, Debug)]
pub enum FinalizeError {
#[error("Failed to parse finalize.yml: {0}")]
YamlParseError(#[from] serde_yaml::Error),
#[error("Failed to validate finalize.yml: {0:?}")]
ValidateError(Vec<String>),
#[error("IO error: {0}")]
IoError(#[from] std::io::Error),
#[error("Artifact not found: {0}")]
ArtifactNotFound(PathBuf),
#[error("Plugin error: {0}")]
PluginError(#[from] PluginError),
#[error("Plugin execution failed: {0}")]
PluginExecutionFailed(String),
#[error("Notification failed: {0}")]
NotificationFailed(String),
#[error("Template error: {0}")]
TemplateError(#[from] minijinja::Error),
#[error("Execution error: {0}")]
ExecutionError(#[from] ExecutionError),
}
impl HasExitCode for FinalizeError {
fn exit_code(&self) -> i32 {
match self {
FinalizeError::YamlParseError(_) => EXITCODE_PARSE_ERROR,
FinalizeError::ValidateError(_) => EXITCODE_PARSE_ERROR,
FinalizeError::IoError(_) => EXITCODE_IO_ERROR,
FinalizeError::ArtifactNotFound(_) => EXITCODE_GENERAL_ERROR,
FinalizeError::PluginError(_) => EXITCODE_GENERAL_ERROR,
FinalizeError::PluginExecutionFailed(_) => EXITCODE_EXECUTION_ERROR,
FinalizeError::NotificationFailed(_) => EXITCODE_GENERAL_ERROR,
FinalizeError::TemplateError(_) => EXITCODE_PARSE_ERROR,
FinalizeError::ExecutionError(_) => EXITCODE_EXECUTION_ERROR,
}
}
}
#[derive(Error, Debug)]
pub enum PluginError {
#[error("Unknown plugin: {0}")]
UnknownPlugin(String),
#[error("Error: {0}")]
GeneralError(String),
#[error("Plugin {name} is not available: {reason}")]
PluginUnavailable { name: String, reason: String },
#[error("Plugin execution failed: {0}")]
ExecutionError(#[from] ExecutionError),
#[error("Failed to fetch plugin({name}:{url}): {reason}")]
FetchError {
name: String,
url: String,
reason: String,
},
#[error("Plugin manifest {0}/manifest.yml not found.")]
ManifestNotFound(PathBuf),
}
+73
View File
@@ -0,0 +1,73 @@
use workshop_engine::{ExecutionEvent, StreamType};
use crate::types::debug::DebugFeature;
/// Receives and processes execution events for the finalize stage.
/// Logs task lifecycle events (started, completed, failed) and streaming output (stdout/stderr)
/// for the pipeline build identified by the given pipeline_name and build_id.
pub async fn event_receiver(
mut rx: tokio::sync::mpsc::Receiver<ExecutionEvent>,
pipeline_name: String,
build_id: String,
debug_flags: u32,
) {
let events_debug = DebugFeature::from_bits_retain(debug_flags).contains(DebugFeature::Events);
while let Some(event) = rx.recv().await {
match event {
ExecutionEvent::TaskStarted { task_id, timestamp } => {
if events_debug {
log::info!(
"[{}] [{}] [{}] Task started: {}",
pipeline_name,
build_id,
timestamp,
task_id
);
} else {
log::debug!(
"Task started: {}",
task_id
);
}
}
ExecutionEvent::OutputChunk {
task_id: _,
stream,
data,
} => match stream {
StreamType::Stdout => print!("{}", data),
StreamType::Stderr => {
eprint!("{}", data)
}
},
ExecutionEvent::TaskCompleted { task_id, result } => {
if result.success {
if events_debug {
log::info!(
"[{}] [{}] Task completed: {} (exit_code={}, duration={:?})",
pipeline_name, build_id, task_id,
result.exit_code, result.duration,
);
} else {
log::debug!(
"Task completed: {} (exit_code={}, duration={:?})",
task_id, result.exit_code, result.duration,
);
}
} else {
log::warn!(
"Task completed with non-zero exit: {} (exit_code={}, duration={:?})",
task_id,
result.exit_code, result.duration,
);
}
}
ExecutionEvent::TaskFailed { task_id, error } => {
log::error!(
"Task failed: {} (error={})",
task_id,
error
);
}
}
}
}
+322
View File
@@ -0,0 +1,322 @@
use crate::finalize::RawPluginMap;
use crate::finalize::constant::{FINALIZE_PLUGIN_MANIFEST, FINALIZE_PLUGIN_MANIFEST_EXTENSION};
use crate::finalize::plugin::metadata::{PluginMetadata, PluginType};
use workshop_engine::{EventSender, ExecutionContext};
use crate::{ finalize::error::PluginError};
use crate::types::resource::ResourceRegistry;
use async_trait::async_trait;
use std::collections::{HashMap, HashSet};
use std::path::{Path, PathBuf};
use std::sync::Arc;
mod dylib;
pub mod internal;
mod metadata;
mod rhai;
mod shell;
#[derive(Clone)]
pub struct FinalizePlugin {
pub metadata: PluginMetadata,
argument: serde_yaml::Value,
pub entry: Arc<dyn AsyncPluginFn>,
}
impl FinalizePlugin {
pub fn new(entry: Box<dyn AsyncPluginFn>) -> Self {
Self {
metadata: PluginMetadata::default(),
argument: serde_yaml::Value::Mapping(Default::default()),
entry: Arc::from(entry),
}
}
pub fn with_metadata(mut self, metadata: PluginMetadata) -> Self {
self.metadata = metadata;
self
}
pub fn with_argument(mut self, argument: serde_yaml::Value) -> Self {
self.argument = argument;
self
}
pub async fn call(
&self,
runtime_argument: serde_yaml::Value,
ctx: &ExecutionContext,
event_tx: &EventSender,
) -> PluginResult {
// shallow merge argument
let _argument = match (&self.argument, runtime_argument) {
(serde_yaml::Value::Mapping(base), serde_yaml::Value::Mapping(runtime)) => {
let mut merged = base.clone();
for (k, v) in runtime {
merged.insert(k.clone(), v.clone());
}
serde_yaml::Value::Mapping(merged)
}
_ => self.argument.clone(), // If not both are mappings, just use the original argument
};
self.entry
.call(&self.metadata, _argument, ctx, event_tx)
.await
}
}
pub type PluginMap = HashMap<String, FinalizePlugin>;
type PluginResult = Result<(), PluginError>;
#[async_trait]
pub trait AsyncPluginFn: Send + Sync {
async fn call(
&self,
metadata: &PluginMetadata,
argument: serde_yaml::Value,
ctx: &ExecutionContext,
event_tx: &EventSender,
) -> PluginResult;
}
/// Reads and parses a plugin's manifest file.
///
/// # Arguments
///
/// * `path` - Optional filesystem path to the plugin directory
/// * `name` - Name of the plugin (used for error reporting)
///
/// # Returns
///
/// Returns parsed [`PluginMetadata`] on success, or a [`PluginError`] if the manifest
/// cannot be read or parsed.
fn read_manifest(path: Option<PathBuf>, name: &str) -> Result<PluginMetadata, PluginError> {
let path = path.ok_or_else(|| PluginError::PluginUnavailable {
name: name.to_string(),
reason: "Plugin not present in filesystem.".to_string(),
})?;
// Extract content from manifest.yml or manifest.yaml
let manifest_content = 'load: {
for ext in FINALIZE_PLUGIN_MANIFEST_EXTENSION {
match load_manifest(&path, name, ext) {
Ok(content) => break 'load content,
Err(PluginError::ManifestNotFound(_)) => {}
Err(e) => return Err(e),
}
}
return Err(PluginError::ManifestNotFound(path));
};
log::debug!("[plugin:{}] Parsing manifest content", name);
let mut metadata: PluginMetadata =
serde_yaml::from_str(manifest_content.as_str()).map_err(|e| {
PluginError::PluginUnavailable {
name: name.to_string(),
reason: format!("Failed to parse plugin manifest: {}", e),
}
})?;
metadata.fspath = path;
Ok(metadata)
}
fn load_manifest(path: &Path, name: &str, suffix: &str) -> Result<String, PluginError> {
let manifest_path = path.join(format!("{}{}", FINALIZE_PLUGIN_MANIFEST, suffix));
log::debug!(
"[plugin:{}] Reading manifest from {}",
name,
manifest_path.display()
);
if !manifest_path.exists() {
return Err(PluginError::ManifestNotFound(manifest_path));
}
std::fs::read_to_string(&manifest_path).map_err(|e| PluginError::PluginUnavailable {
name: name.to_string(),
reason: format!(
"Failed to read plugin manifest from {}: {}",
manifest_path.display(),
e
),
})
}
/// Determines the plugin type based on the manifest type or file extension.
///
/// The function first returns `original` if it's not [`PluginType::Unknown`].
/// Otherwise, it infers the type from the entrypoint file extension:
/// - `.sh` / `.bash` → [`PluginType::Shell`]
/// - `.rhai` → [`PluginType::Rhai`]
/// - `.so` / `.dll` / `.dylib` → [`PluginType::Dylib`]
///
/// # Arguments
///
/// * `original` - Plugin type as specified in the manifest
/// * `entrypoint` - Path or filename of the plugin entrypoint
///
/// # Returns
///
/// The inferred [`PluginType`].
fn get_plugin_type(original: PluginType, entrypoint: &str) -> PluginType {
if original != PluginType::Unknown {
return original;
}
let _suffix = entrypoint.rsplit('.').next().unwrap_or("");
match _suffix {
"sh" | "bash" => PluginType::Shell,
"rhai" => PluginType::Rhai,
"so" | "dll" | "dylib" => PluginType::Dylib,
_ => PluginType::Unknown,
}
}
/// Creates a plugin entry function instance based on the plugin type.
///
/// # Arguments
///
/// * `plugin_type` - The type of plugin to instantiate
/// * `name` - Name of the plugin (used for error reporting)
///
/// # Returns
///
/// Returns a boxed [`AsyncPluginFn`] on success, or a [`PluginError`] if the plugin type
/// is [`PluginType::Unknown`] and cannot be inferred.
fn get_entry(plugin_type: PluginType, name: &str) -> Result<Box<dyn AsyncPluginFn>, PluginError> {
match plugin_type {
PluginType::Shell => Ok(Box::new(shell::Shell)),
PluginType::Rhai => Ok(Box::new(rhai::Rhai)),
PluginType::Dylib => Ok(Box::new(dylib::Dylib)),
PluginType::Unknown => {
log::error!(
"Plugin {} has unknown plugin type and cannot be inferred from entrypoint.",
name
);
Err(PluginError::PluginUnavailable {
name: name.to_string(),
reason: "Unknown plugin type and cannot be inferred from entrypoint".to_string(),
})
}
}
}
/// Registers all available plugins (internal and external) into a plugin map.
///
/// Internal plugins are registered first, followed by external plugins loaded from
/// the provided [`RawPluginMap`]. Plugins that are unavailable (e.g., missing runtime)
/// are logged and skipped.
///
/// # Arguments
///
/// * `external_plugins` - Map of external plugin configurations
///
/// # Returns
///
/// Returns a [`PluginMap`] containing all registered plugins, or a [`PluginError`] if
/// a plugin manifest cannot be read or parsed.
pub fn register(
external_plugins: RawPluginMap,
list: Option<&HashSet<String>>,
notification_config: &mut crate::finalize::NotificationConfig,
registry: &ResourceRegistry,
) -> Result<PluginMap, PluginError> {
let mut plugins: PluginMap = HashMap::new();
let mut count = 0;
// Register internal plugin
for i in internal::get_internal_plugin() {
let name = i.name.to_string();
match list {
Some(list) if !list.contains(&name) => {
log::debug!("Plugin {} not in list, skipping", name);
continue;
}
_ => {}
}
log::debug!("Registering internal plugin {}", i.name);
let mut plugin = FinalizePlugin::new(i.entry);
plugin.metadata.renderer = Some(i.renderer);
plugins.insert(i.name.to_string(), plugin);
count += 1;
}
// Register external plugin
for (name, data) in external_plugins.into_iter() {
match list {
Some(list) if !list.contains(&name) => {
log::debug!("Plugin {} not in list, skipping", name);
continue;
}
_ => {}
}
log::debug!("Registering external plugin {}", name);
let (_source, config) = data.into_iter().next().unwrap_or_else(|| {
panic!(
"Internal Error: Plugin {} has no source configured, this should have been caught by validation!",
name
)
});
// Assume every plugin prepared (or definitely unavailable) now
if let Some(reason) = config.runtime.error {
log::warn!("Plugin {} is not available: {}", name, reason);
continue;
}
// Resolve plugin path from registry instead of config.runtime.fspath
let plugin_path = registry.resolve(&name).ok_or_else(|| {
PluginError::PluginUnavailable {
name: name.clone(),
reason: "plugin not found in resource registry".to_string(),
}
})?;
let metadata = read_manifest(Some(plugin_path), &name)?;
let plugin_type = get_plugin_type(metadata.plugin_type.clone(), &metadata.entrypoint);
let entry = get_entry(plugin_type, &name)?;
plugins.insert(
name,
FinalizePlugin::new(entry)
.with_metadata(metadata)
.with_argument(config.config),
);
count += 1;
}
// Inject plugin-declared templates as .fallback
for (name, plugin) in &plugins {
if let Some(ref templates) = plugin.metadata.templates {
let key = format!("{}.fallback", name);
for tmpl in templates {
notification_config.templates.insert(key.clone(), tmpl.clone());
}
log::debug!("Registered templates from plugin '{}' as '{}'", name, key);
}
}
log::info!("Registered {} plugins", count);
Ok(plugins)
}
/// Calls a registered plugin by name, passing the configuration, execution context and event sender.
///
/// # Arguments
///
/// * `plugins` - Map of registered plugin names to their [`FinalizePlugin`] instances
/// * `name` - Name of the plugin to invoke
/// * `config` - Configuration value passed to the plugin
/// * `ctx` - Execution context for the plugin
/// * `event_tx` - Event sender for plugin lifecycle events
///
/// # Returns
///
/// Returns `Ok(())` on successful plugin execution, or a [`PluginError`] if the plugin
/// is not found or execution fails.
pub async fn call_named_plugin(
plugins: &PluginMap,
name: &str,
config: serde_yaml::Value,
ctx: &ExecutionContext,
event_tx: &EventSender,
) -> PluginResult {
let plugin = plugins
.get(name)
.ok_or_else(|| PluginError::UnknownPlugin(name.to_string()))?;
plugin.call(config, ctx, event_tx).await
}
@@ -0,0 +1,20 @@
use crate::finalize::plugin::PluginError;
use crate::finalize::plugin::PluginMetadata;
use workshop_engine::{EventSender, ExecutionContext};
use crate::{ finalize::plugin::AsyncPluginFn};
use async_trait::async_trait;
pub struct Dylib;
#[async_trait]
impl AsyncPluginFn for Dylib {
async fn call(
&self,
_metadata: &PluginMetadata,
_argument: serde_yaml::Value,
_ctx: &ExecutionContext,
_event_tx: &EventSender,
) -> Result<(), PluginError> {
unimplemented!("Dylib plugin is not implemented yet");
}
}
@@ -0,0 +1,47 @@
use crate::finalize::plugin::AsyncPluginFn;
use crate::notify::types::NotificationRenderer;
pub mod dummy;
mod insitenotify;
mod mail;
mod satori;
mod webhook;
pub struct InternalPlugin {
pub name: &'static str,
pub entry: Box<dyn AsyncPluginFn>,
pub renderer: NotificationRenderer,
}
pub fn get_internal_plugin() -> Vec<InternalPlugin> {
vec![
InternalPlugin {
name: "in-site-notify",
entry: Box::new(insitenotify::InsiteNotify),
renderer: NotificationRenderer::Plugin,
},
InternalPlugin {
name: "mail",
entry: Box::new(mail::Mail),
renderer: NotificationRenderer::Server,
},
InternalPlugin {
name: "satori",
entry: Box::new(satori::Satori),
renderer: NotificationRenderer::Server,
},
InternalPlugin {
name: "webhook",
entry: Box::new(webhook::Webhook),
renderer: NotificationRenderer::Server,
},
]
}
/// Returns dummy only (for `--debug notify.dummy` mode).
/// All real adapters and in-site-notify are excluded.
pub fn get_debug_plugin() -> Vec<InternalPlugin> {
vec![InternalPlugin {
name: "dummy",
entry: Box::new(dummy::Dummy),
renderer: NotificationRenderer::Plugin,
}]
}
@@ -0,0 +1,21 @@
use crate::finalize::plugin::PluginError;
use crate::finalize::plugin::PluginMetadata;
use workshop_engine::{EventSender, ExecutionContext};
use crate::{ finalize::plugin::AsyncPluginFn};
use async_trait::async_trait;
pub struct Dummy;
#[async_trait]
impl AsyncPluginFn for Dummy {
async fn call(
&self,
metadata: &PluginMetadata,
argument: serde_yaml::Value,
_ctx: &ExecutionContext,
_event_tx: &EventSender,
) -> Result<(), PluginError> {
log::info!("[Dummy] metadata={:?}, arg={:?}", metadata, argument);
Ok(())
}
}
@@ -0,0 +1,20 @@
use crate::finalize::plugin::PluginMetadata;
use crate::{
finalize::{error::PluginError, plugin::AsyncPluginFn},
};
use async_trait::async_trait;
use workshop_engine::{EventSender, ExecutionContext};
pub struct InsiteNotify;
#[async_trait]
impl AsyncPluginFn for InsiteNotify {
async fn call(
&self,
_metadata: &PluginMetadata,
_argument: serde_yaml::Value,
_ctx: &ExecutionContext,
_event_tx: &EventSender,
) -> Result<(), PluginError> {
log::info!("in-site-notify: placeholder (always succeeds)");
Ok(())
}
}
@@ -0,0 +1,122 @@
notification:
1:
# 最简配置
mail: # 插件名,不可变动,下同
to: "admin@example.com"
from: "ci@example.com"
subject: "Build {{ build.status }}: {{ pipeline.name }}"
body: "Pipeline {{ pipeline.name }} build #{{ build.id }} finished with status: {{ build.status }}"
smtp:
host: "smtp.gmail.com"
port: 587
username: "{{ secret.smtp_user }}"
password: "{{ secret.smtp_pass }}"
# 高级配置(完整功能)
mail:
# ===== 收件人配置 =====
to:
- "team@example.com"
- "admin@example.com"
- "{{ commit.author_email }}" # 模板变量支持
cc:
- "manager@example.com"
bcc:
- "archive@example.com"
# 多态,单字符串=email
from:
name: "HoneyBiscuitWorkshop CI"
email: "ci@example.com"
reply_to: "support@example.com"
# 可选:使用预定义模板
schema: "custom" # 或 "default",或定义在notification.schema里的字段
# ===== 主题配置,schema=custom时可用 =====
subject: "[{{ build.status | upper }}] {{ pipeline.name }} #{{ build.id }} - {{ commit.message | truncate(50) }}"
# ===== 正文配置,schema=custom时可用 =====
body: |
<html>
<body style="font-family: Arial, sans-serif;">
<h2 style="color: {{ build.status_color }};">
Build {{ build.status | title }}
</h2>
<table>
<tr><td><strong>Pipeline:</strong></td><td>{{ pipeline.name }}</td></tr>
<tr><td><strong>Build ID:</strong></td><td>{{ build.id }}</td></tr>
<tr><td><strong>Status:</strong></td><td>{{ build.status }}</td></tr>
<tr><td><strong>Duration:</strong></td><td>{{ build.duration }}</td></tr>
<tr><td><strong>Commit:</strong></td><td>{{ commit.hash | slice(0, 7) }}</td></tr>
<tr><td><strong>Author:</strong></td><td>{{ commit.author }}</td></tr>
<tr><td><strong>Message:</strong></td><td>{{ commit.message }}</td></tr>
</table>
<p>
<a href="{{ build.url }}" style="background: #007bff; color: white; padding: 10px 20px; text-decoration: none; border-radius: 5px;">
View Build Details
</a>
</p>
</body>
</html>
# ===== SMTP 配置 =====
smtp:
host: "smtp.gmail.com"
port: 587
# 认证方式:password / oauth2 / none
auth:
type: "password"
username: "{{ secret.smtp_user }}"
password: "{{ secret.smtp_pass }}"
# 加密方式:tls / starttls / none
encryption: "starttls"
# 连接超时
connect_timeout: "30s"
timeout: "90s"
retry: 3
fallible: false
# 仅作参考,暂不实现:
# OAuth2 配置示例(Gmail
mail:
to: "user@example.com"
from: "ci@example.com"
subject: "Build Notification"
body: "Build completed"
schema: "default"
smtp:
host: "smtp.gmail.com"
port: 465
auth:
type: "oauth2"
client_id: "{{ secret.gmail_client_id }}"
client_secret: "{{ secret.gmail_client_secret }}"
refresh_token: "{{ secret.gmail_refresh_token }}"
encryption: "tls"
# 企业级配置(Exchange/Office365
mail:
to: "team@company.com"
from: "ci@company.com"
subject: "{{ pipeline.name }} Build {{ build.status }}"
schema: "default"
smtp:
host: "smtp.office365.com"
port: 587
auth:
type: "password"
username: "{{ secret.exchange_user }}"
password: "{{ secret.exchange_pass }}"
encryption: "starttls"
# 企业环境可能需要自定义 TLS 配置
tls:
verify: true
ca_cert: "{{ secret.exchange_cert }}" # 可选

Some files were not shown because too many files have changed in this diff Show More