Compare commits

15 Commits

Author SHA1 Message Date
Catty Steve df5026cdbe refactor(notify): extract handler, rule, util
CI / ${{ matrix.crate }} (workshop-engine) (push) Failing after 3m54s
CI / ${{ matrix.crate }} (workshop-baker) (push) Failing after 27m50s
Split the monolithic notify module into three submodules:
- `handler`: async bucket dispatch with cooldown, retry, escalation
- `rule`: rule parsing from YAML config, template merging, overrides
- `util`: validation, event timing, helper functions
  `NotificationTemplateDef` replaces `AddAssign` with `apply()` for
  non-destructive merging (self wins, other fills gaps).
  `insitenotify` logs instead of panicking.
2026-05-22 00:16:28 +08:00
Catty Steve d1ad08ef3a feat: add workshop-baker-params for config layering
Implement a priority-based merge system for pipeline configuration.
Parameters carry their source (Base, Courier, Bakerd) and only
higher-priority writers override. Migrate notification, prebake, bake,
and finalize config to use ParamVal wrappers. Remove hardcoded
constants and the InteractiveServer component.

BREAKING CHANGE: Remove NotAvailable variant from MethodStatus enum
2026-05-21 20:04:47 +08:00
Catty Steve cf2968e720 refactor: update AGENTS.md and remove osbolete modules
Archive 7 deprecated crates. Add Resource model documentation.
Remove internal fetch module from workshop-baker.
2026-05-20 16:30:50 +08:00
Catty Steve 28abc5d207 refactor: replace internal fetch utilities with ResourceRegistry
Remove the monolithic utils::fetch module (HTTP download, git clone,
checksum, extraction) and utils::fetch_plugin module from the baker.
Introduce a ResourceRegistry type to centrally track pipeline
resources by name and path.

The actual fetching logic is now delegated to the workshop-getterurl
crate, which gains optional md5 and sha1 hash support behind default
features. Also flatten the builtin module in workshop-getterurl.
2026-05-19 23:22:59 +08:00
Catty Steve 6e7b96cd66 refactor: remove unused crates, refactor fetch
- Move fetch_plugin from finalize::plugin::fetch to utils::fetch_plugin
- Delete unused crates: workshop-cert, workshop-deviceid, workshop-vault
- Remove parser benchmarks and empty daemon/socket modules
- Add new workshop-getterurl crate for resource fetching
2026-05-19 21:53:41 +08:00
Catty Steve 3e80825d58 refactor(plugin)!: remove dedicated plugin fetch subsystem
Drop the entire plugin fetch module (HTTP, git, checksums, extraction).
Simplify mail notification plugin to a bare stub.
Remove outdated metadata tests.

BREAKING CHANGE: Not a buildable version!
2026-05-19 16:24:19 +08:00
Catty Steve c97eafab29 refactor(baker): Restructure CLI, drop legacy tests
- Remove Python-based test framework, Dockerfiles, and runner scripts
- Add bare subcommand for standalone pipeline stage execution
- Implement unified worker mode and error module
- Refactor NotificationConfig to support YAML string keys
- Add dummy internal plugin and clean up mail template
2026-05-19 16:19:28 +08:00
Catty Steve f737b6a24d feat(baker): add axum HTTP server and refactor plugin fetching
Refactor plugin imports to use workshop_engine types, consolidate
resource fetching into utils/fetch module with checksum verification,
and add interactive notification server using axum.
2026-05-18 10:01:55 +08:00
Catty Steve 681b7d1333 style: format all code to comply with cargo fmt check
CI / ${{ matrix.crate }} (workshop-engine) (push) Failing after 36m14s
CI / ${{ matrix.crate }} (workshop-baker) (push) Successful in 1h35m51s
2026-04-28 19:17:56 +08:00
Catty Steve f07588a716 test: add Go UPM system tests for bare prebake 2026-04-28 19:12:53 +08:00
Catty Steve b0530fb858 feat(upm): add Python and Go package manager support 2026-04-28 16:56:53 +08:00
Catty Steve fd18b547e5 feat(upm): implement Rust user package manager
Adds the `user.rust` UPM to workshop-engine for managing Rust
toolchains,
components, cross-compilation targets, and cargo operations:

- `rustup default <toolchain>` / `rustup component add` / `rustup target
  add`
- `cargo fetch` for dependency pre-fetch with optional `--locked`
- `cargo install` for global crate tools with registry support

Includes 275-line test suite (`tests/upm/test_rust.py`) covering the
full
lifecycle with optional network tests.
2026-04-28 12:20:05 +08:00
Catty Steve cf317b55c6 feat(pm): add apk/dnf implementations and multi-distro test
CI / ${{ matrix.crate }} (workshop-engine) (push) Failing after 28m8s
CI / ${{ matrix.crate }} (workshop-baker) (push) Failing after 33m39s
infrastructure
2026-04-27 23:01:43 +08:00
Catty Steve 20e2ab4224 refactor: reorganize imports and add lint allows
CI / ${{ matrix.crate }} (workshop-engine) (push) Failing after 13m13s
CI / ${{ matrix.crate }} (workshop-baker) (push) Failing after 28m54s
2026-04-27 14:31:11 +08:00
Catty Steve 50b42e654c refactor(workshop-engine): replace tokio Child with ManagedChild trait
for testability

- Add `ManagedChild` trait abstracting process lifecycle (real
  `TokioChild` + `MockChild` for tests)
- Use Unix process groups (`process_group(0)` + `libc::kill(-pgid,
  SIGKILL)`) for tree-wide termination
- Remove cgroup-related code (`CgroupManager`, `ResourceUsage`,
  `cgroup_path` field)
- Update imports across test files to use `workshop-engine` crate
  directly
2026-04-27 12:02:17 +08:00
171 changed files with 8525 additions and 14740 deletions
+3 -1
View File
@@ -9,7 +9,7 @@ archived
*.bak
*.clean
session*
llm.env
.secret
# System test artifacts
tests/results/
@@ -18,3 +18,5 @@ tests/**/__pycache__/
tests/.pytest_cache/
tests/llm/.cache/
*.pyc
.staging
.pytest_cache
+40 -56
View File
@@ -1,83 +1,72 @@
# PROJECT KNOWLEDGE BASE
**Generated:** 2026-04-22
**Commit:** 7b3b71a
**Generated:** 2026-05-19
**Commit:** 28abc5d
**Branch:** master
## OVERVIEW
HoneyBiscuitWorkshop is a Rust-based CI/CD system ("蜜饼工坊") with LLM-powered pipeline auto-configuration. 9 independent Cargo crates, no workspace.
HoneyBiscuitWorkshop ("蜜饼工坊") — Rust CI/CD system with decorator-driven shell pipeline DSL and LLM-powered configuration. 3 independent Cargo crates, no workspace.
## STRUCTURE
```
./
├── workshop-baker/ # Main orchestrator (CLI + daemon)
├── workshop-agent/ # HTTP agent server
├── workshop-pipeline/ # Config library (prebake/finalize)
├── workshop-llm-detector/ # LLM repo analyzer
├── workshop-vault/ # Secret management client
├── workshop-cert/ # TLS certificate generator
├── workshop-deviceid/ # Device ID library
├── workshop-builder-native/ # Stub (unused)
├── workshop-helper-mac/ # MAC address utility
├── workshop-baker/ # Main orchestrator (CLI + pipeline stages)
├── workshop-engine/ # Execution engine (extracted from baker)
├── workshop-getterurl/ # URL fetching (go-getter style)
├── docs/ # mdBook documentation (zh-CN)
├── playground/ # Experimentation
└── third_party/ # Empty
├── templates/ # Pipeline templates (bake.sh, finalize.yml, prebake.yml)
└── archived/ # Deprecated crates (7 moved here)
```
## WHERE TO LOOK
| Task | Location | Notes |
|------|----------|-------|
| Baker (main) | `workshop-baker/src/{bake,prebake,engine}/` | Core pipeline logic |
| Agent server | `workshop-agent/src/` | Actix-web HTTP + TLS |
| Pipeline config | `workshop-pipeline/src/config/` | prebake/finalize YAML |
| LLM integration | `workshop-llm-detector/src/llm/` | Ollama/OpenAI clients |
| Secrets | `workshop-vault/src/` | Vault client wrapper |
| Baker (CLI) | `workshop-baker/src/{lib,main}.rs` | Clap CLI + tokio entry |
| Pipeline stages | `workshop-baker/src/{bake,prebake,finalize}/` | Stage orchestration |
| Notifications | `workshop-baker/src/notify/` | Plugin-based event notifications |
|| Engine (execution) | `workshop-engine/src/` | Process mgmt, PM detection |
| URL fetching | `workshop-getterurl/src/` | Git/HTTP/File resource getter |
## CODE MAP
| Symbol | Type | Location | Role |
|--------|------|----------|------|
| PipelineConfig | struct | workshop-pipeline/src/lib.rs:27 | Full pipeline config |
| CICDLLMHelper | struct | workshop-llm-detector/src/lib.rs:16 | LLM analysis entry |
| Engine | struct | workshop-baker/src/engine.rs | Build executor |
| VaultClient | struct | workshop-vault/src/client.rs | Secret ops |
| Cli | struct | workshop-baker/src/lib.rs:21 | CLI argument struct |
| ExecutionContext | struct | workshop-engine/src/types.rs:10 | Pipeline context |
| Engine | struct | workshop-engine/src/types.rs:99 | Build executor (placeholder) |
| PrebakeConfig | struct | workshop-baker/src/prebake/config.rs | Prebake YAML schema |
| FinalizeConfig | struct | workshop-baker/src/finalize/config.rs:21 | Finalize YAML schema |
| GetterUrl | struct | workshop-getterurl/src/lib.rs:84 | Parsed resource URL |
| Getter | trait | workshop-getterurl/src/getter.rs:18 | Resource fetch trait |
## CONVENTIONS
### Rust
- **Edition**: 2024 (non-standard, requires Rust 1.85+)
- Deprecate `mod.rs`
- **Edition**: 2024 (requires Rust 1.85+), deprecates `mod.rs`
- **Naming**: `snake_case` files/modules, `PascalCase` types, `SCREAMING_SNAKE_CASE` consts
- **Error handling**: `thiserror` + `anyhow` combo
- **Error handling**: `thiserror` + `anyhow` combo; `HasExitCode` trait for exit codes
- **Async**: `#[tokio::main]` + `tokio` with "full" features
- **Imports**: `std``external_crate``crate::module`
- **Dual-mode**: CLI commands (prebake/bake/finalize) + daemon mode (unimplemented)
### Testing
- **Rust**: `#[test]` inline, `#[tokio::test]` async, `tests/*.rs` integration
- **Python**: `pytest` with `tests/integration/test_*.py` (Docker-based)
- **Benchmarks**: Criterion with `harness = false`
- **Rust**: `#[test]` inline, `#[tokio::test]` async, `tests/` integration
- **Benchmarks**: Criterion (configured in Cargo.toml)
### Build Pipeline
- **prebake.yml**: Environment setup (docker/firecracker/baremetal)
- **bake.sh**: Build execution with `# @pipeline` decorators
- **finalize.yml**: Packaging, deployment, notifications
- **Annotations**: `@timeout()`, `@retry()`, `@parallel`, `@fallible`
### CI
- **GitHub Actions**: rust-toolchain (stable), clippy + rustfmt, `cargo test --all-features`
- **Matrix**: workshop-baker, workshop-engine
## ANTI-PATTERNS (THIS PROJECT)
1. **Python in Rust project** — workshop-baker has Python tests (pytest.ini, test_prebake.py)
2. **temp/ directory** — Non-standard in Rust projects
3. **Certificates in source** — artifact workshop-cert/certs/ not excluded
4. **Deprecated projects** — workshop-executor.old, workshop-monitor.old still present
5. **Hardcoded mirrors** — Tsinghua University mirrors in configs
6. **Rust 2024 edition** — May not work with stable toolchains
1. **Hardcoded user** — "vulcan" username hardcoded throughout prebake
2. **Unsafe libc** — workshop-baker/src/prebake/security.rs has 5+ unsafe blocks
3. **Rust 2024 edition** — May not work with stable toolchains
4. **Empty daemon**`daemon/` dir is empty, Daemon command is `unimplemented!()`
## UNIQUE STYLES
- **Pipeline DSL**: Shell scripts with `# @decorator` annotations
- **Dual-mode baker**: CLI commands + daemon mode
- **LLM cookbook system**: YAML cookbooks for language-specific builds
- **Notification system**: Plugin-based with batching, retry, priority groups
- **LLM cookbook system**: YAML cookbooks for language-specific builds (archived)
- **Chinese docs** — Documentation primarily in zh-CN
- **User "vulcan"** — Custom build user (not root/runner)
## COMMANDS
```bash
@@ -86,23 +75,18 @@ cargo build --release -p workshop-baker
# Test
cargo test -p workshop-baker
pytest tests/integration/test_prebake.py -v
cargo test -p workshop-engine
# Lint
cargo clippy -- -D warnings
cargo fmt --all
# Run baker
cargo run --bin workshop-baker -- prebake config.yml
cargo run --bin workshop-baker -- bake script.sh
# Run agent
cargo run --bin workshop-agent
# Run
cargo run --bin workshop-baker -- --pipeline demo --build-id 1 --prebake prebake.yml --bake bake.sh --finalize finalize.yml
```
## NOTES
- **Docker required** for integration tests
- **workshop-builder-native** is a stub (3-line Hello World)
- No root workspace Cargo.toml — each crate is independent
- `AGENTS.md.old` at root — legacy documentation
- **No root workspace** — each crate independent, build separately
- **7 archived crates** — workshop-agent, workshop-cert, workshop-deviceid, workshop-helper-mac, workshop-llm-detector, workshop-pipeline, workshop-vault
- **Python tests removed** — pytest.ini exists but test files no longer in workspace
+2
View File
@@ -29,8 +29,10 @@
- [测试策略](zh-CN/vol3_dev/testing_strategy.md)
- [插件系统指南](zh-CN/vol3_dev/plugin_system_guide.md)
- [Engine 模块指南](zh-CN/vol3_dev/engine_module_guide.md)
- [设计决策](zh-CN/vol3_dev/design_decisions.md)
- [添加包管理器](zh-CN/vol3_dev/add_package_manager.md)
- [自定义插件](zh-CN/vol3_dev/custom_plugin.md)
- [自定义 Decorator](zh-CN/vol3_dev/custom_decorator.md)
- [模板变量](zh-CN/vol3_dev/template_variables.md)
- [邮件模板](zh-CN/vol3_dev/mail_template.md)
- [Resource 模型](zh-CN/vol3_dev/resource_model.md)
+412
View File
@@ -0,0 +1,412 @@
# 设计决策
本文档记录了 HoneyBiscuitWorkshop 架构演进过程中的关键设计决策及其背后的思考。
> **注意**:本文档面向开发者,记录的是**目标架构**的设计方向,而非当前代码的实现现状。
> 当前 bakerd 尚未完全实现(`daemon.rs` 为 `todo!()`),bare mode 仅用于开发调试。
---
## 1. Bare Mode 与 Daemon Mode
### 决策
**Bare mode 仅用于开发调试,不承载生产语义。**
### 背景
当前代码提供两种运行入口:
| 模式 | 入口 | 状态 | 用途 |
|------|------|------|------|
| **Bare mode** | `cargo run -- prebake/bake/finalize <args>` | 可用 | 开发调试 |
| **Daemon mode** | `cargo run --` (无子命令) | `todo!()` | 生产 |
### 推论
- Bare mode 中 `ctx.privileged` 是写死的 fake 值(`Prebake: true`, `Bake: false`),不代表生产行为
- Bare mode 中 `os_info::get()` 读取的是 HOST 的 OS 信息,不代表环境内的 OS
- 不要从 bare mode 的行为推导生产路径的问题
- 不要为了"修复" bare mode 的问题而给生产设计增加不必要的抽象
### 代码残留
当前 `bake.rs` 中存在以下已在设计层面淘汰、但因 bare mode 未被清理的代码:
- `fn privileged()` — 从未被调用
- `type TaskResult` — 从未被使用
- `type TaskFn` — 从未被使用
这些是 `bake.rs` 废弃功能的残留,不涉及生产设计。清理它们不改变任何行为。
---
## 2. 环境模型
### 决策
**bakerd 先创建环境(容器/VM/裸机),然后 baker 在环境内执行全部三个阶段(prebake → bake → finalize)。**
### 架构
```
bakerd
├── prepare_environment (容器/VM/裸机) ← 先建隔离环境
├── prebake (全在环境内执行) ← 包括 depssystem 的包安装
├── bake (全在环境内执行) ← 构建脚本
├── finalize (全在环境内执行) ← 部署通知
└── cleanup
```
### 推论
- `prepare_environment` 不是 prebake 的一个 stage——它是 prebake 的前置条件
- `depssystem.rs``os_info::get()` 读取的应当是**环境内**的 `/etc/os-release`,逻辑正确
- 容器镜像只需拉取(`prepare_container`),不需要在代码层面管理容器生命周期——那是 bakerd 的职责
- 当前 `prepare_container` 返回的 `_container` 被丢弃,是因为 bakerd 尚未实现,不是设计缺陷
### 环境类型
```yaml
env:
type: container # Docker/Podman 容器
# type: firecracker # Firecracker 微 VM
# type: baremetal # 裸机直接执行
image: ubuntu:24.04
```
环境声明决定了:
1. 隔离级别(进程级 / VM级 / 无隔离)
2. 操作系统和工具链(通过 `image`
3. 包管理器(从 OS 推导)
---
## 3. PM 选择链
### 决策
**包管理器(PM)应从环境声明中推导,而不是通过运行时探测。**
### 选择链
```
用户声明 bakerd 确定 执行时确定
env.image: ubuntu:24.04
容器运行时 (Docker/VM)
│ os_info::get() 在环境内
OS: Ubuntu 24.04
│ 从已知映射推导
PM: apt
│ depssystem 使用 PM trait 操作
apt install -y {packages}
```
### 为什么淘汰 detect()
`pm::detect()` 的当前逻辑:
```rust
pub fn detect(distro: &os_info::Info) -> Option<Box<dyn PackageManager>> {
all_managers().into_iter().find(|pm| pm.adopt(distro))
}
```
它在解决一个**已经知道答案的问题**:
- 用户声明了 `image: ubuntu:24.04`
- 容器内 `/etc/os-release` 确认了 Ubuntu
- `detect()` 遍历所有 PM 并匹配,得出 apt
- **结果和环境声明一致,但绕了一大圈**
`detect()` 的唯一价值场景:
- **Bare mode** 用户没配环境时,省一个配置项
- 这在生产路径中不应出现(bakerd 总是提供环境)
### 推论
- PM trait 保留(见下节),但 `adopt()` 方法在生产路径中不需要
- `detect()` / `adopt()` / `os_info::get()` 这条路径应当是 dev-only 的便利设施
- Daemon 模式中 PM 的选择依据是环境声明,不是运行时探测
- 当前 `os_info::Type` 的 58 个变体覆盖不全(仅 Pacman 可用)——这是 bare mode 的问题,不影响生产设计
---
## 4. PM Trait 的职责边界
### 决策
**PM trait 保留核心操作能力,淘汰探测分类能力。**
### 核心方法
```rust
pub trait PackageManager {
fn name(&self) -> &'static str;
fn update(&self) -> Vec<Command>;
fn install(&self, package: &[String]) -> Vec<Command>;
fn change_mirror(&self, repo: &str, osinfo: Option<&Info>) -> Vec<Command>;
fn add_repository(&self, repo: &Value, osinfo: Option<&Info>) -> Vec<Command>;
}
```
这四个方法是 PM-specific 的操作,具有真实复杂度:
| 操作 | PM 间差异 | 原因 |
|------|-----------|------|
| `install` | 参数完全不同 | `apt install -y` vs `pacman -S --noconfirm` |
| `update` | 命令不同 | `apt update` vs `pacman -Sy` |
| `change_mirror` | 配置格式不同 | sources.list vs mirrorlist vs .repo |
| `add_repository` | 仓库机制不同 | PPA vs AUR vs COPR |
### 为什么镜像源和仓库在容器中仍然需要
容器不解决源管理问题:
```
docker pull ubuntu:24.04
# 容器内 sources.list → archive.ubuntu.com
# 但用户在北京 → 需要 mirror.tuna.tsinghua.edu.cn
# 这不是预置在镜像中的——源是运行态配置
```
`change_mirror``add_repository` 都是**容器运行时需要**的操作,不能预置在基础镜像中。PM trait 的存在价值就在于这些操作。
### 淘汰的方法
```rust
fn adopt(&self, distro: &Info) -> bool; // 生产路径不需要
fn binary(&self) -> &'static str; // 从未被使用
```
### 实现策略
在 daemon 模式下,PM 的选型由环境声明确定(`image: ubuntu:24.04``apt`),PM 实现仅需按需注册,不需要自述适配范围:
```rust
// 不再需要 detect() + adopt()
// PM 实例由工厂方法根据 PM 名称字符串创建
pub fn from_name(name: &str) -> Option<Box<dyn PackageManager>> {
match name {
"apt" => Some(Box::new(Apt)),
"pacman" => Some(Box::new(Pacman)),
"dnf" => Some(Box::new(Dnf)),
_ => None,
}
}
```
---
## 5. 发行版多态
### 决策
**流水线不应是发行版多态的。发行版差异由容器化解决。**
### 原因
流水线的执行体(bake 脚本)天然发行版绑定:
```bash
# @pipeline
function build() {
cmake -B build -DCMAKE_INSTALL_PREFIX=/usr # Linux 路径
make -j$(nproc)
}
```
- 换到 macOS`/usr` 只读,路径不同
- 换到 WindowsMSVC 而不是 GCC
- 换到 Alpinemusl 而不是 glibc
这与 PM 的差异是同一类问题,但 PM 抽象不能解决它。**容器才是解决方案:**
```yaml
# 在 Ubuntu 上构建
env:
type: container
image: ubuntu:24.04
```
```yaml
# 在 Alpine 上构建
env:
type: container
image: alpine:3.19
```
### 各层面的发行版相关性
| 层面 | 发行版相关 | 解决方案 |
|------|-----------|----------|
| DepsSystem(系统包) | ✅ | 容器内 PM |
| DepsUser(用户包管理器) | ❌ | 天然无关(cargo/npm/pip 跨发行版一致) |
| Bake(构建脚本) | ✅ | 容器内执行 |
| Finalize(后处理) | ❌ | 纯产物操作,无关发行版 |
### UPM 的特殊位置
用户包管理器(UPM,如 cargo、npm、pip、nix)天然跨发行版:
```bash
cargo install --locked bat # Ubuntu 和 Arch 上都一样
npm install -g typescript # Ubuntu 和 Arch 上都一样
```
`depsuser.rs` 处理的正是这个层面。但 `collect_upm_sysdeps()` 需要知道系统 PM——这是 UPM 和系统 PM 之间的 bridge。在设计上,这个 bridge 的 PM 信息应来自环境声明,而非 `detect()`
---
## 6. drop_after 安全模型
### 决策
**`drop_after` 指定流水线在哪个 stage 之后从 root 降权到 vulcan 用户。合理的默认值是 `DepsUser`。**
### Stage 顺序
```rust
pub enum PrebakeStage {
Init,
Bootstrap,
EarlyHook,
DepsSystem, // ← 默认
DepsUser,
LateHook,
Ready,
Never, // ← 永远不降权
}
```
### 各 stage 的特权需求
| Stage | 需要 root | 原因 |
|-------|-----------|------|
| Bootstrap | ✅ | 创建系统用户(vulcan)、配置 sudo/doas |
| EarlyHook | ❓ | 用户自定义 |
| DepsSystem | ✅ | `apt install` 需要 root |
| DepsUser | ❌ | `cargo install` 不需要 root |
| LateHook | ❓ | 用户自定义 |
| Ready / Bake | ❌ | 构建应以非特权用户运行 |
### 正确的配置
```yaml
security:
drop_after: "DepsUser"
```
执行效果:
```
Bootstrap root ← 创建用户
EarlyHook root ← hook 以 root 运行
DepsSystem root ← apt install -y {packages}
DepsUser root ← cargo install {packages}
LateHook vulcan ← 降权,用户脚本不能 rm -rf /
Ready vulcan
Bake vulcan ← 构建以 vulcan 运行
```
### 禁止的配置
`drop_after` 设置为 `Bootstrap``EarlyHook` 在当前 stage 顺序下会损坏 DepsStage(需要 root 权限的 `apt install` 在降权后无法执行)。
**配置验证应拒绝 `drop_after < DepsUser`**(以 root 运行的 user 级包安装没有意义,但 DepsSystem 必须 root)。
### 特殊值 Never
```yaml
security:
drop_after: "Never" # 整条流水线以 root 运行
```
适用场景:Docker 构建、内核模块编译等需要 root 权限的操作。Bootstrap 的 `user: root` 配置也会产生相同效果。
---
## 7. 当前代码中的真问题
以下问题是当前代码中确实存在的,与 bare/daemon 模式无关:
### 7.1 Apt::adopt 永远返回 false
```rust
// workshop-engine/src/pm/apt.rs:20-23
fn adopt(&self, distro: &Info) -> bool {
return false; // ← 永远 false
// todo!(); // ← 死代码
}
```
- `adopt()``return false` 后无法执行 `todo!()`
- Apt 永远不会被 `detect()` 选中
- 后果:Debian/Ubuntu 用户在 bare mode 下永远检测不到 PM
- 修复方向:移除 return false,实现真正的 adopt 逻辑(Debian/Ubuntu/Mint/Pop 等发行版的匹配)
### 7.2 PM 覆盖严重不全
当前 `all_managers()` 仅注册了 Pacman
```rust
pub fn all_managers() -> Vec<Box<dyn PackageManager>> {
vec![
// Box::new(apt::Apt), // adopt 坏了
Box::new(pacman::Pacman), // 唯一可用
// Box::new(dnf::Dnf), // 未实现
]
}
```
os_info 3.14.0 定义了 58 种发行版类型,Pacman 仅覆盖其中 9 种(Arch 系)。其余 49 种无 PM 覆盖。
### 7.3 死代码
| 位置 | 内容 | 状态 |
|------|------|------|
| `bake.rs:98` | `fn privileged()` | 定义但未调用 |
| `bake.rs:108` | `type TaskResult` | 定义但未使用 |
| `bake.rs:110` | `type TaskFn` | 定义但未使用 |
| `workshop-baker/src/prebake/stage/environment.rs` | `prepare_environment()` | 定义但未接入执行流 |
---
## 8. Daemon 设计原则(汇总)
以上讨论最终可以提炼为以下设计原则,供 bakerd 重构时参考:
### P1. 环境声明驱动
```
用户声明 env.image → bakerd 创建环境 → 环境内确定 PM → depssystem 操作 PM
```
PM 的选择权在环境声明,不在运行时探测。`detect()` 不出现在生产路径中。
### P2. 容器是发行版隔离的边界
发行版差异在容器边界被隔断。容器内是已知、确定的环境。PM trait 只负责"已知 PM 的操作"install、update、mirror、repo),不负责"猜是什么 PM"。
### P3. 降权边界可配置但需校验
`drop_after` 从 root 降权到 vulcan,默认在 DepsUser 之后。配置验证需确保:降权不会损坏需要 root 的 stage。
### P4. 最小化 trait 抽象
PM trait 保留操作系统所需的核心方法。不在 trait 中嵌入分类功能(adopt)。分类是一个数据映射问题,不是多态问题。
### P5. Bare mode 是开发工具,不承载生产行为
Bare mode 的行为(host 上直接执行、write static privileged 等)不代表生产设计。不要为了优化 bare mode 而增加生产路径的抽象负担。
### P6. 天然跨发行版的部分不需要抽象
UPMcargo/npm/pip/nix)跨发行版一致,不需要 PM 级别的抽象,不需要 detect,不需要 distro 感知。trait 只应用于确实有 PM-specific 差异的部分。
+88
View File
@@ -0,0 +1,88 @@
# Resource 模型
## 核心思想
整个系统只关心两件事:东西从哪来(fetch),东西到哪去(publish)。
fetch 方向:URL → 本地路径 → 使用。
publish 方向:本地路径 → 变换(插件)→ URL。
两个方向共用同一套寻址方式,但 Resource 中间态只存在于 fetch 方向。
## URL 格式
```
getter::url?param1=value1&param2=value2
```
getter 显式声明意图。不是"我猜这是个 git 仓库",而是"使用者告诉我这是什么"。
只有三个 getter
- `git` — 克隆仓库,必要时 checkout 指定版本
- `http` / `https` — 下载文件,必要时校验 checksum
- `file` — 复制本地文件或目录
版本锁定不用 `@v1.0``@` 在 URL 里是认证分隔符),用 `?ref=v1.0`
没有 `github.com/user/repo` 看起来应该 clone 它这种黑魔法。
没有 `.tar.gz` 看起来应该解压它的惊喜。
URL 是 URL,行为是行为,两者用 getter:: 显式绑定。
## Resource 不是网络地址
流水线阶段(prebake / finalize / notify)只接触 Resource:一个名字 + 一个本地路径,已就绪。它们不经由网络、不解析 URL、不碰 getter。
所有 fetch 行为在阶段启动前完成:
- worker 模式:bakerd fetch → 填入 ResourceRegistry → 启动 baker
- standalone 模式:baker 自己 fetch → 填入 ResourceRegistry → 执行阶段
- bare 模式:不 fetch,传入的必须是 Resource
ResourceRegistry 是名字到路径的映射。填进去就只读了。
## getterurl 是独立的 crate
URL 格式解析 + 基本 fetch 放在 workshop-getterurl,和 baker 本身没有耦合。任何人都可以用。
crate 分两层:
```
结构层(parser):
"git::https://host/repo.git?ref=v1.0" → { getter: "git", url: Url, params: Params }
获取层(builtin getters):
getter.fetch(url, params, dst) → local_path
git: git clone + optional checkout
http: HTTP GET + optional checksum
file: std::fs::copy
```
获取层是字面意义的——不解压、不推断、不猜意图。
## Checksum
checksum 作为 URL query 参数传递。三态语义:
- `?checksum=sha256:abc` — 必须匹配
- `?checksum=` — 显式禁用校验
- 无 checksum — 如果服务端返回 Content-Digest 头,校验它
getter 不主动校验,它只执行 URL 里指定的操作。
## Artifact 不经过 Resource
artifact 是 bake 的产物,自产生就开始被使用,不存在中间态。它的路径就是 bake 产出的本地路径,不走 ResourceRegistry。
artifact 的 publish 方向由插件定义行为——可能上传、可能扫描、可能只记录。插件定义最终的行为,只要是自洽的。
## 三种运行模式
```
fetch 执行者 fetch 在阶段内
daemon 模式 bakerd 否
standalone baker 自身 否(阶段启动前已完成)
bare 模式 无 否(必须传入 Resource
```
bare 模式下传入的必须是已经就绪的 Resource。外部 URL 是 daemon 或 standalone 的事,bare 不负责。
-112
View File
@@ -1,112 +0,0 @@
#!/bin/bash
set -euo pipefail
# =============================================================================
# HoneyBiscuitWorkshop System Test Runner
# =============================================================================
# Usage: ./runner.sh [auto|manual] [-- pytest-args...]
#
# Modes:
# auto - Build, run all tests, collect reports, cleanup (default)
# manual - Build, start container shell for interactive testing
#
# Examples:
# ./runner.sh # Run all tests automatically
# ./runner.sh auto -k test_prebake # Run only prebake tests
# ./runner.sh manual # Drop into container shell
# =============================================================================
MODE="${1:-auto}"
shift 2>/dev/null || true
PYTEST_ARGS="$@"
SCRIPT_DIR="$(cd "$(dirname "$0")" && pwd)"
STAGING_DIR="/tmp/hbw-system-test"
IMAGE_NAME="hbw-system-test:latest"
# ---- Cleanup handler ----
cleanup() {
echo "[runner] Cleaning up..."
docker rm -f hbw-system-test-container 2>/dev/null || true
rm -rf "$STAGING_DIR"
}
trap cleanup EXIT
# ---- Step 1: Build the binary ----
build_binary() {
echo "[runner] Building workshop-baker (release)..."
cd "$SCRIPT_DIR/workshop-baker"
cargo build --release 2>&1
echo "[runner] Build complete."
}
# ---- Step 2: Prepare staging directory ----
prepare_staging() {
echo "[runner] Preparing staging directory..."
rm -rf "$STAGING_DIR"
mkdir -p "$STAGING_DIR"
# Copy binary
cp "$SCRIPT_DIR/workshop-baker/target/release/workshop-baker" "$STAGING_DIR/"
chmod +x "$STAGING_DIR/workshop-baker"
# Copy examples if they exist
if [ -d "$SCRIPT_DIR/workshop-baker/examples" ]; then
cp -r "$SCRIPT_DIR/workshop-baker/examples" "$STAGING_DIR/examples"
else
mkdir -p "$STAGING_DIR/examples"
fi
# Copy llm.env if it exists
if [ -f "$SCRIPT_DIR/llm.env" ]; then
cp "$SCRIPT_DIR/llm.env" "$STAGING_DIR/llm.env"
fi
echo "[runner] Staging ready at $STAGING_DIR"
}
# ---- Step 3: Build Docker image ----
build_image() {
echo "[runner] Building Docker image: $IMAGE_NAME ..."
docker build -t "$IMAGE_NAME" -f "$SCRIPT_DIR/tests/Dockerfile" "$SCRIPT_DIR"
echo "[runner] Image built: $IMAGE_NAME"
}
# ---- Step 4: Run container ----
run_container() {
local mode="$1"
echo "[runner] Starting container in $mode mode..."
mkdir -p "$SCRIPT_DIR/tests/results"
if [ "$mode" = "manual" ]; then
docker run -it --rm \
--name hbw-system-test-container \
--privileged \
-v "$STAGING_DIR:/workshop:ro" \
-v "$SCRIPT_DIR/tests/results:/results" \
-e TEST_MODE=manual \
"$IMAGE_NAME"
else
docker run --rm \
--name hbw-system-test-container \
--privileged \
-v "$STAGING_DIR:/workshop:ro" \
-v "$SCRIPT_DIR/tests/results:/results" \
-e TEST_MODE=auto \
"$IMAGE_NAME" $PYTEST_ARGS
fi
}
# ---- Main flow ----
echo "============================================"
echo " HoneyBiscuitWorkshop System Test Runner"
echo " Mode: $MODE"
echo "============================================"
build_binary
prepare_staging
build_image
run_container "$MODE"
echo "[runner] Done."
-32
View File
@@ -1,32 +0,0 @@
FROM archlinux:latest
RUN sed -i 's|https://geo.mirror.pkgbuild.com|https://mirrors.tuna.tsinghua.edu.cn/archlinux|g' /etc/pacman.d/mirrorlist && \
sed -i 's|https://mirror.rackspace.com/archlinux|https://mirrors.tuna.tsinghua.edu.cn/archlinux|g' /etc/pacman.d/mirrorlist && \
echo 'Server = https://mirrors.tuna.tsinghua.edu.cn/archlinux/$repo/os/$arch' >> /etc/pacman.d/mirrorlist
RUN pacman -Syu --noconfirm && \
pacman -S --noconfirm \
python python-pip base-devel git curl wget sudo \
which procps-ng shadow && \
pacman -Scc --noconfirm
# Create test user
RUN useradd -m -s /bin/bash testuser
# Python environment
WORKDIR /app
COPY tests/requirements.txt ./requirements.txt
RUN python -m venv /opt/test-venv && \
/opt/test-venv/bin/pip install -r requirements.txt
# Copy test suite
COPY tests/ ./tests/
COPY tests/entrypoint.sh ./entrypoint.sh
RUN chmod +x entrypoint.sh
# Environment
ENV PATH="/opt/test-venv/bin:$PATH"
ENV PYTHONUNBUFFERED=1
ENV PYTHONDONTWRITEBYTECODE=1
ENTRYPOINT ["./entrypoint.sh"]
-164
View File
@@ -1,164 +0,0 @@
"""
Root conftest for HoneyBiscuitWorkshop system tests.
Provides core fixtures for:
- Locating the baker binary and examples
- Running subprocess commands with output capture
- Managing temporary working directories
"""
import os
import subprocess
import shutil
import tempfile
import pytest
def pytest_configure(config):
"""Register custom markers."""
config.addinivalue_line("markers", "prebake: prebake stage tests")
config.addinivalue_line("markers", "bake: bake stage tests")
config.addinivalue_line("markers", "finalize: finalize stage tests")
config.addinivalue_line("markers", "llm: tests requiring LLM API")
config.addinivalue_line("markers", "slow: slow running tests")
@pytest.fixture(scope="session")
def baker_bin():
"""Path to the workshop-baker binary."""
path = os.environ.get("BAKER_BIN", "/workshop/workshop-baker")
if not os.path.isfile(path):
pytest.skip(f"Baker binary not found at {path}")
if not os.access(path, os.X_OK):
pytest.skip(f"Baker binary not executable at {path}")
return path
@pytest.fixture(scope="session")
def examples_dir():
"""Path to the examples directory (may be None)."""
path = os.environ.get("EXAMPLES_DIR", "/workshop/examples")
if not os.path.isdir(path):
return None
return path
@pytest.fixture(scope="session")
def llm_env():
"""LLM environment variables dict (from llm.env)."""
env = {}
for var in [
"OPENAI_BASE_URL",
"OPENAI_API_KEY",
"ANTHROPIC_BASE_URL",
"ANTHROPIC_API_KEY",
"MODEL",
"REASONING_EFFORT",
]:
val = os.environ.get(var)
if val:
env[var] = val
return env
@pytest.fixture
def work_dir():
"""Create a temporary working directory for a test, cleaned up after."""
d = tempfile.mkdtemp(prefix="hbw-test-")
yield d
shutil.rmtree(d, ignore_errors=True)
@pytest.fixture
def run_cmd():
"""Fixture that provides a command runner function.
Returns a function that runs a command and returns
(stdout, stderr, returncode, success).
"""
def _run(args, cwd=None, env=None, timeout=120):
"""Run a command and capture output.
Args:
args: Command as list of strings.
cwd: Working directory.
env: Environment variables (merged with os.environ).
timeout: Timeout in seconds.
Returns:
dict with keys: stdout, stderr, returncode, success
"""
full_env = {**os.environ}
if env:
full_env.update(env)
try:
result = subprocess.run(
args,
cwd=cwd,
env=full_env,
capture_output=True,
text=True,
timeout=timeout,
)
return {
"stdout": result.stdout,
"stderr": result.stderr,
"returncode": result.returncode,
"success": result.returncode == 0,
}
except subprocess.TimeoutExpired:
return {
"stdout": "",
"stderr": f"Command timed out after {timeout}s",
"returncode": -1,
"success": False,
}
except Exception as e:
return {
"stdout": "",
"stderr": str(e),
"returncode": -1,
"success": False,
}
return _run
@pytest.fixture
def run_baker(baker_bin, run_cmd):
"""Fixture that provides a baker command runner.
Returns a function that runs workshop-baker with common flags.
"""
def _run(subcommand, *args, cwd=None, env=None, timeout=120):
cmd = [
baker_bin,
"-u", "testuser",
"-p", "test-pipeline",
"-b", "test-build-001",
"bare",
subcommand,
]
cmd.extend(args)
return run_cmd(cmd, cwd=cwd, env=env, timeout=timeout)
return _run
@pytest.fixture
def fixtures_dir():
"""Path to the test fixtures directory."""
return os.path.join(os.path.dirname(__file__), "fixtures")
@pytest.fixture
def bake_workspace():
"""Create /workspace for bake tests (baker writes temp scripts there)."""
existed = os.path.exists("/workspace")
if not existed:
os.makedirs("/workspace", exist_ok=True)
yield
if not existed:
shutil.rmtree("/workspace", ignore_errors=True)
-36
View File
@@ -1,36 +0,0 @@
#!/bin/bash
set -e
# Load LLM environment if available
if [ -f /workshop/llm.env ]; then
set -a
source /workshop/llm.env
set +a
fi
# Export test configuration
export BAKER_BIN=/workshop/workshop-baker
export EXAMPLES_DIR=/workshop/examples
export TEST_MODE=${TEST_MODE:-auto}
if [ "$TEST_MODE" = "manual" ]; then
echo "========================================"
echo " HoneyBiscuitWorkshop System Tests"
echo " Mode: MANUAL (interactive shell)"
echo "========================================"
echo ""
echo "Binary: $BAKER_BIN"
echo "Examples: $EXAMPLES_DIR"
echo ""
echo "Run tests manually:"
echo " cd /app && pytest tests/ -v"
echo ""
exec /bin/bash
else
echo "========================================"
echo " HoneyBiscuitWorkshop System Tests"
echo " Mode: AUTO (running pytest)"
echo "========================================"
cd /app
exec python -m pytest tests/ -v --tb=short --junitxml=/results/report.xml "$@"
fi
-6
View File
@@ -1,6 +0,0 @@
#!/bin/bash
[ -f /dev/shm/bakeexport.{{ name }} ] && . /dev/shm/bakeexport.{{ name }} || true
{{ condition }}
{{ main }}
{{ name }}
{{ export }}
-25
View File
@@ -1,25 +0,0 @@
#!/bin/bash
# @pipeline
step_one() {
echo "Step 1: Setup"
mkdir -p /tmp/bake-test-output
echo "step1-done" > /tmp/bake-test-output/step1.txt
}
# @pipeline
step_two() {
echo "Step 2: Build"
echo "step2-done" > /tmp/bake-test-output/step2.txt
}
# @pipeline
step_three() {
echo "Step 3: Verify"
if [ -f /tmp/bake-test-output/step1.txt ] && [ -f /tmp/bake-test-output/step2.txt ]; then
echo "All steps completed successfully"
else
echo "ERROR: Missing step outputs"
exit 1
fi
}
-5
View File
@@ -1,5 +0,0 @@
#!/bin/bash
echo "Hello from simple bake script"
echo "Task ID: $HBW_TASKID"
echo "Pipeline: $HBW_PIPELINE"
exit 0
-22
View File
@@ -1,22 +0,0 @@
#!/bin/bash
# @pipeline
init_workspace() {
echo "Initializing workspace"
mkdir -p /tmp/bake-after-test
echo "init" > /tmp/bake-after-test/status.txt
}
# @pipeline
# @after(init_workspace)
fetch_source() {
echo "Fetching source (depends on init_workspace)"
echo "fetched" > /tmp/bake-after-test/source.txt
}
# @pipeline
# @after(fetch_source)
build_project() {
echo "Building project (depends on fetch_source)"
echo "built" > /tmp/bake-after-test/build.txt
}
-4
View File
@@ -1,4 +0,0 @@
#!/bin/bash
echo "EARLY_HOOK_EXECUTED"
echo "Working directory: $(pwd)"
echo "User: $(whoami)"
-3
View File
@@ -1,3 +0,0 @@
#!/bin/bash
echo "LATE_HOOK_EXECUTED"
echo "Working directory: $(pwd)"
-6
View File
@@ -1,6 +0,0 @@
version: "0.0.1"
plugin: {}
notification: {}
artifact: []
cleanup:
policy: "auto"
-13
View File
@@ -1,13 +0,0 @@
version: "0.0.1"
plugin: {}
notification: {}
artifact: []
cleanup:
policy: "auto"
hooks:
early:
- name: "finalize-early"
command: "echo 'finalize early hook'"
late:
- name: "finalize-late"
command: "echo 'finalize late hook'"
-8
View File
@@ -1,8 +0,0 @@
version: "1.0"
environment:
builder: "baremetal"
bootstrap:
user: "vulcan"
workspace:
path: "/home/vulcan/workspace"
fallback: true
-15
View File
@@ -1,15 +0,0 @@
version: "1.0"
environment:
builder: "baremetal"
bootstrap:
user: "vulcan"
workspace:
path: "/home/vulcan/workspace"
fallback: true
hooks:
early:
- name: "test-early-hook"
command: "echo 'early hook executed'"
late:
- name: "test-late-hook"
command: "echo 'late hook executed'"
-3
View File
@@ -1,3 +0,0 @@
version: "0.5"
environment:
builder: "baremetal"
-8
View File
@@ -1,8 +0,0 @@
version: "1.0"
environment:
builder: "baremetal"
bootstrap:
user: "vulcan"
workspace:
path: "/home/vulcan/workspace"
fallback: true
-12
View File
@@ -1,12 +0,0 @@
version: "1.0"
environment:
builder: "baremetal"
bootstrap:
user: "vulcan"
workspace:
path: "/home/vulcan/workspace"
fallback: true
custom: |
#!/bin/sh
echo 'custom bootstrap executed'
mkdir -p /custom-workspace
-16
View File
@@ -1,16 +0,0 @@
version: "1.0"
environment:
builder: "baremetal"
bootstrap:
user: "vulcan"
workspace:
path: "/home/vulcan/workspace"
fallback: true
dependencies:
system:
pacman:
packages:
- "which"
user:
cargo:
- "ripgrep"
-13
View File
@@ -1,13 +0,0 @@
version: "1.0"
environment:
builder: "baremetal"
bootstrap:
user: "vulcan"
workspace:
path: "/home/vulcan/workspace"
fallback: true
envvars:
prebake:
TEST_PREBAKE_VAR: "prebake_value"
bake:
TEST_BAKE_VAR: "bake_value"
-17
View File
@@ -1,17 +0,0 @@
version: "1.0"
environment:
builder: "baremetal"
bootstrap:
user: "vulcan"
workspace:
path: "/home/vulcan/workspace"
fallback: true
hooks:
early:
- name: "test-early-hook"
command: "echo 'early hook executed'"
working_dir: "/workspace"
late:
- name: "test-late-hook"
command: "echo 'late hook executed'"
working_dir: "/workspace"
-14
View File
@@ -1,14 +0,0 @@
version: "1.0"
environment:
builder: "baremetal"
bootstrap:
user: "vulcan"
workspace:
path: "/home/vulcan/workspace"
fallback: true
dependencies:
system:
pacman:
packages:
- "curl"
- "wget"
-10
View File
@@ -1,10 +0,0 @@
version: "1.0"
environment:
builder: "baremetal"
bootstrap:
user: "vulcan"
workspace:
path: "/home/vulcan/workspace"
fallback: true
security:
drop_after: "Bootstrap"
-12
View File
@@ -1,12 +0,0 @@
"""
LLM-based log judgment system for system tests.
Uses LLM APIs to evaluate non-deterministic test outputs (logs, stderr, etc.)
where simple string matching is insufficient.
"""
from .client import LLMClient
from .cache import LLMCache
from .judge import judge_log, JudgmentResult
__all__ = ["LLMClient", "LLMCache", "judge_log", "JudgmentResult"]
-56
View File
@@ -1,56 +0,0 @@
"""
Disk-based cache for LLM API responses.
Uses SHA256 of (prompt + model + system_prompt) as cache key.
Cache files stored as JSON in ``.cache/`` directory.
"""
import hashlib
import json
from pathlib import Path
class LLMCache:
"""SHA256-based disk cache for LLM API responses."""
def __init__(self, cache_dir: str | None = None):
if cache_dir is None:
import os
cache_dir = os.path.join(os.path.dirname(__file__), ".cache")
self.cache_dir = Path(cache_dir)
self.cache_dir.mkdir(parents=True, exist_ok=True)
def _make_key(self, prompt: str, model: str, system_prompt: str = "") -> str:
"""Create deterministic cache key from request parameters."""
data = json.dumps(
{"prompt": prompt, "model": model, "system": system_prompt},
sort_keys=True,
)
return hashlib.sha256(data.encode()).hexdigest()
def get(self, prompt: str, model: str, system_prompt: str = "") -> dict | None:
"""Retrieve cached response if exists."""
key = self._make_key(prompt, model, system_prompt)
path = self.cache_dir / f"{key}.json"
if path.exists():
try:
with open(path) as f:
return json.load(f)
except (json.JSONDecodeError, IOError):
return None
return None
def set(
self, prompt: str, model: str, system_prompt: str, response: dict
) -> None:
"""Store response in cache."""
key = self._make_key(prompt, model, system_prompt)
path = self.cache_dir / f"{key}.json"
with open(path, "w") as f:
json.dump(response, f, ensure_ascii=False)
def clear(self) -> None:
"""Clear all cached responses."""
for f in self.cache_dir.glob("*.json"):
f.unlink()
-187
View File
@@ -1,187 +0,0 @@
"""
LLM client supporting both Anthropic and OpenAI APIs.
Prefers Anthropic format (per project convention).
Falls back to OpenAI format if Anthropic is unavailable.
"""
import os
import logging
from .cache import LLMCache
logger = logging.getLogger(__name__)
class LLMClient:
"""LLM client with caching support. Prefers Anthropic API."""
def __init__(self, cache: LLMCache | None = None):
self.cache = cache or LLMCache()
self._anthropic_client = None
self._openai_client = None
self._init_clients()
def _init_clients(self):
"""Initialize API clients based on available environment variables."""
# Anthropic (preferred)
anthropic_key = os.environ.get("ANTHROPIC_API_KEY")
anthropic_base = os.environ.get("ANTHROPIC_BASE_URL")
if anthropic_key:
try:
import anthropic
kwargs: dict = {"api_key": anthropic_key}
if anthropic_base:
kwargs["base_url"] = anthropic_base
self._anthropic_client = anthropic.Anthropic(**kwargs)
logger.info("Anthropic client initialized")
except ImportError:
logger.warning("anthropic package not installed")
# OpenAI (fallback)
openai_key = os.environ.get("OPENAI_API_KEY")
openai_base = os.environ.get("OPENAI_BASE_URL")
if openai_key:
try:
import openai
kwargs = {"api_key": openai_key}
if openai_base:
kwargs["base_url"] = openai_base
self._openai_client = openai.OpenAI(**kwargs)
logger.info("OpenAI client initialized")
except ImportError:
logger.warning("openai package not installed")
@property
def available(self) -> bool:
"""Check if any LLM client is available."""
return (
self._anthropic_client is not None or self._openai_client is not None
)
def complete(
self, prompt: str, system_prompt: str = "", max_tokens: int = 2000
) -> str:
"""Send a completion request, with caching.
Args:
prompt: User message.
system_prompt: System message (sets role / persona).
max_tokens: Max response tokens.
Returns:
LLM response text.
Raises:
RuntimeError: If no client is available.
"""
model = os.environ.get("MODEL", "deepseek-v4-flash")
# Check cache
cached = self.cache.get(prompt, model, system_prompt)
if cached:
logger.info("Cache hit for LLM request")
return cached["content"]
# Try Anthropic first
if self._anthropic_client:
content = self._call_anthropic(
prompt, system_prompt, model, max_tokens
)
elif self._openai_client:
content = self._call_openai(prompt, system_prompt, model, max_tokens)
else:
raise RuntimeError(
"No LLM client available. "
"Set ANTHROPIC_API_KEY or OPENAI_API_KEY."
)
# Cache the response
self.cache.set(prompt, model, system_prompt, {"content": content})
return content
# ------------------------------------------------------------------
# Private helpers
# ------------------------------------------------------------------
def _call_anthropic(
self, prompt: str, system_prompt: str, model: str, max_tokens: int
) -> str:
"""Call Anthropic API."""
kwargs: dict = {
"model": model,
"max_tokens": max_tokens,
"messages": [{"role": "user", "content": prompt}],
}
if system_prompt:
kwargs["system"] = system_prompt
# Only add reasoning_effort if the model supports it
reasoning_effort = os.environ.get("REASONING_EFFORT")
if reasoning_effort:
try:
kwargs["reasoning_effort"] = reasoning_effort
response = self._anthropic_client.messages.create(**kwargs)
except Exception as e:
if (
"reasoning_effort" in str(e).lower()
or "unexpected" in str(e).lower()
):
del kwargs["reasoning_effort"]
response = self._anthropic_client.messages.create(**kwargs)
else:
raise
else:
response = self._anthropic_client.messages.create(**kwargs)
# Extract text from response content blocks
if hasattr(response, "content") and response.content:
block = response.content[0]
if hasattr(block, "text"):
return block.text
if hasattr(block, "type") and block.type == "thinking":
for b in response.content:
if hasattr(b, "text"):
return b.text
return str(block)
return str(response)
def _call_openai(
self, prompt: str, system_prompt: str, model: str, max_tokens: int
) -> str:
"""Call OpenAI-compatible API."""
messages: list[dict] = []
if system_prompt:
messages.append({"role": "system", "content": system_prompt})
messages.append({"role": "user", "content": prompt})
kwargs: dict = {
"model": model,
"messages": messages,
"max_tokens": max_tokens,
}
# Only add reasoning_effort if supported
reasoning_effort = os.environ.get("REASONING_EFFORT")
if reasoning_effort:
try:
kwargs["reasoning_effort"] = reasoning_effort
response = self._openai_client.chat.completions.create(
**kwargs
)
except Exception as e:
if (
"reasoning_effort" in str(e).lower()
or "unexpected" in str(e).lower()
):
del kwargs["reasoning_effort"]
response = self._openai_client.chat.completions.create(
**kwargs
)
else:
raise
else:
response = self._openai_client.chat.completions.create(**kwargs)
return response.choices[0].message.content
-149
View File
@@ -1,149 +0,0 @@
"""
LLM-based log judgment for system tests.
Evaluates logs against expected behavior using LLM semantic analysis.
Returns structured pass/fail results with reasoning.
"""
import json
import re
import logging
from dataclasses import dataclass
from .client import LLMClient
logger = logging.getLogger(__name__)
JUDGE_SYSTEM_PROMPT = """\
You are a test result judge for a CI/CD system called HoneyBiscuitWorkshop.
Your job: Evaluate whether actual command output/logs match expected behavior.
RULES:
1. You MUST respond with valid JSON in this exact format:
{"verdict": "pass" | "fail", "reason": "brief explanation"}
2. "pass" = the output confirms the expected behavior occurred
3. "fail" = the output shows the expected behavior did NOT occur, or shows an error
4. Ignore irrelevant noise (timestamps, debug lines, unrelated warnings)
5. Focus ONLY on whether the specific expected behavior is confirmed or contradicted
6. Be strict: if the evidence is ambiguous, verdict is "fail"
7. Do NOT include any text outside the JSON object"""
@dataclass
class JudgmentResult:
"""Result of an LLM log judgment."""
verdict: str # "pass" or "fail"
reason: str
raw_response: str
@property
def passed(self) -> bool:
return self.verdict == "pass"
def _extract_relevant_logs(full_log: str, max_chars: int = 4000) -> str:
"""Extract relevant portions from logs, removing noise.
Keeps the last *max_chars* characters (most recent / relevant).
Removes common noise patterns.
"""
lines = full_log.strip().split("\n")
noise_patterns = [
r"^\d{4}-\d{2}-\d{2}T",
r"^DEBUG ",
r"^TRACE ",
]
filtered = [
line
for line in lines
if not any(re.match(p, line) for p in noise_patterns)
]
text = "\n".join(filtered)
if len(text) > max_chars:
text = "...\n" + text[-max_chars:]
return text
def judge_log(
operation: str,
expected_behavior: str,
actual_log: str,
client: LLMClient | None = None,
) -> JudgmentResult:
"""Judge whether actual logs match expected behavior.
Args:
operation: Description of what was attempted
(e.g. ``"prebake bootstrap stage"``).
expected_behavior: What should have happened
(e.g. ``"user vulcan was created"``).
actual_log: The actual stdout / stderr output.
client: ``LLMClient`` instance (creates default if ``None``).
Returns:
``JudgmentResult`` with verdict and reason.
"""
if client is None:
client = LLMClient()
if not client.available:
logger.warning("No LLM client available, returning fail verdict")
return JudgmentResult(
verdict="fail",
reason="No LLM client available for judgment",
raw_response="",
)
relevant_log = _extract_relevant_logs(actual_log)
prompt = f"""\
## Operation
{operation}
## Expected Behavior
{expected_behavior}
## Actual Output (relevant excerpts)
```
{relevant_log}
```
Evaluate: Does the actual output confirm the expected behavior occurred?
Respond with JSON only."""
try:
response = client.complete(
prompt=prompt,
system_prompt=JUDGE_SYSTEM_PROMPT,
max_tokens=500,
)
# Parse JSON response — handle potential markdown code blocks
text = response.strip()
if text.startswith("```"):
lines = text.split("\n")
text = "\n".join(lines[1:-1]) if len(lines) > 2 else text
result = json.loads(text)
return JudgmentResult(
verdict=result.get("verdict", "fail"),
reason=result.get("reason", "No reason provided"),
raw_response=response,
)
except json.JSONDecodeError as e:
logger.error(f"Failed to parse LLM response as JSON: {e}")
return JudgmentResult(
verdict="fail",
reason=f"LLM response was not valid JSON: {response[:200]}",
raw_response=response,
)
except Exception as e:
logger.error(f"LLM judgment failed: {e}")
return JudgmentResult(
verdict="fail",
reason=f"LLM call failed: {str(e)}",
raw_response="",
)
-34
View File
@@ -1,34 +0,0 @@
"""
Environment probes for verifying system state in workshop-baker tests.
Each probe is a standalone function that can be called from any test.
Probes return bool or raise AssertionError with descriptive message.
"""
from .user import user_exists, user_uid, user_gid, user_in_group, user_shell
from .file import (
file_exists,
dir_exists,
file_permissions,
file_contains,
file_owned_by,
)
from .package import package_installed, package_version
from .process import process_running, process_count
__all__ = [
"user_exists",
"user_uid",
"user_gid",
"user_in_group",
"user_shell",
"file_exists",
"dir_exists",
"file_permissions",
"file_contains",
"file_owned_by",
"package_installed",
"package_version",
"process_running",
"process_count",
]
-43
View File
@@ -1,43 +0,0 @@
"""File and directory existence / permission probes."""
import os
import stat
import grp
import pwd
def file_exists(path: str) -> bool:
"""Check if *path* exists and is a regular file."""
return os.path.isfile(path)
def dir_exists(path: str) -> bool:
"""Check if *path* exists and is a directory."""
return os.path.isdir(path)
def file_permissions(path: str) -> int:
"""Return the permission mode of *path* as an octal int (e.g. ``0o755``).
Raises ``FileNotFoundError`` if *path* doesn't exist.
"""
return stat.S_IMODE(os.stat(path).st_mode)
def file_contains(path: str, text: str) -> bool:
"""Check if a file contains the specified text."""
with open(path, "r", errors="replace") as f:
return text in f.read()
def file_owned_by(path: str, username: str) -> bool:
"""Check if a file is owned by the specified user."""
st = os.stat(path)
owner = pwd.getpwuid(st.st_uid).pw_name
return owner == username
def file_group(path: str) -> str:
"""Get the group owner of a file."""
st = os.stat(path)
return grp.getgrgid(st.st_gid).gr_name
-34
View File
@@ -1,34 +0,0 @@
"""Package installation probes for Arch Linux (pacman)."""
import subprocess
def package_installed(package_name: str) -> bool:
"""Check if a pacman package is installed."""
try:
result = subprocess.run(
["pacman", "-Q", package_name],
capture_output=True,
text=True,
)
return result.returncode == 0
except FileNotFoundError:
return False
def package_version(package_name: str) -> str | None:
"""Get installed version of a package, or ``None`` if not installed."""
try:
result = subprocess.run(
["pacman", "-Q", package_name],
capture_output=True,
text=True,
)
if result.returncode == 0:
# Output format: "package_name version"
parts = result.stdout.strip().split()
if len(parts) >= 2:
return parts[1]
return None
except FileNotFoundError:
return None
-44
View File
@@ -1,44 +0,0 @@
"""Process existence probes."""
import os
import subprocess
def process_running(process_name: str) -> bool:
"""Check if a process with the given name is running."""
try:
result = subprocess.run(
["pgrep", "-x", process_name],
capture_output=True,
text=True,
)
return result.returncode == 0
except FileNotFoundError:
# Fall back to /proc scan if pgrep not available
try:
for pid_dir in os.listdir("/proc"):
if pid_dir.isdigit():
try:
with open(f"/proc/{pid_dir}/comm") as f:
if f.read().strip() == process_name:
return True
except (IOError, OSError):
continue
except OSError:
pass
return False
def process_count(process_name: str) -> int:
"""Count processes with the given name."""
try:
result = subprocess.run(
["pgrep", "-c", "-x", process_name],
capture_output=True,
text=True,
)
if result.returncode == 0:
return int(result.stdout.strip())
return 0
except (FileNotFoundError, ValueError):
return 0
-37
View File
@@ -1,37 +0,0 @@
"""User existence and attribute probes."""
import pwd
import grp
def user_exists(username: str) -> bool:
"""Check if a system user exists."""
try:
pwd.getpwnam(username)
return True
except KeyError:
return False
def user_uid(username: str) -> int:
"""Get the UID of a user. Raises KeyError if user doesn't exist."""
return pwd.getpwnam(username).pw_uid
def user_gid(username: str) -> int:
"""Get the primary GID of a user. Raises KeyError if user doesn't exist."""
return pwd.getpwnam(username).pw_gid
def user_in_group(username: str, groupname: str) -> bool:
"""Check if a user is a member of a group."""
try:
group = grp.getgrnam(groupname)
return username in group.gr_mem
except KeyError:
return False
def user_shell(username: str) -> str:
"""Get the login shell of a user. Raises KeyError if user doesn't exist."""
return pwd.getpwnam(username).pw_shell
-6
View File
@@ -1,6 +0,0 @@
pytest>=8.0.0
pytest-timeout>=2.2.0
pytest-xdist>=3.5.0
anthropic>=0.40.0,<0.90.0
openai>=1.50.0,<2.0.0
pyyaml>=6.0
-129
View File
@@ -1,129 +0,0 @@
import os
import pytest
FIXTURES = os.path.join(os.path.dirname(__file__), "fixtures", "bake")
@pytest.mark.bake
class TestBakeSimpleScript:
def test_simple_script_runs(self, run_baker, bake_workspace, work_dir):
prebake_cfg = os.path.join(
os.path.dirname(__file__), "fixtures", "prebake", "minimal.yml"
)
script = os.path.join(FIXTURES, "simple.sh")
bake_base = os.path.join(FIXTURES, "bake_base.sh")
result = run_baker(
"bake", script,
"--bake-base", bake_base,
"--prebake", prebake_cfg,
cwd=work_dir,
timeout=60,
)
assert result["success"], f"Bake failed: {result['stderr']}"
assert "Hello from simple bake script" in result["stdout"]
def test_simple_script_sets_env_vars(self, run_baker, bake_workspace, work_dir):
prebake_cfg = os.path.join(
os.path.dirname(__file__), "fixtures", "prebake", "minimal.yml"
)
script = os.path.join(FIXTURES, "simple.sh")
bake_base = os.path.join(FIXTURES, "bake_base.sh")
result = run_baker(
"bake", script,
"--bake-base", bake_base,
"--prebake", prebake_cfg,
cwd=work_dir,
timeout=60,
)
assert result["success"]
assert "test-pipeline" in result["stdout"]
@pytest.mark.bake
class TestBakePipeline:
def test_pipeline_functions_execute(self, run_baker, bake_workspace, work_dir):
prebake_cfg = os.path.join(
os.path.dirname(__file__), "fixtures", "prebake", "minimal.yml"
)
script = os.path.join(FIXTURES, "pipeline.sh")
bake_base = os.path.join(FIXTURES, "bake_base.sh")
result = run_baker(
"bake", script,
"--bake-base", bake_base,
"--prebake", prebake_cfg,
cwd=work_dir,
timeout=60,
)
assert result["success"], f"Pipeline bake failed: {result['stderr']}"
assert os.path.exists("/tmp/bake-test-output/step1.txt")
assert os.path.exists("/tmp/bake-test-output/step2.txt")
def test_pipeline_step_outputs_correct(self, run_baker, bake_workspace, work_dir):
prebake_cfg = os.path.join(
os.path.dirname(__file__), "fixtures", "prebake", "minimal.yml"
)
script = os.path.join(FIXTURES, "pipeline.sh")
bake_base = os.path.join(FIXTURES, "bake_base.sh")
result = run_baker(
"bake", script,
"--bake-base", bake_base,
"--prebake", prebake_cfg,
cwd=work_dir,
timeout=60,
)
assert result["success"]
with open("/tmp/bake-test-output/step1.txt") as f:
assert f.read().strip() == "step1-done"
with open("/tmp/bake-test-output/step2.txt") as f:
assert f.read().strip() == "step2-done"
@pytest.mark.bake
class TestBakeAfterDependencies:
def test_after_ordering(self, run_baker, bake_workspace, work_dir):
prebake_cfg = os.path.join(
os.path.dirname(__file__), "fixtures", "prebake", "minimal.yml"
)
script = os.path.join(FIXTURES, "with_after.sh")
bake_base = os.path.join(FIXTURES, "bake_base.sh")
result = run_baker(
"bake", script,
"--bake-base", bake_base,
"--prebake", prebake_cfg,
cwd=work_dir,
timeout=60,
)
assert result["success"], f"@after bake failed: {result['stderr']}"
assert os.path.exists("/tmp/bake-after-test/status.txt")
assert os.path.exists("/tmp/bake-after-test/source.txt")
assert os.path.exists("/tmp/bake-after-test/build.txt")
@pytest.mark.bake
class TestBakeStatusFiles:
def test_status_written_after_bake(self, run_baker, bake_workspace, work_dir):
prebake_cfg = os.path.join(
os.path.dirname(__file__), "fixtures", "prebake", "minimal.yml"
)
script = os.path.join(FIXTURES, "simple.sh")
bake_base = os.path.join(FIXTURES, "bake_base.sh")
result = run_baker(
"bake", script,
"--bake-base", bake_base,
"--prebake", prebake_cfg,
cwd=work_dir,
timeout=60,
)
assert result["success"]
assert os.path.exists("/tmp/.hbwstatus")
-77
View File
@@ -1,77 +0,0 @@
import os
import pytest
from probes import file
from llm import judge_log, LLMClient
FIXTURES = os.path.join(os.path.dirname(__file__), "fixtures", "finalize")
@pytest.mark.finalize
class TestFinalizeMinimal:
def test_minimal_finalize_succeeds(self, run_baker, work_dir):
config = os.path.join(FIXTURES, "minimal.yml")
result = run_baker("finalize", config, cwd=work_dir, timeout=60)
assert result["success"], f"Minimal finalize failed: {result['stderr']}"
@pytest.mark.finalize
class TestFinalizeHooks:
def test_early_hook_executes(self, run_baker, work_dir):
config = os.path.join(FIXTURES, "with_hooks.yml")
result = run_baker("finalize", config, cwd=work_dir, timeout=60)
assert result["success"]
assert "finalize early hook" in result["stdout"] or "finalize early hook" in result["stderr"]
def test_late_hook_executes(self, run_baker, work_dir):
config = os.path.join(FIXTURES, "with_hooks.yml")
result = run_baker("finalize", config, cwd=work_dir, timeout=60)
assert result["success"]
assert "finalize late hook" in result["stdout"] or "finalize late hook" in result["stderr"]
@pytest.mark.finalize
class TestFinalizeStatus:
def test_finalize_writes_status(self, run_baker, work_dir):
config = os.path.join(FIXTURES, "minimal.yml")
result = run_baker("finalize", config, cwd=work_dir, timeout=60)
assert result["success"]
assert os.path.exists("/tmp/.hbwstatus")
@pytest.mark.finalize
@pytest.mark.llm
class TestFinalizeWithLLM:
def test_finalize_hooks_via_llm(self, run_baker, llm_env, work_dir):
if not llm_env:
pytest.skip("LLM environment not configured")
config = os.path.join(FIXTURES, "with_hooks.yml")
result = run_baker("finalize", config, cwd=work_dir, timeout=60)
combined_log = result["stdout"] + "\n" + result["stderr"]
judgment = judge_log(
operation="finalize with early and late hooks",
expected_behavior="Both early and late hooks executed successfully, "
"printing 'finalize early hook' and 'finalize late hook'",
actual_log=combined_log,
)
assert judgment.passed, f"LLM judgment failed: {judgment.reason}"
@pytest.mark.finalize
class TestFinalizePlugins:
@pytest.mark.xfail(reason="Plugin download requires network/git access", strict=False)
def test_plugin_download(self, run_baker, work_dir):
pass
@pytest.mark.xfail(reason="Artifact stage not yet implemented", strict=False)
def test_artifact_collection(self, run_baker, work_dir):
pass
-261
View File
@@ -1,261 +0,0 @@
import os
import json
import pytest
from probes import user, file, package
FIXTURES = os.path.join(os.path.dirname(__file__), "fixtures", "prebake")
@pytest.mark.prebake
class TestPrebakeConfigValidation:
def test_invalid_version_fails(self, run_baker, work_dir):
config = os.path.join(FIXTURES, "invalid_version.yml")
result = run_baker("prebake", config, cwd=work_dir, timeout=60)
assert not result["success"], f"Expected prebake to fail with invalid version"
assert "version" in result["stderr"].lower() or "Version" in result["stderr"]
def test_missing_required_field_fails(self, run_baker, work_dir):
missing_env = os.path.join(work_dir, "missing_env.yml")
with open(missing_env, "w") as f:
f.write('version: "1.0"\n')
result = run_baker("prebake", missing_env, cwd=work_dir, timeout=60)
assert not result["success"]
def test_minimal_config_succeeds(self, run_baker, work_dir):
config = os.path.join(FIXTURES, "minimal.yml")
result = run_baker("prebake", config, cwd=work_dir, timeout=60)
assert result["success"], f"Minimal prebake failed: {result['stderr']}"
@pytest.mark.prebake
class TestPrebakeBootstrap:
def test_bootstrap_creates_vulcan_user(self, run_baker, work_dir):
config = os.path.join(FIXTURES, "bootstrap_only.yml")
result = run_baker("prebake", config, cwd=work_dir, timeout=60)
assert result["success"], f"prebake failed: {result['stderr']}"
assert user.user_exists("vulcan")
def test_bootstrap_creates_workspace(self, run_baker, work_dir):
config = os.path.join(FIXTURES, "bootstrap_only.yml")
result = run_baker("prebake", config, cwd=work_dir, timeout=60)
assert result["success"]
assert file.dir_exists("/home/vulcan/workspace")
def test_bootstrap_creates_symlink(self, run_baker, work_dir):
config = os.path.join(FIXTURES, "bootstrap_only.yml")
result = run_baker("prebake", config, cwd=work_dir, timeout=60)
assert result["success"]
assert os.path.islink("/workspace")
def test_bootstrap_generates_bootstrap_script(self, run_baker, work_dir):
config = os.path.join(FIXTURES, "bootstrap_only.yml")
result = run_baker("prebake", config, cwd=work_dir, timeout=60)
assert result["success"]
assert file.file_exists("/bootstrap.sh")
def test_bootstrap_script_permissions(self, run_baker, work_dir):
config = os.path.join(FIXTURES, "bootstrap_only.yml")
result = run_baker("prebake", config, cwd=work_dir, timeout=60)
assert result["success"]
mode = file.file_permissions("/bootstrap.sh")
assert mode & 0o111, f"/bootstrap.sh not executable, mode: {oct(mode)}"
def test_bootstrap_script_contains_user_creation(self, run_baker, work_dir):
config = os.path.join(FIXTURES, "bootstrap_only.yml")
result = run_baker("prebake", config, cwd=work_dir, timeout=60)
assert result["success"]
assert file.file_exists("/bootstrap.sh")
assert file.file_contains("/bootstrap.sh", "useradd")
def test_bootstrap_script_contains_workspace_setup(self, run_baker, work_dir):
config = os.path.join(FIXTURES, "bootstrap_only.yml")
result = run_baker("prebake", config, cwd=work_dir, timeout=60)
assert result["success"]
assert file.file_contains("/bootstrap.sh", "mkdir -p /home/vulcan/workspace")
def test_bootstrap_with_pacman_creates_sudoers(self, run_baker, work_dir):
config = os.path.join(FIXTURES, "with_pacman.yml")
result = run_baker("prebake", config, cwd=work_dir, timeout=120)
assert result["success"], f"prebake failed: {result['stderr']}"
assert file.file_exists("/etc/sudoers.d/workshop")
def test_custom_bootstrap_executes(self, run_baker, work_dir):
config = os.path.join(FIXTURES, "with_custom_bootstrap.yml")
result = run_baker("prebake", config, cwd=work_dir, timeout=60)
assert result["success"], f"Custom bootstrap failed: {result['stderr']}"
assert file.dir_exists("/custom-workspace")
@pytest.mark.prebake
class TestPrebakeDryRun:
def test_dry_run_does_not_create_user(self, run_baker, work_dir):
config = os.path.join(FIXTURES, "bootstrap_only.yml")
env = {"HBW_DRY_RUN": "true"}
result = run_baker("prebake", config, cwd=work_dir, env=env, timeout=60)
assert result["success"], f"Dry-run failed: {result['stderr']}"
def test_dry_run_logs_script_content(self, run_baker, work_dir):
config = os.path.join(FIXTURES, "bootstrap_only.yml")
env = {"HBW_DRY_RUN": "true", "RUST_LOG": "info"}
result = run_baker("prebake", config, cwd=work_dir, env=env, timeout=60)
assert result["success"]
combined = result["stdout"] + result["stderr"]
assert "DRY_RUN" in combined or "dry_run" in combined.lower()
@pytest.mark.prebake
class TestPrebakeHooks:
#@pytest.mark.xfail(reason="Hook execution fails with Permission denied in container environment", strict=False)
def test_early_hook_executes(self, run_baker, work_dir):
config = os.path.join(FIXTURES, "with_hooks.yml")
result = run_baker("prebake", config, cwd=work_dir, timeout=60)
assert result["success"], f"prebake with hooks failed: {result['stderr']}"
combined = result["stdout"] + result["stderr"]
assert "early hook executed" in combined
#@pytest.mark.xfail(reason="Hook execution fails with Permission denied in container environment", strict=False)
def test_late_hook_executes(self, run_baker, work_dir):
config = os.path.join(FIXTURES, "with_hooks.yml")
result = run_baker("prebake", config, cwd=work_dir, timeout=60)
assert result["success"], f"prebake with hooks failed: {result['stderr']}"
combined = result["stdout"] + result["stderr"]
assert "late hook executed" in combined
#@pytest.mark.xfail(reason="Hook execution fails with Permission denied in container environment", strict=False)
def test_hooks_run_in_order(self, run_baker, work_dir):
config = os.path.join(FIXTURES, "with_hooks.yml")
result = run_baker("prebake", config, cwd=work_dir, timeout=60)
assert result["success"]
combined = result["stdout"] + result["stderr"]
early_pos = combined.find("early hook executed")
late_pos = combined.find("late hook executed")
assert early_pos != -1 and late_pos != -1
assert early_pos < late_pos, "Early hook should execute before late hook"
def test_empty_hooks_succeed(self, run_baker, work_dir):
config = os.path.join(FIXTURES, "minimal.yml")
result = run_baker("prebake", config, cwd=work_dir, timeout=60)
assert result["success"], f"prebake with no hooks failed: {result['stderr']}"
@pytest.mark.prebake
class TestPrebakeDepsSystem:
@pytest.mark.xfail(reason="Package installation may fail in test environment", strict=False)
def test_pacman_packages_installed(self, run_baker, work_dir):
config = os.path.join(FIXTURES, "with_pacman.yml")
result = run_baker("prebake", config, cwd=work_dir, timeout=120)
assert result["success"], f"prebake failed: {result['stderr']}"
assert package.package_installed("curl")
assert package.package_installed("wget")
def test_pacman_mirror_change(self, run_baker, work_dir):
mirror_config = os.path.join(work_dir, "mirror.yml")
with open(mirror_config, "w") as f:
f.write('''
version: "1.0"
environment:
builder: "baremetal"
bootstrap:
user: "vulcan"
dependencies:
system:
pacman:
packages: ["which"]
mirror: "https://mirrors.tuna.tsinghua.edu.cn/archlinux/$repo/os/$arch"
''')
result = run_baker("prebake", mirror_config, cwd=work_dir, timeout=120)
assert result["success"], f"prebake with mirror failed: {result['stderr']}"
@pytest.mark.prebake
class TestPrebakeEnvVars:
def test_prebake_envvars_injected(self, run_baker, work_dir):
hook_config = os.path.join(work_dir, "envvar_hook.yml")
with open(hook_config, "w") as f:
f.write('''
version: "1.0"
environment:
builder: "baremetal"
bootstrap:
user: "vulcan"
envvars:
prebake:
TEST_INJECTED: "injected_value"
hooks:
early:
- name: "check-env"
command: "echo $TEST_INJECTED"
''')
result = run_baker("prebake", hook_config, cwd=work_dir, timeout=60)
assert result["success"], f"prebake with envvars failed: {result['stderr']}"
combined = result["stdout"] + result["stderr"]
assert "injected_value" in combined
@pytest.mark.prebake
class TestPrebakeSecurity:
#@pytest.mark.xfail(reason="Privilege drop may fail in container without proper setup", strict=False)
def test_privilege_drop_after_bootstrap(self, run_baker, work_dir):
config = os.path.join(FIXTURES, "with_security.yml")
result = run_baker("prebake", config, cwd=work_dir, timeout=60)
assert result["success"], f"prebake with security failed: {result['stderr']}"
@pytest.mark.prebake
class TestPrebakeStatus:
def test_status_files_written(self, run_baker, work_dir):
config = os.path.join(FIXTURES, "bootstrap_only.yml")
result = run_baker("prebake", config, cwd=work_dir, timeout=60)
assert result["success"]
assert os.path.exists("/tmp/.hbwstatus")
def test_status_contains_bootstrap_stage(self, run_baker, work_dir):
config = os.path.join(FIXTURES, "bootstrap_only.yml")
result = run_baker("prebake", config, cwd=work_dir, timeout=60)
assert result["success"]
with open("/tmp/.hbwstatus", "r") as f:
lines = f.readlines()
assert len(lines) > 0
bootstrap_lines = [l for l in lines if json.loads(l).get("name") == "bootstrap"]
assert len(bootstrap_lines) > 0, "No bootstrap stage found in status file"
data = json.loads(bootstrap_lines[-1])
assert data.get("phase") == "Prebake"
assert data.get("result") == "Success"
def test_status_contains_duration(self, run_baker, work_dir):
config = os.path.join(FIXTURES, "bootstrap_only.yml")
result = run_baker("prebake", config, cwd=work_dir, timeout=60)
assert result["success"]
with open("/tmp/.hbwstatus", "r") as f:
data = json.loads(f.readline())
assert "duration_ms" in data
assert isinstance(data["duration_ms"], int)
def test_status_contains_timestamps(self, run_baker, work_dir):
config = os.path.join(FIXTURES, "bootstrap_only.yml")
result = run_baker("prebake", config, cwd=work_dir, timeout=60)
assert result["success"]
with open("/tmp/.hbwstatus", "r") as f:
data = json.loads(f.readline())
assert "started_at" in data
assert "finished_at" in data
def test_status_written_for_each_stage(self, run_baker, work_dir):
config = os.path.join(FIXTURES, "with_pacman.yml")
result = run_baker("prebake", config, cwd=work_dir, timeout=120)
assert result["success"], f"prebake failed: {result['stderr']}"
with open("/tmp/.hbwstatus", "r") as f:
lines = f.readlines()
names = [json.loads(line).get("name") for line in lines]
assert "bootstrap" in names
assert "depssystem" in names
+107
View File
@@ -0,0 +1,107 @@
# This file is automatically @generated by Cargo.
# It is not intended for manual editing.
version = 4
[[package]]
name = "itoa"
version = "1.0.18"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "8f42a60cbdf9a97f5d2305f08a87dc4e09308d1276d28c869c684d7777685682"
[[package]]
name = "memchr"
version = "2.8.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "f8ca58f447f06ed17d5fc4043ce1b10dd205e060fb3ce5b979b8ed8e59ff3f79"
[[package]]
name = "proc-macro2"
version = "1.0.106"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "8fd00f0bb2e90d81d1044c2b32617f68fcb9fa3bb7640c23e9c748e53fb30934"
dependencies = [
"unicode-ident",
]
[[package]]
name = "quote"
version = "1.0.45"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "41f2619966050689382d2b44f664f4bc593e129785a36d6ee376ddf37259b924"
dependencies = [
"proc-macro2",
]
[[package]]
name = "serde"
version = "1.0.228"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "9a8e94ea7f378bd32cbbd37198a4a91436180c5bb472411e48b5ec2e2124ae9e"
dependencies = [
"serde_core",
"serde_derive",
]
[[package]]
name = "serde_core"
version = "1.0.228"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "41d385c7d4ca58e59fc732af25c3983b67ac852c1a25000afe1175de458b67ad"
dependencies = [
"serde_derive",
]
[[package]]
name = "serde_derive"
version = "1.0.228"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "d540f220d3187173da220f885ab66608367b6574e925011a9353e4badda91d79"
dependencies = [
"proc-macro2",
"quote",
"syn",
]
[[package]]
name = "serde_json"
version = "1.0.149"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "83fc039473c5595ace860d8c4fafa220ff474b3fc6bfdb4293327f1a37e94d86"
dependencies = [
"itoa",
"memchr",
"serde",
"serde_core",
"zmij",
]
[[package]]
name = "syn"
version = "2.0.117"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "e665b8803e7b1d2a727f4023456bbbbe74da67099c585258af0ad9c5013b9b99"
dependencies = [
"proc-macro2",
"quote",
"unicode-ident",
]
[[package]]
name = "unicode-ident"
version = "1.0.24"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "e6e4313cd5fcd3dad5cafa179702e2b244f760991f45397d14d4ebf38247da75"
[[package]]
name = "workshop-baker-params"
version = "0.1.0"
dependencies = [
"serde",
"serde_json",
]
[[package]]
name = "zmij"
version = "1.0.21"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "b8848ee67ecc8aedbaf3e4122217aff892639231befc6a1b58d29fff4c2cabaa"
+8
View File
@@ -0,0 +1,8 @@
[package]
name = "workshop-baker-params"
version = "0.1.0"
edition = "2024"
[dependencies]
serde = { version = "1.0.228", features = ["derive"] }
serde_json = "1.0.149"
+24
View File
@@ -0,0 +1,24 @@
use crate::apply_field;
use serde::{Deserialize, Serialize};
use crate::{ParamVal, Writer, Overridden, traits::MergeParams};
#[derive(Debug, Clone, Serialize, Deserialize)]
#[serde(deny_unknown_fields)]
pub struct BakeParams {
#[serde(default)] pub workspace: ParamVal<String>,
}
impl MergeParams for BakeParams {
fn defaults() -> Self { Self { workspace: ParamVal::new("/workspace".into()) } }
fn apply(&mut self, i: Self, w: Writer, ov: &mut Overridden, p: &str) {
apply_field!(self.workspace, i.workspace, w, ov, p, "workspace");
}
}
impl Default for BakeParams { fn default() -> Self { Self::defaults() } }
#[derive(Debug, Clone)]
pub struct BakeSettings { pub workspace: String }
impl Default for BakeSettings { fn default() -> Self { BakeParams::defaults().into() } }
impl From<&BakeParams> for BakeSettings { fn from(p: &BakeParams) -> Self { Self { workspace: p.workspace.value.clone() } } }
impl From<BakeParams> for BakeSettings { fn from(p: BakeParams) -> Self { (&p).into() } }
+23
View File
@@ -0,0 +1,23 @@
use crate::apply_field;
use serde::{Deserialize, Serialize};
use crate::{ParamVal, Writer, Overridden, traits::MergeParams};
#[derive(Debug, Clone, Serialize, Deserialize)]
#[serde(deny_unknown_fields)]
pub struct FinalizeParams {
#[serde(default)] pub artifact_retention_days: ParamVal<u32>,
}
impl MergeParams for FinalizeParams {
fn defaults() -> Self { Self { artifact_retention_days: ParamVal::new(7) } }
fn apply(&mut self, i: Self, w: Writer, ov: &mut Overridden, p: &str) {
apply_field!(self.artifact_retention_days, i.artifact_retention_days, w, ov, p, "artifact_retention_days");
}
}
impl Default for FinalizeParams { fn default() -> Self { Self::defaults() } }
#[derive(Debug, Clone)]
pub struct FinalizeSettings { pub artifact_retention_days: u32 }
impl Default for FinalizeSettings { fn default() -> Self { FinalizeParams::defaults().into() } }
impl From<&FinalizeParams> for FinalizeSettings { fn from(p: &FinalizeParams) -> Self { Self { artifact_retention_days: p.artifact_retention_days.value } } }
impl From<FinalizeParams> for FinalizeSettings { fn from(p: FinalizeParams) -> Self { (&p).into() } }
+168
View File
@@ -0,0 +1,168 @@
pub mod notification;
pub mod prebake;
pub mod bake;
pub mod finalize;
pub mod writer;
pub mod param;
pub mod traits;
pub use notification::{NotificationParams, NotificationSettings};
pub use prebake::{PrebakeParams, PrebakeSettings};
pub use bake::{BakeParams, BakeSettings};
pub use finalize::{FinalizeParams, FinalizeSettings};
pub use writer::Writer;
pub use param::ParamVal;
pub use traits::MergeParams;
use serde::{Deserialize, Serialize};
use std::collections::HashMap;
/// Root parameter tree. Each component gets its own subtree.
/// Used for both IPC (Serialized to JSON) and in-process merging.
#[derive(Debug, Clone, Serialize, Deserialize, Default)]
pub struct Params {
#[serde(default, skip_serializing_if = "Option::is_none")]
pub notification: Option<NotificationParams>,
#[serde(default, skip_serializing_if = "Option::is_none")]
pub prebake: Option<PrebakeParams>,
#[serde(default, skip_serializing_if = "Option::is_none")]
pub bake: Option<BakeParams>,
#[serde(default, skip_serializing_if = "Option::is_none")]
pub finalize: Option<FinalizeParams>,
}
/// Apply and merge helpers.
impl Params {
/// Merge a layer from a specific writer into the current params.
/// Returns entries that were overridden (for debug).
pub fn apply(&mut self, incoming: Params, writer: Writer) -> Overridden {
let mut ov = Overridden::new();
if let Some(l) = incoming.notification {
if let Some(ref mut b) = self.notification {
b.apply(l, writer, &mut ov, "notification.");
} else {
self.notification = Some(l);
}
}
if let Some(l) = incoming.prebake {
if let Some(ref mut b) = self.prebake {
b.apply(l, writer, &mut ov, "prebake.");
} else {
self.prebake = Some(l);
}
}
if let Some(l) = incoming.bake {
if let Some(ref mut b) = self.bake {
b.apply(l, writer, &mut ov, "bake.");
} else {
self.bake = Some(l);
}
}
if let Some(l) = incoming.finalize {
if let Some(ref mut b) = self.finalize {
b.apply(l, writer, &mut ov, "finalize.");
} else {
self.finalize = Some(l);
}
}
ov
}
/// Convert to baker-consumable settings. All params guaranteed to have values.
pub fn to_settings(&self) -> Result<AllSettings, String> {
Ok(AllSettings {
notification: self.notification.as_ref().map(NotificationSettings::from).unwrap_or_default(),
prebake: self.prebake.as_ref().map(PrebakeSettings::from).unwrap_or_default(),
bake: self.bake.as_ref().map(BakeSettings::from).unwrap_or_default(),
finalize: self.finalize.as_ref().map(FinalizeSettings::from).unwrap_or_default(),
})
}
}
/// Plain-value settings for baker consumption. No metadata, no Option.
#[derive(Debug, Clone)]
pub struct AllSettings {
pub notification: NotificationSettings,
pub prebake: PrebakeSettings,
pub bake: BakeSettings,
pub finalize: FinalizeSettings,
}
/// Debug — values that were overridden during merge, keyed by dot-path.
#[derive(Debug, Clone, Default)]
pub struct Overridden {
pub entries: std::collections::HashMap<String, serde_json::Value>,
}
impl Overridden {
pub fn new() -> Self { Self { entries: HashMap::new() } }
pub fn is_empty(&self) -> bool { self.entries.is_empty() }
}
#[cfg(test)]
mod tests {
use super::*;
#[test]
fn test_empty_params_serializes_empty() {
let p = Params::default();
let json = serde_json::to_string(&p).unwrap();
assert_eq!(json, "{}");
}
#[test]
fn test_notification_params_serialize() {
let mut p = Params::default();
p.notification = Some(NotificationParams::defaults());
let json = serde_json::to_string_pretty(&p).unwrap();
assert!(json.contains("max_lives"));
assert!(json.contains("3"));
}
#[test]
fn test_apply_courier_overrides_default() {
let mut base = Params::default();
base.notification = Some(NotificationParams::defaults());
let incoming = Params {
notification: Some(NotificationParams {
max_lives: ParamVal::with(5, Writer::Courier),
..Default::default()
}),
..Default::default()
};
let ov = base.apply(incoming, Writer::Courier);
let s = base.to_settings().unwrap();
assert_eq!(s.notification.max_lives, 5);
assert!(ov.is_empty()); // default had no writer, not an "override"
}
#[test]
fn test_apply_upstream_wins() {
let mut base = Params::default();
base.notification = Some(NotificationParams {
max_lives: ParamVal::with(3, Writer::Base),
..Default::default()
});
let incoming = Params {
notification: Some(NotificationParams {
max_lives: ParamVal::with(5, Writer::Courier),
..Default::default()
}),
..Default::default()
};
let ov = base.apply(incoming, Writer::Courier);
let s = base.to_settings().unwrap();
// base(3) > courier(2) — rejected, base stays
assert_eq!(s.notification.max_lives, 3);
assert!(ov.is_empty()); // nothing overridden
}
#[test]
fn test_deny_unknown_fields() {
let bad = r#"{"max_lives":{"value":5},"ghost_param":{"value":1}}"#;
let result: Result<NotificationParams, _> = serde_json::from_str(bad);
assert!(result.is_err()); // unknown field should be denied
}
}
+65
View File
@@ -0,0 +1,65 @@
use crate::apply_field;
use serde::{Deserialize, Serialize};
use crate::{ParamVal, Writer, Overridden, traits::MergeParams};
#[derive(Debug, Clone, Serialize, Deserialize)]
#[serde(deny_unknown_fields)]
pub struct NotificationParams {
#[serde(default)] pub max_lives: ParamVal<u32>,
#[serde(default)] pub timeout_ms: ParamVal<u64>,
#[serde(default)] pub retry_backoff: ParamVal<f64>,
#[serde(default)] pub retry_delay_seconds: ParamVal<f64>,
#[serde(default)] pub max_retry_delay_seconds: ParamVal<f64>,
#[serde(default)] pub life_impact_factor: ParamVal<f64>,
#[serde(default)] pub batch_period: ParamVal<u32>,
#[serde(default)] pub batch_watermark: ParamVal<u32>,
#[serde(default)] pub template_ref_depth: ParamVal<u32>,
}
impl MergeParams for NotificationParams {
fn defaults() -> Self {
Self {
max_lives: ParamVal::new(3), timeout_ms: ParamVal::new(30000),
retry_backoff: ParamVal::new(2.0), retry_delay_seconds: ParamVal::new(5.0),
max_retry_delay_seconds: ParamVal::new(300.0),
life_impact_factor: ParamVal::new(0.0),
batch_period: ParamVal::new(30), batch_watermark: ParamVal::new(5),
template_ref_depth: ParamVal::new(10),
}
}
fn apply(&mut self, incoming: Self, writer: Writer, ov: &mut Overridden, prefix: &str) {
apply_field!(self.max_lives, incoming.max_lives, writer, ov, prefix, "max_lives");
apply_field!(self.timeout_ms, incoming.timeout_ms, writer, ov, prefix, "timeout_ms");
apply_field!(self.retry_backoff, incoming.retry_backoff, writer, ov, prefix, "retry_backoff");
apply_field!(self.retry_delay_seconds, incoming.retry_delay_seconds, writer, ov, prefix, "retry_delay_seconds");
apply_field!(self.max_retry_delay_seconds, incoming.max_retry_delay_seconds, writer, ov, prefix, "max_retry_delay_seconds");
apply_field!(self.life_impact_factor, incoming.life_impact_factor, writer, ov, prefix, "life_impact_factor");
apply_field!(self.batch_period, incoming.batch_period, writer, ov, prefix, "batch_period");
apply_field!(self.batch_watermark, incoming.batch_watermark, writer, ov, prefix, "batch_watermark");
apply_field!(self.template_ref_depth, incoming.template_ref_depth, writer, ov, prefix, "template_ref_depth");
}
}
impl Default for NotificationParams { fn default() -> Self { Self::defaults() } }
#[derive(Debug, Clone)]
pub struct NotificationSettings {
pub max_lives: u32, pub timeout_ms: u64, pub retry_backoff: f64,
pub retry_delay_seconds: f64, pub max_retry_delay_seconds: f64,
pub life_impact_factor: f64,
pub batch_period: u32, pub batch_watermark: u32, pub template_ref_depth: u32,
}
impl Default for NotificationSettings { fn default() -> Self { NotificationParams::defaults().into() } }
impl From<&NotificationParams> for NotificationSettings {
fn from(p: &NotificationParams) -> Self { Self {
max_lives: p.max_lives.value, timeout_ms: p.timeout_ms.value,
retry_backoff: p.retry_backoff.value, retry_delay_seconds: p.retry_delay_seconds.value,
max_retry_delay_seconds: p.max_retry_delay_seconds.value,
life_impact_factor: p.life_impact_factor.value,
batch_period: p.batch_period.value, batch_watermark: p.batch_watermark.value,
template_ref_depth: p.template_ref_depth.value,
}}
}
impl From<NotificationParams> for NotificationSettings { fn from(p: NotificationParams) -> Self { (&p).into() } }
+32
View File
@@ -0,0 +1,32 @@
use serde::{Deserialize, Serialize};
use crate::Writer;
/// A strongly-typed parameter value with writer tracking.
///
/// Serializes as `{ "value": 3, "writer": "base" }`.
#[derive(Debug, Clone, Serialize, Deserialize)]
pub struct ParamVal<T> {
pub value: T,
#[serde(skip_serializing_if = "Option::is_none")]
pub writer: Option<Writer>,
}
impl<T: Default> Default for ParamVal<T> {
fn default() -> Self {
ParamVal { value: T::default(), writer: None }
}
}
impl<T> ParamVal<T> {
pub fn new(value: T) -> Self {
ParamVal { value, writer: None }
}
pub fn with(value: T, writer: Writer) -> Self {
ParamVal { value, writer: Some(writer) }
}
/// Whether an incoming value from `incoming_writer` should override this one.
pub fn should_override(&self, incoming: Writer) -> bool {
let current = self.writer.map_or(0, |w| w.priority());
incoming.priority() > current
}
}
+54
View File
@@ -0,0 +1,54 @@
pub mod network;
pub mod cache;
pub mod engine;
use crate::apply_field;
use crate::{ParamVal, Writer, Overridden, traits::MergeParams};
use serde::{Deserialize, Serialize};
#[derive(Debug, Clone, Serialize, Deserialize)]
#[serde(deny_unknown_fields)]
pub struct PrebakeParams {
#[serde(default)] pub bootstrap_user: ParamVal<String>,
#[serde(default)] pub workspace: ParamVal<String>,
#[serde(default)] pub network: network::NetworkParams,
#[serde(default)] pub cache: cache::CacheParams,
#[serde(default)] pub engine: engine::EngineParams,
#[serde(default)] pub repology_endpoint: ParamVal<String>,
}
impl MergeParams for PrebakeParams {
fn defaults() -> Self {
Self {
bootstrap_user: ParamVal::new("vulcan".into()), workspace: ParamVal::new("/home/vulcan/workspace".into()),
network: network::NetworkParams::defaults(), cache: cache::CacheParams::defaults(),
engine: engine::EngineParams::defaults(),
repology_endpoint: ParamVal::new("default".into()),
}
}
fn apply(&mut self, i: Self, w: Writer, ov: &mut Overridden, p: &str) {
apply_field!(self.bootstrap_user, i.bootstrap_user, w, ov, p, "bootstrap_user");
apply_field!(self.workspace, i.workspace, w, ov, p, "workspace");
apply_field!(self.repology_endpoint, i.repology_endpoint, w, ov, p, "repology_endpoint");
self.network.apply(i.network, w, ov, &format!("{}network.", p));
self.cache.apply(i.cache, w, ov, &format!("{}cache.", p));
self.engine.apply(i.engine, w, ov, &format!("{}engine.", p));
}
}
impl Default for PrebakeParams { fn default() -> Self { Self::defaults() } }
#[derive(Debug, Clone)]
pub struct PrebakeSettings {
pub bootstrap_user: String, pub workspace: String,
pub network: network::NetworkSettings, pub cache: cache::CacheSettings,
pub engine: engine::EngineSettings, pub repology_endpoint: String,
}
impl Default for PrebakeSettings { fn default() -> Self { PrebakeParams::defaults().into() } }
impl From<&PrebakeParams> for PrebakeSettings {
fn from(p: &PrebakeParams) -> Self { Self {
bootstrap_user: p.bootstrap_user.value.clone(), workspace: p.workspace.value.clone(),
network: (&p.network).into(), cache: (&p.cache).into(), engine: (&p.engine).into(),
repology_endpoint: p.repology_endpoint.value.clone(),
}}
}
impl From<PrebakeParams> for PrebakeSettings { fn from(p: PrebakeParams) -> Self { (&p).into() } }
@@ -0,0 +1,37 @@
use crate::apply_field;
use crate::{ParamVal, Writer, Overridden, traits::MergeParams};
use serde::{Deserialize, Serialize};
#[derive(Debug, Clone, Serialize, Deserialize)]
pub struct CacheParams {
#[serde(default)] pub strategy: ParamVal<String>,
#[serde(default)] pub mode: ParamVal<String>,
#[serde(default)] pub ttl_days: ParamVal<u32>,
#[serde(default)] pub max_size_gb: ParamVal<u32>,
#[serde(default)] pub cleanup_policy: ParamVal<String>,
}
impl MergeParams for CacheParams {
fn defaults() -> Self {
Self {
strategy: ParamVal::new("always".into()), mode: ParamVal::new("zstd".into()),
ttl_days: ParamVal::new(30), max_size_gb: ParamVal::new(20),
cleanup_policy: ParamVal::new("lru".into()),
}
}
fn apply(&mut self, i: Self, w: Writer, ov: &mut Overridden, p: &str) {
apply_field!(self.strategy, i.strategy, w, ov, p, "strategy");
apply_field!(self.mode, i.mode, w, ov, p, "mode");
apply_field!(self.ttl_days, i.ttl_days, w, ov, p, "ttl_days");
apply_field!(self.max_size_gb, i.max_size_gb, w, ov, p, "max_size_gb");
apply_field!(self.cleanup_policy, i.cleanup_policy, w, ov, p, "cleanup_policy");
}
}
impl Default for CacheParams { fn default() -> Self { Self::defaults() } }
#[derive(Debug, Clone)]
pub struct CacheSettings { pub strategy: String, pub mode: String, pub ttl_days: u32, pub max_size_gb: u32, pub cleanup_policy: String }
impl Default for CacheSettings { fn default() -> Self { CacheParams::defaults().into() } }
impl From<&CacheParams> for CacheSettings { fn from(p: &CacheParams) -> Self {
Self { strategy: p.strategy.value.clone(), mode: p.mode.value.clone(), ttl_days: p.ttl_days.value, max_size_gb: p.max_size_gb.value, cleanup_policy: p.cleanup_policy.value.clone() }
} }
impl From<CacheParams> for CacheSettings { fn from(p: CacheParams) -> Self { (&p).into() } }
@@ -0,0 +1,21 @@
use crate::apply_field;
use crate::{ParamVal, Writer, Overridden, traits::MergeParams};
use serde::{Deserialize, Serialize};
#[derive(Debug, Clone, Serialize, Deserialize)]
pub struct EngineParams {
#[serde(default)] pub timeout_ms: ParamVal<u64>,
}
impl MergeParams for EngineParams {
fn defaults() -> Self { Self { timeout_ms: ParamVal::new(300000) } }
fn apply(&mut self, i: Self, w: Writer, ov: &mut Overridden, p: &str) {
apply_field!(self.timeout_ms, i.timeout_ms, w, ov, p, "timeout_ms");
}
}
impl Default for EngineParams { fn default() -> Self { Self::defaults() } }
#[derive(Debug, Clone)]
pub struct EngineSettings { pub timeout_ms: u64 }
impl Default for EngineSettings { fn default() -> Self { EngineParams::defaults().into() } }
impl From<&EngineParams> for EngineSettings { fn from(p: &EngineParams) -> Self { Self { timeout_ms: p.timeout_ms.value } } }
impl From<EngineParams> for EngineSettings { fn from(p: EngineParams) -> Self { (&p).into() } }
@@ -0,0 +1,23 @@
use crate::apply_field;
use crate::{ParamVal, Writer, Overridden, traits::MergeParams};
use serde::{Deserialize, Serialize};
#[derive(Debug, Clone, Serialize, Deserialize)]
pub struct NetworkParams {
#[serde(default)] pub enabled: ParamVal<bool>,
#[serde(default)] pub outbound: ParamVal<bool>,
}
impl MergeParams for NetworkParams {
fn defaults() -> Self { Self { enabled: ParamVal::new(true), outbound: ParamVal::new(true) } }
fn apply(&mut self, i: Self, w: Writer, ov: &mut Overridden, p: &str) {
apply_field!(self.enabled, i.enabled, w, ov, p, "enabled");
apply_field!(self.outbound, i.outbound, w, ov, p, "outbound");
}
}
impl Default for NetworkParams { fn default() -> Self { Self::defaults() } }
#[derive(Debug, Clone)]
pub struct NetworkSettings { pub enabled: bool, pub outbound: bool }
impl Default for NetworkSettings { fn default() -> Self { NetworkParams::defaults().into() } }
impl From<&NetworkParams> for NetworkSettings { fn from(p: &NetworkParams) -> Self { Self { enabled: p.enabled.value, outbound: p.outbound.value } } }
impl From<NetworkParams> for NetworkSettings { fn from(p: NetworkParams) -> Self { (&p).into() } }
+28
View File
@@ -0,0 +1,28 @@
use crate::{Writer, Overridden};
/// A parameter tree node that can merge itself with an incoming layer.
pub trait MergeParams: Sized {
fn apply(&mut self, incoming: Self, writer: Writer, ov: &mut Overridden, prefix: &str);
fn defaults() -> Self;
}
/// Apply one field's merge logic.
/// ```ignore
/// apply_field!(self.foo, incoming.foo, writer, ov, prefix, "foo");
/// ```
#[macro_export]
macro_rules! apply_field {
($self:ident.$field:ident, $incoming:ident.$field2:ident, $writer:expr, $ov:ident, $prefix:expr, $name:expr) => {
let old_w = $self.$field.writer;
if $self.$field.should_override($writer) {
let old_v = ::std::mem::replace(&mut $self.$field.value, $incoming.$field2.value);
$self.$field.writer = ::std::option::Option::Some($writer);
if let ::std::option::Option::Some(w) = old_w {
$ov.entries.insert(
format!("{}{}", $prefix, $name),
::serde_json::json!({"value": old_v, "writer": w.as_str()}),
);
}
}
};
}
+26
View File
@@ -0,0 +1,26 @@
use serde::{Deserialize, Serialize};
#[derive(Debug, Clone, Copy, PartialEq, Eq, Serialize, Deserialize)]
#[serde(rename_all = "lowercase")]
pub enum Writer {
Base,
Courier,
Bakerd,
}
impl Writer {
/// Priority: Base=3 > Courier=2 > Bakerd=1
pub fn priority(self) -> u8 {
match self {
Writer::Base => 3,
Writer::Courier => 2,
Writer::Bakerd => 1,
}
}
pub fn as_str(self) -> &'static str {
match self {
Writer::Base => "base", Writer::Courier => "courier", Writer::Bakerd => "bakerd",
}
}
}
impl Default for Writer { fn default() -> Self { Writer::Base } }
+1
View File
@@ -1,3 +1,4 @@
temp
examples
quicktest.sh
_env.sh
+93 -1561
View File
File diff suppressed because it is too large Load Diff
+11 -13
View File
@@ -15,10 +15,6 @@ path = "src/main.rs"
default = []
integration-tests = []
[[test]]
name = "engine_integration"
path = "tests/engine_integration.rs"
[dependencies]
workshop-engine = { path = "../workshop-engine" }
anyhow = "1.0.100"
@@ -32,7 +28,7 @@ env_logger = "0.11.8"
glob = "0.3.2"
futures-util = "0.3.31"
lazy_static = "1.5.0"
lettre = { version = "0.11", features = ["tokio1-native-tls"] }
lettre = { version = "0.11", default-features = false, features = ["tokio1-rustls", "rustls-tls", "builder", "smtp-transport"] }
log = "0.4.28"
minijinja = "2.14.0"
regex = "1.12.2"
@@ -51,17 +47,19 @@ uuid = { version = "1.19.0", features = ["v4"] }
libc = "0.2"
os_info = "3.14.0"
privdrop = "0.5.6"
reqwest = { version = "0.12", features = ["json"] }
which = "8.0.2"
gix = "0.81.0"
sha2 = "0.10"
flate2 = "1.0"
zstd = "0.13"
globset = "0.4.18"
axum = "0.7"
workshop-baker-params = { version = "0.1.0", path = "../workshop-baker-params" }
[dev-dependencies]
criterion = "0.5"
[[bench]]
name = "parser_bench"
harness = false
[lints.rust]
dead_code = "allow"
unreachable_code = "allow"
[lints.clippy]
inherent_to_string = "allow"
non_canonical_partial_ord_impl = "allow"
-93
View File
@@ -1,93 +0,0 @@
#!/usr/bin/env python3
import os
import random
import string
OUTPUT_DIR = "/tmp"
DECORATORS = [
"# @pipeline",
"# @fallible",
"# @parallel",
"# @export",
"# @timeout(60)",
"# @retry(3, 5)",
"# @if(condition)",
"# @after(dep)",
"# @pipe(step1, step2)",
"# @health(/health)",
"# @loop(10)",
]
FUNCTION_BODIES = [
"echo 'Processing...';",
"local var=$(date +%s);",
"if [ -f /tmp/test ]; then echo 'exists'; fi;",
"for i in $(seq 1 10); do echo $i; done;",
"case $1 in start) echo 'starting';; stop) echo 'stopping';; esac;",
"read -r line < /dev/stdin;",
"export PATH=$PATH:/usr/local/bin;",
"set -e; set -u;",
"trap 'echo error' ERR;",
"cd /tmp || exit 1;",
"local result=$(grep -r 'pattern' /var/log/ 2>/dev/null);",
'for f in /tmp/*.tmp; do rm -f "$f"; done',
"if [[ $var -gt 100 ]]; then echo 'large'; fi;",
'while read -r line; do echo "$line"; done < /etc/passwd;',
'select opt in a b c; do echo "Selected $opt"; break; done;',
]
def random_string(length=8):
return "".join(random.choices(string.ascii_letters + string.digits, k=length))
def generate_function(body_lines):
lines = []
func_name = f"func_{random_string(6)}"
for _ in range(random.randint(0, 3)):
lines.append(random.choice(DECORATORS))
lines.append(f"{func_name}() {{")
for _ in range(body_lines):
lines.append(" " + random.choice(FUNCTION_BODIES))
lines.append("}")
lines.append("")
return lines
def generate_script(target_lines, num_functions, target_bytes):
lines = []
total_lines = 0
body_per_func = max(1, (target_lines - num_functions * 3) // num_functions)
for _ in range(num_functions):
func_lines = generate_function(body_per_func)
lines.extend(func_lines)
total_lines += len(func_lines)
while (
total_lines < target_lines
or len("\n".join(lines).encode("utf-8")) < target_bytes
):
lines.append(random.choice(FUNCTION_BODIES))
total_lines += 1
return "\n".join(lines)
def main():
sizes = [
("small", 1000, 10, 50_000),
("medium", 5000, 50, 500_000),
("large", 10000, 100, 950_000),
]
for name, lines, funcs, target_bytes in sizes:
script = generate_script(lines, funcs, target_bytes)
filepath = os.path.join(OUTPUT_DIR, f"bench_{name}.sh")
with open(filepath, "w") as f:
f.write(script)
actual_bytes = len(script.encode("utf-8"))
actual_lines = len(script.splitlines())
print(f"Generated {filepath}: lines={actual_lines}, bytes={actual_bytes}")
if __name__ == "__main__":
main()
-61
View File
@@ -1,61 +0,0 @@
use criterion::{Criterion, black_box, criterion_group, criterion_main};
use std::fs;
use std::path::Path;
fn parse_large_script(c: &mut Criterion) {
let script_path = Path::new("/tmp/bench_large.sh");
let script_content = fs::read_to_string(script_path)
.expect("Failed to read benchmark script. Run `python3 benches/generate.py` first.");
let byte_size = script_content.len();
let line_count = script_content.lines().count();
println!("Benchmark file: {} lines, {} bytes", line_count, byte_size);
c.bench_function("parse_script_large", |b| {
b.iter(|| {
let result =
workshop_baker::bake::parser::parse_script(black_box(&script_content.clone()));
black_box(result)
});
});
}
fn parse_medium_script(c: &mut Criterion) {
let script_path = Path::new("/tmp/bench_medium.sh");
let script_content = fs::read_to_string(script_path)
.expect("Failed to read benchmark script. Run `python3 benches/generate.py` first.");
c.bench_function("parse_script_medium", |b| {
b.iter(|| {
let result =
workshop_baker::bake::parser::parse_script(black_box(&script_content.clone()));
black_box(result)
});
});
}
fn parse_small_script(c: &mut Criterion) {
let script_path = Path::new("/tmp/bench_small.sh");
let script_content = fs::read_to_string(script_path)
.expect("Failed to read benchmark script. Run `python3 benches/generate.py` first.");
c.bench_function("parse_script_small", |b| {
b.iter(|| {
let result =
workshop_baker::bake::parser::parse_script(black_box(&script_content.clone()));
black_box(result)
});
});
}
criterion_group!(
benches,
parse_small_script,
parse_medium_script,
parse_large_script
);
criterion_main!(benches);
+42 -47
View File
@@ -1,62 +1,57 @@
# workshop-baker/src
**Generated:** 2026-04-22
**Commit:** 7b3b71a
**Generated:** 2026-05-19
**Commit:** 28abc5d
Baker CLI + execution engine for HoneyBiscuitWorkshop pipelines.
## OVERVIEW
CLI tool and runtime for executing CI/CD pipelines with resource isolation, privilege dropping, and package management integration.
CLI tool and runtime for executing CI/CD pipelines. Orchestrates prebake → bake → finalize stages with notification dispatch.
## STRUCTURE
```
src/
├── bake.rs # Top-level bake orchestration
├── bake/ # Script parsing and building
├── parser.rs # @decorator-aware shell parser
│ ├── decorator.rs # @pipeline, @retry, @parallel, etc.
├── builder.rs # Script template builder
│ └── schedule.rs # Function ordering via @after deps
├── engine/ # Execution runtime
│ ├── executor.rs # Script execution with cgroups
│ ├── cgroups.rs # Linux cgroup resource limits
│ ├── repology.rs # Package name resolution
│ ├── pm.rs # Package manager detection
│ └── upm.rs # User package manager (Nix/Guix)
├── prebake/ # Build environment setup
├── config.rs # PrebakeConfig YAML schema
│ ├── stage/ # Bootstrap, DepsSystem, DepsUser, Hooks
│ └── security.rs # Privilege dropping logic
├── types/ # Shared type definitions
│ ├── repology.rs # RepologyEndpoint enum
│ ├── builderconfig.rs
│ ├── cache.rs
│ └── memsize.rs
├── monitor/ # Resource usage monitoring
│ ├── cgroups.rs # cgroup-based usage collection
│ └── jobobject.rs # Windows job objects (stub)
├── daemon.rs # Unix socket daemon (TODO: refactor)
└── lib.rs # CLI struct + module exports
├── lib.rs # Module declarations + Cli struct + Commands enum
├── main.rs # tokio::main entry point
├── bare.rs # Bare mode (run individual stages)
├── error.rs # CliError (wraps stage errors)
├── bake.rs # Bake orchestration (script → execution)
├── bake/ # Script parsing, decorator DSL, scheduling
├── prebake.rs # Prebake orchestration (env setup)
├── prebake/ # Config, stages, security, env providers
├── finalize.rs # Finalize orchestration (post-build)
├── finalize/ # Config, plugins, hooks, templates
├── notify.rs # Notification system (batching, retry)
├── notify/ # Types, templates, interactive, queue
├── daemon/ # Empty (Daemon command = unimplemented!)
├── monitor/ # Resource usage monitoring (cgroups)
├── types.rs # Module re-exports
├── types/ # buildstatus, resource types
├── utils.rs # Module root (fetch submodule)
└── utils/ # fetch.rs
```
## WHERE TO LOOK
| Task | Location |
|------|----------|
| Script parsing with @decorators | `bake/parser.rs:29` - `parse_script()` |
| Decorator types | `bake/decorator.rs` - `Decorator` enum |
| Build execution | `engine/executor.rs` - `Executor::execute_script()` |
| Resource limits | `engine/types.rs:56` - `ResourceLimits` struct |
| Prebake stages | `prebake/stage/` subdirectories |
| Package detection | `engine/pm.rs:detection()` |
| CLI entry | `lib.rs:26` - `cli::Cli` struct |
| CLI args | `lib.rs:21` - `Cli` struct |
| Entry point | `main.rs:91` - `#[tokio::main]` |
| Pipeline orchestration | `main.rs:68-74` - prebake→bake→finalize chain |
| Bare mode (single stage) | `bare.rs:14` - `run_bare()` |
| Script parsing | `bake/parser.rs:56` - `parse_script()` |
| Decorator DSL | `bake/decorator.rs` - `Decorator` enum |
| Prebake stages | `prebake/stage/` - Bootstrap→DepsSystem→DepsUser→Hooks→Ready |
| Plugin system | `finalize/plugin.rs` + `finalize/plugin/` |
| Notification dispatch | `notify.rs:66` - `notify()` async loop |
| Notification types | `notify/types.rs` - `NotificationRule`, `ParsedTrigger` |
## CONVENTIONS
- **ExecutionContext**: Passed through all stages, carries task_id, username, env_vars, timeout
- **PrebakeStage**: Ordered enum controlling stage execution sequence (Bootstrap -> Ready)
- **Decorator**: Attached to shell functions via `# @name(args)` comments above function declarations
- **Engine**: Lightweight wrapper around cgroup manager for resource isolation
- **Dual-mode**: CLI commands (prebake, bake, finalize) vs daemon mode via Unix socket
- **PrebakeStage**: Ordered enum Bootstrap→EarlyHook→DepsSystem→DepsUser→LateHook→Ready
- **Decorator**: `# @name(args)` comments above shell function declarations
- **Notification**: Plugin-based dispatch with batching, exponential backoff, priority groups
- **Engine**: Delegated to `workshop-engine` crate (formerly in-tree)
- **Dual-mode**: CLI commands (standalone) vs Daemon mode (unimplemented)
## ANTI-PATTERNS
1. **Empty daemon/** directory — Daemon command is `unimplemented!()`
2. **Dead code allowed**`lints.rust.dead_code = "allow"` in Cargo.toml
3. **Engine was extracted** — workshop-engine is its own crate now, src/engine/ removed
4. **Python test infra exists but files removed** — pytest.ini remains, test dir is empty
+13 -13
View File
@@ -1,26 +1,26 @@
use workshop_engine::{Engine, EventSender, ExecutionContext};
use crate::bake::parser::Function;
use crate::bake::constant::{DEFAULT_WORKSPACE, TRIVIAL_SCRIPT_NAME};
use crate::bake::error::BakeError;
use crate::bake::parser::Function;
use crate::cli::Cli;
use crate::prebake;
use crate::prebake::PrebakeConfig;
use crate::prebake::security::get_drop_after;
use crate::prebake::stage::PrebakeStage;
use crate::prebake;
use std::path::{Path, PathBuf};
use workshop_engine::{Engine, EventSender, ExecutionContext};
mod builder;
pub mod constant;
mod decorator;
pub mod parser;
pub mod error;
pub mod parser;
mod schedule;
pub async fn bake(
script_path: &Path,
bake_base_path: &Path,
prebake_path: &Path,
cli: &Cli,
_cli: &Cli,
ctx: &mut ExecutionContext,
event_tx: EventSender,
) -> Result<(), BakeError> {
@@ -43,11 +43,11 @@ pub async fn bake(
let use_template = true;
let engine = Engine::new();
if let Some(env) = &prebake.envvars {
if let Some(prebake_env) = &env.bake {
if let Some(env) = &prebake.envvars
&& let Some(prebake_env) = &env.bake
{
ctx.env_vars.extend(prebake_env.clone());
}
}
if !has_pipeline || trivial {
let function = Function {
@@ -57,7 +57,7 @@ pub async fn bake(
};
let script =
builder::build_script(&function, bake_base_path, &workspace, None, use_template)?;
let result = engine.execute_script(&script, &ctx, &event_tx).await;
let result = engine.execute_script(&script, ctx, &event_tx).await;
result?;
} else {
let sorted_functions = schedule::sort_function(functions.pipeline_functions)?;
@@ -72,7 +72,7 @@ pub async fn bake(
None,
use_template,
)?;
let result = engine.execute_script(&script, &ctx, &event_tx).await;
let result = engine.execute_script(&script, ctx, &event_tx).await;
result?;
}
}
@@ -96,11 +96,11 @@ fn get_workspace(prebake_config: &PrebakeConfig) -> PathBuf {
}
fn privileged(prebake_config: &PrebakeConfig) -> bool {
if let Some(bootstrap) = &prebake_config.bootstrap {
if bootstrap.user == "root" {
if let Some(bootstrap) = &prebake_config.bootstrap
&& bootstrap.user == "root"
{
return true;
}
}
let drop_after = get_drop_after(&prebake_config.security).unwrap_or_default();
PrebakeStage::Never == drop_after
}
+4 -2
View File
@@ -12,14 +12,16 @@ bake/
├── parser.rs # @decorator-aware shell script parser
├── decorator.rs # Decorator enum + 100+ unit tests
├── builder.rs # Script template assembly
── schedule.rs # Function ordering via @after dependencies
── schedule.rs # Function ordering via @after dependencies
├── error.rs # BakeError enum
└── constant.rs # DEFAULT_WORKSPACE, TRIVIAL_SCRIPT_NAME
```
## WHERE TO LOOK
| Task | Location |
|------|----------|
| Parse script | `parser.rs:29` - `parse_script()` |
Parse script | `parser.rs:56` - `parse_script()` |
| Decorator types | `decorator.rs` - `Decorator` enum variants |
| Build script | `builder.rs` - template builder |
| Schedule deps | `schedule.rs` - @after resolution |
+2 -2
View File
@@ -1,7 +1,7 @@
use crate::bake::decorator::Decorator;
use crate::bake::parser::Function;
use crate::bake::constant::{PIPELINE_SKIP_ERRORCODE, TRIVIAL_SCRIPT_NAME};
use crate::bake::decorator::Decorator;
use crate::bake::error::BakeError;
use crate::bake::parser::Function;
use minijinja::{Environment, context};
use std::fs::Permissions;
use std::os::unix::fs::PermissionsExt;
+1 -4
View File
@@ -2,10 +2,7 @@ use thiserror::Error;
use workshop_engine::{
ExecutionError,
error::{
EXITCODE_IO_ERROR, EXITCODE_PARSE_ERROR,
HasExitCode,
},
error::{EXITCODE_IO_ERROR, EXITCODE_PARSE_ERROR, HasExitCode},
};
#[derive(Error, Debug)]
+50
View File
@@ -0,0 +1,50 @@
use std::path::PathBuf;
use crate::cli::{BareCommands, Cli};
use crate::error::CliError;
use crate::types::resource::ResourceRegistry;
use crate::{bake::bake, finalize::finalize, prebake::prebake};
use workshop_engine::{EventSender, ExecutionContext};
fn or_workshop(path: &Option<PathBuf>, name: &str) -> PathBuf {
path
.clone()
.unwrap_or_else(|| PathBuf::from(format!(".workshop/{}", name)))
}
pub async fn run_bare(
cli: &Cli,
command: &BareCommands,
ctx: &mut ExecutionContext,
event_tx: EventSender,
pipeline: &str,
build_id: &str,
registry: &ResourceRegistry,
) -> Result<(), CliError> {
let (stage, config) = match command {
BareCommands::Prebake { .. } => ("prebake", or_workshop(&cli.prebake, "prebake.yml")),
BareCommands::Bake { .. } => ("bake", or_workshop(&cli.bake, "bake.sh")),
BareCommands::Finalize { .. } => {
("finalize", or_workshop(&cli.finalize, "finalize.yml"))
}
};
let prebake_path = or_workshop(&cli.prebake, "prebake.yml");
let bake_base = or_workshop(&cli.bake_base, "bake_base.sh");
ctx.task_id = format!("{}-{}-{}", pipeline, build_id, stage);
ctx.pipeline_name = pipeline.to_string();
ctx.standalone = true;
match command {
BareCommands::Prebake { .. } => {
prebake(&config, cli, None, ctx, event_tx, None).await?;
}
BareCommands::Bake { .. } => {
bake(&config, &bake_base, &prebake_path, cli, ctx, event_tx).await?;
}
BareCommands::Finalize { .. } => {
finalize(&config, cli, ctx, event_tx, registry).await?;
}
}
Ok(())
}
-193
View File
@@ -1,193 +0,0 @@
use crate::cli::Cli;
// use serde::Serialize;
use serde_json::Value;
// use std::path::PathBuf;
// use std::sync::Arc;
use tokio::io::{AsyncBufReadExt, BufReader};
use tokio::net::UnixStream;
// use tokio::process::{Child, Command};
// use tokio::signal;
// use tokio::sync::RwLock;
// use tokio::time::{Duration, interval};
// #[derive(Debug, Clone, Serialize)]
// struct Heartbeat {
// pipeline_id: String,
// build_id: String,
// stage: String,
// timestamp: u64,
// status: String,
// subtasks: Vec<String>,
// }
pub async fn daemon(_cli: &Cli) {
todo!("Daemon should be refactored");
// let socket = PathBuf::from(settings.get_string("socket").unwrap());
// let agentsocket = settings.get_string("agentsocket").unwrap();
// let _ = std::fs::remove_file(&socket);
// let heartbeat = Arc::new(RwLock::new(Heartbeat {
// pipeline_id: cli.pipeline.clone(),
// build_id: cli.build_id.clone(),
// stage: String::new(),
// timestamp: 0,
// status: String::new(),
// subtasks: Vec::new(),
// }));
// // let heartbeat_clone = heartbeat.clone();
// let heartbeat_task = {
// let hb = heartbeat.clone();
// tokio::spawn(async move {
// let mut tick = interval(Duration::from_secs(1));
// loop {
// tick.tick().await;
// let mut hb = hb.write().await;
// hb.timestamp = std::time::SystemTime::now()
// .duration_since(std::time::UNIX_EPOCH)
// .unwrap()
// .as_millis() as u64;
// // 只输出到stdout(模拟未来发送给Agent
// println!("[HEARTBEAT] {}", serde_json::to_string(&*hb).unwrap());
// }
// })
// };
// let socket_task = {
// let socket = socket.clone();
// tokio::spawn(async move {
// log::info!("Creating Unix socket at {}", socket.display());
// let listener =
// UnixListener::bind(&socket).expect("Failed to bind Unix socket");
// loop {
// match listener.accept().await {
// Ok((stream, _)) => {
// tokio::spawn(handle_connection(stream));
// }
// Err(e) => {
// log::error!("Accept error: {}", e);
// break;
// }
// }
// }
// })
// };
// let mut tasks: Vec<TaskSpec> = Vec::new();
// let execution_task = {
// let hb = heartbeat.clone();
// tokio::spawn(async move {
// let mut hb = hb.write().await;
// let mut children: Vec<(String, Child)> = vec![];
// let executable = std::env::current_exe().unwrap();
// for (index, task) in tasks.iter().enumerate() {
// let mut command = task.command(&executable);
// command.stdout(std::process::Stdio::piped());
// command.stderr(std::process::Stdio::piped());
// let child = command
// .spawn()
// .expect(format!("Failed to spawn child {}", index).as_str());
// let pid = child.id().unwrap_or(0).to_string();
// children.push((pid.clone(), child));
// hb.subtasks.push(pid.clone());
// }
// for (pid, mut child) in children {
// let status = child.wait().await.expect("Failed to wait child");
// log::info!("Subtask {} exited with: {}", pid, status);
// hb.subtasks.retain(|p| p != &pid);
// }
// hb.stage = "Success".to_string();
// })
// };
// tokio::select! {
// _ = heartbeat_task => {
// log::error!("Heartbeat task died");
// }
// _ = execution_task => {
// log::info!("All subtasks completed");
// }
// _ = socket_task => {
// log::error!("Socket task died");
// }
// _ = signal::ctrl_c() => {
// log::info!("Shutting down...");
// }
// }
// for pid in heartbeat.read().await.subtasks.clone() {
// let _ = Command::new("kill").arg("-TERM").arg(&pid).status().await;
// }
// let _ = std::fs::remove_file(&socket);
}
async fn _handle_connection(stream: UnixStream) {
let peer_addr = match stream.peer_addr() {
Ok(addr) => format!("{:?}", addr),
Err(_) => "unknown".to_string(),
};
log::debug!("New connection from: {}", peer_addr);
let mut reader = BufReader::new(stream);
let mut line = String::new();
loop {
line.clear();
match reader.read_line(&mut line).await {
Ok(0) => {
log::debug!("Connection closed by peer: {}", peer_addr);
break;
}
Ok(_) => {
// 简单打印原始JSON(预留字段解析空间)
match serde_json::from_str::<Value>(&line) {
Ok(json) => {
// 只提取关键字段打印(字段可能变化)
let msgtype = json
.get("msgtype")
.and_then(|v| v.as_str())
.unwrap_or("unknown");
let name = json
.get("name")
.and_then(|v| v.as_str())
.unwrap_or("unnamed");
log::info!("[{}] Message from '{}': {}", msgtype, name, line.trim());
// TODO: 字段稳定后,可扩展为结构化打印
// match msgtype {
// "ResourceUsage" => print_resource(&json),
// "Log" => print_log(&json),
// _ => log::debug!("Unknown type: {}", msgtype),
// }
}
Err(e) => {
log::warn!(
"Invalid JSON from {}: {} - Error: {}",
peer_addr,
line.trim(),
e
);
}
}
}
Err(e) => {
log::error!("Failed to read from {}: {}", peer_addr, e);
break;
}
}
}
log::debug!("Connection handler for {} exiting", peer_addr);
}
+30
View File
@@ -0,0 +1,30 @@
use thiserror::Error;
use workshop_engine::HasExitCode;
#[derive(Error, Debug)]
pub enum CliError {
#[error("{0}")]
Prebake(#[from] crate::prebake::error::PrebakeError),
#[error("{0}")]
Bake(#[from] crate::bake::error::BakeError),
#[error("{0}")]
Finalize(#[from] crate::finalize::FinalizeError),
#[error("{0}")]
General(anyhow::Error),
}
impl From<anyhow::Error> for CliError {
fn from(e: anyhow::Error) -> Self {
CliError::General(e)
}
}
impl CliError {
pub fn exit_code(&self) -> i32 {
match self {
CliError::Prebake(e) => e.exit_code(),
CliError::Bake(e) => e.exit_code(),
CliError::Finalize(e) => e.exit_code(),
CliError::General(_) => 1,
}
}
}
+7 -31
View File
@@ -8,13 +8,11 @@ pub mod template;
pub mod types;
use crate::cli::Cli;
use crate::finalize::constant::FINALIZE_STANDALONE_PLUGIN_PATH;
use workshop_engine::EventSender;
use crate::finalize::plugin::fetch::{self, FetchArgument};
use std::path::PathBuf;
use crate::types::resource::ResourceRegistry;
pub use config::*;
pub use error::FinalizeError;
use std::path::Path;
use workshop_engine::EventSender;
pub fn parse(path: &Path) -> Result<FinalizeConfig, FinalizeError> {
let content = std::fs::read_to_string(path).map_err(FinalizeError::IoError)?;
@@ -24,9 +22,10 @@ pub fn parse(path: &Path) -> Result<FinalizeConfig, FinalizeError> {
pub async fn finalize(
finalize_path: &Path,
cli: &Cli,
_cli: &Cli,
ctx: &mut workshop_engine::ExecutionContext,
event_tx: EventSender,
registry: &ResourceRegistry,
) -> Result<(), FinalizeError> {
let mut finalize = parse(finalize_path)?;
let validate_result = finalize.validate();
@@ -36,33 +35,10 @@ pub async fn finalize(
}
return Err(FinalizeError::ValidateError(errors));
};
if ctx.standalone {
// In standalone mode, finalize should download plugin by itself
log::debug!("Downloading {} plugins", finalize.plugin.len());
let plugin_path = PathBuf::from(FINALIZE_STANDALONE_PLUGIN_PATH);
// if plugin_path.exists() {
// std::fs::remove_dir_all(&plugin_path).map_err(FinalizeError::IoError)?;
// }
for (name, data) in finalize.plugin.iter_mut() {
log::info!("Downloading plugin {}", name);
let (source, config) = data.iter_mut().next().unwrap_or_else(|| {
panic!(
"Internal Error: Plugin {} has no source configured, this should have been caught by validation!",
name
)
});
let fetch_argument = FetchArgument {
checksum: config.checksum.clone(),
shallow: config.shallow,
};
fetch::fetch_plugin(&source, &name, &plugin_path, fetch_argument).await?;
config.runtime.fspath = Some(plugin_path.join(name));
}
}
// Register Plugins
// Register Plugins (paths resolved from registry)
log::info!("Registering {} plugins", finalize.plugin.len());
let registered_plugins = plugin::register(finalize.plugin, None)?;
let registered_plugins = plugin::register(finalize.plugin, None, &mut finalize.notification, registry)?;
// Early hook
let earlyhook_result = stage::hook::hook(
@@ -75,7 +51,7 @@ pub async fn finalize(
// Determine if we should continue with artifacts/latehook
// If prior stages failed, we still send notification but skip artifacts
let prior_failed = earlyhook_result.is_err();
let _prior_failed = earlyhook_result.is_err();
// Prepare artifacts
// TODO: artifact stage implementation
+28 -32
View File
@@ -1,56 +1,52 @@
# workshop-baker/src/finalize
**Generated:** 2026-04-22
**Commit:** 7b3b71a
**Generated:** 2026-05-19
**Commit:** 28abc5d
Artifact packaging, distribution, deployment, and notifications.
Artifact packaging, plugin-based notifications, and post-build hooks.
## STRUCTURE
```
finalize/
├── config.rs # FinalizeConfig YAML (artifacts, deploy, notify)
├── config.rs # FinalizeConfig YAML (379 lines, plugins, notifications)
├── stage.rs # FinalizeStage enum
├── stage/hook.rs # Post-build hooks
├── stage/hook.rs # Post-build hook execution
├── event.rs # Event propagation
├── template.rs # Variable substitution {{VAR}}
├── plugin.rs # Plugin trait + registry
├── template.rs # Variable substitution {{VARIABLE}}
├── plugin.rs # Plugin trait + registration
├── plugin/
│ ├── metadata.rs # Plugin manifest parsing
│ ├── dylib.rs # Dynamic library plugins
│ ├── rhai.rs # Rhai scripting (todo!())
│ ├── shell.rs # Shell command plugins
│ └── internal/ # Built-in plugins
│ ├── mail/ # Email notifications (SMTP)
│ ├── webhook.rs # HTTP webhooks
├── satori.rs # Satori integration
│ └── insitenotify.rs # In-site notifications
│ └── fetch/ # Artifact fetching
│ ├── git.rs # Git repository fetch
│ ├── http.rs # HTTP download
│ ├── extract.rs # Archive extraction
│ └── checksum.rs# SHA256 verification (SHA1 deprecated)
│ └── internal.rs + internal/ # Built-in plugins (mail, webhook, satori, insitenotify)
├── types.rs
├── types/compression.rs
├── error.rs # FinalizeError
└── constant.rs
```
## WHERE TO LOOK
| Task | Location |
|------|----------|
| Config loading | `config.rs` - FinalizeConfig |
| Email notify | `plugin/internal/mail/` - SMTP templates |
| Config loading | `config.rs:17` - `FinalizeConfig` + `parse()` |
| Plugin registration | `plugin.rs` - `register()` |
| Email notifications | `plugin/internal/mail.rs` - SMTP |
| Webhooks | `plugin/internal/webhook.rs` |
| Artifact fetch | `plugin/fetch/` - git/http/extract |
| Checksums | `plugin/fetch/checksum.rs:13` - SHA1 deprecated |
| Satori integration | `plugin/internal/satori.rs` |
| Shell plugins | `plugin/shell.rs` |
| Template rendering | `template.rs` - `{{VARIABLE}}` handlebars-style |
## CONVENTIONS
- **Plugin System**: Dynamic + internal plugins via Plugin trait
- **Template Syntax**: `{{VARIABLE}}` handlebars-style
- **Stages**: Hook → Fetch → Build → Publish → Notify → Cleanup
- **Fetch**: Supports git, http, with extraction and checksum verify
- **Plugin System**: `FinalizePlugin` trait + registry with both internal and dynamic plugins
- **Template Syntax**: `{{VARIABLE}}` handlebars-style substitution via `template.rs`
- **Stages**: EarlyHook → (Artifacts) → LateHook → Notifications
- **Notification**: `notify.rs` in parent module drives plugin-based notification dispatch
- **ResourceRegistry**: Shared resource cache passed through to plugins
## ANTI-PATTERNS
1. **SHA1 deprecated**`checksum.rs:13` warns SHA1 is deprecated
2. **Unimplemented**`rhai.rs:17` - "todo!()" for Rhai scripting
3. **TODO markers** — Multiple TODOs in config.rs (lines 305-306)
1. **fetch/ subdirectory removed** — was `plugin/fetch/{git,http,extract,checksum}`, no longer exists
2. **SHA1 deprecated**workshop-getterurl handles checksums now
3. **Unimplemented**`rhai.rs:17` - "todo!()" for Rhai scripting
4. **TODO markers** — Multiple TODOs in config.rs (lines 305-306)
5. **Artifact stage not implemented**`finalize.rs:57` — "TODO: artifact stage implementation"
+113 -11
View File
@@ -3,13 +3,13 @@
//! This module defines the configuration schema for the finalize stage,
//! which handles artifact collection, publishing, and notifications.
use workshop_engine::{CustomCommand, deserialize_duration_ms};
use crate::finalize::types::compression::CompressionMethod;
use semver::{Version, VersionReq};
use serde::{Deserialize, Serialize};
use serde_yaml::Value;
use std::collections::HashMap;
use std::path::PathBuf;
use workshop_engine::{CustomCommand, deserialize_duration_ms};
/// Version requirement for finalize.yml schema compatibility
pub const VERSION_REQUIREMENT: &str = "^0.0.1";
@@ -115,6 +115,17 @@ pub struct NotificationMethod {
pub policy: Vec<NotifyPolicy>,
}
impl Default for NotificationMethod {
fn default() -> Self {
Self {
template: NotificationTemplate::default(),
trigger: Vec::new(),
config: Value::default(),
policy: Vec::new(),
}
}
}
#[derive(Debug, Deserialize, Serialize, Clone, Default)]
pub struct NotificationTemplate {
/// Payload schema: "default" (plugin-defined) or "custom" (requires on_* templates)
@@ -134,17 +145,59 @@ pub struct NotificationTemplate {
pub on_finish_of: Option<OnFinishOf>,
}
impl std::ops::AddAssign for NotificationTemplate {
fn add_assign(&mut self, other: Self) {
if self.schema.is_none() {
self.schema = other.schema;
}
if self.on_success.is_none() {
self.on_success = other.on_success;
}
if self.on_failure.is_none() {
self.on_failure = other.on_failure;
}
if self.on_finish_of.is_none() {
self.on_finish_of = other.on_finish_of;
}
}
}
/// Notification configuration container
#[derive(Debug, Deserialize, Serialize, Clone, Default)]
#[derive(Debug, Serialize, Clone, Default)]
pub struct NotificationConfig {
/// Template definitions for notification content
#[serde(default)]
pub templates: HashMap<String, NotificationTemplateDef>,
#[serde(flatten)]
/// Priority groups (YAML keys like `0`, `1` are parsed as u32).
/// Custom Deserialize handles serde_yaml string→u32 conversion.
#[serde(default)]
pub groups: HashMap<u32, HashMap<String, NotificationMethod>>,
}
/// Serde helper: YAML map keys are always strings, convert to u32.
impl<'de> serde::Deserialize<'de> for NotificationConfig {
fn deserialize<D>(deserializer: D) -> Result<Self, D::Error>
where D: serde::Deserializer<'de>
{
#[derive(serde::Deserialize)]
struct Helper {
#[serde(default)]
templates: HashMap<String, NotificationTemplateDef>,
#[serde(flatten)]
groups_raw: HashMap<String, HashMap<String, NotificationMethod>>,
}
let h = Helper::deserialize(deserializer)?;
let groups: HashMap<u32, _> = h.groups_raw.into_iter()
.map(|(k, v)| {
k.parse::<u32>().map(|n| (n, v))
.map_err(|_| serde::de::Error::custom(format!("priority key must be integer, got {:?}", k)))
})
.collect::<Result<_, _>>()?;
Ok(NotificationConfig { templates: h.templates, groups })
}
}
/// Stage-triggered notification configuration
#[derive(Debug, Deserialize, Serialize, Clone)]
pub struct OnFinishOf {
@@ -158,7 +211,7 @@ pub struct OnFinishOf {
/// Notification template definition.
/// Supports inline content (on_success/on_failure/on_finish_of) or external reference (use).
#[derive(Debug, Deserialize, Serialize, Clone)]
#[derive(Debug, Deserialize, Serialize, Clone, Default)]
pub struct NotificationTemplateDef {
/// External template reference (git URL).
/// Cannot be combined with on_* fields in MVP (validation will reject).
@@ -178,6 +231,53 @@ pub struct NotificationTemplateDef {
pub on_finish_of: Option<String>,
}
impl NotificationTemplateDef {
/// Merge `other` into `self`, keeping self's values when present.
///
/// `other` fills in gaps where `self` is `None`.
/// Chaining: `a.apply(b).apply(c)` = a wins, then b fills gaps, then c fills gaps.
pub fn apply(mut self, other: Self) -> Self {
if self.on_success.is_none() {
self.on_success = other.on_success;
}
if self.on_failure.is_none() {
self.on_failure = other.on_failure;
}
if self.on_finish_of.is_none() {
self.on_finish_of = other.on_finish_of;
}
self
}
}
/// Builder for [`NotificationTemplateDef`].
pub struct NotificationTemplateDefBuilder(NotificationTemplateDef);
impl NotificationTemplateDefBuilder {
pub fn new() -> Self {
Self(NotificationTemplateDef::default())
}
pub fn with_use(mut self, val: impl Into<String>) -> Self {
self.0.use_ = Some(val.into());
self
}
pub fn with_on_success(mut self, val: impl Into<String>) -> Self {
self.0.on_success = Some(val.into());
self
}
pub fn with_on_failure(mut self, val: impl Into<String>) -> Self {
self.0.on_failure = Some(val.into());
self
}
pub fn with_on_finish_of(mut self, val: impl Into<String>) -> Self {
self.0.on_finish_of = Some(val.into());
self
}
pub fn build(self) -> NotificationTemplateDef {
self.0
}
}
/// Notification trigger type.
#[derive(Debug, Clone, PartialEq, Eq)]
pub enum Trigger {
@@ -253,7 +353,6 @@ impl Default for ArtifactDef {
/// Artifact
/// TODO: Under heavy refactor
/// Cleanup configuration
#[derive(Debug, Deserialize, Serialize, Clone, Default)]
pub struct CleanupConfig {
@@ -337,16 +436,19 @@ impl FinalizeConfig {
fn validate_notification(&self, errors: &mut Vec<String>) {
// Validate templates
for (name, tmpl) in &self.notification.templates {
if tmpl.use_.is_some()
&& (tmpl.on_success.is_some()
|| tmpl.on_failure.is_some()
|| tmpl.on_finish_of.is_some())
{
if let Some(ref u) = tmpl.use_ {
if u.contains("://") && (
tmpl.on_success.is_some() ||
tmpl.on_failure.is_some() ||
tmpl.on_finish_of.is_some()
) {
errors.push(format!(
"Template '{}' cannot combine 'use' with 'on_*' fields (MVP restriction)",
"Template '{}' cannot combine external 'use' with 'on_*' fields",
name
));
}
// Local references (no ://) are allowed to have on_* overrides
}
}
}
}
+28 -8
View File
@@ -1,14 +1,15 @@
use crate::finalize::constant::{FINALIZE_PLUGIN_MANIFEST, FINALIZE_PLUGIN_MANIFEST_EXTENSION};
use crate::finalize::RawPluginMap;
use crate::finalize::constant::{FINALIZE_PLUGIN_MANIFEST, FINALIZE_PLUGIN_MANIFEST_EXTENSION};
use crate::finalize::plugin::metadata::{PluginMetadata, PluginType};
use crate::{EventSender, ExecutionContext, finalize::error::PluginError};
use workshop_engine::{EventSender, ExecutionContext};
use crate::{ finalize::error::PluginError};
use crate::types::resource::ResourceRegistry;
use async_trait::async_trait;
use std::collections::{HashMap, HashSet};
use std::path::{Path, PathBuf};
use std::sync::Arc;
mod dylib;
pub mod fetch;
mod internal;
mod metadata;
mod rhai;
@@ -134,7 +135,7 @@ fn load_manifest(path: &Path, name: &str, suffix: &str) -> Result<String, Plugin
reason: format!(
"Failed to read plugin manifest from {}: {}",
manifest_path.display(),
e.to_string()
e
),
})
}
@@ -189,10 +190,10 @@ fn get_entry(plugin_type: PluginType, name: &str) -> Result<Box<dyn AsyncPluginF
"Plugin {} has unknown plugin type and cannot be inferred from entrypoint.",
name
);
return Err(PluginError::PluginUnavailable {
Err(PluginError::PluginUnavailable {
name: name.to_string(),
reason: "Unknown plugin type and cannot be inferred from entrypoint".to_string(),
});
})
}
}
}
@@ -214,6 +215,8 @@ fn get_entry(plugin_type: PluginType, name: &str) -> Result<Box<dyn AsyncPluginF
pub fn register(
external_plugins: RawPluginMap,
list: Option<&HashSet<String>>,
notification_config: &mut crate::finalize::NotificationConfig,
registry: &ResourceRegistry,
) -> Result<PluginMap, PluginError> {
let mut plugins: PluginMap = HashMap::new();
let mut count = 0;
@@ -255,7 +258,14 @@ pub fn register(
log::warn!("Plugin {} is not available: {}", name, reason);
continue;
}
let metadata = read_manifest(config.runtime.fspath, &name)?;
// Resolve plugin path from registry instead of config.runtime.fspath
let plugin_path = registry.resolve(&name).ok_or_else(|| {
PluginError::PluginUnavailable {
name: name.clone(),
reason: "plugin not found in resource registry".to_string(),
}
})?;
let metadata = read_manifest(Some(plugin_path), &name)?;
let plugin_type = get_plugin_type(metadata.plugin_type.clone(), &metadata.entrypoint);
let entry = get_entry(plugin_type, &name)?;
plugins.insert(
@@ -267,8 +277,18 @@ pub fn register(
count += 1;
}
log::info!("Registered {} plugins", count);
// Inject plugin-declared templates as .fallback
for (name, plugin) in &plugins {
if let Some(ref templates) = plugin.metadata.templates {
let key = format!("{}.fallback", name);
for tmpl in templates {
notification_config.templates.insert(key.clone(), tmpl.clone());
}
log::debug!("Registered templates from plugin '{}' as '{}'", name, key);
}
}
log::info!("Registered {} plugins", count);
Ok(plugins)
}
+2 -1
View File
@@ -1,6 +1,7 @@
use crate::finalize::plugin::PluginError;
use crate::finalize::plugin::PluginMetadata;
use crate::{EventSender, ExecutionContext, finalize::plugin::AsyncPluginFn};
use workshop_engine::{EventSender, ExecutionContext};
use crate::{ finalize::plugin::AsyncPluginFn};
use async_trait::async_trait;
pub struct Dylib;
-112
View File
@@ -1,112 +0,0 @@
// Code in this module PARTIALLY OR FULLY utilized AI Coding Agent.
use crate::finalize::fetch;
use crate::finalize::{
PluginConfig,
plugin::{PluginError, RawPluginMap},
};
use std::collections::HashSet;
use std::path::Path;
mod checksum;
mod extract;
mod git;
mod http;
mod types;
pub use types::FetchArgument;
enum URLScheme {
Unknown,
Git,
Http,
Https,
File,
}
impl URLScheme {
fn parse(url: &str) -> (Self, String) {
match url.split_once("://") {
Some((prefix, rest)) => {
let scheme = match prefix {
"git" => URLScheme::Git,
"https" => URLScheme::Https,
"http" => URLScheme::Http,
"file" => URLScheme::File,
_ => URLScheme::Unknown,
};
(scheme, rest.to_string())
}
None => (URLScheme::Unknown, url.to_string()),
}
}
}
pub async fn fetch_plugin(
url: &str,
name: &str,
basedir: &Path,
argument: FetchArgument,
) -> Result<(), PluginError> {
let (scheme, path) = URLScheme::parse(url);
match scheme {
URLScheme::Git => {
// NOTE: as a practical compromise, we use git command for git clone
// gix will be supported later, with ONLY https scheme
// TODO: Implement gix
log::info!("Fetching git plugin: {}", url);
git::fetch_git_plugin(&path, name, basedir, argument.shallow).await
}
URLScheme::Http | URLScheme::Https => {
log::info!("Fetching http plugin: {}", url);
let checksum = argument.get_checksum();
http::fetch_http_plugin(url, name, basedir, checksum).await
}
URLScheme::File => {
// TODO: Support client mode
// NOTE: In standalone mode, we just assume path is well-defined and utilize
log::info!("Fetching file plugin: {}", url);
return Ok(());
}
URLScheme::Unknown => {
log::error!("Unknown URL scheme in plugin URL: {}", url);
return Err(PluginError::GeneralError(format!(
"Unknown URL scheme in plugin URL: {}",
url
)));
}
}
}
pub async fn fetch_plugins(
plugin_path: &Path,
plugin_config: &mut RawPluginMap,
list: Option<&HashSet<String>>,
) -> Result<(), PluginError> {
log::info!("Preparing to download {} plugins", plugin_config.len());
for (name, data) in plugin_config.iter_mut() {
match list {
Some(list) if !list.contains(name) => {
log::debug!("Plugin {} not in list, skipping", name);
continue;
}
_ => {}
}
log::debug!("Downloading plugin {}", name);
let (source, config) =
data.iter_mut()
.next()
.ok_or_else(|| PluginError::PluginUnavailable {
name: name.clone(),
reason: "no source configured".to_string(),
})?;
let fetch_argument = FetchArgument {
checksum: config.checksum.clone(),
shallow: config.shallow,
};
fetch::fetch_plugin(&source, &name, &plugin_path, fetch_argument).await?;
log::debug!("Storing plugin {} to {}", name, plugin_path.display());
config.runtime.fspath = Some(plugin_path.join(name));
}
Ok(())
}
@@ -1,137 +0,0 @@
// Code in this module PARTIALLY OR FULLY utilized AI Coding Agent.
use super::types::ChecksumType;
use sha2::{Digest, Sha256, Sha512};
/// Detects the checksum type based on the length of the checksum string.
///
/// Returns `Ok(None)` for empty string (checksum disabled).
/// Returns `Ok(Some(ChecksumType))` for valid SHA256 (64 chars) or SHA512 (128 chars).
/// Returns `Err` for invalid lengths (32=MD5, 40=SHA1) or unsupported lengths.
pub fn detect_checksum_type(checksum: &str) -> Result<Option<ChecksumType>, String> {
match checksum.len() {
0 => Ok(None),
32 => Err(format!(
"MD5 (first 8 chars: {}) is not supported for security reasons. Use SHA256 (64 chars) or SHA512 (128 chars)",
&checksum[..8.min(checksum.len())]
)),
40 => Err(format!(
"SHA1 (first 8 chars: {}) is deprecated and not supported. Use SHA256 (64 chars) or SHA512 (128 chars)",
&checksum[..8.min(checksum.len())]
)),
64 => Ok(Some(ChecksumType::Sha256)),
128 => Ok(Some(ChecksumType::Sha512)),
_ => Err(format!(
"Invalid checksum length: {}. Expected 0 (disabled), 64 (SHA256), or 128 (SHA512). First 8 chars: {}",
checksum.len(),
&checksum[..8.min(checksum.len())]
)),
}
}
/// Verifies that the data matches the expected checksum.
///
/// Compares the computed hash of `data` against `expected` using the specified `checksum_type`.
/// Returns `Ok(())` if the checksums match, `Err` otherwise.
pub fn verify_checksum(
data: &[u8],
expected: &str,
checksum_type: ChecksumType,
) -> Result<(), String> {
let actual = match checksum_type {
ChecksumType::Sha256 => {
let mut hasher = Sha256::new();
hasher.update(data);
format!("{:x}", hasher.finalize())
}
ChecksumType::Sha512 => {
let mut hasher = Sha512::new();
hasher.update(data);
format!("{:x}", hasher.finalize())
}
};
if actual.eq_ignore_ascii_case(expected) {
Ok(())
} else {
Err(format!(
"Checksum mismatch: expected {}..., got {}...",
&expected[..8.min(expected.len())],
&actual[..8.min(actual.len())]
))
}
}
#[cfg(test)]
mod tests {
use super::*;
#[test]
fn test_detect_checksum_type_empty() {
assert_eq!(detect_checksum_type("").unwrap(), None);
}
#[test]
fn test_detect_checksum_type_md5_rejected() {
let result = detect_checksum_type("a".repeat(32).as_str());
assert!(result.is_err());
assert!(result.unwrap_err().contains("MD5"));
}
#[test]
fn test_detect_checksum_type_sha1_rejected() {
let result = detect_checksum_type("a".repeat(40).as_str());
assert!(result.is_err());
assert!(result.unwrap_err().contains("SHA1"));
}
#[test]
fn test_detect_checksum_type_sha256() {
assert_eq!(
detect_checksum_type("a".repeat(64).as_str()).unwrap(),
Some(ChecksumType::Sha256)
);
}
#[test]
fn test_detect_checksum_type_sha512() {
assert_eq!(
detect_checksum_type("a".repeat(128).as_str()).unwrap(),
Some(ChecksumType::Sha512)
);
}
#[test]
fn test_detect_checksum_type_invalid_length() {
let result = detect_checksum_type("a".repeat(50).as_str());
assert!(result.is_err());
assert!(result.unwrap_err().contains("Invalid checksum length"));
}
#[test]
fn test_verify_checksum_sha256_valid() {
let data = b"hello";
let expected = "2cf24dba5fb0a30e26e83b2ac5b9e29e1b161e5c1fa7425e73043362938b9824";
assert!(verify_checksum(data, expected, ChecksumType::Sha256).is_ok());
}
#[test]
fn test_verify_checksum_sha256_invalid() {
let data = b"hello";
let expected = "0000000000000000000000000000000000000000000000000000000000000000";
assert!(verify_checksum(data, expected, ChecksumType::Sha256).is_err());
}
#[test]
fn test_verify_checksum_sha512_valid() {
let data = b"hello";
let expected = "9b71d224bd62f3785d96d46ad3ea3d73319bfbc2890caadae2dff72519673ca72323c3d99ba5c11d7c7acc6e14b8c5da0c4663475c2e5c3adef46f73bcdec043";
assert!(verify_checksum(data, expected, ChecksumType::Sha512).is_ok());
}
#[test]
fn test_verify_checksum_sha512_invalid() {
let data = b"hello";
let expected = "0000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000";
assert!(verify_checksum(data, expected, ChecksumType::Sha512).is_err());
}
}
@@ -1,169 +0,0 @@
// Code in this module PARTIALLY OR FULLY utilized AI Coding Agent.
use crate::finalize::plugin::PluginError;
use std::path::PathBuf;
use tokio::task;
/// Compression type for tar archives.
#[derive(Debug, Clone, Copy, PartialEq)]
pub enum CompressionType {
/// Gzip compressed (.tar.gz, .tgz)
Gzip,
/// Zstd compressed (.tar.zst, .tar.zstd, .tzst)
Zstd,
/// Uncompressed tar (.tar)
None,
}
impl CompressionType {
/// Detects compression type from file path extension.
///
/// Supports: .tar.gz, .tgz (Gzip), .tar.zst, .tar.zstd, .tzst (Zstd), .tar (None).
pub fn from_path(path: &std::path::Path) -> Result<Self, String> {
let name = path
.file_name()
.and_then(|n| n.to_str())
.unwrap_or("")
.to_lowercase();
if name.ends_with(".tar.gz") || name.ends_with(".tgz") {
Ok(CompressionType::Gzip)
} else if name.ends_with(".tar.zst")
|| name.ends_with(".tar.zstd")
|| name.ends_with(".tzst")
{
Ok(CompressionType::Zstd)
} else if name.ends_with(".tar") {
Ok(CompressionType::None)
} else {
Err(format!(
"Unsupported archive format: '{}'. Supported formats: .tar, .tar.gz, .tgz, .tar.zst, .tar.zstd, .tzst",
name
))
}
}
}
/// Extracts a tar archive to the destination directory.
///
/// Removes existing destination directory if present, creates a fresh directory,
/// then unpacks the archive using the appropriate decompression based on `compression`.
pub async fn extract_tar(
archive: &PathBuf,
destdir: &PathBuf,
compression: CompressionType,
) -> Result<(), PluginError> {
if destdir.exists() {
tokio::fs::remove_dir_all(destdir).await.map_err(|e| {
PluginError::GeneralError(format!("Failed to remove existing directory: {}", e))
})?;
}
tokio::fs::create_dir_all(destdir)
.await
.map_err(|e| PluginError::GeneralError(format!("Failed to create directory: {}", e)))?;
let archive = archive.clone();
let destdir = destdir.clone();
task::spawn_blocking(move || {
let file = std::fs::File::open(&archive)
.map_err(|e| PluginError::GeneralError(format!("Failed to open archive: {}", e)))?;
match compression {
CompressionType::Gzip => {
let decoder = flate2::read::GzDecoder::new(file);
let mut archive = tar::Archive::new(decoder);
archive.unpack(&destdir).map_err(|e| {
PluginError::GeneralError(format!("Failed to unpack archive: {}", e))
})?;
}
CompressionType::Zstd => {
let decoder = zstd::stream::read::Decoder::new(file).map_err(|e| {
PluginError::GeneralError(format!("Failed to create zstd decoder: {}", e))
})?;
let mut archive = tar::Archive::new(decoder);
archive.unpack(&destdir).map_err(|e| {
PluginError::GeneralError(format!("Failed to unpack archive: {}", e))
})?;
}
CompressionType::None => {
let mut archive = tar::Archive::new(file);
archive.unpack(&destdir).map_err(|e| {
PluginError::GeneralError(format!("Failed to unpack archive: {}", e))
})?;
}
};
Ok::<(), PluginError>(())
})
.await
.map_err(|e| PluginError::GeneralError(format!("Task join error: {}", e)))??;
Ok(())
}
#[cfg(test)]
mod tests {
use super::*;
use std::path::Path;
#[test]
fn test_from_path_tar_gz() {
assert_eq!(
CompressionType::from_path(Path::new("archive.tar.gz")).unwrap(),
CompressionType::Gzip
);
}
#[test]
fn test_from_path_tgz() {
assert_eq!(
CompressionType::from_path(Path::new("archive.tgz")).unwrap(),
CompressionType::Gzip
);
}
#[test]
fn test_from_path_tar_zst() {
assert_eq!(
CompressionType::from_path(Path::new("archive.tar.zst")).unwrap(),
CompressionType::Zstd
);
}
#[test]
fn test_from_path_tar_zstd() {
assert_eq!(
CompressionType::from_path(Path::new("archive.tar.zstd")).unwrap(),
CompressionType::Zstd
);
}
#[test]
fn test_from_path_tzst() {
assert_eq!(
CompressionType::from_path(Path::new("archive.tzst")).unwrap(),
CompressionType::Zstd
);
}
#[test]
fn test_from_path_tar() {
assert_eq!(
CompressionType::from_path(Path::new("archive.tar")).unwrap(),
CompressionType::None
);
}
#[test]
fn test_from_path_unsupported() {
assert!(CompressionType::from_path(Path::new("archive.zip")).is_err());
}
#[test]
fn test_from_path_case_insensitive() {
assert_eq!(
CompressionType::from_path(Path::new("archive.TAR.GZ")).unwrap(),
CompressionType::Gzip
);
}
}
@@ -1,185 +0,0 @@
// Code in this module PARTIALLY OR FULLY utilized AI Coding Agent.
use crate::finalize::plugin::PluginError;
use std::path::Path;
use tokio::process::Command;
/// Version specification for a git repository.
#[derive(Debug, PartialEq)]
pub enum GitVersion {
/// No version specified (invalid for fetch operations)
None,
/// Single version string (tag or commit hash)
Single(String),
/// Mixed: try tag first, fallback to commit hash
Mixed(String, String),
}
/// Parses a git URL with optional version specification.
///
/// Format: `url` (no version), `url@tag` (single version), `url@tag:commit` (mixed version).
/// Returns (base_url, GitVersion).
pub fn parse_git_url(url: &str) -> (String, GitVersion) {
if let Some((base, version)) = url.split_once('@') {
if version.contains(":") {
let (tag, commit) = version.split_once(':').unwrap();
return (
base.to_string(),
GitVersion::Mixed(tag.to_string(), commit.to_string()),
);
}
return (base.to_string(), GitVersion::Single(version.to_string()));
}
(url.to_string(), GitVersion::None)
}
/// Clones a git repository and checks out the specified version.
///
/// Removes existing destination directory, performs shallow or full clone,
/// then checks out the given version (tag or commit).
pub async fn git_clone_and_checkout(
baseurl: &str,
version: &str,
dest: &Path,
name: &str,
shallow: bool,
) -> Result<(), PluginError> {
if dest.exists() {
tokio::fs::remove_dir_all(dest)
.await
.map_err(|e| PluginError::FetchError {
name: name.to_string(),
url: baseurl.to_string(),
reason: format!("Failed to remove existing directory: {}", e),
})?;
}
let mut cmd = Command::new("git");
cmd.arg("clone");
if shallow {
cmd.arg("--depth").arg("1");
}
cmd.arg(baseurl).arg(dest);
let output = cmd.output().await.map_err(|e| PluginError::FetchError {
name: name.to_string(),
url: baseurl.to_string(),
reason: e.to_string(),
})?;
if !output.status.success() {
return Err(PluginError::FetchError {
name: name.to_string(),
url: baseurl.to_string(),
reason: String::from_utf8_lossy(&output.stderr).to_string(),
});
}
let mut cmd = Command::new("git");
cmd.arg("checkout").arg(version).current_dir(dest);
let output = cmd.output().await.map_err(|e| PluginError::FetchError {
name: name.to_string(),
url: baseurl.to_string(),
reason: e.to_string(),
})?;
if !output.status.success() {
return Err(PluginError::FetchError {
name: name.to_string(),
url: baseurl.to_string(),
reason: String::from_utf8_lossy(&output.stderr).to_string(),
});
}
Ok(())
}
/// Fetches a plugin from a git repository.
///
/// Parses the URL for version info, then clones and checks out the appropriate ref.
/// For Mixed version, tries the tag first and falls back to the commit hash.
pub async fn fetch_git_plugin(
path: &str,
name: &str,
basedir: &Path,
shallow: Option<bool>,
) -> Result<(), PluginError> {
let (baseurl, version) = parse_git_url(path);
match version {
GitVersion::None => {
log::error!("Version should be specified for git plugin URL: {}", path);
Err(PluginError::FetchError {
name: name.to_string(),
url: path.to_string(),
reason: "Version should be specified as either a tag or a commit hash.".to_string(),
})
}
GitVersion::Single(version) => {
log::debug!("Fetching git plugin: {}@{}", name, version);
let shallow_flag = shallow.unwrap_or(false);
git_clone_and_checkout(
&baseurl,
&version.to_string(),
&basedir.join(name),
name,
shallow_flag,
)
.await
}
GitVersion::Mixed(tag, commit) => {
log::debug!("Fetching git plugin: {}@{}/{}", name, tag, commit);
let shallow_flag = shallow.unwrap_or(false);
let result =
git_clone_and_checkout(&baseurl, &tag, &basedir.join(name), name, shallow_flag)
.await;
if result.is_err() {
log::warn!(
"Failed to clone with tag, trying commit hash. URL: {}",
path
);
return git_clone_and_checkout(&baseurl, &commit, &basedir.join(name), name, false)
.await;
}
result
}
}
}
#[cfg(test)]
mod tests {
use super::*;
#[test]
fn test_parse_git_url_no_version() {
let (url, version) = parse_git_url("https://github.com/user/repo");
assert_eq!(url, "https://github.com/user/repo");
assert_eq!(version, GitVersion::None);
}
#[test]
fn test_parse_git_url_single_tag() {
let (url, version) = parse_git_url("https://github.com/user/repo@v1.0.0");
assert_eq!(url, "https://github.com/user/repo");
assert_eq!(version, GitVersion::Single("v1.0.0".to_string()));
}
#[test]
fn test_parse_git_url_mixed_version() {
let (url, version) = parse_git_url("https://github.com/user/repo@main:abc123");
assert_eq!(url, "https://github.com/user/repo");
assert_eq!(
version,
GitVersion::Mixed("main".to_string(), "abc123".to_string())
);
}
#[test]
fn test_parse_git_url_complex_url() {
let (url, version) = parse_git_url("git@github.com:user/repo.git@tag:commit");
assert_eq!(url, "git@github.com:user/repo.git");
assert_eq!(
version,
GitVersion::Mixed("tag".to_string(), "commit".to_string())
);
}
#[test]
fn test_parse_git_url_single_with_at_in_path() {
let (url, version) = parse_git_url("https://github.com/user/repo@my-repo@v1.0");
assert_eq!(url, "https://github.com/user/repo@my-repo");
assert_eq!(version, GitVersion::Single("v1.0".to_string()));
}
}
@@ -1,112 +0,0 @@
// Code in this module PARTIALLY OR FULLY utilized AI Coding Agent.
use super::checksum::{detect_checksum_type, verify_checksum};
use super::extract::{CompressionType, extract_tar};
use crate::finalize::plugin::PluginError;
use std::path::Path;
pub async fn download_to_file(url: &str, dest: &Path, name: &str) -> Result<(), PluginError> {
let client = reqwest::Client::builder()
.timeout(std::time::Duration::from_secs(300))
.build()
.map_err(|e| PluginError::GeneralError(format!("Failed to build HTTP client: {}", e)))?;
let response = client
.get(url)
.send()
.await
.map_err(|e| PluginError::FetchError {
name: name.to_string(),
url: url.to_string(),
reason: format!("HTTP request failed: {}", e),
})?;
let status = response.status();
if !status.is_success() {
return Err(PluginError::FetchError {
name: name.to_string(),
url: url.to_string(),
reason: format!("HTTP error: {}", status),
});
}
let bytes = response
.bytes()
.await
.map_err(|e| PluginError::FetchError {
name: name.to_string(),
url: url.to_string(),
reason: format!("Failed to read response body: {}", e),
})?;
tokio::fs::write(dest, &bytes)
.await
.map_err(|e| PluginError::FetchError {
name: name.to_string(),
url: url.to_string(),
reason: format!("Failed to write file: {}", e),
})?;
Ok(())
}
fn extract_archive_extension(url: &str) -> &str {
let path_part = url
.split('?')
.next()
.unwrap_or(url)
.split('#')
.next()
.unwrap_or(url);
let file_part = path_part.rsplit('/').next().unwrap_or("");
let lower = file_part.to_lowercase();
if lower.ends_with(".tar.gz") || lower.ends_with(".tgz") {
".tar.gz"
} else if lower.ends_with(".tar.zst")
|| lower.ends_with(".tar.zstd")
|| lower.ends_with(".tzst")
{
".tar.zst"
} else if lower.ends_with(".tar") {
".tar"
} else {
".tar"
}
}
pub async fn fetch_http_plugin(
url: &str,
name: &str,
basedir: &Path,
checksum_str: Option<&str>,
) -> Result<(), PluginError> {
let temp_dir = std::env::temp_dir().join("workshop-plugins");
tokio::fs::create_dir_all(&temp_dir).await.ok();
let ext = extract_archive_extension(url);
let temp_file = temp_dir.join(format!("{}-{}{}", name, uuid::Uuid::new_v4(), ext));
download_to_file(url, &temp_file, name).await?;
if let Some(checksum) = checksum_str {
if !checksum.is_empty() {
let bytes = tokio::fs::read(&temp_file).await.map_err(|e| {
PluginError::GeneralError(format!("Failed to read downloaded file: {}", e))
})?;
let checksum_type =
detect_checksum_type(checksum).map_err(|e| PluginError::GeneralError(e))?;
if let Some(ct) = checksum_type {
verify_checksum(&bytes, checksum, ct).map_err(|e| PluginError::GeneralError(e))?;
}
}
}
let compression =
CompressionType::from_path(&temp_file).map_err(|e| PluginError::GeneralError(e))?;
let destdir = basedir.join(name);
extract_tar(&temp_file, &destdir, compression).await?;
tokio::fs::remove_file(&temp_file).await.ok();
Ok(())
}
@@ -1,84 +0,0 @@
// Code in this module PARTIALLY OR FULLY utilized AI Coding Agent.
use serde::{Deserialize, Serialize};
/// Supported checksum types for verifying downloaded artifacts.
#[derive(Debug, Clone, PartialEq)]
pub enum ChecksumType {
/// SHA-256 hash (64 hex characters)
Sha256,
/// SHA-512 hash (128 hex characters)
Sha512,
}
/// Arguments for fetching a plugin artifact.
///
/// Contains optional checksum verification and shallow clone settings.
#[derive(Debug, Deserialize, Serialize, Clone, Default)]
pub struct FetchArgument {
/// SHA256 or SHA512 hex string for integrity verification.
#[serde(default)]
pub checksum: Option<String>,
/// Perform a shallow clone (single revision only).
#[serde(default)]
pub shallow: Option<bool>,
}
impl FetchArgument {
/// Returns the checksum if present and non-empty.
pub fn get_checksum(&self) -> Option<&str> {
self.checksum.as_deref().filter(|s| !s.is_empty())
}
}
#[cfg(test)]
mod tests {
use super::*;
#[test]
fn test_get_checksum_some_non_empty() {
let arg = FetchArgument {
checksum: Some("abc123".to_string()),
shallow: None,
};
assert_eq!(arg.get_checksum(), Some("abc123"));
}
#[test]
fn test_get_checksum_empty_string() {
let arg = FetchArgument {
checksum: Some("".to_string()),
shallow: None,
};
assert_eq!(arg.get_checksum(), None);
}
#[test]
fn test_get_checksum_none() {
let arg = FetchArgument {
checksum: None,
shallow: None,
};
assert_eq!(arg.get_checksum(), None);
}
#[test]
fn test_serde_roundtrip() {
let arg = FetchArgument {
checksum: Some("deadbeef".to_string()),
shallow: Some(true),
};
let json = serde_json::to_string(&arg).unwrap();
let parsed: FetchArgument = serde_json::from_str(&json).unwrap();
assert_eq!(parsed.checksum, arg.checksum);
assert_eq!(parsed.shallow, arg.shallow);
}
#[test]
fn test_serde_roundtrip_empty() {
let arg = FetchArgument::default();
let json = serde_json::to_string(&arg).unwrap();
let parsed: FetchArgument = serde_json::from_str(&json).unwrap();
assert_eq!(parsed.checksum, arg.checksum);
assert_eq!(parsed.shallow, arg.shallow);
}
}
+17 -10
View File
@@ -10,22 +10,29 @@ pub struct InternalPlugin {
}
pub fn get_internal_plugin() -> Vec<InternalPlugin> {
let mut internal_plugins: Vec<InternalPlugin> = Vec::new();
internal_plugins.push(InternalPlugin {
vec![
InternalPlugin {
name: "in-site-notify",
entry: Box::new(insitenotify::InsiteNotify),
});
internal_plugins.push(InternalPlugin {
},
InternalPlugin {
name: "mail",
entry: Box::new(mail::Mail),
});
internal_plugins.push(InternalPlugin {
},
InternalPlugin {
name: "satori",
entry: Box::new(satori::Satori),
});
internal_plugins.push(InternalPlugin {
},
InternalPlugin {
name: "webhook",
entry: Box::new(webhook::Webhook),
});
internal_plugins
},
]
}
pub fn get_dummy_plugin() -> Vec<InternalPlugin> {
vec![InternalPlugin {
name: "dummy",
entry: Box::new(dummy::Dummy),
}]
}
@@ -1,12 +1,13 @@
use crate::finalize::plugin::PluginError;
use crate::finalize::plugin::PluginMetadata;
use crate::{EventSender, ExecutionContext, finalize::plugin::AsyncPluginFn};
use workshop_engine::{EventSender, ExecutionContext};
use crate::{ finalize::plugin::AsyncPluginFn};
use async_trait::async_trait;
pub struct _Dummy;
pub struct Dummy;
#[async_trait]
impl AsyncPluginFn for _Dummy {
impl AsyncPluginFn for Dummy {
async fn call(
&self,
_metadata: &PluginMetadata,
@@ -14,6 +15,7 @@ impl AsyncPluginFn for _Dummy {
_ctx: &ExecutionContext,
_event_tx: &EventSender,
) -> Result<(), PluginError> {
log::debug!("[NOTIFY_DEBUG] Successfully registered dummy plugin.");
Ok(())
}
}
@@ -1,12 +1,10 @@
use crate::finalize::plugin::PluginMetadata;
use crate::{
EventSender, ExecutionContext,
finalize::{error::PluginError, plugin::AsyncPluginFn},
};
use async_trait::async_trait;
use workshop_engine::{EventSender, ExecutionContext};
pub struct InsiteNotify;
#[async_trait]
impl AsyncPluginFn for InsiteNotify {
async fn call(
@@ -16,6 +14,7 @@ impl AsyncPluginFn for InsiteNotify {
_ctx: &ExecutionContext,
_event_tx: &EventSender,
) -> Result<(), PluginError> {
unimplemented!("Requires workshop-base to be implemented!");
log::info!("in-site-notify: placeholder (always succeeds)");
Ok(())
}
}
@@ -1,18 +1,9 @@
// Code in this module PARTIALLY OR FULLY utilized AI Coding Agent.
use crate::finalize::plugin::PluginError;
use crate::finalize::plugin::PluginMetadata;
use crate::{EventSender, ExecutionContext, finalize::plugin::AsyncPluginFn};
use crate::{
finalize::{error::PluginError, plugin::AsyncPluginFn},
};
use async_trait::async_trait;
use lettre::message::{MultiPart, SinglePart, header};
use lettre::{AsyncSmtpTransport, AsyncTransport, Message, Tokio1Executor};
use std::time::Duration;
use tokio::time::timeout;
mod config;
mod template;
pub use config::*;
use template::*;
use workshop_engine::{EventSender, ExecutionContext};
pub struct Mail;
@@ -21,182 +12,10 @@ impl AsyncPluginFn for Mail {
async fn call(
&self,
_metadata: &PluginMetadata,
argument: serde_yaml::Value,
ctx: &ExecutionContext,
_argument: serde_yaml::Value,
_ctx: &ExecutionContext,
_event_tx: &EventSender,
) -> Result<(), PluginError> {
let config: MailConfig = serde_yaml::from_value(argument)
.map_err(|e| PluginError::GeneralError(format!("Invalid mail config: {}", e)))?;
if let Err(errors) = config.validate() {
return Err(PluginError::GeneralError(format!(
"Mail config validation failed: {:?}",
errors
)));
}
let template_data = prepare_template_data(ctx);
let (subject_template, body_template_html, body_template_text) = match config.schema {
Schema::Default => {
let subject = config
.subject
.as_deref()
.unwrap_or(DefaultTemplates::SUBJECT);
let body_html = config
.body
.as_deref()
.unwrap_or(DefaultTemplates::BODY_HTML);
(
subject.to_string(),
body_html.to_string(),
DefaultTemplates::BODY_TEXT.to_string(),
)
}
Schema::Custom => {
let subject = config.subject.clone().unwrap();
let body_html = config.body.clone().unwrap();
(subject, body_html.clone(), body_html)
}
};
let subject = render_template(&subject_template, &template_data)
.map_err(|e| PluginError::GeneralError(format!("Failed to render subject: {}", e)))?;
let body_html = render_template(&body_template_html, &template_data)
.map_err(|e| PluginError::GeneralError(format!("Failed to render body_html: {}", e)))?;
let body_text = render_template(&body_template_text, &template_data).map_err(|e| {
PluginError::GeneralError(format!("Failed to render body_text(?): {}", e))
})?;
let email = build_email(&config, subject, body_html, body_text)?;
send_email(&config, email).await
todo!();
}
}
fn build_email(
config: &MailConfig,
subject: String,
body_html: String,
body_text: String,
) -> Result<Message, PluginError> {
let from_str = config.from.to_string();
let from: lettre::message::Mailbox = from_str.parse().map_err(|e| {
PluginError::GeneralError(format!("Invalid from address '{}': {}", from_str, e))
})?;
let to_addrs: Vec<String> = config.to.to_vec();
if to_addrs.is_empty() {
return Err(PluginError::GeneralError(
"No recipients specified".to_string(),
));
}
let mut message_builder = Message::builder().from(from).subject(subject);
for addr in &to_addrs {
let mailbox: lettre::message::Mailbox = addr.parse().map_err(|e| {
PluginError::GeneralError(format!("Invalid to address '{}': {}", addr, e))
})?;
message_builder = message_builder.to(mailbox);
}
if let Some(cc) = &config.cc {
for addr in cc.to_vec() {
let mailbox: lettre::message::Mailbox = addr.parse().map_err(|e| {
PluginError::GeneralError(format!("Invalid cc address '{}': {}", addr, e))
})?;
message_builder = message_builder.cc(mailbox);
}
}
if let Some(bcc) = &config.bcc {
for addr in bcc.to_vec() {
let mailbox: lettre::message::Mailbox = addr.parse().map_err(|e| {
PluginError::GeneralError(format!("Invalid bcc address '{}': {}", addr, e))
})?;
message_builder = message_builder.bcc(mailbox);
}
}
if let Some(reply_to) = &config.reply_to {
let mailbox: lettre::message::Mailbox = reply_to.parse().map_err(|e| {
PluginError::GeneralError(format!("Invalid reply-to address '{}': {}", reply_to, e))
})?;
message_builder = message_builder.reply_to(mailbox);
}
let message = message_builder
.multipart(
MultiPart::alternative()
.singlepart(
SinglePart::builder()
.header(header::ContentType::TEXT_PLAIN)
.body(body_text),
)
.singlepart(
SinglePart::builder()
.header(header::ContentType::TEXT_HTML)
.body(body_html),
),
)
.map_err(|e| PluginError::GeneralError(format!("Failed to build email: {}", e)))?;
Ok(message)
}
async fn send_email(config: &MailConfig, email: Message) -> Result<(), PluginError> {
let connect_timeout = if config.smtp.connect_timeout_ms == 0 {
None
} else {
Some(Duration::from_millis(config.smtp.connect_timeout_ms))
};
let mut transport_builder = match config.smtp.encryption {
Encryption::Tls => {
log::debug!("Using TLS encryption");
AsyncSmtpTransport::<Tokio1Executor>::relay(&config.smtp.host)
.map_err(|e| PluginError::GeneralError(format!("Invalid SMTP host: {}", e)))?
}
Encryption::Starttls => {
log::debug!("Using StartTLS encryption");
AsyncSmtpTransport::<Tokio1Executor>::starttls_relay(&config.smtp.host)
.map_err(|e| PluginError::GeneralError(format!("Invalid SMTP host: {}", e)))?
}
Encryption::None => {
log::debug!("Using NO encryption");
AsyncSmtpTransport::<Tokio1Executor>::builder_dangerous(&config.smtp.host)
}
};
transport_builder = transport_builder.port(config.smtp.port);
match config.smtp.auth.auth_type {
AuthType::Password => {
log::debug!("Using password auth");
transport_builder = transport_builder.credentials(
(
config.smtp.auth.username.clone(),
config.smtp.auth.password.clone(),
)
.into(),
);
}
AuthType::None => {
log::debug!("Using no auth");
}
}
let transport = transport_builder.build();
let result = match connect_timeout {
Some(t) => timeout(t, transport.send(email)).await,
None => Ok(transport.send(email).await),
};
result
.map_err(|_| PluginError::GeneralError("SMTP connection timed out".to_string()))?
.map_err(|e| PluginError::GeneralError(format!("SMTP error: {}", e)))?;
Ok(())
}
@@ -1,490 +0,0 @@
// Code in this module PARTIALLY OR FULLY utilized AI Coding Agent.
use workshop_engine::deserialize_duration_ms;
use serde::Deserialize;
/// Email address representation supporting plain string or structured object.
#[derive(Debug, Deserialize, Clone)]
#[serde(untagged)]
pub enum EmailAddress {
/// Plain email address string.
String(String),
/// Structured email with display name.
Object { name: String, email: String },
}
impl EmailAddress {
/// Returns the formatted address string (e.g., "user@example.com" or "Name <user@example.com>").
pub fn to_string(&self) -> String {
match self {
EmailAddress::String(email) => email.clone(),
EmailAddress::Object { name, email } => {
if name.is_empty() {
email.clone()
} else {
format!("{} <{}>", name, email)
}
}
}
}
/// Returns the raw email address without formatting.
pub fn email(&self) -> &str {
match self {
EmailAddress::String(email) => email.as_str(),
EmailAddress::Object { email, .. } => email.as_str(),
}
}
}
/// Recipients container supporting single or multiple addresses.
#[derive(Debug, Deserialize, Clone)]
#[serde(untagged)]
pub enum Recipients {
/// Single recipient.
Single(String),
/// Multiple recipients.
Multiple(Vec<String>),
}
impl Recipients {
/// Converts recipients to a flat vector of email strings.
pub fn to_vec(&self) -> Vec<String> {
match self {
Recipients::Single(s) => vec![s.clone()],
Recipients::Multiple(v) => v.clone(),
}
}
}
/// SMTP authentication mechanism.
#[derive(Debug, Deserialize, Clone)]
#[serde(rename_all = "lowercase")]
pub enum AuthType {
/// No authentication.
None,
/// Password-based authentication.
Password,
}
/// SMTP authentication configuration.
#[derive(Debug, Deserialize, Clone)]
pub struct AuthConfig {
/// Authentication mechanism type.
#[serde(rename = "type")]
pub auth_type: AuthType,
/// Username for authentication.
pub username: String,
/// Password for authentication.
pub password: String,
}
/// SMTP connection encryption mode.
#[derive(Debug, Deserialize, Clone)]
#[serde(rename_all = "lowercase")]
pub enum Encryption {
/// Implicit TLS/SSL on connection.
Tls,
/// STARTTLS upgrade after initial plaintext connection.
Starttls,
/// No encryption.
None,
}
/// SMTP server connection configuration.
#[derive(Debug, Deserialize, Clone)]
pub struct SmtpConfig {
/// SMTP server hostname.
pub host: String,
/// SMTP server port.
pub port: u16,
/// Authentication configuration.
pub auth: AuthConfig,
/// Encryption mode.
pub encryption: Encryption,
/// Connection timeout in milliseconds.
#[serde(
default = "default_connect_timeout_ms",
deserialize_with = "deserialize_duration_ms"
)]
pub connect_timeout_ms: u64,
}
fn default_connect_timeout_ms() -> u64 {
30_000
}
/// Mail schema type determining required fields.
#[derive(Debug, Deserialize, Clone)]
#[serde(rename_all = "lowercase")]
pub enum Schema {
/// Default schema with minimal required fields.
Default,
/// Custom schema requiring explicit subject and body.
Custom,
}
fn default_schema() -> Schema {
Schema::Default
}
/// Complete mail notification configuration.
#[derive(Debug, Deserialize, Clone)]
pub struct MailConfig {
/// Primary recipients.
pub to: Recipients,
/// Carbon copy recipients.
#[serde(default)]
pub cc: Option<Recipients>,
/// Blind carbon copy recipients.
#[serde(default)]
pub bcc: Option<Recipients>,
/// Sender address.
pub from: EmailAddress,
/// Reply-to address.
#[serde(default)]
pub reply_to: Option<String>,
/// Schema type determining validation rules.
#[serde(default = "default_schema")]
pub schema: Schema,
/// Email subject (required for custom schema).
pub subject: Option<String>,
/// Email body content (required for custom schema).
pub body: Option<String>,
/// SMTP server configuration.
pub smtp: SmtpConfig,
}
impl MailConfig {
/// Validates the mail configuration according to schema rules.
/// Returns Ok(()) if valid, or Err(vec) with error messages if invalid.
pub fn validate(&self) -> Result<(), Vec<String>> {
let mut errors = Vec::new();
if matches!(self.schema, Schema::Custom) {
if self.subject.is_none() {
errors.push("subject: required when schema=custom".to_string());
}
if self.body.is_none() {
errors.push("body: required when schema=custom".to_string());
}
}
if self.to.to_vec().is_empty() {
errors.push("to: must have at least one recipient".to_string());
}
if errors.is_empty() {
Ok(())
} else {
Err(errors)
}
}
}
/// Built-in email notification templates.
pub struct DefaultTemplates;
impl DefaultTemplates {
/// Default subject template with build status.
pub const SUBJECT: &'static str =
"[{{ build.status | upper }}] {{ pipeline.name }} #{{ build.id }}";
/// Default HTML email body template.
pub const BODY_HTML: &'static str = r#"<!DOCTYPE html>
<html>
<body style="font-family: Arial, sans-serif;">
<h2 style="color: {{ build.status_color }};">
Build {{ build.status | title }}
</h2>
<table>
<tr><td><strong>Pipeline:</strong></td><td>{{ pipeline.name }}</td></tr>
<tr><td><strong>Build ID:</strong></td><td>{{ build.id }}</td></tr>
<tr><td><strong>Status:</strong></td><td>{{ build.status }}</td></tr>
<tr><td><strong>Duration:</strong></td><td>{{ build.duration }}</td></tr>
<tr><td><strong>Author:</strong></td><td>{{ commit.author }}</td></tr>
<tr><td><strong>Message:</strong></td><td>{{ commit.message }}</td></tr>
</table>
<p>
<a href="{{ build.url }}" style="background: #007bff; color: white; padding: 10px 20px; text-decoration: none; border-radius: 5px;">
View Build Details
</a>
</p>
</body>
</html>"#;
/// Default plain text email body template.
pub const BODY_TEXT: &'static str = r#"Build {{ build.status | title }}
==============================
Pipeline: {{ pipeline.name }}
Build ID: {{ build.id }}
Status: {{ build.status }}
Duration: {{ build.duration }}
Commit: {{ commit.hash }}
Author: {{ commit.author }}
Message: {{ commit.message }}
View details: {{ build.url }}"#;
}
#[cfg(test)]
mod tests {
use super::*;
#[test]
fn test_email_address_string_to_string() {
let addr = EmailAddress::String("test@example.com".to_string());
assert_eq!(addr.to_string(), "test@example.com");
}
#[test]
fn test_email_address_object_with_name_to_string() {
let addr = EmailAddress::Object {
name: "Test User".to_string(),
email: "test@example.com".to_string(),
};
assert_eq!(addr.to_string(), "Test User <test@example.com>");
}
#[test]
fn test_email_address_object_without_name_to_string() {
let addr = EmailAddress::Object {
name: "".to_string(),
email: "test@example.com".to_string(),
};
assert_eq!(addr.to_string(), "test@example.com");
}
#[test]
fn test_email_address_string_email() {
let addr = EmailAddress::String("test@example.com".to_string());
assert_eq!(addr.email(), "test@example.com");
}
#[test]
fn test_email_address_object_email() {
let addr = EmailAddress::Object {
name: "Test User".to_string(),
email: "test@example.com".to_string(),
};
assert_eq!(addr.email(), "test@example.com");
}
#[test]
fn test_recipients_single_to_vec() {
let recipients = Recipients::Single("test@example.com".to_string());
let vec = recipients.to_vec();
assert_eq!(vec, vec!["test@example.com"]);
}
#[test]
fn test_recipients_multiple_to_vec() {
let recipients = Recipients::Multiple(vec![
"a@example.com".to_string(),
"b@example.com".to_string(),
]);
let vec = recipients.to_vec();
assert_eq!(vec, vec!["a@example.com", "b@example.com"]);
}
#[test]
fn test_mail_config_validate_valid_default_schema() {
let config = MailConfig {
to: Recipients::Single("to@example.com".to_string()),
cc: None,
bcc: None,
from: EmailAddress::String("from@example.com".to_string()),
reply_to: None,
schema: Schema::Default,
subject: None,
body: None,
smtp: SmtpConfig {
host: "smtp.example.com".to_string(),
port: 587,
auth: AuthConfig {
auth_type: AuthType::None,
username: "".to_string(),
password: "".to_string(),
},
encryption: Encryption::Tls,
connect_timeout_ms: 30_000,
},
};
assert!(config.validate().is_ok());
}
#[test]
fn test_mail_config_validate_valid_custom_schema() {
let config = MailConfig {
to: Recipients::Single("to@example.com".to_string()),
cc: None,
bcc: None,
from: EmailAddress::String("from@example.com".to_string()),
reply_to: None,
schema: Schema::Custom,
subject: Some("Test Subject".to_string()),
body: Some("Test Body".to_string()),
smtp: SmtpConfig {
host: "smtp.example.com".to_string(),
port: 587,
auth: AuthConfig {
auth_type: AuthType::None,
username: "".to_string(),
password: "".to_string(),
},
encryption: Encryption::Tls,
connect_timeout_ms: 30_000,
},
};
assert!(config.validate().is_ok());
}
#[test]
fn test_mail_config_validate_custom_schema_missing_subject() {
let config = MailConfig {
to: Recipients::Single("to@example.com".to_string()),
cc: None,
bcc: None,
from: EmailAddress::String("from@example.com".to_string()),
reply_to: None,
schema: Schema::Custom,
subject: None,
body: Some("Test Body".to_string()),
smtp: SmtpConfig {
host: "smtp.example.com".to_string(),
port: 587,
auth: AuthConfig {
auth_type: AuthType::None,
username: "".to_string(),
password: "".to_string(),
},
encryption: Encryption::Tls,
connect_timeout_ms: 30_000,
},
};
let result = config.validate();
assert!(result.is_err());
assert!(result.unwrap_err().iter().any(|e| e.contains("subject")));
}
#[test]
fn test_mail_config_validate_custom_schema_missing_body() {
let config = MailConfig {
to: Recipients::Single("to@example.com".to_string()),
cc: None,
bcc: None,
from: EmailAddress::String("from@example.com".to_string()),
reply_to: None,
schema: Schema::Custom,
subject: Some("Test Subject".to_string()),
body: None,
smtp: SmtpConfig {
host: "smtp.example.com".to_string(),
port: 587,
auth: AuthConfig {
auth_type: AuthType::None,
username: "".to_string(),
password: "".to_string(),
},
encryption: Encryption::Tls,
connect_timeout_ms: 30_000,
},
};
let result = config.validate();
assert!(result.is_err());
assert!(result.unwrap_err().iter().any(|e| e.contains("body")));
}
#[test]
fn test_mail_config_validate_empty_recipients() {
let config = MailConfig {
to: Recipients::Multiple(vec![]),
cc: None,
bcc: None,
from: EmailAddress::String("from@example.com".to_string()),
reply_to: None,
schema: Schema::Default,
subject: None,
body: None,
smtp: SmtpConfig {
host: "smtp.example.com".to_string(),
port: 587,
auth: AuthConfig {
auth_type: AuthType::None,
username: "".to_string(),
password: "".to_string(),
},
encryption: Encryption::Tls,
connect_timeout_ms: 30_000,
},
};
let result = config.validate();
assert!(result.is_err());
assert!(result.unwrap_err().iter().any(|e| e.contains("recipient")));
}
#[test]
fn test_email_address_serde_string() {
let json = r#""test@example.com""#;
let addr: EmailAddress = serde_json::from_str(json).unwrap();
assert!(matches!(addr, EmailAddress::String(s) if s == "test@example.com"));
}
#[test]
fn test_email_address_serde_object() {
let json = r#"{"name":"Test User","email":"test@example.com"}"#;
let addr: EmailAddress = serde_json::from_str(json).unwrap();
assert!(
matches!(addr, EmailAddress::Object { name, email } if name == "Test User" && email == "test@example.com")
);
}
#[test]
fn test_recipients_serde_single() {
let json = r#""single@example.com""#;
let recipients: Recipients = serde_json::from_str(json).unwrap();
assert!(matches!(recipients, Recipients::Single(s) if s == "single@example.com"));
}
#[test]
fn test_recipients_serde_multiple() {
let json = r#"["a@example.com","b@example.com"]"#;
let recipients: Recipients = serde_json::from_str(json).unwrap();
assert!(
matches!(recipients, Recipients::Multiple(v) if v == vec!["a@example.com".to_string(), "b@example.com".to_string()])
);
}
#[test]
fn test_auth_type_serde() {
#[derive(serde::Deserialize, Debug)]
struct Test {
#[serde(rename = "type")]
auth_type: AuthType,
}
let json = r#"{"type":"password"}"#;
let t: Test = serde_json::from_str(json).unwrap();
assert!(matches!(t.auth_type, AuthType::Password));
}
#[test]
fn test_encryption_serde() {
#[derive(serde::Deserialize, Debug)]
struct Test {
encryption: Encryption,
}
let json = r#"{"encryption":"tls"}"#;
let t: Test = serde_json::from_str(json).unwrap();
assert!(matches!(t.encryption, Encryption::Tls));
}
#[test]
fn test_schema_serde() {
#[derive(serde::Deserialize, Debug)]
struct Test {
schema: Schema,
}
let json = r#"{"schema":"custom"}"#;
let t: Test = serde_json::from_str(json).unwrap();
assert!(matches!(t.schema, Schema::Custom));
}
}
@@ -1,101 +0,0 @@
// Code in this module PARTIALLY OR FULLY utilized AI Coding Agent.
use crate::ExecutionContext;
use minijinja::Environment;
use std::collections::HashMap;
/// Builds template context data from execution context.
/// Populates pipeline, build, and commit information for template rendering.
pub fn prepare_template_data(ctx: &ExecutionContext) -> HashMap<String, serde_json::Value> {
let mut data = HashMap::new();
data.insert(
"pipeline".to_string(),
serde_json::json!({
"name": ctx.pipeline_name,
}),
);
data.insert(
"build".to_string(),
serde_json::json!({
"id": ctx.task_id,
"status": "success",
"status_color": "green",
"duration": "0s",
"url": "",
}),
);
data.insert(
"commit".to_string(),
serde_json::json!({
"hash": "",
"short_hash": "",
"message": "",
"author": "",
"author_email": "",
}),
);
data
}
/// Renders a minijinja template string with the provided data map.
/// Returns the rendered string on success, or a String error message on failure.
pub fn render_template(
template: &str,
data: &HashMap<String, serde_json::Value>,
) -> Result<String, String> {
let env = Environment::new();
let tmpl = env
.template_from_str(template)
.map_err(|e| format!("Template parse error: {}", e))?;
let result = tmpl
.render(data)
.map_err(|e| format!("Template render error: {}", e))?;
Ok(result)
}
#[cfg(test)]
mod tests {
use super::*;
#[test]
fn test_render_template_simple() {
let template = "Hello {{ name }}!";
let mut data = HashMap::new();
data.insert(
"name".to_string(),
serde_json::Value::String("World".to_string()),
);
let result = render_template(template, &data).unwrap();
assert_eq!(result, "Hello World!");
}
#[test]
fn test_render_template_with_build_status() {
let template = "Build {{ build.status }} for pipeline {{ pipeline.name }}";
let mut data = HashMap::new();
data.insert(
"build".to_string(),
serde_json::json!({ "status": "success" }),
);
data.insert(
"pipeline".to_string(),
serde_json::json!({ "name": "test-pipeline" }),
);
let result = render_template(template, &data).unwrap();
assert_eq!(result, "Build success for pipeline test-pipeline");
}
#[test]
fn test_render_template_invalid_syntax() {
let template = "{{ Unterminated";
let data = HashMap::new();
let result = render_template(template, &data);
assert!(result.is_err());
}
}
@@ -1,6 +1,7 @@
use crate::finalize::plugin::PluginError;
use crate::finalize::plugin::PluginMetadata;
use crate::{EventSender, ExecutionContext, finalize::plugin::AsyncPluginFn};
use workshop_engine::{EventSender, ExecutionContext};
use crate::{ finalize::plugin::AsyncPluginFn};
use async_trait::async_trait;
pub struct Satori;
@@ -1,6 +1,7 @@
use crate::finalize::plugin::PluginError;
use crate::finalize::plugin::PluginMetadata;
use crate::{EventSender, ExecutionContext, finalize::plugin::AsyncPluginFn};
use workshop_engine::{EventSender, ExecutionContext};
use crate::{ finalize::plugin::AsyncPluginFn};
use async_trait::async_trait;
pub struct Webhook;
+7 -74
View File
@@ -1,3 +1,4 @@
use crate::finalize::NotificationTemplateDef;
use crate::notify::types::NotificationRenderer;
use serde::Deserialize;
use std::path::PathBuf;
@@ -18,10 +19,14 @@ pub struct PluginMetadata {
#[serde(default)]
pub renderer: Option<NotificationRenderer>,
/// Additional YAML fields not captured by struct.
/// Additional YAML fields.
#[serde(flatten)]
pub value: serde_yaml::Value,
/// Plugin-defined default templates, auto-registered as <name>.fallback
#[serde(default)]
pub templates: Option<Vec<NotificationTemplateDef>>,
/// Filesystem path to plugin definition.
#[serde(skip)]
pub fspath: PathBuf,
@@ -36,6 +41,7 @@ impl Default for PluginMetadata {
plugin_type: PluginType::Unknown,
value: serde_yaml::Value::Null,
renderer: None,
templates: None,
fspath: PathBuf::new(),
}
}
@@ -50,76 +56,3 @@ pub enum PluginType {
Rhai,
Dylib,
}
#[cfg(test)]
mod tests {
use super::*;
#[test]
fn test_plugin_metadata_default() {
let meta = PluginMetadata::default();
assert_eq!(meta.name, "");
assert_eq!(meta.version, "");
assert_eq!(meta.entrypoint, "");
assert_eq!(meta.plugin_type, PluginType::Unknown);
}
#[test]
fn test_plugin_type_default() {
let pt = PluginType::default();
assert_eq!(pt, PluginType::Unknown);
}
#[test]
fn test_plugin_type_equality() {
assert_eq!(PluginType::Shell, PluginType::Shell);
assert_eq!(PluginType::Dylib, PluginType::Dylib);
assert_ne!(PluginType::Shell, PluginType::Dylib);
}
#[test]
fn test_plugin_metadata_clone() {
let meta = PluginMetadata {
name: "test".to_string(),
version: "1.0".to_string(),
entrypoint: "/bin/test".to_string(),
plugin_type: PluginType::Shell,
value: serde_yaml::Value::Null,
fspath: PathBuf::from("/path/to/plugin"),
renderer: None,
};
let cloned = meta.clone();
assert_eq!(cloned.name, meta.name);
assert_eq!(cloned.version, meta.version);
assert_eq!(cloned.plugin_type, meta.plugin_type);
}
#[test]
fn test_plugin_type_serde_unknown() {
let yaml = "Unknown";
let pt: PluginType = serde_yaml::from_str(yaml).unwrap();
assert_eq!(pt, PluginType::Unknown);
}
#[test]
fn test_plugin_type_serde_shell() {
let yaml = "Shell";
let pt: PluginType = serde_yaml::from_str(yaml).unwrap();
assert_eq!(pt, PluginType::Shell);
}
#[test]
fn test_plugin_metadata_serde() {
let yaml = r#"
name: test-plugin
version: "2.0"
entrypoint: ./bin/test
plugin_type: Shell
"#;
let meta: PluginMetadata = serde_yaml::from_str(yaml).unwrap();
assert_eq!(meta.name, "test-plugin");
assert_eq!(meta.version, "2.0");
assert_eq!(meta.entrypoint, "./bin/test");
assert_eq!(meta.plugin_type, PluginType::Shell);
}
}
+2 -1
View File
@@ -1,6 +1,7 @@
use crate::finalize::plugin::PluginError;
use crate::finalize::plugin::PluginMetadata;
use crate::{EventSender, ExecutionContext, finalize::plugin::AsyncPluginFn};
use workshop_engine::{EventSender, ExecutionContext};
use crate::{ finalize::plugin::AsyncPluginFn};
use async_trait::async_trait;
pub struct Rhai;
+7 -6
View File
@@ -1,8 +1,9 @@
use crate::Engine;
use workshop_engine::Engine;
use crate::finalize::constant::FINALIZE_SHELL_ENVPREFIX;
use crate::finalize::plugin::PluginError;
use crate::finalize::plugin::PluginMetadata;
use crate::{EventSender, ExecutionContext, finalize::plugin::AsyncPluginFn};
use workshop_engine::{EventSender, ExecutionContext};
use crate::{ finalize::plugin::AsyncPluginFn};
use async_trait::async_trait;
use std::collections::HashMap;
@@ -94,7 +95,7 @@ fn convert_argument(
envvars.insert(prefix.to_string(), value.to_string());
}
serde_yaml::Value::String(value) => {
envvars.insert(prefix.to_string(), format!("{}", value));
envvars.insert(prefix.to_string(), value.to_string());
}
serde_yaml::Value::Number(value) => {
envvars.insert(prefix.to_string(), value.to_string());
@@ -112,12 +113,12 @@ fn convert_argument(
return envvars;
}
let mut array = String::new();
array.push_str("(");
array.push('(');
let mut has_mapping = false;
let mut _envvars: HashMap<String, String> = HashMap::new();
for (index, item) in seq.into_iter().enumerate() {
if index > 0 {
array.push_str(" ");
array.push(' ');
}
let new_prefix = format!("{}_{}", prefix, index);
let subenvvars = convert_argument(new_prefix.as_str(), item.clone(), true);
@@ -136,7 +137,7 @@ fn convert_argument(
if has_mapping {
envvars.extend(_envvars);
} else {
array.push_str(")");
array.push(')');
envvars.insert(prefix.to_string(), array);
}
}
+2 -2
View File
@@ -1,7 +1,7 @@
use crate::ExecutionContext;
use workshop_engine::{DeltaExecutionContext, Engine, EventSender, ExecutionError, CustomCommand};
use workshop_engine::ExecutionContext;
use crate::finalize::plugin::{PluginMap, call_named_plugin};
use std::path::PathBuf;
use workshop_engine::{CustomCommand, DeltaExecutionContext, Engine, EventSender, ExecutionError};
/// Executes post-build finalize hooks, supporting both plugin and shell command hooks.
///
+25 -57
View File
@@ -1,27 +1,16 @@
// Library crate for workshop-executor
// Selective exports: only modules used by external callers or tests
pub mod bare;
pub mod bake;
pub mod daemon;
pub mod error;
pub mod finalize;
pub mod monitor;
pub mod notify;
pub mod prebake;
pub mod socket;
pub mod types;
pub mod utils;
// External crates re-exports
pub use config;
// Re-export commonly used types from workshop-engine
pub use workshop_engine::{Engine, EventSender, ExecutionContext, ExecutionError, ExecutionResult};
// CLI types used by main (keep here since they're used in daemon)
pub use clap::{Parser, Subcommand};
pub use serde::Deserialize;
pub use std::collections::HashMap;
pub use std::path::PathBuf;
pub mod cli {
pub use super::{Deserialize, HashMap, Parser, PathBuf, Subcommand};
pub use clap::Command;
@@ -46,64 +35,43 @@ pub mod cli {
#[arg(long, default_value = "false", env = "HBW_DRY_RUN")]
pub dry_run: bool,
/// Override prebake config path
#[arg(long)]
pub prebake: Option<PathBuf>,
/// Override bake script path
#[arg(long)]
pub bake: Option<PathBuf>,
/// Override bake_base.sh path
#[arg(long)]
pub bake_base: Option<PathBuf>,
/// Override finalize config path
#[arg(long)]
pub finalize: Option<PathBuf>,
/// Debug features (comma-separated). "notify" enables notification debug logging.
#[arg(long, env = "HBW_DEBUG")]
pub debug: Option<String>,
#[command(subcommand)]
pub command: Option<Commands>,
}
#[derive(Subcommand, Debug)]
pub enum Commands {
/// Run pipeline stages in standalone (bare) mode
/// Run individual pipeline stages
Bare {
#[command(subcommand)]
command: BareCommands,
},
/// Run in client mode
Client { config: String },
Daemon {},
}
#[derive(Subcommand, Debug)]
pub enum BareCommands {
/// Prepare a task
Prebake { config: PathBuf },
/// Run a task
Bake {
/// Path to the script to execute
script: PathBuf,
/// Path to bake_base.sh
#[arg(short, long, default_value = "./bake_base.sh")]
bake_base: PathBuf,
#[arg(short, long, default_value = "./prebake.yml")]
prebake: PathBuf,
},
/// Finish a task
Finalize { config: String },
Prebake {},
Bake {},
Finalize {},
}
#[derive(Debug, Deserialize)]
pub struct TaskSpec {
pub subcommand: String,
pub arguments: Vec<String>,
pub options: HashMap<String, String>,
}
impl TaskSpec {
pub fn command(&self, executable: &PathBuf) -> tokio::process::Command {
let mut command = tokio::process::Command::new(executable);
command.arg(&self.subcommand);
for arg in &self.arguments {
command.arg(arg);
}
for (flag, value) in self.options.iter() {
if value.is_empty() {
command.arg(format!("--{}", flag));
} else {
command.arg(format!("--{}={}", flag, value));
}
}
command
}
}
}
+110 -63
View File
@@ -1,13 +1,91 @@
use std::path::Path;
use workshop_baker::types::resource::ResourceRegistry;
use std::path::PathBuf;
use workshop_baker::{
bake::bake,
cli::{BareCommands, Cli, Commands, Parser},
daemon,
finalize::finalize,
prebake::prebake,
bake::bake, cli::{Cli, Commands, Parser},
error::CliError,
finalize::finalize, notify::notify, prebake::prebake,
};
use workshop_engine::{error::HasExitCode, EventSender, ExecutionContext};
use workshop_baker::bare::run_bare;
use workshop_baker::notify::queue::NotificationQueue;
use workshop_engine::{EventSender, ExecutionContext};
fn or_workshop(path: &Option<PathBuf>, name: &str) -> PathBuf {
path.clone().unwrap_or_else(|| PathBuf::from(format!(".workshop/{}", name)))
}
async fn worker_main(cli: Cli) -> Result<(), CliError> {
let prebake_path = cli.prebake.clone().unwrap_or_else(|| {
log::error!("--prebake is required in worker standalone mode");
std::process::exit(1);
});
let bake_path = cli.bake.clone().unwrap_or_else(|| {
log::error!("--bake is required in worker standalone mode");
std::process::exit(1);
});
let finalize_path = cli.finalize.clone().unwrap_or_else(|| {
log::error!("--finalize is required in worker standalone mode");
std::process::exit(1);
});
log::info!("Worker standalone mode");
let mut ctx = ExecutionContext {
standalone: true,
task_id: format!("{}-{}-worker", cli.pipeline, cli.build_id),
pipeline_name: cli.pipeline.clone(),
username: cli.username.clone(),
dry_run: cli.dry_run,
privileged: false,
..Default::default()
};
let (tx, event_rx) = tokio::sync::mpsc::channel(256);
let event_tx: EventSender = Some(tx);
let _event_handle = tokio::spawn(workshop_baker::prebake::event::event_receiver(
event_rx, cli.pipeline.clone(), cli.build_id.clone(),
));
let (to_tx, to_rx) = tokio::sync::mpsc::channel(256);
let (retry_tx, retry_rx) = tokio::sync::mpsc::channel(256);
// Resource registry — populated before stages, consumed by stages
let registry = ResourceRegistry::new();
// TODO: pre-fetch plugins and templates into registry
let notify_path = finalize_path.clone();
let notify_ctx = ctx.clone();
let notify_etx = event_tx.clone();
let debug = cli.debug.clone();
let notify_registry = registry.clone();
let notify_handle = tokio::spawn(async move {
notify(&notify_path, &mut notify_ctx.clone(), notify_etx, to_rx, retry_tx, debug.map(|d| d.contains("notify")).unwrap_or(false), &notify_registry).await
});
let nevent_tx = to_tx.clone();
let mut queue = NotificationQueue::new(to_tx, retry_rx);
let pipe_result: anyhow::Result<()> = async {
prebake(&prebake_path, &cli, None, &mut ctx, event_tx.clone(), Some(nevent_tx)).await?;
let bake_base_path = or_workshop(&cli.bake_base, "bake_base.sh");
let pb_path = or_workshop(&cli.prebake, "prebake.yml");
bake(&bake_path, &bake_base_path, &pb_path, &cli, &mut ctx, event_tx.clone()).await?;
finalize(&finalize_path, &cli, &mut ctx, event_tx.clone(), &registry).await?;
Ok(())
}.await;
if let Err(e) = pipe_result {
log::error!("Pipeline stage failed: {}", e);
return Err(CliError::General(e));
}
log::info!("Pipeline complete. Draining notifications...");
if let Err(e) = queue.drain().await {
log::error!("Notification queue error: {}", e);
}
let _ = notify_handle.await;
Ok(())
}
#[tokio::main]
async fn main() {
@@ -16,77 +94,46 @@ async fn main() {
if cli.dry_run {
log::warn!("Dry run enabled, no changes will be made.");
}
match &cli.command {
Some(Commands::Client { config: _ }) => {
unimplemented!("Client mode not yet implemented.")
}
Some(Commands::Bare { command }) => {
log::info!("Running in standalone mode.");
let (stage, privileged) = match command {
BareCommands::Prebake { .. } => ("init", true),
BareCommands::Bake { .. } => ("bake", false),
BareCommands::Finalize { .. } => ("finalize", false),
};
let pipeline = cli.pipeline.clone();
let build_id = cli.build_id.clone();
match &cli.command {
Some(Commands::Bare { command }) => {
let mut ctx = ExecutionContext {
standalone: true,
task_id: format!("{}-{}-{}", cli.pipeline, cli.build_id, stage),
pipeline_name: cli.pipeline.clone(),
task_id: format!("{}-{}-bare", pipeline, build_id),
pipeline_name: pipeline.clone(),
username: cli.username.clone(),
dry_run: cli.dry_run,
privileged,
privileged: false,
..Default::default()
};
let (tx, event_rx) = tokio::sync::mpsc::channel(256);
let event_tx: EventSender = Some(tx);
let handle = tokio::spawn(workshop_baker::prebake::event::event_receiver(
event_rx,
cli.pipeline.clone(),
cli.build_id.clone(),
let event_handle = tokio::spawn(workshop_baker::prebake::event::event_receiver(
event_rx, pipeline.clone(), build_id.clone(),
));
let result: anyhow::Result<()> = match command {
BareCommands::Prebake { config } => {
prebake(config, &cli, None, &mut ctx, event_tx).await
}
BareCommands::Bake {
script,
bake_base,
prebake: prebake_path,
} => bake(script, bake_base, prebake_path, &cli, &mut ctx, event_tx)
.await
.map_err(|e| e.into()),
BareCommands::Finalize { config } => {
finalize(Path::new(config), &cli, &mut ctx, event_tx)
.await
.map_err(|e| e.into())
}
};
let registry = ResourceRegistry::new();
let result = run_bare(&cli, command, &mut ctx, event_tx, &pipeline, &build_id, &registry).await;
drop(ctx);
let _ = handle.await;
dbg!(&result);
if let Err(ref e) = result {
let code = match command {
BareCommands::Prebake { .. } => e
.downcast_ref::<workshop_baker::prebake::error::PrebakeError>()
.map(|pe| pe.exit_code())
.unwrap_or(1),
BareCommands::Bake { .. } => e
.downcast_ref::<workshop_baker::bake::error::BakeError>()
.map(|be| be.exit_code())
.unwrap_or(1),
BareCommands::Finalize { .. } => e
.downcast_ref::<workshop_baker::finalize::FinalizeError>()
.map(|fe| fe.exit_code())
.unwrap_or(1),
};
log::error!("{} stage failed: {}", stage, e);
std::process::exit(code);
let _ = event_handle.await;
if let Err(e) = result {
log::error!("{}", e);
std::process::exit(e.exit_code());
}
}
Some(Commands::Daemon {}) => {
unimplemented!("Daemon mode not implemented");
}
None => {
log::info!("Executor will be running in daemon mode.");
let _result = daemon::daemon(&cli).await;
if let Err(e) = worker_main(cli).await {
log::error!("{}", e);
std::process::exit(e.exit_code());
}
}
}
}
-89
View File
@@ -1,89 +0,0 @@
mod cgroups;
mod jobobject;
use crate::socket::{establish_connection, get_socket_addr};
use config::Config;
use serde::{Deserialize, Serialize};
use std::time::Instant;
use tokio::io::AsyncWriteExt;
#[derive(Debug, Serialize, Deserialize)]
pub struct ResourceUsageInfo {
pub msgtype: String,
pub name: String,
pub os_type: OsType,
pub statistics: UsageStats,
pub timestamp: u64,
}
#[derive(Debug, Serialize, Deserialize)]
pub enum OsType {
Linux,
Windows,
MacOS,
}
#[derive(Debug, Serialize, Deserialize, Clone)]
pub struct UsageStats {
pub cpu_usage_percent: f64,
pub memory_usage_bytes: u64,
pub memory_limit_bytes: i64,
// pub memory_usage_percent: f64,
pub process_count: u64,
pub cpu_usage_us: u64,
/// Internal timestamp, using std::Instant
#[serde(skip, default = "default_internal_timestamp")]
_timestamp: Instant,
}
fn default_internal_timestamp() -> Instant {
Instant::now()
}
pub async fn monitor(
name: &String,
interval_ms: Option<u64>,
settings: Config,
) -> anyhow::Result<()> {
// let socket_addr = get_wsagent_socket_addr()?;
let socket_addr = get_socket_addr(settings.get_string("socket")?)?;
let interval = tokio::time::Duration::from_millis(interval_ms.unwrap_or(1000));
let mut socket = establish_connection(&socket_addr).await?;
let mut prev_usage = UsageStats {
cpu_usage_percent: 0.0,
memory_usage_bytes: 0,
memory_limit_bytes: 0,
process_count: 0,
cpu_usage_us: 0,
_timestamp: default_internal_timestamp(),
};
loop {
let usage: UsageStats = cgroups::get_usage(name, prev_usage)?; // TODO: 多系统支持?
prev_usage = usage.clone();
// dbg!(&usage);
let usage_info = ResourceUsageInfo {
msgtype: "ResourceUsage".to_string(),
name: name.clone(),
os_type: OsType::Linux, // TODO: 多系统支持?
statistics: usage,
timestamp: std::time::SystemTime::now()
.duration_since(std::time::UNIX_EPOCH)?
.as_millis() as u64,
};
let usage_json = serde_json::to_string(&usage_info)?;
send_usage_info(&mut socket, &usage_json).await?;
// TODO: 增加消息接收,提供(伪)双向通信功能
tokio::time::sleep(interval).await;
}
}
async fn send_usage_info<S>(stream: &mut S, usage_json: &str) -> anyhow::Result<()>
where
S: tokio::io::AsyncWrite + Unpin,
{
stream.write_all(usage_json.as_bytes()).await?;
stream.write_all(b"\n").await?;
Ok(())
}
+431 -346
View File
@@ -1,383 +1,468 @@
use crate::finalize::constant::FINALIZE_STANDALONE_PLUGIN_PATH;
use crate::finalize::FinalizeConfig;
use crate::finalize::NotificationConfig;
use crate::finalize::parse;
use crate::finalize::plugin;
use crate::finalize::plugin::fetch;
use crate::notify::types::Method;
use crate::notify::types::NOTIFICATION_TEMPORARY_DELAY_FACTOR;
use crate::notify::types::NOTIFICATION_TEMPORARY_DELAY_TIME;
use crate::notify::types::NOTIFICATION_TEMPORARY_MAX_TIME;
use crate::notify::types::NotificationEvent;
use crate::notify::types::DEFAULT_NOTIFY_PARAMS;
use crate::notify::types::NotificationEventReceiver;
use crate::notify::types::NotificationEventSender;
use crate::notify::types::NotificationPluginMap;
use crate::notify::types::NotificationRenderer;
use crate::notify::types::NotificationRule;
use crate::notify::types::NotificationRulesMap;
use crate::{EventSender, ExecutionContext};
use std::collections::HashMap;
use std::collections::HashSet;
use crate::notify::handler::NotificationHandler;
use crate::types::resource::ResourceRegistry;
use std::path::Path;
use std::path::PathBuf;
use crate::notify::types::ParsedTrigger;
use crate::notify::types::Matchable;
use globset::{Glob, GlobSetBuilder};
use crate::finalize::config::TriggerItemRaw;
use crate::finalize::NotificationTemplate;
use crate::notify::types::NOTIFICATION_DEFAULT_FALLIBLE;
use crate::notify::types::NOTIFICATION_DEFAULT_MAX_LIVES;
use crate::notify::types::NOTIFICATION_DEFAULT_TIMEOUT_MS;
use crate::notify::types::NOTIFICATION_DEFAULT_RETRY_BACKOFF;
use std::sync::Arc;
use workshop_engine::{EventSender, ExecutionContext};
mod error;
pub mod handler;
pub mod queue;
pub mod rule;
pub mod template;
pub mod types;
use error::NotifyError;
fn validate(finalize: &FinalizeConfig) -> anyhow::Result<()> {
let validate_result = finalize.validate();
if let Err(errors) = validate_result {
for error in &errors {
log::error!("Error: {}", error);
}
anyhow::bail!(NotifyError::FinalizeValidationError(errors))
};
Ok(())
}
fn extract_plugin(finalize: &FinalizeConfig) -> HashSet<String> {
finalize
.notification
.groups
.iter()
.flat_map(|(_, methods)| methods.keys())
.cloned()
.collect()
}
pub mod util;
use rule::parse_rules;
use template::{flatten_template_refs, resolve_external_template_refs};
use util::{extract_plugin, validate};
pub async fn notify(
finalize_path: &Path,
ctx: &mut ExecutionContext,
event_tx: EventSender,
notifyevent_rx: &mut NotificationEventReceiver,
notifyevent_tx: NotificationEventSender,
_event_tx: EventSender,
mut rx: NotificationEventReceiver,
retry_tx: NotificationEventSender,
_debug: bool,
registry: &ResourceRegistry,
) -> anyhow::Result<()> {
let mut finalize = parse(finalize_path)?;
// Always inject in-site-notify as the lowest priority fallback
{
let max_p = finalize.notification.groups.keys().max().copied().unwrap_or(0);
let mut m = std::collections::HashMap::new();
m.insert(
"in-site-notify".to_string(),
crate::finalize::NotificationMethod::default(),
);
finalize.notification.groups.insert(max_p + 1, m);
}
validate(&finalize)?;
let plugin_list = extract_plugin(&finalize);
if plugin_list.is_empty() {
log::warn!("No notification method provided, skipping notification");
// TODO: send event to event_tx
return Ok(());
}
if ctx.standalone {
fetch::fetch_plugins(
&PathBuf::from(FINALIZE_STANDALONE_PLUGIN_PATH),
&mut finalize.plugin,
let notify_plugins = plugin::register(
finalize.plugin,
Some(&plugin_list),
)
.await?;
}
let notify_plugins = plugin::register(finalize.plugin, Some(&plugin_list))?;
&mut finalize.notification,
registry,
)?;
resolve_external_template_refs(&mut finalize.notification.templates, registry);
flatten_template_refs(&mut finalize.notification.templates)?;
let notification_rules = parse_rules(&finalize.notification);
// loop {
// tokio::select! {
// Some(events) = notifyevent_rx.recv() => {
// match notify_handler(&notify_plugins, &finalize.notification, &events){
// Ok(_) => {}, // Notification succeeded / failed with fallible
// Err(e) => {
// log::debug!("Failed to send notification: {:?}", e);
// match e {
// NotifyError::PluginNotAvailable(_) => {
// // Should keep lives, add static delay and give less priority
// for mut event in events {
// event.not_before = Utc::now() + NOTIFY_NOTREADY_DELAY;
// event.effective_priority = event.effective_priority.saturating_add(1);
// notifyevent_tx.send(event).await?;
// }
// },
// NotifyError::TemporaryError(_) => {
// // Should cost 1 life, add dynamic (exponential) delay
// for mut event in events {
// event.lives -= 1;
// event.not_before = Utc::now() + notify_delay(&event);
// notifyevent_tx.send(event).await?;
// }
// },
// NotifyError::PermanentError(_) => {
// // Unexpected!
// // Should be dropped
// log::error!("None of the configured notification plugins works.");
// log::error!("Message will be dropped.");
// // TODO: Send audit/metric information
// }
// error => {
// // Unexpected!
// log::error!("Internal error: notify_handler returned unexpected error: {:?}", error);
// }
// };
// }
// };
// }
// }
// }
todo!();
}
let handler = Arc::new(NotificationHandler::new(
Arc::new(notification_rules),
Arc::new(notify_plugins),
Arc::new(finalize.notification.templates),
Arc::new(ctx.clone()),
retry_tx,
DEFAULT_NOTIFY_PARAMS.batch_period.value,
DEFAULT_NOTIFY_PARAMS.batch_watermark.value,
));
fn notification_handler(
plugins: &NotificationPluginMap,
config: &NotificationRulesMap,
event: &NotificationEvent,
) -> Result<(), NotifyError> {
for (priority, ruleset) in config.iter() {
for (method, rules) in ruleset.iter() {
if !event.target.contains(method) {
log::trace!("target does not contain method name: {}", method);
continue;
}
for (rule_id, rule) in rules.iter().enumerate() {
if !rule.enabled {
log::trace!("rule is not enabled: {}", rule_id);
continue;
}
if rule.accept(&event.outcome, &event.stage, &event.substage) {
let plugin = plugins
.get(method)
.ok_or(NotifyError::UnknownPlugin(method.clone()))?;
let renderer = plugin.metadata.renderer.clone().unwrap_or_default();
match renderer {
NotificationRenderer::Server => {
// ...
todo!();
}
NotificationRenderer::Plugin => {
unimplemented!();
return Err(NotifyError::UnsupportedRenderer(method.clone(), renderer));
}
NotificationRenderer::Interactive => {
unimplemented!();
return Err(NotifyError::UnsupportedRenderer(method.clone(), renderer));
}
}
}
}
}
}
todo!();
}
fn parse_trigger_item_raw(item: &TriggerItemRaw) -> Vec<ParsedTrigger> {
match item {
TriggerItemRaw::Simple(s) => {
match s.as_str() {
"on_success" => vec![ParsedTrigger::OnSuccess],
"on_failure" => vec![ParsedTrigger::OnFailure],
s if s.starts_with("on_finish_of:") => {
let pattern = s.trim_start_matches("on_finish_of:").trim();
let mut builder = GlobSetBuilder::new();
builder.add(Glob::new(pattern).expect("Invalid glob pattern"));
let glob_set = builder.build().expect("Failed to compile glob set");
vec![ParsedTrigger::OnFinishOf(Matchable::Glob(glob_set))]
}
_ => {
log::warn!("Unknown trigger string: {}", s);
vec![]
}
}
}
TriggerItemRaw::Map(map) => {
let mut triggers = Vec::new();
for (key, patterns) in map {
match key.as_str() {
"on_finish_of" => {
for pattern in patterns {
let mut builder = GlobSetBuilder::new();
builder.add(Glob::new(pattern).expect("Invalid glob pattern"));
let glob_set = builder.build().expect("Failed to compile glob set");
triggers.push(ParsedTrigger::OnFinishOf(Matchable::Glob(glob_set)));
}
}
"on_success" => triggers.push(ParsedTrigger::OnSuccess),
"on_failure" => triggers.push(ParsedTrigger::OnFailure),
_ => {
log::warn!("Unknown trigger key: {}", key);
}
}
}
triggers
}
}
}
fn merge_templates(
method_template: &NotificationTemplate,
policy_template: &Option<serde_yaml::Value>,
) -> NotificationTemplate {
let mut merged = method_template.clone();
if let Some(policy_tmpl) = policy_template {
if let Ok(policy_template) =
serde_yaml::from_value::<NotificationTemplate>(policy_tmpl.clone())
{
if policy_template.schema.is_some() {
merged.schema = policy_template.schema;
}
if policy_template.on_success.is_some() {
merged.on_success = policy_template.on_success;
}
if policy_template.on_failure.is_some() {
merged.on_failure = policy_template.on_failure;
}
if policy_template.on_finish_of.is_some() {
merged.on_finish_of = policy_template.on_finish_of;
}
}
}
merged
}
fn extract_overrides(overrides: &serde_yaml::Value) -> (bool, u32, std::time::Duration, f64) {
let fallible = overrides
.get("fallible")
.and_then(|v| v.as_bool())
.unwrap_or(NOTIFICATION_DEFAULT_FALLIBLE);
let max_lives = overrides
.get("max_lives")
.and_then(|v| v.as_u64())
.unwrap_or(NOTIFICATION_DEFAULT_MAX_LIVES as u64) as u32;
let timeout_ms = overrides
.get("timeout_ms")
.and_then(|v| v.as_u64())
.unwrap_or(NOTIFICATION_DEFAULT_TIMEOUT_MS);
let retry_backoff = overrides
.get("retry_backoff")
.and_then(|v| v.as_f64())
.unwrap_or(NOTIFICATION_DEFAULT_RETRY_BACKOFF);
(
fallible,
max_lives,
std::time::Duration::from_millis(timeout_ms),
retry_backoff,
)
}
fn parse_rules(config: &NotificationConfig) -> NotificationRulesMap {
let mut rules = NotificationRulesMap::default();
for (priority, group) in &config.groups {
let mut ruleset: HashMap<Method, Vec<NotificationRule>> = HashMap::new();
for (method_name, method) in group {
let mut method_rules: Vec<NotificationRule> = Vec::new();
if method.policy.is_empty() {
let trigger: Vec<ParsedTrigger> = method
.trigger
.iter()
.flat_map(parse_trigger_item_raw)
.collect();
let template = method.template.clone();
let (fallible, max_lives, timeout, retry_backoff) =
extract_overrides(&method.config);
method_rules.push(NotificationRule {
enabled: true,
trigger,
template,
fallible,
max_lives,
timeout,
config: method.config.clone(),
retry_backoff,
});
} else {
for policy in &method.policy {
let trigger: Vec<ParsedTrigger> = if policy.trigger.is_empty() {
method
.trigger
.iter()
.flat_map(parse_trigger_item_raw)
.collect()
} else {
policy
.trigger
.iter()
.flat_map(parse_trigger_item_raw)
.collect()
};
let template = merge_templates(&method.template, &Some(policy.template.clone()));
let method_overrides = extract_overrides(&method.config);
let policy_overrides = extract_overrides(&policy.overrides);
let fallible = if policy.overrides.get("fallible").is_some() {
policy_overrides.0
} else {
method_overrides.0
};
let max_lives = if policy.overrides.get("max_lives").is_some() {
policy_overrides.1
} else {
method_overrides.1
};
let timeout = if policy.overrides.get("timeout_ms").is_some() {
policy_overrides.2
} else {
method_overrides.2
};
let retry_backoff = if policy.overrides.get("retry_backoff").is_some() {
policy_overrides.3
} else {
method_overrides.3
};
method_rules.push(NotificationRule {
enabled: true,
trigger,
template,
fallible,
max_lives,
timeout,
config: method.config.clone(),
retry_backoff,
loop {
let Some(event) = rx.recv().await else { break };
let h = handler.clone();
tokio::spawn(async move {
h.process_event(event).await;
});
}
Ok(())
}
ruleset.insert(method_name.clone(), method_rules);
// TODO: PIPELINE_NOTIFICATION_EXHAUSTED — set when any method returns Exhausted
// TODO: PIPELINE_NOTIFICATION_FAILED — set when any method returns PermanentError
// Markers sent via event_tx as ExecutionEvent variants for future WebUI/CLI display.
#[cfg(test)]
mod tests {
use super::*;
use crate::finalize::NotificationTemplateDef;
use crate::notify::types::NotificationRule;
use crate::notify::types::NotificationTarget;
use std::collections::HashMap;
use crate::notify::types::NotificationEvent;
use crate::notify::types::NotificationRulesMap;
use crate::notify::error::NotifyError;
use crate::notify::template::resolve_template;
use crate::notify::util::{find_rule, notify_delay, wait_time};
// ── helper ──
fn make_tmpl(on_success: &str, on_failure: &str) -> NotificationTemplateDef {
NotificationTemplateDef {
use_: None,
on_success: if on_success.is_empty() {
None
} else {
Some(on_success.into())
},
on_failure: if on_failure.is_empty() {
None
} else {
Some(on_failure.into())
},
on_finish_of: None,
}
rules.insert(*priority, ruleset);
}
rules
}
// calculate min(1, k/x)*t as notification grouping period
// to avoid excessive notification frequency
fn wait_time(group_period: u32, group_watermark: u32, count: u32) -> f32 {
if count == 0 {
log::warn!("Unexpected notification count: {}", count);
return group_period as f32;
fn make_tmpl_with_use(use_ref: &str) -> NotificationTemplateDef {
NotificationTemplateDef {
use_: Some(use_ref.into()),
on_success: None,
on_failure: None,
on_finish_of: None,
}
f32::min(1.0, group_watermark as f32 / count as f32) * group_period as f32
}
// calculate p^(m-l)*t as notification delay
fn notify_delay(event: &NotificationEvent) -> std::time::Duration {
let exponent = (event.max_lives - event.lives) as i32;
if exponent < 0 {
log::warn!(
"Unexpected lives in notification: lives({}) > max_lives({})",
event.lives,
event.max_lives
// ═══════════════════════════════════════════
// resolve_template_content
// ═══════════════════════════════════════════
#[test]
fn resolve_none_cd_default_found() {
let mut ws = HashMap::new();
ws.insert("mail".into(), make_tmpl("OK", "FAIL"));
let templates = HashMap::new();
let result = resolve_template(&None, "mail", &templates, false, Some(&ws));
assert!(result.is_ok());
assert_eq!(result.unwrap().on_success.unwrap(), "OK");
}
#[test]
fn resolve_none_cd_default_missing_fallback() {
let ws: HashMap<String, NotificationTemplateDef> = HashMap::new();
let mut templates = HashMap::new();
templates.insert("mail.fallback".into(), make_tmpl("FALLBACK", ""));
let result = resolve_template(&None, "mail", &templates, false, Some(&ws));
assert!(result.is_ok());
assert_eq!(result.unwrap().on_success.unwrap(), "FALLBACK");
}
#[test]
fn resolve_none_cd_no_default_no_fallback_error() {
let ws: HashMap<String, NotificationTemplateDef> = HashMap::new();
let templates = HashMap::new();
let result = resolve_template(&None, "mail", &templates, false, Some(&ws));
assert!(result.is_err());
assert!(format!("{}", result.unwrap_err()).contains("failed"));
}
#[test]
fn resolve_none_standalone_fallback() {
let mut templates = HashMap::new();
templates.insert("mail.fallback".into(), make_tmpl("STANDALONE", ""));
let result = resolve_template(&None, "mail", &templates, true, None);
assert!(result.is_ok());
assert_eq!(result.unwrap().on_success.unwrap(), "STANDALONE");
}
#[test]
fn resolve_none_standalone_no_fallback_error() {
let result = resolve_template(&None, "mail", &HashMap::new(), true, None);
assert!(result.is_err());
}
#[test]
fn resolve_custom() {
let result = resolve_template(&Some("custom".into()), "mail", &HashMap::new(), false, None);
assert!(result.is_ok());
// "custom" returns an empty default — caller reads rule.template.on_* directly
}
#[test]
fn resolve_default_standalone_fallback() {
let mut templates = HashMap::new();
templates.insert("mail.fallback".into(), make_tmpl("FB", ""));
let result = resolve_template(&Some("default".into()), "mail", &templates, true, None);
assert!(result.is_ok());
assert_eq!(result.unwrap().on_success.unwrap(), "FB");
}
#[test]
fn resolve_default_standalone_no_fallback_error() {
let result = resolve_template(&Some("default".into()), "mail", &HashMap::new(), true, None);
assert!(result.is_err());
}
#[test]
fn resolve_default_cd_wsbase() {
let mut ws = HashMap::new();
ws.insert("mail".into(), make_tmpl("WS_OK", ""));
let result = resolve_template(
&Some("default".into()),
"mail",
&HashMap::new(),
false,
Some(&ws),
);
return std::time::Duration::from_secs_f64(NOTIFICATION_TEMPORARY_MAX_TIME);
assert!(result.is_ok());
assert_eq!(result.unwrap().on_success.unwrap(), "WS_OK");
}
let multiplier = NOTIFICATION_TEMPORARY_DELAY_FACTOR.powi(exponent);
let delay = multiplier * NOTIFICATION_TEMPORARY_DELAY_TIME;
if !delay.is_finite() || delay < 0.0 {
log::warn!(
"Unexpected calculated delay: {}, treating as infinite",
delay
#[test]
fn resolve_default_cd_no_wsbase_error() {
let result = resolve_template(
&Some("default".into()),
"mail",
&HashMap::new(),
false,
None,
);
return std::time::Duration::from_secs_f64(NOTIFICATION_TEMPORARY_MAX_TIME);
assert!(result.is_err());
}
#[test]
fn resolve_named_found() {
let mut templates = HashMap::new();
templates.insert("sms-tmpl1".into(), make_tmpl("NAMED", ""));
let result = resolve_template(&Some("sms-tmpl1".into()), "mail", &templates, false, None);
assert!(result.is_ok());
assert_eq!(result.unwrap().on_success.unwrap(), "NAMED");
}
#[test]
fn resolve_named_not_found_method_match_fallback() {
let mut templates = HashMap::new();
templates.insert("mail.fallback".into(), make_tmpl("FB", ""));
let result = resolve_template(&Some("mail".into()), "mail", &templates, false, None);
assert!(result.is_ok());
assert_eq!(result.unwrap().on_success.unwrap(), "FB");
}
#[test]
fn resolve_named_not_found_no_match_error() {
let result = resolve_template(&Some("ghost".into()), "mail", &HashMap::new(), false, None);
assert!(result.is_err());
}
// ═══════════════════════════════════════════
// flatten_template_refs
// ═══════════════════════════════════════════
#[test]
fn flatten_no_refs_unchanged() {
let mut templates = HashMap::new();
templates.insert("a".into(), make_tmpl("A_OK", "A_FAIL"));
let result = flatten_template_refs(&mut templates);
assert!(result.is_ok());
assert_eq!(templates["a"].on_success.as_deref(), Some("A_OK"));
assert!(templates["a"].use_.is_none());
}
#[test]
fn flatten_single_ref_merged() {
let mut templates = HashMap::new();
templates.insert("base".into(), make_tmpl("BASE_OK", "BASE_FAIL"));
templates.insert("child".into(), make_tmpl_with_use("base"));
let result = flatten_template_refs(&mut templates);
assert!(result.is_ok());
assert_eq!(templates["child"].on_success.as_deref(), Some("BASE_OK"));
assert_eq!(templates["child"].on_failure.as_deref(), Some("BASE_FAIL"));
assert!(templates["child"].use_.is_none());
}
#[test]
fn flatten_child_overrides_parent() {
let mut templates = HashMap::new();
templates.insert("base".into(), make_tmpl("BASE_OK", "BASE_FAIL"));
templates.insert(
"child".into(),
NotificationTemplateDef {
use_: Some("base".into()),
on_success: Some("MY_OK".into()),
on_failure: None,
on_finish_of: None,
},
);
let result = flatten_template_refs(&mut templates);
assert!(result.is_ok());
assert_eq!(templates["child"].on_success.as_deref(), Some("MY_OK"));
assert_eq!(templates["child"].on_failure.as_deref(), Some("BASE_FAIL"));
}
#[test]
fn flatten_chain_a_b_c() {
let mut templates = HashMap::new();
templates.insert("c".into(), make_tmpl("C_OK", "C_FAIL"));
templates.insert("b".into(), make_tmpl_with_use("c"));
templates.insert("a".into(), make_tmpl_with_use("b"));
let result = flatten_template_refs(&mut templates);
assert!(result.is_ok());
assert_eq!(templates["a"].on_success.as_deref(), Some("C_OK"));
assert_eq!(templates["b"].on_success.as_deref(), Some("C_OK"));
}
#[test]
fn flatten_cycle_detected() {
let mut templates = HashMap::new();
templates.insert("x".into(), make_tmpl_with_use("y"));
templates.insert("y".into(), make_tmpl_with_use("x"));
let result = flatten_template_refs(&mut templates);
assert!(result.is_err());
match result.unwrap_err() {
NotifyError::TemplateLoop(_) => {}
other => panic!("expected TemplateLoop, got {:?}", other),
}
}
#[test]
fn flatten_self_ref_detected() {
let mut templates = HashMap::new();
templates.insert("self".into(), make_tmpl_with_use("self"));
let result = flatten_template_refs(&mut templates);
assert!(result.is_err());
match result.unwrap_err() {
NotifyError::TemplateLoop(_) => {}
other => panic!("expected TemplateLoop, got {:?}", other),
}
}
#[test]
fn flatten_nonexistent_ref_error() {
let mut templates = HashMap::new();
templates.insert("orphan".into(), make_tmpl_with_use("ghost"));
let result = flatten_template_refs(&mut templates);
assert!(result.is_err());
}
#[test]
fn flatten_external_url_preserved() {
let mut templates = HashMap::new();
templates.insert(
"ext".into(),
NotificationTemplateDef {
use_: Some("git://example.com/repo@v1".into()),
on_success: None,
on_failure: None,
on_finish_of: None,
},
);
let result = flatten_template_refs(&mut templates);
assert!(result.is_ok());
// External URLs are NOT flattened — use_ preserved
assert_eq!(
templates["ext"].use_.as_deref(),
Some("git://example.com/repo@v1")
);
}
// ═══════════════════════════════════════════
// wait_time
// ═══════════════════════════════════════════
#[test]
fn wait_time_zero_count() {
let w = wait_time(30, 5, 0);
assert_eq!(w, 30.0);
}
#[test]
fn wait_time_at_watermark() {
let w = wait_time(30, 5, 5);
assert_eq!(w, 30.0);
}
#[test]
fn wait_time_below_watermark() {
let w = wait_time(30, 5, 2);
assert!((w - 30.0).abs() < 0.01);
}
#[test]
fn wait_time_above_watermark() {
let w = wait_time(30, 5, 10);
assert!((w - 15.0).abs() < 0.01);
}
#[test]
fn wait_time_far_above_watermark() {
let w = wait_time(30, 5, 100);
assert!((w - 1.5).abs() < 0.01);
}
// ═══════════════════════════════════════════
// notify_delay
// ═══════════════════════════════════════════
fn make_event(count: u32, retry_backoff: f64) -> NotificationEvent {
NotificationEvent {
id: uuid::Uuid::new_v4(),
stage: crate::types::buildstatus::StagePhase::Bake,
substage: "build".into(),
outcome: crate::types::buildstatus::StageOutcome::PipelineSuccess,
not_before: chrono::Utc::now(),
priority: 0,
effective_priority: 0,
attempted: 0,
retry_backoff,
target: NotificationTarget {
priority: 0,
rules: None,
count,
},
}
}
#[test]
fn notify_delay_default_factor() {
let event = make_event(1, 2.0);
let delay = notify_delay(&event);
assert!(delay.as_secs_f64() > 0.0);
}
#[test]
fn notify_delay_larger_factor() {
let event1 = make_event(1, 2.0);
let event2 = make_event(1, 4.0);
let d1 = notify_delay(&event1).as_secs_f64();
let d2 = notify_delay(&event2).as_secs_f64();
assert!(d2 > d1, "larger retry_backoff should produce longer delay");
}
// ═══════════════════════════════════════════
// find_rule
// ═══════════════════════════════════════════
#[test]
fn find_rule_found() {
use crate::finalize::NotificationTemplate;
let rule = NotificationRule {
enabled: true,
trigger: vec![],
template: NotificationTemplate {
schema: None,
on_success: None,
on_failure: None,
on_finish_of: None,
},
fallible: false,
max_lives: 3,
timeout: std::time::Duration::from_secs(30),
config: serde_yaml::Value::Null,
retry_backoff: 2.0,
};
let mut rules_map: NotificationRulesMap = HashMap::new();
rules_map.insert(0, {
let mut m: HashMap<String, Vec<NotificationRule>> = HashMap::new();
m.insert("mail".into(), vec![rule]);
m
});
let result = find_rule(&rules_map, "mail", 0);
assert!(result.is_ok());
}
#[test]
fn find_rule_not_found() {
let rules_map: NotificationRulesMap = HashMap::new();
let result = find_rule(&rules_map, "mail", 0);
assert!(result.is_err());
}
std::time::Duration::from_secs_f64(delay)
}
+13 -1
View File
@@ -1,4 +1,4 @@
use crate::notify::NotificationRenderer;
use crate::notify::types::NotificationRenderer;
use thiserror::Error;
#[derive(Error, Debug)]
@@ -23,4 +23,16 @@ pub enum NotifyError {
#[error("Unsupported renderer for {0}: {1}")]
UnsupportedRenderer(String, NotificationRenderer),
#[error("Schema error: {0}")]
SchemaError(String),
#[error("Template render error: {0}")]
TemplateRenderError(String),
#[error("Circular template reference: {0}")]
TemplateLoop(String),
#[error("Template reference depth exceeded (max {max}): {name}")]
TemplateDepthExceeded { name: String, max: u32 },
}
+345
View File
@@ -0,0 +1,345 @@
use std::collections::HashMap;
use std::sync::atomic::{AtomicBool, Ordering};
use std::sync::{Arc, Mutex};
use std::time::Duration;
use chrono::Utc;
use tokio::sync::Notify;
use crate::finalize::NotificationTemplateDef;
use crate::notify::template::{
build_batch_template_context, render_template, resolve_template,
select_template_str,
};
use crate::notify::types::{
MethodStatus, NotificationEvent,
NotificationEventSender, NotificationPluginMap, NotificationRenderer,
NotificationRulesMap,
};
use crate::notify::util::{
find_rule, get_fallible, is_temporary_error, notify_delay, wait_time,
};
/// Shared state for a single dispatch bucket.
struct BucketEntry {
events: Arc<Mutex<Vec<NotificationEvent>>>,
result: Arc<Mutex<Option<(MethodStatus, Vec<NotificationEvent>)>>>,
done: Arc<AtomicBool>,
notify: Arc<Notify>,
method: String,
rule_id: usize,
_handle: tokio::task::JoinHandle<()>,
}
/// Concurrent notification handler with bucket-based dispatch.
///
/// Events are routed to matching buckets. Buckets cool down and dispatch
/// plugins concurrently. The caller awaits bucket completion, then handles
/// retry/escalation decisions.
pub struct NotificationHandler {
buckets: Mutex<HashMap<(String, usize), BucketEntry>>,
pub config: Arc<NotificationRulesMap>,
plugins: Arc<NotificationPluginMap>,
templates: Arc<HashMap<String, NotificationTemplateDef>>,
ctx: Arc<workshop_engine::ExecutionContext>,
retry_tx: NotificationEventSender,
period: u32,
watermark: u32,
}
impl NotificationHandler {
#[allow(clippy::too_many_arguments)]
pub fn new(
config: Arc<NotificationRulesMap>,
plugins: Arc<NotificationPluginMap>,
templates: Arc<HashMap<String, NotificationTemplateDef>>,
ctx: Arc<workshop_engine::ExecutionContext>,
retry_tx: NotificationEventSender,
period: u32,
watermark: u32,
) -> Self {
NotificationHandler {
buckets: Mutex::new(HashMap::new()),
config,
plugins,
templates,
ctx,
retry_tx,
period,
watermark,
}
}
/// Process a single event: route to buckets, await completion,
/// then handle retry/escalation for each bucket result.
pub async fn process_event(self: Arc<Self>, event: NotificationEvent) {
let entries = self.route_event(&event);
if entries.is_empty() {
return;
}
// Await all buckets — yields CPU
for (_, notify) in &entries {
notify.notified().await;
}
// Collect results and remove entries
let mut results: Vec<(String, usize, Option<(MethodStatus, Vec<NotificationEvent>)>)> =
Vec::new();
{
let mut buckets = self.buckets.lock().unwrap();
for (key, _) in &entries {
if let Some(entry) = buckets.remove(key) {
let r = entry.result.lock().unwrap().take();
results.push((entry.method, entry.rule_id, r));
}
}
}
// Handle retry/escalation per method
for (method, _rule_id, result) in results {
let Some((status, mut events)) = result else { continue };
match &status {
MethodStatus::Success => {
log::debug!("[NOTIFY] {}: success ({} events)", method, events.len());
}
MethodStatus::TemporaryError(_) => {
log::warn!("[NOTIFY] {}: temporary error, retrying", method);
for event in events.iter_mut() {
event.target.rules = Some(vec![method.clone()]);
event.target.count += 1;
event.not_before = Utc::now() + notify_delay(event);
let _ = self.retry_tx.send(event.clone()).await;
}
}
MethodStatus::PermanentError(_) | MethodStatus::Exhausted(_) => {
let fallible = get_fallible(&self.config, &method);
if fallible {
log::error!("[NOTIFY] {}: discard (fallible)", method);
} else {
log::warn!("[NOTIFY] {}: escalating priority", method);
for event in events.iter_mut() {
event.target.priority += 1;
event.target.rules = None;
event.attempted = event.target.count;
event.target.count = 0;
event.effective_priority = event.priority;
let _ = self.retry_tx.send(event.clone()).await;
}
}
}
}
}
}
/// Route a single event to all matching buckets.
///
/// Returns Notify handles for awaiting bucket completion.
fn route_event(
&self,
event: &NotificationEvent,
) -> Vec<((String, usize), Arc<Notify>)> {
let mut matches: Vec<(String, usize)> = Vec::new();
for ruleset in self.config.values() {
for (method, rules) in ruleset.iter() {
if let Some(ref method_rules) = event.target.rules {
if !method_rules.contains(method) {
continue;
}
}
for (rule_id, rule) in rules.iter().enumerate() {
if !rule.enabled {
continue;
}
if rule.accept(&event.outcome, &event.stage, &event.substage) {
matches.push((method.clone(), rule_id));
}
}
}
}
let mut buckets = self.buckets.lock().unwrap();
let mut result: Vec<((String, usize), Arc<Notify>)> = Vec::new();
for key in matches {
if let Some(entry) = buckets.get(&key) {
if entry.done.load(Ordering::Acquire) {
buckets.remove(&key);
}
}
if let Some(entry) = buckets.get(&key) {
entry.events.lock().unwrap().push(event.clone());
result.push((key, entry.notify.clone()));
} else {
let (entry, notify) = Self::spawn_bucket(
&key.0,
key.1,
&self.config,
&self.plugins,
&self.templates,
&self.ctx,
self.period,
self.watermark,
);
entry.events.lock().unwrap().push(event.clone());
result.push((key.clone(), notify));
buckets.insert(key, entry);
}
}
result
}
/// Spawn a bucket task: cooldown → take → dispatch → store → notify.
fn spawn_bucket(
method: &str,
rule_id: usize,
config: &Arc<NotificationRulesMap>,
plugins: &Arc<NotificationPluginMap>,
templates: &Arc<HashMap<String, NotificationTemplateDef>>,
ctx: &Arc<workshop_engine::ExecutionContext>,
period: u32,
watermark: u32,
) -> (BucketEntry, Arc<Notify>) {
let events: Arc<Mutex<Vec<NotificationEvent>>> = Arc::new(Mutex::new(Vec::new()));
let result: Arc<Mutex<Option<(MethodStatus, Vec<NotificationEvent>)>>> =
Arc::new(Mutex::new(None));
let notify = Arc::new(Notify::new());
let done = Arc::new(AtomicBool::new(false));
let ec = events.clone();
let rc = result.clone();
let nc = notify.clone();
let dc = done.clone();
let m = method.to_string();
let c = config.clone();
let p = plugins.clone();
let t = templates.clone();
let x = ctx.clone();
let handle = tokio::spawn(async move {
// Phase 1: Cooldown
let count = ec.lock().unwrap().len() as u32;
let wait_secs = wait_time(period, watermark, count);
tokio::time::sleep(Duration::from_secs_f32(wait_secs)).await;
// Phase 2: Take events
let batch = std::mem::take(&mut *ec.lock().unwrap());
if batch.is_empty() {
dc.store(true, Ordering::Release);
nc.notify_waiters();
return;
}
// Phase 3: Dispatch
let rule = match find_rule(&c, &m, rule_id) {
Ok(r) => r,
Err(_) => {
*rc.lock().unwrap() = Some((MethodStatus::PermanentError("rule not found".into()), batch));
dc.store(true, Ordering::Release);
nc.notify_waiters();
return;
}
};
let plugin = match p.get(&m) {
Some(p) => p,
None => {
*rc.lock().unwrap() = Some((MethodStatus::PermanentError("plugin not found".into()), batch));
dc.store(true, Ordering::Release);
nc.notify_waiters();
return;
}
};
let renderer = plugin.metadata.renderer.clone().unwrap_or_default();
let argument = build_plugin_argument(&renderer, rule, &m, &t, &x, &batch);
let status = match argument {
Some(arg) => {
match tokio::time::timeout(rule.timeout, plugin.call(arg, &x, &None)).await {
Ok(Ok(())) => MethodStatus::Success,
Ok(Err(e)) if is_temporary_error(&e) => {
MethodStatus::TemporaryError(format!("{}", e))
}
Ok(Err(e)) => MethodStatus::PermanentError(format!("{}", e)),
Err(_) => MethodStatus::TemporaryError("timeout".into()),
}
}
None => MethodStatus::PermanentError("argument build failed".into()),
};
*rc.lock().unwrap() = Some((status, batch));
dc.store(true, Ordering::Release);
nc.notify_waiters();
});
let entry = BucketEntry {
events,
result,
done,
notify: notify.clone(),
method: method.to_string(),
rule_id,
_handle: handle,
};
(entry, notify)
}
}
/// Build the YAML argument for a plugin call based on renderer mode.
fn build_plugin_argument(
renderer: &NotificationRenderer,
rule: &crate::notify::types::NotificationRule,
method: &str,
templates: &HashMap<String, NotificationTemplateDef>,
ctx: &workshop_engine::ExecutionContext,
events: &[NotificationEvent],
) -> Option<serde_yaml::Value> {
let first = events.first()?;
match renderer {
NotificationRenderer::Server => {
let template_def = resolve_template(
&rule.template.schema,
method,
templates,
ctx.standalone,
None,
)
.ok()?;
let template_str = select_template_str(&template_def, &first.outcome)
.or_else(|| rule.template.on_success.as_deref())
.or_else(|| rule.template.on_failure.as_deref())
.or_else(|| {
rule.template
.on_finish_of
.as_ref()
.map(|o| o.content.as_str())
})?;
let data = build_batch_template_context(ctx, events);
let rendered = render_template(template_str, &data).ok()?;
let mut argument = rule.config.clone();
if let serde_yaml::Value::Mapping(ref mut map) = argument {
map.insert(
serde_yaml::Value::String("_rendered_content".into()),
serde_yaml::Value::String(rendered),
);
}
log::info!("SSR: rendered content for method '{}'", method);
Some(argument)
}
NotificationRenderer::Plugin => {
unimplemented!("Plugin renderer is not yet implemented")
}
NotificationRenderer::Interactive => {
unimplemented!("Interactive renderer is not yet implemented")
}
}
}
+86
View File
@@ -0,0 +1,86 @@
use crate::notify::types::{NotificationEvent, NotificationEventReceiver, NotificationEventSender};
use std::collections::BinaryHeap;
/// Priority queue for notification events.
///
/// Integrates with `notify()`:
/// - `tx` sends events to notify() for processing
/// - `rx` receives retry events back from notify()
pub struct NotificationQueue {
pub pq: BinaryHeap<NotificationEvent>,
pub tx: NotificationEventSender,
pub rx: NotificationEventReceiver,
}
impl NotificationQueue {
pub fn new(tx: NotificationEventSender, rx: NotificationEventReceiver) -> Self {
Self {
pq: BinaryHeap::new(),
tx,
rx,
}
}
/// Push an event into the priority queue.
pub fn enqueue(&mut self, event: NotificationEvent) {
self.pq.push(event);
}
/// Non-blocking single round: recycle retry events into PQ, then
/// pop and send all ready events (not_before ≤ now).
pub async fn flush_now(&mut self) -> anyhow::Result<()> {
// Recycle retry events
while let Ok(event) = self.rx.try_recv() {
self.pq.push(event);
}
// Send ready events
let now = chrono::Utc::now();
while let Some(peek) = self.pq.peek() {
if peek.not_before > now {
break;
}
let event = self.pq.pop().unwrap();
self.tx.send(event).await?;
}
Ok(())
}
/// Blocking drain until all events are processed.
/// For standalone mode: called after prebake/bake/finalize to wait for completion.
/// For c/d mode: the main execution loop.
pub async fn drain(&mut self) -> anyhow::Result<()> {
loop {
// Drain retry channel
while let Ok(event) = self.rx.try_recv() {
self.pq.push(event);
}
// Determine sleep deadline
let until = self
.pq
.peek()
.map(|e| e.not_before)
.unwrap_or_else(|| chrono::Utc::now() + chrono::Duration::seconds(3600));
let std_until = (until - chrono::Utc::now())
.to_std()
.unwrap_or(std::time::Duration::from_secs(1));
tokio::select! {
event = self.rx.recv() => {
match event {
Some(event) => self.pq.push(event),
None => break Ok(()),
}
}
_ = tokio::time::sleep(std_until) => {
let now = chrono::Utc::now();
while let Some(peek) = self.pq.peek() {
if peek.not_before > now { break; }
let event = self.pq.pop().unwrap();
self.tx.send(event).await?;
}
}
}
}
}
}
+148
View File
@@ -0,0 +1,148 @@
use crate::finalize::NotificationConfig;
use crate::finalize::NotificationTemplate;
use crate::finalize::config::{NotifyPolicy, TriggerItemRaw};
use crate::notify::types::{DEFAULT_NOTIFY_PARAMS, Matchable, Method, NotificationRule, NotificationRulesMap, ParsedTrigger};
use globset::{Glob, GlobSetBuilder};
use std::collections::HashMap;
/// Parse a single `on_finish_of` pattern string into a ParsedTrigger, or None.
/// Supports `re:` prefix for regex patterns, otherwise treated as glob.
fn parse_on_finish_of(pattern: &str) -> Option<ParsedTrigger> {
let pattern = pattern.trim();
if let Some(re_str) = pattern.strip_prefix("re:") {
regex::Regex::new(re_str).ok().map(|r| ParsedTrigger::OnFinishOf(Matchable::Regex(r)))
} else {
Glob::new(pattern).ok().and_then(|g| {
let mut builder = GlobSetBuilder::new();
builder.add(g);
builder.build().ok()
}).map(|gs| ParsedTrigger::OnFinishOf(Matchable::Glob(gs)))
}
}
fn parse_trigger_item_raw(item: &TriggerItemRaw) -> Vec<ParsedTrigger> {
// Normalize Simple → single-entry Vec, then unify both branches
let entries: Vec<(&str, Vec<&str>)> = match item {
TriggerItemRaw::Simple(s) => {
if let Some(pattern) = s.strip_prefix("on_finish_of:") {
vec![("on_finish_of", vec![pattern.trim()])]
} else {
vec![(s.as_str(), vec![])]
}
}
TriggerItemRaw::Map(map) => map.iter().map(|(k, v)|
(k.as_str(), v.iter().map(|s| s.as_str()).collect())
).collect(),
};
let mut triggers = Vec::new();
for (key, patterns) in entries {
match key {
"on_finish_of" => {
for pattern in patterns {
if let Some(t) = parse_on_finish_of(pattern) {
triggers.push(t);
}
}
}
"on_success" => triggers.push(ParsedTrigger::OnSuccess),
"on_failure" => triggers.push(ParsedTrigger::OnFailure),
_ => log::warn!("Unknown trigger key: {}", key),
}
}
triggers
}
fn merge_templates(
method_template: &NotificationTemplate,
policy_template: &Option<serde_yaml::Value>,
) -> NotificationTemplate {
let mut merged = method_template.clone();
if let Some(policy_tmpl) = policy_template
&& let Ok(policy) = serde_yaml::from_value::<NotificationTemplate>(policy_tmpl.clone())
{
merged += policy;
}
merged
}
struct RuleOverrides {
fallible: bool,
max_lives: u32,
timeout: std::time::Duration,
retry_backoff: f64,
}
impl RuleOverrides {
fn from_value(v: &serde_yaml::Value) -> Self {
RuleOverrides {
fallible: v.get("fallible").and_then(|v| v.as_bool()).unwrap_or(false),
max_lives: v.get("max_lives").and_then(|v| v.as_u64()).unwrap_or(DEFAULT_NOTIFY_PARAMS.max_lives.value as u64) as u32,
timeout: std::time::Duration::from_millis(
v.get("timeout_ms").and_then(|v| v.as_u64()).unwrap_or(DEFAULT_NOTIFY_PARAMS.timeout_ms.value)
),
retry_backoff: v.get("retry_backoff").and_then(|v| v.as_f64()).unwrap_or(DEFAULT_NOTIFY_PARAMS.retry_backoff.value),
}
}
/// Chain: apply `source` atop existing values (source wins when present).
fn apply(self, source: &serde_yaml::Value) -> Self {
RuleOverrides {
fallible: source.get("fallible").and_then(|v| v.as_bool()).unwrap_or(self.fallible),
max_lives: source.get("max_lives").and_then(|v| v.as_u64()).unwrap_or(self.max_lives as u64) as u32,
timeout: source.get("timeout_ms").and_then(|v| v.as_u64()).map(std::time::Duration::from_millis).unwrap_or(self.timeout),
retry_backoff: source.get("retry_backoff").and_then(|v| v.as_f64()).unwrap_or(self.retry_backoff),
}
}
}
pub fn parse_rules(config: &NotificationConfig) -> NotificationRulesMap {
log::info!("[NOTIFY] parse_rules: groups={}", config.groups.len());
let mut rules = NotificationRulesMap::default();
for (priority, group) in &config.groups {
let mut ruleset: HashMap<Method, Vec<NotificationRule>> = HashMap::new();
for (method_name, method) in group {
let mut method_rules = Vec::new();
// Unify: no policy → one empty policy (same merge path)
let empty_policy = NotifyPolicy {
trigger: vec![],
template: serde_yaml::Value::Null,
overrides: serde_yaml::Value::Null,
};
let policies: &[NotifyPolicy] = if method.policy.is_empty() {
std::slice::from_ref(&empty_policy)
} else {
&method.policy
};
for policy in policies {
let trigger: Vec<ParsedTrigger> = if policy.trigger.is_empty() {
method.trigger.iter().flat_map(parse_trigger_item_raw).collect()
} else {
policy.trigger.iter().flat_map(parse_trigger_item_raw).collect()
};
let template =
merge_templates(&method.template, &Some(policy.template.clone()));
let vals = RuleOverrides::from_value(&serde_yaml::Value::Null)
.apply(&method.config)
.apply(&policy.overrides);
method_rules.push(NotificationRule {
enabled: true,
trigger,
template,
fallible: vals.fallible,
max_lives: vals.max_lives,
timeout: vals.timeout,
config: method.config.clone(),
retry_backoff: vals.retry_backoff,
});
}
ruleset.insert(method_name.clone(), method_rules);
}
rules.insert(*priority, ruleset);
}
rules
}
+281
View File
@@ -0,0 +1,281 @@
use std::collections::HashMap;
use minijinja::Environment;
use workshop_engine::ExecutionContext;
use crate::finalize::NotificationTemplateDef;
use crate::notify::error::NotifyError;
use crate::notify::types::NotificationEvent;
use crate::types::buildstatus::StageOutcome;
use crate::types::resource::ResourceRegistry;
/// Maximum depth for template cross-references (prevent infinite recursion)
pub const TEMPLATE_REFERENCE_DEPTH: u32 = 10;
/// Build template context for a batch of notification events.
pub fn build_batch_template_context(
ctx: &ExecutionContext,
events: &[NotificationEvent],
) -> HashMap<String, serde_json::Value> {
let mut data = HashMap::new();
data.insert("pipeline".into(), serde_json::json!({
"name": ctx.pipeline_name,
}));
data.insert("count".into(), serde_json::json!(events.len()));
let event_list: Vec<_> = events
.iter()
.map(|e| {
serde_json::json!({
"stage": format!("{}.{}", e.stage.as_str(), e.substage),
"status": e.outcome.to_string(),
})
})
.collect();
data.insert("events".into(), serde_json::json!(event_list));
// Single event compatibility: provide direct access
if events.len() == 1 {
let e = &events[0];
data.insert("build".into(), serde_json::json!({
"id": ctx.task_id,
"status": e.outcome.to_string(),
"stage": format!("{}.{}", e.stage.as_str(), e.substage),
"duration": "0s",
"url": "",
}));
} else {
data.insert("build".into(), serde_json::json!({
"id": ctx.task_id,
"status": "batched",
"stage": "batch",
"duration": "0s",
"url": "",
}));
}
data.insert("commit".into(), serde_json::json!({
"hash": "",
"short_hash": "",
"message": "",
"author": "",
"author_email": "",
}));
data
}
/// Build template context for a single notification event (convenience wrapper).
pub fn build_template_context(
ctx: &ExecutionContext,
event: &NotificationEvent,
) -> HashMap<String, serde_json::Value> {
build_batch_template_context(ctx, std::slice::from_ref(event))
}
/// Render a minijinja template string with the provided data.
pub fn render_template(
template: &str,
data: &HashMap<String, serde_json::Value>,
) -> Result<String, String> {
let env = Environment::new();
let tmpl = env
.template_from_str(template)
.map_err(|e| format!("Template parse error: {}", e))?;
let result = tmpl
.render(data)
.map_err(|e| format!("Template render error: {}", e))?;
Ok(result)
}
/// Select template string from NotificationTemplateDef based on outcome.
pub fn select_template_str<'a>(
tmpl: &'a NotificationTemplateDef,
outcome: &StageOutcome,
) -> Option<&'a str> {
match outcome {
StageOutcome::PipelineSuccess => tmpl.on_success.as_deref(),
StageOutcome::PipelineFailure => tmpl.on_failure.as_deref(),
_ => None,
}
}
/// Resolve external template references from the resource registry.
pub fn resolve_external_template_refs(
templates: &mut HashMap<String, NotificationTemplateDef>,
registry: &ResourceRegistry,
) {
for (name, tmpl_def) in templates.iter_mut() {
if tmpl_def.use_.is_none() {
continue;
}
let Some(path) = registry.resolve(name) else {
continue;
};
let Ok(content) = std::fs::read_to_string(path.join("template.yml"))
.or_else(|_| std::fs::read_to_string(path.join("template.yaml")))
else {
continue;
};
let Ok(fetched) =
serde_yaml::from_str::<NotificationTemplateDef>(&content)
else {
continue;
};
*tmpl_def = fetched.apply(std::mem::take(tmpl_def));
tmpl_def.use_ = None;
}
}
/// Resolve a schema reference into a NotificationTemplateDef by walking a fallback chain.
pub fn resolve_template(
schema: &Option<String>,
method_name: &str,
templates: &HashMap<String, NotificationTemplateDef>,
_standalone: bool,
server_templates: Option<&HashMap<String, NotificationTemplateDef>>,
) -> Result<NotificationTemplateDef, NotifyError> {
// 1. Try the initial schema directly
if let Some(schema_name) = schema.as_deref() {
if let Some(template) =
resolve_template_by_schema(schema_name, method_name, templates, server_templates)
{
return Ok(template);
}
}
// 2. Walk the fallback chain
let mut schema_next = resolve_template_next_schema(schema);
while let Some(ref schema_name) = schema_next {
if let Some(template) =
resolve_template_by_schema(schema_name, method_name, templates, server_templates)
{
return Ok(template);
}
log::warn!("Failed to resolve template for schema {}.", schema_name);
schema_next = resolve_template_next_schema(&schema_next);
}
Err(NotifyError::SchemaError(format!("All template resolution attempts beginning from {} failed.", schema.as_deref().unwrap_or("[system default]"))))
}
/// Return the next schema name in the fallback chain, or None to terminate.
pub fn resolve_template_next_schema(schema: &Option<String>) -> Option<String> {
match schema.as_deref() {
Some("custom") => None,
Some("default") => Some("fallback".to_string()),
Some("fallback") => None,
Some(_) => Some("default".to_string()),
None => Some("default".to_string()),
}
}
/// Perform a single lookup for a given schema name. Returns None if not found.
pub fn resolve_template_by_schema(
schema: &str,
method_name: &str,
templates: &HashMap<String, NotificationTemplateDef>,
server_templates: Option<&HashMap<String, NotificationTemplateDef>>,
) -> Option<NotificationTemplateDef> {
match schema {
"custom" => Some(NotificationTemplateDef::default()),
"fallback" => templates.get(&format!("{}.fallback", method_name)).cloned(),
"default" => server_templates.and_then(|b| b.get(method_name)).cloned(),
name => templates.get(name).cloned(),
}
}
/// Flatten all local template cross-references (use: "other-name").
pub fn flatten_template_refs(
templates: &mut HashMap<String, NotificationTemplateDef>,
) -> Result<(), NotifyError> {
let names: Vec<String> = templates.keys().cloned().collect();
for name in names {
let mut visited = std::collections::HashSet::new();
resolve_chain(&name, templates, 0, &mut visited)?;
}
Ok(())
}
fn resolve_chain(
name: &str,
templates: &mut HashMap<String, NotificationTemplateDef>,
depth: u32,
visited: &mut std::collections::HashSet<String>,
) -> Result<(), NotifyError> {
if depth > TEMPLATE_REFERENCE_DEPTH {
return Err(NotifyError::TemplateDepthExceeded {
name: name.to_string(),
max: TEMPLATE_REFERENCE_DEPTH,
});
}
if !visited.insert(name.to_string()) {
return Err(NotifyError::TemplateLoop(name.to_string()));
}
// Only process local references (no ://)
if let Some(ref_name) = templates.get(name).and_then(|t| t.use_.clone()) {
if !ref_name.contains("://") {
if !templates.contains_key(&ref_name) {
return Err(NotifyError::SchemaError(format!(
"template '{}' references non-existent template '{}'",
name, ref_name
)));
}
resolve_chain(&ref_name, templates, depth + 1, visited)?;
// Merge: base provides defaults, referencing template overrides
let base = templates[&ref_name].clone();
let template = templates.get_mut(name).unwrap();
*template = std::mem::take(template).apply(base);
template.use_ = None;
}
}
visited.remove(name);
Ok(())
}
#[cfg(test)]
mod tests {
use super::*;
#[test]
fn test_render_template_simple() {
let template = "Hello {{ name }}!";
let mut data = HashMap::new();
data.insert(
"name".to_string(),
serde_json::Value::String("World".to_string()),
);
let result = render_template(template, &data).unwrap();
assert_eq!(result, "Hello World!");
}
#[test]
fn test_render_template_with_build_status() {
let template = "Build {{ build.status }} for pipeline {{ pipeline.name }}";
let mut data = HashMap::new();
data.insert(
"build".to_string(),
serde_json::json!({ "status": "success" }),
);
data.insert(
"pipeline".to_string(),
serde_json::json!({ "name": "test-pipeline" }),
);
let result = render_template(template, &data).unwrap();
assert_eq!(result, "Build success for pipeline test-pipeline");
}
#[test]
fn test_render_template_invalid_syntax() {
let template = "{{ Unterminated";
let data = HashMap::new();
let result = render_template(template, &data);
assert!(result.is_err());
}
}
+57 -23
View File
@@ -7,14 +7,13 @@ use regex::Regex;
use serde::Deserialize;
use serde::Serialize;
use std::collections::HashMap;
use std::collections::HashSet;
use std::fmt::Display;
use std::hash::Hash;
use std::time::Duration;
use uuid::Uuid;
use crate::finalize::NotificationTemplate;
use crate::finalize::config::Trigger;
/// Final runtime configuration for notification, restored and flattened from yaml
pub struct NotificationRule {
pub enabled: bool,
@@ -78,12 +77,39 @@ impl ParsedTrigger {
impl NotificationRule {
pub fn accept(&self, result: &StageOutcome, stage: &StagePhase, substage: &str) -> bool {
if self.trigger.is_empty() {
eprintln!("[ACCEPT] trigger empty, returning true");
return true;
}
self.trigger
.iter()
.any(|t| t.accept(result, stage, substage))
}
}
/// Status of a single notification method after plugin execution.
#[derive(Debug)]
pub enum MethodStatus {
Success,
TemporaryError(String),
PermanentError(String),
Exhausted(String),
}
/// Tracks which notification methods still need attention within a priority group.
#[derive(Debug, Clone, Serialize)]
pub struct NotificationTarget {
pub priority: u32,
pub rules: Option<Vec<String>>,
pub count: u32,
}
impl Default for NotificationTarget {
fn default() -> Self {
Self { priority: 0, rules: None, count: 0 }
}
}
pub type Priority = u32;
pub type Method = String;
pub type NotificationRulesMap = HashMap<Priority, HashMap<Method, Vec<NotificationRule>>>;
@@ -101,9 +127,9 @@ pub struct NotificationEvent {
pub not_before: DateTime<Utc>,
pub priority: u32,
pub effective_priority: u32,
pub lives: u32,
pub max_lives: u32,
pub target: HashSet<String>, // Target audience
pub retry_backoff: f64, // Exponential backoff factor
pub attempted: u32, // Cumulative cross-group retry count
pub target: NotificationTarget, // Current audience
}
impl Hash for NotificationEvent {
@@ -137,25 +163,11 @@ impl Ord for NotificationEvent {
}
}
/// Delay before retrying an unavailable plugin (5 seconds)
pub const NOTIFICATION_NOTREADY_DELAY: std::time::Duration = std::time::Duration::from_secs(5);
use std::sync::LazyLock;
use workshop_baker_params::{NotificationParams, MergeParams};
/// Exponential backoff base (2.0 = double each retry)
pub const NOTIFICATION_TEMPORARY_DELAY_FACTOR: f64 = 2.0;
/// Initial backoff delay for temporary errors (5 seconds)
pub const NOTIFICATION_TEMPORARY_DELAY_TIME: f64 = 5.0;
/// Upper bound for backoff delay (unbounded)
pub const NOTIFICATION_TEMPORARY_MAX_TIME: f64 = f64::MAX;
/// Lives recovered after downgrade as fraction of max_lives (0.0 = none)
pub const NOTIFICATION_LIFE_RECOVERY_FACTOR: f64 = 0.0;
pub const NOTIFICATION_DEFAULT_FALLIBLE: bool = false;
pub const NOTIFICATION_DEFAULT_MAX_LIVES: u32 = 3;
pub const NOTIFICATION_DEFAULT_TIMEOUT_MS: u64 = 30000;
pub const NOTIFICATION_DEFAULT_RETRY_BACKOFF: f64 = 2.0;
/// Default parameters used when no external config is provided.
pub static DEFAULT_NOTIFY_PARAMS: LazyLock<NotificationParams> = LazyLock::new(|| NotificationParams::defaults());
#[derive(Default, Debug, Deserialize, Clone)]
pub enum NotificationRenderer {
@@ -174,3 +186,25 @@ impl Display for NotificationRenderer {
}
}
}
/// Notification batching configuration.
/// Controls how notification events are grouped before dispatch.
#[derive(Debug, Clone)]
pub struct BatchConfig {
/// Whether batching is enabled
pub enabled: bool,
/// Group period in seconds (the maximum wait time for a batch)
pub period: u32,
/// Group watermark threshold (the minimum number of events before batching)
pub watermark: u32,
}
impl Default for BatchConfig {
fn default() -> Self {
Self {
enabled: true,
period: 30,
watermark: 5,
}
}
}
+83
View File
@@ -0,0 +1,83 @@
use crate::finalize::FinalizeConfig;
use crate::finalize::error::PluginError;
use crate::notify::error::NotifyError;
use crate::notify::types::{DEFAULT_NOTIFY_PARAMS, NotificationEvent, NotificationRule, NotificationRulesMap};
use std::collections::HashSet;
pub fn validate(finalize: &FinalizeConfig) -> anyhow::Result<()> {
let validate_result = finalize.validate();
if let Err(errors) = validate_result {
for error in &errors {
log::error!("Error: {}", error);
}
anyhow::bail!(NotifyError::FinalizeValidationError(errors))
};
Ok(())
}
pub fn extract_plugin(finalize: &FinalizeConfig) -> HashSet<String> {
finalize
.notification
.groups
.values()
.flat_map(|methods| methods.keys())
.cloned()
.collect()
}
pub fn is_temporary_error(e: &PluginError) -> bool {
!matches!(e, PluginError::GeneralError(_))
}
// calculate min(1, k/x)*t as notification grouping period
// to avoid excessive notification frequency
pub fn wait_time(group_period: u32, group_watermark: u32, count: u32) -> f32 {
if count == 0 {
log::warn!("Unexpected notification count: {}", count);
return group_period as f32;
}
f32::min(1.0, group_watermark as f32 / count as f32) * group_period as f32
}
pub fn notify_delay(event: &NotificationEvent) -> std::time::Duration {
let exponent = event.target.count as i32;
if exponent < 0 {
log::warn!("Unexpected count: {}", event.target.count);
return std::time::Duration::from_secs_f64(DEFAULT_NOTIFY_PARAMS.max_retry_delay_seconds.value);
}
let factor = if event.retry_backoff > 0.0 {
event.retry_backoff
} else {
DEFAULT_NOTIFY_PARAMS.retry_backoff.value
};
let multiplier = factor.powi(exponent);
let delay = multiplier * DEFAULT_NOTIFY_PARAMS.retry_delay_seconds.value;
if !delay.is_finite() || delay < 0.0 {
return std::time::Duration::from_secs_f64(DEFAULT_NOTIFY_PARAMS.max_retry_delay_seconds.value);
}
std::time::Duration::from_secs_f64(delay)
}
pub fn find_rule<'a>(
config: &'a NotificationRulesMap,
method: &str,
rule_id: usize,
) -> Result<&'a NotificationRule, NotifyError> {
for ruleset in config.values() {
if let Some(rules) = ruleset.get(method) {
if let Some(rule) = rules.get(rule_id) {
return Ok(rule);
}
}
}
Err(NotifyError::SchemaError("rule not found".into()))
}
pub fn get_fallible(config: &NotificationRulesMap, method: &str) -> bool {
for ruleset in config.values() {
if let Some(rules) = ruleset.get(method) {
return rules.iter().all(|r| r.fallible);
}
}
false
}

Some files were not shown because too many files have changed in this diff Show More