Compare commits
12 Commits
cf317b55c6
...
master
| Author | SHA1 | Date | |
|---|---|---|---|
| df5026cdbe | |||
| d1ad08ef3a | |||
| cf2968e720 | |||
| 28abc5d207 | |||
| 6e7b96cd66 | |||
| 3e80825d58 | |||
| c97eafab29 | |||
| f737b6a24d | |||
| 681b7d1333 | |||
| f07588a716 | |||
| b0530fb858 | |||
| fd18b547e5 |
+3
-1
@@ -9,7 +9,7 @@ archived
|
|||||||
*.bak
|
*.bak
|
||||||
*.clean
|
*.clean
|
||||||
session*
|
session*
|
||||||
llm.env
|
.secret
|
||||||
|
|
||||||
# System test artifacts
|
# System test artifacts
|
||||||
tests/results/
|
tests/results/
|
||||||
@@ -18,3 +18,5 @@ tests/**/__pycache__/
|
|||||||
tests/.pytest_cache/
|
tests/.pytest_cache/
|
||||||
tests/llm/.cache/
|
tests/llm/.cache/
|
||||||
*.pyc
|
*.pyc
|
||||||
|
.staging
|
||||||
|
.pytest_cache
|
||||||
|
|||||||
@@ -1,83 +1,72 @@
|
|||||||
# PROJECT KNOWLEDGE BASE
|
# PROJECT KNOWLEDGE BASE
|
||||||
|
|
||||||
**Generated:** 2026-04-22
|
**Generated:** 2026-05-19
|
||||||
**Commit:** 7b3b71a
|
**Commit:** 28abc5d
|
||||||
**Branch:** master
|
**Branch:** master
|
||||||
|
|
||||||
## OVERVIEW
|
## OVERVIEW
|
||||||
HoneyBiscuitWorkshop is a Rust-based CI/CD system ("蜜饼工坊") with LLM-powered pipeline auto-configuration. 9 independent Cargo crates, no workspace.
|
HoneyBiscuitWorkshop ("蜜饼工坊") — Rust CI/CD system with decorator-driven shell pipeline DSL and LLM-powered configuration. 3 independent Cargo crates, no workspace.
|
||||||
|
|
||||||
## STRUCTURE
|
## STRUCTURE
|
||||||
```
|
```
|
||||||
./
|
./
|
||||||
├── workshop-baker/ # Main orchestrator (CLI + daemon)
|
├── workshop-baker/ # Main orchestrator (CLI + pipeline stages)
|
||||||
├── workshop-agent/ # HTTP agent server
|
├── workshop-engine/ # Execution engine (extracted from baker)
|
||||||
├── workshop-pipeline/ # Config library (prebake/finalize)
|
├── workshop-getterurl/ # URL fetching (go-getter style)
|
||||||
├── workshop-llm-detector/ # LLM repo analyzer
|
├── docs/ # mdBook documentation (zh-CN)
|
||||||
├── workshop-vault/ # Secret management client
|
├── templates/ # Pipeline templates (bake.sh, finalize.yml, prebake.yml)
|
||||||
├── workshop-cert/ # TLS certificate generator
|
└── archived/ # Deprecated crates (7 moved here)
|
||||||
├── workshop-deviceid/ # Device ID library
|
|
||||||
├── workshop-builder-native/ # Stub (unused)
|
|
||||||
├── workshop-helper-mac/ # MAC address utility
|
|
||||||
├── docs/ # mdBook documentation (zh-CN)
|
|
||||||
├── playground/ # Experimentation
|
|
||||||
└── third_party/ # Empty
|
|
||||||
```
|
```
|
||||||
|
|
||||||
## WHERE TO LOOK
|
## WHERE TO LOOK
|
||||||
| Task | Location | Notes |
|
| Task | Location | Notes |
|
||||||
|------|----------|-------|
|
|------|----------|-------|
|
||||||
| Baker (main) | `workshop-baker/src/{bake,prebake,engine}/` | Core pipeline logic |
|
| Baker (CLI) | `workshop-baker/src/{lib,main}.rs` | Clap CLI + tokio entry |
|
||||||
| Agent server | `workshop-agent/src/` | Actix-web HTTP + TLS |
|
| Pipeline stages | `workshop-baker/src/{bake,prebake,finalize}/` | Stage orchestration |
|
||||||
| Pipeline config | `workshop-pipeline/src/config/` | prebake/finalize YAML |
|
| Notifications | `workshop-baker/src/notify/` | Plugin-based event notifications |
|
||||||
| LLM integration | `workshop-llm-detector/src/llm/` | Ollama/OpenAI clients |
|
|| Engine (execution) | `workshop-engine/src/` | Process mgmt, PM detection |
|
||||||
| Secrets | `workshop-vault/src/` | Vault client wrapper |
|
| URL fetching | `workshop-getterurl/src/` | Git/HTTP/File resource getter |
|
||||||
|
|
||||||
## CODE MAP
|
## CODE MAP
|
||||||
| Symbol | Type | Location | Role |
|
| Symbol | Type | Location | Role |
|
||||||
|--------|------|----------|------|
|
|--------|------|----------|------|
|
||||||
| PipelineConfig | struct | workshop-pipeline/src/lib.rs:27 | Full pipeline config |
|
| Cli | struct | workshop-baker/src/lib.rs:21 | CLI argument struct |
|
||||||
| CICDLLMHelper | struct | workshop-llm-detector/src/lib.rs:16 | LLM analysis entry |
|
| ExecutionContext | struct | workshop-engine/src/types.rs:10 | Pipeline context |
|
||||||
| Engine | struct | workshop-baker/src/engine.rs | Build executor |
|
| Engine | struct | workshop-engine/src/types.rs:99 | Build executor (placeholder) |
|
||||||
| VaultClient | struct | workshop-vault/src/client.rs | Secret ops |
|
| PrebakeConfig | struct | workshop-baker/src/prebake/config.rs | Prebake YAML schema |
|
||||||
|
| FinalizeConfig | struct | workshop-baker/src/finalize/config.rs:21 | Finalize YAML schema |
|
||||||
|
| GetterUrl | struct | workshop-getterurl/src/lib.rs:84 | Parsed resource URL |
|
||||||
|
| Getter | trait | workshop-getterurl/src/getter.rs:18 | Resource fetch trait |
|
||||||
|
|
||||||
## CONVENTIONS
|
## CONVENTIONS
|
||||||
|
|
||||||
### Rust
|
### Rust
|
||||||
- **Edition**: 2024 (non-standard, requires Rust 1.85+)
|
- **Edition**: 2024 (requires Rust 1.85+), deprecates `mod.rs`
|
||||||
- Deprecate `mod.rs`
|
|
||||||
- **Naming**: `snake_case` files/modules, `PascalCase` types, `SCREAMING_SNAKE_CASE` consts
|
- **Naming**: `snake_case` files/modules, `PascalCase` types, `SCREAMING_SNAKE_CASE` consts
|
||||||
- **Error handling**: `thiserror` + `anyhow` combo
|
- **Error handling**: `thiserror` + `anyhow` combo; `HasExitCode` trait for exit codes
|
||||||
- **Async**: `#[tokio::main]` + `tokio` with "full" features
|
- **Async**: `#[tokio::main]` + `tokio` with "full" features
|
||||||
- **Imports**: `std` → `external_crate` → `crate::module`
|
- **Imports**: `std` → `external_crate` → `crate::module`
|
||||||
|
- **Dual-mode**: CLI commands (prebake/bake/finalize) + daemon mode (unimplemented)
|
||||||
|
|
||||||
### Testing
|
### Testing
|
||||||
- **Rust**: `#[test]` inline, `#[tokio::test]` async, `tests/*.rs` integration
|
- **Rust**: `#[test]` inline, `#[tokio::test]` async, `tests/` integration
|
||||||
- **Python**: `pytest` with `tests/integration/test_*.py` (Docker-based)
|
- **Benchmarks**: Criterion (configured in Cargo.toml)
|
||||||
- **Benchmarks**: Criterion with `harness = false`
|
|
||||||
|
|
||||||
### Build Pipeline
|
### CI
|
||||||
- **prebake.yml**: Environment setup (docker/firecracker/baremetal)
|
- **GitHub Actions**: rust-toolchain (stable), clippy + rustfmt, `cargo test --all-features`
|
||||||
- **bake.sh**: Build execution with `# @pipeline` decorators
|
- **Matrix**: workshop-baker, workshop-engine
|
||||||
- **finalize.yml**: Packaging, deployment, notifications
|
|
||||||
- **Annotations**: `@timeout()`, `@retry()`, `@parallel`, `@fallible`
|
|
||||||
|
|
||||||
## ANTI-PATTERNS (THIS PROJECT)
|
## ANTI-PATTERNS (THIS PROJECT)
|
||||||
|
1. **Hardcoded user** — "vulcan" username hardcoded throughout prebake
|
||||||
1. **Python in Rust project** — workshop-baker has Python tests (pytest.ini, test_prebake.py)
|
2. **Unsafe libc** — workshop-baker/src/prebake/security.rs has 5+ unsafe blocks
|
||||||
2. **temp/ directory** — Non-standard in Rust projects
|
3. **Rust 2024 edition** — May not work with stable toolchains
|
||||||
3. **Certificates in source** — artifact workshop-cert/certs/ not excluded
|
4. **Empty daemon** — `daemon/` dir is empty, Daemon command is `unimplemented!()`
|
||||||
4. **Deprecated projects** — workshop-executor.old, workshop-monitor.old still present
|
|
||||||
5. **Hardcoded mirrors** — Tsinghua University mirrors in configs
|
|
||||||
6. **Rust 2024 edition** — May not work with stable toolchains
|
|
||||||
|
|
||||||
## UNIQUE STYLES
|
## UNIQUE STYLES
|
||||||
|
|
||||||
- **Pipeline DSL**: Shell scripts with `# @decorator` annotations
|
- **Pipeline DSL**: Shell scripts with `# @decorator` annotations
|
||||||
- **Dual-mode baker**: CLI commands + daemon mode
|
- **Notification system**: Plugin-based with batching, retry, priority groups
|
||||||
- **LLM cookbook system**: YAML cookbooks for language-specific builds
|
- **LLM cookbook system**: YAML cookbooks for language-specific builds (archived)
|
||||||
- **Chinese docs** — Documentation primarily in zh-CN
|
- **Chinese docs** — Documentation primarily in zh-CN
|
||||||
- **User "vulcan"** — Custom build user (not root/runner)
|
|
||||||
|
|
||||||
## COMMANDS
|
## COMMANDS
|
||||||
```bash
|
```bash
|
||||||
@@ -86,23 +75,18 @@ cargo build --release -p workshop-baker
|
|||||||
|
|
||||||
# Test
|
# Test
|
||||||
cargo test -p workshop-baker
|
cargo test -p workshop-baker
|
||||||
pytest tests/integration/test_prebake.py -v
|
cargo test -p workshop-engine
|
||||||
|
|
||||||
# Lint
|
# Lint
|
||||||
cargo clippy -- -D warnings
|
cargo clippy -- -D warnings
|
||||||
cargo fmt --all
|
cargo fmt --all
|
||||||
|
|
||||||
# Run baker
|
# Run
|
||||||
cargo run --bin workshop-baker -- prebake config.yml
|
cargo run --bin workshop-baker -- --pipeline demo --build-id 1 --prebake prebake.yml --bake bake.sh --finalize finalize.yml
|
||||||
cargo run --bin workshop-baker -- bake script.sh
|
|
||||||
|
|
||||||
# Run agent
|
|
||||||
cargo run --bin workshop-agent
|
|
||||||
```
|
```
|
||||||
|
|
||||||
## NOTES
|
## NOTES
|
||||||
|
|
||||||
- **Docker required** for integration tests
|
- **Docker required** for integration tests
|
||||||
- **workshop-builder-native** is a stub (3-line Hello World)
|
- **No root workspace** — each crate independent, build separately
|
||||||
- No root workspace Cargo.toml — each crate is independent
|
- **7 archived crates** — workshop-agent, workshop-cert, workshop-deviceid, workshop-helper-mac, workshop-llm-detector, workshop-pipeline, workshop-vault
|
||||||
- `AGENTS.md.old` at root — legacy documentation
|
- **Python tests removed** — pytest.ini exists but test files no longer in workspace
|
||||||
|
|||||||
@@ -35,3 +35,4 @@
|
|||||||
- [自定义 Decorator](zh-CN/vol3_dev/custom_decorator.md)
|
- [自定义 Decorator](zh-CN/vol3_dev/custom_decorator.md)
|
||||||
- [模板变量](zh-CN/vol3_dev/template_variables.md)
|
- [模板变量](zh-CN/vol3_dev/template_variables.md)
|
||||||
- [邮件模板](zh-CN/vol3_dev/mail_template.md)
|
- [邮件模板](zh-CN/vol3_dev/mail_template.md)
|
||||||
|
- [Resource 模型](zh-CN/vol3_dev/resource_model.md)
|
||||||
|
|||||||
@@ -0,0 +1,88 @@
|
|||||||
|
# Resource 模型
|
||||||
|
|
||||||
|
## 核心思想
|
||||||
|
|
||||||
|
整个系统只关心两件事:东西从哪来(fetch),东西到哪去(publish)。
|
||||||
|
|
||||||
|
fetch 方向:URL → 本地路径 → 使用。
|
||||||
|
publish 方向:本地路径 → 变换(插件)→ URL。
|
||||||
|
|
||||||
|
两个方向共用同一套寻址方式,但 Resource 中间态只存在于 fetch 方向。
|
||||||
|
|
||||||
|
## URL 格式
|
||||||
|
|
||||||
|
```
|
||||||
|
getter::url?param1=value1¶m2=value2
|
||||||
|
```
|
||||||
|
|
||||||
|
getter 显式声明意图。不是"我猜这是个 git 仓库",而是"使用者告诉我这是什么"。
|
||||||
|
|
||||||
|
只有三个 getter:
|
||||||
|
|
||||||
|
- `git` — 克隆仓库,必要时 checkout 指定版本
|
||||||
|
- `http` / `https` — 下载文件,必要时校验 checksum
|
||||||
|
- `file` — 复制本地文件或目录
|
||||||
|
|
||||||
|
版本锁定不用 `@v1.0`(`@` 在 URL 里是认证分隔符),用 `?ref=v1.0`。
|
||||||
|
|
||||||
|
没有 `github.com/user/repo` 看起来应该 clone 它这种黑魔法。
|
||||||
|
没有 `.tar.gz` 看起来应该解压它的惊喜。
|
||||||
|
URL 是 URL,行为是行为,两者用 getter:: 显式绑定。
|
||||||
|
|
||||||
|
## Resource 不是网络地址
|
||||||
|
|
||||||
|
流水线阶段(prebake / finalize / notify)只接触 Resource:一个名字 + 一个本地路径,已就绪。它们不经由网络、不解析 URL、不碰 getter。
|
||||||
|
|
||||||
|
所有 fetch 行为在阶段启动前完成:
|
||||||
|
|
||||||
|
- worker 模式:bakerd fetch → 填入 ResourceRegistry → 启动 baker
|
||||||
|
- standalone 模式:baker 自己 fetch → 填入 ResourceRegistry → 执行阶段
|
||||||
|
- bare 模式:不 fetch,传入的必须是 Resource
|
||||||
|
|
||||||
|
ResourceRegistry 是名字到路径的映射。填进去就只读了。
|
||||||
|
|
||||||
|
## getterurl 是独立的 crate
|
||||||
|
|
||||||
|
URL 格式解析 + 基本 fetch 放在 workshop-getterurl,和 baker 本身没有耦合。任何人都可以用。
|
||||||
|
|
||||||
|
crate 分两层:
|
||||||
|
|
||||||
|
```
|
||||||
|
结构层(parser):
|
||||||
|
"git::https://host/repo.git?ref=v1.0" → { getter: "git", url: Url, params: Params }
|
||||||
|
|
||||||
|
获取层(builtin getters):
|
||||||
|
getter.fetch(url, params, dst) → local_path
|
||||||
|
git: git clone + optional checkout
|
||||||
|
http: HTTP GET + optional checksum
|
||||||
|
file: std::fs::copy
|
||||||
|
```
|
||||||
|
|
||||||
|
获取层是字面意义的——不解压、不推断、不猜意图。
|
||||||
|
|
||||||
|
## Checksum
|
||||||
|
|
||||||
|
checksum 作为 URL query 参数传递。三态语义:
|
||||||
|
|
||||||
|
- `?checksum=sha256:abc` — 必须匹配
|
||||||
|
- `?checksum=` — 显式禁用校验
|
||||||
|
- 无 checksum — 如果服务端返回 Content-Digest 头,校验它
|
||||||
|
|
||||||
|
getter 不主动校验,它只执行 URL 里指定的操作。
|
||||||
|
|
||||||
|
## Artifact 不经过 Resource
|
||||||
|
|
||||||
|
artifact 是 bake 的产物,自产生就开始被使用,不存在中间态。它的路径就是 bake 产出的本地路径,不走 ResourceRegistry。
|
||||||
|
|
||||||
|
artifact 的 publish 方向由插件定义行为——可能上传、可能扫描、可能只记录。插件定义最终的行为,只要是自洽的。
|
||||||
|
|
||||||
|
## 三种运行模式
|
||||||
|
|
||||||
|
```
|
||||||
|
fetch 执行者 fetch 在阶段内
|
||||||
|
daemon 模式 bakerd 否
|
||||||
|
standalone baker 自身 否(阶段启动前已完成)
|
||||||
|
bare 模式 无 否(必须传入 Resource)
|
||||||
|
```
|
||||||
|
|
||||||
|
bare 模式下传入的必须是已经就绪的 Resource。外部 URL 是 daemon 或 standalone 的事,bare 不负责。
|
||||||
@@ -1,101 +0,0 @@
|
|||||||
#!/bin/bash
|
|
||||||
set -euo pipefail
|
|
||||||
|
|
||||||
# HoneyBiscuitWorkshop System Test Runner
|
|
||||||
#
|
|
||||||
# Usage:
|
|
||||||
# ./runner.sh # Arch baker tests (hbw-system-test:latest)
|
|
||||||
# ./runner.sh --image hbw-test-pm:test-apt # Debian engine PM tests
|
|
||||||
# ./runner.sh -k test_install # Arch baker, filtered
|
|
||||||
|
|
||||||
MODE="auto"
|
|
||||||
TEST_IMAGE=""
|
|
||||||
TEST_DIR="tests/baker"
|
|
||||||
PYTEST_ARGS=""
|
|
||||||
|
|
||||||
while [[ $# -gt 0 ]]; do
|
|
||||||
case "$1" in
|
|
||||||
--image) TEST_IMAGE="$2"; TEST_DIR="tests/engine"; shift 2 ;;
|
|
||||||
manual) MODE="manual"; shift ;;
|
|
||||||
auto) MODE="auto"; shift ;;
|
|
||||||
--) shift; PYTEST_ARGS="$@"; break ;;
|
|
||||||
*) PYTEST_ARGS="$PYTEST_ARGS $1"; shift ;;
|
|
||||||
esac
|
|
||||||
done
|
|
||||||
|
|
||||||
# Default image: Arch-based baker test image
|
|
||||||
: ${TEST_IMAGE:="hbw-system-test:latest"}
|
|
||||||
|
|
||||||
SCRIPT_DIR="$(cd "$(dirname "$0")" && pwd)"
|
|
||||||
CONTAINER_NAME="hbw-system-test-container"
|
|
||||||
STAGING_DIR="/tmp/hbw-system-test"
|
|
||||||
|
|
||||||
cleanup() {
|
|
||||||
docker rm -f "$CONTAINER_NAME" 2>/dev/null || true
|
|
||||||
[ "$TEST_DIR" = "tests/baker" ] && rm -rf "$STAGING_DIR"
|
|
||||||
}
|
|
||||||
trap cleanup EXIT
|
|
||||||
|
|
||||||
build_binary() {
|
|
||||||
local target="x86_64-unknown-linux-musl"
|
|
||||||
local binary="$SCRIPT_DIR/workshop-baker/target/$target/debug/workshop-baker"
|
|
||||||
if [ ! -f "$binary" ]; then
|
|
||||||
echo "[runner] Building static (musl) baker binary..."
|
|
||||||
cd "$SCRIPT_DIR/workshop-baker"
|
|
||||||
cargo build -p workshop-baker --target "$target" 2>&1 | tail -n 1
|
|
||||||
fi
|
|
||||||
}
|
|
||||||
|
|
||||||
prepare_staging() {
|
|
||||||
# Only needed for baker tests (hbw-system-test:latest mounts /workshop)
|
|
||||||
if [ "$TEST_DIR" = "tests/baker" ]; then
|
|
||||||
mkdir -p "$STAGING_DIR"
|
|
||||||
cp "$SCRIPT_DIR/workshop-baker/target/x86_64-unknown-linux-musl/debug/workshop-baker" "$STAGING_DIR/"
|
|
||||||
chmod +x "$STAGING_DIR/workshop-baker"
|
|
||||||
fi
|
|
||||||
}
|
|
||||||
|
|
||||||
run_container() {
|
|
||||||
local tag="${TEST_IMAGE##*:}"
|
|
||||||
local filter=""
|
|
||||||
case "$tag" in
|
|
||||||
test-arch) filter="-k pacman" ;;
|
|
||||||
test-apt) filter="-k apt" ;;
|
|
||||||
test-dnf) filter="-k dnf" ;;
|
|
||||||
test-apk) filter="-k apk" ;;
|
|
||||||
esac
|
|
||||||
|
|
||||||
echo "[runner] Starting: $TEST_IMAGE (tests=$TEST_DIR${filter:+, filter=$filter})"
|
|
||||||
mkdir -p "$SCRIPT_DIR/tests/results"
|
|
||||||
|
|
||||||
local mounts="-v $SCRIPT_DIR:/src:ro"
|
|
||||||
local entrypoint=""
|
|
||||||
|
|
||||||
if [ "$TEST_DIR" = "tests/baker" ]; then
|
|
||||||
mounts="$mounts -v $STAGING_DIR:/workshop:ro"
|
|
||||||
entrypoint='--entrypoint ""'
|
|
||||||
fi
|
|
||||||
|
|
||||||
local cmd="sh -c \"python3 -m venv /tmp/v && . /tmp/v/bin/activate && \
|
|
||||||
pip install pytest pytest-timeout -q && \
|
|
||||||
exec python -m pytest $TEST_DIR $PYTEST_ARGS $filter\""
|
|
||||||
|
|
||||||
if [ "$MODE" = "manual" ]; then
|
|
||||||
docker run -it --rm --name "$CONTAINER_NAME" --privileged $entrypoint \
|
|
||||||
$mounts "$TEST_IMAGE"
|
|
||||||
else
|
|
||||||
docker run --rm --name "$CONTAINER_NAME" --privileged $entrypoint \
|
|
||||||
$mounts -w /src \
|
|
||||||
"$TEST_IMAGE" "$cmd"
|
|
||||||
fi
|
|
||||||
}
|
|
||||||
|
|
||||||
echo "============================================"
|
|
||||||
echo " HBW System Test Runner"
|
|
||||||
echo " Image: $TEST_IMAGE"
|
|
||||||
echo " Tests: $TEST_DIR"
|
|
||||||
echo "============================================"
|
|
||||||
build_binary
|
|
||||||
prepare_staging
|
|
||||||
run_container
|
|
||||||
echo "[runner] Done."
|
|
||||||
@@ -1,32 +0,0 @@
|
|||||||
FROM archlinux:latest
|
|
||||||
|
|
||||||
RUN sed -i 's|https://geo.mirror.pkgbuild.com|https://mirrors.tuna.tsinghua.edu.cn/archlinux|g' /etc/pacman.d/mirrorlist && \
|
|
||||||
sed -i 's|https://mirror.rackspace.com/archlinux|https://mirrors.tuna.tsinghua.edu.cn/archlinux|g' /etc/pacman.d/mirrorlist && \
|
|
||||||
echo 'Server = https://mirrors.tuna.tsinghua.edu.cn/archlinux/$repo/os/$arch' >> /etc/pacman.d/mirrorlist
|
|
||||||
|
|
||||||
RUN pacman -Syu --noconfirm && \
|
|
||||||
pacman -S --noconfirm \
|
|
||||||
python python-pip base-devel git curl wget sudo \
|
|
||||||
which procps-ng shadow && \
|
|
||||||
pacman -Scc --noconfirm
|
|
||||||
|
|
||||||
# Create test user
|
|
||||||
RUN useradd -m -s /bin/bash testuser
|
|
||||||
|
|
||||||
# Python environment
|
|
||||||
WORKDIR /app
|
|
||||||
COPY tests/requirements.txt ./requirements.txt
|
|
||||||
RUN python -m venv /opt/test-venv && \
|
|
||||||
/opt/test-venv/bin/pip install -r requirements.txt
|
|
||||||
|
|
||||||
# Copy test suite
|
|
||||||
COPY tests/ ./tests/
|
|
||||||
COPY tests/entrypoint.sh ./entrypoint.sh
|
|
||||||
RUN chmod +x entrypoint.sh
|
|
||||||
|
|
||||||
# Environment
|
|
||||||
ENV PATH="/opt/test-venv/bin:$PATH"
|
|
||||||
ENV PYTHONUNBUFFERED=1
|
|
||||||
ENV PYTHONDONTWRITEBYTECODE=1
|
|
||||||
|
|
||||||
ENTRYPOINT ["./entrypoint.sh"]
|
|
||||||
@@ -1,129 +0,0 @@
|
|||||||
import os
|
|
||||||
import pytest
|
|
||||||
|
|
||||||
|
|
||||||
FIXTURES = os.path.join(os.path.dirname(__file__), "fixtures", "bake")
|
|
||||||
|
|
||||||
|
|
||||||
@pytest.mark.bake
|
|
||||||
class TestBakeSimpleScript:
|
|
||||||
|
|
||||||
def test_simple_script_runs(self, run_baker, bake_workspace, work_dir):
|
|
||||||
prebake_cfg = os.path.join(
|
|
||||||
os.path.dirname(__file__), "fixtures", "prebake", "minimal.yml"
|
|
||||||
)
|
|
||||||
script = os.path.join(FIXTURES, "simple.sh")
|
|
||||||
bake_base = os.path.join(FIXTURES, "bake_base.sh")
|
|
||||||
|
|
||||||
result = run_baker(
|
|
||||||
"bake", script,
|
|
||||||
"--bake-base", bake_base,
|
|
||||||
"--prebake", prebake_cfg,
|
|
||||||
cwd=work_dir,
|
|
||||||
timeout=60,
|
|
||||||
)
|
|
||||||
assert result["success"], f"Bake failed: {result['stderr']}"
|
|
||||||
assert "Hello from simple bake script" in result["stdout"]
|
|
||||||
|
|
||||||
def test_simple_script_sets_env_vars(self, run_baker, bake_workspace, work_dir):
|
|
||||||
prebake_cfg = os.path.join(
|
|
||||||
os.path.dirname(__file__), "fixtures", "prebake", "minimal.yml"
|
|
||||||
)
|
|
||||||
script = os.path.join(FIXTURES, "simple.sh")
|
|
||||||
bake_base = os.path.join(FIXTURES, "bake_base.sh")
|
|
||||||
|
|
||||||
result = run_baker(
|
|
||||||
"bake", script,
|
|
||||||
"--bake-base", bake_base,
|
|
||||||
"--prebake", prebake_cfg,
|
|
||||||
cwd=work_dir,
|
|
||||||
timeout=60,
|
|
||||||
)
|
|
||||||
assert result["success"]
|
|
||||||
assert "test-pipeline" in result["stdout"]
|
|
||||||
|
|
||||||
|
|
||||||
@pytest.mark.bake
|
|
||||||
class TestBakePipeline:
|
|
||||||
|
|
||||||
def test_pipeline_functions_execute(self, run_baker, bake_workspace, work_dir):
|
|
||||||
prebake_cfg = os.path.join(
|
|
||||||
os.path.dirname(__file__), "fixtures", "prebake", "minimal.yml"
|
|
||||||
)
|
|
||||||
script = os.path.join(FIXTURES, "pipeline.sh")
|
|
||||||
bake_base = os.path.join(FIXTURES, "bake_base.sh")
|
|
||||||
|
|
||||||
result = run_baker(
|
|
||||||
"bake", script,
|
|
||||||
"--bake-base", bake_base,
|
|
||||||
"--prebake", prebake_cfg,
|
|
||||||
cwd=work_dir,
|
|
||||||
timeout=60,
|
|
||||||
)
|
|
||||||
assert result["success"], f"Pipeline bake failed: {result['stderr']}"
|
|
||||||
assert os.path.exists("/tmp/bake-test-output/step1.txt")
|
|
||||||
assert os.path.exists("/tmp/bake-test-output/step2.txt")
|
|
||||||
|
|
||||||
def test_pipeline_step_outputs_correct(self, run_baker, bake_workspace, work_dir):
|
|
||||||
prebake_cfg = os.path.join(
|
|
||||||
os.path.dirname(__file__), "fixtures", "prebake", "minimal.yml"
|
|
||||||
)
|
|
||||||
script = os.path.join(FIXTURES, "pipeline.sh")
|
|
||||||
bake_base = os.path.join(FIXTURES, "bake_base.sh")
|
|
||||||
|
|
||||||
result = run_baker(
|
|
||||||
"bake", script,
|
|
||||||
"--bake-base", bake_base,
|
|
||||||
"--prebake", prebake_cfg,
|
|
||||||
cwd=work_dir,
|
|
||||||
timeout=60,
|
|
||||||
)
|
|
||||||
assert result["success"]
|
|
||||||
with open("/tmp/bake-test-output/step1.txt") as f:
|
|
||||||
assert f.read().strip() == "step1-done"
|
|
||||||
with open("/tmp/bake-test-output/step2.txt") as f:
|
|
||||||
assert f.read().strip() == "step2-done"
|
|
||||||
|
|
||||||
|
|
||||||
@pytest.mark.bake
|
|
||||||
class TestBakeAfterDependencies:
|
|
||||||
|
|
||||||
def test_after_ordering(self, run_baker, bake_workspace, work_dir):
|
|
||||||
prebake_cfg = os.path.join(
|
|
||||||
os.path.dirname(__file__), "fixtures", "prebake", "minimal.yml"
|
|
||||||
)
|
|
||||||
script = os.path.join(FIXTURES, "with_after.sh")
|
|
||||||
bake_base = os.path.join(FIXTURES, "bake_base.sh")
|
|
||||||
|
|
||||||
result = run_baker(
|
|
||||||
"bake", script,
|
|
||||||
"--bake-base", bake_base,
|
|
||||||
"--prebake", prebake_cfg,
|
|
||||||
cwd=work_dir,
|
|
||||||
timeout=60,
|
|
||||||
)
|
|
||||||
assert result["success"], f"@after bake failed: {result['stderr']}"
|
|
||||||
assert os.path.exists("/tmp/bake-after-test/status.txt")
|
|
||||||
assert os.path.exists("/tmp/bake-after-test/source.txt")
|
|
||||||
assert os.path.exists("/tmp/bake-after-test/build.txt")
|
|
||||||
|
|
||||||
|
|
||||||
@pytest.mark.bake
|
|
||||||
class TestBakeStatusFiles:
|
|
||||||
|
|
||||||
def test_status_written_after_bake(self, run_baker, bake_workspace, work_dir):
|
|
||||||
prebake_cfg = os.path.join(
|
|
||||||
os.path.dirname(__file__), "fixtures", "prebake", "minimal.yml"
|
|
||||||
)
|
|
||||||
script = os.path.join(FIXTURES, "simple.sh")
|
|
||||||
bake_base = os.path.join(FIXTURES, "bake_base.sh")
|
|
||||||
|
|
||||||
result = run_baker(
|
|
||||||
"bake", script,
|
|
||||||
"--bake-base", bake_base,
|
|
||||||
"--prebake", prebake_cfg,
|
|
||||||
cwd=work_dir,
|
|
||||||
timeout=60,
|
|
||||||
)
|
|
||||||
assert result["success"]
|
|
||||||
assert os.path.exists("/tmp/.hbwstatus")
|
|
||||||
@@ -1,77 +0,0 @@
|
|||||||
import os
|
|
||||||
import pytest
|
|
||||||
|
|
||||||
from probes import file
|
|
||||||
from llm import judge_log, LLMClient
|
|
||||||
|
|
||||||
|
|
||||||
FIXTURES = os.path.join(os.path.dirname(__file__), "fixtures", "finalize")
|
|
||||||
|
|
||||||
|
|
||||||
@pytest.mark.finalize
|
|
||||||
class TestFinalizeMinimal:
|
|
||||||
|
|
||||||
def test_minimal_finalize_succeeds(self, run_baker, work_dir):
|
|
||||||
config = os.path.join(FIXTURES, "minimal.yml")
|
|
||||||
result = run_baker("finalize", config, cwd=work_dir, timeout=60)
|
|
||||||
assert result["success"], f"Minimal finalize failed: {result['stderr']}"
|
|
||||||
|
|
||||||
|
|
||||||
@pytest.mark.finalize
|
|
||||||
class TestFinalizeHooks:
|
|
||||||
|
|
||||||
def test_early_hook_executes(self, run_baker, work_dir):
|
|
||||||
config = os.path.join(FIXTURES, "with_hooks.yml")
|
|
||||||
result = run_baker("finalize", config, cwd=work_dir, timeout=60)
|
|
||||||
assert result["success"]
|
|
||||||
assert "finalize early hook" in result["stdout"] or "finalize early hook" in result["stderr"]
|
|
||||||
|
|
||||||
def test_late_hook_executes(self, run_baker, work_dir):
|
|
||||||
config = os.path.join(FIXTURES, "with_hooks.yml")
|
|
||||||
result = run_baker("finalize", config, cwd=work_dir, timeout=60)
|
|
||||||
assert result["success"]
|
|
||||||
assert "finalize late hook" in result["stdout"] or "finalize late hook" in result["stderr"]
|
|
||||||
|
|
||||||
|
|
||||||
@pytest.mark.finalize
|
|
||||||
class TestFinalizeStatus:
|
|
||||||
|
|
||||||
def test_finalize_writes_status(self, run_baker, work_dir):
|
|
||||||
config = os.path.join(FIXTURES, "minimal.yml")
|
|
||||||
result = run_baker("finalize", config, cwd=work_dir, timeout=60)
|
|
||||||
assert result["success"]
|
|
||||||
assert os.path.exists("/tmp/.hbwstatus")
|
|
||||||
|
|
||||||
|
|
||||||
@pytest.mark.finalize
|
|
||||||
@pytest.mark.llm
|
|
||||||
class TestFinalizeWithLLM:
|
|
||||||
|
|
||||||
def test_finalize_hooks_via_llm(self, run_baker, llm_env, work_dir):
|
|
||||||
if not llm_env:
|
|
||||||
pytest.skip("LLM environment not configured")
|
|
||||||
|
|
||||||
config = os.path.join(FIXTURES, "with_hooks.yml")
|
|
||||||
result = run_baker("finalize", config, cwd=work_dir, timeout=60)
|
|
||||||
|
|
||||||
combined_log = result["stdout"] + "\n" + result["stderr"]
|
|
||||||
|
|
||||||
judgment = judge_log(
|
|
||||||
operation="finalize with early and late hooks",
|
|
||||||
expected_behavior="Both early and late hooks executed successfully, "
|
|
||||||
"printing 'finalize early hook' and 'finalize late hook'",
|
|
||||||
actual_log=combined_log,
|
|
||||||
)
|
|
||||||
assert judgment.passed, f"LLM judgment failed: {judgment.reason}"
|
|
||||||
|
|
||||||
|
|
||||||
@pytest.mark.finalize
|
|
||||||
class TestFinalizePlugins:
|
|
||||||
|
|
||||||
@pytest.mark.xfail(reason="Plugin download requires network/git access", strict=False)
|
|
||||||
def test_plugin_download(self, run_baker, work_dir):
|
|
||||||
pass
|
|
||||||
|
|
||||||
@pytest.mark.xfail(reason="Artifact stage not yet implemented", strict=False)
|
|
||||||
def test_artifact_collection(self, run_baker, work_dir):
|
|
||||||
pass
|
|
||||||
@@ -1,261 +0,0 @@
|
|||||||
import os
|
|
||||||
import json
|
|
||||||
import pytest
|
|
||||||
|
|
||||||
from probes import user, file, package
|
|
||||||
|
|
||||||
|
|
||||||
FIXTURES = os.path.join(os.path.dirname(__file__), "fixtures", "prebake")
|
|
||||||
|
|
||||||
|
|
||||||
@pytest.mark.prebake
|
|
||||||
class TestPrebakeConfigValidation:
|
|
||||||
|
|
||||||
def test_invalid_version_fails(self, run_baker, work_dir):
|
|
||||||
config = os.path.join(FIXTURES, "invalid_version.yml")
|
|
||||||
result = run_baker("prebake", config, cwd=work_dir, timeout=60)
|
|
||||||
assert not result["success"], f"Expected prebake to fail with invalid version"
|
|
||||||
assert "version" in result["stderr"].lower() or "Version" in result["stderr"]
|
|
||||||
|
|
||||||
def test_missing_required_field_fails(self, run_baker, work_dir):
|
|
||||||
missing_env = os.path.join(work_dir, "missing_env.yml")
|
|
||||||
with open(missing_env, "w") as f:
|
|
||||||
f.write('version: "1.0"\n')
|
|
||||||
result = run_baker("prebake", missing_env, cwd=work_dir, timeout=60)
|
|
||||||
assert not result["success"]
|
|
||||||
|
|
||||||
def test_minimal_config_succeeds(self, run_baker, work_dir):
|
|
||||||
config = os.path.join(FIXTURES, "minimal.yml")
|
|
||||||
result = run_baker("prebake", config, cwd=work_dir, timeout=60)
|
|
||||||
assert result["success"], f"Minimal prebake failed: {result['stderr']}"
|
|
||||||
|
|
||||||
|
|
||||||
@pytest.mark.prebake
|
|
||||||
class TestPrebakeBootstrap:
|
|
||||||
|
|
||||||
def test_bootstrap_creates_vulcan_user(self, run_baker, work_dir):
|
|
||||||
config = os.path.join(FIXTURES, "bootstrap_only.yml")
|
|
||||||
result = run_baker("prebake", config, cwd=work_dir, timeout=60)
|
|
||||||
assert result["success"], f"prebake failed: {result['stderr']}"
|
|
||||||
assert user.user_exists("vulcan")
|
|
||||||
|
|
||||||
def test_bootstrap_creates_workspace(self, run_baker, work_dir):
|
|
||||||
config = os.path.join(FIXTURES, "bootstrap_only.yml")
|
|
||||||
result = run_baker("prebake", config, cwd=work_dir, timeout=60)
|
|
||||||
assert result["success"]
|
|
||||||
assert file.dir_exists("/home/vulcan/workspace")
|
|
||||||
|
|
||||||
def test_bootstrap_creates_symlink(self, run_baker, work_dir):
|
|
||||||
config = os.path.join(FIXTURES, "bootstrap_only.yml")
|
|
||||||
result = run_baker("prebake", config, cwd=work_dir, timeout=60)
|
|
||||||
assert result["success"]
|
|
||||||
assert os.path.islink("/workspace")
|
|
||||||
|
|
||||||
def test_bootstrap_generates_bootstrap_script(self, run_baker, work_dir):
|
|
||||||
config = os.path.join(FIXTURES, "bootstrap_only.yml")
|
|
||||||
result = run_baker("prebake", config, cwd=work_dir, timeout=60)
|
|
||||||
assert result["success"]
|
|
||||||
assert file.file_exists("/bootstrap.sh")
|
|
||||||
|
|
||||||
def test_bootstrap_script_permissions(self, run_baker, work_dir):
|
|
||||||
config = os.path.join(FIXTURES, "bootstrap_only.yml")
|
|
||||||
result = run_baker("prebake", config, cwd=work_dir, timeout=60)
|
|
||||||
assert result["success"]
|
|
||||||
mode = file.file_permissions("/bootstrap.sh")
|
|
||||||
assert mode & 0o111, f"/bootstrap.sh not executable, mode: {oct(mode)}"
|
|
||||||
|
|
||||||
def test_bootstrap_script_contains_user_creation(self, run_baker, work_dir):
|
|
||||||
config = os.path.join(FIXTURES, "bootstrap_only.yml")
|
|
||||||
result = run_baker("prebake", config, cwd=work_dir, timeout=60)
|
|
||||||
assert result["success"]
|
|
||||||
assert file.file_exists("/bootstrap.sh")
|
|
||||||
assert file.file_contains("/bootstrap.sh", "useradd")
|
|
||||||
|
|
||||||
def test_bootstrap_script_contains_workspace_setup(self, run_baker, work_dir):
|
|
||||||
config = os.path.join(FIXTURES, "bootstrap_only.yml")
|
|
||||||
result = run_baker("prebake", config, cwd=work_dir, timeout=60)
|
|
||||||
assert result["success"]
|
|
||||||
assert file.file_contains("/bootstrap.sh", "mkdir -p /home/vulcan/workspace")
|
|
||||||
|
|
||||||
def test_bootstrap_with_pacman_creates_sudoers(self, run_baker, work_dir):
|
|
||||||
config = os.path.join(FIXTURES, "with_pacman.yml")
|
|
||||||
result = run_baker("prebake", config, cwd=work_dir, timeout=120)
|
|
||||||
assert result["success"], f"prebake failed: {result['stderr']}"
|
|
||||||
assert file.file_exists("/etc/sudoers.d/workshop")
|
|
||||||
|
|
||||||
def test_custom_bootstrap_executes(self, run_baker, work_dir):
|
|
||||||
config = os.path.join(FIXTURES, "with_custom_bootstrap.yml")
|
|
||||||
result = run_baker("prebake", config, cwd=work_dir, timeout=60)
|
|
||||||
assert result["success"], f"Custom bootstrap failed: {result['stderr']}"
|
|
||||||
assert file.dir_exists("/custom-workspace")
|
|
||||||
|
|
||||||
|
|
||||||
@pytest.mark.prebake
|
|
||||||
class TestPrebakeDryRun:
|
|
||||||
|
|
||||||
def test_dry_run_does_not_create_user(self, run_baker, work_dir):
|
|
||||||
config = os.path.join(FIXTURES, "bootstrap_only.yml")
|
|
||||||
env = {"HBW_DRY_RUN": "true"}
|
|
||||||
result = run_baker("prebake", config, cwd=work_dir, env=env, timeout=60)
|
|
||||||
assert result["success"], f"Dry-run failed: {result['stderr']}"
|
|
||||||
|
|
||||||
def test_dry_run_logs_script_content(self, run_baker, work_dir):
|
|
||||||
config = os.path.join(FIXTURES, "bootstrap_only.yml")
|
|
||||||
env = {"HBW_DRY_RUN": "true", "RUST_LOG": "info"}
|
|
||||||
result = run_baker("prebake", config, cwd=work_dir, env=env, timeout=60)
|
|
||||||
assert result["success"]
|
|
||||||
combined = result["stdout"] + result["stderr"]
|
|
||||||
assert "DRY_RUN" in combined or "dry_run" in combined.lower()
|
|
||||||
|
|
||||||
|
|
||||||
@pytest.mark.prebake
|
|
||||||
class TestPrebakeHooks:
|
|
||||||
|
|
||||||
#@pytest.mark.xfail(reason="Hook execution fails with Permission denied in container environment", strict=False)
|
|
||||||
def test_early_hook_executes(self, run_baker, work_dir):
|
|
||||||
config = os.path.join(FIXTURES, "with_hooks.yml")
|
|
||||||
result = run_baker("prebake", config, cwd=work_dir, timeout=60)
|
|
||||||
assert result["success"], f"prebake with hooks failed: {result['stderr']}"
|
|
||||||
combined = result["stdout"] + result["stderr"]
|
|
||||||
assert "early hook executed" in combined
|
|
||||||
|
|
||||||
#@pytest.mark.xfail(reason="Hook execution fails with Permission denied in container environment", strict=False)
|
|
||||||
def test_late_hook_executes(self, run_baker, work_dir):
|
|
||||||
config = os.path.join(FIXTURES, "with_hooks.yml")
|
|
||||||
result = run_baker("prebake", config, cwd=work_dir, timeout=60)
|
|
||||||
assert result["success"], f"prebake with hooks failed: {result['stderr']}"
|
|
||||||
combined = result["stdout"] + result["stderr"]
|
|
||||||
assert "late hook executed" in combined
|
|
||||||
|
|
||||||
#@pytest.mark.xfail(reason="Hook execution fails with Permission denied in container environment", strict=False)
|
|
||||||
def test_hooks_run_in_order(self, run_baker, work_dir):
|
|
||||||
config = os.path.join(FIXTURES, "with_hooks.yml")
|
|
||||||
result = run_baker("prebake", config, cwd=work_dir, timeout=60)
|
|
||||||
assert result["success"]
|
|
||||||
combined = result["stdout"] + result["stderr"]
|
|
||||||
early_pos = combined.find("early hook executed")
|
|
||||||
late_pos = combined.find("late hook executed")
|
|
||||||
assert early_pos != -1 and late_pos != -1
|
|
||||||
assert early_pos < late_pos, "Early hook should execute before late hook"
|
|
||||||
|
|
||||||
def test_empty_hooks_succeed(self, run_baker, work_dir):
|
|
||||||
config = os.path.join(FIXTURES, "minimal.yml")
|
|
||||||
result = run_baker("prebake", config, cwd=work_dir, timeout=60)
|
|
||||||
assert result["success"], f"prebake with no hooks failed: {result['stderr']}"
|
|
||||||
|
|
||||||
|
|
||||||
@pytest.mark.prebake
|
|
||||||
class TestPrebakeDepsSystem:
|
|
||||||
|
|
||||||
@pytest.mark.xfail(reason="Package installation may fail in test environment", strict=False)
|
|
||||||
def test_pacman_packages_installed(self, run_baker, work_dir):
|
|
||||||
config = os.path.join(FIXTURES, "with_pacman.yml")
|
|
||||||
result = run_baker("prebake", config, cwd=work_dir, timeout=120)
|
|
||||||
assert result["success"], f"prebake failed: {result['stderr']}"
|
|
||||||
assert package.package_installed("curl")
|
|
||||||
assert package.package_installed("wget")
|
|
||||||
|
|
||||||
def test_pacman_mirror_change(self, run_baker, work_dir):
|
|
||||||
mirror_config = os.path.join(work_dir, "mirror.yml")
|
|
||||||
with open(mirror_config, "w") as f:
|
|
||||||
f.write('''
|
|
||||||
version: "1.0"
|
|
||||||
environment:
|
|
||||||
builder: "baremetal"
|
|
||||||
bootstrap:
|
|
||||||
user: "vulcan"
|
|
||||||
dependencies:
|
|
||||||
system:
|
|
||||||
pacman:
|
|
||||||
packages: ["which"]
|
|
||||||
mirror: "https://mirrors.tuna.tsinghua.edu.cn/archlinux/$repo/os/$arch"
|
|
||||||
''')
|
|
||||||
result = run_baker("prebake", mirror_config, cwd=work_dir, timeout=120)
|
|
||||||
assert result["success"], f"prebake with mirror failed: {result['stderr']}"
|
|
||||||
|
|
||||||
|
|
||||||
@pytest.mark.prebake
|
|
||||||
class TestPrebakeEnvVars:
|
|
||||||
|
|
||||||
def test_prebake_envvars_injected(self, run_baker, work_dir):
|
|
||||||
hook_config = os.path.join(work_dir, "envvar_hook.yml")
|
|
||||||
with open(hook_config, "w") as f:
|
|
||||||
f.write('''
|
|
||||||
version: "1.0"
|
|
||||||
environment:
|
|
||||||
builder: "baremetal"
|
|
||||||
bootstrap:
|
|
||||||
user: "vulcan"
|
|
||||||
envvars:
|
|
||||||
prebake:
|
|
||||||
TEST_INJECTED: "injected_value"
|
|
||||||
hooks:
|
|
||||||
early:
|
|
||||||
- name: "check-env"
|
|
||||||
command: "echo $TEST_INJECTED"
|
|
||||||
''')
|
|
||||||
result = run_baker("prebake", hook_config, cwd=work_dir, timeout=60)
|
|
||||||
assert result["success"], f"prebake with envvars failed: {result['stderr']}"
|
|
||||||
combined = result["stdout"] + result["stderr"]
|
|
||||||
assert "injected_value" in combined
|
|
||||||
|
|
||||||
|
|
||||||
@pytest.mark.prebake
|
|
||||||
class TestPrebakeSecurity:
|
|
||||||
|
|
||||||
#@pytest.mark.xfail(reason="Privilege drop may fail in container without proper setup", strict=False)
|
|
||||||
def test_privilege_drop_after_bootstrap(self, run_baker, work_dir):
|
|
||||||
config = os.path.join(FIXTURES, "with_security.yml")
|
|
||||||
result = run_baker("prebake", config, cwd=work_dir, timeout=60)
|
|
||||||
assert result["success"], f"prebake with security failed: {result['stderr']}"
|
|
||||||
|
|
||||||
|
|
||||||
@pytest.mark.prebake
|
|
||||||
class TestPrebakeStatus:
|
|
||||||
|
|
||||||
def test_status_files_written(self, run_baker, work_dir):
|
|
||||||
config = os.path.join(FIXTURES, "bootstrap_only.yml")
|
|
||||||
result = run_baker("prebake", config, cwd=work_dir, timeout=60)
|
|
||||||
assert result["success"]
|
|
||||||
assert os.path.exists("/tmp/.hbwstatus")
|
|
||||||
|
|
||||||
def test_status_contains_bootstrap_stage(self, run_baker, work_dir):
|
|
||||||
config = os.path.join(FIXTURES, "bootstrap_only.yml")
|
|
||||||
result = run_baker("prebake", config, cwd=work_dir, timeout=60)
|
|
||||||
assert result["success"]
|
|
||||||
with open("/tmp/.hbwstatus", "r") as f:
|
|
||||||
lines = f.readlines()
|
|
||||||
assert len(lines) > 0
|
|
||||||
bootstrap_lines = [l for l in lines if json.loads(l).get("name") == "bootstrap"]
|
|
||||||
assert len(bootstrap_lines) > 0, "No bootstrap stage found in status file"
|
|
||||||
data = json.loads(bootstrap_lines[-1])
|
|
||||||
assert data.get("phase") == "Prebake"
|
|
||||||
assert data.get("result") == "Success"
|
|
||||||
|
|
||||||
def test_status_contains_duration(self, run_baker, work_dir):
|
|
||||||
config = os.path.join(FIXTURES, "bootstrap_only.yml")
|
|
||||||
result = run_baker("prebake", config, cwd=work_dir, timeout=60)
|
|
||||||
assert result["success"]
|
|
||||||
with open("/tmp/.hbwstatus", "r") as f:
|
|
||||||
data = json.loads(f.readline())
|
|
||||||
assert "duration_ms" in data
|
|
||||||
assert isinstance(data["duration_ms"], int)
|
|
||||||
|
|
||||||
def test_status_contains_timestamps(self, run_baker, work_dir):
|
|
||||||
config = os.path.join(FIXTURES, "bootstrap_only.yml")
|
|
||||||
result = run_baker("prebake", config, cwd=work_dir, timeout=60)
|
|
||||||
assert result["success"]
|
|
||||||
with open("/tmp/.hbwstatus", "r") as f:
|
|
||||||
data = json.loads(f.readline())
|
|
||||||
assert "started_at" in data
|
|
||||||
assert "finished_at" in data
|
|
||||||
|
|
||||||
def test_status_written_for_each_stage(self, run_baker, work_dir):
|
|
||||||
config = os.path.join(FIXTURES, "with_pacman.yml")
|
|
||||||
result = run_baker("prebake", config, cwd=work_dir, timeout=120)
|
|
||||||
assert result["success"], f"prebake failed: {result['stderr']}"
|
|
||||||
with open("/tmp/.hbwstatus", "r") as f:
|
|
||||||
lines = f.readlines()
|
|
||||||
names = [json.loads(line).get("name") for line in lines]
|
|
||||||
assert "bootstrap" in names
|
|
||||||
assert "depssystem" in names
|
|
||||||
@@ -1,164 +0,0 @@
|
|||||||
"""
|
|
||||||
Root conftest for HoneyBiscuitWorkshop system tests.
|
|
||||||
|
|
||||||
Provides core fixtures for:
|
|
||||||
- Locating the baker binary and examples
|
|
||||||
- Running subprocess commands with output capture
|
|
||||||
- Managing temporary working directories
|
|
||||||
"""
|
|
||||||
import os
|
|
||||||
import subprocess
|
|
||||||
import shutil
|
|
||||||
import tempfile
|
|
||||||
import pytest
|
|
||||||
|
|
||||||
|
|
||||||
def pytest_configure(config):
|
|
||||||
"""Register custom markers."""
|
|
||||||
config.addinivalue_line("markers", "prebake: prebake stage tests")
|
|
||||||
config.addinivalue_line("markers", "bake: bake stage tests")
|
|
||||||
config.addinivalue_line("markers", "finalize: finalize stage tests")
|
|
||||||
config.addinivalue_line("markers", "llm: tests requiring LLM API")
|
|
||||||
config.addinivalue_line("markers", "slow: slow running tests")
|
|
||||||
|
|
||||||
|
|
||||||
@pytest.fixture(scope="session")
|
|
||||||
def baker_bin():
|
|
||||||
"""Path to the workshop-baker binary."""
|
|
||||||
path = os.environ.get("BAKER_BIN", "/workshop/workshop-baker")
|
|
||||||
if not os.path.isfile(path):
|
|
||||||
pytest.skip(f"Baker binary not found at {path}")
|
|
||||||
if not os.access(path, os.X_OK):
|
|
||||||
pytest.skip(f"Baker binary not executable at {path}")
|
|
||||||
return path
|
|
||||||
|
|
||||||
|
|
||||||
@pytest.fixture(scope="session")
|
|
||||||
def examples_dir():
|
|
||||||
"""Path to the examples directory (may be None)."""
|
|
||||||
path = os.environ.get("EXAMPLES_DIR", "/workshop/examples")
|
|
||||||
if not os.path.isdir(path):
|
|
||||||
return None
|
|
||||||
return path
|
|
||||||
|
|
||||||
|
|
||||||
@pytest.fixture(scope="session")
|
|
||||||
def llm_env():
|
|
||||||
"""LLM environment variables dict (from llm.env)."""
|
|
||||||
env = {}
|
|
||||||
for var in [
|
|
||||||
"OPENAI_BASE_URL",
|
|
||||||
"OPENAI_API_KEY",
|
|
||||||
"ANTHROPIC_BASE_URL",
|
|
||||||
"ANTHROPIC_API_KEY",
|
|
||||||
"MODEL",
|
|
||||||
"REASONING_EFFORT",
|
|
||||||
]:
|
|
||||||
val = os.environ.get(var)
|
|
||||||
if val:
|
|
||||||
env[var] = val
|
|
||||||
return env
|
|
||||||
|
|
||||||
|
|
||||||
@pytest.fixture
|
|
||||||
def work_dir():
|
|
||||||
"""Create a temporary working directory for a test, cleaned up after."""
|
|
||||||
d = tempfile.mkdtemp(prefix="hbw-test-")
|
|
||||||
yield d
|
|
||||||
shutil.rmtree(d, ignore_errors=True)
|
|
||||||
|
|
||||||
|
|
||||||
@pytest.fixture
|
|
||||||
def run_cmd():
|
|
||||||
"""Fixture that provides a command runner function.
|
|
||||||
|
|
||||||
Returns a function that runs a command and returns
|
|
||||||
(stdout, stderr, returncode, success).
|
|
||||||
"""
|
|
||||||
|
|
||||||
def _run(args, cwd=None, env=None, timeout=120):
|
|
||||||
"""Run a command and capture output.
|
|
||||||
|
|
||||||
Args:
|
|
||||||
args: Command as list of strings.
|
|
||||||
cwd: Working directory.
|
|
||||||
env: Environment variables (merged with os.environ).
|
|
||||||
timeout: Timeout in seconds.
|
|
||||||
|
|
||||||
Returns:
|
|
||||||
dict with keys: stdout, stderr, returncode, success
|
|
||||||
"""
|
|
||||||
full_env = {**os.environ}
|
|
||||||
if env:
|
|
||||||
full_env.update(env)
|
|
||||||
|
|
||||||
try:
|
|
||||||
result = subprocess.run(
|
|
||||||
args,
|
|
||||||
cwd=cwd,
|
|
||||||
env=full_env,
|
|
||||||
capture_output=True,
|
|
||||||
text=True,
|
|
||||||
timeout=timeout,
|
|
||||||
)
|
|
||||||
return {
|
|
||||||
"stdout": result.stdout,
|
|
||||||
"stderr": result.stderr,
|
|
||||||
"returncode": result.returncode,
|
|
||||||
"success": result.returncode == 0,
|
|
||||||
}
|
|
||||||
except subprocess.TimeoutExpired:
|
|
||||||
return {
|
|
||||||
"stdout": "",
|
|
||||||
"stderr": f"Command timed out after {timeout}s",
|
|
||||||
"returncode": -1,
|
|
||||||
"success": False,
|
|
||||||
}
|
|
||||||
except Exception as e:
|
|
||||||
return {
|
|
||||||
"stdout": "",
|
|
||||||
"stderr": str(e),
|
|
||||||
"returncode": -1,
|
|
||||||
"success": False,
|
|
||||||
}
|
|
||||||
|
|
||||||
return _run
|
|
||||||
|
|
||||||
|
|
||||||
@pytest.fixture
|
|
||||||
def run_baker(baker_bin, run_cmd):
|
|
||||||
"""Fixture that provides a baker command runner.
|
|
||||||
|
|
||||||
Returns a function that runs workshop-baker with common flags.
|
|
||||||
"""
|
|
||||||
|
|
||||||
def _run(subcommand, *args, cwd=None, env=None, timeout=120):
|
|
||||||
cmd = [
|
|
||||||
baker_bin,
|
|
||||||
"-u", "testuser",
|
|
||||||
"-p", "test-pipeline",
|
|
||||||
"-b", "test-build-001",
|
|
||||||
"bare",
|
|
||||||
subcommand,
|
|
||||||
]
|
|
||||||
cmd.extend(args)
|
|
||||||
return run_cmd(cmd, cwd=cwd, env=env, timeout=timeout)
|
|
||||||
|
|
||||||
return _run
|
|
||||||
|
|
||||||
|
|
||||||
@pytest.fixture
|
|
||||||
def fixtures_dir():
|
|
||||||
"""Path to the test fixtures directory."""
|
|
||||||
return os.path.join(os.path.dirname(__file__), "fixtures")
|
|
||||||
|
|
||||||
|
|
||||||
@pytest.fixture
|
|
||||||
def bake_workspace():
|
|
||||||
"""Create /workspace for bake tests (baker writes temp scripts there)."""
|
|
||||||
existed = os.path.exists("/workspace")
|
|
||||||
if not existed:
|
|
||||||
os.makedirs("/workspace", exist_ok=True)
|
|
||||||
yield
|
|
||||||
if not existed:
|
|
||||||
shutil.rmtree("/workspace", ignore_errors=True)
|
|
||||||
@@ -1,102 +0,0 @@
|
|||||||
"""PM system tests driven by bare workshop-baker prebake."""
|
|
||||||
|
|
||||||
import os
|
|
||||||
import pytest
|
|
||||||
from probes import binary_exists
|
|
||||||
|
|
||||||
PACMAN_MIRROR = "https://mirrors.tuna.tsinghua.edu.cn/archlinux/$repo/os/$arch"
|
|
||||||
|
|
||||||
|
|
||||||
def _yml(pm, pkgs, mirror="", repo=None):
|
|
||||||
"""Build prebake.yml. repo is a list of lines (without lead indent)."""
|
|
||||||
m = ""
|
|
||||||
if mirror:
|
|
||||||
m += ' mirror: "' + mirror + '"\n'
|
|
||||||
if repo:
|
|
||||||
m += " repositories:\n"
|
|
||||||
for line in repo:
|
|
||||||
m += " " + line + "\n"
|
|
||||||
return """\
|
|
||||||
version: "1.0"
|
|
||||||
environment:
|
|
||||||
builder: "baremetal"
|
|
||||||
bootstrap:
|
|
||||||
user: "vulcan"
|
|
||||||
dependencies:
|
|
||||||
system:
|
|
||||||
{pm}:
|
|
||||||
packages: [{pkgs}]
|
|
||||||
{m}""".format(pm=pm, pkgs=pkgs, m=m)
|
|
||||||
|
|
||||||
|
|
||||||
# ── install ──────────────────────────────────────────────────────────
|
|
||||||
|
|
||||||
@pytest.mark.parametrize("pm_name,pkgs", [
|
|
||||||
("pacman", '"curl", "git"'),
|
|
||||||
("apt", '"curl", "git"'),
|
|
||||||
("dnf", '"curl", "git"'),
|
|
||||||
("apk", '"curl", "git"'),
|
|
||||||
])
|
|
||||||
def test_install(run_cmd, work_dir, pm_name, pkgs):
|
|
||||||
cfg = os.path.join(work_dir, "pm.yml")
|
|
||||||
mirror = PACMAN_MIRROR if pm_name == "pacman" else ""
|
|
||||||
with open(cfg, "w") as f:
|
|
||||||
f.write(_yml(pm_name, pkgs, mirror=mirror))
|
|
||||||
r = run_cmd(["/baker", "-p", "t", "-b", "1", "-u", "testuser",
|
|
||||||
"bare", "prebake", cfg], timeout=120)
|
|
||||||
assert r["success"], "[{}] prebake: {}".format(pm_name, r["stderr"][:300])
|
|
||||||
for pkg in ["curl", "git"]:
|
|
||||||
assert binary_exists("/usr/bin/" + pkg), "[{}] {} missing".format(pm_name, pkg)
|
|
||||||
|
|
||||||
|
|
||||||
@pytest.mark.parametrize("pm_name", ["pacman", "apt", "dnf", "apk"])
|
|
||||||
def test_reinstall(run_cmd, work_dir, pm_name):
|
|
||||||
cfg = os.path.join(work_dir, "pm.yml")
|
|
||||||
mirror = PACMAN_MIRROR if pm_name == "pacman" else ""
|
|
||||||
with open(cfg, "w") as f:
|
|
||||||
f.write(_yml(pm_name, '"curl"', mirror=mirror))
|
|
||||||
cmd = ["/baker", "-p", "t", "-b", "1", "-u", "testuser", "bare", "prebake", cfg]
|
|
||||||
r1 = run_cmd(cmd, timeout=120)
|
|
||||||
assert r1["success"], "[{}] first install failed".format(pm_name)
|
|
||||||
r2 = run_cmd(cmd, timeout=120)
|
|
||||||
assert r2["success"], "[{}] reinstall failed".format(pm_name)
|
|
||||||
|
|
||||||
|
|
||||||
# ── change_mirror ────────────────────────────────────────────────────
|
|
||||||
|
|
||||||
@pytest.mark.parametrize("pm_name", ["pacman", "apt", "dnf", "apk"])
|
|
||||||
def test_mirror(run_cmd, work_dir, pm_name):
|
|
||||||
cfg = os.path.join(work_dir, "pm.yml")
|
|
||||||
mirror = PACMAN_MIRROR if pm_name == "pacman" else ""
|
|
||||||
with open(cfg, "w") as f:
|
|
||||||
f.write(_yml(pm_name, '"which"', mirror=mirror))
|
|
||||||
r = run_cmd(["/baker", "-p", "t", "-b", "1", "-u", "testuser",
|
|
||||||
"bare", "prebake", cfg], timeout=120)
|
|
||||||
assert r["success"], "[{}] mirror: {}".format(pm_name, r["stderr"][:300])
|
|
||||||
assert binary_exists("/usr/bin/which"), "[{}] which not installed".format(pm_name)
|
|
||||||
|
|
||||||
|
|
||||||
# ── add_repository ───────────────────────────────────────────────────
|
|
||||||
|
|
||||||
@pytest.mark.parametrize("pm_name,repo_lines,result_file", [
|
|
||||||
("pacman", ['- name: custom', ' server: "https://repo.example.com/$arch"'],
|
|
||||||
"/etc/pacman.conf"),
|
|
||||||
("apt", ['- url: "https://repo.example.com/debian"', ' distro: bookworm'],
|
|
||||||
"/etc/apt/sources.list.d/custom.list"),
|
|
||||||
("dnf", ['- name: custom', ' baseurl: "https://repo.example.com/fedora"'],
|
|
||||||
"/etc/yum.repos.d/custom.repo"),
|
|
||||||
("apk", ['- url: "https://repo.example.com/v3.19/main"'],
|
|
||||||
"/etc/apk/repositories"),
|
|
||||||
])
|
|
||||||
@pytest.mark.xfail(reason="fake repo URL, package install may fail", strict=False)
|
|
||||||
def test_add_repo(run_cmd, work_dir, pm_name, repo_lines, result_file):
|
|
||||||
cfg = os.path.join(work_dir, "pm.yml")
|
|
||||||
mirror = PACMAN_MIRROR if pm_name == "pacman" else ""
|
|
||||||
with open(cfg, "w") as f:
|
|
||||||
f.write(_yml(pm_name, '"curl"', mirror=mirror, repo=repo_lines))
|
|
||||||
r = run_cmd(["/baker", "-p", "t", "-b", "1", "-u", "testuser",
|
|
||||||
"bare", "prebake", cfg], timeout=120)
|
|
||||||
# Prebake may fail if repo URL is unreachable — check only the config file
|
|
||||||
# was written by add_repository before the (possibly failing) install step.
|
|
||||||
assert binary_exists(result_file), \
|
|
||||||
"[{}] repo file missing: {}".format(pm_name, result_file)
|
|
||||||
@@ -1,36 +0,0 @@
|
|||||||
#!/bin/bash
|
|
||||||
set -e
|
|
||||||
|
|
||||||
# Load LLM environment if available
|
|
||||||
if [ -f /workshop/llm.env ]; then
|
|
||||||
set -a
|
|
||||||
source /workshop/llm.env
|
|
||||||
set +a
|
|
||||||
fi
|
|
||||||
|
|
||||||
# Export test configuration
|
|
||||||
export BAKER_BIN=/workshop/workshop-baker
|
|
||||||
export EXAMPLES_DIR=/workshop/examples
|
|
||||||
export TEST_MODE=${TEST_MODE:-auto}
|
|
||||||
|
|
||||||
if [ "$TEST_MODE" = "manual" ]; then
|
|
||||||
echo "========================================"
|
|
||||||
echo " HoneyBiscuitWorkshop System Tests"
|
|
||||||
echo " Mode: MANUAL (interactive shell)"
|
|
||||||
echo "========================================"
|
|
||||||
echo ""
|
|
||||||
echo "Binary: $BAKER_BIN"
|
|
||||||
echo "Examples: $EXAMPLES_DIR"
|
|
||||||
echo ""
|
|
||||||
echo "Run tests manually:"
|
|
||||||
echo " cd /app && pytest tests/ -v"
|
|
||||||
echo ""
|
|
||||||
exec /bin/bash
|
|
||||||
else
|
|
||||||
echo "========================================"
|
|
||||||
echo " HoneyBiscuitWorkshop System Tests"
|
|
||||||
echo " Mode: AUTO (running pytest)"
|
|
||||||
echo "========================================"
|
|
||||||
cd /app
|
|
||||||
exec python -m pytest tests/ -v --tb=short --junitxml=/results/report.xml "$@"
|
|
||||||
fi
|
|
||||||
Vendored
-6
@@ -1,6 +0,0 @@
|
|||||||
#!/bin/bash
|
|
||||||
[ -f /dev/shm/bakeexport.{{ name }} ] && . /dev/shm/bakeexport.{{ name }} || true
|
|
||||||
{{ condition }}
|
|
||||||
{{ main }}
|
|
||||||
{{ name }}
|
|
||||||
{{ export }}
|
|
||||||
Vendored
-25
@@ -1,25 +0,0 @@
|
|||||||
#!/bin/bash
|
|
||||||
|
|
||||||
# @pipeline
|
|
||||||
step_one() {
|
|
||||||
echo "Step 1: Setup"
|
|
||||||
mkdir -p /tmp/bake-test-output
|
|
||||||
echo "step1-done" > /tmp/bake-test-output/step1.txt
|
|
||||||
}
|
|
||||||
|
|
||||||
# @pipeline
|
|
||||||
step_two() {
|
|
||||||
echo "Step 2: Build"
|
|
||||||
echo "step2-done" > /tmp/bake-test-output/step2.txt
|
|
||||||
}
|
|
||||||
|
|
||||||
# @pipeline
|
|
||||||
step_three() {
|
|
||||||
echo "Step 3: Verify"
|
|
||||||
if [ -f /tmp/bake-test-output/step1.txt ] && [ -f /tmp/bake-test-output/step2.txt ]; then
|
|
||||||
echo "All steps completed successfully"
|
|
||||||
else
|
|
||||||
echo "ERROR: Missing step outputs"
|
|
||||||
exit 1
|
|
||||||
fi
|
|
||||||
}
|
|
||||||
Vendored
-5
@@ -1,5 +0,0 @@
|
|||||||
#!/bin/bash
|
|
||||||
echo "Hello from simple bake script"
|
|
||||||
echo "Task ID: $HBW_TASKID"
|
|
||||||
echo "Pipeline: $HBW_PIPELINE"
|
|
||||||
exit 0
|
|
||||||
Vendored
-22
@@ -1,22 +0,0 @@
|
|||||||
#!/bin/bash
|
|
||||||
|
|
||||||
# @pipeline
|
|
||||||
init_workspace() {
|
|
||||||
echo "Initializing workspace"
|
|
||||||
mkdir -p /tmp/bake-after-test
|
|
||||||
echo "init" > /tmp/bake-after-test/status.txt
|
|
||||||
}
|
|
||||||
|
|
||||||
# @pipeline
|
|
||||||
# @after(init_workspace)
|
|
||||||
fetch_source() {
|
|
||||||
echo "Fetching source (depends on init_workspace)"
|
|
||||||
echo "fetched" > /tmp/bake-after-test/source.txt
|
|
||||||
}
|
|
||||||
|
|
||||||
# @pipeline
|
|
||||||
# @after(fetch_source)
|
|
||||||
build_project() {
|
|
||||||
echo "Building project (depends on fetch_source)"
|
|
||||||
echo "built" > /tmp/bake-after-test/build.txt
|
|
||||||
}
|
|
||||||
-4
@@ -1,4 +0,0 @@
|
|||||||
#!/bin/bash
|
|
||||||
echo "EARLY_HOOK_EXECUTED"
|
|
||||||
echo "Working directory: $(pwd)"
|
|
||||||
echo "User: $(whoami)"
|
|
||||||
Vendored
-3
@@ -1,3 +0,0 @@
|
|||||||
#!/bin/bash
|
|
||||||
echo "LATE_HOOK_EXECUTED"
|
|
||||||
echo "Working directory: $(pwd)"
|
|
||||||
Vendored
-6
@@ -1,6 +0,0 @@
|
|||||||
version: "0.0.1"
|
|
||||||
plugin: {}
|
|
||||||
notification: {}
|
|
||||||
artifact: []
|
|
||||||
cleanup:
|
|
||||||
policy: "auto"
|
|
||||||
-13
@@ -1,13 +0,0 @@
|
|||||||
version: "0.0.1"
|
|
||||||
plugin: {}
|
|
||||||
notification: {}
|
|
||||||
artifact: []
|
|
||||||
cleanup:
|
|
||||||
policy: "auto"
|
|
||||||
hooks:
|
|
||||||
early:
|
|
||||||
- name: "finalize-early"
|
|
||||||
command: "echo 'finalize early hook'"
|
|
||||||
late:
|
|
||||||
- name: "finalize-late"
|
|
||||||
command: "echo 'finalize late hook'"
|
|
||||||
Vendored
-6
@@ -1,6 +0,0 @@
|
|||||||
FROM docker.1ms.run/library/alpine:3.19
|
|
||||||
RUN sed -i 's|dl-cdn.alpinelinux.org|mirrors.tuna.tsinghua.edu.cn|g' /etc/apk/repositories
|
|
||||||
RUN apk add --no-cache python3 py3-pip bash
|
|
||||||
COPY workshop-baker/target/x86_64-unknown-linux-musl/debug/workshop-baker /baker
|
|
||||||
RUN chmod +x /baker
|
|
||||||
ENTRYPOINT ["/bin/sh", "-c"]
|
|
||||||
Vendored
-6
@@ -1,6 +0,0 @@
|
|||||||
FROM docker.1ms.run/library/debian:bookworm-slim
|
|
||||||
RUN sed -i 's|deb.debian.org|mirrors.tuna.tsinghua.edu.cn|g' /etc/apt/sources.list.d/debian.sources 2>/dev/null; sed -i 's|security.debian.org|mirrors.tuna.tsinghua.edu.cn/debian-security|g' /etc/apt/sources.list.d/debian.sources 2>/dev/null; true
|
|
||||||
RUN apt-get update -qq && apt-get install -y -qq --no-install-recommends python3 python3-venv ca-certificates 2>&1 | tail -n 2
|
|
||||||
COPY workshop-baker/target/x86_64-unknown-linux-musl/debug/workshop-baker /baker
|
|
||||||
RUN chmod +x /baker
|
|
||||||
ENTRYPOINT ["/bin/sh", "-c"]
|
|
||||||
Vendored
-6
@@ -1,6 +0,0 @@
|
|||||||
FROM docker.1ms.run/library/fedora:43
|
|
||||||
RUN sed -e 's|^metalink=|#metalink=|g' -e 's|^#baseurl=http://download.example/pub/fedora/linux|baseurl=https://mirrors.tuna.tsinghua.edu.cn/fedora|g' -i.bak /etc/yum.repos.d/fedora.repo /etc/yum.repos.d/fedora-updates.repo
|
|
||||||
RUN dnf install -y python3 2>&1 | tail -n 2
|
|
||||||
COPY workshop-baker/target/x86_64-unknown-linux-musl/debug/workshop-baker /baker
|
|
||||||
RUN chmod +x /baker
|
|
||||||
ENTRYPOINT ["/bin/sh", "-c"]
|
|
||||||
Vendored
-2
@@ -1,2 +0,0 @@
|
|||||||
FROM docker.1ms.run/library/archlinux:latest
|
|
||||||
RUN echo 'Server = https://mirrors.tuna.tsinghua.edu.cn/archlinux/$repo/os/$arch' > /etc/pacman.d/mirrorlist
|
|
||||||
-8
@@ -1,8 +0,0 @@
|
|||||||
version: "1.0"
|
|
||||||
environment:
|
|
||||||
builder: "baremetal"
|
|
||||||
bootstrap:
|
|
||||||
user: "vulcan"
|
|
||||||
workspace:
|
|
||||||
path: "/home/vulcan/workspace"
|
|
||||||
fallback: true
|
|
||||||
Vendored
-15
@@ -1,15 +0,0 @@
|
|||||||
version: "1.0"
|
|
||||||
environment:
|
|
||||||
builder: "baremetal"
|
|
||||||
bootstrap:
|
|
||||||
user: "vulcan"
|
|
||||||
workspace:
|
|
||||||
path: "/home/vulcan/workspace"
|
|
||||||
fallback: true
|
|
||||||
hooks:
|
|
||||||
early:
|
|
||||||
- name: "test-early-hook"
|
|
||||||
command: "echo 'early hook executed'"
|
|
||||||
late:
|
|
||||||
- name: "test-late-hook"
|
|
||||||
command: "echo 'late hook executed'"
|
|
||||||
-3
@@ -1,3 +0,0 @@
|
|||||||
version: "0.5"
|
|
||||||
environment:
|
|
||||||
builder: "baremetal"
|
|
||||||
Vendored
-8
@@ -1,8 +0,0 @@
|
|||||||
version: "1.0"
|
|
||||||
environment:
|
|
||||||
builder: "baremetal"
|
|
||||||
bootstrap:
|
|
||||||
user: "vulcan"
|
|
||||||
workspace:
|
|
||||||
path: "/home/vulcan/workspace"
|
|
||||||
fallback: true
|
|
||||||
@@ -1,12 +0,0 @@
|
|||||||
version: "1.0"
|
|
||||||
environment:
|
|
||||||
builder: "baremetal"
|
|
||||||
bootstrap:
|
|
||||||
user: "vulcan"
|
|
||||||
workspace:
|
|
||||||
path: "/home/vulcan/workspace"
|
|
||||||
fallback: true
|
|
||||||
custom: |
|
|
||||||
#!/bin/sh
|
|
||||||
echo 'custom bootstrap executed'
|
|
||||||
mkdir -p /custom-workspace
|
|
||||||
-16
@@ -1,16 +0,0 @@
|
|||||||
version: "1.0"
|
|
||||||
environment:
|
|
||||||
builder: "baremetal"
|
|
||||||
bootstrap:
|
|
||||||
user: "vulcan"
|
|
||||||
workspace:
|
|
||||||
path: "/home/vulcan/workspace"
|
|
||||||
fallback: true
|
|
||||||
dependencies:
|
|
||||||
system:
|
|
||||||
pacman:
|
|
||||||
packages:
|
|
||||||
- "which"
|
|
||||||
user:
|
|
||||||
cargo:
|
|
||||||
- "ripgrep"
|
|
||||||
-13
@@ -1,13 +0,0 @@
|
|||||||
version: "1.0"
|
|
||||||
environment:
|
|
||||||
builder: "baremetal"
|
|
||||||
bootstrap:
|
|
||||||
user: "vulcan"
|
|
||||||
workspace:
|
|
||||||
path: "/home/vulcan/workspace"
|
|
||||||
fallback: true
|
|
||||||
envvars:
|
|
||||||
prebake:
|
|
||||||
TEST_PREBAKE_VAR: "prebake_value"
|
|
||||||
bake:
|
|
||||||
TEST_BAKE_VAR: "bake_value"
|
|
||||||
-17
@@ -1,17 +0,0 @@
|
|||||||
version: "1.0"
|
|
||||||
environment:
|
|
||||||
builder: "baremetal"
|
|
||||||
bootstrap:
|
|
||||||
user: "vulcan"
|
|
||||||
workspace:
|
|
||||||
path: "/home/vulcan/workspace"
|
|
||||||
fallback: true
|
|
||||||
hooks:
|
|
||||||
early:
|
|
||||||
- name: "test-early-hook"
|
|
||||||
command: "echo 'early hook executed'"
|
|
||||||
working_dir: "/workspace"
|
|
||||||
late:
|
|
||||||
- name: "test-late-hook"
|
|
||||||
command: "echo 'late hook executed'"
|
|
||||||
working_dir: "/workspace"
|
|
||||||
-14
@@ -1,14 +0,0 @@
|
|||||||
version: "1.0"
|
|
||||||
environment:
|
|
||||||
builder: "baremetal"
|
|
||||||
bootstrap:
|
|
||||||
user: "vulcan"
|
|
||||||
workspace:
|
|
||||||
path: "/home/vulcan/workspace"
|
|
||||||
fallback: true
|
|
||||||
dependencies:
|
|
||||||
system:
|
|
||||||
pacman:
|
|
||||||
packages:
|
|
||||||
- "curl"
|
|
||||||
- "wget"
|
|
||||||
-10
@@ -1,10 +0,0 @@
|
|||||||
version: "1.0"
|
|
||||||
environment:
|
|
||||||
builder: "baremetal"
|
|
||||||
bootstrap:
|
|
||||||
user: "vulcan"
|
|
||||||
workspace:
|
|
||||||
path: "/home/vulcan/workspace"
|
|
||||||
fallback: true
|
|
||||||
security:
|
|
||||||
drop_after: "Bootstrap"
|
|
||||||
@@ -1,12 +0,0 @@
|
|||||||
"""
|
|
||||||
LLM-based log judgment system for system tests.
|
|
||||||
|
|
||||||
Uses LLM APIs to evaluate non-deterministic test outputs (logs, stderr, etc.)
|
|
||||||
where simple string matching is insufficient.
|
|
||||||
"""
|
|
||||||
|
|
||||||
from .client import LLMClient
|
|
||||||
from .cache import LLMCache
|
|
||||||
from .judge import judge_log, JudgmentResult
|
|
||||||
|
|
||||||
__all__ = ["LLMClient", "LLMCache", "judge_log", "JudgmentResult"]
|
|
||||||
@@ -1,56 +0,0 @@
|
|||||||
"""
|
|
||||||
Disk-based cache for LLM API responses.
|
|
||||||
|
|
||||||
Uses SHA256 of (prompt + model + system_prompt) as cache key.
|
|
||||||
Cache files stored as JSON in ``.cache/`` directory.
|
|
||||||
"""
|
|
||||||
|
|
||||||
import hashlib
|
|
||||||
import json
|
|
||||||
from pathlib import Path
|
|
||||||
|
|
||||||
|
|
||||||
class LLMCache:
|
|
||||||
"""SHA256-based disk cache for LLM API responses."""
|
|
||||||
|
|
||||||
def __init__(self, cache_dir: str | None = None):
|
|
||||||
if cache_dir is None:
|
|
||||||
import os
|
|
||||||
|
|
||||||
cache_dir = os.path.join(os.path.dirname(__file__), ".cache")
|
|
||||||
self.cache_dir = Path(cache_dir)
|
|
||||||
self.cache_dir.mkdir(parents=True, exist_ok=True)
|
|
||||||
|
|
||||||
def _make_key(self, prompt: str, model: str, system_prompt: str = "") -> str:
|
|
||||||
"""Create deterministic cache key from request parameters."""
|
|
||||||
data = json.dumps(
|
|
||||||
{"prompt": prompt, "model": model, "system": system_prompt},
|
|
||||||
sort_keys=True,
|
|
||||||
)
|
|
||||||
return hashlib.sha256(data.encode()).hexdigest()
|
|
||||||
|
|
||||||
def get(self, prompt: str, model: str, system_prompt: str = "") -> dict | None:
|
|
||||||
"""Retrieve cached response if exists."""
|
|
||||||
key = self._make_key(prompt, model, system_prompt)
|
|
||||||
path = self.cache_dir / f"{key}.json"
|
|
||||||
if path.exists():
|
|
||||||
try:
|
|
||||||
with open(path) as f:
|
|
||||||
return json.load(f)
|
|
||||||
except (json.JSONDecodeError, IOError):
|
|
||||||
return None
|
|
||||||
return None
|
|
||||||
|
|
||||||
def set(
|
|
||||||
self, prompt: str, model: str, system_prompt: str, response: dict
|
|
||||||
) -> None:
|
|
||||||
"""Store response in cache."""
|
|
||||||
key = self._make_key(prompt, model, system_prompt)
|
|
||||||
path = self.cache_dir / f"{key}.json"
|
|
||||||
with open(path, "w") as f:
|
|
||||||
json.dump(response, f, ensure_ascii=False)
|
|
||||||
|
|
||||||
def clear(self) -> None:
|
|
||||||
"""Clear all cached responses."""
|
|
||||||
for f in self.cache_dir.glob("*.json"):
|
|
||||||
f.unlink()
|
|
||||||
@@ -1,187 +0,0 @@
|
|||||||
"""
|
|
||||||
LLM client supporting both Anthropic and OpenAI APIs.
|
|
||||||
|
|
||||||
Prefers Anthropic format (per project convention).
|
|
||||||
Falls back to OpenAI format if Anthropic is unavailable.
|
|
||||||
"""
|
|
||||||
|
|
||||||
import os
|
|
||||||
import logging
|
|
||||||
from .cache import LLMCache
|
|
||||||
|
|
||||||
logger = logging.getLogger(__name__)
|
|
||||||
|
|
||||||
|
|
||||||
class LLMClient:
|
|
||||||
"""LLM client with caching support. Prefers Anthropic API."""
|
|
||||||
|
|
||||||
def __init__(self, cache: LLMCache | None = None):
|
|
||||||
self.cache = cache or LLMCache()
|
|
||||||
self._anthropic_client = None
|
|
||||||
self._openai_client = None
|
|
||||||
self._init_clients()
|
|
||||||
|
|
||||||
def _init_clients(self):
|
|
||||||
"""Initialize API clients based on available environment variables."""
|
|
||||||
# Anthropic (preferred)
|
|
||||||
anthropic_key = os.environ.get("ANTHROPIC_API_KEY")
|
|
||||||
anthropic_base = os.environ.get("ANTHROPIC_BASE_URL")
|
|
||||||
if anthropic_key:
|
|
||||||
try:
|
|
||||||
import anthropic
|
|
||||||
|
|
||||||
kwargs: dict = {"api_key": anthropic_key}
|
|
||||||
if anthropic_base:
|
|
||||||
kwargs["base_url"] = anthropic_base
|
|
||||||
self._anthropic_client = anthropic.Anthropic(**kwargs)
|
|
||||||
logger.info("Anthropic client initialized")
|
|
||||||
except ImportError:
|
|
||||||
logger.warning("anthropic package not installed")
|
|
||||||
|
|
||||||
# OpenAI (fallback)
|
|
||||||
openai_key = os.environ.get("OPENAI_API_KEY")
|
|
||||||
openai_base = os.environ.get("OPENAI_BASE_URL")
|
|
||||||
if openai_key:
|
|
||||||
try:
|
|
||||||
import openai
|
|
||||||
|
|
||||||
kwargs = {"api_key": openai_key}
|
|
||||||
if openai_base:
|
|
||||||
kwargs["base_url"] = openai_base
|
|
||||||
self._openai_client = openai.OpenAI(**kwargs)
|
|
||||||
logger.info("OpenAI client initialized")
|
|
||||||
except ImportError:
|
|
||||||
logger.warning("openai package not installed")
|
|
||||||
|
|
||||||
@property
|
|
||||||
def available(self) -> bool:
|
|
||||||
"""Check if any LLM client is available."""
|
|
||||||
return (
|
|
||||||
self._anthropic_client is not None or self._openai_client is not None
|
|
||||||
)
|
|
||||||
|
|
||||||
def complete(
|
|
||||||
self, prompt: str, system_prompt: str = "", max_tokens: int = 2000
|
|
||||||
) -> str:
|
|
||||||
"""Send a completion request, with caching.
|
|
||||||
|
|
||||||
Args:
|
|
||||||
prompt: User message.
|
|
||||||
system_prompt: System message (sets role / persona).
|
|
||||||
max_tokens: Max response tokens.
|
|
||||||
|
|
||||||
Returns:
|
|
||||||
LLM response text.
|
|
||||||
|
|
||||||
Raises:
|
|
||||||
RuntimeError: If no client is available.
|
|
||||||
"""
|
|
||||||
model = os.environ.get("MODEL", "deepseek-v4-flash")
|
|
||||||
|
|
||||||
# Check cache
|
|
||||||
cached = self.cache.get(prompt, model, system_prompt)
|
|
||||||
if cached:
|
|
||||||
logger.info("Cache hit for LLM request")
|
|
||||||
return cached["content"]
|
|
||||||
|
|
||||||
# Try Anthropic first
|
|
||||||
if self._anthropic_client:
|
|
||||||
content = self._call_anthropic(
|
|
||||||
prompt, system_prompt, model, max_tokens
|
|
||||||
)
|
|
||||||
elif self._openai_client:
|
|
||||||
content = self._call_openai(prompt, system_prompt, model, max_tokens)
|
|
||||||
else:
|
|
||||||
raise RuntimeError(
|
|
||||||
"No LLM client available. "
|
|
||||||
"Set ANTHROPIC_API_KEY or OPENAI_API_KEY."
|
|
||||||
)
|
|
||||||
|
|
||||||
# Cache the response
|
|
||||||
self.cache.set(prompt, model, system_prompt, {"content": content})
|
|
||||||
return content
|
|
||||||
|
|
||||||
# ------------------------------------------------------------------
|
|
||||||
# Private helpers
|
|
||||||
# ------------------------------------------------------------------
|
|
||||||
|
|
||||||
def _call_anthropic(
|
|
||||||
self, prompt: str, system_prompt: str, model: str, max_tokens: int
|
|
||||||
) -> str:
|
|
||||||
"""Call Anthropic API."""
|
|
||||||
kwargs: dict = {
|
|
||||||
"model": model,
|
|
||||||
"max_tokens": max_tokens,
|
|
||||||
"messages": [{"role": "user", "content": prompt}],
|
|
||||||
}
|
|
||||||
if system_prompt:
|
|
||||||
kwargs["system"] = system_prompt
|
|
||||||
|
|
||||||
# Only add reasoning_effort if the model supports it
|
|
||||||
reasoning_effort = os.environ.get("REASONING_EFFORT")
|
|
||||||
if reasoning_effort:
|
|
||||||
try:
|
|
||||||
kwargs["reasoning_effort"] = reasoning_effort
|
|
||||||
response = self._anthropic_client.messages.create(**kwargs)
|
|
||||||
except Exception as e:
|
|
||||||
if (
|
|
||||||
"reasoning_effort" in str(e).lower()
|
|
||||||
or "unexpected" in str(e).lower()
|
|
||||||
):
|
|
||||||
del kwargs["reasoning_effort"]
|
|
||||||
response = self._anthropic_client.messages.create(**kwargs)
|
|
||||||
else:
|
|
||||||
raise
|
|
||||||
else:
|
|
||||||
response = self._anthropic_client.messages.create(**kwargs)
|
|
||||||
|
|
||||||
# Extract text from response content blocks
|
|
||||||
if hasattr(response, "content") and response.content:
|
|
||||||
block = response.content[0]
|
|
||||||
if hasattr(block, "text"):
|
|
||||||
return block.text
|
|
||||||
if hasattr(block, "type") and block.type == "thinking":
|
|
||||||
for b in response.content:
|
|
||||||
if hasattr(b, "text"):
|
|
||||||
return b.text
|
|
||||||
return str(block)
|
|
||||||
return str(response)
|
|
||||||
|
|
||||||
def _call_openai(
|
|
||||||
self, prompt: str, system_prompt: str, model: str, max_tokens: int
|
|
||||||
) -> str:
|
|
||||||
"""Call OpenAI-compatible API."""
|
|
||||||
messages: list[dict] = []
|
|
||||||
if system_prompt:
|
|
||||||
messages.append({"role": "system", "content": system_prompt})
|
|
||||||
messages.append({"role": "user", "content": prompt})
|
|
||||||
|
|
||||||
kwargs: dict = {
|
|
||||||
"model": model,
|
|
||||||
"messages": messages,
|
|
||||||
"max_tokens": max_tokens,
|
|
||||||
}
|
|
||||||
|
|
||||||
# Only add reasoning_effort if supported
|
|
||||||
reasoning_effort = os.environ.get("REASONING_EFFORT")
|
|
||||||
if reasoning_effort:
|
|
||||||
try:
|
|
||||||
kwargs["reasoning_effort"] = reasoning_effort
|
|
||||||
response = self._openai_client.chat.completions.create(
|
|
||||||
**kwargs
|
|
||||||
)
|
|
||||||
except Exception as e:
|
|
||||||
if (
|
|
||||||
"reasoning_effort" in str(e).lower()
|
|
||||||
or "unexpected" in str(e).lower()
|
|
||||||
):
|
|
||||||
del kwargs["reasoning_effort"]
|
|
||||||
response = self._openai_client.chat.completions.create(
|
|
||||||
**kwargs
|
|
||||||
)
|
|
||||||
else:
|
|
||||||
raise
|
|
||||||
else:
|
|
||||||
response = self._openai_client.chat.completions.create(**kwargs)
|
|
||||||
|
|
||||||
return response.choices[0].message.content
|
|
||||||
@@ -1,149 +0,0 @@
|
|||||||
"""
|
|
||||||
LLM-based log judgment for system tests.
|
|
||||||
|
|
||||||
Evaluates logs against expected behavior using LLM semantic analysis.
|
|
||||||
Returns structured pass/fail results with reasoning.
|
|
||||||
"""
|
|
||||||
|
|
||||||
import json
|
|
||||||
import re
|
|
||||||
import logging
|
|
||||||
from dataclasses import dataclass
|
|
||||||
from .client import LLMClient
|
|
||||||
|
|
||||||
logger = logging.getLogger(__name__)
|
|
||||||
|
|
||||||
JUDGE_SYSTEM_PROMPT = """\
|
|
||||||
You are a test result judge for a CI/CD system called HoneyBiscuitWorkshop.
|
|
||||||
|
|
||||||
Your job: Evaluate whether actual command output/logs match expected behavior.
|
|
||||||
|
|
||||||
RULES:
|
|
||||||
1. You MUST respond with valid JSON in this exact format:
|
|
||||||
{"verdict": "pass" | "fail", "reason": "brief explanation"}
|
|
||||||
2. "pass" = the output confirms the expected behavior occurred
|
|
||||||
3. "fail" = the output shows the expected behavior did NOT occur, or shows an error
|
|
||||||
4. Ignore irrelevant noise (timestamps, debug lines, unrelated warnings)
|
|
||||||
5. Focus ONLY on whether the specific expected behavior is confirmed or contradicted
|
|
||||||
6. Be strict: if the evidence is ambiguous, verdict is "fail"
|
|
||||||
7. Do NOT include any text outside the JSON object"""
|
|
||||||
|
|
||||||
|
|
||||||
@dataclass
|
|
||||||
class JudgmentResult:
|
|
||||||
"""Result of an LLM log judgment."""
|
|
||||||
|
|
||||||
verdict: str # "pass" or "fail"
|
|
||||||
reason: str
|
|
||||||
raw_response: str
|
|
||||||
|
|
||||||
@property
|
|
||||||
def passed(self) -> bool:
|
|
||||||
return self.verdict == "pass"
|
|
||||||
|
|
||||||
|
|
||||||
def _extract_relevant_logs(full_log: str, max_chars: int = 4000) -> str:
|
|
||||||
"""Extract relevant portions from logs, removing noise.
|
|
||||||
|
|
||||||
Keeps the last *max_chars* characters (most recent / relevant).
|
|
||||||
Removes common noise patterns.
|
|
||||||
"""
|
|
||||||
lines = full_log.strip().split("\n")
|
|
||||||
|
|
||||||
noise_patterns = [
|
|
||||||
r"^\d{4}-\d{2}-\d{2}T",
|
|
||||||
r"^DEBUG ",
|
|
||||||
r"^TRACE ",
|
|
||||||
]
|
|
||||||
filtered = [
|
|
||||||
line
|
|
||||||
for line in lines
|
|
||||||
if not any(re.match(p, line) for p in noise_patterns)
|
|
||||||
]
|
|
||||||
|
|
||||||
text = "\n".join(filtered)
|
|
||||||
if len(text) > max_chars:
|
|
||||||
text = "...\n" + text[-max_chars:]
|
|
||||||
return text
|
|
||||||
|
|
||||||
|
|
||||||
def judge_log(
|
|
||||||
operation: str,
|
|
||||||
expected_behavior: str,
|
|
||||||
actual_log: str,
|
|
||||||
client: LLMClient | None = None,
|
|
||||||
) -> JudgmentResult:
|
|
||||||
"""Judge whether actual logs match expected behavior.
|
|
||||||
|
|
||||||
Args:
|
|
||||||
operation: Description of what was attempted
|
|
||||||
(e.g. ``"prebake bootstrap stage"``).
|
|
||||||
expected_behavior: What should have happened
|
|
||||||
(e.g. ``"user vulcan was created"``).
|
|
||||||
actual_log: The actual stdout / stderr output.
|
|
||||||
client: ``LLMClient`` instance (creates default if ``None``).
|
|
||||||
|
|
||||||
Returns:
|
|
||||||
``JudgmentResult`` with verdict and reason.
|
|
||||||
"""
|
|
||||||
if client is None:
|
|
||||||
client = LLMClient()
|
|
||||||
|
|
||||||
if not client.available:
|
|
||||||
logger.warning("No LLM client available, returning fail verdict")
|
|
||||||
return JudgmentResult(
|
|
||||||
verdict="fail",
|
|
||||||
reason="No LLM client available for judgment",
|
|
||||||
raw_response="",
|
|
||||||
)
|
|
||||||
|
|
||||||
relevant_log = _extract_relevant_logs(actual_log)
|
|
||||||
|
|
||||||
prompt = f"""\
|
|
||||||
## Operation
|
|
||||||
{operation}
|
|
||||||
|
|
||||||
## Expected Behavior
|
|
||||||
{expected_behavior}
|
|
||||||
|
|
||||||
## Actual Output (relevant excerpts)
|
|
||||||
```
|
|
||||||
{relevant_log}
|
|
||||||
```
|
|
||||||
|
|
||||||
Evaluate: Does the actual output confirm the expected behavior occurred?
|
|
||||||
Respond with JSON only."""
|
|
||||||
|
|
||||||
try:
|
|
||||||
response = client.complete(
|
|
||||||
prompt=prompt,
|
|
||||||
system_prompt=JUDGE_SYSTEM_PROMPT,
|
|
||||||
max_tokens=500,
|
|
||||||
)
|
|
||||||
|
|
||||||
# Parse JSON response — handle potential markdown code blocks
|
|
||||||
text = response.strip()
|
|
||||||
if text.startswith("```"):
|
|
||||||
lines = text.split("\n")
|
|
||||||
text = "\n".join(lines[1:-1]) if len(lines) > 2 else text
|
|
||||||
|
|
||||||
result = json.loads(text)
|
|
||||||
return JudgmentResult(
|
|
||||||
verdict=result.get("verdict", "fail"),
|
|
||||||
reason=result.get("reason", "No reason provided"),
|
|
||||||
raw_response=response,
|
|
||||||
)
|
|
||||||
except json.JSONDecodeError as e:
|
|
||||||
logger.error(f"Failed to parse LLM response as JSON: {e}")
|
|
||||||
return JudgmentResult(
|
|
||||||
verdict="fail",
|
|
||||||
reason=f"LLM response was not valid JSON: {response[:200]}",
|
|
||||||
raw_response=response,
|
|
||||||
)
|
|
||||||
except Exception as e:
|
|
||||||
logger.error(f"LLM judgment failed: {e}")
|
|
||||||
return JudgmentResult(
|
|
||||||
verdict="fail",
|
|
||||||
reason=f"LLM call failed: {str(e)}",
|
|
||||||
raw_response="",
|
|
||||||
)
|
|
||||||
@@ -1,45 +0,0 @@
|
|||||||
"""
|
|
||||||
Environment probes for verifying system state in workshop-baker tests.
|
|
||||||
|
|
||||||
Each probe is a standalone function that can be called from any test.
|
|
||||||
Probes return bool or Optional[int/str]; they do not raise on failure.
|
|
||||||
|
|
||||||
A probe accepts an optional *runner* callable (default ``subprocess.run``)
|
|
||||||
that decouples it from the execution context (host or Docker).
|
|
||||||
"""
|
|
||||||
|
|
||||||
from .user import user_exists, user_uid, user_gid, user_in_group, user_shell
|
|
||||||
from .file import (
|
|
||||||
file_exists,
|
|
||||||
file_absent,
|
|
||||||
dir_exists,
|
|
||||||
dir_absent,
|
|
||||||
file_permissions,
|
|
||||||
file_contains,
|
|
||||||
file_owned_by,
|
|
||||||
file_group,
|
|
||||||
)
|
|
||||||
from .package import binary_exists, binary_on_path, package_installed, package_version
|
|
||||||
from .process import process_running, process_count
|
|
||||||
|
|
||||||
__all__ = [
|
|
||||||
"user_exists",
|
|
||||||
"user_uid",
|
|
||||||
"user_gid",
|
|
||||||
"user_in_group",
|
|
||||||
"user_shell",
|
|
||||||
"file_exists",
|
|
||||||
"file_absent",
|
|
||||||
"dir_exists",
|
|
||||||
"dir_absent",
|
|
||||||
"file_permissions",
|
|
||||||
"file_contains",
|
|
||||||
"file_owned_by",
|
|
||||||
"file_group",
|
|
||||||
"package_installed",
|
|
||||||
"package_version",
|
|
||||||
"binary_exists",
|
|
||||||
"binary_on_path",
|
|
||||||
"process_running",
|
|
||||||
"process_count",
|
|
||||||
]
|
|
||||||
@@ -1,53 +0,0 @@
|
|||||||
"""File and directory existence / permission probes."""
|
|
||||||
|
|
||||||
import os
|
|
||||||
import stat
|
|
||||||
import grp
|
|
||||||
import pwd
|
|
||||||
|
|
||||||
|
|
||||||
def file_exists(path: str) -> bool:
|
|
||||||
"""Check if *path* exists and is a regular file."""
|
|
||||||
return os.path.isfile(path)
|
|
||||||
|
|
||||||
|
|
||||||
def dir_exists(path: str) -> bool:
|
|
||||||
"""Check if *path* exists and is a directory."""
|
|
||||||
return os.path.isdir(path)
|
|
||||||
|
|
||||||
|
|
||||||
def file_permissions(path: str) -> int:
|
|
||||||
"""Return the permission mode of *path* as an octal int (e.g. ``0o755``).
|
|
||||||
|
|
||||||
Raises ``FileNotFoundError`` if *path* doesn't exist.
|
|
||||||
"""
|
|
||||||
return stat.S_IMODE(os.stat(path).st_mode)
|
|
||||||
|
|
||||||
|
|
||||||
def file_contains(path: str, text: str) -> bool:
|
|
||||||
"""Check if a file contains the specified text."""
|
|
||||||
with open(path, "r", errors="replace") as f:
|
|
||||||
return text in f.read()
|
|
||||||
|
|
||||||
|
|
||||||
def file_owned_by(path: str, username: str) -> bool:
|
|
||||||
"""Check if a file is owned by the specified user."""
|
|
||||||
st = os.stat(path)
|
|
||||||
owner = pwd.getpwuid(st.st_uid).pw_name
|
|
||||||
return owner == username
|
|
||||||
|
|
||||||
|
|
||||||
def file_group(path: str) -> str:
|
|
||||||
"""Get the group owner of a file."""
|
|
||||||
st = os.stat(path)
|
|
||||||
return grp.getgrgid(st.st_gid).gr_name
|
|
||||||
|
|
||||||
|
|
||||||
def file_absent(path: str) -> bool:
|
|
||||||
"""Check that *path* does not exist."""
|
|
||||||
return not os.path.lexists(path)
|
|
||||||
|
|
||||||
|
|
||||||
def dir_absent(path: str) -> bool:
|
|
||||||
"""Check that *path* does not exist or is not a directory."""
|
|
||||||
return not os.path.isdir(path)
|
|
||||||
@@ -1,42 +0,0 @@
|
|||||||
"""Package / binary existence probes (cross-distro).
|
|
||||||
|
|
||||||
Each probe accepts an optional *runner* callable with the same signature
|
|
||||||
as ``subprocess.run``. Defaults to running on the host; for Docker tests,
|
|
||||||
pass a runner that wraps ``docker run --rm <image>``.
|
|
||||||
"""
|
|
||||||
|
|
||||||
import subprocess
|
|
||||||
from typing import Any, Callable
|
|
||||||
|
|
||||||
Runner = Callable[..., Any]
|
|
||||||
|
|
||||||
|
|
||||||
def _run(cmd: list[str], runner: Runner = subprocess.run, **kw: Any) -> Any:
|
|
||||||
opts = {"capture_output": True, "text": True, "timeout": 30}
|
|
||||||
opts.update(kw)
|
|
||||||
return runner(cmd, **opts)
|
|
||||||
|
|
||||||
|
|
||||||
def _succeeded(r: Any) -> bool:
|
|
||||||
""".returncode==0 for CompletedProcess, or ["success"] for dict."""
|
|
||||||
if hasattr(r, "returncode"):
|
|
||||||
return r.returncode == 0
|
|
||||||
return bool(r.get("success", False))
|
|
||||||
|
|
||||||
|
|
||||||
def binary_exists(path: str, runner: Runner = subprocess.run) -> bool:
|
|
||||||
"""Check if a binary exists at *path* (e.g. ``/usr/bin/curl``).
|
|
||||||
|
|
||||||
Uses ``test -f`` (POSIX, guaranteed available).
|
|
||||||
"""
|
|
||||||
return _succeeded(_run(["test", "-f", path], runner))
|
|
||||||
|
|
||||||
|
|
||||||
def binary_on_path(name: str, runner: Runner = subprocess.run) -> bool:
|
|
||||||
"""Check if a command is on PATH (via ``command -v``)."""
|
|
||||||
return _succeeded(_run(["sh", "-c", "command -v " + name], runner))
|
|
||||||
|
|
||||||
|
|
||||||
# Backward-compatible aliases
|
|
||||||
package_installed = binary_exists
|
|
||||||
package_version = binary_on_path # used in __init__ export, now binary-based
|
|
||||||
@@ -1,44 +0,0 @@
|
|||||||
"""Process existence probes."""
|
|
||||||
|
|
||||||
import os
|
|
||||||
import subprocess
|
|
||||||
|
|
||||||
|
|
||||||
def process_running(process_name: str) -> bool:
|
|
||||||
"""Check if a process with the given name is running."""
|
|
||||||
try:
|
|
||||||
result = subprocess.run(
|
|
||||||
["pgrep", "-x", process_name],
|
|
||||||
capture_output=True,
|
|
||||||
text=True,
|
|
||||||
)
|
|
||||||
return result.returncode == 0
|
|
||||||
except FileNotFoundError:
|
|
||||||
# Fall back to /proc scan if pgrep not available
|
|
||||||
try:
|
|
||||||
for pid_dir in os.listdir("/proc"):
|
|
||||||
if pid_dir.isdigit():
|
|
||||||
try:
|
|
||||||
with open(f"/proc/{pid_dir}/comm") as f:
|
|
||||||
if f.read().strip() == process_name:
|
|
||||||
return True
|
|
||||||
except (IOError, OSError):
|
|
||||||
continue
|
|
||||||
except OSError:
|
|
||||||
pass
|
|
||||||
return False
|
|
||||||
|
|
||||||
|
|
||||||
def process_count(process_name: str) -> int:
|
|
||||||
"""Count processes with the given name."""
|
|
||||||
try:
|
|
||||||
result = subprocess.run(
|
|
||||||
["pgrep", "-c", "-x", process_name],
|
|
||||||
capture_output=True,
|
|
||||||
text=True,
|
|
||||||
)
|
|
||||||
if result.returncode == 0:
|
|
||||||
return int(result.stdout.strip())
|
|
||||||
return 0
|
|
||||||
except (FileNotFoundError, ValueError):
|
|
||||||
return 0
|
|
||||||
@@ -1,41 +0,0 @@
|
|||||||
"""User existence and attribute probes."""
|
|
||||||
|
|
||||||
import pwd
|
|
||||||
import grp
|
|
||||||
|
|
||||||
|
|
||||||
def user_exists(username: str) -> bool:
|
|
||||||
"""Check if a system user exists."""
|
|
||||||
try:
|
|
||||||
pwd.getpwnam(username)
|
|
||||||
return True
|
|
||||||
except KeyError:
|
|
||||||
return False
|
|
||||||
|
|
||||||
|
|
||||||
def user_uid(username: str) -> int | None:
|
|
||||||
try:
|
|
||||||
return pwd.getpwnam(username).pw_uid
|
|
||||||
except KeyError:
|
|
||||||
return None
|
|
||||||
|
|
||||||
def user_gid(username: str) -> int | None:
|
|
||||||
try:
|
|
||||||
return pwd.getpwnam(username).pw_gid
|
|
||||||
except KeyError:
|
|
||||||
return None
|
|
||||||
|
|
||||||
def user_in_group(username: str, groupname: str) -> bool:
|
|
||||||
"""Check if a user is a member of a group."""
|
|
||||||
try:
|
|
||||||
group = grp.getgrnam(groupname)
|
|
||||||
return username in group.gr_mem
|
|
||||||
except KeyError:
|
|
||||||
return False
|
|
||||||
|
|
||||||
|
|
||||||
def user_shell(username: str) -> str | None:
|
|
||||||
try:
|
|
||||||
return pwd.getpwnam(username).pw_shell
|
|
||||||
except KeyError:
|
|
||||||
return None
|
|
||||||
@@ -1,6 +0,0 @@
|
|||||||
pytest>=8.0.0
|
|
||||||
pytest-timeout>=2.2.0
|
|
||||||
pytest-xdist>=3.5.0
|
|
||||||
anthropic>=0.40.0,<0.90.0
|
|
||||||
openai>=1.50.0,<2.0.0
|
|
||||||
pyyaml>=6.0
|
|
||||||
Generated
+107
@@ -0,0 +1,107 @@
|
|||||||
|
# This file is automatically @generated by Cargo.
|
||||||
|
# It is not intended for manual editing.
|
||||||
|
version = 4
|
||||||
|
|
||||||
|
[[package]]
|
||||||
|
name = "itoa"
|
||||||
|
version = "1.0.18"
|
||||||
|
source = "registry+https://github.com/rust-lang/crates.io-index"
|
||||||
|
checksum = "8f42a60cbdf9a97f5d2305f08a87dc4e09308d1276d28c869c684d7777685682"
|
||||||
|
|
||||||
|
[[package]]
|
||||||
|
name = "memchr"
|
||||||
|
version = "2.8.0"
|
||||||
|
source = "registry+https://github.com/rust-lang/crates.io-index"
|
||||||
|
checksum = "f8ca58f447f06ed17d5fc4043ce1b10dd205e060fb3ce5b979b8ed8e59ff3f79"
|
||||||
|
|
||||||
|
[[package]]
|
||||||
|
name = "proc-macro2"
|
||||||
|
version = "1.0.106"
|
||||||
|
source = "registry+https://github.com/rust-lang/crates.io-index"
|
||||||
|
checksum = "8fd00f0bb2e90d81d1044c2b32617f68fcb9fa3bb7640c23e9c748e53fb30934"
|
||||||
|
dependencies = [
|
||||||
|
"unicode-ident",
|
||||||
|
]
|
||||||
|
|
||||||
|
[[package]]
|
||||||
|
name = "quote"
|
||||||
|
version = "1.0.45"
|
||||||
|
source = "registry+https://github.com/rust-lang/crates.io-index"
|
||||||
|
checksum = "41f2619966050689382d2b44f664f4bc593e129785a36d6ee376ddf37259b924"
|
||||||
|
dependencies = [
|
||||||
|
"proc-macro2",
|
||||||
|
]
|
||||||
|
|
||||||
|
[[package]]
|
||||||
|
name = "serde"
|
||||||
|
version = "1.0.228"
|
||||||
|
source = "registry+https://github.com/rust-lang/crates.io-index"
|
||||||
|
checksum = "9a8e94ea7f378bd32cbbd37198a4a91436180c5bb472411e48b5ec2e2124ae9e"
|
||||||
|
dependencies = [
|
||||||
|
"serde_core",
|
||||||
|
"serde_derive",
|
||||||
|
]
|
||||||
|
|
||||||
|
[[package]]
|
||||||
|
name = "serde_core"
|
||||||
|
version = "1.0.228"
|
||||||
|
source = "registry+https://github.com/rust-lang/crates.io-index"
|
||||||
|
checksum = "41d385c7d4ca58e59fc732af25c3983b67ac852c1a25000afe1175de458b67ad"
|
||||||
|
dependencies = [
|
||||||
|
"serde_derive",
|
||||||
|
]
|
||||||
|
|
||||||
|
[[package]]
|
||||||
|
name = "serde_derive"
|
||||||
|
version = "1.0.228"
|
||||||
|
source = "registry+https://github.com/rust-lang/crates.io-index"
|
||||||
|
checksum = "d540f220d3187173da220f885ab66608367b6574e925011a9353e4badda91d79"
|
||||||
|
dependencies = [
|
||||||
|
"proc-macro2",
|
||||||
|
"quote",
|
||||||
|
"syn",
|
||||||
|
]
|
||||||
|
|
||||||
|
[[package]]
|
||||||
|
name = "serde_json"
|
||||||
|
version = "1.0.149"
|
||||||
|
source = "registry+https://github.com/rust-lang/crates.io-index"
|
||||||
|
checksum = "83fc039473c5595ace860d8c4fafa220ff474b3fc6bfdb4293327f1a37e94d86"
|
||||||
|
dependencies = [
|
||||||
|
"itoa",
|
||||||
|
"memchr",
|
||||||
|
"serde",
|
||||||
|
"serde_core",
|
||||||
|
"zmij",
|
||||||
|
]
|
||||||
|
|
||||||
|
[[package]]
|
||||||
|
name = "syn"
|
||||||
|
version = "2.0.117"
|
||||||
|
source = "registry+https://github.com/rust-lang/crates.io-index"
|
||||||
|
checksum = "e665b8803e7b1d2a727f4023456bbbbe74da67099c585258af0ad9c5013b9b99"
|
||||||
|
dependencies = [
|
||||||
|
"proc-macro2",
|
||||||
|
"quote",
|
||||||
|
"unicode-ident",
|
||||||
|
]
|
||||||
|
|
||||||
|
[[package]]
|
||||||
|
name = "unicode-ident"
|
||||||
|
version = "1.0.24"
|
||||||
|
source = "registry+https://github.com/rust-lang/crates.io-index"
|
||||||
|
checksum = "e6e4313cd5fcd3dad5cafa179702e2b244f760991f45397d14d4ebf38247da75"
|
||||||
|
|
||||||
|
[[package]]
|
||||||
|
name = "workshop-baker-params"
|
||||||
|
version = "0.1.0"
|
||||||
|
dependencies = [
|
||||||
|
"serde",
|
||||||
|
"serde_json",
|
||||||
|
]
|
||||||
|
|
||||||
|
[[package]]
|
||||||
|
name = "zmij"
|
||||||
|
version = "1.0.21"
|
||||||
|
source = "registry+https://github.com/rust-lang/crates.io-index"
|
||||||
|
checksum = "b8848ee67ecc8aedbaf3e4122217aff892639231befc6a1b58d29fff4c2cabaa"
|
||||||
@@ -0,0 +1,8 @@
|
|||||||
|
[package]
|
||||||
|
name = "workshop-baker-params"
|
||||||
|
version = "0.1.0"
|
||||||
|
edition = "2024"
|
||||||
|
|
||||||
|
[dependencies]
|
||||||
|
serde = { version = "1.0.228", features = ["derive"] }
|
||||||
|
serde_json = "1.0.149"
|
||||||
@@ -0,0 +1,24 @@
|
|||||||
|
use crate::apply_field;
|
||||||
|
use serde::{Deserialize, Serialize};
|
||||||
|
use crate::{ParamVal, Writer, Overridden, traits::MergeParams};
|
||||||
|
|
||||||
|
#[derive(Debug, Clone, Serialize, Deserialize)]
|
||||||
|
#[serde(deny_unknown_fields)]
|
||||||
|
pub struct BakeParams {
|
||||||
|
#[serde(default)] pub workspace: ParamVal<String>,
|
||||||
|
}
|
||||||
|
|
||||||
|
impl MergeParams for BakeParams {
|
||||||
|
fn defaults() -> Self { Self { workspace: ParamVal::new("/workspace".into()) } }
|
||||||
|
fn apply(&mut self, i: Self, w: Writer, ov: &mut Overridden, p: &str) {
|
||||||
|
apply_field!(self.workspace, i.workspace, w, ov, p, "workspace");
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
impl Default for BakeParams { fn default() -> Self { Self::defaults() } }
|
||||||
|
|
||||||
|
#[derive(Debug, Clone)]
|
||||||
|
pub struct BakeSettings { pub workspace: String }
|
||||||
|
impl Default for BakeSettings { fn default() -> Self { BakeParams::defaults().into() } }
|
||||||
|
impl From<&BakeParams> for BakeSettings { fn from(p: &BakeParams) -> Self { Self { workspace: p.workspace.value.clone() } } }
|
||||||
|
impl From<BakeParams> for BakeSettings { fn from(p: BakeParams) -> Self { (&p).into() } }
|
||||||
@@ -0,0 +1,23 @@
|
|||||||
|
use crate::apply_field;
|
||||||
|
use serde::{Deserialize, Serialize};
|
||||||
|
use crate::{ParamVal, Writer, Overridden, traits::MergeParams};
|
||||||
|
|
||||||
|
#[derive(Debug, Clone, Serialize, Deserialize)]
|
||||||
|
#[serde(deny_unknown_fields)]
|
||||||
|
pub struct FinalizeParams {
|
||||||
|
#[serde(default)] pub artifact_retention_days: ParamVal<u32>,
|
||||||
|
}
|
||||||
|
|
||||||
|
impl MergeParams for FinalizeParams {
|
||||||
|
fn defaults() -> Self { Self { artifact_retention_days: ParamVal::new(7) } }
|
||||||
|
fn apply(&mut self, i: Self, w: Writer, ov: &mut Overridden, p: &str) {
|
||||||
|
apply_field!(self.artifact_retention_days, i.artifact_retention_days, w, ov, p, "artifact_retention_days");
|
||||||
|
}
|
||||||
|
}
|
||||||
|
impl Default for FinalizeParams { fn default() -> Self { Self::defaults() } }
|
||||||
|
|
||||||
|
#[derive(Debug, Clone)]
|
||||||
|
pub struct FinalizeSettings { pub artifact_retention_days: u32 }
|
||||||
|
impl Default for FinalizeSettings { fn default() -> Self { FinalizeParams::defaults().into() } }
|
||||||
|
impl From<&FinalizeParams> for FinalizeSettings { fn from(p: &FinalizeParams) -> Self { Self { artifact_retention_days: p.artifact_retention_days.value } } }
|
||||||
|
impl From<FinalizeParams> for FinalizeSettings { fn from(p: FinalizeParams) -> Self { (&p).into() } }
|
||||||
@@ -0,0 +1,168 @@
|
|||||||
|
pub mod notification;
|
||||||
|
pub mod prebake;
|
||||||
|
pub mod bake;
|
||||||
|
pub mod finalize;
|
||||||
|
pub mod writer;
|
||||||
|
pub mod param;
|
||||||
|
pub mod traits;
|
||||||
|
|
||||||
|
pub use notification::{NotificationParams, NotificationSettings};
|
||||||
|
pub use prebake::{PrebakeParams, PrebakeSettings};
|
||||||
|
pub use bake::{BakeParams, BakeSettings};
|
||||||
|
pub use finalize::{FinalizeParams, FinalizeSettings};
|
||||||
|
pub use writer::Writer;
|
||||||
|
pub use param::ParamVal;
|
||||||
|
pub use traits::MergeParams;
|
||||||
|
|
||||||
|
use serde::{Deserialize, Serialize};
|
||||||
|
use std::collections::HashMap;
|
||||||
|
|
||||||
|
/// Root parameter tree. Each component gets its own subtree.
|
||||||
|
/// Used for both IPC (Serialized to JSON) and in-process merging.
|
||||||
|
#[derive(Debug, Clone, Serialize, Deserialize, Default)]
|
||||||
|
pub struct Params {
|
||||||
|
#[serde(default, skip_serializing_if = "Option::is_none")]
|
||||||
|
pub notification: Option<NotificationParams>,
|
||||||
|
#[serde(default, skip_serializing_if = "Option::is_none")]
|
||||||
|
pub prebake: Option<PrebakeParams>,
|
||||||
|
#[serde(default, skip_serializing_if = "Option::is_none")]
|
||||||
|
pub bake: Option<BakeParams>,
|
||||||
|
#[serde(default, skip_serializing_if = "Option::is_none")]
|
||||||
|
pub finalize: Option<FinalizeParams>,
|
||||||
|
}
|
||||||
|
|
||||||
|
/// Apply and merge helpers.
|
||||||
|
impl Params {
|
||||||
|
/// Merge a layer from a specific writer into the current params.
|
||||||
|
/// Returns entries that were overridden (for debug).
|
||||||
|
pub fn apply(&mut self, incoming: Params, writer: Writer) -> Overridden {
|
||||||
|
let mut ov = Overridden::new();
|
||||||
|
if let Some(l) = incoming.notification {
|
||||||
|
if let Some(ref mut b) = self.notification {
|
||||||
|
b.apply(l, writer, &mut ov, "notification.");
|
||||||
|
} else {
|
||||||
|
self.notification = Some(l);
|
||||||
|
}
|
||||||
|
}
|
||||||
|
if let Some(l) = incoming.prebake {
|
||||||
|
if let Some(ref mut b) = self.prebake {
|
||||||
|
b.apply(l, writer, &mut ov, "prebake.");
|
||||||
|
} else {
|
||||||
|
self.prebake = Some(l);
|
||||||
|
}
|
||||||
|
}
|
||||||
|
if let Some(l) = incoming.bake {
|
||||||
|
if let Some(ref mut b) = self.bake {
|
||||||
|
b.apply(l, writer, &mut ov, "bake.");
|
||||||
|
} else {
|
||||||
|
self.bake = Some(l);
|
||||||
|
}
|
||||||
|
}
|
||||||
|
if let Some(l) = incoming.finalize {
|
||||||
|
if let Some(ref mut b) = self.finalize {
|
||||||
|
b.apply(l, writer, &mut ov, "finalize.");
|
||||||
|
} else {
|
||||||
|
self.finalize = Some(l);
|
||||||
|
}
|
||||||
|
}
|
||||||
|
ov
|
||||||
|
}
|
||||||
|
|
||||||
|
/// Convert to baker-consumable settings. All params guaranteed to have values.
|
||||||
|
pub fn to_settings(&self) -> Result<AllSettings, String> {
|
||||||
|
Ok(AllSettings {
|
||||||
|
notification: self.notification.as_ref().map(NotificationSettings::from).unwrap_or_default(),
|
||||||
|
prebake: self.prebake.as_ref().map(PrebakeSettings::from).unwrap_or_default(),
|
||||||
|
bake: self.bake.as_ref().map(BakeSettings::from).unwrap_or_default(),
|
||||||
|
finalize: self.finalize.as_ref().map(FinalizeSettings::from).unwrap_or_default(),
|
||||||
|
})
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
/// Plain-value settings for baker consumption. No metadata, no Option.
|
||||||
|
#[derive(Debug, Clone)]
|
||||||
|
pub struct AllSettings {
|
||||||
|
pub notification: NotificationSettings,
|
||||||
|
pub prebake: PrebakeSettings,
|
||||||
|
pub bake: BakeSettings,
|
||||||
|
pub finalize: FinalizeSettings,
|
||||||
|
}
|
||||||
|
|
||||||
|
/// Debug — values that were overridden during merge, keyed by dot-path.
|
||||||
|
#[derive(Debug, Clone, Default)]
|
||||||
|
pub struct Overridden {
|
||||||
|
pub entries: std::collections::HashMap<String, serde_json::Value>,
|
||||||
|
}
|
||||||
|
|
||||||
|
impl Overridden {
|
||||||
|
pub fn new() -> Self { Self { entries: HashMap::new() } }
|
||||||
|
pub fn is_empty(&self) -> bool { self.entries.is_empty() }
|
||||||
|
}
|
||||||
|
|
||||||
|
|
||||||
|
|
||||||
|
#[cfg(test)]
|
||||||
|
mod tests {
|
||||||
|
use super::*;
|
||||||
|
|
||||||
|
#[test]
|
||||||
|
fn test_empty_params_serializes_empty() {
|
||||||
|
let p = Params::default();
|
||||||
|
let json = serde_json::to_string(&p).unwrap();
|
||||||
|
assert_eq!(json, "{}");
|
||||||
|
}
|
||||||
|
|
||||||
|
#[test]
|
||||||
|
fn test_notification_params_serialize() {
|
||||||
|
let mut p = Params::default();
|
||||||
|
p.notification = Some(NotificationParams::defaults());
|
||||||
|
let json = serde_json::to_string_pretty(&p).unwrap();
|
||||||
|
assert!(json.contains("max_lives"));
|
||||||
|
assert!(json.contains("3"));
|
||||||
|
}
|
||||||
|
|
||||||
|
#[test]
|
||||||
|
fn test_apply_courier_overrides_default() {
|
||||||
|
let mut base = Params::default();
|
||||||
|
base.notification = Some(NotificationParams::defaults());
|
||||||
|
let incoming = Params {
|
||||||
|
notification: Some(NotificationParams {
|
||||||
|
max_lives: ParamVal::with(5, Writer::Courier),
|
||||||
|
..Default::default()
|
||||||
|
}),
|
||||||
|
..Default::default()
|
||||||
|
};
|
||||||
|
let ov = base.apply(incoming, Writer::Courier);
|
||||||
|
let s = base.to_settings().unwrap();
|
||||||
|
assert_eq!(s.notification.max_lives, 5);
|
||||||
|
assert!(ov.is_empty()); // default had no writer, not an "override"
|
||||||
|
}
|
||||||
|
|
||||||
|
#[test]
|
||||||
|
fn test_apply_upstream_wins() {
|
||||||
|
let mut base = Params::default();
|
||||||
|
base.notification = Some(NotificationParams {
|
||||||
|
max_lives: ParamVal::with(3, Writer::Base),
|
||||||
|
..Default::default()
|
||||||
|
});
|
||||||
|
let incoming = Params {
|
||||||
|
notification: Some(NotificationParams {
|
||||||
|
max_lives: ParamVal::with(5, Writer::Courier),
|
||||||
|
..Default::default()
|
||||||
|
}),
|
||||||
|
..Default::default()
|
||||||
|
};
|
||||||
|
let ov = base.apply(incoming, Writer::Courier);
|
||||||
|
let s = base.to_settings().unwrap();
|
||||||
|
// base(3) > courier(2) — rejected, base stays
|
||||||
|
assert_eq!(s.notification.max_lives, 3);
|
||||||
|
assert!(ov.is_empty()); // nothing overridden
|
||||||
|
}
|
||||||
|
|
||||||
|
#[test]
|
||||||
|
fn test_deny_unknown_fields() {
|
||||||
|
let bad = r#"{"max_lives":{"value":5},"ghost_param":{"value":1}}"#;
|
||||||
|
let result: Result<NotificationParams, _> = serde_json::from_str(bad);
|
||||||
|
assert!(result.is_err()); // unknown field should be denied
|
||||||
|
}
|
||||||
|
}
|
||||||
@@ -0,0 +1,65 @@
|
|||||||
|
use crate::apply_field;
|
||||||
|
use serde::{Deserialize, Serialize};
|
||||||
|
use crate::{ParamVal, Writer, Overridden, traits::MergeParams};
|
||||||
|
|
||||||
|
#[derive(Debug, Clone, Serialize, Deserialize)]
|
||||||
|
#[serde(deny_unknown_fields)]
|
||||||
|
pub struct NotificationParams {
|
||||||
|
#[serde(default)] pub max_lives: ParamVal<u32>,
|
||||||
|
#[serde(default)] pub timeout_ms: ParamVal<u64>,
|
||||||
|
#[serde(default)] pub retry_backoff: ParamVal<f64>,
|
||||||
|
#[serde(default)] pub retry_delay_seconds: ParamVal<f64>,
|
||||||
|
#[serde(default)] pub max_retry_delay_seconds: ParamVal<f64>,
|
||||||
|
#[serde(default)] pub life_impact_factor: ParamVal<f64>,
|
||||||
|
#[serde(default)] pub batch_period: ParamVal<u32>,
|
||||||
|
#[serde(default)] pub batch_watermark: ParamVal<u32>,
|
||||||
|
#[serde(default)] pub template_ref_depth: ParamVal<u32>,
|
||||||
|
}
|
||||||
|
|
||||||
|
impl MergeParams for NotificationParams {
|
||||||
|
fn defaults() -> Self {
|
||||||
|
Self {
|
||||||
|
max_lives: ParamVal::new(3), timeout_ms: ParamVal::new(30000),
|
||||||
|
retry_backoff: ParamVal::new(2.0), retry_delay_seconds: ParamVal::new(5.0),
|
||||||
|
max_retry_delay_seconds: ParamVal::new(300.0),
|
||||||
|
life_impact_factor: ParamVal::new(0.0),
|
||||||
|
batch_period: ParamVal::new(30), batch_watermark: ParamVal::new(5),
|
||||||
|
template_ref_depth: ParamVal::new(10),
|
||||||
|
}
|
||||||
|
}
|
||||||
|
fn apply(&mut self, incoming: Self, writer: Writer, ov: &mut Overridden, prefix: &str) {
|
||||||
|
apply_field!(self.max_lives, incoming.max_lives, writer, ov, prefix, "max_lives");
|
||||||
|
apply_field!(self.timeout_ms, incoming.timeout_ms, writer, ov, prefix, "timeout_ms");
|
||||||
|
apply_field!(self.retry_backoff, incoming.retry_backoff, writer, ov, prefix, "retry_backoff");
|
||||||
|
apply_field!(self.retry_delay_seconds, incoming.retry_delay_seconds, writer, ov, prefix, "retry_delay_seconds");
|
||||||
|
apply_field!(self.max_retry_delay_seconds, incoming.max_retry_delay_seconds, writer, ov, prefix, "max_retry_delay_seconds");
|
||||||
|
apply_field!(self.life_impact_factor, incoming.life_impact_factor, writer, ov, prefix, "life_impact_factor");
|
||||||
|
apply_field!(self.batch_period, incoming.batch_period, writer, ov, prefix, "batch_period");
|
||||||
|
apply_field!(self.batch_watermark, incoming.batch_watermark, writer, ov, prefix, "batch_watermark");
|
||||||
|
apply_field!(self.template_ref_depth, incoming.template_ref_depth, writer, ov, prefix, "template_ref_depth");
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
impl Default for NotificationParams { fn default() -> Self { Self::defaults() } }
|
||||||
|
|
||||||
|
#[derive(Debug, Clone)]
|
||||||
|
pub struct NotificationSettings {
|
||||||
|
pub max_lives: u32, pub timeout_ms: u64, pub retry_backoff: f64,
|
||||||
|
pub retry_delay_seconds: f64, pub max_retry_delay_seconds: f64,
|
||||||
|
pub life_impact_factor: f64,
|
||||||
|
pub batch_period: u32, pub batch_watermark: u32, pub template_ref_depth: u32,
|
||||||
|
}
|
||||||
|
|
||||||
|
impl Default for NotificationSettings { fn default() -> Self { NotificationParams::defaults().into() } }
|
||||||
|
|
||||||
|
impl From<&NotificationParams> for NotificationSettings {
|
||||||
|
fn from(p: &NotificationParams) -> Self { Self {
|
||||||
|
max_lives: p.max_lives.value, timeout_ms: p.timeout_ms.value,
|
||||||
|
retry_backoff: p.retry_backoff.value, retry_delay_seconds: p.retry_delay_seconds.value,
|
||||||
|
max_retry_delay_seconds: p.max_retry_delay_seconds.value,
|
||||||
|
life_impact_factor: p.life_impact_factor.value,
|
||||||
|
batch_period: p.batch_period.value, batch_watermark: p.batch_watermark.value,
|
||||||
|
template_ref_depth: p.template_ref_depth.value,
|
||||||
|
}}
|
||||||
|
}
|
||||||
|
impl From<NotificationParams> for NotificationSettings { fn from(p: NotificationParams) -> Self { (&p).into() } }
|
||||||
@@ -0,0 +1,32 @@
|
|||||||
|
use serde::{Deserialize, Serialize};
|
||||||
|
use crate::Writer;
|
||||||
|
|
||||||
|
/// A strongly-typed parameter value with writer tracking.
|
||||||
|
///
|
||||||
|
/// Serializes as `{ "value": 3, "writer": "base" }`.
|
||||||
|
#[derive(Debug, Clone, Serialize, Deserialize)]
|
||||||
|
pub struct ParamVal<T> {
|
||||||
|
pub value: T,
|
||||||
|
#[serde(skip_serializing_if = "Option::is_none")]
|
||||||
|
pub writer: Option<Writer>,
|
||||||
|
}
|
||||||
|
|
||||||
|
impl<T: Default> Default for ParamVal<T> {
|
||||||
|
fn default() -> Self {
|
||||||
|
ParamVal { value: T::default(), writer: None }
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
impl<T> ParamVal<T> {
|
||||||
|
pub fn new(value: T) -> Self {
|
||||||
|
ParamVal { value, writer: None }
|
||||||
|
}
|
||||||
|
pub fn with(value: T, writer: Writer) -> Self {
|
||||||
|
ParamVal { value, writer: Some(writer) }
|
||||||
|
}
|
||||||
|
/// Whether an incoming value from `incoming_writer` should override this one.
|
||||||
|
pub fn should_override(&self, incoming: Writer) -> bool {
|
||||||
|
let current = self.writer.map_or(0, |w| w.priority());
|
||||||
|
incoming.priority() > current
|
||||||
|
}
|
||||||
|
}
|
||||||
@@ -0,0 +1,54 @@
|
|||||||
|
pub mod network;
|
||||||
|
pub mod cache;
|
||||||
|
pub mod engine;
|
||||||
|
|
||||||
|
use crate::apply_field;
|
||||||
|
use crate::{ParamVal, Writer, Overridden, traits::MergeParams};
|
||||||
|
use serde::{Deserialize, Serialize};
|
||||||
|
|
||||||
|
#[derive(Debug, Clone, Serialize, Deserialize)]
|
||||||
|
#[serde(deny_unknown_fields)]
|
||||||
|
pub struct PrebakeParams {
|
||||||
|
#[serde(default)] pub bootstrap_user: ParamVal<String>,
|
||||||
|
#[serde(default)] pub workspace: ParamVal<String>,
|
||||||
|
#[serde(default)] pub network: network::NetworkParams,
|
||||||
|
#[serde(default)] pub cache: cache::CacheParams,
|
||||||
|
#[serde(default)] pub engine: engine::EngineParams,
|
||||||
|
#[serde(default)] pub repology_endpoint: ParamVal<String>,
|
||||||
|
}
|
||||||
|
|
||||||
|
impl MergeParams for PrebakeParams {
|
||||||
|
fn defaults() -> Self {
|
||||||
|
Self {
|
||||||
|
bootstrap_user: ParamVal::new("vulcan".into()), workspace: ParamVal::new("/home/vulcan/workspace".into()),
|
||||||
|
network: network::NetworkParams::defaults(), cache: cache::CacheParams::defaults(),
|
||||||
|
engine: engine::EngineParams::defaults(),
|
||||||
|
repology_endpoint: ParamVal::new("default".into()),
|
||||||
|
}
|
||||||
|
}
|
||||||
|
fn apply(&mut self, i: Self, w: Writer, ov: &mut Overridden, p: &str) {
|
||||||
|
apply_field!(self.bootstrap_user, i.bootstrap_user, w, ov, p, "bootstrap_user");
|
||||||
|
apply_field!(self.workspace, i.workspace, w, ov, p, "workspace");
|
||||||
|
apply_field!(self.repology_endpoint, i.repology_endpoint, w, ov, p, "repology_endpoint");
|
||||||
|
self.network.apply(i.network, w, ov, &format!("{}network.", p));
|
||||||
|
self.cache.apply(i.cache, w, ov, &format!("{}cache.", p));
|
||||||
|
self.engine.apply(i.engine, w, ov, &format!("{}engine.", p));
|
||||||
|
}
|
||||||
|
}
|
||||||
|
impl Default for PrebakeParams { fn default() -> Self { Self::defaults() } }
|
||||||
|
|
||||||
|
#[derive(Debug, Clone)]
|
||||||
|
pub struct PrebakeSettings {
|
||||||
|
pub bootstrap_user: String, pub workspace: String,
|
||||||
|
pub network: network::NetworkSettings, pub cache: cache::CacheSettings,
|
||||||
|
pub engine: engine::EngineSettings, pub repology_endpoint: String,
|
||||||
|
}
|
||||||
|
impl Default for PrebakeSettings { fn default() -> Self { PrebakeParams::defaults().into() } }
|
||||||
|
impl From<&PrebakeParams> for PrebakeSettings {
|
||||||
|
fn from(p: &PrebakeParams) -> Self { Self {
|
||||||
|
bootstrap_user: p.bootstrap_user.value.clone(), workspace: p.workspace.value.clone(),
|
||||||
|
network: (&p.network).into(), cache: (&p.cache).into(), engine: (&p.engine).into(),
|
||||||
|
repology_endpoint: p.repology_endpoint.value.clone(),
|
||||||
|
}}
|
||||||
|
}
|
||||||
|
impl From<PrebakeParams> for PrebakeSettings { fn from(p: PrebakeParams) -> Self { (&p).into() } }
|
||||||
@@ -0,0 +1,37 @@
|
|||||||
|
use crate::apply_field;
|
||||||
|
use crate::{ParamVal, Writer, Overridden, traits::MergeParams};
|
||||||
|
use serde::{Deserialize, Serialize};
|
||||||
|
|
||||||
|
#[derive(Debug, Clone, Serialize, Deserialize)]
|
||||||
|
pub struct CacheParams {
|
||||||
|
#[serde(default)] pub strategy: ParamVal<String>,
|
||||||
|
#[serde(default)] pub mode: ParamVal<String>,
|
||||||
|
#[serde(default)] pub ttl_days: ParamVal<u32>,
|
||||||
|
#[serde(default)] pub max_size_gb: ParamVal<u32>,
|
||||||
|
#[serde(default)] pub cleanup_policy: ParamVal<String>,
|
||||||
|
}
|
||||||
|
impl MergeParams for CacheParams {
|
||||||
|
fn defaults() -> Self {
|
||||||
|
Self {
|
||||||
|
strategy: ParamVal::new("always".into()), mode: ParamVal::new("zstd".into()),
|
||||||
|
ttl_days: ParamVal::new(30), max_size_gb: ParamVal::new(20),
|
||||||
|
cleanup_policy: ParamVal::new("lru".into()),
|
||||||
|
}
|
||||||
|
}
|
||||||
|
fn apply(&mut self, i: Self, w: Writer, ov: &mut Overridden, p: &str) {
|
||||||
|
apply_field!(self.strategy, i.strategy, w, ov, p, "strategy");
|
||||||
|
apply_field!(self.mode, i.mode, w, ov, p, "mode");
|
||||||
|
apply_field!(self.ttl_days, i.ttl_days, w, ov, p, "ttl_days");
|
||||||
|
apply_field!(self.max_size_gb, i.max_size_gb, w, ov, p, "max_size_gb");
|
||||||
|
apply_field!(self.cleanup_policy, i.cleanup_policy, w, ov, p, "cleanup_policy");
|
||||||
|
}
|
||||||
|
}
|
||||||
|
impl Default for CacheParams { fn default() -> Self { Self::defaults() } }
|
||||||
|
|
||||||
|
#[derive(Debug, Clone)]
|
||||||
|
pub struct CacheSettings { pub strategy: String, pub mode: String, pub ttl_days: u32, pub max_size_gb: u32, pub cleanup_policy: String }
|
||||||
|
impl Default for CacheSettings { fn default() -> Self { CacheParams::defaults().into() } }
|
||||||
|
impl From<&CacheParams> for CacheSettings { fn from(p: &CacheParams) -> Self {
|
||||||
|
Self { strategy: p.strategy.value.clone(), mode: p.mode.value.clone(), ttl_days: p.ttl_days.value, max_size_gb: p.max_size_gb.value, cleanup_policy: p.cleanup_policy.value.clone() }
|
||||||
|
} }
|
||||||
|
impl From<CacheParams> for CacheSettings { fn from(p: CacheParams) -> Self { (&p).into() } }
|
||||||
@@ -0,0 +1,21 @@
|
|||||||
|
use crate::apply_field;
|
||||||
|
use crate::{ParamVal, Writer, Overridden, traits::MergeParams};
|
||||||
|
use serde::{Deserialize, Serialize};
|
||||||
|
|
||||||
|
#[derive(Debug, Clone, Serialize, Deserialize)]
|
||||||
|
pub struct EngineParams {
|
||||||
|
#[serde(default)] pub timeout_ms: ParamVal<u64>,
|
||||||
|
}
|
||||||
|
impl MergeParams for EngineParams {
|
||||||
|
fn defaults() -> Self { Self { timeout_ms: ParamVal::new(300000) } }
|
||||||
|
fn apply(&mut self, i: Self, w: Writer, ov: &mut Overridden, p: &str) {
|
||||||
|
apply_field!(self.timeout_ms, i.timeout_ms, w, ov, p, "timeout_ms");
|
||||||
|
}
|
||||||
|
}
|
||||||
|
impl Default for EngineParams { fn default() -> Self { Self::defaults() } }
|
||||||
|
|
||||||
|
#[derive(Debug, Clone)]
|
||||||
|
pub struct EngineSettings { pub timeout_ms: u64 }
|
||||||
|
impl Default for EngineSettings { fn default() -> Self { EngineParams::defaults().into() } }
|
||||||
|
impl From<&EngineParams> for EngineSettings { fn from(p: &EngineParams) -> Self { Self { timeout_ms: p.timeout_ms.value } } }
|
||||||
|
impl From<EngineParams> for EngineSettings { fn from(p: EngineParams) -> Self { (&p).into() } }
|
||||||
@@ -0,0 +1,23 @@
|
|||||||
|
use crate::apply_field;
|
||||||
|
use crate::{ParamVal, Writer, Overridden, traits::MergeParams};
|
||||||
|
use serde::{Deserialize, Serialize};
|
||||||
|
|
||||||
|
#[derive(Debug, Clone, Serialize, Deserialize)]
|
||||||
|
pub struct NetworkParams {
|
||||||
|
#[serde(default)] pub enabled: ParamVal<bool>,
|
||||||
|
#[serde(default)] pub outbound: ParamVal<bool>,
|
||||||
|
}
|
||||||
|
impl MergeParams for NetworkParams {
|
||||||
|
fn defaults() -> Self { Self { enabled: ParamVal::new(true), outbound: ParamVal::new(true) } }
|
||||||
|
fn apply(&mut self, i: Self, w: Writer, ov: &mut Overridden, p: &str) {
|
||||||
|
apply_field!(self.enabled, i.enabled, w, ov, p, "enabled");
|
||||||
|
apply_field!(self.outbound, i.outbound, w, ov, p, "outbound");
|
||||||
|
}
|
||||||
|
}
|
||||||
|
impl Default for NetworkParams { fn default() -> Self { Self::defaults() } }
|
||||||
|
|
||||||
|
#[derive(Debug, Clone)]
|
||||||
|
pub struct NetworkSettings { pub enabled: bool, pub outbound: bool }
|
||||||
|
impl Default for NetworkSettings { fn default() -> Self { NetworkParams::defaults().into() } }
|
||||||
|
impl From<&NetworkParams> for NetworkSettings { fn from(p: &NetworkParams) -> Self { Self { enabled: p.enabled.value, outbound: p.outbound.value } } }
|
||||||
|
impl From<NetworkParams> for NetworkSettings { fn from(p: NetworkParams) -> Self { (&p).into() } }
|
||||||
@@ -0,0 +1,28 @@
|
|||||||
|
use crate::{Writer, Overridden};
|
||||||
|
|
||||||
|
/// A parameter tree node that can merge itself with an incoming layer.
|
||||||
|
pub trait MergeParams: Sized {
|
||||||
|
fn apply(&mut self, incoming: Self, writer: Writer, ov: &mut Overridden, prefix: &str);
|
||||||
|
fn defaults() -> Self;
|
||||||
|
}
|
||||||
|
|
||||||
|
/// Apply one field's merge logic.
|
||||||
|
/// ```ignore
|
||||||
|
/// apply_field!(self.foo, incoming.foo, writer, ov, prefix, "foo");
|
||||||
|
/// ```
|
||||||
|
#[macro_export]
|
||||||
|
macro_rules! apply_field {
|
||||||
|
($self:ident.$field:ident, $incoming:ident.$field2:ident, $writer:expr, $ov:ident, $prefix:expr, $name:expr) => {
|
||||||
|
let old_w = $self.$field.writer;
|
||||||
|
if $self.$field.should_override($writer) {
|
||||||
|
let old_v = ::std::mem::replace(&mut $self.$field.value, $incoming.$field2.value);
|
||||||
|
$self.$field.writer = ::std::option::Option::Some($writer);
|
||||||
|
if let ::std::option::Option::Some(w) = old_w {
|
||||||
|
$ov.entries.insert(
|
||||||
|
format!("{}{}", $prefix, $name),
|
||||||
|
::serde_json::json!({"value": old_v, "writer": w.as_str()}),
|
||||||
|
);
|
||||||
|
}
|
||||||
|
}
|
||||||
|
};
|
||||||
|
}
|
||||||
@@ -0,0 +1,26 @@
|
|||||||
|
use serde::{Deserialize, Serialize};
|
||||||
|
|
||||||
|
#[derive(Debug, Clone, Copy, PartialEq, Eq, Serialize, Deserialize)]
|
||||||
|
#[serde(rename_all = "lowercase")]
|
||||||
|
pub enum Writer {
|
||||||
|
Base,
|
||||||
|
Courier,
|
||||||
|
Bakerd,
|
||||||
|
}
|
||||||
|
|
||||||
|
impl Writer {
|
||||||
|
/// Priority: Base=3 > Courier=2 > Bakerd=1
|
||||||
|
pub fn priority(self) -> u8 {
|
||||||
|
match self {
|
||||||
|
Writer::Base => 3,
|
||||||
|
Writer::Courier => 2,
|
||||||
|
Writer::Bakerd => 1,
|
||||||
|
}
|
||||||
|
}
|
||||||
|
pub fn as_str(self) -> &'static str {
|
||||||
|
match self {
|
||||||
|
Writer::Base => "base", Writer::Courier => "courier", Writer::Bakerd => "bakerd",
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
impl Default for Writer { fn default() -> Self { Writer::Base } }
|
||||||
@@ -1,3 +1,4 @@
|
|||||||
temp
|
temp
|
||||||
examples
|
examples
|
||||||
quicktest.sh
|
quicktest.sh
|
||||||
|
_env.sh
|
||||||
|
|||||||
Generated
+84
-1471
File diff suppressed because it is too large
Load Diff
@@ -47,13 +47,10 @@ uuid = { version = "1.19.0", features = ["v4"] }
|
|||||||
libc = "0.2"
|
libc = "0.2"
|
||||||
os_info = "3.14.0"
|
os_info = "3.14.0"
|
||||||
privdrop = "0.5.6"
|
privdrop = "0.5.6"
|
||||||
reqwest = { version = "0.12", default-features = false, features = ["json", "rustls-tls"] }
|
|
||||||
which = "8.0.2"
|
which = "8.0.2"
|
||||||
gix = "0.81.0"
|
|
||||||
sha2 = "0.10"
|
|
||||||
flate2 = "1.0"
|
|
||||||
zstd = "0.13"
|
|
||||||
globset = "0.4.18"
|
globset = "0.4.18"
|
||||||
|
axum = "0.7"
|
||||||
|
workshop-baker-params = { version = "0.1.0", path = "../workshop-baker-params" }
|
||||||
|
|
||||||
[dev-dependencies]
|
[dev-dependencies]
|
||||||
criterion = "0.5"
|
criterion = "0.5"
|
||||||
@@ -66,6 +63,3 @@ unreachable_code = "allow"
|
|||||||
inherent_to_string = "allow"
|
inherent_to_string = "allow"
|
||||||
non_canonical_partial_ord_impl = "allow"
|
non_canonical_partial_ord_impl = "allow"
|
||||||
|
|
||||||
[[bench]]
|
|
||||||
name = "parser_bench"
|
|
||||||
harness = false
|
|
||||||
|
|||||||
@@ -1,93 +0,0 @@
|
|||||||
#!/usr/bin/env python3
|
|
||||||
import os
|
|
||||||
import random
|
|
||||||
import string
|
|
||||||
|
|
||||||
OUTPUT_DIR = "/tmp"
|
|
||||||
DECORATORS = [
|
|
||||||
"# @pipeline",
|
|
||||||
"# @fallible",
|
|
||||||
"# @parallel",
|
|
||||||
"# @export",
|
|
||||||
"# @timeout(60)",
|
|
||||||
"# @retry(3, 5)",
|
|
||||||
"# @if(condition)",
|
|
||||||
"# @after(dep)",
|
|
||||||
"# @pipe(step1, step2)",
|
|
||||||
"# @health(/health)",
|
|
||||||
"# @loop(10)",
|
|
||||||
]
|
|
||||||
FUNCTION_BODIES = [
|
|
||||||
"echo 'Processing...';",
|
|
||||||
"local var=$(date +%s);",
|
|
||||||
"if [ -f /tmp/test ]; then echo 'exists'; fi;",
|
|
||||||
"for i in $(seq 1 10); do echo $i; done;",
|
|
||||||
"case $1 in start) echo 'starting';; stop) echo 'stopping';; esac;",
|
|
||||||
"read -r line < /dev/stdin;",
|
|
||||||
"export PATH=$PATH:/usr/local/bin;",
|
|
||||||
"set -e; set -u;",
|
|
||||||
"trap 'echo error' ERR;",
|
|
||||||
"cd /tmp || exit 1;",
|
|
||||||
"local result=$(grep -r 'pattern' /var/log/ 2>/dev/null);",
|
|
||||||
'for f in /tmp/*.tmp; do rm -f "$f"; done',
|
|
||||||
"if [[ $var -gt 100 ]]; then echo 'large'; fi;",
|
|
||||||
'while read -r line; do echo "$line"; done < /etc/passwd;',
|
|
||||||
'select opt in a b c; do echo "Selected $opt"; break; done;',
|
|
||||||
]
|
|
||||||
|
|
||||||
|
|
||||||
def random_string(length=8):
|
|
||||||
return "".join(random.choices(string.ascii_letters + string.digits, k=length))
|
|
||||||
|
|
||||||
|
|
||||||
def generate_function(body_lines):
|
|
||||||
lines = []
|
|
||||||
func_name = f"func_{random_string(6)}"
|
|
||||||
for _ in range(random.randint(0, 3)):
|
|
||||||
lines.append(random.choice(DECORATORS))
|
|
||||||
lines.append(f"{func_name}() {{")
|
|
||||||
for _ in range(body_lines):
|
|
||||||
lines.append(" " + random.choice(FUNCTION_BODIES))
|
|
||||||
lines.append("}")
|
|
||||||
lines.append("")
|
|
||||||
return lines
|
|
||||||
|
|
||||||
|
|
||||||
def generate_script(target_lines, num_functions, target_bytes):
|
|
||||||
lines = []
|
|
||||||
total_lines = 0
|
|
||||||
body_per_func = max(1, (target_lines - num_functions * 3) // num_functions)
|
|
||||||
|
|
||||||
for _ in range(num_functions):
|
|
||||||
func_lines = generate_function(body_per_func)
|
|
||||||
lines.extend(func_lines)
|
|
||||||
total_lines += len(func_lines)
|
|
||||||
|
|
||||||
while (
|
|
||||||
total_lines < target_lines
|
|
||||||
or len("\n".join(lines).encode("utf-8")) < target_bytes
|
|
||||||
):
|
|
||||||
lines.append(random.choice(FUNCTION_BODIES))
|
|
||||||
total_lines += 1
|
|
||||||
|
|
||||||
return "\n".join(lines)
|
|
||||||
|
|
||||||
|
|
||||||
def main():
|
|
||||||
sizes = [
|
|
||||||
("small", 1000, 10, 50_000),
|
|
||||||
("medium", 5000, 50, 500_000),
|
|
||||||
("large", 10000, 100, 950_000),
|
|
||||||
]
|
|
||||||
for name, lines, funcs, target_bytes in sizes:
|
|
||||||
script = generate_script(lines, funcs, target_bytes)
|
|
||||||
filepath = os.path.join(OUTPUT_DIR, f"bench_{name}.sh")
|
|
||||||
with open(filepath, "w") as f:
|
|
||||||
f.write(script)
|
|
||||||
actual_bytes = len(script.encode("utf-8"))
|
|
||||||
actual_lines = len(script.splitlines())
|
|
||||||
print(f"Generated {filepath}: lines={actual_lines}, bytes={actual_bytes}")
|
|
||||||
|
|
||||||
|
|
||||||
if __name__ == "__main__":
|
|
||||||
main()
|
|
||||||
@@ -1,61 +0,0 @@
|
|||||||
use criterion::{Criterion, black_box, criterion_group, criterion_main};
|
|
||||||
use std::fs;
|
|
||||||
use std::path::Path;
|
|
||||||
|
|
||||||
fn parse_large_script(c: &mut Criterion) {
|
|
||||||
let script_path = Path::new("/tmp/bench_large.sh");
|
|
||||||
|
|
||||||
let script_content = fs::read_to_string(script_path)
|
|
||||||
.expect("Failed to read benchmark script. Run `python3 benches/generate.py` first.");
|
|
||||||
|
|
||||||
let byte_size = script_content.len();
|
|
||||||
let line_count = script_content.lines().count();
|
|
||||||
|
|
||||||
println!("Benchmark file: {} lines, {} bytes", line_count, byte_size);
|
|
||||||
|
|
||||||
c.bench_function("parse_script_large", |b| {
|
|
||||||
b.iter(|| {
|
|
||||||
let result =
|
|
||||||
workshop_baker::bake::parser::parse_script(black_box(&script_content.clone()));
|
|
||||||
black_box(result)
|
|
||||||
});
|
|
||||||
});
|
|
||||||
}
|
|
||||||
|
|
||||||
fn parse_medium_script(c: &mut Criterion) {
|
|
||||||
let script_path = Path::new("/tmp/bench_medium.sh");
|
|
||||||
|
|
||||||
let script_content = fs::read_to_string(script_path)
|
|
||||||
.expect("Failed to read benchmark script. Run `python3 benches/generate.py` first.");
|
|
||||||
|
|
||||||
c.bench_function("parse_script_medium", |b| {
|
|
||||||
b.iter(|| {
|
|
||||||
let result =
|
|
||||||
workshop_baker::bake::parser::parse_script(black_box(&script_content.clone()));
|
|
||||||
black_box(result)
|
|
||||||
});
|
|
||||||
});
|
|
||||||
}
|
|
||||||
|
|
||||||
fn parse_small_script(c: &mut Criterion) {
|
|
||||||
let script_path = Path::new("/tmp/bench_small.sh");
|
|
||||||
|
|
||||||
let script_content = fs::read_to_string(script_path)
|
|
||||||
.expect("Failed to read benchmark script. Run `python3 benches/generate.py` first.");
|
|
||||||
|
|
||||||
c.bench_function("parse_script_small", |b| {
|
|
||||||
b.iter(|| {
|
|
||||||
let result =
|
|
||||||
workshop_baker::bake::parser::parse_script(black_box(&script_content.clone()));
|
|
||||||
black_box(result)
|
|
||||||
});
|
|
||||||
});
|
|
||||||
}
|
|
||||||
|
|
||||||
criterion_group!(
|
|
||||||
benches,
|
|
||||||
parse_small_script,
|
|
||||||
parse_medium_script,
|
|
||||||
parse_large_script
|
|
||||||
);
|
|
||||||
criterion_main!(benches);
|
|
||||||
@@ -1,62 +1,57 @@
|
|||||||
# workshop-baker/src
|
# workshop-baker/src
|
||||||
|
|
||||||
**Generated:** 2026-04-22
|
**Generated:** 2026-05-19
|
||||||
**Commit:** 7b3b71a
|
**Commit:** 28abc5d
|
||||||
|
|
||||||
Baker CLI + execution engine for HoneyBiscuitWorkshop pipelines.
|
CLI tool and runtime for executing CI/CD pipelines. Orchestrates prebake → bake → finalize stages with notification dispatch.
|
||||||
|
|
||||||
## OVERVIEW
|
|
||||||
|
|
||||||
CLI tool and runtime for executing CI/CD pipelines with resource isolation, privilege dropping, and package management integration.
|
|
||||||
|
|
||||||
## STRUCTURE
|
## STRUCTURE
|
||||||
|
|
||||||
```
|
```
|
||||||
src/
|
src/
|
||||||
├── bake.rs # Top-level bake orchestration
|
├── lib.rs # Module declarations + Cli struct + Commands enum
|
||||||
├── bake/ # Script parsing and building
|
├── main.rs # tokio::main entry point
|
||||||
│ ├── parser.rs # @decorator-aware shell parser
|
├── bare.rs # Bare mode (run individual stages)
|
||||||
│ ├── decorator.rs # @pipeline, @retry, @parallel, etc.
|
├── error.rs # CliError (wraps stage errors)
|
||||||
│ ├── builder.rs # Script template builder
|
├── bake.rs # Bake orchestration (script → execution)
|
||||||
│ └── schedule.rs # Function ordering via @after deps
|
├── bake/ # Script parsing, decorator DSL, scheduling
|
||||||
├── engine/ # Execution runtime
|
├── prebake.rs # Prebake orchestration (env setup)
|
||||||
│ ├── executor.rs # Script execution with cgroups
|
├── prebake/ # Config, stages, security, env providers
|
||||||
│ ├── cgroups.rs # Linux cgroup resource limits
|
├── finalize.rs # Finalize orchestration (post-build)
|
||||||
│ ├── repology.rs # Package name resolution
|
├── finalize/ # Config, plugins, hooks, templates
|
||||||
│ ├── pm.rs # Package manager detection
|
├── notify.rs # Notification system (batching, retry)
|
||||||
│ └── upm.rs # User package manager (Nix/Guix)
|
├── notify/ # Types, templates, interactive, queue
|
||||||
├── prebake/ # Build environment setup
|
├── daemon/ # Empty (Daemon command = unimplemented!)
|
||||||
│ ├── config.rs # PrebakeConfig YAML schema
|
├── monitor/ # Resource usage monitoring (cgroups)
|
||||||
│ ├── stage/ # Bootstrap, DepsSystem, DepsUser, Hooks
|
├── types.rs # Module re-exports
|
||||||
│ └── security.rs # Privilege dropping logic
|
├── types/ # buildstatus, resource types
|
||||||
├── types/ # Shared type definitions
|
├── utils.rs # Module root (fetch submodule)
|
||||||
│ ├── repology.rs # RepologyEndpoint enum
|
└── utils/ # fetch.rs
|
||||||
│ ├── builderconfig.rs
|
|
||||||
│ ├── cache.rs
|
|
||||||
│ └── memsize.rs
|
|
||||||
├── monitor/ # Resource usage monitoring
|
|
||||||
│ ├── cgroups.rs # cgroup-based usage collection
|
|
||||||
│ └── jobobject.rs # Windows job objects (stub)
|
|
||||||
├── daemon.rs # Unix socket daemon (TODO: refactor)
|
|
||||||
└── lib.rs # CLI struct + module exports
|
|
||||||
```
|
```
|
||||||
|
|
||||||
## WHERE TO LOOK
|
## WHERE TO LOOK
|
||||||
|
|
||||||
| Task | Location |
|
| Task | Location |
|
||||||
|------|----------|
|
|------|----------|
|
||||||
| Script parsing with @decorators | `bake/parser.rs:29` - `parse_script()` |
|
| CLI args | `lib.rs:21` - `Cli` struct |
|
||||||
| Decorator types | `bake/decorator.rs` - `Decorator` enum |
|
| Entry point | `main.rs:91` - `#[tokio::main]` |
|
||||||
| Build execution | `engine/executor.rs` - `Executor::execute_script()` |
|
| Pipeline orchestration | `main.rs:68-74` - prebake→bake→finalize chain |
|
||||||
| Resource limits | `engine/types.rs:56` - `ResourceLimits` struct |
|
| Bare mode (single stage) | `bare.rs:14` - `run_bare()` |
|
||||||
| Prebake stages | `prebake/stage/` subdirectories |
|
| Script parsing | `bake/parser.rs:56` - `parse_script()` |
|
||||||
| Package detection | `engine/pm.rs:detection()` |
|
| Decorator DSL | `bake/decorator.rs` - `Decorator` enum |
|
||||||
| CLI entry | `lib.rs:26` - `cli::Cli` struct |
|
| Prebake stages | `prebake/stage/` - Bootstrap→DepsSystem→DepsUser→Hooks→Ready |
|
||||||
|
| Plugin system | `finalize/plugin.rs` + `finalize/plugin/` |
|
||||||
|
| Notification dispatch | `notify.rs:66` - `notify()` async loop |
|
||||||
|
| Notification types | `notify/types.rs` - `NotificationRule`, `ParsedTrigger` |
|
||||||
|
|
||||||
## CONVENTIONS
|
## CONVENTIONS
|
||||||
|
|
||||||
- **ExecutionContext**: Passed through all stages, carries task_id, username, env_vars, timeout
|
- **ExecutionContext**: Passed through all stages, carries task_id, username, env_vars, timeout
|
||||||
- **PrebakeStage**: Ordered enum controlling stage execution sequence (Bootstrap -> Ready)
|
- **PrebakeStage**: Ordered enum Bootstrap→EarlyHook→DepsSystem→DepsUser→LateHook→Ready
|
||||||
- **Decorator**: Attached to shell functions via `# @name(args)` comments above function declarations
|
- **Decorator**: `# @name(args)` comments above shell function declarations
|
||||||
- **Engine**: Lightweight wrapper around cgroup manager for resource isolation
|
- **Notification**: Plugin-based dispatch with batching, exponential backoff, priority groups
|
||||||
- **Dual-mode**: CLI commands (prebake, bake, finalize) vs daemon mode via Unix socket
|
- **Engine**: Delegated to `workshop-engine` crate (formerly in-tree)
|
||||||
|
- **Dual-mode**: CLI commands (standalone) vs Daemon mode (unimplemented)
|
||||||
|
|
||||||
|
## ANTI-PATTERNS
|
||||||
|
1. **Empty daemon/** directory — Daemon command is `unimplemented!()`
|
||||||
|
2. **Dead code allowed** — `lints.rust.dead_code = "allow"` in Cargo.toml
|
||||||
|
3. **Engine was extracted** — workshop-engine is its own crate now, src/engine/ removed
|
||||||
|
4. **Python test infra exists but files removed** — pytest.ini remains, test dir is empty
|
||||||
|
|||||||
@@ -12,14 +12,16 @@ bake/
|
|||||||
├── parser.rs # @decorator-aware shell script parser
|
├── parser.rs # @decorator-aware shell script parser
|
||||||
├── decorator.rs # Decorator enum + 100+ unit tests
|
├── decorator.rs # Decorator enum + 100+ unit tests
|
||||||
├── builder.rs # Script template assembly
|
├── builder.rs # Script template assembly
|
||||||
└── schedule.rs # Function ordering via @after dependencies
|
├── schedule.rs # Function ordering via @after dependencies
|
||||||
|
├── error.rs # BakeError enum
|
||||||
|
└── constant.rs # DEFAULT_WORKSPACE, TRIVIAL_SCRIPT_NAME
|
||||||
```
|
```
|
||||||
|
|
||||||
## WHERE TO LOOK
|
## WHERE TO LOOK
|
||||||
|
|
||||||
| Task | Location |
|
| Task | Location |
|
||||||
|------|----------|
|
|------|----------|
|
||||||
| Parse script | `parser.rs:29` - `parse_script()` |
|
Parse script | `parser.rs:56` - `parse_script()` |
|
||||||
| Decorator types | `decorator.rs` - `Decorator` enum variants |
|
| Decorator types | `decorator.rs` - `Decorator` enum variants |
|
||||||
| Build script | `builder.rs` - template builder |
|
| Build script | `builder.rs` - template builder |
|
||||||
| Schedule deps | `schedule.rs` - @after resolution |
|
| Schedule deps | `schedule.rs` - @after resolution |
|
||||||
|
|||||||
@@ -0,0 +1,50 @@
|
|||||||
|
use std::path::PathBuf;
|
||||||
|
|
||||||
|
use crate::cli::{BareCommands, Cli};
|
||||||
|
use crate::error::CliError;
|
||||||
|
use crate::types::resource::ResourceRegistry;
|
||||||
|
use crate::{bake::bake, finalize::finalize, prebake::prebake};
|
||||||
|
use workshop_engine::{EventSender, ExecutionContext};
|
||||||
|
fn or_workshop(path: &Option<PathBuf>, name: &str) -> PathBuf {
|
||||||
|
path
|
||||||
|
.clone()
|
||||||
|
.unwrap_or_else(|| PathBuf::from(format!(".workshop/{}", name)))
|
||||||
|
}
|
||||||
|
|
||||||
|
pub async fn run_bare(
|
||||||
|
cli: &Cli,
|
||||||
|
command: &BareCommands,
|
||||||
|
ctx: &mut ExecutionContext,
|
||||||
|
event_tx: EventSender,
|
||||||
|
pipeline: &str,
|
||||||
|
build_id: &str,
|
||||||
|
registry: &ResourceRegistry,
|
||||||
|
) -> Result<(), CliError> {
|
||||||
|
let (stage, config) = match command {
|
||||||
|
BareCommands::Prebake { .. } => ("prebake", or_workshop(&cli.prebake, "prebake.yml")),
|
||||||
|
BareCommands::Bake { .. } => ("bake", or_workshop(&cli.bake, "bake.sh")),
|
||||||
|
BareCommands::Finalize { .. } => {
|
||||||
|
("finalize", or_workshop(&cli.finalize, "finalize.yml"))
|
||||||
|
}
|
||||||
|
};
|
||||||
|
|
||||||
|
let prebake_path = or_workshop(&cli.prebake, "prebake.yml");
|
||||||
|
let bake_base = or_workshop(&cli.bake_base, "bake_base.sh");
|
||||||
|
|
||||||
|
ctx.task_id = format!("{}-{}-{}", pipeline, build_id, stage);
|
||||||
|
ctx.pipeline_name = pipeline.to_string();
|
||||||
|
ctx.standalone = true;
|
||||||
|
|
||||||
|
match command {
|
||||||
|
BareCommands::Prebake { .. } => {
|
||||||
|
prebake(&config, cli, None, ctx, event_tx, None).await?;
|
||||||
|
}
|
||||||
|
BareCommands::Bake { .. } => {
|
||||||
|
bake(&config, &bake_base, &prebake_path, cli, ctx, event_tx).await?;
|
||||||
|
}
|
||||||
|
BareCommands::Finalize { .. } => {
|
||||||
|
finalize(&config, cli, ctx, event_tx, registry).await?;
|
||||||
|
}
|
||||||
|
}
|
||||||
|
Ok(())
|
||||||
|
}
|
||||||
@@ -1,193 +0,0 @@
|
|||||||
use crate::cli::Cli;
|
|
||||||
// use serde::Serialize;
|
|
||||||
use serde_json::Value;
|
|
||||||
// use std::path::PathBuf;
|
|
||||||
// use std::sync::Arc;
|
|
||||||
use tokio::io::{AsyncBufReadExt, BufReader};
|
|
||||||
use tokio::net::UnixStream;
|
|
||||||
// use tokio::process::{Child, Command};
|
|
||||||
// use tokio::signal;
|
|
||||||
// use tokio::sync::RwLock;
|
|
||||||
// use tokio::time::{Duration, interval};
|
|
||||||
|
|
||||||
// #[derive(Debug, Clone, Serialize)]
|
|
||||||
// struct Heartbeat {
|
|
||||||
// pipeline_id: String,
|
|
||||||
// build_id: String,
|
|
||||||
// stage: String,
|
|
||||||
// timestamp: u64,
|
|
||||||
// status: String,
|
|
||||||
// subtasks: Vec<String>,
|
|
||||||
// }
|
|
||||||
|
|
||||||
pub async fn daemon(_cli: &Cli) {
|
|
||||||
todo!("Daemon should be refactored");
|
|
||||||
// let socket = PathBuf::from(settings.get_string("socket").unwrap());
|
|
||||||
// let agentsocket = settings.get_string("agentsocket").unwrap();
|
|
||||||
// let _ = std::fs::remove_file(&socket);
|
|
||||||
|
|
||||||
// let heartbeat = Arc::new(RwLock::new(Heartbeat {
|
|
||||||
// pipeline_id: cli.pipeline.clone(),
|
|
||||||
// build_id: cli.build_id.clone(),
|
|
||||||
// stage: String::new(),
|
|
||||||
// timestamp: 0,
|
|
||||||
// status: String::new(),
|
|
||||||
// subtasks: Vec::new(),
|
|
||||||
// }));
|
|
||||||
|
|
||||||
// // let heartbeat_clone = heartbeat.clone();
|
|
||||||
// let heartbeat_task = {
|
|
||||||
// let hb = heartbeat.clone();
|
|
||||||
// tokio::spawn(async move {
|
|
||||||
// let mut tick = interval(Duration::from_secs(1));
|
|
||||||
// loop {
|
|
||||||
// tick.tick().await;
|
|
||||||
// let mut hb = hb.write().await;
|
|
||||||
// hb.timestamp = std::time::SystemTime::now()
|
|
||||||
// .duration_since(std::time::UNIX_EPOCH)
|
|
||||||
// .unwrap()
|
|
||||||
// .as_millis() as u64;
|
|
||||||
|
|
||||||
// // 只输出到stdout(模拟未来发送给Agent)
|
|
||||||
// println!("[HEARTBEAT] {}", serde_json::to_string(&*hb).unwrap());
|
|
||||||
// }
|
|
||||||
// })
|
|
||||||
// };
|
|
||||||
|
|
||||||
// let socket_task = {
|
|
||||||
// let socket = socket.clone();
|
|
||||||
// tokio::spawn(async move {
|
|
||||||
// log::info!("Creating Unix socket at {}", socket.display());
|
|
||||||
// let listener =
|
|
||||||
// UnixListener::bind(&socket).expect("Failed to bind Unix socket");
|
|
||||||
|
|
||||||
// loop {
|
|
||||||
// match listener.accept().await {
|
|
||||||
// Ok((stream, _)) => {
|
|
||||||
// tokio::spawn(handle_connection(stream));
|
|
||||||
// }
|
|
||||||
// Err(e) => {
|
|
||||||
// log::error!("Accept error: {}", e);
|
|
||||||
// break;
|
|
||||||
// }
|
|
||||||
// }
|
|
||||||
// }
|
|
||||||
// })
|
|
||||||
// };
|
|
||||||
|
|
||||||
// let mut tasks: Vec<TaskSpec> = Vec::new();
|
|
||||||
|
|
||||||
// let execution_task = {
|
|
||||||
// let hb = heartbeat.clone();
|
|
||||||
// tokio::spawn(async move {
|
|
||||||
// let mut hb = hb.write().await;
|
|
||||||
// let mut children: Vec<(String, Child)> = vec![];
|
|
||||||
|
|
||||||
// let executable = std::env::current_exe().unwrap();
|
|
||||||
|
|
||||||
// for (index, task) in tasks.iter().enumerate() {
|
|
||||||
// let mut command = task.command(&executable);
|
|
||||||
// command.stdout(std::process::Stdio::piped());
|
|
||||||
// command.stderr(std::process::Stdio::piped());
|
|
||||||
// let child = command
|
|
||||||
// .spawn()
|
|
||||||
// .expect(format!("Failed to spawn child {}", index).as_str());
|
|
||||||
|
|
||||||
// let pid = child.id().unwrap_or(0).to_string();
|
|
||||||
// children.push((pid.clone(), child));
|
|
||||||
|
|
||||||
// hb.subtasks.push(pid.clone());
|
|
||||||
// }
|
|
||||||
|
|
||||||
// for (pid, mut child) in children {
|
|
||||||
// let status = child.wait().await.expect("Failed to wait child");
|
|
||||||
// log::info!("Subtask {} exited with: {}", pid, status);
|
|
||||||
|
|
||||||
// hb.subtasks.retain(|p| p != &pid);
|
|
||||||
// }
|
|
||||||
|
|
||||||
// hb.stage = "Success".to_string();
|
|
||||||
// })
|
|
||||||
// };
|
|
||||||
|
|
||||||
// tokio::select! {
|
|
||||||
// _ = heartbeat_task => {
|
|
||||||
// log::error!("Heartbeat task died");
|
|
||||||
// }
|
|
||||||
// _ = execution_task => {
|
|
||||||
// log::info!("All subtasks completed");
|
|
||||||
// }
|
|
||||||
// _ = socket_task => {
|
|
||||||
// log::error!("Socket task died");
|
|
||||||
// }
|
|
||||||
// _ = signal::ctrl_c() => {
|
|
||||||
// log::info!("Shutting down...");
|
|
||||||
// }
|
|
||||||
// }
|
|
||||||
// for pid in heartbeat.read().await.subtasks.clone() {
|
|
||||||
// let _ = Command::new("kill").arg("-TERM").arg(&pid).status().await;
|
|
||||||
// }
|
|
||||||
|
|
||||||
// let _ = std::fs::remove_file(&socket);
|
|
||||||
}
|
|
||||||
|
|
||||||
async fn _handle_connection(stream: UnixStream) {
|
|
||||||
let peer_addr = match stream.peer_addr() {
|
|
||||||
Ok(addr) => format!("{:?}", addr),
|
|
||||||
Err(_) => "unknown".to_string(),
|
|
||||||
};
|
|
||||||
|
|
||||||
log::debug!("New connection from: {}", peer_addr);
|
|
||||||
|
|
||||||
let mut reader = BufReader::new(stream);
|
|
||||||
let mut line = String::new();
|
|
||||||
|
|
||||||
loop {
|
|
||||||
line.clear();
|
|
||||||
match reader.read_line(&mut line).await {
|
|
||||||
Ok(0) => {
|
|
||||||
log::debug!("Connection closed by peer: {}", peer_addr);
|
|
||||||
break;
|
|
||||||
}
|
|
||||||
Ok(_) => {
|
|
||||||
// 简单打印原始JSON(预留字段解析空间)
|
|
||||||
match serde_json::from_str::<Value>(&line) {
|
|
||||||
Ok(json) => {
|
|
||||||
// 只提取关键字段打印(字段可能变化)
|
|
||||||
let msgtype = json
|
|
||||||
.get("msgtype")
|
|
||||||
.and_then(|v| v.as_str())
|
|
||||||
.unwrap_or("unknown");
|
|
||||||
let name = json
|
|
||||||
.get("name")
|
|
||||||
.and_then(|v| v.as_str())
|
|
||||||
.unwrap_or("unnamed");
|
|
||||||
|
|
||||||
log::info!("[{}] Message from '{}': {}", msgtype, name, line.trim());
|
|
||||||
|
|
||||||
// TODO: 字段稳定后,可扩展为结构化打印
|
|
||||||
// match msgtype {
|
|
||||||
// "ResourceUsage" => print_resource(&json),
|
|
||||||
// "Log" => print_log(&json),
|
|
||||||
// _ => log::debug!("Unknown type: {}", msgtype),
|
|
||||||
// }
|
|
||||||
}
|
|
||||||
Err(e) => {
|
|
||||||
log::warn!(
|
|
||||||
"Invalid JSON from {}: {} - Error: {}",
|
|
||||||
peer_addr,
|
|
||||||
line.trim(),
|
|
||||||
e
|
|
||||||
);
|
|
||||||
}
|
|
||||||
}
|
|
||||||
}
|
|
||||||
Err(e) => {
|
|
||||||
log::error!("Failed to read from {}: {}", peer_addr, e);
|
|
||||||
break;
|
|
||||||
}
|
|
||||||
}
|
|
||||||
}
|
|
||||||
|
|
||||||
log::debug!("Connection handler for {} exiting", peer_addr);
|
|
||||||
}
|
|
||||||
@@ -0,0 +1,30 @@
|
|||||||
|
use thiserror::Error;
|
||||||
|
use workshop_engine::HasExitCode;
|
||||||
|
|
||||||
|
#[derive(Error, Debug)]
|
||||||
|
pub enum CliError {
|
||||||
|
#[error("{0}")]
|
||||||
|
Prebake(#[from] crate::prebake::error::PrebakeError),
|
||||||
|
#[error("{0}")]
|
||||||
|
Bake(#[from] crate::bake::error::BakeError),
|
||||||
|
#[error("{0}")]
|
||||||
|
Finalize(#[from] crate::finalize::FinalizeError),
|
||||||
|
#[error("{0}")]
|
||||||
|
General(anyhow::Error),
|
||||||
|
}
|
||||||
|
|
||||||
|
impl From<anyhow::Error> for CliError {
|
||||||
|
fn from(e: anyhow::Error) -> Self {
|
||||||
|
CliError::General(e)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
impl CliError {
|
||||||
|
pub fn exit_code(&self) -> i32 {
|
||||||
|
match self {
|
||||||
|
CliError::Prebake(e) => e.exit_code(),
|
||||||
|
CliError::Bake(e) => e.exit_code(),
|
||||||
|
CliError::Finalize(e) => e.exit_code(),
|
||||||
|
CliError::General(_) => 1,
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
@@ -8,12 +8,10 @@ pub mod template;
|
|||||||
pub mod types;
|
pub mod types;
|
||||||
|
|
||||||
use crate::cli::Cli;
|
use crate::cli::Cli;
|
||||||
use crate::finalize::constant::FINALIZE_STANDALONE_PLUGIN_PATH;
|
use crate::types::resource::ResourceRegistry;
|
||||||
use crate::finalize::plugin::fetch::{self, FetchArgument};
|
|
||||||
pub use config::*;
|
pub use config::*;
|
||||||
pub use error::FinalizeError;
|
pub use error::FinalizeError;
|
||||||
use std::path::Path;
|
use std::path::Path;
|
||||||
use std::path::PathBuf;
|
|
||||||
use workshop_engine::EventSender;
|
use workshop_engine::EventSender;
|
||||||
|
|
||||||
pub fn parse(path: &Path) -> Result<FinalizeConfig, FinalizeError> {
|
pub fn parse(path: &Path) -> Result<FinalizeConfig, FinalizeError> {
|
||||||
@@ -27,6 +25,7 @@ pub async fn finalize(
|
|||||||
_cli: &Cli,
|
_cli: &Cli,
|
||||||
ctx: &mut workshop_engine::ExecutionContext,
|
ctx: &mut workshop_engine::ExecutionContext,
|
||||||
event_tx: EventSender,
|
event_tx: EventSender,
|
||||||
|
registry: &ResourceRegistry,
|
||||||
) -> Result<(), FinalizeError> {
|
) -> Result<(), FinalizeError> {
|
||||||
let mut finalize = parse(finalize_path)?;
|
let mut finalize = parse(finalize_path)?;
|
||||||
let validate_result = finalize.validate();
|
let validate_result = finalize.validate();
|
||||||
@@ -36,33 +35,10 @@ pub async fn finalize(
|
|||||||
}
|
}
|
||||||
return Err(FinalizeError::ValidateError(errors));
|
return Err(FinalizeError::ValidateError(errors));
|
||||||
};
|
};
|
||||||
if ctx.standalone {
|
|
||||||
// In standalone mode, finalize should download plugin by itself
|
|
||||||
log::debug!("Downloading {} plugins", finalize.plugin.len());
|
|
||||||
let plugin_path = PathBuf::from(FINALIZE_STANDALONE_PLUGIN_PATH);
|
|
||||||
// if plugin_path.exists() {
|
|
||||||
// std::fs::remove_dir_all(&plugin_path).map_err(FinalizeError::IoError)?;
|
|
||||||
// }
|
|
||||||
for (name, data) in finalize.plugin.iter_mut() {
|
|
||||||
log::info!("Downloading plugin {}", name);
|
|
||||||
let (source, config) = data.iter_mut().next().unwrap_or_else(|| {
|
|
||||||
panic!(
|
|
||||||
"Internal Error: Plugin {} has no source configured, this should have been caught by validation!",
|
|
||||||
name
|
|
||||||
)
|
|
||||||
});
|
|
||||||
let fetch_argument = FetchArgument {
|
|
||||||
checksum: config.checksum.clone(),
|
|
||||||
shallow: config.shallow,
|
|
||||||
};
|
|
||||||
fetch::fetch_plugin(source, name, &plugin_path, fetch_argument).await?;
|
|
||||||
config.runtime.fspath = Some(plugin_path.join(name));
|
|
||||||
}
|
|
||||||
}
|
|
||||||
|
|
||||||
// Register Plugins
|
// Register Plugins (paths resolved from registry)
|
||||||
log::info!("Registering {} plugins", finalize.plugin.len());
|
log::info!("Registering {} plugins", finalize.plugin.len());
|
||||||
let registered_plugins = plugin::register(finalize.plugin, None)?;
|
let registered_plugins = plugin::register(finalize.plugin, None, &mut finalize.notification, registry)?;
|
||||||
|
|
||||||
// Early hook
|
// Early hook
|
||||||
let earlyhook_result = stage::hook::hook(
|
let earlyhook_result = stage::hook::hook(
|
||||||
|
|||||||
@@ -1,56 +1,52 @@
|
|||||||
# workshop-baker/src/finalize
|
# workshop-baker/src/finalize
|
||||||
|
|
||||||
**Generated:** 2026-04-22
|
**Generated:** 2026-05-19
|
||||||
**Commit:** 7b3b71a
|
**Commit:** 28abc5d
|
||||||
|
|
||||||
Artifact packaging, distribution, deployment, and notifications.
|
Artifact packaging, plugin-based notifications, and post-build hooks.
|
||||||
|
|
||||||
## STRUCTURE
|
## STRUCTURE
|
||||||
|
|
||||||
```
|
```
|
||||||
finalize/
|
finalize/
|
||||||
├── config.rs # FinalizeConfig YAML (artifacts, deploy, notify)
|
├── config.rs # FinalizeConfig YAML (379 lines, plugins, notifications)
|
||||||
├── stage.rs # FinalizeStage enum
|
├── stage.rs # FinalizeStage enum
|
||||||
├── stage/hook.rs # Post-build hooks
|
├── stage/hook.rs # Post-build hook execution
|
||||||
├── event.rs # Event propagation
|
├── event.rs # Event propagation
|
||||||
├── template.rs # Variable substitution {{VAR}}
|
├── template.rs # Variable substitution {{VARIABLE}}
|
||||||
├── plugin.rs # Plugin trait + registry
|
├── plugin.rs # Plugin trait + registration
|
||||||
├── plugin/
|
├── plugin/
|
||||||
│ ├── metadata.rs # Plugin manifest parsing
|
│ ├── metadata.rs # Plugin manifest parsing
|
||||||
│ ├── dylib.rs # Dynamic library plugins
|
│ ├── dylib.rs # Dynamic library plugins
|
||||||
│ ├── rhai.rs # Rhai scripting (todo!())
|
│ ├── rhai.rs # Rhai scripting (todo!())
|
||||||
│ ├── shell.rs # Shell command plugins
|
│ ├── shell.rs # Shell command plugins
|
||||||
│ └── internal/ # Built-in plugins
|
│ └── internal.rs + internal/ # Built-in plugins (mail, webhook, satori, insitenotify)
|
||||||
│ ├── mail/ # Email notifications (SMTP)
|
├── types.rs
|
||||||
│ ├── webhook.rs # HTTP webhooks
|
├── types/compression.rs
|
||||||
│ ├── satori.rs # Satori integration
|
├── error.rs # FinalizeError
|
||||||
│ └── insitenotify.rs # In-site notifications
|
└── constant.rs
|
||||||
│ └── fetch/ # Artifact fetching
|
|
||||||
│ ├── git.rs # Git repository fetch
|
|
||||||
│ ├── http.rs # HTTP download
|
|
||||||
│ ├── extract.rs # Archive extraction
|
|
||||||
│ └── checksum.rs# SHA256 verification (SHA1 deprecated)
|
|
||||||
```
|
```
|
||||||
|
|
||||||
## WHERE TO LOOK
|
## WHERE TO LOOK
|
||||||
|
|
||||||
| Task | Location |
|
| Task | Location |
|
||||||
|------|----------|
|
|------|----------|
|
||||||
| Config loading | `config.rs` - FinalizeConfig |
|
| Config loading | `config.rs:17` - `FinalizeConfig` + `parse()` |
|
||||||
| Email notify | `plugin/internal/mail/` - SMTP templates |
|
| Plugin registration | `plugin.rs` - `register()` |
|
||||||
|
| Email notifications | `plugin/internal/mail.rs` - SMTP |
|
||||||
| Webhooks | `plugin/internal/webhook.rs` |
|
| Webhooks | `plugin/internal/webhook.rs` |
|
||||||
| Artifact fetch | `plugin/fetch/` - git/http/extract |
|
| Satori integration | `plugin/internal/satori.rs` |
|
||||||
| Checksums | `plugin/fetch/checksum.rs:13` - SHA1 deprecated |
|
| Shell plugins | `plugin/shell.rs` |
|
||||||
|
| Template rendering | `template.rs` - `{{VARIABLE}}` handlebars-style |
|
||||||
|
|
||||||
## CONVENTIONS
|
## CONVENTIONS
|
||||||
|
- **Plugin System**: `FinalizePlugin` trait + registry with both internal and dynamic plugins
|
||||||
- **Plugin System**: Dynamic + internal plugins via Plugin trait
|
- **Template Syntax**: `{{VARIABLE}}` handlebars-style substitution via `template.rs`
|
||||||
- **Template Syntax**: `{{VARIABLE}}` handlebars-style
|
- **Stages**: EarlyHook → (Artifacts) → LateHook → Notifications
|
||||||
- **Stages**: Hook → Fetch → Build → Publish → Notify → Cleanup
|
- **Notification**: `notify.rs` in parent module drives plugin-based notification dispatch
|
||||||
- **Fetch**: Supports git, http, with extraction and checksum verify
|
- **ResourceRegistry**: Shared resource cache passed through to plugins
|
||||||
|
|
||||||
## ANTI-PATTERNS
|
## ANTI-PATTERNS
|
||||||
|
1. **fetch/ subdirectory removed** — was `plugin/fetch/{git,http,extract,checksum}`, no longer exists
|
||||||
1. **SHA1 deprecated** — `checksum.rs:13` warns SHA1 is deprecated
|
2. **SHA1 deprecated** — workshop-getterurl handles checksums now
|
||||||
2. **Unimplemented** — `rhai.rs:17` - "todo!()" for Rhai scripting
|
3. **Unimplemented** — `rhai.rs:17` - "todo!()" for Rhai scripting
|
||||||
3. **TODO markers** — Multiple TODOs in config.rs (lines 305-306)
|
4. **TODO markers** — Multiple TODOs in config.rs (lines 305-306)
|
||||||
|
5. **Artifact stage not implemented** — `finalize.rs:57` — "TODO: artifact stage implementation"
|
||||||
|
|||||||
@@ -115,6 +115,17 @@ pub struct NotificationMethod {
|
|||||||
pub policy: Vec<NotifyPolicy>,
|
pub policy: Vec<NotifyPolicy>,
|
||||||
}
|
}
|
||||||
|
|
||||||
|
impl Default for NotificationMethod {
|
||||||
|
fn default() -> Self {
|
||||||
|
Self {
|
||||||
|
template: NotificationTemplate::default(),
|
||||||
|
trigger: Vec::new(),
|
||||||
|
config: Value::default(),
|
||||||
|
policy: Vec::new(),
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
#[derive(Debug, Deserialize, Serialize, Clone, Default)]
|
#[derive(Debug, Deserialize, Serialize, Clone, Default)]
|
||||||
pub struct NotificationTemplate {
|
pub struct NotificationTemplate {
|
||||||
/// Payload schema: "default" (plugin-defined) or "custom" (requires on_* templates)
|
/// Payload schema: "default" (plugin-defined) or "custom" (requires on_* templates)
|
||||||
@@ -134,17 +145,59 @@ pub struct NotificationTemplate {
|
|||||||
pub on_finish_of: Option<OnFinishOf>,
|
pub on_finish_of: Option<OnFinishOf>,
|
||||||
}
|
}
|
||||||
|
|
||||||
|
impl std::ops::AddAssign for NotificationTemplate {
|
||||||
|
fn add_assign(&mut self, other: Self) {
|
||||||
|
if self.schema.is_none() {
|
||||||
|
self.schema = other.schema;
|
||||||
|
}
|
||||||
|
if self.on_success.is_none() {
|
||||||
|
self.on_success = other.on_success;
|
||||||
|
}
|
||||||
|
if self.on_failure.is_none() {
|
||||||
|
self.on_failure = other.on_failure;
|
||||||
|
}
|
||||||
|
if self.on_finish_of.is_none() {
|
||||||
|
self.on_finish_of = other.on_finish_of;
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
/// Notification configuration container
|
/// Notification configuration container
|
||||||
#[derive(Debug, Deserialize, Serialize, Clone, Default)]
|
#[derive(Debug, Serialize, Clone, Default)]
|
||||||
pub struct NotificationConfig {
|
pub struct NotificationConfig {
|
||||||
/// Template definitions for notification content
|
/// Template definitions for notification content
|
||||||
#[serde(default)]
|
#[serde(default)]
|
||||||
pub templates: HashMap<String, NotificationTemplateDef>,
|
pub templates: HashMap<String, NotificationTemplateDef>,
|
||||||
|
|
||||||
#[serde(flatten)]
|
/// Priority groups (YAML keys like `0`, `1` are parsed as u32).
|
||||||
|
/// Custom Deserialize handles serde_yaml string→u32 conversion.
|
||||||
|
#[serde(default)]
|
||||||
pub groups: HashMap<u32, HashMap<String, NotificationMethod>>,
|
pub groups: HashMap<u32, HashMap<String, NotificationMethod>>,
|
||||||
}
|
}
|
||||||
|
|
||||||
|
/// Serde helper: YAML map keys are always strings, convert to u32.
|
||||||
|
impl<'de> serde::Deserialize<'de> for NotificationConfig {
|
||||||
|
fn deserialize<D>(deserializer: D) -> Result<Self, D::Error>
|
||||||
|
where D: serde::Deserializer<'de>
|
||||||
|
{
|
||||||
|
#[derive(serde::Deserialize)]
|
||||||
|
struct Helper {
|
||||||
|
#[serde(default)]
|
||||||
|
templates: HashMap<String, NotificationTemplateDef>,
|
||||||
|
#[serde(flatten)]
|
||||||
|
groups_raw: HashMap<String, HashMap<String, NotificationMethod>>,
|
||||||
|
}
|
||||||
|
let h = Helper::deserialize(deserializer)?;
|
||||||
|
let groups: HashMap<u32, _> = h.groups_raw.into_iter()
|
||||||
|
.map(|(k, v)| {
|
||||||
|
k.parse::<u32>().map(|n| (n, v))
|
||||||
|
.map_err(|_| serde::de::Error::custom(format!("priority key must be integer, got {:?}", k)))
|
||||||
|
})
|
||||||
|
.collect::<Result<_, _>>()?;
|
||||||
|
Ok(NotificationConfig { templates: h.templates, groups })
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
/// Stage-triggered notification configuration
|
/// Stage-triggered notification configuration
|
||||||
#[derive(Debug, Deserialize, Serialize, Clone)]
|
#[derive(Debug, Deserialize, Serialize, Clone)]
|
||||||
pub struct OnFinishOf {
|
pub struct OnFinishOf {
|
||||||
@@ -158,7 +211,7 @@ pub struct OnFinishOf {
|
|||||||
|
|
||||||
/// Notification template definition.
|
/// Notification template definition.
|
||||||
/// Supports inline content (on_success/on_failure/on_finish_of) or external reference (use).
|
/// Supports inline content (on_success/on_failure/on_finish_of) or external reference (use).
|
||||||
#[derive(Debug, Deserialize, Serialize, Clone)]
|
#[derive(Debug, Deserialize, Serialize, Clone, Default)]
|
||||||
pub struct NotificationTemplateDef {
|
pub struct NotificationTemplateDef {
|
||||||
/// External template reference (git URL).
|
/// External template reference (git URL).
|
||||||
/// Cannot be combined with on_* fields in MVP (validation will reject).
|
/// Cannot be combined with on_* fields in MVP (validation will reject).
|
||||||
@@ -178,6 +231,53 @@ pub struct NotificationTemplateDef {
|
|||||||
pub on_finish_of: Option<String>,
|
pub on_finish_of: Option<String>,
|
||||||
}
|
}
|
||||||
|
|
||||||
|
impl NotificationTemplateDef {
|
||||||
|
/// Merge `other` into `self`, keeping self's values when present.
|
||||||
|
///
|
||||||
|
/// `other` fills in gaps where `self` is `None`.
|
||||||
|
/// Chaining: `a.apply(b).apply(c)` = a wins, then b fills gaps, then c fills gaps.
|
||||||
|
pub fn apply(mut self, other: Self) -> Self {
|
||||||
|
if self.on_success.is_none() {
|
||||||
|
self.on_success = other.on_success;
|
||||||
|
}
|
||||||
|
if self.on_failure.is_none() {
|
||||||
|
self.on_failure = other.on_failure;
|
||||||
|
}
|
||||||
|
if self.on_finish_of.is_none() {
|
||||||
|
self.on_finish_of = other.on_finish_of;
|
||||||
|
}
|
||||||
|
self
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
/// Builder for [`NotificationTemplateDef`].
|
||||||
|
pub struct NotificationTemplateDefBuilder(NotificationTemplateDef);
|
||||||
|
|
||||||
|
impl NotificationTemplateDefBuilder {
|
||||||
|
pub fn new() -> Self {
|
||||||
|
Self(NotificationTemplateDef::default())
|
||||||
|
}
|
||||||
|
pub fn with_use(mut self, val: impl Into<String>) -> Self {
|
||||||
|
self.0.use_ = Some(val.into());
|
||||||
|
self
|
||||||
|
}
|
||||||
|
pub fn with_on_success(mut self, val: impl Into<String>) -> Self {
|
||||||
|
self.0.on_success = Some(val.into());
|
||||||
|
self
|
||||||
|
}
|
||||||
|
pub fn with_on_failure(mut self, val: impl Into<String>) -> Self {
|
||||||
|
self.0.on_failure = Some(val.into());
|
||||||
|
self
|
||||||
|
}
|
||||||
|
pub fn with_on_finish_of(mut self, val: impl Into<String>) -> Self {
|
||||||
|
self.0.on_finish_of = Some(val.into());
|
||||||
|
self
|
||||||
|
}
|
||||||
|
pub fn build(self) -> NotificationTemplateDef {
|
||||||
|
self.0
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
/// Notification trigger type.
|
/// Notification trigger type.
|
||||||
#[derive(Debug, Clone, PartialEq, Eq)]
|
#[derive(Debug, Clone, PartialEq, Eq)]
|
||||||
pub enum Trigger {
|
pub enum Trigger {
|
||||||
@@ -336,15 +436,18 @@ impl FinalizeConfig {
|
|||||||
fn validate_notification(&self, errors: &mut Vec<String>) {
|
fn validate_notification(&self, errors: &mut Vec<String>) {
|
||||||
// Validate templates
|
// Validate templates
|
||||||
for (name, tmpl) in &self.notification.templates {
|
for (name, tmpl) in &self.notification.templates {
|
||||||
if tmpl.use_.is_some()
|
if let Some(ref u) = tmpl.use_ {
|
||||||
&& (tmpl.on_success.is_some()
|
if u.contains("://") && (
|
||||||
|| tmpl.on_failure.is_some()
|
tmpl.on_success.is_some() ||
|
||||||
|| tmpl.on_finish_of.is_some())
|
tmpl.on_failure.is_some() ||
|
||||||
{
|
tmpl.on_finish_of.is_some()
|
||||||
errors.push(format!(
|
) {
|
||||||
"Template '{}' cannot combine 'use' with 'on_*' fields (MVP restriction)",
|
errors.push(format!(
|
||||||
name
|
"Template '{}' cannot combine external 'use' with 'on_*' fields",
|
||||||
));
|
name
|
||||||
|
));
|
||||||
|
}
|
||||||
|
// Local references (no ://) are allowed to have on_* overrides
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|||||||
@@ -1,14 +1,15 @@
|
|||||||
use crate::finalize::RawPluginMap;
|
use crate::finalize::RawPluginMap;
|
||||||
use crate::finalize::constant::{FINALIZE_PLUGIN_MANIFEST, FINALIZE_PLUGIN_MANIFEST_EXTENSION};
|
use crate::finalize::constant::{FINALIZE_PLUGIN_MANIFEST, FINALIZE_PLUGIN_MANIFEST_EXTENSION};
|
||||||
use crate::finalize::plugin::metadata::{PluginMetadata, PluginType};
|
use crate::finalize::plugin::metadata::{PluginMetadata, PluginType};
|
||||||
use crate::{EventSender, ExecutionContext, finalize::error::PluginError};
|
use workshop_engine::{EventSender, ExecutionContext};
|
||||||
|
use crate::{ finalize::error::PluginError};
|
||||||
|
use crate::types::resource::ResourceRegistry;
|
||||||
use async_trait::async_trait;
|
use async_trait::async_trait;
|
||||||
use std::collections::{HashMap, HashSet};
|
use std::collections::{HashMap, HashSet};
|
||||||
use std::path::{Path, PathBuf};
|
use std::path::{Path, PathBuf};
|
||||||
use std::sync::Arc;
|
use std::sync::Arc;
|
||||||
|
|
||||||
mod dylib;
|
mod dylib;
|
||||||
pub mod fetch;
|
|
||||||
mod internal;
|
mod internal;
|
||||||
mod metadata;
|
mod metadata;
|
||||||
mod rhai;
|
mod rhai;
|
||||||
@@ -214,6 +215,8 @@ fn get_entry(plugin_type: PluginType, name: &str) -> Result<Box<dyn AsyncPluginF
|
|||||||
pub fn register(
|
pub fn register(
|
||||||
external_plugins: RawPluginMap,
|
external_plugins: RawPluginMap,
|
||||||
list: Option<&HashSet<String>>,
|
list: Option<&HashSet<String>>,
|
||||||
|
notification_config: &mut crate::finalize::NotificationConfig,
|
||||||
|
registry: &ResourceRegistry,
|
||||||
) -> Result<PluginMap, PluginError> {
|
) -> Result<PluginMap, PluginError> {
|
||||||
let mut plugins: PluginMap = HashMap::new();
|
let mut plugins: PluginMap = HashMap::new();
|
||||||
let mut count = 0;
|
let mut count = 0;
|
||||||
@@ -255,7 +258,14 @@ pub fn register(
|
|||||||
log::warn!("Plugin {} is not available: {}", name, reason);
|
log::warn!("Plugin {} is not available: {}", name, reason);
|
||||||
continue;
|
continue;
|
||||||
}
|
}
|
||||||
let metadata = read_manifest(config.runtime.fspath, &name)?;
|
// Resolve plugin path from registry instead of config.runtime.fspath
|
||||||
|
let plugin_path = registry.resolve(&name).ok_or_else(|| {
|
||||||
|
PluginError::PluginUnavailable {
|
||||||
|
name: name.clone(),
|
||||||
|
reason: "plugin not found in resource registry".to_string(),
|
||||||
|
}
|
||||||
|
})?;
|
||||||
|
let metadata = read_manifest(Some(plugin_path), &name)?;
|
||||||
let plugin_type = get_plugin_type(metadata.plugin_type.clone(), &metadata.entrypoint);
|
let plugin_type = get_plugin_type(metadata.plugin_type.clone(), &metadata.entrypoint);
|
||||||
let entry = get_entry(plugin_type, &name)?;
|
let entry = get_entry(plugin_type, &name)?;
|
||||||
plugins.insert(
|
plugins.insert(
|
||||||
@@ -267,8 +277,18 @@ pub fn register(
|
|||||||
count += 1;
|
count += 1;
|
||||||
}
|
}
|
||||||
|
|
||||||
log::info!("Registered {} plugins", count);
|
// Inject plugin-declared templates as .fallback
|
||||||
|
for (name, plugin) in &plugins {
|
||||||
|
if let Some(ref templates) = plugin.metadata.templates {
|
||||||
|
let key = format!("{}.fallback", name);
|
||||||
|
for tmpl in templates {
|
||||||
|
notification_config.templates.insert(key.clone(), tmpl.clone());
|
||||||
|
}
|
||||||
|
log::debug!("Registered templates from plugin '{}' as '{}'", name, key);
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
log::info!("Registered {} plugins", count);
|
||||||
Ok(plugins)
|
Ok(plugins)
|
||||||
}
|
}
|
||||||
|
|
||||||
|
|||||||
@@ -1,6 +1,7 @@
|
|||||||
use crate::finalize::plugin::PluginError;
|
use crate::finalize::plugin::PluginError;
|
||||||
use crate::finalize::plugin::PluginMetadata;
|
use crate::finalize::plugin::PluginMetadata;
|
||||||
use crate::{EventSender, ExecutionContext, finalize::plugin::AsyncPluginFn};
|
use workshop_engine::{EventSender, ExecutionContext};
|
||||||
|
use crate::{ finalize::plugin::AsyncPluginFn};
|
||||||
use async_trait::async_trait;
|
use async_trait::async_trait;
|
||||||
|
|
||||||
pub struct Dylib;
|
pub struct Dylib;
|
||||||
|
|||||||
@@ -1,109 +0,0 @@
|
|||||||
// Code in this module PARTIALLY OR FULLY utilized AI Coding Agent.
|
|
||||||
use crate::finalize::fetch;
|
|
||||||
use crate::finalize::plugin::{PluginError, RawPluginMap};
|
|
||||||
use std::collections::HashSet;
|
|
||||||
use std::path::Path;
|
|
||||||
|
|
||||||
mod checksum;
|
|
||||||
mod extract;
|
|
||||||
mod git;
|
|
||||||
mod http;
|
|
||||||
mod types;
|
|
||||||
|
|
||||||
pub use types::FetchArgument;
|
|
||||||
|
|
||||||
enum URLScheme {
|
|
||||||
Unknown,
|
|
||||||
Git,
|
|
||||||
Http,
|
|
||||||
Https,
|
|
||||||
File,
|
|
||||||
}
|
|
||||||
|
|
||||||
impl URLScheme {
|
|
||||||
fn parse(url: &str) -> (Self, String) {
|
|
||||||
match url.split_once("://") {
|
|
||||||
Some((prefix, rest)) => {
|
|
||||||
let scheme = match prefix {
|
|
||||||
"git" => URLScheme::Git,
|
|
||||||
"https" => URLScheme::Https,
|
|
||||||
"http" => URLScheme::Http,
|
|
||||||
"file" => URLScheme::File,
|
|
||||||
_ => URLScheme::Unknown,
|
|
||||||
};
|
|
||||||
(scheme, rest.to_string())
|
|
||||||
}
|
|
||||||
None => (URLScheme::Unknown, url.to_string()),
|
|
||||||
}
|
|
||||||
}
|
|
||||||
}
|
|
||||||
|
|
||||||
pub async fn fetch_plugin(
|
|
||||||
url: &str,
|
|
||||||
name: &str,
|
|
||||||
basedir: &Path,
|
|
||||||
argument: FetchArgument,
|
|
||||||
) -> Result<(), PluginError> {
|
|
||||||
let (scheme, path) = URLScheme::parse(url);
|
|
||||||
|
|
||||||
match scheme {
|
|
||||||
URLScheme::Git => {
|
|
||||||
// NOTE: as a practical compromise, we use git command for git clone
|
|
||||||
// gix will be supported later, with ONLY https scheme
|
|
||||||
// TODO: Implement gix
|
|
||||||
log::info!("Fetching git plugin: {}", url);
|
|
||||||
git::fetch_git_plugin(&path, name, basedir, argument.shallow).await
|
|
||||||
}
|
|
||||||
URLScheme::Http | URLScheme::Https => {
|
|
||||||
log::info!("Fetching http plugin: {}", url);
|
|
||||||
let checksum = argument.get_checksum();
|
|
||||||
http::fetch_http_plugin(url, name, basedir, checksum).await
|
|
||||||
}
|
|
||||||
URLScheme::File => {
|
|
||||||
// TODO: Support client mode
|
|
||||||
// NOTE: In standalone mode, we just assume path is well-defined and utilize
|
|
||||||
log::info!("Fetching file plugin: {}", url);
|
|
||||||
Ok(())
|
|
||||||
}
|
|
||||||
URLScheme::Unknown => {
|
|
||||||
log::error!("Unknown URL scheme in plugin URL: {}", url);
|
|
||||||
Err(PluginError::GeneralError(format!(
|
|
||||||
"Unknown URL scheme in plugin URL: {}",
|
|
||||||
url
|
|
||||||
)))
|
|
||||||
}
|
|
||||||
}
|
|
||||||
}
|
|
||||||
|
|
||||||
pub async fn fetch_plugins(
|
|
||||||
plugin_path: &Path,
|
|
||||||
plugin_config: &mut RawPluginMap,
|
|
||||||
list: Option<&HashSet<String>>,
|
|
||||||
) -> Result<(), PluginError> {
|
|
||||||
log::info!("Preparing to download {} plugins", plugin_config.len());
|
|
||||||
for (name, data) in plugin_config.iter_mut() {
|
|
||||||
match list {
|
|
||||||
Some(list) if !list.contains(name) => {
|
|
||||||
log::debug!("Plugin {} not in list, skipping", name);
|
|
||||||
continue;
|
|
||||||
}
|
|
||||||
_ => {}
|
|
||||||
}
|
|
||||||
log::debug!("Downloading plugin {}", name);
|
|
||||||
let (source, config) =
|
|
||||||
data.iter_mut()
|
|
||||||
.next()
|
|
||||||
.ok_or_else(|| PluginError::PluginUnavailable {
|
|
||||||
name: name.clone(),
|
|
||||||
reason: "no source configured".to_string(),
|
|
||||||
})?;
|
|
||||||
let fetch_argument = FetchArgument {
|
|
||||||
checksum: config.checksum.clone(),
|
|
||||||
shallow: config.shallow,
|
|
||||||
};
|
|
||||||
fetch::fetch_plugin(source, name, plugin_path, fetch_argument).await?;
|
|
||||||
log::debug!("Storing plugin {} to {}", name, plugin_path.display());
|
|
||||||
config.runtime.fspath = Some(plugin_path.join(name));
|
|
||||||
}
|
|
||||||
Ok(())
|
|
||||||
}
|
|
||||||
@@ -1,137 +0,0 @@
|
|||||||
// Code in this module PARTIALLY OR FULLY utilized AI Coding Agent.
|
|
||||||
use super::types::ChecksumType;
|
|
||||||
use sha2::{Digest, Sha256, Sha512};
|
|
||||||
|
|
||||||
/// Detects the checksum type based on the length of the checksum string.
|
|
||||||
///
|
|
||||||
/// Returns `Ok(None)` for empty string (checksum disabled).
|
|
||||||
/// Returns `Ok(Some(ChecksumType))` for valid SHA256 (64 chars) or SHA512 (128 chars).
|
|
||||||
/// Returns `Err` for invalid lengths (32=MD5, 40=SHA1) or unsupported lengths.
|
|
||||||
pub fn detect_checksum_type(checksum: &str) -> Result<Option<ChecksumType>, String> {
|
|
||||||
match checksum.len() {
|
|
||||||
0 => Ok(None),
|
|
||||||
32 => Err(format!(
|
|
||||||
"MD5 (first 8 chars: {}) is not supported for security reasons. Use SHA256 (64 chars) or SHA512 (128 chars)",
|
|
||||||
&checksum[..8.min(checksum.len())]
|
|
||||||
)),
|
|
||||||
40 => Err(format!(
|
|
||||||
"SHA1 (first 8 chars: {}) is deprecated and not supported. Use SHA256 (64 chars) or SHA512 (128 chars)",
|
|
||||||
&checksum[..8.min(checksum.len())]
|
|
||||||
)),
|
|
||||||
64 => Ok(Some(ChecksumType::Sha256)),
|
|
||||||
128 => Ok(Some(ChecksumType::Sha512)),
|
|
||||||
_ => Err(format!(
|
|
||||||
"Invalid checksum length: {}. Expected 0 (disabled), 64 (SHA256), or 128 (SHA512). First 8 chars: {}",
|
|
||||||
checksum.len(),
|
|
||||||
&checksum[..8.min(checksum.len())]
|
|
||||||
)),
|
|
||||||
}
|
|
||||||
}
|
|
||||||
|
|
||||||
/// Verifies that the data matches the expected checksum.
|
|
||||||
///
|
|
||||||
/// Compares the computed hash of `data` against `expected` using the specified `checksum_type`.
|
|
||||||
/// Returns `Ok(())` if the checksums match, `Err` otherwise.
|
|
||||||
pub fn verify_checksum(
|
|
||||||
data: &[u8],
|
|
||||||
expected: &str,
|
|
||||||
checksum_type: ChecksumType,
|
|
||||||
) -> Result<(), String> {
|
|
||||||
let actual = match checksum_type {
|
|
||||||
ChecksumType::Sha256 => {
|
|
||||||
let mut hasher = Sha256::new();
|
|
||||||
hasher.update(data);
|
|
||||||
format!("{:x}", hasher.finalize())
|
|
||||||
}
|
|
||||||
ChecksumType::Sha512 => {
|
|
||||||
let mut hasher = Sha512::new();
|
|
||||||
hasher.update(data);
|
|
||||||
format!("{:x}", hasher.finalize())
|
|
||||||
}
|
|
||||||
};
|
|
||||||
|
|
||||||
if actual.eq_ignore_ascii_case(expected) {
|
|
||||||
Ok(())
|
|
||||||
} else {
|
|
||||||
Err(format!(
|
|
||||||
"Checksum mismatch: expected {}..., got {}...",
|
|
||||||
&expected[..8.min(expected.len())],
|
|
||||||
&actual[..8.min(actual.len())]
|
|
||||||
))
|
|
||||||
}
|
|
||||||
}
|
|
||||||
|
|
||||||
#[cfg(test)]
|
|
||||||
mod tests {
|
|
||||||
use super::*;
|
|
||||||
|
|
||||||
#[test]
|
|
||||||
fn test_detect_checksum_type_empty() {
|
|
||||||
assert_eq!(detect_checksum_type("").unwrap(), None);
|
|
||||||
}
|
|
||||||
|
|
||||||
#[test]
|
|
||||||
fn test_detect_checksum_type_md5_rejected() {
|
|
||||||
let result = detect_checksum_type("a".repeat(32).as_str());
|
|
||||||
assert!(result.is_err());
|
|
||||||
assert!(result.unwrap_err().contains("MD5"));
|
|
||||||
}
|
|
||||||
|
|
||||||
#[test]
|
|
||||||
fn test_detect_checksum_type_sha1_rejected() {
|
|
||||||
let result = detect_checksum_type("a".repeat(40).as_str());
|
|
||||||
assert!(result.is_err());
|
|
||||||
assert!(result.unwrap_err().contains("SHA1"));
|
|
||||||
}
|
|
||||||
|
|
||||||
#[test]
|
|
||||||
fn test_detect_checksum_type_sha256() {
|
|
||||||
assert_eq!(
|
|
||||||
detect_checksum_type("a".repeat(64).as_str()).unwrap(),
|
|
||||||
Some(ChecksumType::Sha256)
|
|
||||||
);
|
|
||||||
}
|
|
||||||
|
|
||||||
#[test]
|
|
||||||
fn test_detect_checksum_type_sha512() {
|
|
||||||
assert_eq!(
|
|
||||||
detect_checksum_type("a".repeat(128).as_str()).unwrap(),
|
|
||||||
Some(ChecksumType::Sha512)
|
|
||||||
);
|
|
||||||
}
|
|
||||||
|
|
||||||
#[test]
|
|
||||||
fn test_detect_checksum_type_invalid_length() {
|
|
||||||
let result = detect_checksum_type("a".repeat(50).as_str());
|
|
||||||
assert!(result.is_err());
|
|
||||||
assert!(result.unwrap_err().contains("Invalid checksum length"));
|
|
||||||
}
|
|
||||||
|
|
||||||
#[test]
|
|
||||||
fn test_verify_checksum_sha256_valid() {
|
|
||||||
let data = b"hello";
|
|
||||||
let expected = "2cf24dba5fb0a30e26e83b2ac5b9e29e1b161e5c1fa7425e73043362938b9824";
|
|
||||||
assert!(verify_checksum(data, expected, ChecksumType::Sha256).is_ok());
|
|
||||||
}
|
|
||||||
|
|
||||||
#[test]
|
|
||||||
fn test_verify_checksum_sha256_invalid() {
|
|
||||||
let data = b"hello";
|
|
||||||
let expected = "0000000000000000000000000000000000000000000000000000000000000000";
|
|
||||||
assert!(verify_checksum(data, expected, ChecksumType::Sha256).is_err());
|
|
||||||
}
|
|
||||||
|
|
||||||
#[test]
|
|
||||||
fn test_verify_checksum_sha512_valid() {
|
|
||||||
let data = b"hello";
|
|
||||||
let expected = "9b71d224bd62f3785d96d46ad3ea3d73319bfbc2890caadae2dff72519673ca72323c3d99ba5c11d7c7acc6e14b8c5da0c4663475c2e5c3adef46f73bcdec043";
|
|
||||||
assert!(verify_checksum(data, expected, ChecksumType::Sha512).is_ok());
|
|
||||||
}
|
|
||||||
|
|
||||||
#[test]
|
|
||||||
fn test_verify_checksum_sha512_invalid() {
|
|
||||||
let data = b"hello";
|
|
||||||
let expected = "0000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000";
|
|
||||||
assert!(verify_checksum(data, expected, ChecksumType::Sha512).is_err());
|
|
||||||
}
|
|
||||||
}
|
|
||||||
@@ -1,169 +0,0 @@
|
|||||||
// Code in this module PARTIALLY OR FULLY utilized AI Coding Agent.
|
|
||||||
use crate::finalize::plugin::PluginError;
|
|
||||||
use std::path::{Path, PathBuf};
|
|
||||||
use tokio::task;
|
|
||||||
|
|
||||||
/// Compression type for tar archives.
|
|
||||||
#[derive(Debug, Clone, Copy, PartialEq)]
|
|
||||||
pub enum CompressionType {
|
|
||||||
/// Gzip compressed (.tar.gz, .tgz)
|
|
||||||
Gzip,
|
|
||||||
/// Zstd compressed (.tar.zst, .tar.zstd, .tzst)
|
|
||||||
Zstd,
|
|
||||||
/// Uncompressed tar (.tar)
|
|
||||||
None,
|
|
||||||
}
|
|
||||||
|
|
||||||
impl CompressionType {
|
|
||||||
/// Detects compression type from file path extension.
|
|
||||||
///
|
|
||||||
/// Supports: .tar.gz, .tgz (Gzip), .tar.zst, .tar.zstd, .tzst (Zstd), .tar (None).
|
|
||||||
pub fn from_path(path: &std::path::Path) -> Result<Self, String> {
|
|
||||||
let name = path
|
|
||||||
.file_name()
|
|
||||||
.and_then(|n| n.to_str())
|
|
||||||
.unwrap_or("")
|
|
||||||
.to_lowercase();
|
|
||||||
|
|
||||||
if name.ends_with(".tar.gz") || name.ends_with(".tgz") {
|
|
||||||
Ok(CompressionType::Gzip)
|
|
||||||
} else if name.ends_with(".tar.zst")
|
|
||||||
|| name.ends_with(".tar.zstd")
|
|
||||||
|| name.ends_with(".tzst")
|
|
||||||
{
|
|
||||||
Ok(CompressionType::Zstd)
|
|
||||||
} else if name.ends_with(".tar") {
|
|
||||||
Ok(CompressionType::None)
|
|
||||||
} else {
|
|
||||||
Err(format!(
|
|
||||||
"Unsupported archive format: '{}'. Supported formats: .tar, .tar.gz, .tgz, .tar.zst, .tar.zstd, .tzst",
|
|
||||||
name
|
|
||||||
))
|
|
||||||
}
|
|
||||||
}
|
|
||||||
}
|
|
||||||
|
|
||||||
/// Extracts a tar archive to the destination directory.
|
|
||||||
///
|
|
||||||
/// Removes existing destination directory if present, creates a fresh directory,
|
|
||||||
/// then unpacks the archive using the appropriate decompression based on `compression`.
|
|
||||||
pub async fn extract_tar(
|
|
||||||
archive: &Path,
|
|
||||||
destdir: &PathBuf,
|
|
||||||
compression: CompressionType,
|
|
||||||
) -> Result<(), PluginError> {
|
|
||||||
if destdir.exists() {
|
|
||||||
tokio::fs::remove_dir_all(destdir).await.map_err(|e| {
|
|
||||||
PluginError::GeneralError(format!("Failed to remove existing directory: {}", e))
|
|
||||||
})?;
|
|
||||||
}
|
|
||||||
tokio::fs::create_dir_all(destdir)
|
|
||||||
.await
|
|
||||||
.map_err(|e| PluginError::GeneralError(format!("Failed to create directory: {}", e)))?;
|
|
||||||
|
|
||||||
let archive = archive.to_path_buf();
|
|
||||||
let destdir = destdir.clone();
|
|
||||||
|
|
||||||
task::spawn_blocking(move || {
|
|
||||||
let file = std::fs::File::open(&archive)
|
|
||||||
.map_err(|e| PluginError::GeneralError(format!("Failed to open archive: {}", e)))?;
|
|
||||||
|
|
||||||
match compression {
|
|
||||||
CompressionType::Gzip => {
|
|
||||||
let decoder = flate2::read::GzDecoder::new(file);
|
|
||||||
let mut archive = tar::Archive::new(decoder);
|
|
||||||
archive.unpack(&destdir).map_err(|e| {
|
|
||||||
PluginError::GeneralError(format!("Failed to unpack archive: {}", e))
|
|
||||||
})?;
|
|
||||||
}
|
|
||||||
CompressionType::Zstd => {
|
|
||||||
let decoder = zstd::stream::read::Decoder::new(file).map_err(|e| {
|
|
||||||
PluginError::GeneralError(format!("Failed to create zstd decoder: {}", e))
|
|
||||||
})?;
|
|
||||||
let mut archive = tar::Archive::new(decoder);
|
|
||||||
archive.unpack(&destdir).map_err(|e| {
|
|
||||||
PluginError::GeneralError(format!("Failed to unpack archive: {}", e))
|
|
||||||
})?;
|
|
||||||
}
|
|
||||||
CompressionType::None => {
|
|
||||||
let mut archive = tar::Archive::new(file);
|
|
||||||
archive.unpack(&destdir).map_err(|e| {
|
|
||||||
PluginError::GeneralError(format!("Failed to unpack archive: {}", e))
|
|
||||||
})?;
|
|
||||||
}
|
|
||||||
};
|
|
||||||
|
|
||||||
Ok::<(), PluginError>(())
|
|
||||||
})
|
|
||||||
.await
|
|
||||||
.map_err(|e| PluginError::GeneralError(format!("Task join error: {}", e)))??;
|
|
||||||
|
|
||||||
Ok(())
|
|
||||||
}
|
|
||||||
|
|
||||||
#[cfg(test)]
|
|
||||||
mod tests {
|
|
||||||
use super::*;
|
|
||||||
use std::path::Path;
|
|
||||||
|
|
||||||
#[test]
|
|
||||||
fn test_from_path_tar_gz() {
|
|
||||||
assert_eq!(
|
|
||||||
CompressionType::from_path(Path::new("archive.tar.gz")).unwrap(),
|
|
||||||
CompressionType::Gzip
|
|
||||||
);
|
|
||||||
}
|
|
||||||
|
|
||||||
#[test]
|
|
||||||
fn test_from_path_tgz() {
|
|
||||||
assert_eq!(
|
|
||||||
CompressionType::from_path(Path::new("archive.tgz")).unwrap(),
|
|
||||||
CompressionType::Gzip
|
|
||||||
);
|
|
||||||
}
|
|
||||||
|
|
||||||
#[test]
|
|
||||||
fn test_from_path_tar_zst() {
|
|
||||||
assert_eq!(
|
|
||||||
CompressionType::from_path(Path::new("archive.tar.zst")).unwrap(),
|
|
||||||
CompressionType::Zstd
|
|
||||||
);
|
|
||||||
}
|
|
||||||
|
|
||||||
#[test]
|
|
||||||
fn test_from_path_tar_zstd() {
|
|
||||||
assert_eq!(
|
|
||||||
CompressionType::from_path(Path::new("archive.tar.zstd")).unwrap(),
|
|
||||||
CompressionType::Zstd
|
|
||||||
);
|
|
||||||
}
|
|
||||||
|
|
||||||
#[test]
|
|
||||||
fn test_from_path_tzst() {
|
|
||||||
assert_eq!(
|
|
||||||
CompressionType::from_path(Path::new("archive.tzst")).unwrap(),
|
|
||||||
CompressionType::Zstd
|
|
||||||
);
|
|
||||||
}
|
|
||||||
|
|
||||||
#[test]
|
|
||||||
fn test_from_path_tar() {
|
|
||||||
assert_eq!(
|
|
||||||
CompressionType::from_path(Path::new("archive.tar")).unwrap(),
|
|
||||||
CompressionType::None
|
|
||||||
);
|
|
||||||
}
|
|
||||||
|
|
||||||
#[test]
|
|
||||||
fn test_from_path_unsupported() {
|
|
||||||
assert!(CompressionType::from_path(Path::new("archive.zip")).is_err());
|
|
||||||
}
|
|
||||||
|
|
||||||
#[test]
|
|
||||||
fn test_from_path_case_insensitive() {
|
|
||||||
assert_eq!(
|
|
||||||
CompressionType::from_path(Path::new("archive.TAR.GZ")).unwrap(),
|
|
||||||
CompressionType::Gzip
|
|
||||||
);
|
|
||||||
}
|
|
||||||
}
|
|
||||||
@@ -1,185 +0,0 @@
|
|||||||
// Code in this module PARTIALLY OR FULLY utilized AI Coding Agent.
|
|
||||||
use crate::finalize::plugin::PluginError;
|
|
||||||
use std::path::Path;
|
|
||||||
use tokio::process::Command;
|
|
||||||
|
|
||||||
/// Version specification for a git repository.
|
|
||||||
#[derive(Debug, PartialEq)]
|
|
||||||
pub enum GitVersion {
|
|
||||||
/// No version specified (invalid for fetch operations)
|
|
||||||
None,
|
|
||||||
/// Single version string (tag or commit hash)
|
|
||||||
Single(String),
|
|
||||||
/// Mixed: try tag first, fallback to commit hash
|
|
||||||
Mixed(String, String),
|
|
||||||
}
|
|
||||||
|
|
||||||
/// Parses a git URL with optional version specification.
|
|
||||||
///
|
|
||||||
/// Format: `url` (no version), `url@tag` (single version), `url@tag:commit` (mixed version).
|
|
||||||
/// Returns (base_url, GitVersion).
|
|
||||||
pub fn parse_git_url(url: &str) -> (String, GitVersion) {
|
|
||||||
if let Some((base, version)) = url.rsplit_once('@') {
|
|
||||||
if version.contains(":") {
|
|
||||||
let (tag, commit) = version.split_once(':').unwrap();
|
|
||||||
return (
|
|
||||||
base.to_string(),
|
|
||||||
GitVersion::Mixed(tag.to_string(), commit.to_string()),
|
|
||||||
);
|
|
||||||
}
|
|
||||||
return (base.to_string(), GitVersion::Single(version.to_string()));
|
|
||||||
}
|
|
||||||
(url.to_string(), GitVersion::None)
|
|
||||||
}
|
|
||||||
|
|
||||||
/// Clones a git repository and checks out the specified version.
|
|
||||||
///
|
|
||||||
/// Removes existing destination directory, performs shallow or full clone,
|
|
||||||
/// then checks out the given version (tag or commit).
|
|
||||||
pub async fn git_clone_and_checkout(
|
|
||||||
baseurl: &str,
|
|
||||||
version: &str,
|
|
||||||
dest: &Path,
|
|
||||||
name: &str,
|
|
||||||
shallow: bool,
|
|
||||||
) -> Result<(), PluginError> {
|
|
||||||
if dest.exists() {
|
|
||||||
tokio::fs::remove_dir_all(dest)
|
|
||||||
.await
|
|
||||||
.map_err(|e| PluginError::FetchError {
|
|
||||||
name: name.to_string(),
|
|
||||||
url: baseurl.to_string(),
|
|
||||||
reason: format!("Failed to remove existing directory: {}", e),
|
|
||||||
})?;
|
|
||||||
}
|
|
||||||
let mut cmd = Command::new("git");
|
|
||||||
cmd.arg("clone");
|
|
||||||
if shallow {
|
|
||||||
cmd.arg("--depth").arg("1");
|
|
||||||
}
|
|
||||||
cmd.arg(baseurl).arg(dest);
|
|
||||||
let output = cmd.output().await.map_err(|e| PluginError::FetchError {
|
|
||||||
name: name.to_string(),
|
|
||||||
url: baseurl.to_string(),
|
|
||||||
reason: e.to_string(),
|
|
||||||
})?;
|
|
||||||
if !output.status.success() {
|
|
||||||
return Err(PluginError::FetchError {
|
|
||||||
name: name.to_string(),
|
|
||||||
url: baseurl.to_string(),
|
|
||||||
reason: String::from_utf8_lossy(&output.stderr).to_string(),
|
|
||||||
});
|
|
||||||
}
|
|
||||||
let mut cmd = Command::new("git");
|
|
||||||
cmd.arg("checkout").arg(version).current_dir(dest);
|
|
||||||
let output = cmd.output().await.map_err(|e| PluginError::FetchError {
|
|
||||||
name: name.to_string(),
|
|
||||||
url: baseurl.to_string(),
|
|
||||||
reason: e.to_string(),
|
|
||||||
})?;
|
|
||||||
if !output.status.success() {
|
|
||||||
return Err(PluginError::FetchError {
|
|
||||||
name: name.to_string(),
|
|
||||||
url: baseurl.to_string(),
|
|
||||||
reason: String::from_utf8_lossy(&output.stderr).to_string(),
|
|
||||||
});
|
|
||||||
}
|
|
||||||
Ok(())
|
|
||||||
}
|
|
||||||
|
|
||||||
/// Fetches a plugin from a git repository.
|
|
||||||
///
|
|
||||||
/// Parses the URL for version info, then clones and checks out the appropriate ref.
|
|
||||||
/// For Mixed version, tries the tag first and falls back to the commit hash.
|
|
||||||
pub async fn fetch_git_plugin(
|
|
||||||
path: &str,
|
|
||||||
name: &str,
|
|
||||||
basedir: &Path,
|
|
||||||
shallow: Option<bool>,
|
|
||||||
) -> Result<(), PluginError> {
|
|
||||||
let (baseurl, version) = parse_git_url(path);
|
|
||||||
match version {
|
|
||||||
GitVersion::None => {
|
|
||||||
log::error!("Version should be specified for git plugin URL: {}", path);
|
|
||||||
Err(PluginError::FetchError {
|
|
||||||
name: name.to_string(),
|
|
||||||
url: path.to_string(),
|
|
||||||
reason: "Version should be specified as either a tag or a commit hash.".to_string(),
|
|
||||||
})
|
|
||||||
}
|
|
||||||
GitVersion::Single(version) => {
|
|
||||||
log::debug!("Fetching git plugin: {}@{}", name, version);
|
|
||||||
let shallow_flag = shallow.unwrap_or(false);
|
|
||||||
git_clone_and_checkout(
|
|
||||||
&baseurl,
|
|
||||||
&version.to_string(),
|
|
||||||
&basedir.join(name),
|
|
||||||
name,
|
|
||||||
shallow_flag,
|
|
||||||
)
|
|
||||||
.await
|
|
||||||
}
|
|
||||||
GitVersion::Mixed(tag, commit) => {
|
|
||||||
log::debug!("Fetching git plugin: {}@{}/{}", name, tag, commit);
|
|
||||||
let shallow_flag = shallow.unwrap_or(false);
|
|
||||||
let result =
|
|
||||||
git_clone_and_checkout(&baseurl, &tag, &basedir.join(name), name, shallow_flag)
|
|
||||||
.await;
|
|
||||||
if result.is_err() {
|
|
||||||
log::warn!(
|
|
||||||
"Failed to clone with tag, trying commit hash. URL: {}",
|
|
||||||
path
|
|
||||||
);
|
|
||||||
return git_clone_and_checkout(&baseurl, &commit, &basedir.join(name), name, false)
|
|
||||||
.await;
|
|
||||||
}
|
|
||||||
result
|
|
||||||
}
|
|
||||||
}
|
|
||||||
}
|
|
||||||
|
|
||||||
#[cfg(test)]
|
|
||||||
mod tests {
|
|
||||||
use super::*;
|
|
||||||
|
|
||||||
#[test]
|
|
||||||
fn test_parse_git_url_no_version() {
|
|
||||||
let (url, version) = parse_git_url("https://github.com/user/repo");
|
|
||||||
assert_eq!(url, "https://github.com/user/repo");
|
|
||||||
assert_eq!(version, GitVersion::None);
|
|
||||||
}
|
|
||||||
|
|
||||||
#[test]
|
|
||||||
fn test_parse_git_url_single_tag() {
|
|
||||||
let (url, version) = parse_git_url("https://github.com/user/repo@v1.0.0");
|
|
||||||
assert_eq!(url, "https://github.com/user/repo");
|
|
||||||
assert_eq!(version, GitVersion::Single("v1.0.0".to_string()));
|
|
||||||
}
|
|
||||||
|
|
||||||
#[test]
|
|
||||||
fn test_parse_git_url_mixed_version() {
|
|
||||||
let (url, version) = parse_git_url("https://github.com/user/repo@main:abc123");
|
|
||||||
assert_eq!(url, "https://github.com/user/repo");
|
|
||||||
assert_eq!(
|
|
||||||
version,
|
|
||||||
GitVersion::Mixed("main".to_string(), "abc123".to_string())
|
|
||||||
);
|
|
||||||
}
|
|
||||||
|
|
||||||
#[test]
|
|
||||||
fn test_parse_git_url_complex_url() {
|
|
||||||
let (url, version) = parse_git_url("git@github.com:user/repo.git@tag:commit");
|
|
||||||
assert_eq!(url, "git@github.com:user/repo.git");
|
|
||||||
assert_eq!(
|
|
||||||
version,
|
|
||||||
GitVersion::Mixed("tag".to_string(), "commit".to_string())
|
|
||||||
);
|
|
||||||
}
|
|
||||||
|
|
||||||
#[test]
|
|
||||||
fn test_parse_git_url_single_with_at_in_path() {
|
|
||||||
let (url, version) = parse_git_url("https://github.com/user/repo@my-repo@v1.0");
|
|
||||||
assert_eq!(url, "https://github.com/user/repo@my-repo");
|
|
||||||
assert_eq!(version, GitVersion::Single("v1.0".to_string()));
|
|
||||||
}
|
|
||||||
}
|
|
||||||
@@ -1,110 +0,0 @@
|
|||||||
// Code in this module PARTIALLY OR FULLY utilized AI Coding Agent.
|
|
||||||
use super::checksum::{detect_checksum_type, verify_checksum};
|
|
||||||
use super::extract::{CompressionType, extract_tar};
|
|
||||||
use crate::finalize::plugin::PluginError;
|
|
||||||
use std::path::Path;
|
|
||||||
|
|
||||||
pub async fn download_to_file(url: &str, dest: &Path, name: &str) -> Result<(), PluginError> {
|
|
||||||
let client = reqwest::Client::builder()
|
|
||||||
.timeout(std::time::Duration::from_secs(300))
|
|
||||||
.build()
|
|
||||||
.map_err(|e| PluginError::GeneralError(format!("Failed to build HTTP client: {}", e)))?;
|
|
||||||
|
|
||||||
let response = client
|
|
||||||
.get(url)
|
|
||||||
.send()
|
|
||||||
.await
|
|
||||||
.map_err(|e| PluginError::FetchError {
|
|
||||||
name: name.to_string(),
|
|
||||||
url: url.to_string(),
|
|
||||||
reason: format!("HTTP request failed: {}", e),
|
|
||||||
})?;
|
|
||||||
|
|
||||||
let status = response.status();
|
|
||||||
if !status.is_success() {
|
|
||||||
return Err(PluginError::FetchError {
|
|
||||||
name: name.to_string(),
|
|
||||||
url: url.to_string(),
|
|
||||||
reason: format!("HTTP error: {}", status),
|
|
||||||
});
|
|
||||||
}
|
|
||||||
|
|
||||||
let bytes = response
|
|
||||||
.bytes()
|
|
||||||
.await
|
|
||||||
.map_err(|e| PluginError::FetchError {
|
|
||||||
name: name.to_string(),
|
|
||||||
url: url.to_string(),
|
|
||||||
reason: format!("Failed to read response body: {}", e),
|
|
||||||
})?;
|
|
||||||
|
|
||||||
tokio::fs::write(dest, &bytes)
|
|
||||||
.await
|
|
||||||
.map_err(|e| PluginError::FetchError {
|
|
||||||
name: name.to_string(),
|
|
||||||
url: url.to_string(),
|
|
||||||
reason: format!("Failed to write file: {}", e),
|
|
||||||
})?;
|
|
||||||
|
|
||||||
Ok(())
|
|
||||||
}
|
|
||||||
|
|
||||||
fn extract_archive_extension(url: &str) -> &str {
|
|
||||||
let path_part = url
|
|
||||||
.split('?')
|
|
||||||
.next()
|
|
||||||
.unwrap_or(url)
|
|
||||||
.split('#')
|
|
||||||
.next()
|
|
||||||
.unwrap_or(url);
|
|
||||||
let file_part = path_part.rsplit('/').next().unwrap_or("");
|
|
||||||
let lower = file_part.to_lowercase();
|
|
||||||
if lower.ends_with(".tar.gz") || lower.ends_with(".tgz") {
|
|
||||||
".tar.gz"
|
|
||||||
} else if lower.ends_with(".tar.zst")
|
|
||||||
|| lower.ends_with(".tar.zstd")
|
|
||||||
|| lower.ends_with(".tzst")
|
|
||||||
{
|
|
||||||
".tar.zst"
|
|
||||||
} else if lower.ends_with(".tar") {
|
|
||||||
".tar"
|
|
||||||
} else {
|
|
||||||
".unknown"
|
|
||||||
}
|
|
||||||
}
|
|
||||||
|
|
||||||
pub async fn fetch_http_plugin(
|
|
||||||
url: &str,
|
|
||||||
name: &str,
|
|
||||||
basedir: &Path,
|
|
||||||
checksum_str: Option<&str>,
|
|
||||||
) -> Result<(), PluginError> {
|
|
||||||
let temp_dir = std::env::temp_dir().join("workshop-plugins");
|
|
||||||
tokio::fs::create_dir_all(&temp_dir).await.ok();
|
|
||||||
let ext = extract_archive_extension(url);
|
|
||||||
let temp_file = temp_dir.join(format!("{}-{}{}", name, uuid::Uuid::new_v4(), ext));
|
|
||||||
|
|
||||||
download_to_file(url, &temp_file, name).await?;
|
|
||||||
|
|
||||||
if let Some(checksum) = checksum_str
|
|
||||||
&& !checksum.is_empty()
|
|
||||||
{
|
|
||||||
let bytes = tokio::fs::read(&temp_file).await.map_err(|e| {
|
|
||||||
PluginError::GeneralError(format!("Failed to read downloaded file: {}", e))
|
|
||||||
})?;
|
|
||||||
|
|
||||||
let checksum_type = detect_checksum_type(checksum).map_err(PluginError::GeneralError)?;
|
|
||||||
|
|
||||||
if let Some(ct) = checksum_type {
|
|
||||||
verify_checksum(&bytes, checksum, ct).map_err(PluginError::GeneralError)?;
|
|
||||||
}
|
|
||||||
}
|
|
||||||
|
|
||||||
let compression = CompressionType::from_path(&temp_file).map_err(PluginError::GeneralError)?;
|
|
||||||
let destdir = basedir.join(name);
|
|
||||||
extract_tar(&temp_file, &destdir, compression).await?;
|
|
||||||
|
|
||||||
tokio::fs::remove_file(&temp_file).await.ok();
|
|
||||||
|
|
||||||
Ok(())
|
|
||||||
}
|
|
||||||
@@ -1,84 +0,0 @@
|
|||||||
// Code in this module PARTIALLY OR FULLY utilized AI Coding Agent.
|
|
||||||
use serde::{Deserialize, Serialize};
|
|
||||||
|
|
||||||
/// Supported checksum types for verifying downloaded artifacts.
|
|
||||||
#[derive(Debug, Clone, PartialEq)]
|
|
||||||
pub enum ChecksumType {
|
|
||||||
/// SHA-256 hash (64 hex characters)
|
|
||||||
Sha256,
|
|
||||||
/// SHA-512 hash (128 hex characters)
|
|
||||||
Sha512,
|
|
||||||
}
|
|
||||||
|
|
||||||
/// Arguments for fetching a plugin artifact.
|
|
||||||
///
|
|
||||||
/// Contains optional checksum verification and shallow clone settings.
|
|
||||||
#[derive(Debug, Deserialize, Serialize, Clone, Default)]
|
|
||||||
pub struct FetchArgument {
|
|
||||||
/// SHA256 or SHA512 hex string for integrity verification.
|
|
||||||
#[serde(default)]
|
|
||||||
pub checksum: Option<String>,
|
|
||||||
/// Perform a shallow clone (single revision only).
|
|
||||||
#[serde(default)]
|
|
||||||
pub shallow: Option<bool>,
|
|
||||||
}
|
|
||||||
|
|
||||||
impl FetchArgument {
|
|
||||||
/// Returns the checksum if present and non-empty.
|
|
||||||
pub fn get_checksum(&self) -> Option<&str> {
|
|
||||||
self.checksum.as_deref().filter(|s| !s.is_empty())
|
|
||||||
}
|
|
||||||
}
|
|
||||||
|
|
||||||
#[cfg(test)]
|
|
||||||
mod tests {
|
|
||||||
use super::*;
|
|
||||||
|
|
||||||
#[test]
|
|
||||||
fn test_get_checksum_some_non_empty() {
|
|
||||||
let arg = FetchArgument {
|
|
||||||
checksum: Some("abc123".to_string()),
|
|
||||||
shallow: None,
|
|
||||||
};
|
|
||||||
assert_eq!(arg.get_checksum(), Some("abc123"));
|
|
||||||
}
|
|
||||||
|
|
||||||
#[test]
|
|
||||||
fn test_get_checksum_empty_string() {
|
|
||||||
let arg = FetchArgument {
|
|
||||||
checksum: Some("".to_string()),
|
|
||||||
shallow: None,
|
|
||||||
};
|
|
||||||
assert_eq!(arg.get_checksum(), None);
|
|
||||||
}
|
|
||||||
|
|
||||||
#[test]
|
|
||||||
fn test_get_checksum_none() {
|
|
||||||
let arg = FetchArgument {
|
|
||||||
checksum: None,
|
|
||||||
shallow: None,
|
|
||||||
};
|
|
||||||
assert_eq!(arg.get_checksum(), None);
|
|
||||||
}
|
|
||||||
|
|
||||||
#[test]
|
|
||||||
fn test_serde_roundtrip() {
|
|
||||||
let arg = FetchArgument {
|
|
||||||
checksum: Some("deadbeef".to_string()),
|
|
||||||
shallow: Some(true),
|
|
||||||
};
|
|
||||||
let json = serde_json::to_string(&arg).unwrap();
|
|
||||||
let parsed: FetchArgument = serde_json::from_str(&json).unwrap();
|
|
||||||
assert_eq!(parsed.checksum, arg.checksum);
|
|
||||||
assert_eq!(parsed.shallow, arg.shallow);
|
|
||||||
}
|
|
||||||
|
|
||||||
#[test]
|
|
||||||
fn test_serde_roundtrip_empty() {
|
|
||||||
let arg = FetchArgument::default();
|
|
||||||
let json = serde_json::to_string(&arg).unwrap();
|
|
||||||
let parsed: FetchArgument = serde_json::from_str(&json).unwrap();
|
|
||||||
assert_eq!(parsed.checksum, arg.checksum);
|
|
||||||
assert_eq!(parsed.shallow, arg.shallow);
|
|
||||||
}
|
|
||||||
}
|
|
||||||
@@ -29,3 +29,10 @@ pub fn get_internal_plugin() -> Vec<InternalPlugin> {
|
|||||||
},
|
},
|
||||||
]
|
]
|
||||||
}
|
}
|
||||||
|
|
||||||
|
pub fn get_dummy_plugin() -> Vec<InternalPlugin> {
|
||||||
|
vec![InternalPlugin {
|
||||||
|
name: "dummy",
|
||||||
|
entry: Box::new(dummy::Dummy),
|
||||||
|
}]
|
||||||
|
}
|
||||||
|
|||||||
@@ -1,12 +1,13 @@
|
|||||||
use crate::finalize::plugin::PluginError;
|
use crate::finalize::plugin::PluginError;
|
||||||
use crate::finalize::plugin::PluginMetadata;
|
use crate::finalize::plugin::PluginMetadata;
|
||||||
use crate::{EventSender, ExecutionContext, finalize::plugin::AsyncPluginFn};
|
use workshop_engine::{EventSender, ExecutionContext};
|
||||||
|
use crate::{ finalize::plugin::AsyncPluginFn};
|
||||||
use async_trait::async_trait;
|
use async_trait::async_trait;
|
||||||
|
|
||||||
pub struct _Dummy;
|
pub struct Dummy;
|
||||||
|
|
||||||
#[async_trait]
|
#[async_trait]
|
||||||
impl AsyncPluginFn for _Dummy {
|
impl AsyncPluginFn for Dummy {
|
||||||
async fn call(
|
async fn call(
|
||||||
&self,
|
&self,
|
||||||
_metadata: &PluginMetadata,
|
_metadata: &PluginMetadata,
|
||||||
@@ -14,6 +15,7 @@ impl AsyncPluginFn for _Dummy {
|
|||||||
_ctx: &ExecutionContext,
|
_ctx: &ExecutionContext,
|
||||||
_event_tx: &EventSender,
|
_event_tx: &EventSender,
|
||||||
) -> Result<(), PluginError> {
|
) -> Result<(), PluginError> {
|
||||||
|
log::debug!("[NOTIFY_DEBUG] Successfully registered dummy plugin.");
|
||||||
Ok(())
|
Ok(())
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|||||||
@@ -1,12 +1,10 @@
|
|||||||
use crate::finalize::plugin::PluginMetadata;
|
use crate::finalize::plugin::PluginMetadata;
|
||||||
use crate::{
|
use crate::{
|
||||||
EventSender, ExecutionContext,
|
|
||||||
finalize::{error::PluginError, plugin::AsyncPluginFn},
|
finalize::{error::PluginError, plugin::AsyncPluginFn},
|
||||||
};
|
};
|
||||||
use async_trait::async_trait;
|
use async_trait::async_trait;
|
||||||
|
use workshop_engine::{EventSender, ExecutionContext};
|
||||||
pub struct InsiteNotify;
|
pub struct InsiteNotify;
|
||||||
|
|
||||||
#[async_trait]
|
#[async_trait]
|
||||||
impl AsyncPluginFn for InsiteNotify {
|
impl AsyncPluginFn for InsiteNotify {
|
||||||
async fn call(
|
async fn call(
|
||||||
@@ -16,6 +14,7 @@ impl AsyncPluginFn for InsiteNotify {
|
|||||||
_ctx: &ExecutionContext,
|
_ctx: &ExecutionContext,
|
||||||
_event_tx: &EventSender,
|
_event_tx: &EventSender,
|
||||||
) -> Result<(), PluginError> {
|
) -> Result<(), PluginError> {
|
||||||
unimplemented!("Requires workshop-base to be implemented!");
|
log::info!("in-site-notify: placeholder (always succeeds)");
|
||||||
|
Ok(())
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|||||||
@@ -1,18 +1,9 @@
|
|||||||
// Code in this module PARTIALLY OR FULLY utilized AI Coding Agent.
|
|
||||||
use crate::finalize::plugin::PluginError;
|
|
||||||
use crate::finalize::plugin::PluginMetadata;
|
use crate::finalize::plugin::PluginMetadata;
|
||||||
use crate::{EventSender, ExecutionContext, finalize::plugin::AsyncPluginFn};
|
use crate::{
|
||||||
|
finalize::{error::PluginError, plugin::AsyncPluginFn},
|
||||||
|
};
|
||||||
use async_trait::async_trait;
|
use async_trait::async_trait;
|
||||||
use lettre::message::{MultiPart, SinglePart, header};
|
use workshop_engine::{EventSender, ExecutionContext};
|
||||||
use lettre::{AsyncSmtpTransport, AsyncTransport, Message, Tokio1Executor};
|
|
||||||
use std::time::Duration;
|
|
||||||
use tokio::time::timeout;
|
|
||||||
|
|
||||||
mod config;
|
|
||||||
mod template;
|
|
||||||
|
|
||||||
pub use config::*;
|
|
||||||
use template::*;
|
|
||||||
|
|
||||||
pub struct Mail;
|
pub struct Mail;
|
||||||
|
|
||||||
@@ -21,182 +12,10 @@ impl AsyncPluginFn for Mail {
|
|||||||
async fn call(
|
async fn call(
|
||||||
&self,
|
&self,
|
||||||
_metadata: &PluginMetadata,
|
_metadata: &PluginMetadata,
|
||||||
argument: serde_yaml::Value,
|
_argument: serde_yaml::Value,
|
||||||
ctx: &ExecutionContext,
|
_ctx: &ExecutionContext,
|
||||||
_event_tx: &EventSender,
|
_event_tx: &EventSender,
|
||||||
) -> Result<(), PluginError> {
|
) -> Result<(), PluginError> {
|
||||||
let config: MailConfig = serde_yaml::from_value(argument)
|
todo!();
|
||||||
.map_err(|e| PluginError::GeneralError(format!("Invalid mail config: {}", e)))?;
|
|
||||||
|
|
||||||
if let Err(errors) = config.validate() {
|
|
||||||
return Err(PluginError::GeneralError(format!(
|
|
||||||
"Mail config validation failed: {:?}",
|
|
||||||
errors
|
|
||||||
)));
|
|
||||||
}
|
|
||||||
|
|
||||||
let template_data = prepare_template_data(ctx);
|
|
||||||
|
|
||||||
let (subject_template, body_template_html, body_template_text) = match config.schema {
|
|
||||||
Schema::Default => {
|
|
||||||
let subject = config
|
|
||||||
.subject
|
|
||||||
.as_deref()
|
|
||||||
.unwrap_or(DefaultTemplates::SUBJECT);
|
|
||||||
let body_html = config
|
|
||||||
.body
|
|
||||||
.as_deref()
|
|
||||||
.unwrap_or(DefaultTemplates::BODY_HTML);
|
|
||||||
(
|
|
||||||
subject.to_string(),
|
|
||||||
body_html.to_string(),
|
|
||||||
DefaultTemplates::BODY_TEXT.to_string(),
|
|
||||||
)
|
|
||||||
}
|
|
||||||
Schema::Custom => {
|
|
||||||
let subject = config.subject.clone().unwrap();
|
|
||||||
let body_html = config.body.clone().unwrap();
|
|
||||||
(subject, body_html.clone(), body_html)
|
|
||||||
}
|
|
||||||
};
|
|
||||||
|
|
||||||
let subject = render_template(&subject_template, &template_data)
|
|
||||||
.map_err(|e| PluginError::GeneralError(format!("Failed to render subject: {}", e)))?;
|
|
||||||
let body_html = render_template(&body_template_html, &template_data)
|
|
||||||
.map_err(|e| PluginError::GeneralError(format!("Failed to render body_html: {}", e)))?;
|
|
||||||
let body_text = render_template(&body_template_text, &template_data).map_err(|e| {
|
|
||||||
PluginError::GeneralError(format!("Failed to render body_text(?): {}", e))
|
|
||||||
})?;
|
|
||||||
|
|
||||||
let email = build_email(&config, subject, body_html, body_text)?;
|
|
||||||
|
|
||||||
send_email(&config, email).await
|
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
fn build_email(
|
|
||||||
config: &MailConfig,
|
|
||||||
subject: String,
|
|
||||||
body_html: String,
|
|
||||||
body_text: String,
|
|
||||||
) -> Result<Message, PluginError> {
|
|
||||||
let from_str = config.from.to_string();
|
|
||||||
let from: lettre::message::Mailbox = from_str.parse().map_err(|e| {
|
|
||||||
PluginError::GeneralError(format!("Invalid from address '{}': {}", from_str, e))
|
|
||||||
})?;
|
|
||||||
|
|
||||||
let to_addrs: Vec<String> = config.to.to_vec();
|
|
||||||
if to_addrs.is_empty() {
|
|
||||||
return Err(PluginError::GeneralError(
|
|
||||||
"No recipients specified".to_string(),
|
|
||||||
));
|
|
||||||
}
|
|
||||||
|
|
||||||
let mut message_builder = Message::builder().from(from).subject(subject);
|
|
||||||
|
|
||||||
for addr in &to_addrs {
|
|
||||||
let mailbox: lettre::message::Mailbox = addr.parse().map_err(|e| {
|
|
||||||
PluginError::GeneralError(format!("Invalid to address '{}': {}", addr, e))
|
|
||||||
})?;
|
|
||||||
message_builder = message_builder.to(mailbox);
|
|
||||||
}
|
|
||||||
|
|
||||||
if let Some(cc) = &config.cc {
|
|
||||||
for addr in cc.to_vec() {
|
|
||||||
let mailbox: lettre::message::Mailbox = addr.parse().map_err(|e| {
|
|
||||||
PluginError::GeneralError(format!("Invalid cc address '{}': {}", addr, e))
|
|
||||||
})?;
|
|
||||||
message_builder = message_builder.cc(mailbox);
|
|
||||||
}
|
|
||||||
}
|
|
||||||
|
|
||||||
if let Some(bcc) = &config.bcc {
|
|
||||||
for addr in bcc.to_vec() {
|
|
||||||
let mailbox: lettre::message::Mailbox = addr.parse().map_err(|e| {
|
|
||||||
PluginError::GeneralError(format!("Invalid bcc address '{}': {}", addr, e))
|
|
||||||
})?;
|
|
||||||
message_builder = message_builder.bcc(mailbox);
|
|
||||||
}
|
|
||||||
}
|
|
||||||
|
|
||||||
if let Some(reply_to) = &config.reply_to {
|
|
||||||
let mailbox: lettre::message::Mailbox = reply_to.parse().map_err(|e| {
|
|
||||||
PluginError::GeneralError(format!("Invalid reply-to address '{}': {}", reply_to, e))
|
|
||||||
})?;
|
|
||||||
message_builder = message_builder.reply_to(mailbox);
|
|
||||||
}
|
|
||||||
|
|
||||||
let message = message_builder
|
|
||||||
.multipart(
|
|
||||||
MultiPart::alternative()
|
|
||||||
.singlepart(
|
|
||||||
SinglePart::builder()
|
|
||||||
.header(header::ContentType::TEXT_PLAIN)
|
|
||||||
.body(body_text),
|
|
||||||
)
|
|
||||||
.singlepart(
|
|
||||||
SinglePart::builder()
|
|
||||||
.header(header::ContentType::TEXT_HTML)
|
|
||||||
.body(body_html),
|
|
||||||
),
|
|
||||||
)
|
|
||||||
.map_err(|e| PluginError::GeneralError(format!("Failed to build email: {}", e)))?;
|
|
||||||
|
|
||||||
Ok(message)
|
|
||||||
}
|
|
||||||
|
|
||||||
async fn send_email(config: &MailConfig, email: Message) -> Result<(), PluginError> {
|
|
||||||
let connect_timeout = if config.smtp.connect_timeout_ms == 0 {
|
|
||||||
None
|
|
||||||
} else {
|
|
||||||
Some(Duration::from_millis(config.smtp.connect_timeout_ms))
|
|
||||||
};
|
|
||||||
|
|
||||||
let mut transport_builder = match config.smtp.encryption {
|
|
||||||
Encryption::Tls => {
|
|
||||||
log::debug!("Using TLS encryption");
|
|
||||||
AsyncSmtpTransport::<Tokio1Executor>::relay(&config.smtp.host)
|
|
||||||
.map_err(|e| PluginError::GeneralError(format!("Invalid SMTP host: {}", e)))?
|
|
||||||
}
|
|
||||||
Encryption::Starttls => {
|
|
||||||
log::debug!("Using StartTLS encryption");
|
|
||||||
AsyncSmtpTransport::<Tokio1Executor>::starttls_relay(&config.smtp.host)
|
|
||||||
.map_err(|e| PluginError::GeneralError(format!("Invalid SMTP host: {}", e)))?
|
|
||||||
}
|
|
||||||
Encryption::None => {
|
|
||||||
log::debug!("Using NO encryption");
|
|
||||||
AsyncSmtpTransport::<Tokio1Executor>::builder_dangerous(&config.smtp.host)
|
|
||||||
}
|
|
||||||
};
|
|
||||||
|
|
||||||
transport_builder = transport_builder.port(config.smtp.port);
|
|
||||||
|
|
||||||
match config.smtp.auth.auth_type {
|
|
||||||
AuthType::Password => {
|
|
||||||
log::debug!("Using password auth");
|
|
||||||
transport_builder = transport_builder.credentials(
|
|
||||||
(
|
|
||||||
config.smtp.auth.username.clone(),
|
|
||||||
config.smtp.auth.password.clone(),
|
|
||||||
)
|
|
||||||
.into(),
|
|
||||||
);
|
|
||||||
}
|
|
||||||
AuthType::None => {
|
|
||||||
log::debug!("Using no auth");
|
|
||||||
}
|
|
||||||
}
|
|
||||||
|
|
||||||
let transport = transport_builder.build();
|
|
||||||
|
|
||||||
let result = match connect_timeout {
|
|
||||||
Some(t) => timeout(t, transport.send(email)).await,
|
|
||||||
None => Ok(transport.send(email).await),
|
|
||||||
};
|
|
||||||
|
|
||||||
result
|
|
||||||
.map_err(|_| PluginError::GeneralError("SMTP connection timed out".to_string()))?
|
|
||||||
.map_err(|e| PluginError::GeneralError(format!("SMTP error: {}", e)))?;
|
|
||||||
|
|
||||||
Ok(())
|
|
||||||
}
|
|
||||||
|
|||||||
@@ -1,490 +0,0 @@
|
|||||||
// Code in this module PARTIALLY OR FULLY utilized AI Coding Agent.
|
|
||||||
use serde::Deserialize;
|
|
||||||
use workshop_engine::deserialize_duration_ms;
|
|
||||||
|
|
||||||
/// Email address representation supporting plain string or structured object.
|
|
||||||
#[derive(Debug, Deserialize, Clone)]
|
|
||||||
#[serde(untagged)]
|
|
||||||
pub enum EmailAddress {
|
|
||||||
/// Plain email address string.
|
|
||||||
String(String),
|
|
||||||
/// Structured email with display name.
|
|
||||||
Object { name: String, email: String },
|
|
||||||
}
|
|
||||||
|
|
||||||
impl EmailAddress {
|
|
||||||
/// Returns the formatted address string (e.g., "user@example.com" or "Name <user@example.com>").
|
|
||||||
pub fn to_string(&self) -> String {
|
|
||||||
match self {
|
|
||||||
EmailAddress::String(email) => email.clone(),
|
|
||||||
EmailAddress::Object { name, email } => {
|
|
||||||
if name.is_empty() {
|
|
||||||
email.clone()
|
|
||||||
} else {
|
|
||||||
format!("{} <{}>", name, email)
|
|
||||||
}
|
|
||||||
}
|
|
||||||
}
|
|
||||||
}
|
|
||||||
|
|
||||||
/// Returns the raw email address without formatting.
|
|
||||||
pub fn email(&self) -> &str {
|
|
||||||
match self {
|
|
||||||
EmailAddress::String(email) => email.as_str(),
|
|
||||||
EmailAddress::Object { email, .. } => email.as_str(),
|
|
||||||
}
|
|
||||||
}
|
|
||||||
}
|
|
||||||
|
|
||||||
/// Recipients container supporting single or multiple addresses.
|
|
||||||
#[derive(Debug, Deserialize, Clone)]
|
|
||||||
#[serde(untagged)]
|
|
||||||
pub enum Recipients {
|
|
||||||
/// Single recipient.
|
|
||||||
Single(String),
|
|
||||||
/// Multiple recipients.
|
|
||||||
Multiple(Vec<String>),
|
|
||||||
}
|
|
||||||
|
|
||||||
impl Recipients {
|
|
||||||
/// Converts recipients to a flat vector of email strings.
|
|
||||||
pub fn to_vec(&self) -> Vec<String> {
|
|
||||||
match self {
|
|
||||||
Recipients::Single(s) => vec![s.clone()],
|
|
||||||
Recipients::Multiple(v) => v.clone(),
|
|
||||||
}
|
|
||||||
}
|
|
||||||
}
|
|
||||||
|
|
||||||
/// SMTP authentication mechanism.
|
|
||||||
#[derive(Debug, Deserialize, Clone)]
|
|
||||||
#[serde(rename_all = "lowercase")]
|
|
||||||
pub enum AuthType {
|
|
||||||
/// No authentication.
|
|
||||||
None,
|
|
||||||
/// Password-based authentication.
|
|
||||||
Password,
|
|
||||||
}
|
|
||||||
|
|
||||||
/// SMTP authentication configuration.
|
|
||||||
#[derive(Debug, Deserialize, Clone)]
|
|
||||||
pub struct AuthConfig {
|
|
||||||
/// Authentication mechanism type.
|
|
||||||
#[serde(rename = "type")]
|
|
||||||
pub auth_type: AuthType,
|
|
||||||
/// Username for authentication.
|
|
||||||
pub username: String,
|
|
||||||
/// Password for authentication.
|
|
||||||
pub password: String,
|
|
||||||
}
|
|
||||||
|
|
||||||
/// SMTP connection encryption mode.
|
|
||||||
#[derive(Debug, Deserialize, Clone)]
|
|
||||||
#[serde(rename_all = "lowercase")]
|
|
||||||
pub enum Encryption {
|
|
||||||
/// Implicit TLS/SSL on connection.
|
|
||||||
Tls,
|
|
||||||
/// STARTTLS upgrade after initial plaintext connection.
|
|
||||||
Starttls,
|
|
||||||
/// No encryption.
|
|
||||||
None,
|
|
||||||
}
|
|
||||||
|
|
||||||
/// SMTP server connection configuration.
|
|
||||||
#[derive(Debug, Deserialize, Clone)]
|
|
||||||
pub struct SmtpConfig {
|
|
||||||
/// SMTP server hostname.
|
|
||||||
pub host: String,
|
|
||||||
/// SMTP server port.
|
|
||||||
pub port: u16,
|
|
||||||
/// Authentication configuration.
|
|
||||||
pub auth: AuthConfig,
|
|
||||||
/// Encryption mode.
|
|
||||||
pub encryption: Encryption,
|
|
||||||
/// Connection timeout in milliseconds.
|
|
||||||
#[serde(
|
|
||||||
default = "default_connect_timeout_ms",
|
|
||||||
deserialize_with = "deserialize_duration_ms"
|
|
||||||
)]
|
|
||||||
pub connect_timeout_ms: u64,
|
|
||||||
}
|
|
||||||
|
|
||||||
fn default_connect_timeout_ms() -> u64 {
|
|
||||||
30_000
|
|
||||||
}
|
|
||||||
|
|
||||||
/// Mail schema type determining required fields.
|
|
||||||
#[derive(Debug, Deserialize, Clone)]
|
|
||||||
#[serde(rename_all = "lowercase")]
|
|
||||||
pub enum Schema {
|
|
||||||
/// Default schema with minimal required fields.
|
|
||||||
Default,
|
|
||||||
/// Custom schema requiring explicit subject and body.
|
|
||||||
Custom,
|
|
||||||
}
|
|
||||||
|
|
||||||
fn default_schema() -> Schema {
|
|
||||||
Schema::Default
|
|
||||||
}
|
|
||||||
|
|
||||||
/// Complete mail notification configuration.
|
|
||||||
#[derive(Debug, Deserialize, Clone)]
|
|
||||||
pub struct MailConfig {
|
|
||||||
/// Primary recipients.
|
|
||||||
pub to: Recipients,
|
|
||||||
/// Carbon copy recipients.
|
|
||||||
#[serde(default)]
|
|
||||||
pub cc: Option<Recipients>,
|
|
||||||
/// Blind carbon copy recipients.
|
|
||||||
#[serde(default)]
|
|
||||||
pub bcc: Option<Recipients>,
|
|
||||||
/// Sender address.
|
|
||||||
pub from: EmailAddress,
|
|
||||||
/// Reply-to address.
|
|
||||||
#[serde(default)]
|
|
||||||
pub reply_to: Option<String>,
|
|
||||||
/// Schema type determining validation rules.
|
|
||||||
#[serde(default = "default_schema")]
|
|
||||||
pub schema: Schema,
|
|
||||||
/// Email subject (required for custom schema).
|
|
||||||
pub subject: Option<String>,
|
|
||||||
/// Email body content (required for custom schema).
|
|
||||||
pub body: Option<String>,
|
|
||||||
/// SMTP server configuration.
|
|
||||||
pub smtp: SmtpConfig,
|
|
||||||
}
|
|
||||||
|
|
||||||
impl MailConfig {
|
|
||||||
/// Validates the mail configuration according to schema rules.
|
|
||||||
/// Returns Ok(()) if valid, or Err(vec) with error messages if invalid.
|
|
||||||
pub fn validate(&self) -> Result<(), Vec<String>> {
|
|
||||||
let mut errors = Vec::new();
|
|
||||||
|
|
||||||
if matches!(self.schema, Schema::Custom) {
|
|
||||||
if self.subject.is_none() {
|
|
||||||
errors.push("subject: required when schema=custom".to_string());
|
|
||||||
}
|
|
||||||
if self.body.is_none() {
|
|
||||||
errors.push("body: required when schema=custom".to_string());
|
|
||||||
}
|
|
||||||
}
|
|
||||||
|
|
||||||
if self.to.to_vec().is_empty() {
|
|
||||||
errors.push("to: must have at least one recipient".to_string());
|
|
||||||
}
|
|
||||||
|
|
||||||
if errors.is_empty() {
|
|
||||||
Ok(())
|
|
||||||
} else {
|
|
||||||
Err(errors)
|
|
||||||
}
|
|
||||||
}
|
|
||||||
}
|
|
||||||
|
|
||||||
/// Built-in email notification templates.
|
|
||||||
pub struct DefaultTemplates;
|
|
||||||
|
|
||||||
impl DefaultTemplates {
|
|
||||||
/// Default subject template with build status.
|
|
||||||
pub const SUBJECT: &'static str =
|
|
||||||
"[{{ build.status | upper }}] {{ pipeline.name }} #{{ build.id }}";
|
|
||||||
|
|
||||||
/// Default HTML email body template.
|
|
||||||
pub const BODY_HTML: &'static str = r#"<!DOCTYPE html>
|
|
||||||
<html>
|
|
||||||
<body style="font-family: Arial, sans-serif;">
|
|
||||||
<h2 style="color: {{ build.status_color }};">
|
|
||||||
Build {{ build.status | title }}
|
|
||||||
</h2>
|
|
||||||
<table>
|
|
||||||
<tr><td><strong>Pipeline:</strong></td><td>{{ pipeline.name }}</td></tr>
|
|
||||||
<tr><td><strong>Build ID:</strong></td><td>{{ build.id }}</td></tr>
|
|
||||||
<tr><td><strong>Status:</strong></td><td>{{ build.status }}</td></tr>
|
|
||||||
<tr><td><strong>Duration:</strong></td><td>{{ build.duration }}</td></tr>
|
|
||||||
<tr><td><strong>Author:</strong></td><td>{{ commit.author }}</td></tr>
|
|
||||||
<tr><td><strong>Message:</strong></td><td>{{ commit.message }}</td></tr>
|
|
||||||
</table>
|
|
||||||
<p>
|
|
||||||
<a href="{{ build.url }}" style="background: #007bff; color: white; padding: 10px 20px; text-decoration: none; border-radius: 5px;">
|
|
||||||
View Build Details
|
|
||||||
</a>
|
|
||||||
</p>
|
|
||||||
</body>
|
|
||||||
</html>"#;
|
|
||||||
|
|
||||||
/// Default plain text email body template.
|
|
||||||
pub const BODY_TEXT: &'static str = r#"Build {{ build.status | title }}
|
|
||||||
==============================
|
|
||||||
Pipeline: {{ pipeline.name }}
|
|
||||||
Build ID: {{ build.id }}
|
|
||||||
Status: {{ build.status }}
|
|
||||||
Duration: {{ build.duration }}
|
|
||||||
Commit: {{ commit.hash }}
|
|
||||||
Author: {{ commit.author }}
|
|
||||||
Message: {{ commit.message }}
|
|
||||||
|
|
||||||
View details: {{ build.url }}"#;
|
|
||||||
}
|
|
||||||
|
|
||||||
#[cfg(test)]
|
|
||||||
mod tests {
|
|
||||||
use super::*;
|
|
||||||
|
|
||||||
#[test]
|
|
||||||
fn test_email_address_string_to_string() {
|
|
||||||
let addr = EmailAddress::String("test@example.com".to_string());
|
|
||||||
assert_eq!(addr.to_string(), "test@example.com");
|
|
||||||
}
|
|
||||||
|
|
||||||
#[test]
|
|
||||||
fn test_email_address_object_with_name_to_string() {
|
|
||||||
let addr = EmailAddress::Object {
|
|
||||||
name: "Test User".to_string(),
|
|
||||||
email: "test@example.com".to_string(),
|
|
||||||
};
|
|
||||||
assert_eq!(addr.to_string(), "Test User <test@example.com>");
|
|
||||||
}
|
|
||||||
|
|
||||||
#[test]
|
|
||||||
fn test_email_address_object_without_name_to_string() {
|
|
||||||
let addr = EmailAddress::Object {
|
|
||||||
name: "".to_string(),
|
|
||||||
email: "test@example.com".to_string(),
|
|
||||||
};
|
|
||||||
assert_eq!(addr.to_string(), "test@example.com");
|
|
||||||
}
|
|
||||||
|
|
||||||
#[test]
|
|
||||||
fn test_email_address_string_email() {
|
|
||||||
let addr = EmailAddress::String("test@example.com".to_string());
|
|
||||||
assert_eq!(addr.email(), "test@example.com");
|
|
||||||
}
|
|
||||||
|
|
||||||
#[test]
|
|
||||||
fn test_email_address_object_email() {
|
|
||||||
let addr = EmailAddress::Object {
|
|
||||||
name: "Test User".to_string(),
|
|
||||||
email: "test@example.com".to_string(),
|
|
||||||
};
|
|
||||||
assert_eq!(addr.email(), "test@example.com");
|
|
||||||
}
|
|
||||||
|
|
||||||
#[test]
|
|
||||||
fn test_recipients_single_to_vec() {
|
|
||||||
let recipients = Recipients::Single("test@example.com".to_string());
|
|
||||||
let vec = recipients.to_vec();
|
|
||||||
assert_eq!(vec, vec!["test@example.com"]);
|
|
||||||
}
|
|
||||||
|
|
||||||
#[test]
|
|
||||||
fn test_recipients_multiple_to_vec() {
|
|
||||||
let recipients = Recipients::Multiple(vec![
|
|
||||||
"a@example.com".to_string(),
|
|
||||||
"b@example.com".to_string(),
|
|
||||||
]);
|
|
||||||
let vec = recipients.to_vec();
|
|
||||||
assert_eq!(vec, vec!["a@example.com", "b@example.com"]);
|
|
||||||
}
|
|
||||||
|
|
||||||
#[test]
|
|
||||||
fn test_mail_config_validate_valid_default_schema() {
|
|
||||||
let config = MailConfig {
|
|
||||||
to: Recipients::Single("to@example.com".to_string()),
|
|
||||||
cc: None,
|
|
||||||
bcc: None,
|
|
||||||
from: EmailAddress::String("from@example.com".to_string()),
|
|
||||||
reply_to: None,
|
|
||||||
schema: Schema::Default,
|
|
||||||
subject: None,
|
|
||||||
body: None,
|
|
||||||
smtp: SmtpConfig {
|
|
||||||
host: "smtp.example.com".to_string(),
|
|
||||||
port: 587,
|
|
||||||
auth: AuthConfig {
|
|
||||||
auth_type: AuthType::None,
|
|
||||||
username: "".to_string(),
|
|
||||||
password: "".to_string(),
|
|
||||||
},
|
|
||||||
encryption: Encryption::Tls,
|
|
||||||
connect_timeout_ms: 30_000,
|
|
||||||
},
|
|
||||||
};
|
|
||||||
assert!(config.validate().is_ok());
|
|
||||||
}
|
|
||||||
|
|
||||||
#[test]
|
|
||||||
fn test_mail_config_validate_valid_custom_schema() {
|
|
||||||
let config = MailConfig {
|
|
||||||
to: Recipients::Single("to@example.com".to_string()),
|
|
||||||
cc: None,
|
|
||||||
bcc: None,
|
|
||||||
from: EmailAddress::String("from@example.com".to_string()),
|
|
||||||
reply_to: None,
|
|
||||||
schema: Schema::Custom,
|
|
||||||
subject: Some("Test Subject".to_string()),
|
|
||||||
body: Some("Test Body".to_string()),
|
|
||||||
smtp: SmtpConfig {
|
|
||||||
host: "smtp.example.com".to_string(),
|
|
||||||
port: 587,
|
|
||||||
auth: AuthConfig {
|
|
||||||
auth_type: AuthType::None,
|
|
||||||
username: "".to_string(),
|
|
||||||
password: "".to_string(),
|
|
||||||
},
|
|
||||||
encryption: Encryption::Tls,
|
|
||||||
connect_timeout_ms: 30_000,
|
|
||||||
},
|
|
||||||
};
|
|
||||||
assert!(config.validate().is_ok());
|
|
||||||
}
|
|
||||||
|
|
||||||
#[test]
|
|
||||||
fn test_mail_config_validate_custom_schema_missing_subject() {
|
|
||||||
let config = MailConfig {
|
|
||||||
to: Recipients::Single("to@example.com".to_string()),
|
|
||||||
cc: None,
|
|
||||||
bcc: None,
|
|
||||||
from: EmailAddress::String("from@example.com".to_string()),
|
|
||||||
reply_to: None,
|
|
||||||
schema: Schema::Custom,
|
|
||||||
subject: None,
|
|
||||||
body: Some("Test Body".to_string()),
|
|
||||||
smtp: SmtpConfig {
|
|
||||||
host: "smtp.example.com".to_string(),
|
|
||||||
port: 587,
|
|
||||||
auth: AuthConfig {
|
|
||||||
auth_type: AuthType::None,
|
|
||||||
username: "".to_string(),
|
|
||||||
password: "".to_string(),
|
|
||||||
},
|
|
||||||
encryption: Encryption::Tls,
|
|
||||||
connect_timeout_ms: 30_000,
|
|
||||||
},
|
|
||||||
};
|
|
||||||
let result = config.validate();
|
|
||||||
assert!(result.is_err());
|
|
||||||
assert!(result.unwrap_err().iter().any(|e| e.contains("subject")));
|
|
||||||
}
|
|
||||||
|
|
||||||
#[test]
|
|
||||||
fn test_mail_config_validate_custom_schema_missing_body() {
|
|
||||||
let config = MailConfig {
|
|
||||||
to: Recipients::Single("to@example.com".to_string()),
|
|
||||||
cc: None,
|
|
||||||
bcc: None,
|
|
||||||
from: EmailAddress::String("from@example.com".to_string()),
|
|
||||||
reply_to: None,
|
|
||||||
schema: Schema::Custom,
|
|
||||||
subject: Some("Test Subject".to_string()),
|
|
||||||
body: None,
|
|
||||||
smtp: SmtpConfig {
|
|
||||||
host: "smtp.example.com".to_string(),
|
|
||||||
port: 587,
|
|
||||||
auth: AuthConfig {
|
|
||||||
auth_type: AuthType::None,
|
|
||||||
username: "".to_string(),
|
|
||||||
password: "".to_string(),
|
|
||||||
},
|
|
||||||
encryption: Encryption::Tls,
|
|
||||||
connect_timeout_ms: 30_000,
|
|
||||||
},
|
|
||||||
};
|
|
||||||
let result = config.validate();
|
|
||||||
assert!(result.is_err());
|
|
||||||
assert!(result.unwrap_err().iter().any(|e| e.contains("body")));
|
|
||||||
}
|
|
||||||
|
|
||||||
#[test]
|
|
||||||
fn test_mail_config_validate_empty_recipients() {
|
|
||||||
let config = MailConfig {
|
|
||||||
to: Recipients::Multiple(vec![]),
|
|
||||||
cc: None,
|
|
||||||
bcc: None,
|
|
||||||
from: EmailAddress::String("from@example.com".to_string()),
|
|
||||||
reply_to: None,
|
|
||||||
schema: Schema::Default,
|
|
||||||
subject: None,
|
|
||||||
body: None,
|
|
||||||
smtp: SmtpConfig {
|
|
||||||
host: "smtp.example.com".to_string(),
|
|
||||||
port: 587,
|
|
||||||
auth: AuthConfig {
|
|
||||||
auth_type: AuthType::None,
|
|
||||||
username: "".to_string(),
|
|
||||||
password: "".to_string(),
|
|
||||||
},
|
|
||||||
encryption: Encryption::Tls,
|
|
||||||
connect_timeout_ms: 30_000,
|
|
||||||
},
|
|
||||||
};
|
|
||||||
let result = config.validate();
|
|
||||||
assert!(result.is_err());
|
|
||||||
assert!(result.unwrap_err().iter().any(|e| e.contains("recipient")));
|
|
||||||
}
|
|
||||||
|
|
||||||
#[test]
|
|
||||||
fn test_email_address_serde_string() {
|
|
||||||
let json = r#""test@example.com""#;
|
|
||||||
let addr: EmailAddress = serde_json::from_str(json).unwrap();
|
|
||||||
assert!(matches!(addr, EmailAddress::String(s) if s == "test@example.com"));
|
|
||||||
}
|
|
||||||
|
|
||||||
#[test]
|
|
||||||
fn test_email_address_serde_object() {
|
|
||||||
let json = r#"{"name":"Test User","email":"test@example.com"}"#;
|
|
||||||
let addr: EmailAddress = serde_json::from_str(json).unwrap();
|
|
||||||
assert!(
|
|
||||||
matches!(addr, EmailAddress::Object { name, email } if name == "Test User" && email == "test@example.com")
|
|
||||||
);
|
|
||||||
}
|
|
||||||
|
|
||||||
#[test]
|
|
||||||
fn test_recipients_serde_single() {
|
|
||||||
let json = r#""single@example.com""#;
|
|
||||||
let recipients: Recipients = serde_json::from_str(json).unwrap();
|
|
||||||
assert!(matches!(recipients, Recipients::Single(s) if s == "single@example.com"));
|
|
||||||
}
|
|
||||||
|
|
||||||
#[test]
|
|
||||||
fn test_recipients_serde_multiple() {
|
|
||||||
let json = r#"["a@example.com","b@example.com"]"#;
|
|
||||||
let recipients: Recipients = serde_json::from_str(json).unwrap();
|
|
||||||
assert!(
|
|
||||||
matches!(recipients, Recipients::Multiple(v) if v == vec!["a@example.com".to_string(), "b@example.com".to_string()])
|
|
||||||
);
|
|
||||||
}
|
|
||||||
|
|
||||||
#[test]
|
|
||||||
fn test_auth_type_serde() {
|
|
||||||
#[derive(serde::Deserialize, Debug)]
|
|
||||||
struct Test {
|
|
||||||
#[serde(rename = "type")]
|
|
||||||
auth_type: AuthType,
|
|
||||||
}
|
|
||||||
let json = r#"{"type":"password"}"#;
|
|
||||||
let t: Test = serde_json::from_str(json).unwrap();
|
|
||||||
assert!(matches!(t.auth_type, AuthType::Password));
|
|
||||||
}
|
|
||||||
|
|
||||||
#[test]
|
|
||||||
fn test_encryption_serde() {
|
|
||||||
#[derive(serde::Deserialize, Debug)]
|
|
||||||
struct Test {
|
|
||||||
encryption: Encryption,
|
|
||||||
}
|
|
||||||
let json = r#"{"encryption":"tls"}"#;
|
|
||||||
let t: Test = serde_json::from_str(json).unwrap();
|
|
||||||
assert!(matches!(t.encryption, Encryption::Tls));
|
|
||||||
}
|
|
||||||
|
|
||||||
#[test]
|
|
||||||
fn test_schema_serde() {
|
|
||||||
#[derive(serde::Deserialize, Debug)]
|
|
||||||
struct Test {
|
|
||||||
schema: Schema,
|
|
||||||
}
|
|
||||||
let json = r#"{"schema":"custom"}"#;
|
|
||||||
let t: Test = serde_json::from_str(json).unwrap();
|
|
||||||
assert!(matches!(t.schema, Schema::Custom));
|
|
||||||
}
|
|
||||||
}
|
|
||||||
@@ -1,101 +0,0 @@
|
|||||||
// Code in this module PARTIALLY OR FULLY utilized AI Coding Agent.
|
|
||||||
use crate::ExecutionContext;
|
|
||||||
use minijinja::Environment;
|
|
||||||
use std::collections::HashMap;
|
|
||||||
|
|
||||||
/// Builds template context data from execution context.
|
|
||||||
/// Populates pipeline, build, and commit information for template rendering.
|
|
||||||
pub fn prepare_template_data(ctx: &ExecutionContext) -> HashMap<String, serde_json::Value> {
|
|
||||||
let mut data = HashMap::new();
|
|
||||||
|
|
||||||
data.insert(
|
|
||||||
"pipeline".to_string(),
|
|
||||||
serde_json::json!({
|
|
||||||
"name": ctx.pipeline_name,
|
|
||||||
}),
|
|
||||||
);
|
|
||||||
|
|
||||||
data.insert(
|
|
||||||
"build".to_string(),
|
|
||||||
serde_json::json!({
|
|
||||||
"id": ctx.task_id,
|
|
||||||
"status": "success",
|
|
||||||
"status_color": "green",
|
|
||||||
"duration": "0s",
|
|
||||||
"url": "",
|
|
||||||
}),
|
|
||||||
);
|
|
||||||
|
|
||||||
data.insert(
|
|
||||||
"commit".to_string(),
|
|
||||||
serde_json::json!({
|
|
||||||
"hash": "",
|
|
||||||
"short_hash": "",
|
|
||||||
"message": "",
|
|
||||||
"author": "",
|
|
||||||
"author_email": "",
|
|
||||||
}),
|
|
||||||
);
|
|
||||||
|
|
||||||
data
|
|
||||||
}
|
|
||||||
|
|
||||||
/// Renders a minijinja template string with the provided data map.
|
|
||||||
/// Returns the rendered string on success, or a String error message on failure.
|
|
||||||
pub fn render_template(
|
|
||||||
template: &str,
|
|
||||||
data: &HashMap<String, serde_json::Value>,
|
|
||||||
) -> Result<String, String> {
|
|
||||||
let env = Environment::new();
|
|
||||||
|
|
||||||
let tmpl = env
|
|
||||||
.template_from_str(template)
|
|
||||||
.map_err(|e| format!("Template parse error: {}", e))?;
|
|
||||||
|
|
||||||
let result = tmpl
|
|
||||||
.render(data)
|
|
||||||
.map_err(|e| format!("Template render error: {}", e))?;
|
|
||||||
|
|
||||||
Ok(result)
|
|
||||||
}
|
|
||||||
|
|
||||||
#[cfg(test)]
|
|
||||||
mod tests {
|
|
||||||
use super::*;
|
|
||||||
|
|
||||||
#[test]
|
|
||||||
fn test_render_template_simple() {
|
|
||||||
let template = "Hello {{ name }}!";
|
|
||||||
let mut data = HashMap::new();
|
|
||||||
data.insert(
|
|
||||||
"name".to_string(),
|
|
||||||
serde_json::Value::String("World".to_string()),
|
|
||||||
);
|
|
||||||
let result = render_template(template, &data).unwrap();
|
|
||||||
assert_eq!(result, "Hello World!");
|
|
||||||
}
|
|
||||||
|
|
||||||
#[test]
|
|
||||||
fn test_render_template_with_build_status() {
|
|
||||||
let template = "Build {{ build.status }} for pipeline {{ pipeline.name }}";
|
|
||||||
let mut data = HashMap::new();
|
|
||||||
data.insert(
|
|
||||||
"build".to_string(),
|
|
||||||
serde_json::json!({ "status": "success" }),
|
|
||||||
);
|
|
||||||
data.insert(
|
|
||||||
"pipeline".to_string(),
|
|
||||||
serde_json::json!({ "name": "test-pipeline" }),
|
|
||||||
);
|
|
||||||
let result = render_template(template, &data).unwrap();
|
|
||||||
assert_eq!(result, "Build success for pipeline test-pipeline");
|
|
||||||
}
|
|
||||||
|
|
||||||
#[test]
|
|
||||||
fn test_render_template_invalid_syntax() {
|
|
||||||
let template = "{{ Unterminated";
|
|
||||||
let data = HashMap::new();
|
|
||||||
let result = render_template(template, &data);
|
|
||||||
assert!(result.is_err());
|
|
||||||
}
|
|
||||||
}
|
|
||||||
@@ -1,6 +1,7 @@
|
|||||||
use crate::finalize::plugin::PluginError;
|
use crate::finalize::plugin::PluginError;
|
||||||
use crate::finalize::plugin::PluginMetadata;
|
use crate::finalize::plugin::PluginMetadata;
|
||||||
use crate::{EventSender, ExecutionContext, finalize::plugin::AsyncPluginFn};
|
use workshop_engine::{EventSender, ExecutionContext};
|
||||||
|
use crate::{ finalize::plugin::AsyncPluginFn};
|
||||||
use async_trait::async_trait;
|
use async_trait::async_trait;
|
||||||
|
|
||||||
pub struct Satori;
|
pub struct Satori;
|
||||||
|
|||||||
@@ -1,6 +1,7 @@
|
|||||||
use crate::finalize::plugin::PluginError;
|
use crate::finalize::plugin::PluginError;
|
||||||
use crate::finalize::plugin::PluginMetadata;
|
use crate::finalize::plugin::PluginMetadata;
|
||||||
use crate::{EventSender, ExecutionContext, finalize::plugin::AsyncPluginFn};
|
use workshop_engine::{EventSender, ExecutionContext};
|
||||||
|
use crate::{ finalize::plugin::AsyncPluginFn};
|
||||||
use async_trait::async_trait;
|
use async_trait::async_trait;
|
||||||
|
|
||||||
pub struct Webhook;
|
pub struct Webhook;
|
||||||
|
|||||||
@@ -1,3 +1,4 @@
|
|||||||
|
use crate::finalize::NotificationTemplateDef;
|
||||||
use crate::notify::types::NotificationRenderer;
|
use crate::notify::types::NotificationRenderer;
|
||||||
use serde::Deserialize;
|
use serde::Deserialize;
|
||||||
use std::path::PathBuf;
|
use std::path::PathBuf;
|
||||||
@@ -18,10 +19,14 @@ pub struct PluginMetadata {
|
|||||||
#[serde(default)]
|
#[serde(default)]
|
||||||
pub renderer: Option<NotificationRenderer>,
|
pub renderer: Option<NotificationRenderer>,
|
||||||
|
|
||||||
/// Additional YAML fields not captured by struct.
|
/// Additional YAML fields.
|
||||||
#[serde(flatten)]
|
#[serde(flatten)]
|
||||||
pub value: serde_yaml::Value,
|
pub value: serde_yaml::Value,
|
||||||
|
|
||||||
|
/// Plugin-defined default templates, auto-registered as <name>.fallback
|
||||||
|
#[serde(default)]
|
||||||
|
pub templates: Option<Vec<NotificationTemplateDef>>,
|
||||||
|
|
||||||
/// Filesystem path to plugin definition.
|
/// Filesystem path to plugin definition.
|
||||||
#[serde(skip)]
|
#[serde(skip)]
|
||||||
pub fspath: PathBuf,
|
pub fspath: PathBuf,
|
||||||
@@ -36,6 +41,7 @@ impl Default for PluginMetadata {
|
|||||||
plugin_type: PluginType::Unknown,
|
plugin_type: PluginType::Unknown,
|
||||||
value: serde_yaml::Value::Null,
|
value: serde_yaml::Value::Null,
|
||||||
renderer: None,
|
renderer: None,
|
||||||
|
templates: None,
|
||||||
fspath: PathBuf::new(),
|
fspath: PathBuf::new(),
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
@@ -50,76 +56,3 @@ pub enum PluginType {
|
|||||||
Rhai,
|
Rhai,
|
||||||
Dylib,
|
Dylib,
|
||||||
}
|
}
|
||||||
|
|
||||||
#[cfg(test)]
|
|
||||||
mod tests {
|
|
||||||
use super::*;
|
|
||||||
|
|
||||||
#[test]
|
|
||||||
fn test_plugin_metadata_default() {
|
|
||||||
let meta = PluginMetadata::default();
|
|
||||||
assert_eq!(meta.name, "");
|
|
||||||
assert_eq!(meta.version, "");
|
|
||||||
assert_eq!(meta.entrypoint, "");
|
|
||||||
assert_eq!(meta.plugin_type, PluginType::Unknown);
|
|
||||||
}
|
|
||||||
|
|
||||||
#[test]
|
|
||||||
fn test_plugin_type_default() {
|
|
||||||
let pt = PluginType::default();
|
|
||||||
assert_eq!(pt, PluginType::Unknown);
|
|
||||||
}
|
|
||||||
|
|
||||||
#[test]
|
|
||||||
fn test_plugin_type_equality() {
|
|
||||||
assert_eq!(PluginType::Shell, PluginType::Shell);
|
|
||||||
assert_eq!(PluginType::Dylib, PluginType::Dylib);
|
|
||||||
assert_ne!(PluginType::Shell, PluginType::Dylib);
|
|
||||||
}
|
|
||||||
|
|
||||||
#[test]
|
|
||||||
fn test_plugin_metadata_clone() {
|
|
||||||
let meta = PluginMetadata {
|
|
||||||
name: "test".to_string(),
|
|
||||||
version: "1.0".to_string(),
|
|
||||||
entrypoint: "/bin/test".to_string(),
|
|
||||||
plugin_type: PluginType::Shell,
|
|
||||||
value: serde_yaml::Value::Null,
|
|
||||||
fspath: PathBuf::from("/path/to/plugin"),
|
|
||||||
renderer: None,
|
|
||||||
};
|
|
||||||
let cloned = meta.clone();
|
|
||||||
assert_eq!(cloned.name, meta.name);
|
|
||||||
assert_eq!(cloned.version, meta.version);
|
|
||||||
assert_eq!(cloned.plugin_type, meta.plugin_type);
|
|
||||||
}
|
|
||||||
|
|
||||||
#[test]
|
|
||||||
fn test_plugin_type_serde_unknown() {
|
|
||||||
let yaml = "Unknown";
|
|
||||||
let pt: PluginType = serde_yaml::from_str(yaml).unwrap();
|
|
||||||
assert_eq!(pt, PluginType::Unknown);
|
|
||||||
}
|
|
||||||
|
|
||||||
#[test]
|
|
||||||
fn test_plugin_type_serde_shell() {
|
|
||||||
let yaml = "Shell";
|
|
||||||
let pt: PluginType = serde_yaml::from_str(yaml).unwrap();
|
|
||||||
assert_eq!(pt, PluginType::Shell);
|
|
||||||
}
|
|
||||||
|
|
||||||
#[test]
|
|
||||||
fn test_plugin_metadata_serde() {
|
|
||||||
let yaml = r#"
|
|
||||||
name: test-plugin
|
|
||||||
version: "2.0"
|
|
||||||
entrypoint: ./bin/test
|
|
||||||
plugin_type: Shell
|
|
||||||
"#;
|
|
||||||
let meta: PluginMetadata = serde_yaml::from_str(yaml).unwrap();
|
|
||||||
assert_eq!(meta.name, "test-plugin");
|
|
||||||
assert_eq!(meta.version, "2.0");
|
|
||||||
assert_eq!(meta.entrypoint, "./bin/test");
|
|
||||||
assert_eq!(meta.plugin_type, PluginType::Shell);
|
|
||||||
}
|
|
||||||
}
|
|
||||||
|
|||||||
@@ -1,6 +1,7 @@
|
|||||||
use crate::finalize::plugin::PluginError;
|
use crate::finalize::plugin::PluginError;
|
||||||
use crate::finalize::plugin::PluginMetadata;
|
use crate::finalize::plugin::PluginMetadata;
|
||||||
use crate::{EventSender, ExecutionContext, finalize::plugin::AsyncPluginFn};
|
use workshop_engine::{EventSender, ExecutionContext};
|
||||||
|
use crate::{ finalize::plugin::AsyncPluginFn};
|
||||||
use async_trait::async_trait;
|
use async_trait::async_trait;
|
||||||
|
|
||||||
pub struct Rhai;
|
pub struct Rhai;
|
||||||
|
|||||||
@@ -1,8 +1,9 @@
|
|||||||
use crate::Engine;
|
use workshop_engine::Engine;
|
||||||
use crate::finalize::constant::FINALIZE_SHELL_ENVPREFIX;
|
use crate::finalize::constant::FINALIZE_SHELL_ENVPREFIX;
|
||||||
use crate::finalize::plugin::PluginError;
|
use crate::finalize::plugin::PluginError;
|
||||||
use crate::finalize::plugin::PluginMetadata;
|
use crate::finalize::plugin::PluginMetadata;
|
||||||
use crate::{EventSender, ExecutionContext, finalize::plugin::AsyncPluginFn};
|
use workshop_engine::{EventSender, ExecutionContext};
|
||||||
|
use crate::{ finalize::plugin::AsyncPluginFn};
|
||||||
use async_trait::async_trait;
|
use async_trait::async_trait;
|
||||||
use std::collections::HashMap;
|
use std::collections::HashMap;
|
||||||
|
|
||||||
|
|||||||
@@ -1,4 +1,4 @@
|
|||||||
use crate::ExecutionContext;
|
use workshop_engine::ExecutionContext;
|
||||||
use crate::finalize::plugin::{PluginMap, call_named_plugin};
|
use crate::finalize::plugin::{PluginMap, call_named_plugin};
|
||||||
use std::path::PathBuf;
|
use std::path::PathBuf;
|
||||||
use workshop_engine::{CustomCommand, DeltaExecutionContext, Engine, EventSender, ExecutionError};
|
use workshop_engine::{CustomCommand, DeltaExecutionContext, Engine, EventSender, ExecutionError};
|
||||||
|
|||||||
+25
-57
@@ -1,27 +1,16 @@
|
|||||||
// Library crate for workshop-executor
|
pub mod bare;
|
||||||
// Selective exports: only modules used by external callers or tests
|
|
||||||
|
|
||||||
pub mod bake;
|
pub mod bake;
|
||||||
pub mod daemon;
|
pub mod error;
|
||||||
pub mod finalize;
|
pub mod finalize;
|
||||||
pub mod monitor;
|
|
||||||
pub mod notify;
|
pub mod notify;
|
||||||
pub mod prebake;
|
pub mod prebake;
|
||||||
pub mod socket;
|
|
||||||
pub mod types;
|
pub mod types;
|
||||||
|
pub mod utils;
|
||||||
|
|
||||||
// External crates re-exports
|
|
||||||
pub use config;
|
|
||||||
|
|
||||||
// Re-export commonly used types from workshop-engine
|
|
||||||
pub use workshop_engine::{Engine, EventSender, ExecutionContext, ExecutionError, ExecutionResult};
|
|
||||||
|
|
||||||
// CLI types used by main (keep here since they're used in daemon)
|
|
||||||
pub use clap::{Parser, Subcommand};
|
pub use clap::{Parser, Subcommand};
|
||||||
pub use serde::Deserialize;
|
pub use serde::Deserialize;
|
||||||
pub use std::collections::HashMap;
|
pub use std::collections::HashMap;
|
||||||
pub use std::path::PathBuf;
|
pub use std::path::PathBuf;
|
||||||
|
|
||||||
pub mod cli {
|
pub mod cli {
|
||||||
pub use super::{Deserialize, HashMap, Parser, PathBuf, Subcommand};
|
pub use super::{Deserialize, HashMap, Parser, PathBuf, Subcommand};
|
||||||
pub use clap::Command;
|
pub use clap::Command;
|
||||||
@@ -46,64 +35,43 @@ pub mod cli {
|
|||||||
#[arg(long, default_value = "false", env = "HBW_DRY_RUN")]
|
#[arg(long, default_value = "false", env = "HBW_DRY_RUN")]
|
||||||
pub dry_run: bool,
|
pub dry_run: bool,
|
||||||
|
|
||||||
|
/// Override prebake config path
|
||||||
|
#[arg(long)]
|
||||||
|
pub prebake: Option<PathBuf>,
|
||||||
|
/// Override bake script path
|
||||||
|
#[arg(long)]
|
||||||
|
pub bake: Option<PathBuf>,
|
||||||
|
/// Override bake_base.sh path
|
||||||
|
#[arg(long)]
|
||||||
|
pub bake_base: Option<PathBuf>,
|
||||||
|
/// Override finalize config path
|
||||||
|
#[arg(long)]
|
||||||
|
pub finalize: Option<PathBuf>,
|
||||||
|
|
||||||
|
/// Debug features (comma-separated). "notify" enables notification debug logging.
|
||||||
|
#[arg(long, env = "HBW_DEBUG")]
|
||||||
|
pub debug: Option<String>,
|
||||||
|
|
||||||
#[command(subcommand)]
|
#[command(subcommand)]
|
||||||
pub command: Option<Commands>,
|
pub command: Option<Commands>,
|
||||||
}
|
}
|
||||||
|
|
||||||
#[derive(Subcommand, Debug)]
|
#[derive(Subcommand, Debug)]
|
||||||
pub enum Commands {
|
pub enum Commands {
|
||||||
/// Run pipeline stages in standalone (bare) mode
|
/// Run individual pipeline stages
|
||||||
Bare {
|
Bare {
|
||||||
#[command(subcommand)]
|
#[command(subcommand)]
|
||||||
command: BareCommands,
|
command: BareCommands,
|
||||||
},
|
},
|
||||||
|
|
||||||
/// Run in client mode
|
Daemon {},
|
||||||
Client { config: String },
|
|
||||||
}
|
}
|
||||||
|
|
||||||
#[derive(Subcommand, Debug)]
|
#[derive(Subcommand, Debug)]
|
||||||
pub enum BareCommands {
|
pub enum BareCommands {
|
||||||
/// Prepare a task
|
Prebake {},
|
||||||
Prebake { config: PathBuf },
|
Bake {},
|
||||||
|
Finalize {},
|
||||||
/// Run a task
|
|
||||||
Bake {
|
|
||||||
/// Path to the script to execute
|
|
||||||
script: PathBuf,
|
|
||||||
/// Path to bake_base.sh
|
|
||||||
#[arg(short, long, default_value = "./bake_base.sh")]
|
|
||||||
bake_base: PathBuf,
|
|
||||||
#[arg(short, long, default_value = "./prebake.yml")]
|
|
||||||
prebake: PathBuf,
|
|
||||||
},
|
|
||||||
|
|
||||||
/// Finish a task
|
|
||||||
Finalize { config: String },
|
|
||||||
}
|
}
|
||||||
|
|
||||||
#[derive(Debug, Deserialize)]
|
|
||||||
pub struct TaskSpec {
|
|
||||||
pub subcommand: String,
|
|
||||||
pub arguments: Vec<String>,
|
|
||||||
pub options: HashMap<String, String>,
|
|
||||||
}
|
|
||||||
|
|
||||||
impl TaskSpec {
|
|
||||||
pub fn command(&self, executable: &PathBuf) -> tokio::process::Command {
|
|
||||||
let mut command = tokio::process::Command::new(executable);
|
|
||||||
command.arg(&self.subcommand);
|
|
||||||
for arg in &self.arguments {
|
|
||||||
command.arg(arg);
|
|
||||||
}
|
|
||||||
for (flag, value) in self.options.iter() {
|
|
||||||
if value.is_empty() {
|
|
||||||
command.arg(format!("--{}", flag));
|
|
||||||
} else {
|
|
||||||
command.arg(format!("--{}={}", flag, value));
|
|
||||||
}
|
|
||||||
}
|
|
||||||
command
|
|
||||||
}
|
|
||||||
}
|
|
||||||
}
|
}
|
||||||
|
|||||||
+110
-63
@@ -1,13 +1,91 @@
|
|||||||
use std::path::Path;
|
use workshop_baker::types::resource::ResourceRegistry;
|
||||||
|
use std::path::PathBuf;
|
||||||
|
|
||||||
use workshop_baker::{
|
use workshop_baker::{
|
||||||
bake::bake,
|
bake::bake, cli::{Cli, Commands, Parser},
|
||||||
cli::{BareCommands, Cli, Commands, Parser},
|
error::CliError,
|
||||||
daemon,
|
finalize::finalize, notify::notify, prebake::prebake,
|
||||||
finalize::finalize,
|
|
||||||
prebake::prebake,
|
|
||||||
};
|
};
|
||||||
use workshop_engine::{EventSender, ExecutionContext, error::HasExitCode};
|
use workshop_baker::bare::run_bare;
|
||||||
|
use workshop_baker::notify::queue::NotificationQueue;
|
||||||
|
use workshop_engine::{EventSender, ExecutionContext};
|
||||||
|
|
||||||
|
fn or_workshop(path: &Option<PathBuf>, name: &str) -> PathBuf {
|
||||||
|
path.clone().unwrap_or_else(|| PathBuf::from(format!(".workshop/{}", name)))
|
||||||
|
}
|
||||||
|
|
||||||
|
async fn worker_main(cli: Cli) -> Result<(), CliError> {
|
||||||
|
let prebake_path = cli.prebake.clone().unwrap_or_else(|| {
|
||||||
|
log::error!("--prebake is required in worker standalone mode");
|
||||||
|
std::process::exit(1);
|
||||||
|
});
|
||||||
|
let bake_path = cli.bake.clone().unwrap_or_else(|| {
|
||||||
|
log::error!("--bake is required in worker standalone mode");
|
||||||
|
std::process::exit(1);
|
||||||
|
});
|
||||||
|
let finalize_path = cli.finalize.clone().unwrap_or_else(|| {
|
||||||
|
log::error!("--finalize is required in worker standalone mode");
|
||||||
|
std::process::exit(1);
|
||||||
|
});
|
||||||
|
|
||||||
|
log::info!("Worker standalone mode");
|
||||||
|
|
||||||
|
let mut ctx = ExecutionContext {
|
||||||
|
standalone: true,
|
||||||
|
task_id: format!("{}-{}-worker", cli.pipeline, cli.build_id),
|
||||||
|
pipeline_name: cli.pipeline.clone(),
|
||||||
|
username: cli.username.clone(),
|
||||||
|
dry_run: cli.dry_run,
|
||||||
|
privileged: false,
|
||||||
|
..Default::default()
|
||||||
|
};
|
||||||
|
|
||||||
|
let (tx, event_rx) = tokio::sync::mpsc::channel(256);
|
||||||
|
let event_tx: EventSender = Some(tx);
|
||||||
|
let _event_handle = tokio::spawn(workshop_baker::prebake::event::event_receiver(
|
||||||
|
event_rx, cli.pipeline.clone(), cli.build_id.clone(),
|
||||||
|
));
|
||||||
|
|
||||||
|
let (to_tx, to_rx) = tokio::sync::mpsc::channel(256);
|
||||||
|
let (retry_tx, retry_rx) = tokio::sync::mpsc::channel(256);
|
||||||
|
|
||||||
|
// Resource registry — populated before stages, consumed by stages
|
||||||
|
let registry = ResourceRegistry::new();
|
||||||
|
// TODO: pre-fetch plugins and templates into registry
|
||||||
|
|
||||||
|
let notify_path = finalize_path.clone();
|
||||||
|
let notify_ctx = ctx.clone();
|
||||||
|
let notify_etx = event_tx.clone();
|
||||||
|
let debug = cli.debug.clone();
|
||||||
|
let notify_registry = registry.clone();
|
||||||
|
let notify_handle = tokio::spawn(async move {
|
||||||
|
notify(¬ify_path, &mut notify_ctx.clone(), notify_etx, to_rx, retry_tx, debug.map(|d| d.contains("notify")).unwrap_or(false), ¬ify_registry).await
|
||||||
|
});
|
||||||
|
|
||||||
|
let nevent_tx = to_tx.clone();
|
||||||
|
let mut queue = NotificationQueue::new(to_tx, retry_rx);
|
||||||
|
|
||||||
|
let pipe_result: anyhow::Result<()> = async {
|
||||||
|
prebake(&prebake_path, &cli, None, &mut ctx, event_tx.clone(), Some(nevent_tx)).await?;
|
||||||
|
let bake_base_path = or_workshop(&cli.bake_base, "bake_base.sh");
|
||||||
|
let pb_path = or_workshop(&cli.prebake, "prebake.yml");
|
||||||
|
bake(&bake_path, &bake_base_path, &pb_path, &cli, &mut ctx, event_tx.clone()).await?;
|
||||||
|
finalize(&finalize_path, &cli, &mut ctx, event_tx.clone(), ®istry).await?;
|
||||||
|
Ok(())
|
||||||
|
}.await;
|
||||||
|
|
||||||
|
if let Err(e) = pipe_result {
|
||||||
|
log::error!("Pipeline stage failed: {}", e);
|
||||||
|
return Err(CliError::General(e));
|
||||||
|
}
|
||||||
|
|
||||||
|
log::info!("Pipeline complete. Draining notifications...");
|
||||||
|
if let Err(e) = queue.drain().await {
|
||||||
|
log::error!("Notification queue error: {}", e);
|
||||||
|
}
|
||||||
|
let _ = notify_handle.await;
|
||||||
|
Ok(())
|
||||||
|
}
|
||||||
|
|
||||||
#[tokio::main]
|
#[tokio::main]
|
||||||
async fn main() {
|
async fn main() {
|
||||||
@@ -16,77 +94,46 @@ async fn main() {
|
|||||||
if cli.dry_run {
|
if cli.dry_run {
|
||||||
log::warn!("Dry run enabled, no changes will be made.");
|
log::warn!("Dry run enabled, no changes will be made.");
|
||||||
}
|
}
|
||||||
match &cli.command {
|
|
||||||
Some(Commands::Client { config: _ }) => {
|
|
||||||
unimplemented!("Client mode not yet implemented.")
|
|
||||||
}
|
|
||||||
Some(Commands::Bare { command }) => {
|
|
||||||
log::info!("Running in standalone mode.");
|
|
||||||
|
|
||||||
let (stage, privileged) = match command {
|
let pipeline = cli.pipeline.clone();
|
||||||
BareCommands::Prebake { .. } => ("init", true),
|
let build_id = cli.build_id.clone();
|
||||||
BareCommands::Bake { .. } => ("bake", false),
|
|
||||||
BareCommands::Finalize { .. } => ("finalize", false),
|
match &cli.command {
|
||||||
};
|
Some(Commands::Bare { command }) => {
|
||||||
let mut ctx = ExecutionContext {
|
let mut ctx = ExecutionContext {
|
||||||
standalone: true,
|
standalone: true,
|
||||||
task_id: format!("{}-{}-{}", cli.pipeline, cli.build_id, stage),
|
task_id: format!("{}-{}-bare", pipeline, build_id),
|
||||||
pipeline_name: cli.pipeline.clone(),
|
pipeline_name: pipeline.clone(),
|
||||||
username: cli.username.clone(),
|
username: cli.username.clone(),
|
||||||
dry_run: cli.dry_run,
|
dry_run: cli.dry_run,
|
||||||
privileged,
|
privileged: false,
|
||||||
..Default::default()
|
..Default::default()
|
||||||
};
|
};
|
||||||
let (tx, event_rx) = tokio::sync::mpsc::channel(256);
|
let (tx, event_rx) = tokio::sync::mpsc::channel(256);
|
||||||
let event_tx: EventSender = Some(tx);
|
let event_tx: EventSender = Some(tx);
|
||||||
let handle = tokio::spawn(workshop_baker::prebake::event::event_receiver(
|
let event_handle = tokio::spawn(workshop_baker::prebake::event::event_receiver(
|
||||||
event_rx,
|
event_rx, pipeline.clone(), build_id.clone(),
|
||||||
cli.pipeline.clone(),
|
|
||||||
cli.build_id.clone(),
|
|
||||||
));
|
));
|
||||||
|
|
||||||
let result: anyhow::Result<()> = match command {
|
let registry = ResourceRegistry::new();
|
||||||
BareCommands::Prebake { config } => {
|
let result = run_bare(&cli, command, &mut ctx, event_tx, &pipeline, &build_id, ®istry).await;
|
||||||
prebake(config, &cli, None, &mut ctx, event_tx).await
|
|
||||||
}
|
|
||||||
BareCommands::Bake {
|
|
||||||
script,
|
|
||||||
bake_base,
|
|
||||||
prebake: prebake_path,
|
|
||||||
} => bake(script, bake_base, prebake_path, &cli, &mut ctx, event_tx)
|
|
||||||
.await
|
|
||||||
.map_err(|e| e.into()),
|
|
||||||
BareCommands::Finalize { config } => {
|
|
||||||
finalize(Path::new(config), &cli, &mut ctx, event_tx)
|
|
||||||
.await
|
|
||||||
.map_err(|e| e.into())
|
|
||||||
}
|
|
||||||
};
|
|
||||||
drop(ctx);
|
drop(ctx);
|
||||||
let _ = handle.await;
|
let _ = event_handle.await;
|
||||||
dbg!(&result);
|
|
||||||
if let Err(ref e) = result {
|
if let Err(e) = result {
|
||||||
let code = match command {
|
log::error!("{}", e);
|
||||||
BareCommands::Prebake { .. } => e
|
std::process::exit(e.exit_code());
|
||||||
.downcast_ref::<workshop_baker::prebake::error::PrebakeError>()
|
|
||||||
.map(|pe| pe.exit_code())
|
|
||||||
.unwrap_or(1),
|
|
||||||
BareCommands::Bake { .. } => e
|
|
||||||
.downcast_ref::<workshop_baker::bake::error::BakeError>()
|
|
||||||
.map(|be| be.exit_code())
|
|
||||||
.unwrap_or(1),
|
|
||||||
BareCommands::Finalize { .. } => e
|
|
||||||
.downcast_ref::<workshop_baker::finalize::FinalizeError>()
|
|
||||||
.map(|fe| fe.exit_code())
|
|
||||||
.unwrap_or(1),
|
|
||||||
};
|
|
||||||
log::error!("{} stage failed: {}", stage, e);
|
|
||||||
std::process::exit(code);
|
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
Some(Commands::Daemon {}) => {
|
||||||
|
unimplemented!("Daemon mode not implemented");
|
||||||
|
}
|
||||||
None => {
|
None => {
|
||||||
log::info!("Executor will be running in daemon mode.");
|
if let Err(e) = worker_main(cli).await {
|
||||||
let _result = daemon::daemon(&cli).await;
|
log::error!("{}", e);
|
||||||
|
std::process::exit(e.exit_code());
|
||||||
|
}
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|||||||
@@ -1,89 +0,0 @@
|
|||||||
mod cgroups;
|
|
||||||
mod jobobject;
|
|
||||||
use crate::socket::{establish_connection, get_socket_addr};
|
|
||||||
use config::Config;
|
|
||||||
use serde::{Deserialize, Serialize};
|
|
||||||
use std::time::Instant;
|
|
||||||
use tokio::io::AsyncWriteExt;
|
|
||||||
|
|
||||||
#[derive(Debug, Serialize, Deserialize)]
|
|
||||||
pub struct ResourceUsageInfo {
|
|
||||||
pub msgtype: String,
|
|
||||||
pub name: String,
|
|
||||||
pub os_type: OsType,
|
|
||||||
pub statistics: UsageStats,
|
|
||||||
pub timestamp: u64,
|
|
||||||
}
|
|
||||||
|
|
||||||
#[derive(Debug, Serialize, Deserialize)]
|
|
||||||
pub enum OsType {
|
|
||||||
Linux,
|
|
||||||
Windows,
|
|
||||||
MacOS,
|
|
||||||
}
|
|
||||||
|
|
||||||
#[derive(Debug, Serialize, Deserialize, Clone)]
|
|
||||||
pub struct UsageStats {
|
|
||||||
pub cpu_usage_percent: f64,
|
|
||||||
pub memory_usage_bytes: u64,
|
|
||||||
pub memory_limit_bytes: i64,
|
|
||||||
// pub memory_usage_percent: f64,
|
|
||||||
pub process_count: u64,
|
|
||||||
pub cpu_usage_us: u64,
|
|
||||||
/// Internal timestamp, using std::Instant
|
|
||||||
#[serde(skip, default = "default_internal_timestamp")]
|
|
||||||
_timestamp: Instant,
|
|
||||||
}
|
|
||||||
|
|
||||||
fn default_internal_timestamp() -> Instant {
|
|
||||||
Instant::now()
|
|
||||||
}
|
|
||||||
|
|
||||||
pub async fn monitor(
|
|
||||||
name: &String,
|
|
||||||
interval_ms: Option<u64>,
|
|
||||||
settings: Config,
|
|
||||||
) -> anyhow::Result<()> {
|
|
||||||
// let socket_addr = get_wsagent_socket_addr()?;
|
|
||||||
let socket_addr = get_socket_addr(settings.get_string("socket")?)?;
|
|
||||||
let interval = tokio::time::Duration::from_millis(interval_ms.unwrap_or(1000));
|
|
||||||
let mut socket = establish_connection(&socket_addr).await?;
|
|
||||||
let mut prev_usage = UsageStats {
|
|
||||||
cpu_usage_percent: 0.0,
|
|
||||||
memory_usage_bytes: 0,
|
|
||||||
memory_limit_bytes: 0,
|
|
||||||
process_count: 0,
|
|
||||||
cpu_usage_us: 0,
|
|
||||||
_timestamp: default_internal_timestamp(),
|
|
||||||
};
|
|
||||||
loop {
|
|
||||||
let usage: UsageStats = cgroups::get_usage(name, prev_usage)?; // TODO: 多系统支持?
|
|
||||||
prev_usage = usage.clone();
|
|
||||||
// dbg!(&usage);
|
|
||||||
let usage_info = ResourceUsageInfo {
|
|
||||||
msgtype: "ResourceUsage".to_string(),
|
|
||||||
name: name.clone(),
|
|
||||||
os_type: OsType::Linux, // TODO: 多系统支持?
|
|
||||||
statistics: usage,
|
|
||||||
timestamp: std::time::SystemTime::now()
|
|
||||||
.duration_since(std::time::UNIX_EPOCH)?
|
|
||||||
.as_millis() as u64,
|
|
||||||
};
|
|
||||||
|
|
||||||
let usage_json = serde_json::to_string(&usage_info)?;
|
|
||||||
|
|
||||||
send_usage_info(&mut socket, &usage_json).await?;
|
|
||||||
// TODO: 增加消息接收,提供(伪)双向通信功能
|
|
||||||
|
|
||||||
tokio::time::sleep(interval).await;
|
|
||||||
}
|
|
||||||
}
|
|
||||||
|
|
||||||
async fn send_usage_info<S>(stream: &mut S, usage_json: &str) -> anyhow::Result<()>
|
|
||||||
where
|
|
||||||
S: tokio::io::AsyncWrite + Unpin,
|
|
||||||
{
|
|
||||||
stream.write_all(usage_json.as_bytes()).await?;
|
|
||||||
stream.write_all(b"\n").await?;
|
|
||||||
Ok(())
|
|
||||||
}
|
|
||||||
+429
-361
@@ -1,400 +1,468 @@
|
|||||||
use crate::finalize::FinalizeConfig;
|
|
||||||
use crate::finalize::NotificationConfig;
|
|
||||||
use crate::finalize::NotificationTemplate;
|
|
||||||
use crate::finalize::config::TriggerItemRaw;
|
|
||||||
use crate::finalize::constant::FINALIZE_STANDALONE_PLUGIN_PATH;
|
|
||||||
use crate::finalize::parse;
|
use crate::finalize::parse;
|
||||||
use crate::finalize::plugin;
|
use crate::finalize::plugin;
|
||||||
use crate::finalize::plugin::fetch;
|
use crate::notify::types::DEFAULT_NOTIFY_PARAMS;
|
||||||
use crate::notify::types::Matchable;
|
|
||||||
use crate::notify::types::Method;
|
|
||||||
use crate::notify::types::NOTIFICATION_DEFAULT_FALLIBLE;
|
|
||||||
use crate::notify::types::NOTIFICATION_DEFAULT_MAX_LIVES;
|
|
||||||
use crate::notify::types::NOTIFICATION_DEFAULT_RETRY_BACKOFF;
|
|
||||||
use crate::notify::types::NOTIFICATION_DEFAULT_TIMEOUT_MS;
|
|
||||||
use crate::notify::types::NOTIFICATION_NOTREADY_DELAY;
|
|
||||||
use crate::notify::types::NOTIFICATION_TEMPORARY_DELAY_FACTOR;
|
|
||||||
use crate::notify::types::NOTIFICATION_TEMPORARY_DELAY_TIME;
|
|
||||||
use crate::notify::types::NOTIFICATION_TEMPORARY_MAX_TIME;
|
|
||||||
use crate::notify::types::NotificationEvent;
|
|
||||||
use crate::notify::types::NotificationEventReceiver;
|
use crate::notify::types::NotificationEventReceiver;
|
||||||
use crate::notify::types::NotificationEventSender;
|
use crate::notify::types::NotificationEventSender;
|
||||||
use crate::notify::types::NotificationPluginMap;
|
use crate::notify::handler::NotificationHandler;
|
||||||
use crate::notify::types::NotificationRenderer;
|
use crate::types::resource::ResourceRegistry;
|
||||||
use crate::notify::types::NotificationRule;
|
|
||||||
use crate::notify::types::NotificationRulesMap;
|
|
||||||
use crate::notify::types::ParsedTrigger;
|
|
||||||
use crate::{EventSender, ExecutionContext};
|
|
||||||
use chrono::Utc;
|
|
||||||
use globset::{Glob, GlobSetBuilder};
|
|
||||||
use std::collections::HashMap;
|
|
||||||
use std::collections::HashSet;
|
|
||||||
use std::path::Path;
|
use std::path::Path;
|
||||||
use std::path::PathBuf;
|
use std::sync::Arc;
|
||||||
|
use workshop_engine::{EventSender, ExecutionContext};
|
||||||
|
|
||||||
mod error;
|
mod error;
|
||||||
|
pub mod handler;
|
||||||
|
pub mod queue;
|
||||||
|
pub mod rule;
|
||||||
|
pub mod template;
|
||||||
pub mod types;
|
pub mod types;
|
||||||
use error::NotifyError;
|
pub mod util;
|
||||||
|
use rule::parse_rules;
|
||||||
fn validate(finalize: &FinalizeConfig) -> anyhow::Result<()> {
|
use template::{flatten_template_refs, resolve_external_template_refs};
|
||||||
let validate_result = finalize.validate();
|
use util::{extract_plugin, validate};
|
||||||
if let Err(errors) = validate_result {
|
|
||||||
for error in &errors {
|
|
||||||
log::error!("Error: {}", error);
|
|
||||||
}
|
|
||||||
anyhow::bail!(NotifyError::FinalizeValidationError(errors))
|
|
||||||
};
|
|
||||||
Ok(())
|
|
||||||
}
|
|
||||||
|
|
||||||
fn extract_plugin(finalize: &FinalizeConfig) -> HashSet<String> {
|
|
||||||
finalize
|
|
||||||
.notification
|
|
||||||
.groups
|
|
||||||
.values()
|
|
||||||
.flat_map(|methods| methods.keys())
|
|
||||||
.cloned()
|
|
||||||
.collect()
|
|
||||||
}
|
|
||||||
|
|
||||||
pub async fn notify(
|
pub async fn notify(
|
||||||
finalize_path: &Path,
|
finalize_path: &Path,
|
||||||
ctx: &mut ExecutionContext,
|
ctx: &mut ExecutionContext,
|
||||||
_event_tx: EventSender,
|
_event_tx: EventSender,
|
||||||
notifyevent_rx: &mut NotificationEventReceiver,
|
mut rx: NotificationEventReceiver,
|
||||||
notifyevent_tx: NotificationEventSender,
|
retry_tx: NotificationEventSender,
|
||||||
|
_debug: bool,
|
||||||
|
registry: &ResourceRegistry,
|
||||||
) -> anyhow::Result<()> {
|
) -> anyhow::Result<()> {
|
||||||
let mut finalize = parse(finalize_path)?;
|
let mut finalize = parse(finalize_path)?;
|
||||||
|
|
||||||
|
// Always inject in-site-notify as the lowest priority fallback
|
||||||
|
{
|
||||||
|
let max_p = finalize.notification.groups.keys().max().copied().unwrap_or(0);
|
||||||
|
let mut m = std::collections::HashMap::new();
|
||||||
|
m.insert(
|
||||||
|
"in-site-notify".to_string(),
|
||||||
|
crate::finalize::NotificationMethod::default(),
|
||||||
|
);
|
||||||
|
finalize.notification.groups.insert(max_p + 1, m);
|
||||||
|
}
|
||||||
|
|
||||||
validate(&finalize)?;
|
validate(&finalize)?;
|
||||||
let plugin_list = extract_plugin(&finalize);
|
let plugin_list = extract_plugin(&finalize);
|
||||||
|
|
||||||
if plugin_list.is_empty() {
|
let notify_plugins = plugin::register(
|
||||||
log::warn!("No notification method provided, skipping notification");
|
finalize.plugin,
|
||||||
// TODO: send event to event_tx
|
Some(&plugin_list),
|
||||||
return Ok(());
|
&mut finalize.notification,
|
||||||
}
|
registry,
|
||||||
|
)?;
|
||||||
|
|
||||||
if ctx.standalone {
|
resolve_external_template_refs(&mut finalize.notification.templates, registry);
|
||||||
fetch::fetch_plugins(
|
flatten_template_refs(&mut finalize.notification.templates)?;
|
||||||
&PathBuf::from(FINALIZE_STANDALONE_PLUGIN_PATH),
|
|
||||||
&mut finalize.plugin,
|
|
||||||
Some(&plugin_list),
|
|
||||||
)
|
|
||||||
.await?;
|
|
||||||
}
|
|
||||||
let notify_plugins = plugin::register(finalize.plugin, Some(&plugin_list))?;
|
|
||||||
|
|
||||||
let notification_rules = parse_rules(&finalize.notification);
|
let notification_rules = parse_rules(&finalize.notification);
|
||||||
|
|
||||||
loop {
|
let handler = Arc::new(NotificationHandler::new(
|
||||||
tokio::select! {
|
Arc::new(notification_rules),
|
||||||
Some(event) = notifyevent_rx.recv() => {
|
Arc::new(notify_plugins),
|
||||||
handle_notification_event(
|
Arc::new(finalize.notification.templates),
|
||||||
event,
|
Arc::new(ctx.clone()),
|
||||||
¬ify_plugins,
|
retry_tx,
|
||||||
¬ification_rules,
|
DEFAULT_NOTIFY_PARAMS.batch_period.value,
|
||||||
¬ifyevent_tx,
|
DEFAULT_NOTIFY_PARAMS.batch_watermark.value,
|
||||||
)
|
));
|
||||||
.await?;
|
|
||||||
}
|
|
||||||
}
|
|
||||||
}
|
|
||||||
}
|
|
||||||
|
|
||||||
/// Process a single notification event.
|
loop {
|
||||||
///
|
let Some(event) = rx.recv().await else { break };
|
||||||
/// Delegates to [`notification_handler`] and handles errors:
|
let h = handler.clone();
|
||||||
/// - [`NotifyError::PluginNotAvailable`] — reschedule with static delay, lower priority
|
tokio::spawn(async move {
|
||||||
/// - [`NotifyError::TemporaryError`] — consume one life, reschedule with exponential backoff
|
h.process_event(event).await;
|
||||||
/// - [`NotifyError::PermanentError`] — drop the event
|
});
|
||||||
/// - other — log and drop
|
|
||||||
async fn handle_notification_event(
|
|
||||||
event: NotificationEvent,
|
|
||||||
plugins: &NotificationPluginMap,
|
|
||||||
rules: &NotificationRulesMap,
|
|
||||||
notifyevent_tx: &NotificationEventSender,
|
|
||||||
) -> anyhow::Result<()> {
|
|
||||||
match notification_handler(plugins, rules, &event) {
|
|
||||||
Ok(_) => {}
|
|
||||||
Err(e) => {
|
|
||||||
log::debug!("Failed to send notification: {:?}", e);
|
|
||||||
match e {
|
|
||||||
NotifyError::PluginNotAvailable(_) => {
|
|
||||||
let mut event = event;
|
|
||||||
event.not_before = Utc::now() + NOTIFICATION_NOTREADY_DELAY;
|
|
||||||
event.effective_priority = event.effective_priority.saturating_add(1);
|
|
||||||
notifyevent_tx.send(event).await?;
|
|
||||||
}
|
|
||||||
NotifyError::TemporaryError(_) => {
|
|
||||||
let mut event = event;
|
|
||||||
event.lives -= 1;
|
|
||||||
event.not_before = Utc::now() + notify_delay(&event);
|
|
||||||
notifyevent_tx.send(event).await?;
|
|
||||||
}
|
|
||||||
NotifyError::PermanentError(_) => {
|
|
||||||
log::error!("None of the configured notification plugins works.");
|
|
||||||
log::error!("Message will be dropped.");
|
|
||||||
}
|
|
||||||
error => {
|
|
||||||
log::error!(
|
|
||||||
"Internal error: notify_handler returned unexpected error: {:?}",
|
|
||||||
error
|
|
||||||
);
|
|
||||||
}
|
|
||||||
}
|
|
||||||
}
|
|
||||||
}
|
}
|
||||||
Ok(())
|
Ok(())
|
||||||
}
|
}
|
||||||
|
|
||||||
fn notification_handler(
|
// TODO: PIPELINE_NOTIFICATION_EXHAUSTED — set when any method returns Exhausted
|
||||||
plugins: &NotificationPluginMap,
|
// TODO: PIPELINE_NOTIFICATION_FAILED — set when any method returns PermanentError
|
||||||
config: &NotificationRulesMap,
|
// Markers sent via event_tx as ExecutionEvent variants for future WebUI/CLI display.
|
||||||
event: &NotificationEvent,
|
|
||||||
) -> Result<(), NotifyError> {
|
|
||||||
for (_priority, ruleset) in config.iter() {
|
|
||||||
for (method, rules) in ruleset.iter() {
|
|
||||||
if !event.target.contains(method) {
|
|
||||||
log::trace!("target does not contain method name: {}", method);
|
|
||||||
continue;
|
|
||||||
}
|
|
||||||
for (rule_id, rule) in rules.iter().enumerate() {
|
|
||||||
if !rule.enabled {
|
|
||||||
log::trace!("rule is not enabled: {}", rule_id);
|
|
||||||
continue;
|
|
||||||
}
|
|
||||||
if rule.accept(&event.outcome, &event.stage, &event.substage) {
|
|
||||||
let plugin = plugins
|
|
||||||
.get(method)
|
|
||||||
.ok_or(NotifyError::UnknownPlugin(method.clone()))?;
|
|
||||||
let renderer = plugin.metadata.renderer.clone().unwrap_or_default();
|
|
||||||
match renderer {
|
|
||||||
NotificationRenderer::Server => {
|
|
||||||
// ...
|
|
||||||
todo!();
|
|
||||||
}
|
|
||||||
NotificationRenderer::Plugin => {
|
|
||||||
unimplemented!();
|
|
||||||
return Err(NotifyError::UnsupportedRenderer(method.clone(), renderer));
|
|
||||||
}
|
|
||||||
NotificationRenderer::Interactive => {
|
|
||||||
unimplemented!();
|
|
||||||
return Err(NotifyError::UnsupportedRenderer(method.clone(), renderer));
|
|
||||||
}
|
|
||||||
}
|
|
||||||
}
|
|
||||||
}
|
|
||||||
}
|
|
||||||
}
|
|
||||||
|
|
||||||
todo!();
|
#[cfg(test)]
|
||||||
}
|
mod tests {
|
||||||
|
use super::*;
|
||||||
|
use crate::finalize::NotificationTemplateDef;
|
||||||
|
use crate::notify::types::NotificationRule;
|
||||||
|
use crate::notify::types::NotificationTarget;
|
||||||
|
use std::collections::HashMap;
|
||||||
|
use crate::notify::types::NotificationEvent;
|
||||||
|
use crate::notify::types::NotificationRulesMap;
|
||||||
|
use crate::notify::error::NotifyError;
|
||||||
|
use crate::notify::template::resolve_template;
|
||||||
|
use crate::notify::util::{find_rule, notify_delay, wait_time};
|
||||||
|
|
||||||
fn parse_trigger_item_raw(item: &TriggerItemRaw) -> Vec<ParsedTrigger> {
|
// ── helper ──
|
||||||
match item {
|
|
||||||
TriggerItemRaw::Simple(s) => match s.as_str() {
|
|
||||||
"on_success" => vec![ParsedTrigger::OnSuccess],
|
|
||||||
"on_failure" => vec![ParsedTrigger::OnFailure],
|
|
||||||
s if s.starts_with("on_finish_of:") => {
|
|
||||||
let pattern = s.trim_start_matches("on_finish_of:").trim();
|
|
||||||
let mut builder = GlobSetBuilder::new();
|
|
||||||
builder.add(Glob::new(pattern).expect("Invalid glob pattern"));
|
|
||||||
let glob_set = builder.build().expect("Failed to compile glob set");
|
|
||||||
vec![ParsedTrigger::OnFinishOf(Matchable::Glob(glob_set))]
|
|
||||||
}
|
|
||||||
_ => {
|
|
||||||
log::warn!("Unknown trigger string: {}", s);
|
|
||||||
vec![]
|
|
||||||
}
|
|
||||||
},
|
|
||||||
TriggerItemRaw::Map(map) => {
|
|
||||||
let mut triggers = Vec::new();
|
|
||||||
for (key, patterns) in map {
|
|
||||||
match key.as_str() {
|
|
||||||
"on_finish_of" => {
|
|
||||||
for pattern in patterns {
|
|
||||||
let mut builder = GlobSetBuilder::new();
|
|
||||||
builder.add(Glob::new(pattern).expect("Invalid glob pattern"));
|
|
||||||
let glob_set = builder.build().expect("Failed to compile glob set");
|
|
||||||
triggers.push(ParsedTrigger::OnFinishOf(Matchable::Glob(glob_set)));
|
|
||||||
}
|
|
||||||
}
|
|
||||||
"on_success" => triggers.push(ParsedTrigger::OnSuccess),
|
|
||||||
"on_failure" => triggers.push(ParsedTrigger::OnFailure),
|
|
||||||
_ => {
|
|
||||||
log::warn!("Unknown trigger key: {}", key);
|
|
||||||
}
|
|
||||||
}
|
|
||||||
}
|
|
||||||
triggers
|
|
||||||
}
|
|
||||||
}
|
|
||||||
}
|
|
||||||
|
|
||||||
fn merge_templates(
|
fn make_tmpl(on_success: &str, on_failure: &str) -> NotificationTemplateDef {
|
||||||
method_template: &NotificationTemplate,
|
NotificationTemplateDef {
|
||||||
policy_template: &Option<serde_yaml::Value>,
|
use_: None,
|
||||||
) -> NotificationTemplate {
|
on_success: if on_success.is_empty() {
|
||||||
let mut merged = method_template.clone();
|
None
|
||||||
if let Some(policy_tmpl) = policy_template
|
|
||||||
&& let Ok(policy_template) =
|
|
||||||
serde_yaml::from_value::<NotificationTemplate>(policy_tmpl.clone())
|
|
||||||
{
|
|
||||||
if policy_template.schema.is_some() {
|
|
||||||
merged.schema = policy_template.schema;
|
|
||||||
}
|
|
||||||
if policy_template.on_success.is_some() {
|
|
||||||
merged.on_success = policy_template.on_success;
|
|
||||||
}
|
|
||||||
if policy_template.on_failure.is_some() {
|
|
||||||
merged.on_failure = policy_template.on_failure;
|
|
||||||
}
|
|
||||||
if policy_template.on_finish_of.is_some() {
|
|
||||||
merged.on_finish_of = policy_template.on_finish_of;
|
|
||||||
}
|
|
||||||
}
|
|
||||||
merged
|
|
||||||
}
|
|
||||||
|
|
||||||
fn extract_overrides(overrides: &serde_yaml::Value) -> (bool, u32, std::time::Duration, f64) {
|
|
||||||
let fallible = overrides
|
|
||||||
.get("fallible")
|
|
||||||
.and_then(|v| v.as_bool())
|
|
||||||
.unwrap_or(NOTIFICATION_DEFAULT_FALLIBLE);
|
|
||||||
let max_lives = overrides
|
|
||||||
.get("max_lives")
|
|
||||||
.and_then(|v| v.as_u64())
|
|
||||||
.unwrap_or(NOTIFICATION_DEFAULT_MAX_LIVES as u64) as u32;
|
|
||||||
let timeout_ms = overrides
|
|
||||||
.get("timeout_ms")
|
|
||||||
.and_then(|v| v.as_u64())
|
|
||||||
.unwrap_or(NOTIFICATION_DEFAULT_TIMEOUT_MS);
|
|
||||||
let retry_backoff = overrides
|
|
||||||
.get("retry_backoff")
|
|
||||||
.and_then(|v| v.as_f64())
|
|
||||||
.unwrap_or(NOTIFICATION_DEFAULT_RETRY_BACKOFF);
|
|
||||||
(
|
|
||||||
fallible,
|
|
||||||
max_lives,
|
|
||||||
std::time::Duration::from_millis(timeout_ms),
|
|
||||||
retry_backoff,
|
|
||||||
)
|
|
||||||
}
|
|
||||||
|
|
||||||
fn parse_rules(config: &NotificationConfig) -> NotificationRulesMap {
|
|
||||||
let mut rules = NotificationRulesMap::default();
|
|
||||||
for (priority, group) in &config.groups {
|
|
||||||
let mut ruleset: HashMap<Method, Vec<NotificationRule>> = HashMap::new();
|
|
||||||
for (method_name, method) in group {
|
|
||||||
let mut method_rules: Vec<NotificationRule> = Vec::new();
|
|
||||||
|
|
||||||
if method.policy.is_empty() {
|
|
||||||
let trigger: Vec<ParsedTrigger> = method
|
|
||||||
.trigger
|
|
||||||
.iter()
|
|
||||||
.flat_map(parse_trigger_item_raw)
|
|
||||||
.collect();
|
|
||||||
let template = method.template.clone();
|
|
||||||
let (fallible, max_lives, timeout, retry_backoff) =
|
|
||||||
extract_overrides(&method.config);
|
|
||||||
method_rules.push(NotificationRule {
|
|
||||||
enabled: true,
|
|
||||||
trigger,
|
|
||||||
template,
|
|
||||||
fallible,
|
|
||||||
max_lives,
|
|
||||||
timeout,
|
|
||||||
config: method.config.clone(),
|
|
||||||
retry_backoff,
|
|
||||||
});
|
|
||||||
} else {
|
} else {
|
||||||
for policy in &method.policy {
|
Some(on_success.into())
|
||||||
let trigger: Vec<ParsedTrigger> = if policy.trigger.is_empty() {
|
},
|
||||||
method
|
on_failure: if on_failure.is_empty() {
|
||||||
.trigger
|
None
|
||||||
.iter()
|
} else {
|
||||||
.flat_map(parse_trigger_item_raw)
|
Some(on_failure.into())
|
||||||
.collect()
|
},
|
||||||
} else {
|
on_finish_of: None,
|
||||||
policy
|
|
||||||
.trigger
|
|
||||||
.iter()
|
|
||||||
.flat_map(parse_trigger_item_raw)
|
|
||||||
.collect()
|
|
||||||
};
|
|
||||||
let template =
|
|
||||||
merge_templates(&method.template, &Some(policy.template.clone()));
|
|
||||||
let method_overrides = extract_overrides(&method.config);
|
|
||||||
let policy_overrides = extract_overrides(&policy.overrides);
|
|
||||||
let fallible = if policy.overrides.get("fallible").is_some() {
|
|
||||||
policy_overrides.0
|
|
||||||
} else {
|
|
||||||
method_overrides.0
|
|
||||||
};
|
|
||||||
let max_lives = if policy.overrides.get("max_lives").is_some() {
|
|
||||||
policy_overrides.1
|
|
||||||
} else {
|
|
||||||
method_overrides.1
|
|
||||||
};
|
|
||||||
let timeout = if policy.overrides.get("timeout_ms").is_some() {
|
|
||||||
policy_overrides.2
|
|
||||||
} else {
|
|
||||||
method_overrides.2
|
|
||||||
};
|
|
||||||
let retry_backoff = if policy.overrides.get("retry_backoff").is_some() {
|
|
||||||
policy_overrides.3
|
|
||||||
} else {
|
|
||||||
method_overrides.3
|
|
||||||
};
|
|
||||||
method_rules.push(NotificationRule {
|
|
||||||
enabled: true,
|
|
||||||
trigger,
|
|
||||||
template,
|
|
||||||
fallible,
|
|
||||||
max_lives,
|
|
||||||
timeout,
|
|
||||||
config: method.config.clone(),
|
|
||||||
retry_backoff,
|
|
||||||
});
|
|
||||||
}
|
|
||||||
}
|
|
||||||
|
|
||||||
ruleset.insert(method_name.clone(), method_rules);
|
|
||||||
}
|
}
|
||||||
rules.insert(*priority, ruleset);
|
|
||||||
}
|
}
|
||||||
rules
|
|
||||||
}
|
|
||||||
|
|
||||||
// calculate min(1, k/x)*t as notification grouping period
|
fn make_tmpl_with_use(use_ref: &str) -> NotificationTemplateDef {
|
||||||
// to avoid excessive notification frequency
|
NotificationTemplateDef {
|
||||||
fn wait_time(group_period: u32, group_watermark: u32, count: u32) -> f32 {
|
use_: Some(use_ref.into()),
|
||||||
if count == 0 {
|
on_success: None,
|
||||||
log::warn!("Unexpected notification count: {}", count);
|
on_failure: None,
|
||||||
return group_period as f32;
|
on_finish_of: None,
|
||||||
|
}
|
||||||
}
|
}
|
||||||
f32::min(1.0, group_watermark as f32 / count as f32) * group_period as f32
|
|
||||||
}
|
|
||||||
|
|
||||||
// calculate p^(m-l)*t as notification delay
|
// ═══════════════════════════════════════════
|
||||||
fn notify_delay(event: &NotificationEvent) -> std::time::Duration {
|
// resolve_template_content
|
||||||
let exponent = (event.max_lives - event.lives) as i32;
|
// ═══════════════════════════════════════════
|
||||||
if exponent < 0 {
|
|
||||||
log::warn!(
|
#[test]
|
||||||
"Unexpected lives in notification: lives({}) > max_lives({})",
|
fn resolve_none_cd_default_found() {
|
||||||
event.lives,
|
let mut ws = HashMap::new();
|
||||||
event.max_lives
|
ws.insert("mail".into(), make_tmpl("OK", "FAIL"));
|
||||||
);
|
let templates = HashMap::new();
|
||||||
return std::time::Duration::from_secs_f64(NOTIFICATION_TEMPORARY_MAX_TIME);
|
let result = resolve_template(&None, "mail", &templates, false, Some(&ws));
|
||||||
|
assert!(result.is_ok());
|
||||||
|
assert_eq!(result.unwrap().on_success.unwrap(), "OK");
|
||||||
}
|
}
|
||||||
let multiplier = NOTIFICATION_TEMPORARY_DELAY_FACTOR.powi(exponent);
|
|
||||||
let delay = multiplier * NOTIFICATION_TEMPORARY_DELAY_TIME;
|
#[test]
|
||||||
if !delay.is_finite() || delay < 0.0 {
|
fn resolve_none_cd_default_missing_fallback() {
|
||||||
log::warn!(
|
let ws: HashMap<String, NotificationTemplateDef> = HashMap::new();
|
||||||
"Unexpected calculated delay: {}, treating as infinite",
|
let mut templates = HashMap::new();
|
||||||
delay
|
templates.insert("mail.fallback".into(), make_tmpl("FALLBACK", ""));
|
||||||
);
|
let result = resolve_template(&None, "mail", &templates, false, Some(&ws));
|
||||||
return std::time::Duration::from_secs_f64(NOTIFICATION_TEMPORARY_MAX_TIME);
|
assert!(result.is_ok());
|
||||||
|
assert_eq!(result.unwrap().on_success.unwrap(), "FALLBACK");
|
||||||
|
}
|
||||||
|
|
||||||
|
#[test]
|
||||||
|
fn resolve_none_cd_no_default_no_fallback_error() {
|
||||||
|
let ws: HashMap<String, NotificationTemplateDef> = HashMap::new();
|
||||||
|
let templates = HashMap::new();
|
||||||
|
let result = resolve_template(&None, "mail", &templates, false, Some(&ws));
|
||||||
|
assert!(result.is_err());
|
||||||
|
assert!(format!("{}", result.unwrap_err()).contains("failed"));
|
||||||
|
}
|
||||||
|
|
||||||
|
#[test]
|
||||||
|
fn resolve_none_standalone_fallback() {
|
||||||
|
let mut templates = HashMap::new();
|
||||||
|
templates.insert("mail.fallback".into(), make_tmpl("STANDALONE", ""));
|
||||||
|
let result = resolve_template(&None, "mail", &templates, true, None);
|
||||||
|
assert!(result.is_ok());
|
||||||
|
assert_eq!(result.unwrap().on_success.unwrap(), "STANDALONE");
|
||||||
|
}
|
||||||
|
|
||||||
|
#[test]
|
||||||
|
fn resolve_none_standalone_no_fallback_error() {
|
||||||
|
let result = resolve_template(&None, "mail", &HashMap::new(), true, None);
|
||||||
|
assert!(result.is_err());
|
||||||
|
}
|
||||||
|
|
||||||
|
#[test]
|
||||||
|
fn resolve_custom() {
|
||||||
|
let result = resolve_template(&Some("custom".into()), "mail", &HashMap::new(), false, None);
|
||||||
|
assert!(result.is_ok());
|
||||||
|
// "custom" returns an empty default — caller reads rule.template.on_* directly
|
||||||
|
}
|
||||||
|
|
||||||
|
#[test]
|
||||||
|
fn resolve_default_standalone_fallback() {
|
||||||
|
let mut templates = HashMap::new();
|
||||||
|
templates.insert("mail.fallback".into(), make_tmpl("FB", ""));
|
||||||
|
let result = resolve_template(&Some("default".into()), "mail", &templates, true, None);
|
||||||
|
assert!(result.is_ok());
|
||||||
|
assert_eq!(result.unwrap().on_success.unwrap(), "FB");
|
||||||
|
}
|
||||||
|
|
||||||
|
#[test]
|
||||||
|
fn resolve_default_standalone_no_fallback_error() {
|
||||||
|
let result = resolve_template(&Some("default".into()), "mail", &HashMap::new(), true, None);
|
||||||
|
assert!(result.is_err());
|
||||||
|
}
|
||||||
|
|
||||||
|
#[test]
|
||||||
|
fn resolve_default_cd_wsbase() {
|
||||||
|
let mut ws = HashMap::new();
|
||||||
|
ws.insert("mail".into(), make_tmpl("WS_OK", ""));
|
||||||
|
let result = resolve_template(
|
||||||
|
&Some("default".into()),
|
||||||
|
"mail",
|
||||||
|
&HashMap::new(),
|
||||||
|
false,
|
||||||
|
Some(&ws),
|
||||||
|
);
|
||||||
|
assert!(result.is_ok());
|
||||||
|
assert_eq!(result.unwrap().on_success.unwrap(), "WS_OK");
|
||||||
|
}
|
||||||
|
|
||||||
|
#[test]
|
||||||
|
fn resolve_default_cd_no_wsbase_error() {
|
||||||
|
let result = resolve_template(
|
||||||
|
&Some("default".into()),
|
||||||
|
"mail",
|
||||||
|
&HashMap::new(),
|
||||||
|
false,
|
||||||
|
None,
|
||||||
|
);
|
||||||
|
assert!(result.is_err());
|
||||||
|
}
|
||||||
|
|
||||||
|
#[test]
|
||||||
|
fn resolve_named_found() {
|
||||||
|
let mut templates = HashMap::new();
|
||||||
|
templates.insert("sms-tmpl1".into(), make_tmpl("NAMED", ""));
|
||||||
|
let result = resolve_template(&Some("sms-tmpl1".into()), "mail", &templates, false, None);
|
||||||
|
assert!(result.is_ok());
|
||||||
|
assert_eq!(result.unwrap().on_success.unwrap(), "NAMED");
|
||||||
|
}
|
||||||
|
|
||||||
|
#[test]
|
||||||
|
fn resolve_named_not_found_method_match_fallback() {
|
||||||
|
let mut templates = HashMap::new();
|
||||||
|
templates.insert("mail.fallback".into(), make_tmpl("FB", ""));
|
||||||
|
let result = resolve_template(&Some("mail".into()), "mail", &templates, false, None);
|
||||||
|
assert!(result.is_ok());
|
||||||
|
assert_eq!(result.unwrap().on_success.unwrap(), "FB");
|
||||||
|
}
|
||||||
|
|
||||||
|
#[test]
|
||||||
|
fn resolve_named_not_found_no_match_error() {
|
||||||
|
let result = resolve_template(&Some("ghost".into()), "mail", &HashMap::new(), false, None);
|
||||||
|
assert!(result.is_err());
|
||||||
|
}
|
||||||
|
|
||||||
|
// ═══════════════════════════════════════════
|
||||||
|
// flatten_template_refs
|
||||||
|
// ═══════════════════════════════════════════
|
||||||
|
|
||||||
|
#[test]
|
||||||
|
fn flatten_no_refs_unchanged() {
|
||||||
|
let mut templates = HashMap::new();
|
||||||
|
templates.insert("a".into(), make_tmpl("A_OK", "A_FAIL"));
|
||||||
|
let result = flatten_template_refs(&mut templates);
|
||||||
|
assert!(result.is_ok());
|
||||||
|
assert_eq!(templates["a"].on_success.as_deref(), Some("A_OK"));
|
||||||
|
assert!(templates["a"].use_.is_none());
|
||||||
|
}
|
||||||
|
|
||||||
|
#[test]
|
||||||
|
fn flatten_single_ref_merged() {
|
||||||
|
let mut templates = HashMap::new();
|
||||||
|
templates.insert("base".into(), make_tmpl("BASE_OK", "BASE_FAIL"));
|
||||||
|
templates.insert("child".into(), make_tmpl_with_use("base"));
|
||||||
|
let result = flatten_template_refs(&mut templates);
|
||||||
|
assert!(result.is_ok());
|
||||||
|
assert_eq!(templates["child"].on_success.as_deref(), Some("BASE_OK"));
|
||||||
|
assert_eq!(templates["child"].on_failure.as_deref(), Some("BASE_FAIL"));
|
||||||
|
assert!(templates["child"].use_.is_none());
|
||||||
|
}
|
||||||
|
|
||||||
|
#[test]
|
||||||
|
fn flatten_child_overrides_parent() {
|
||||||
|
let mut templates = HashMap::new();
|
||||||
|
templates.insert("base".into(), make_tmpl("BASE_OK", "BASE_FAIL"));
|
||||||
|
templates.insert(
|
||||||
|
"child".into(),
|
||||||
|
NotificationTemplateDef {
|
||||||
|
use_: Some("base".into()),
|
||||||
|
on_success: Some("MY_OK".into()),
|
||||||
|
on_failure: None,
|
||||||
|
on_finish_of: None,
|
||||||
|
},
|
||||||
|
);
|
||||||
|
let result = flatten_template_refs(&mut templates);
|
||||||
|
assert!(result.is_ok());
|
||||||
|
assert_eq!(templates["child"].on_success.as_deref(), Some("MY_OK"));
|
||||||
|
assert_eq!(templates["child"].on_failure.as_deref(), Some("BASE_FAIL"));
|
||||||
|
}
|
||||||
|
|
||||||
|
#[test]
|
||||||
|
fn flatten_chain_a_b_c() {
|
||||||
|
let mut templates = HashMap::new();
|
||||||
|
templates.insert("c".into(), make_tmpl("C_OK", "C_FAIL"));
|
||||||
|
templates.insert("b".into(), make_tmpl_with_use("c"));
|
||||||
|
templates.insert("a".into(), make_tmpl_with_use("b"));
|
||||||
|
let result = flatten_template_refs(&mut templates);
|
||||||
|
assert!(result.is_ok());
|
||||||
|
assert_eq!(templates["a"].on_success.as_deref(), Some("C_OK"));
|
||||||
|
assert_eq!(templates["b"].on_success.as_deref(), Some("C_OK"));
|
||||||
|
}
|
||||||
|
|
||||||
|
#[test]
|
||||||
|
fn flatten_cycle_detected() {
|
||||||
|
let mut templates = HashMap::new();
|
||||||
|
templates.insert("x".into(), make_tmpl_with_use("y"));
|
||||||
|
templates.insert("y".into(), make_tmpl_with_use("x"));
|
||||||
|
let result = flatten_template_refs(&mut templates);
|
||||||
|
assert!(result.is_err());
|
||||||
|
match result.unwrap_err() {
|
||||||
|
NotifyError::TemplateLoop(_) => {}
|
||||||
|
other => panic!("expected TemplateLoop, got {:?}", other),
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
#[test]
|
||||||
|
fn flatten_self_ref_detected() {
|
||||||
|
let mut templates = HashMap::new();
|
||||||
|
templates.insert("self".into(), make_tmpl_with_use("self"));
|
||||||
|
let result = flatten_template_refs(&mut templates);
|
||||||
|
assert!(result.is_err());
|
||||||
|
match result.unwrap_err() {
|
||||||
|
NotifyError::TemplateLoop(_) => {}
|
||||||
|
other => panic!("expected TemplateLoop, got {:?}", other),
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
#[test]
|
||||||
|
fn flatten_nonexistent_ref_error() {
|
||||||
|
let mut templates = HashMap::new();
|
||||||
|
templates.insert("orphan".into(), make_tmpl_with_use("ghost"));
|
||||||
|
let result = flatten_template_refs(&mut templates);
|
||||||
|
assert!(result.is_err());
|
||||||
|
}
|
||||||
|
|
||||||
|
#[test]
|
||||||
|
fn flatten_external_url_preserved() {
|
||||||
|
let mut templates = HashMap::new();
|
||||||
|
templates.insert(
|
||||||
|
"ext".into(),
|
||||||
|
NotificationTemplateDef {
|
||||||
|
use_: Some("git://example.com/repo@v1".into()),
|
||||||
|
on_success: None,
|
||||||
|
on_failure: None,
|
||||||
|
on_finish_of: None,
|
||||||
|
},
|
||||||
|
);
|
||||||
|
let result = flatten_template_refs(&mut templates);
|
||||||
|
assert!(result.is_ok());
|
||||||
|
// External URLs are NOT flattened — use_ preserved
|
||||||
|
assert_eq!(
|
||||||
|
templates["ext"].use_.as_deref(),
|
||||||
|
Some("git://example.com/repo@v1")
|
||||||
|
);
|
||||||
|
}
|
||||||
|
|
||||||
|
// ═══════════════════════════════════════════
|
||||||
|
// wait_time
|
||||||
|
// ═══════════════════════════════════════════
|
||||||
|
|
||||||
|
#[test]
|
||||||
|
fn wait_time_zero_count() {
|
||||||
|
let w = wait_time(30, 5, 0);
|
||||||
|
assert_eq!(w, 30.0);
|
||||||
|
}
|
||||||
|
|
||||||
|
#[test]
|
||||||
|
fn wait_time_at_watermark() {
|
||||||
|
let w = wait_time(30, 5, 5);
|
||||||
|
assert_eq!(w, 30.0);
|
||||||
|
}
|
||||||
|
|
||||||
|
#[test]
|
||||||
|
fn wait_time_below_watermark() {
|
||||||
|
let w = wait_time(30, 5, 2);
|
||||||
|
assert!((w - 30.0).abs() < 0.01);
|
||||||
|
}
|
||||||
|
|
||||||
|
#[test]
|
||||||
|
fn wait_time_above_watermark() {
|
||||||
|
let w = wait_time(30, 5, 10);
|
||||||
|
assert!((w - 15.0).abs() < 0.01);
|
||||||
|
}
|
||||||
|
|
||||||
|
#[test]
|
||||||
|
fn wait_time_far_above_watermark() {
|
||||||
|
let w = wait_time(30, 5, 100);
|
||||||
|
assert!((w - 1.5).abs() < 0.01);
|
||||||
|
}
|
||||||
|
|
||||||
|
// ═══════════════════════════════════════════
|
||||||
|
// notify_delay
|
||||||
|
// ═══════════════════════════════════════════
|
||||||
|
|
||||||
|
fn make_event(count: u32, retry_backoff: f64) -> NotificationEvent {
|
||||||
|
NotificationEvent {
|
||||||
|
id: uuid::Uuid::new_v4(),
|
||||||
|
stage: crate::types::buildstatus::StagePhase::Bake,
|
||||||
|
substage: "build".into(),
|
||||||
|
outcome: crate::types::buildstatus::StageOutcome::PipelineSuccess,
|
||||||
|
not_before: chrono::Utc::now(),
|
||||||
|
priority: 0,
|
||||||
|
effective_priority: 0,
|
||||||
|
attempted: 0,
|
||||||
|
retry_backoff,
|
||||||
|
target: NotificationTarget {
|
||||||
|
priority: 0,
|
||||||
|
rules: None,
|
||||||
|
count,
|
||||||
|
},
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
#[test]
|
||||||
|
fn notify_delay_default_factor() {
|
||||||
|
let event = make_event(1, 2.0);
|
||||||
|
let delay = notify_delay(&event);
|
||||||
|
assert!(delay.as_secs_f64() > 0.0);
|
||||||
|
}
|
||||||
|
|
||||||
|
#[test]
|
||||||
|
fn notify_delay_larger_factor() {
|
||||||
|
let event1 = make_event(1, 2.0);
|
||||||
|
let event2 = make_event(1, 4.0);
|
||||||
|
let d1 = notify_delay(&event1).as_secs_f64();
|
||||||
|
let d2 = notify_delay(&event2).as_secs_f64();
|
||||||
|
assert!(d2 > d1, "larger retry_backoff should produce longer delay");
|
||||||
|
}
|
||||||
|
|
||||||
|
// ═══════════════════════════════════════════
|
||||||
|
// find_rule
|
||||||
|
// ═══════════════════════════════════════════
|
||||||
|
|
||||||
|
#[test]
|
||||||
|
fn find_rule_found() {
|
||||||
|
use crate::finalize::NotificationTemplate;
|
||||||
|
let rule = NotificationRule {
|
||||||
|
enabled: true,
|
||||||
|
trigger: vec![],
|
||||||
|
template: NotificationTemplate {
|
||||||
|
schema: None,
|
||||||
|
on_success: None,
|
||||||
|
on_failure: None,
|
||||||
|
on_finish_of: None,
|
||||||
|
},
|
||||||
|
fallible: false,
|
||||||
|
max_lives: 3,
|
||||||
|
timeout: std::time::Duration::from_secs(30),
|
||||||
|
config: serde_yaml::Value::Null,
|
||||||
|
retry_backoff: 2.0,
|
||||||
|
};
|
||||||
|
let mut rules_map: NotificationRulesMap = HashMap::new();
|
||||||
|
rules_map.insert(0, {
|
||||||
|
let mut m: HashMap<String, Vec<NotificationRule>> = HashMap::new();
|
||||||
|
m.insert("mail".into(), vec![rule]);
|
||||||
|
m
|
||||||
|
});
|
||||||
|
let result = find_rule(&rules_map, "mail", 0);
|
||||||
|
assert!(result.is_ok());
|
||||||
|
}
|
||||||
|
|
||||||
|
#[test]
|
||||||
|
fn find_rule_not_found() {
|
||||||
|
let rules_map: NotificationRulesMap = HashMap::new();
|
||||||
|
let result = find_rule(&rules_map, "mail", 0);
|
||||||
|
assert!(result.is_err());
|
||||||
}
|
}
|
||||||
std::time::Duration::from_secs_f64(delay)
|
|
||||||
}
|
}
|
||||||
|
|||||||
@@ -1,4 +1,4 @@
|
|||||||
use crate::notify::NotificationRenderer;
|
use crate::notify::types::NotificationRenderer;
|
||||||
use thiserror::Error;
|
use thiserror::Error;
|
||||||
|
|
||||||
#[derive(Error, Debug)]
|
#[derive(Error, Debug)]
|
||||||
@@ -23,4 +23,16 @@ pub enum NotifyError {
|
|||||||
|
|
||||||
#[error("Unsupported renderer for {0}: {1}")]
|
#[error("Unsupported renderer for {0}: {1}")]
|
||||||
UnsupportedRenderer(String, NotificationRenderer),
|
UnsupportedRenderer(String, NotificationRenderer),
|
||||||
|
|
||||||
|
#[error("Schema error: {0}")]
|
||||||
|
SchemaError(String),
|
||||||
|
|
||||||
|
#[error("Template render error: {0}")]
|
||||||
|
TemplateRenderError(String),
|
||||||
|
|
||||||
|
#[error("Circular template reference: {0}")]
|
||||||
|
TemplateLoop(String),
|
||||||
|
|
||||||
|
#[error("Template reference depth exceeded (max {max}): {name}")]
|
||||||
|
TemplateDepthExceeded { name: String, max: u32 },
|
||||||
}
|
}
|
||||||
|
|||||||
@@ -0,0 +1,345 @@
|
|||||||
|
use std::collections::HashMap;
|
||||||
|
use std::sync::atomic::{AtomicBool, Ordering};
|
||||||
|
use std::sync::{Arc, Mutex};
|
||||||
|
use std::time::Duration;
|
||||||
|
|
||||||
|
use chrono::Utc;
|
||||||
|
use tokio::sync::Notify;
|
||||||
|
|
||||||
|
use crate::finalize::NotificationTemplateDef;
|
||||||
|
use crate::notify::template::{
|
||||||
|
build_batch_template_context, render_template, resolve_template,
|
||||||
|
select_template_str,
|
||||||
|
};
|
||||||
|
use crate::notify::types::{
|
||||||
|
MethodStatus, NotificationEvent,
|
||||||
|
NotificationEventSender, NotificationPluginMap, NotificationRenderer,
|
||||||
|
NotificationRulesMap,
|
||||||
|
};
|
||||||
|
use crate::notify::util::{
|
||||||
|
find_rule, get_fallible, is_temporary_error, notify_delay, wait_time,
|
||||||
|
};
|
||||||
|
|
||||||
|
/// Shared state for a single dispatch bucket.
|
||||||
|
struct BucketEntry {
|
||||||
|
events: Arc<Mutex<Vec<NotificationEvent>>>,
|
||||||
|
result: Arc<Mutex<Option<(MethodStatus, Vec<NotificationEvent>)>>>,
|
||||||
|
done: Arc<AtomicBool>,
|
||||||
|
notify: Arc<Notify>,
|
||||||
|
method: String,
|
||||||
|
rule_id: usize,
|
||||||
|
_handle: tokio::task::JoinHandle<()>,
|
||||||
|
}
|
||||||
|
|
||||||
|
/// Concurrent notification handler with bucket-based dispatch.
|
||||||
|
///
|
||||||
|
/// Events are routed to matching buckets. Buckets cool down and dispatch
|
||||||
|
/// plugins concurrently. The caller awaits bucket completion, then handles
|
||||||
|
/// retry/escalation decisions.
|
||||||
|
pub struct NotificationHandler {
|
||||||
|
buckets: Mutex<HashMap<(String, usize), BucketEntry>>,
|
||||||
|
pub config: Arc<NotificationRulesMap>,
|
||||||
|
plugins: Arc<NotificationPluginMap>,
|
||||||
|
templates: Arc<HashMap<String, NotificationTemplateDef>>,
|
||||||
|
ctx: Arc<workshop_engine::ExecutionContext>,
|
||||||
|
retry_tx: NotificationEventSender,
|
||||||
|
period: u32,
|
||||||
|
watermark: u32,
|
||||||
|
}
|
||||||
|
|
||||||
|
impl NotificationHandler {
|
||||||
|
#[allow(clippy::too_many_arguments)]
|
||||||
|
pub fn new(
|
||||||
|
config: Arc<NotificationRulesMap>,
|
||||||
|
plugins: Arc<NotificationPluginMap>,
|
||||||
|
templates: Arc<HashMap<String, NotificationTemplateDef>>,
|
||||||
|
ctx: Arc<workshop_engine::ExecutionContext>,
|
||||||
|
retry_tx: NotificationEventSender,
|
||||||
|
period: u32,
|
||||||
|
watermark: u32,
|
||||||
|
) -> Self {
|
||||||
|
NotificationHandler {
|
||||||
|
buckets: Mutex::new(HashMap::new()),
|
||||||
|
config,
|
||||||
|
plugins,
|
||||||
|
templates,
|
||||||
|
ctx,
|
||||||
|
retry_tx,
|
||||||
|
period,
|
||||||
|
watermark,
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
/// Process a single event: route to buckets, await completion,
|
||||||
|
/// then handle retry/escalation for each bucket result.
|
||||||
|
pub async fn process_event(self: Arc<Self>, event: NotificationEvent) {
|
||||||
|
let entries = self.route_event(&event);
|
||||||
|
if entries.is_empty() {
|
||||||
|
return;
|
||||||
|
}
|
||||||
|
|
||||||
|
// Await all buckets — yields CPU
|
||||||
|
for (_, notify) in &entries {
|
||||||
|
notify.notified().await;
|
||||||
|
}
|
||||||
|
|
||||||
|
// Collect results and remove entries
|
||||||
|
let mut results: Vec<(String, usize, Option<(MethodStatus, Vec<NotificationEvent>)>)> =
|
||||||
|
Vec::new();
|
||||||
|
{
|
||||||
|
let mut buckets = self.buckets.lock().unwrap();
|
||||||
|
for (key, _) in &entries {
|
||||||
|
if let Some(entry) = buckets.remove(key) {
|
||||||
|
let r = entry.result.lock().unwrap().take();
|
||||||
|
results.push((entry.method, entry.rule_id, r));
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
// Handle retry/escalation per method
|
||||||
|
for (method, _rule_id, result) in results {
|
||||||
|
let Some((status, mut events)) = result else { continue };
|
||||||
|
|
||||||
|
match &status {
|
||||||
|
MethodStatus::Success => {
|
||||||
|
log::debug!("[NOTIFY] {}: success ({} events)", method, events.len());
|
||||||
|
}
|
||||||
|
MethodStatus::TemporaryError(_) => {
|
||||||
|
log::warn!("[NOTIFY] {}: temporary error, retrying", method);
|
||||||
|
for event in events.iter_mut() {
|
||||||
|
event.target.rules = Some(vec![method.clone()]);
|
||||||
|
event.target.count += 1;
|
||||||
|
event.not_before = Utc::now() + notify_delay(event);
|
||||||
|
let _ = self.retry_tx.send(event.clone()).await;
|
||||||
|
}
|
||||||
|
}
|
||||||
|
MethodStatus::PermanentError(_) | MethodStatus::Exhausted(_) => {
|
||||||
|
let fallible = get_fallible(&self.config, &method);
|
||||||
|
if fallible {
|
||||||
|
log::error!("[NOTIFY] {}: discard (fallible)", method);
|
||||||
|
} else {
|
||||||
|
log::warn!("[NOTIFY] {}: escalating priority", method);
|
||||||
|
for event in events.iter_mut() {
|
||||||
|
event.target.priority += 1;
|
||||||
|
event.target.rules = None;
|
||||||
|
event.attempted = event.target.count;
|
||||||
|
event.target.count = 0;
|
||||||
|
event.effective_priority = event.priority;
|
||||||
|
let _ = self.retry_tx.send(event.clone()).await;
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
/// Route a single event to all matching buckets.
|
||||||
|
///
|
||||||
|
/// Returns Notify handles for awaiting bucket completion.
|
||||||
|
fn route_event(
|
||||||
|
&self,
|
||||||
|
event: &NotificationEvent,
|
||||||
|
) -> Vec<((String, usize), Arc<Notify>)> {
|
||||||
|
let mut matches: Vec<(String, usize)> = Vec::new();
|
||||||
|
|
||||||
|
for ruleset in self.config.values() {
|
||||||
|
for (method, rules) in ruleset.iter() {
|
||||||
|
if let Some(ref method_rules) = event.target.rules {
|
||||||
|
if !method_rules.contains(method) {
|
||||||
|
continue;
|
||||||
|
}
|
||||||
|
}
|
||||||
|
for (rule_id, rule) in rules.iter().enumerate() {
|
||||||
|
if !rule.enabled {
|
||||||
|
continue;
|
||||||
|
}
|
||||||
|
if rule.accept(&event.outcome, &event.stage, &event.substage) {
|
||||||
|
matches.push((method.clone(), rule_id));
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
let mut buckets = self.buckets.lock().unwrap();
|
||||||
|
let mut result: Vec<((String, usize), Arc<Notify>)> = Vec::new();
|
||||||
|
|
||||||
|
for key in matches {
|
||||||
|
if let Some(entry) = buckets.get(&key) {
|
||||||
|
if entry.done.load(Ordering::Acquire) {
|
||||||
|
buckets.remove(&key);
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
if let Some(entry) = buckets.get(&key) {
|
||||||
|
entry.events.lock().unwrap().push(event.clone());
|
||||||
|
result.push((key, entry.notify.clone()));
|
||||||
|
} else {
|
||||||
|
let (entry, notify) = Self::spawn_bucket(
|
||||||
|
&key.0,
|
||||||
|
key.1,
|
||||||
|
&self.config,
|
||||||
|
&self.plugins,
|
||||||
|
&self.templates,
|
||||||
|
&self.ctx,
|
||||||
|
self.period,
|
||||||
|
self.watermark,
|
||||||
|
);
|
||||||
|
entry.events.lock().unwrap().push(event.clone());
|
||||||
|
result.push((key.clone(), notify));
|
||||||
|
buckets.insert(key, entry);
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
result
|
||||||
|
}
|
||||||
|
|
||||||
|
/// Spawn a bucket task: cooldown → take → dispatch → store → notify.
|
||||||
|
fn spawn_bucket(
|
||||||
|
method: &str,
|
||||||
|
rule_id: usize,
|
||||||
|
config: &Arc<NotificationRulesMap>,
|
||||||
|
plugins: &Arc<NotificationPluginMap>,
|
||||||
|
templates: &Arc<HashMap<String, NotificationTemplateDef>>,
|
||||||
|
ctx: &Arc<workshop_engine::ExecutionContext>,
|
||||||
|
period: u32,
|
||||||
|
watermark: u32,
|
||||||
|
) -> (BucketEntry, Arc<Notify>) {
|
||||||
|
let events: Arc<Mutex<Vec<NotificationEvent>>> = Arc::new(Mutex::new(Vec::new()));
|
||||||
|
let result: Arc<Mutex<Option<(MethodStatus, Vec<NotificationEvent>)>>> =
|
||||||
|
Arc::new(Mutex::new(None));
|
||||||
|
let notify = Arc::new(Notify::new());
|
||||||
|
let done = Arc::new(AtomicBool::new(false));
|
||||||
|
|
||||||
|
let ec = events.clone();
|
||||||
|
let rc = result.clone();
|
||||||
|
let nc = notify.clone();
|
||||||
|
let dc = done.clone();
|
||||||
|
let m = method.to_string();
|
||||||
|
let c = config.clone();
|
||||||
|
let p = plugins.clone();
|
||||||
|
let t = templates.clone();
|
||||||
|
let x = ctx.clone();
|
||||||
|
|
||||||
|
let handle = tokio::spawn(async move {
|
||||||
|
// Phase 1: Cooldown
|
||||||
|
let count = ec.lock().unwrap().len() as u32;
|
||||||
|
let wait_secs = wait_time(period, watermark, count);
|
||||||
|
tokio::time::sleep(Duration::from_secs_f32(wait_secs)).await;
|
||||||
|
|
||||||
|
// Phase 2: Take events
|
||||||
|
let batch = std::mem::take(&mut *ec.lock().unwrap());
|
||||||
|
if batch.is_empty() {
|
||||||
|
dc.store(true, Ordering::Release);
|
||||||
|
nc.notify_waiters();
|
||||||
|
return;
|
||||||
|
}
|
||||||
|
|
||||||
|
// Phase 3: Dispatch
|
||||||
|
let rule = match find_rule(&c, &m, rule_id) {
|
||||||
|
Ok(r) => r,
|
||||||
|
Err(_) => {
|
||||||
|
*rc.lock().unwrap() = Some((MethodStatus::PermanentError("rule not found".into()), batch));
|
||||||
|
dc.store(true, Ordering::Release);
|
||||||
|
nc.notify_waiters();
|
||||||
|
return;
|
||||||
|
}
|
||||||
|
};
|
||||||
|
|
||||||
|
let plugin = match p.get(&m) {
|
||||||
|
Some(p) => p,
|
||||||
|
None => {
|
||||||
|
*rc.lock().unwrap() = Some((MethodStatus::PermanentError("plugin not found".into()), batch));
|
||||||
|
dc.store(true, Ordering::Release);
|
||||||
|
nc.notify_waiters();
|
||||||
|
return;
|
||||||
|
}
|
||||||
|
};
|
||||||
|
|
||||||
|
let renderer = plugin.metadata.renderer.clone().unwrap_or_default();
|
||||||
|
let argument = build_plugin_argument(&renderer, rule, &m, &t, &x, &batch);
|
||||||
|
let status = match argument {
|
||||||
|
Some(arg) => {
|
||||||
|
match tokio::time::timeout(rule.timeout, plugin.call(arg, &x, &None)).await {
|
||||||
|
Ok(Ok(())) => MethodStatus::Success,
|
||||||
|
Ok(Err(e)) if is_temporary_error(&e) => {
|
||||||
|
MethodStatus::TemporaryError(format!("{}", e))
|
||||||
|
}
|
||||||
|
Ok(Err(e)) => MethodStatus::PermanentError(format!("{}", e)),
|
||||||
|
Err(_) => MethodStatus::TemporaryError("timeout".into()),
|
||||||
|
}
|
||||||
|
}
|
||||||
|
None => MethodStatus::PermanentError("argument build failed".into()),
|
||||||
|
};
|
||||||
|
|
||||||
|
*rc.lock().unwrap() = Some((status, batch));
|
||||||
|
dc.store(true, Ordering::Release);
|
||||||
|
nc.notify_waiters();
|
||||||
|
});
|
||||||
|
|
||||||
|
let entry = BucketEntry {
|
||||||
|
events,
|
||||||
|
result,
|
||||||
|
done,
|
||||||
|
notify: notify.clone(),
|
||||||
|
method: method.to_string(),
|
||||||
|
rule_id,
|
||||||
|
_handle: handle,
|
||||||
|
};
|
||||||
|
|
||||||
|
(entry, notify)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
/// Build the YAML argument for a plugin call based on renderer mode.
|
||||||
|
fn build_plugin_argument(
|
||||||
|
renderer: &NotificationRenderer,
|
||||||
|
rule: &crate::notify::types::NotificationRule,
|
||||||
|
method: &str,
|
||||||
|
templates: &HashMap<String, NotificationTemplateDef>,
|
||||||
|
ctx: &workshop_engine::ExecutionContext,
|
||||||
|
events: &[NotificationEvent],
|
||||||
|
) -> Option<serde_yaml::Value> {
|
||||||
|
let first = events.first()?;
|
||||||
|
|
||||||
|
match renderer {
|
||||||
|
NotificationRenderer::Server => {
|
||||||
|
let template_def = resolve_template(
|
||||||
|
&rule.template.schema,
|
||||||
|
method,
|
||||||
|
templates,
|
||||||
|
ctx.standalone,
|
||||||
|
None,
|
||||||
|
)
|
||||||
|
.ok()?;
|
||||||
|
|
||||||
|
let template_str = select_template_str(&template_def, &first.outcome)
|
||||||
|
.or_else(|| rule.template.on_success.as_deref())
|
||||||
|
.or_else(|| rule.template.on_failure.as_deref())
|
||||||
|
.or_else(|| {
|
||||||
|
rule.template
|
||||||
|
.on_finish_of
|
||||||
|
.as_ref()
|
||||||
|
.map(|o| o.content.as_str())
|
||||||
|
})?;
|
||||||
|
|
||||||
|
let data = build_batch_template_context(ctx, events);
|
||||||
|
let rendered = render_template(template_str, &data).ok()?;
|
||||||
|
|
||||||
|
let mut argument = rule.config.clone();
|
||||||
|
if let serde_yaml::Value::Mapping(ref mut map) = argument {
|
||||||
|
map.insert(
|
||||||
|
serde_yaml::Value::String("_rendered_content".into()),
|
||||||
|
serde_yaml::Value::String(rendered),
|
||||||
|
);
|
||||||
|
}
|
||||||
|
log::info!("SSR: rendered content for method '{}'", method);
|
||||||
|
Some(argument)
|
||||||
|
}
|
||||||
|
NotificationRenderer::Plugin => {
|
||||||
|
unimplemented!("Plugin renderer is not yet implemented")
|
||||||
|
}
|
||||||
|
NotificationRenderer::Interactive => {
|
||||||
|
unimplemented!("Interactive renderer is not yet implemented")
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
@@ -0,0 +1,86 @@
|
|||||||
|
use crate::notify::types::{NotificationEvent, NotificationEventReceiver, NotificationEventSender};
|
||||||
|
use std::collections::BinaryHeap;
|
||||||
|
|
||||||
|
/// Priority queue for notification events.
|
||||||
|
///
|
||||||
|
/// Integrates with `notify()`:
|
||||||
|
/// - `tx` sends events to notify() for processing
|
||||||
|
/// - `rx` receives retry events back from notify()
|
||||||
|
pub struct NotificationQueue {
|
||||||
|
pub pq: BinaryHeap<NotificationEvent>,
|
||||||
|
pub tx: NotificationEventSender,
|
||||||
|
pub rx: NotificationEventReceiver,
|
||||||
|
}
|
||||||
|
|
||||||
|
impl NotificationQueue {
|
||||||
|
pub fn new(tx: NotificationEventSender, rx: NotificationEventReceiver) -> Self {
|
||||||
|
Self {
|
||||||
|
pq: BinaryHeap::new(),
|
||||||
|
tx,
|
||||||
|
rx,
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
/// Push an event into the priority queue.
|
||||||
|
pub fn enqueue(&mut self, event: NotificationEvent) {
|
||||||
|
self.pq.push(event);
|
||||||
|
}
|
||||||
|
|
||||||
|
/// Non-blocking single round: recycle retry events into PQ, then
|
||||||
|
/// pop and send all ready events (not_before ≤ now).
|
||||||
|
pub async fn flush_now(&mut self) -> anyhow::Result<()> {
|
||||||
|
// Recycle retry events
|
||||||
|
while let Ok(event) = self.rx.try_recv() {
|
||||||
|
self.pq.push(event);
|
||||||
|
}
|
||||||
|
// Send ready events
|
||||||
|
let now = chrono::Utc::now();
|
||||||
|
while let Some(peek) = self.pq.peek() {
|
||||||
|
if peek.not_before > now {
|
||||||
|
break;
|
||||||
|
}
|
||||||
|
let event = self.pq.pop().unwrap();
|
||||||
|
self.tx.send(event).await?;
|
||||||
|
}
|
||||||
|
Ok(())
|
||||||
|
}
|
||||||
|
|
||||||
|
/// Blocking drain until all events are processed.
|
||||||
|
/// For standalone mode: called after prebake/bake/finalize to wait for completion.
|
||||||
|
/// For c/d mode: the main execution loop.
|
||||||
|
pub async fn drain(&mut self) -> anyhow::Result<()> {
|
||||||
|
loop {
|
||||||
|
// Drain retry channel
|
||||||
|
while let Ok(event) = self.rx.try_recv() {
|
||||||
|
self.pq.push(event);
|
||||||
|
}
|
||||||
|
// Determine sleep deadline
|
||||||
|
let until = self
|
||||||
|
.pq
|
||||||
|
.peek()
|
||||||
|
.map(|e| e.not_before)
|
||||||
|
.unwrap_or_else(|| chrono::Utc::now() + chrono::Duration::seconds(3600));
|
||||||
|
|
||||||
|
let std_until = (until - chrono::Utc::now())
|
||||||
|
.to_std()
|
||||||
|
.unwrap_or(std::time::Duration::from_secs(1));
|
||||||
|
|
||||||
|
tokio::select! {
|
||||||
|
event = self.rx.recv() => {
|
||||||
|
match event {
|
||||||
|
Some(event) => self.pq.push(event),
|
||||||
|
None => break Ok(()),
|
||||||
|
}
|
||||||
|
}
|
||||||
|
_ = tokio::time::sleep(std_until) => {
|
||||||
|
let now = chrono::Utc::now();
|
||||||
|
while let Some(peek) = self.pq.peek() {
|
||||||
|
if peek.not_before > now { break; }
|
||||||
|
let event = self.pq.pop().unwrap();
|
||||||
|
self.tx.send(event).await?;
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
@@ -0,0 +1,148 @@
|
|||||||
|
use crate::finalize::NotificationConfig;
|
||||||
|
use crate::finalize::NotificationTemplate;
|
||||||
|
use crate::finalize::config::{NotifyPolicy, TriggerItemRaw};
|
||||||
|
use crate::notify::types::{DEFAULT_NOTIFY_PARAMS, Matchable, Method, NotificationRule, NotificationRulesMap, ParsedTrigger};
|
||||||
|
use globset::{Glob, GlobSetBuilder};
|
||||||
|
use std::collections::HashMap;
|
||||||
|
|
||||||
|
/// Parse a single `on_finish_of` pattern string into a ParsedTrigger, or None.
|
||||||
|
/// Supports `re:` prefix for regex patterns, otherwise treated as glob.
|
||||||
|
fn parse_on_finish_of(pattern: &str) -> Option<ParsedTrigger> {
|
||||||
|
let pattern = pattern.trim();
|
||||||
|
if let Some(re_str) = pattern.strip_prefix("re:") {
|
||||||
|
regex::Regex::new(re_str).ok().map(|r| ParsedTrigger::OnFinishOf(Matchable::Regex(r)))
|
||||||
|
} else {
|
||||||
|
Glob::new(pattern).ok().and_then(|g| {
|
||||||
|
let mut builder = GlobSetBuilder::new();
|
||||||
|
builder.add(g);
|
||||||
|
builder.build().ok()
|
||||||
|
}).map(|gs| ParsedTrigger::OnFinishOf(Matchable::Glob(gs)))
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
fn parse_trigger_item_raw(item: &TriggerItemRaw) -> Vec<ParsedTrigger> {
|
||||||
|
// Normalize Simple → single-entry Vec, then unify both branches
|
||||||
|
let entries: Vec<(&str, Vec<&str>)> = match item {
|
||||||
|
TriggerItemRaw::Simple(s) => {
|
||||||
|
if let Some(pattern) = s.strip_prefix("on_finish_of:") {
|
||||||
|
vec![("on_finish_of", vec![pattern.trim()])]
|
||||||
|
} else {
|
||||||
|
vec![(s.as_str(), vec![])]
|
||||||
|
}
|
||||||
|
}
|
||||||
|
TriggerItemRaw::Map(map) => map.iter().map(|(k, v)|
|
||||||
|
(k.as_str(), v.iter().map(|s| s.as_str()).collect())
|
||||||
|
).collect(),
|
||||||
|
};
|
||||||
|
|
||||||
|
let mut triggers = Vec::new();
|
||||||
|
for (key, patterns) in entries {
|
||||||
|
match key {
|
||||||
|
"on_finish_of" => {
|
||||||
|
for pattern in patterns {
|
||||||
|
if let Some(t) = parse_on_finish_of(pattern) {
|
||||||
|
triggers.push(t);
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
"on_success" => triggers.push(ParsedTrigger::OnSuccess),
|
||||||
|
"on_failure" => triggers.push(ParsedTrigger::OnFailure),
|
||||||
|
_ => log::warn!("Unknown trigger key: {}", key),
|
||||||
|
}
|
||||||
|
}
|
||||||
|
triggers
|
||||||
|
}
|
||||||
|
|
||||||
|
fn merge_templates(
|
||||||
|
method_template: &NotificationTemplate,
|
||||||
|
policy_template: &Option<serde_yaml::Value>,
|
||||||
|
) -> NotificationTemplate {
|
||||||
|
let mut merged = method_template.clone();
|
||||||
|
if let Some(policy_tmpl) = policy_template
|
||||||
|
&& let Ok(policy) = serde_yaml::from_value::<NotificationTemplate>(policy_tmpl.clone())
|
||||||
|
{
|
||||||
|
merged += policy;
|
||||||
|
}
|
||||||
|
merged
|
||||||
|
}
|
||||||
|
|
||||||
|
struct RuleOverrides {
|
||||||
|
fallible: bool,
|
||||||
|
max_lives: u32,
|
||||||
|
timeout: std::time::Duration,
|
||||||
|
retry_backoff: f64,
|
||||||
|
}
|
||||||
|
|
||||||
|
impl RuleOverrides {
|
||||||
|
fn from_value(v: &serde_yaml::Value) -> Self {
|
||||||
|
RuleOverrides {
|
||||||
|
fallible: v.get("fallible").and_then(|v| v.as_bool()).unwrap_or(false),
|
||||||
|
max_lives: v.get("max_lives").and_then(|v| v.as_u64()).unwrap_or(DEFAULT_NOTIFY_PARAMS.max_lives.value as u64) as u32,
|
||||||
|
timeout: std::time::Duration::from_millis(
|
||||||
|
v.get("timeout_ms").and_then(|v| v.as_u64()).unwrap_or(DEFAULT_NOTIFY_PARAMS.timeout_ms.value)
|
||||||
|
),
|
||||||
|
retry_backoff: v.get("retry_backoff").and_then(|v| v.as_f64()).unwrap_or(DEFAULT_NOTIFY_PARAMS.retry_backoff.value),
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
/// Chain: apply `source` atop existing values (source wins when present).
|
||||||
|
fn apply(self, source: &serde_yaml::Value) -> Self {
|
||||||
|
RuleOverrides {
|
||||||
|
fallible: source.get("fallible").and_then(|v| v.as_bool()).unwrap_or(self.fallible),
|
||||||
|
max_lives: source.get("max_lives").and_then(|v| v.as_u64()).unwrap_or(self.max_lives as u64) as u32,
|
||||||
|
timeout: source.get("timeout_ms").and_then(|v| v.as_u64()).map(std::time::Duration::from_millis).unwrap_or(self.timeout),
|
||||||
|
retry_backoff: source.get("retry_backoff").and_then(|v| v.as_f64()).unwrap_or(self.retry_backoff),
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
pub fn parse_rules(config: &NotificationConfig) -> NotificationRulesMap {
|
||||||
|
log::info!("[NOTIFY] parse_rules: groups={}", config.groups.len());
|
||||||
|
let mut rules = NotificationRulesMap::default();
|
||||||
|
for (priority, group) in &config.groups {
|
||||||
|
let mut ruleset: HashMap<Method, Vec<NotificationRule>> = HashMap::new();
|
||||||
|
for (method_name, method) in group {
|
||||||
|
let mut method_rules = Vec::new();
|
||||||
|
|
||||||
|
// Unify: no policy → one empty policy (same merge path)
|
||||||
|
let empty_policy = NotifyPolicy {
|
||||||
|
trigger: vec![],
|
||||||
|
template: serde_yaml::Value::Null,
|
||||||
|
overrides: serde_yaml::Value::Null,
|
||||||
|
};
|
||||||
|
let policies: &[NotifyPolicy] = if method.policy.is_empty() {
|
||||||
|
std::slice::from_ref(&empty_policy)
|
||||||
|
} else {
|
||||||
|
&method.policy
|
||||||
|
};
|
||||||
|
|
||||||
|
for policy in policies {
|
||||||
|
let trigger: Vec<ParsedTrigger> = if policy.trigger.is_empty() {
|
||||||
|
method.trigger.iter().flat_map(parse_trigger_item_raw).collect()
|
||||||
|
} else {
|
||||||
|
policy.trigger.iter().flat_map(parse_trigger_item_raw).collect()
|
||||||
|
};
|
||||||
|
let template =
|
||||||
|
merge_templates(&method.template, &Some(policy.template.clone()));
|
||||||
|
let vals = RuleOverrides::from_value(&serde_yaml::Value::Null)
|
||||||
|
.apply(&method.config)
|
||||||
|
.apply(&policy.overrides);
|
||||||
|
method_rules.push(NotificationRule {
|
||||||
|
enabled: true,
|
||||||
|
trigger,
|
||||||
|
template,
|
||||||
|
fallible: vals.fallible,
|
||||||
|
max_lives: vals.max_lives,
|
||||||
|
timeout: vals.timeout,
|
||||||
|
config: method.config.clone(),
|
||||||
|
retry_backoff: vals.retry_backoff,
|
||||||
|
});
|
||||||
|
}
|
||||||
|
|
||||||
|
ruleset.insert(method_name.clone(), method_rules);
|
||||||
|
}
|
||||||
|
rules.insert(*priority, ruleset);
|
||||||
|
}
|
||||||
|
rules
|
||||||
|
}
|
||||||
|
|
||||||
@@ -0,0 +1,281 @@
|
|||||||
|
use std::collections::HashMap;
|
||||||
|
|
||||||
|
use minijinja::Environment;
|
||||||
|
use workshop_engine::ExecutionContext;
|
||||||
|
|
||||||
|
use crate::finalize::NotificationTemplateDef;
|
||||||
|
use crate::notify::error::NotifyError;
|
||||||
|
use crate::notify::types::NotificationEvent;
|
||||||
|
use crate::types::buildstatus::StageOutcome;
|
||||||
|
use crate::types::resource::ResourceRegistry;
|
||||||
|
|
||||||
|
/// Maximum depth for template cross-references (prevent infinite recursion)
|
||||||
|
pub const TEMPLATE_REFERENCE_DEPTH: u32 = 10;
|
||||||
|
|
||||||
|
/// Build template context for a batch of notification events.
|
||||||
|
pub fn build_batch_template_context(
|
||||||
|
ctx: &ExecutionContext,
|
||||||
|
events: &[NotificationEvent],
|
||||||
|
) -> HashMap<String, serde_json::Value> {
|
||||||
|
let mut data = HashMap::new();
|
||||||
|
|
||||||
|
data.insert("pipeline".into(), serde_json::json!({
|
||||||
|
"name": ctx.pipeline_name,
|
||||||
|
}));
|
||||||
|
|
||||||
|
data.insert("count".into(), serde_json::json!(events.len()));
|
||||||
|
|
||||||
|
let event_list: Vec<_> = events
|
||||||
|
.iter()
|
||||||
|
.map(|e| {
|
||||||
|
serde_json::json!({
|
||||||
|
"stage": format!("{}.{}", e.stage.as_str(), e.substage),
|
||||||
|
"status": e.outcome.to_string(),
|
||||||
|
})
|
||||||
|
})
|
||||||
|
.collect();
|
||||||
|
data.insert("events".into(), serde_json::json!(event_list));
|
||||||
|
|
||||||
|
// Single event compatibility: provide direct access
|
||||||
|
if events.len() == 1 {
|
||||||
|
let e = &events[0];
|
||||||
|
data.insert("build".into(), serde_json::json!({
|
||||||
|
"id": ctx.task_id,
|
||||||
|
"status": e.outcome.to_string(),
|
||||||
|
"stage": format!("{}.{}", e.stage.as_str(), e.substage),
|
||||||
|
"duration": "0s",
|
||||||
|
"url": "",
|
||||||
|
}));
|
||||||
|
} else {
|
||||||
|
data.insert("build".into(), serde_json::json!({
|
||||||
|
"id": ctx.task_id,
|
||||||
|
"status": "batched",
|
||||||
|
"stage": "batch",
|
||||||
|
"duration": "0s",
|
||||||
|
"url": "",
|
||||||
|
}));
|
||||||
|
}
|
||||||
|
|
||||||
|
data.insert("commit".into(), serde_json::json!({
|
||||||
|
"hash": "",
|
||||||
|
"short_hash": "",
|
||||||
|
"message": "",
|
||||||
|
"author": "",
|
||||||
|
"author_email": "",
|
||||||
|
}));
|
||||||
|
|
||||||
|
data
|
||||||
|
}
|
||||||
|
|
||||||
|
/// Build template context for a single notification event (convenience wrapper).
|
||||||
|
pub fn build_template_context(
|
||||||
|
ctx: &ExecutionContext,
|
||||||
|
event: &NotificationEvent,
|
||||||
|
) -> HashMap<String, serde_json::Value> {
|
||||||
|
build_batch_template_context(ctx, std::slice::from_ref(event))
|
||||||
|
}
|
||||||
|
|
||||||
|
/// Render a minijinja template string with the provided data.
|
||||||
|
pub fn render_template(
|
||||||
|
template: &str,
|
||||||
|
data: &HashMap<String, serde_json::Value>,
|
||||||
|
) -> Result<String, String> {
|
||||||
|
let env = Environment::new();
|
||||||
|
let tmpl = env
|
||||||
|
.template_from_str(template)
|
||||||
|
.map_err(|e| format!("Template parse error: {}", e))?;
|
||||||
|
|
||||||
|
let result = tmpl
|
||||||
|
.render(data)
|
||||||
|
.map_err(|e| format!("Template render error: {}", e))?;
|
||||||
|
|
||||||
|
Ok(result)
|
||||||
|
}
|
||||||
|
|
||||||
|
/// Select template string from NotificationTemplateDef based on outcome.
|
||||||
|
pub fn select_template_str<'a>(
|
||||||
|
tmpl: &'a NotificationTemplateDef,
|
||||||
|
outcome: &StageOutcome,
|
||||||
|
) -> Option<&'a str> {
|
||||||
|
match outcome {
|
||||||
|
StageOutcome::PipelineSuccess => tmpl.on_success.as_deref(),
|
||||||
|
StageOutcome::PipelineFailure => tmpl.on_failure.as_deref(),
|
||||||
|
_ => None,
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
/// Resolve external template references from the resource registry.
|
||||||
|
pub fn resolve_external_template_refs(
|
||||||
|
templates: &mut HashMap<String, NotificationTemplateDef>,
|
||||||
|
registry: &ResourceRegistry,
|
||||||
|
) {
|
||||||
|
for (name, tmpl_def) in templates.iter_mut() {
|
||||||
|
if tmpl_def.use_.is_none() {
|
||||||
|
continue;
|
||||||
|
}
|
||||||
|
let Some(path) = registry.resolve(name) else {
|
||||||
|
continue;
|
||||||
|
};
|
||||||
|
let Ok(content) = std::fs::read_to_string(path.join("template.yml"))
|
||||||
|
.or_else(|_| std::fs::read_to_string(path.join("template.yaml")))
|
||||||
|
else {
|
||||||
|
continue;
|
||||||
|
};
|
||||||
|
let Ok(fetched) =
|
||||||
|
serde_yaml::from_str::<NotificationTemplateDef>(&content)
|
||||||
|
else {
|
||||||
|
continue;
|
||||||
|
};
|
||||||
|
*tmpl_def = fetched.apply(std::mem::take(tmpl_def));
|
||||||
|
tmpl_def.use_ = None;
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
/// Resolve a schema reference into a NotificationTemplateDef by walking a fallback chain.
|
||||||
|
pub fn resolve_template(
|
||||||
|
schema: &Option<String>,
|
||||||
|
method_name: &str,
|
||||||
|
templates: &HashMap<String, NotificationTemplateDef>,
|
||||||
|
_standalone: bool,
|
||||||
|
server_templates: Option<&HashMap<String, NotificationTemplateDef>>,
|
||||||
|
) -> Result<NotificationTemplateDef, NotifyError> {
|
||||||
|
// 1. Try the initial schema directly
|
||||||
|
if let Some(schema_name) = schema.as_deref() {
|
||||||
|
if let Some(template) =
|
||||||
|
resolve_template_by_schema(schema_name, method_name, templates, server_templates)
|
||||||
|
{
|
||||||
|
return Ok(template);
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
// 2. Walk the fallback chain
|
||||||
|
let mut schema_next = resolve_template_next_schema(schema);
|
||||||
|
while let Some(ref schema_name) = schema_next {
|
||||||
|
if let Some(template) =
|
||||||
|
resolve_template_by_schema(schema_name, method_name, templates, server_templates)
|
||||||
|
{
|
||||||
|
return Ok(template);
|
||||||
|
}
|
||||||
|
log::warn!("Failed to resolve template for schema {}.", schema_name);
|
||||||
|
schema_next = resolve_template_next_schema(&schema_next);
|
||||||
|
}
|
||||||
|
Err(NotifyError::SchemaError(format!("All template resolution attempts beginning from {} failed.", schema.as_deref().unwrap_or("[system default]"))))
|
||||||
|
}
|
||||||
|
|
||||||
|
/// Return the next schema name in the fallback chain, or None to terminate.
|
||||||
|
pub fn resolve_template_next_schema(schema: &Option<String>) -> Option<String> {
|
||||||
|
match schema.as_deref() {
|
||||||
|
Some("custom") => None,
|
||||||
|
Some("default") => Some("fallback".to_string()),
|
||||||
|
Some("fallback") => None,
|
||||||
|
Some(_) => Some("default".to_string()),
|
||||||
|
None => Some("default".to_string()),
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
/// Perform a single lookup for a given schema name. Returns None if not found.
|
||||||
|
pub fn resolve_template_by_schema(
|
||||||
|
schema: &str,
|
||||||
|
method_name: &str,
|
||||||
|
templates: &HashMap<String, NotificationTemplateDef>,
|
||||||
|
server_templates: Option<&HashMap<String, NotificationTemplateDef>>,
|
||||||
|
) -> Option<NotificationTemplateDef> {
|
||||||
|
match schema {
|
||||||
|
"custom" => Some(NotificationTemplateDef::default()),
|
||||||
|
"fallback" => templates.get(&format!("{}.fallback", method_name)).cloned(),
|
||||||
|
"default" => server_templates.and_then(|b| b.get(method_name)).cloned(),
|
||||||
|
name => templates.get(name).cloned(),
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
/// Flatten all local template cross-references (use: "other-name").
|
||||||
|
pub fn flatten_template_refs(
|
||||||
|
templates: &mut HashMap<String, NotificationTemplateDef>,
|
||||||
|
) -> Result<(), NotifyError> {
|
||||||
|
let names: Vec<String> = templates.keys().cloned().collect();
|
||||||
|
for name in names {
|
||||||
|
let mut visited = std::collections::HashSet::new();
|
||||||
|
resolve_chain(&name, templates, 0, &mut visited)?;
|
||||||
|
}
|
||||||
|
Ok(())
|
||||||
|
}
|
||||||
|
|
||||||
|
fn resolve_chain(
|
||||||
|
name: &str,
|
||||||
|
templates: &mut HashMap<String, NotificationTemplateDef>,
|
||||||
|
depth: u32,
|
||||||
|
visited: &mut std::collections::HashSet<String>,
|
||||||
|
) -> Result<(), NotifyError> {
|
||||||
|
if depth > TEMPLATE_REFERENCE_DEPTH {
|
||||||
|
return Err(NotifyError::TemplateDepthExceeded {
|
||||||
|
name: name.to_string(),
|
||||||
|
max: TEMPLATE_REFERENCE_DEPTH,
|
||||||
|
});
|
||||||
|
}
|
||||||
|
if !visited.insert(name.to_string()) {
|
||||||
|
return Err(NotifyError::TemplateLoop(name.to_string()));
|
||||||
|
}
|
||||||
|
|
||||||
|
// Only process local references (no ://)
|
||||||
|
if let Some(ref_name) = templates.get(name).and_then(|t| t.use_.clone()) {
|
||||||
|
if !ref_name.contains("://") {
|
||||||
|
if !templates.contains_key(&ref_name) {
|
||||||
|
return Err(NotifyError::SchemaError(format!(
|
||||||
|
"template '{}' references non-existent template '{}'",
|
||||||
|
name, ref_name
|
||||||
|
)));
|
||||||
|
}
|
||||||
|
resolve_chain(&ref_name, templates, depth + 1, visited)?;
|
||||||
|
|
||||||
|
// Merge: base provides defaults, referencing template overrides
|
||||||
|
let base = templates[&ref_name].clone();
|
||||||
|
let template = templates.get_mut(name).unwrap();
|
||||||
|
*template = std::mem::take(template).apply(base);
|
||||||
|
template.use_ = None;
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
visited.remove(name);
|
||||||
|
Ok(())
|
||||||
|
}
|
||||||
|
|
||||||
|
#[cfg(test)]
|
||||||
|
mod tests {
|
||||||
|
use super::*;
|
||||||
|
|
||||||
|
#[test]
|
||||||
|
fn test_render_template_simple() {
|
||||||
|
let template = "Hello {{ name }}!";
|
||||||
|
let mut data = HashMap::new();
|
||||||
|
data.insert(
|
||||||
|
"name".to_string(),
|
||||||
|
serde_json::Value::String("World".to_string()),
|
||||||
|
);
|
||||||
|
let result = render_template(template, &data).unwrap();
|
||||||
|
assert_eq!(result, "Hello World!");
|
||||||
|
}
|
||||||
|
|
||||||
|
#[test]
|
||||||
|
fn test_render_template_with_build_status() {
|
||||||
|
let template = "Build {{ build.status }} for pipeline {{ pipeline.name }}";
|
||||||
|
let mut data = HashMap::new();
|
||||||
|
data.insert(
|
||||||
|
"build".to_string(),
|
||||||
|
serde_json::json!({ "status": "success" }),
|
||||||
|
);
|
||||||
|
data.insert(
|
||||||
|
"pipeline".to_string(),
|
||||||
|
serde_json::json!({ "name": "test-pipeline" }),
|
||||||
|
);
|
||||||
|
let result = render_template(template, &data).unwrap();
|
||||||
|
assert_eq!(result, "Build success for pipeline test-pipeline");
|
||||||
|
}
|
||||||
|
|
||||||
|
#[test]
|
||||||
|
fn test_render_template_invalid_syntax() {
|
||||||
|
let template = "{{ Unterminated";
|
||||||
|
let data = HashMap::new();
|
||||||
|
let result = render_template(template, &data);
|
||||||
|
assert!(result.is_err());
|
||||||
|
}
|
||||||
|
}
|
||||||
@@ -7,7 +7,6 @@ use regex::Regex;
|
|||||||
use serde::Deserialize;
|
use serde::Deserialize;
|
||||||
use serde::Serialize;
|
use serde::Serialize;
|
||||||
use std::collections::HashMap;
|
use std::collections::HashMap;
|
||||||
use std::collections::HashSet;
|
|
||||||
use std::fmt::Display;
|
use std::fmt::Display;
|
||||||
use std::hash::Hash;
|
use std::hash::Hash;
|
||||||
use std::time::Duration;
|
use std::time::Duration;
|
||||||
@@ -78,12 +77,39 @@ impl ParsedTrigger {
|
|||||||
|
|
||||||
impl NotificationRule {
|
impl NotificationRule {
|
||||||
pub fn accept(&self, result: &StageOutcome, stage: &StagePhase, substage: &str) -> bool {
|
pub fn accept(&self, result: &StageOutcome, stage: &StagePhase, substage: &str) -> bool {
|
||||||
|
if self.trigger.is_empty() {
|
||||||
|
eprintln!("[ACCEPT] trigger empty, returning true");
|
||||||
|
return true;
|
||||||
|
}
|
||||||
self.trigger
|
self.trigger
|
||||||
.iter()
|
.iter()
|
||||||
.any(|t| t.accept(result, stage, substage))
|
.any(|t| t.accept(result, stage, substage))
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
|
/// Status of a single notification method after plugin execution.
|
||||||
|
#[derive(Debug)]
|
||||||
|
pub enum MethodStatus {
|
||||||
|
Success,
|
||||||
|
TemporaryError(String),
|
||||||
|
PermanentError(String),
|
||||||
|
Exhausted(String),
|
||||||
|
}
|
||||||
|
|
||||||
|
/// Tracks which notification methods still need attention within a priority group.
|
||||||
|
#[derive(Debug, Clone, Serialize)]
|
||||||
|
pub struct NotificationTarget {
|
||||||
|
pub priority: u32,
|
||||||
|
pub rules: Option<Vec<String>>,
|
||||||
|
pub count: u32,
|
||||||
|
}
|
||||||
|
|
||||||
|
impl Default for NotificationTarget {
|
||||||
|
fn default() -> Self {
|
||||||
|
Self { priority: 0, rules: None, count: 0 }
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
pub type Priority = u32;
|
pub type Priority = u32;
|
||||||
pub type Method = String;
|
pub type Method = String;
|
||||||
pub type NotificationRulesMap = HashMap<Priority, HashMap<Method, Vec<NotificationRule>>>;
|
pub type NotificationRulesMap = HashMap<Priority, HashMap<Method, Vec<NotificationRule>>>;
|
||||||
@@ -101,9 +127,9 @@ pub struct NotificationEvent {
|
|||||||
pub not_before: DateTime<Utc>,
|
pub not_before: DateTime<Utc>,
|
||||||
pub priority: u32,
|
pub priority: u32,
|
||||||
pub effective_priority: u32,
|
pub effective_priority: u32,
|
||||||
pub lives: u32,
|
pub retry_backoff: f64, // Exponential backoff factor
|
||||||
pub max_lives: u32,
|
pub attempted: u32, // Cumulative cross-group retry count
|
||||||
pub target: HashSet<String>, // Target audience
|
pub target: NotificationTarget, // Current audience
|
||||||
}
|
}
|
||||||
|
|
||||||
impl Hash for NotificationEvent {
|
impl Hash for NotificationEvent {
|
||||||
@@ -137,25 +163,11 @@ impl Ord for NotificationEvent {
|
|||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
/// Delay before retrying an unavailable plugin (5 seconds)
|
use std::sync::LazyLock;
|
||||||
pub const NOTIFICATION_NOTREADY_DELAY: std::time::Duration = std::time::Duration::from_secs(5);
|
use workshop_baker_params::{NotificationParams, MergeParams};
|
||||||
|
|
||||||
/// Exponential backoff base (2.0 = double each retry)
|
/// Default parameters used when no external config is provided.
|
||||||
pub const NOTIFICATION_TEMPORARY_DELAY_FACTOR: f64 = 2.0;
|
pub static DEFAULT_NOTIFY_PARAMS: LazyLock<NotificationParams> = LazyLock::new(|| NotificationParams::defaults());
|
||||||
|
|
||||||
/// Initial backoff delay for temporary errors (5 seconds)
|
|
||||||
pub const NOTIFICATION_TEMPORARY_DELAY_TIME: f64 = 5.0;
|
|
||||||
|
|
||||||
/// Upper bound for backoff delay (unbounded)
|
|
||||||
pub const NOTIFICATION_TEMPORARY_MAX_TIME: f64 = f64::MAX;
|
|
||||||
|
|
||||||
/// Lives recovered after downgrade as fraction of max_lives (0.0 = none)
|
|
||||||
pub const NOTIFICATION_LIFE_RECOVERY_FACTOR: f64 = 0.0;
|
|
||||||
|
|
||||||
pub const NOTIFICATION_DEFAULT_FALLIBLE: bool = false;
|
|
||||||
pub const NOTIFICATION_DEFAULT_MAX_LIVES: u32 = 3;
|
|
||||||
pub const NOTIFICATION_DEFAULT_TIMEOUT_MS: u64 = 30000;
|
|
||||||
pub const NOTIFICATION_DEFAULT_RETRY_BACKOFF: f64 = 2.0;
|
|
||||||
|
|
||||||
#[derive(Default, Debug, Deserialize, Clone)]
|
#[derive(Default, Debug, Deserialize, Clone)]
|
||||||
pub enum NotificationRenderer {
|
pub enum NotificationRenderer {
|
||||||
@@ -174,3 +186,25 @@ impl Display for NotificationRenderer {
|
|||||||
}
|
}
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
|
/// Notification batching configuration.
|
||||||
|
/// Controls how notification events are grouped before dispatch.
|
||||||
|
#[derive(Debug, Clone)]
|
||||||
|
pub struct BatchConfig {
|
||||||
|
/// Whether batching is enabled
|
||||||
|
pub enabled: bool,
|
||||||
|
/// Group period in seconds (the maximum wait time for a batch)
|
||||||
|
pub period: u32,
|
||||||
|
/// Group watermark threshold (the minimum number of events before batching)
|
||||||
|
pub watermark: u32,
|
||||||
|
}
|
||||||
|
|
||||||
|
impl Default for BatchConfig {
|
||||||
|
fn default() -> Self {
|
||||||
|
Self {
|
||||||
|
enabled: true,
|
||||||
|
period: 30,
|
||||||
|
watermark: 5,
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|||||||
Some files were not shown because too many files have changed in this diff Show More
Reference in New Issue
Block a user