pig2014/honey-biscuit-workshop
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

refactor: update AGENTS.md and remove osbolete modules

Browse Source 
Archive 7 deprecated crates. Add Resource model documentation.
Remove internal fetch module from workshop-baker.
This commit is contained in:
Catty Steve
2026-05-20 16:30:50 +08:00
parent 28abc5d207
commit cf2968e720
12 changed files with 264 additions and 1779 deletions
Download Patch File Download Diff File Expand all files Collapse all files
AGENTS.md
+41 -57
View File
@@ -1,83 +1,72 @@
# PROJECT KNOWLEDGE BASE
**Generated:** 2026-04-22
**Commit:** 7b3b71a
**Generated:** 2026-05-19
**Commit:** 28abc5d
**Branch:** master
## OVERVIEW
HoneyBiscuitWorkshop is a Rust-based CI/CD system ("蜜饼工坊") with LLM-powered pipeline auto-configuration. 9 independent Cargo crates, no workspace.
HoneyBiscuitWorkshop ("蜜饼工坊") — Rust CI/CD system with decorator-driven shell pipeline DSL and LLM-powered configuration. 3 independent Cargo crates, no workspace.
## STRUCTURE
```
./
├── workshop-baker/ # Main orchestrator (CLI + daemon)
├── workshop-agent/ # HTTP agent server
├── workshop-pipeline/ # Config library (prebake/finalize)
├── workshop-llm-detector/ # LLM repo analyzer
├── workshop-vault/ # Secret management client
├── workshop-cert/ # TLS certificate generator
├── workshop-deviceid/ # Device ID library
├── workshop-builder-native/ # Stub (unused)
├── workshop-helper-mac/ # MAC address utility
├── docs/ # mdBook documentation (zh-CN)
├── playground/ # Experimentation
└── third_party/ # Empty
├── workshop-baker/ # Main orchestrator (CLI + pipeline stages)
├── workshop-engine/ # Execution engine (extracted from baker)
├── workshop-getterurl/ # URL fetching (go-getter style)
├── docs/ # mdBook documentation (zh-CN)
├── templates/ # Pipeline templates (bake.sh, finalize.yml, prebake.yml)
└── archived/ # Deprecated crates (7 moved here)
```
## WHERE TO LOOK
| Task | Location | Notes |
|------|----------|-------|
| Baker (main) | `workshop-baker/src/{bake,prebake,engine}/` | Core pipeline logic |
| Agent server | `workshop-agent/src/` | Actix-web HTTP + TLS |
| Pipeline config | `workshop-pipeline/src/config/` | prebake/finalize YAML |
| LLM integration | `workshop-llm-detector/src/llm/` | Ollama/OpenAI clients |
| Secrets | `workshop-vault/src/` | Vault client wrapper |
| Baker (CLI) | `workshop-baker/src/{lib,main}.rs` | Clap CLI + tokio entry |
| Pipeline stages | `workshop-baker/src/{bake,prebake,finalize}/` | Stage orchestration |
| Notifications | `workshop-baker/src/notify/` | Plugin-based event notifications |
|| Engine (execution) | `workshop-engine/src/` | Process mgmt, PM detection |
| URL fetching | `workshop-getterurl/src/` | Git/HTTP/File resource getter |
## CODE MAP
| Symbol | Type | Location | Role |
|--------|------|----------|------|
| PipelineConfig | struct | workshop-pipeline/src/lib.rs:27 | Full pipeline config |
| CICDLLMHelper | struct | workshop-llm-detector/src/lib.rs:16 | LLM analysis entry |
| Engine | struct | workshop-baker/src/engine.rs | Build executor |
| VaultClient | struct | workshop-vault/src/client.rs | Secret ops |
| Cli | struct | workshop-baker/src/lib.rs:21 | CLI argument struct |
| ExecutionContext | struct | workshop-engine/src/types.rs:10 | Pipeline context |
| Engine | struct | workshop-engine/src/types.rs:99 | Build executor (placeholder) |
| PrebakeConfig | struct | workshop-baker/src/prebake/config.rs | Prebake YAML schema |
| FinalizeConfig | struct | workshop-baker/src/finalize/config.rs:21 | Finalize YAML schema |
| GetterUrl | struct | workshop-getterurl/src/lib.rs:84 | Parsed resource URL |
| Getter | trait | workshop-getterurl/src/getter.rs:18 | Resource fetch trait |
## CONVENTIONS
### Rust
- **Edition**: 2024 (non-standard, requires Rust 1.85+)
- Deprecate `mod.rs`
- **Edition**: 2024 (requires Rust 1.85+), deprecates `mod.rs`
- **Naming**: `snake_case` files/modules, `PascalCase` types, `SCREAMING_SNAKE_CASE` consts
- **Error handling**: `thiserror` + `anyhow` combo
- **Error handling**: `thiserror` + `anyhow` combo; `HasExitCode` trait for exit codes
- **Async**: `#[tokio::main]` + `tokio` with "full" features
- **Imports**: `std` → `external_crate` → `crate::module`
- **Dual-mode**: CLI commands (prebake/bake/finalize) + daemon mode (unimplemented)
### Testing
- **Rust**: `#[test]` inline, `#[tokio::test]` async, `tests/*.rs` integration
- **Python**: `pytest` with `tests/integration/test_*.py` (Docker-based)
- **Benchmarks**: Criterion with `harness = false`
- **Rust**: `#[test]` inline, `#[tokio::test]` async, `tests/` integration
- **Benchmarks**: Criterion (configured in Cargo.toml)
### Build Pipeline
- **prebake.yml**: Environment setup (docker/firecracker/baremetal)
- **bake.sh**: Build execution with `# @pipeline` decorators
- **finalize.yml**: Packaging, deployment, notifications
- **Annotations**: `@timeout()`, `@retry()`, `@parallel`, `@fallible`
### CI
- **GitHub Actions**: rust-toolchain (stable), clippy + rustfmt, `cargo test --all-features`
- **Matrix**: workshop-baker, workshop-engine
## ANTI-PATTERNS (THIS PROJECT)
1. **Python in Rust project** — workshop-baker has Python tests (pytest.ini, test_prebake.py)
2. **temp/ directory** — Non-standard in Rust projects
3. **Certificates in source** — artifact workshop-cert/certs/ not excluded
4. **Deprecated projects** — workshop-executor.old, workshop-monitor.old still present
5. **Hardcoded mirrors** — Tsinghua University mirrors in configs
6. **Rust 2024 edition** — May not work with stable toolchains
1. **Hardcoded user** — "vulcan" username hardcoded throughout prebake
2. **Unsafe libc** — workshop-baker/src/prebake/security.rs has 5+ unsafe blocks
3. **Rust 2024 edition** — May not work with stable toolchains
4. **Empty daemon** — `daemon/` dir is empty, Daemon command is `unimplemented!()`
## UNIQUE STYLES
- **Pipeline DSL**: Shell scripts with `# @decorator` annotations
- **Dual-mode baker**: CLI commands + daemon mode
- **LLM cookbook system**: YAML cookbooks for language-specific builds
- **Notification system**: Plugin-based with batching, retry, priority groups
- **LLM cookbook system**: YAML cookbooks for language-specific builds (archived)
- **Chinese docs** — Documentation primarily in zh-CN
- **User "vulcan"** — Custom build user (not root/runner)
## COMMANDS
```bash
@@ -86,23 +75,18 @@ cargo build --release -p workshop-baker
# Test
cargo test -p workshop-baker
pytest tests/integration/test_prebake.py -v
cargo test -p workshop-engine
# Lint
cargo clippy -- -D warnings
cargo fmt --all
# Run baker
cargo run --bin workshop-baker -- prebake config.yml
cargo run --bin workshop-baker -- bake script.sh
# Run agent
cargo run --bin workshop-agent
# Run
cargo run --bin workshop-baker -- --pipeline demo --build-id 1 --prebake prebake.yml --bake bake.sh --finalize finalize.yml
```
## NOTES
- **Docker required** for integration tests
- **workshop-builder-native** is a stub (3-line Hello World)
- No root workspace Cargo.toml — each crate is independent
- `AGENTS.md.old` at root — legacy documentation
- **No root workspace** — each crate independent, build separately
- **7 archived crates** — workshop-agent, workshop-cert, workshop-deviceid, workshop-helper-mac, workshop-llm-detector, workshop-pipeline, workshop-vault
- **Python tests removed** — pytest.ini exists but test files no longer in workspace
docs/src/SUMMARY.md
+1
View File
@@ -35,3 +35,4 @@
- [自定义 Decorator](zh-CN/vol3_dev/custom_decorator.md)
- [模板变量](zh-CN/vol3_dev/template_variables.md)
- [邮件模板](zh-CN/vol3_dev/mail_template.md)
- [Resource 模型](zh-CN/vol3_dev/resource_model.md)
docs/src/zh-CN/vol3_dev/resource_model.md
+88
View File
@@ -0,0 +1,88 @@
# Resource 模型
## 核心思想
整个系统只关心两件事：东西从哪来（fetch），东西到哪去（publish）。
fetch 方向：URL → 本地路径 → 使用。
publish 方向：本地路径 → 变换（插件）→ URL。
两个方向共用同一套寻址方式，但 Resource 中间态只存在于 fetch 方向。
## URL 格式
```
getter::url?param1=value1&param2=value2
```
getter 显式声明意图。不是"我猜这是个 git 仓库"，而是"使用者告诉我这是什么"。
只有三个 getter：
- `git` — 克隆仓库，必要时 checkout 指定版本
- `http` / `https` — 下载文件，必要时校验 checksum
- `file` — 复制本地文件或目录
版本锁定不用 `@v1.0`（`@` 在 URL 里是认证分隔符），用 `?ref=v1.0`。
没有 `github.com/user/repo` 看起来应该 clone 它这种黑魔法。
没有 `.tar.gz` 看起来应该解压它的惊喜。
URL 是 URL，行为是行为，两者用 getter:: 显式绑定。
## Resource 不是网络地址
流水线阶段（prebake / finalize / notify）只接触 Resource：一个名字 + 一个本地路径，已就绪。它们不经由网络、不解析 URL、不碰 getter。
所有 fetch 行为在阶段启动前完成：
- worker 模式：bakerd fetch → 填入 ResourceRegistry → 启动 baker
- standalone 模式：baker 自己 fetch → 填入 ResourceRegistry → 执行阶段
- bare 模式：不 fetch，传入的必须是 Resource
ResourceRegistry 是名字到路径的映射。填进去就只读了。
## getterurl 是独立的 crate
URL 格式解析 + 基本 fetch 放在 workshop-getterurl，和 baker 本身没有耦合。任何人都可以用。
crate 分两层：
```
结构层（parser）：
"git::https://host/repo.git?ref=v1.0" → { getter: "git", url: Url, params: Params }
获取层（builtin getters）：
getter.fetch(url, params, dst) → local_path
git: git clone + optional checkout
http: HTTP GET + optional checksum
file: std::fs::copy
```
获取层是字面意义的——不解压、不推断、不猜意图。
## Checksum
checksum 作为 URL query 参数传递。三态语义：
- `?checksum=sha256:abc` — 必须匹配
- `?checksum=` — 显式禁用校验
- 无 checksum — 如果服务端返回 Content-Digest 头，校验它
getter 不主动校验，它只执行 URL 里指定的操作。
## Artifact 不经过 Resource
artifact 是 bake 的产物，自产生就开始被使用，不存在中间态。它的路径就是 bake 产出的本地路径，不走 ResourceRegistry。
artifact 的 publish 方向由插件定义行为——可能上传、可能扫描、可能只记录。插件定义最终的行为，只要是自洽的。
## 三种运行模式
```
fetch 执行者 fetch 在阶段内
daemon 模式 bakerd 否
standalone baker 自身 否（阶段启动前已完成）
bare 模式 无 否（必须传入 Resource）
```
bare 模式下传入的必须是已经就绪的 Resource。外部 URL 是 daemon 或 standalone 的事，bare 不负责。
workshop-baker/Cargo.lock
Generated
+1 -1471
View File
File diff suppressed because it is too large Load Diff
workshop-baker/Cargo.toml
-5
View File
@@ -47,12 +47,7 @@ uuid = { version = "1.19.0", features = ["v4"] }
libc = "0.2"
os_info = "3.14.0"
privdrop = "0.5.6"
reqwest = { version = "0.12", default-features = false, features = ["json", "rustls-tls"] }
which = "8.0.2"
gix = "0.81.0"
sha2 = "0.10"
flate2 = "1.0"
zstd = "0.13"
globset = "0.4.18"
axum = "0.7"
workshop-baker/src/AGENTS.md
+42 -47
View File
@@ -1,62 +1,57 @@
# workshop-baker/src
**Generated:** 2026-04-22
**Commit:** 7b3b71a
**Generated:** 2026-05-19
**Commit:** 28abc5d
Baker CLI + execution engine for HoneyBiscuitWorkshop pipelines.
## OVERVIEW
CLI tool and runtime for executing CI/CD pipelines with resource isolation, privilege dropping, and package management integration.
CLI tool and runtime for executing CI/CD pipelines. Orchestrates prebake → bake → finalize stages with notification dispatch.
## STRUCTURE
```
src/
├── bake.rs # Top-level bake orchestration
├── bake/ # Script parsing and building
│ ├── parser.rs # @decorator-aware shell parser
│ ├── decorator.rs # @pipeline, @retry, @parallel, etc.
│ ├── builder.rs # Script template builder
│ └── schedule.rs # Function ordering via @after deps
├── engine/ # Execution runtime
│ ├── executor.rs # Script execution with cgroups
│ ├── cgroups.rs # Linux cgroup resource limits
│ ├── repology.rs # Package name resolution
│ ├── pm.rs # Package manager detection
│ └── upm.rs # User package manager (Nix/Guix)
├── prebake/ # Build environment setup
│ ├── config.rs # PrebakeConfig YAML schema
│ ├── stage/ # Bootstrap, DepsSystem, DepsUser, Hooks
│ └── security.rs # Privilege dropping logic
├── types/ # Shared type definitions
│ ├── repology.rs # RepologyEndpoint enum
│ ├── builderconfig.rs
│ ├── cache.rs
│ └── memsize.rs
├── monitor/ # Resource usage monitoring
│ ├── cgroups.rs # cgroup-based usage collection
│ └── jobobject.rs # Windows job objects (stub)
├── daemon.rs # Unix socket daemon (TODO: refactor)
└── lib.rs # CLI struct + module exports
├── lib.rs # Module declarations + Cli struct + Commands enum
├── main.rs # tokio::main entry point
├── bare.rs # Bare mode (run individual stages)
├── error.rs # CliError (wraps stage errors)
├── bake.rs # Bake orchestration (script → execution)
├── bake/ # Script parsing, decorator DSL, scheduling
├── prebake.rs # Prebake orchestration (env setup)
├── prebake/ # Config, stages, security, env providers
├── finalize.rs # Finalize orchestration (post-build)
├── finalize/ # Config, plugins, hooks, templates
├── notify.rs # Notification system (batching, retry)
├── notify/ # Types, templates, interactive, queue
├── daemon/ # Empty (Daemon command = unimplemented!)
├── monitor/ # Resource usage monitoring (cgroups)
├── types.rs # Module re-exports
├── types/ # buildstatus, resource types
├── utils.rs # Module root (fetch submodule)
└── utils/ # fetch.rs
```
## WHERE TO LOOK
| Task | Location |
|------|----------|
| Script parsing with @decorators | `bake/parser.rs:29` - `parse_script()` |
| Decorator types | `bake/decorator.rs` - `Decorator` enum |
| Build execution | `engine/executor.rs` - `Executor::execute_script()` |
| Resource limits | `engine/types.rs:56` - `ResourceLimits` struct |
| Prebake stages | `prebake/stage/` subdirectories |
| Package detection | `engine/pm.rs:detection()` |
| CLI entry | `lib.rs:26` - `cli::Cli` struct |
| CLI args | `lib.rs:21` - `Cli` struct |
| Entry point | `main.rs:91` - `#[tokio::main]` |
| Pipeline orchestration | `main.rs:68-74` - prebake→bake→finalize chain |
| Bare mode (single stage) | `bare.rs:14` - `run_bare()` |
| Script parsing | `bake/parser.rs:56` - `parse_script()` |
| Decorator DSL | `bake/decorator.rs` - `Decorator` enum |
| Prebake stages | `prebake/stage/` - Bootstrap→DepsSystem→DepsUser→Hooks→Ready |
| Plugin system | `finalize/plugin.rs` + `finalize/plugin/` |
| Notification dispatch | `notify.rs:66` - `notify()` async loop |
| Notification types | `notify/types.rs` - `NotificationRule`, `ParsedTrigger` |
## CONVENTIONS
- **ExecutionContext**: Passed through all stages, carries task_id, username, env_vars, timeout
- **PrebakeStage**: Ordered enum controlling stage execution sequence (Bootstrap -> Ready)
- **Decorator**: Attached to shell functions via `# @name(args)` comments above function declarations
- **Engine**: Lightweight wrapper around cgroup manager for resource isolation
- **Dual-mode**: CLI commands (prebake, bake, finalize) vs daemon mode via Unix socket
- **PrebakeStage**: Ordered enum Bootstrap→EarlyHook→DepsSystem→DepsUser→LateHook→Ready
- **Decorator**: `# @name(args)` comments above shell function declarations
- **Notification**: Plugin-based dispatch with batching, exponential backoff, priority groups
- **Engine**: Delegated to `workshop-engine` crate (formerly in-tree)
- **Dual-mode**: CLI commands (standalone) vs Daemon mode (unimplemented)
## ANTI-PATTERNS
1. **Empty daemon/** directory — Daemon command is `unimplemented!()`
2. **Dead code allowed** — `lints.rust.dead_code = "allow"` in Cargo.toml
3. **Engine was extracted** — workshop-engine is its own crate now, src/engine/ removed
4. **Python test infra exists but files removed** — pytest.ini remains, test dir is empty
workshop-baker/src/bake/AGENTS.md
+4 -2
View File
@@ -12,14 +12,16 @@ bake/
├── parser.rs # @decorator-aware shell script parser
├── decorator.rs # Decorator enum + 100+ unit tests
├── builder.rs # Script template assembly
└── schedule.rs # Function ordering via @after dependencies
├── schedule.rs # Function ordering via @after dependencies
├── error.rs # BakeError enum
└── constant.rs # DEFAULT_WORKSPACE, TRIVIAL_SCRIPT_NAME
```
## WHERE TO LOOK
| Task | Location |
|------|----------|
| Parse script | `parser.rs:29` - `parse_script()` |
Parse script | `parser.rs:56` - `parse_script()` |
| Decorator types | `decorator.rs` - `Decorator` enum variants |
| Build script | `builder.rs` - template builder |
| Schedule deps | `schedule.rs` - @after resolution |
workshop-baker/src/finalize/AGENTS.md
+28 -32
View File
@@ -1,56 +1,52 @@
# workshop-baker/src/finalize
**Generated:** 2026-04-22
**Commit:** 7b3b71a
**Generated:** 2026-05-19
**Commit:** 28abc5d
Artifact packaging, distribution, deployment, and notifications.
Artifact packaging, plugin-based notifications, and post-build hooks.
## STRUCTURE
```
finalize/
├── config.rs # FinalizeConfig YAML (artifacts, deploy, notify)
├── config.rs # FinalizeConfig YAML (379 lines, plugins, notifications)
├── stage.rs # FinalizeStage enum
├── stage/hook.rs # Post-build hooks
├── stage/hook.rs # Post-build hook execution
├── event.rs # Event propagation
├── template.rs # Variable substitution {{VAR}}
├── plugin.rs # Plugin trait + registry
├── template.rs # Variable substitution {{VARIABLE}}
├── plugin.rs # Plugin trait + registration
├── plugin/
│ ├── metadata.rs # Plugin manifest parsing
│ ├── dylib.rs # Dynamic library plugins
│ ├── rhai.rs # Rhai scripting (todo!())
│ ├── shell.rs # Shell command plugins
│ └── internal/ # Built-in plugins
│ ├── mail/ # Email notifications (SMTP)
│ ├── webhook.rs # HTTP webhooks
│ ├── satori.rs # Satori integration
│ └── insitenotify.rs # In-site notifications
│ └── fetch/ # Artifact fetching
│ ├── git.rs # Git repository fetch
│ ├── http.rs # HTTP download
│ ├── extract.rs # Archive extraction
│ └── checksum.rs# SHA256 verification (SHA1 deprecated)
│ └── internal.rs + internal/ # Built-in plugins (mail, webhook, satori, insitenotify)
├── types.rs
├── types/compression.rs
├── error.rs # FinalizeError
└── constant.rs
```
## WHERE TO LOOK
| Task | Location |
|------|----------|
| Config loading | `config.rs` - FinalizeConfig |
| Email notify | `plugin/internal/mail/` - SMTP templates |
| Config loading | `config.rs:17` - `FinalizeConfig` + `parse()` |
| Plugin registration | `plugin.rs` - `register()` |
| Email notifications | `plugin/internal/mail.rs` - SMTP |
| Webhooks | `plugin/internal/webhook.rs` |
| Artifact fetch | `plugin/fetch/` - git/http/extract |
| Checksums | `plugin/fetch/checksum.rs:13` - SHA1 deprecated |
| Satori integration | `plugin/internal/satori.rs` |
| Shell plugins | `plugin/shell.rs` |
| Template rendering | `template.rs` - `{{VARIABLE}}` handlebars-style |
## CONVENTIONS
- **Plugin System**: Dynamic + internal plugins via Plugin trait
- **Template Syntax**: `{{VARIABLE}}` handlebars-style
- **Stages**: Hook → Fetch → Build → Publish → Notify → Cleanup
- **Fetch**: Supports git, http, with extraction and checksum verify
- **Plugin System**: `FinalizePlugin` trait + registry with both internal and dynamic plugins
- **Template Syntax**: `{{VARIABLE}}` handlebars-style substitution via `template.rs`
- **Stages**: EarlyHook → (Artifacts) → LateHook → Notifications
- **Notification**: `notify.rs` in parent module drives plugin-based notification dispatch
- **ResourceRegistry**: Shared resource cache passed through to plugins
## ANTI-PATTERNS
1. **SHA1 deprecated** — `checksum.rs:13` warns SHA1 is deprecated
2. **Unimplemented** — `rhai.rs:17` - "todo!()" for Rhai scripting
3. **TODO markers** — Multiple TODOs in config.rs (lines 305-306)
1. **fetch/ subdirectory removed** — was `plugin/fetch/{git,http,extract,checksum}`, no longer exists
2. **SHA1 deprecated** — workshop-getterurl handles checksums now
3. **Unimplemented** — `rhai.rs:17` - "todo!()" for Rhai scripting
4. **TODO markers** — Multiple TODOs in config.rs (lines 305-306)
5. **Artifact stage not implemented** — `finalize.rs:57` — "TODO: artifact stage implementation"
workshop-baker/src/prebake/AGENTS.md
+8 -5
View File
@@ -10,7 +10,7 @@ Build environment setup: bootstrap, dependencies, security, hooks.
```
prebake/
├── config.rs # PrebakeConfig YAML schema (environment, deps, cache)
├── stage.rs # PrebakeStage enum (Bootstrap→Ready ordering)
├── stage.rs # PrebakeStage enum (Bootstrap→EarlyHook→DepsSystem→DepsUser→LateHook→Ready)
├── stage/
│ ├── bootstrap.rs # User creation (vulcan), sudo/doas setup
│ ├── environment.rs # Env var injection
@@ -24,7 +24,12 @@ prebake/
│ ├── baremetal.rs # Direct host execution
│ └── custom.rs # Custom env (todo!())
├── security.rs # Privilege dropping (unsafe libc calls)
└── event.rs # Event logging
├── event.rs # Event logging
├── error.rs # PrebakeError + BootstrapError
├── types.rs # Module re-exports
├── types/ # memsize, cache, builderconfig, architecture
├── constant.rs # Module constants
└── env.rs # Environment module root
```
## WHERE TO LOOK
@@ -38,13 +43,11 @@ prebake/
| Hooks | `stage/hook.rs` - pre/post stage hooks |
## CONVENTIONS
- **PrebakeStage**: Ordered enum (Bootstrap→DepsSystem→DepsUser→Environment→Hooks→Ready)
- **PrebakeStage**: Ordered enum Bootstrap→EarlyHook→DepsSystem→DepsUser→LateHook→Ready
- **ExecutionContext**: Carries task_id, username, env_vars through all stages
- **Target User**: "vulcan" (hardcoded in bootstrap.rs:6-24)
## ANTI-PATTERNS
1. **Hardcoded user** — "vulcan" username hardcoded throughout
2. **Unsafe blocks** — `security.rs` has 5+ unsafe libc calls
3. **Incomplete** — custom.rs and apt.rs have `todo!()` stubs
workshop-baker/src/utils/fetch.rs
-45
View File
@@ -1,45 +0,0 @@
mod checksum;
mod error;
mod extract;
mod file;
mod git;
mod http;
mod scheme;
mod types;
use std::path::{Path, PathBuf};
pub use error::FetchError;
pub use scheme::Scheme;
pub use types::FetchArgument;
/// Fetch an external resource to a target directory. Returns the local path.
///
/// - `git://repo@tag` → clone
/// - `https://url` → download (+ checksum + extract)
/// - `file:///path` → copy to target
///
/// Note: `workspace://` must be converted to `file://` by the caller before passing.
pub async fn fetch_resource(
name: &str,
url: &str,
target: &Path,
) -> Result<PathBuf, FetchError> {
let scheme = scheme::parse_url(url)?;
match scheme {
Scheme::Git { repo, git_ref } => {
let full_url = if git_ref.is_empty() {
repo.clone()
} else {
format!("{}@{}", repo, git_ref)
};
git::fetch_git_plugin(&full_url, name, target, None).await?;
Ok(target.join(name))
}
Scheme::Http { url, checksum } => {
http::fetch_http_plugin(&url, name, target, checksum.as_deref()).await?;
Ok(target.join(name))
}
Scheme::File { path } => file::fetch_file(&path, name, target).await,
}
}
workshop-engine/src/AGENTS.md
+51
View File
@@ -0,0 +1,51 @@
# workshop-engine/src
**Generated:** 2026-05-19
**Commit:** 28abc5d
Execution engine extracted from workshop-baker. Process management, package detection, resource isolation.
## STRUCTURE
```
src/
├── lib.rs # Module declarations + 10+ convenience re-exports
├── error.rs # ExecutionError + DependencyError + HasExitCode trait
├── types.rs # ExecutionContext, ExecutionResult, Engine, etc.
├── child.rs # ManagedChild + TokioChild process wrappers
├── command.rs # CustomCommand builder
├── executor.rs # Script execution
├── time.rs # Duration parsing helpers
├── pm.rs # PackageManager trait + detection
├── pm/ # Backends: apt, apk, dnf, pacman
├── upm.rs # UserPackageManager trait
├── upm/ # Backends: rust (cargo), python (pip), go, custom
├── repology.rs # RepologyEndpoint enum
└── repology/ # Package name resolution (local/)
```
## KEY EXPORTS
| Symbol | Type | Role |
|--------|------|------|
| ExecutionContext | struct | Pipeline context (task_id, username, env, etc.) |
| Engine | struct | Script executor (placeholder) |
| ExecutionError | enum | Error types with exit codes |
| HasExitCode | trait | Exit code protocol for all error types |
| ManagedChild | struct | Child process wrapper with managed lifecycle |
| TokioChild | struct | Tokio-based child process wrapper |
| CustomCommand | struct | Builder for command execution |
| PackageManager | trait | System package manager abstraction |
| UserPackageManager | trait | User-level package manager abstraction |
| RepologyEndpoint | enum | Package name resolution endpoint |
| ExecutionEvent | enum | Event stream (TaskStarted, OutputChunk, etc.) |
| ExecutionResult | struct | Command execution result (exit_code, stdout, stderr) |
| EventSender | type alias | `Option<mpsc::Sender<ExecutionEvent>>` |
## CONVENTIONS
- **Exit codes**: Centralized in `error.rs` — EXITCODE_OK=0 through EXITCODE_PRIV_DROP_FAILED=201
- **ExecutionContext**: Defaults to current directory, bash shell, non-privileged
- **PackageManagers**: Auto-detected via `os_info`, fallback chain (apt→apk→dnf→pacman)
- **Engine**: Currently a unit struct with `execute_script()` method
- **All errors implement HasExitCode**: Enables uniform error propagation up to CLI
## ANTI-PATTERNS
1. **Engine is a stub** — `types.rs:99` — `pub struct Engine;` with no fields, minimal implementation
workshop-engine/src/cgroups.rs
-115
View File
@@ -1,115 +0,0 @@
use std::fs;
use std::io::{self, Write};
use std::path::PathBuf;
/// Linux cgroup resource manager for ws-executor.
///
/// Manages cgroup v2 hierarchy under `/sys/fs/cgroup/ws-executor/` for
/// CPU, memory, and process isolation.
pub struct CgroupManager {
path: PathBuf,
}
impl CgroupManager {
/// Creates a new cgroup for the given build ID.
pub fn new(build_id: &str) -> io::Result<Self> {
let path = PathBuf::from(format!("/sys/fs/cgroup/ws-executor/{}", build_id));
fs::create_dir_all(&path)?;
Ok(Self { path })
}
/// Sets memory limit in bytes. Use 0 for unlimited.
pub fn set_memory_limit(&self, limit_bytes: u64) -> io::Result<()> {
let mem_max_path = self.path.join("memory.max");
let content = if limit_bytes == 0 {
"max".to_string()
} else {
limit_bytes.to_string()
};
fs::write(&mem_max_path, content)?;
let mem_swap_path = self.path.join("memory.swap.max");
fs::write(&mem_swap_path, "max")?;
Ok(())
}
/// Sets CPU weight (1-10000, higher = more CPU time).
pub fn set_cpu_weight(&self, weight: u64) -> io::Result<()> {
let cpu_weight_path = self.path.join("cpu.weight");
let weight = weight.clamp(1, 10000);
fs::write(&cpu_weight_path, weight.to_string())?;
Ok(())
}
/// Sets CPU max burst in microseconds with optional period.
pub fn set_cpu_max(&self, max_us: u64, period_us: u64) -> io::Result<()> {
let cpu_max_path = self.path.join("cpu.max");
let content = if max_us == 0 {
"max".to_string()
} else {
format!("{} {}", max_us, period_us)
};
fs::write(&cpu_max_path, content)?;
Ok(())
}
/// Adds a process to this cgroup.
pub fn add_process(&self, pid: u32) -> io::Result<()> {
let cgroup_procs_path = self.path.join("cgroup.procs");
let mut file = fs::OpenOptions::new()
.append(true)
.open(&cgroup_procs_path)?;
writeln!(file, "{}", pid)?;
Ok(())
}
/// Destroys the cgroup directory.
pub fn destroy(&self) -> io::Result<()> {
if self.path.exists() {
fs::remove_dir(&self.path)?;
}
Ok(())
}
/// Returns the cgroup path.
pub fn path(&self) -> &PathBuf {
&self.path
}
}
#[cfg(test)]
mod tests {
use super::*;
#[test]
#[ignore]
fn test_cgroup_manager_new() {
let build_id = "test-build-123";
let manager = CgroupManager::new(build_id).unwrap();
assert!(manager.path().exists());
manager.destroy().unwrap();
}
#[test]
#[ignore]
fn test_set_memory_limit() {
let build_id = "test-mem-123";
let manager = CgroupManager::new(build_id).unwrap();
manager.set_memory_limit(1024 * 1024 * 1024).unwrap();
manager.destroy().unwrap();
}
#[test]
#[ignore]
fn test_set_cpu_weight() {
let build_id = "test-cpu-123";
let manager = CgroupManager::new(build_id).unwrap();
manager.set_cpu_weight(1024).unwrap();
manager.destroy().unwrap();
}
}