diff --git a/docs/src/SUMMARY.md b/docs/src/SUMMARY.md index c127b54..ecb8faa 100644 --- a/docs/src/SUMMARY.md +++ b/docs/src/SUMMARY.md @@ -29,6 +29,7 @@ - [测试策略](zh-CN/vol3_dev/testing_strategy.md) - [插件系统指南](zh-CN/vol3_dev/plugin_system_guide.md) - [Engine 模块指南](zh-CN/vol3_dev/engine_module_guide.md) + - [设计决策](zh-CN/vol3_dev/design_decisions.md) - [添加包管理器](zh-CN/vol3_dev/add_package_manager.md) - [自定义插件](zh-CN/vol3_dev/custom_plugin.md) - [自定义 Decorator](zh-CN/vol3_dev/custom_decorator.md) diff --git a/docs/src/zh-CN/vol3_dev/design_decisions.md b/docs/src/zh-CN/vol3_dev/design_decisions.md new file mode 100644 index 0000000..d879b2b --- /dev/null +++ b/docs/src/zh-CN/vol3_dev/design_decisions.md @@ -0,0 +1,412 @@ +# 设计决策 + +本文档记录了 HoneyBiscuitWorkshop 架构演进过程中的关键设计决策及其背后的思考。 + +> **注意**：本文档面向开发者，记录的是**目标架构**的设计方向，而非当前代码的实现现状。 +> 当前 bakerd 尚未完全实现（`daemon.rs` 为 `todo!()`），bare mode 仅用于开发调试。 + +--- + +## 1. Bare Mode 与 Daemon Mode + +### 决策 + +**Bare mode 仅用于开发调试，不承载生产语义。** + +### 背景 + +当前代码提供两种运行入口： + +| 模式 | 入口 | 状态 | 用途 | +|------|------|------|------| +| **Bare mode** | `cargo run -- prebake/bake/finalize ` | 可用 | 开发调试 | +| **Daemon mode** | `cargo run --` (无子命令) | `todo!()` | 生产 | + +### 推论 + +- Bare mode 中 `ctx.privileged` 是写死的 fake 值（`Prebake: true`, `Bake: false`），不代表生产行为 +- Bare mode 中 `os_info::get()` 读取的是 HOST 的 OS 信息，不代表环境内的 OS +- 不要从 bare mode 的行为推导生产路径的问题 +- 不要为了"修复" bare mode 的问题而给生产设计增加不必要的抽象 + +### 代码残留 + +当前 `bake.rs` 中存在以下已在设计层面淘汰、但因 bare mode 未被清理的代码： + +- `fn privileged()` — 从未被调用 +- `type TaskResult` — 从未被使用 +- `type TaskFn` — 从未被使用 + +这些是 `bake.rs` 废弃功能的残留，不涉及生产设计。清理它们不改变任何行为。 + +--- + +## 2. 环境模型 + +### 决策 + +**bakerd 先创建环境（容器/VM/裸机），然后 baker 在环境内执行全部三个阶段（prebake → bake → finalize）。** + +### 架构 + +``` +bakerd + ├── prepare_environment (容器/VM/裸机) ← 先建隔离环境 + ├── prebake (全在环境内执行) ← 包括 depssystem 的包安装 + ├── bake (全在环境内执行) ← 构建脚本 + ├── finalize (全在环境内执行) ← 部署通知 + └── cleanup +``` + +### 推论 + +- `prepare_environment` 不是 prebake 的一个 stage——它是 prebake 的前置条件 +- `depssystem.rs` 中 `os_info::get()` 读取的应当是**环境内**的 `/etc/os-release`，逻辑正确 +- 容器镜像只需拉取（`prepare_container`），不需要在代码层面管理容器生命周期——那是 bakerd 的职责 +- 当前 `prepare_container` 返回的 `_container` 被丢弃，是因为 bakerd 尚未实现，不是设计缺陷 + +### 环境类型 + +```yaml +env: + type: container # Docker/Podman 容器 + # type: firecracker # Firecracker 微 VM + # type: baremetal # 裸机直接执行 + image: ubuntu:24.04 +``` + +环境声明决定了： + +1. 隔离级别（进程级 / VM级 / 无隔离） +2. 操作系统和工具链（通过 `image`） +3. 包管理器（从 OS 推导） + +--- + +## 3. PM 选择链 + +### 决策 + +**包管理器（PM）应从环境声明中推导，而不是通过运行时探测。** + +### 选择链 + +``` +用户声明 bakerd 确定 执行时确定 +env.image: ubuntu:24.04 + │ + ▼ + 容器运行时 (Docker/VM) + │ os_info::get() 在环境内 + ▼ + OS: Ubuntu 24.04 + │ 从已知映射推导 + ▼ + PM: apt + │ depssystem 使用 PM trait 操作 + ▼ + apt install -y {packages} +``` + +### 为什么淘汰 detect() + +`pm::detect()` 的当前逻辑： + +```rust +pub fn detect(distro: &os_info::Info) -> Option> { + all_managers().into_iter().find(|pm| pm.adopt(distro)) +} +``` + +它在解决一个**已经知道答案的问题**： + +- 用户声明了 `image: ubuntu:24.04` +- 容器内 `/etc/os-release` 确认了 Ubuntu +- `detect()` 遍历所有 PM 并匹配，得出 apt +- **结果和环境声明一致，但绕了一大圈** + +`detect()` 的唯一价值场景： + +- **Bare mode** 用户没配环境时，省一个配置项 +- 这在生产路径中不应出现（bakerd 总是提供环境） + +### 推论 + +- PM trait 保留（见下节），但 `adopt()` 方法在生产路径中不需要 +- `detect()` / `adopt()` / `os_info::get()` 这条路径应当是 dev-only 的便利设施 +- Daemon 模式中 PM 的选择依据是环境声明，不是运行时探测 +- 当前 `os_info::Type` 的 58 个变体覆盖不全（仅 Pacman 可用）——这是 bare mode 的问题，不影响生产设计 + +--- + +## 4. PM Trait 的职责边界 + +### 决策 + +**PM trait 保留核心操作能力，淘汰探测分类能力。** + +### 核心方法 + +```rust +pub trait PackageManager { + fn name(&self) -> &'static str; + fn update(&self) -> Vec; + fn install(&self, package: &[String]) -> Vec; + fn change_mirror(&self, repo: &str, osinfo: Option<&Info>) -> Vec; + fn add_repository(&self, repo: &Value, osinfo: Option<&Info>) -> Vec; +} +``` + +这四个方法是 PM-specific 的操作，具有真实复杂度： + +| 操作 | PM 间差异 | 原因 | +|------|-----------|------| +| `install` | 参数完全不同 | `apt install -y` vs `pacman -S --noconfirm` | +| `update` | 命令不同 | `apt update` vs `pacman -Sy` | +| `change_mirror` | 配置格式不同 | sources.list vs mirrorlist vs .repo | +| `add_repository` | 仓库机制不同 | PPA vs AUR vs COPR | + +### 为什么镜像源和仓库在容器中仍然需要 + +容器不解决源管理问题： + +``` +docker pull ubuntu:24.04 +# 容器内 sources.list → archive.ubuntu.com +# 但用户在北京 → 需要 mirror.tuna.tsinghua.edu.cn +# 这不是预置在镜像中的——源是运行态配置 +``` + +`change_mirror` 和 `add_repository` 都是**容器运行时需要**的操作，不能预置在基础镜像中。PM trait 的存在价值就在于这些操作。 + +### 淘汰的方法 + +```rust +fn adopt(&self, distro: &Info) -> bool; // 生产路径不需要 +fn binary(&self) -> &'static str; // 从未被使用 +``` + +### 实现策略 + +在 daemon 模式下，PM 的选型由环境声明确定（`image: ubuntu:24.04` → `apt`），PM 实现仅需按需注册，不需要自述适配范围： + +```rust +// 不再需要 detect() + adopt() +// PM 实例由工厂方法根据 PM 名称字符串创建 +pub fn from_name(name: &str) -> Option> { + match name { + "apt" => Some(Box::new(Apt)), + "pacman" => Some(Box::new(Pacman)), + "dnf" => Some(Box::new(Dnf)), + _ => None, + } +} +``` + +--- + +## 5. 发行版多态 + +### 决策 + +**流水线不应是发行版多态的。发行版差异由容器化解决。** + +### 原因 + +流水线的执行体（bake 脚本）天然发行版绑定： + +```bash +# @pipeline +function build() { + cmake -B build -DCMAKE_INSTALL_PREFIX=/usr # Linux 路径 + make -j$(nproc) +} +``` + +- 换到 macOS：`/usr` 只读，路径不同 +- 换到 Windows：MSVC 而不是 GCC +- 换到 Alpine：musl 而不是 glibc + +这与 PM 的差异是同一类问题，但 PM 抽象不能解决它。**容器才是解决方案：** + +```yaml +# 在 Ubuntu 上构建 +env: + type: container + image: ubuntu:24.04 +``` + +```yaml +# 在 Alpine 上构建 +env: + type: container + image: alpine:3.19 +``` + +### 各层面的发行版相关性 + +| 层面 | 发行版相关 | 解决方案 | +|------|-----------|----------| +| DepsSystem（系统包） | ✅ | 容器内 PM | +| DepsUser（用户包管理器） | ❌ | 天然无关（cargo/npm/pip 跨发行版一致） | +| Bake（构建脚本） | ✅ | 容器内执行 | +| Finalize（后处理） | ❌ | 纯产物操作，无关发行版 | + +### UPM 的特殊位置 + +用户包管理器（UPM，如 cargo、npm、pip、nix）天然跨发行版： + +```bash +cargo install --locked bat # Ubuntu 和 Arch 上都一样 +npm install -g typescript # Ubuntu 和 Arch 上都一样 +``` + +`depsuser.rs` 处理的正是这个层面。但 `collect_upm_sysdeps()` 需要知道系统 PM——这是 UPM 和系统 PM 之间的 bridge。在设计上，这个 bridge 的 PM 信息应来自环境声明，而非 `detect()`。 + +--- + +## 6. drop_after 安全模型 + +### 决策 + +**`drop_after` 指定流水线在哪个 stage 之后从 root 降权到 vulcan 用户。合理的默认值是 `DepsUser`。** + +### Stage 顺序 + +```rust +pub enum PrebakeStage { + Init, + Bootstrap, + EarlyHook, + DepsSystem, // ← 默认 + DepsUser, + LateHook, + Ready, + Never, // ← 永远不降权 +} +``` + +### 各 stage 的特权需求 + +| Stage | 需要 root | 原因 | +|-------|-----------|------| +| Bootstrap | ✅ | 创建系统用户（vulcan）、配置 sudo/doas | +| EarlyHook | ❓ | 用户自定义 | +| DepsSystem | ✅ | `apt install` 需要 root | +| DepsUser | ❌ | `cargo install` 不需要 root | +| LateHook | ❓ | 用户自定义 | +| Ready / Bake | ❌ | 构建应以非特权用户运行 | + +### 正确的配置 + +```yaml +security: + drop_after: "DepsUser" +``` + +执行效果： + +``` +Bootstrap root ← 创建用户 +EarlyHook root ← hook 以 root 运行 +DepsSystem root ← apt install -y {packages} +DepsUser root ← cargo install {packages} +LateHook vulcan ← 降权，用户脚本不能 rm -rf / +Ready vulcan +Bake vulcan ← 构建以 vulcan 运行 +``` + +### 禁止的配置 + +`drop_after` 设置为 `Bootstrap` 或 `EarlyHook` 在当前 stage 顺序下会损坏 DepsStage（需要 root 权限的 `apt install` 在降权后无法执行）。 + +**配置验证应拒绝 `drop_after < DepsUser`**（以 root 运行的 user 级包安装没有意义，但 DepsSystem 必须 root）。 + +### 特殊值 Never + +```yaml +security: + drop_after: "Never" # 整条流水线以 root 运行 +``` + +适用场景：Docker 构建、内核模块编译等需要 root 权限的操作。Bootstrap 的 `user: root` 配置也会产生相同效果。 + +--- + +## 7. 当前代码中的真问题 + +以下问题是当前代码中确实存在的，与 bare/daemon 模式无关： + +### 7.1 Apt::adopt 永远返回 false + +```rust +// workshop-engine/src/pm/apt.rs:20-23 +fn adopt(&self, distro: &Info) -> bool { + return false; // ← 永远 false + // todo!(); // ← 死代码 +} +``` + +- `adopt()` 在 `return false` 后无法执行 `todo!()` +- Apt 永远不会被 `detect()` 选中 +- 后果：Debian/Ubuntu 用户在 bare mode 下永远检测不到 PM +- 修复方向：移除 return false，实现真正的 adopt 逻辑（Debian/Ubuntu/Mint/Pop 等发行版的匹配） + +### 7.2 PM 覆盖严重不全 + +当前 `all_managers()` 仅注册了 Pacman： + +```rust +pub fn all_managers() -> Vec> { + vec![ + // Box::new(apt::Apt), // adopt 坏了 + Box::new(pacman::Pacman), // 唯一可用 + // Box::new(dnf::Dnf), // 未实现 + ] +} +``` + +os_info 3.14.0 定义了 58 种发行版类型，Pacman 仅覆盖其中 9 种（Arch 系）。其余 49 种无 PM 覆盖。 + +### 7.3 死代码 + +| 位置 | 内容 | 状态 | +|------|------|------| +| `bake.rs:98` | `fn privileged()` | 定义但未调用 | +| `bake.rs:108` | `type TaskResult` | 定义但未使用 | +| `bake.rs:110` | `type TaskFn` | 定义但未使用 | +| `workshop-baker/src/prebake/stage/environment.rs` | `prepare_environment()` | 定义但未接入执行流 | + +--- + +## 8. Daemon 设计原则（汇总） + +以上讨论最终可以提炼为以下设计原则，供 bakerd 重构时参考： + +### P1. 环境声明驱动 + +``` +用户声明 env.image → bakerd 创建环境 → 环境内确定 PM → depssystem 操作 PM +``` + +PM 的选择权在环境声明，不在运行时探测。`detect()` 不出现在生产路径中。 + +### P2. 容器是发行版隔离的边界 + +发行版差异在容器边界被隔断。容器内是已知、确定的环境。PM trait 只负责"已知 PM 的操作"（install、update、mirror、repo），不负责"猜是什么 PM"。 + +### P3. 降权边界可配置但需校验 + +`drop_after` 从 root 降权到 vulcan，默认在 DepsUser 之后。配置验证需确保：降权不会损坏需要 root 的 stage。 + +### P4. 最小化 trait 抽象 + +PM trait 保留操作系统所需的核心方法。不在 trait 中嵌入分类功能（adopt）。分类是一个数据映射问题，不是多态问题。 + +### P5. Bare mode 是开发工具，不承载生产行为 + +Bare mode 的行为（host 上直接执行、write static privileged 等）不代表生产设计。不要为了优化 bare mode 而增加生产路径的抽象负担。 + +### P6. 天然跨发行版的部分不需要抽象 + +UPM（cargo/npm/pip/nix）跨发行版一致，不需要 PM 级别的抽象，不需要 detect，不需要 distro 感知。trait 只应用于确实有 PM-specific 差异的部分。 diff --git a/runner.sh b/runner.sh index a849795..c324eec 100755 --- a/runner.sh +++ b/runner.sh @@ -1,112 +1,101 @@ #!/bin/bash set -euo pipefail -# ============================================================================= # HoneyBiscuitWorkshop System Test Runner -# ============================================================================= -# Usage: ./runner.sh [auto|manual] [-- pytest-args...] # -# Modes: -# auto - Build, run all tests, collect reports, cleanup (default) -# manual - Build, start container shell for interactive testing -# -# Examples: -# ./runner.sh # Run all tests automatically -# ./runner.sh auto -k test_prebake # Run only prebake tests -# ./runner.sh manual # Drop into container shell -# ============================================================================= +# Usage: +# ./runner.sh # Arch baker tests (hbw-system-test:latest) +# ./runner.sh --image hbw-test-pm:test-apt # Debian engine PM tests +# ./runner.sh -k test_install # Arch baker, filtered -MODE="${1:-auto}" -shift 2>/dev/null || true -PYTEST_ARGS="$@" +MODE="auto" +TEST_IMAGE="" +TEST_DIR="tests/baker" +PYTEST_ARGS="" + +while [[ $# -gt 0 ]]; do + case "$1" in + --image) TEST_IMAGE="$2"; TEST_DIR="tests/engine"; shift 2 ;; + manual) MODE="manual"; shift ;; + auto) MODE="auto"; shift ;; + --) shift; PYTEST_ARGS="$@"; break ;; + *) PYTEST_ARGS="$PYTEST_ARGS $1"; shift ;; + esac +done + +# Default image: Arch-based baker test image +: ${TEST_IMAGE:="hbw-system-test:latest"} SCRIPT_DIR="$(cd "$(dirname "$0")" && pwd)" +CONTAINER_NAME="hbw-system-test-container" STAGING_DIR="/tmp/hbw-system-test" -IMAGE_NAME="hbw-system-test:latest" -# ---- Cleanup handler ---- cleanup() { - echo "[runner] Cleaning up..." - docker rm -f hbw-system-test-container 2>/dev/null || true - rm -rf "$STAGING_DIR" + docker rm -f "$CONTAINER_NAME" 2>/dev/null || true + [ "$TEST_DIR" = "tests/baker" ] && rm -rf "$STAGING_DIR" } trap cleanup EXIT -# ---- Step 1: Build the binary ---- build_binary() { - echo "[runner] Building workshop-baker (release)..." + local target="x86_64-unknown-linux-musl" + local binary="$SCRIPT_DIR/workshop-baker/target/$target/debug/workshop-baker" + if [ ! -f "$binary" ]; then + echo "[runner] Building static (musl) baker binary..." cd "$SCRIPT_DIR/workshop-baker" - cargo build --release 2>&1 - echo "[runner] Build complete." + cargo build -p workshop-baker --target "$target" 2>&1 | tail -n 1 + fi } -# ---- Step 2: Prepare staging directory ---- prepare_staging() { - echo "[runner] Preparing staging directory..." - rm -rf "$STAGING_DIR" + # Only needed for baker tests (hbw-system-test:latest mounts /workshop) + if [ "$TEST_DIR" = "tests/baker" ]; then mkdir -p "$STAGING_DIR" - - # Copy binary - cp "$SCRIPT_DIR/workshop-baker/target/release/workshop-baker" "$STAGING_DIR/" + cp "$SCRIPT_DIR/workshop-baker/target/x86_64-unknown-linux-musl/debug/workshop-baker" "$STAGING_DIR/" chmod +x "$STAGING_DIR/workshop-baker" - - # Copy examples if they exist - if [ -d "$SCRIPT_DIR/workshop-baker/examples" ]; then - cp -r "$SCRIPT_DIR/workshop-baker/examples" "$STAGING_DIR/examples" - else - mkdir -p "$STAGING_DIR/examples" - fi - - # Copy llm.env if it exists - if [ -f "$SCRIPT_DIR/llm.env" ]; then - cp "$SCRIPT_DIR/llm.env" "$STAGING_DIR/llm.env" - fi - - echo "[runner] Staging ready at $STAGING_DIR" + fi } -# ---- Step 3: Build Docker image ---- -build_image() { - echo "[runner] Building Docker image: $IMAGE_NAME ..." - docker build -t "$IMAGE_NAME" -f "$SCRIPT_DIR/tests/Dockerfile" "$SCRIPT_DIR" - echo "[runner] Image built: $IMAGE_NAME" -} - -# ---- Step 4: Run container ---- run_container() { - local mode="$1" - echo "[runner] Starting container in $mode mode..." + local tag="${TEST_IMAGE##*:}" + local filter="" + case "$tag" in + test-arch) filter="-k pacman" ;; + test-apt) filter="-k apt" ;; + test-dnf) filter="-k dnf" ;; + test-apk) filter="-k apk" ;; + esac - mkdir -p "$SCRIPT_DIR/tests/results" + echo "[runner] Starting: $TEST_IMAGE (tests=$TEST_DIR${filter:+, filter=$filter})" + mkdir -p "$SCRIPT_DIR/tests/results" - if [ "$mode" = "manual" ]; then - docker run -it --rm \ - --name hbw-system-test-container \ - --privileged \ - -v "$STAGING_DIR:/workshop:ro" \ - -v "$SCRIPT_DIR/tests/results:/results" \ - -e TEST_MODE=manual \ - "$IMAGE_NAME" - else - docker run --rm \ - --name hbw-system-test-container \ - --privileged \ - -v "$STAGING_DIR:/workshop:ro" \ - -v "$SCRIPT_DIR/tests/results:/results" \ - -e TEST_MODE=auto \ - "$IMAGE_NAME" $PYTEST_ARGS - fi + local mounts="-v $SCRIPT_DIR:/src:ro" + local entrypoint="" + + if [ "$TEST_DIR" = "tests/baker" ]; then + mounts="$mounts -v $STAGING_DIR:/workshop:ro" + entrypoint='--entrypoint ""' + fi + + local cmd="sh -c \"python3 -m venv /tmp/v && . /tmp/v/bin/activate && \ + pip install pytest pytest-timeout -q && \ + exec python -m pytest $TEST_DIR $PYTEST_ARGS $filter\"" + + if [ "$MODE" = "manual" ]; then + docker run -it --rm --name "$CONTAINER_NAME" --privileged $entrypoint \ + $mounts "$TEST_IMAGE" + else + docker run --rm --name "$CONTAINER_NAME" --privileged $entrypoint \ + $mounts -w /src \ + "$TEST_IMAGE" "$cmd" + fi } -# ---- Main flow ---- echo "============================================" -echo " HoneyBiscuitWorkshop System Test Runner" -echo " Mode: $MODE" +echo " HBW System Test Runner" +echo " Image: $TEST_IMAGE" +echo " Tests: $TEST_DIR" echo "============================================" - build_binary prepare_staging -build_image -run_container "$MODE" - +run_container echo "[runner] Done." diff --git a/tests/test_bake.py b/tests/baker/test_bake.py similarity index 100% rename from tests/test_bake.py rename to tests/baker/test_bake.py diff --git a/tests/test_finalize.py b/tests/baker/test_finalize.py similarity index 100% rename from tests/test_finalize.py rename to tests/baker/test_finalize.py diff --git a/tests/test_prebake.py b/tests/baker/test_prebake.py similarity index 100% rename from tests/test_prebake.py rename to tests/baker/test_prebake.py diff --git a/tests/engine/test_pm.py b/tests/engine/test_pm.py new file mode 100644 index 0000000..c8ae541 --- /dev/null +++ b/tests/engine/test_pm.py @@ -0,0 +1,102 @@ +"""PM system tests driven by bare workshop-baker prebake.""" + +import os +import pytest +from probes import binary_exists + +PACMAN_MIRROR = "https://mirrors.tuna.tsinghua.edu.cn/archlinux/$repo/os/$arch" + + +def _yml(pm, pkgs, mirror="", repo=None): + """Build prebake.yml. repo is a list of lines (without lead indent).""" + m = "" + if mirror: + m += ' mirror: "' + mirror + '"\n' + if repo: + m += " repositories:\n" + for line in repo: + m += " " + line + "\n" + return """\ +version: "1.0" +environment: + builder: "baremetal" +bootstrap: + user: "vulcan" +dependencies: + system: + {pm}: + packages: [{pkgs}] +{m}""".format(pm=pm, pkgs=pkgs, m=m) + + +# ── install ────────────────────────────────────────────────────────── + +@pytest.mark.parametrize("pm_name,pkgs", [ + ("pacman", '"curl", "git"'), + ("apt", '"curl", "git"'), + ("dnf", '"curl", "git"'), + ("apk", '"curl", "git"'), +]) +def test_install(run_cmd, work_dir, pm_name, pkgs): + cfg = os.path.join(work_dir, "pm.yml") + mirror = PACMAN_MIRROR if pm_name == "pacman" else "" + with open(cfg, "w") as f: + f.write(_yml(pm_name, pkgs, mirror=mirror)) + r = run_cmd(["/baker", "-p", "t", "-b", "1", "-u", "testuser", + "bare", "prebake", cfg], timeout=120) + assert r["success"], "[{}] prebake: {}".format(pm_name, r["stderr"][:300]) + for pkg in ["curl", "git"]: + assert binary_exists("/usr/bin/" + pkg), "[{}] {} missing".format(pm_name, pkg) + + +@pytest.mark.parametrize("pm_name", ["pacman", "apt", "dnf", "apk"]) +def test_reinstall(run_cmd, work_dir, pm_name): + cfg = os.path.join(work_dir, "pm.yml") + mirror = PACMAN_MIRROR if pm_name == "pacman" else "" + with open(cfg, "w") as f: + f.write(_yml(pm_name, '"curl"', mirror=mirror)) + cmd = ["/baker", "-p", "t", "-b", "1", "-u", "testuser", "bare", "prebake", cfg] + r1 = run_cmd(cmd, timeout=120) + assert r1["success"], "[{}] first install failed".format(pm_name) + r2 = run_cmd(cmd, timeout=120) + assert r2["success"], "[{}] reinstall failed".format(pm_name) + + +# ── change_mirror ──────────────────────────────────────────────────── + +@pytest.mark.parametrize("pm_name", ["pacman", "apt", "dnf", "apk"]) +def test_mirror(run_cmd, work_dir, pm_name): + cfg = os.path.join(work_dir, "pm.yml") + mirror = PACMAN_MIRROR if pm_name == "pacman" else "" + with open(cfg, "w") as f: + f.write(_yml(pm_name, '"which"', mirror=mirror)) + r = run_cmd(["/baker", "-p", "t", "-b", "1", "-u", "testuser", + "bare", "prebake", cfg], timeout=120) + assert r["success"], "[{}] mirror: {}".format(pm_name, r["stderr"][:300]) + assert binary_exists("/usr/bin/which"), "[{}] which not installed".format(pm_name) + + +# ── add_repository ─────────────────────────────────────────────────── + +@pytest.mark.parametrize("pm_name,repo_lines,result_file", [ + ("pacman", ['- name: custom', ' server: "https://repo.example.com/$arch"'], + "/etc/pacman.conf"), + ("apt", ['- url: "https://repo.example.com/debian"', ' distro: bookworm'], + "/etc/apt/sources.list.d/custom.list"), + ("dnf", ['- name: custom', ' baseurl: "https://repo.example.com/fedora"'], + "/etc/yum.repos.d/custom.repo"), + ("apk", ['- url: "https://repo.example.com/v3.19/main"'], + "/etc/apk/repositories"), +]) +@pytest.mark.xfail(reason="fake repo URL, package install may fail", strict=False) +def test_add_repo(run_cmd, work_dir, pm_name, repo_lines, result_file): + cfg = os.path.join(work_dir, "pm.yml") + mirror = PACMAN_MIRROR if pm_name == "pacman" else "" + with open(cfg, "w") as f: + f.write(_yml(pm_name, '"curl"', mirror=mirror, repo=repo_lines)) + r = run_cmd(["/baker", "-p", "t", "-b", "1", "-u", "testuser", + "bare", "prebake", cfg], timeout=120) + # Prebake may fail if repo URL is unreachable — check only the config file + # was written by add_repository before the (possibly failing) install step. + assert binary_exists(result_file), \ + "[{}] repo file missing: {}".format(pm_name, result_file) diff --git a/tests/fixtures/pm/Dockerfile.apk b/tests/fixtures/pm/Dockerfile.apk new file mode 100644 index 0000000..4316071 --- /dev/null +++ b/tests/fixtures/pm/Dockerfile.apk @@ -0,0 +1,6 @@ +FROM docker.1ms.run/library/alpine:3.19 +RUN sed -i 's|dl-cdn.alpinelinux.org|mirrors.tuna.tsinghua.edu.cn|g' /etc/apk/repositories +RUN apk add --no-cache python3 py3-pip bash +COPY workshop-baker/target/x86_64-unknown-linux-musl/debug/workshop-baker /baker +RUN chmod +x /baker +ENTRYPOINT ["/bin/sh", "-c"] diff --git a/tests/fixtures/pm/Dockerfile.apt b/tests/fixtures/pm/Dockerfile.apt new file mode 100644 index 0000000..ebd6ef5 --- /dev/null +++ b/tests/fixtures/pm/Dockerfile.apt @@ -0,0 +1,6 @@ +FROM docker.1ms.run/library/debian:bookworm-slim +RUN sed -i 's|deb.debian.org|mirrors.tuna.tsinghua.edu.cn|g' /etc/apt/sources.list.d/debian.sources 2>/dev/null; sed -i 's|security.debian.org|mirrors.tuna.tsinghua.edu.cn/debian-security|g' /etc/apt/sources.list.d/debian.sources 2>/dev/null; true +RUN apt-get update -qq && apt-get install -y -qq --no-install-recommends python3 python3-venv ca-certificates 2>&1 | tail -n 2 +COPY workshop-baker/target/x86_64-unknown-linux-musl/debug/workshop-baker /baker +RUN chmod +x /baker +ENTRYPOINT ["/bin/sh", "-c"] diff --git a/tests/fixtures/pm/Dockerfile.dnf b/tests/fixtures/pm/Dockerfile.dnf new file mode 100644 index 0000000..91cd387 --- /dev/null +++ b/tests/fixtures/pm/Dockerfile.dnf @@ -0,0 +1,6 @@ +FROM docker.1ms.run/library/fedora:43 +RUN sed -e 's|^metalink=|#metalink=|g' -e 's|^#baseurl=http://download.example/pub/fedora/linux|baseurl=https://mirrors.tuna.tsinghua.edu.cn/fedora|g' -i.bak /etc/yum.repos.d/fedora.repo /etc/yum.repos.d/fedora-updates.repo +RUN dnf install -y python3 2>&1 | tail -n 2 +COPY workshop-baker/target/x86_64-unknown-linux-musl/debug/workshop-baker /baker +RUN chmod +x /baker +ENTRYPOINT ["/bin/sh", "-c"] diff --git a/tests/fixtures/pm/Dockerfile.pacman b/tests/fixtures/pm/Dockerfile.pacman new file mode 100644 index 0000000..83eccbc --- /dev/null +++ b/tests/fixtures/pm/Dockerfile.pacman @@ -0,0 +1,2 @@ +FROM docker.1ms.run/library/archlinux:latest +RUN echo 'Server = https://mirrors.tuna.tsinghua.edu.cn/archlinux/$repo/os/$arch' > /etc/pacman.d/mirrorlist diff --git a/tests/probes/__init__.py b/tests/probes/__init__.py index 4d71a54..411f772 100644 --- a/tests/probes/__init__.py +++ b/tests/probes/__init__.py @@ -2,18 +2,24 @@ Environment probes for verifying system state in workshop-baker tests. Each probe is a standalone function that can be called from any test. -Probes return bool or raise AssertionError with descriptive message. +Probes return bool or Optional[int/str]; they do not raise on failure. + +A probe accepts an optional *runner* callable (default ``subprocess.run``) +that decouples it from the execution context (host or Docker). """ from .user import user_exists, user_uid, user_gid, user_in_group, user_shell from .file import ( file_exists, + file_absent, dir_exists, + dir_absent, file_permissions, file_contains, file_owned_by, + file_group, ) -from .package import package_installed, package_version +from .package import binary_exists, binary_on_path, package_installed, package_version from .process import process_running, process_count __all__ = [ @@ -23,12 +29,17 @@ __all__ = [ "user_in_group", "user_shell", "file_exists", + "file_absent", "dir_exists", + "dir_absent", "file_permissions", "file_contains", "file_owned_by", + "file_group", "package_installed", "package_version", + "binary_exists", + "binary_on_path", "process_running", "process_count", ] diff --git a/tests/probes/file.py b/tests/probes/file.py index 7f73fb1..86c4b05 100644 --- a/tests/probes/file.py +++ b/tests/probes/file.py @@ -41,3 +41,13 @@ def file_group(path: str) -> str: """Get the group owner of a file.""" st = os.stat(path) return grp.getgrgid(st.st_gid).gr_name + + +def file_absent(path: str) -> bool: + """Check that *path* does not exist.""" + return not os.path.lexists(path) + + +def dir_absent(path: str) -> bool: + """Check that *path* does not exist or is not a directory.""" + return not os.path.isdir(path) diff --git a/tests/probes/package.py b/tests/probes/package.py index 295fcfa..81b4607 100644 --- a/tests/probes/package.py +++ b/tests/probes/package.py @@ -1,34 +1,42 @@ -"""Package installation probes for Arch Linux (pacman).""" +"""Package / binary existence probes (cross-distro). + +Each probe accepts an optional *runner* callable with the same signature +as ``subprocess.run``. Defaults to running on the host; for Docker tests, +pass a runner that wraps ``docker run --rm ``. +""" import subprocess +from typing import Any, Callable + +Runner = Callable[..., Any] -def package_installed(package_name: str) -> bool: - """Check if a pacman package is installed.""" - try: - result = subprocess.run( - ["pacman", "-Q", package_name], - capture_output=True, - text=True, - ) - return result.returncode == 0 - except FileNotFoundError: - return False +def _run(cmd: list[str], runner: Runner = subprocess.run, **kw: Any) -> Any: + opts = {"capture_output": True, "text": True, "timeout": 30} + opts.update(kw) + return runner(cmd, **opts) -def package_version(package_name: str) -> str | None: - """Get installed version of a package, or ``None`` if not installed.""" - try: - result = subprocess.run( - ["pacman", "-Q", package_name], - capture_output=True, - text=True, - ) - if result.returncode == 0: - # Output format: "package_name version" - parts = result.stdout.strip().split() - if len(parts) >= 2: - return parts[1] - return None - except FileNotFoundError: - return None +def _succeeded(r: Any) -> bool: + """.returncode==0 for CompletedProcess, or ["success"] for dict.""" + if hasattr(r, "returncode"): + return r.returncode == 0 + return bool(r.get("success", False)) + + +def binary_exists(path: str, runner: Runner = subprocess.run) -> bool: + """Check if a binary exists at *path* (e.g. ``/usr/bin/curl``). + + Uses ``test -f`` (POSIX, guaranteed available). + """ + return _succeeded(_run(["test", "-f", path], runner)) + + +def binary_on_path(name: str, runner: Runner = subprocess.run) -> bool: + """Check if a command is on PATH (via ``command -v``).""" + return _succeeded(_run(["sh", "-c", "command -v " + name], runner)) + + +# Backward-compatible aliases +package_installed = binary_exists +package_version = binary_on_path # used in __init__ export, now binary-based diff --git a/tests/probes/user.py b/tests/probes/user.py index 94ade6d..490643b 100644 --- a/tests/probes/user.py +++ b/tests/probes/user.py @@ -13,15 +13,17 @@ def user_exists(username: str) -> bool: return False -def user_uid(username: str) -> int: - """Get the UID of a user. Raises KeyError if user doesn't exist.""" - return pwd.getpwnam(username).pw_uid - - -def user_gid(username: str) -> int: - """Get the primary GID of a user. Raises KeyError if user doesn't exist.""" - return pwd.getpwnam(username).pw_gid +def user_uid(username: str) -> int | None: + try: + return pwd.getpwnam(username).pw_uid + except KeyError: + return None +def user_gid(username: str) -> int | None: + try: + return pwd.getpwnam(username).pw_gid + except KeyError: + return None def user_in_group(username: str, groupname: str) -> bool: """Check if a user is a member of a group.""" @@ -32,6 +34,8 @@ def user_in_group(username: str, groupname: str) -> bool: return False -def user_shell(username: str) -> str: - """Get the login shell of a user. Raises KeyError if user doesn't exist.""" - return pwd.getpwnam(username).pw_shell +def user_shell(username: str) -> str | None: + try: + return pwd.getpwnam(username).pw_shell + except KeyError: + return None diff --git a/workshop-baker/Cargo.lock b/workshop-baker/Cargo.lock index 24173d7..a048bb7 100644 --- a/workshop-baker/Cargo.lock +++ b/workshop-baker/Cargo.lock @@ -691,16 +691,6 @@ dependencies = [ "unicode-segmentation", ] -[[package]] -name = "core-foundation" -version = "0.9.4" -source = "registry+https://github.com/rust-lang/crates.io-index" -checksum = "91e195e091a93c46f7102ec7818a2aa394e1e1771c3ab4825963fa03e45afb8f" -dependencies = [ - "core-foundation-sys", - "libc", -] - [[package]] name = "core-foundation" version = "0.10.1" @@ -1101,21 +1091,6 @@ version = "0.2.0" source = "registry+https://github.com/rust-lang/crates.io-index" checksum = "77ce24cb58228fbb8aa041425bb1050850ac19177686ea6e0f41a70416f56fdb" -[[package]] -name = "foreign-types" -version = "0.3.2" -source = "registry+https://github.com/rust-lang/crates.io-index" -checksum = "f6f339eb8adc052cd2ca78910fda869aefa38d22d5cb648e6485e4d3fc06f3b1" -dependencies = [ - "foreign-types-shared", -] - -[[package]] -name = "foreign-types-shared" -version = "0.1.1" -source = "registry+https://github.com/rust-lang/crates.io-index" -checksum = "00b0228411908ca8685dba7fc2cdd70ec9990a6e753e89b6ac91a84c40fbaf4b" - [[package]] name = "form_urlencoded" version = "1.2.2" @@ -1215,8 +1190,10 @@ source = "registry+https://github.com/rust-lang/crates.io-index" checksum = "335ff9f135e4384c8150d6f27c6daed433577f86b4750418338c01a1a2528592" dependencies = [ "cfg-if", + "js-sys", "libc", "wasi", + "wasm-bindgen", ] [[package]] @@ -1226,9 +1203,11 @@ source = "registry+https://github.com/rust-lang/crates.io-index" checksum = "899def5c37c4fd7b2664648c28120ecec138e4d395b459e5ca34f9cce2dd77fd" dependencies = [ "cfg-if", + "js-sys", "libc", "r-efi", "wasip2", + "wasm-bindgen", ] [[package]] @@ -2285,17 +2264,6 @@ dependencies = [ "windows-sys 0.61.2", ] -[[package]] -name = "hostname" -version = "0.4.2" -source = "registry+https://github.com/rust-lang/crates.io-index" -checksum = "617aaa3557aef3810a6369d0a99fac8a080891b68bd9f9812a1eeda0c0730cbd" -dependencies = [ - "cfg-if", - "libc", - "windows-link", -] - [[package]] name = "http" version = "1.4.0" @@ -2399,6 +2367,7 @@ dependencies = [ "tokio", "tokio-rustls", "tower-service", + "webpki-roots", ] [[package]] @@ -2414,22 +2383,6 @@ dependencies = [ "tower-service", ] -[[package]] -name = "hyper-tls" -version = "0.6.0" -source = "registry+https://github.com/rust-lang/crates.io-index" -checksum = "70206fc6890eaca9fde8a0bf71caa2ddfc9fe045ac9e5c70df101a7dbde866e0" -dependencies = [ - "bytes", - "http-body-util", - "hyper", - "hyper-util", - "native-tls", - "tokio", - "tokio-native-tls", - "tower-service", -] - [[package]] name = "hyper-util" version = "0.1.19" @@ -2449,11 +2402,9 @@ dependencies = [ "percent-encoding", "pin-project-lite", "socket2", - "system-configuration", "tokio", "tower-service", "tracing", - "windows-registry", ] [[package]] @@ -2806,18 +2757,18 @@ dependencies = [ "fastrand", "futures-io", "futures-util", - "hostname", "httpdate", "idna", "mime", - "native-tls", "nom", "percent-encoding", "quoted_printable", + "rustls", "socket2", "tokio", - "tokio-native-tls", + "tokio-rustls", "url", + "webpki-roots", ] [[package]] @@ -2865,6 +2816,12 @@ version = "0.4.28" source = "registry+https://github.com/rust-lang/crates.io-index" checksum = "34080505efa8e45a4b816c349525ebe327ceaa8559756f0356cba97ef3bf7432" +[[package]] +name = "lru-slab" +version = "0.1.2" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "112b39cec0b298b6c1999fee3e31427f74f676e4cb9879ed1a121b43661a4154" + [[package]] name = "matchit" version = "0.8.4" @@ -2942,23 +2899,6 @@ dependencies = [ "windows-sys 0.61.2", ] -[[package]] -name = "native-tls" -version = "0.2.18" -source = "registry+https://github.com/rust-lang/crates.io-index" -checksum = "465500e14ea162429d264d44189adc38b199b62b1c21eea9f69e4b73cb03bbf2" -dependencies = [ - "libc", - "log", - "openssl", - "openssl-probe 0.2.1", - "openssl-sys", - "schannel", - "security-framework", - "security-framework-sys", - "tempfile", -] - [[package]] name = "nix" version = "0.25.1" @@ -3269,56 +3209,12 @@ dependencies = [ "tokio", ] -[[package]] -name = "openssl" -version = "0.10.76" -source = "registry+https://github.com/rust-lang/crates.io-index" -checksum = "951c002c75e16ea2c65b8c7e4d3d51d5530d8dfa7d060b4776828c88cfb18ecf" -dependencies = [ - "bitflags 2.10.0", - "cfg-if", - "foreign-types", - "libc", - "once_cell", - "openssl-macros", - "openssl-sys", -] - -[[package]] -name = "openssl-macros" -version = "0.1.1" -source = "registry+https://github.com/rust-lang/crates.io-index" -checksum = "a948666b637a0f465e8564c73e89d4dde00d72d4d473cc972f390fc3dcee7d9c" -dependencies = [ - "proc-macro2", - "quote", - "syn", -] - [[package]] name = "openssl-probe" version = "0.1.6" source = "registry+https://github.com/rust-lang/crates.io-index" checksum = "d05e27ee213611ffe7d6348b942e8f942b37114c00cc03cec254295a4a17852e" -[[package]] -name = "openssl-probe" -version = "0.2.1" -source = "registry+https://github.com/rust-lang/crates.io-index" -checksum = "7c87def4c32ab89d880effc9e097653c8da5d6ef28e6b539d313baaacfbafcbe" - -[[package]] -name = "openssl-sys" -version = "0.9.112" -source = "registry+https://github.com/rust-lang/crates.io-index" -checksum = "57d55af3b3e226502be1526dfdba67ab0e9c96fc293004e79576b2b9edb0dbdb" -dependencies = [ - "cc", - "libc", - "pkg-config", - "vcpkg", -] - [[package]] name = "ordered-multimap" version = "0.7.3" @@ -3646,6 +3542,61 @@ dependencies = [ "prost", ] +[[package]] +name = "quinn" +version = "0.11.9" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "b9e20a958963c291dc322d98411f541009df2ced7b5a4f2bd52337638cfccf20" +dependencies = [ + "bytes", + "cfg_aliases", + "pin-project-lite", + "quinn-proto", + "quinn-udp", + "rustc-hash", + "rustls", + "socket2", + "thiserror 2.0.18", + "tokio", + "tracing", + "web-time", +] + +[[package]] +name = "quinn-proto" +version = "0.11.14" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "434b42fec591c96ef50e21e886936e66d3cc3f737104fdb9b737c40ffb94c098" +dependencies = [ + "bytes", + "getrandom 0.3.4", + "lru-slab", + "rand", + "ring", + "rustc-hash", + "rustls", + "rustls-pki-types", + "slab", + "thiserror 2.0.18", + "tinyvec", + "tracing", + "web-time", +] + +[[package]] +name = "quinn-udp" +version = "0.5.14" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "addec6a0dcad8a8d96a771f815f0eaf55f9d1805756410b39f5fa81332574cbd" +dependencies = [ + "cfg_aliases", + "libc", + "once_cell", + "socket2", + "tracing", + "windows-sys 0.60.2", +] + [[package]] name = "quote" version = "1.0.42" @@ -3791,29 +3742,26 @@ checksum = "eddd3ca559203180a307f12d114c268abf583f59b03cb906fd0b3ff8646c1147" dependencies = [ "base64", "bytes", - "encoding_rs", "futures-core", - "h2", "http", "http-body", "http-body-util", "hyper", "hyper-rustls", - "hyper-tls", "hyper-util", "js-sys", "log", - "mime", - "native-tls", "percent-encoding", "pin-project-lite", + "quinn", + "rustls", "rustls-pki-types", "serde", "serde_json", "serde_urlencoded", "sync_wrapper", "tokio", - "tokio-native-tls", + "tokio-rustls", "tower", "tower-http", "tower-service", @@ -3821,6 +3769,7 @@ dependencies = [ "wasm-bindgen", "wasm-bindgen-futures", "web-sys", + "webpki-roots", ] [[package]] @@ -3871,6 +3820,12 @@ dependencies = [ "num-traits", ] +[[package]] +name = "rustc-hash" +version = "2.1.2" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "94300abf3f1ae2e2b8ffb7b58043de3d399c73fa6f4b73826402a5c457614dbe" + [[package]] name = "rustix" version = "1.1.4" @@ -3905,7 +3860,7 @@ version = "0.8.1" source = "registry+https://github.com/rust-lang/crates.io-index" checksum = "7fcff2dd52b58a8d98a70243663a0d234c4e2b79235637849d15913394a247d3" dependencies = [ - "openssl-probe 0.1.6", + "openssl-probe", "rustls-pki-types", "schannel", "security-framework", @@ -3917,6 +3872,7 @@ version = "1.13.2" source = "registry+https://github.com/rust-lang/crates.io-index" checksum = "21e6f2ab2928ca4291b86736a8bd920a277a399bba1589409d72154ff87c1282" dependencies = [ + "web-time", "zeroize", ] @@ -3998,7 +3954,7 @@ source = "registry+https://github.com/rust-lang/crates.io-index" checksum = "b3297343eaf830f66ede390ea39da1d462b6b0c1b000f420d0a83f898bbbe6ef" dependencies = [ "bitflags 2.10.0", - "core-foundation 0.10.1", + "core-foundation", "core-foundation-sys", "libc", "security-framework-sys", @@ -4291,27 +4247,6 @@ dependencies = [ "syn", ] -[[package]] -name = "system-configuration" -version = "0.6.1" -source = "registry+https://github.com/rust-lang/crates.io-index" -checksum = "3c879d448e9d986b661742763247d3693ed13609438cf3d006f51f5368a5ba6b" -dependencies = [ - "bitflags 2.10.0", - "core-foundation 0.9.4", - "system-configuration-sys", -] - -[[package]] -name = "system-configuration-sys" -version = "0.6.0" -source = "registry+https://github.com/rust-lang/crates.io-index" -checksum = "8e1d1b10ced5ca923a1fcb8d03e96b8d3268065d724548c0211415ff6ac6bac4" -dependencies = [ - "core-foundation-sys", - "libc", -] - [[package]] name = "tar" version = "0.4.44" @@ -4479,16 +4414,6 @@ dependencies = [ "syn", ] -[[package]] -name = "tokio-native-tls" -version = "0.3.1" -source = "registry+https://github.com/rust-lang/crates.io-index" -checksum = "bbae76ab933c85776efabc971569dd6119c580d8f5d448769dec1764bf796ef2" -dependencies = [ - "native-tls", - "tokio", -] - [[package]] name = "tokio-rustls" version = "0.26.4" @@ -4845,12 +4770,6 @@ dependencies = [ "wasm-bindgen", ] -[[package]] -name = "vcpkg" -version = "0.2.15" -source = "registry+https://github.com/rust-lang/crates.io-index" -checksum = "accd4ea62f7bb7a82fe23066fb0957d48ef677f6eeb8215f372f52e48bb32426" - [[package]] name = "version_check" version = "0.9.5" @@ -4959,6 +4878,16 @@ dependencies = [ "wasm-bindgen", ] +[[package]] +name = "web-time" +version = "1.1.0" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "5a6580f308b1fad9207618087a65c04e7a10bc77e02c8e84e9b00dd4b12fa0bb" +dependencies = [ + "js-sys", + "wasm-bindgen", +] + [[package]] name = "webpki-roots" version = "1.0.4" @@ -5049,17 +4978,6 @@ version = "0.2.1" source = "registry+https://github.com/rust-lang/crates.io-index" checksum = "f0805222e57f7521d6a62e36fa9163bc891acd422f971defe97d64e70d0a4fe5" -[[package]] -name = "windows-registry" -version = "0.6.1" -source = "registry+https://github.com/rust-lang/crates.io-index" -checksum = "02752bf7fbdcce7f2a27a742f798510f3e5ad88dbe84871e5168e2120c3d5720" -dependencies = [ - "windows-link", - "windows-result", - "windows-strings", -] - [[package]] name = "windows-result" version = "0.4.1" diff --git a/workshop-baker/Cargo.toml b/workshop-baker/Cargo.toml index 8cda569..d5a3d04 100644 --- a/workshop-baker/Cargo.toml +++ b/workshop-baker/Cargo.toml @@ -28,7 +28,7 @@ env_logger = "0.11.8" glob = "0.3.2" futures-util = "0.3.31" lazy_static = "1.5.0" -lettre = { version = "0.11", features = ["tokio1-native-tls"] } +lettre = { version = "0.11", default-features = false, features = ["tokio1-rustls", "rustls-tls", "builder", "smtp-transport"] } log = "0.4.28" minijinja = "2.14.0" regex = "1.12.2" @@ -47,7 +47,7 @@ uuid = { version = "1.19.0", features = ["v4"] } libc = "0.2" os_info = "3.14.0" privdrop = "0.5.6" -reqwest = { version = "0.12", features = ["json"] } +reqwest = { version = "0.12", default-features = false, features = ["json", "rustls-tls"] } which = "8.0.2" gix = "0.81.0" sha2 = "0.10" diff --git a/workshop-baker/src/finalize/plugin/fetch/git.rs b/workshop-baker/src/finalize/plugin/fetch/git.rs index 0f12ec1..79cf79a 100644 --- a/workshop-baker/src/finalize/plugin/fetch/git.rs +++ b/workshop-baker/src/finalize/plugin/fetch/git.rs @@ -19,7 +19,7 @@ pub enum GitVersion { /// Format: `url` (no version), `url@tag` (single version), `url@tag:commit` (mixed version). /// Returns (base_url, GitVersion). pub fn parse_git_url(url: &str) -> (String, GitVersion) { - if let Some((base, version)) = url.split_once('@') { + if let Some((base, version)) = url.rsplit_once('@') { if version.contains(":") { let (tag, commit) = version.split_once(':').unwrap(); return ( diff --git a/workshop-baker/src/prebake.rs b/workshop-baker/src/prebake.rs index dcc7ea8..b214ee0 100644 --- a/workshop-baker/src/prebake.rs +++ b/workshop-baker/src/prebake.rs @@ -154,8 +154,9 @@ pub async fn prebake( .map(|c| c.repology_endpoint) .unwrap_or(workshop_engine::RepologyEndpoint::Default); - let osinfo = os_info::get(); - let detected_pm = workshop_engine::pm::detect(&osinfo).map(|pm| pm.name().to_string()); + let pm_name = dependencies.system + .as_ref() + .and_then(|sys| sys.keys().next().cloned()); let upm_sys_deps = if let Some(user) = &dependencies.user { if stage <= PrebakeStage::DepsSystem { @@ -170,7 +171,7 @@ pub async fn prebake( let mut system_config = dependencies.system.clone().unwrap_or_default(); if !upm_sys_deps.deps.is_empty() { - if let Some(pm_name) = &detected_pm { + if let Some(ref pm_name) = pm_name { for (upm_name, sys_deps) in &upm_sys_deps.deps { let resolved = workshop_engine::repology::resolve_package_names( &sys_deps.packages, diff --git a/workshop-baker/src/prebake/error.rs b/workshop-baker/src/prebake/error.rs index 7805ebf..2656f38 100644 --- a/workshop-baker/src/prebake/error.rs +++ b/workshop-baker/src/prebake/error.rs @@ -61,4 +61,7 @@ pub enum BootstrapError { #[error("Failed to create user {0}: {1}")] UserCreationFailed(String, String), -} + + #[error("Invalid bootstrap configuration: {0}")] + InvalidConfig(String), + } diff --git a/workshop-baker/src/prebake/stage/bootstrap.rs b/workshop-baker/src/prebake/stage/bootstrap.rs index 62702cc..104d725 100644 --- a/workshop-baker/src/prebake/stage/bootstrap.rs +++ b/workshop-baker/src/prebake/stage/bootstrap.rs @@ -53,21 +53,48 @@ pub async fn bootstrap( let bootstrap_config = config.bootstrap.clone().unwrap_or_default(); - if let Some(custom_bootstrap) = bootstrap_config.custom { - if ctx.dry_run { - log::info!( - "[DRY_RUN] User provided bootstrap script:\n{}", - custom_bootstrap - ); - return Ok(()); - } - // TODO: Parse custom bootstrap URL and download them - log::debug!("User provided bootstrap script:\n{}", custom_bootstrap); - if ctx.standalone { - write_bootstrap_content(&bootstrap_path, custom_bootstrap)?; - } else { - // Requires PIP(Pipeline Implicit Parameters) to be implemented - unimplemented!("Permission check is not implemented, failing..."); + if let Some(raw) = bootstrap_config.custom { + match BootstrapSource::parse(&raw) { + BootstrapSource::Inline(script) => { + if ctx.dry_run { + log::info!("[DRY_RUN] User provided inline bootstrap script:\n{}", script); + return Ok(()); + } + log::debug!("Using inline bootstrap script ({} bytes)", script.len()); + if ctx.standalone { + write_bootstrap_content(&bootstrap_path, script)?; + } else { + // Requires PIP(Pipeline Implicit Parameters) to be implemented + unimplemented!("Permission check is not implemented, failing..."); + } + } + BootstrapSource::Workspace(relpath) => { + // Project-relative path, resolved by bakerd against the project root. + // Routing path relpath through bakerd's workspace manager ensures + // proper sandboxing and access control. + log::info!("Custom bootstrap from workspace: {}", relpath); + unimplemented!("Workspace-relative bootstrap requires bakerd's workspace manager"); + } + BootstrapSource::Server(name) => { + // Script provided by the bake server (bakerd), keyed by name. + // The server maintains a registry of trusted bootstrap scripts. + log::info!("Custom bootstrap from server: {}", name); + unimplemented!("Server-provided bootstrap requires bakerd's script registry"); + } + BootstrapSource::Http(url) => { + // HTTP download, similar to finalize::plugin::fetch::fetch_http_plugin. + // Should support: checksum verification, retry, timeout. + log::info!("Custom bootstrap from URL: {}", url); + log::info!(" Target: {}", bootstrap_path.display()); + unimplemented!("HTTP bootstrap fetching requires bakerd-level network infrastructure"); + } + BootstrapSource::File(path) => { + // Local file copy. This is the simplest case — we can implement + // this directly without bakerd infrastructure. + log::info!("Custom bootstrap from local file: {}", path); + tokio::fs::copy(&path, &bootstrap_path).await + .map_err(BootstrapError::IOError)?; + } } } else { let script = generate_bootstrap(&bootstrap_config, config, osinfo, ctx.standalone)?; @@ -102,6 +129,52 @@ fn write_bootstrap_content(bootstrap_path: &Path, content: String) -> Result<(), Ok(()) } +/// Source type for a custom bootstrap script. +/// +/// Mirrors the scheme-based resolution in `finalize::plugin::fetch` URL parsing, +/// and supports the schemes defined in the prebake.yml.tmpl spec: +/// `workspace://`, `file://`, `server://`, `https?://`, plus inline content. +enum BootstrapSource { + /// Inline script content (no scheme match). + Inline(String), + /// Project-relative file: `workspace://path/to/bootstrap.sh` + Workspace(String), + /// Script provided by bake server: `server://script-name` + Server(String), + /// HTTP(S) download: `https://example.com/bootstrap.sh` + Http(String), + /// Local file path: `file:///bin/bootstrap.sh` + File(String), +} + +impl BootstrapSource { + /// Parses a raw custom bootstrap string into its source type. + /// + /// Scheme detection follows the template spec: + /// - `workspace://` → Workspace + /// - `file://` → File (local path) + /// - `server://` → Server (bakerd registry) + /// - `http://`/`https://`→ Http + /// - No recognized scheme → Inline + fn parse(raw: &str) -> Self { + if let Some(rest) = raw.strip_prefix("workspace://") { + return BootstrapSource::Workspace(rest.to_string()); + } + if let Some(path) = raw.strip_prefix("file://") { + return BootstrapSource::File(path.to_string()); + } + if let Some(name) = raw.strip_prefix("server://") { + return BootstrapSource::Server(name.to_string()); + } + if let Some(url) = raw.strip_prefix("https://") + .or_else(|| raw.strip_prefix("http://")) + { + return BootstrapSource::Http(url.to_string()); + } + BootstrapSource::Inline(raw.to_string()) + } +} + /// Generates a bootstrap script based on configuration and target OS. /// /// The generated script performs the following setup tasks: @@ -119,7 +192,7 @@ fn write_bootstrap_content(bootstrap_path: &Path, content: String) -> Result<(), /// /// * `bootstrap_config` - Bootstrap-specific configuration (user, workspace, sudoers) /// * `config` - Full prebake configuration for dependency and package manager detection -/// * `osinfo` - Operating system information for package manager detection +/// * `_osinfo` - Operating system information (reserved) /// /// # Returns /// @@ -134,7 +207,7 @@ fn write_bootstrap_content(bootstrap_path: &Path, content: String) -> Result<(), fn generate_bootstrap( bootstrap_config: &Bootstrap, config: &PrebakeConfig, - osinfo: &os_info::Info, + _osinfo: &os_info::Info, standalone: bool, ) -> Result { let mut script = String::new(); @@ -190,7 +263,7 @@ fn generate_bootstrap( // Setup privilege executor script.push_str("# Setup privilege executor\n"); - let pkg_manager = get_package_manager(config, osinfo); + let pkg_manager = get_package_manager(config); if which("sudo").is_ok() { if let Some(sudoers) = &bootstrap_config.sudoers { if standalone { @@ -242,60 +315,32 @@ fn generate_bootstrap( Ok(script) } -/// Detects and selects an appropriate package manager based on configuration and OS. -/// -/// This function first attempts to detect the OS's native package manager, then -/// checks if it's configured in the prebake.yml. If the detected package manager -/// is not configured, it falls back to the first configured package manager. -/// -/// # Detection Priority -/// -/// 1. **Detected Package Manager**: Uses `pm::detect()` to find the OS-native package manager -/// (e.g., `apt` on Debian/Ubuntu, `pacman` on Arch Linux) -/// 2. **Configured Package Manager**: If detected manager is not in config, uses the -/// first configured package manager from the dependencies section +/// Selects the package manager from the first configured system dependency. /// /// # Arguments /// /// * `config` - Prebake configuration containing dependencies.system configuration -/// * `osinfo` - Operating system information for package manager detection /// /// # Returns /// -/// Returns `Some(Box)` if a suitable package manager is found, -/// or `None` if no package manager is configured or detectable. +/// Returns `Some(Box)` if a configured package manager is known, +/// or `None` if no package manager is configured or recognized. fn get_package_manager( config: &PrebakeConfig, - osinfo: &os_info::Info, ) -> Option> { config .dependencies .as_ref() .and_then(|deps| deps.system.as_ref()) .and_then(|sysdeps| { - if let Some(detected) = pm::detect(osinfo) { - if sysdeps.contains_key(detected.name()) { - log::info!("Using package manager: {} (detected)", detected.name()); - return Some(detected); - } else { - log::warn!( - "Detected package manager '{}' but not configured in prebake.yml", - detected.name() - ); - } - } - if let Some((name, _)) = sysdeps.iter().next() && let Some(pm) = pm::select(name) { - log::warn!( - "Using package manager: {} (first configured, not detected)", - name - ); + log::info!("Using package manager: {}", name); return Some(pm); } - log::warn!("No valid package manager found in configuration"); + log::warn!("No known package manager found in dependencies.system"); None }) } diff --git a/workshop-baker/src/prebake/stage/depssystem.rs b/workshop-baker/src/prebake/stage/depssystem.rs index 58d0f22..159adcf 100644 --- a/workshop-baker/src/prebake/stage/depssystem.rs +++ b/workshop-baker/src/prebake/stage/depssystem.rs @@ -6,7 +6,7 @@ use workshop_engine::{Engine, EventSender, error::DependencyError, pm}; /// Handles system dependency installation for the target platform. /// /// This function: -/// - Detects the operating system and package manager +/// - Selects the package manager from the configured dependencies /// - Optionally changes the package manager mirror /// - Optionally adds custom repositories /// - Updates the package index @@ -19,21 +19,21 @@ pub async fn depssystem( let osinfo = os_info::get(); let engine = Engine::new(); - // Detect package manager - let package_manager = pm::detect(&osinfo).ok_or_else(|| { - log::error!("No supported package manager found for {}", &osinfo); - DependencyError::UnsupportedPlatform(osinfo.clone()) - })?; - log::info!("Detected package manager: {}", package_manager.name()); + // Select package manager from the first configured system dependency + let package_manager = config + .keys() + .find_map(|name| pm::select(name)) + .ok_or_else(|| { + log::error!( + "No known package manager configured in dependencies.system for {}", + &osinfo + ); + DependencyError::UnsupportedPlatform(osinfo.clone()) + })?; + log::info!("Using package manager: {}", package_manager.name()); // Retrieve configuration - let Some(pm_config) = config.get(package_manager.name()) else { - log::info!( - "No configuration found for {}, skipping system dependencies", - package_manager.name() - ); - return Ok(()); - }; + let pm_config = &config[package_manager.name()]; // Change mirror if let Some(mirror) = &pm_config.mirror { diff --git a/workshop-baker/src/types/buildstatus.rs b/workshop-baker/src/types/buildstatus.rs index 527bd1d..b3e5949 100644 --- a/workshop-baker/src/types/buildstatus.rs +++ b/workshop-baker/src/types/buildstatus.rs @@ -88,13 +88,13 @@ impl StageInfo { /// /// # Example /// ``` - /// # use workshop_baker::types::buildstatus::{StageInfo, StagePhase, StageResult}; + /// # use workshop_baker::types::buildstatus::{StageInfo, StagePhase, StageOutcome}; /// # use chrono::Utc; /// let info = StageInfo { /// name: "bootstrap".to_string(), /// phase: StagePhase::Prebake, /// substage: "bootstrap".to_string(), - /// result: StageResult::Success, + /// result: StageOutcome::Success, /// duration_ms: 1000, /// started_at: Utc::now(), /// finished_at: Utc::now(), diff --git a/workshop-engine/src/pm.rs b/workshop-engine/src/pm.rs index 5862e35..dd0f746 100644 --- a/workshop-engine/src/pm.rs +++ b/workshop-engine/src/pm.rs @@ -1,8 +1,9 @@ use os_info::Info; use serde_yaml::Value; use tokio::process::Command; -// TODO: apt -// mod apt; +mod apk; +mod apt; +mod dnf; mod pacman; /// Trait for package manager implementations. @@ -13,8 +14,6 @@ pub trait PackageManager { /// Returns the name of the package manager. fn name(&self) -> &'static str; - /// Determines whether this package manager supports the given distribution. - fn adopt(&self, distro: &Info) -> bool; /// Returns the binary name of the package manager. fn binary(&self) -> &'static str; @@ -25,12 +24,6 @@ pub trait PackageManager { /// Generates commands to install the specified packages. fn install(&self, package: &[String]) -> Vec; - // Check whether a package is installed - // Currently not planned - // Package manager should handle installed packages themselves - // just as --needed in pacman - // Also the PM module should generate command, not execute them - // fn is_installed(&self, _package: &str) -> Option {unimplemented!()} /// Generates commands to change the primary mirror repository. fn change_mirror(&self, repo: &str, osinfo: Option<&Info>) -> Vec; @@ -42,22 +35,15 @@ pub trait PackageManager { /// Returns a list of all available package managers. pub fn all_managers() -> Vec> { vec![ - // Box::new(apt::Apt), + Box::new(apk::Apk), + Box::new(apt::Apt), + Box::new(dnf::Dnf), Box::new(pacman::Pacman), - // Box::new(dnf::Dnf), ] } -/// Detects the appropriate package manager for the given distribution. -pub fn detect(distro: &os_info::Info) -> Option> { - all_managers().into_iter().find(|pm| pm.adopt(distro)) -} -/// Selects a package manager by name. pub fn select(manager: &str) -> Option> { - // TODO: allow specify PM - // log::warn!("Manually select package manager is not recommended."); - // log::warn!("Use at your own risk."); all_managers().into_iter().find(|pm| pm.name() == manager) } @@ -72,15 +58,60 @@ mod tests { } #[test] - fn test_select_known_manager() { + fn test_select_pacman() { let pm = select("pacman"); assert!(pm.is_some()); assert_eq!(pm.unwrap().name(), "pacman"); } + #[test] + fn test_select_apt() { + let pm = select("apt"); + assert!(pm.is_some()); + assert_eq!(pm.unwrap().name(), "apt"); + } + #[test] fn test_select_unknown_manager() { let pm = select("nonexistent_pm"); assert!(pm.is_none()); } + + #[test] + fn test_binary_pacman() { + let pm = select("pacman").unwrap(); + assert_eq!(pm.binary(), "pacman"); + } + + #[test] + fn test_binary_apt() { + let pm = select("apt").unwrap(); + assert_eq!(pm.binary(), "apt-get"); + } + + #[test] + fn test_select_dnf() { + let pm = select("dnf"); + assert!(pm.is_some()); + assert_eq!(pm.unwrap().name(), "dnf"); + } + + #[test] + fn test_select_apk() { + let pm = select("apk"); + assert!(pm.is_some()); + assert_eq!(pm.unwrap().name(), "apk"); + } + + #[test] + fn test_binary_dnf() { + let pm = select("dnf").unwrap(); + assert_eq!(pm.binary(), "dnf"); + } + + #[test] + fn test_binary_apk() { + let pm = select("apk").unwrap(); + assert_eq!(pm.binary(), "apk"); + } } diff --git a/workshop-engine/src/pm/apk.rs b/workshop-engine/src/pm/apk.rs new file mode 100644 index 0000000..7a5d2c4 --- /dev/null +++ b/workshop-engine/src/pm/apk.rs @@ -0,0 +1,272 @@ +use crate::pm::PackageManager; +use os_info::Info; +use serde::Deserialize; +use serde_yaml::Value; +use tokio::process::Command; + +#[derive(Clone)] +pub struct Apk; + +/// Repository configuration for apk (Alpine Linux). +#[derive(Deserialize, Clone)] +struct ApkRepo { + /// Full repository URL (e.g., `https://dl-cdn.alpinelinux.org/alpine/v3.19/main`). + url: String, + /// Whether this is a community repo (adds @community tag, optional). + #[serde(default)] + community: bool, + /// GPG key URL to download and trust (optional). + #[serde(default)] + key_url: Option, +} + +impl PackageManager for Apk { + fn name(&self) -> &'static str { + "apk" + } + + fn binary(&self) -> &'static str { + "apk" + } + + fn update(&self) -> Vec { + let mut cmd = Command::new("apk"); + cmd.arg("update"); + vec![cmd] + } + + fn install(&self, package: &[String]) -> Vec { + let mut cmd = Command::new("apk"); + cmd.arg("add"); + cmd.args(package); + vec![cmd] + } + + /// Changes the primary apk mirror by replacing /etc/apk/repositories. + /// + /// Alpine's repository config is a simple line-per-mirror file. + /// This function replaces the entire file with the specified mirror URL. + fn change_mirror(&self, repo: &str, _osinfo: Option<&Info>) -> Vec { + let script = format!( + "cat > /etc/apk/repositories << 'APK_EOF'\n{}\nAPK_EOF", + repo + ); + let mut cmd = Command::new("sh"); + cmd.arg("-c").arg(script); + vec![cmd] + } + + /// Adds a custom apk repository by appending a line to /etc/apk/repositories + /// and optionally installing a GPG key. + fn add_repository(&self, repo: &Value, _osinfo: Option<&Info>) -> Vec { + let repos: Vec = match serde_yaml::from_value(repo.clone()) { + Ok(r) => r, + Err(e) => { + log::error!("Failed to parse apk repository: {}", e); + return Vec::new(); + } + }; + + let mut cmds: Vec = vec![]; + + for repo_cfg in &repos { + // Download and trust GPG key if provided + if let Some(key_url) = &repo_cfg.key_url { + let mut cmd = Command::new("sh"); + cmd.arg("-c").arg(format!( + "wget -qO- '{}' | apk keys --stdin", + key_url + )); + cmds.push(cmd); + } + + // Append the repository URL to /etc/apk/repositories + let line = if repo_cfg.community { + format!("{}@community\n", repo_cfg.url) + } else { + format!("{}\n", repo_cfg.url) + }; + let mut cmd = Command::new("sh"); + cmd.arg("-c").arg(format!( + "echo '{}' >> /etc/apk/repositories", + line.trim() + )); + cmds.push(cmd); + } + + cmds + } +} + +#[cfg(test)] +mod tests { + use super::*; + + #[test] + fn test_name() { + assert_eq!(Apk.name(), "apk"); + } + + #[test] + fn test_binary() { + assert_eq!(Apk.binary(), "apk"); + } + + #[test] + fn test_update_command_contents() { + let cmds = Apk.update(); + let cmd = cmds[0].as_std(); + assert_eq!(cmd.get_program().to_string_lossy(), "apk"); + let args: Vec = cmd + .get_args() + .map(|a| a.to_string_lossy().into_owned()) + .collect(); + assert_eq!(args, vec!["update"]); + } + + #[test] + fn test_install_command_contents() { + let cmds = Apk.install(&["curl".to_string()]); + let cmd = cmds[0].as_std(); + assert_eq!(cmd.get_program().to_string_lossy(), "apk"); + let args: Vec = cmd + .get_args() + .map(|a| a.to_string_lossy().into_owned()) + .collect(); + assert_eq!(args, vec!["add", "curl"]); + } + + #[test] + fn test_install_multiple_packages() { + let cmds = Apk.install(&["curl".to_string(), "git".to_string()]); + let cmd = cmds[0].as_std(); + let args: Vec = cmd + .get_args() + .map(|a| a.to_string_lossy().into_owned()) + .collect(); + assert_eq!(args, vec!["add", "curl", "git"]); + } + + #[test] + fn test_change_mirror_command() { + let cmds = Apk.change_mirror( + "https://mirrors.tuna.tsinghua.edu.cn/alpine/v3.19/main", + None, + ); + let cmd = cmds[0].as_std(); + assert_eq!(cmd.get_program().to_string_lossy(), "sh"); + let args: Vec = cmd + .get_args() + .map(|a| a.to_string_lossy().into_owned()) + .collect(); + let full_arg = args.join(" "); + assert!(full_arg.contains("cat > /etc/apk/repositories")); + assert!(full_arg.contains("mirrors.tuna.tsinghua.edu.cn")); + } + + #[test] + fn test_add_repository_simple() { + let repo_yaml = serde_yaml::from_value(serde_yaml::Value::Sequence(vec![ + serde_yaml::Value::Mapping( + [( + serde_yaml::Value::String("url".to_string()), + serde_yaml::Value::String( + "https://dl-cdn.alpinelinux.org/alpine/v3.19/main".to_string(), + ), + )] + .into_iter() + .collect(), + ), + ])) + .unwrap(); + + let cmds = Apk.add_repository(&repo_yaml, None); + assert!(!cmds.is_empty()); + let last_cmd = cmds.last().unwrap().as_std(); + let args: Vec = last_cmd + .get_args() + .map(|a| a.to_string_lossy().into_owned()) + .collect(); + let full_arg = args.join(" "); + assert!(full_arg.contains("echo")); + assert!(full_arg.contains(">> /etc/apk/repositories")); + assert!(full_arg.contains("dl-cdn.alpinelinux.org")); + } + + #[test] + fn test_add_repository_with_key() { + let repo_yaml = serde_yaml::from_value(serde_yaml::Value::Sequence(vec![ + serde_yaml::Value::Mapping( + [ + ( + serde_yaml::Value::String("url".to_string()), + serde_yaml::Value::String( + "https://dl-cdn.alpinelinux.org/alpine/v3.19/main".to_string(), + ), + ), + ( + serde_yaml::Value::String("key_url".to_string()), + serde_yaml::Value::String( + "https://example.com/alpine-key".to_string(), + ), + ), + ] + .into_iter() + .collect(), + ), + ])) + .unwrap(); + + let cmds = Apk.add_repository(&repo_yaml, None); + // Should have 2 commands: key download, repo append + assert_eq!(cmds.len(), 2); + let key_cmd = cmds[0].as_std(); + let args: Vec = key_cmd + .get_args() + .map(|a| a.to_string_lossy().into_owned()) + .collect(); + let full_arg = args.join(" "); + assert!(full_arg.contains("wget")); + assert!(full_arg.contains("apk keys")); + } + + #[test] + fn test_add_repository_community() { + let repo_yaml = serde_yaml::from_value(serde_yaml::Value::Sequence(vec![ + serde_yaml::Value::Mapping( + [ + ( + serde_yaml::Value::String("url".to_string()), + serde_yaml::Value::String( + "https://dl-cdn.alpinelinux.org/alpine/v3.19/community" + .to_string(), + ), + ), + ( + serde_yaml::Value::String("community".to_string()), + serde_yaml::Value::Bool(true), + ), + ] + .into_iter() + .collect(), + ), + ])) + .unwrap(); + + let cmds = Apk.add_repository(&repo_yaml, None); + let last_cmd = cmds.last().unwrap().as_std(); + let args: Vec = last_cmd + .get_args() + .map(|a| a.to_string_lossy().into_owned()) + .collect(); + let full_arg = args.join(" "); + assert!(full_arg.contains("@community")); + } + + #[test] + fn test_add_repository_invalid_yaml_returns_empty() { + let invalid_yaml = serde_yaml::Value::String("invalid".to_string()); + let cmds = Apk.add_repository(&invalid_yaml, None); + assert!(cmds.is_empty()); + } +} diff --git a/workshop-engine/src/pm/apt.rs b/workshop-engine/src/pm/apt.rs index a84a9d9..02208ac 100644 --- a/workshop-engine/src/pm/apt.rs +++ b/workshop-engine/src/pm/apt.rs @@ -1,15 +1,48 @@ use crate::pm::PackageManager; use os_info::{Info, Version}; +use serde::Deserialize; use serde_yaml::Value; use tokio::process::Command; #[derive(Clone)] pub struct Apt; -#[derive(Clone)] -struct AptRepositories { +/// Repository configuration for apt. +#[derive(Deserialize, Clone)] +struct AptRepo { + /// The deb line URL (e.g., `https://repo.example.com/ubuntu`) url: String, - key: Option, + /// Distribution codename override (e.g., `focal`, `bookworm`). + /// If not set, extracted from os_info at runtime. + #[serde(default)] + distro: Option, + /// Components (e.g., `main`, `universe`, `contrib`). + /// Defaults to `["main"]` if not specified. + #[serde(default = "default_components")] + components: Vec, + /// GPG key URL to download and install. + #[serde(default)] + key_url: Option, + /// GPG key ID to fetch from keyserver. + #[serde(default)] + key_id: Option, + /// Path to save the GPG keyring file. + #[serde(default = "default_keyring_dir")] + keyring_path: Option, + /// Architecture constraint (e.g., `amd64`, `arm64`). + #[serde(default)] + arch: Option, + /// Whether this is a source repo (`deb-src`). + #[serde(default)] + source: bool, +} + +fn default_components() -> Vec { + vec!["main".to_string()] +} + +fn default_keyring_dir() -> Option { + Some("/usr/share/keyrings".to_string()) } impl PackageManager for Apt { @@ -17,49 +50,200 @@ impl PackageManager for Apt { "apt" } - fn adopt(&self, distro: &Info) -> bool { - return false; - // todo!(); - } - fn binary(&self) -> &'static str { - "apt" + "apt-get" } - fn update(&self) -> Command { - let mut cmd = Command::new("apt"); + fn update(&self) -> Vec { + let mut cmd = Command::new("apt-get"); cmd.arg("update"); - cmd + vec![cmd] } - fn install(&self, package: &[String]) -> Command { - let mut cmd = Command::new("apt"); + fn install(&self, package: &[String]) -> Vec { + let mut cmd = Command::new("apt-get"); cmd.arg("install"); - cmd.arg("-y"); // 自动确认 + cmd.arg("-y"); cmd.args(package); - cmd + vec![cmd] } - fn change_mirror(&self, repo: &str, osinfo: Option<&Info>) -> Command { - - todo!(); + /// Changes the primary apt mirror by replacing the URL in /etc/apt/sources.list. + /// + /// Uses sed to replace the existing mirror URL with the user-specified repo URL. + fn change_mirror(&self, repo: &str, _osinfo: Option<&Info>) -> Vec { + let script = format!( + "sed -i 's|^deb http[^ ]*|deb {}|' /etc/apt/sources.list", + repo + ); + let mut cmd = Command::new("sh"); + cmd.arg("-c").arg(script); + vec![cmd] } + /// Adds a custom apt repository by creating a .list file in /etc/apt/sources.list.d/ + /// and optionally installing a GPG key. fn add_repository(&self, repo: &Value, osinfo: Option<&Info>) -> Vec { + let repos: Vec = match serde_yaml::from_value(repo.clone()) { + Ok(r) => r, + Err(e) => { + log::error!("Failed to parse apt repository: {}", e); + return Vec::new(); + } + }; - todo!(); + let codename = repos + .first() + .and_then(|r| r.distro.clone()) + .or_else(|| extract_codename(osinfo)); + + let codename = match codename { + Some(c) => c, + None => { + log::error!( + "Cannot determine distribution codename for apt repository, \ + set 'distro' field in config or provide os_info" + ); + return Vec::new(); + } + }; + + let mut cmds: Vec = vec![]; + + for repo_cfg in &repos { + // Install GPG key if configured + if let Some(key_url) = &repo_cfg.key_url { + let keyring = repo_cfg + .keyring_path + .clone() + .unwrap_or_else(|| "/usr/share/keyrings".to_string()); + + let keyring_name = format!("{}-keyring.gpg", sanitize_repo_name(&repo_cfg.url)); + let keyring_path = format!("{}/{}", keyring, keyring_name); + + let mut cmd = Command::new("sh"); + cmd.arg("-c").arg(format!( + "curl -fsSL '{}' | gpg --dearmor -o '{}' && chmod 644 '{}'", + key_url, keyring_path, keyring_path + )); + cmds.push(cmd); + } + + if let Some(key_id) = &repo_cfg.key_id { + let mut cmd = Command::new("sh"); + cmd.arg("-c").arg(format!( + "gpg --keyserver keyserver.ubuntu.com --recv-keys {} && \ + gpg --export {} | gpg --dearmor > /usr/share/keyrings/{}-keyring.gpg", + key_id, + key_id, + sanitize_repo_name(&repo_cfg.url) + )); + cmds.push(cmd); + } + + // Build the deb line + let arch_part = repo_cfg + .arch + .as_ref() + .map(|a| format!("[arch={}] ", a)) + .unwrap_or_default(); + + let keyring_name = if repo_cfg.key_url.is_some() || repo_cfg.key_id.is_some() { + format!( + " [signed-by=/usr/share/keyrings/{}-keyring.gpg]", + sanitize_repo_name(&repo_cfg.url) + ) + } else { + String::new() + }; + + let deb_type = if repo_cfg.source { "deb-src" } else { "deb" }; + let components = repo_cfg.components.join(" "); + let deb_line = format!( + "{deb_type} {arch_part}{keyring_name} {url} {codename} {components}\n", + deb_type = deb_type, + arch_part = arch_part, + keyring_name = keyring_name, + url = repo_cfg.url, + codename = codename, + components = components, + ); + + // Write the .list file + let list_name = format!( + "/etc/apt/sources.list.d/{}.list", + sanitize_repo_name(&repo_cfg.url) + ); + let mut cmd = Command::new("sh"); + cmd.arg("-c").arg(format!( + "cat > '{}' << 'APT_EOF'\n{}APT_EOF", + list_name, deb_line + )); + cmds.push(cmd); + } + + cmds } } +/// Sanitizes a URL to a safe filename component. +fn sanitize_repo_name(url: &str) -> String { + url.trim_start_matches("https://") + .trim_start_matches("http://") + .trim_start_matches("ppa:") + .replace(|c: char| !c.is_alphanumeric() && c != '.' && c != '-', "_") + .trim_matches('_') + .to_string() +} + +/// Extracts the distribution codename from os_info, falling back to version-based heuristics. +fn extract_codename(info: Option<&Info>) -> Option { + let info = info?; + + // 1. Direct codename from os_info (supported in newer versions) + if let Some(codename) = info.codename() + && !codename.is_empty() + { + return Some(codename.to_string()); + } + + // 2. Try to infer from version string + match info.version() { + Version::Custom(v) + if !v.chars().next().map(|c| c.is_numeric()).unwrap_or(true) + => { + // Some systems put codename in Custom field + return Some(v.clone()); + } + Version::Semantic(major, minor, _) => { + // For Ubuntu/Debian, fall back to version number + return Some(format!("{}.{}", major, minor)); + } + _ => {} + } + + None +} + #[cfg(test)] mod tests { use super::*; + #[test] + fn test_name() { + assert_eq!(Apt.name(), "apt"); + } + + #[test] + fn test_binary() { + assert_eq!(Apt.binary(), "apt-get"); + } + #[test] fn test_update_command_contents() { - let cmd = Apt.update(); - let cmd = cmd.as_std(); - assert_eq!(cmd.get_program().to_string_lossy(), "apt"); + let cmds = Apt.update(); + let cmd = cmds[0].as_std(); + assert_eq!(cmd.get_program().to_string_lossy(), "apt-get"); let args: Vec = cmd .get_args() .map(|a| a.to_string_lossy().into_owned()) @@ -69,47 +253,165 @@ mod tests { #[test] fn test_install_command_contents() { - let cmd = Apt.install(&["curl".to_string()]); - let cmd = cmd.as_std(); - assert_eq!(cmd.get_program().to_string_lossy(), "apt"); + let cmds = Apt.install(&["curl".to_string()]); + let cmd = cmds[0].as_std(); + assert_eq!(cmd.get_program().to_string_lossy(), "apt-get"); let args: Vec = cmd .get_args() .map(|a| a.to_string_lossy().into_owned()) .collect(); assert_eq!(args, vec!["install", "-y", "curl"]); } -} -/// 辅助函数：从 os_info 中提取 Codename -fn extract_codename(info: Option<&Info>) -> Option { - let info = info?; - - // 1. 直接获取 codename (os_info 较新版本支持) - if let Some(codename) = info.codename() { - if !codename.is_empty() { - return Some(codename.to_string()); - } + #[test] + fn test_install_multiple_packages() { + let cmds = Apt.install(&["curl".to_string(), "git".to_string()]); + let cmd = cmds[0].as_std(); + let args: Vec = cmd + .get_args() + .map(|a| a.to_string_lossy().into_owned()) + .collect(); + assert_eq!(args, vec!["install", "-y", "curl", "git"]); } - // 2. 尝试从 version 中推断 (针对某些把 codename 放在 Custom 里的情况) - match info.version() { - Version::Custom(v) => { - // 有些系统可能把 codename 放在这里，虽然不标准 - if !v.chars().next().map(|c| c.is_numeric()).unwrap_or(true) { - return Some(v.clone()); - } - } - Version::Semantic(major, minor, _) => { - // 对于 Ubuntu/Debian，如果没有 codename，有时可以用版本号兜底 (如 22.04) - // 但 apt 源通常更喜欢 codename。如果实在没有，返回版本号字符串作为最后手段 - return Some(format!("{}.{}", major, minor)); - } - _ => {} + #[test] + fn test_change_mirror_command() { + let cmds = Apt.change_mirror("http://mirror.tuna.tsinghua.edu.cn/ubuntu", None); + let cmd = cmds[0].as_std(); + assert_eq!(cmd.get_program().to_string_lossy(), "sh"); + let args: Vec = cmd + .get_args() + .map(|a| a.to_string_lossy().into_owned()) + .collect(); + let full_arg = args.join(" "); + assert!(full_arg.contains("sed -i")); + assert!(full_arg.contains("/etc/apt/sources.list")); + assert!(full_arg.contains("mirror.tuna.tsinghua.edu.cn")); } - // 3. 如果 os_info 没提供，且是常见发行版，可以尝试根据 Type 猜测？ - // 不推荐硬编码猜测，因为版本太多。 - // 最好返回 None，让调用者决定是报错还是让用户手动指定。 + #[test] + fn test_add_repository_simple_deb_line() { + let repo_yaml = serde_yaml::from_value(serde_yaml::Value::Sequence(vec![ + serde_yaml::Value::Mapping( + [ + ( + serde_yaml::Value::String("url".to_string()), + serde_yaml::Value::String( + "https://repo.example.com/ubuntu".to_string(), + ), + ), + ( + serde_yaml::Value::String("distro".to_string()), + serde_yaml::Value::String("focal".to_string()), + ), + ( + serde_yaml::Value::String("components".to_string()), + serde_yaml::Value::Sequence(vec![ + serde_yaml::Value::String("main".to_string()), + serde_yaml::Value::String("universe".to_string()), + ]), + ), + ] + .into_iter() + .collect(), + ), + ])) + .unwrap(); - None + let cmds = Apt.add_repository(&repo_yaml, None); + assert!(!cmds.is_empty()); + let last_cmd = cmds.last().unwrap().as_std(); + let args: Vec = last_cmd + .get_args() + .map(|a| a.to_string_lossy().into_owned()) + .collect(); + let full_arg = args.join(" "); + assert!(full_arg.contains("cat >")); + assert!(full_arg.contains("sources.list.d")); + assert!(full_arg.contains("repo.example.com")); + assert!(full_arg.contains("focal")); + assert!(full_arg.contains("main universe")); + } + + #[test] + fn test_add_repository_with_key_url() { + let repo_yaml = serde_yaml::from_value(serde_yaml::Value::Sequence(vec![ + serde_yaml::Value::Mapping( + [ + ( + serde_yaml::Value::String("url".to_string()), + serde_yaml::Value::String( + "https://repo.example.com/ubuntu".to_string(), + ), + ), + ( + serde_yaml::Value::String("distro".to_string()), + serde_yaml::Value::String("focal".to_string()), + ), + ( + serde_yaml::Value::String("key_url".to_string()), + serde_yaml::Value::String( + "https://repo.example.com/key.gpg".to_string(), + ), + ), + ] + .into_iter() + .collect(), + ), + ])) + .unwrap(); + + let cmds = Apt.add_repository(&repo_yaml, None); + // First command: key download, last command: list file + assert!(cmds.len() >= 2); + let first_cmd = cmds[0].as_std(); + let args: Vec = first_cmd + .get_args() + .map(|a| a.to_string_lossy().into_owned()) + .collect(); + let full_arg = args.join(" "); + assert!(full_arg.contains("curl")); + assert!(full_arg.contains("key.gpg")); + assert!(full_arg.contains("gpg --dearmor")); + } + + #[test] + fn test_sanitize_repo_name() { + assert_eq!(sanitize_repo_name("https://example.com/repo"), "example.com_repo"); + assert_eq!(sanitize_repo_name("http://mirror.tuna.tsinghua.edu.cn/ubuntu"), "mirror.tuna.tsinghua.edu.cn_ubuntu"); + assert_eq!(sanitize_repo_name("ppa:user/name"), "user_name"); + } + + #[test] + fn test_extract_codename_from_codename_field() { + let info = Info::with_type(os_info::Type::Ubuntu); + assert!(extract_codename(Some(&info)).is_none()); + } + + #[test] + fn test_add_repository_invalid_yaml_returns_empty() { + let invalid_yaml = serde_yaml::Value::String("invalid".to_string()); + let cmds = Apt.add_repository(&invalid_yaml, None); + assert!(cmds.is_empty()); + } + + #[test] + fn test_add_repository_missing_codename_returns_empty() { + let repo_yaml = serde_yaml::from_value(serde_yaml::Value::Sequence(vec![ + serde_yaml::Value::Mapping( + [( + serde_yaml::Value::String("url".to_string()), + serde_yaml::Value::String( + "https://repo.example.com/ubuntu".to_string(), + ), + )] + .into_iter() + .collect(), + ), + ])) + .unwrap(); + + let cmds = Apt.add_repository(&repo_yaml, None); + assert!(cmds.is_empty()); + } } diff --git a/workshop-engine/src/pm/dnf.rs b/workshop-engine/src/pm/dnf.rs new file mode 100644 index 0000000..18f5045 --- /dev/null +++ b/workshop-engine/src/pm/dnf.rs @@ -0,0 +1,275 @@ +use crate::pm::PackageManager; +use os_info::Info; +use serde::Deserialize; +use serde_yaml::Value; +use tokio::process::Command; + +#[derive(Clone)] +pub struct Dnf; + +/// Repository configuration for dnf. +#[derive(Deserialize, Clone)] +struct DnfRepo { + /// Repository name (used as section header and filename). + name: String, + /// Base URL for packages. + baseurl: String, + /// Whether the repository is enabled (default: true). + #[serde(default = "default_enabled")] + enabled: bool, + /// GPG key URL. + #[serde(default)] + gpgkey: Option, + /// Additional metadata (name, gpgcheck, etc.). + #[serde(default)] + description: Option, +} + +fn default_enabled() -> bool { + true +} + +impl PackageManager for Dnf { + fn name(&self) -> &'static str { + "dnf" + } + + fn binary(&self) -> &'static str { + "dnf" + } + + fn update(&self) -> Vec { + let mut cmd = Command::new("dnf"); + cmd.arg("makecache"); + vec![cmd] + } + + fn install(&self, package: &[String]) -> Vec { + let mut cmd = Command::new("dnf"); + cmd.arg("install"); + cmd.arg("-y"); + cmd.args(package); + vec![cmd] + } + + /// Changes the primary dnf mirror by replacing baseurl in the main repo file. + /// + /// Uses sed to replace `baseurl=` lines in `/etc/yum.repos.d/` with the + /// user-specified mirror URL. + fn change_mirror(&self, repo: &str, _osinfo: Option<&Info>) -> Vec { + let script = format!( + "sed -i 's|^baseurl=[^ ].*|baseurl={}|' /etc/yum.repos.d/*.repo", + repo + ); + let mut cmd = Command::new("sh"); + cmd.arg("-c").arg(script); + vec![cmd] + } + + /// Adds a custom dnf repository by creating a .repo file in /etc/yum.repos.d/. + fn add_repository(&self, repo: &Value, _osinfo: Option<&Info>) -> Vec { + let repos: Vec = match serde_yaml::from_value(repo.clone()) { + Ok(r) => r, + Err(e) => { + log::error!("Failed to parse dnf repository: {}", e); + return Vec::new(); + } + }; + + let mut cmds: Vec = vec![]; + + for repo_cfg in &repos { + // Build the .repo file content + let mut content = format!( + "[{}]\nname={}\nbaseurl={}\nenabled={}\n", + repo_cfg.name, + repo_cfg + .description + .clone() + .unwrap_or_else(|| repo_cfg.name.clone()), + repo_cfg.baseurl, + if repo_cfg.enabled { "1" } else { "0" }, + ); + + if let Some(gpgkey) = &repo_cfg.gpgkey { + content.push_str(&format!("gpgcheck=1\ngpgkey={}\n", gpgkey)); + + // Download and import the GPG key + let mut key_cmd = Command::new("sh"); + key_cmd.arg("-c").arg(format!( + "rpm --import '{}'", + gpgkey + )); + cmds.push(key_cmd); + } else { + content.push_str("gpgcheck=0\n"); + } + + // Write the .repo file + let repo_path = format!("/etc/yum.repos.d/{}.repo", sanitize_repo_name(&repo_cfg.name)); + let mut cmd = Command::new("sh"); + cmd.arg("-c").arg(format!( + "cat > '{}' << 'DNF_EOF'\n{}DNF_EOF", + repo_path, content + )); + cmds.push(cmd); + } + + cmds + } +} + +/// Sanitizes a string to a safe filename component. +fn sanitize_repo_name(name: &str) -> String { + name.replace(|c: char| !c.is_alphanumeric() && c != '-' && c != '_', "_") +} + +#[cfg(test)] +mod tests { + use super::*; + + #[test] + fn test_name() { + assert_eq!(Dnf.name(), "dnf"); + } + + #[test] + fn test_binary() { + assert_eq!(Dnf.binary(), "dnf"); + } + + #[test] + fn test_update_command_contents() { + let cmds = Dnf.update(); + let cmd = cmds[0].as_std(); + assert_eq!(cmd.get_program().to_string_lossy(), "dnf"); + let args: Vec = cmd + .get_args() + .map(|a| a.to_string_lossy().into_owned()) + .collect(); + assert_eq!(args, vec!["makecache"]); + } + + #[test] + fn test_install_command_contents() { + let cmds = Dnf.install(&["curl".to_string()]); + let cmd = cmds[0].as_std(); + assert_eq!(cmd.get_program().to_string_lossy(), "dnf"); + let args: Vec = cmd + .get_args() + .map(|a| a.to_string_lossy().into_owned()) + .collect(); + assert_eq!(args, vec!["install", "-y", "curl"]); + } + + #[test] + fn test_install_multiple_packages() { + let cmds = Dnf.install(&["curl".to_string(), "git".to_string()]); + let cmd = cmds[0].as_std(); + let args: Vec = cmd + .get_args() + .map(|a| a.to_string_lossy().into_owned()) + .collect(); + assert_eq!(args, vec!["install", "-y", "curl", "git"]); + } + + #[test] + fn test_change_mirror_command() { + let cmds = Dnf.change_mirror("http://mirror.tuna.tsinghua.edu.cn/fedora", None); + let cmd = cmds[0].as_std(); + assert_eq!(cmd.get_program().to_string_lossy(), "sh"); + let args: Vec = cmd + .get_args() + .map(|a| a.to_string_lossy().into_owned()) + .collect(); + let full_arg = args.join(" "); + assert!(full_arg.contains("sed -i")); + assert!(full_arg.contains("baseurl=")); + assert!(full_arg.contains("yum.repos.d")); + assert!(full_arg.contains("mirror.tuna.tsinghua.edu.cn")); + } + + #[test] + fn test_add_repository_simple() { + let repo_yaml = serde_yaml::from_value(serde_yaml::Value::Sequence(vec![ + serde_yaml::Value::Mapping( + [ + ( + serde_yaml::Value::String("name".to_string()), + serde_yaml::Value::String("custom-repo".to_string()), + ), + ( + serde_yaml::Value::String("baseurl".to_string()), + serde_yaml::Value::String( + "https://repo.example.com/fedora".to_string(), + ), + ), + ] + .into_iter() + .collect(), + ), + ])) + .unwrap(); + + let cmds = Dnf.add_repository(&repo_yaml, None); + // Should have one command (write .repo file, no GPG key) + assert!(!cmds.is_empty()); + let last_cmd = cmds.last().unwrap().as_std(); + let args: Vec = last_cmd + .get_args() + .map(|a| a.to_string_lossy().into_owned()) + .collect(); + let full_arg = args.join(" "); + assert!(full_arg.contains("cat >")); + assert!(full_arg.contains("custom-repo.repo")); + assert!(full_arg.contains("repo.example.com")); + } + + #[test] + fn test_add_repository_with_gpgkey() { + let repo_yaml = serde_yaml::from_value(serde_yaml::Value::Sequence(vec![ + serde_yaml::Value::Mapping( + [ + ( + serde_yaml::Value::String("name".to_string()), + serde_yaml::Value::String("custom-repo".to_string()), + ), + ( + serde_yaml::Value::String("baseurl".to_string()), + serde_yaml::Value::String( + "https://repo.example.com/fedora".to_string(), + ), + ), + ( + serde_yaml::Value::String("gpgkey".to_string()), + serde_yaml::Value::String( + "https://repo.example.com/gpg".to_string(), + ), + ), + ] + .into_iter() + .collect(), + ), + ])) + .unwrap(); + + let cmds = Dnf.add_repository(&repo_yaml, None); + // Should have 2 commands: import GPG key, write .repo file + assert_eq!(cmds.len(), 2); + let key_cmd = cmds[0].as_std(); + let args: Vec = key_cmd + .get_args() + .map(|a| a.to_string_lossy().into_owned()) + .collect(); + let full_arg = args.join(" "); + assert!(full_arg.contains("rpm --import")); + assert!(full_arg.contains("repo.example.com/gpg")); + } + + #[test] + fn test_add_repository_invalid_yaml_returns_empty() { + let invalid_yaml = serde_yaml::Value::String("invalid".to_string()); + let cmds = Dnf.add_repository(&invalid_yaml, None); + assert!(cmds.is_empty()); + } +} diff --git a/workshop-engine/src/pm/pacman.rs b/workshop-engine/src/pm/pacman.rs index 9150f2a..901fa13 100644 --- a/workshop-engine/src/pm/pacman.rs +++ b/workshop-engine/src/pm/pacman.rs @@ -1,5 +1,5 @@ use crate::pm::PackageManager; -use os_info::{Info, Type}; +use os_info::Info; use serde::Deserialize; use serde_yaml::Value; use tokio::process::Command; @@ -77,20 +77,6 @@ impl PackageManager for Pacman { "pacman" } - fn adopt(&self, distro: &Info) -> bool { - matches!( - distro.os_type(), - Type::Arch - | Type::Manjaro - | Type::EndeavourOS - | Type::Garuda - | Type::CachyOS - | Type::Artix - | Type::Mabox - | Type::Nobara - | Type::InstantOS - ) - } fn binary(&self) -> &'static str { "pacman" @@ -212,54 +198,6 @@ mod tests { assert_eq!(pacman.binary(), "pacman"); } - #[test] - fn test_adopt_true_for_arch_based_distros() { - let pacman = Pacman; - let arch_distros = [ - Type::Arch, - Type::Manjaro, - Type::EndeavourOS, - Type::Garuda, - Type::CachyOS, - Type::Artix, - Type::Mabox, - Type::Nobara, - Type::InstantOS, - ]; - for distro in arch_distros { - let info = Info::with_type(distro); - assert!( - pacman.adopt(&info), - "Pacman should adopt distro: {:?}", - distro - ); - } - } - - #[test] - fn test_adopt_false_for_non_arch_distros() { - let pacman = Pacman; - let non_arch_distros = [ - Type::Ubuntu, - Type::Debian, - Type::Fedora, - Type::CentOS, - Type::RedHatEnterprise, - Type::openSUSE, - Type::Gentoo, - Type::FreeBSD, - Type::Macos, - Type::Windows, - ]; - for distro in non_arch_distros { - let info = Info::with_type(distro); - assert!( - !pacman.adopt(&info), - "Pacman should NOT adopt distro: {:?}", - distro - ); - } - } #[test] fn test_update_command_contents() {