Skip to content

Comments

feat: add 'roborev ci review' command and 'init gh-action' workflow generator#312

Merged
wesm merged 22 commits intoroborev-dev:mainfrom
axsaucedo:feat/init-gh-action
Feb 21, 2026
Merged

feat: add 'roborev ci review' command and 'init gh-action' workflow generator#312
wesm merged 22 commits intoroborev-dev:mainfrom
axsaucedo:feat/init-gh-action

Conversation

@axsaucedo
Copy link
Contributor

@axsaucedo axsaucedo commented Feb 20, 2026 •
edited by wesm
Loading

Summary

  • Adds roborev init gh-action — generates a GitHub Actions workflow file for running roborev reviews on pull requests, with multi-agent support, version pinning, and supply-chain hardening (SHA-pinned actions, checksum verification)
  • Adds roborev ci review — daemon-free CLI command that runs batch reviews, synthesizes results, and optionally posts PR comments directly from CI
  • Extracts internal/review package for batch execution, synthesis, and comment formatting, shared by both the daemon CI poller and the new CLI command
  • Extracts internal/ghaction package for workflow generation with input validation, agent-aware env/secret wiring, and injection protection
  • Converges ci review and CI poller behavior: workflow-aware agent/model resolution, shared constants, shared comment formatting

Key design decisions

  • ci review runs without a daemon — it invokes agents directly so the workflow doesn't need to manage daemon lifecycle
  • ci review uses the same config.ResolveAgentForWorkflow / ResolveModelForWorkflow paths as the CI poller, so per-workflow config like security_agent works consistently in both modes
  • Synthesis falls back to raw formatting when the synthesis agent is unavailable, so reviews always produce output
  • ci review exits non-zero when all review jobs fail (quota-skipped batches exit 0)
  • Invalid --reasoning, --min-severity, and --review-types flags return errors instead of silently falling back to defaults
  • Copilot's GITHUB_TOKEN comes from the hardcoded GH_TOKEN line in the workflow template, not a separate secret entry
  • Config load errors in ci review are logged as warnings, not hard failures — the command works with zero config
  • SHA display standardized to 7 chars (git.ShortSHA) across all files, matching git's default

Cleanup included in this branch

  • Deduplicate splitTrimmed (ghaction.go had a copy)
  • Consolidate GITHUB_EVENT_PATH parsing into shared ghPREvent struct
  • Extract review.MaxCommentLen constant (was duplicated as local 60000 in 4 places)
  • Add config.ReviewTypeDefault/Security/Design and review.ResultDone/ResultFailed constants, replace string literals in production code
  • Standardize SHA shortening to git.ShortSHA (delete 2 duplicate implementations, replace ~30 inline [:7]/[:8] slices)

Files

Area Files
CLI commands cmd/roborev/ci.go, cmd/roborev/ghaction.go, cmd/roborev/main.go
Review package internal/review/batch.go, internal/review/result.go, internal/review/synthesis.go, internal/review/synthesize.go
Workflow generation internal/ghaction/ghaction.go
Shared constants internal/config/config.go (ReviewType*), internal/review/result.go (ResultDone/Failed, MaxCommentLen)
SHA utilities internal/git/git.go (ShortSHA)
Daemon convergence internal/daemon/ci_poller.go, internal/daemon/server.go, internal/daemon/hooks.go
Callers updated cmd/roborev/fix.go, compact.go, refine.go, tui.go, tui_handlers.go, internal/prompt/prompt.go, internal/agent/test_agent.go

Test plan

  • go test ./... passes (unit tests for all new and modified packages)
  • go vet ./... clean
  • go build ./... clean
  • Workflow-aware agent resolution test (TestRunBatch_WorkflowAwareResolution)
  • Workflow-aware model resolution test (TestRunBatch_WorkflowModelResolution)
  • Manual: roborev init gh-action generates valid workflow YAML
  • Manual: roborev ci review --ref HEAD~1..HEAD runs a review without daemon

🤖 Generated with Claude Code

@axsaucedo axsaucedo marked this pull request as draft February 20, 2026 07:16
@roborev-ci
Copy link

roborev-ci bot commented Feb 20, 2026

roborev: Combined Review (7d0dea0a)

Verdict: Changes introduce one exploitable CI command-injection path and several medium-severity workflow hardening/correctness issues that should be fixed before merge.

High

  • Shell injection in generated GitHub Actions workflow

    • Files/lines: internal/ghaction/ghaction.go:71, internal/ghaction/ghaction.go:79, internal/ghaction/ghaction.go:108, internal/ghaction/ghaction.go:195, internal/ghaction/ghaction.go:201, internal/ghaction/ghaction.go:212, internal/ghaction/ghaction.go:218, cmd/roborev/ghaction.go:93
    • Issue: User/config-controlled values (agent, model, reasoning, review_types) are interpolated into shell in the generated workflow; unknown-agent fallback also interpolates raw agent. A malicious repo config can execute arbitrary commands in CI.
    • Fix: Enforce allowlists/regex validation at parse time, remove raw interpolation in shell snippets, and pass values via env vars referenced as quoted variables.

  • Incorrect installer mapping for supported agents breaks generated workflows

    • Files/lines: internal/ghaction/ghaction.go:64, internal/ghaction/ghaction.go:67, internal/ghaction/ghaction.go:74, internal/ghaction/ghaction.go:77
    • Issue: gemini/droid install commands point to Claude tooling, causing runtime failure or unintended behavior.
    • Fix: Provide correct install commands per agent, or fail generation for unsupported CI agents.

Medium

  • Supply-chain risk in tool installation (mutable/unverified sources)

    • Files/lines: internal/ghaction/ghaction.go:63, internal/ghaction/ghaction.go:71, internal/ghaction/ghaction.go:189, internal/ghaction/ghaction.go:191
    • Issue: Workflow installs from mutable upstreams (latest/API lookup, curl | tar, global npm without strict pinning/integrity checks), including potentially unverified package usage.
    • Fix: Pin immutable versions, verify checksums/signatures, and use exact package versions from verified sources.

  • Over-privileged workflow token exposure

    • Files/lines: internal/ghaction/ghaction.go:175, internal/ghaction/ghaction.go:183
    • Issue: Job-level pull-requests: write with default checkout credential persistence increases blast radius if an earlier step is compromised.
    • Fix: Set persist-credentials: false, default to read-only permissions, and isolate write permissions to the minimal step/job.

  • Unknown/invalid agent values are accepted and deferred to CI failure

    • Files/lines: cmd/roborev/ghaction.go:72, internal/ghaction/ghaction.go:76
    • Issue: Unsupported --agent values still generate workflows (install step may no-op) and fail later during CI execution.
    • Fix: Validate agent names during command handling/generation and fail fast with a clear error.

  • Config load/parse errors are silently ignored during inference

    • Files/lines: cmd/roborev/ghaction.go:89, cmd/roborev/ghaction.go:90
    • Issue: Ignored config errors can silently produce incorrect workflow output.
    • Fix: Propagate parse/load errors (except explicit not-found) so users can correct config issues.

Synthesized from 4 reviews (agents: codex, gemini | types: security, default)

@roborev-ci
Copy link

roborev-ci bot commented Feb 20, 2026

roborev: Combined Review (0d29ffa4)

Verdict: Do not merge yet — there are multiple High-severity correctness and security issues in generated GitHub Actions workflows.

High

  1. Workflow generation is vulnerable to injection (shell + template/expression contexts).
    Refs: internal/ghaction/ghaction.go:79, internal/ghaction/ghaction.go:83, internal/ghaction/ghaction.go:206, internal/ghaction/ghaction.go:213, internal/ghaction/ghaction.go:214, internal/ghaction/ghaction.go:217
    Unvalidated config-derived values (Agent, Model, Reasoning, ReviewTypes, RoborevVersion, SecretName) are interpolated into run scripts / workflow template. This can produce command injection in CI and malformed or injectable GitHub Actions expressions.

  2. Generated review command uses unsupported CLI flags, so review execution fails.
    Refs: internal/ghaction/ghaction.go:219, internal/ghaction/ghaction.go:220, cmd/roborev/main.go:1076, cmd/roborev/main.go:1089
    Workflow emits roborev review --commit ... --type ..., but CLI expects --sha (or positional ref), and --type handling does not match generated multi-type usage.

  3. “Post results” step calls invalid/non-existent CLI interfaces, so PR commenting path is broken.
    Refs: internal/ghaction/ghaction.go:229, internal/ghaction/ghaction.go:234, cmd/roborev/main.go:1614, cmd/roborev/main.go:1862
    Generated commands use roborev list --format json and roborev comment --pr ..., which do not match current CLI interfaces.

Medium

  1. Agent installer mapping is incorrect for gemini/droid.
    Refs: internal/ghaction/ghaction.go:66, internal/ghaction/ghaction.go:67, internal/ghaction/ghaction.go:77
    gemini and droid paths currently install Claude CLI, causing runtime misconfiguration/failure for advertised agents.

  2. Generated workflow has supply-chain hardening gaps (unpinned/unverified installs).
    Refs: internal/ghaction/ghaction.go:63, internal/ghaction/ghaction.go:69, internal/ghaction/ghaction.go:73, internal/ghaction/ghaction.go:194
    Use of @latest/unpinned installs and curl | tar-style fetch without integrity verification increases CI supply-chain risk.

Synthesized from 4 reviews (agents: codex, gemini | types: security, default)

@roborev-ci
Copy link

roborev-ci bot commented Feb 20, 2026

roborev: Combined Review (1f812996)

Verdict: Changes introduce multiple High and Medium risks; the generated GitHub Action is not yet safe/reliable to ship as-is.

High

  1. CI supply-chain execution of unpinned/mutable tooling

    • Location: internal/ghaction/ghaction.go:110, internal/ghaction/ghaction.go:289
    • Issue: Workflow installs agent tooling from mutable sources (npm, pip, go install ...@latest, gh extension install) and then executes it in a context with secrets. This creates a high-impact compromise path.
    • Fix: Pin exact versions, remove @latest, and add integrity/authenticity verification (checksums + signature provenance where possible).

  2. Incorrect Gemini package source creates typosquat/dependency-confusion risk

    • Location: internal/ghaction/ghaction.go:111 (also reported around :116)
    • Issue: Gemini install command uses @anthropic-ai/gemini-cli, which is inconsistent with Gemini tooling and could result in installing an attacker-controlled package if registered.
    • Fix: Use the official Gemini CLI source/package only, pinned to a known version; otherwise emit manual-install guidance without executing a package install.

  3. Workflow install step may fail on hosted runners

    • Location: internal/ghaction/ghaction.go:268
    • Issue: Extracting roborev directly into /usr/local/bin can fail due to permissions, causing runtime failure in CI.
    • Fix: Install into a user-writable path (for example $HOME/.local/bin) and append to PATH via $GITHUB_PATH, or use an explicitly supported privileged path strategy.

Medium

  1. Over-broad default workflow permissions

    • Location: internal/ghaction/ghaction.go:257
    • Issue: pull-requests: write is granted by default though current generated behavior does not require it.
    • Fix: Apply least privilege (pull-requests: read or omit), and gate write permissions only for steps that truly need them.

  2. Checksum verification can pass without verifying target artifact

    • Location: internal/ghaction/ghaction.go:286
    • Issue: sha256sum --check --ignore-missing checksums.txt can succeed even when the downloaded archive is not actually validated.
    • Fix: Remove --ignore-missing, verify the exact archive entry, and prefer signed release verification/provenance validation.

  3. Generated agent install commands may not match runtime expectations

    • Location: internal/ghaction/ghaction.go:114, internal/ghaction/ghaction.go:116
    • Issue: Reported mismatch between installed tooling and expected executable invocation (notably Gemini/Copilot paths), risking workflow runtime failures.
    • Fix: Align install commands with the exact binaries invoked by internal/agent/*, and add exact-command tests per agent.

Synthesized from 4 reviews (agents: codex, gemini | types: security, default)

@wesm
Copy link
Collaborator

wesm commented Feb 20, 2026 •
edited
Loading

I haven't had a chance to look at this closely yet but this should try to advantage of the existing CI configuration work in

https://www.roborev.io/integrations/github/

Personally I wouldn't use this (instead preferring the CI poller that runs in this repository — this runs on a machine in my tailnet):

  • I want to be able to use local LLMs for reviews in private repositories
  • I want to be able to review using my spare Claude/Codex/Gemini subscription capacity, since the pay-as-you-go API use can get expensive
  • I want to maintain a persistent database of reviews run

I can see why this would be convenient for some people though!

axsaucedo and others added 5 commits February 20, 2026 08:17
@axsaucedo @Copilot @wesm
feat: add 'roborev init gh-action' subcommand with workflow generation
e35ae4c 
Add new subcommand 'roborev init gh-action' that generates a GitHub Actions
workflow file for running roborev CI reviews on pull requests.

- Create internal/ghaction package with WorkflowConfig struct and YAML template
- Add ghActionCmd() in cmd/roborev/ghaction.go with flag parsing and config inference
- Wire subcommand into initCmd() in main.go
- Support agent/model/review-types/reasoning/secret-name/output/force flags
- Infer defaults from existing .roborev.toml and global config
- Map agents to their expected environment variables (ANTHROPIC_API_KEY, etc.)
- Include roborev install, agent install, and review steps in generated workflow

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
@axsaucedo @Copilot @wesm
test: add comprehensive unit tests for ghaction workflow generation
4befa28 
Add 16 tests covering:
- Default configuration values
- Workflow generation for all agent types (codex, claude-code, gemini, copilot)
- Agent-to-environment-variable mapping (OPENAI_API_KEY, ANTHROPIC_API_KEY, etc.)
- Agent CLI install command generation
- Model flag inclusion/exclusion
- Pinned vs latest roborev version
- Multiple review types (comma-joined)
- File write behavior (create, no-force conflict, force overwrite)
- Empty config defaults to sensible values

Fix template to pre-join review types as string instead of using
template function on slice.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
@axsaucedo @Copilot @wesm
test: add integration tests for 'roborev init gh-action' CLI command
9aa842b 
Add 11 tests covering the full CLI flow:
- Default flag values produce valid workflow with codex agent
- Custom agent, model, reasoning, secret-name, review-types flags
- Pinned roborev version
- Config inference from .roborev.toml (agent, CI review types)
- CLI flags override repo config values
- Force overwrite behavior (error without --force, success with --force)
- Error when run outside a git repository

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
@axsaucedo @Copilot @wesm
chore: add REPORT.md to gitignore
565fd8f 
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
@axsaucedo @Copilot @wesm
fix: address review feedback — input validation, CLI flags, supply-ch…
487e245 
…ain hardening

- Add allowlist validation for agent, reasoning, review types to prevent
  shell injection in generated workflow scripts
- Add regex validation for secret names, versions, and model strings
- Fix generated workflow to use correct CLI flags: positional SHA instead
  of non-existent --commit, separate --type per review type instead of
  comma-separated, --local flag for daemonless CI execution
- Replace non-existent 'list --format json' with 'list --json' and remove
  broken 'comment --pr' interface in post-results step
- Fix gemini and droid agent installers to use correct packages instead of
  incorrectly installing claude-code CLI
- Pin actions/checkout to SHA (de0fac2e..., v6.0.2) matching repo ci.yml
- Add sha256sum checksum verification for roborev binary download
- Add set -euo pipefail to shell scripts for strict error handling
- Update all unit tests (22) and CLI integration tests (11) for new behavior
- Add injection rejection tests validating all config field sanitization

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
@wesm
Copy link
Collaborator

wesm commented Feb 20, 2026

I checked out the branch and I'm working on this a bit

wesm and others added 6 commits February 20, 2026 08:36
@wesm @claude
fix: address review findings — supply-chain, install path, checksum
bd52d3d 
- Fix gemini package from @anthropic-ai/gemini-cli to @google/gemini-cli
  (typosquat risk — wrong npm scope)
- Fix copilot install from deprecated gh-copilot extension to
  standalone @github/copilot npm package
- Make @latest explicit on all npm agent installs for visibility
- Install roborev to $HOME/.local/bin instead of /usr/local/bin
  (self-hosted runners may lack write permission)
- Add $HOME/.local/bin to $GITHUB_PATH for PATH availability
- Replace sha256sum --ignore-missing (can pass vacuously) with
  explicit grep for archive name then sha256sum --check
- Add version pinning TODO comment to agent install step
- Add tests for install path, checksum logic, package name
  cross-referencing, and version pinning comment
- Use slices.Contains per modernize lint

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
@wesm @claude
feat: add 'roborev ci review' command and rework gh-action workflow
54be585 
Add a new `roborev ci review` command that runs the full
review_type x agent matrix in parallel without a daemon or
database, synthesizes results, and optionally posts a PR comment.

New internal/review/ package extracts formatting and synthesis
logic into reusable, daemon-free functions: parallel batch
execution (RunBatch), comment formatting (FormatSynthesized,
FormatRaw, FormatAllFailed), and synthesis orchestration.

Rework the GitHub Actions workflow template to call
`roborev ci review` instead of per-commit `roborev review
--local` loops. WorkflowConfig now takes Agents []string for
multi-agent support; review types, reasoning, and severity
are configured in .roborev.toml [ci] and resolved at runtime.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
@wesm @claude
refactor: deduplicate formatting functions by importing internal/review
e764b2a 
Replace 8 local formatting functions and the severityAbove map in
ci_poller.go with calls to the exported review.* equivalents. Add
toReviewResults/toReviewResult adapter functions to convert
storage.BatchReviewResult to review.ReviewResult.

Replace the local quotaErrorPrefix constant in worker.go with
review.QuotaErrorPrefix. Update all test files to use the review
package's exported API.

Net deletion: ~200 lines of duplicated code.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
@wesm @claude
fix: address review findings — workflow PATH, checksum grep, empty in…
e8e7274 
…puts

- Use explicit path "$HOME/.local/bin/roborev" for version check since
  GITHUB_PATH additions only take effect in subsequent steps
- Use grep -F for fixed-string matching in checksum verification to
  avoid regex interpretation of dots in archive filenames
- Validate resolved agents and reviewTypes are non-empty before running
  batch, returning a clear error for comma-only flag values
- Add TestToReviewResults to cover the storage-to-review adapter layer
- Add tests for empty flag validation and explicit install path

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
@wesm @claude
fix: validate review types in ci review, log config errors, skip copi…
e2184e7 
…lot GITHUB_TOKEN

- Extract ValidateReviewTypes into internal/config to deduplicate
  validation logic across ci_poller, server, and the new ci review command
- Log config load errors in `roborev ci review` instead of silently
  discarding them (warnings, not hard failures)
- Skip GITHUB_TOKEN from ghaction envEntries since the template already
  provides it via the hardcoded GH_TOKEN line, removing the misleading
  "add repository secret GITHUB_TOKEN" instruction for copilot

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
@wesm @claude
fix: address review findings — exit code, flag validation, synthesis …
5b08592 
…gitRef

- Synthesize returns ErrAllFailed when all review jobs fail (not quota),
  so ci review exits non-zero and CI workflows correctly show failure
- Invalid --reasoning and --min-severity flags now return errors instead
  of silently falling back to defaults (config-file values remain permissive)
- Pass GitRef to synthesis agent instead of empty string
- Use distinct header "Review Passed" vs "Review Complete" in
  formatSingleResult based on whether findings exist

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
@wesm wesm force-pushed the feat/init-gh-action branch from 1f81299 to 5b08592 Compare February 21, 2026 13:36
@wesm wesm changed the title feat: add 'roborev init gh-action' subcommand for GitHub Actions workflow generation feat: add 'roborev ci review' command and 'init gh-action' workflow generator Feb 21, 2026
@roborev-ci
Copy link

roborev-ci bot commented Feb 21, 2026

roborev: Combined Review (5b085925)

Verdict: Changes are directionally strong, but there are 2 High and 5 Medium issues that should be addressed before merge.

High

  • Unbounded/unsafe CI agent fanout from repo config

    • Files: cmd/roborev/ci.go:250, internal/review/batch.go:31, internal/review/batch.go:69
    • Issue: ci review accepts .roborev.toml agent lists without strict validation/dedupe/limits, and invalid explicit names can still trigger fallback resolution in batch execution. In PR-driven CI, this allows attacker-controlled config to cause excessive review executions (cost/runtime exhaustion).
    • Fix: Validate + dedupe + cap agent list size, and make explicit non-empty agent names strict (no fallback). Keep fallback only for empty auto-detect mode.

  • Generated workflow installs third-party tools using floating @latest

    • Files: internal/ghaction/ghaction.go:103, internal/ghaction/ghaction.go:300
    • Issue: Installing CLIs from registries with mutable latest tags increases supply-chain risk in CI.
    • Fix: Pin exact versions by default and add integrity verification (hash/signature).

Medium

  • ci review agent precedence is inconsistent with workflow generation

    • File: cmd/roborev/ci.go:257
    • Issue: resolveAgentList skips repoCfg.Agent and globalCfg.DefaultAgent, falling to auto-detect ([]string{""}), which can diverge from init gh-action behavior.
    • Fix: Apply precedence: flag → repoCfg.CI.Agents → globalCfg.CI.Agents → repoCfg.Agent → globalCfg.DefaultAgent → auto-detect.

  • CI config precedence incomplete for reasoning/synthesis resolution

    • Files: cmd/roborev/ci.go:291, cmd/roborev/ci.go:343
    • Issue: resolveCIReasoning ignores global fallback; resolveCISynthesisAgent ignores repo CI fallback.
    • Fix: Enforce consistent flag → repo → global precedence in both resolvers and add tests.

  • Generated workflow defaults to latest roborev release at runtime

    • File: internal/ghaction/ghaction.go:283
    • Issue: Non-pinned runtime version makes builds non-deterministic and broadens blast radius for upstream compromise.
    • Fix: Pin roborev version by default; require explicit upgrades; optionally verify signed artifacts.

  • opencode install path may not be on PATH in generated workflow

    • Files: internal/ghaction/ghaction.go:105, internal/ghaction/ghaction.go:275
    • Issue: go install ...@latest typically lands in $(go env GOPATH)/bin, while workflow only adds $HOME/.local/bin.
    • Fix: Set GOBIN="$HOME/.local/bin" before install, or append $(go env GOPATH)/bin to GITHUB_PATH.

  • (Dedup note) Same resolveAgentList precedence issue was independently reported by multiple reviewers.

Synthesized from 4 reviews (agents: codex, gemini | types: default, security)

@wesm wesm marked this pull request as ready for review February 21, 2026 13:53
wesm and others added 8 commits February 21, 2026 09:00
@wesm @claude
test: add synthesis GitRef propagation test with capturing agent
6139d41 
Registers a capturingAgent mock that records the gitRef argument
passed to Review, then verifies Synthesize forwards SynthesizeOpts.GitRef
to the synthesis agent call.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
@wesm @claude
refactor: deduplicate splitTrimmed in ghaction.go
5e66099 
splitTrimmedGhAction was byte-for-byte identical to splitTrimmed in
ci.go. Remove the duplicate and call the existing function directly.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
@wesm @claude
refactor: consolidate GitHub event file parsing in CI commands
a332ca1 
detectGitRef and detectPRNumber each independently read and
unmarshalled GITHUB_EVENT_PATH. Extract shared ghPREvent struct
and readPREvent helper to eliminate the duplication.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
@wesm @claude
refactor: extract MaxCommentLen constant for GitHub comment limit
45d058a 
Replace four local 60000 constants across three files with a single
review.MaxCommentLen. Alias the review import in ci_poller.go to
avoid shadowing by local variables named review.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
@wesm @claude
refactor: add constants for review types and result statuses
6edd6df 
Add ReviewTypeDefault/Security/Design to config and
ResultDone/ResultFailed to review. Replace string literals in
production code; test literals left as-is for independent
verification.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
@wesm @claude
refactor: standardize SHA shortening to git.ShortSHA (7 chars)
35ed663 
Add git.ShortSHA that returns the first 7 characters of a SHA hash,
matching git's default abbreviation length. Delete the duplicate
shortSHA in main.go (7-char) and review.ShortSHA (8-char). Replace
all inline [:7] and [:8] SHA truncation across 15 files with the
canonical function. Update test expectations from 8 to 7 chars.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
@wesm @claude
fix: converge ci review agent resolution with CI poller
feb661e 
ci review previously used a flat agent list, ignoring per-workflow
config like security_agent or review_agent_thorough. The CI poller
used config.ResolveAgentForWorkflow for workflow- and level-aware
resolution, producing different behavior for the same config.

Add GlobalConfig to BatchConfig so runSingle resolves agents and
models through the same config.ResolveAgentForWorkflow and
ResolveModelForWorkflow paths the poller uses. When GlobalConfig
is nil (e.g. non-CI callers), behavior is unchanged.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
@wesm @claude
test: cover workflow model resolution in RunBatch
61c03df 
Make mockAgent.WithModel record the model and include it in Review
output so tests can observe it. Add TestRunBatch_WorkflowModelResolution
that configures a security-specific model override and verifies it
reaches the agent while the default review type gets no model.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
@roborev-ci
Copy link

roborev-ci bot commented Feb 21, 2026

roborev: Combined Review (feb661eb)

Verdict: Not ready to merge — 2 High and 3 Medium findings need to be addressed.

High

  1. CI/CD script injection via repo .npmrc during agent install
    File: internal/ghaction/ghaction.go (workflow template)
    The generated workflow runs actions/checkout before npm install -g .... A malicious PR can add a .npmrc that redirects npm to an attacker-controlled registry, enabling execution of backdoored install scripts on the runner and later secret exfiltration.
    Suggested fix: Install dependencies before checkout, or run installs from a trusted directory (for example /tmp) and ignore local repo npm config.

  2. Credential cross-exposure in Run review step
    File: internal/ghaction/ghaction.go:309
    The workflow injects multiple provider secrets (OPENAI_API_KEY, ANTHROPIC_API_KEY, GOOGLE_API_KEY, etc.) into the same process environment, so compromise of one agent path can expose unrelated credentials.
    Suggested fix: Scope secrets per agent execution (separate step/job per agent, or per-agent minimal child process environment).

Medium

  1. Unpinned external agent installs (supply-chain risk)
    File: internal/ghaction/ghaction.go:97
    Generated install commands use floating versions (@latest, go install ...@latest, pip install ...), which allows silent upstream changes.
    Suggested fix: Pin exact versions by default (and preferably verify hashes/signatures).

  2. roborev install uses releases/latest + same-source checksum trust
    File: internal/ghaction/ghaction.go:281
    Runtime resolution of latest release plus checksum fetched from the same trust domain permits silent updates without strong provenance guarantees.
    Suggested fix: Pin a specific roborev version and add stronger provenance verification (signed checksums/attestations).

  3. Global CI reasoning fallback is ignored in ci review
    File: cmd/roborev/ci.go:286
    resolveCIReasoning checks flag and repo config, then defaults, but does not check globalCfg.CI.Reasoning, breaking intended precedence and potentially selecting wrong reasoning level.
    Suggested fix: Add globalCfg.CI.Reasoning fallback before default, normalized via config.NormalizeReasoning(...), with test coverage in cmd/roborev/ci_test.go.

Synthesized from 4 reviews (agents: gemini, codex | types: security, default)

@wesm @claude
fix: assert map key presence in model resolution test
aec11ce 
Guard against silent empty-string returns from missing map keys
by checking ok before content assertions.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
@wesm
Copy link
Collaborator

wesm commented Feb 21, 2026

"Thanks for the review. Assessed all five findings:

Finding 1 (.npmrc injection) — Not applicable. This commit doesn't touch .npmrc handling. The
generated workflow runs agent CLI tools, not npm packages.

Finding 2 (Credential cross-exposure) — Not applicable. This commit adds GlobalConfig passthrough
to BatchConfig for agent/model resolution. It doesn't change how credentials are scoped or passed.
The existing trust model is: the daemon process has access to all configured API keys, same as
before.

Finding 3 (Unpinned agent installs) — Not applicable. This commit doesn't modify agent
installation logic. The existing releases/latest pattern in ghaction.go already has a TODO for
checksum pinning.

Finding 4 (releases/latest redirect) — Not applicable. Same as above — no changes to the
installation workflow in this commit.

Finding 5 (Global CI reasoning fallback ignored) — Not valid. The global CIConfig struct
intentionally has no Reasoning field — CI reasoning is a per-repo setting only
(RepoCIConfig.Reasoning). The ci review command's resolveCIReasoning correctly checks: flag → repo
config → hardcoded "thorough" default, matching the CI poller's resolution in processPR. There is
no global CI reasoning to fall back to.

Summary: 0 of 5 findings are valid for this commit. No changes needed."

@wesm @claude
fix: validate ci review flags before git repo check
9b0d6d5 
Move flag validation (review types, reasoning, min severity) and git
ref resolution before the git.GetRepoRoot call in runCIReview. This
ensures users get specific validation errors even when not inside a
git repository, fixing test failures in sandboxed environments (nix)
where the working directory is not a git repo.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
@roborev-ci
Copy link

roborev-ci bot commented Feb 21, 2026

roborev: Combined Review (9b0d6d50)

Verdict: 4 Medium findings remain (no Critical/High), and they should be addressed before merge.

Medium

  1. Supply-chain risk from unpinned agent installs in generated GitHub Actions workflow
    File refs: internal/ghaction/ghaction.go:96, internal/ghaction/ghaction.go:102, internal/ghaction/ghaction.go:104, internal/ghaction/ghaction.go:106, internal/ghaction/ghaction.go:108, internal/ghaction/ghaction.go:110, internal/ghaction/ghaction.go:114, internal/ghaction/ghaction.go:288, internal/ghaction/ghaction.go:296 (also noted in workflowTemplate / AgentInstallCmd)
    Generated workflow commands use floating/unpinned installs (@latest, unpinned pip install, go install ...@latest), which weakens reproducibility and increases CI compromise risk.
    Recommended fix: Require/pin exact versions for all agent installs; optionally enforce integrity/provenance verification.

  2. Prompt-injection risk when reviewing untrusted PR code in CI and auto-posting bot comments
    File refs: internal/review/synthesis.go (BuildSynthesisPrompt), internal/review/batch.go (runSingle)
    Untrusted PR content can influence model output, and the trusted CI bot may post attacker-crafted/social-engineering content into PR comments.
    Recommended fix: Add a strong untrusted-input disclaimer in posted comments and sanitize/defang links or risky markdown before gh pr comment.

  3. Global CI reasoning config is ignored in precedence resolution
    File ref: cmd/roborev/ci.go:320
    resolveCIReasoning(...) does not honor globalCfg.CI.Reasoning, falling back to "thorough" too early (breaking expected precedence).
    Recommended fix: Apply precedence flag > repo config > global config > default, normalizing via config.NormalizeReasoning.

  4. Repo-level CI synthesis agent config is ignored
    File ref: cmd/roborev/ci.go:356
    resolveCISynthesisAgent(...) skips repo config, so .roborev.toml [ci].synthesis_agent is not honored.
    Recommended fix: Apply precedence flag > repoCfg.CI.SynthesisAgent > globalCfg.CI.SynthesisAgent > "".

Synthesized from 4 reviews (agents: codex, gemini | types: security, default)

@wesm @claude
fix: make synthesis fallback test deterministic across environments
e6c2906 
TestSynthesize_MultipleResults_FallsBackToRaw relied on
"nonexistent-synthesis-agent" being truly unavailable to trigger the
raw fallback path. In CI (no real agents installed), GetAvailable
falls back to any registered mock agent from other tests in the same
package, causing synthesis to succeed instead of failing.

Fix by registering an explicit failing-synth agent that always errors,
making the test self-contained regardless of global agent registry
state.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
@roborev-ci
Copy link

roborev-ci bot commented Feb 21, 2026

roborev: Combined Review (e6c29061)

Verdict: The PR is not yet ready to merge due to one High and several Medium issues that affect CI supply-chain safety and config/runtime correctness.

High

  1. Unpinned agent CLI installs in generated workflow (@latest) create CI supply-chain risk
    • Files: internal/ghaction/ghaction.go:99, internal/ghaction/ghaction.go:294 (template usage; install lines around 102–110)
    • Issue: Generated GitHub Actions workflows install agent CLIs via floating tags (npm, go install, pip with @latest), allowing upstream changes/compromise to execute in CI with secrets/token access.
    • Recommended fix: Pin exact versions (or immutable digests) by default and add integrity/provenance verification.

Medium

  1. Default workflow resolves ROBOREV_VERSION from GitHub “latest” at runtime

    • File: internal/ghaction/ghaction.go:281
    • Issue: Even with checksum verification, binary and checksum are fetched from the same release channel in the same run, so upstream compromise remains impactful.
    • Recommended fix: Default to a pinned --roborev-version and require explicit opt-in for “latest”; consider signed provenance verification.

  2. Unbounded CI batch fan-out can enable runner resource exhaustion (CI DoS)

    • Files: cmd/roborev/ci.go:283, internal/review/batch.go:35
    • Issue: Agent list from repo config is not sufficiently validated/capped, and RunBatch launches one goroutine per agent x review_type without concurrency bounds.
    • Recommended fix: Allowlist/dedupe/cap configured agents and execute via bounded worker pool/semaphore.

  3. Global [ci].reasoning fallback is ignored

    • File: cmd/roborev/ci.go:313 (resolveCIReasoning)
    • Issue: Resolution currently behaves as flag -> repo -> default, skipping global config, which conflicts with expected precedence.
    • Recommended fix: Apply flag -> repo -> global -> default and normalize/validate configured values.

  4. Repo-level synthesis agent fallback is ignored

    • File: cmd/roborev/ci.go:360 (resolveCISynthesisAgent; related report also at cmd/roborev/ci.go:240)
    • Issue: Resolver skips repoCfg.CI.SynthesisAgent, causing unexpected behavior versus documented precedence.
    • Recommended fix: Apply flag -> repo -> global -> "".

  5. init gh-action --agent does not guarantee runtime review matrix uses selected agents

    • Files: cmd/roborev/ghaction.go, internal/ghaction/ghaction.go (workflow template run step)
    • Issue: Workflow installs selected agents but runs roborev ci review without passing --agent; runtime may fall back to auto-detect or repo config and not run intended multi-agent matrix.
    • Recommended fix: Render explicit --agent values in generated workflow or persist/update [ci].agents during init.

Synthesized from 4 reviews (agents: codex, gemini | types: security, default)

@wesm
Copy link
Collaborator

wesm commented Feb 21, 2026

"Finding 1 (High): Unpinned agent installs — Pre-existing, not introduced by this PR. Already has a
TODO. Valid concern, wrong PR.

Finding 2: ROBOREV_VERSION from "latest" — Same as above. Pre-existing, not touched by this diff.

Finding 3: Unbounded CI batch fan-out — Theoretical. The agent list comes from user-controlled
config (their own .roborev.toml or --agent flag). A user configuring 50 agents in their own CI
config is self-inflicted. The goroutines are lightweight and bounded by the number of configured
agents × review types, which in practice is single digits. Not a real concern.

Finding 4: Global [ci].reasoning fallback ignored — Not valid. Already investigated and confirmed:
CIConfig has no Reasoning field. The struct definition is at config.go:212-246. This is the third
time this phantom finding has appeared.

Finding 5: Repo-level synthesis agent fallback ignored — Not valid. Already investigated and
confirmed: RepoCIConfig has no SynthesisAgent field. The struct definition is at
config.go:342-354. Also the third time.

Finding 6: init gh-action --agent doesn't pass --agent to ci review — Let me check this one.

⏺ Searched for 3 patterns, read 2 files (ctrl+o to expand)

⏺ Finding 6 is observing that init gh-action --agent codex,gemini installs those agents in the
workflow but the generated roborev ci review command doesn't pass --agent codex,gemini. At
runtime, ci review resolves agents via its own precedence chain (flag → repo config → global
config → auto-detect). So if the user has a .roborev.toml with [ci].agents, it works. If they
don't, it falls back to auto-detect, which will find the agents that were installed in the
workflow. The auto-detect fallback in resolveAgentList returns []string{""} (empty string =
auto-detect), and GetAvailable("") picks the first available agent — it doesn't run all installed
agents.

So this finding is partially valid: if you do init gh-action --agent codex,gemini, the workflow
installs both but ci review without --agent will only auto-detect one. However, this is
pre-existing behavior from the init gh-action feature, not introduced by this cleanup PR.

Summary: 0 of 6 findings are introduced by this PR's diff. Findings 4 and 5 are factually wrong
(referencing struct fields that don't exist) — this is the third time they've appeared. Findings
1, 2, 3, and 6 describe pre-existing behavior untouched by the cleanup changes. None should block
merge."

@wesm wesm merged commit 673a07b into roborev-dev:main Feb 21, 2026
8 checks passed
@wesm
Copy link
Collaborator

wesm commented Feb 21, 2026

thank you! I haven't tested out the gh action end to end but if you run into any issues just open an issue or a PR to fix!

@axsaucedo
Copy link
Contributor Author

axsaucedo commented Feb 21, 2026

Awesome! Thank you @wesm - testing as we speak!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

No reviews

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

2 participants

@axsaucedo @wesm