Repository navigation
Run the hosted Go flow on real Bazel in CI - #2107
Mikola Lysenko (mikolalysenko) wants to merge 5 commits into
Conversation
socket-patch has no Bazel-specific code, and until now no test ran it
in a Bazel workspace. This adds a test target that does, and a workflow
that runs it on pinned Bazel releases. No file under crates/*/src
changes.
The first tests hold hosted mode for a Go module that comes through
gazelle's go_deps.from_file under Bzlmod, on Bazel 9.3.2, 8.8.2 and
7.7.2. Each one, in a fresh output base:
- builds the fixture workspace and reads "OUT: PRISTINE" from
`bazel run //:consumer`;
- runs `scan --mode hosted --yes`, which rewrites go.mod and go.sum
and nothing else;
- runs the same two Bazel commands and reads the patched marker, which
holds an id made for the run: the Socket module came from the
stand-in patch host, Bazel changed no file of the checkout, and a
`--lockfile_mode=error` build passes;
- runs `rollback --yes`: every file of the checkout is the one of
before, byte for byte, and the build prints "OUT: PRISTINE" again;
- builds the rewritten checkout in a second output base against a
patch host that serves other bytes: the build fails on the sum.
The Socket API, the Go proxy, the patch host and the checksum-database
lookup are one loopback wiremock server, and `go` is given no public
proxy. The Bazel registry, the ruleset archives, the Go release list
and the Go SDK come from the network. Bazel 9's repository contents
cache stays on, as a user has it, with one cache per workspace copy.
- crates/socket-patch-cli/tests/e2e_bazel_build/: harness.rs keeps
every Bazel write inside a per-test scratch root (startup options,
environment, repository cache, contents cache, server shutdown) and
runs the socket-patch child; go.rs builds the two modules, the
stand-in and the flow. The tests are #[ignore]d and print SKIP
without a Bazel, or fail when SOCKET_PATCH_BAZEL_E2E_REQUIRED is
set. Two hermetic tests run on every Unix runner.
- crates/socket-patch-cli/tests/fixtures/bazel/gazelle/bzlmod/{e9,e7}:
the two workspaces (gazelle 0.54.0 with rules_go 0.65.0, and gazelle
0.45.0 with rules_go 0.53.0 for Bazel 7).
- scripts/install-bazel.sh and scripts/bazel-pins.txt: one Bazel
release binary, checked against a pinned sha256 before it is
installed.
- .github/workflows/bazel-compatibility.yml: one Linux leg per Bazel
release on a pull request that touches these files or a Go file the
tests run through, on a main push that also touches the shared
engine code and test helpers, nightly and on dispatch; one macOS leg
nightly and on dispatch. Not required for merge.
A later test of the same flow is one fixture directory and one test
function. A new Bazel major adds its pin lines and one matrix line; a
new ecosystem adds its module and one matrix line per release.
Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
The real-Bazel test of the hosted Go flow asserts what it asserted. This changes how a leg selects it, what it takes from the machine it runs on, and how its Bazel is installed. - A flow is one `#[ignore]` test, `go::hosted_go_deps`. It takes its release from the Bazel it is given (`bazel --version`) and its fixture era from one table in the test code, `go::FIXTURE`; a release the table does not hold fails with a message that names the table. The workflow passes the module filter `go::`: no name prefix derived from the release, no assert on test names, no scrape of the log. `SOCKET_PATCH_BAZEL_E2E_VERSION` fails a leg whose Bazel is not its release, as `SOCKET_PATCH_GO_E2E_VERSION` does for Go. - The fixture is one `common` directory and one `MODULE.bazel` per era, in place of two full copies. A hermetic test holds the directories to the table and the one `go.sum` to the module the test serves. - The Go fetch settings that gazelle hands to `go mod download` (`GOPRIVATE`, `GONOPROXY`, `GONOSUMDB`, `GOSUMDB`, `GOAUTH`), and `GOFLAGS` and `GOINSECURE`, are pinned beside `GOPROXY`, as variables of the Bazel client and as `--repo_env`. An exported `GOPRIVATE=patch.socket.dev` used to send the fetch past the loopback stand-ins to the real host. - `XDG_CONFIG_HOME`, `XDG_DATA_HOME` and `XDG_STATE_HOME` are removed from every Bazel child, so on Linux the telemetry counters of the Go SDK land under the scratch home. `GOTELEMETRY` is not set: `go env` reports it, and Go does not read it from the environment. - A bazelisk launcher gets one `BAZELISK_HOME`, the caller's own cache, so it downloads a release at most once, not once per workspace copy. - A relative `SOCKET_PATCH_BAZEL_E2E_BAZEL` or `SOCKET_PATCH_BAZEL_E2E_SOCKET_BIN` fails at once with a message that names the variable, with or without `SOCKET_PATCH_BAZEL_E2E_REQUIRED`. - `scripts/install-bazel.sh` gives curl a connect timeout, a low-speed limit and a total time, and retries only transient failures: a 404 goes to the second origin at once and a stalled transfer ends. It prints one line on stdout, and the documented recipe is `export "$(scripts/install-bazel.sh <version> <dir>)"`, which keeps a space in the path. - The workflow's jobs have no `name:`. GitHub showed the matrix expressions of the name as text for a job that its `if` skipped. Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
PR SummaryLow Risk Overview A new
Reviewed by Cursor Bugbot for commit 39893d3. Configure here. |
The Go flow kept to itself what any flow on a real Bazel needs, and the workflow had one path list and one matrix for every ecosystem. A second ecosystem would copy the first, and each would start its legs on every change to the other. harness.rs now holds what the flows share: - StandIn, the loopback server behind the hosts of a flow (start, origin, serve, api, requests, asked), and the steps a flow is made of: the unpatched build with Bazel's lock, the envelope and the files of a hosted scan, the patched build that changes no file and passes with a frozen lock, and the undo. go.rs uses them and asserts what it asserted before. - Cell::route_downloads and rewrite, for a ruleset that fetches with Bazel's own downloader: a downloader config beside the workspace copy, named on every later command of the cell and of its twin by the option of the release (--experimental_downloader_config before Bazel 8.2.0, --downloader_config from there on; 8.2.0 and later still take the old name, with a warning). A rule quotes its URL prefix, so a `.` or a `+` in a file name is no pattern. Cell::numbers gives a flow the parsed release. - The environment of every Bazel child. The tool settings of a developer's shell no longer reach the repository rules and `bazel run`: GEM_HOME, GEM_PATH, NODE_OPTIONS, NODE_PATH, PYTHONHOME, PYTHONPATH, PYTHONSTARTUP, PIP_* and UV_*, and through the scrubs of tests/common/hermetic.rs BUNDLE_*, RUBYOPT, RUBYLIB, npm_config_*, PNPM_* and VIRTUAL_ENV, beside the Go settings removed before. The proxy of the machine stays, because a build may need it to reach the Bazel registry, and loopback is added to no_proxy and NO_PROXY: without it Bazel's downloader asks the proxy for a stand-in's URL. Two hermetic tests hold this. One checks the option name, and the config a cell and its twin name. The other starts the test binary again with every removed variable and a proxy set, and reads the environment of a child spawned there. .github/workflows/bazel-compatibility.yml becomes a reusable workflow that holds the legs: the Linux matrix, the macOS job, the installer, the caches, the REQUIRED and VERSION settings and the BAZEL_COMPAT_DISABLED switch. .github/workflows/bazel-go.yml calls it with the Go module, fixture and releases, under its own triggers, path lists and concurrency group. Another ecosystem adds a caller and touches neither file. A pull request run covers the ecosystem's own flow and fixture, the harness, the installer with its pins and the two workflows; shared engine code is covered on the push to main and nightly, as before. The repository cache is one per ecosystem and fixture era now, not one per Bazel release: the legs of one era share it. A caller names the era of each release it runs, and Fixture::check fails when that is not the era the test code's table gives the release. The run step fails when its module filter matches no test, which would pass with nothing run. Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
|
[agent] PR shepherd: review brief What it does Risk: low. The new workflow is not required for merge: it has no Look here
Verified
Changes I made Open questions
Generated by Claude Code |
…by exact name The test that a Bazel child sees none of a shell's tool settings read its names from the two lists it was meant to guard, so a name dropped from either still passed. The names are now written out in the test, and the test also fails when a list gains a name it does not cover. Dropping GEM_HOME, the PIP_ prefix or the virtualenv scrub each fails it. The run step chose an ecosystem's tests with a libtest filter, which is a substring: `go::` is also inside `cargo::`. It now lists the ignored tests, keeps the names that start with the module, and runs those by exact name; it still fails when the module has no test. The header of bazel-pins.txt names the callers as the place that says which releases run. Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
…s own A concurrency group holds one pending run. In the group of main's pushes, a second push replaces a nightly run that waits behind the first, and the nightly is the only run with the macOS leg and the only run for a main change outside the push path list. Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
|
[agent] PR shepherd: review brief for What changed since the last brief: Risk: low. The workflow is not required: it has no Look here
Verified
Changes I made: none. Open questions (all minor, none block merging)
Generated by Claude Code |
No user-visible change.
What this is
The first pull request of a small series that proves socket-patch in a Bazel workspace on a real Bazel, in GitHub Actions. Tests only: no file under
crates/*/srcchanges, and the tests run the binary that main builds.socket-patch has no Bazel-specific code. Hosted mode for Go already works under gazelle's
go_deps, because it editsgo.modandgo.sum, which gazelle reads. Until now no test said so.Contents
Implementation 0 lines. Tests, scripts and workflows 2,025 lines. Fixtures 53 lines in 6 files, no binary file. Five commits: the second answers the first code review below; the third and fourth move into the harness what the reviews of the three ecosystem pull requests stacked on this one (#2109, #2110, #2111) found in common; the fifth gives the nightly run a concurrency group of its own.
crates/socket-patch-cli/tests/e2e_bazel_build/main.rs#![cfg(unix)]), how to run it, how to add a test.../e2e_bazel_build/harness.rssocket-patchchild;bazel shutdownon drop; hermetic tests.../e2e_bazel_build/go.rsgo::hosted_go_deps) and a hermetic testtests/fixtures/bazel/gazelle/bzlmod/{common,e9,e7}MODULE.bazeleras: gazelle 0.54.0 + rules_go 0.65.0 (Bazel 8 and 9), gazelle 0.45.0 + rules_go 0.53.0 (Bazel 7)scripts/install-bazel.sh,scripts/bazel-pins.txt.github/workflows/bazel-compatibility.yml.github/workflows/bazel-go.ymlWhat it proves, and where
Hosted mode, gazelle
go_deps.from_file(go_mod = "//:go.mod")under Bzlmod, on Bazel 9.3.2, 8.8.2 and 7.7.2. The test, in a fresh output base:bazel buildandbazel run //:consumerprintOUT: PRISTINE.socket-patch scan --mode hosted --yesrewritesgo.modandgo.sumand nothing else.OUT: PATCHED <id of the run>. The Socket module came from the stand-in patch host, Bazel changed no file of the checkout, and a--lockfile_mode=errorbuild passes.socket-patch rollback --yes: every file of the checkout is the one of before, byte for byte, and the build printsOUT: PRISTINEagain.The Socket API, the Go proxy, the patch host and the checksum-database lookup are one loopback wiremock server.
gois given no public proxy, and the Go settings that could route around the stand-in (GOPRIVATE,GONOPROXY,GONOSUMDB,GOSUMDB,GOAUTH,GOFLAGS,GOINSECURE) are pinned.Linux x86_64 (the CI runner): all three legs pass at the present head, in two to three minutes per leg including the cargo build (run 44054611385, checks
go / bazel (9.3.2, e9),go / bazel (8.8.2, e9),go / bazel (7.7.2, e7)). They also passed on their very first run, cold, before the harness was reshaped (run 44019323002). Test time over the last two runs (the spread is the runner and the download hosts; no cache is saved before main has the workflow):Each log has the line
Bazel <release> built "OUT: PATCHED <id>" from patch.socket.dev/gopatch/<id>@v1.0.0-socketpatch.1 afterscan --mode hosted;rollbackrestored all 6 files and the "OUT: PRISTINE" build; other bytes at the patch host failed a second machine's build.macOS arm64, one developer machine (the installer's binaries, a fresh output base per run, whole
cargo testcommand):The first commit also passed against the released 5.1.0 binary on all three releases.
The test was broken on purpose, on 9.3.2, and each change failed it: the scan left out (
//:consumer did not print "OUT: PATCHED ..."), the stand-in serving the upstream bytes under the patched name (the same assertion), the tamper left out (the tampered module built), and a Bazel path that does not exist with the required setting on (a failure, not a skip).What starts on main
bazel-compatibility.yml; each ecosystem has a small caller with its own triggers and path lists, herebazel-go.yml. A change to one ecosystem starts that ecosystem's legs only.*golang*andvendor/go*.rsundercrates/*/src, asgo-compatibility.ymllists them); on a main push that also touchespatch/redirect/,hosted/,commands/or the shared test helpers; nightly; on dispatch.e9one.merge_grouptrigger, not inci-ok. A leg depends on five outside hosts, so a bad day at one of them must not block the queue: github.com and release-assets.githubusercontent.com (the Bazel release and the ruleset archives), dl.google.com (the Go SDK) and, on every run, bcr.bazel.build (the registry) and go.dev (the Go release list). That is what a run asked on macOS arm64; a repository cache saved from main holds the archives and the SDK. The repository variableBAZEL_COMPAT_DISABLED=trueturns every leg off without a merge.SKIPand passes; in the workflowSOCKET_PATCH_BAZEL_E2E_REQUIRED=1makes that a failure, andSOCKET_PATCH_BAZEL_E2E_VERSIONfails a leg whose Bazel is another release than its matrix line names. The run step picks the module's tests by exact name and fails when the module has none. A host that does not answer fails the test everywhere.What a later pull request adds: for one more Bazel release in an existing era, its pin lines and one matrix line; for one more era, one
MODULE.bazeland one line of the era table; for one more ecosystem, its module with one ignored test per flow, its fixture directory, a caller workflow of its own and onemodline inmain.rs. It changes neither the reusable workflow nor another ecosystem's caller. A hermetic test holds each caller's releases and eras to the table in the test code.Not claimed
#![cfg(unix)]).GOPROXY. No request leaves loopback for the package under test.What the first run on Linux showed
Nothing here had run on Linux x86_64 before. The three legs passed without a change: Bazel's sandbox on Ubuntu 24.04, the Go SDK download and the C toolchain rules_go finds all worked as on macOS.
Code review
/code-review highran on the first commit and reported ten findings, none blocking; the second commit answers nine and one is left on purpose.GOPROXYwas pinned, so an exportedGOPRIVATE=patch.socket.devsent gazelle'sgo mod downloadpast the loopback stand-ins to the real host. The settings listed above are now pinned beside it, as variables of the Bazel client and as--repo_env. WithGOPRIVATEandGONOPROXYexported the test passes on all three releases, and fails without the change.HOME. Every Bazel child now gets oneBAZELISK_HOME, the caller's own cache, so a launcher downloads at most once. The pinned, verified path is the installer, which CI uses.curlnow has a connect timeout, a low-speed limit and a total time, and retries only transient failures. A 404 at the first origin falls through in 0.2 s (was 31 s).export "$(scripts/install-bazel.sh <version> <dir>)"; the script prints one line on stdout.SOCKET_PATCH_BAZEL_E2E_BAZELorSOCKET_PATCH_BAZEL_E2E_SOCKET_BINnow fails at once with a message that names the variable.name:now, as in the other compatibility workflows.XDG_CONFIG_HOME,XDG_DATA_HOMEandXDG_STATE_HOMEare removed from every Bazel child. This is read from the Go and rules_go sources; it could not be reproduced on macOS, where Go usesHOME.bazel --versionand its fixture era from one table; a release the table does not hold fails with a message that names the table. The workflow passes the module filtergo::, with no name prefix, name assert or log scrape.commondirectory and oneMODULE.bazelper era; a hermetic test holds the directories to the table and thego.sumto the module the test serves.h1in go.rs (13 lines) and the recursive make-writable walk in harness.rs (17 lines) duplicate helpers in four other test targets. Hoisting them intotests/common/edits unrelated test targets, so it is a follow-up and not part of this pull request.Second round: what the three ecosystem reviews found in common
/code-review highon #2109 (npm), #2110 (PyPI) and #2111 (RubyGems) found no blocker and the same four structural problems in all three. They are fixed here, in the third commit, so that each ecosystem pull request gets smaller. An independent check of that commit found one more defect, fixed in the fourth.harness.rsnow holds the stand-in (start, serve a route, count the requests) and the four steps every hosted flow shares.go.rsuses them and asserts what it asserted before.no_proxyandNO_PROXY, so a machine that needs a proxy for github.com can still run the suite. The Ruby, Bundler, Node, npm, Python, pip and uv settings of a shell no longer reach a Bazel child. With a deadhttp_proxyexported the Go flow passes on 9.3.2.GEM_HOME, thePIP_prefix or the virtualenv scrub each fails it.gradle-compatibility.yml.go::is also insidecargo::, so a latercargomodule would have run in the Go legs. The run step now runs the module's tests by exact name.Not fixed, by choice: a caller's pull-request path list leaves out shared engine code (
patch/redirect/mod.rs,hosted/,commands/). A change there gets its Bazel run on the push to main or in the nightly run. That is the stated price of not running every ecosystem's legs on every shared change.One run of
--test clifailedtelemetry_e2e::hosted_scan_failure_reports_patch_apply_failedon event order and passed on three reruns; that file is not part of this pull request.Decisions for the owner
None beyond approval.
🤖 Generated with Claude Code