Repository navigation
Test RubyGems hosted mode under rules_ruby on real Bazel - #2111
Mikola Lysenko (mikolalysenko) wants to merge 7 commits into
Conversation
The real-Bazel suite gets its RubyGems flow: hosted mode for a gem that rules_ruby's ruby.bundle_fetch reads from Gemfile and Gemfile.lock under Bzlmod, with no gem_checksums in MODULE.bazel and a CHECKSUMS section in the lock, on Bazel 9.3.2, 8.8.2 and 7.7.2 (rules_ruby 1.0.0-rc2 on all three). Tests only: no file under crates/*/src changes, and harness.rs is used as it is. One #[ignore] test, ruby::hosted_bundle_fetch, in a fresh output base: - builds the fixture workspace and reads "OUT: PRISTINE" from `bazel run //:consumer`; - runs `scan --mode hosted --yes`, which rewrites Gemfile (a `source:` option on the gem's declaration) and Gemfile.lock (a GEM section of the gem's own with the patch registry as its remote, and the sha256 of the patched gem in CHECKSUMS) and nothing else; - runs the same two Bazel commands and reads the patched marker, which holds an id made for the run: the gem came from the stand-in patch registry, Bazel changed no file of the checkout, and a `--lockfile_mode=error` build passes; - runs `rollback --yes`: every file of the checkout is the one of before, byte for byte, and the build prints "OUT: PRISTINE" again; - builds the rewritten checkout in a second output base against a patch registry that serves other bytes: the build fails in its `bundle install` action, where Bundler holds the gem to the lock's CHECKSUMS, and the upstream gem is not fetched in its place. Only the lock's CHECKSUMS section verifies the gem. rules_ruby gives Bazel a sha256 for a gem that gem_checksums names and none otherwise, so a lock without the section would build the other bytes. That lock is not this fixture's, and the docs of the test say so. The Socket API, rubygems.org (the fixture's two gems and the info file `rollback` reads) and the patch registry are one loopback wiremock server. Bazel drops a plain-http URL that comes without a checksum, so the lock and the grant keep the https URLs of production, and a downloader config that the test writes into the workspace copy sends the two gems and the patch host to the stand-in. The Bazel registry, the ruleset, the prebuilt Ruby 3.4.10 it downloads, the JDK its rules resolve and the bundler gem come from the network. - crates/socket-patch-cli/tests/e2e_bazel_build/ruby.rs: the flow, the stand-in and the gems. A gem is built in the test (a tar of the specification and of the files, each gzipped as one stored block), so no binary is committed and its sha256 does not depend on a compressor. A hermetic test holds the CHECKSUMS lines of the fixture's lock to the gems the test serves. - crates/socket-patch-cli/tests/fixtures/bazel/rules_ruby/bzlmod: one era for all three releases. The lock lists the `ruby` platform only, so it names no host: both gems are pure Ruby. The .bazelrc names a remote JDK, because rules_ruby's rules resolve a Java runtime, and gives build actions the HOME of the Bazel client, which the harness sets, so Bundler does not look into the home directory of the machine. - .github/workflows/bazel-compatibility.yml: one Linux leg per Bazel release, one macOS leg (9.3.2, nightly and on dispatch), and the RubyGems files in both path filters. Not required for merge. Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
…ygems-rules-ruby The harness now holds the stand-in server, the shared steps of a flow, the downloader config and the environment of every Bazel child, and bazel-compatibility.yml is a reusable workflow with one caller per ecosystem. The conflict in bazel-compatibility.yml is resolved to the harness branch's file. The RubyGems legs move to a caller of their own in the next commit, which also ports ruby.rs to the harness; until then the ruby module does not build against the merged harness. Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
ruby.rs is written against the harness as it is now. The stand-in server, the four steps of a flow and the downloader config are the harness's, and the module's own copies are gone: StandIn, NOTHING, the assertions on the scan, the patched build and the rollback, and route_downloads, which parsed the release a second time and rewrote .bazelrc. The flow asserts what it asserted before. Every Bazel child now runs without the Ruby and Bundler settings of the shell and with loopback in no_proxy, which is the harness's doing: the flow passes with RUBYOPT=-rnonexistent_library, BUNDLE_GEMFILE and GEM_HOME set to paths that do not exist, and with http_proxy set to a closed port, and fails in both cases when the harness's lines are taken out. .github/workflows/bazel-ruby.yml is the RubyGems caller of bazel-compatibility.yml, which is the harness's file again: one Linux leg per Bazel release (9.3.2, 8.8.2, 7.7.2) and one macOS leg, nightly and on demand. All of them build the fixture's one era, so they share one repository cache. The pull request paths name the RubyGems files the flow runs through, each by its path: formats/gem, lock discovery, the lock inventory, the Ruby crawler, the upstream restore and the stale-install guard. A `*gem*` pattern would also start the legs for the vendored backend and for the native extension rebuild of agent mode. rewrite_gem and its planner live in patch/redirect/mod.rs, the file of every ecosystem's rewriter; like the rest of the shared engine it gets its run on the push to main and in the nightly run. The fixture no longer downloads a JDK. rules_ruby's rb_binary and rb_bundle_install name @bazel_tools//tools/jdk:runtime_toolchain_type as a required toolchain and read it under JRuby only. The fixture's .bazelrc named remotejdk_21 for it, a download of about 200 MB that no action ran; without the line Bazel takes the JDK of the machine and stops at "Cannot find Java binary" where there is none. The fixture now registers a Java runtime toolchain with nothing behind it (no_java.bzl) ahead of those, so a build needs no JDK from anywhere. On macOS arm64 the repository cache after a 9.3.2 run goes from 269 MB to 72 MB, and is 83 MB once 8.8.2 and 7.7.2 have added what they resolve. A gem is still built in the test. Its gzip members come from flate2's GzBuilder at no compression, in place of a member assembled by hand that could hold 65,535 bytes at most. They are stored as before and differ in one header byte, so the two sums in the fixture's Gemfile.lock change; the hermetic test holds them to the gems the test serves. A sha256 is common::sha256_hex. The gem builder stays beside real_gem of e2e_redirect_gem_stale_install.rs, which builds a gem the same way in a test target of its own; sharing the two is left for later. Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
…helpers again One call to sha256_hex pulled every file of tests/common into the target a second time. The flow now hashes inline, as the Go flow does. Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
…he fixture The fixture's Gemfile.lock held the sha256 of the two upstream gems, which the test builds with flate2 and tar. A release of either crate that emits other bytes for the same input would have failed the hermetic test on a pull request that only bumps a dependency. The lock now holds a placeholder for each sum (64 zeros, the sum of no file), and the flow writes the sums of the gems it built into the lock of its workspace copy before the first build. A build on the fixture's own lock fails in Bundler, so no run passes without that step. The upstream gem now prints an id made for the run, as the patched gem does, so the unpatched build can only print its marker from the gem that this run's stand-in served. The hermetic test that held the two committed sums now holds what is still fixed: the fixture's directories and the legs of its workflow are those of the era table, and the lock's CHECKSUMS section is one placeholder line per gem the test serves, so that a pinned lock holds those two sums and no other. The format of a CHECKSUMS line is written once (`checksum`), for the pinned lock, for the assertions on the rewritten lock and for the hermetic test. The module docs said that only Bundler before 2.6 writes a lock without a CHECKSUMS section. Bundler 2.6 and 2.7 write none either unless asked (`bundle lock --add-checksums`, or the `lockfile_checksums` setting for a new lock). A new lock gets the section by default from Bundler 4.0.0 on, and a lock that exists keeps what it has (feature_flag.rb and definition.rb of Bundler 2.6.9, settings.rb and CHANGELOG.md of Bundler 4.0.15). The docs now say that the fixture's lock has the section, and that the flow proves nothing for a lock without one, where the build takes whatever bytes the patch registry serves. bazel-ruby.yml: the nightly run gets a concurrency group of its own, as in bazel-go.yml, so that a second push to main cannot replace a nightly that waits behind the first; the nightly is the only run with the macOS leg. Six path patterns that named a directory beside a source file (`vex/discover/gem/**` and the like) matched no file and are gone: both lists now name only paths that exist. Tests and one workflow only; no file under crates/*/src changes. Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
PR SummaryLow Risk Overview Introduces a Adds Reviewed by Cursor Bugbot for commit fa284e1. Configure here. |
|
[agent] PR shepherd: review brief for What it does: It runs a hosted Risk: low. It adds only tests and an optional, non-required workflow. The unusual part is the empty Java-toolchain stub ( Look here
Verified: merged onto its base in a scratch tree, the non-ignored tests pass and clippy is clean. The selection picks Changes I made: none. Open questions (minor)
Stacking and merge order: this PR's base is Shared notes (#2109–#2111): none of these PRs touches Generated by Claude Code |
No user-visible change.
What this is
Tests only. The real-Bazel suite of #2107 (
e2e_bazel_build, run by the reusable workflowbazel-compatibility.yml) gets its RubyGems flow: hosted mode for a gem that rules_ruby 1.0.0-rc2 reads throughruby.bundle_fetchfromGemfileandGemfile.lockunder Bzlmod, with nogem_checksumsinMODULE.bazeland a lock that has aCHECKSUMSsection.No file under
crates/*/srcchanges.harness.rs,go.rs,bazel-compatibility.ymlandbazel-go.ymlare #2107's and are not changed here.Contents
10 files, 609 added lines, none removed, against #2107's branch at f1aa02b (
git diff --shortstat). Implementation: 0 lines. Tests and workflow: 524 lines. Fixture: 85 lines. No binary file.crates/socket-patch-cli/tests/e2e_bazel_build/ruby.rscrates/socket-patch-cli/tests/e2e_bazel_build/main.rsmod ruby;.github/workflows/bazel-ruby.ymlcrates/socket-patch-cli/tests/fixtures/bazel/rules_ruby/bzlmod/common/.bazelrc4,BUILD.bazel23,Gemfile4,Gemfile.lock19,consumer.rb3,no_java.bzl6crates/socket-patch-cli/tests/fixtures/bazel/rules_ruby/bzlmod/e9/MODULE.bazelruby.bundle_fetch. One era for all three Bazel releases.About the fixture:
Gemfile.lockholds a placeholder, 64 zeros, as the sha256 of each of its two gems, and the test writes the sums of the gems it built into the lock of its workspace copy before the first build. A build on the fixture's own lock fails in Bundler, so no run passes without that step, and no committed value depends on the bytes that the gzip and tar libraries of the test emit.CHECKSUMSsection and names Bundler 4.0.15, which rules_ruby downloads and runs.rb_binaryandrb_bundle_installrequire a Java runtime toolchain that only JRuby reads. The fixture registers one with nothing behind it, so no build downloads a JDK or needs one on the machine..bazelrcis one setting,build --action_env=HOME: Bundler runs in a build action, and with it the action gets the home directory of the Bazel client, which in a test is a scratch directory.What it proves, and where
One
#[ignore]test,ruby::hosted_bundle_fetch, on Bazel 9.3.2, 8.8.2 and 7.7.2, each in a fresh output base. The gem under test,sp-victim1.0.0, prints one line throughbazel run //:consumer. Both the upstream gem and the patched gem print an id made for the run, so no cache of an earlier run can supply either.bazel buildandbazel run //:consumerprintOUT: PRISTINE <run id>, Bazel wrote itsMODULE.bazel.lock, and Bazel fetched the upstream gem from the rubygems.org stand-in.scan --mode hosted --yes. Exit 0, oneappliedevent for the gem with the patch's id, andGemfileandGemfile.lockare the files it names and the only files of the checkout that changed. The gem's declaration gainssource: "<patch registry>". The lock gets aGEMsection of the gem's own with the patch registry as its remote, and the gem'sCHECKSUMSline holds the sha256 of the patched gem in place of the upstream one. The command did not fetch the patched gem.OUT: PATCHED <patch id>. Bazel fetched the patched gem from the patch-registry stand-in and did not fetch the upstream gem again. Bazel changed no file, its own lock included, and a--lockfile_mode=errorbuild passes.rollback --yes. Exit 0 with one package rolled back. Every file of the checkout is the one of before, byte for byte, right after the command and again after the next build, which printsOUT: PRISTINE <run id>.GemfileandGemfile.lock, and the patch registry serves another gem under the same name. The build fails in thebundle installaction with Bundler'smismatched checksums, which names the lock's sha256 and the file's. The patch registry was asked, and the upstream gem was not fetched in its place.A second test needs no Bazel and runs with the workspace's tests in ci.yml: the fixture's directories and the legs of
bazel-ruby.ymlare those of the era table in the test code, and theCHECKSUMSsection of the fixture's lock is one placeholder line per gem the test serves.Runs on macOS arm64, at this head (fa284e1), with the three release binaries
scripts/bazel-pins.txtpins, run as the workflow's run step runs them: list the ignored tests, keep the names that start withruby::, run them by exact name withSOCKET_PATCH_BAZEL_E2E_REQUIRED=1and the release inSOCKET_PATCH_BAZEL_E2E_VERSION. Seconds are those of the test command once the test target is built.The 139 seconds are one run; the machine was shared and busy, and the cause was not looked into.
Control runs on 9.3.2 at this head. Each is one temporary edit that must fail the test, restored afterwards:
mismatched checksums.//:consumer did not print "OUT: PRISTINE <run id>".Three more control runs come from the first review round, on an earlier revision of this branch whose assertions for those steps are the same as now. They were not repeated at this head: the scan step left out (the patched marker is not printed), the upstream bytes served under the patched name (Bundler's checksum stops step (c)), and the other bytes of step (e) left out (
the tampered gem built).Linux x86_64 (the CI runner): the flow passed on the first run there, on all three releases (run 44062759795; checks
ruby / bazel (9.3.2, e9),(8.8.2, e9),(7.7.2, e9)). Test time on the runner, with no cache saved yet: 68 s on 9.3.2, 40 s on 8.8.2, 44 s on 7.7.2.What starts on main
.github/workflows/bazel-ruby.ymlcalls the reusablebazel-compatibility.ymlwith therubymodule and therules_ruby/bzlmodfixture.e9; checks are namedruby / bazel (9.3.2, e9)and so on. One macOS leg on 9.3.2, nightly and on demand only.scripts/install-bazel.shorscripts/bazel-pins.txt,ruby.rsorharness.rs, the rules_ruby fixture, or one of the RubyGems source files the flow runs through, each named by its path:formats/gem/**(the readers of a lock and a Gemfile, and the lock half of the hosted rewrite),vex/discover/gem.rs,vendor/lock_inventory/gem.rs,crawlers/ruby_crawler.rs,patch/redirect/upstream/gem.rs(the restore ofrollback), andgem_refresh.rsin both crates. Today that is 13 patterns and 27 files, 14 of them source files. Nothing another ecosystem's flow owns is in the list.main.rsof the test target,vendor/gem.rs,patch/redirect/**,hosted/**, the CLI'scommands/**,tests/common/**,tests/hosted_memory_common/**,Cargo.lock, theCargo.tomlfiles,rust-toolchain.toml,.cargo/**. So a change to the shared engine code,rewrite_geminpatch/redirect/mod.rsincluded, gets its Ruby run after it is on main and not on its pull request.workflow_dispatchfor any branch. The nightly run has a concurrency group of its own, so a second push to main cannot replace a nightly that waits behind the first.merge_grouptrigger, and no job is aneedsof ci.yml'sci-ok. Setting the repository variableBAZEL_COMPAT_DISABLEDtotrueturns every leg off.Not claimed
CHECKSUMSsection. The fixture's lock is one with the section, and the flow proves that a build fails on other bytes only for such a lock. See the first item of the next section for what happens without one.MODULE.bazelthat pins the gem ingem_checksums. A transitive gem, a gem with platform variants or a native extension. WORKSPACErb_bundle_fetch. Another rules_ruby than 1.0.0-rc2, another Ruby than 3.4.10, another Bundler than 4.0.15, a lock that Bundler 2 wrote.httpsURLs of production, and a downloader config that the harness names on every Bazel command sends them to the stand-in.socket-patch. The tests run the binary cargo builds from this branch.bundle lockorbundle installby hand, outside Bazel, after the scan.Known behaviour the flow records
What a user of hosted mode under rules_ruby meets. Where an item was seen in a one-off run and is not asserted by the committed test, it says so.
CHECKSUMSsection is not verified, and the build takes whatever bytes the patch host serves. rules_ruby gives Bazel's downloader a sha256 only for a gem thatgem_checksumsnames, so only Bundler checks the file, and only against aCHECKSUMSline. Without the section,scan --mode hostedstill applies the patch (exit 0) with the warningredirect_gem_no_checksums_section, and the gem is pinned by its source alone. Seen once on Bazel 9.3.2 with a temporary variant of the flow that is not in this diff: a second output base built and ran the other bytes (OUT: TAMPERED). It is the lock Bundler 2 writes unless it is asked for the section. Read in Bundler's sources: in 2.6.9, the Bundler of Ruby 3.4.10,lockfile_checksumsis off (feature_flag.rb:settings_flag(:lockfile_checksums) { bundler_3_mode? }), and a lock that exists keeps what it has; 2.6.0 added the ways to ask for the section (bundle lock --add-checksums, or thelockfile_checksumssetting for a new lock); 4.0.0 made it the default for a new lock (its changelog, and"BUNDLE_LOCKFILE_CHECKSUMS" => truein 4.0.15'ssettings.rb). The warning's own text says "bundler < 2.6 does not write one", which leaves out that 2.6 and 2.7 do not write one unless asked.bundle installaction withBundler found mismatched checksums, and the message also names Bundler's own switch to turn the check off.scanrewrites the gem's line inGemfileas well asGemfile.lock. The patch registry URL it writes into both carries the token and the patch id of the grant.MODULE.bazelandMODULE.bazel.lockstay as they are, no Bazel command is needed between the scan and the build, and a--lockfile_mode=errorbuild passes.rollbackneeds the gem index. It readsinfo/<gem>of rubygems.org to restore the upstreamCHECKSUMSline; the test answers it from the stand-in throughSOCKET_RUBYGEMS_URL. Seen once on 9.3.2 with the answer taken away:rollbackexits 1 withhosted_restore_refused, reports no edited file, and namesgit checkout -- Gemfile Gemfile.lockas the way out.Cannot write a changed lockfile while frozen.Seen in the output of the unpatched and of the patched build on 9.3.2; the test does not assert it. It is printed before the scan too, so the patch does not cause it, and it is not an error. A lock without the section did not print it.gem_checksums. After the scan the hint names the sum of the patched gem. A workspace that follows it is thegem_checksumsshape that this flow does not cover.Code review
First round
GEM_HOME,GEM_PATH,BUNDLE_*,RUBYOPTandRUBYLIBfrom every Bazel child. The flow passed on 9.3.2 with hostile values exported and failed in step (a) with the harness's scrub taken out.http_proxy. The harness adds loopback tono_proxyandNO_PROXY. The flow passed with a deadhttp_proxyexported and failed with those lines taken out.rewrite_gemand its planner are inpatch/redirect/mod.rs, the file of every ecosystem's rewriter, which stays out of the pull request list so that a change to another ecosystem does not start the Ruby legs. It is covered by the push to main and the nightly run. See the decisions below.bazel-ruby.yml.ruby.rsuses the harness's and asserts what it asserted before..bazelrcline into the workspace. Deleted; the flow uses the harness's downloader config, which lives beside the workspace copy.remotejdk_21. The repository cache of a 9.3.2 run on macOS arm64 went from 269 MB to 72 MB, and the releases now share one cache.sha256()duplicated a shared test helper. Left as three lines in the module, over the same two crates the Go flow uses: the shared helper lives intests/common/mod.rs, which this test target does not load, and loading it for one function compiled three shared files a second time.gzip()assembled a gzip member by hand and failed above 65,535 bytes. It isflate2::GzBuilderat no compression.Second round
CHECKSUMSsection. Wrong, as read in Bundler 2.6.9 and 4.0.15 (see above). The docs, 'Not claimed' and 'Known behaviour' now say which lock the fixture is and what the flow does not prove.CHECKSUMSentry was written twice. One function writes it, for the pinned lock, the assertions on the rewritten lock and the test without Bazel.gem(),archive()andgzip()are one more in-test gem builder besidereal_gemine2e_redirect_gem_stale_install.rsand themake_gemfunctions of the vendor tests. Not fixed here: a shared builder undertests/commonwould change other test targets, which a pull request that adds one flow should not do. A comment atgem()namesreal_gem.Two more changes in this round did not come from a finding. The nightly run of the caller has a concurrency group of its own, as in
bazel-go.yml. And six path patterns that named a directory beside a source file (vex/discover/gem/**and the like) matched no file and are removed, so both lists name only paths that exist; if one of those files is later split into a directory, its pattern has to follow.Decisions for the owner
patch/redirect/mod.rsstart the Ruby legs on its pull request?rewrite_gemlives there. It is one line inbazel-ruby.yml. The price, as counted in the first round and not recounted: about 40 commits to main in a month touch that file and no RubyGems file, and each would run three legs.CHECKSUMSsection. Today hosted mode applies with a warning, and the build then takes whatever the patch host serves. Whether that should stay so, and the wording of the warning, are product questions outside this pull request.🤖 Generated with Claude Code