Skip to content
4 changes: 4 additions & 0 deletions .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -30,6 +30,7 @@
!/assets/fleet/
!/bin/
!/docs/
!/docs/dynamic-sbom-inference-settings.md
!/patches/
!/scripts/
!/scripts/build-package-matrix.mts
Expand Down Expand Up @@ -556,6 +557,8 @@
!/src/commands/manifest/bazel/extract_bazel_to_pypi.constructed.test.mts
!/src/commands/manifest/bazel/extract_bazel_to_pypi.mts
!/src/commands/manifest/bazel/extract_bazel_to_pypi.test.mts
!/src/commands/manifest/build-root-config.mts
!/src/commands/manifest/build-root-config.test.mts
!/src/commands/manifest/cmd-manifest-auto.mts
!/src/commands/manifest/cmd-manifest-auto.test.mts
!/src/commands/manifest/cmd-manifest-cdxgen.mts
Expand Down Expand Up @@ -603,6 +606,7 @@
!/src/commands/manifest/handle-manifest-dynamic-sbom-inference.test.mts
!/src/commands/manifest/handle-manifest-setup.mts
!/src/commands/manifest/init.gradle
!/src/commands/manifest/load-build.mts
!/src/commands/manifest/manifest-flags.mts
!/src/commands/manifest/output-manifest-dynamic-sbom-inference.mts
!/src/commands/manifest/output-manifest-setup.mts
Expand Down
10 changes: 10 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -6,7 +6,17 @@ The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.1.0/).

## [Unreleased]

### Added
- `socket manifest setup --dynamic-sbom-inference` can skip any build: one Socket can load is skipped together with its modules, leaving the builds beside and beneath it alone. The setup shows the files it will write before writing them.
- Configure or skip one build of a directory on its own, with `socket.json` settings under `defaults.manifest.builds` keyed by its build file name.

### Changed
- `socket.json` settings apply from every directory above a scan's starting point, not only from that directory down.
- `--dynamic-sbom-inference` builds a module that lives outside its reactor's directory (such as Maven's `<module>../core</module>`) only inside that reactor, instead of also building it on its own.
- A build that fails to generate Socket facts now stops only the builds in and beneath its directory, and its failure is ignored when another build that includes it succeeds.

### Fixed
- `--dynamic-sbom-inference` builds a Gradle or sbt build nested in another build on its own as well, so the projects only its own settings include are no longer missed.
- Maven Socket facts honor artifact types that build extensions define, so they no longer list dependencies Maven doesn't resolve, or fetch files under the wrong extension.
- Maven Socket facts no longer drop dependencies whose scope Maven 3 doesn't define, such as Maven 4's `compile-only` or `none`.

Expand Down
147 changes: 147 additions & 0 deletions docs/dynamic-sbom-inference-settings.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,147 @@
# Build settings for `--dynamic-sbom-inference`

How to write the `socket.json` settings that `--dynamic-sbom-inference` reads,
by hand or with an agent. `socket manifest setup --dynamic-sbom-inference`
asks the same questions interactively and writes the same files.

## What the scan does

1. Finds every file that looks like a build: `pom.xml` and its variants
(`*-*.pom.xml`, `pom`), `.sln`, `.slnx`, `.csproj`, `.fsproj`, `.vbproj`,
`build.gradle(.kts)`, `settings.gradle(.kts)`, `build.sbt`. It honours
`.gitignore`, `socket.yml` ignores and `--exclude-paths`.
2. Runs each build's own tool (Maven, Gradle, sbt, the .NET SDK) to resolve
its dependencies, unless another build already loads it. A Maven reactor
builds its modules, including ones outside its directory such as
`<module>../core</module>`; a Gradle or sbt build builds its subprojects.
3. Fails if any build it runs fails, naming the build.

Settings tell the build tools how to run each build, and which builds not to
run.

## Prefer builds Socket can load

Socket learns what a build includes only by letting its tool load it: Maven
profiles, Gradle settings scripts and sbt definitions are code. When a build
loads:

- each module is built once, inside the build that owns it;
- skipping the build skips exactly it and its modules, wherever they are, and
leaves the builds next to and beneath it alone.

So fix a build that fails to load (usually a JDK, build tool or option) before
reaching for a skip.

## Where settings live

- Any directory may hold a `socket.json`. Its settings apply to that directory
and everything beneath it.
- Every `socket.json` from the filesystem root down to a build's directory
applies; for each field the nearest one wins. `null` clears an inherited
value.
- `defaults.manifest.builds` in a directory's `socket.json` holds settings for
one build of that directory, keyed by its build file name. They apply after
all directory settings, to that build alone.

```json
{
"version": 1,
"defaults": {
"manifest": {
"maven": { "javaHome": "$JAVA17_HOME" },
"builds": {
"pom.xml": { "disabled": true },
"tools-cli.pom.xml": { "mavenOpts": "-Pcli" }
}
}
}
}
```

A Gradle or sbt build is keyed by any one of its build files that exists
(`settings.gradle`, `build.gradle.kts`, `build.sbt`, ...); two entries for the
same build are an error.

## Fields

Under `defaults.manifest.maven`, `.gradle`, `.sbt` or `.dotnet`, or inside a
`builds` entry:

| Field | Meaning |
|---|---|
| `bin` | Build tool command. Relative paths are relative to the build's directory. Defaults to the project's wrapper (`mvnw`, `gradlew`) if present, else `mvn`, `gradle`, `sbt`, `dotnet`. |
| `javaHome` | JDK for Maven, Gradle and sbt. `$VAR` and `${VAR}` expand from the environment. |
| `mavenOpts`, `gradleOpts`, `sbtOpts`, `dotnetOpts` | Arguments passed to the build tool, for example profiles or `-p:Configuration=Release`. Maven's `-f`/`--file` is ignored: each POM is built by its own file. |
| `includeConfigs`, `excludeConfigs` | Comma-separated globs over Maven scopes, Gradle and sbt configurations, or .NET target frameworks. |
| `ignoreUnresolved` | `true` warns about dependencies that don't resolve instead of failing. |
| `disabled` | `true` skips builds; see below. |
| `facts` | Gradle and sbt: `false` skips the build. |
| `verbose` | `true` streams the build tool's output. |

## Skipping builds

| To skip | Set | Effect |
|---|---|---|
| One build and what it loads | `builds["<build file>"].disabled: true` in its directory | Socket still loads the build, resolving nothing, to find its modules. If it can't load it, the scan fails. |
| A directory and everything beneath it | `defaults.manifest.<tool>.disabled: true` in that directory | Nothing is loaded. A nearer `socket.json` with `disabled: false` turns builds beneath it back on. |

Don't use `--exclude-paths` to skip builds: it also removes those files from
the scan and from reachability analysis.

Socket can't skip a module of a build that runs: the build resolves its
modules itself. Settings for such a module are ignored, with a warning.

## Procedure for an agent

1. Run `socket manifest dynamic-sbom-inference <dir> --json` from the
repository root. It runs every build and lists each with a status:

| Status | Meaning |
|---|---|
| `generated` | Built. |
| `empty` | Built; nothing to resolve. |
| `skippedCovered` | Built as part of another build. |
| `skippedDisabled` | Skipped by a setting, or loaded by a skipped build. |
| `failed` | Its build tool failed; the output above it says why. |
| `aborted` | Not run because a build in or above its directory failed. |

2. For each `failed` build, read the build tool's error and fix the cause in
the nearest fitting `socket.json`: `javaHome` for a wrong JDK, `bin` for a
missing tool, `<tool>Opts` for missing profiles or properties.
3. Skip a build only when it is not meant to be built (a test fixture, a
template, an abandoned project), using the table above. Prefer
`builds["<file>"].disabled` when the build loads; use the directory form
only when it doesn't.
4. Repeat until no build is `failed` or `aborted`.
5. Commit the `socket.json` files.

## Example

```
repo/
pom.xml reactor with modules api, web
codegen/
pom.xml not shipped: skip it
codegen-cli.pom.xml shipped
legacy/
pom.xml doesn't load any more, with modules old-core, old-ui
tool/pom.xml separate build, shipped
```

`codegen/socket.json` skips one of two builds in the directory:

```json
{ "version": 1, "defaults": { "manifest": { "builds": { "pom.xml": { "disabled": true } } } } }
```

`legacy/pom.xml` doesn't load, so `legacy/socket.json` skips the directory:

```json
{ "version": 1, "defaults": { "manifest": { "maven": { "disabled": true } } } }
```

and `legacy/tool/socket.json` turns the build beneath it back on:

```json
{ "version": 1, "defaults": { "manifest": { "maven": { "disabled": false } } } }
```
7 changes: 6 additions & 1 deletion src/commands/fix/coana-fix.mts
Original file line number Diff line number Diff line change
Expand Up @@ -56,6 +56,7 @@ import { getPackageFilesForScan } from '../../utils/path-resolve.mts'
import { setupSdk } from '../../utils/sdk.mts'
import { excludePathToScanIgnores } from '../scan/exclude-paths.mts'
import { fetchSupportedScanFileNames } from '../scan/fetch-supported-scan-file-names.mts'
import { withoutDiscardedFacts } from '../scan/run-dynamic-sbom-inference.mts'

import type { GeneratedSocketFacts } from './generated-socket-facts.mts'
import type { FixConfig } from './types.mts'
Expand Down Expand Up @@ -390,7 +391,7 @@ async function coanaFixWithFacts(
outputPath: DOT_SOCKET_DOT_FACTS_JSON,
}),
)
const scanFilepaths = await findScanFilepaths()
let scanFilepaths = await findScanFilepaths()
if (factsSlot) {
if (!silence) {
spinner?.stop()
Expand All @@ -404,6 +405,10 @@ async function coanaFixWithFacts(
(await findScanFilepaths()).filter(isSocketFactsFile),
tmpDir: factsSlot.tmpDir,
})
scanFilepaths = await withoutDiscardedFacts(
scanFilepaths,
factsSlot.generated.discardedPaths,
)
const known = new Set(scanFilepaths)
scanFilepaths.push(...factsSlot.generated.paths.filter(p => !known.has(p)))
if (!silence) {
Expand Down
28 changes: 19 additions & 9 deletions src/commands/fix/generated-socket-facts.mts
Original file line number Diff line number Diff line change
Expand Up @@ -9,6 +9,8 @@ import type { PURL_Type } from '../../utils/ecosystem.mts'
export type GeneratedSocketFacts = {
// Coana's flags selecting what the facts files resolve.
artifactsFlags: string[]
// Facts files of dropped runs, which Coana must not read.
discardedPaths: string[]
paths: string[]
// The facts files' per-project classpaths, written outside the repository.
sidecarFile: string
Expand Down Expand Up @@ -60,18 +62,24 @@ export async function generateSocketFactsForFix({
await restoreAll(originals)
}
let artifactsFlags: string[]
let discardedPaths: string[]
let paths: string[]
const sidecarFile = path.join(tmpDir, 'sidecar.json')
try {
const { factsOnlyTypes, factsPaths, resolvedPathsSidecar } =
await runDynamicSbomInference({
consumer: 'fix',
cwd,
excludePaths,
purlTypes,
sbtTmpDir: undefined,
})
const {
discardedFactsPaths,
factsOnlyTypes,
factsPaths,
resolvedPathsSidecar,
} = await runDynamicSbomInference({
consumer: 'fix',
cwd,
excludePaths,
purlTypes,
sbtTmpDir: undefined,
})
artifactsFlags = useOnlySocketFactsArtifactsFlags(factsOnlyTypes)
discardedPaths = discardedFactsPaths
paths = factsPaths.map(p => path.resolve(cwd, p))
// Written even when empty, as for reach; applying fixes looks each facts
// file up in it, so an empty one is the same as none.
Expand All @@ -88,9 +96,11 @@ export async function generateSocketFactsForFix({
]
return {
artifactsFlags,
discardedPaths,
paths,
sidecarFile,
remove: () => reset(paths),
// A discarded file a run left as written is removed too.
remove: () => reset([...paths, ...discardedPaths]),
restore: () => restoreAll(backups),
}
}
32 changes: 30 additions & 2 deletions src/commands/fix/generated-socket-facts.test.mts
Original file line number Diff line number Diff line change
Expand Up @@ -46,7 +46,11 @@ describe('generateSocketFactsForFix', () => {
mockRunDynamicSbomInference.mockImplementation(async () => {
await fs.writeFile(existing, 'regenerated')
await fs.writeFile(fresh, 'generated')
return { factsOnlyTypes: ['maven'], factsPaths: [existing, fresh] }
return {
discardedFactsPaths: [],
factsOnlyTypes: ['maven'],
factsPaths: [existing, fresh],
}
})

const generated = await generate()
Expand All @@ -63,10 +67,29 @@ describe('generateSocketFactsForFix', () => {
expect(existsSync(fresh)).toBe(false)
})

it('removes a discarded facts file a dropped run left behind', async () => {
const discarded = path.join(cwd, 'sub', 'pom.xml.socket.facts.json')
mockRunDynamicSbomInference.mockImplementation(async () => {
await fs.writeFile(discarded, 'dropped')
return {
discardedFactsPaths: [discarded, existing],
factsOnlyTypes: ['maven'],
factsPaths: [],
}
})

const generated = await generate()
await generated.remove()

expect(existsSync(discarded)).toBe(false)
expect(await fs.readFile(existing, 'utf8')).toBe('original')
})

it('writes the sidecar outside the repository', async () => {
mockRunDynamicSbomInference.mockImplementation(async () => {
await fs.writeFile(fresh, 'generated')
return {
discardedFactsPaths: [],
factsOnlyTypes: ['maven', 'nuget'],
factsPaths: [fresh],
resolvedPathsSidecar: { [fresh]: { components: [], projects: [] } },
Expand All @@ -90,6 +113,7 @@ describe('generateSocketFactsForFix', () => {

it('writes an empty sidecar when no build root resolved any paths', async () => {
mockRunDynamicSbomInference.mockResolvedValue({
discardedFactsPaths: [],
factsOnlyTypes: ['maven'],
factsPaths: [],
})
Expand All @@ -103,7 +127,11 @@ describe('generateSocketFactsForFix', () => {
it('restores an existing facts file that generation does not rewrite', async () => {
mockRunDynamicSbomInference.mockImplementation(async () => {
await fs.writeFile(fresh, 'generated')
return { factsOnlyTypes: ['maven'], factsPaths: [fresh] }
return {
discardedFactsPaths: [],
factsOnlyTypes: ['maven'],
factsPaths: [fresh],
}
})

const generated = await generate()
Expand Down
5 changes: 5 additions & 0 deletions src/commands/manifest/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -167,6 +167,11 @@ Generates a manifest file (`pom.xml`) from Scala's `build.sbt` file.
Starts an interactive configurator that writes default flag values for
`socket manifest` into a `socket.json` in the current directory.

With `--dynamic-sbom-inference`, it walks every build beneath the current
directory and writes the settings for each. The settings themselves are
documented in [`docs/dynamic-sbom-inference-settings.md`](../../../docs/dynamic-sbom-inference-settings.md),
for anyone, or any agent, writing them without the configurator.

## Dev

Run it like these examples:
Expand Down
Loading
Loading