Skip to content

fix: route buildkit dns through host's systemd-resolved cache#63

Merged
pbardea merged 5 commits intomainfrom
pb/dns-local-cache
Feb 13, 2026
Merged

fix: route buildkit dns through host's systemd-resolved cache#63
pbardea merged 5 commits intomainfrom
pb/dns-local-cache

Conversation

@pbardea
Copy link
Contributor

@pbardea pbardea commented Feb 12, 2026

Summary

  • BuildKit on Blacksmith runners currently hardcodes public DNS servers (8.8.8.8, 1.1.1.1, etc.) in buildkitd.toml, bypassing the host's systemd-resolved DNS cache
  • This causes DNS cache misses and EAI_AGAIN/ETIMEOUT errors during parallel Docker builds with heavy DNS traffic (e.g. pnpm install in monorepos)
  • This PR configures systemd-resolved to listen on the host's routable IP (not just loopback 127.0.0.53), then injects that IP into buildkitd.toml's [dns] section
  • Build containers on any network mode (host, bridge, custom) can now reach the host's DNS cache, matching Docker's built-in behavior for the docker-container driver
  • Public DNS servers are kept as fallback if the routable IP can't be determined

See: moby/buildkit#5009

Made with Cursor


Note

Medium Risk
Touches host-level DNS configuration and BuildKit daemon startup behavior; misconfiguration could disrupt DNS resolution on runners or in builds.

Overview
BuildKit DNS is reworked to prefer the host’s systemd-resolved cache instead of hardcoded public resolvers.

startBuildkitd now (1) installs a systemd-resolved drop-in to listen on a routable interface (DNSStubListenerExtra=0.0.0.0) and restarts the service, and (2) writes buildkitd.toml with a [dns] nameservers list set to the host’s primary routable IP (derived via ip route get), falling back to public DNS only when the host IP can’t be determined.

Written by Cursor Bugbot for commit 5d82af7. This will update automatically on new commits. Configure here.

@pbardea
Copy link
Contributor Author

pbardea commented Feb 12, 2026

@codesmith fix ci

@blacksmith-staging
Copy link

I'm now helping with this PR. Give me a moment to analyze the code and I'll respond shortly.

@linear
Copy link

linear bot commented Feb 12, 2026

@blacksmith-staging
Copy link

@pbardea

fix ci

Pushed the fix. All 4 CI failures (Build and Test, Lint, Verify Build Output, Test Action) were caused by the same issue: package.json had a packageManager: "pnpm@10.29.3" field (likely auto-added by Cursor) that conflicted with the version: 8.15.9 specified in all CI workflow files. pnpm/action-setup@v4 rejects this mismatch.

Removed the packageManager field to restore package.json to match main. CI should pass now.

@pbardea
Copy link
Contributor Author

pbardea commented Feb 13, 2026

@pbardea pbardea requested a review from adityamaru February 13, 2026 15:03
BuildKit falls back to hardcoded public DNS (8.8.8.8/8.8.4.4) when the
host uses systemd-resolved (127.0.0.53), because that loopback address
is unreachable from build containers in their own network namespace.

This causes DNS cache misses and EAI_AGAIN/ETIMEOUT errors during
parallel builds that make many DNS queries (e.g. pnpm install).

Fix:
1. Configure systemd-resolved to listen on all interfaces (0.0.0.0)
   via a drop-in config, not just loopback
2. Resolve the host's routable IP at startup and inject it into
   buildkitd.toml's [dns] nameservers
3. Keep public DNS as fallback if the host IP can't be determined

This makes the host's DNS cache reachable from build containers on
any network mode (host, bridge, custom), matching Docker's built-in
behavior for the docker-container driver.

See: moby/buildkit#5009
BuildKit round-robins across all nameservers rather than using them
as ordered fallbacks. Including public DNS alongside the host IP
caused ~50% of queries to bypass the local cache.

Now only the host's routable IP (backed by systemd-resolved) is used.
systemd-resolved itself already handles upstream fallback to external
resolvers when needed.
Regenerate dist artifacts with ncc after rebasing onto origin/main.
@pbardea pbardea merged commit 6078dc1 into main Feb 13, 2026
10 checks passed
@pbardea
Copy link
Contributor Author

pbardea commented Feb 19, 2026

ubuntu@paul-dev-131:~$ ip route get 1.1.1.1
1.1.1.1 via 131.153.225.122 dev bond0.65 src 131.153.225.123 uid 1000
    cache
    ```
    
    for context

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants

Comments