Commit Graph

84 Commits

Author SHA1 Message Date
CrazyMax
af24d72dd8
kubernetes: fix context load test
Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
2022-12-05 17:13:03 +01:00
CrazyMax
3f6a5ab6ba
kubernetes: move context config logic to its own pkg
Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
2022-11-17 00:34:42 +01:00
Justin Chadwell
7f008a7d1e driver: don't create tracer delegate opt if tracer is nil
The error handling for the cast to client.TracerDelegate was incorrect,
and previously, a client would unconditionally append an opt.

This results in the scenario that while the ClientOpt was not nil, the
tracer delegate in the ClientOpt was, which isn't an error case
explicitly handled by buildkit.

Signed-off-by: Justin Chadwell <me@jedevc.com>
2022-10-31 11:19:47 +00:00
Justin Chadwell
33ef1b3a30 docker-container: move userns detection into driver
This moves the detection of the docker daemon's security options into
the driver from the factory, handling them in a similar way to how we do
cgroups.

Because of recent changes that modify error detection in driver
creation, this attempt to contact the docker daemon during builder
creation meant that a docker-container builder could not be created
without access to the docker socket. This patch resolves this, by
defering the Info call to the driver, when the container is actually
created.

Signed-off-by: Justin Chadwell <me@jedevc.com>
2022-10-21 13:44:44 +01:00
Justin Chadwell
a6caf4b948 chore: tidy up duplicate dockertypes import
Signed-off-by: Justin Chadwell <me@jedevc.com>
2022-10-21 13:40:17 +01:00
Akihiro Suda
33e5f47c6c
kubernetes: rootless: support Google Container-Optimized OS
Tested with GKE Autopilot 1.24.3-gke.200 (kernel 5.10.123+, containerd 1.6.6).

ref: moby/buildkit PR 3097

Signed-off-by: Akihiro Suda <akihiro.suda.cz@hco.ntt.co.jp>
2022-09-09 17:16:35 +09:00
Justin Chadwell
17d4369866 create: improve interface when attempting to create docker driver
Previously, the help information for buildx indicated that users could
create a new instance of the docker driver - which is explicitly
something we don't support, driver of this form are automatically
derived from the available list of docker contexts.

This patch ensures that don't have AllowsInstance set will not appear in
the help text, and additionally provide a new more specific error
message instead of the generic "failed to find driver". This should help
point users in the correct direction.

Signed-off-by: Justin Chadwell <me@jedevc.com>
2022-08-16 10:58:23 +01:00
Tonis Tiigi
3bc7d4bec6 enable other exporters if docker driver uses containerd
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2022-08-09 14:16:30 -07:00
Nicolas De Loof
1d8db8a738
detect moby worker supports multiplatform feature through containerd snapshotter
Signed-off-by: Nicolas De Loof <nicolas.deloof@gmail.com>
2022-08-09 14:41:08 +02:00
Sebastiaan van Stijn
4983b98005
vendor: v20.10.3-0.20220803220330-418ca3b4d46f (v22.06.0-dev)
full diff: f1615facb1...418ca3b4d4

relevant changes;

- cli/command: remove unused args from ResolveDefaultContext()
- consider empty DOCKER_HOST and DOCKER_CONTEXT env-vars equivalent to "not set"
- cli: set timeout connection ping on sockets as well

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-08-04 00:45:51 +02:00
CrazyMax
b0deb8bdd7
vendor: update docker/cli to f1615fa
also needs to update docker/docker to a60b458 (22.06 branch) otherwise
build breaks since docker/cli#3512 with:

    # github.com/docker/cli/cli/flags
    vendor/github.com/docker/cli/cli/flags/common.go:40:37: undefined: client.EnvOverrideCertPath
    vendor/github.com/docker/cli/cli/flags/common.go:41:37: undefined: client.EnvTLSVerify
    vendor/github.com/docker/cli/cli/flags/common.go:89:76: undefined: client.EnvOverrideHost

needs also to update github.com/spf13/cobra to v1.5.0 otherwise
build breaks with:

    # github.com/docker/cli/cli-plugins/plugin
    vendor/github.com/docker/cli/cli-plugins/plugin/plugin.go:130:4: unknown field 'HiddenDefaultCmd' in struct literal of type cobra.CompletionOptions

Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
2022-07-22 10:53:37 +02:00
Justin Chadwell
d9ef9bec34 kubernetes: add error when no pods available
This prevents the fall-through to the panic from division by zero in the
modulus below, and presents a neater error to the user.

Signed-off-by: Justin Chadwell <me@jedevc.com>
2022-07-06 16:02:48 +01:00
Justin Chadwell
1eff9310f8 remote: add additional connhelpers to buildx
Signed-off-by: Justin Chadwell <me@jedevc.com>
2022-05-25 10:39:16 +01:00
Tõnis Tiigi
062cf29de2
Merge pull request #1103 from jedevc/remote-driver-bootstrap
Use --bootstrap to wait for remote to become active
2022-05-12 10:21:15 -07:00
CrazyMax
57156ee95c
ls: adds fallback if buildkit version info unimplemented
Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
2022-05-09 16:17:30 +02:00
Justin Chadwell
c245f30a94 remote: use --bootstrap to wait for remote to become active
Signed-off-by: Justin Chadwell <me@jedevc.com>
2022-05-09 13:51:07 +01:00
Justin Chadwell
d7e4affe98 Complete remote driver
This patch completes the work started in creating a remote driver:

- Renames the env driver to the remote driver (an alternative suggestion
  that should be more user-friendly)
- Adds support for TLS to encrypt connections with buildkitd
- Fixes outstanding review comments
- Reworks the buildx create command endpoint construction to be clearer
  and include better support for this new driver.

Signed-off-by: Justin Chadwell <me@jedevc.com>
2022-04-28 11:34:06 +01:00
Batuhan Apaydın
3dc83e5dd8 feat: env driver
Co-authored-by: Furkan Türkal <furkan.turkal@trendyol.com>
Signed-off-by: Batuhan Apaydın <batuhan.apaydin@trendyol.com>
2022-04-28 11:34:06 +01:00
Zsolt
4a226568a0 Fix tolerations not parsing its options correctly, add tests
Signed-off-by: Zsolt <zsolt.szeberenyi@figured.com>
2022-04-22 12:12:05 +12:00
Doug Borg
79de2c5d82 Only set default rootless image if it is not already customized
Only change the image to the default rootless image when using the
--rootless option if the image has not already customized with the
--image option.

Fix #938

Signed-off-by: Doug Borg <dougborg@apple.com>
2022-04-12 10:13:43 -06:00
Akihiro Suda
add4301ed6
kubernetes: replace deprecated seccomp annotations with securityContext
Kubernetes added the official `securityContext.seccompProfile` support in Kubernetes 1.19.
Seccomp is still disabled by default.

The legacy `container.seccomp.security.alpha.kubernetes.io/<PODNAME>` annotation has been deprecated and will be unsupported in Kubernetes 1.25.
https://kubernetes.io/docs/tutorials/security/seccomp/

A test cluster can be created with the following minikube command:
```
minikube start --feature-gates SeccompDefault=true --extra-config kubelet.seccomp-default=true
```

Related to moby/buildkit PR 2782

Signed-off-by: Akihiro Suda <akihiro.suda.cz@hco.ntt.co.jp>
2022-04-07 17:23:42 +09:00
Zsolt
3f6517747e Add support for defining kubernetes tolerations
Signed-off-by: Zsolt <zsolt.szeberenyi@figured.com>
2022-04-06 17:10:21 +12:00
Sebastiaan van Stijn
fc7ba75fd7
Remove uses of deprecated io/ioutil
The package has been deprecated since Go 1.16: https://go.dev/doc/go1.16#ioutil

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-03-10 01:46:06 +01:00
CrazyMax
971b5d2b73
vendor: update docker/cli to 8667ccd
Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
2022-02-27 00:35:39 +01:00
Akihiro Suda
5f8600f098
docker-container: set UsernsMode only when needed
Set `UsernsMode="host"` only when the daemon is running in userns-remapping mode.

Fix issue 561

The issue will be also fixed in moby/moby PR 43084 (Docker 20.10.13).
This buildx PR helps users of old releases of Docker.

Signed-off-by: Akihiro Suda <akihiro.suda.cz@hco.ntt.co.jp>
2021-12-15 18:42:58 +09:00
Mayeul Blanzat
72dab552b5 Add an option to buildx rm to keep the buildkitd daemon running
Add --keep-daemon to the `rm` command option to preserve the buildkitd daemon after the buildx context is deleted.

Signed-off-by: Mayeul Blanzat <mayeul.blanzat@datadoghq.com>
2021-11-22 13:24:47 +01:00
Tonis Tiigi
40121c671c kubernetes: store config files for k8s
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2021-11-03 18:24:16 -07:00
Tonis Tiigi
4c1621cccd store snapshot of config files on create
Files can be reused when container needs to be booted again.

Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2021-11-03 16:34:43 -07:00
Tõnis Tiigi
49342dd54d
Merge pull request #787 from crazy-max/inject-certs
container driver: copy ca and user tls registries certs
2021-10-28 15:11:57 -07:00
CrazyMax
3f716f00fa
container driver: copy ca and user tls registries certs
Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
2021-10-28 22:30:11 +02:00
Tonis Tiigi
2644d56a6d docker: dial session directly with http hijack
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2021-10-20 18:54:31 -07:00
Tõnis Tiigi
461369748c
Merge pull request #782 from djs55/cgroup-parent
docker-container: place build containers in a separate cgroup
2021-09-30 09:06:29 -07:00
David Scott
d5908cdddf docker-container: use /docker/buildx cgroup by default
This allows resource limits to be applied to all builds on a host.
For example to limit the total amount of CPU used by builds:

https://medium.com/@asishrs/docker-limit-resource-utilization-using-cgroup-parent-72a646651f9d

Signed-off-by: David Scott <dave@recoil.org>
2021-09-29 19:58:22 +01:00
David Scott
b5bc754bad docker-container: support --driver-opt cgroup-parent=...
This allows the parent cgroup to be customised, which allows resource
limits to be imposed on build containers separately from "user"
containers.

Signed-off-by: David Scott <dave@recoil.org>
2021-09-29 19:57:46 +01:00
CrazyMax
9e62c9f074
Fix --driver flag usage markdown output
Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
2021-09-29 15:33:44 +02:00
Sebastiaan van Stijn
93867d02f0
container-driver: fix volume destination for cache
The container-driver creates a Linux container (as there currently isn't a
Windows version of buildkitd). However, the defaults are platform specific.

Buildx was using the defaults from the buildkit `util/appdefault' package,
which resulted in Buildx running on a Windows client to create a Linux
container that used the Windows location, which causes it to fail:

    invalid mount config for type "volume": invalid mount path: 'C:/ProgramData/buildkitd/.buildstate' mount path must be absolute

This patch hard-codes the destination to the default Linux path.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2021-08-27 09:48:31 +02:00
Tonis Tiigi
422ba60b04 use long-running context for client initialization
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2021-08-19 20:36:24 -07:00
Morlay
c76b5eac03 feat(driver/kubernetes): support mount buildkit.toml and qemu installing
Signed-off-by: Morlay <morlay.null@gmail.com>
2021-08-04 21:32:27 +08:00
Tonis Tiigi
0b6ba1cd32 don't error on deleting old build containers without state volume
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2021-07-16 12:00:27 -07:00
CrazyMax
258d12b2e7
Keep BuildKit state in a volume
Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
2021-07-13 18:09:35 +02:00
Tonis Tiigi
9d88450118 enable opentelemetry support
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2021-07-12 13:42:52 -07:00
Michal Augustyn
a53e392afb feat: add resources handling to kubernetes driver
Signed-off-by: Michal Augustyn <michal.augustyn@mail.com>
2021-05-29 13:45:52 +02:00
Akihiro Suda
8b8725d1fd
set DefaultRootlessImage to "moby/buildkit:buildx-stable-1-rootless"
Signed-off-by: Akihiro Suda <akihiro.suda.cz@hco.ntt.co.jp>
2020-12-16 17:35:38 +09:00
Tibor Vass
381dc8fb43 driver: do not insert "platform" as driver-opt
Addresses https://github.com/docker/setup-buildx-action/issues/45

Simple repro:
```
$ buildx create --platform linux/amd64 --use
$ buildx build - <<EOF
from scratch
EOF
```

Since https://github.com/docker/buildx/pull/370 a `platform` driver-opt was automatically inserted with the value specified by `--platform` flag on regardless of the type of driver, even though it was only used in the kubernetes driver. However, because the docker-container driver is pedantic about the options being passed, it errored out.

Another side-effect I suspect is that with the kubernetes driver it was now possible to specify the platforms in two different ways: `--driver-opt platform=...` and `--platform`.

This patch reverts completely the `platform` driver-opt and instead ensures the platforms information is passed onto the kubernetes driver via variables.

Signed-off-by: Tibor Vass <tibor@docker.com>
2020-12-15 07:09:46 +00:00
Tonis Tiigi
f68f42cb11 build: add push support to docker driver
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2020-12-08 13:54:41 -08:00
Tonis Tiigi
232af9aa0d move moby check to driver interface
Driver caching masked the method detection

Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2020-12-07 22:02:51 -08:00
Tonis Tiigi
1496ac9b55 util: simplify progress syncronization
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2020-12-07 22:02:51 -08:00
Tonis Tiigi
290e25917c build: allow dockerfile from URL
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2020-12-07 22:02:51 -08:00
Tõnis Tiigi
35a60b8e04
Merge pull request #441 from tonistiigi/buildkit-pull-creds2
refactor driver auth for easier passing
2020-12-05 00:02:38 -08:00
Andrew Haines
37020dc8da
Disable user namespace remapping in docker-container driver
Signed-off-by: Andrew Haines <andrew.haines@zencargo.com>
2020-12-03 10:50:00 +00:00