This measures the duration of exec operations. It does not factor in
whether the operation was cached or not so this should include the
amount of time to determine whether an operation was cached.
Signed-off-by: Jonathan A. Sternberg <jonathan.sternberg@docker.com>
This measures the transfer size and duration for image pulls along with
the time spent extracting the image contents.
Signed-off-by: Jonathan A. Sternberg <jonathan.sternberg@docker.com>
Measure the transfer size and duration of context transfers for various
categories of local source transfers from the progress stream that's
returned during the build.
Local source transfers are split into one of four categories:
* context
* dockerfile
* dockerignore
* namedcontext
Named contexts that are different names will be categorized under the
same metric.
Signed-off-by: Jonathan A. Sternberg <jonathan.sternberg@docker.com>
The `BUILDX_EXPERIMENTAL` check is removed from the docker otel
collector. We'll send metrics to the OTLP endpoint for docker desktop if
it is present and enabled regardless of experimental status.
The user-facing `OTEL` endpoints for enabling the metric reporting for
external use is still hidden behind the experimental flag. We'll likely
remove the experimental flag for this feature for v0.14.
Signed-off-by: Jonathan A. Sternberg <jonathan.sternberg@docker.com>
This adds a build duration metric for the build command with attributes
related to the buildx driver, the error type (if any), and which options
were used to perform the build from a subset of the options.
This also refactors some of the utility methods used by the git tool to
determine filepaths into its own separate package so they can be reused
in another place.
Also adds a test to ensure the resource is initialized correctly and
doesn't error. The otel handler logging message is suppressed on buildx
invocations so we never see the error if there's a problem with the
schema url. It's so easy to mess up the schema url when upgrading OTEL
that we need a proper test to make sure we haven't broken the
functionality.
Signed-off-by: Jonathan A. Sternberg <jonathan.sternberg@docker.com>
Add the service instance id to the resource attributes to prevent
downstream OTEL processors and exporters from thinking that the CLI
invocations are a single process that keeps restarting. The unique id
can be removed through downstream aggregation to prevent cardinality
issues, but we need some way to tell OTEL that it shouldn't reset the
counters.
Move the check for the experimental flag to its own package and then use
that invocation to prevent creating exporters so metrics are disabled
completely. This makes it so we don't have to check for the experimental
flag in every place we add metrics until we decide to make metrics
stable in general.
This also moves the OTEL initialization to a `util/metricutil` package
to be more consistent with the existing util naming and to differentiate
it from the upstream `metric` name. Using both `metrics` and `metric` as
import names was confusing since `metric` was an upstream dependency and
`metrics` was a local utility. `metricutil` matches with the existing
utilities and makes clear that it isn't a spelling mistake.
The record version metric has been removed since we weren't planning on
keeping that metric anyway and most of the information is now included
in the instrumentation library name and version. That function is
included as a utility in the `otel/sdk/metric` package to retrieve the
appropriate meter from the meter provider.
Signed-off-by: Jonathan A. Sternberg <jonathan.sternberg@docker.com>
Prior to this commit, experimental flags were not distinguishable from
regular flags in `--help`
Signed-off-by: Akihiro Suda <akihiro.suda.cz@hco.ntt.co.jp>
Introduce a meter provider to the buildx cli that will send metrics to
the otel-collector included in docker desktop if enabled.
This will send usage metrics to the desktop application but also send
metrics to a user-provided otlp receiver endpoint through the standard
environment variables.
This introduces a single metric which is the cli count for build and
bake along with the command name and a few additional attributes.
Signed-off-by: Jonathan A. Sternberg <jonathan.sternberg@docker.com>
Using this command resolves remote based on remote
tracking branch of the curently selected branch and
should be more precise in case we can't predict if user
uses origin to mark upstream or their fork.
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
Without this, we attempt to overwrite the previous display in the lines
above which have been filled with logs from the monitor (see when used
with `--invoke`).
Signed-off-by: Justin Chadwell <me@jedevc.com>
This extracts the same logic for parsing annotations from the imagetools
create command, and allows the same flags to be attached to the build
command.
These annotations are then merged into all provided exporters.
Signed-off-by: Justin Chadwell <me@jedevc.com>
No other parts of the progress rendering modify the inputs, so we should
avoid this as well.
This actually fixes an edge case in pushWithMoby which writes the same
VertexStatus multiple times, modifying the timestamps and similar.
However, if the operation takes long enough the small time difference
can accumulate, and move the Start time far into the past.
Signed-off-by: Justin Chadwell <me@jedevc.com>
Use buildkit's parsing of entitlements to make sure that accepted
values match what's accepted by BuildKit.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
ReadLocalFiles should allow passing the stdin file as an argument, which
allows us to read from dockerCli.Stdin() to be consistent with other
commands in the same package.
Signed-off-by: Justin Chadwell <me@jedevc.com>
- Adds a new GitServeHTTP function to start an http server to serve a
target git repository.
- Adds a new GitDir helper method to get the path to the .git
directory
- Updates the GitAdd method to take a variable number of files
Signed-off-by: Justin Chadwell <me@jedevc.com>
Replace invalid characters in the name of the base directory of
certificates using the name of the registry which can contain
for example a colon when the port is specified.
Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
When the user supply static creds, we must not enrich them with a
session token which is unrelated.
Signed-off-by: Bertrand Paquet <bertrand.paquet@gmail.com>
This ensures that `target.attest=["type=sbom,<value>"]` can be
appropriately merged when `--sbom=true` or `--set
target.attest=type=sbom`.
To merge, we simply naively take the last valid value.
Signed-off-by: Justin Chadwell <me@jedevc.com>
When using imagetools create and combining multiple sources
we should check the media type of each manifest and set
the right media type for the manifest list.
If there is a mismatch we set OCI index as best effort.
Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
This fixes a possible race where messages printed directly after calls
to NewPrinter may appear before the printer starts. With this change, we
delay all of the logs until after.
Signed-off-by: Justin Chadwell <me@jedevc.com>
This will allow result printing to work with the remote controller
(though this currently causes a panic, to be fixed in a follow-up).
Signed-off-by: Justin Chadwell <me@jedevc.com>
Refactor the progress printer creation to the caller-side of the
controller api. Then, instead of passing around status channels (and
progressMode strings), we can simply pass around the higher level
interface progress.Writer.
This has a couple of benefits:
- A simplified interface to the controller
- Allows us to correctly extract warnings out of the controller, so that
they can be displayed correctly from the client side.
Some extra work is required to make sure that we can pass a
progress.Printer into the debug monitor. If we want to keep it
persistent, then we need a way to temporarily suspend output from it,
otherwise it will continue printing as the monitor is prompting for
input from the user, and forwarding output from debug containers.
To handle this, we add two methods to the printer, `Pause` and
`Unpause`. `Pause` acts similarly to `Wait`, closing the printer, and
cleanly shutting down the display - however, the printer does not
terminate, and can later be resumed by a call to `Unpause`. This
provides a neater interface to the caller, instead of needing to
continually reconstruct printers for every single time we want to
produce progress output.
Signed-off-by: Justin Chadwell <me@jedevc.com>
We can perform all attestation processing, handling how the sbom and
provenance arguments interact on the client, while applying defaults on
the server.
Additionally, this allows us to start pulling fields out of CommonOpts.
Signed-off-by: Justin Chadwell <me@jedevc.com>
Strongly typing the API allows us to perform all command line parsing
fully on the client-side, where we have access to the client local
directory and all the client environment variables, which may not be
available on the remote server.
Additionally, the controller api starts to look a lot like
build.Options, so at some point in the future there may be an
oppportunity to merge the two, which would allow both build and bake to
execute through the controller, instead of needing to maintain multiple
code paths.
Signed-off-by: Justin Chadwell <me@jedevc.com>
To give us the option later down the road of producing recommended OCI
names in BuildKit (using com instead of vnd, woops), we need to update
Buildx to be able to process both.
Ideally, if a Buildx/BuildKit release hadn't been made we could just
switch over, but since we have, we'd need to support both (at least for
a while, eventually we could consider deprecating+removing the vnd
variant).
Signed-off-by: Justin Chadwell <me@jedevc.com>
Delay loading the attestation data immediately, and only compute it upon
request. We do this using a deferred function which allows to define the
computation in the same place as before, but perform the computation
later.
With this patch, we ensure that the attestation data is only pulled from
the remote if it is actually referenced in the format string -
otherwise, we can skip it, for improved performance.
Signed-off-by: Justin Chadwell <me@jedevc.com>
This refactor ensures that the attestations are not output in the JSON
output for "{{ json . }}", and additionally allows future refactors to
dynamically load the attestation contents, ensuring faster performance
when attestations are not used in the output.
Signed-off-by: Justin Chadwell <me@jedevc.com>
Jonathan A. Sternberg