[REQ] Implement Copa Metrics

[ashnamehrotra]

What kind of request is this?

None

What is your request or suggestion?

Have copa return metrics like total time to run, this could be helpful to compare update all vs scanner approach and when adding new features.

Are you willing to submit PRs to contribute to this feature request?

• Yes, I am willing to implement it.

[pradhans0906]

I believe copa uses buildkit and buildKit provides extensive logging and can expose detailed metrics about the build process.

so we can have this. Let me know , if I can start working on this ?

[sozercan]

@pradhans0906 thanks for your interest! it would be great if you can provide link to existing buildkit metrics and create a doc/list of metrics that can be useful for copa specifically

[pradhans0906]

So what I meant from from my comment was : We can get the build kit to run on debug mode (--debug)
Plain progress (BUILDKIT_PROGRESS=plain) and we can utilise this with copa and we able to get the full operation like image pull and extraction and package operation and data transfer details as well. Buildkit doesn't export any metrics out of the box only debug and plain-progress where it spits out all the information and from there we can programmatically parse these debug logs rather than relying on pre-built metrics.

`BUILDKIT_PROGRESS=plain copa patch --scanner docker-scout -i node:18 --debug

DEBU[0000] Trying docker driver
DEBU[0000] serving grpc connection
DEBU[0000] stopping session
DEBU[0000] serving grpc connection
#1 resolve image config for docker-image://docker.io/library/node:18
#1 DONE 1.5s

#2 resolve image config for docker-image://docker.io/library/node:18
#2 DONE 0.2s

#3 docker-image://docker.io/library/node:18
#3 resolve docker.io/library/node:18
#3 resolve docker.io/library/node:18 0.2s done
#3 DONE 0.2s

#3 docker-image://docker.io/library/node:18
#3 sha256:b36b8e92e3a7953f2a5fd00e4fd5fc29cb0d70958d32c19601eb23186b6bc3d9 0B / 446B 0.2s
#3 sha256:b36b8e92e3a7953f2a5fd00e4fd5fc29cb0d70958d32c19601eb23186b6bc3d9 446B / 446B 0.2s done
#3 sha256:7ad827fdfa492a1d157db8b502be89b4c74e853dfd47f418da31de9cdd492bc7 0B / 45.69MB 0.3s
#3 sha256:ff943d243a0d6b882ab5e03100c4c2382b7fa1306208088f351c27de72d97fb0 0B / 1.25MB 0.3s
#3 sha256:4767b1635f4a478c9c34f6169008202af623e1c05ea663f697e49bbb25ac0166 0B / 3.33kB 0.2s
#3 sha256:ff943d243a0d6b882ab5e03100c4c2382b7fa1306208088f351c27de72d97fb0 1.25MB / 1.25MB 0.4s done
#3 sha256:59d4884f85282b9a352dbcedf2cccd073a63e60b151be84375ce9279dec1c553 0B / 202.64MB 0.2s
#3 sha256:7ad827fdfa492a1d157db8b502be89b4c74e853dfd47f418da31de9cdd492bc7 4.19MB / 45.69MB 0.6s
#3 sha256:4767b1635f4a478c9c34f6169008202af623e1c05ea663f697e49bbb25ac0166 3.33kB / 3.33kB 0.5s done
#3 sha256:cb5594266b1bacf9ad6855b00d9c5c98e721001eb115218eda673e548e04fdbf 0B / 64.35MB 0.2s
#3 sha256:7ad827fdfa492a1d157db8b502be89b4c74e853dfd47f418da31de9cdd492bc7 8.39MB / 45.69MB 0.8s
#3 sha256:95b894d63c771a6056bc65ff25192b251413259ba7d160b0076f0f5d7975dc39 0B / 23.59MB 0.2s
#3 sha256:7ad827fdfa492a1d157db8b502be89b4c74e853dfd47f418da31de9cdd492bc7 15.73MB / 45.69MB 0.9s
#3 sha256:59d4884f85282b9a352dbcedf2cccd073a63e60b151be84375ce9279dec1c553 17.83MB / 202.64MB 0.6s
#3 sha256:cb5594266b1bacf9ad6855b00d9c5c98e721001eb115218eda673e548e04fdbf 4.19MB / 64.35MB 0.5s
#3 sha256:cb5594266b1bacf9ad6855b00d9c5c98e721001eb115218eda673e548e04fdbf 9.44MB / 64.35MB 0.6s
#3 sha256:7ad827fdfa492a1d157db8b502be89b4c74e853dfd47f418da31de9cdd492bc7 20.97MB / 45.69MB 1.2s
#3 sha256:cb5594266b1bacf9ad6855b00d9c5c98e721001eb115218eda673e548e04fdbf 16.78MB / 64.35MB 0.8s
#3 sha256:95b894d63c771a6056bc65ff25192b251413259ba7d160b0076f0f5d7975dc39 6.29MB / 23.59MB 0.6s
#3 sha256:7ad827fdfa492a1d157db8b502be89b4c74e853dfd47f418da31de9cdd492bc7 24.12MB / 45.69MB 1.4s
#3 sha256:59d4884f85282b9a352dbcedf2cccd073a63e60b151be84375ce9279dec1c553 31.46MB / 202.64MB 1.1s
#3 sha256:cb5594266b1bacf9ad6855b00d9c5c98e721001eb115218eda673e548e04fdbf 23.07MB / 64.35MB 0.9s
#3 sha256:95b894d63c771a6056bc65ff25192b251413259ba7d160b0076f0f5d7975dc39 10.49MB / 23.59MB 0.8s
#3 sha256:7ad827fdfa492a1d157db8b502be89b4c74e853dfd47f418da31de9cdd492bc7 27.26MB / 45.69MB 1.5s
#3 sha256:cb5594266b1bacf9ad6855b00d9c5c98e721001eb115218eda673e548e04fdbf 32.51MB / 64.35MB 1.2s
#3 sha256:95b894d63c771a6056bc65ff25192b251413259ba7d160b0076f0f5d7975dc39 16.78MB / 23.59MB 1.1s
#3 sha256:7ad827fdfa492a1d157db8b502be89b4c74e853dfd47f418da31de9cdd492bc7 34.60MB / 45.69MB 1.8s
#3 sha256:59d4884f85282b9a352dbcedf2cccd073a63e60b151be84375ce9279dec1c553 41.94MB / 202.64MB 1.5s
#3 sha256:cb5594266b1bacf9ad6855b00d9c5c98e721001eb115218eda673e548e04fdbf 40.89MB / 64.35MB 1.4s
#3 sha256:7ad827fdfa492a1d157db8b502be89b4c74e853dfd47f418da31de9cdd492bc7 38.80MB / 45.69MB 2.0s
#3 sha256:cb5594266b1bacf9ad6855b00d9c5c98e721001eb115218eda673e548e04fdbf 47.19MB / 64.35MB 1.5s
#3 sha256:95b894d63c771a6056bc65ff25192b251413259ba7d160b0076f0f5d7975dc39 23.59MB / 23.59MB 1.3s done
#3 sha256:59d4884f85282b9a352dbcedf2cccd073a63e60b151be84375ce9279dec1c553 56.62MB / 202.64MB 1.8s
#3 sha256:cb5594266b1bacf9ad6855b00d9c5c98e721001eb115218eda673e548e04fdbf 55.57MB / 64.35MB 1.7s
#3 sha256:c1e0ef7b956a07c7b090256aa16cbb0550a34d0625d1d23c5b1a76e92a58d01e 0B / 49.58MB 0.2s
#3 sha256:7ad827fdfa492a1d157db8b502be89b4c74e853dfd47f418da31de9cdd492bc7 42.99MB / 45.69MB 2.3s
#3 sha256:cb5594266b1bacf9ad6855b00d9c5c98e721001eb115218eda673e548e04fdbf 59.77MB / 64.35MB 1.8s
#3 sha256:7ad827fdfa492a1d157db8b502be89b4c74e853dfd47f418da31de9cdd492bc7 45.69MB / 45.69MB 2.3s done
#3 sha256:59d4884f85282b9a352dbcedf2cccd073a63e60b151be84375ce9279dec1c553 70.25MB / 202.64MB 2.1s
#3 sha256:cb5594266b1bacf9ad6855b00d9c5c98e721001eb115218eda673e548e04fdbf 64.35MB / 64.35MB 1.9s done
#3 sha256:c1e0ef7b956a07c7b090256aa16cbb0550a34d0625d1d23c5b1a76e92a58d01e 9.44MB / 49.58MB 0.6s
#3 sha256:c1e0ef7b956a07c7b090256aa16cbb0550a34d0625d1d23c5b1a76e92a58d01e 14.68MB / 49.58MB 0.8s
#3 sha256:59d4884f85282b9a352dbcedf2cccd073a63e60b151be84375ce9279dec1c553 80.74MB / 202.64MB 2.4s
#3 sha256:c1e0ef7b956a07c7b090256aa16cbb0550a34d0625d1d23c5b1a76e92a58d01e 17.83MB / 49.58MB 0.9s
#3 sha256:c1e0ef7b956a07c7b090256aa16cbb0550a34d0625d1d23c5b1a76e92a58d01e 22.02MB / 49.58MB 1.1s
#3 sha256:59d4884f85282b9a352dbcedf2cccd073a63e60b151be84375ce9279dec1c553 95.42MB / 202.64MB 2.7s
#3 sha256:c1e0ef7b956a07c7b090256aa16cbb0550a34d0625d1d23c5b1a76e92a58d01e 25.17MB / 49.58MB 1.4s
#3 sha256:59d4884f85282b9a352dbcedf2cccd073a63e60b151be84375ce9279dec1c553 110.10MB / 202.64MB 3.0s
#3 sha256:c1e0ef7b956a07c7b090256aa16cbb0550a34d0625d1d23c5b1a76e92a58d01e 28.31MB / 49.58MB 1.5s
#3 sha256:59d4884f85282b9a352dbcedf2cccd073a63e60b151be84375ce9279dec1c553 120.59MB / 202.64MB 3.3s
#3 sha256:c1e0ef7b956a07c7b090256aa16cbb0550a34d0625d1d23c5b1a76e92a58d01e 36.70MB / 49.58MB 1.8s
#3 sha256:c1e0ef7b956a07c7b090256aa16cbb0550a34d0625d1d23c5b1a76e92a58d01e 41.94MB / 49.58MB 2.0s
#3 sha256:59d4884f85282b9a352dbcedf2cccd073a63e60b151be84375ce9279dec1c553 135.27MB / 202.64MB 3.6s
#3 sha256:c1e0ef7b956a07c7b090256aa16cbb0550a34d0625d1d23c5b1a76e92a58d01e 48.23MB / 49.58MB 2.1s
#3 sha256:59d4884f85282b9a352dbcedf2cccd073a63e60b151be84375ce9279dec1c553 154.14MB / 202.64MB 3.9s
#3 sha256:c1e0ef7b956a07c7b090256aa16cbb0550a34d0625d1d23c5b1a76e92a58d01e 49.58MB / 49.58MB 2.1s done
#3 extracting sha256:c1e0ef7b956a07c7b090256aa16cbb0550a34d0625d1d23c5b1a76e92a58d01e
#3 sha256:59d4884f85282b9a352dbcedf2cccd073a63e60b151be84375ce9279dec1c553 164.63MB / 202.64MB 4.2s
#3 sha256:59d4884f85282b9a352dbcedf2cccd073a63e60b151be84375ce9279dec1c553 175.11MB / 202.64MB 4.4s
#3 extracting sha256:c1e0ef7b956a07c7b090256aa16cbb0550a34d0625d1d23c5b1a76e92a58d01e 0.6s done
#3 extracting sha256:95b894d63c771a6056bc65ff25192b251413259ba7d160b0076f0f5d7975dc39
#3 extracting sha256:95b894d63c771a6056bc65ff25192b251413259ba7d160b0076f0f5d7975dc39 0.2s done
#3 extracting sha256:cb5594266b1bacf9ad6855b00d9c5c98e721001eb115218eda673e548e04fdbf
#3 sha256:59d4884f85282b9a352dbcedf2cccd073a63e60b151be84375ce9279dec1c553 185.60MB / 202.64MB 4.8s
#3 sha256:59d4884f85282b9a352dbcedf2cccd073a63e60b151be84375ce9279dec1c553 200.28MB / 202.64MB 5.1s
#3 sha256:59d4884f85282b9a352dbcedf2cccd073a63e60b151be84375ce9279dec1c553 202.64MB / 202.64MB 5.4s done
#3 extracting sha256:cb5594266b1bacf9ad6855b00d9c5c98e721001eb115218eda673e548e04fdbf 0.7s done
#3 DONE 5.8s

#3 docker-image://docker.io/library/node:18
#3 extracting sha256:59d4884f85282b9a352dbcedf2cccd073a63e60b151be84375ce9279dec1c553
#3 extracting sha256:59d4884f85282b9a352dbcedf2cccd073a63e60b151be84375ce9279dec1c553 2.2s done
#3 DONE 7.9s

#3 docker-image://docker.io/library/node:18
#3 extracting sha256:4767b1635f4a478c9c34f6169008202af623e1c05ea663f697e49bbb25ac0166 done
#3 extracting sha256:7ad827fdfa492a1d157db8b502be89b4c74e853dfd47f418da31de9cdd492bc7
#3 extracting sha256:7ad827fdfa492a1d157db8b502be89b4c74e853dfd47f418da31de9cdd492bc7 0.6s done
#3 DONE 8.6s
DEBU[0010] Using debian:12-slim as basis for tooling image

#3 docker-image://docker.io/library/node:18
#3 extracting sha256:ff943d243a0d6b882ab5e03100c4c2382b7fa1306208088f351c27de72d97fb0 0.0s done
#3 extracting sha256:b36b8e92e3a7953f2a5fd00e4fd5fc29cb0d70958d32c19601eb23186b6bc3d9 done
#3 DONE 8.6s

#4 docker-image://docker.io/library/debian:12-slim
#4 resolve docker.io/library/debian:12-slim
#4 resolve docker.io/library/debian:12-slim 0.4s done
#4 DONE 0.4s

#4 docker-image://docker.io/library/debian:12-slim
#4 CACHED

#5 apt-get update
#5 0.265 Get:1 http://deb.debian.org/debian bookworm InRelease [151 kB]
#5 0.288 Get:2 http://deb.debian.org/debian bookworm-updates InRelease [55.4 kB]
#5 0.316 Get:3 http://deb.debian.org/debian-security bookworm-security InRelease [48.0 kB]
#5 0.348 Get:4 http://deb.debian.org/debian bookworm/main arm64 Packages [8689 kB]
#5 0.528 Get:5 http://deb.debian.org/debian bookworm-updates/main arm64 Packages [2468 B]
#5 0.531 Get:6 http://deb.debian.org/debian-security bookworm-security/main arm64 Packages [187 kB]
#5 1.109 Fetched 9133 kB in 1s (9480 kB/s)
#5 1.109 Reading package lists...
#5 DONE 1.6s

#6 apt-get install busybox-static
#6 0.124 Reading package lists...
#6 0.349 Building dependency tree...
#6 0.395 Reading state information...
#6 0.462 The following NEW packages will be installed:
#6 0.462 busybox-static
#6 0.515 0 upgraded, 1 newly installed, 0 to remove and 0 not upgraded.
#6 0.515 Need to get 819 kB of archives.
#6 0.515 After this operation, 1936 kB of additional disk space will be used.
#6 0.515 Get:1 http://deb.debian.org/debian bookworm/main arm64 busybox-static arm64 1:1.35.0-4+b3 [819 kB]
#6 0.594 debconf: delaying package configuration, since apt-utils is not installed
#6 0.601 Fetched 819 kB in 0s (13.2 MB/s)
#6 0.609 Selecting previously unselected package busybox-static.
(Reading database ... 6083 files and directories currently installed.)
#6 0.618 Preparing to unpack .../busybox-static_1%3a1.35.0-4+b3_arm64.deb ...
#6 0.619 Unpacking busybox-static (1:1.35.0-4+b3) ...
#6 0.662 Setting up busybox-static (1:1.35.0-4+b3) ...
#6 DONE 0.7s

#7 copy /bin/busybox /bin/busybox
#7 DONE 0.0s

#8 mkdir /copa-out
#8 DONE 0.0s

#9 /bin/busybox sh -c
status="$DPKG_STATUS_IS_UNKNOWN"
if [ -f "$DPKG_STATUS_PATH" ]; then
status="$DPKG_STATUS_IS_FILE"
cp "$DPKG_STATUS_PATH" "$RESULTS_PATH"
elif [ -d "$DPKG_STATUS_FOLDER" ]; then
status="$DPKG_STATUS_IS_DIRECTORY"
ls -1 "$DPKG_STATUS_FOLDER" > "$RESULT_STATUSD_PATH"
mv "$DPKG_STATUS_FOLDER"/* "$RESULTS_PATH"
fi
echo -n "$status" > "${RESULTS_PATH}/${STATUSD_OUTPUT_FILENAME}"

#9 DONE 0.1s

#10 apt-get update
#10 0.255 Get:1 http://deb.debian.org/debian bookworm InRelease [151 kB]
#10 0.282 Get:2 http://deb.debian.org/debian bookworm-updates InRelease [55.4 kB]
#10 0.307 Get:3 http://deb.debian.org/debian-security bookworm-security InRelease [48.0 kB]
#10 0.327 Get:4 http://deb.debian.org/debian bookworm/main arm64 Packages [8689 kB]
#10 0.507 Get:5 http://deb.debian.org/debian bookworm-updates/main arm64 Packages [2468 B]
#10 0.508 Get:6 http://deb.debian.org/debian-security bookworm-security/main arm64 Packages [187 kB]
#10 1.082 Fetched 9133 kB in 1s (9529 kB/s)
#10 1.082 Reading package lists...
#10 DONE 1.3s

#11 sh -c apt-get -s upgrade 2>/dev/null | grep -q ^Inst || exit 1
#11 DONE 0.6s

#12 sh -c output=$(apt-get upgrade -y && apt-get clean -y && apt-get autoremove 2>&1); if [ $? -ne 0 ]; then echo $output >>error_log.txt; fi
#12 0.566 debconf: delaying package configuration, since apt-utils is not installed
#12 DONE 1.0s

#13 /bin/sh -c if [ -s error_log.txt ]; then cat error_log.txt; exit 1; fi
#13 DONE 0.1s

#14 sh -c grep "^Package:|^Version:" "/var/lib/dpkg/status" >> "results.manifest"
#14 DONE 0.1s

#15 diff (/bin/sh -c if [ -s error_log.txt ]; then cat error_log.txt; exit 1; fi) -> (sh -c grep "^Package:|^Version:" "/var/lib/dpkg/status" >> "results.manifest")
#15 diffing done
#15 DONE 0.0s

#16 diff (sh -c apt-get -s upgrade 2>/dev/null | grep -q ^Inst || exit 1) -> (/bin/sh -c if [ -s error_log.txt ]; then cat error_log.txt; exit 1; fi)
#16 DONE 0.0s

#17 merge (docker-image://docker.io/library/node:18, diff (sh -c apt-get -s upgrade 2>/dev/null | grep -q ^Inst || exit 1) -> (/bin/sh -c if [ -s error_log.txt ]; then cat error_log.txt; exit 1; fi))
#17 DONE 0.0s

#18 exporting to docker image format
#18 exporting layers 0.1s done
#18 exporting manifest sha256:023378fb462d7c77279f4dae2e2405ee6c914cfa00f09ad25fac6282426b6f5e done
#18 exporting config sha256:0986daafe608eb639049a933e55566ea532eda140c3918a9606e8b3ee859dd07 done
#18 sending tarball
#18 sending tarball 4.5s done
#18 DONE 4.6s
DEBU[0020] stopping session
INFO[0020] Loaded image: node:18-patched
WARN[0020] --debug specified, working folder at /var/folders/4n/xjysvzys2kj451rs9yl9qrkm0000gn/T/copa-2630944265 needs to be manually cleaned up`

Let me know if my understanding matches yours, or if you had a different approach in mind for incorporating BuildKit's debug output as metrics for Copa.