[BUG/Question] MANIFEST_UNKNOWN while using selfhosted registry

[Whisper40]

I'm trying to connect your tool to our private registry (artifactory) with a docker-remote to dockerhub but it seems that we encounter errors like "MANIFEST_UNKNOWN"

Expected behavior

Version of image should be displayed, but it is not

Logs

time="2025-01-24T09:43:54Z" level=error msg="docker-remote.registry.domain.fr/v2/library/haproxy/manifests/1.6.13: failed to get manifest response for tag, skipping (404): {\"errors\":[{\"code\":\"MANIFEST_UNKNOWN\",\"message\":\"The named manifest is not known to the registry.\",\"detail\":{\"manifest\":\"library/haproxy\"}}]}" client=docker-remote.registry.domain.fr
time="2025-01-24T09:43:57Z" level=error msg="docker-remote.registry.domain.fr/v2/library/haproxy/manifests/1.6.13-alpine: failed to get manifest response for tag, skipping (404): {\"errors\":[{\"code\":\"MANIFEST_UNKNOWN\",\"message\":\"The named manifest is not known to the registry.\",\"detail\":{\"manifest\":\"library/haproxy\"}}]}" client=docker-remote.registry.domain.fr
time="2025-01-24T09:44:01Z" level=error msg="docker-remote.registry.domain.fr/v2/library/haproxy/manifests/1.6.14: failed to get manifest response for tag, skipping (404): {\"errors\":[{\"code\":\"MANIFEST_UNKNOWN\",\"message\":\"The named manifest is not known to the registry.\",\"detail\":{\"manifest\":\"library/haproxy\"}}]}" client=docker-remote.registry.domain.fr
time="2025-01-24T09:44:07Z" level=error msg="docker-remote.registry.domain.fr/v2/library/haproxy/manifests/1.6.14-alpine: failed to get manifest response for tag, skipping (404): {\"errors\":[{\"code\":\"MANIFEST_UNKNOWN\",\"message\":\"The named manifest is not known to the registry.\",\"detail\":{\"manifest\":\"library/haproxy\"}}]}" client=docker-remote.registry.domain.fr
time="2025-01-24T09:44:11Z" level=error msg="docker-remote.registry.domain.fr/v2/library/haproxy/manifests/1.6.15: failed to get manifest response for tag, skipping (404): {\"errors\":[{\"code\":\"MANIFEST_UNKNOWN\",\"message\":\"The named manifest is not known to the registry.\",\"detail\":{\"manifest\":\"library/haproxy\"}}]}" client=docker-remote.registry.domain.fr
time="2025-01-24T09:44:14Z" level=error msg="docker-remote.registry.domain.fr/v2/library/haproxy/manifests/1.6.15-alpine: failed to get manifest response for tag, skipping (404): {\"errors\":[{\"code\":\"MANIFEST_UNKNOWN\",\"message\":\"The named manifest is not known to the registry.\",\"detail\":{\"manifest\":\"library/haproxy\"}}]}" client=docker-remote.registry.domain.fr
time="2025-01-24T09:45:18Z" level=debug msg="removing deleted pod containers from metrics: miscellaneous/kgb-agent-28961856-vt5l7/kgb-agent" module=controller
time="2025-01-24T09:45:32Z" level=error msg="docker-remote.registry.domain.fr/v2/library/haproxy/manifests/1.7: failed to get manifest response for tag, skipping (404): {\"errors\":[{\"code\":\"MANIFEST_UNKNOWN\",\"message\":\"The named manifest is not known to the registry.\",\"detail\":{\"manifest\":\"library/haproxy\"}}]}" client=docker-remote.registry.domain.fr
time="2025-01-24T09:45:42Z" level=error msg="error syncing 'argo-cd-redis-ha-haproxy-84c4877f96-fmqmp/argocd': failed to sync pod argocd/argo-cd-redis-ha-haproxy-84c4877f96-fmqmp: failed to check container image \"config-init\": failed to get tags from remote registry for \"docker-remote.registry.domain.fr/library/haproxy\": failed to get docker image: Get \"https://docker-remote.registry.domain.fr/v2/library/haproxy/tags/list?n=500\": context deadline exceeded (Client.Timeout exceeded while awaiting headers),failed to check container image \"install-oneagent\": failed to get tags from remote registry for \"docker-remote.registry.domain.fr/dynatrace/dynatrace-operator\": failed to get docker image: Get \"https://docker-remote.registry.domain.fr/v2/dynatrace/dynatrace-operator/manifests/1.0.0\": context deadline exceeded (Client.Timeout exceeded while awaiting headers),failed to check container image \"haproxy\": failed to get tags from remote registry for \"docker-remote.registry.domain.fr/library/haproxy\": failed to get docker image: Get \"https://docker-remote.registry.domain.fr/v2/library/haproxy/manifests/1.7-alpine\": context deadline exceeded (Client.Timeout exceeded while awaiting headers), requeuing" module=controller
time="2025-01-24T09:45:42Z" level=debug msg="processing container image" container=config-init module=controller name=argo-cd-redis-ha-server-1 namespace=argocd
time="2025-01-24T09:45:45Z" level=error msg="docker-remote.registry.domain.fr/v2/library/haproxy/manifests/1: failed to get manifest response for tag, skipping (404): {\"errors\":[{\"code\":\"MANIFEST_UNKNOWN\",\"message\":\"The named manifest is not known to the registry.\",\"detail\":{\"manifest\":\"library/haproxy\"}}]}" client=docker-remote.registry.domain.fr
time="2025-01-24T10:03:17Z" level=debug msg="removing deleted pod containers from metrics: miscellaneous/kgb-agent-28961874-t5ptk/kgb-agent" module=controller
time="2025-01-24T10:05:38Z" level=error msg="docker-remote.registry.domain.fr/v2/dynatrace/dynatrace-operator/manifests/gktest: failed to get manifest response for tag, skipping (404): {\"errors\":[{\"code\":\"MANIFEST_UNKNOWN\",\"message\":\"The named manifest is not known to the registry.\",\"detail\":{\"manifest\":\"dynatrace/dynatrace-operator\"}}]}" client=docker-remote.registry.domain.fr
time="2025-01-24T10:09:01Z" level=error msg="error syncing 'argo-rollout-argo-rollouts-697c9d754c-42m45/argo-rollouts': failed to sync pod argo-rollouts/argo-rollout-argo-rollouts-697c9d754c-42m45: failed to check container image \"argo-rollouts\": failed to get tags from remote registry for \"docker-remote.registry.domain.fr/argoproj/argo-rollouts\": failed to get docker image: Get \"https://docker-remote.registry.domain.fr/v2/argoproj/argo-rollouts/manifests/sha256-248bac515858915fbe52e4a6926d269ba3f167fe9c35a1a3546a56e891a9c2f7.sig\": context deadline exceeded (Client.Timeout exceeded while awaiting headers), requeuing" module=controller

time="2025-01-24T10:18:09Z" level=error msg="registry.k8s.io/v2/ingress-nginx/controller/manifests/v1.11.1: failed to get manifest response for tag, skipping (404): {\"errors\":[{\"code\":\"MANIFEST_UNKNOWN\",\"message\":\"Manifest has media type \\\"application/vnd.docker.distribution.manifest.list.v2+json\\\" but client accepts [\\\"application/vnd.docker.distribution.manifest.v1+json\\\"]\"}]}\n" client=docker-remote.registry.domain.fr
time="2025-01-24T10:18:09Z" level=error msg="ghcr.io/v2/fluxcd/notification-controller/manifests/sha256-80a99e9ad4f41abd8549dd3ddb75c6c698550d6179c926d18ce98edbcaba8782.sig: failed to get manifest response for tag, skipping (404): {\"errors\":[{\"code\":\"MANIFEST_UNKNOWN\",\"message\":\"OCI manifest found, but Accept header does not support OCI manifests\"}]}\n" client=docker-remote.registry.domain.fr
time="2025-01-24T10:18:09Z" level=error msg="registry.k8s.io/v2/coredns/coredns/manifests/v1.8.0: failed to get manifest response for tag, skipping (404): {\"errors\":[{\"code\":\"MANIFEST_UNKNOWN\",\"message\":\"Manifest has media type \\\"application/vnd.docker.distribution.manifest.list.v2+json\\\" but client accepts [\\\"application/vnd.docker.distribution.manifest.v1+json\\\"]\"}]}\n" client=docker-remote.registry.domain.fr
time="2025-01-24T10:18:09Z" level=error msg="registry.k8s.io/v2/ingress-nginx/controller/manifests/v1.11.2: failed to get manifest response for tag, skipping (404): {\"errors\":[{\"code\":\"MANIFEST_UNKNOWN\",\"message\":\"Manifest has media type \\\"application/vnd.oci.image.index.v1+json\\\" but client accepts [\\\"application/vnd.docker.distribution.manifest.v1+json\\\"]\"}]}\n" client=docker-remote.registry.domain.fr
time="2025-01-24T10:18:09Z" level=error msg="registry.k8s.io/v2/conformance/manifests/v1.17.5: failed to get manifest response for tag, skipping (404): {\"errors\":[{\"code\":\"MANIFEST_UNKNOWN\",\"message\":\"Manifest has media type \\\"application/vnd.docker.distribution.manifest.list.v2+json\\\" but client accepts [\\\"application/vnd.docker.distribution.manifest.v1+json\\\"]\"}]}\n" client=docker-remote.registry.domain.fr
time="2025-01-24T10:18:09Z" level=error msg="registry.k8s.io/v2/coredns/coredns/manifests/v1.8.3: failed to get manifest response for tag, skipping (404): {\"errors\":[{\"code\":\"MANIFEST_UNKNOWN\",\"message\":\"Manifest has media type \\\"application/vnd.docker.distribution.manifest.list.v2+json\\\" but client accepts [\\\"application/vnd.docker.distribution.manifest.v1+json\\\"]\"}]}\n" client=docker-remote.registry.domain.fr
time="2025-01-24T10:18:09Z" level=error msg="registry.k8s.io/v2/ingress-nginx/controller/manifests/v1.11.3: failed to get manifest response for tag, skipping (404): {\"errors\":[{\"code\":\"MANIFEST_UNKNOWN\",\"message\":\"Manifest has media type \\\"application/vnd.oci.image.index.v1+json\\\" but client accepts [\\\"application/vnd.docker.distribution.manifest.v1+json\\\"]\"}]}\n" client=docker-remote.registry.domain.fr
time="2025-01-24T10:18:09Z" level=error msg="registry.k8s.io/v2/conformance/manifests/v1.17.5-beta.0: failed to get manifest response for tag, skipping (404): {\"errors\":[{\"code\":\"MANIFEST_UNKNOWN\",\"message\":\"Manifest has media type \\\"application/vnd.docker.distribution.manifest.list.v2+json\\\" but client accepts [\\\"application/vnd.docker.distribution.manifest.v1+json\\\"]\"}]}\n" client=docker-remote.registry.domain.fr

Versions

Version-cecker : 0.8.5
K8S : 1.28

[davidcollom]

Hi, Thanks for raising this issue. As you may have seen (linked issues), this is a long-standing issue in supporting OCI Style repositories and is something we need to address in the future. Unfortunately, we don't have a full ETA on delivery as we try and maintain version-checker on a best-efforts basis.

There are key changes with OCI repositories that make version detection a little harder and we wish to implement the best solution to our users.