Monday, December 15, 2025

Podman Build Fails: HTTP vs HTTPS Registry


Problem Statement

A podman build command worked on one machine but failed on another with the following error:

http: server gave HTTP response to HTTPS client

The failing build was attempting to pull a base image from a private registry:

FROM server1.mycompany.com:9001/mccbase_jdk21:base_25.2

Interestingly:

  • The working machine did not have insecure=true set system-wide

  • The same user, Dockerfile, and registry were involved

This made the issue non-obvious and confusing.


Initial Observations

  • Podman printed a warning about a missing build argument:

    missing "MAVEN_BUILD_ENV" build argument
    

    This was not fatal and unrelated to the failure.

  • The actual failure happened when Podman tried to pull the base image:

    Error: pinging container registry ... https://host:9001/v2/
    http: server gave HTTP response to HTTPS client
    

This indicates:

  • The registry only supports HTTP

  • Podman attempted to connect using HTTPS


Key Insight

Podman’s registry behavior is client-side configurable and can differ per machine and per user.

Even if:

  • /etc/containers/registries.conf does not list the registry as insecure

Podman may still allow HTTP due to:

  • User-specific configuration

  • Different Podman versions

  • Rootless vs root Podman

  • Cached or legacy behavior


Root Cause

The working machine had a user-specific Podman registry configuration, while the failing machine did not.

Podman reads registry config in this order (simplified):

  1. ~/.config/containers/registries.conf (user-specific)

  2. /etc/containers/registries.conf (system-wide)

The working machine already allowed HTTP at the user level, which explained why:

  • It worked without --tls-verify=false

  • No system-wide insecure registry was needed


Final Working Configuration (User-Specific)

Creating the following file resolved the issue on the failing machine:

~/.config/containers/registries.conf
unqualified-search-registries = [
  "registry.access.redhat.com",
  "registry.redhat.io",
  "docker.io"
]

[[registry]]
location = "server1.mycompany.com:9001"
insecure = true [[registry]] location = "server1.mycompany.com:8082"
insecure = true

After this change:

  • Podman correctly used HTTP for the private registry

  • The build succeeded without --tls-verify=false

  • Behavior matched the working machine


Why This Matters

This issue is common when:

  • Moving builds between machines

  • Upgrading Podman or OS versions

  • Switching users or using rootless Podman

  • Working with private, HTTP-only registries

Relying on implicit behavior can cause sudden failures.


Best Practice Recommendation

Even if things “work” on one machine, make registry behavior explicit:

  • Prefer user-specific registries.conf if you don’t control the system

  • Use system-wide config for shared build hosts

  • Avoid --tls-verify=false except for temporary debugging

Explicit configuration prevents:

  • Silent breakage after upgrades

  • Environment-specific surprises

  • CI/CD inconsistencies



Symptom

  • Podman build fails with HTTP/HTTPS mismatch

Cause

  • Missing user-level registry configuration

Fix

  • Add the registry as insecure=true in:

    ~/.config/containers/registries.conf
    

No comments:

Post a Comment