Host Your Own Docker Registry with Kamal 2

DHH commented in his 2024 RailsWorld Keynote that "you shouldn't have to use DockerHub or any other registry service to run [Kamal 2]."

Until Kamal 2 has first-class support for running your own private Docker registry, we can use a minor trick with kamal-proxy to make it work today.

This technique also works well for hosting other accessories that require a web interface. Hans Schnedlitz recently tweeted about using the same technique to deploy Puny Monitor as an accessory on the same server as his app.

We will use several new features of Kamal 2 to make this happen:

• kamal-proxy: Simplifies routing and SSL termination
• kamal network: Allows containers to communicate with each other privately
• Aliases: Enables more flexible configuration and maintains an "Infrastructure as Code" workflow even with the few tricks we need to perform

We will also need to bypass the default kamal setup steps and manually bootstrap our server because docker login will not work until the registry is up and running.

Step-by-Step Guide to Host Your Own Docker Registry with Kamal 2

Prerequisites

This guide assumes that you have:

• An app you want to deploy with Kamal 2
• A server you want to deploy it to that is ready for Kamal 2 but does not have anything set up yet
• A domain name that you want to use for the registry
• Kamal 2 installed on your local machine
• You've run kamal init locally for your app

See the Kamal 2 installation guide for help getting these set up if you haven't done so already.

1. Create a password file for the Docker Registry

We will use basic HTTP auth to secure the private registry. The Docker Registry image already supports including a password file generated with htpasswd, so we just need to create that file.

We can generate that file locally at config/registry.htpasswd with this command:

docker run --entrypoint htpasswd httpd:2 -Bbn testuser testpassword > config/registry.htpasswd

Remember to replace testuser and testpassword with your own username and password.

htpasswd stores these credentials hashed, so we can safely store the file in our repository. But remember to store the plaintext password securely in your secrets management tool of choice.

2. Update your [object Object] file to add and use the registry

We will add/update three components in the app's deploy.yml file:

1. Configure registry: settings to use the server and credentials of our private registry
2. Add the registry accessory using the official registry Docker image
3. Add an alias to configure kamal-proxy to route traffic to the registry accessory on the registry's domain name

Here's an example deploy.yml that adds these components:

registry:
  server: registry.myhost.com # replace with your domain name
  username: testuser
  password: 
    - MY_REGISTRY_PASSWORD # retreive password from kamal secrets

aliases:
  # Replace "SERVICE" with the service name configured for this app
  # Set "--host" to the domain name you want to use for the registry
  add-registry-to-proxy: |
    server exec docker exec kamal-proxy kamal-proxy deploy registry 
    --target "SERVICE-registry:5000"
    --host "registry.myhost.com" 
    --tls 
    --deploy-timeout "30s" 
    --drain-timeout "30s" 
    --health-check-path "/" 
    --buffer-requests 
    --buffer-responses 
    --log-request-header "Cache-Control" 
    --log-request-header "Last-Modified" 
    --log-request-header "User-Agent"

accessories:
  registry:
    image: registry:latest
    host: HOST # replace with server ip address
    port: "127.0.0.1:5000:5000" # exposes port 5000 only to the kamal network
    env:
      clear:
        REGISTRY_AUTH: htpasswd
        REGISTRY_AUTH_HTPASSWD_REALM: Registry Realm
        REGISTRY_AUTH_HTPASSWD_PATH: /auth/htpasswd
    files:
      - config/registry.htpasswd:/auth/htpasswd # mount the password file
    directories:
      - data:/var/lib/registry # data directory for the registry

3. Bootstrap the server

As previously mentioned, we cannot use the default kamal setup command because docker login will not work until the registry is up and running.

Instead, run a couple of commands to set up kamal on the server, install docker, and start kamal-proxy:

kamal server bootstrap
kamal proxy boot 

4. Boot the accessory

We can now manually boot the registry accessory:

kamal accessory boot registry

5. Add the registry to the proxy

Finally, we will configure the proxy to route traffic to the registry. We already set up an alias to make this step easy:

kamal add-registry-to-proxy

The alias uses kamal-proxy deploy to connect the registry container to the public internet on the desired domain name with --tls.

Adjust the alias as needed to fit your specific use case.

Test the registry and deploy

We can now test the registry by running docker login on our local machine to make sure we can connect to it:

docker login registry.myhost.com -u testuser

If everything is working, you should see output that looks like this:

Login Succeeded

And we are ready to deploy!

Now we can run kamal deploy and everything will work as expected with your images built by kamal pushed to your new private registry:

kamal deploy

Conclusion

With Kamal 2 and a few techniques to self-host a private Docker registry, we can now fully self-host our app on a single server with no external dependencies!