1. Unveiling the Proxy Trio: http_proxy, https_proxy, and no_proxy

Ever tried downloading a file online but encountered a restrictive firewall (security barrier) blocking your way?

Fear not! This guide unveils the magic behind http_proxy and https_proxy, the environment variables that act as passports for your tools to navigate these gatekeepers.

Imagine curl, a popular Swiss army knife tool for downloading files from the internet, needs to access a website, but a firewall stands between them, potentially for restriction of information access, and security reasons. Here is where the environment variables http_proxy and https_proxy come in:

  • http_proxy specifies the address and port of a proxy server specifically for handling regular, unencrypted HTTP traffic.

    Think of it as a translator, converting your request into a format the proxy understands before forwarding it to the website.

  • https_proxy: caters to HTTPS traffic, which is encrypted for security.

    When set, curl establishes a secure tunnel with the proxy server using TLS/SSL before sending any data. It’s like whispering your request through a hidden passage.

  • no_proxy specifies a list of hostnames, domains, or IP addresses that should bypass the proxy server. Traffic destined for these entries will be sent directly to the internet without going through the proxy.

    export no_proxy="localhost,127.0.0.1,internal.mycompany.com"

    The no_proxy list takes precedence over http_proxy and https_proxy. Any traffic destined for a hostname or IP address listed in no_proxy will bypass the proxy, even if a proxy server is defined using the other variables.

In most cases, setting http_proxy and https_proxy to the same value (pointing to the same proxy server) is the common practice for proxying all traffic.

It’s important to note that no_proxy might support wildcard patterns (e.g., *.mycompany.com) depending on the specific tool or library interpreting the environment variable. However, it is not a guaranteed feature, and its behavior can vary across applications.

  • Whenever possible, it’s safer and more reliable to specify exact hostnames or IP addresses in no_proxy instead of relying on wildcards.

  • If you must use wildcards, thoroughly consult the documentation for the specific tool or library to understand how it interprets wildcard patterns in no_proxy.

Imagine you want to download a file using curl but a firewall with a proxy server is in place. Here’s how you would use the environment variables:

# Set the proxy server address and port (replace with your actual details)
export https_proxy=http://your_proxy_server:port

# Now, use curl to download the file
curl https://example.com/file.txt

By setting https_proxy, curl knows to route its request through the specified proxy server, allowing you to download the file despite the firewall and proxy combination.

While curl itself is generally case-insensitive for these variables, using lowercase (http_proxy and https_proxy) is recommended for consistency.

While http_proxy and https_proxy can be configured for SOCKS proxies by including the socks5:// prefix in the URL, it’s important to understand some limitations:

export http_proxy=socks5://proxy_server:port
export https_proxy=socks5://proxy_server:port

  • SOCKS proxies typically don’t handle DNS resolution themselves. You might need to configure a separate DNS server for your system to work correctly with a SOCKS proxy.

  • Not all applications are SOCKS proxy aware. Some applications might require additional configuration or may not work correctly through a SOCKS proxy.

With the understanding of http_proxy and https_proxy, you can navigate proxy servers and access the resources you need!

2. Unveiling the Proxy Trio with Docker and containerd

When working with containerized applications, pulling images from registries is a common task. But what if your network environment requires a proxy server for internet access? This guide explores how to configure https_proxy for secure communication with container image registries using Docker and containerd.

  • Docker: A container engine built on top of containerd, offering a user-friendly interface and additional functionalities for managing images and containers.

  • containerd: A container runtime engine that manages the lifecycle of containers (creation, starting, stopping, and deletion).

While Docker, built on top of containerd, manages container image pulling by default, containerd also has its own built-in image pulling plugin (e.g., used by Kubernetes).

To ensure consistent proxy behavior regardless of which tool pulls the image, it’s recommended to configure the proxy for both Docker and containerd.

  • Set the HTTPS_PROXY environment variable on systemd service files:

    • Locate the systemd service file for containerd and dockerd (e.g., /etc/systemd/system/docker.service).

    • Edit the file (e.g., sudo systemctl edit docker.service) and add the following lines under the [Service] section:

      [Service]
Environment="HTTPS_PROXY=https://your_proxy_server:port"

    • Reload systemd and restart the service:

      sudo systemctl daemon-reload
sudo systemctl restart docker

  • Configure the Docker daemon to use a proxy server in the daemon.json file (Recommended for Docker only):

    {
  "proxies": {
    "http-proxy": "http://proxy.example.com:3128",
    "https-proxy": "https://proxy.example.com:3129",
    "no-proxy": "*.test.example.com,.example.org,127.0.0.0/8"
  }
}

3. Install Docker Engine on Debian

  1. Set up Docker’s apt repository.

    # Add Docker's official GPG key:
sudo apt-get update
sudo apt-get install ca-certificates curl
sudo install -m 0755 -d /etc/apt/keyrings
sudo curl -fsSL https://download.docker.com/linux/debian/gpg -o /etc/apt/keyrings/docker.asc
sudo chmod a+r /etc/apt/keyrings/docker.asc

# Add the repository to Apt sources:
echo \
  "deb [arch=$(dpkg --print-architecture) signed-by=/etc/apt/keyrings/docker.asc] https://download.docker.com/linux/debian \
  $(. /etc/os-release && echo "$VERSION_CODENAME") stable" | \
  sudo tee /etc/apt/sources.list.d/docker.list > /dev/null
sudo apt-get update

    To use the APT source mirro from Alibaba Cloud, replace the https://download.docker.com with https://mirrors.aliyun.com/docker-ce at the /etc/apt/sources.list.d/docker.list.

  2. Install the Docker packages.

    • To install the latest version, run:

      sudo apt-get install docker-ce docker-ce-cli containerd.io docker-buildx-plugin docker-compose-plugin

    • To install a specific version of Docker Engine, start by listing the available versions in the repository:

      # List the available versions:
apt-cache madison docker-ce | awk '{ print $3 }'

5:26.1.0-1~debian.12~bookworm
5:26.0.2-1~debian.12~bookworm
...
      # Select the desired version and install:
VERSION_STRING=5:26.1.0-1~debian.12~bookworm
sudo apt-get install docker-ce=$VERSION_STRING docker-ce-cli=$VERSION_STRING containerd.io docker-buildx-plugin docker-compose-plugin