Reading · Andy Freeland · RSS Amplifier

This page did not load. You can still read it on the original site — the toolbar below keeps your place in the directory.
If you want small Docker images and fast builds, you're likely already using Docker's multi-stage builds . For Python applications, this often means one stage to download and compile any wheels and another stage to install them and add your application source code. For example: FROM python:3.10 AS build COPY requirements.txt . RUN pip wheel -r requirements.txt \ --wheel-dir /whl \…
If you want small Docker images and fast builds, you're likely already using Docker's multi-stage builds</a>. For Python applications, this often means one stage to download and compile any wheels and another stage to install them and add your application source code. For example:</p>
FROM</span> python:3.10 </span>AS</span> build</span></span>
COPY</span> requirements.txt .</span></span>
RUN</span> pip wheel -r requirements.txt \</span></span>
    --wheel-dir /whl \</span></span>
    --no-cache-dir</span></span>
</span>
FROM</span> python:3.10-slim</span></span>
COPY</span> --from=build /whl/*.whl /whl/</span></span>
RUN</span> pip install --no-deps /whl/*.whl</span></span></code></pre>
Given a small requirements.txt</code>, we can build this image with Docker
BuildKit</a>:</p>
❯</span> cat requirements.txt</span></span>
ciso8601</span></span>
flask</span></span>
numpy</span></span>
pandas</span></span>
</span>
❯</span> DOCKER_BUILDKIT</span>=</span>1</span> docker</span> build</span> -</span>-no-cache</span> -</span>t</span> python-with-copy</span> .</span></span>
[+] Building 22.3s (12/12) FINISHED</span></span>
 => [internal] load build definition from copy.Dockerfile                       0.0s</span></span>
 => => transferring dockerfile: 42B                                             0.0s</span></span>
 => [internal] load .dockerignore                                               0.0s</span></span>
 => => transferring context: 2B                                                 0.0s</span></span>
 => [internal] load metadata for docker.io/library/python:3.10-slim             0.0s</span></span>
 => [internal] load metadata for docker.io/library/python:3.10                  0.0s</span></span>
 => [internal] load build context                                               0.0s</span></span>
 => => transferring context: 37B                                                0.0s</span></span>
 => CACHED [build 1/3] FROM docker.io/library/python:3.10                       0.0s</span></span>
 => CACHED [stage-1 1/3] FROM docker.io/library/python:3.10-slim                0.0s</span></span>
 => [build 2/3] COPY requirements.txt .                                         0.0s</span></span>
 => [build 3/3] RUN pip wheel -r requirements.txt     --wheel-dir /whl     --n  9.5s</span></span>
 => [stage-1 2/3] COPY --from=build /whl/*.whl /whl/                            0.1s</span></span>
 => [stage-1 3/3] RUN pip install --no-deps /whl/*.whl                          9.5s</span></span>
 => exporting to image                                                          2.2s</span></span>
 => => exporting layers                                                         2.2s</span></span>
 => => writing image sha256:242b56c7f1b6f181986d62fcd80afde5287d5f2d8cdaddd96d  0.0s</span></span>
 => => naming to docker.io/library/python-with-copy                             0.0s</span></span></code></pre>
We can inspect the history of the image to view the sizes of each layer:</p>
❯</span> docker history python-with-copy</span></span>
IMAGE          CREATED         CREATED BY                                      SIZE      COMMENT</span></span>
242b56c7f1b6   8 minutes ago   RUN /bin/sh -c pip install --no-deps /whl/*.…   127MB     buildkit.dockerfile.v0</span></span>
<missing>      8 minutes ago   COPY /whl/*.whl /whl/ # buildkit                28.9MB    buildkit.dockerfile.v0</span></span>
<missing>      2 weeks ago     /bin/sh -c #(nop)  CMD ["python3"]              0B</span></span>
<missing>      2 weeks ago     /bin/sh -c set -ex;   savedAptMark="$(apt-ma…   9.51MB</span></span>
<missing>      2 weeks ago     /bin/sh -c #(nop)  ENV PYTHON_GET_PIP_SHA256…   0B</span></span>
<missing>      2 weeks ago     /bin/sh -c #(nop)  ENV PYTHON_GET_PIP_URL=ht…   0B</span></span>
<missing>      2 weeks ago     /bin/sh -c #(nop)  ENV PYTHON_SETUPTOOLS_VER…   0B</span></span>
<missing>      2 weeks ago     /bin/sh -c #(nop)  ENV PYTHON_PIP_VERSION=21…   0B</span></span>
<missing>      2 weeks ago     /bin/sh -c cd /usr/local/bin  && ln -s idle3…   32B</span></span>
<missing>      2 weeks ago     /bin/sh -c set -ex   && savedAptMark="$(apt-…   29.5MB</span></span>
<missing>      4 weeks ago     /bin/sh -c #(nop)  ENV PYTHON_VERSION=3.10.0    0B</span></span>
<missing>      4 weeks ago     /bin/sh -c #(nop)  ENV GPG_KEY=A035C8C19219B…   0B</span></span>
<missing>      4 weeks ago     /bin/sh -c set -eux;  apt-get update;  apt-g…   3.11MB</span></span>
<missing>      4 weeks ago     /bin/sh -c #(nop)  ENV LANG=C.UTF-8             0B</span></span>
<missing>      4 weeks ago     /bin/sh -c #(nop)  ENV PATH=/usr/local/bin:/…   0B</span></span>
<missing>      4 weeks ago     /bin/sh -c #(nop)  CMD ["bash"]                 0B</span></span>
<missing>      4 weeks ago     /bin/sh -c #(nop) ADD file:16dc2c6d1932194ed…   80.4MB</span></span></code></pre>
From this, we can see that COPY /whl/*.whl /whl/</code> adds 28.9MB to our Docker
image. We don't need the wheels in our final image, but we can't free up this
space with a RUN rm -rf /whl/</code> instruction, so what do we do?</p>
We can use Docker BuildKit's build
mounts</a>
feature. This is most commonly used to pass secrets or SSH keys to the build,
but we're interested in the bind</code> mount type, which "can be used to bind
files from other part of the build without copying"</em>.</p>
We use it like so:</p>
#</span> syntax=docker/dockerfile:1.2</span></span>
FROM</span> python:3.10 </span>AS</span> build</span></span>
COPY</span> requirements.txt .</span></span>
RUN</span> pip wheel -r requirements.txt \</span></span>
    --wheel-dir /whl \</span></span>
    --no-cache-dir</span></span>
</span>
FROM</span> python:3.10-slim</span></span>
RUN</span> --mount=type=bind,target=/whl,source=/whl,from=build \</span></span>
    pip install --no-deps /whl/*.whl</span></span></code></pre>
We can build this image:</p>
❯</span> DOCKER_BUILDKIT</span>=</span>1</span> docker</span> build</span> -</span>-no-cache</span> -</span>t</span> python-with-mount</span> .</span></span>
[+] Building 22.6s (15/15) FINISHED</span></span>
 => [internal] load build definition from mount.Dockerfile                      0.0s</span></span>
 => => transferring dockerfile: 43B                                             0.0s</span></span>
 => [internal] load .dockerignore                                               0.0s</span></span>
 => => transferring context: 2B                                                 0.0s</span></span>
 => resolve image config for docker.io/docker/dockerfile:1.3                    0.6s</span></span>
 => CACHED docker-image://docker.io/docker/dockerfile:1.3@sha256:42399d4635edd  0.0s</span></span>
 => [internal] load build definition from mount.Dockerfile                      0.0s</span></span>
 => [internal] load .dockerignore                                               0.0s</span></span>
 => [internal] load metadata for docker.io/library/python:3.10-slim             0.0s</span></span>
 => [internal] load metadata for docker.io/library/python:3.10                  0.0s</span></span>
 => [internal] load build context                                               0.0s</span></span>
 => => transferring context: 37B                                                0.0s</span></span>
 => CACHED [stage-1 1/2] FROM docker.io/library/python:3.10-slim                0.0s</span></span>
 => CACHED [build 1/3] FROM docker.io/library/python:3.10                       0.0s</span></span>
 => [build 2/3] COPY requirements.txt .                                         0.0s</span></span>
 => [build 3/3] RUN pip wheel -r requirements.txt     --wheel-dir /whl     --n  9.7s</span></span>
 => [stage-1 2/2] RUN --mount=type=bind,target=/whl,source=/whl,from=build      9.2s</span></span>
 => exporting to image                                                          2.1s</span></span>
 => => exporting layers                                                         2.0s</span></span>
 => => writing image sha256:1d677e47533d54467300be2c1b4898cb7b5d07b539877c2f07  0.0s</span></span>
 => => naming to docker.io/library/python-with-mount                            0.0s</span></span></code></pre>
And inspect its layers:</p>
❯</span> docker history python-with-mount</span></span>
IMAGE          CREATED          CREATED BY                                      SIZE      COMMENT</span></span>
1d677e47533d   59 seconds ago   RUN /bin/sh -c pip install --no-deps /whl/*.…   127MB     buildkit.dockerfile.v0</span></span>
<missing>      2 weeks ago      /bin/sh -c #(nop)  CMD ["python3"]              0B</span></span>
<missing>      2 weeks ago      /bin/sh -c set -ex;   savedAptMark="$(apt-ma…   9.51MB</span></span>
<missing>      2 weeks ago      /bin/sh -c #(nop)  ENV PYTHON_GET_PIP_SHA256…   0B</span></span>
<missing>      2 weeks ago      /bin/sh -c #(nop)  ENV PYTHON_GET_PIP_URL=ht…   0B</span></span>
<missing>      2 weeks ago      /bin/sh -c #(nop)  ENV PYTHON_SETUPTOOLS_VER…   0B</span></span>
<missing>      2 weeks ago      /bin/sh -c #(nop)  ENV PYTHON_PIP_VERSION=21…   0B</span></span>
<missing>      2 weeks ago      /bin/sh -c cd /usr/local/bin  && ln -s idle3…   32B</span></span>
<missing>      2 weeks ago      /bin/sh -c set -ex   && savedAptMark="$(apt-…   29.5MB</span></span>
<missing>      4 weeks ago      /bin/sh -c #(nop)  ENV PYTHON_VERSION=3.10.0    0B</span></span>
<missing>      4 weeks ago      /bin/sh -c #(nop)  ENV GPG_KEY=A035C8C19219B…   0B</span></span>
<missing>      4 weeks ago      /bin/sh -c set -eux;  apt-get update;  apt-g…   3.11MB</span></span>
<missing>      4 weeks ago      /bin/sh -c #(nop)  ENV LANG=C.UTF-8             0B</span></span>
<missing>      4 weeks ago      /bin/sh -c #(nop)  ENV PATH=/usr/local/bin:/…   0B</span></span>
<missing>      4 weeks ago      /bin/sh -c #(nop)  CMD ["bash"]                 0B</span></span>
<missing>      4 weeks ago      /bin/sh -c #(nop) ADD file:16dc2c6d1932194ed…   80.4MB</span></span></code></pre>
As expected, the COPY</code> step and its 28.9MB are gone. This is about 10% of our
initial image size freed up!</p>
Smaller Python Docker Images with Build Mounts

Comments
