When you pull a Docker image, you will notice that it is pulled as different layers. Also, when you create your own Docker image, several layers are created. In this post we will try to get a better understanding of Docker layers.
1. What Is a Docker Layer?
A Docker image consists of several layers. Each layer corresponds to certain instructions in your Dockerfile. The following instructions create a layer: RUN, COPY, ADD. The other instructions will create intermediate layers and do not influence the size of your image. Let’s take a look at an example. We will use a Spring Boot MVC application which we have created before and where the Maven build creates our Docker image. The sources are available at GitHub. We use the feature/dockerbenchsecurity branch, which is a more secure version of the master branch. Here is the Dockerfile:
FROM openjdk:10-jdk
VOLUME /tmp
RUN useradd -d /home/appuser -m -s /bin/bash appuser
USER appuser
HEALTHCHECK --interval=5m --timeout=3s CMD curl -f http://localhost:8080/actuator/health/ || exit 1
ARG JAR_FILE
COPY ${JAR_FILE} app.jar
ENTRYPOINT ["java","-Djava.security.egd=file:/dev/./urandom","-jar","/app.jar"]
We build the application with mvn clean install which will also create the Docker image. We don’t list the pulling of all layers of the openjdk:10-jdk image for brevity reasons.
Image will be built as mydeveloperplanet/mykubernetesplanet:0.0.3-SNAPSHOT
Step 1/8 : FROM openjdk:10-jdk
Pulling from library/openjdk
Image 16e82e17faef: Pulling fs layer
...
Image a9448aba0bc3: Pull complete
Digest: sha256:9f17c917630d5e95667840029487b6561b752f1be6a3c4a90c4716907c1aad65
Status: Downloaded newer image for openjdk:10-jdk
---> b11e88dd885d
Step 2/8 : VOLUME /tmp
---> Running in 21329898c3a6
Removing intermediate container 21329898c3a6
---> b6f9ca000de6
Step 3/8 : RUN useradd -d /home/appuser -m -s /bin/bash appuser
---> Running in 82645047e6e7
Removing intermediate container 82645047e6e7
---> 04f6b2716819
Step 4/8 : USER appuser
---> Running in 697b663dadbb
Removing intermediate container 697b663dadbb
---> eaf6b8af5709
Step 5/8 : HEALTHCHECK --interval=5m --timeout=3s CMD curl -f http://localhost:8080/actuator/health/ || exit 1
---> Running in f420b9d060c5
Removing intermediate container f420b9d060c5
---> 77f95436a3ff
Step 6/8 : ARG JAR_FILE
---> Running in 60b9d25ad2ac
Removing intermediate container 60b9d25ad2ac
---> 135fa7df95ac
Step 7/8 : COPY ${JAR_FILE} app.jar
---> 63c18567012b
Step 8/8 : ENTRYPOINT ["java","-Djava.security.egd=file:/dev/./urandom","-jar","/app.jar"]
---> Running in 79203446934a
Removing intermediate container 79203446934a
---> 8e2b049f9783
Successfully built 8e2b049f9783
Successfully tagged mydeveloperplanet/mykubernetesplanet:0.0.3-SNAPSHOT
What happens here? We notice that layers are created and most of them are removed (removing intermediate container). So why does it say removing intermediate container and not removing intermediate layer? That’s because a build step is executed in an intermediate container. When the build step is finished executing, the intermediate container can be removed. Besides that, a layer is read-only. A layer contains the differences between the preceding layer and the current layer. On top of the layers, there is a writable layer (the current one) which is called the container layer. As mentioned before, only specific instructions will create a new layer. Let’s take a look at our Docker images:
$ docker image ls REPOSITORY TAG IMAGE ID CREATED SIZE mydeveloperplanet/mykubernetesplanet 0.0.3-SNAPSHOT 8e2b049f9783 About a minute ago 1GB openjdk 10-jdk b11e88dd885d 2 months ago 987MB
And take a look at the history of our mykubernetesplanet Docker image:
$ docker history 8e2b049f9783
IMAGE CREATED CREATED BY SIZE COMMENT
8e2b049f9783 About a minute ago /bin/sh -c #(nop) ENTRYPOINT ["java" "-Djav… 0B
63c18567012b About a minute ago /bin/sh -c #(nop) COPY file:2a5b71774c60e0f6… 17.4MB
135fa7df95ac About a minute ago /bin/sh -c #(nop) ARG JAR_FILE 0B
77f95436a3ff 2 minutes ago /bin/sh -c #(nop) HEALTHCHECK &{["CMD-SHELL… 0B
eaf6b8af5709 2 minutes ago /bin/sh -c #(nop) USER appuser 0B
04f6b2716819 2 minutes ago /bin/sh -c useradd -d /home/appuser -m -s /b… 399kB
b6f9ca000de6 2 minutes ago /bin/sh -c #(nop) VOLUME [/tmp] 0B
b11e88dd885d 2 months ago /bin/sh -c #(nop) CMD ["jshell"] 0B
<missing> 2 months ago /bin/sh -c set -ex; if [ ! -d /usr/share/m… 697MB
<missing> 2 months ago /bin/sh -c #(nop) ENV JAVA_DEBIAN_VERSION=1… 0B
...
We notice here, that the intermediate containers do have a size of 0B, just what was expected. Only the RUN and COPY command from the Dockerfile contribute to the size of the Docker image. The layers of the openjdk:10-jdk image are also listed and are recognized by the missing keyword. This only means that those layers are built on a different system and are not available locally.
2. Recreate the Docker Image
What happens if we run our Maven build again without making any changes to the sources?
Image will be built as mydeveloperplanet/mykubernetesplanet:0.0.3-SNAPSHOT
Step 1/8 : FROM openjdk:10-jdk
Pulling from library/openjdk
Digest: sha256:9f17c917630d5e95667840029487b6561b752f1be6a3c4a90c4716907c1aad65
Status: Image is up to date for openjdk:10-jdk
---> b11e88dd885d
Step 2/8 : VOLUME /tmp
---> Using cache
---> b6f9ca000de6
...
Step 6/8 : ARG JAR_FILE
---> Using cache
---> 135fa7df95ac
Step 7/8 : COPY ${JAR_FILE} app.jar
---> 409f2fee0cde
Step 8/8 : ENTRYPOINT ["java","-Djava.security.egd=file:/dev/./urandom","-jar","/app.jar"]
---> Running in 75f07955bbc8
Removing intermediate container 75f07955bbc8
---> e5d7b72aad05
Successfully built e5d7b72aad05
Successfully tagged mydeveloperplanet/mykubernetesplanet:0.0.3-SNAPSHOT
We notice that the first layers are identical to our previous build. The layer ID’s are the same. In the log we notice that the layers are taken from the cache. In step 7 a new layer is created with a new ID. We did create a new JAR file, Docker interprets this as a new file and therefore a new layer is created. In step 8 also a new layer is created, because it is build on top of the new layer.
Let’s list the Docker images again:
$ docker image ls REPOSITORY TAG IMAGE ID CREATED SIZE mydeveloperplanet/mykubernetesplanet 0.0.3-SNAPSHOT e5d7b72aad05 13 seconds ago 1GB <none> <none> 8e2b049f9783 5 minutes ago 1GB openjdk 10-jdk b11e88dd885d 2 months ago 987MB
Our tag 0.0.3-SNAPSHOT has received the image ID of our last build. The repository and tag of our old image ID are removed, which is indicated with the none keyword. This is called a dangling image. We will explain this in more detail at the end of this post.
When we take a look at the history of the newly created image, we notice that the two top layers are new, just as in the build log:
$ docker history e5d7b72aad05 IMAGE CREATED CREATED BY SIZE COMMENT e5d7b72aad05 38 seconds ago /bin/sh -c #(nop) ENTRYPOINT ["java" "-Djav… 0B 409f2fee0cde 42 seconds ago /bin/sh -c #(nop) COPY file:4b04c6500d340c9e… 17.4MB 135fa7df95ac 6 minutes ago /bin/sh -c #(nop) ARG JAR_FILE 0B ...
When we make a source code change, the result is identical because also in this case a new JAR file is generated.
$ docker image ls REPOSITORY TAG IMAGE ID CREATED SIZE mydeveloperplanet/mykubernetesplanet 0.0.3-SNAPSHOT eced642d4f5c 30 seconds ago 1GB <none> <none> e5d7b72aad05 3 minutes ago 1GB <none> <none> 8e2b049f9783 8 minutes ago 1GB openjdk 10-jdk b11e88dd885d 2 months ago 987MB $ docker history eced642d4f5c IMAGE CREATED CREATED BY SIZE COMMENT eced642d4f5c About a minute ago /bin/sh -c #(nop) ENTRYPOINT ["java" "-Djav… 0B 44a9097b8bad About a minute ago /bin/sh -c #(nop) COPY file:1d5276778b53310e… 17.4MB 135fa7df95ac 9 minutes ago /bin/sh -c #(nop) ARG JAR_FILE 0B ...
3. What About Size?
Let’s take a closer look at the latest output of the docker image ls command, we notice two dangling images, which are 1 GB in size. But what does this really mean for storage? First, we need to know where the data for our images is stored. Use the following command in order to retrieve the storage location:
$ docker image inspect eced642d4f5c
...
"GraphDriver": {
"Data": {
"LowerDir": "/var/lib/docker/overlay2/655be8bea8e54c31ebb7e3adf05db227d194a49c1e2f95552d593d623e024b92/diff:/var/lib/docker/overlay2/993f77b91a487e19b3696836efee23c8a17791d71096d348c54c38fba3dc8478/diff:/var/lib/docker/overlay2/d62d6ca8ce1960d057e11d163d458563628e5a337de06455e714900f72005589/diff:/var/lib/docker/overlay2/cabdf4de81557a8047e3670bd2eecb5449de7de8fe9dfd4ad0c81d7dd2c61e9d/diff:/var/lib/docker/overlay2/062bf99d6a563ee2ef7824ec02ff5cd09fb8721cb23f6a55f8927edc2607f9c1/diff:/var/lib/docker/overlay2/ba024c24b20771dbf409f501423273e13225cf675f30896720cadace1c7be000/diff:/var/lib/docker/overlay2/d15f4477b53508127bebd1224c9ea09cd767f7db7429ffb1e8aa79b01ab77506/diff:/var/lib/docker/overlay2/ea434348d6625bc49875d0aba886b24ff0e1e204a350099981dcfc4029bc688d/diff:/var/lib/docker/overlay2/05e003c0522c7049110aa3ce09814ff2167da1e53ec83481fef03324011ce6e6/diff",
"MergedDir": "/var/lib/docker/overlay2/205b55ee2f0e06394b6d17067338845410609887ccd18f53bf0646ff60452ffb/merged",
"UpperDir": "/var/lib/docker/overlay2/205b55ee2f0e06394b6d17067338845410609887ccd18f53bf0646ff60452ffb/diff",
"WorkDir": "/var/lib/docker/overlay2/205b55ee2f0e06394b6d17067338845410609887ccd18f53bf0646ff60452ffb/work"
},
"Name": "overlay2"
},
...
Our Docker images are stored at /var/lib/docker/overlay2. We can simply retrieve the size of the overlay2 directory in order to have an idea of the allocated storage:
$ du -sh -m overlay2 1059 overlay2
The openjdk:10-jdk image is 987 MB. The JAR file in our image is 17,4 MB. The total size should be somewhere around 987 MB + 3 * 17,4 MB (two dangling images and one real image). This is about 1040 MB. We can conclude that there is some sort of smart storage for the Docker images and that we cannot simply add all the sizes of the Docker images in order to retrieve the real storage size. The difference is due to the existence of intermediate images. These can be revealed as follows:
$ docker images -a REPOSITORY TAG IMAGE ID CREATED SIZE mydeveloperplanet/mykubernetesplanet 0.0.3-SNAPSHOT eced642d4f5c 7 days ago 1GB <none> <none> 44a9097b8bad 7 days ago 1GB <none> <none> e5d7b72aad05 7 days ago 1GB <none> <none> 409f2fee0cde 7 days ago 1GB <none> <none> 8e2b049f9783 7 days ago 1GB <none> <none> 63c18567012b 7 days ago 1GB <none> <none> 135fa7df95ac 7 days ago 987MB <none> <none> 77f95436a3ff 7 days ago 987MB <none> <none> eaf6b8af5709 7 days ago 987MB <none> <none> 04f6b2716819 7 days ago 987MB <none> <none> b6f9ca000de6 7 days ago 987MB openjdk 10-jdk b11e88dd885d 2 months ago 987MB
4. Get Rid of Dangling Images
How do we get rid of these dangling images? We don’t need them anymore and they only allocate storage. First, list the dangling images:
$ docker images -f dangling=true REPOSITORY TAG IMAGE ID CREATED SIZE <none> <none> e5d7b72aad05 7 days ago 1GB <none> <none> 8e2b049f9783 7 days ago 1GB
We can use the docker rmi command to remove the images:
$ docker rmi e5d7b72aad05 Deleted: sha256:e5d7b72aad054100d142d99467c218062a2ef3bc2a0994fb589f9fc7ff004afe Deleted: sha256:409f2fee0cde9b5144f8e92887b61e49f3ccbd2b0e601f536941d3b9be32ff47 Deleted: sha256:2162a2af22ee26f7ac9bd95c39818312dc9714b8fbfbeb892ff827be15c7795b
Or you can use the docker image prune command to do so.
Now that we have removed the dangling images, let’s take a look at the size of the overlay2 directory:
$ du -sh -m overlay2 1026 overlay2
We got rid of 33 MB. This seems not so much, but when you often build Docker images, this can grow significantly over time.
5. Conclusion
In this post we tried to get a better understanding of Docker layers. We noticed that intermediate layers are created and that dangling images remain on our system if we don’t clean them regularly. We also inspected the size of the Docker images on our system.
mydeveloperplanet.com 