Automatic Builds at Your Fingertips With GCP Cloud Build

You are looking for an easy way to automatically build your application in the Cloud? Then maybe Google Cloud Platform (GCP) Cloud Build is something for you. In this post, we will build a Spring Boot Maven project with Cloud Build, create a Docker image for it and push it to GCP Container Registry.

1. Introduction

Cloud Build is the build server tooling of GCP, something similar as Jenkins. But, Cloud Build is available out-of-the-box in your GCP account and that is a major advantage. The only thing you will need, is a build configuration file in your git repository containing the build steps and that is about it. Each build step is running in its own Docker container. Several cloud builders which can be used as a build step are generally available. You can read more about Cloud Build on the overview and concepts website of GCP. There are three categories of build steps:

• Official cloud builders provided by GCP, e.g. a Maven builder is available here;
• Community cloud builders, e.g. a SonarQube builder is available here;
• You can create your own Docker image which can be used in a build step. We will create a custom Docker image in this post later on.

Before getting started, you will need to create a GCP account if you don’t already have one. Check out the first paragraph of a previous post how to do so.

First, we need to create a new project mygcpcloudbuild.

Go to the left menu to Tools – Cloud Build and click e.g. the History item.

A message is shown that we first need to enable the Cloud Build API.

After enabling the Cloud Build API, we are ready to go!

2. Create the Example Application

We are going to create a Spring Boot application, therefore we go to the SpringInitializr website. We select Web and Java 11 and we add a HelloController to the project which returns a simple welcome message. Sources can be found at GitHub (use the tag withoutcustommavendocker first, the master branch contains changes needed at the end of this post). Before we pushed everything to GitHub, we verified whether the project builds locally:

$ mvn clean install -DskipTests

3. Create the Build File

As mentioned in the introduction, we need to create a build configuration file cloudbuild.yaml in the root of our git repository. First, we will only add a Maven build step and verify whether this builds successfully on Cloud Build. It would be logical to choose for the official Maven Cloud builder, but, at the time of writing, only JDK 8 and Maven version 3.3.9 and 3.5.0 are supported. We are using JDK 11 in the application and therefore we will use the official Maven Docker image with Maven 3.6.0. We add the following cloudbuild.yaml file to the root of the git repository:

steps:
  - name: maven:3.6.0-jdk-11-slim
    entrypoint: 'mvn'
    args: ['clean', 'install', '-DskipTests']

4. Build on Cloud Build

Now that our git repository is ready, we return to GCP. We want a build to be triggered on every commit to our repository. Navigate to the menu and click Cloud Build – Triggers. The triggers are still a beta feature, but it worked fine during our experiment with Cloud Build. Click the Create Trigger button.

We need to select a repository. We choose GitHub and click the Continue button.

In the next step, we need to authorize GCP to access our repository. Enter your GitHub credentials and continue.

We select the repository we want to build and click Continue.

And finally, we need to configure some settings. In the Build Configuration section, we choose for Cloud Build configuration file (yaml or json) and click the Create Trigger button. You can specify more things over here, e.g. which branches you want to build.

The build triggers overview now contains the trigger we just created. We start our first build by clicking the Run Trigger button.

The build history now contains an entry for our successful build.

5. Add Docker Build Step

Now that our Maven build is successful, it is time to add a build step for creating a Docker image for our application. We therefore create the following Dockerfile and add it to the root of our git repository:

FROM openjdk:11-jdk
VOLUME /tmp
ADD target/mygcpcloudbuildplanet-0.0.1-SNAPSHOT.jar app.jar
ENTRYPOINT ["java","-Djava.security.egd=file:/dev/./urandom","-jar","/app.jar"]

We adapt the cloudbuild.yaml file with an extra build step for creating the Docker image and for pushing it to the GCP Container Registry.

  - name: 'gcr.io/cloud-builders/docker'
    args: ['build', '-t', 'gcr.io/$PROJECT_ID/mygcpcloudbuildplanet', '.']
images: ['gcr.io/$PROJECT_ID/mygcpcloudbuildplanet']

After pushing these changes to the git repository, a build is automatically triggered at Cloud Build. The Docker image is available in the Container Registry after the build has finished.

The Maven build is successful, the Docker build is successful, the Docker image is available in the Docker repository. But does the application run successfully? Let’s find out. Open the Cloud Shell in GCP and run the Docker image in detached mode.

$ docker run -p 127.0.0.1:8080:8080/tcp -d gcr.io/mygcpcloudbuild/mygcpcloudbuildplanet

Verify whether the Docker container is running with the docker ps command:

$ docker ps
CONTAINER ID    IMAGE                                            COMMAND                  CREATED           STATUS           PORTS                       NAMES
f17203279c03    gcr.io/mygcpcloudbuild/mygcpcloudbuildplanet    "java -Djava.securit…"    18 seconds ago    Up 17 seconds    127.0.0.1:8080->8080/tcp    jolly_mcclintock

The container is up-and-running, we verify the response of our welcome message:

$ curl http://localhost:8080/hello
Hello Google Cloud Build! From host: f17203279c03/172.18.0.2

The URL can also be accessed via the Web Preview feature of Cloud Shell, you only need to add hello to the URL otherwise a 404 HTTP error is presented:

https://8080-dot-6340638-dot-devshell.appspot.com/hello/?authuser=0

Pretty cool, isn’t it? We just created in a few minutes a basic build pipeline with Cloud Build.

6. Something About Maven Dependencies

We are using a Docker image for building the application. This means that for each build, all the Maven dependencies are downloaded again. This isn’t a major problem for a very small application, but the number of Maven dependencies can grow significantly over time. If this happens, it will contribute for a specific amount of your build time. Three options are available for resolving this issue:

1. Do nothing, we can accept the extra build time and it doesn’t bother us;
2. We can try to cache the Maven repository between builds, a solution which is provided at the Optimizing build speed page;
3. We can create a custom Docker Maven image which contains the Maven dependencies for our build as described here.

We already tried solution 1, let’s take a closer look at the two other solutions.

6.1 Cache Maven Repository Between Builds

The solution in paragraph Caching directories with Google Cloud Storage at the Optimizing build speed page suggests to copy the results of a previous build to a Google Cloud Storage bucket and then copy them again before a next build. The changes to the git repository for this experiment can be found in branch feature/mavencache at GitHub.

Via the menu in GCP, go to Storage – Browser and create a bucket com-mydeveloperplanet-mavenrepo. We will use this storage for our Maven repository.

Change the cloudbuild.yaml file. We have 3 build steps:

1. Copy the Maven dependencies from the bucket to the Docker volume mavenrepo;
2. Run the Maven build where we also mapped the Docker volume mavenrepo in order that the Maven dependencies don’t need to be downloaded;
3. Copy the Maven dependencies back to the bucket. New downloaded Maven dependencies will always be available in the bucket for the next build.

steps:
  # Copy Maven dependencies to volume mavenrepo
  - name: gcr.io/cloud-builders/gsutil
    volumes:
      - name: 'mavenrepo'
        path: '/root/.m2'
    args: ['cp', '-r', 'gs://com-mydeveloperplanet-mavenrepo', '/root/.m2']
  # Run the Maven build
  - name: maven:3.6.0-jdk-11-slim
    volumes:
      - name: 'mavenrepo'
        path: '/root/.m2'
    entrypoint: 'mvn'
    args: ['clean', 'install', '-DskipTests']
  # Preserve the Maven dependencies into the bucket
  - name: gcr.io/cloud-builders/gsutil
    volumes:
      - name: 'mavenrepo'
        path: '/root/.m2'
    args: ['cp', '-r', '/root/.m2', 'gs://com-mydeveloperplanet-mavenrepo']

Running this build the first time returns the following error:

Starting Step #0
Step #0: Already have image (with digest): gcr.io/cloud-builders/gsutil
Step #0: CommandException: No URLs matched: gs://com-mydeveloperplanet-mavenrepo
Finished Step #0
ERROR
ERROR: build step 0 "gcr.io/cloud-builders/gsutil" failed: exit status 1

This seems to be an existing issue with the gsutil command. Adding a random file into the bucket solves the problem. After the build, the complete Maven repository needed for our build is located in the bucket. Initially, the first build step did not need to copy anything. The next builds we executed always failed due to a build timeout. When analyzing the build step time a bit closer, we notice that copying the Maven repository to the bucket and vice versa takes more than 6 minutes. To conclude with, the build with downloading the Maven dependencies takes approximately 54 seconds, the build with caching the Maven repository between builds will take approximately 13 minutes. Not really an increase in speed. This isn’t a good solution.

6.2 Custom Docker Maven Image

Another solution for caching Maven dependencies is to create a custom Docker image based on the Maven Docker image but extended with the Maven dependencies. We create a dockerfile named Dockerfile-maven in order to create the custom Docker image.

FROM maven:3.6.0-jdk-11-slim as target
WORKDIR /build
COPY pom.xml .
RUN mvn dependency:go-offline

Go to Cloud Shell and clone the git repository:

$ git clone https://github.com/mydeveloperplanet/mygcpcloudbuildplanet.git

Navigate to the directory mygcpcloudbuildplanet and build the Docker image:

$ docker build -f Dockerfile-maven -t gcr.io/mygcpcloudbuild/mymavengcpcloudbuild .

List the Docker images and verify the presence of the Docker image:

$ docker images
REPOSITORY                                    TAG                  IMAGE ID        CREATED           SIZE
gcr.io/mygcpcloudbuild/mymavengcpcloudbuild   latest               8371e285038d    17 seconds ago    572MB
maven                                         3.6.0-jdk-11-slim    c7428be691f8    3 weeks ago       489MB

Now push the Docker image to the Container Registry:

$ docker push gcr.io/mygcpcloudbuild/mymavengcpcloudbuild

Verify the presence of the Docker image in the Container Registry.

Change the cloudbuild.yaml file in order that the Maven build step uses the custom Docker image instead of the official Maven Docker image:

steps:
  # Run the Maven build
  - name: gcr.io/mygcpcloudbuild/mymavengcpcloudbuild
    entrypoint: 'mvn'
    args: ['clean', 'install', '-DskipTests']
  - name: 'gcr.io/cloud-builders/docker'
    args: ['build', '-t', 'gcr.io/$PROJECT_ID/mygcpcloudbuildplanet', '.']
images: ['gcr.io/$PROJECT_ID/mygcpcloudbuildplanet']

It is important that you adapt the cloudbuild.yaml in the master branch, because this is the build configuration which is being used.

Let’s take a look at the results. A build with the official Maven Docker image takes 1 minute and 8 seconds including downloading the Maven dependencies and creating and pushing the Docker image to the Container Registry. A build with the custom Maven Docker image takes 47 seconds. A difference of approximately 15 to 20 seconds for a simple project like ours. It is definitely beneficial to use a custom Maven Docker image for your builds.

7. Conclusion

GCP Cloud Build is a very easy way to set up a build pipeline in just a few minutes. You can make use of provided cloud builders, but it is also easy to create custom ones. It is advised to use a custom cloud builder for Maven builds in order to speed up build time.