Against Entropy: gradle

Showing posts with label gradle. Show all posts

Friday, September 1, 2017

Docker Java Example Part 4: Bmuschko and Nebula Gradle Docker Plugins

Docker Java Example Series

Converting from the transmode gradle plugin to the bmuschko remote api gradle plugin was pretty straightforward. Other than importing and applying the plugin, the code to get local docker image creation working is as follows:

Note that bmuschko does support multiple image tags, and I took advantage of that to get the versioned tag as well as the "latest" tag.

REPOSITORY                      TAG                 IMAGE ID            CREATED             SIZE
ryanmckay/java-docker-example   0.0.1-SNAPSHOT      7fd01d5b247f        6 seconds ago       115MB
ryanmckay/java-docker-example   latest              7fd01d5b247f        6 seconds ago       115MB

I tagged the code repo at this point v0.4.1

Java Application plugin

In addition to the low-level remote api plugin, bmuschko offers an opinionated docker-java-application plugin based on the application gradle plugin. Using the opinionated plugin cuts down dramatically on the boilerplate in the build.gradle:

Unfortunately, this task only supports one tag. By default, you get the versioned one.

REPOSITORY                      TAG                 IMAGE ID            CREATED             SIZE
ryanmckay/java-docker-example   0.0.1-snapshot      415a9e4b201d        3 seconds ago       115MB

The generated Dockerfile looks like this:

As an interesting side note, the ADD Dockerfile directive has special behavior when the file being added is a tar file. In that case, it unpacks it to the destination.

The application gradle plugin is a more generic method of packaging up a java application than that offered by the spring boot plugin. It creates a tar file containing the application jar and all the dependency jars. It also contains a shell script for launching the application, which has OS detection and some OS-specific config.

$ tar tf build/distributions/java-docker-example-0.0.1-SNAPSHOT.tar 
java-docker-example-0.0.1-SNAPSHOT/
java-docker-example-0.0.1-SNAPSHOT/lib/
java-docker-example-0.0.1-SNAPSHOT/lib/java-docker-example-0.0.1-SNAPSHOT.jar
java-docker-example-0.0.1-SNAPSHOT/lib/spring-boot-starter-1.5.4.RELEASE.jar
java-docker-example-0.0.1-SNAPSHOT/lib/spring-boot-starter-web-1.5.4.RELEASE.jar
...
java-docker-example-0.0.1-SNAPSHOT/bin/
java-docker-example-0.0.1-SNAPSHOT/bin/java-docker-example
java-docker-example-0.0.1-SNAPSHOT/bin/java-docker-example.bat

I started using gradle about the same time I started using spring boot (which has its own gradle plugin with executable jar packaging), so wasn't familiar with the application plugin. It makes sense that bmuschko would base the opinionated plugin on that, so it can support all types of java applications, not just spring boot. However, since I plan to exclusively use spring boot for the foreseeable future, and can completely specify the execution environment in Docker (so don't need the OS-related functionality provided by the application plugin), I want to stick with Spring Boot application packaging and running.
I left the modifications in a branch tagged as v0.4.2

Nebula docker gradle plugin

Netflix publishes a set of plugins for gradle called Nebula. The nebula-docker-plugin is another opinionated plugin built on top of the bmuschko and application plugins. It doesn't seem to add a lot beyond the bmuschko application plugin, other than the concept of separate test and production repositories for publishing docker images. I'm going to look into docker deployment models next so it might come into play there.

Tuesday, August 29, 2017

Docker Java Example Part 3: Transmode Gradle plugin

Docker Java Example Series

At last we get to some Docker in this Docker example.

Gradle Docker Plugin

There are a few prominent docker plugins for gradle: transmode, bmuschko, and Netflix nebula. First, I used transmode, as recommended in the spring boot guide Spring Boot with Docker. After adding the Dockerfile:

FROM openjdk:8-jdk-alpine
VOLUME /tmp
ADD target/java-docker-example-0.0.1-SNAPSHOT.jar app.jar
ENV JAVA_OPTS=""
ENTRYPOINT [ "sh", "-c", "java $JAVA_OPTS -Djava.security.egd=file:/dev/./urandom -jar /app.jar" ]

, and updating build.gradle as described in the guide, I was able to build a docker image for my application:

$ gw clean build buildDocker --info

Setting up staging directory.
Creating Dockerfile from file /Users/ryanmckay/projects/java-docker-example/java-docker-example/src/main/docker/Dockerfile.
Determining image tag: ryanmckay/java-docker-example:0.0.1-SNAPSHOT
Using the native docker binary.
Sending build context to Docker daemon  14.43MB
Step 1/5 : FROM openjdk:8-jdk-alpine
---> 478bf389b75b
Step 2/5 : VOLUME /tmp
---> Using cache
---> 136f2d4e58dc
Step 3/5 : ADD target/java-docker-example-0.0.1-SNAPSHOT.jar app.jar
---> b3b47b89bbf1
Removing intermediate container 92f637bc67e0
Step 4/5 : ENV JAVA_OPTS ""
---> Running in e90c9a3557eb
---> 1d3f6526e8e5
Removing intermediate container e90c9a3557eb
Step 5/5 : ENTRYPOINT sh -c java $JAVA_OPTS -Djava.security.egd=file:/dev/./urandom -jar /app.jar
---> Running in 2fbfb52f836d
---> f001bdddc80b
Removing intermediate container 2fbfb52f836d
Successfully built f001bdddc80b
Successfully tagged ryanmckay/java-docker-example:0.0.1-SNAPSHOT

$ docker images
REPOSITORY                       TAG               IMAGE ID        CREATED          SIZE
ryanmckay/java-docker-example    0.0.1-SNAPSHOT    f001bdddc80b    3 minutes ago    115MB

$ docker run -p 8080:8080 -t ryanmckay/java-docker-example:0.0.1-SNAPSHOT

Automatically Tracking Application Version

Note that the Dockerfile at this point has the application version hard-coded in it. This duplication must not stand. The transmode gradle plugin also supports a dsl for specifying the Dockerfile in build.gradle. Then as part of the build process, it produces the actual Dockerfile.

I set about moving line by line of the Dockerfile into the dsl. With one exception it went smoothly. You can see the result in v0.3 of the app. The relevant portion of build.gradle is listed here. You can see its pretty much a line for line translation of the Dockerfile. And since we have access to the jar filename in the build script, nothing needs to be hard coded for docker.

// for docker
group = 'ryanmckay'

docker {
 baseImage 'openjdk:8-jdk-alpine'
}

task buildDocker(type: Docker, dependsOn: build) {
 applicationName = jar.baseName
 volume('/tmp')
 addFile(jar.archivePath, 'app.jar')
 setEnvironment('JAVA_OPTS', '""')
 entryPoint([ 'sh', '-c', 'java $JAVA_OPTS -Djava.security.egd=file:/dev/./urandom -jar /app.jar' ])
}

ENV is used at build time And at run time

One little gotcha in the previous section lead to an interesting learning. My initial attempt to set the JAVA_OPTS env variable looked like this:

setEnvironment('JAVA_OPTS', '')

but that produced an illegal line in the Dockerfile:

ENV JAVA_OPTS

That led me to read about the ENV directive in the Dockerfile reference docs. I was confused about whether ENV directives are used at build time, run time, or both. Turns out, the answer is both, as I was able to prove to myself with the following. The ENV THE_FILE is used at build time to decide which file to add to the image, and at run time as an environment variable, which can be overriden at the command line.

$ cat somefile
somefile contents

$ cat Dockerfile
FROM alpine:latest
ENV THE_FILE="somefile"
ADD $THE_FILE containerizedfile
ENTRYPOINT ["sh", "-c", "cat containerizedfile && echo '-----' && env | sort"]

$ docker build -t envtest .
Sending build context to Docker daemon  3.072kB
Step 1/4 : FROM alpine:latest
 ---> 7328f6f8b418
Step 2/4 : ENV THE_FILE "somefile"
 ---> Using cache
 ---> 148a4236ce19
Step 3/4 : ADD $THE_FILE containerizedfile
 ---> Using cache
 ---> d44f9e242685
Step 4/4 : ENTRYPOINT sh -c cat containerizedfile && echo '-----' && env | sort
 ---> Running in 70de8ceac5ef
 ---> 5d875712904a
Removing intermediate container 70de8ceac5ef
Successfully built 5d875712904a
Successfully tagged envtest:latest

$ docker run envtest
somefile contents
-----
HOME=/root
HOSTNAME=36b3233697df
PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin
PWD=/
SHLVL=1
THE_FILE=somefile
no_proxy=*.local, 169.254/16

$ docker run -e THE_FILE=blah envtest
somefile contents
-----
HOME=/root
HOSTNAME=6a0f8c183a18
PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin
PWD=/
SHLVL=1
THE_FILE=blah
no_proxy=*.local, 169.254/16

Version Tag and Latest Tag

So that all worked fine for creating and publishing a versioned docker image locally. But I really want that image tagged with the semantic version and also the latest tag. The transmode plugin currently does not support multiple tags for the produced docker image. I'm not the only one who wants this feature. I took a look at the source code, and it wouldn't be a minor change. At this point, I'm only publishing locally, so given the choice between version tag and latest tag, I'm going to go for latest for now. This is a simple matter of adding tagVersion = 'latest' to the buildDocker task.

I tagged the code repo at v0.3.2 at this point.

I'm going to move on to evaluating the bmuschko and Netflix Nebula Docker Gradle plugins next.

Sunday, July 23, 2017

Docker Java Example Part 1: Initializing a new Spring Boot Project

Docker Java Example Series

I've been wanting to learn more about Docker for a while. I'm almost done with this udemy course Docker for Java Developers. Its a good course, and the concepts are straightforward. To help me commit it to memory, I wanted to do my own project to apply what I'm learning.
https://github.com/ryanmckaytx/java-docker-example
In this part, I'm just going to initialize a new project. Part 2 covers Spring Web MVC testing.

Set up your dev machine

Every once in a while, like when you switch jobs and get a new laptop, you need to set up a machine for java development.

For this, I like to use sdkman. It helps install the tools you need to do java development. It also helps switch between multiple versions of those tools. I've installed java, groovy, gradle, and maven.

$ sdk c

Using:

gradle: 4.0
groovy: 2.4.11
java: 8u131-zulu
maven: 3.5.0

Create a new project

There are a few good ways to do this. The typical way I do this is to copy another project. Within an organization, or at least within a team, there is typically some amount of infrastructure and institutional knowledge built into existing projects that you want in a new project. But for this project, I wanted to practice starting completely from scratch. There are a couple of good options.

Gradle init

I like gradle as a build tool, and gradle has a built in project initializer. It supports a few project archtypes, including pom (converting a maven project to gradle), java library, and java application. It even explicitly supports spock testing framework.

$ gradle init --type java-application --test-framework spock

BUILD SUCCESSFUL in 0s
2 actionable tasks: 2 executed
$ tree
.
|____build.gradle
|____gradle
| |____wrapper
| | |____gradle-wrapper.jar
| | |____gradle-wrapper.properties
|____gradlew
|____gradlew.bat
|____settings.gradle
|____src
| |____main
| | |____java
| | | |____App.java
| |____test
| | |____groovy
| | | |____AppTest.groovy

You can see it also generates a demo App and AppTest.

Spring Boot Initializr

For Spring Boot apps, Spring provides the Spring Boot Initializr. This lets you choose from a curated (but extensive) set of options and dependencies, and then generates a project in a zip file for download. Similarly to gradle init, it includes a default basic app and test.

$ unzip demo.zip
Archive:  demo.zip
   creating: demo/
  inflating: demo/gradlew
   creating: demo/gradle/
   creating: demo/gradle/wrapper/
   creating: demo/src/
   creating: demo/src/main/
   creating: demo/src/main/java/
   creating: demo/src/main/java/net/
   creating: demo/src/main/java/net/ryanmckay/
   creating: demo/src/main/java/net/ryanmckay/demo/
   creating: demo/src/main/resources/
   creating: demo/src/main/resources/static/
   creating: demo/src/main/resources/templates/
   creating: demo/src/test/
   creating: demo/src/test/java/
   creating: demo/src/test/java/net/
   creating: demo/src/test/java/net/ryanmckay/
   creating: demo/src/test/java/net/ryanmckay/demo/
  inflating: demo/.gitignore
  inflating: demo/build.gradle
  inflating: demo/gradle/wrapper/gradle-wrapper.jar
  inflating: demo/gradle/wrapper/gradle-wrapper.properties
  inflating: demo/gradlew.bat
  inflating: demo/src/main/java/net/ryanmckay/demo/DemoApplication.java
  inflating: demo/src/main/resources/application.properties
  inflating: demo/src/test/java/net/ryanmckay/demo/DemoApplicationTests.java

Pretty much the only thing I don't like here is that the test isn't spock and there doesn't seem to be a way to choose it. Not a big deal, its easy to change afterward. I went with initializr for this project.

JHipster

JHipster is an opinionated full-stack project generator for Spring Boot + Angular apps. It has a lot of features that I want to explore later, so for now I stuck with Spring Boot Initializr.

Add a Rest Controller

I could have gone straight to CI at this point, but I wanted to do a little extra work that I wish the Initializr could have done for me. I copied in a basic hello world spring rest controller (and its tests) from the Spring Restful Web Service Guide. The repo is now at this point https://github.com/ryanmckaytx/java-docker-example/tree/v0.1