Tuesday, August 29, 2017

Docker Java Example Part 3: Transmode Gradle plugin

At last we get to some Docker in this Docker example.

Gradle Docker Plugin

There are a few prominent docker plugins for gradle: transmode, bmuschko, and Netflix nebula. First, I used transmode, as recommended in the spring boot guide Spring Boot with Docker. After adding the Dockerfile:

FROM openjdk:8-jdk-alpine
VOLUME /tmp
ADD target/java-docker-example-0.0.1-SNAPSHOT.jar app.jar
ENV JAVA_OPTS=""
ENTRYPOINT [ "sh", "-c", "java $JAVA_OPTS -Djava.security.egd=file:/dev/./urandom -jar /app.jar" ]

, and updating build.gradle as described in the guide, I was able to build a docker image for my application:

$ gw clean build buildDocker --info

Setting up staging directory.
Creating Dockerfile from file /Users/ryanmckay/projects/java-docker-example/java-docker-example/src/main/docker/Dockerfile.
Determining image tag: ryanmckay/java-docker-example:0.0.1-SNAPSHOT
Using the native docker binary.
Sending build context to Docker daemon  14.43MB
Step 1/5 : FROM openjdk:8-jdk-alpine
---> 478bf389b75b
Step 2/5 : VOLUME /tmp
---> Using cache
---> 136f2d4e58dc
Step 3/5 : ADD target/java-docker-example-0.0.1-SNAPSHOT.jar app.jar
---> b3b47b89bbf1
Removing intermediate container 92f637bc67e0
Step 4/5 : ENV JAVA_OPTS ""
---> Running in e90c9a3557eb
---> 1d3f6526e8e5
Removing intermediate container e90c9a3557eb
Step 5/5 : ENTRYPOINT sh -c java $JAVA_OPTS -Djava.security.egd=file:/dev/./urandom -jar /app.jar
---> Running in 2fbfb52f836d
---> f001bdddc80b
Removing intermediate container 2fbfb52f836d
Successfully built f001bdddc80b
Successfully tagged ryanmckay/java-docker-example:0.0.1-SNAPSHOT

$ docker images
REPOSITORY                       TAG               IMAGE ID        CREATED          SIZE
ryanmckay/java-docker-example    0.0.1-SNAPSHOT    f001bdddc80b    3 minutes ago    115MB

$ docker run -p 8080:8080 -t ryanmckay/java-docker-example:0.0.1-SNAPSHOT 

Automatically Tracking Application Version

Note that the Dockerfile at this point has the application version hard-coded in it. This duplication must not stand. The transmode gradle plugin also supports a dsl for specifying the Dockerfile in build.gradle. Then as part of the build process, it produces the actual Dockerfile.

I set about moving line by line of the Dockerfile into the dsl. With one exception it went smoothly. You can see the result in v0.3 of the app. The relevant portion of build.gradle is listed here. You can see its pretty much a line for line translation of the Dockerfile. And since we have access to the jar filename in the build script, nothing needs to be hard coded for docker.

// for docker
group = 'ryanmckay'

docker {
 baseImage 'openjdk:8-jdk-alpine'
}

task buildDocker(type: Docker, dependsOn: build) {
 applicationName = jar.baseName
 volume('/tmp')
 addFile(jar.archivePath, 'app.jar')
 setEnvironment('JAVA_OPTS', '""')
 entryPoint([ 'sh', '-c', 'java $JAVA_OPTS -Djava.security.egd=file:/dev/./urandom -jar /app.jar' ])
}

ENV is used at build time And at run time

One little gotcha in the previous section lead to an interesting learning. My initial attempt to set the JAVA_OPTS env variable looked like this:
setEnvironment('JAVA_OPTS', '')
but that produced an illegal line in the Dockerfile:
ENV JAVA_OPTS
That led me to read about the ENV directive in the Dockerfile reference docs.  I was confused about whether ENV directives are used at build time, run time, or both.  Turns out, the answer is both, as I was able to prove to myself with the following. The ENV THE_FILE is used at build time to decide which file to add to the image, and at run time as an environment variable, which can be overriden at the command line.

$ cat somefile
somefile contents

$ cat Dockerfile
FROM alpine:latest
ENV THE_FILE="somefile"
ADD $THE_FILE containerizedfile
ENTRYPOINT ["sh", "-c", "cat containerizedfile && echo '-----' && env | sort"]

$ docker build -t envtest .
Sending build context to Docker daemon  3.072kB
Step 1/4 : FROM alpine:latest
 ---> 7328f6f8b418
Step 2/4 : ENV THE_FILE "somefile"
 ---> Using cache
 ---> 148a4236ce19
Step 3/4 : ADD $THE_FILE containerizedfile
 ---> Using cache
 ---> d44f9e242685
Step 4/4 : ENTRYPOINT sh -c cat containerizedfile && echo '-----' && env | sort
 ---> Running in 70de8ceac5ef
 ---> 5d875712904a
Removing intermediate container 70de8ceac5ef
Successfully built 5d875712904a
Successfully tagged envtest:latest

$ docker run envtest
somefile contents
-----
HOME=/root
HOSTNAME=36b3233697df
PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin
PWD=/
SHLVL=1
THE_FILE=somefile
no_proxy=*.local, 169.254/16

$ docker run -e THE_FILE=blah envtest
somefile contents
-----
HOME=/root
HOSTNAME=6a0f8c183a18
PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin
PWD=/
SHLVL=1
THE_FILE=blah
no_proxy=*.local, 169.254/16

Version Tag and Latest Tag

So that all worked fine for creating and publishing a versioned docker image locally.  But I really want that image tagged with the semantic version and also the latest tag.  The transmode plugin currently does not support multiple tags for the produced docker image.  I'm not the only one who wants this feature.  I took a look at the source code, and it wouldn't be a minor change.  At this point, I'm only publishing locally, so given the choice between version tag and latest tag, I'm going to go for latest for now.  This is a simple matter of adding tagVersion = 'latest' to the buildDocker task.

I tagged the code repo at v0.3.2 at this point.

I'm going to move on to evaluating the bmuschko and Netflix Nebula Docker Gradle plugins next.

Wednesday, August 23, 2017

Docker Java Example Part 2: Spring Web MVC Testing

The next step was to add some tests. The tests that came with the demo controller used a Spring feature I was not familiar with, MockMvc. The Spring Guide "Testing the Web Layer" provides a good discussion of various levels of testing, focusing on how much of the Spring context to load. There are 3 main levels: 1) start the full Tomcat server with full Spring context, 2) full Spring context without server, and 3) narrower MVC-focused context without server. I wanted to compare all three, plus add in variation in testing framework and assertion framework. Specifically I wanted to add Spock with groovy power assert.  The aspects I wanted to compare were: test speed, readability of test code, readability of test output.  I intentionally made one of the tests fail in each approach to compare output.


Spock with Full Tomcat Server

This is the approach I am most familiar with.
https://github.com/ryanmckaytx/java-docker-example/blob/v0.2/src/test/groovy/net/ryanmckay/demo/GreetingControllerSpec.groovy

Timing

I ran and timed the test in isolation with
$ ./gradlew test --tests '*GreetingControllerSpec' --profile

Total 'test' task time (reported by gradle profile output): 13.734s
Total test run time (reported by junit test output): 12.690s
Time to start GreetingControllerSpec (load full context and start tomcat): 12.157s
So, not fast. Maybe one of the other approaches can do better.

Test Code Readability

def "no Param greeting should return default message"() {

    when:
    ResponseEntity<Greeting> responseGreeting = restTemplate
                .getForEntity("http://localhost:" + port + "/greeting", Greeting.class)

    then:
    responseGreeting.statusCode == HttpStatus.OK
    responseGreeting.body.content == "blah"
}
I really like Spock. I like the plain English test names. I like the separate sections for given, when, then, etc. I think it reads well and makes it obvious what is under test.

Test Output Readability

When a test fails, you want to see why, right?  In this aspect, groovy power assertions are simply unparalleled.
Condition not satisfied:

responseGreeting.body.content == "blah"
|                |    |       |
|                |    |       false
|                |    |       12 differences (7% similarity)
|                |    |       (He)l(lo, World!)
|                |    |       (b-)l(ah--------)
|                |    Hello, World!
|                Greeting(id=1, content=Hello, World!)
<200 OK,Greeting(id=1, content=Hello, World!),{Content-Type=[application/json;charset=UTF-8], Transfer-Encoding=[chunked], Date=[Tue, 22 Aug 2017 22:06:33 GMT]}>

 at net.ryanmckay.demo.GreetingControllerSpec.no Param greeting should return default message(GreetingControllerSpec.groovy:27)
Note that the nice output for responseGreeting itself comes from ResponseEntity.toString(), and from Greeting.toString(), which is provided by Lombok.

Spock with MockMvc

By adding @AutoConfigureMockMvc to your test class, you can inject a MockMvc instance, which facilitates making calls directly to Springs HTTP request handling layer.  This allows you to skip starting up a Tomcat server, so should save some time and/or memory.  On the other hand, you are testing less of the round trip, so the time savings would need to be significant to justify this approach.
https://github.com/ryanmckaytx/java-docker-example/blob/v0.2/src/test/groovy/net/ryanmckay/demo/GreetingControllerMockMvcSpec.groovy

Timing

This approach was about 500ms faster than with tomcat.  Not significant enough to justify for me, considering the overall time scale.

Total 'test' task time (reported by gradle profile output): 13.263s
Total test run time (reported by junit test output): 12.281s
Time to start GreetingControllerSpec (load full context, no tomcat): 11.804s

Test Code Readability

def "no Param greeting should return default message"() {

    when:
    def resultActions = mockMvc.perform(get("/greeting")).andDo(print())

    then:
    resultActions
            .andExpect(status().isOk())
            .andExpect(jsonPath('$.content').value("blah"))
}
This reads reasonably well.  Capturing the resultActions in the when block to use later in the then block is a little awkward, but not too bad.  Being able to express arbitrary JSON path expectations is convenient.  I didn't see an obvious way to get a ResponseEntity as was done in the full Tomcat example.

Test Output Readability

Condition failed with Exception:

resultActions .andExpect(status().isOk()) .andExpect(jsonPath('$.content').value("blah"))
|              |         |        |        |         |                     |
|              |         |        |        |         |                     org.springframework.test.web.servlet.result.JsonPathResultMatchers$2@1f977413
|              |         |        |        |         org.springframework.test.web.servlet.result.JsonPathResultMatchers@6cd50e89
|              |         |        |        java.lang.AssertionError: JSON path "$.content" expected:<blah> but was:<Hello, World!&rt;
|              |         |        org.springframework.test.web.servlet.result.StatusResultMatchers$10@660dd332
|              |         org.springframework.test.web.servlet.result.StatusResultMatchers@251379e8
|              org.springframework.test.web.servlet.MockMvc$1@68837646
org.springframework.test.web.servlet.MockMvc$1@68837646

 at net.ryanmckay.demo.GreetingControllerMockMvcSpec.no Param greeting should return default message(GreetingControllerMockMvcSpec.groovy:29)
Caused by: java.lang.AssertionError: JSON path "$.content" expected:<blah> but was:<Hello, World!&rt;

This test output does not read well at all. Spock and the Spring MockMvc library are both tripping over each other trying to provide verbose output.  I think you need choose either Spock or MockMvc, but not both.

JUnit with WebMvcTest and MockMvc

This configuration is on the far other end of the spectrum from full service Spock.  With @WebMvcTest, not only does it not start a Tomcat server, it doesn't even load a full context.  In the current state of the project this doesn't make much of a difference because the GreetingController has no injected dependencies.  If it did, I would have to mock those out.  Again, because of the differences from "real" configuration, time savings would need to be significant.
https://github.com/ryanmckaytx/java-docker-example/blob/v0.2/src/test/groovy/net/ryanmckay/demo/GreetingControllerTests.java

Timing

This approach was also about 500ms faster overall than full context with Tomcat.

Total 'test' task time (reported by gradle profile output): 13.275s
Total test run time (reported by junit test output): 0.269s
Time to start GreetingControllerSpec (load narrow context, no tomcat): 11.88s

Test Code Readability

@Testpublic void noParamGreetingShouldReturnDefaultMessage() throws Exception {

    this.mockMvc.perform(get("/greeting")).andDo(print()).andExpect(status().isOk())
            .andExpect(jsonPath("$.content").value("blah"));
}

This is the least readable for me.  Again, I like separating the call under test from the assertions.

Test Output Readability

The failure message for MockMvc-based assertion failures isn't as informative as Spock in this case.
java.lang.AssertionError: JSON path "$.content" expected:<blah> but was:<Hello, World!>" type="java.lang.AssertionError">java.lang.AssertionError: JSON path "$.content" expected:<blah> but was:<Hello, World!>

Because the test called .andDo(print()), some additional information is available in the standard out of the test, including the full response status code and body.

Conclusion

I'm as convinced as ever that Spock is the premier Java testing framework.  I'm reserving judgment on the Spring annotations that let you avoid starting a Tomcat server or load the full context.  If the project gets more complicated, those could potentially provide a nice speedup.

I tagged the code repo at v0.2 at this point.