Continous integration on Android with travis CI

  6 mins read

Testing is trending topic. At least, in my twitter timeline and the guys that I’m used to talk about Android with. This is a good thing because testing is mandatory in a serious project, but assuming that all of us have tests added we can benefit from this by using it often.

If you are developing using TDD you should run your tests to be sure that you don’t break something in every TDD phase. (remember the TDD phases, usually represented as red-green-blue). But we are not perfect and we can commit changes that break our system and we should have something that can alert us if we are doing something wrong asap.

Here is where Continous Integration comes into play. Every time that you push changes into a branch in your system you can check if everything is ok before you merge this branch into develop (assuming you are using git-flow). What to check? Well, this is up to you and your team members, but for me the common things that I like to check are:

  • Android Lint, and force that we don’t have any error. It will be nice if we can also recover the reports to see the warnings.
  • Checkstyle, to ensure that the new code is matching the style that out team has decided.
  • Jacoco, to get a percentage of code coverage, I really don’t care so much about the percentage but and multiple occasions it helped me to detect code that I forgot to test.
  • Unit tests, those located in the test/ folder in our project
  • Instrumentation tests, those located in the androidTest/ folder in our project

Id addition you can run PMD and FindBugs.

Configuring Travis

I am going to use Travis CI for this article, I don’t know if it is better than other options like CircleCI, Codeship… But is one of the most popular options and it is free for open source projects. Of course, you can make it work in other CIs.

To accomplish that requirements I already configured the project with this travis.yml file. How does this work? I’m going to avoid the obvious lines, and explain line by line the most difficult parts, and the problems that I found while I was configuring it.

env:
  matrix:
    - ANDROID_TARGET=android-18 ANDROID_ABI=armeabi-v7a

This lines re-executes the script for each variable so, if I write another line with: “- ANDROID_TARGET=android-16 ANDROID_ABI=armeabi-v7a” It will execute the whole script with the 18 and 16 Apis. Really useful to test different API levels.

In the android components part, you should specify “tools” and “platform-tools” to use the last version available. If you are going to use more than one API level, remember to define all the emulators needed with “sys-img-armeabi-v7a-android-23”, “sys-img-armeabi-v7a-android-21”, etc…

Travis can cache directories that you need to speedup the following builds, to do that you have to pay or you can use the new container infrastructure, by specifing “sudo: false”. To enable cache, you have to specify which directories you want to cache, in this case I’m caching some .gradle folders with:

sudo: false
cache:
  directories:
    - $HOME/.gradle/caches/2.8
    - $HOME/.gradle/caches/jars-1
    - $HOME/.gradle/daemon
    - $HOME/.gradle/native
    - $HOME/.gradle/wrapper

Remember to change 2.8 with your gradle version if you are using a different one and remember also that the version is specified in the file gradle-wrapper.properties in your root gradle folder.

To execute android instrumentation tests we need an emulator, so to create it I included in the before_script section:

- echo no | android create avd --force --name test --target $ANDROID_TARGET --abi $ANDROID_ABI
- emulator -avd test -no-skin -no-audio -no-window &

Notice the $ANDROID_TARGET and $ANDROID_ABI variables that come from the env/matrix section that I explained before. Again, I got these lines to execute the emulator from another project. This line creates the emulator and launches the created emulator without a screen.

If you don’t specify any script section travis, by default, is going to execute ./gradlew connectedCheck. But I want to have some more checks, so I overrided a bit the behaviour of this section with this custom script:

script:
  - ./gradlew check -PdisablePreDex --continue --stacktrace
  - android-wait-for-emulator
  - adb devices
  - adb shell input keyevent 82 &
  - ./gradlew connectedAndroidTest -PdisablePreDex --continue --stacktrace

The first line executes the check task that executes unit tests, Android lint and the checkstyle if you have applied the plugin in your project with a valid configuration. To apply checkstyle is as easy as add this to your gradle file:

apply plugin: 'checkstyle'
  checkstyle {
    configFile rootProject.file('checkstyle.xml')
    showViolations true
  }

And defining a valid checkstyle.xml file or grab one from the checkstyle website.
With the –continue flag we tell gradle to continue if there is any error in the unit test, so we can get the full error report.

We launched an emulator, but we need to wait until the emulator is ready, meanwhile we launched the check task, but it can happen that the emulator is not ready yet, so we wait with android-wait-emulator. The next two lines are printing the available devices and sending the key “menu” to the emulator to unlock it.

Now that we have a running emulator, we can run the task connectedAndroidTest to run all the tests that need the Android device connected.

If the build fails, I’m using the Imgur script to upload an image of the current status of the emulator, it can be useful to debug and I also print the testing reports. To do this I have the following after_failure section:

after_failure:
  - wget http://imgur.com/tools/imgurbash.sh
  - chmod a+x imgurbash.sh
  - adb shell screencap -p | sed 's/\r$//' > screen.png
  - ./imgurbash.sh screen.png
  - pandoc builder/build/reports/tests/index.html -t plain | sed -n '/^Failed tests/,/default-package/p'

In the after_script section you can find:

after_success:
  - ./gradlew jacocoFullReport
  - pip install --user codecov
  - codecov

Now that we launched the tests and connected tests, we can generate the code coverage report with all the results, but I have another problem. I use a multi-module project with some projects using Java and some others with the Android gradle plugin, at this moment I’ve seen some solutions on the internet, but neither of all works for me, so I had to merge tons of solutions to make it work. The result is the jacoco.gradle file that I created to make all of this work, in some time I will fix some hard coded values that I have and publish this as a gradle plugin, for now, is enough. So the “jacocoFullReport” task of gradle is a custom one.

The last line is an utility called codecov that after a jacoco report is generated, it uploads all the reports to their server and you can see the reports online and you can have also a badge like this: Coverage via Codecov continous integration pretty cool! This is an example of the output.

Bonus

We specified at the beginning of the script the API 18, what happens if we need to test with the API 23 which as this moment is the last one? We are going to find problems 🙂
The API 23 includes a bug that prevents the tasks connectedAndroidTest and connectedCheck to work, well, in reality any task that needs a connected device because it has a timeout hardcoded, you can find more details here and the Stackoverflow thread that an user provides a valid solution. In order to solve this with the API 23 I overrided the default behaviour of the script section with:

script:
  - ./gradlew assembleDebug -PdisablePreDex
  - ./gradlew assembleAndroidTest -PdisablePreDex
  - android-wait-for-emulator
  - adb devices
  - adb shell input keyevent 82 &
  - adb install -r app/build/outputs/apk/app-mock-debug.apk
  - adb install -r app/build/outputs/apk/app-mock-debug-androidTest-unaligned.apk
  - ./gradlew check -PdisablePreDex --continue --stacktrace
  - .buildscript/launchInstrumentationTests.sh

The 2 first lines compiles the project and the androidTest variant. This is going to create 2 different apks. In the next line we are waiting for the emulator that we launched before, until the emulator is not ready, we can’t launch the tests.
After this lines we are going to install the 2 generated apks, notice the “-mock-” section in the path of my apk, that corresponds to my flavor, you have to replace it with yours and in the next line, again we launch the unit tests.
To launch the instrumentation tests, is a bit trickier, so I created an script with this lines:

#!/bin/sh
instrumentationResult=$(adb shell 'am instrument -w me.panavtec.cleancontacts.mock.test/me.panavtec.cleancontacts.CleanContactsRunner ; printf "$?"')
printf "$instrumentationResult\n"
exitCode=$(printf "$instrumentationResult" | tail -1)
if [ $exitCode != "0" ]; then
    exit 1
fi

The complex part was that the adb shell command does not redirect the exit code if it fails, so if you execute this command and the tests fails, they return 0 as the exit code of the application and travis thougt that the build was ok. so in order to workarround it I had to concat an echo $? that gives me the real exit code of the adb shell command to later on get the last line of the complete output with “tail -1” to be able to compare it and execute “exit 1” to make the script fail if the tests fails. All of this again with some help.

And that is all for now, as you can see work with Android is always a pleasure 🙂 I hope you enjoyed the article, and if you have some improvements you can comment here in the comments sections or open an issue/pull request in Clean-Contacts.

Written by:

Christian Panadero Martinez