Functional Testing of Android Applications

by Tim Lavers
(Test. Crash? Code.)+
Thats my preferred workflow as a programmer. So when I started writing a simple Android
application my first task was to get this productive cycle happening. This was not completely
straightforward. In fact Im not finished yet. But Ive made enough progress that I think that its
worth writing down what Ive found so far, for two reasons. Firstly, Ive got a working system that
others can use and can learn from. Secondly, I hope that readers can point out problems with my
approach and suggest alternatives.
By a functional test I mean a program that runs the application as a user would and checks that
some particular requirement is implemented correctly. In general, these tests use more than a single
screen of the application.
In this article we will look at the ui proxy pattern allows for a fluid programming style in writing
functional tests. We will also look at the code that is needed in order to allow a sequence of
independent tests to be run as an ant task. Finally, we will discuss the Robotium toolkit, which is
great for simulating user actions in tests.

The IndoFlash App

The application that Ive been writing is a flashcard application for learning Indonesian. The user is
presented with a word and tries to remember the translation. If they have trouble with the word
they add it to their favourites list. There are buttons for showing words and their definitions, adding
words to the favourites list, shuffling the list, and for showing word pairs with either Indonesian or
English first:

There are three screens, or Activities: a WordListDisplay for showing the words, a
WordListSelecter for choosing a list of words to work on, and a ChapterSelecter for choosing
a collection of word lists. The underlying Android application is an instance of a class called
IndoFlash. This class manages the state of the system, such as which list is selected and what is in
the list of favourites.

Fluent Functional Tests

To test IndoFlash I wanted to write tests that operate the application as a user would: pressing
buttons, scrolling lists, and moving between different screens. I also wanted to test that navigational
information and user choices were preserved between application invocations. For example, if a
user chooses to view the lists as shuffled, then if the application is closed and re-started, the lists
should be presented as shuffled.
The Android platform supplies an API for reading the state of screens and for injecting user events
such as button presses and scroll gestures. This library is quite low level and one of my aims was to
convert this into a much more fluid style of programming in which method names very clearly state
what the method does. This has worked well and the tests are largely self-documenting. For
example, here is a test that checks that a word can be added to the list of favourites:
ui.checkThatCurrentWordIs("you");//Sanity check.
ui.addCurrentWordAsFavourite();
ui.showDefinitionOfCurrentWord();
ui.checkThatCurrentTranslationIs("anda");
ui.showNextWord();
ui.checkThatCurrentWordIs("what");
openFavouritesList();
ui.checkThatCurrentWordIs("you");//It was previously showing "what".
ui.checkThatCurrentTranslationIsEmpty();//It was previously showing "anda".
ui.showDefinitionOfCurrentWord();
ui.checkThatCurrentTranslationIs("anda");//Check that the definition is
carried along with a word.

UI Proxy Classes
In the code shown above, the object ui is a test proxy for the WordListDisplay, which is the
main application Activity. The class of ui is UIWordListDisplay. Every user operation of the
WordListDisplay is matched by a method in this class.
The call openFavouritesList(); midway through the test is an example of a multi-screen
operation. It is implemented as follows:
protected void openFavouritesList() {
UIWordListSelecter uiWordListSelecter = ui.showWordLists();
ui = uiWordListSelecter.selectFavourites();
}

We see here that there is also a test proxy class for WordListSelecter, which is the Activity that is
used to switch between different wordlists. There is also a ui proxy, UIChapterSelecter, for the
chapter selection activity.
It is the use of ui proxy classes that allows the fluent programming style in writing tests. This pattern
can be used for pretty much any user interface platform. In the book Swing Extreme Testing it is
used to test desktop applications.

The user actions that are performed in the function tests, such as clicking a button or scrolling, make
use of the TouchUtils class of the android.test package. For example, to select the favourites
list, we need to do some scrolling:
public UIWordListDisplay selectFavourites() {
TouchUtils.scrollToBottom(testCase(), activity(), listView());
return selectList(listView().getChildCount() - 1);
}

Android Functional Tests

Android tests are applications that exercise the main application within an emulated Android device.
The use of the emulator provides a lot of advantages. For example, we can see our app being
exercised by the tests and we can easily try out our app on a variety of different devices. If the tests
pass then we have very high degree of certainty that the app will work on a physical device. The
downside of testing in an emulator is that the tests are very slow to run. For example, it takes about
seven minutes for me to build the software, install it on the emulator and run a suite of 20 or so
tests.
Android function tests extend android.test.ActivityInstrumentationTestCase2. This
class provides access to an Instrumentation instance, which manages the interactions between
the test harness and the app that is being tested. These interactions are controlled by the use of
Instrumentation.ActivityMonitor and Intent objects. I found this part of the Android
testing API hard to use because if methods are called in the wrong order or in the wrong context,
they can simply hang. The documentation doesnt give much information about how things work,
and the sheer slowness of running tests on an emulator makes it hard to discover more about the
API through experimentation.
However, by trial and error and Google, I came across this magical incantation that will launch an
Activity:
//Add a Monitor for the Activity to the Instrumentation instance.
//Create an Intent to start the Activity.
//Call Instrumentation.startActivity with the Intent.
//Wait for the Activity to be launched, using the Monitor.

Within the tests for IndoFlash this is implemented as the launch() method of
UIWordListDisplay:
public void launch() {
Instrumentation.ActivityMonitor monitor =
instrumentation().addMonitor(WordListDisplay.class.getName(), null, false);
setMonitor(monitor);
Intent intent = new Intent(Intent.ACTION_MAIN);
intent.setFlags(Intent.FLAG_ACTIVITY_NEW_TASK);
intent.setClassName(instrumentation().getTargetContext(),
WordListDisplay.class.getName());
instrumentation().startActivitySync(intent);
Activity activity =
instrumentation().waitForMonitorWithTimeout(monitor(), 5);
setActivity(activity);
}

Each of the function tests used in the development of IndoFlash extends from FunctionTest,
which provides this basic test structure:
//Launch the main activity, which is WordListDisplay
//Run the specific test, which is the doIt() method of subclasses.
//Get the application back to a known state.

Specifically, this is implemented as follows:

public void test() {
//Launch the main activity, which is WordListDisplay.
ui = new UIWordListDisplay(this);
ui.launch();
//Run the actual test.
doIt();
//Get the application back to a known state.
IndoFlash application = (IndoFlash) ui.activity().getApplication();
ChapterSpec chapter1Spec =
application.applicationSpec().chapterSpecs().get(0);
application.setCurrentChapter(chapter1Spec);
application.setWordList(chapter1Spec.wordLists().get(0));
ui.finish();
application.clearFavourites();
if (application.shuffle()) {
application.toggleShuffle();
}
if (application.showIndonesianFirst()) {
application.toggleShowIndonesianFirst();
}
//Cleanup.
pause(5);
}

Most of the code is in getting the application, which is an instance of IndoFlash, back to its initial
state: showing the first word list in the first chapter, with the English translation first and the word
list unshuffled. The pause at the very end of the test is annoying, but without it the tests are flaky.
So far there are 16 function tests and the entire build-test cycle takes almost 7 minutes on a fairly
fast laptop. Of course, 80 seconds of that 7 minutes is caused by the 5 second pause at the end of
each test an example of how evil these pauses are.

Robotium
In a code snippet earlier we saw how TouchUtils can be used to perform simple actions such as
pressing buttons and scrolling through lists. To write tests that involve more complex actions it is
best to make use of the Robotium tool. Robotium has a rich API for simulating user actions and also
has very useful methods for investigating the state of the application.
Lets see an example of a Robotium test. One of the requirements of IndoFlash is that the action bar
always shows a help button and an information button. When these are pressed, message windows
should show that display the appropriate information. For example:

UIActivity has a solo() method that returns a Robotium Solo object, which is the workhorse

of the Robotium API. This can be used to check that the help button shows the correct information:
ui.solo().clickOnActionBarItem(R.id.action_help);
ui.solo().waitForDialogToOpen();
Assert.assertTrue(checkThatAlertDialogShows("How to use IndoFlash",
"Press the Show button to see"));
ui.solo().clickOnButton("OK");
ui.solo().waitForDialogToClose();

The trickiest part of this test is the method checkThatAlertDialogShows. This is why. The
IndoFlash code that creates this dialog uses the AlertDialog.Builder class:

Our difficulty is that in the resulting View object, we dont know the id of either of the important
TextView instances. However, with the help of Robotium, we can track them down:
boolean checkThatAlertDialogShows(@NotNull CharSequence title, @NotNull
CharSequence messageStart) {
boolean titleFound = false;
boolean messageFound = false;
ArrayList<View> currentViews = ui.solo().getCurrentViews();
for (View view : currentViews) {
if (view instanceof TextView) {
CharSequence text = ((TextView) view).getText();

if (title.equals(text)) {
titleFound = true;
} else if (isPrefixFor(messageStart, text)) {
messageFound = true;
}
}
}
return titleFound && messageFound;
}
private boolean isPrefixFor(CharSequence pref, CharSequence text) {
if (pref.length() > text.length()) return false;
CharSequence sub = text.subSequence(0, pref.length());
return sub.equals(pref);
}

In a more complex application, in which there were more tests involving ActionDialog, it might be
worth refactoring this kind of code into a UIActionDialog class.

Summary
This article has presented code for running suites multi-screen functional tests of an Android
application. The tests are written in a very fluid style, thanks to the use of the ui proxy design
pattern. By harnessing the power of Robotium, it is possible to write tests that are even more
expressive and powerful. The source code for IndoFlash and its tests are an excellent starting point
for anyone implementing multi-screen functional tests for their Android applications.

Resources
The code for IndoFlash and its tests can be downloaded from here.
The Robotium tool is available from Google code: https://code.google.com/p/robotium/
The book Android in Practice: http://www.manning.com/collins/ contains a good overview of
Android testing.
This excellent article contained some code snippets that helped me get started with multi-screen
tests: http://www.vogella.com/tutorials/AndroidTesting/article.html
I have tried both the Eclipse and IntelliJ IDEs for developing Android applications and have a strong
preference for IntelliJ.