One of my first assignments when I joined my project was to add unit tests to our Android application. I had read about the testing framework that was part of the platform, however I wanted to make sure that it was the right option. I started to look into several alternatives and found the following to be the most comprehensive:

  • Android testing framework
  • Roboelectric
  • Robotium

In the next sections you will find an overview of each one of them with a test example. All the tests correspond to a simple activity with a single button that triggers a transition to another activity.

Android testing framework

This is the framework included as part of the platform. It provides mock objects for the following:

  • Context
  • ContentProvider
  • ContentResolver
  • Service

It also includes control methods (instrumentation) that allows for calls to Android lifecycle methods (onCreate, onPause, etc). Overall it allows you to test the setup of the UI elements and the main lifecycle events of the activities. The simulation of input interactions is performed by the sendKeys method in the instrumentation test class or via the UI/Application Exerciser Monkey. It only supports JUnit3 style of unit tests and requires that the application is running on a device or emulator. Here is how a test would look for the simple activity that I described before:

public class AssignmentOneActivityTest extends
		ActivityInstrumentationTestCase2AssignmentOneActivity> {
 
	private AssignmentOneActivity mActivity;
	private Button mButton;
	private String resourceString;
 
	public AssignmentOneActivityTest(){
		super("com.captech.assignmentone", AssignmentOneActivity.class);
	}
 
	public AssignmentOneActivityTest(ClassAssignmentOneActivity> activityClass) {
		super(activityClass);
	}
 
	/* (non-Javadoc)
	 * @see android.test.ActivityInstrumentationTestCase2#setUp()
	 */
	@Override
	protected void setUp() throws Exception {
		super.setUp();
		mActivity = this.getActivity();
		mButton = (Button) mActivity.findViewById(com.captech.assignmentone.R.id.audit);
		resourceString = mActivity.getString(com.captech.assignmentone.R.string.buttonText);
	}
 
	public void testPreconditions(){
		assertNotNull(mButton);
	}
 
	public void testText(){
		assertEquals(resourceString, (String) mButton.getText());
	}
}

In this example we get the current activity and verify that the button and text have been added correctly. More information regarding this framework can be found here.

Roboelectric

The approach of this framework is to create "shadow" objects to simulate the Android SDK behavior. Due to this, the Roboelectric tests can run outside an emulator/device which makes it faster and more convenient than the other frameworks. It supports JUnit4 style of unit tests. Our example test would look like this using Roboelectric:

@RunWith(RobolectricTestRunner.class)
public class AssignmentOneActivityRoboelectricTest {
	private Button auditButton;
	private AssignmentOneActivity activity;
 
 
	@Before
	public void setup() throws Exception{
		activity = new AssignmentOneActivity();
		activity.onCreate(null);
		auditButton = (Button) activity.findViewById(R.id.audit);
 
	}
 
	@Test
	public void shouldHaveButtonThatSaysAudit() throws Exception{
		assertThat((String)auditButton.getText(), equalTo("Audit"));
	}
 
	@Test
	public void pressingButtonForNextActivity() throws Exception {
		auditButton.performClick();
		ShadowActivity shadowActivity = shadowOf(activity);
		Intent startedIntent = shadowActivity.getNextStartedActivity();
		ShadowIntent shadowIntent = shadowOf(startedIntent);
		assertThat(shadowIntent.getComponent().getClassName(), equalTo(AuditLog.class.getName()));
 
	}
 
}

In this case we simulate the clicking of the button and verify that the new intent created is the correct one. When I started looking deeper into Roboelectric I noticed several instances where some functionality wasn't covered and people had to wait for the next build or release. The claim on their website is that "Coverage of the SDK is improving every day". You can find additional information on Roboelectric here.

Robotium

This framework is designed to provide black box tests for Android applications. This means that you test for expected outcomes instead of specific methods, which allows for testing of applications for which you don't have the source code. It also means that these tests are brittle by nature, since simple changes in the data, order of elements, or processing time can break an existing test. It does require the application to be running in an emulator or device. Here is the Robotium test for our simple activity:

public class AssignmentOneActivityRobotiumTest extends
		ActivityInstrumentationTestCase2AssignmentOneActivity> {
 
	private Solo solo;
 
	public AssignmentOneActivityRobotiumTest() {
		super("com.captech.assignmentone", AssignmentOneActivity.class);
	}
 
	public void setUp() throws Exception {
		solo = new Solo(getInstrumentation(), getActivity());
	}
 
	@Smoke
	public void testAudit() throws Exception {
		solo.clickOnButton("Audit");
		solo.assertCurrentActivity("Expected AuditLog activity", "AuditLog");
 
		// Found at least one entry
		assertTrue("No entries found", solo.searchText("May 22", true));
 
		solo.goBack();
	}
 
}

This test clicks on the created button, verifies that the correct activity is launched and that some data exists in that activity. I found Robotium really simple to use but I had to work around the latency between activities, which was different depending on which machine I tested from. More information about Robotium can be found here.

And the winner is ...

Each one of these frameworks provide a different approach and will fit better depending on the application architecture. In my case a combination of 2 frameworks, the Android testing framework and Robotium, turned out to be the best approach. Happy testing!