This series of articles focuses on testing on the Android platform. In the first article, we will talk about the concept of testing and its benefits. Moreover, I will explain how tests can be categorized. Understanding this subject is necessary for real-world projects.
Let us start with a bit of theory.
What is testing, and why should we care about it?
Testing is simply a process where we verify that our software is doing what it is supposed to do. In other words, we want to check if business requirements are fulfilled and if there are no defects or bugs of any kind.
There are plenty of benefits when it comes to testing. Some of them are:
- reduced development costs
- improved performance
- bug prevention
Manual vs Automated
Manual testing happens when a tester is manually using the software and is going through test cases and scenarios to ensure correct behavior. While it is a great way to spot bugs, it might not be enough for larger-scale projects.
Luckily, with the use of additional tools, we can help ourselves with automated tests. Generally speaking, these are the tests that are set up once and can be executed later. It is great for repetitive and tedious tasks that must be done periodically.
Let us list a few differences between manual and automated testing:
Human presence – when a tester is manually going through the application, he can spot bugs outside of the test scope. Automated tests will only verify the behavior defined in the test. Humans are better observers, and with the use of common sense, they can provide more insights.
Development cost – while automated tests require some development time to set them up, we will benefit from them in the long run. In the case of manual tests, every execution requires some effort from the tester.
Usually, manual and automated testing can be combined. Manual testing is a great approach and it is always useful, but the bigger our app is, the more benefits we will see from having a set of automated tests.
Local vs Instrumented
Let us look more closely at automated tests from the Android Developer’s point of view. They can be split into:
Local tests – are executed on the JVM (Java Virtual Machine). We do not need a device or emulator to run them. They are faster than instrumented tests, and generally, they do not depend on the Android framework. There is one exception though – Robolectric library.
I will help myself with a quote from the Robolectric documentation:
Unlike traditional emulator-based Android tests, Robolectric tests run inside a sandbox which allows the Android environment to be precisely configured to the desired conditions for each test, isolates each test from its neighbors and extends the Android framework with test APIs which provide minute control over the Android framework’s behavior and visibility of state for assertions.
http://robolectric.org/
Local tests have their place in the Android project structure: /src/test/
Instrumented tests are dependent on the Android framework. That is why they need a device or emulator to run on. While slower than local tests, they can be executed simultaneously on different devices. It is also possible to use a device farm, which is an outside service that runs tests across various devices and returns test results.
Instrumented tests are located in src/androidTest.
Scope
Tests can also be divided by scope, or in other words – the size of the code that is being tested.
Unit tests are responsible for covering the smallest pieces of code. For example, if we have a method that can return 3 different results, we can write 3 unit tests that will verify if the method is working correctly.
UI tests (also known as End-to-end tests (E2E)) are used to verify bigger parts of the application. They always include a UI interaction. UI tests resemble a real-world usage of the software. Whole screens and user flows can be tested with them.
There are also integration tests, which can be considered middle-sized. There is no UI interaction needed. Here, we test if our code is “integrating” correctly with outside dependencies, like SharedPreferences or a database.
Test Pyramid
The test pyramid is a concept that describes the ratio between the types of testing. Simply speaking, we should have more low-level tests than high-level.
Unit tests can be written and executed quickly. They are the base of the pyramid and can pinpoint the exact location of the defect (because of their smallest scope). They can be called the first line of defense.
Integration tests require more resources to run, but compared to unit tests, there are not that many cases that need to be covered.
UI tests are the slowest and most expensive in terms of the needed resources and maintenance. However, they are designed to cover bigger parts of the app – this means we do not need a lot of them.
Also, failed end-to-end test usually means that you are missing a unit test that should cover this specific case.
This is it for today. In the next article, we will learn more about unit tests and examine practical examples. See you there!
