Maestro end-to-end testing for Android
The end-to-end tests for the Android app are built using Maestro and
generate reports. The scripts have been written and tested on Linux. They may work on other systems,
but that has not been thoroughly tested.
Prerequisites:
- Install Maestro as per the official instructions.
- Build the android release apk from the app-android module and the
HTTP server from the app-ktor-server module. This would be done by the normal
gradlew build command.
- Start an Android emulator or connect a physical device and get the serial of the device (e.g. using the adb devices command)
- MacOS only: install the realpath command.
Run all tests:
$ ./run-maestro-test.sh --serial1 emulator-5554
Where emulator-5554 is the serial of the emulator as per the adb devices command.
Options:
- –test specify a test to run as per the test flows found in e2e-tests e.g. –test 001_001_admin_can_add_content_001-h5p
- –apk specify a particular apk file to test and install. By default the test runner expects to
use the release APK file built from source
- –console-output use Maestro console output instead of saving to a JUnit XML. Helpful to see
more verbose output when designing/running/debugging flows etc.
Output
Result artifacts (screenshots, screen recording, etc) will be saved as follows:
- build/results/emulator-serial/report.xml - JUnit XML report for the given serial generated by Maestro
- build/results/emulator-serial/test-name - artifacts for each end-to-end test including screenshots
- build/results/emulator-serial/test-name/screenrecord.mp4 - screen recording of the end to end test
The combined report (with videos for each test) HTML will be saved to build/results/index.html
Run a single test using Maestro command line:
-
Build and install APK on emulator or device as per app-android README
-
Clear Data Folder in app-ktor-server:Start the server using runserver.sh or runserver.bat as per the Main README development environment setup
-
Run Maestro command to run test
cd test-end-to-end/android-maestro
maestro test e2e-tests/<test name>.yaml -e USERNAME=admin -e PASSWORD=testpass -e TESTSERIAL=emulator-5554 -e ENDPOINT=http://<ip address>:8087/
Replace the values forPASSWORD with the server admin password, TESTSERIAL with the emulator or device serial (as per adb devices command), ENDPOINT with the server url as per the –siteUrl parameter used with runserver.