Automatic mobile application testing with Xamarin

Introduction

The diversity of mobile devices means that interfaces need to be optimised for many resolutions and screen sizes. Traditional testing on individual devices and emulators is costly and scales poorly: time increases with the number of devices and application versions. Xamarin offers tools that automate this process, including Test Cloud for running tests on hundreds of real devices. The aim of this article is to review selected Xamarin features that make automatic mobile application testing easier.

Why mobile testing is critical

To cover around 80% of the market, an application should be tested on a very large number of device models. Coverage grows exponentially with the number of tested configurations, so a broad test matrix genuinely increases reach.

Users are unforgiving when it comes to performance: if an application does not load in around 3 seconds, most will close it, and a significant proportion will never return. On the other hand, applications that work well are gladly recommended to others. With the vast availability in app stores, quality becomes a key success factor.

Fragmentation of the mobile ecosystem

One of the main challenges is fragmentation: different systems, versions, device classes, and hardware parameters. This multiplies the combinations and the risk of errors, making testing a real challenge for teams.

- different operating system versions (iOS, Android, Windows Phone, WinRT, Windows 10) - different device classes (smartphones, tablets) - smartphones: resolutions, screen sizes, processor types, amount of memory - tablets: resolutions, screen sizes, processor types, amount of memory - different releases of the same system, e.g. Android 2.3–6.x, iOS 7–9.x, Windows Phone 8.0–10

Methods of testing mobile applications

The testing arsenal includes manual testing, crowd testing, unit, integration, and system testing, as well as automatic UI testing. The latter simulates real user actions: tapping buttons, selecting from lists, scrolling views, and so on.

Manual testing offers the highest realism, but it is time-consuming. Unit tests are fast, but they do not verify the interface layer. Automatic UI tests combine realism with speed, which is why they are the focus of this article. Importantly, Xamarin tools work not only with applications written in Xamarin.

Tools from Xamarin

Xamarin.UITest is a framework for writing tests that allows you to emulate user behaviour and run tests on an emulator or a connected device. Tests can be written in C# or Ruby.

Xamarin Test Cloud (XTC) makes it possible to run these tests in the cloud on more than 2000 real devices. The service automatically installs the application on the selected device matrix, runs the tests, and returns reports with screenshots, logs, and assertion results.

Typical workflow

- design and refine tests locally on an emulator and a physical device - once the tests run stably, publish them to XTC - select a device set and run tests in parallel - analyse reports and artefacts (screenshots, logs), quickly identifying regressions

Such a process shortens verification time across many hardware and system configurations without the need to own hundreds of devices in-house.

Designing UI tests in practice

Tests are created in C# or Ruby by describing user scenarios. A sample Xamarin.Forms application has a welcome screen with a Start app button that takes the user to a second screen with the text Hello!.

Example test scenario

- launch the application and set portrait orientation - take a screenshot of the start screen - verify that the Start app button is visible - rotate the device to landscape orientation and verify the button visibility again - tap the Start app button - verify that the text Hello! is displayed on the second screen - return to the start screen and take a screenshot

In Xamarin.UITest, the scenario consists of simple commands reflecting interactions, such as WaitForElement, Tap, SetOrientationPortrait, SetOrientationLandscape, Back, or Screenshot. First, you run the test locally, and then, without any changes, execute it in XTC on the selected device pool.

Summary

Automating UI tests with Xamarin.UITest and Xamarin Test Cloud enables faster bug detection, broader device coverage, and a better user experience. It is a practical way to reduce hardware costs and accelerate the delivery of high-quality mobile applications.