The advantages of testing an application using Xamarin Test Cloud

Authors: Tomasz Soroka

08.03.2016



There’s no need to convince anyone, anymore about the necessity of  testing mobile applications, everyone who at least once had a problem with errors in the application knows that for sure. Surveys of users opinions suggest that a first impression is the most important. A significant portion of respondents didn’t give a second chance to the application that didn’t run at the first time. 60% of users abandon applications if it doesn’t load from the very beginning - after approx. 3 seconds, 43 %. of them won’t come back. This means that 1/4 of users that wanted to use an application, resigned and will never return. Fortunately, this problem can be relatively easily resolved by careful application tests and making it available for users.

There are various methods for testing the application. Manual testing, as the name suggests are the tests carried out on real devices, eg. by people within a company or a group of the developer’s friends. Crowd testing or group tests are carried out simultaneously on a large group of testers corresponding to typical users to whom the application is adressed. And finally, automatic or automated tests - carried out using special tools, application testing software for eg Xamarin Test Cloud. The biggest advantage is the wide availability of various devices and operating system configurations. Xamarin Test Cloud significantly simplifies and shortens the testing time.


The following table shows the benefits of testing an application using the Xamarin test Cloud


Selected functionalities

Xamarin test Cloud 

Crowd testing

Manual testing

Certainty of checking each scenario of users’ behaviour in exactly the same way [on different devices]

Yes - Tests are carried out automatically

No - tests are conducted by different people according to an individual model of behaviour

No - tests are conducted by different people according to an individual model of behaviour

Tests provided on more than 1,000 different configurations of mobile devices

and operating systems

Yes

Unlikely

[possible with a very large group of testers = large cost of tests]

No

Regression Error reporting [errors

occured after the amendments were made] and bottlenecks - places

in which the largest number of users has problems

Yes

Very inaccurate

Very inaccurate

Testing on new devices, that have only appeared on the market

Yes

Unlikely

 

Unlikely

 

Testing applications on the beta versions of operating systems

Yes

Unlikely

 

Unlikely

 

Integration with test progress controlling software

Yes

No

No

Tests on proper application loading up
[on 50 devices at the same time]

Yes

No

No

The ability to track application backend loading up

Yes

No

No

Graphics layer tests on more than 100 devices

Yes

No

No

Application loading up speed tests

Yes

Very inaccurate

Very inaccurate

Ability to specify max. loading up time

Yes

Very inaccurate

Very inaccurate

Estimated time of testing in 25 steps on

50 devices

10 minutes

10 hours

25 hours

Screenshots of each step

Yes

No

No

A detailed report is generated at each stage of the test

Yes

No

No