At Leaware we already developed more than 70 mobile apps using Xamarin technology. Most of them are multiplatform, means developed in most cases for IOS, Android and Windows platform.
In recent years we spent a lot of time against finding best way to test our applications in different stages of their life cycle.
Most typical stages during app life cycle is like on following picture.
In this stage we work a lot on developing app from scratch. Typically we work in scrum approach, so this means that we work in sprint cycles, where at end of every cycle we deliver to customer a shippable product.
But we know that later or faster we will have to perform a transition of our development effort into something what we can name a product version 1.0. During this stage we transit development into alpha version, then into beta, then into release candidate and finally into first production release.
Is means that from beta version we are more focused on polishing our product, we are not so focused on adding new functions, but more on fixing these which we have already, doing some change requests … still using scrum approach.
When we are in release candidate stage, then we in most cases don’t add anything new, event don’t implement change requests. We are focused on polishing, fixing and improving stability, scalability, user interface and so on.
Here is how it looks:
So many different screens, resolutions, proportions….
We also divided testing into two stages.
As you remember we internally divide development into two stages:
What’s a difference between manual testing and automated testing? It’s something totally different…
When we use automated testing we can test app for example on 300 DIFFERENT devices in 20 minutes. In manual… eghh 300 DIFFERENT devices? Impossible, no one has so many.
When we use automated testing we can choose 300 DIFFERENT devices from more than 1000 DIFFERENT devices which are in the cloud – in manual? Impossible!
We can perform a test scenario on for example 100, 200, 300, 400 different devices in parallel – such test is performed in average 10-20 minutes (with reports, information about bugs and so on).
Let’s calculate performing such test manually. Let’s imagine
We have 50 different configurations in company (we don’t even think about 100 or more –it’s almost impossible in typical software house)
We have one person who is doing the test.
So let’s assume:
Taking device – 1 minute
Connecting device to computer – 1 minute
Loading a build of app to device – 5 minutes
Opening test scenario excel file – 2 minutes
Performing a test – 10 minutes
Filling bug reports document – 10 minutes (bugs description, print screens)
If we have more bugs – another 5 minutes for filling
Assume = 1+1+5+2+10+10+5=34 minutes.
Its optimistic scenario and it doesn’t cover work of developer after getting a report regarding checking if bugs are repeatable or not, on which exactly devices bugs were found and so on.
But okay. Let’s say that such test takes 30 minutes so if we have to perform a test for 50 devices, our person will need 30*50=1500 minutes = 25 hours, so seems like 4 days of testing.
SO THIS IS WHY MANUAL TESTING IN LEAWARE IS NOT A SOLUTION. It’s only helpful in early stages of development where we are focused only on one, maybe two devices which are used by us for showing progress to customer.
When it’s going to RC stage or PRODUCTION we need a way of proving that application will work properly on thousands of different configurations. This is why we test our apps on cloud of real devices.
If You are interested in testing your apps on many different real devices, contact with us. We will show You how much more differences.
For example Do You imagine to get screenshots of every screen of app during testing? To find bugs like squished images, keyboard overlays and so on? In manual testing its almost impossible, in our approach – it works. See example:
There is much more differences, contact with us to get detailed information.
Application is already released
Great day for every team of developers… as well as for hole company and of course for our customers as well.
Let’s say that we released an app a month ago. No bugs, everything was working well. No ANY CHANGE IN CODE in last 30 days.
But what happens when Apple release new version of IOS or Google new version of Android? Are you prepared?
What happens when any producer will release new version of smartphone or tablet?
Are you prepared for that?
MobileTAS application. Released several months ago for some enterprise customers and then Samsung Galaxy 6 Edge appears on market.
Normally we wouldn’t do anything, but our mobile testing strategy push us to test every device on new smartphones when only they appears on market.
So we did a test:
And what happened… Application didn’t even start on Galaxy S6 Edge.
So thanks to our strategy we found problem with application before MobileTAS users. Without strategy for testing – it wouldn’t be possible.
Conclusion? We test our mobile apps all the time, not only during development, not only when we release something new… It’s not enough in current sophisticated world of mobile.
If you wish to get more information – how we can help with testing your apps, please contact us.