Cross-Platform Testing: Myths and Mysteries

Cross-Platform testing (aka Platform testing) is often a confusing term and it means different things to different people. This post aims to bring together some common concepts in this area. Feel free to add your thoughts.

What is cross-platform testing really ?

Cross platform testing is one form of specialized testing where you would verify the suitability of your solution to work across various different platforms.  Platform can include pretty much anything including OS, browser or a device that will be necessary to run your solution.

It can be achieved in many ways. Two of the most used categories are –

1. Cross Browser : Application simulation across a number of browsers
2. Cross OS :   Application simulation in Operating Systems

Both of them can be performed across a range of devices including desktop, PC, mobile devices etc. An ideal solution is to have a combination of both, where you will have the ability to test against a multiple OS/Browser combinations, giving you an edge to test against exhaustive list of configurations.

Why is it so important?

You are putting so much effort and emotion to build your solution, can be a website or an app.  Its important that equally resonates with your end-users who will be using the system in long run. If they love it, there is nothing more satisfying.  In  an era of cloud computing and cloud based services, most of the solutions are public facing which means it is in easy reach of common users and they can access the solution ( website, Apps) is limitless, diverse ways . Why not test your system across some of these and make it work ? At least majority of your users will be happy!

Again, you can’t possibly test every single combinations that a user might use to access your applications but a handful of them will cover a chunk of your user base ( pareto principle of 80-20 applies !)

 How do you approach it?

Testing applications for cross platform capability can be immensely complicated if not approached in a proper way. Moreover, a lack of an agreed principles may lead to mis-communications and bad reputation of your website, product. So how exactly you should approach cross –platform testing?

1. Know your audience: Probably the most important principle of testing. If you have a good knowledge on who are going to use your applications and may be a bit of number crunching, past statistics, then you are in control ! This will drive the next set of actions on what device/platforms you will make your applications available and work seamlessly.

2. Decide the platform you will support: Once you know your user-base, you will have to take an informed decision on how do you want to implement your solution, targeting your maximum user base.

3. Know how much you can test: The fallacy with testing is ‘endless possibilities’; you can test against all possible combinations but your time (and money) will limit your aspirations. So why not invest the time at the right places ? Figure out which of the platforms you will test that covers off your maximum risk (and hence majority users !). A multi-dimensional matrix along with a heat map will be handy to determine what all things you will target first and things that can wait.The Yellow ones below are your ‘possible Test conditions’ and the Red ones are the ‘High risk test conditions’ that you should cover as part of your testing.

4. Think about automation: At the end of the day, testing your solution across multiple platforms/devices is nothing but an extremely boring piece of work, quite repetitive in nature.  You might or might not see any differences and by the time you are about to find something fishy, your morale will be low! So what .? Automation is there to help you out. There are quite a few automated Test tools in the market that can help you to access your applications and probably can save you an enormous amount of time.

A word of caution – many of these automated tools can get you to a certain point ( e.g. example can get you the screenshots across a series of platforms ) but there is still a certain level of manual intervention needed to ensure the usability and accessibility of your applications. Fortunately, that is not so boring!

6. Virtual world cost less than physical: Platform Testing could be extremely expensive if you think about arranging the time and money that you need to get your device with multiple browsers, OS or any other dependencies. Fortunately, virtualization is the saviour. You can create your own VMs with different platform combinations to test your application. Alternatively, there are quite a few companies in the market who will let to use their VMs to test your apps.

What are some of the Testing good practice?

1. First things first; defining a clear boundary for Platform Testing is always important as it takes out any unnecessary confusion of the game.
2. Create a Test matrix that you will test against. A matrix will always come handy when we define the boundary, design testing scenarios or communicate to the end users.
3. Identify elements in the matrix that are of high risk of failure; A heat map might be a good idea to define those high risk elements and prioritize your tests, especially in a time/resource constrained environment.
4. Use of Tools: There is no denying fact that use of the right tools makes the whole work much more efficient and exciting at the same. This will not only save in time and effort but also a great deal in arranging and procuring resources.

What should be my Tool Strategy?

Selecting the right-fit tool for Platform Testing is no different to any other Test tools selection process.  These are some of the things that might help you –

1. Have a quick scan of the market – to see what are your options
2. Look through the leading forums where people share their experience about certain tools and its usage
3. List down top 3-4 tools that makes the most impact
4. If possible, define a list of criteria against which the tools should be evaluated
5. Prepare a quick comparison chart to see the relative position of each tools. You might use some elementary scoring against the criteria
6. And finally, do a Proof-of-concept against top two tools to make sure you are comfortable with the decision.

And voila..rest is easy.  You can get started when you like.

What are some of my options?

These days you will have quite a few options to choose from (it is quite a different situation few years back when you were lucky to have one workable solution). Here are some of the options worth looking ( not necessarily in any order of preference)

1. Browserstack ( www.browserstack.com)
2. Testize (www.testize.com)
3. TestPlant cross-browser Testing (www.testplant.com)
4. Browsershots (https://browsershots.org)

 So what is holding you back ?  Go and enjoy your test !

 If you have any questions about testing your applications against multi platforms, reach out to us at kloud.com.au