Tools of the Mobile QA Trade

In our last blog post, we provided an overview of the mobile QA process. This week’s article dives a little deeper into the QA engineer’s toolbox to take a look at what software supports their work when testing a mobile app or website. It might be an emulator or even mirroring software to help them record a mobile app in action on an iOS or Android device.

Let’s take a closer look at some stand alone emulators and other applications used by QA teams involved in mobile testing.

Mirroring Software helps when Testing Mobile Devices

Even considering the overall convenience of mobile device emulators, most QA professionals prefer to test with an actual piece of hardware. It is truly the best way to properly benchmark an app or website’s performance on a mobile footprint. Desktop computers are too powerful to properly simulate the speed in which an app runs on a smartphone or a tablet.

QA teams use mirroring software, like Reflector on the iOS platform and Mobizen for Android, for two major reasons. Mirroring allows the tester to view the app in action on a larger screen giving them a better view of the user interface. They are also able to make a recording of the interaction with the app or website being tested.

Being able to record an app’s user interface in action definitely helps when logging bugs and other issues with the software’s performance. Having a visual recording of an app also allows software engineers to more easily replicate any issues discovered by the QA team.

Stand Alone Mobile Emulators

While both major mobile app development environments — Apple’s Xcode and Google’s Android SDK — contain built-in emulators, some QA teams prefer to use stand alone emulation software to perform their mobile testing. In the Android world, some device manufacturers, most notably Samsung, offer their own emulation software, providing benefits when testing user interfaces for a specific device. The company even has a web-based emulator for the Galaxy Tab.

A variety of Android emulators are actually aimed at end-users, allowing them to run games and other Android apps on their Windows computers. QA teams are also able to take advantage of these emulators in a pinch. BlueStacks and GenyMotion are two of the more popular examples of stand-alone Android emulators.

In addition to the iOS and Android platforms, emulators exist for some of the less popular mobile platforms, including BlackBerry, Java Micro Edition, and Windows 10 Mobile. Redmond also provides an integrated mobile emulator as part of the Visual Studio IDE, in the same manner as Apple and Google.

The wearable market is still in nascent stage, but emulators for the Apple Watch and Android Wear help QA engineers test apps targeted at these platforms. Expect more examples in this arena as wearables see wider acceptance from consumers. When it comes to most mobile testing, however, smartphones and tablets remain the focus for app and website development.

Stay tuned to the Betica blog for upcoming articles as we continue to explore the wide world of QA!

Testing Apps and Websites on Mobile Devices

With more users leveraging their smartphones and/or tablets instead of a personal computer, the importance of QA on mobile devices has increased over the past few years. The resource-constrained environment of a portable device — less RAM, smaller screen — raises issues for both developers as well as testers. Testing mobile websites and applications has become a convoluted process.

Adding to the complexity is the cornucopia of OS versions available for both the iOS and Android platforms, which doesn’t even take into account the differences in hardware, especially screen sizes. The fragmentation of the Android market also causes frustration for many mobile development shops.

This next series of blog posts focuses on the process of QA on mobile devices.

Mobile Website QA on iOS and Android

Testing web applications on desktops is a detailed task by itself considering the multitude of browsers (and different versions) all with their own slightly unique way of handling JavaScript and rendering CSS layout. Both developers and testers need to ensure websites operate in a consistent manner no matter the browser and its version number. Migrate that process onto a mobile device and other issues can arise.

The iOS platform partially benefits from Apple’s “walled garden.” Apple handles the manufacturing of all devices and keeps tight rein on what apps become available at the iTunes App Store. Only recently did they allow alternatives to Safari as a mobile web browser for the iPhone and iPad.

Android, on the other hand, is more of a free for all. Different device makers and a wider array of mobile web browsers make testing mobile web apps on Android a more difficult proposition. An open platform is also more prone to mobile malware, which means testing for security becomes paramount.

Emulators are necessary for Mobile QA

One of the difficulties facing QA managers responsible for mobile testing is the procurement of enough devices to properly vet apps and websites. Finding a good supply of older iPhones, iPads, as well as Android smartphones and tablets from different manufacturers is challenging. Because of this issue, the use of emulators simulating the operation of a mobile device plays a large role in mobile QA.

Software engineers also use emulators as part of the mobile development process. Apple’s Xcode includes iPhone and iPad emulators capable of running multiple iOS versions. Google also offers similar functionality in its Android IDE, but the wider variety of device makers adds an extra layer of complexity.

Since QA staff also employs emulation as a major aspect of their testing process, many companies are now offering stand-alone mobile emulators as an alternative to those contained within a software development IDE. One caveat to remember is an emulator is running with the horsepower of a desktop PC, so the overall performance tends to be better when compared to an actual device. The QA team needs to factor that in to their final test results.

Stay tuned to upcoming Betica blog posts as we dive deeper into the use of mobile emulators in QA.

The Importance of QA in Software Development

There’s no denying the importance of the QA role within the software development lifecycle. Some companies try to save money by giving the process short shrift. Relying on only acceptance testing, which doesn’t properly vet an application in a real world environment, only leads to higher losses if and when software fails in the future.

The old Fram oil filter commercial — “you can pay me now, or you can pay me later” — definitely applies in this case. The cost of fixing software bugs rises exponentially the later they are discovered in the SDLC. Software errors caused the U.S. economy to lose nearly $60 billion in 2002; today that number is probably closer to $100 billion.

To truly illustrate the importance of the QA process within software development, here’s a look back at a few famous software failures caused by inadequate testing. Hopefully learning about these mistakes helps prevents similar issues with the software developed and tested by your company.

Faulty Trading Software costs Firm $440 Million in 45 Minutes

In 2012, the U.S. stock trader, Knight Capital, reported a loss of $440 million due to a software bug within its trading application. The program confused the “bid” and “ask” price for a stock, essentially buying shares at the higher ask price and selling them instantaneously at the cheaper bid price. Introduce automated trading into the equation and Knight Capital lost nearly a half-billion dollars in less than an hour.

A Calculation Error in the Pentium taught Intel Customer Relations

Intel’s new Pentium microprocessor earned the chipmaker accolades in 1994, until a college mathematics professor discovered a calculation error in the chip’s embedded code. Intel tried to ignore the problem, reasoning that most users would never encounter the issue. However, enough disgruntled customers demanded a replacement, which Intel reluctantly agreed to, costing the company $475 million.

A mea culpa in Intel’s annual report to shareholders that year revealed the importance of QA to any company’s reputation. “We received a crash course in consumer relations. In the future, we intend to be better prepared to meet the public expectations that come with our dramatically higher profile,” the shareholder report commented.

Toyota Prius Software Glitch costs Billions

Toyota recalled nearly 400,000 Prius models in 2010 due a software bug in the car’s anti-lock braking system. The glitch caused the ABS to lag, which led to a potentially dangerous situation for drivers and passengers. The resultant class-action lawsuits due to this and other Toyota quality issues led to three billon dollars worth of payments.

The Great Blackout of 2003

In August of 2003, the Northeast United States suffered one of its worst blackouts in history. A glitch occurred when multiple power systems tried to simultaneously access the same data. Finding the software error that caused the problem required the analysis of millions of lines of code. The total cost of the bug: nearly $10 billion!

These examples of the real cost of software bugs reveal the need for QA at all points of a software project. Remember this golden rule: The earlier any bugs are found, the cheaper they are to fix!