Testing Apps and Websites on Mobile Devices

With more users leveraging their smartphones and/or tablets instead of a personal computer, the importance of QA on mobile devices has increased over the past few years. The resource-constrained environment of a portable device — less RAM, smaller screen — raises issues for both developers as well as testers. Testing mobile websites and applications has become a convoluted process.

Adding to the complexity is the cornucopia of OS versions available for both the iOS and Android platforms, which doesn’t even take into account the differences in hardware, especially screen sizes. The fragmentation of the Android market also causes frustration for many mobile development shops.

This next series of blog posts focuses on the process of QA on mobile devices.

Mobile Website QA on iOS and Android

Testing web applications on desktops is a detailed task by itself considering the multitude of browsers (and different versions) all with their own slightly unique way of handling JavaScript and rendering CSS layout. Both developers and testers need to ensure websites operate in a consistent manner no matter the browser and its version number. Migrate that process onto a mobile device and other issues can arise.

The iOS platform partially benefits from Apple’s “walled garden.” Apple handles the manufacturing of all devices and keeps tight rein on what apps become available at the iTunes App Store. Only recently did they allow alternatives to Safari as a mobile web browser for the iPhone and iPad.

Android, on the other hand, is more of a free for all. Different device makers and a wider array of mobile web browsers make testing mobile web apps on Android a more difficult proposition. An open platform is also more prone to mobile malware, which means testing for security becomes paramount.

Emulators are necessary for Mobile QA

One of the difficulties facing QA managers responsible for mobile testing is the procurement of enough devices to properly vet apps and websites. Finding a good supply of older iPhones, iPads, as well as Android smartphones and tablets from different manufacturers is challenging. Because of this issue, the use of emulators simulating the operation of a mobile device plays a large role in mobile QA.

Software engineers also use emulators as part of the mobile development process. Apple’s Xcode includes iPhone and iPad emulators capable of running multiple iOS versions. Google also offers similar functionality in its Android IDE, but the wider variety of device makers adds an extra layer of complexity.

Since QA staff also employs emulation as a major aspect of their testing process, many companies are now offering stand-alone mobile emulators as an alternative to those contained within a software development IDE. One caveat to remember is an emulator is running with the horsepower of a desktop PC, so the overall performance tends to be better when compared to an actual device. The QA team needs to factor that in to their final test results.

Stay tuned to upcoming Betica blog posts as we dive deeper into the use of mobile emulators in QA.

The Importance of QA in Software Development

There’s no denying the importance of the QA role within the software development lifecycle. Some companies try to save money by giving the process short shrift. Relying on only acceptance testing, which doesn’t properly vet an application in a real world environment, only leads to higher losses if and when software fails in the future.

The old Fram oil filter commercial — “you can pay me now, or you can pay me later” — definitely applies in this case. The cost of fixing software bugs rises exponentially the later they are discovered in the SDLC. Software errors caused the U.S. economy to lose nearly $60 billion in 2002; today that number is probably closer to $100 billion.

To truly illustrate the importance of the QA process within software development, here’s a look back at a few famous software failures caused by inadequate testing. Hopefully learning about these mistakes helps prevents similar issues with the software developed and tested by your company.

Faulty Trading Software costs Firm $440 Million in 45 Minutes

In 2012, the U.S. stock trader, Knight Capital, reported a loss of $440 million due to a software bug within its trading application. The program confused the “bid” and “ask” price for a stock, essentially buying shares at the higher ask price and selling them instantaneously at the cheaper bid price. Introduce automated trading into the equation and Knight Capital lost nearly a half-billion dollars in less than an hour.

A Calculation Error in the Pentium taught Intel Customer Relations

Intel’s new Pentium microprocessor earned the chipmaker accolades in 1994, until a college mathematics professor discovered a calculation error in the chip’s embedded code. Intel tried to ignore the problem, reasoning that most users would never encounter the issue. However, enough disgruntled customers demanded a replacement, which Intel reluctantly agreed to, costing the company $475 million.

A mea culpa in Intel’s annual report to shareholders that year revealed the importance of QA to any company’s reputation. “We received a crash course in consumer relations. In the future, we intend to be better prepared to meet the public expectations that come with our dramatically higher profile,” the shareholder report commented.

Toyota Prius Software Glitch costs Billions

Toyota recalled nearly 400,000 Prius models in 2010 due a software bug in the car’s anti-lock braking system. The glitch caused the ABS to lag, which led to a potentially dangerous situation for drivers and passengers. The resultant class-action lawsuits due to this and other Toyota quality issues led to three billon dollars worth of payments.

The Great Blackout of 2003

In August of 2003, the Northeast United States suffered one of its worst blackouts in history. A glitch occurred when multiple power systems tried to simultaneously access the same data. Finding the software error that caused the problem required the analysis of millions of lines of code. The total cost of the bug: nearly $10 billion!

These examples of the real cost of software bugs reveal the need for QA at all points of a software project. Remember this golden rule: The earlier any bugs are found, the cheaper they are to fix!

A Closer Look at the API Hierarchy of Needs

In this article in our series covering API testing — check out the introduction and an analysis of what features make up any quality testing tool — we train our eyes towards the API Hierarchy of Needs. While it is primarily relevant to public-facing interfaces, the same principles are applicable those developed and tested in-house. It is essentially a programming version of the human hierarchy of needs created by psychologist, Abraham Maslow.

API Technologist, Bruno Pedro, published the API Hierarchy of Needs at APIUX.com. Even though the hierarchy was written primarily for programmers, QA personnel are also able to wean valuable information about the qualities making up the best APIs.

Read further to gain a better understanding of what capabilities any good API needs to provide to programmers as well as those QA engineers responsible for its testing. Since the hierarchy follows an inverted pyramid structure, we’ll take a look at each of the five sections, starting at the bottom — the most important layer.

Usability

An API needs to be easy to setup and implement. This simple concept applies to both programmers and the QA team. Strong documentation, well-written RESTful URLs, and the use of standard HTTP error codes all play an important role in an API’s overall usability.

Functionality

All functional points within an API need to perform as expected; the documentation is vital in explaining their ultimate behavior. The QA team plays an important role in validating the underlying code, as well as ensuring the proper error codes bubble up when exceptions are thrown. Whether the API is public or private, this hierarchy layer is arguably the most important for the QA role.

Reliability

APIs need to perform as expected each and every time they are accessed. This is an area where stress testing becomes important. The QA team needs to help ensure uptime is maximized. This becomes especially vital for public-facing APIs which provide revenue for the company responsible for the API’s development and hosting.

Proficiency

The best APIs actually increase the development skills of the programmers working with them. In some respects, they also offer a similar benefit to the QA personnel responsible for testing and validation. Rich documentation of the APIs functionality plays a big role in increasing the technical knowledge of the development and QA teams.

Creativity

The top layer of the API Hierarchy of Needs relates to creativity. In short, does the API inspire developers to use the interface in creative and unexpected ways? Bruno Pedro describes a scenario where APIs are mixed together to ultimately create a stronger product. QA personnel needn’t be shy about providing their own insight into new uses for an API.

While the API Hierarchy of Needs focuses on development staff, QA team members responsible for API testing should also take the time to understand these concepts as it improves their overall skill set, enabling them to offer meaningful suggestions to improve all aspects of an API. In this era of DevOps, collaboration is vital to ensure a strong and useful product.

Stay tuned to the Betica blog for future discussions covering the Wide World of QA.