The Importance of QA in Software Development

There’s no denying the importance of the QA role within the software development lifecycle. Some companies try to save money by giving the process short shrift. Relying on only acceptance testing, which doesn’t properly vet an application in a real world environment, only leads to higher losses if and when software fails in the future.

The old Fram oil filter commercial — “you can pay me now, or you can pay me later” — definitely applies in this case. The cost of fixing software bugs rises exponentially the later they are discovered in the SDLC. Software errors caused the U.S. economy to lose nearly $60 billion in 2002; today that number is probably closer to $100 billion.

To truly illustrate the importance of the QA process within software development, here’s a look back at a few famous software failures caused by inadequate testing. Hopefully learning about these mistakes helps prevents similar issues with the software developed and tested by your company.

Faulty Trading Software costs Firm $440 Million in 45 Minutes

In 2012, the U.S. stock trader, Knight Capital, reported a loss of $440 million due to a software bug within its trading application. The program confused the “bid” and “ask” price for a stock, essentially buying shares at the higher ask price and selling them instantaneously at the cheaper bid price. Introduce automated trading into the equation and Knight Capital lost nearly a half-billion dollars in less than an hour.

A Calculation Error in the Pentium taught Intel Customer Relations

Intel’s new Pentium microprocessor earned the chipmaker accolades in 1994, until a college mathematics professor discovered a calculation error in the chip’s embedded code. Intel tried to ignore the problem, reasoning that most users would never encounter the issue. However, enough disgruntled customers demanded a replacement, which Intel reluctantly agreed to, costing the company $475 million.

A mea culpa in Intel’s annual report to shareholders that year revealed the importance of QA to any company’s reputation. “We received a crash course in consumer relations. In the future, we intend to be better prepared to meet the public expectations that come with our dramatically higher profile,” the shareholder report commented.

Toyota Prius Software Glitch costs Billions

Toyota recalled nearly 400,000 Prius models in 2010 due a software bug in the car’s anti-lock braking system. The glitch caused the ABS to lag, which led to a potentially dangerous situation for drivers and passengers. The resultant class-action lawsuits due to this and other Toyota quality issues led to three billon dollars worth of payments.

The Great Blackout of 2003

In August of 2003, the Northeast United States suffered one of its worst blackouts in history. A glitch occurred when multiple power systems tried to simultaneously access the same data. Finding the software error that caused the problem required the analysis of millions of lines of code. The total cost of the bug: nearly $10 billion!

These examples of the real cost of software bugs reveal the need for QA at all points of a software project. Remember this golden rule: The earlier any bugs are found, the cheaper they are to fix!

A Closer Look at the API Hierarchy of Needs

In this article in our series covering API testing — check out the introduction and an analysis of what features make up any quality testing tool — we train our eyes towards the API Hierarchy of Needs. While it is primarily relevant to public-facing interfaces, the same principles are applicable those developed and tested in-house. It is essentially a programming version of the human hierarchy of needs created by psychologist, Abraham Maslow.

API Technologist, Bruno Pedro, published the API Hierarchy of Needs at APIUX.com. Even though the hierarchy was written primarily for programmers, QA personnel are also able to wean valuable information about the qualities making up the best APIs.

Read further to gain a better understanding of what capabilities any good API needs to provide to programmers as well as those QA engineers responsible for its testing. Since the hierarchy follows an inverted pyramid structure, we’ll take a look at each of the five sections, starting at the bottom — the most important layer.

Usability

An API needs to be easy to setup and implement. This simple concept applies to both programmers and the QA team. Strong documentation, well-written RESTful URLs, and the use of standard HTTP error codes all play an important role in an API’s overall usability.

Functionality

All functional points within an API need to perform as expected; the documentation is vital in explaining their ultimate behavior. The QA team plays an important role in validating the underlying code, as well as ensuring the proper error codes bubble up when exceptions are thrown. Whether the API is public or private, this hierarchy layer is arguably the most important for the QA role.

Reliability

APIs need to perform as expected each and every time they are accessed. This is an area where stress testing becomes important. The QA team needs to help ensure uptime is maximized. This becomes especially vital for public-facing APIs which provide revenue for the company responsible for the API’s development and hosting.

Proficiency

The best APIs actually increase the development skills of the programmers working with them. In some respects, they also offer a similar benefit to the QA personnel responsible for testing and validation. Rich documentation of the APIs functionality plays a big role in increasing the technical knowledge of the development and QA teams.

Creativity

The top layer of the API Hierarchy of Needs relates to creativity. In short, does the API inspire developers to use the interface in creative and unexpected ways? Bruno Pedro describes a scenario where APIs are mixed together to ultimately create a stronger product. QA personnel needn’t be shy about providing their own insight into new uses for an API.

While the API Hierarchy of Needs focuses on development staff, QA team members responsible for API testing should also take the time to understand these concepts as it improves their overall skill set, enabling them to offer meaningful suggestions to improve all aspects of an API. In this era of DevOps, collaboration is vital to ensure a strong and useful product.

Stay tuned to the Betica blog for future discussions covering the Wide World of QA.

What makes a Good API Testing Tool

Last week, we looked at API testing and its importance in ensuring a web application’s business rules and other functional areas are properly vetted before release. A QA process that only tests the presentation layer is insufficient for applications using a service-oriented architecture. In short, full API testing leads to a more robust and stable product.

This time out, we train our eyes at what functionality is necessary in any top notch API testing tool. In this collaborative era of DevOps and the Agile Manifesto, QA teams are tasked with accomplishing more in less time with fewer resources. Any good testing tool needs to function in a transparent manner, supporting the QA professional as they validate the application in question.

Here are some features any worthy API testing application needs to provide.

Easy Generation of Test Scripts

The modern QA process works at the speed of business — fast! Your QA software needs to support the quick generation of test skeletons and other relevant scripts. Even large APIs with complex testing scenarios shouldn’t require an inordinate amount of time to build test cases.

The Ability to quickly target Multiple Environments

One of the advantages of SOA development is the ability to swap out different tiers as development and testing needs change. Testers are faced with having to target multiple QA environments, sometimes using different data sources depending on the scenario. Any testing tool worth its salt must allow them to easily switch targets as the need arises.

Automation!

The modern QA professional works in a collaborative manner — this is the era of DevOps after all. The capability to automate the running of test scripts gives them time to attend meetings, helping ensure the QA voice is properly heard during all parts of the software development lifecycle — the earlier the better! Providing command-line support also allows tests to be included as part of the overall build process.

Platform and Framework Agnostic

No matter if a web application is written in Java, PHP, Ruby on Rails, or for the .NET framework, your testing tool needs to function in an agnostic fashion. A QA engineer can’t worry themselves with the language used for an application. They need to focus on testing each business rule and the other functionality contained within the API.

This same rule also applies to web service messaging formats — REST, SOAP, etc. Any API testing tool needs to support all major web development frameworks in a seamless manner.

Robust Test History and Reporting

When something goes wrong, the tester or developer needs to see at a glance what happened. Any API testing tool needs to offer a detailed test history, easily allowing the comparison of different test results. Meaningful metrics and other reporting features also help the QA team quickly discern the overall quality of any application.

If you or your QA staff are in the market for API testing software, hopefully this article offered a measure of insight before a purchase is made. Stay tuned for additional posts on Betica blog covering all aspects of the QA process.