SoapUI vs. JMeter — Options for Testing Web Services

Testing web programming goes beyond what a user sees in their browser or desktop application. Back end functionality residing within web services or APIs also depends on the QA role to ensure error-free operation. Software development shops increasingly leverage open source tools like SoapUI and JMeter to help test their web services.

What follows is a quick overview of both applications to give you a better feel of their features as well as their differences. Maybe one or both need to be part of your team’s QA arsenal?

A Closer Look at SoapUI

SoapUI was first released in 2005. Its basic version remains a freely available open source application, but a commercial edition, called SoapUI NG Pro, is offered by SmartBear Software, primary developers of both versions. Experienced enterprise users looking for automated testing and other productivity improvements are the main audience for the commercial application.

Web service and API QA is the main focus of SoapUI, making it a valuable tool for applications designed using a service-oriented architecture (SOA). A whole host of useful features include functional API testing, web service simulation, security QA, and load testing. It easily handles most messaging formats: SOAP, REST, WSDL, and more.

Some basic automation functionality comes with the free version of SoapUI; mostly command line interactivity with other open source build and scheduling tools. Pay for SoapUI NG Pro for advanced automation capabilities. An analytics/reporting suite in addition to test recording and generation round out the feature set.

Both versions of SoapUI run on all the major operating systems while integrating with many popular IDEs.

Apache JMeter focuses on Load Testing

The first version of JMeter became available way back in the 20th Century (1998). It remains one of the most popular applications developed by the open source Apache Software Foundation. Many developers continue to leverage JMeter’s plug-in architecture to customize its features.

While offering a measure of API testing functionality, JMeter focuses on web application performance analysis, including APIs, web services, and database connectivity, in addition to load testing. It seamlessly handles most popular web protocols, including HTTP/HTTPS, SOAP, REST, FTP, LDAP, and more. An easy-to-use interface facilitates the creation of test plans.

In addition to testing web services, JMeter also analyzes the performance of web applications written in a variety of languages — PHP, ASP.NET (C#, VB.NET), and Java. A basic reporting engine is provided, with additional functionality available through the use of plug-ins developed by the application’s robust user community.

Choosing between SoapUI and JMeter

One of the benefits of open source software is the ability to try out an application with your own team before investing any resources on a commercial option. This same rule applies when deciding to use SoapUI, JMeter, or potentially both. There is some crossover between the two applications, but SoapUI offers more features aimed at functional testing, while performance analysis and load testing remain JMeter’s specialties.

The smart plan is to get hands-on with both applications to get a feel for their strengths and weaknesses. For bigger shops with large budgets, SoapUI NG Pro might be the closest thing to a true turnkey solution for web service and load testing. Ultimately, having both applications in your team’s toolbox is a wise choice.

Stay tuned to the Betica Blog for additional insights into the world of software development and testing.

News from the Worlds of Software Development and QA — June 2016

As June draws to a close, here are a few interesting stories from the related worlds of software development and QA. If you want to check out last month’s news, simply click on this link. Hopefully, this month’s edition offers some insights relevant to your daily development and testing work.

Need QA for Continuous Deployment? Enter Continuous Testing.

We’ve talked previously about the growing adoption of Agile and DevOps methodologies by companies hoping to gain a competitive advantage through faster software development. Many of these firms strive for a continuous deployment model where software enhancements happen at a rapid pace.

So how does the QA process keep up? Enter continuous testing.

Many forward-looking companies are beginning to leverage continuous testing to ensure software gets released at a speed letting them successfully compete in the modern business landscape. Ashley Dotterweich recently talked about the emerging practice for DevOps Zone. Faster release cycles, better code, and ultimately a decreased risk of production failures are its major impacts.

One of the key steps in implementing continuous testing involves shifting QA to the left. In short, this involves starting testing earlier in the software development process in the hope of catching problems at a point before a fix becomes too costly. Some would argue a QA presence also needs to be involved during the requirements gathering and design phases.

The use of a continuous integration server infrastructure to execute unit tests also facilitates the implementation of continuous testing. Other forms of automated testing need to be considered as part of a migration to this new testing model. It is something worth considering for organizations hoping to achieve continuous deployment.

Game Development Shops want Standardized Testing Practices

A myriad of AAA video games suffering from high profile bugs and server hiccups over the last year has led to a call for standardized QA practices in the industry. James Batchelor covered the growing issue in a recent post on Develop, a game industry website. Many testing managers in gaming feel standards are vital for keeping (or regaining) the trust of video gamers. 

Testology CEO, Andy Robson, commented on the growing problem caused by a lack of QA standards. “Why do we think it is acceptable to release games that don’t meet the quality level consumers expect? We should have a standard where no Class A bugs are released in a product along with Class B bugs, whether functional or LOC issues. Class C bugs are always going to be in games, but don’t affect the experience, so we could be more lenient,” said Robson.

Stay tuned to see if their efforts at standardized QA are successful.

Is Software Development only for the Young?

A recent study noted the average age for the software developer is under 30. Nick Heath, writing for Tech Republic, wondered if this was due to a glut of new programmers entering the industry or if older developers are leaving? Natalia Radcliffe-Brine, marketing manager at Stack Overflow feels it is the former trend.

“I don’t think it’s that the older developers aren’t there anymore, I think there’s been momentum around technology and you’ve got so many more young people going into computer science,” said Radcliffe-Brine. Whatever the reason behind the hard data, there’s no denying the right mixture between younger workers excited about the industry and the wisdom of veteran developers is a smart call when building an efficient development team.

Stay tuned to the Betica Blog for additional news and insights from the world of software development and testing.

Powering Mobile QA with Smartphone Test Farm

Previously on the Betica Blog, we covered a variety of topics related to mobile QA, including how to build a device testing lab, as well as taking a look at some useful tools to make mobile testing more efficient. Validating software quality on a resource-constrained smartphone or tablet brings with it a host of issues, not the least of which is the wide array of mobile devices, especially on the Android platform. There has to be an easier way?

A web application called the Smartphone Test Farm hopes to make Android application testing an easier process. This tool actually lets you perform remote testing of Android devices as if they were directly connected to your computer — all within your browser. Let’s look more closely at this powerful mobile testing framework.

A Closer Look at Smartphone Test Farm

Smartphone Test Farm (STF) was first developed by the software firm, CyberAgent, for use in their own mobile development projects. The tool was so successful they decided to make it available for everyone last year as an open source application on GitHub. The Rails web application framework is another example of a useful internal project ultimately released to the wider development community.

STF is compatible with nearly all modern web browsers and supports over 170 Android devices — including their specific screen dimensions. The tool features a simple drag and drop interface for installing Android APK files onto a remotely connected device. It is useful for testing both standalone and web applications with the system logs for each device accessible within the web browser.

Other STF features helpful to the QA professional include: bidirectional copy and paste on each device, screenshots, running shell commands from within the browser, remote debugging with Android Studio, as well as keyboard and mouse input with multi-touch. An inventory management interface, including a powerful search feature, makes managing a large number of testing devices a simple process.

The application supports all Android versions from 2.3 to 5.1 (the Android N preview as well) and is also able to handle many Android variants like the Amazon FireOS. While STF doesn’t test Apple iOS apps, you are able to use the tool from within the iOS Safari browser. Considering screen real estate issues, using STF from an iPhone is more of a curiosity, but it may actually be useful on an iPad Pro.

An Easy Installation using GitHub

Installing Smartphone Test Farm is a breeze; just follow the simple instructions on the STF GitHub page. Also keep an eye on the GitHub page for the latest information about the tool as well as new feature announcements. For support, the GitHub Issues list and a Google Groups mailing list are available; CyberAgent also offers an email address for direct questions.

If your organization is responsible for Android app and web application development, Smartphone Test Farm is definitely worthy of further exploration. It might end up as an invaluable part of your mobile QA toolbox.

Keep an eye on the Betica Blog for additional insights and ideas from the worlds of software development and QA.