News from the World of Software Development – November 2017

Welcome to this month’s edition of the Betica Blog news digest, looking at a few recent stories of interest to the software development community. With the Holidays rapidly approaching, it becomes time to ramp up the effort to successfully reach your organization’s year end goals. Perhaps the stories within provide a measure of insight to inspire your team? If interested in last month’s digest, simply click on the following link.

Is DevOps’ Maturity Level being exaggerated by CIOs?

We regularly talk about DevOps: its growing adoption rate, the advantages it brings to the software development process, strategies on how to implement it, and more. A recent study by the tech industry research group, Forrester, notes that executives exaggerate the maturity level of their DevOps adoption. ZDNet reported on the study findings earlier this week.

In short, Forrester feels CIOs overstate their progress at implementing DevOps. According to the research firm: “[businesses] are gaining some speed, but the primary benefit connecting development and operations for both speed and quality is still elusive for many.” A major issue hampering full adoption involves the large cultural changes that need to happen for a successful implementation.

Increasing the speed of software delivery is one thing, but sometimes additional velocity comes at a cost in overall quality. This appears to be another issue affecting the perception of DevOps maturity. In fact, different perspectives exist between CIOs and DevOps practitioners as far as the overall success of the practice at their business.

Another reason executives exaggerate their organization’s success level with DevOps relates to overestimating the number of automated processes in place in their software development practice. Automation remains a key indicator of DevOps adding value to a company, according to Forrester.

Ultimately, Forrester concludes that “[f]or an enterprise to implement DevOps successfully, both executives and practitioners need to understand the current maturity state. Executives and practitioners differ widely in their perspectives on strategy, customer experience and progress they have made on their DevOps journey. When pros and executives are on the same page, your business wins!”

Microsoft lets F# target .NET Core Projects

Microsoft introduced .NET Core in August, an open source version of its software framework; an example of Redmond’s friendlier stance towards the open source software community. The preview version of Visual Studio, the company’s flagship IDE, allows F# programmers to target .NET Core as well the standard version of the framework. News about this new VS feature appeared this week in Visual Studio Magazine.

F# is Microsoft’s largely functional programming language that includes a measure of object-oriented and imperative methodologies. The F# Software Foundation also contributed to its development and produced an open source compiler for the language.

Microsoft’s Phillip Carter commented on the new F# support for .NET Core. “Finally, we are laying the groundwork for a long-term effort of migrating all F# projects to the new project system that .NET Core and .NET Standard projects use,” said Carter. Microsoft maintains a repository for their open source F# compiler and other related tools on GitHub.

Uses for F# abound; the language is especially known for its brevity and ease of maintenance. For example, Credit Suisse relies on the language for writing quantitative models for the financial industry. Other applications include asset portfolio optimization, machine learning, business intelligence, and more.

Hopefully this edition of the Betica Blog news digest provided some insight to help in your daily duties. As always, thanks for reading!

PostgreSQL 10 – New Features and Functionality

PostgreSQL remains a popular option for organizations that need a traditional SQL database, but don’t want to spend the money required for Oracle. We’ve covered this open source database in the past here on the blog. For those companies who want extra support, a commercial Postgre option like EnterpriseDB needs to be considered.

With PostgreSQL 10 scheduled for release later this year, many users are undoubtedly curious about the new features and functionality. Let take a closer look at what’s in the feature set so you can consider either an upgrade or using this new version on your next development project.

Improved Query Performance

One of the most important enhancements in PostgreSQL 10 is its faster query executor. The database is already known for performing essentially as fast as Oracle, so any additional speed boost is sure to make those benchmark comparisons even closer.

Robert Haas, Vice President for Enterprise DB and a major contributor to the PostgreSQL codebase, commented on the technical changes behind the executor’s performance boost. “Hash aggregation has been rewritten to use a more efficient hash table and store narrower tuples in it, and work has also been done to speed up queries that compute multiple aggregates and joins where one side can be proven unique,” said Haas.

Improved parallelism is another enhancement in Postgre aimed at boosting query performance. Haas noted that parallel queries now run two to four times faster in version 10. Index scanning is another function now faster because of parallel processing.

The new XMLTABLE support improves query processing against data stored internally as XML. This is the one PostgreSQL 10 enhancement aimed at the NoSQL market.

Replication is now Better – and Easier

PostgreSQL 10 now supports replication at the table level; previous versions required the full database to be replicated. This additional flexibility comes with the bonus of being easier to use as well. Called Logical Replication, it is a feature greatly anticipated in the PostgreSQL community.

Extended Statistics help with Query Planning

Developers who write complex queries against a PostgreSQL 10 instance enjoy the benefit of expanded statistics that help the query planning process. Haas explains this in more detail: “If the query planner makes a bad row count estimate resulting in a terrible plan, how do you fix it?  With extended statistics, you can tell the system to gather additional statistics according to parameters that you specify, which may help it get the plan right.”

Other PostgreSQL 10 Enhancements

Other significant Postgre 10 improvements include Declarative Partitioning which makes inserting new records faster, among other benefits. Support for SCRAM authentication enhances the security of a database instance. Durable Hash Indexes are another new feature aimed at boosting database performance.

One future enhancement potentially coming out in a point release is just-in-time compilation. This is expected to add yet another performance boost to any PostgreSQL implementation.

PostgreSQL 10 definitely adds enough new functionality for current users as well as organizations interested in an alternative to Oracle. While its NoSQL support remains limited, it is definitely a traditional SQL database worthy of your interest. EnterpriseDB also offers commercial-level support for companies still wary of an open source solution.

Keep returning to the Betica Blog for additional dispatches from the software development world. Thanks for reading!

Implementing Continuous Delivery with Screwdriver

Continuous Delivery is the Holy Grail driving many companies to initiate Agile and DevOps programs with the hope for rapid software development, enhancements, and fixes. In a competitive business world, quality applications are a big differentiator between the top organizations and those merely treading water. CD helps enterprises manage their development process in a more efficient manner.

Screwdriver is an application from Yahoo that facilitates Continuous Delivery; the company recently released it into open source status. Let’s take a closer look at the software to see if its features make it worthy of exploration as part of your company’s CD program.

A Closer Look at Screwdriver

Offering scalable continuous delivery, Screwdriver is a tool well-suited for organizations with a widely distributed application infrastructure. Yahoo developed the app for its own array of internal applications. It is possible they are open sourcing it before the company is fully acquired by Verizon.

James Collins, Yahoo’s senior director of engineering commented on the tool’s major advantages. “Screwdriver handles over 25,000 builds per day and 12,000 daily Git commits as a single shared entry point for Yahoo. It supports multiple languages and handles both virtual machine and container-based builds and deployment,” said Collins. Enterprises with large development teams stand to benefit from the application as part of their own CD initiatives.

The application’s architecture is relatively straightforward. A front-end manages handles users input and receives status messages from the rest of the software. The back-end includes a stateless API, which controls and launches builds and other functionality. Docker Swarm is used within Screwdriver to manage environment clusters.

According to Collins, the three main benefits of the application are making deployment pipelines easy, optimizing for trunk development, and to facilitate change rollbacks. Yahoo wanted their trunks shippable, so they leveraged Selenium to handle automated testing as part of Screwdriver’s build process. In short, Screwdriver allows for the efficient management of the build process – an important component of any Continuous Delivery initiative.

Screwdriver has been in use at Yahoo for over five years and is a major part of the company being able to implement CD and DevOps. Collins feels the tool lets programmers fully control the state of an application’s production infrastructure – a key part of the company’s DevOps organizational structure.

The Future of an Open Source Screwdriver

Yahoo is still planning to make enhancements to Screwdriver. These include the storage of build metadata, improved metrics and reporting, system templates, as well as more detailed log analysis. All these features are expected to let companies better refine their Continuous Delivery processes.

With a great proprietary CD-optimized build tool, what prompted Yahoo to release the application to the vibrant open source community? It appears to be an act of charity with a potential benefit to many organizations hoping to implement Continuous Delivery. These companies need to fully explore what Screwdriver brings to the table to ensure they are able to stay competitive with the rest of the business world.

James Collins sums up how Screwdriver enabled CD at Yahoo. “Yahoo’s engineering has modernized as it has embraced Continuous Delivery as a strategy for improving product quality and engineering agility. All our active products deliver from commit to production with full automation and this has greatly improved Yahoo’s ability to deliver products,” said Collins.

Keep returning to the Betica Blog for additional news and insights from the robust world of software development. Thanks for reading!