With the development and testing of builds continuing to grow not just in complexity but also in scale, the need for improved, efficient management of changes, approvals, and merges are evolving as well. It is from these expanding requirements that the concepts and practices of CI/CD pipeline management and CI/CD tools have stemmed from, resulting in the development of Zuul, a flexible, high-performing continuous integration platform tool.
Zuul was initially developed back in 2012 for open source CI testing, enabling its users to efficiently and effectively develop together across multiple projects. Essentially, the problem that Zuul was designed to solve is that of providing developers with a tool to test serialized future states faster than was possible when executing the tests for them serially. The concept of processor design borrowing pipeline provided a sort of “optimistic speculation” by which one could effectively parallelize this testing in cases where jobs were likely to pass. It later became a tool for the automatic merging, building, and testing of any new changes being made to a project, delivering an automated gated workflow pipeline. It is through this gating that new or changed code is prevented from making it into production without passing the unit integration tests designed and defined by the project keeper.
As open-source software, Zuul is designed to run irrespective of your cloud infrastructure driver, be it OpenStack, Openshift, Kubernetes, or AWS EC2.
Within the control plane, Zuul requests single-use test nodes to execute Zuul jobs. Upon Zuul’s request, Nodepool launches a clean node every time to run tests. Zuul’s compatibility allows for the nodes to be chosen from any of the infrastructures mentioned above, ensuring that the selected node is best suited for your infrastructure driver.
By automating the multi-cloud code merging process, workloads are effectively and efficiently coordinated, while hybrid workflows are easily managed. The DevOps process is also integrated, promoting innovation by enabling both halves to focus on their core competencies. Fundamentally, developers can concentrate on code as opposed to the production process, and the operations team no longer needs to act as a gate-keeper.
Zuul CI has had an extensive integration with Gerrit and is known for its use of Nodepool for the on-demand launch of required servers. Additionally, the latest release, Version 3, supports a much broader range of use cases and development platforms. Some of the most notable being its integration with GitHub. The Zuul CI GitHub compatibility is a step further in the tools support of multi-repository dependencies, hence, proving to be advantageous for both developers and users.
With its OpenStack Ansible integration, Zuul empowers users to deploy their software anywhere between bare metal, virtual machines, containers, and network switches. Another benefit is that the same Ansible playbooks can be used in the deployment of systems and running of tests, making it repeatable without side effects.
Owing to Ansible’s clear syntax and task-based nature, you also gain a more straightforward way of orchestrating the deployment of an application. Ansible’s library of modules and extensibility empowers users to orchestrate between multiple machines, all of which take place using a straightforward language while ensuring that tasks in the playbook are carried out in the appropriate sequence.
When looking into the available software for managing your continuous integration/continuous delivery pipeline, Zuul stands out through its co-gating capabilities. Co-gating functions the same way as gating, with the critical difference being that more than one software is tested across numerous repositories, all of which are working towards a common goal. The goal is to create a failsafe testing environment as not only will a change not pass if it breaks its project, but it also won’t pass if it breaks a different project.
Regardless of whether code is dependant on multiple other systems, Zuul ensures that they are all tested together using a shared queue before applying any patches, taking the complications out of multi-project testing.
Additionally, for any repository that is known to Zuul, users can specify a dependency, meaning that one change is dependent on another. It includes changes in other projects with projects residing in different systems, as well as testing across branches of the same repositories. The testing of such changes can be managed through both dependent and independent pipelines, ensuring that dependencies are always accounted for.
Parallelization & Testing
Through Zuul’s knowledge of parallelization, multiple contributors can apply code changes all at the same time. Zuul itself knows not only where to run things and the order they should be run in but is also aware of the states they should run in and the way it should respond to the results.
ZUUL AND NODEPOOL
Supporting a Multi-Cloud Environment
Nodepool acts as a companion pool manager who has access to multiple cloud infrastructure resources. Through Nodepool, Zuul can execute jobs on nodes or pods most compatible with your infrastructure. Therefore, AWS Zuul is just one of many other deployment options available to Zuul users. Moreover, Nodepool’s integration with Ansible eases the automation and multi-cloud merging process. Nodepool enables multi-cloud support through its ability to launch nodes from the following infrastructure drivers: OpenStack Driver, Static Driver, Openshift Driver, Kubernetes Driver, Openshift Pods Driver, and AWS EC2 Driver.
Through its origins of ensuring that all the components for each OpenStack release work together, which span over 800 repositories, along with OpenDev currently hosting 2129 Git repositories, OpenStack Zuul is ideal for enterprise-grade organizations running large and complex environments. However, both small and medium-sized businesses are also apt to reap the benefits of Zuul’s CI/CD management. To illustrate Zuul’s flexibility, the following are excerpts from Superuser that highlight its use case for the automobile industry, through a BMW case study, as well as for OpenLab, an open-source ecosystem testing, development, and collaboration project.
“Software has been an integral part of cars for several decades and has become one of the key enablers for many modern safety and comfort features. The amount of software required to implement all these features and complexity inducted by the many configuration options of current cars are constantly rising.” Says Tobias Henkel, a BMW software engineer.
“After using CI/CD systems for many years for an ever-increasing amount of projects, the limitations of the existing CI solutions were starting to impact our software development efforts. With the increasing size and complexity of today’s software projects, such as autonomous driving, the scaling capabilities of our CI/CD solution have become a crucial prerequisite for future development. The Zuul solution, especially after the release of version 3.0, fully supports all our requirements to provide a centrally hosted solution that can be shared by many internal software projects. It dramatically reduces operations overhead and frees up valuable developer time to improve all aspects of our CI system setup continuously.”
In the use case of OpenLab, Melvin Hillsman, a member of its governing board, explains, “Zuul is currently being used as an offering within OpenLab for projects/applications/tools that need CI gating and/or automation around testing. With the companion tool Nodepool we’re able to keep OpenStack VMs available, speeding up the process for developers of testing code changes.
Primarily the benefit we see is the ease of integration with other projects. Along those lines, Zuul is beneficial in general for open source users who utilize Ansible for other problem domains because it allows for a common language/workflow to be utilized both within the CI/CD system and outside it, opening the door for other tools to be more easily integrated.”