Continuous Integration (CI) and Continuous Delivery (CD)

Use continuous Integration (CI) and continuous delivery (CD) to improve the speed of software delivery. This book presents a game changer—how to use pipelines to automate the software delivery process. The theories about CI/CD are much the same, but the book covers what the development of pipelines looks like and how testing of pipelines themselves should be performed.

Most teams just plunge into coding, without thinking about the CI/CD process itself. Why don’t we use the same development method for pipelines that we use for apps?

Pipelines code development undergoes similar stages as application code development, such as requirements analysis, development, testing, implementation, operations, and monitoring. This is the starting point of the book. It describes the current challenges with pipeline development and how this process can be improved and structured. It describes in detail how to design pipelines and shows examples in BPMN 2.0 notation.

WHAT YOU’LL LEARN

* Know the shortcomings and challenges of current pipeline development such as misalignment between the pipeline engineer and the team’s workflow, the use of infrastructure as code (IaC), and pipeline security
* Understand the need for CI/CD requirements through the book's non-exhaustive list of more than 60 CI/CD requirements provided to inspire and increase awareness
* See how certain choices affect the way a pipeline is designed (and realized)
* Become familiar with branching strategy, build strategy, test strategy, release strategy, and deployment strategy that are explained in detail in the book, including their effect on pipeline design
* Know how pipelines can be unit tested, using a real-world example
* Know how performance bottlenecks in a pipeline occur, how they can be detected, and how they can be solved
* View a complete implementation, including code, showing how the guidelines in this book are applied to a real use case

WHO THIS BOOK IS FOR

DevOps engineers and solution architects involved with automating the software supply chain and using application lifecycle management (ALM)/integration platforms such as Jenkins, CircleCI, Bamboo, and Azure DevOps; intermediate and experienced DevOps engineers (developers, ops engineers, test engineers); and ICT managers interested in the CI/CD pipeline development domain

HENRY VAN MERODE is a Solution Architect with over 30 years of experience in ICT within several financial organizations. His experience spans a wide range of technologies and platforms, from IBM Mainframe to Cloud systems on AWS and Azure. He developed, designed, and architected major financial systems such as Internet Banking and Order Management systems, with a focus on performance, high availability, reliability, maintainability, and security. Last 8 years, Henry’s expertise has been extended with Continuous Integration, Continuous Delivery, and automated pipelines. As an Azure DevOps community lead, Henry likes to talk about this subject and promotes automating the software supply chain to the teams at his work.

CHAPTER 1. THE CI/CD PITFALL

Challenges

CHAPTER 2. CONCEPTS

Principles

Positioning of CI/CD

Application Lifecycle Management (ALM)

CI/CD strategy

Naming convention

CHAPTER 3. REQUIREMENTS ANALYSIS

Generic

Workflow

Technology

Information

Security

Resource constraints

Manageability

Operations

Quality Assurance

Metrics

Monitoring

Governance

CHAPTER 4. PIPELINE DESIGN

CI/CD and pipeline design approach

BPMN 2.0

BPMN elements overview

BPMN in action

Level of detail

Logical design versus realization

The Generic CI/CD Pipeline

Validate entry criteria

Execute build

Perform unit tests

Analyze code

Package artifact

Publish artifact

Provision test environment

Deploy artifact to test

Perform test

Validate infrastructure compliancy

Validate exit criteria

Perform dual control

Provision production environment

Deploy artifact to production

Notify actor

Design strategies

Context diagram

Branching strategy

Trunk-based workflow

Feature branch workflow

Gitflow

Build strategy

Build time – Vertical scaling

Build time – Full builds versus incremental builds

Build time – Parallel builds

Build targets

Cross-platform builds

Multi-team build strategy

Test strategy

Automated versus manual tests

Functional versus non-functional tests

Parallel execution versus sequential execution

Manual tests performed by specialists

Long execution time versus short execution time

Production deployment strategy

Recreate deployment strategy

Blue/green deployment

Canary testing deployment

A/B test strategy

Separation of concerns

Delegation

Application architecture

Orchestrator

Event-based CI/CD

Resource constraints

Parallelize stages and tasks

Timeboxed delivery

Commercial of the Shelf

CHAPTER 5. PIPELINE DEVELOPMENT

Pipeline specification

Multi-branch, multi-stage pipeline

User interface-based pipelines

Scripted pipelines

Declarative pipelines

Declarative Jenkins pipeline

Declarative Azure DevOps pipeline

Repositories - Everything as code (EaC)

Development in the value streams

Simplified pipeline development

Extended pipeline development

Advanced pipeline development

Develop a base pipeline

Pipeline generation

Pipeline of pipelines (DevOps assembly line)

Trigger

Validate entry criteria

Execute build

Perform unit tests

Analyze code

Package artifact

Publish artifact

Provision test environment

Deploy artifact to test

Perform test

Validate infrastructure compliancy

Deploy artifact to production

Notify actor

Constructs

Triggers

Execution environment

Connections

Conditions and conditional variables

Matrix

Deployment strategy

Auto-cancel

On success/failure

Fail fast

Priority

Parallelism

Shards

Templates

Decorator

Gates and approvals

Workflow

Plugins

Feature management

Third-party libraries and containers

Versioning and tagging

Environment repository

Secrets management

Sustainable pipeline development

CHAPTER 6. TEST PIPELINES

Testability of pipelines

Unit tests

Performance tests

Pipeline compliance

Acceptance tests

CHAPTER 7. PIPELINE IMPLEMENTATION

Organizational preparations

CI/CD Infrastructure preparations

System

Security

Target environment preparations

System

Security

Pipeline preparations

Release note

Runbook

Artifact promotion

CHAPTER 8. OPERATE AND MONITOR

Operations

Monitoring

Systems monitoring

Platform monitoring

Security monitoring

Share information

Events, alerts, incidents and notifications

REFERENCES