News Update: IOI's Metadata Driven ETL Tool ™ CI/CD process is up and running in Microsoft Azure June 3, 2019 - Kevin Herbert The CI/CD for the ETL engine starts with the creation of a DevOps project in Azure. This project provides the building blocks (version control, build pipelines, release pipelines, etc.) necessary to automate the continuous delivery of extract, transform, load (ETL) engine releases into the different staging areas up through production where customers interact with the final product. The continuous integration (CI) portion of our software development life cycle (SDLC) starts with a build pipeline that monitors for changes in version control. As changes are detected, an instance of the build pipeline gathers up all source control artifacts and begins building a cross-platform docker-based image of the application and all required dependencies. The pipeline then tags the image with a unique build number and pushes the image to an Azure container registry. The final step of the build pipeline is to publish deployment artifacts that will be needed in the next stage (continuous delivery). The continuous delivery (CD) stage, involves the creation of a release pipeline. The release pipeline listens for new builds as well as changes to the container registry to initiate a new release. The release pipeline provides the mechanism to promote code from one environment to the next with optional approval processes and test feedback loops. The first step in the release pipeline is to leverage the artifacts from the build pipeline. In this case, the YAML file that describes the Kubernetes configuration required to host the ETL image container along with the published build number to identify the correct image that was tagged during the CI stage. The YAML file is updated with build specific values followed by executing KUBECTL commands to push the image into the development Kubernetes cluster (AKS). Once the deployment to development AKS cluster succeeds, a notification is sent indicating a new release is available for review and/or testing. The assigned resource can then pull up an instance of the release and choose to approve/deny the promotion to the production AKS cluster. This concludes the CI/CD cycle until the next check in occurs.
Way to go team! Kevin Herbert Founder, CEO & Chief Architect