|
Data Warehouse & ETL Tools |
We have over 130 years of combined experience building and deploying data warehouse, ETL, business intelligence, artificial intelligence, and data analytics solutions. We have leveraged that experience to develop innovative, cutting-edge tools to help ensure the solutions we build and deploy drive value for our customers and are efficiently and effectively developed and managed.
Below are some of our data transfer and integration products and tools, with their features:
Below are some of our data transfer and integration products and tools, with their features:
|
IOI Metadata-driven ETL Tool ™
Contact us to learn more
|
IOI Route Manager ™
|
Here is more information on the ETL Tool:
The CI/CD for the ETL engine starts with the creation of a DevOps project in Azure. This project provides the building blocks (version control, build pipelines, release pipelines, etc.) necessary to automate the continuous delivery of extract, transform, load (ETL) engine releases into the different staging areas up through production where customers interact with the final product.
The continuous integration (CI) portion of our software development life cycle (SDLC) starts with a build pipeline that monitors for changes in version control. As changes are detected, an instance of the build pipeline gathers up all source control artifacts and begins building a cross-platform docker-based image of the application and all required dependencies. The pipeline then tags the image with a unique build number and pushes the image to an Azure container registry. The final step of the build pipeline is to publish deployment artifacts that will be needed in the next stage (continuous delivery).
The continuous delivery (CD) stage, involves the creation of a release pipeline. The release pipeline listens for new builds as well as changes to the container registry to initiate a new release. The release pipeline provides the mechanism to promote code from one environment to the next with optional approval processes and test feedback loops. The first step in the release pipeline is to leverage the artifacts from the build pipeline. In this case, the YAML file that describes the Kubernetes configuration required to host the ETL image container along with the published build number to identify the correct image that was tagged during the CI stage. The YAML file is updated with build specific values followed by executing KUBECTL commands to push the image into the development Kubernetes cluster (AKS). Once the deployment to development AKS cluster succeeds, a notification is sent indicating a new release is available for review and/or testing. The assigned resource can then pull up an instance of the release and choose to approve/deny the promotion to the production AKS cluster. This concludes the CI/CD cycle until the next check in occurs.
The CI/CD for the ETL engine starts with the creation of a DevOps project in Azure. This project provides the building blocks (version control, build pipelines, release pipelines, etc.) necessary to automate the continuous delivery of extract, transform, load (ETL) engine releases into the different staging areas up through production where customers interact with the final product.
The continuous integration (CI) portion of our software development life cycle (SDLC) starts with a build pipeline that monitors for changes in version control. As changes are detected, an instance of the build pipeline gathers up all source control artifacts and begins building a cross-platform docker-based image of the application and all required dependencies. The pipeline then tags the image with a unique build number and pushes the image to an Azure container registry. The final step of the build pipeline is to publish deployment artifacts that will be needed in the next stage (continuous delivery).
The continuous delivery (CD) stage, involves the creation of a release pipeline. The release pipeline listens for new builds as well as changes to the container registry to initiate a new release. The release pipeline provides the mechanism to promote code from one environment to the next with optional approval processes and test feedback loops. The first step in the release pipeline is to leverage the artifacts from the build pipeline. In this case, the YAML file that describes the Kubernetes configuration required to host the ETL image container along with the published build number to identify the correct image that was tagged during the CI stage. The YAML file is updated with build specific values followed by executing KUBECTL commands to push the image into the development Kubernetes cluster (AKS). Once the deployment to development AKS cluster succeeds, a notification is sent indicating a new release is available for review and/or testing. The assigned resource can then pull up an instance of the release and choose to approve/deny the promotion to the production AKS cluster. This concludes the CI/CD cycle until the next check in occurs.
