# Containerization

Once you've set up a Meltano project and run some pipelines on your local machine, it'll be time to repeat this trick in production!

The Deployment in Production guide will walk you through getting Meltano, your project, and all of its plugins onto a new environment one-by-one, among other things, but you can greatly simplify this process (and prevent issues caused by inconsistencies between environments!) by wrapping them all up into a project-specific Docker container image: "a lightweight, standalone, executable package of software that includes everything needed to run an application: code, runtime, system tools, system libraries and settings."

This image can then be used on any environment running Docker (or a compatible tool like Kubernetes) to directly run meltano commands in the context of your project, without needing to separately manage the installation of Meltano, your project's plugins, or any of their dependencies.

If you're storing your Meltano project in version control on a platform like GitLab or GitHub, you can set up a CI/CD pipeline to run every time a change is made to your project, which can automatically build a new version of the image and push it to a container registry. The image can then be pulled from that registry onto any local or cloud environment on which you'd like to run your project's pipelines.

If you'd like to containerize your Meltano project, you can easily add the appropriate Dockerfile and .dockerignore files to your project by adding the docker file bundle:

# For these examples to work, ensure that
# Docker has been installed
docker --version

# Add Docker files to your project
meltano add files docker

# Build Docker image containing
# Meltano, your project, and all of its plugins
docker build --tag meltano-demo-project:dev .

# Docker Compose

If you'd like to use Docker Compose to experiment with a production-grade setup of your containerized project, you can add the appropriate docker-compose.prod.yml file to your project by adding the docker-compose file bundle:

# For these examples to work, ensure that
# Docker Compose has been installed
docker-compose --version

# Add Docker Compose files to your project
meltano add files docker-compose

# Start the `meltano-ui` and `meltano-system-db` services in the background
docker-compose -f docker-compose.prod.yml up -d

Meltano UI will now be available at http://localhost:5000.

For more details and instructions, refer to README contained in the file bundle.

# GitLab CI/CD

If you'd like to use GitLab CI/CD to continuously build your Meltano project's Docker image and push it to GitLab's built-in Container Registry, you can add the appropriate .gitlab-ci.yml and .gitlab/ci/docker.gitlab-ci.yml files to your project by adding the gitlab-ci file bundle:

# For these examples to work, ensure that
# you have an account on GitLab.com or
# a self-hosted GitLab instance with
# GitLab CI/CD and Container Registry enabled

# Add GitLab CI/CD files to your project
meltano add files gitlab-ci

# Initialize Git repository, if you haven't already
git init

# Add and commit all files
git add -A
git commit -m "Set up Meltano project with Docker and GitLab CI"

# Push to GitLab, which will automatically create
# a new private project at the specified path
NAMESPACE="<your-gitlab-username-or-group>"
git push git@gitlab.com:$NAMESPACE/meltano-demo-project.git master

GitLab CI/CD will now start building your Meltano project's dedicated Docker image, which will be available at registry.gitlab.com/$NAMESPACE/meltano-demo-project:latest once the CI/CD pipeline completes.