This section provides step-by-step guides for deploying Meltano on various environments. Currently, we provide detailed intructions for:
We are working toward one-click installers, and will update this page as soon as those become available. In the meantime, Meltano can be deployed locally or to the cloud using these instructions.
DigitalOcean provides a simple container for spinning up a server where Meltano can be deployed to the Cloud. You will need to create an account with DigitalOcean, if you don't have one already.
We are actively working on a one-click installer for DigitalOcean (Meltano Issue #926)
From the "Create" dropdown menu in DigitalOcean, choose "Droplets"
For "Choose an Image" select Ubuntu
Choose your plan (the lowest cost "Standard" option is great if you setting this up for the first time) and make any other selections (the defaults are all fine).
Click "Create Droplet" and wait until the progress bar completes. Congratulations, you now have a DigitalOcean Droplet!
Click on the three-dot dropdown menu to the right of your Droplet's name and select "Access console" to launch the web-based command line. You will be prompted to use your login of "root" and the password that was emailed to you (or SSH key, if you chose that option). Once you enter your username root and password (from the email) you will be prompted to change the password to something more secure.
You can also connect to your Droplet using SSH, from the command line:
Your new server will not have any of Meltano's requirements installed by default, so you will need to install them.
Your Ubuntu image will not come with Python 3.6+ set as the sysyem wide version by default, so you will need to complete the following steps.
Get the most updated version of all packages:
And then install Python:
apt install python
Now, if you run
python --version, you will see the system wide version is 2.7.15+ but Meltano requires 3.6+. Now we need to update the system wide version we want to be using with the command:
update-alternatives --install /usr/bin/python python /usr/bin/python3.6 1
To confirm the version is now correct, run
python --version again
python --version #Python 3.6.8
Next up, we need to install our package manager, pip.
apt install python3-pip
Now that you have Python configured system wide and pip installed, we'll use pip to install your virtual environment management tools:
pip3 install virtualenv
And then install it with:
apt-get install python3-venv
You may be tempted to create your DigitalOcean Droplet without a virtual environment, but due to Python-related installation issues and locked down dependencies for Meltano we highly recommend that you take advantage of the virtual environment.
Create a directory where you want your virtual environments to be saved:
Then create a new virtual environment inside that directory:
python -m venv ~/virtualenvs/meltano
Activate the virtual environment using:
Now that you are inside your virtual environment, follow the Meltano installation instructions
meltano init YOUR_PROJECT_NAME
Launch the Meltano UI on your server:
cd YOUR_PROJECT_NAME meltano ui
When you run
meltano ui you will be prompted to view the Meltano UI at http://localhost:5000, however this will not work for DigitalOcean Droplets because they are hosted in the cloud.
Instead, use the IP address of your DigitalOcean Droplet and port 5000.
You can also install PostgreSQL to your DigitalOcean Droplet, and then use that database when you configure the Postgres target. DigitalOcean provides installation instructions here.
This guide assumes that you have a functioning Docker image where your Meltano project is already bundled with the Meltano installation. To track this issue, follow meltano#624.
In this section, we will be going over how you can deploy a Meltano Docker image to AWS.
Configurebutton in the custom card
Updatebutton to finish setting up your container defintion
Editnext to the Task defintion heading
Nextto move to the next step
Nextto move on to the next step
The main configuration here is the Cluster name. We provide a suggestion below, but feel free to name it as you wish.
After you click
Next, you will have the opportunity to review all of the properties that you set. Once you confirm that the settings are correct, click
Create to setup your ECS.
You should now see a page where Amazon prepares the services we configured. There will be spinning icons on the right of each service that will live update as it finished. Once you see everything has setup properly, you're cluster has been successfully deployed!
RUNNINGfor Last Status
The IP address can be mapped to a domain using Route53. We will be writing up a guide on how to do this. You can follow along at meltano#625.
This section is only necessary if you do not have a Security Group that allows for port 5000,5010 inbound.
Once you complete the cluster setup, you should be brought to the detail page for the service. You should be default on a tab called Details with a Network Access section.
Add Rulewith the following properties: