Meltano

# SQLite

The target-sqlite loader loads extracted data into a SQLite database.

To learn more about target-sqlite, refer to the repository at https://gitlab.com/meltano/target-sqlite.

# Getting Started

# Prerequisites

If you haven't already, follow the initial steps of the Getting Started guide:

  1. Install Meltano
  2. Create your Meltano project
  3. Add an extractor to pull data from a source

# Installation and configuration

# Using the Command Line Interface

  1. Add the target-sqlite loader to your project using meltano add:

    meltano add loader target-sqlite

  2. Configure the settings below using meltano config.

# Using Meltano UI

  1. Start Meltano UI using meltano ui:

    meltano ui

  2. Open the Loaders interface at http://localhost:5000/loaders.

  3. Click the "Add to project" button for "SQLite".

  4. Configure the settings below in the "Configuration" interface that opens automatically.

# Next steps

Follow the remaining step of the Getting Started guide:

  1. Run a data integration (EL) pipeline

# Settings

target-sqlite requires the configuration of the following settings:

These and other supported settings are documented below. To quickly find the setting you're looking for, use the Table of Contents in the sidebar.

# Minimal configuration

A minimal configuration of target-sqlite in your meltano.yml project file will look like this:






 
 

plugins:
  loaders:
  - name: target-sqlite
    variant: meltano
    pip_url: git+https://gitlab.com/meltano/target-sqlite.git
    config:
      database: my_database.db

# Database

  • Name: database
  • Environment variable: TARGET_SQLITE_DATABASE, alias: SQLITE_DATABASE
  • Default: warehouse

Name of the SQLite database file to be used or created, relative to the project root.

The .db extension is optional and will be added automatically when omitted.

# How to use

Manage this setting using Meltano UI, meltano config, or an environment variable:

meltano config target-sqlite set database <database>

export TARGET_SQLITE_DATABASE=<database>

# Batch Size

How many records are sent to SQLite at a time?

# How to use

Manage this setting using Meltano UI, meltano config, or an environment variable:

meltano config target-sqlite set batch_size 100

export TARGET_SQLITE_BATCH_SIZE=100

# Timestamp Column

Name of the column used for recording the timestamp when Data are loaded to SQLite.

# How to use

Manage this setting using Meltano UI, meltano config, or an environment variable:

meltano config target-sqlite set timestamp_column <column>

export TARGET_SQLITE_TIMESTAMP_COLUMN=<column>