Meltano

# JSON Lines (JSONL)

The target-jsonl loader loads extracted data into JSON Lines (JSONL) files.

To learn more about target-jsonl, refer to the repository at https://github.com/andyh1203/target-jsonl.

# Getting Started

# Prerequisites

If you haven't already, follow the initial steps of the Getting Started guide:

  1. Install Meltano
  2. Create your Meltano project
  3. Add an extractor to pull data from a source

# Installation and configuration

# Using the Command Line Interface

  1. Add the target-jsonl loader to your project using meltano add:

    meltano add loader target-jsonl

  2. Configure the settings below using meltano config.

# Using Meltano UI

  1. Start Meltano UI using meltano ui:

    meltano ui

  2. Open the Loaders interface at http://localhost:5000/loaders.

  3. Click the "Add to project" button for "JSON Lines (JSONL)".

  4. Configure the settings below in the "Configuration" interface that opens automatically.

# Next steps

Follow the remaining step of the Getting Started guide:

  1. Run a data integration (EL) pipeline

# Settings

target-jsonl requires the configuration of the following settings:

These and other supported settings are documented below. To quickly find the setting you're looking for, use the Table of Contents in the sidebar.

# Minimal configuration

A minimal configuration of target-jsonl in your meltano.yml project file will look like this:






 
 

plugins:
  loaders:
  - name: target-jsonl
    variant: andyh1203
    pip_url: target-jsonl
    config:
      destination_path: my_jsonl_files

# Destination Path

Sets the destination path the JSONL files are written to, relative to the project root.

The directory needs to exist already, it will not be created automatically.

To write JSONL files to the project root, set an empty string ("").

# How to use

Manage this setting using Meltano UI, meltano config, or an environment variable:

meltano config target-jsonl set destination_path <path>

export TARGET_JSONL_DESTINATION_PATH=<path>

# Do Timestamp File

Specifies if the files should get timestamped.

By default, the resulting file will not have a timestamp in the file name (i.e. exchange_rate.jsonl).

If this option gets set to true, the resulting file will have a timestamp associated with it (i.e. exchange_rate-{timestamp}.jsonl).

# How to use

Manage this setting using Meltano UI, meltano config, or an environment variable:

meltano config target-jsonl set do_timestamp_file true

export TARGET_JSONL_DO_TIMESTAMP_FILE=true