JSON Lines (JSONL)

The target-jsonl loader loads extracted data into JSON Lines (JSONL) files.

• Repository: https://github.com/andyh1203/target-jsonl
• Maintainer: Andy Huynh
• Maintenance status: Active

Getting Started

Prerequisites

If you haven't already, follow the initial steps of the Getting Started guide:

1. Install Meltano
2. Create your Meltano project
3. Add an extractor to pull data from a source

Installation and configuration

Using the Command Line Interface

1. Add the target-jsonl loader to your project using meltano add:

   meltano add loader target-jsonl
   

2. Configure the settings below using meltano config.

Using Meltano UI

1. Start Meltano UI using meltano ui:

   meltano ui
   

2. Open the Loaders interface at http://localhost:5000/loaders.

3. Click the "Add to project" button for "JSON Lines (JSONL)".

4. Configure the settings below in the "Configuration" interface that opens automatically.

Next steps

Follow the remaining step of the Getting Started guide:

1. Run a data integration (EL) pipeline

If you run into any issues, learn how to get help.

Settings

target-jsonl requires the configuration of the following settings:

• Destination Path

These and other supported settings are documented below. To quickly find the setting you're looking for, use the Table of Contents in the sidebar.

Minimal configuration

A minimal configuration of target-jsonl in your meltano.yml project file will look like this:

plugins:
  loaders:
  - name: target-jsonl
    variant: andyh1203
    config:
      destination_path: my_jsonl_files

Destination Path

• Name: destination_path
• Environment variable: TARGET_JSONL_DESTINATION_PATH
• Default: output

Sets the destination path the JSONL files are written to, relative to the project root.

The directory needs to exist already, it will not be created automatically.

To write JSONL files to the project root, set an empty string ("").

How to use

Manage this setting using Meltano UI, meltano config, or an environment variable:

meltano config target-jsonl set destination_path <path>

export TARGET_JSONL_DESTINATION_PATH=<path>

Do Timestamp File

• Name: do_timestamp_file
• Environment variable: TARGET_JSONL_DO_TIMESTAMP_FILE
• Default: false

Specifies if the files should get timestamped.

By default, the resulting file will not have a timestamp in the file name (i.e. exchange_rate.jsonl).

If this option gets set to true, the resulting file will have a timestamp associated with it (i.e. exchange_rate-{timestamp}.jsonl).

How to use

Manage this setting using Meltano UI, meltano config, or an environment variable:

meltano config target-jsonl set do_timestamp_file true

export TARGET_JSONL_DO_TIMESTAMP_FILE=true