Now Available: Meltano v1.36.1

Today, we are excited to release Meltano version 1.36.1, which (among other things) adds the ability to specify Singer tap stream and property metadata (like replication-method and replication-key) as if it were any other plugin configuration, using special nested config properties metadata.<entity>.<key> and metadata.<entity>.<attribute>.<key>:

meltano config <plugin_name> set metadata <entity> <key> <value>
meltano config <plugin_name> set metadata <entity> <attribute> <key> <value>

# For example:
meltano config tap-postgres set metadata "some_schema-*" replication-method INCREMENTAL
meltano config tap-postgres set metadata "some_schema-*" replication-key created_at

meltano config tap-postgres set metadata some_schema-some_table some_column custom-metadata custom-value

Whenever an extractor is run using meltano elt or meltano invoke, Meltano will generate the desired catalog on the fly by running the tap in discovery mode and applying the selection and metadata rules to the resulting catalog file before passing it to the tap in sync mode.

Excited to try it out?

To upgrade Meltano and your Meltano project to the latest version, navigate to your project directory, activate the appropriate virtual environment, and run meltano upgrade. This will upgrade the meltano package and apply any necessary changes to your project.

What else is new?

The list below (copied from the changelog) covers all of the changes made to Meltano since the release of v1.36.0 on June 15:

New

  • #2067 Add pipeline name, loader, and transform columns to Pipelines table
  • #2103 Support nested objects in meltano.yml config objects
  • #2103 Allow nested properties to be set using meltano config by specifying a list of property names: meltano config <plugin_name> set <property> <subproperty> <value>
  • #2026 Allow Singer stream and property metadata to be configured using special nested config properties metadata.<entity>.<key> and metadata.<entity>.<attribute>.<key>.

Fixes

  • #2102 Fix potential meltano upgrade failures by having it invoke itself with --skip-package after upgrading the package to ensure it always uses the latest code.