Today, we are excited to release Meltano version 1.48.0, which (among other things) lets you extract a subset of selected entities using new --select
and --exclude
options on meltano elt
, improves plugin invocation and extractor catalog discovery error messages, and changes where meltano elt
logs and generated plugin config files are stored.
Users who were depending on the previous locations of meltano elt
logs ( .meltano/run/elt/<job ID>/<run ID>/elt.log
) and extractor catalog files (.meltano/run/<extractor>/tap.properties.json
) will want to use the new locations instead: .meltano/logs/elt/<job ID>/<run ID>/elt.log
and .meltano/extractors/<extractor>/tap.properties.json
.
Excited to try it out?
To upgrade Meltano and your Meltano project to the latest version, navigate to your project directory, activate the appropriate virtual environment, and run meltano upgrade
. This will upgrade the meltano
package and apply any necessary changes to your project.
What else is new?
The list below (copied from the changelog) covers all of the changes made to Meltano since the release of v1.47.0 on September 3:
New
- #2340 Print full error message when extractor catalog discovery fails
- #2223 Print clear error message when
meltano invoke
ormeltano elt
attempts to execute a plugin that hasn’t been installed yet - #2345 Make
tap-csv
files
a known setting - #2155 Add
select_filter
extractor extra to allow extracting a subset of selected entities - #2155 Add
--select
and--exclude
options tomeltano elt
to allow extracting a subset of selected entities
Changes
- #2167 Include extractor and loader name in autogenerated
meltano elt
job ID - #2343 Automatically delete generated plugin config files at end of
meltano elt
andmeltano invoke
Fixes
- #2167 Make sure autogenerated
meltano elt
job ID matches in system database and.meltano/{run,logs}/elt
- #2347 Have
meltano config <plugin> set --store=dotenv
store valid JSON values for arrays and objects - #2346 Correctly cast environment variable string value when overriding custom array and object settings
Breaks
- #2344 Move
meltano elt
output logs from.meltano/run/elt
to.meltano/logs/elt
- #2342 Store pipeline-specific generated plugin config files (
tap.config.json
,tap.properties.json
, etc) under.meltano/run/elt/<job_id>/<run_id>
instead of.meltano/run/<plugin_name>
. Users who were explicitly putting a catalog file at.meltano/run/<plugin_name>/tap.properties.json
should use.meltano/extractors/<plugin_name>/tap.properties.json
instead.