Today, we are excited to release Meltano version 1.65.0, which lets you tweak the size of the buffer that stores records (and other messages) output by the extractor (tap) while they are waiting to be processed by the loader (target), using a new elt.buffer_size setting with a default value of 10MiB.
The length of a single line of extractor output is limited to half the buffer size, making the default maximum message size 5MiB.
Excited to try it out?
To upgrade Meltano and your Meltano project to the latest version, navigate to your project directory, activate the appropriate virtual environment, and run meltano upgrade. This will upgrade the meltano package and apply any necessary changes to your project.
What else is new?
The list below (copied from the changelog) covers all of the changes made to Meltano since the release of v1.64.0 on January 7:
New
- #2392 Add ‘elt.buffer_size’ setting with default value of 10MiB to let extractor output buffer size and line length limit (maximum message size) be configured as appropriate for the extractor and loader in question.
Fixes
- #2501 Don’t lose
versionwhen cachingdiscovery.yml.