Today, we are excited to release Meltano version 1.65.0, which lets you tweak the size of the buffer that stores records (and other messages) output by the extractor (tap) while they are waiting to be processed by the loader (target), using a new
elt.buffer_size setting with a default value of 10MiB.
The length of a single line of extractor output is limited to half the buffer size, making the default maximum message size 5MiB.
Excited to try it out?
To upgrade Meltano and your Meltano project to the latest version, navigate to your project directory, activate the appropriate virtual environment, and run
meltano upgrade. This will upgrade the
meltano package and apply any necessary changes to your project.
What else is new?
- #2392 Add ‘elt.buffer_size’ setting with default value of 10MiB to let extractor output buffer size and line length limit (maximum message size) be configured as appropriate for the extractor and loader in question.
- #2501 Don’t lose