Today we are excited to announce the latest version of Meltano SDK for taps! In this release, we are adding the ability to create Parent-Child stream relationships, which was the most-requested feature by developers using the SDK.
Where previously, tap developers needed to create complex nested loops and error checking, this new SDK feature quickly and easily define relationships between parent and child streams. The SDK automatically takes care of orchestration, error handling, and state tracking. Check out the updated docs for instructions and sample code to get you started.
What else is new?
- Added support for parent-child streams (#97, !79)
- Added support for configurable metrics logging (#91, !79)
- Added ability to use fewer state bookmarks by setting
Stream.state_partitioning_keysto a subset of available context keys (!79)
- Renamed the optional
partitiondictionary arg in method signatures to the more generic
- The methods
Stream.get_stream_or_partition_state()have been deprecated in favor of the new and simpler
- Code coverage is now tracked and available as a tool for SDK contributors to further improve overall stability and help prioritize unit test development. (#39, !89)
Come see the SDK in action
Tomorrow in our fortnightly Demo Day, we’ll be demoing an end-to-end Meltano implementation which utilizes the new Parent-Child streams capability. We’ll also demo an AWS Athena loader built with the Target SDK Preview and our Athena dbt transformation pipeline running natively against S3 files.
Hope to see you soon!