The target-parquet Singer target sends data into Parquet after it was pulled from a source using a Singer tap.

Alternative variants #

Multiple variants of target-parquet are available. This document describes the default estrategiahq variant, which is recommended for new users.

Alternative variants are:

Standalone usage #

Install the package using pip: 

pip install git+https://github.com/estrategiahq/target-parquet.git

For additional instructions, refer to the README in the repository.

Usage with Meltano #

Meltano helps you manage your configuration, incremental replication, and scheduled pipelines.

View the Meltano-specific target-parquet instructions to learn more.

Capabilities #

target-parquet does not have any capabilities defined in its metadata. Please consider adding them by making a pull request to the YAML file that defines the capabilities for this target.

Settings #

These and other supported settings are documented below. To quickly find the setting you're looking for, use the Table of Contents at the top of the page.

Disable Collection (disable_collection) #

A boolean of whether to disable Singer anonymous tracking.

Logging Level (logging_level) #

(Default - INFO) The log level. Can also be set using environment variables.

Destination Path (destination_path) #

(Default - ‘.’) The path to write files out to.

Compression Method (compression_method) #

Compression methods have to be supported by Pyarrow, and currently the compression modes available are - snappy (recommended), zstd, brotli and gzip.

Streams In Separate Folder (streams_in_separate_folder) #

(Default - False) The option to create each stream in a different folder, as these are expected to come in different schema.

File Size (file_size) #

The number of rows to write per file. The default is to write to a single file.

Looking for help? #

If you're having trouble getting the target-parquet target to work, look for an existing issue in its repository, file a new issue, or join the Meltano Slack community and ask for help in the #plugins-general channel.

Found an issue on this page? #

This page is generated from a YAML file that you can contribute changes to. Edit it on GitHub!