The target-yaml Meltano loader sends data into YAML after it was pulled from a source using an extractor.

Getting Started #

Prerequisites #

If you haven't already, follow the initial steps of the Getting Started guide:

  1. Install Meltano
  2. Create your Meltano project
  3. Add an extractor to pull data from a source

Installation and configuration #

Using the Command Line Interface #

  1. Add the target-yaml loader to your project using meltano add : 

    meltano add loader target-yaml
  2. Configure the settings below using meltano config .

Next steps #

Follow the remaining steps of the Getting Started guide:

  1. Run a data integration (EL) pipeline

If you run into any issues, learn how to get help.

Capabilities #

Settings #

These and other supported settings are documented below. To quickly find the setting you're looking for, use the Table of Contents at the top of the page.

File Naming Scheme (file_naming_scheme) #

The scheme with which output files will be named. Naming scheme may leverage any of the following substitutions - \n\n- {stream_name}- {datestamp}- {timestamp}

How to use #

Manage this setting using meltano config or an environment variable: 

meltano config target-yaml set file_naming_scheme <file_naming_scheme>

export TARGET_YAML_FILE_NAMING_SCHEME=<file_naming_scheme>

Datestamp Format (datestamp_format) #

A python format string to use when outputting the {datestamp} string. For reference, see - https://docs.python.org/3/library/datetime.html#strftime-and-strptime-format-codes

How to use #

Manage this setting using meltano config or an environment variable: 

meltano config target-yaml set datestamp_format <datestamp_format>

export TARGET_YAML_DATESTAMP_FORMAT=<datestamp_format>

Timestamp Format (timestamp_format) #

A python format string to use when outputting the {datestamp} string. For reference, see - https://docs.python.org/3/library/datetime.html#strftime-and-strptime-format-codes

How to use #

Manage this setting using meltano config or an environment variable: 

meltano config target-yaml set timestamp_format <timestamp_format>

export TARGET_YAML_TIMESTAMP_FORMAT=<timestamp_format>

Timestamp Timezone (timestamp_timezone) #

The timezone code or name to use when generating {timestamp} and {datestamp}. Defaults to “UTC”. For a list of possible values, please see - https://en.wikipedia.org/wiki/List_of_tz_database_time_zones

How to use #

Manage this setting using meltano config or an environment variable: 

meltano config target-yaml set timestamp_timezone <timestamp_timezone>

export TARGET_YAML_TIMESTAMP_TIMEZONE=<timestamp_timezone>

Stream Maps (stream_maps) #

Allows inline stream transformations and aliasing. For more information see - https://sdk.meltano.com/en/latest/stream_maps.html

How to use #

Manage this setting using meltano config or an environment variable: 

meltano config target-yaml set stream_maps '{...}'

export TARGET_YAML_STREAM_MAPS='{...}'

Record Insert Jsonpath (record_insert_jsonpath) #

A jsonpath string determining the insertion point for new records. Currently, this must be the path to a map key which will be populated by a list of records. \n\nFor example “$.metrics” will populate the file with metrics - [{<record1>},{<record2>},...] \n\nFor JSONPath syntax help, see - https://jsonpath.com

How to use #

Manage this setting using meltano config or an environment variable: 

meltano config target-yaml set record_insert_jsonpath <record_insert_jsonpath>

export TARGET_YAML_RECORD_INSERT_JSONPATH=<record_insert_jsonpath>

Record Key Property Name (record_key_property_name) #

A property in the record which will be used as the dictionary key.\n\nIf this property is provided, records will be written as key-value objects; if omitted, records are written as a list.

How to use #

Manage this setting using meltano config or an environment variable: 

meltano config target-yaml set record_key_property_name <record_key_property_name>

export TARGET_YAML_RECORD_KEY_PROPERTY_NAME=<record_key_property_name>

Record Sort Property Name (record_sort_property_name) #

A property in the record which will be used as a sort key.\n\nIf this property is omitted, records will not be sorted.

How to use #

Manage this setting using meltano config or an environment variable: 

meltano config target-yaml set record_sort_property_name <record_sort_property_name>

export TARGET_YAML_RECORD_SORT_PROPERTY_NAME=<record_sort_property_name>

Overwrite Behavior (overwrite_behavior) #

Determines the overwrite behavior if destination file already exists. Must be one of the following string values - \n\n- append_records (default) - append records at the insertion point\n- replace_records - replace all records at the insertion point\n- replace_file - replace entire file using default_yaml_template\n

How to use #

Manage this setting using meltano config or an environment variable: 

meltano config target-yaml set overwrite_behavior <overwrite_behavior>

export TARGET_YAML_OVERWRITE_BEHAVIOR=<overwrite_behavior>

Default Yaml Template (default_yaml_template) #

Text string to use for a yaml template file. This text will be used to create a new file if the destination file does not exist.

How to use #

Manage this setting using meltano config or an environment variable: 

meltano config target-yaml set default_yaml_template <default_yaml_template>

export TARGET_YAML_DEFAULT_YAML_TEMPLATE=<default_yaml_template>

Looking for help? #

If you're having trouble getting the target-yaml loader to work, look for an existing issue in its repository, file a new issue, or join the Meltano Slack community and ask for help in the #plugins-general channel.

Found an issue on this page? #

This page is generated from a YAML file that you can contribute changes to. Edit it on GitHub!