Documentation

csv importing options

Define table with external shell script or command with arguments

external:
  shell: ./import_table.sh --from "{{{db_url.raw}}}" --table "{{{table.db}}}"

or > external: > command: ./import_table.sh > arguments: > - --from "{{{db_url.raw}}}" > - --table "{{{table.db}}}"

Define table with SQL query specified directly in YAML

sql_query:
  query: SELECT * FROM foo

Define table with SQL query specified directly in YAML. vars is an optional object used to specify variables used in mustache interpolation.

sql_file:
  source: some_file.sql
  vars:
    agg: sum

Define table with Explicit DDL to create schema and (optionally) SQL Insert Statements to add data, with ExtendedStatement specified directly in YAML. vars is an optional object used to specify variables used in mustache interpolation. The table will always be recreated if update strategy calls for it, and any existing data on table will be lost.

sql_file_with_schema:
  source: some_file_extended.sql
  vars:
    agg: sum

Insert into a table (or create if non-existent) with SQL query specified directly in YAML. The data is appended to the table, and the table is never truncated. vars is an optional object used to specify variables used in mustache interpolation.

sql_file_insert:
  source: some_file.sql
  vars:
    agg: sum

Insert into a table (or create if non-existent) with explicit DDL to create schema and SQL Insert Statements to add data, with ExtendedStatement specified directly in YAML. The data is appended to the table, and the table is never truncated. vars is an optional object used to specify variables used in mustache interpolation.

sql_file_with_schema_insert:
  source: some_file.sql
  vars:
    agg: sum

Create a table from yaml schema specification and load content from a given CSV file. source CSV file path type can be int, double, text, day, datetime header CSV header is present in source file columns ordered (as in CSV source file columns) list of column name-type association create create table by default is true chunks optional chunks size in which source file is loaded csv_separator optional character providing the csv separator character (',' by default) csv_quote_char optional character providing quotes character ('"' by default)

csv_file:
  source: input.csv
  chunks: 50
  header: true
  create: false
  csv_separator: ','
  csv_quote_char: '"'
  columns:
    - field1: type1
    ..
    - fieldn: typen

Copy a table

copy:
  table: table_to_be_copied
  if_exists: append