napkin-runtime
Safe HaskellNone
LanguageGHC2024

Napkin.Spec.Yaml.Preprocessors.LogicalDataset

Synopsis

Documentation

preprocessor :: PreprocessorForYaml b #

Rewrites logical schemas (Postgres) or datasets (BigQuery) into "physical" to facilitate multiple environments.

Both datasets and default_dataset can be overridden with meta arguments.

YAML

logical_dataset:
  default_dataset: someproject.somedataset # default, it the table was referenced without dataset
  datasets:
    input: someotherproject.raw_analytics_data
    derived: production_analytics
    billing: someotherproject.billing
allow_unknown: false # will fail when a table from unknown dataset is referenced to prevent typos (default)