Configuration Reference

SqlSynthGen is configured using a YAML file, which is passed to several commands with the --config-file option. Throughout the docs, we will refer to this file as config.yaml but it can be called anything (the exception being that there will be a naming conflict if you have a vocabulary table called config).

Below, we see the schema for the configuration file. Note that our config file format includes a section of SmartNoise SQL metadata, which is explained more fully here.

SQLSynthGen Config

SQLSynthGen Config

Type: object

A SQLSynthGen configuration YAML file

 No Additional Properties

root use-asyncio
Type: boolean

Run source-statistics queries using asyncpg.

root row_generators_module
Type: string

The name of a local Python module of row generators (excluding .py).

root story_generators_module
Type: string

The name of a local Python module of story generators (excluding .py).

root src-stats
Type: array

An array of source statistics queries.

Each item of this array must be:

root src-stats src-stats items
Type: object
No Additional Properties

root src-stats src-stats items name
Type: string

A name for the query, which will be used in the stats file.

root src-stats src-stats items query
Type: string

A SQL query.

root src-stats src-stats items dp-query
Type: string

A SmartNoise SQL query.

root src-stats src-stats items epsilon
Type: number

The differential privacy epsilon value for the DP query.

root src-stats src-stats items delta
Type: number

The differential privacy delta value for the DP query.

root src-stats src-stats items snsql-metadata
Type: object

See https://docs.smartnoise.org/sql/metadata.html#yaml-format.

All properties whose name matches the following regular expression must respect the following conditions

Property name regular expression: ^(?!(max_ids|row_privacy|sample_max_ids|censor_dims|clamp_counts|clamp_columns|use_dpsu)).*$
root src-stats src-stats items snsql-metadata ^(?!(max_ids|row_privacy|sample_max_ids|censor_dims|clamp_counts|clamp_columns|use_dpsu)).*$
Type: object
No Additional Properties

root story_generators
Type: array of object

An array of story generators.

Each item of this array must be:

root story_generators story_generators items
Type: object
No Additional Properties

root story_generators story_generators items name
Type: string

The full name of a story generator (e.g. mystorygenerators.short_story).

root story_generators story_generators items args
Type: array

Positional arguments to pass to the story generator.

root story_generators story_generators items kwargs
Type: object

Keyword arguments to pass to the story generator.

root story_generators story_generators items num_stories_per_pass
Type: integer

The number of times to call the story generator per pass.

root max-unique-constraint-tries
Type: integer

The maximum number of tries to respect a uniqueness constraint.

root tables
Type: object

Table configurations.

All properties whose name matches the following regular expression must respect the following conditions

Property name regular expression: .*
root tables .*
Type: object

A table configuration.

 No Additional Properties

root tables .* ignore
Type: boolean

Whether to completely ignore this table.

root tables .* vocabulary_table
Type: boolean

Whether to export the table data.

root tables .* num_rows_per_pass
Type: integer

The number of rows to generate per pass.

root tables .* row_generators
Type: array of object

An array of row generators to create column values.

Each item of this array must be:

root tables .* row_generators row_generators items
Type: object

root tables .* row_generators row_generators items name
Type: string

The name of a (built-in or custom) function (e.g. max or myrowgenerators.my_gen).

root tables .* row_generators row_generators items args
Type: array

Positional arguments to pass to the function.

root tables .* row_generators row_generators items kwargs
Type: object

Keyword arguments to pass to the function.

root tables .* row_generators row_generators items columns_assigned
Type: array of string or string

One or more columns to assign the return value to.

Each item of this array must be:

root tables .* row_generators row_generators items columns_assigned columns_assigned items
Type: string