Skip to content

Configuration

Environment Configuration Overview

Once installed, additional environment configuration can be set up using environment specific yaml files.

In the base folder of the database config (base working directory), create a yml file for each environment.

For example:

deploy_config_dev.yml deploy_config_prod.yml

Referencing a Config File

There are 2 ways to tell Snowflake Deployer which config file to use during execution.

Option 1 - Command Line Parameter

Use the -c parameter when calling 

snowflake-deployer deploy -c "deploy_config_dev.yml"

Option 2 - Environment Var

Use the environment variable DEPLOY_CONFIG_PATH.

export DEPLOY_CONFIG_PATH="deploy_config_dev.yml"

Usage - If both the command line parameter AND environment variable are set, execution will use the command line parameter.

Supported Config Parameters

Parameter
 Description
ENV_DATABASE_PREFIX (String) Database Prefix within environment
  • Default = ""
  • Valid Values = ["A","B"]
ENV_WAREHOUSE_PREFIX (String) Warehoue Prefix within environment
  • Default = ""
  • Valid Values = ["A","B"]
ENV_ROLE_PREFIX (String) Role Prefix within environment
  • Default = ""
  • Valid Values = ["A","B"]
ENV_PROCEDURE_PREFIX (String) Procedure Prefix within environment
  • Default = ""
  • Valid Values = ["A","B"]
ENV_FUNCTION_PREFIX (String) Function Prefix within environment
  • Default = ""
  • Valid Values = ["A","B"]
OBJECT_METADATA_ONLY (Boolean) Flag to support the creation of Tables & Views or only the metadata.
  • Currently deployer does not support creation of the tables and views, only the metadata, tags & grants on an object
  • Default = True
MAX_THREADS (Int) Number of threads the deployer will process in parrellel during deployment. Balance Snowflake warehouse load with python execution environment.
  • Default = 3
DEPLOY_ENV (String) Name of the current environment.
  • Used in conjunction with the DEPLOY_ENV list on an object.
  • If an object should only be deployed to certain environments, set the current environment name here, and then list the environments to deploy to within the DEPLOY_ENV config on each object.
  • Default = None
VARS List({key:value}) A list of key value pairs of variables within an environment.
    Vars can be referenced as Jinja vars in any object config and will take precedence over any values returned from the reverse engineering process.
  • Default = None
HANDLE_OWNERSHIP (String) How to handle the ownership of existing objects.
  • For deployments in existing environments, the deployer can manage existing options. But the DEPLOY_ROLE may not have ownership of the objects to manage. This config tells the deployer how to handle that scenario.
  • Default = ERROR
  • Valid Values = [ERROR,GRANT]
  • GRANT = Grant ownership role of object to DEPLOY_ROLE so the DEPLOY_ROLE inherits ownership capabilities. Be careful of any objects that are object by ACCOUNTADMIN as this tells the deployer to make DEPLOY_ROLE the parent role to ACCOUNTADMIN
  • ERROR = Deployer will error out if it does not have ownership privilege on objects it's managing. These errors will need to be fixed manually by granting ownership to a role that the DEPLOY role is a parent role to.
DEPLOY_DATABASE_NAME (String) Name of the MetaOps Deploy Database.
  • Only needing configuration if modified database in the Snowflake set up script. Else exclude.
  • For multiple environments within a single instance, environments can reference the same deploy db
  • Default = "_DEPLOY"
DEPLOY_ROLE (String) Name of the MetaOps Deploy Role.
  • Only needing configuration if modified fole in the Snowflake set up script. Else exclude.
  • For multiple environments within a single instance, environments can use the same role
  • Default = "INSTANCEADMIN"
IMPORT_DATABASES (List) List of databases to import for subcommand = IMPORT only. If not included, all databases imported. List values can either include or not include the ENV_DATABASE_PREFIX. If not included, then ENV_DATABASE_PREFIX is applied when finding databases to import.
IMPORT_OBJECT_TYPES (List) List of object types to import for subcommand = IMPORT only. If empty list of not included, all object types will be imported. Valid object types:
  • DATABASE
  • FUNCTION
  • MASKING POLICY
  • OBJECT
  • PROCEDURE
  • ROLE
  • ROW ACCESS POLICY
  • SCHEMA
  • TAG
  • TASK
  • WAREHOUSE
CLASSIFY_DATABASES (List) List of databases to classify for subcommand = CLASSIFY only. If not included, all databases classified. List values can either include or not include the ENV_DATABASE_PREFIX. If not included, then ENV_DATABASE_PREFIX is applied when finding databases to classify.
CLASSIFY_TAGS_DB_SCHEMA (String) Database.Schema location of where governance tags to be stored for subcommand = CLASSIFY only. Classify command auto creates necessary tags.
CLASSIFY_IGNORE_TAGS (List) List of tags to signify that column should be ignored from classification for subcommand = CLASSIFY only.
CLASSIFY_MAX_SAMPLE_SIZE (Int) Max sample size of Snowflake's auto-classification for subcommand = CLASSIFY only. This helps reduce processing time of classification. Default = 10000

Example config

deploy_config_dev.yml 

ENV_DATABASE_PREFIX: DEV_
ENV_WAREHOUSE_PREFIX: 
ENV_ROLE_PREFIX: 
ENV_PROCEDURE_PREFIX: 
ENV_FUNCTION_PREFIX: 
OBJECT_METADATA_ONLY: True
MAX_THREADS: 5
HANDLE_OWNERSHIP: ERROR
IMPORT_DATABASES:
- ANALYTICS
- CONTROL
- RAW
- SECURITY
CLASSIFY_DATABASES:
- ANALYTICS
- RAW
DEPLOY_ENV: dev
VARS:
- my_comment: some dev comment
- another_var: some other var in dev