Snowflake
Snowflake is a data warehousing platform, its unique architecture provides complete relational database support for any structural data formats. A data platform that’s built from the ground up for the cloud. It’s designed with a patented new architecture to handle all aspects of data and analytics.
Sprinkle data now integrates with Snowflake’s warehouse to support unstructured data formats like CSV and semi-structured data formats such as JSON all within a single, logically integrated solution by syncing the varied data into Snowflake.
The seamlessness in integrating with Snowflake warehouse and the ability to update to the changing schemas of it is what sets Sprinkle apart.
How to configure snowflake data warehouse in Sprinkle?
Admin -> Drivers
Admin tab consists of a number of subcategories of which Driver is one. The drivers tab routes the user to Storage, Warehouse and Compute tabs.
On clicking the Create new warehouse, the pop up displays a number of data warehouses. In this case, Snowflake is selected.
This routes the user to a page where the user is required to fill in the unique credentials. On clicking Create, a new data warehouse is created.
Creating stage in snowflake
Snowflake supports two types of stages for storing data files used for loading/unloading: Internal stages store the files internally within Snowflake. External stages store the files in an external location (i.e. S3 bucket) that is referenced by the stage.
How to create a stage?
Internal storage
CREATE [ OR REPLACE ] [ TEMPORARY ] STAGE [ IF NOT EXISTS ] <internal_stage_name>
[ FILE_FORMAT = ( { FORMAT_NAME = '<file_format_name>' | TYPE = { CSV | JSON | AVRO | ORC | PARQUET | XML } [ formatTypeOptions ] ) } ]
[ COPY_OPTIONS = ( copyOptions ) ]
[ COMMENT = '<string_literal>' ]
External stage
CREATE [ OR REPLACE ] [ TEMPORARY ] STAGE [ IF NOT EXISTS ] <external_stage_name>
externalStageParams
[ FILE_FORMAT = ( { FORMAT_NAME = '<file_format_name>' | TYPE = { CSV | JSON | AVRO | ORC | PARQUET | XML } [ formatTypeOptions ] ) } ]
[ COPY_OPTIONS = ( copyOptions ) ]
[ COMMENT = '<string_literal>' ]
In Sprinkle we create CSV data sources and csv files are stored in the storage using stage. Below is the query used to create the S3 stage. In the same way you can create for other storages.
_create or replace stage DB.SCHEMA.STAGE_NAME url='s3://bucket_name_
_credentials=(aws_key_id='XXXXXXXXXXXXXXXX' aws_secret_key='XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX')_
_ FILE_FORMAT = (TYPE = CSV );_