Datrics supports data export to databases and object storages.
Export to the database
Create the data source to your object storage in the Datasets tab.
2/ In the pipeline scene, add the Export Data brick and select the data source.
3/ Select the datasource
4/ Define the target table name, and schema if required
5/ Chunk size
5/ Action is the table exist
Export to the object storage
The connector supports export to csv and parquet file formats. We also provide the possibility to export the dataset into multiple parquet files using the data in the partition columns as the key.
Here is how to set up the export brick:
1/ Create the data source to your object storage in the Datasets tab.
2/ In the pipeline scene, add the Export Data brick and select the data source.
3/ Define the file format - csv or parquet
4/ Define the target file path:
- For csv set the full path to the file.
- For parquet there are two options - static or dynamic path. Static path - is the full path to the file Dynamic path consist of: prefix - the static part of the path, partition columns - key defining how to split the dataset into the files. As the result, folders with files will be created: prefix/partition_column=value/uuid.parquet
5/ Action is the file or folder exist