Types of Datasets
For efficiency, Prequel Import requires each Dataset to be assigned a type: Dimension or Fact.- Dimension: A table containing rows that may be updated over time (e.g., users, products, accounts). Prequel Import tracks changes and syncs updates to existing records.
- Fact: An append-only table containing immutable events or transactions (e.g., events, page views, logs). New records are added but existing records are never modified. Prequel Import assumes records that have been successfully delivered to the Destination do not need to be monitored for changes.
Dataset configuration options
| Field | Type | Required | Description |
|---|---|---|---|
| source_id | uuid | Yes | The ID of the Source this Dataset is connected to. |
| source_object_name | string | Yes | The fully qualified name of the table or object in the Source (e.g., schema.table_name). |
| cache_table_name | string | Yes | The name of the table in the Datalake where this Dataset is cached. |
| type | string | Yes | The type of Dataset. Supported values: dimension, fact. |
| primary_key_column | string | Yes | The column used to uniquely identify records in this Dataset. |
| last_modified_column | string | No | The column used to track when records were last modified. Used for incremental sync. Set to null for full sync. |