Skip to main content

Documentation Index

Fetch the complete documentation index at: https://import-docs.prequel.co/llms.txt

Use this file to discover all available pages before exploring further.

This guide walks through the order in which to create resources for a complete Prequel Import pipeline. Each step links to the corresponding API reference page and includes an expandable curl example. If you would prefer an agent to drive the setup conversationally, install the Prequel Import agent skill. Open the raw skill file, copy the contents, and save the file to your agent’s skills directory.

​
Prerequisites

1

Access your Prequel API key

You will need a Prequel API key. If you do not have one, contact your Prequel representative to get access. The API key used in the X-API-Key header will route your request to either production or staging based on the key.
export PREQUEL_API_KEY="<your-key>"
2

Stand up a delivery target

Stand up a spec-compliant API endpoint to receive delivered records.
3

Verify Prequel API access

Confirm your key works against the API. A 200 response means your environment is ready.
curl -H "X-API-Key: $PREQUEL_API_KEY" https://api.prequel.co/

​
Setup overview

Prequel Import should follow a three phase setup to prevent errors from missing dependencies:
  1. Prepare your environment by creating a Datalake. The Datalake is a one-time setup shared across all Providers.
  2. Configure and validate a data source by creating a Provider, Source, and Dataset, then triggering an Extract to confirm the Source, Dataset, and Datalake are configured correctly before adding delivery.
  3. Connect and deliver by standing up a delivery endpoint, creating a Destination (and optionally a custom Product), creating a Stream, testing the transform, and triggering a Load.

​
Set up your pipeline

Each call returns a resource ID in the response body. Record the IDs as you go, since later steps reference them with placeholders such as <DATALAKE_ID> and <DESTINATION_ID>.

​
Phase 1: Prepare your environment

1

Create a Datalake

Provision a bucket on S3, GCS, Azure Blob Storage, or an S3-compatible store to back the Datalake, then create the Datalake referencing that bucket. Typically, a single multi-tenant Datalake is created and reused across Providers, since Prequel automatically partitions the bucket internally by Provider. You may create and assign additional Datalakes if you prefer bucket-level isolation for Providers. See Datalake for configuration steps and open the API reference for the full request schema.
curl -X POST https://api.prequel.co/import/datalakes \
  -H "X-API-Key: $PREQUEL_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "datalake": {
      "name": "primary-datalake",
      "storage": "s3",
      "s3": {
        "bucket_name": "<your-bucket>",
        "bucket_region": "us-east-1",
        "prefix": "prequel-import",
        "auth_method": "aws_iam_role",
        "aws_iam_role": {
          "aws_iam_role_arn": "<your-role-arn>"
        }
      }
    }
  }'

​
Phase 2: Configure and validate a data source

Repeat this phase for each Provider you onboard. Each Provider can have multiple Sources and multiple Datasets.
1

Create a Provider

A Provider represents a customer or tenant of your platform. The Provider references the Datalake by ID and the assigned Products. Pass the special ["all"] Product to grant access to every Destination when customers share all Destinations. If a customer should only have access to a subset of Destinations, use Products to restrict Destinations by Provider. See Providers, or open the API reference for the full request schema.
curl -X POST https://api.prequel.co/import/providers \
  -H "X-API-Key: $PREQUEL_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "provider": {
      "name": "Acme Corp",
      "id": "acme",
      "datalake_id": "<DATALAKE_ID>",
      "products": ["all"]
    }
  }'
2

Create a Source

A Source is the Provider’s database Prequel will read from. See Sources for the supported vendors and per-vendor fields and open the API reference for the full request schema.
curl -X POST https://api.prequel.co/import/sources \
  -H "X-API-Key: $PREQUEL_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "source": {
      "name": "acme-postgres",
      "vendor": "postgres",
      "provider_id": "<PROVIDER_ID>",
      "postgres": {
        "host": "db.acme.example.com",
        "port": 5432,
        "database": "production",
        "auth_method": "password_auth",
        "password_auth": {
          "username": "prequel",
          "password": "<password>"
        }
      }
    }
  }'
3

Create a Dataset

A Dataset represents a structured table of data that Prequel will replicate from the Source. The Dataset declares its extraction method (table for a single table, sql for a custom query, glob for object-storage paths) and certain special columns and configuration options. This step is repeated for each Provider’s table to be imported. See Datasets and Change Detection for incremental sync configuration, or open the API reference for the full request schema.
curl -X POST https://api.prequel.co/import/datasets \
  -H "X-API-Key: $PREQUEL_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "dataset": {
      "source_id": "<SOURCE_ID>",
      "method": "table",
      "table": {
        "source_table_namespace": "public",
        "source_table_name": "users"
      },
      "type": "dimension",
      "dimension": {
        "primary_key_column": "id",
        "last_modified_column": "updated_at"
      },
      "frequency_minutes": 60
    }
  }'
4

Trigger an Extract to validate

The Extract pulls Dataset rows into the Datalake. Trigger one now to confirm the Source, Dataset, and Datalake are configured correctly before moving on to delivery. A successful Extract verifies authentication to the Source, read access to the Dataset’s table, and write access to the Datalake. Open the API reference for the full request schema.
curl -X POST https://api.prequel.co/import/datasets/<DATASET_ID>/extract \
  -H "X-API-Key: $PREQUEL_API_KEY"

​
Phase 3: Connect and deliver

Repeat for each Dataset/Destination pair.
1

Create a Destination

A Destination is one of your delivery endpoints. Each Destination is configured with a record_schema that describes the records your endpoint expects and a request_template that defines the URL and auth headers. See Destinations and the Batch API specification, or open the API reference for the full request schema.Repeat this step for each Destination you need.
Confirm your delivery endpoint from the prerequisites is reachable. Prequel validates connectivity during creation.
curl -X POST https://api.prequel.co/import/destinations \
  -H "X-API-Key: $PREQUEL_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "destination": {
      "name": "primary-destination",
      "type": "webhook_record",
      "record_schema": { "type": "object", "required": ["id"], "properties": { "id": { "type": "string" } } },
      "webhook_record": {
        "request_template": {
          "uri": "https://your-app.example.com/prequel/records",
          "method": "POST"
        }
      }
    }
  }'
2

Create a custom Product (optional)

By default, every Destination you create is included in a built-in all Product. Skip this step and reference all when you create a Provider, unless you want to group Destinations differently. To define a custom group, create a Product and include the Destination IDs you want it to cover. Open the API reference for the full request schema.
curl -X POST https://api.prequel.co/import/products \
  -H "X-API-Key: $PREQUEL_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "product": {
      "name": "primary-product",
      "destinations": ["<DESTINATION_ID>"]
    }
  }'
3

Create a Stream

A Stream maps one Dataset to one Destination through field-level mappings. Before authoring the mappings, fetch the Destination’s record_schema so you know which fields your mappings need to cover. See Streams and Advanced Table Mapping, or open the API reference for the full request schema.
curl -X POST https://api.prequel.co/import/streams \
  -H "X-API-Key: $PREQUEL_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "stream": {
      "name": "users-stream",
      "dataset_id": "<DATASET_ID>",
      "destination_id": "<DESTINATION_ID>",
      "mappings": [
        { "target_field": "id", "field_type": "field", "column": { "name": "id" } }
      ]
    }
  }'
4

Test the transform

Apply the Stream’s mappings to a sample of extracted records and confirm the output covers every required field in the Destination’s record_schema with the correct type. Open the API reference for the full request schema.
curl -X POST https://api.prequel.co/import/streams/<STREAM_ID>/test-transform \
  -H "X-API-Key: $PREQUEL_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{ "record_ids": ["<RECORD_ID>"] }'
5

Trigger a Load

The Load applies the Stream’s mappings and delivers records to the Destination. Open the API reference for the full request schema. Then, verify records have reached your delivery endpoint.
curl -X POST https://api.prequel.co/import/streams/<STREAM_ID>/load \
  -H "X-API-Key: $PREQUEL_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{}'
That’s it. Your Prequel Import pipeline is now configured to automatically extract data from the Source and load to the Destination.
To onboard each additional Provider, repeat Phase 2: Configure and validate a data source and Phase 3: Connect and deliver.

​
Next steps

Building Your Customer Experience

Wrap the API in an embedded setup and monitoring UX for your customers.

Webhooks and Monitoring

Get notified on transfer failures and track sync health.

Change Detection

Tune incremental sync behavior for dimension Datasets.

Advanced Table Mapping

Use expressions and complex mappings to reshape source data.