On this page
Setting it up
- In HailMate: Settings → Export data → Scheduled → Your storage → Add storage (owners and admins).
- Amazon S3, Google Cloud Storage (an HMAC key), Cloudflare R2, Backblaze B2, Wasabi, DigitalOcean Spaces, or any other S3-compatible storage. The key needs to write to the bucket (
s3:PutObject) and, to tidy a day that ran twice, to read and delete in its folder. - We write a small check file (
<folder>/_hailmate_check.txt) before saving, and keep the secret where only our server reads it. - Then pick the storage on a scheduled export. CSV of all time is a full delivery.
The layout
The layout never changes under you: point a load job at it once.
Text
<folder>/schedule-<id>/<table>/dt=<YYYY-MM-DD>/<table>-<run>-<NNN>.csv
<folder>/schedule-<id>/_manifests/dt=<YYYY-MM-DD>.json
<folder>/schedule-<id>/_manifests/latest.json| Part | What it is |
|---|---|
schedule-<id> | Each scheduled export has its own folder, named by its id: renaming the schedule never moves it, and two schedules (one per location) never write over each other. |
<table> | One folder per table, its name in lower case with dashes: jobs, payments, time-in-stage. |
dt=<YYYY-MM-DD> | Your company’s day the run was made: a date partition BigQuery, Athena, Snowflake and Databricks read as one. |
<run>-<NNN> | The run, then the file’s number: a big table comes in several files, each with the header row. |
One run per day
A day’s folder holds exactly one run. When a schedule runs twice in a day (Send now, or a retry), the second run writes its own files, then removes the first run’s, after its manifest is written. A load that reads a day’s folder never reads it twice.
The manifest
Each run writes what it wrote: every file, its rows, its size and its columns. Read latest.json (or the dated one) rather than listing the bucket.
_manifests/dt=2026-11-19.json
{
"schedule": "Nightly — every table",
"schedule_id": "3f2a9c1e-77aa-4b2b-9e00-123456789abc",
"run_id": "9e1b44d0-5a6b-4c7d-8e9f-0a1b2c3d4e5f",
"made_at": "2026-11-19T07:02:11Z",
"day": "2026-11-19",
"format": "csv",
"files": [
{
"table": "Jobs",
"key": "hailmate/schedule-3f2a9c1e/jobs/dt=2026-11-19/jobs-9e1b44d0-001.csv",
"part": 1,
"rows": 6652,
"bytes": 4183022,
"columns": [
"Job",
"Job number",
"Location",
"Stage",
"…"
]
},
{
"table": "Payments",
"key": "hailmate/schedule-3f2a9c1e/payments/dt=2026-11-19/payments-9e1b44d0-001.csv",
"part": 1,
"rows": 6677,
"bytes": 1920443,
"columns": [
"Paid by",
"Amount",
"…"
]
}
]
}Loading it
BigQuery
Shell
bq load --source_format=CSV --skip_leading_rows=1 --autodetect \
--hive_partitioning_mode=AUTO \
--hive_partitioning_source_uri_prefix=gs://acme-exports/hailmate/schedule-3f2a9c1e/jobs \
hailmate.jobs "gs://acme-exports/hailmate/schedule-3f2a9c1e/jobs/dt=2026-11-19/*.csv"Snowflake
Text
COPY INTO hailmate.jobs
FROM @hailmate_stage/schedule-3f2a9c1e/jobs/dt=2026-11-19/
FILE_FORMAT = (TYPE = CSV SKIP_HEADER = 1 FIELD_OPTIONALLY_ENCLOSED_BY = '"');Power BI and Excel
Load the files into your warehouse and point Power BI at it, or, for one report that refreshes by itself, use a live link (Getting all of it).
Keep reading
OverviewKeys, your first request, pagination, idempotency, limits and errors.API referenceEvery endpoint and every object, with cURL, Node.js and Python.WebhooksThirty events, filters, signed deliveries and retries.Zapier, Make & n8nConnect HailMate to thousands of apps today with built-in webhook and HTTP steps.Claude, ChatGPT & GeminiThe HailMate MCP server: let an AI assistant work your jobs, contacts, estimates and messages.ActionsEvery change an API key or an AI assistant can make, with its fields and an example.Data dictionaryEvery kind of record and every field, and how to keep your own copy current.ChangelogWhat changed in the API, and when.