Data ingestion
Read from anywhere your data lives.
SQL databases
Read from PostgreSQL, SQLite, MS SQL, and other JDBC/ODBC sources via standard drivers, with schema discovery and incremental sync.
REST & third-party APIs
Authenticated pulls from external APIs with pagination, retry/backoff, schema mapping, and credential management.
Files & folder drops
Excel, CSV, and JSON ingestion from network shares, SFTP, and watched folder drops, with archival and de-duplication.
Email & webhook intake
Pull attachments from mailboxes and consume webhook events from third-party platforms, normalised into consistent rows.
Transformation & validation
Make data trustworthy before it reaches the warehouse.
Normalisation
Standardise column names, datatypes, date formats, and codes across heterogeneous sources into one consistent schema.
Quality gates
Required-column checks, null-rate thresholds, regex patterns, range checks, and pluggable custom rules.
Storage
Two warehouses, deliberately split.
Cloud SQL warehouse
PostgreSQL as the authoritative store, with append, replace, and upsert load strategies. Powers BI dashboards directly.
Local SQL bridge
Credential-free SQLite mirror so spreadsheets can link to live data without embedding cloud secrets in shared files.
Reporting & exports
Outputs in every format your stakeholders need.
Excel reports
Refresh model-linked workbooks and produce branded, formatted Excel files ready for distribution.
CSV / JSON exports
Generate machine-readable outputs for downstream pipelines, automation triggers, and partner deliveries.
Email delivery
Templated HTML emails with attachments, recipient management, and run-tracking for operational distribution.
Scheduled notifications
Operational alerts when feeds fail, datasets go stale, or thresholds break โ before stakeholders ask.
Integrations & push
Send data outward, not just inward.
Power BI refresh
Trigger dataset refreshes via gateway/ODBC integration as soon as new data lands โ no manual button-clicking.
Power Automate flows
Fire webhook events into Power Automate to drive cross-system business workflows from your data layer.
Push to APIs
Send computed datasets back to internal services or third-party APIs with auth and error handling built in.
CLI & GUI runners
Run any pipeline from the command line for power users, or a one-click GUI for non-technical stakeholders.