Export all active Matillion jobs, groups, and schedules. For each job, document: name, source, target warehouse, transformations, frequency, and any dependencies. Use Matillion's API to extract metadata. Create a spreadsheet categorizing jobs by complexity (simple cloud ELT vs. complex transformation).
⚠️ Watch Out For:
- Matillion jobs can be organized in groups—ensure you capture all of them
- Nested jobs and dependencies may not be immediately obvious from the UI
Sort Matillion jobs into: (1) Cloud source → warehouse (ideal for Fivetran), (2) Cloud source + transformation (Fivetran + dbt), (3) On-premises or complex (keep in Matillion). Most Matillion users will find 70-80% of jobs fall into categories 1-2.
⚠️ Watch Out For:
- Don't overestimate transformation complexity—some Matillion jobs look complex but are actually simple
- On-premises sources may block migration—identify these early
Sign up for Fivetran Cloud. Create workspace. Set up connections for your data warehouses (Snowflake, BigQuery, Redshift, etc.). Test warehouse connections with sample queries. Configure schema naming conventions if needed.
⚠️ Watch Out For:
- Warehouse credentials must have schema/table creation permissions—verify this early
- Network firewall rules must be updated for Fivetran's IP ranges
Select the simplest Matillion job (single source, single target, minimal transformation). Create the equivalent Fivetran connector. Configure table selection and sync mode (full refresh vs. incremental). Test the sync. Compare outputs with the original Matillion run.
⚠️ Watch Out For:
- Matillion's incremental sync logic may differ from Fivetran's—test thoroughly
- Column ordering and data types may differ—verify sample rows match
For jobs with transformation, create dbt models that build on Fivetran-loaded tables. Rewrite Matillion transformations as dbt SQL. Configure dbt to run after Fivetran syncs (via Fivetran webhooks or external orchestrator). Test dbt output.
⚠️ Watch Out For:
- Matillion's visual job logic requires translation to dbt SQL—may uncover hidden business logic
- Scheduling dependencies between Fivetran and dbt must be carefully orchestrated
Set up Fivetran sync schedules (hourly, daily, weekly). Map Matillion's schedule to Fivetran's options. Configure alerts for failed syncs (email, Slack, webhooks). Set up monitoring dashboards to track sync success rates and latency.
⚠️ Watch Out For:
- Fivetran's schedule granularity may differ from Matillion's—adjust expectations
- Alert thresholds should account for expected variability (weekend vs. weekday traffic)
Progressively migrate remaining Matillion jobs to Fivetran. Start with simple ones (category 1). Move to more complex ones (category 2) once you have confidence. For each job, validate that outputs match the original Matillion run.
⚠️ Watch Out For:
- Later jobs may have hidden complexity—don't assume smooth sailing after first few
- Some jobs may require custom connectors or workarounds—plan alternatives early
Keep both Matillion and Fivetran jobs running in parallel for 1-2 full execution cycles. Compare outputs: record counts, data accuracy, timing. Validate that downstream analytics and dashboards produce identical results with Fivetran data.
⚠️ Watch Out For:
- Timing differences between Matillion and Fivetran can complicate comparison—align schedules temporarily
- Small data mismatches often stem from NULL handling or data type precision—investigate thoroughly
Review Fivetran's consumption costs (MAR—Monthly Active Rows) for the first month. Identify large tables and optimize by adjusting column selection or sync frequency. Configure cost alerts. Update team dashboards and reporting to reflect new load times.
⚠️ Watch Out For:
- First month of Fivetran costs may be higher than expected if large tables are fully synced—review data volume
- Some Fivetran sources have per-connector costs regardless of data volume—plan accordingly
Once Fivetran passes validation, disable Matillion jobs. Keep Matillion running read-only for 2 weeks for reference. Update documentation and team runbooks to reflect Fivetran-based pipelines. Archive Matillion jobs. Decommission Matillion infrastructure.
⚠️ Watch Out For:
- Don't delete Matillion jobs immediately—archive for 2 weeks in case rollback is needed
- Update downstream alerts and SLAs to account for any timing changes in Fivetran schedules
