Skip to content
Outcomes · Case study

Synapse toDatabricks.

An insurer ran claims and policy reporting on a Synapse dedicated pool with ADF pipelines around it and a retirement date approaching. We moved the estate to Databricks with Lakebridge: the analyzer inventoried it, the converter translated the T-SQL for engineers to review, and each table passed reconciliation checks before its pool copy was switched off.

01 / The situation

Before and after

Before

A pool with a retirement date

Claims and policy marts in a Synapse dedicated pool, loaded by ADF pipelines and a few notebooks; stored procedures nobody wanted to touch; Power BI refreshing from all of it every night. The pool was billed at rest and slow at month end.

After

Databricks, proven table by table

Delta tables in Unity Catalog, procedures translated to Databricks SQL and reviewed, ADF orchestration rebuilt as Lakeflow Jobs, the semantic models repointed to a serverless SQL warehouse. Each object was reconciled against the pool across a full reporting cycle before the pool lost it.

02 / Architecture

The platform

Claims and policies, refined into trusted marts.

The logical target runtime retains raw inputs in Bronze, validates them in Silver and publishes reporting marts in Gold. Lakeflow runs the reviewed transformations.

Data flowControlSelect a stage to inspect
Full architecture
The logical target runtime retains raw inputs in Bronze, validates them in Silver and publishes reporting marts in Gold. Lakeflow runs the reviewed transformations.
Silver layer · persisted data

Validated data

04 / 09

Silver represents the validated claims and policy tables. Lakeflow Declarative Pipelines apply the reviewed transformations and quality expectations before these tables feed the reporting models.

  • Delta Lake
  • Lakeflow Declarative Pipelines
  • Expectations
Data in
Bronze · Raw claims
Data out
Gold · Reporting marts
Control in
Lakeflow pipelines

Bronze, Silver and Gold describe the logical data layers in the target estate. Unity Catalog governs the Delta tables; the migration path separately shows how each object was approved for cutover.

03 / The delivery

Inside the delivery

Assessment with Lakebridge

Assessment inventoried the pool and pipelines with the Lakebridge analyzer: tables, procedures, views and their dependencies. The estate was ranked by use, and the part of it that nothing read was retired without being migrated at all.

  • Lakebridge analyzer
  • Inventory
  • Dead objects retired

Translation, reviewed

The converter translated T-SQL procedures and views to Databricks SQL; engineers reviewed every diff, kept the business logic and rewrote only the dialect. Nothing was translated blind, and the reviewed SQL went into Git with its history.

  • Lakebridge converter
  • T-SQL to Databricks SQL
  • Reviewed diffs

Orchestration

ADF pipelines became Lakeflow Jobs with the same schedules and dependencies; the loads that were really transformations became Lakeflow Declarative Pipelines with expectations, so a month-end run no longer failed on one bad extract.

  • Lakeflow Jobs
  • Declarative Pipelines
  • Same schedules

Reconciliation

The reconciler compared row counts, aggregates and samples between the pool and Databricks for every table on every run, until the numbers matched across a full reporting cycle. Only then was a table cut over and its pool copy frozen.

  • Lakebridge reconciler
  • Every table
  • A full cycle

Serving and sharing

The Power BI models were repointed to a serverless Databricks SQL warehouse, measures and reports untouched. Reinsurers who received extract files now read governed tables through Delta Sharing.

  • Databricks SQL
  • Delta Sharing
  • Reports unchanged
04 / The outcome

What changed

Nothing was switched off on faith.

Row counts, aggregates and sample comparisons agreed across a full reporting cycle before each table’s pool copy was frozen.

The procedures kept their meaning.

Translation changed the dialect; reviewers kept the business rules, and the SQL now lives in Git.

Runs finish on bad data too.

Expectations validate each extract; explicit quarantine logic retains malformed records while valid data continues through the pipeline.

Compute runs when there is work.

Serverless SQL and job clusters replace a pool billed at rest; the system tables show what each job costs.

Shares instead of files.

Reinsurers read governed tables through Delta Sharing; the extract jobs and their failures are gone.

TechnologyDatabricksLakebridgeUnity CatalogLakeflow JobsLakeflow Declarative PipelinesDatabricks SQLDelta SharingPower BI