Six applications needed a shared data foundation for a healthcare organization. This Microsoft Fabric platform was built from scratch around configuration, generated pipelines and notebook processing. All five REST applications and the SQL Server source are integrated through Bronze, Silver and Gold. The prepared data now feeds Salesforce Data Cloud for cross-source identity resolution and Power BI for reporting through Direct Lake.
The brief called for five API integrations and one SQL Server integration, with consistent data for a CDP and reporting. The platform needed to establish ingestion, contracts, quality checks, run history and deployment from the ground up.
All six integrations run through the shared Fabric framework, with generated orchestration, recorded checks and isolated rejects. Gold data is ready and in use for Salesforce Data Cloud and Power BI. The reporting model reads Gold Delta tables through Direct Lake; cross-source identity resolution remains in Salesforce.
Five REST applications and one SQL application feed Bronze, Silver and Gold. The prepared data is in use for Salesforce Data Cloud and Power BI reporting.
A dedicated schema-enabled Silver lakehouse holds rebuilt, validated tables. Transformations normalize names, types, timestamps and flags, and expand JSON arrays into child tables. Configured contracts and relationships produce check records and separate rejected-row outputs.
The architecture uses one Fabric workspace with separate schema-enabled Control, Bronze, Silver and Gold lakehouses. Lakehouse permissions remain separate while workspace roles retain their broader access. Salesforce Data Cloud ingests Gold through its SQL connector path; Power BI reads Gold Delta tables through Direct Lake.
One workspace contains separate schema-enabled lakehouses for Control, Bronze, Silver and Gold. Application, entity and column definitions live as reviewed JSON and CSV in the repository. Deployment applies that configuration, and the generator produces the pipeline definitions. Runtime batches, entity runs, watermarks, quality results and audit records live in the Control lakehouse. All five REST applications and the SQL Server application use this shared framework.
SQL Server lands through an on-premises gateway and a generated Copy activity per connection. The REST notebook framework integrates all five API applications through source-specific configuration. It supports OAuth refresh-token rotation, incremental reads by modification time and a weekly full-reload path. Inactive and deceased source records are included. Files land first, then merge into Delta using null-safe business keys and a row-loss guard. Notebooks open and close each batch; every entity receives its own run record.
A Fabric pipeline invokes the Silver build notebook to rebuild each table from Bronze, with failures isolated by table. Names become snake_case, columns receive explicit types and source-local timestamps become UTC. Sentinel values and Y/N flags are normalized; nested JSON arrays become child tables. Contracts, row rules and relationships are configuration, so transformations and validation can be reviewed together.
Each check writes a result to runtime.data_quality. Invalid rows are retained in table-specific reject outputs, separate from accepted Silver data. A build lock and a batch precondition block processing when Bronze has failed or is stale. Rules are added from measured evidence, with their results retained for investigation. Classification also supplies column comments, keeping the data contract visible to the next engineer.
Gold publishes person records at one row per person per source, retaining native identifiers. Salesforce Data Cloud ingests the selected tables through its Fabric connector and a schema-scoped SQL analytics endpoint connection. Cross-source identity resolution happens in Salesforce Data Cloud. The Power BI semantic model uses Direct Lake on the same Gold lakehouse, reading Delta data from OneLake through a separate consumption path.
The parent pipeline processes load groups sequentially, invokes Bronze and starts Silver only after success. Invoke Pipeline activities use the workspace identity. Notification notebooks compose application-grouped summaries from the control tables: one parent Bronze summary, a Silver summary, quality alerts when checks find issues and a failure alert when a run fails. Office 365 Email uses its authenticated connection; message copies are retained with the run evidence.
Notebooks, configuration, SQL, deployment scripts and architecture documentation live in the repository, with 77 offline framework tests. Deployment and run tools support notebook logs and Silver-run verification; a status file records the handover. The maintenance notebook is prepared for weekly OPTIMIZE, VACUUM with seven-day retention and runtime-history retention. Large tables use Z-Order.
Five REST applications and one SQL Server application are integrated through the same configuration-driven Fabric platform.
Silver tables have explicit contracts and recorded checks. Rejected rows remain inspectable, and failed or stale Bronze batches cannot silently feed a new build.
Batch and entity records connect ingestion, watermarks, quality results and notification copies. Failures produce an alert instead of an ordinary completion summary.
Reviewed configuration generates the pipelines. Deployment scripts, offline tests, run verification and the status handover document accompany the notebooks.
Salesforce Data Cloud consumes the prepared source-level person tables for CDP identity resolution. Power BI reads the same Gold lakehouse through Direct Lake for reporting.