I’ve been hearing about Microsoft Fabric for almost 3 years. Most of what I heard about Fabric was not very encouraging. There was a lot of talk about a “single end-to-end platform” for business intelligence, but it sounded like marketing more than reality.

At the same time, it’s clear that Microsoft is investing heavily in Fabric. That alone made me take a closer look. When I checked the certification paths, I noticed that Fabric certifications are replacing the old ones. Many non-Fabric certificates are no longer renewed. It looks like Fabric is the only direction now.

How Did We Make Microsoft Fabric Work?

I started by building a regular meta-driven data pipeline to ingest, load, and transform data from the source system into Fabric Lakehouse. In the same way, I previously built pipelines in Azure Data Factory. The outcome was miserable. It worked just like data pipelines in Azure Data Factory, with high capacity ($$$) consumption. Totally not what I expected.

Then I started to think a bit differently, more Fabric-wise. I created notebooks and put the logic into those. Then I orchestrated these notebooks with other notebooks. As a result, I got a Directed Acyclic Graph (DAG) with all dependencies drawn. This architecture worked significantly better.

We can:

• run notebooks in parallel
• monitor everything in a clean GUI
• schedule jobs easily

Apache Spark really shines in this scenario.

What is the game-changer then?

1. Zero-copy data integration.
   
   With Dataverse, we can replicate tables from Microsoft systems like Business Central or Dynamics 365 and create shortcuts in the Lakehouse. This is a managed service, so it requires almost no code and no maintenance. We get ready-to-use delta tables as a landing zone, without building pipelines. There is also mirroring.
   
   Currently supported sources include:
   
   · Snowflake
   · Oracle
   · Cosmos DB
   · SQL Server
   · Azure SQL DB / Managed Instance
   · SAP
   · SharePoint
   · BigQuery
   · Dremio
   · Databricks Catalog and more coming
   
   
   
2. Everything in one place. Fabric includes:
   
   · Ingestion pipelines
   · Transformation notebooks
   · Custom modules and libraries
   · Git integration
   · Semantic models and reports
   · Scheduling
   · Event streams and event houses
   · Lakehouses and Warehouses
   · … Since this is Microsoft’s main product, more features will come.
   
   
   
3. OneLake. You can access your Lakehouse like a folder on your laptop. This is very useful as a simple landing area where users can drop files.

What is not working?

Microsoft Fabric still feels immature in many areas. We have a fresh tool that is totally unpredictable. Some features work, and some don’t. We do not have most of the features Databricks offers. And the worst thing is that the features are not working, without notifying us. A good example is the table properties. When running our table creation script, Fabric silently omits unsupported properties. And most of them are not supported. As a result, we think all went fine, but no. Nothing went fine.

DELTA_TBL_PROPERTIES = {

    “managed.table.purge”: True,

    “external.table.purge”: True,

    “delta.columnMapping.mode”: “name”,

    “delta.minReaderVersion”: “2”,

    “delta.minWriterVersion”: “5”,

    “delta.autoOptimize.autoCompact”: True,

    “delta.enableChangeDataFeed”: False,

    “delta.logRetentionDuration”: “interval 1 hours”,

    “delta.deletedFileRetentionDuration”: “interval 1 hours”

}

Is it worth trying?

Yes. Not because it’s perfect, and there is no real alternative. There are real alternatives with proven results. Regardless, the data platform market will likely move further into the Microsoft ecosystem, just as data warehouses did before. Decision-makers often choose Microsoft tools — like SSIS, SQL Server, or ADF — and Fabric will follow the same path. Organisations already invested in Microsoft technologies will likely continue to adopt Microsoft Fabric. In many cases, the choice is already made. So instead of fighting it, it’s better to learn how to work with it.

Does Fabric actually work in practice? Answer for yourself.

Author

Marcin Wasilewsk
Technical Solutions Architect | Microsoft Fabric Data Engineer
Holisticon Insight
DP-600 & DP-700 Certified