Hey everyone,
I want to share an experiment I ran on cross-workspace orchestration across 50 and 500 Microsoft Fabric workspaces.
My motivation comes from my background running enterprise analytics for a large organisation. One of the distinctive strengths of Fabric, compared with other platforms, is its suitability for self-service BI across a large organisation. OneLake is pivotal to that.
This raises some common challenges I've experienced. If a central team manages shared data, while dozens of business teams independently build their own data products, how do we coordinate dependencies, understand the impact of failures and manage recovery?
So I set up an experiment with 50 workspaces, then scaled it to 500 to see whether the approach would hold.
In particular, I wanted to see whether Weaver, an open-source data engineering framework I've been building for Fabric, could handle the challenge.
The setup
I created 50 real Fabric workspaces on an FTL64 capacity:
- 3 central workspaces covering Bronze, Silver, Gold and shared semantic models.
- 47 business workspaces, each with its own catalogue.
- A mixture of Lakehouses, Warehouses, semantic models and reports.
- 53 cross-workspace connections, including a deliberate dependency cycle.
Every workspace was independently initialised, built and loaded. Together, they formed one dependency graph across the organisation.
I then deliberately broke a central Bronze customer table, BRZ.Customer.
The results
1. Enterprise-wide impact analysis
- 161 downstream objects identified across 49 other catalogues.
- Full assessment in 134 seconds.
- The assessment distinguished stale objects from those still operational.
2. The central team could recover its own estate independently.
I then tested recovery from the central team's perspective, across its three workspaces.
A single load --stale identified and planned just six affected objects, in dependency order:
- Bronze workspace: Reload the affected central Bronze data.
- Silver/Gold workspace: Propagate the repair through the Silver Lakehouse and Gold Warehouse.
- Semantic model workspace: Refresh the central semantic model against the repaired data.
The central team recovered its own estate without reloading any business team's data.
There was even a network interruption during the semantic model's load recording. On rerun, only the semantic model needed refreshing rather than repeating the entire recovery.
3. Business teams could see what was affected on their side.
I tested this from the perspective of business team 05.
- Its own Health assessment identified two affected Mart tables.
- It traced the cause back to
BRZ.Customer.
- The assessment took just 10 seconds.
No need to contact the central engineering team.
4. Business teams could decide how to recover.
After the central team repaired its estate, team 05 could see that its own tables were now behind the upstream data.
It reloaded only its two stale tables, taking 24 seconds, and returned to green.
No need to rebuild the whole workspace or wait for a central orchestrator.
5. The system tolerated dependency cycles.
I deliberately created a cycle between two business workspaces.
The cycle was identified and reported, and execution proceeded by cutting one dependency edge. The rest of the estate continued operating normally.
One object remained correctly stale afterwards because it had consumed the other team's previous load.
One final experiment
I also tried running one federated load across all 50 workspaces, with independent branches executing concurrently while respecting dependencies.
The result:
- 81 execution nodes across 47 segments.
- All successful under one workflow ID.
- End-to-end time of 9 minutes 21 seconds.
This opens up another possibility: a central team could coordinate an organisation-wide refresh when required, while ordinary day-to-day execution remains decentralised.
Then I tried 500 workspaces.
I expanded the experiment to 500 workspaces: three central workspaces and 497 business teams, using the same FTL64 capacity.
The results:
- 1,160 execution steps successfully coordinated in approximately 39 minutes.
- 752 downstream objects traced to a single failure, spanning 450 workspaces.
- Full Health assessment across 500 catalogues in approximately 5–7 minutes.
- Individual team assessment still took around 13 seconds.
What I learned
OneLake is absolutely wonderful, and I think the Microsoft Fabric architecture is extremely strong for large enterprises.
With Fabric, teams can achieve two things:
- Complete decentralised freedom when appropriate: Teams can own their workspaces, choose their tools and development processes, and use a common data hub.
- Complete coordination when required: Dependencies, failure impacts, execution and recovery can be understood across the entire organisation.
My conclusion is that:
- Microsoft Fabric is one of the strongest platforms for enterprise data analytics, where central information can be shared across multiple business areas without pigeonholing people into high-code development. People can choose Python, SQL, semantic models or simply reports. OneLake is a brilliant architectural foundation for this.
- Open-source tools like Weaver can help data teams get the most out of Fabric. Not just with coordination challenges like this, but by giving teams a head start on robust data engineering across Python, Spark and T-SQL. And the more teams use it in an organisation, the better it gets.
I suspect I'm not alone in facing these challenges. Keen to hear about other people's experiences and techniques for managing multiple workspaces.
Do you have cross-workspace or cross-item dependency workloads? How do you deal with them?
Full experiment and technical details:
https://weaverstack.dev/guides/organising-microsoft-fabric-workspaces/
Happy to answer any questions!