Microsoft Fabric Community Conference 2024 – Recap 

Last week, Lytix had the opportunity to attend the first edition of the European Microsoft Fabric Conference in Stockholm. The four-day event was filled with insightful sessions showcasing the latest advancements in Microsoft Fabric. The first day featured a full schedule of hands-on workshops during the tutorial day, offering attendees practical learning experiences. Over the next three days, a diverse lineup of speakers from around the globe, including customers, industry experts and Microsoft employees, delivered engaging sessions to share their knowledge with the audience. In this blog post, we will share our experiences and some of the key announcements. 

Strong Partnerships 

The strong presence of industry partners was evident throughout the conference. Microsoft announced the ability to mirror Unity Catalog Tables from Databricks to Microsoft Fabric, highlighting the power of collaboration. Additionally, integrations with platforms like Snowflake, Amazon S3 and Google Cloud underscored the value of these strategic partnerships. When competitors work together, it’s the customers who benefit the most. Besides that, it is also clear that Microsoft is embracing open-source formats such as Delta Lake and Iceberg. Another feature now in public preview is the ability to sync Oracle databases with Microsoft Fabric using Oracle’s GoldenGate. Last but not least, Microsoft continues to support SAP data ingestion into the platform by offering a wide range of standard connectors to various SAP systems, ensuring seamless integration. 

Enterprise Readiness 

To accelerate enterprise-wide adoption, Microsoft announced the Public Preview of Fabric’s Terraform provider, a significant step towards simplifying the rollout of Fabric. Another key update is the ability to use Fabric REST APIs with service principals, which will greatly enhance deployment and management processes. They also introduced a revamped deployment pipeline interface in Fabric, streamlining the user experience and making deployment processes more intuitive and efficient. To further streamline the developer experience, git integration went into general availability. Not all items are yet supported but full support is planned before the end of the year. 

Data Engineering 

The core of the Lakehouse architecture received significant upgrades with the introduction of the new Fabric 1.3 runtime, which includes Spark 3.5 and Delta 3.2, bringing it up to speed with the latest features. Additionally, the Native Execution Engine was launched in public preview, offering a substantial performance boost for Spark jobs and queries without added cost. It promises a 4x performance improvement! Developers can now create user-defined functions in Python, currently in public preview. Another valuable update is the support for spaces in Delta column names, a previously unavailable feature. This allows for more business-friendly naming in Power BI while still taking advantage of Direct Lake capabilities. 

Data Warehousing 

Let’s not forget about an old friend, the SQL Datawarehouse, which also received some exciting updates. One notable addition is the ability to use T-SQL directly within notebooks. A migration assistant, now in private preview, was introduced to streamline the migration of both on-premise and cloud data warehouses to Fabric, making the process much easier. Speaking of ease, Copilot for Data Warehouse is now in public preview, enabling users to write SQL queries faster than ever with AI assistance. 

AI Capabilities 

In the age of AI, the announcement of Copilot is impossible to overlook. Copilot has now reached General Availability for Dataflows Gen 2 and has been introduced in public preview for the Data Warehouse experience. It was already integrated into Data Factory and Power BI, expanding its role across the platform. 

Data Factory 

Several exciting announcements were made for Data Factory as well. You can now bring your existing Azure Data Factory into Fabric, thanks to a new preview feature that allows a seamless connection between the two. A new job type, Copy Job, has also been introduced. Since many pipelines only use the copy activity, this job lets you copy data without needing to create an entire pipeline. Additionally, the Copy Job now supports incremental copies, a valuable enhancement. Another highly requested feature is session reuse for notebook activities. You can now pass session tags to reuse sessions, eliminating the delay caused by cold starts. 

Example of the Copy Job Assistant Source: Microsoft 

Data Governance 

A topic often overlooked but crucial for improving data literacy within organizations is the integration of Microsoft Purview with Fabric. This integration is now stronger than ever and continues to evolve. The new user interface for Purview is available to all users, and one of the most significant updates is the ability to add data quality rules. This empowers data stewards to track and monitor data quality over time. Purview will also suggest new rules based on profiling results. This feature supports multi-cloud environments, including Azure Synapse, Databricks Unity Catalog, Fabric Lakehouse, Google BigQuery and Snowflake—once again highlighting the value of collaboration across platforms for customer benefit. Another valuable addition is the ability to apply sensitivity labels in Purview, which are seamlessly reflected within Fabric. 

Looking Back 

We reflect on an incredible conference filled with valuable learning and networking opportunities. FabCon Europe brought together a community of like-minded individuals with a shared passion for data. One of the exciting new challenges unveiled at the event was the DP-700 Fabric Data Engineer Associate certification. As a Microsoft Gold Partner, we stay at the forefront of Microsoft’s Data and AI technologies. In fact, more than 15 of our colleagues are already certified as DP-600 Fabric Analytics Engineers, and we’re eager to explore the learning opportunities this newly announced certification will offer. 

Curious about how Microsoft Fabric can benefit your organization? Don’t hesitate to reach out—we’d be happy to discuss how it can create value for you! 

Written by