Microsoft Fabric now offers low-cost, low-latency data replication into OneLake through mirroring, facilitating near real-time analytics without complex ingestion. The Fabric Accelerator 5.0 introduces a OneLake-Native ELT Pattern, eliminating duplicate data and accelerating the creation of reliable data products directly from shared assets while maintaining governance and quality standards.
Fabric Accelerator setup is now fully automated, cutting deployment time from 2 hours to just 15–20 minutes. The new process uses Fabric CLI for streamlined configuration, eliminating Git dependencies and tedious ID replacements. Deploy all Fabric items with full support for re-runnability and asynchronous Long Running Operations (LRO). Git-based CI/CD via GitHub Actions ensures version-controlled…
The Fabric Accelerator is a reusable code component suite for Microsoft Fabric, supporting batch and real-time workloads. It features a metadata-driven ELT framework and has transitioned to Fabric SQL, enhancing management and integration. Minor updates were made for streamlined deployment while removing certain features for better efficiency. Data engineers are encouraged to explore its…
Introducing observability in Fabric Accelerator, the first RTI workload enhancing data platform insights. It monitors workspace activities, job events, OneLake actions, and alerts for execution anomalies and user trends. Leveraging a medallion architecture, it provides real-time data filters and visualizations, enabling timely notifications and efficient data tracking.
The Fabric Accelerator is a framework for building modern data platforms using Microsoft Fabric. It utilizes the ELT framework, offering ease of use, scalability, and flexibility. Key features include automated monitoring, pre-built artifacts, and a modular architecture. It aims to accelerate development, streamline operations, and incorporates industry best practices.
The ability to generate concise summary from pages of text is one of the built-in capabilities of Large Language Models like GPT. This post explains how to summarize text using Azure Open AI.
This post is all about using PySpark libraries to apply common transformations to a Spark Dataframe.
Time Zone Conversion in PySpark
While type 2 dimensions are great to track attribute history, in some instances type 2 SCD may not be appropriate. In this post, I would like to mention few common anti-patterns I have encountered around type 2 SCD.
Power Query can discover and import data from websites. Often data warehouses rely on external data which is readily available in public websites for e.g. public holidays, school holidays, SIC codes, SWIFT codes, post codes etc. Power Query is perfectly suitable for such situations. Power Query can discover, fetch and transform data from a HTML table in a web page into a format that can be easily…