Learn how the Variant data type in Spark 4.x and Delta Lake 4.x eliminates repeated JSON parsing, replacing fragile STRING-based patterns with a binary, query-optimized semi-structured type in Fabric Runtime 2.0.
Continue reading
Reliable surrogate keys for modern lakehouse workloads As data platforms evolve from traditional data warehouses to open lakehouse architectures, one requirement has remained constant: the need for stable, unique surrogate keys. With Fabric Runtime 2.0, built on Apache Spark 4.x and Delta...
Continue reading
Historical context: This article describes the April 2026 preview baseline. For newer Spark 4.1 walkthroughs, see the SQL enhancements guide and the SQL pipe syntax tutorial. A modern Spark and Delta foundation for the next generation of analytics and AI in Microsoft...
Continue reading
Welcome to Arshad Ali's blog - practical insights on data and analytics engineering and product management.
Continue reading