Power BI

short-term financial goals pushes longer-term

established hybrid organizations, which include centers or excellence, analytic sandboxes, or innovation labs in efforts to derive benefits more rapidly from their data investments

One suggestion was not to focus on overall data-driven transformation in a large enterprise, but rather to identify specific projects and business initiatives that move a company in the right direction

Another was trying to implement agile methods in key programs, while avoiding terms like “data governance” that have a negative connotation for many executives

Analytical decisions and actions continue to be generally superior to those based on intuition and experience

data-driven organizations and cultures isn’t going away

Microsoft is pushing dataflows to “data engineer, data integrator, and business analyst”. My guidance is to consider dataflows only if you want to open data ingestion to business users (something you must carefully think about and definitely surround it with a log of supervision). As its predecessor (Power BI dataflows), Power Query is notoriously difficult to troubleshoot or optimize. It doesn’t support the ELT pattern (my favorite), such as to handle Type 2 changes. This could be partially ramified by implementing a pipeline that mixes dataflows with other ADF artifacts, such as calling stored procedures in Fabric Warehouse. Moreover, I consider Power Query as a Microsoft proprietary tool, irrespective that the M language is documented. If one day you decide to leave Fabric, you’d need to rewrite your flows. Finally, the only output options supported are append or replace (no update).Moving to ADF, the copy activity supports only append or replace (no update). Outside Fabric, Azure Data Fabric doesn’t have connectors to connect to Fabric yet.

Direct Lake mode is a groundbreaking new dataset capability for analyzing very large data volumes in Power BI. Direct Lake is based on loading parquet-formatted files directly from a data lake without having to query a Lakehouse endpoint

When I think about the Lakehouse is I think of files as the source files just being loaded in their raw form from their source. While the tables in Lakehouse are the parquet files that have been processed via Dataflow Gen2 or Spark Notebooks

Creating a DirectLake Power BI Dataset Now this is the most amazing part where I am going to create my DirectLake dataset, where I will not have to import the data into Power BI, but rather simply create my dataset directly from the tables in the Lakehouse.