Implement your TimeXtender on Microsoft Fabric

With the latest TimeXtender release, it is now possible to base your TimeXtender implementation entirely on Microsoft Fabric. This means Fabric Lakehouse for both Ingest and Prepare instances, and XMLA endpoints for Power BI models. As a result, the entire data pipeline lands in Microsoft Fabric.

Although not all transformation functions are available in Prepare instances yet, the most commonly used ones are. For details on Fabric support per TimeXtender layer, see:

• Ingest
• Prepare
• Deliver

With this release we were able to set up a Fabric Capacity to experience what it is like to build solutions entirely on Fabric. Setting up Fabric is easy, and you don't have many choices to make:

• Where do you want your infrastructure to be hosted?
• How much Capacity do you want to give your Fabric workspace? (in other words, how much horsepower you have at your disposal)

Beyond that, you need to take a few technical steps to automate Fabric from TimeXtender, and then you're done. Microsoft Fabric has a unified scale/cost model that depends on "Capacity": an abstraction of compute power that can be used for every Fabric feature. Fabric storage is also abstracted in OneLake. This means there is less to think about (but also fewer options for fine-tuning) during implementation. The smallest available Fabric Capacity already gives you access to everything you need to build a TimeXtender implementation.

TimeXtender builds its solutions from three different types of layers: Ingest (extracting data from the source and storing it in a uniform format), Prepare (transforming data into its final form), Deliver (bringing data to consuming BI tools). I will give you the current status of each type of Instance with regard to how they work in Microsoft Fabric:

Visual representation of the 3 layers in TimeXtender

Ingest

For the most part, the Ingest instance works the same way you are used to in TimeXtender, with only a few limitations at the moment. You can find a full overview of these here.

If you want to try out Fabric Lakehouse Prepare, for the time being you also need to run your Ingest against Lakehouse. Because TimeXtender leaves part of the costly compute to the Virtual Machine that runs the Ingest service, Capacity consumption is lower than you would expect for Ingest. At the moment, overall performance is lower compared to what you can achieve with manual tuning in a SQL IaaS / ADLS Gen2 infrastructure; this is a trade-off against simplified management, since Fabric and OneLake make this a lot easier.

Prepare

For Prepare instances the situation is a bit more dynamic: most implementations will currently not be able to work fully "the TimeXtender way", because they rely on functionality that is not yet available for Fabric Lakehouse.

On the other hand, you can customize your code by linking Spark Notebooks to your tables. This makes your solution less portable and more code-intensive. One of the nice things is that when you simply move data, Shortcuts to the data in Ingest are used, without having to make copies of the data. Performance is currently not as good as other infrastructure options, but this is largely due to cluster start-up times. TimeXtender is working on improvements here, so expect this to get better soon.

Deliver

Working against Power BI XMLA endpoints has always worked in TimeXtender Data Integration, and the move from Premium Capacity to Fabric Capacity doesn't change that. Being able to deliver the same data model to Qlik and Tableau alongside Power BI gives you maximum flexibility.

Right now it is useful to experiment and see what using Fabric in a Data Platform can bring. In that context, evaluating the state of the art with TimeXtender Data Integration is a good idea; one of my favorite TimeXtender features is that you can change the infrastructure along the way to carry out functional and technical evaluations without much work.

If the option to use Service Principal authentication on Run On Demand Item Jobs becomes available from Microsoft, this will remove a security drawback that currently exists. Read the Microsoft documentation on this topic here.

New developments

Both Microsoft Fabric and TimeXtender are developing rapidly, which can make it hard to keep up with everything. For TimeXtender, the annual Xtend event is a great way to stay informed about developments and to meet partners. For Microsoft Fabric, we will be at the Microsoft Fabric Community Conference in Las Vegas at the end of March, so hopefully we will hear more exciting news there. TimeXtender will also be at Fabcon, so stop by if you are there!