In today’s fast-paced, data-driven world, businesses generate vast amounts of information daily. With over 328.77 million terabytes of data being created every day, the need for scalable, efficient data pipelines has never been more critical. Many organizations struggle with the complexity of traditional ETL (Extract, Transform, Load) processes—where moving, cleaning, and transforming data can be […]
Tag: SQL
Performance tuning using indexes
Do you know that feeling when you finally have your data warehouse up and running using dozens of stored procedures written in SQL? You feel satisfied when your pipelines have been running for days without throwing an error. But then you notice that the time it takes to run your pipeline is tremendous and the […]
Synapse Dedicated SQL Pool vs. Serverless SQL Pool
Microsoft’s Synapse Analytics Suite aims to provide solutions for various data usage, consumption, and exploratory situations while also ensuring that clients can control costs in a straightforward and predictable manner. To address these distinct circumstances, Microsoft has built two separate techniques to controlling and allocating computation. In this blog we will go through both techniques. […]
Azure Synapse Analytics
Organizations understand the value of data more than ever. A Data Warehouse as a single source of truth, a data lake to store data for analytical exploration, self-service tools for data transformation, visualisation, and consumption as well as clusters to process immense data volumes. All these different use cases require other specialised tools resulting in […]
5 Tips for Power BI/Analysis Services to improve the use of Excel
Wow, you may recall our previous post ‘5 tips for Excel users when connecting to Power BI’, its popularity needed us to upgrade our webservers. Luckily we run on azure and could easily scale up Don’t fear, we are more than happy to provide you with five (5!) additional tips to improve your BI environment. In […]
Loading mechanisms – Part I
As there are huge amounts of data available within companies, data is also moved in increasing quantities from one data storage to another for multiple reasons. As copying data can come with a longer load time and higher costs, you want to make this process as efficient as possible. Luckily, there are multiple loading mechanisms […]
Feature Store
Everyone who has already come in touch with data science, has already heard of features used in such models. One aspect that can become quite challenging, is reusing features in a consistent way, across several team members, projects and in environments. In this article, I will explain the most commonly used way to resolve these […]
Data Services Python Script Call
In the magical world of data extraction Data Services offers great out of the box functionality to extract data from databases like SQL Server, MySQL, PostgreSQL, Oracle… and from SAP ECC. However, to extract data from REST APIs it lacks some functionality. Therefore, we will use one of the most currently popular programming/scripting language: Python. […]
