Tutorial – Start and Stop Microsoft Fabric Capacity

Cost management is a common challenge across all projects. When implementing solutions, it is crucial to prioritize cost-saving measures to optimize expenses. 

Introduction 

Microsoft Fabric offers the capability to pause and resume its capacity. By pausing the Microsoft Fabric capacity during periods of non-operation, you can achieve significant cost savings for your organization. This document aims to help you set up and execute a scheduled runbook in charge of starting or stopping the capacity. The function of the runbook is to make a dynamic API call that can interact with the Fabric capacity.  

Things to consider 

What does “Pause” capacity really mean? While pausing a Fabric capacity sounds harmless, it is important to understand the full impact of this action. When the Fabric is paused, Fabric cannot handle new information. No new data can be added, and any processes requiring real-time updates will be disrupted. Furthermore, you cannot access your data on the data lake. In essence, your Fabric capacity becomes inoperative. That is why you need to be sure no users are actively working on the platform and consider if any critical processes need to run continuously. Moreover, be aware that resuming Fabric after a pause might take some time for everything to be fully operational again. 

When the fabric capacity is paused, reports in workspaces linked to “Fabric capacity” will become unavailable. Fabric workspaces are specifically tied to “Fabric capacity,” so any reports within these workspaces will not be accessible during the pause. Pro and PPU workspaces will continue to operate normally when the fabric capacity is paused. 

Another important aspect is authentication for the API calls. Authentication plays a central role when calling through an API. For the API call to be authenticated, we will need to set up an app registration that will generate an authentication token for accessing the Azure resources. Each time the runbook is executed, a new token will be generated. 

Main Steps 

Fabric Capacity creation 

The first step is the creation of a fabric capacity. For this we refer to the documentation of Microsoft itself.

The result will look something similar as to what is shown on the figure below. To continue, make sure you have an administrator role within the Fabric capacity. 

App Registration set up 

As previously mentioned, we need to generate a token for accessing the Azure resources including Fabric. For this we will register an application in the Azure Active Directory.  

Click on the + New Registration button. 

Fill in all the necessary fields and click on the “Register” button.

In this example we called the app registration: serviceprincipaldata. The next step is to create a secret and store it in a KeyVault. Navigate to the app registration you just created and open it. Next, under “Manage”, click the “Certificates & secrets” tab. Next, click the button “+ New client secret”.  

The secret value is only accessible after creating the secret. It is a secret string that the application uses to prove its identity when requesting a token. Since this value is only accessible after the creation, it is recommended to store it in a key vault as shown in the picture below.  

Next, you need to assign the “Contributor” role to the app registration within Azure’s Access Control (IAM) settings of the Fabric Capacity to allow service principal to perform action on the Fabric capacity. 

Click the “+ Add” and next the “+Add role assignment” buttons and navigate to the privileged administrator role tab. There you can select the contributor role. Select the role and click on next. 

Next click on the “+Select members” and type in the name of the app registration (service principal) that you just created.  

Click on the review+assign button to assign the role.  

Automation Account 

The next step is to create an Automation Account. This is necessary to schedule and run your runbooks. For this we refer to the Azure documentation.

In the Automation Account, you’ll create a runbook that contains the code to execute actions for starting and stopping capacity in Fabric. The required code will be explained in the next section. 

In the left pane, click on the “Runbooks” section and click on the “+Create a runbook” button. The following screen will appear. 

Give the runbook a name and select the runbook type. For this documentation, we used Python as the runbook type. Pay specific attention to the runtime version of the runbook, as this will be important for a later stage when we install the required Python Packages. In this example we utilized runtime 3.8 (recommended).  

Next, associate two triggers with the runbook to schedule the starting and stopping the Fabric capacity. 

These two triggers will provide parameters to the runbook to perform the respective actions (resume or suspend). 

The only step left is to put the code into the runbook. Which is expanded upon in the next section. 

Runbook Creation 

Requirements 

The runbook is written in Python and requires certain packages and libraries to function properly. 

These packages can be downloaded from a specific website (PyPI · The Python Package Index) and should be stored locally. You can download either the “.tar.gz” or “.whl file” extension. In this example we downloaded the .whl extension. After downloading, they can be imported directly into the Python script under the “Package section”. In this solution, the following packages were installed on the automation account: 

1. Azure_core 

2. Azure_identity 

3. Azure_keyvault_secret 

4. Requests 

An important note is that when downloading a package and installing it on the runbook is that dependencies on other packages are not automatically installed. Therefore, you need to ensure that dependent packages are installed as well. In this case, azure_identity is dependent on azure_core.  

Furthermore, it is also imperative that when installing the package, the correct runtime is enabled. Otherwise, the runbook cannot recognize the installed Python packages. As previously mentioned, in this solution we chose to use runtime 3.8. Therefore, when installing the packages we should utilize the same version.  

To import a package, click on the Python packages tab in the left panel. Next, press the “+ Add a Python package” button.  

Here you upload your local file (.whl or .tar.gz format) downloaded from the (PyPI · The Python Package Index) website. Azure will automatically detect the name of the package, but you can customize the name accordingly. Select the correct runtime version (in this case 3.8) and click on the “Import” button to import the package. Only a single package can be uploaded at a time. 

Code 

1. Import libraries needed 

2. Retrieve secret from Keyvault 

DefaultAzureCredential: It’s a part of the Azure SDK for Python and provides a way to use multiple credentials (like environment variables, managed identities, or shared tokens) to authenticate requests. 

SecretClient: This is used to interact with Azure Key Vault to retrieve secrets. The script retrieves various secrets like client_id, client_secret, subscription_id, resource_group, and resource_name from Azure Key Vault. 

3. Retrieve Action as parameter 

4. Generate Authorization Token 

This section uses the retrieved client_id and client_secret to request an OAuth2 token from Azure Active Directory. The token is then extracted from the response and stored in authorization_token to be used for API requests. 

5. API Call 

• url_post: This URL is used for making a POST request to either suspend or resume a resource based on the provided action (suspend or resume). 
• url_get: This URL is used for making a GET request to retrieve information about the specified resource. 
• headers: The headers include the authorization token and content type. 
• res: The POST request is made to either suspend or resume the resource. 
• response: The GET request is made to retrieve information about the resource. 

Closing 

In this tutorial, we explored one method to automate the start and stop of Microsoft Fabric capacity using scheduled runbooks. Other methods include Logic Apps, Power Automate or Data Factory can also be used to achieve this goal. By leveraging these tools, you can further optimize the management and operational efficiency of your Microsoft Fabric. 

Written by