Integrating with Microsoft Azure Event Hubs

Overview

Microsoft Azure Event Hubs is a real-time logging and data ingestion service with integration across the Microsoft Azure platform.

Blumira integrates with Microsoft Azure Event Hubs to stream Azure cloud security event logs and alerts to the Blumira service for threat detection, alerting, and actionable response.

The Azure Event Hubs integration can also be used to collect logs from Microsoft Intune and Microsoft 365 Defender.

Before you begin

This integration requires a Blumira sensor to be installed before you can complete the steps below. Ensure that you complete the steps in Building a Blumira sensor with Ubuntu before you continue.

To connect Azure Event Hubs and send logs to Blumira from your supported data sources, complete the following steps:

Configuring Azure to obtain credentials

We provide two methods below as options to set up and gather credentials for your Azure Event Hub. Choose one of the following methods:

• (Recommended) Running a script to create and obtain Event Hub credentials
• Manually configuring Azure and obtaining credentials

Running a script to create and obtain Event Hub credentials

You can use Blumira's AzBluMon script to automatically set up an event hub and obtain the necessary credentials for the Blumira sensor module. The pre-requisites and limitations of using the script include:

• This script must be run from a Bash shell with Azure CLI installed locally or through Azure Cloud Shell using Bash, which is our recommended method.
• This script only works with one Azure subscription at a time. If you have multiple subscriptions you will need to run it within each subscription.
• This script only works with your Azure subscription and does not integrate with Azure AD, Intune, or Defender. Those products are not included in your Azure subscription.
• All the resources you wish to monitor such as Storage Accounts, Network Security Groups, Virtual Networks, etc. need to be in the same region as your Event Hub Namespace. If you are using multiple regions for the resources you wish to monitor you must configure those manually and create a new Event Hub Namespace for those regions.

Note: You can run this script multiple times. When run again, the script will update resource diagnostic settings, if applicable. This will update logging configurations for Azure resources that were not enabled or present during the initial run. It still provides the connection string and event hub name in case you need to make any changes to the module in-app. Usually, updating the module in-app is not required unless your module was deleted between runs.

To prepare a Cloud Shell and run the script:

1. In Azure, launch Cloud Shell from the top navigation of the portal.
   
2. If this is the first time you are using Cloud Shell, you will be prompted to select Bash or Powershell.
3. Click Create storage.
4. (Optional) If you need a specialized or custom storage account, click Show advanced settings to customize the Cloud Shell configuration.
   
5. Wait for the Cloud Shell to show “Succeeded” and verify that you are in the Bash environment (i.e., that Bash is selected in the environment dropdown in the left-hand side of the shell window) before proceeding with the steps below.
   
6. In the Cloud Shell window run this command:
   git clone https://github.com/Blumira/AzBluMon.git
   
7. After the successful clone, run the following separate commands:
   cd ./AzBluMon
chmod +x ./AzBluMon.azcli
./AzBluMon.azcli
8. After the prompt What is your subscription ID?, paste your subscription ID into the command line.
   Note: The prompt also includes a link to help you find your subscription ID. If you have multiple subscriptions you will need to run this script separately for each subscription. Do not enter more than one subscription ID.
   
9. Press the Enter key.
10. After the prompt Where are the majority of your resources located?, type your region code. This is used to create the Event Hub namespace.
    Note: Determine your Azure region code by referring to the Name column of the table provided in Current Azure Region Names - Reference. For this integration, consider the region as the place where most of your resources are located.
11. Press the Enter key.
12. After the prompt What would you like to name your Event Hub Namespace?, type the name you want for your namespace.
    Note: You must provide a unique name for your namespace. No spaces are allowed.
13. Press the Enter key.
14. Copy the primary connection string and the Event Hub Name for use in Providing your Event Hub credentials to Blumira.
15. Skip to Sending logs from Azure AD.

Manually configuring Azure and obtaining credentials

Manually configuring an Event Hub Namespace

To manually configure a namespace in Event Hubs:

1. Go to https://portal.azure.com/ and log in.
2. Click Event Hubs.
3. Click Add.
4. On the Basics screen, under Project Details, complete the following fields:
   • Select a Resource Group (or click Create New to add a new resource group option).
   • Type an event hub namespace name (example: Blumiralogs).
   • Select the Location and Pricing tier you want to use.
5. (Optional) Add Availability Zone Features and Tags.
6. Click Review + Create.
   

Configuring an Event Hub

Note: This step is not required for all types of logs. Some log sources automatically create their own event hub within the namespace you created above. Blumira recommends creating an event hub to provide clarity in your configurations, including when integrating Blumira with Microsoft Defender. 

1. In the Azure portal, click Event Hubs.
2. Click on the event hub namespace you want to use for Blumira logs.
3. Click Event Hubs
4. Click + Event Hub to add a new event hub.
5. Type an event hub name (example: blumira-log-stream).
6. Select Create.

After Azure is done with the creation process, complete these steps:

1. Click on the event hub you just created.
2. Click Shared access policies.
3. Click + Add to add a new policy.
4. Type a name, such as ReadOnlyAccessKey.
5. Select the Listen check box.
6. Click Create.
7. In the Shared access policies list, click on the policy you just created.
8. In the policy's detail window, copy and save the Connection string-primary key for use in later steps.
   

Providing your Event Hub credentials to Blumira

The Event Hubs integration requires the Azure Event Hubs sensor module in Blumira, configured with these Event Hub credentials, which you gathered above:

• Connection string-primary key
• Event Hub Name

To add a module on an existing sensor and provide credentials:

1. In Blumira, click Settings.
2. Click Sensors.
3. Click the sensor on which you want to add a module.
4. On the detail page for the sensor, scroll down and click Add Module.
5. In the Add New Module window, select the newest version of this integration's module. Note: For the best stability and performance, Blumira will update the module version when old versions are deprecated.
6. Enter the credentials that you gathered in the "Before you begin" section above.
7. (Optional) Type a name for this log deployment in the Log Source Name box. This name is what will appear in the "device_address" column in the results of your event data queries. If you might have additional modules collect logs for different integrations in the future, this will help you distinguish them. Note: The name can only contain alphanumeric characters, periods, and hyphens; no spaces or underscores are allowed.
8. Click Install.

Requirement: If you filter your outbound traffic, you need to allow the following ports for Event Hubs communication and authentication:

• TCP/443
• TCP/5671
• TCP/5672

Sending logs through an Event Hub to Blumira

After configuring the Event Hubs module in Blumira, you must configure your log source applications so that they send logs through the event hub to your Blumira sensor module. Complete the procedures below for the applications that your organization uses:

• Azure Monitor
• Azure AD
• Microsoft Intune
• Microsoft 365 Defender

Sending logs from Azure Monitor

There are two types of Azure platform logs collected with this integration: Activity logs and Resource logs. These logs give you insight into the actions taken within your Subscription such as the creation, modification, and deletion of resources, as well as the use of Azure CLI and Azure Powershell. Resource logs can take a few different forms and are not supported on all Azure resources; for example, some Compute resources and Managed Disks do not have the ability to provide audit logs and security logs. 

Note: If you used the script to automate the setup, Azure Monitor creates its own event hub called “insights-operational-logs”. You must still create an event hub namespace but can skip creating an event hub.

1. Go to https://portal.azure.com/ and log in.
2. Navigate to All Services > Subscriptions.
3. Select the Azure subscription you want to monitor.
4. Click Resource providers.
5. Search for and select Microsoft.Insights.
6. Click Register. (If already registered, move on to next the step.)
7. Navigate back to the Azure subscription that you selected in Step 3.
8. Click Monitor.
9. Click Activity Log.
10. Click Export Activity Logs.
    
    
    
    
11. Click Add Diagnostic Setting and complete these steps:
    • Under Category details, click the check boxes next to all log types.
    • Under Destination details:
      • Click the Stream to an event hub check box.
      • Select or verify the event hub's Subscription and namespace.
      • Select the Event hub name (i.e., blumira-log-stream).
      • In the Event hub policy name box, select RootManageSharedAccessKey.
    • Click Save. 

Sending logs from Azure AD

Azure AD logs are critical to organizations that use Defender products or have active resources within Azure such as a subscription. These logs are also recommended if the organization uses Conditional Access and other Azure AD resources. Sign-in logs, Azure AD alerts, and more are available from this integration.

Because Azure AD exists outside of your Azure subscription, you must configure Azure AD separately to point to your Event Hub and Event Hub Namespace, which you created manually or via the automated script in previous steps.

Note: Azure AD can optionally create its own event hub called “insights-logs-audit”.  If you prefer to use a different event hub, you must create it first using the procedure above.

1. Go to https://aad.portal.azure.com and log in.
2. Click Azure Active Directory.
3. Click Audit Logs (in monitoring section of left menu).
4. Scroll down the left menu and click Add Diagnostics Setting.
5. Enter a name for this setting, such as “Blumira events“.
6. Select Stream to an event hub.
7. Click Event hub Configure.
8. Select your event hub namespace.
9. Select the event hub that you previously identified (or use default “insights-logs-audit”).
10. Use policy RootManageSharedAccessKey.
11. Click OK to save the event hub configuration.
12. Select the check box for all of the available log categories (AuditLogs, SignInLogs, etc).
    Note: Categories appear based on your Microsoft licensing level.
    Note: Azure AD must be configured manually to point to your Event Hub and Event Hub Namespace that was created manually or via the automated script. This is due to Azure AD existing outside of your Azure subscription.
13. Click Save (at the top).