PDF

Azure Resource Monitoring using Telegraf in NetCrunch

This document describes how to configure Telegraf to collect metrics from various Azure resources (such as Virtual Machines, Storage Accounts, and Databases) and send them to NetCrunch via the Telemetry Node endpoint.

Overview

Telegraf can collect Azure resource metrics using the Azure Monitor input plugin. This enables monitoring of Azure resources without requiring direct network access from NetCrunch to Azure.

The configuration examples in this document use Azure Virtual Machines as a reference, but the plugin supports any Azure resource type that exposes metrics through Azure Monitor API.

How NetCrunch Supports Azure VM Telemetry

NetCrunch receives data from Telegraf through a Telemetry Node REST endpoint. Telemetry Nodes accept JSON-formatted data and store received values as counters or alert statuses.

The endpoint, its URL shape and how it is authorized are described once in Monitoring with Telegraf. Everything below assumes a Telemetry Node already exists — see Telemetry Node.

Data Flow

Azure VM monitoring via Telegraf follows this process:

  1. Azure Monitor API Query - Telegraf queries Azure Monitor API for VM metrics using service principal credentials.
  2. Metric Collection - Azure Monitor returns requested metrics for the specified resource.
  3. Data Forwarding - Telegraf forwards collected metrics to the NetCrunch Telemetry Node via HTTP POST.
  4. NetCrunch Processing - The Telemetry Node anchors incoming metrics and stores them as counters or alert statuses.

Monitoring Methods

The Azure Monitor input plugin supports three collection methods:

Resource Target

Collect metrics from specific resources using resource ID. This method is suitable for monitoring individual resources.


[[inputs.azure_monitor.resource_target]]
  resource_id = "resourceGroups/<group>/providers/Microsoft.Compute/virtualMachines/<vm-name>"
  metrics = ["Percentage CPU", "Available Memory Bytes"]
  aggregations = ["Average", "Maximum"]

Resource Group Target

Collect metrics from all resources of a specific type within a resource group.


[[inputs.azure_monitor.resource_group_target]]
  resource_group = "production-rg"


[[inputs.azure_monitor.resource_group_target.resource]]
    resource_type = "Microsoft.Compute/virtualMachines"
    metrics = ["Percentage CPU", "Available Memory Bytes"]
    aggregations = ["Average"]

Subscription Target

Collect metrics from all resources of a specific type across the entire subscription.


[[inputs.azure_monitor.subscription_target]]
  resource_type = "Microsoft.Compute/virtualMachines"
  metrics = ["Percentage CPU"]
  aggregations = ["Average"]

Supported Azure Resources

The Azure Monitor plugin can collect metrics from any Azure resource type that exposes metrics through Azure Monitor API. Common resource types include:

Compute Resources

  • Virtual Machines: Microsoft.Compute/virtualMachines
  • Virtual Machine Scale Sets: Microsoft.Compute/virtualMachineScaleSets
  • App Services: Microsoft.Web/sites
  • Azure Functions: Microsoft.Web/sites
  • Container Instances: Microsoft.ContainerInstance/containerGroups
  • Kubernetes Service: Microsoft.ContainerService/managedClusters

Storage Resources

  • Storage Accounts: Microsoft.Storage/storageAccounts
  • Blob Storage: Microsoft.Storage/storageAccounts/blobServices
  • File Storage: Microsoft.Storage/storageAccounts/fileServices
  • Queue Storage: Microsoft.Storage/storageAccounts/queueServices
  • Table Storage: Microsoft.Storage/StorageAccounts/tableServices

Database Resources

  • SQL Database: Microsoft.Sql/servers/databases
  • SQL Managed Instance: Microsoft.Sql/managedInstances
  • Cosmos DB: Microsoft.DocumentDB/databaseAccounts
  • MySQL Database: Microsoft.DBforMySQL/servers
  • PostgreSQL Database: Microsoft.DBforPostgreSQL/servers
  • Redis Cache: Microsoft.Cache/redis

Network Resources

  • Load Balancer: Microsoft.Network/loadBalancers
  • Application Gateway: Microsoft.Network/applicationGateways
  • Virtual Network Gateway: Microsoft.Network/virtualNetworkGateways
  • ExpressRoute Circuit: Microsoft.Network/expressRouteCircuits
  • Public IP Address: Microsoft.Network/publicIPAddresses
  • Network Interface: Microsoft.Network/networkInterfaces

Integration Resources

  • Service Bus: Microsoft.ServiceBus/namespaces
  • Event Hub: Microsoft.EventHub/namespaces
  • Event Grid: Microsoft.EventGrid/topics
  • Logic Apps: Microsoft.Logic/workflows

Analytics Resources

  • Data Factory: Microsoft.DataFactory/factories
  • Stream Analytics: Microsoft.StreamAnalytics/streamingjobs
  • Synapse Analytics: Microsoft.Synapse/workspaces

Monitoring Resources

  • Application Insights: Microsoft.Insights/components
  • Log Analytics Workspace: Microsoft.OperationalInsights/workspaces

For a complete list of supported resources and their available metrics, refer to Azure Monitor supported metrics documentation.

Azure Configuration

Create Service Principal

Azure Monitor input plugin requires authentication via service principal.

  • Register an application in Azure Active Directory
  • Create a client secret
  • Assign the Monitoring Reader role to the application at the subscription or resource group level

Required information: - Tenant ID - Client ID - Client Secret - Subscription ID

Telegraf Configuration

The primary configuration file is /etc/telegraf/telegraf.conf.

Basic Configuration


[agent]
  interval = "10m"
  flush_interval = "10m"
  metric_buffer_limit = 10000
  debug = false
  quiet = true


[[inputs.azure_monitor]]
  tenant_id = "<your-tenant-id>"
  client_id = "<your-client-id>"
  client_secret = "<your-client-secret>"
  subscription_id = "<your-subscription-id>"


[[inputs.azure_monitor.resource_target]]
    resource_id = "resourceGroups/<resource-group>/providers/Microsoft.Compute/virtualMachines/<vm-name>"


metrics = [
  "Percentage CPU",
  "Available Memory Bytes",
  "Network In Total",
  "Network Out Total",
  "Disk Read Bytes",
  "Disk Write Bytes",
  "Disk Read Operations/Sec",
  "Disk Write Operations/Sec",
  "OS Disk Queue Depth",
  "Data Disk Queue Depth"
]

aggregations = ["Average", "Maximum", "Minimum"]




[[outputs.http]]
  url = "https://gw.netcrunch.io/tm/v1/SRV-001@sensor01@node100/update"
  method = "POST"
  data_format = "json"
  content_encoding = "identity"
  [outputs.http.headers]
    Content-Type = "application/json"
  timeout = "30s"

Configuration Parameters

Agent Section:

  • interval - Metric collection frequency
  • flush_interval - How often data is sent to outputs
  • metric_buffer_limit - Maximum number of unwritten metrics per output
  • debug - Enable detailed logging
  • quiet - Suppress non-error messages

Azure Monitor Input:

  • tenant_id - Azure tenant identifier
  • client_id - Service principal client identifier
  • client_secret - Service principal secret
  • subscription_id - Azure subscription identifier
  • resource_id - Full Azure resource identifier
  • metrics - List of metrics to collect
  • aggregations - Data aggregation methods (Average, Maximum, Minimum, Total, Count)

HTTP Output:

  • url - NetCrunch Telemetry Node endpoint
  • method - HTTP method (POST)
  • data_format - Output format (JSON)
  • content_encoding - Encoding type
  • timeout - Request timeout
  • headers - HTTP headers including content type

Collected Metrics

The Azure Monitor input plugin collects the following VM metrics:

CPU Metrics

  • Percentage CPU - CPU utilization percentage

Memory Metrics

  • Available Memory Bytes - Available physical memory in bytes

Network Metrics

  • Network In Total - Total bytes received on all network interfaces
  • Network Out Total - Total bytes transmitted on all network interfaces

Disk I/O Metrics

  • Disk Read Bytes - Bytes read from disk per second
  • Disk Write Bytes - Bytes written to disk per second
  • Disk Read Operations/Sec - Read IOPS
  • Disk Write Operations/Sec - Write IOPS

Disk Queue Metrics

  • OS Disk Queue Depth - Operating system disk queue depth
  • Data Disk Queue Depth - Data disk queue depth

Configuration Examples for Other Azure Resources

Azure Storage Account


[[inputs.azure_monitor]]
  tenant_id = "<your-tenant-id>"
  client_id = "<your-client-id>"
  client_secret = "<your-client-secret>"
  subscription_id = "<your-subscription-id>"


[[inputs.azure_monitor.resource_target]]
    resource_id = "resourceGroups/<rg-name>/providers/Microsoft.Storage/storageAccounts/<storage-account>"
    metrics = [
      "UsedCapacity",
      "Transactions",
      "Ingress",
      "Egress",
      "SuccessServerLatency",
      "SuccessE2ELatency",
      "Availability"
    ]
    aggregations = ["Average", "Total"]


[[outputs.http]]
  url = "https://gw.netcrunch.io/tm/v1/SRV-001@sensor02@node101/update"
  method = "POST"
  data_format = "json"
  content_encoding = "identity"
  [outputs.http.headers]
    Content-Type = "application/json"

Azure SQL Database


[[inputs.azure_monitor]]
  tenant_id = "<your-tenant-id>"
  client_id = "<your-client-id>"
  client_secret = "<your-client-secret>"
  subscription_id = "<your-subscription-id>"


[[inputs.azure_monitor.resource_target]]
    resource_id = "resourceGroups/<rg-name>/providers/Microsoft.Sql/servers/<server>/databases/<database>"
    metrics = [
      "cpu_percent",
      "physical_data_read_percent",
      "log_write_percent",
      "dtu_consumption_percent",
      "storage_percent",
      "connection_successful",
      "connection_failed",
      "blocked_by_firewall"
    ]
    aggregations = ["Average", "Maximum"]


[[outputs.http]]
  url = "https://gw.netcrunch.io/tm/v1/SRV-001@sensor03@node102/update"
  method = "POST"
  data_format = "json"
  content_encoding = "identity"
  [outputs.http.headers]
    Content-Type = "application/json"

Azure Kubernetes Service


[[inputs.azure_monitor]]
  tenant_id = "<your-tenant-id>"
  client_id = "<your-client-id>"
  client_secret = "<your-client-secret>"
  subscription_id = "<your-subscription-id>"


[[inputs.azure_monitor.resource_target]]
    resource_id = "resourceGroups/<rg-name>/providers/Microsoft.ContainerService/managedClusters/<cluster-name>"
    metrics = [
      "node_cpu_usage_percentage",
      "node_memory_working_set_percentage",
      "node_disk_usage_percentage",
      "node_network_in_bytes",
      "node_network_out_bytes"
    ]
    aggregations = ["Average"]


[[outputs.http]]
  url = "https://gw.netcrunch.io/tm/v1/SRV-001@sensor04@node103/update"
  method = "POST"
  data_format = "json"
  content_encoding = "identity"
  [outputs.http.headers]
    Content-Type = "application/json"

Monitoring All VMs in Resource Group


[[inputs.azure_monitor]]
  tenant_id = "<your-tenant-id>"
  client_id = "<your-client-id>"
  client_secret = "<your-client-secret>"
  subscription_id = "<your-subscription-id>"


[[inputs.azure_monitor.resource_group_target]]
    resource_group = "production-rg"


[[inputs.azure_monitor.resource_group_target.resource]]
  resource_type = "Microsoft.Compute/virtualMachines"
  metrics = ["Percentage CPU", "Available Memory Bytes", "Network In Total", "Network Out Total"]
  aggregations = ["Average", "Maximum"]




[[outputs.http]]
  url = "https://gw.netcrunch.io/tm/v1/SRV-001@sensor05@node104/update"
  method = "POST"
  data_format = "json"
  content_encoding = "identity"
  [outputs.http.headers]
    Content-Type = "application/json"

Advanced Configuration

Multiple Virtual Machines

Monitor multiple Azure VMs:


[[inputs.azure_monitor]]
  tenant_id = "<your-tenant-id>"
  client_id = "<your-client-id>"
  client_secret = "<your-client-secret>"
  subscription_id = "<your-subscription-id>"


[[inputs.azure_monitor.resource_target]]
    resource_id = "resourceGroups/rg-prod/providers/Microsoft.Compute/virtualMachines/vm-web-01"
    metrics = ["Percentage CPU", "Available Memory Bytes"]
    aggregations = ["Average"]


[[inputs.azure_monitor.resource_target]]
    resource_id = "resourceGroups/rg-prod/providers/Microsoft.Compute/virtualMachines/vm-db-01"
    metrics = ["Percentage CPU", "Available Memory Bytes", "Disk Read Bytes", "Disk Write Bytes"]
    aggregations = ["Average", "Maximum"]

Multiple Output Targets

Send data to multiple NetCrunch instances:


[[outputs.http]]
  url = "https://gw.netcrunch.io/tm/v1/SRV-001@sensor01@node100/update"
  method = "POST"
  data_format = "json"
  content_encoding = "identity"
  [outputs.http.headers]
    Content-Type = "application/json"
  timeout = "30s"


[[outputs.http]]
  url = "https://gw.netcrunch.io/tm/v1/SRV-002@sensor02@node200/update"
  method = "POST"
  data_format = "json"
  content_encoding = "identity"
  [outputs.http.headers]
    Content-Type = "application/json"
  timeout = "30s"

Resource Usage Optimization

For large-scale deployments:


[agent]
  interval = "15m"
  flush_interval = "15m"
  metric_buffer_limit = 5000
  debug = false

Longer intervals reduce Azure API calls and network traffic.

Local System Metrics

Telegraf can simultaneously collect local system metrics from the host running the agent.

Combined Configuration Example


[agent]
  interval = "60s"
  flush_interval = "60s"


# Azure VM metrics
[[inputs.azure_monitor]]
  tenant_id = "<your-tenant-id>"
  client_id = "<your-client-id>"
  client_secret = "<your-client-secret>"
  subscription_id = "<your-subscription-id>"


[[inputs.azure_monitor.resource_target]]
    resource_id = "resourceGroups/rg-prod/providers/Microsoft.Compute/virtualMachines/vm-01"
    metrics = ["Percentage CPU", "Available Memory Bytes"]
    aggregations = ["Average"]


# Local system metrics
[[inputs.cpu]]
  percpu = true
  totalcpu = true


[[inputs.mem]]


[[inputs.disk]]
  ignore_fs = ["tmpfs", "devtmpfs", "devfs"]


[[inputs.diskio]]


[[inputs.net]]


# Send to NetCrunch
[[outputs.http]]
  url = "https://gw.netcrunch.io/tm/v1/SRV-001@sensor01@node100/update"
  method = "POST"
  data_format = "json"
  content_encoding = "identity"
  [outputs.http.headers]
    Content-Type = "application/json"

This configuration collects both Azure VM metrics and local system metrics from the Telegraf host.

Use Cases

Cloud Infrastructure Monitoring

  • Monitor Azure resources across compute, storage, database, and network services without requiring direct network connectivity from on-premises NetCrunch servers to Azure.

Hybrid Cloud Environments

  • Collect metrics from both Azure resources and on-premises systems using a single Telegraf instance deployed in a hybrid network.

Multi-Subscription Monitoring

  • Deploy Telegraf agents to monitor resources across multiple Azure subscriptions using different service principals with appropriate permissions.

Resource Group Monitoring

  • Monitor all resources of a specific type within a resource group, automatically including new resources as they are created.

Multi-Resource Type Monitoring

  • Combine monitoring of different Azure resource types (VMs, databases, storage, networking) in a single Telegraf configuration for centralized visibility.

Cost-Effective Monitoring

  • Reduce Azure API costs by controlling collection intervals and selecting only required metrics. The Azure Monitor API has a read limit of 12,000 requests per hour.

Summary

Telegraf provides native integration with Azure Monitor for collecting metrics from any Azure resource type. The plugin supports three collection methods: resource target, resource group target, and subscription target. Data is forwarded to NetCrunch Telemetry Nodes for centralized monitoring and alerting.

This approach enables comprehensive cloud monitoring without requiring VPN connections or direct network access from NetCrunch to Azure. The plugin can monitor compute resources, storage accounts, databases, network services, and other Azure resources through a unified configuration.

Key capabilities: - Native Azure Monitor API integration - Support for all Azure resource types with exposed metrics - Three collection methods: resource, resource group, and subscription level - Configurable collection intervals to manage API rate limits - Centralized monitoring through NetCrunch Telemetry Nodes

azureazure monitorcloud infrastructurecloud metricshybrid cloudresource groupservice principaltelegraftelemetryvm monitoring