PDF

MQTT Telemetry via Telegraf in NetCrunch

This topic explains how to collect system metrics published via MQTT, process them using Telegraf, and forward them to a NetCrunch Telemetry Node endpoint using JSON-based telemetry data.

Overview

MQTT is a lightweight publish/subscribe messaging protocol commonly used in IoT and distributed systems. Telegraf can subscribe to MQTT topics, parse incoming JSON messages, and forward them to a NetCrunch Telemetry Node via the HTTP output plugin.

How NetCrunch Supports MQTT Telemetry

NetCrunch receives MQTT-based telemetry through Telemetry Nodes, which accept data in JSON format and anchor incoming metrics or status values. Telemetry Nodes do not require network discovery and can receive data from distributed or isolated systems.

The endpoint, its URL shape and how it is authorized are described once in Monitoring with Telegraf. Everything below assumes a Telemetry Node already exists — see Telemetry Node.

Data Flow

  1. Metric Generation - Systems generate metrics using scripts, applications, or monitoring agents.
  2. MQTT Publishing - Metrics are published as JSON messages to MQTT broker topics.
  3. Telegraf Subscription - Telegraf subscribes to designated MQTT topics.
  4. Data Forwarding - Telegraf sends received messages to the NetCrunch Telemetry Node endpoint.
  5. NetCrunch Processing - Incoming data is stored as counters or alert statuses.

Telegraf Configuration

The primary configuration file is typically located at:

  • Linux: /etc/telegraf/telegraf.conf
  • Windows: C:\Program Files\Telegraf\telegraf.conf

Basic Configuration

MQTT messages must be valid JSON for Telegraf to parse them correctly.


[agent]
  interval = "30s"
  flush_interval = "30s"
  debug = false
  quiet = true


[[inputs.mqtt_consumer]]
  servers = ["tcp://localhost:1883"]
  topics = [
    "linux/kernel/errors",
    "linux/fd/usage",
    "linux/systemd/failed",
    "linux/packages/health",
    "linux/security/entropy"
  ]
  data_format = "json"
  json_name_key = "measurement_name"
  tag_keys = ["hostname"]


[[outputs.http]]
  url = "https://gw.netcrunch.io/tm/v1/SRV-001@sensor01@node100/update"
  method = "POST"
  data_format = "json"
  content_encoding = "identity"
  [outputs.http.headers]
    Content-Type = "application/json"

Configuration Parameters

Agent Section

  • interval - How frequently data is collected
  • flush_interval - How often data is forwarded
  • debug - Enables verbose debugging information
  • quiet - Suppresses non-error output

MQTT Consumer Input

  • servers - MQTT broker address
  • topics - Subscribed MQTT topics
  • data_format - Expected format (json)
  • json_name_key - JSON field used as measurement name
  • tag_keys - Fields extracted as tags

HTTP Output

  • url - NetCrunch Telemetry Node endpoint
  • method - Must be POST
  • data_format - JSON payload format
  • headers - HTTP headers

MQTT Message Format

Messages published to MQTT topics must be JSON objects containing relevant metadata and metric fields.

Required Fields

  • timestamp - ISO 8601 timestamp
  • hostname - System identifier
  • Metric fields- Numeric or string values representing counters or statuses

Example Messages

Kernel Errors


{
  "timestamp": "2025-10-29T15:09:08+01:00",
  "hostname": "server01.example.com",
  "kernel_errors_5min": 0
}

File Descriptor Usage


{
  "timestamp": "2025-10-29T15:05:12+01:00",
  "hostname": "server01.example.com",
  "total_fd_count": 1471
}

Package Health Status


{
  "timestamp": "2025-10-29T15:07:54+01:00",
  "hostname": "server01.example.com",
  "upgradable_packages": 1,
  "broken_packages": 0
}

System Entropy Level


{
  "timestamp": "2025-10-29T15:09:08+01:00",
  "hostname": "server01.example.com",
  "entropy_available": 256
}

Failed Systemd Units


{
  "timestamp": "2025-10-29T15:06:41+01:00",
  "hostname": "server01.example.com",
  "failed_units_count": 0
}

Use Cases

IoT Device Monitoring

Devices publish telemetry to an MQTT broker. Telegraf consumes the messages and sends them to NetCrunch for visualization and alerting.

Distributed System Metrics

Systems across remote networks publish metrics to centralized MQTT brokers. NetCrunch receives telemetry without requiring direct connectivity.

Custom Application Telemetry

Applications publish structured metrics to MQTT topics, avoiding the need to implement HTTP endpoints.

Edge Computing

Edge devices publish telemetry locally into an MQTT broker. Telegraf aggregates and forwards it to NetCrunch.

Multi-Tenant Monitoring

Topic structures and tag extraction allow routing metrics to separate Telemetry Nodes based on tenant or subsystem.

Summary

Using MQTT with Telegraf and NetCrunch provides a scalable and flexible telemetry pipeline. Publishers send JSON metrics to an MQTT broker, Telegraf subscribes to relevant topics, and telemetry is forwarded to NetCrunch using REST endpoints. This model supports IoT, distributed architectures, and custom monitoring scenarios without requiring SNMP or WMI.

brokeredgeiotjsonmqttmqtt_consumerpublish subscribepushtelegraftelemetry node