PDF

Linux Sysctl Filesystem Monitoring via Telegraf in NetCrunch

This topic explains how to monitor Linux kernel filesystem parameters using Telegraf and send collected metrics to NetCrunch Telemetry Nodes. The Linux Sysctl Filesystem input plugin reads values from the proc sys fs directory and forwards them to NetCrunch using the HTTP output plugin.

Overview

Telegraf can collect Linux kernel filesystem parameters using the Linux Sysctl Filesystem input plugin. The plugin reads metrics from the proc sys fs directory, providing visibility into kernel level file system resource usage. The plugin requires no configuration and automatically collects all available metrics. Data is forwarded to a NetCrunch Telemetry Node endpoint using HTTP POST.

How NetCrunch Supports Linux Sysctl Metrics

NetCrunch receives incoming sysctl filesystem metrics through a Telemetry Node endpoint. Telemetry Nodes accept JSON formatted data and store collected values as counters or alert statuses.

The endpoint, its URL shape and how it is authorized are described once in Monitoring with Telegraf. Everything below assumes a Telemetry Node already exists — see Telemetry Node.

Data Flow

  1. Kernel metrics are read from proc sys fs by Telegraf.
  2. Telegraf collects the kernel level filesystem statistics.
  3. Telegraf sends JSON metrics to the NetCrunch Telemetry Node endpoint.
  4. NetCrunch processes and stores the collected counters and status values.

Telegraf Configuration

Primary configuration file location:

  • Linux path: /etc/telegraf/telegraf.conf

Basic Configuration

[agent] interval = "10s" flush_interval = "10s" metric_buffer_limit = 10000 debug = false

[[inputs.linux_sysctl_fs]]

[[outputs.http]] url = "https://gw.netcrunch.io/tm/v1/SRV-001@sensor01@node100/update" timeout = "5s" method = "POST" data_format = "json"

Configuration Parameters

Agent Section

  • interval sets metric collection frequency
  • flush_interval sets data submission frequency
  • metric_buffer_limit sets the internal buffer size
  • debug enables detailed output

Linux Sysctl Filesystem Input

  • Requires no configuration
  • Automatically collects all filesystem sysctl metrics

HTTP Output

  • url sets the NetCrunch Telemetry Node endpoint
  • timeout defines request timeout
  • method specifies HTTP POST
  • data_format sets JSON output

Collected Metrics

The Linux Sysctl Filesystem plugin collects kernel level filesystem metrics from proc sys fs.

File Descriptor Metrics

file nr
Number of allocated file handles.

file max
Maximum number of file handles Linux can allocate.

Asynchronous IO Metrics

aio nr
Current number of asynchronous IO requests.

aio max nr
Maximum number of concurrent asynchronous IO requests allowed.

Dentry Cache Metrics

dentry nr
Number of directory cache entries in use.

dentry unused nr
Number of unused directory cache entries.

dentry age limit
Age limit for directory cache entries.

dentry want pages
Indicates memory pressure related to the dentry cache.

Inode Metrics

inode nr
Allocated inodes.

inode free nr
Free inodes.

inode preshrink nr
Inodes that will be freed in the next shrink cycle.

Disk Quota Metrics

dquot nr
Allocated disk quota structures.

dquot max
Maximum number of disk quota structures.

Superblock Metrics

super nr
Allocated superblocks.

super max
Maximum number of superblocks.

Metric Interpretation

File Handle Monitoring

Monitor file nr relative to file max to identify file handle exhaustion risks. If file nr becomes close to file max, new file creation or connection establishment may fail.

Dentry Cache Monitoring

  • dentry nr indicates how many entries are active
  • dentry unused nr indicates reusable entries
  • high dentry want pages indicates memory pressure on directory cache

Inode Monitoring

  • inode nr indicates allocated inodes
  • inode free nr indicates available inodes
  • low free inodes can block file creation

Asynchronous IO Monitoring

  • aio nr indicates active asynchronous IO load
  • compare against aio max nr to detect limits

Multiple Output Targets

Metrics can be delivered to more than one NetCrunch instance.

[agent] interval = "10s" flush_interval = "10s"

[[inputs.linux_sysctl_fs]]

[[outputs.http]] url = "https://gw.netcrunch.io/tm/v1/SRV-001@sensor01@node100/update" timeout = "5s" method = "POST" data_format = "json"

[[outputs.http]] url = "https://gw.netcrunch.io/tm/v1/SRV-002@sensor02@node200/update" timeout = "5s" method = "POST" data_format = "json"

[[outputs.http]] url = "https://gw.netcrunch.io/tm/v1/SRV-003@sensor03@node300/update" timeout = "5s" method = "POST" data_format = "json"

Use Cases

File Descriptor Capacity Monitoring

Track file handle usage to prevent service outages due to handle exhaustion.

Filesystem Cache Health Monitoring

Monitor dentry and inode metrics to detect memory pressure conditions.

High Load Application Monitoring

Track asynchronous IO metrics for high throughput applications like databases and servers.

Long Term Capacity Planning

Review trends in inode and file descriptor usage over time.

Multi System Comparison

Compare filesystem resource usage between several Linux systems.

Advanced Configuration

Lower Collection Frequency

Suitable for stable systems.

[agent] interval = "60s" flush_interval = "60s"

High Frequency Monitoring

Suitable for fast changing systems.

[agent] interval = "5s" flush_interval = "5s"

Combining Sysctl Metrics with Other Inputs

[agent] interval = "10s" flush_interval = "10s"

[[inputs.linux_sysctl_fs]]

[[inputs.cpu]] percpu = false totalcpu = true

[[inputs.mem]]

[[inputs.disk]] ignore_fs = ["tmpfs", "devtmpfs"]

[[inputs.diskio]]

[[outputs.http]] url = "https://gw.netcrunch.io/tm/v1/SRV-001@sensor01@node100/update" timeout = "5s" method = "POST" data_format = "json"

Summary

The Linux Sysctl Filesystem input plugin provides kernel level filesystem metrics with zero configuration required. Telegraf collects file descriptor metrics, inode counts, dentry cache details, asynchronous IO statistics, superblock metrics, quota metrics, and forwards them to NetCrunch Telemetry Nodes where they can be monitored, charted, and alerted on.

aiodentryfile descriptorfilesysteminodekernellinuxprocpushsysctltelegraftelemetry node