Contents PDF

Monitoring External Sources

How NetCrunch monitors data originating outside built-in collectors using telemetry, scripts, files, and external APIs.

data-ingestiondata-parserseventsexternal-sourceshttplogsmonitoringrestscriptstelemetry

Purpose and Scope

Monitoring External Sources describes how NetCrunch collects and monitors data that is not produced by built-in collectors such as SNMP, WMI, or native device sensors.

An external source is any system, service, or interface that provides monitoring data but requires custom retrieval or transformation before it can be used by NetCrunch.

Examples include:

External applications exporting metrics
Custom scripts and command-line tools
Files generated by other systems
REST and HTTP-based APIs
Third-party monitoring agents or exporters

This chapter explains how external data becomes NetCrunch monitoring data, regardless of its origin.

Conceptual Model

Monitoring external sources always involves two distinct concerns:

Where data comes from – retrieval or reception method
What the data represents – counters and status objects

These concerns are intentionally separated.

Counters and statuses describe measurement identity and state, not location or source
Scope is resolved by the sensor and node context in which the data is produced

External Monitoring Methods

NetCrunch supports several complementary methods for monitoring external sources. Each method addresses a different integration scenario.

Telemetry (Receive Data)

Telemetry is a push-based model where external systems actively send data to NetCrunch.

Use telemetry when:

You control the data producer
Near-real-time delivery is required
Polling is not suitable or possible

Key components:

Telemetry Node
Data Receiver or Data Sensor
Local or cloud endpoints

Preferred data format:

NetCrunch Native Data Formats

Other formats can be accepted if transformed using Custom Data Parsers.

Scripts (Execute and Collect)

Script-based monitoring executes code and collects its output.

Use scripts when:

Data is only available via CLI or shell
SSH, PowerShell, or local execution is required
Logic must be executed close to the target system

Supported scenarios:

Remote SSH scripts
PowerShell scripts
Local scripts on the NetCrunch Server

Script output can be provided in:

NetCrunch Native Data Formats
Arbitrary formats parsed by Custom Data Parsers

Data Retrieval (Pull External Data)

Data retrieval is a polling-based model in which NetCrunch retrieves external data via standard protocols.

Use this method when:

Data is exposed as files or endpoints
Periodic polling is acceptable
Direct execution is not required

Supported protocols include:

HTTP and HTTPS
FTP, SFTP, TFTP
SSH command output
Windows or SMB file access

Retrieved data can be:

Directly interpreted using Native Data Formats
Transformed using Custom Data Parsers

REST and HTTP APIs

Many modern systems expose monitoring data through REST or HTTP APIs.

NetCrunch supports:

Custom HTTP methods
Query parameters
Headers and authentication
Cookies and session handling

REST APIs are typically used together with:

Native Data Formats preferred
JSON, XML, or text parsed by Custom Data Parsers

External Event Sources

In addition to metric and status collection, NetCrunch can receive external events that represent discrete occurrences rather than continuous measurements.

External event sources are typically used for:

Log-based alerting
Security and audit events
Infrastructure notifications
Application and platform events

Supported external event sources include:

Text log files
Windows Event Log
Syslog
SNMP traps and notifications, including SNMP v3
Webhook and HTTP-based messages
Telemetry events delivered via OTLP (OpenTelemetry)

External events are ingested, classified, and correlated with nodes and services, allowing them to:

Trigger alerts
Contribute to node and service status
Be stored, filtered, and searched

Event ingestion focuses on event semantics and context, not on metric extraction.

Data Formats and Transformation

Regardless of the monitoring method used, external data must be converted into NetCrunch counters and status objects.

Native Data Formats

NetCrunch Native Data Formats provide a direct and explicit way to define counters and statuses.

They are:

Stable and version-safe
Independent of the ingestion method
Preferred whenever possible

If an external source can produce data in Native Data Formats, no parsing is required.

Custom Data Parsers

Custom Data Parsers are used when incoming data cannot be provided in Native Data Formats.

They allow:

Extracting values from XML, JSON, HTML, or text
Transforming arbitrary structures into counters and statuses
Applying logic using declarative selectors or scripts

Custom Data Parsers are a transformation layer, not a data source.

They are typically used together with:

Data Retrieval
Script-based monitoring
REST and HTTP APIs

Choosing the Right Approach

The following guidelines can help select the appropriate monitoring method:

Can the external system push data
→ Use Telemetry
Can NetCrunch execute code to obtain data
→ Use Scripts
Is data exposed via files or network endpoints
→ Use Data Retrieval
Can data be produced in Native Data Formats
→ Prefer Native Data Formats
Is the data format arbitrary or legacy
→ Use Custom Data Parsers
Are you ingesting discrete notifications or logs
→ Use External Event Sources

Summary

Monitoring External Sources allows NetCrunch to integrate with virtually any system or data producer.

Key principles:

The monitoring method defines how data is obtained
Data formats define how measurements are represented
Counters and statuses describe identity, not source
Events represent discrete occurrences, not metrics
Scope is resolved by sensor and node context
Native Data Formats are preferred
Custom Data Parsers enable compatibility with non-native formats

This separation ensures flexibility, scalability, and long-term maintainability of external monitoring integrations.