Starlink metrics: offline vs online availability

Audience: Platform engineers and SREs evaluating on-premise Agent collection vs cloud telemetry API.

Source: Validated against a real rev4_catapult_proto1 dish (firmware 2025.12.04) running indoors with no satellite lock. Collection behavior below reflects the on-premise Agent check.

At a glance

The on-premise Agent always collects all 67 metrics each cycle over the local LAN, regardless of satellite lock. Without a connection, many values reflect a degraded or “no link” state—but hardware alerts, alignment, ready states, and outage cause codes still populate. That is the core offline advantage over the cloud API.

All 67 metrics collected without satellite lock

The Agent polls the terminal every 15 seconds on the local network. All 67 curated metrics are emitted each cycle—no internet or satellite lock required for collection. Interpretation differs: some series carry full hardware state offline; others report sentinel values (-1, 0, 1.0 loss) until the link is up.

Meaningful offline — hardware & state

Meaningful offline — alert flags

Collected offline — values reflect no satellite link

These metrics are collected but their values indicate the dish isn't connected:

Metrics that require satellite lock for useful values

These are part of the 67-metric set but only produce operational values when the terminal has constellation contact:

Cloud API vs On-Premise: What You Get When

Key takeaway — maritime and remote deployments

The on-premise Agent delivers 67 curated metrics from 200+ analyzed terminal fields—not the raw device dump. During cloud API blind spots it still collects locally:

• Is it a thermal issue? (alerts.thermal_throttle, alerts.thermal_shutdown)
• Is it obstructed? (alerts.mast_not_near_vertical, obstruction metrics)
• Is it a hardware failure? (alerts.motors_stuck, alerts.low_motor_current)
• Is it a power issue? (alerts.power_supply_thermal_throttle)
• Is it water damage? (alerts.dish_water_detected, alerts.router_water_detected)
• Is it a network issue? (alerts.no_ethernet_link, alerts.slow_ethernet_speeds)
• Which subsystems are ready? (ready states show exactly what's working and what isn't)

This data is buffered locally by the Datadog Agent and uploaded to Datadog cloud as soon as connectivity returns — giving the shore-side ops team a complete picture of what happened during the outage.

Data Delivery: Real-Time Collection vs Buffered Upload

Important distinction for maritime/aviation deployments:

Collection (Agent → Dish): Always Real-Time

• The Agent collects from the dish every 15 seconds over the ship's LAN
• This works regardless of satellite/internet status — it's local network traffic only
• Data is immediately available to anything running locally on the ship

Upload (Agent → Datadog Cloud): Buffered When Offline

• The Agent tries to send metrics to datadoghq.com over the internet every 15 seconds
• If the satellite link is down, metrics queue in the Agent's local buffer
• When connectivity returns, the buffer flushes and Datadog receives all historical data with correct timestamps
• The dashboard timeline fills in retroactively — shore-side ops can see exactly what happened during the gap

Buffer Limits

• Default Agent buffer: ~4 hours of metric data
• Configurable via forwarder_storage_max_size_in_bytes in datadog.yaml
• For extended offline periods (days), increase the buffer or implement local persistent storage

What Shore-Side Ops Sees

For Extended Offline Scenarios (days without connectivity)

If a customer needs guaranteed zero data loss during multi-day outages:

• Increase Agent buffer size (forwarder_storage_max_size_in_bytes)
• Or implement local persistent storage on the ship with a custom forwarder
• For typical maritime Starlink usage (intermittent gaps of minutes to hours), the default 4-hour buffer is sufficient

Bandwidth Impact: Metrics Payload Size

Concern: Cruise lines report that Starlink throttles when packets exceed 2MB during weak signal. Does the integration's outbound traffic risk hitting this limit?

Answer: No. The metrics payload is negligible.

Measured Payload Size (67 metrics with full maritime tags)

The Datadog Agent compresses all outbound payloads with gzip/zlib before transmission. Metric data compresses extremely well (~95% reduction) because of repeated tag strings and similar structure across metrics.

Scaled to Fleet Deployments

Values below assume the default 15 second min_collection_interval. Doubling the interval (e.g. 30s) halves daily/hourly volume; payload per cycle stays ~1.1 KB per dish.

Context

• A single 1080p video stream uses ~5 Mbps = ~37 MB per minute
• Our integration at 50 dishes uses 220 KB per minute — 170x less than one video stream
• The 2MB packet size concern applies to individual HTTP requests, not cumulative bandwidth
• Each Agent payload is well under 100 KB compressed even at 50 dishes (the Agent batches intelligently)

Conclusion

The integration's bandwidth footprint is a rounding error on any maritime satellite link. Even during severe throttling (e.g. 1 Mbps available), the metrics payload would consume less than 0.2% of available bandwidth. This is not a concern for deployment.

How We Know It's Compressed (verified)

The Datadog Agent compresses all outbound payloads by default. This is confirmed in the Agent's own configuration file (datadog.yaml):

## @param use_compression - boolean - optional - default: true
## compresses logs before sending them.
# use_compression: true

## @param compression_level - integer - optional - default: 6
## The compression_level parameter accepts values from 0 (no compression)
## to 9 (maximum compression but higher resource usage).
# compression_level: 6

• Compression is enabled by default — no configuration needed
• Default compression level is 6 (out of 9)
• This applies to both logs and metrics payloads
• The Agent sends Content-Encoding: deflate headers on all outbound HTTP requests
• The 1.1 KB compressed figure is what actually traverses the satellite link — the raw 22.5 KB never leaves the machine uncompressed

Per-Metric Size Breakdown

All metrics are nearly the same size (287–325 bytes raw) because the payload structure is identical—only the metric name and value differ. Tags dominate byte count and are shared across all 67 metrics per terminal.

Key insight: Reducing metric count barely reduces bandwidth—the shared tags compress efficiently. Disabling whole metric groups via collect_* toggles is the right lever for constrained links, not trimming individual metrics from the 67-metric set. See the Configuration reference.

How Satellite Link Issues Affect Our Integration's Data Delivery

These are the operational constraints cruise lines face with Starlink. The question is: does our integration's outbound traffic (Agent sending metrics to Datadog cloud) get affected by these same issues?

Bottom line: Yes, our data travels over the same satellite link. But our payload is so small that these issues don't meaningfully affect us.

1. Satellite Handover (brief dropouts every few minutes)

What happens: Starlink satellites move fast. The dish switches from one satellite to another every few minutes, causing micro-second to millisecond connection blips.

Impact on our integration: None in practice. The Agent sends metrics over HTTPS (which runs on TCP). TCP automatically retransmits any packet that gets lost during a handover blip — the application layer never even sees it. A micro-second dropout is far too short to trigger a connection timeout. The Agent wouldn't know a handover happened.

Worst case: If a handover happens at the exact moment the Agent is mid-transmission, TCP retransmits the 1.1 KB payload in the next fraction of a second. No data loss, no user-visible impact.

2. Rain Fade (bandwidth drops sharply)

What happens: Heavy rain, snow, or dense clouds weaken the satellite signal. Available bandwidth drops significantly — sometimes from 200 Mbps down to single-digit Mbps.

Impact on our integration: Negligible. Our compressed payload is 1.1 KB per dish per cycle. Even if bandwidth drops to 100 Kbps (extremely degraded), transmitting 1.1 KB takes less than 0.1 seconds. The 2MB packet size concern the cruise lines mentioned is about their application traffic — video calls, passenger internet, operational systems. Our monitoring payload is 170x smaller than a single video stream.

Analogy: If the satellite link is a highway that narrows from 6 lanes to 1 lane during a storm, our metrics are a bicycle. The cars (video, passenger internet) get stuck in traffic. The bicycle still gets through fine.

3. Line-of-Sight Obstructions (signal blocked entirely)

What happens: Physical objects (ship structures, cranes, other vessels) temporarily block the dish's view of the sky. The connection drops completely for seconds to minutes.

Impact on our integration: The Agent can't send data during the blockage — same as everything else on the ship. But:

• The Agent buffers metrics locally during the obstruction
• When line-of-sight returns (seconds to minutes later), the buffer flushes automatically
• Datadog receives the data with correct timestamps — the timeline fills in retroactively
• No data is lost unless the obstruction lasts longer than the buffer window (~4 hours)

The irony: Our integration actually measures this exact problem. The obstruction.fraction_obstructed and pop_ping_drop_rate metrics show the customer exactly when and how often obstructions occur. So after the blockage clears, the ops team can see it in the dashboard.

Summary Table

What We Monitor vs What Affects Us

This is the important distinction for the customer:

These satellite issues affect the customer's operational traffic (passenger internet, video, VoIP, POS systems). That's their pain point — that's why they want monitoring.

These satellite issues do NOT meaningfully affect our monitoring traffic because our payload is 1.1 KB compressed. We're monitoring the problem without contributing to it.

Our integration tells the customer:

• pop_ping_drop_rate → shows handover gaps and signal loss in real-time
• obstruction.fraction_obstructed → quantifies how much sky view is blocked
• alerts.lower_signal_than_predicted → indicates rain fade or atmospheric interference
• throughput.download_bps / upload_bps → shows actual available bandwidth
• outage.cause → tells them WHY connectivity dropped

We don't add to the bandwidth problem. We measure it.

Verification: What These Metrics Actually Report During an Outage

Tested against a real dish with no satellite connection (indoors, no sky view). This simulates a total connectivity loss scenario — the worst case for a ship.

Reliable During Total Connectivity Loss ✅

These metrics correctly indicate "something is wrong" even when the dish has zero satellite contact:

These are the metrics that reliably answer "is the dish connected?" and "why not?" regardless of the failure mode.

Only Useful During PARTIAL Degradation (not total outage) ⚠️

These metrics require the dish to have some satellite contact to produce meaningful values:

When these DO work: During rain fade or partial obstruction — the dish still has satellite contact but the link is degraded. That's when obstruction.fraction_obstructed shows a real percentage and lower_signal_than_predicted fires. These are the "bandwidth is dropping" scenarios, not the "connection is dead" scenarios.

How to Interpret the Data for Each Scenario

Honest Limitations

We should be transparent with customers:

• The integration cannot distinguish between "no obstruction" and "can't measure obstruction" when the dish is fully offline
• The lower_signal_than_predicted alert is a rain fade indicator only when the dish has partial connectivity
• For total outage diagnosis, rely on pop_ping_drop_rate, outage.cause, and the alert flags (thermal, motors, water, ethernet) rather than obstruction/signal metrics

Critical Clarification: Shore-Side Visibility During Total Outage

During a total satellite outage, the shore-side ops team cannot see anything in Datadog until connectivity returns.

Here's the timeline:

What this means practically:

• Shore-side can detect an outage by the absence of data (metrics stop arriving)
• Shore-side cannot diagnose the outage in real-time — they have to wait for connectivity to return
• Shore-side gets full post-incident visibility once the buffer flushes — they can see exactly what happened, when, and why
• The Agent does NOT lose data during the outage (up to ~4 hours of buffer)

The only way to get real-time shore-side visibility during a total outage:

The ship would need a backup communication path that doesn't depend on Starlink — for example:

• A secondary VSAT link (many cruise ships have these as backup)
• Cellular connection (only works near shore)
• Iridium/Inmarsat low-bandwidth satellite (expensive, very low data rate)

This is a network architecture decision for the customer, not something our integration controls. If the ship has a backup link, the Agent will use it automatically (it just needs any internet path to reach datadoghq.com).

Setting expectations with customers:

Be upfront: "During a total Starlink outage, you'll see a gap in your dashboard. When connectivity returns, the gap fills in automatically with full detail. The Agent never stops collecting — you just can't see it in real-time until the data can reach the cloud."

This is not a limitation of our integration — it's a fundamental constraint of monitoring remote sites over the same link being monitored. Every monitoring solution has this same limitation unless there's a redundant communication path.

Why On-Premise Agent vs Cloud API (Both Are Blind During Outage)

Neither approach gives shore-side real-time visibility during a total outage. The difference is what happens after:

The value of on-premise is post-incident forensics, not real-time outage visibility.

For a cruise line ops team doing root cause analysis: the cloud API tells them that an outage happened. The on-premise Agent tells them why it happened and what the dish was doing the entire time it was offline.