Monitoring the network: SNMP, logs and flows
Lesson 1 of 4
1Monitoring the network: SNMP, logs and flows
The big pictureIn the junior stage you learned to diagnose a fault after it happens. A real company has hundreds of devices, and nobody wants to hear about a fault from the boss. Monitoring makes the network talk: a link 90% full, a port with rising errors, a router running hot. The question is not whether you collect data, but which data, and how to turn it into an alert worth waking someone for.
What the picture shows: Devices send or are polled, data is stored as time series, charts show the trend, and an alert goes out only when something crosses a line that needs action.
Map of this part
The concepts in order, click any one to jump to it:
Two ways into the same idea
A network is like a patient in intensive care. SNMP is the nurse taking blood pressure every five minutes. Logs (syslog) are the notes: "3:12, port 7 went down". Flows (NetFlow) say who visited and how long they stayed. Streaming telemetry is a monitor sending the heartbeat every second. All of it is useless if the clocks in each room differ, which is why NTP sets every clock to one time.
SNMP: the manager polls the device on UDP 161 for OIDs from a MIB (such as the 64-bit ifHCInOctets counters), and the device sends a trap on UDP 162 when something happens. Use SNMPv3 (authentication and encryption with authPriv), not v2c with a plain-text community. syslog (RFC 5424) has eight levels from 0 Emergency to 7 Debug, on UDP 514 or TLS on 6514. NetFlow v9 / IPFIX (RFC 7011) exports flow records (source, destination, port, bytes) to see who consumes what. Streaming telemetry (gNMI over gRPC) pushes data every few seconds using YANG models instead of polling every five minutes. Billing and planning use the 95th percentile: drop the top 5% of readings and take the highest left.
Before You Start
🧰Diagnostic tools
You know ping, traceroute and the troubleshooting ladder.
🌐Modern networks
You met monitoring and automation in brief.
The Core Idea
1What do you measure?
Do not collect everything. Start with what answers "is the network healthy?". For every link and device, five indicators cover most faults:
| Indicator | What it reveals | From where | Typical alert line |
|---|---|---|---|
| Up/down state | A link or device down | SNMP trap and syslog | At once for core links |
| Utilisation % | A link filling up | SNMP counters | Above 80% for 15 minutes |
| Errors and discards | A bad cable or congestion | ifInErrors and ifOutDiscards | Any steady rise |
| Latency and loss | User experience | ping and IP SLA | Loss above 1% |
| CPU, memory, temperature | An overloaded device or a dead fan | SNMP | Above 85% sustained |
2SNMP: polling and traps
The manager asks each device every one to five minutes for counters that only go up, such as bytes in. Utilisation is worked out from the difference between two readings: (difference × 8) ÷ (seconds × speed). Use the 64-bit (HC) counters, because a 32-bit counter on a 10 Gbps link wraps to zero about every 3.4 seconds.
This is how every monitoring tool draws its utilisation chart. Try changing the speed to 100 Mbps.
speed = 1_000_000_000 # 1 Gbps link
interval = 300 # polled every 5 minutes
readings = [812_440_120_331, 824_470_350_995, 851_521_903_117, 885_559_100_004] # ifHCInOctets
for before, after in zip(readings, readings[1:]):
bits = (after - before) * 8
print(f"{bits / interval / 1e6:7.1f} Mbps -> {100 * bits / (interval * speed):5.1f}% used")
wrap32 = 2 ** 32 * 8 / 10e9
print(f"a 32-bit counter wraps every {wrap32:.1f} s on 10 Gbps")3Logs and time: syslog and NTP
Every device writes down its events, and you send them to a central server because the device forgets on reboot. Each message has a level from 0 to 7: the lower the number, the more serious. Without NTP every device keeps a different time, and you cannot say what happened first. Point every device at the same NTP servers and log in one time zone (UTC).
The number in < > packs the facility and level together: the level is the remainder after dividing by 8.
import re
from collections import Counter
LEVELS = ["emerg", "alert", "crit", "err", "warning", "notice", "info", "debug"]
log = """<187>Nov 2 03:12:44 sw-acc-3 %LINK-3-UPDOWN: Interface Gi1/0/7, changed state to down
<189>Nov 2 03:12:45 sw-acc-3 %LINEPROTO-5-UPDOWN: Line protocol on Gi1/0/7, changed state to down
<187>Nov 2 03:13:02 sw-acc-3 %LINK-3-UPDOWN: Interface Gi1/0/7, changed state to up
<187>Nov 2 03:13:09 sw-acc-3 %LINK-3-UPDOWN: Interface Gi1/0/7, changed state to down
<186>Nov 2 03:15:30 rtr-edge-1 %SYS-2-MALLOCFAIL: Memory allocation failed
<190>Nov 2 03:16:00 sw-core-1 %SYS-6-LOGGINGHOST_STARTSTOP: Logging to host started"""
counts, flaps = Counter(), Counter()
for line in log.splitlines():
pri = int(re.match(r"<(\d+)>", line).group(1))
level = LEVELS[pri % 8] # priority = facility * 8 + severity
counts[level] += 1
port = re.search(r"Interface (\S+), changed state to down", line)
if port:
flaps[port.group(1)] += 1
print(dict(counts))
for port, n in flaps.items():
if n >= 2:
print(f"flapping port {port}: down {n} times -> check the cable")4Flows, streaming telemetry and the baseline
SNMP tells you a link is full but not who is filling it. NetFlow/IPFIX answers: the router summarises every conversation (source, destination, port, bytes) and sends it to a collector. Streaming telemetry pushes counters every one to ten seconds, so you see short bursts a five-minute average hides. Before you set alert lines, collect two weeks and build a baseline: what is normal on a Sunday at 9?
The 95th percentile ignores one odd moment, which is why providers bill on it and engineers plan capacity with it.
samples = [42, 38, 51, 47, 300, 45, 49, 55, 61, 58, 44, 39, 47, 52, 60,
57, 49, 41, 46, 53] # Mbps, one reading every 5 minutes
ordered = sorted(samples)
p95 = ordered[int(len(ordered) * 0.95) - 1]
print("average:", sum(samples) / len(samples), "Mbps")
print("peak :", max(samples), "Mbps")
print("95th :", p95, "Mbps (the short 300 Mbps burst is ignored)")Worked example
The task: branch users complain the internet "drags" between 10 and 11 am. The SNMP dashboard says the link averages 55%. What do you do?
The answer: a five-minute average hides bursts, so turn on streaming telemetry or poll every 30 seconds for that hour to see the peaks. Then open NetFlow for the same window to see who consumes: if a cloud backup starts at 10:00, move it to the night or cap it with QoS. And check ifOutDiscards on the outbound port: a rise means the queue is filling and dropping packets.
Myth vs Reality
| Myth | Reality |
|---|---|
| The more data we collect, the better. | Data with no question is storage cost and noise. Collect what answers specific operational questions. |
| A 50% average means the link is comfortable. | It may be 100% full for scattered minutes. Look at peaks, the 95th percentile and dropped packets. |
| SNMP is old and finished. | It is still in almost every device. Streaming telemetry adds second-by-second detail; it does not replace it. |
Where you meet it at work
🖥️Network operations centre
Live screens of state and utilisation, with alerts going to whoever is on call.
💰Capacity planning
Six months of 95th percentile tells you when you need a bigger link.
🛡️Security
Flows reveal a device sending lots of data to an odd address at night.
Test your understanding
Which port does a device send SNMP traps to?
Polls go to 161; traps from the device to the manager go to 162.
A syslog message has level 2. What does it mean?
0 emergency, 1 alert, 2 critical, 3 error, 4 warning, 5 notice, 6 info, 7 debug.
The link is full. Which tool tells you who is filling it?
Flows record source, destination and bytes for every conversation.
Why point every device at NTP?
Without one clock you cannot tell which event came first.
Why prefer 64-bit counters on 10 Gbps?
2³² bytes pass in about 3.4 seconds at 10 Gbps.
Quick review cards
Check Your Understanding
The dashboard shows 40% and people complain. Why might it mislead?
Because it averages long periods and hides short bursts that fill the queue. Look at peaks and dropped packets.
What is the difference between a poll and a trap?
In a poll the manager asks on a fixed schedule; a trap is sent by the device itself the moment something happens.
When does something not deserve to be an alert?
When it needs no human action right now; it belongs in a report or a dashboard.
Resources for this lesson
Grouped by type. Start with the official docs for precision, or an article or video for the explanation.
Done with this lesson?
Sign in to save your progress.