Linux and the command line for defenders
Lesson 1 of 4
1Linux and the command line for defenders
The big pictureAn alert at midnight: "many sign-in attempts on the web server". There is no graphical interface, just a black terminal. Within two minutes you must know: who tried, how many times, and whether anyone got in. This lesson gives you those two minutes.
What the picture shows: A pipe passes each command's output to the next: grep picks the lines, awk cuts the column you want, sort and uniq count and rank. Four small commands answer a big question.
Map of this part
The concepts in order, click any one to jump to it:
Two ways into the same idea
Picture the server as a house with a visitors' book at the door, where the guard writes down everyone who knocked, when, where they came from, and whether they were let in. That book is the log. The command line is a quick way to leaf through it: "show me only the ones refused" (grep), "count how many times each person came" (sort and uniq). You do not have to memorise the commands now; what matters is knowing that every action on a server is written somewhere, and that an analyst reads it to learn what happened.
Permissions are three sets (owner, group, others), each with r, w and x, read as numbers: 640 = owner read and write, group read, others nothing. sudo runs as root and is logged in auth.log. The key logs: /var/log/auth.log (or /var/log/secure on RHEL family systems) for sign-ins, sudo and SSH, /var/log/syslog (or messages) for system events, journalctl -u ssh for systemd logs, /var/log/audit/audit.log for auditd, and ~/.bash_history for commands. The tools: grep -E, awk, cut, sort, uniq -c, head and tail -f, plus ps aux and ss -tulpn for processes and ports, and find / -perm -4000 for SUID files. The hands-on reference: TryHackMe's Linux Logging for SOC room.
Before You Start
⌨️The terminal (stage 0 of software engineering)
If you have tried ls and cd you are ready; if not, this lesson starts from the first command.
🛡️What is cybersecurity?
The least-privilege principle.
The Core Idea
1Files, users and permissions
On Linux everything is a file: settings in /etc, logs in /var/log, users' files in /home. And every file has an owner, a group and permissions. The line -rw-r----- root adm auth.log means: the owner root reads and writes, the adm group only reads, everyone else nothing. That is why you need sudo or adm membership to read the sign-in log.
| Command | What it does | Why a defender cares |
|---|---|---|
ls -l | Lists files with permissions and owner | A world-writable config file is a flaw |
chmod 640 file | Changes permissions | Applying least privilege |
id · who · last | Who am I, who is on, who signed in lately | A strange account signed in last night? |
sudo -l | What I may run as root | Excess sudo rights are a common path to escalation |
find / -perm -4000 2>/dev/null | Files that run with their owner's rights (SUID) | An unexpected SUID file is a red flag |
chmod 777. That makes the file readable, writable and executable by any user, so by any attacker who gets in on the weakest account.2Where do the logs live?
Knowing these locations saves you hours. With every alert ask: which log answers this question?
| Log | What is in it | A question it answers |
|---|---|---|
/var/log/auth.log | Sign-ins, SSH, sudo, user creation (/var/log/secure on RHEL) | Who tried to sign in and who succeeded? |
/var/log/syslog | System and service events (messages on RHEL) | When did the service stop? |
journalctl -u ssh | systemd logs for one service | What did the SSH service say today? |
/var/log/nginx/access.log | Every request the web server got | Who requested the admin page? |
/var/log/audit/audit.log | auditd events: watched files and commands | Who changed /etc/passwd? |
~/.bash_history | Commands the user typed | What did the attacker run after getting in? |
.bash_history. Its sudden absence or emptiness is evidence in itself, which is why organisations ship logs to a SIEM off the server.3Pipes: a security question in one line
The command line's power is the pipe |: one command's output becomes the next one's input. The question "which IP addresses failed to sign in the most?" becomes one line. The script below imitates the same pipeline step by step in Python, so you can run it here and see what each command does.
$(NF-3) in awk means "the fourth field from the end", which is the IP address in a Failed password line. Run these on a Linux machine you own.
# Which IPs failed SSH sign-in the most?
sudo grep "Failed password" /var/log/auth.log \
| awk '{print $(NF-3)}' \
| sort | uniq -c | sort -rn | head -5
# Did any of them get in?
sudo grep "Accepted" /var/log/auth.log | grep 203.0.113.7
# Who used sudo, and to run what?
sudo grep "sudo:" /var/log/auth.log | grep COMMAND
# Watch new lines live during an incident
sudo tail -f /var/log/auth.logThe whole story in six lines: guessing from 203.0.113.7, then a successful sign-in as admin, then a new user backup2 created as root (a known persistence trick). The addresses are fictional, from the ranges reserved for documentation.
from collections import Counter
auth_log = """\
Nov 02 01:14:02 web sshd[911]: Failed password for root from 203.0.113.7 port 50122 ssh2
Nov 02 01:14:05 web sshd[911]: Failed password for root from 203.0.113.7 port 50131 ssh2
Nov 02 01:14:09 web sshd[913]: Failed password for invalid user oracle from 198.51.100.40 port 41000 ssh2
Nov 02 01:14:11 web sshd[911]: Failed password for admin from 203.0.113.7 port 50140 ssh2
Nov 02 01:15:30 web sshd[920]: Accepted password for admin from 203.0.113.7 port 50201 ssh2
Nov 02 01:16:02 web sudo: admin : TTY=pts/0 ; PWD=/home/admin ; USER=root ; COMMAND=/usr/bin/useradd -m backup2
"""
lines = auth_log.splitlines()
failed = [l for l in lines if "Failed password" in l] # grep
ips = [l.split()[-4] for l in failed] # awk '{print $(NF-3)}'
for ip, n in Counter(ips).most_common(): # sort | uniq -c | sort -rn
print(f"{n:>3} {ip}")
print("--- did any of them get in? ---")
for l in lines:
if "Accepted" in l and any(ip in l for ip in ips):
print(l)
print("--- what ran as root? ---")
for l in lines:
if "COMMAND=" in l:
print(l.split("COMMAND=")[1])4Processes and ports: what is running now?
The log tells you what happened; processes and ports tell you what is happening now. ps aux shows every running program and who started it, and ss -tulpn shows the open ports and the program listening on each. A strange program listening on a high port, or a process named like a system service but running from /tmp, are two signs worth investigating.
Port 4444 is common in remote-control tools, the name imitates the kernel's kworker, and the path is hidden in /tmp. Immediate containment: isolate the server and keep evidence before deleting anything.
$ sudo ss -tulpn
Netid State Local Address:Port Process
tcp LISTEN 0.0.0.0:22 users:(("sshd",pid=640))
tcp LISTEN 0.0.0.0:443 users:(("nginx",pid=702))
tcp LISTEN 0.0.0.0:4444 users:(("kworkerd",pid=3112))
$ ps -o pid,user,cmd -p 3112
PID USER CMD
3112 www-data /tmp/.x/kworkerdWorked example
The task: from the log above, write the incident summary in three lines as a tier 1 analyst would, and propose two immediate steps.
The answer: "(1) Between 01:14 and 01:15, 203.0.113.7 made three failed SSH attempts on root and admin, then succeeded as admin with a password. (2) At 01:16 that account created a new user backup2 via sudo, a persistence technique (ATT&CK T1136 Create Account). (3) Severity high: a successful sign-in and account changes." Immediate steps: disable admin and backup2 and end their sessions, block the address at the firewall, then escalate to tier 2 with the log. Longer term: SSH by keys only (PasswordAuthentication no) and a tool such as fail2ban.
Myth vs Reality
| Myth | Reality |
|---|---|
| Cybersecurity is graphical consoles and colourful dashboards. | Dashboards show the alert, but investigating a server usually goes through the terminal and text logs. |
| I have to memorise hundreds of commands. | About twenty commands cover most daily work, and you look up the rest in man. |
| Linux is secure by nature so it needs no monitoring. | Most internet servers run on it, so it is a constant target; configuration, updates and monitoring are what protect it. |
Where you meet it at work
🚨SOC analyst
Checks a SIEM alert by reading the original log on the server.
🧯Incident responder
Collects processes, ports and logs before isolating the server.
☁️Cloud security
Cloud servers are mostly Linux, and every defensive tool there is driven from the terminal.
Test your understanding
Where do you look for SSH sign-in attempts on an Ubuntu server?
auth.log records sign-ins, SSH and sudo; on RHEL it is called secure.
What do the permissions 640 mean?
6 = r+w, 4 = r, 0 = nothing.
Which command shows open ports and the program listening on each?
t and u for TCP and UDP, l for listening, p for the program, n for numbers.
What does sort | uniq -c | sort -rn do at the end of a pipe?
uniq -c counts identical neighbouring lines, which is why we sort first.
A process called kworkerd runs from /tmp and listens on 4444. Best first step?
Deleting and rebooting wipe evidence; contain first.
Why is chmod 777 a security mistake?
It completely breaks least privilege.
Quick review cards
Check Your Understanding
Why do organisations ship logs to a SIEM instead of only reading them on the server?
Because an attacker who reaches the server can wipe its logs, and because correlating many servers only works in one place.
An auth.log line says Accepted publickey instead of Accepted password. What is the security difference?
Key sign-in is far stronger because a key cannot be guessed; but from a strange address it may mean a stolen key.
How do you practise the commands safely?
On a Linux virtual machine you own, the OverTheWire Bandit game, and TryHackMe rooms. Never practise on servers you do not own.
Resources for this lesson
Grouped by type. Start with the official docs for precision, or an article or video for the explanation.
Done with this lesson?
Sign in to save your progress.