Linux

Basics a backend engineer needs to debug a container or a host. Not kernel internals.

Signals

Signals are how the OS (and you) communicate with running processes. The graceful-shutdown handler for SIGTERM is the single most-interviewed detail.

Name	#	Behavior	Catchable?	Use
SIGTERM	15	Polite termination request	Yes — the program can run cleanup handlers	Default for `kill`, Docker `stop`, Kubernetes pre-deletion. Programs SHOULD catch this and shut down gracefully
SIGINT	2	Interactive interrupt	Yes	What Ctrl-C sends. Same spirit as SIGTERM but from the keyboard
SIGHUP	1	Terminal hangup or controlling-terminal death	Yes	Historically: "reload your config". Many daemons still use it that way (nginx, postgres)
SIGQUIT	3	Terminate + core dump	Yes	Ctrl-\. JVMs dump thread stacks on SIGQUIT — handy for "what is my service doing?"
SIGKILL	9	Immediate kernel-level termination	No — cannot be caught, blocked, or ignored	Last resort. After this the process is gone — no cleanup, no flushing, open files unsynced
SIGSTOP	19	Pause the process (cannot be caught)	No	Useful for freezing a misbehaving process without killing it — resume with SIGCONT
SIGPIPE	13	Write to a pipe / socket with no reader	Yes (default: terminate)	`curl foo \| head` ends because head exits; curl's next write raises SIGPIPE. A subtle cause of 'silently missing output' in pipelines

The graceful shutdown pattern

Catch SIGTERM → stop accepting new work → finish in-flight work within a grace period → exit. Containers that exit quickly on SIGTERM avoid the kubelet / Docker escalating to SIGKILL, which means no lost requests mid-shutdown.

Process & resource inspection

Process tree

ps auxf | head -30

Top-like interactive view

htop

Falls back to `top` if htop not installed

Who owns this port?

sudo ss -tlnp | grep :8080

ss > netstat — faster, actively maintained

What files does this process have open?

lsof -p <pid>

Syscall trace of a running process

sudo strace -p <pid> -f -e trace=network

Expensive — do not leave running on prod

systemd service logs

journalctl -u my-service.service -f --since "10 min ago"

Kernel / all logs

dmesg -T | tail -50

Memory / CPU summary

free -h && uptime

Disk usage of this directory tree

du -sh */ | sort -h

Which process is writing to disk?

sudo iotop -o

iotop may need a package install

File descriptors & redirection

Every open resource (file, socket, pipe, TTY) is a file descriptor — an integer index into a per-process table. 0=stdin, 1=stdout, 2=stderr are fixed; the rest are allocated as low as possible on open.

Shape	Meaning
> file	Redirect stdout (fd 1) to file, truncate
>> file	Redirect stdout, append
2> file	Redirect stderr (fd 2)
2>&1	Merge stderr into stdout — usually written after other redirects
&> file	Shorthand for both stdout and stderr to file (bash)
< file	Use file as stdin (fd 0)
cmd1 \| cmd2	Pipe: stdout of cmd1 becomes stdin of cmd2
2>/dev/null	Discard stderr
cmd <(other)	Process substitution — feed the output of `other` as if it were a file

Redirection order gotcha

Order matters: `cmd > out 2>&1` redirects stdout to `out` then points stderr at "wherever stdout currently goes" — so both land in `out`. `cmd 2>&1 > out` does the opposite: stderr ends up on the terminal; stdout goes to `out`.

Permissions

Symbolic	Octal	Meaning
rwx------	700	Owner full; group/other nothing
rwxr-xr-x	755	Owner full; others read + execute (typical for scripts, binaries)
rw-r--r--	644	Owner read/write; others read (typical for config files)
rw-------	600	Owner read/write only (SSH keys, secrets)
rwxrwxrwx	777	Everyone everything — almost always wrong

Special bits

setuid (4000): binary runs as its owner (e.g., /usr/bin/passwd runs as root regardless of who launched it)
setgid (2000): binary runs as its group; on directories, new files inherit the dir's group
sticky (1000): in a dir with this bit, only the file's owner can delete it (e.g., /tmp)

Search & text wrangling

Find files by name

find . -name "*.log" -type f

Find + delete (in one pass)

find . -name "*.tmp" -type f -delete

Case-insensitive recursive grep

grep -riI "TODO" src/

-I skips binary files

Grep with surrounding lines

grep -n -B 2 -A 2 "ERROR" app.log

sed — in-place substitute

sed -i 's/foo/bar/g' config.yaml

GNU sed. BSD (macOS) sed requires: sed -i '' ...

awk — print a column

ps auxf | awk '{print $2, $11}'