Linux Administration

Linux Networking Basics: ip, Routing, DNS, Sockets, Firewalls and SSH Tunnels

Intermediate8 min readUpdated

Networking problems are usually diagnosed in the wrong order, which is why they take so long. This guide walks the stack the way a packet does — interface, address, route, name, port — and gives the modern command for inspecting each layer. It then covers firewalls with nftables and its front ends, the diagnostic value of curl and dig, and the SSH tunnelling tricks that get you through a bastion host.

What you will learn

  • Inspect and configure interfaces, addresses and routes with the ip command
  • Explain how a name becomes an address and debug resolution with dig and resolvectl
  • List listening and established sockets with ss and map them back to a process
  • Write and inspect firewall rules with nftables, ufw or firewalld
  • Isolate a connectivity failure layer by layer and reach a private service through an SSH tunnel

Most Linux networking questions turn out to be one of five things: the interface has no address, there is no route to the destination, the name does not resolve, nothing is listening on the port, or a firewall is dropping the packet. Knowing that list, and having one command for each item, converts an afternoon of guessing into a five-minute check.

The tooling has also changed. ifconfig, netstat, route and arp came from the net-tools package, which is unmaintained and absent from default installations; ip and ss from iproute2 replaced them and expose things the old tools cannot see. This guide uses the modern commands, mentions the old names once so you can read older documentation, and assumes a current Debian, Ubuntu or RHEL-family host.

The layers you actually touch

You do not need the full seven-layer model to work effectively, but four levels are unavoidable in daily diagnosis:

  1. Link — a network interface with a MAC address, connected to a segment. The ip link layer. ARP (ip neigh) maps IP addresses to MAC addresses inside a segment.
  2. Network — IP addresses and routes. Getting a packet from this host towards a destination is ip addr and ip route.
  3. Transport — TCP and UDP ports, connections, listening sockets. The ss layer.
  4. Application — HTTP, DNS, TLS. curl, dig, openssl s_client.

An address is written with a prefix length: 10.0.3.17/24 means the first 24 bits identify the network, so 10.0.3.0 through 10.0.3.255 are local and everything else must go via a router. Getting that number wrong produces the maddening symptom where some destinations work and others do not.

Three private ranges — 10.0.0.0/8, 172.16.0.0/12 and 192.168.0.0/16 — are reserved for internal use and never routed on the public internet, which is why cloud VPCs and home networks all look alike. 127.0.0.0/8 is the loopback, entirely inside the kernel. IPv6 addresses appear alongside IPv4 on most modern networks; fe80::/10 addresses are link-local, valid only on one segment, and always carry a %interface suffix when you use them.

Interfaces, addresses and routes

$ ip -brief addr
lo               UNKNOWN        127.0.0.1/8 ::1/128
eth0             UP             10.0.3.17/24 fe80::5054:ff:fe12:3456/64

$ ip route
default via 10.0.3.1 dev eth0 proto dhcp src 10.0.3.17 metric 100
10.0.3.0/24 dev eth0 proto kernel scope link src 10.0.3.17

The -brief flag gives the summary view; plain ip addr adds MTU, flags and lifetimes. Read the routing table from the most specific prefix to the least: the kernel picks the longest matching prefix, so 10.0.3.0/24 handles the local segment and default catches everything else by sending it to the gateway 10.0.3.1.

ip route get answers the question directly and saves a lot of squinting:

$ ip route get 93.184.216.34
93.184.216.34 via 10.0.3.1 dev eth0 src 10.0.3.17 uid 1000

Changes made with ip are immediate and lost at reboot, which makes them ideal for testing and wrong for configuration:

sudo ip link set eth0 up
sudo ip addr add 10.0.3.200/24 dev eth0
sudo ip route add 10.50.0.0/16 via 10.0.3.254
sudo ip addr flush dev eth0            # careful: this drops your SSH session

Persistent configuration belongs to whichever manager the distribution uses: netplan YAML in /etc/netplan/ on Ubuntu server, NetworkManager connection profiles (nmcli) on Fedora and RHEL-family desktops and servers, or systemd-networkd .network files on minimal images.

sudo netplan try                    # applies with an automatic rollback after 120 s
nmcli con mod eth0 ipv4.addresses 10.0.3.17/24 ipv4.gateway 10.0.3.1 ipv4.method manual
nmcli con up eth0

netplan try and NetworkManager's rollback exist because a bad address change on a remote host locks you out. Use them, or arrange a scheduled reboot before you make the change.

Name resolution

Programs do not call DNS directly. They call the C library's resolver, which consults /etc/nsswitch.conf to decide the order of sources — typically files (meaning /etc/hosts), then a local stub resolver, then DNS. /etc/hosts therefore wins over DNS, which is both a useful override and a frequent cause of "it works on that server only".

On systems running systemd-resolved — the default on Ubuntu and increasingly elsewhere — /etc/resolv.conf is a symlink pointing at a stub file that lists 127.0.0.53. Editing it by hand is pointless: it is regenerated. Ask the actual configuration instead:

$ resolvectl status | sed -n '1,12p'
Global
       Protocols: LLMNR=resolve -mDNS +DNSOverTLS DNSSEC=no/unsupported
Link 2 (eth0)
Current Scopes: DNS
     DNS Servers: 10.0.0.2 10.0.0.3
      DNS Domain: internal.example.com

$ resolvectl query api.internal.example.com
api.internal.example.com: 10.0.4.21

dig is the tool for interrogating DNS itself, bypassing the library entirely:

dig +short api.example.com            # just the answer
dig api.example.com A +noall +answer  # the answer section with TTLs
dig @1.1.1.1 example.com MX           # ask a specific server
dig +trace example.com                # follow delegation from the root
dig -x 93.184.216.34                  # reverse lookup

The distinction in the FAQ above is worth internalising: dig tells you what DNS says, getent hosts name tells you what your applications will get. When they disagree, the answer is in /etc/hosts, nsswitch.conf or the stub resolver, not in the zone.

Caching adds its own confusion. A record's TTL governs how long resolvers may keep the old answer, so a change that "has not propagated" is usually just a TTL that has not expired. Flush the local cache with sudo resolvectl flush-caches and check the authoritative server with dig @ns1.example.com before assuming something is broken.

Ports and sockets

ss replaces netstat and is dramatically faster on busy hosts. The flags worth committing to memory are -l (listening), -t/-u (TCP/UDP), -n (numeric, no name lookups), -p (owning process, needs root).

$ sudo ss -ltnp
State  Recv-Q Send-Q  Local Address:Port  Peer Address:Port Process
LISTEN 0      4096        127.0.0.1:5432       0.0.0.0:*     users:(("postgres",pid=1842,fd=5))
LISTEN 0      511           0.0.0.0:80         0.0.0.0:*     users:(("nginx",pid=1201,fd=6))
LISTEN 0      128           0.0.0.0:22         0.0.0.0:*     users:(("sshd",pid=980,fd=3))

The local address column is the first thing to read when a service is unreachable. 127.0.0.1:5432 accepts connections only from the same host; 0.0.0.0:80 accepts from anywhere on any interface; [::]:80 is the IPv6 equivalent, and on Linux a socket bound to [::] usually also accepts IPv4 connections. Binding a service to loopback and then fronting it with a reverse proxy is a good default, and it is exactly what the LAMP stack guide does with PHP-FPM.

Other useful forms:

ss -tan state established             # current TCP connections
ss -tn dst 10.0.4.21                  # connections to one host
ss -s                                 # summary counters by protocol
ss -tlnp 'sport = :8080'              # who owns port 8080

Recv-Q on a listening socket is the number of connections waiting to be accepted; a persistently non-zero value means the application is not accepting fast enough. On an established socket the queues show unread and unacknowledged bytes, which is how you spot a stalled reader.

Ports below 1024 are privileged: only root, or a process granted CAP_NET_BIND_SERVICE, may bind them. That is why application servers listen on 8080 and a proxy or a systemd socket unit handles 80 and 443.

Firewalls

Modern kernels filter with nftables, which replaced iptables. Most administrators drive it through a front end.

ufw (Debian and Ubuntu) is deliberately minimal:

sudo ufw default deny incoming
sudo ufw default allow outgoing
sudo ufw allow OpenSSH                 # named application profile
sudo ufw allow 80,443/tcp
sudo ufw allow from 10.0.0.0/8 to any port 5432 proto tcp
sudo ufw enable
sudo ufw status verbose

firewalld (RHEL family) organises rules into zones, each zone applying to a set of interfaces or sources:

sudo firewall-cmd --get-active-zones
sudo firewall-cmd --zone=public --add-service=https --permanent
sudo firewall-cmd --zone=public --add-port=8080/tcp --permanent
sudo firewall-cmd --reload
sudo firewall-cmd --list-all

The --permanent flag writes the rule to disk without applying it; omitting it applies without saving. Doing both, then --reload, is the reliable sequence — and the runtime-only form is a genuine safety feature when you are editing rules over SSH, because a reboot undoes a mistake.

Writing nftables rules directly is worth understanding even if you use a front end:

table inet filter {
  chain input {
    type filter hook input priority filter; policy drop;
    ct state established,related accept
    iif lo accept
    ip protocol icmp accept
    tcp dport { 22, 80, 443 } accept
    tcp dport 5432 ip saddr 10.0.0.0/8 accept
  }
}

Read it top down: the default is to drop, replies to connections this host started are accepted by connection tracking, loopback is unrestricted, and three ports plus a source-restricted database port are open. sudo nft list ruleset shows what is loaded, /etc/nftables.conf with systemctl enable nftables makes it persistent.

Two cautions. First, always allow SSH before setting a default deny policy, and keep a second session open. Second, in a cloud environment there is a second firewall — a security group or network ACL — outside the machine entirely; a packet blocked there never reaches the host, and nothing in nft list ruleset will hint at it.

Diagnosing from the outside in

ping tests ICMP reachability, which many networks and most cloud load balancers deliberately block, so a failed ping proves very little. These prove more:

curl -sS -o /dev/null -w '%{http_code} %{time_total}s\n' https://example.com/health
curl -v --connect-timeout 3 https://api.internal:8443/status
curl --resolve api.example.com:443:10.0.4.21 https://api.example.com/   # bypass DNS, test one backend
openssl s_client -connect example.com:443 -servername example.com </dev/null | head -20
mtr -rwc 20 10.0.4.21                 # traceroute plus per-hop loss, over 20 cycles
tcpdump -ni eth0 'tcp port 5432 and host 10.0.4.21'

curl -v is the highest-value diagnostic on this list because it narrates each stage: DNS resolution, TCP connect, TLS handshake with the certificate chain, request, response headers. Where it stops tells you which layer failed. A hang at "Trying 10.0.4.21:443" is a firewall dropping packets silently; an immediate "Connection refused" means the packet arrived and nothing was listening — very different problems with the same user-visible symptom.

tcpdump is the last resort and the definitive one: if you see the SYN leave and no SYN-ACK return, the problem is beyond this host. Capture to a file with -w capture.pcap when you need to look at it properly.

SSH as network plumbing

Beyond logins, SSH forwards TCP connections, which is how you reach a database or an admin interface that is not exposed publicly.

# Local forward: localhost:15432 on my laptop -> db.internal:5432, via the bastion
ssh -N -L 15432:db.internal:5432 [email protected]
psql -h 127.0.0.1 -p 15432 -U app orders

# SOCKS proxy: a browser can then reach anything the bastion can
ssh -N -D 1080 [email protected]

# Jump host in one hop, without an agent on the bastion
ssh -J [email protected] [email protected]

-N means "no remote command", so you get a tunnel and no shell. Put the details in ~/.ssh/config once and the complexity disappears:

Host bastion
  HostName bastion.example.com
  User alice
  IdentityFile ~/.ssh/id_ed25519

Host db
  HostName db.internal
  User alice
  ProxyJump bastion
  LocalForward 15432 db.internal:5432

Agent forwarding (-A) is convenient and risky: root on the intermediate host can use your agent to authenticate elsewhere as you. ProxyJump achieves the same result without exposing the agent, and should be the default choice.

An order of operations

When something cannot reach something else, work outwards and stop at the first failure:

  1. Is it listening, and on what? sudo ss -ltnp | grep :<port>.
  2. Does it answer locally? curl -v http://127.0.0.1:<port>/. If not, the problem is the application — check its logs with journalctl -u, as described in processes and systemd.
  3. Does the name resolve, and to the right address? getent hosts <name> and dig +short <name>.
  4. Is there a route? ip route get <address>.
  5. Does the host firewall allow it? sudo nft list ruleset, ufw status, firewall-cmd --list-all.
  6. Does anything outside the host block it? Cloud security groups, network ACLs, a corporate proxy. Test from a third machine on the same segment to separate host rules from network rules.
  7. What do the packets say? tcpdump on both ends, if you get this far.

Common mistakes

  • Binding to 127.0.0.1 and expecting remote access. Check the local address in ss before touching the firewall.
  • Editing /etc/resolv.conf when systemd-resolved manages it. The file is regenerated; configure the resolver instead.
  • Trusting ping as a reachability test. ICMP is filtered in many environments; test the actual port with curl or nc -vz.
  • Applying a default-deny firewall policy without an SSH rule. Add the allow rule first, then the policy, and keep a second session open.
  • Configuring an address with ip addr add and calling it done. It disappears at the next boot; write it into netplan, NetworkManager or systemd-networkd.
  • Forgetting the cloud-side firewall. Half of "the port is open but I cannot connect" tickets end at a security group.
  • Ignoring IPv6. A name with both A and AAAA records may be reached over IPv6, where your firewall rules or routing may differ entirely. Test with curl -4 and curl -6 separately.

Key takeaways

  • Diagnose in layer order — interface, address, route, name, port, firewall — and stop at the first thing that fails.
  • ip addr, ip route and ip route get replace ifconfig and route; ip changes are immediate and non-persistent by design.
  • Resolution goes through nsswitch.conf and possibly a stub resolver, so getent hosts reflects what applications see while dig reflects what DNS says.
  • ss -ltnp and the local address column answer most "unreachable service" questions before the firewall is even involved.
  • nftables is the kernel's filter; ufw and firewalld are convenient front ends, and a cloud security group is a second, invisible firewall.
  • curl -v narrates DNS, TCP, TLS and HTTP in one command, and tcpdump settles arguments about whether packets ever left.
  • ssh -L, -D and ProxyJump, written once into ~/.ssh/config, give you safe access to private services without exposing them.

Sources and further reading

Frequently asked questions

Why is ifconfig missing on my server?
The net-tools package that provided ifconfig, netstat, route and arp has been unmaintained for years and is no longer installed by default. Its replacements come from iproute2: ip addr for ifconfig, ip route for route, ip neigh for arp and ss for netstat. They understand features net-tools never learned, such as multiple addresses per interface and policy routing, so learn the new names rather than installing the old package.
The service is running but I cannot reach it from another machine. Where do I start?
Check what it is bound to before blaming the network. Run ss -ltnp and look at the local address column: 127.0.0.1:8080 accepts only local connections, while 0.0.0.0:8080 accepts any. If the binding is correct, test from the server itself with curl, then from outside, then check the host firewall, then any cloud security group. Working outwards in that order isolates the layer in a couple of minutes.
Why does dig return an answer when ping cannot resolve the same name?
dig queries a DNS server directly and ignores the rest of the resolution machinery. Ordinary programs go through the C library, which consults /etc/nsswitch.conf and therefore /etc/hosts, mDNS and any local stub resolver before or instead of DNS. Use getent hosts name to reproduce exactly what an application sees; a difference between the two means the problem is in the configuration, not in DNS.
What is the difference between local and remote SSH port forwarding?
Local forwarding, ssh -L, opens a port on your machine and tunnels connections through the SSH server to a destination the server can reach — the usual way to reach a database that only listens on a private network. Remote forwarding, ssh -R, opens a port on the SSH server and tunnels connections back to your machine, which exposes something local to the remote side. Local is what you want in almost every case.