Linux Administration

Bash Shell Basics: Commands, Redirection, Pipes, Variables and Configuration

Beginner9 min readUpdated

This guide explains what actually happens between pressing Enter and a program running: command lookup, quoting, expansion, and the three standard streams. You will learn to redirect output and errors, build pipelines, use the history and line-editing shortcuts that make the shell fast, manage environment variables and the PATH, and configure Bash through .bashrc and .profile without breaking your login.

What you will learn

  • Explain how Bash parses a command line and locates the program to run
  • Redirect standard output and standard error to files and connect commands with pipes
  • Use quoting and globbing correctly so that spaces and special characters do not cause surprises
  • Set, export and inspect shell and environment variables, including PATH
  • Configure aliases and environment in the right startup file and use the man pages efficiently

A shell is a program that reads lines of text, works out which programs to run and with what arguments, runs them, and shows you the result. Bash (the "Bourne-Again SHell") is the default interactive shell on almost every Linux distribution and the interpreter most administrative scripts still assume. Learning what Bash does to a command line before running anything — expansion, quoting, redirection — is what separates people who type commands from people who can predict what a command will do.

What happens when you press Enter

Take a line such as grep -i error /var/log/*.log > errors.txt. Bash processes it in stages:

  1. It splits the line into words on unquoted whitespace.
  2. It performs expansions: ~ becomes your home directory, $VAR is replaced by its value, $(cmd) by the output of a command, {a,b} by a b, and glob patterns such as *.log by the matching file names.
  3. It sets up redirections (>, <, |) and removes them from the argument list.
  4. It looks up the first word: an alias, a shell function, a builtin (cd, echo, export…) or an external program found by searching the directories in PATH, in that order.
  5. It forks a child process, applies the redirections in the child, and executes the program with the remaining words as arguments.

The program never sees the *, the > or the quotes; it receives a list of strings. grep does not know that *.log was a pattern — it just gets /var/log/syslog /var/log/kern.log …. This is why quoting matters so much and why every program on Linux handles wildcards consistently: none of them do, the shell does.

type tells you which category a name falls into, and command -v prints the path of an external command:

$ type cd ls ll
cd is a shell builtin
ls is /usr/bin/ls
ll is aliased to `ls -alF'
$ echo "$PATH"
/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin

PATH is searched left to right and the first match wins. The current directory is deliberately not in it, so a program in your working directory must be run as ./program — that stops a malicious ls dropped in /tmp from hijacking your commands.

Anatomy of a command

Most commands follow command [options] [arguments]. Options come in short (-l), bundled short (-la = -l -a) and long (--all) forms; some take a value (-n 5, --lines=5). Arguments are usually file names. A bare -- marks the end of options, so rm -- -rf deletes a file literally named -rf.

The manual is built in. man ls opens the page for ls; inside it, /pattern searches, n jumps to the next hit and q quits. Pages are grouped into numbered sections — 1 for commands, 5 for file formats, 8 for administration — and the number resolves ambiguity: man 5 passwd is the file format, man 1 passwd the command. man -k keyword (or apropos) searches page summaries when you do not know the command name. Most programs also print a summary with --help.

The three standard streams

Every process starts with three open file descriptors: standard input (fd 0), standard output (fd 1) and standard error (fd 2). In a terminal all three point at the terminal, which is why output and error messages appear intermixed. Redirection re-points them before the program starts.

SyntaxEffect
cmd > filestdout to file, truncating it first
cmd >> filestdout appended to file
cmd 2> filestderr to file
cmd > file 2>&1 or cmd &> fileboth streams to file
cmd 2> /dev/nulldiscard errors
cmd < filestdin read from file
cmd <<< "text"stdin from a string (a "here string")
cmd1 | cmd2stdout of cmd1 becomes stdin of cmd2

Because the program only sees descriptors, it cannot tell whether it is writing to a terminal, a file or another process. That is the design decision that makes pipelines possible: small programs that read stdin and write stdout can be chained arbitrarily.

# Count distinct IP addresses that hit the server, most active first
cut -d' ' -f1 /var/log/apache2/access.log | sort | uniq -c | sort -rn | head

tee copies its input to a file and to stdout, so you can keep a record while still watching: ./build.sh 2>&1 | tee build.log. xargs turns lines of input into arguments for another command: find . -name '*.tmp' -print0 | xargs -0 rm (the -print0/-0 pair keeps file names with spaces intact).

Two builtins complete the picture. set -o noclobber makes > refuse to overwrite an existing file (>| forces it). And exec > logfile redirects the shell's own stdout for the rest of a script.

Quoting and expansion

Quoting controls which expansions happen.

  • Single quotes '…' suppress everything. '$HOME' is the five characters $HOME.
  • Double quotes "…" suppress globbing and word splitting but still perform $var, $(cmd) and $((math)) expansion.
  • A backslash escapes the single following character.

The practical consequence: an unquoted $file containing my report.txt becomes two arguments after word splitting. "$file" stays one. Get into the habit of double-quoting every variable expansion unless you specifically want the split.

name="Ada Lovelace"
echo Hello $name        # echo receives 3 arguments: Hello, Ada, Lovelace
echo "Hello $name"      # echo receives 1 argument
echo 'Hello $name'      # prints Hello $name
echo "Today is $(date +%A)"
echo "Free memory: $(( $(grep MemAvailable /proc/meminfo | tr -s ' ' | cut -d' ' -f2) / 1024 )) MB"

Glob patterns match file names: * any run of characters, ? one character, [a-z] one character from a set, [!0-9] one character not in the set. shopt -s globstar enables ** for recursive matching (**/*.java). An important quirk: a pattern that matches nothing is passed through literally by default, so rm *.bak in a directory with no .bak files tries to delete a file called *.bak and fails. shopt -s nullglob makes unmatched patterns expand to nothing, which is usually what scripts want.

Brace expansion is not globbing — it generates strings whether or not files exist: mkdir -p project/{src,test,docs}, cp config.yml{,.bak}, echo {01..12}.

Variables and the environment

Assign with no spaces around =: retries=3. Read with $retries or, more safely, ${retries}. Bash provides many automatic variables: $? is the exit status of the last command, $$ the shell's PID, $HOME, $USER, $PWD, $RANDOM, $SECONDS.

A plain assignment creates a shell variable, private to this process. export promotes it to the environment, the block of NAME=value strings handed to every child process. env and printenv list the environment; set lists everything including shell variables and functions.

JAVA_HOME=/usr/lib/jvm/java-21-openjdk-amd64
export JAVA_HOME                       # now visible to programs you start
export PATH="$JAVA_HOME/bin:$PATH"     # prepend: found before the system java
LANG=C sort names.txt                  # one-off: only for this command
unset JAVA_HOME

Parameter expansion offers defaults and transformations that save a great deal of scripting:

echo "${EDITOR:-nano}"          # value, or "nano" if unset/empty
: "${DB_HOST:?must be set}"     # abort with a message if unset
f=/var/log/app/server.log
echo "${f##*/}"                 # server.log   (strip longest prefix matching */)
echo "${f%.log}"                # /var/log/app/server (strip suffix)
echo "${#f}"                    # length of the string

Exit status and command chaining

Every command ends with an integer exit status: 0 means success, anything else a failure (the meaning of specific non-zero values is program-defined). Bash exposes it as $? and uses it for control flow. Three operators chain commands:

make && sudo make install      # install only if the build succeeded
ping -c1 db01 || echo "db01 unreachable"
cd /tmp; ls                    # run both regardless

grep returns 1 when it finds nothing, so grep -q pattern file && do_something is idiomatic. true and false exist purely to produce exit codes.

Working faster: history and line editing

Bash keeps a history of commands (history, saved to ~/.bash_history on exit). Recalling them is quicker than retyping:

  • Ctrl-R then type — incremental search backwards; press Ctrl-R again for older matches, Enter to run, Esc to edit.
  • !! repeats the last command (sudo !! is the classic use); !$ is the last argument of the previous command.
  • Alt-. inserts the last argument of the previous command interactively.

The line editor uses Emacs keys by default: Ctrl-A/Ctrl-E jump to start/end, Ctrl-W deletes the previous word, Ctrl-U deletes to the start of the line, Ctrl-K to the end, Ctrl-L clears the screen, and Tab completes file names, command names and — with the bash-completion package — options and arguments for hundreds of tools. Press Tab twice to list the candidates.

Ctrl-C interrupts the running command; Ctrl-D at an empty prompt sends end-of-file and logs out; Ctrl-Z suspends the foreground job (covered with the rest of job control in Processes and systemd).

Aliases, functions and startup files

An alias is a textual shortcut expanded when it is the first word of a command: alias ll='ls -alF', alias gs='git status'. Aliases cannot take arguments in the middle; for that, define a function:

mkcd() { mkdir -p -- "$1" && cd -- "$1"; }

Bash reads different files depending on how it was started, which is why "I added it to .bashrc and it doesn't work" is such a frequent complaint.

Shell typeFiles readExample
Interactive login/etc/profile, then the first of ~/.bash_profile, ~/.bash_login, ~/.profileSSH session, console login
Interactive non-login/etc/bash.bashrc (Debian) or /etc/bashrc (RHEL), then ~/.bashrcNew terminal window, bash typed at a prompt
Non-interactiveNothing, unless BASH_ENV is setScripts, cron, ssh host command

The conventional arrangement: put environment variables (PATH, JAVA_HOME, EDITOR) in ~/.profile so they are set once at login and inherited by everything, and put aliases, functions, prompt settings and shopt options in ~/.bashrc because those must be redefined in every interactive shell. Debian's and Ubuntu's default ~/.profile sources ~/.bashrc, so both apply to SSH logins. System-wide settings for all users go in a new file under /etc/profile.d/.

After editing, apply changes to the current shell with source ~/.bashrc (or . ~/.bashrc) — running bash ~/.bashrc would execute it in a child that then exits, changing nothing.

Getting help without leaving the terminal

Beyond man and --help: help cd documents a builtin (the man page for Bash is 100+ pages, help is targeted); info coreutils has the fuller GNU manuals; whatis cmd prints the one-line summary; and /usr/share/doc/<package>/ holds README files, examples and changelogs installed with each package. When a command's output is long, pipe it into less and search with /.

Common mistakes

Spaces around =. count = 3 runs a command called count with two arguments. Assignments have no spaces.

Unquoted variables with spaces or globs. rm $file where file="report *.txt" deletes every .txt file. Quote it.

Expecting .bashrc to affect cron, systemd or scripts. Non-interactive shells skip it. Set PATH and other variables in the script or unit.

Redirection order. cmd 2>&1 > file still shows errors on the terminal. Put > file first, or use &> file.

Running scripts with sh that were written for Bash. sh script.sh uses Dash on Debian and Ubuntu and fails on [[, arrays and local -n. Use the shebang and chmod +x, then run ./script.sh.

Using cat file | grep x. Harmless, but grep x file is simpler and lets grep show file names when given several. Reserve cat for concatenating.

Overwriting the file you are reading. sort names.txt > names.txt truncates the file before sort reads it, leaving it empty. Write to a temporary file and rename, or use sponge from moreutils.

Key takeaways

  • Bash expands the command line — variables, globs, braces, command substitution — before the program runs; programs receive plain argument strings.
  • Command lookup order is alias, function, builtin, then PATH search; type reveals which applies.
  • The three standard streams and the redirection operators (>, >>, 2>, 2>&1, <, |) are what make small tools composable.
  • Double-quote variable expansions; single-quote literal text with $ or backslashes.
  • export moves a variable into the environment of child processes; unexported variables stay in the shell.
  • Environment goes in ~/.profile, aliases and prompt settings in ~/.bashrc; non-interactive shells read neither.
  • Exit status 0 means success, and &&, || and $? let you act on it.

Sources and further reading

Frequently asked questions

What is the difference between a shell variable and an environment variable?
A shell variable exists only inside the current shell process. Exporting it (export NAME=value) copies it into the environment block that is handed to every child process the shell starts. Programs such as java, python or curl only see exported variables. Variables set in a script are never visible to the parent shell, because a child cannot modify its parent's memory.
Why does my alias or PATH change work in a terminal but not in cron or a systemd service?
Aliases and .bashrc are only read by interactive shells. Cron jobs and services run without a terminal and with a minimal environment, so they see neither your aliases nor the PATH additions you made in .bashrc. Use absolute paths in scripts, set PATH explicitly at the top of the script, or put environment in the service unit or crontab itself.
When should I use single quotes and when double quotes?
Single quotes are literal: nothing inside is expanded, so they are right for regular expressions, awk programs and anything containing $ or backslashes you want kept. Double quotes keep spaces together but still expand $variables and $(commands), which is what you want when passing a variable as one argument. Rule of thumb: double-quote every variable reference, single-quote fixed text with special characters.
What does 2>&1 mean and why does the order matter?
2>&1 makes file descriptor 2 (standard error) a copy of wherever descriptor 1 (standard output) currently points. Redirections are applied left to right, so cmd > log 2>&1 sends both streams to the file, whereas cmd 2>&1 > log copies stderr to the terminal first and only then points stdout at the file. Bash also accepts the shorthand &> log.