sed
MASTERY

What is sed?

sed (stream editor) reads text from input, transforms it, and outputs the result. It's perfect for batch text operations like find-and-replace.

Basic Syntax

sed [OPTIONS] 'COMMAND' [FILE...]
sed [OPTIONS] -f SCRIPTFILE [FILE...]

Simple Example

# Replace first 'hello' with 'world' in file.txt
sed 's/hello/world/' file.txt

# Use pipe
echo "hello world" | sed 's/hello/goodbye/'

sed vs Editor

• Non-interactive: Works in scripts and pipelines
• Stream-based: Processes line by line
• Powerful: Regex support, substitutions, deletion
• Fast: Can process huge files efficiently

Quiz

The s Command

The substitution command `s` finds a pattern and replaces it:

s/pattern/replacement/

Basic Examples

# Replace 'error' with 'ERROR'
sed 's/error/ERROR/' file.log

# Replace all occurrences on each line (default is first only)
sed 's/error/ERROR/' file.log

# Delimiter can be any character
sed 's|/bin/bash|/bin/zsh|' /etc/passwd

Quiz

Common Flags

g  Replace all occurrences on each line (global)
i  Case insensitive matching
p  Print the modified line
I  Case insensitive (GNU extension)
E  Extended regex (ERE)

Examples

# Replace ALL occurrences (not just first)
sed 's/error/ERROR/g' file.log

# Case insensitive
sed 's/error/ERROR/gi' file.log

# Print only changed lines (-n suppresses unless p)
sed -n 's/error/ERROR/p' file.log

# Only print modified lines
sed -n 's/root/ADMIN/p' /etc/passwd

Quiz

What are Addresses?

Addresses specify which lines to operate on. Without an address, sed operates on all lines.

Address Types

N          Line number N
$          Last line
N,M        Lines N through M
/pattern/  Lines matching pattern
/pat1/,/pat2/  From first match to next match

Examples

# Replace on line 5 only
sed '5s/old/new/' file.txt

# Replace on lines 10-20
sed '10,20s/old/new/' file.txt

# Replace on last line
sed '$s/old/new/' file.txt

# Replace on lines containing 'error'
sed '/error/s/old/new/' file.txt

# From line matching 'start' to 'end'
sed '/^start/,/^end/s/old/new/' file.txt

Quiz

The d Command

The delete command removes lines from the output (or file with -i).

Examples

# Delete line 5
sed '5d' file.txt

# Delete lines 1-10
sed '1,10d' file.txt

# Delete last line
sed '$d' file.txt

# Delete empty lines
sed '/^$/d' file.txt

# Delete lines containing 'error'
sed '/error/d' file.txt

# Delete lines NOT containing 'error' (invert)
sed '/error/!d' file.txt

Quiz

Text Addition Commands

i\  Insert text BEFORE line
a\  Append text AFTER line
c\  Change (replace) entire line

Examples

# Insert before line 3
sed '3i\New line here' file.txt

# Append after line 3
sed '3a\New line here' file.txt

# Append at end of file
sed '$a\Last line added' file.txt

# Change (replace) line 5
sed '5c\This replaces the entire line' file.txt

# Insert at beginning of file
sed '1i\Header line' file.txt

Quiz

File Operations

r FILE  Read file and insert after line
w FILE  Write lines to file

Examples

# Insert contents of header.txt after line 1
sed '1r header.txt' file.txt

# Insert footer at end of file
sed '$r footer.txt' file.txt

# Write all lines containing 'error' to errors.log
sed -n '/error/w errors.log' file.txt

# Write lines 1-10 to newfile.txt
sed -n '1,10w newfile.txt' file.txt

Quiz

sed and Regex

sed uses basic regular expressions (BRE) by default. Use -E for extended regex (ERE).

Regex Metacharacters

.   Any character
*   Zero or more of preceding
^   Beginning of line
$   End of line
[]  Character class
\(\) Group (BRE)
\|  OR (ERE with -E)

Examples

# Replace lines starting with 'error'
sed 's/^error/ERROR/' file.log

# Replace lines ending with '.log'
sed 's/\.log$/LOG_FILE/' file.txt

# Replace any digit
sed 's/[0-9]/X/g' file.txt

# Extended regex (-E)
sed -E 's/(error|warning|critical)/[ALERT]/gi' file.log

# Remove leading whitespace
sed 's/^[[:space:]]*//' file.txt

Quiz

Capture Groups

Use \( \) to group patterns and \1, \2, etc. to reference them in the replacement.

Examples

# Swap first two words
echo "hello world" | sed 's/\(hello\) \(world\)/\2 \1/'
# Output: world hello

# Remove duplicate words
sed 's/\([a-z]*\) \1/\1/g' file.txt

# Add quotes around words
echo "one two three" | sed 's/\([a-z]*\)/"\1"/g'
# Output: "one" "two" "three"

# Extract and reorder
echo "John:Doe:123" | sed 's/\([^:]*\):\([^:]*\):\(.*\)/\2, \1 (\3)/'
# Output: Doe, John (123)

Quiz

The -i Option

The -i flag edits files in place (modifies the original file). Use -i.bak to create a backup.

Examples

# Edit file.txt in place
sed -i 's/old/new/g' file.txt

# Create backup before editing
sed -i.bak 's/old/new/g' file.txt
# Creates file.txt.bak as backup

# Edit multiple files
sed -i 's/old/new/g' file1.txt file2.txt file3.txt

# Edit only .conf files
for f in *.conf; do sed -i 's/port 80/port 8080/g' "$f"; done

# In-place with regex
sed -i -E 's/[0-9]{4}-[0-9]{2}-[0-9]{2}/YYYY-MM-DD/g' log.txt

WARNING

Always create backups or test with redirection first!

# Test first (redirect to new file)
sed 's/old/new/g' file.txt > newfile.txt

# When satisfied, use -i
sed -i.bak 's/old/new/g' file.txt

Quiz

The N Command

N appends the next line to the pattern space, allowing multi-line operations.

Examples

# Join lines (replace newlines with space)
sed ':a; N; s/\n/ /g; ba' file.txt

# Delete blank lines
sed '/^$/d' file.txt
# Or
sed -n '/./p' file.txt

# Print lines 1-5 only
sed -n '1,5p' file.txt

# Find lines containing 'error' and the next line
sed -n '/error/{p; n; p}' file.txt

Practical Example

# Convert DOS line endings to Unix
sed -i 's/\r$//' file.txt

# Remove ^M characters (DOS line endings)
sed -i 's/\r//g' file.txt

Quiz

Script Examples

#!/bin/bash
# Replace URLs in config file

sed -i.bak \
    -e 's|http://old.com|http://new.com|g' \
    -e 's|8080|80|g' \
    -e 's|/var/data|/var/www|g' \
    config.txt

echo "Config updated. Backup: config.txt.bak"

Log Processing

#!/bin/bash
# Extract and format errors from log

sed -n 's/.*\(ERROR\).*/\1/p' app.log | \
    sed 's/ERROR/\[ERROR\]/g' | \
    sort | uniq -c | sort -rn

Multiple Transformations

# Multiple -e options
sed -e 's/a/A/g' \
    -e 's/e/E/g' \
    -e 's/i/I/g' \
    -e 's/o/O/g' \
    -e 's/u/U/g' \
    file.txt

# Use semicolons
sed 's/a/A/g; s/e/E/g; s/i/I/g; s/o/O/g; s/u/U/g' file.txt

Congratulations!

You've mastered sed! You now understand:

• Substitution with s command
• Flags: g, i, p
• Addressing lines and patterns
• Deleting, inserting, appending lines
• Regex patterns in sed
• Backreferences
• In-place editing
• Multi-line processing