jq
MASTERY

What is jq?

jq is a lightweight command-line JSON processor. It lets you filter, transform, and extract data from JSON with a powerful, expressive syntax.

Installation

# Debian/Ubuntu
sudo apt install jq

# macOS
brew install jq

# Check version
jq --version

Basic Syntax

jq 'FILTER' [FILE...]
echo 'JSON' | jq 'FILTER'

Simple Examples

# Pretty print JSON
echo '{"name":"Alice","age":30}' | jq '.'

# Identity filter (.)
cat data.json | jq '.'

# Validate JSON
cat data.json | jq . > /dev/null && echo "Valid JSON"

Quiz

Identity Filter (.)

The dot (.) filter returns the entire input unchanged. It's the most basic jq filter.

Examples

# Identity - output same as input
echo '{"a":1,"b":2}' | jq '.'

# Array
echo '[1,2,3]' | jq '.'

# Nested
echo '{"user":{"name":"Bob"}}' | jq '.'

Quiz

Dot Notation

Access object properties with dot notation: `.key`

Examples

# Access property
echo '{"name":"Alice","age":30}' | jq '.name'

# Nested access
echo '{"user":{"name":"Bob","email":"bob@example.com"}}' | jq '.user.email'

# Multiple properties
echo '{"name":"Alice","age":30}' | jq '.name, .age'

# Optional: ? for missing keys
echo '{"name":"Alice"}' | jq '.age?'

Bracket Notation

# Bracket notation (useful for special chars)
echo '{"user-name":"Alice"}' | jq '.["user-name"]'

# Keys with spaces
echo '{"first name":"Alice"}' | jq '.["first name"]'

Quiz

Array Indices

# First element
echo '[10,20,30]' | jq '.[0]'

# Last element
echo '[10,20,30]' | jq '.[-1]'

# Second element
echo '[10,20,30]' | jq '.[1]'

# Array length
echo '[10,20,30]' | jq 'length'

Slicing

# First two elements
echo '[10,20,30,40]' | jq '.[:2]'

# Elements 1-3 (indices 1, 2, 3)
echo '[10,20,30,40]' | jq '.[1:4]'

# Last two elements
echo '[10,20,30,40]' | jq '.[-2:]'

Quiz

Chaining Filters

The pipe (|) passes output of one filter as input to the next, just like shell pipes.

Examples

# Get array, then first element
echo '{"items":[10,20,30]}' | jq '.items | .[0]'

# Shorthand
echo '{"items":[10,20,30]}' | jq '.items[0]'

# Chain multiple
echo '{"user":{"name":"Alice","posts":[{"title":"Hello"}]}}' | jq '.user.posts[0].title'

# Get all titles
echo '{"posts":[{"title":"A"},{"title":"B"}]}' | jq '.posts[].title'

Recursion Operator

# Recursively descend
echo '{"a":{"b":{"c":1}}}' | jq '..'

# Find all objects with key
echo '{"name":"Alice"}' | jq '.. | objects'

Quiz

Create Objects

# Object construction
echo '{}' | jq '{name: "Alice", age: 30}'

# From input
echo '{"first":"John","last":"Doe"}' | jq '{fullname: .first, last: .last}'

# With computed values
echo '{"name":"Alice","age":30}' | jq '{name: .name, adult: .age >= 18}'

Create Arrays

# Array construction
echo '[]' | jq '[.a, .b, .c]'

# From array
echo '[1,2,3]' | jq '[.[] | . * 2]'

# Collecting
echo '{"items":[1,2,3]}' | jq '[.items[] | select(. > 1)]'

Quiz

if-then-else

# Basic if
echo '{"age":25}' | jq 'if .age >= 18 then "adult" else "minor" end'

# Multiple conditions
echo '{"score":85}' | jq 'if .score >= 90 then "A" elif .score >= 80 then "B" else "C" end'

Comparisons

==    Equal
!=    Not equal
< >   Less, greater
<= >=  Less or equal, greater or equal
and   Boolean AND
or     Boolean OR
not   Boolean NOT

Quiz

Common Functions

map(FILTER)     Apply filter to each element
select(COND)    Keep elements matching condition
flatten         Flatten nested arrays
unique          Remove duplicates
sort            Sort arrays
keys            Get object keys
values          Get object values
type            Get value type

Examples

# Double each element
echo '[1,2,3]' | jq 'map(. * 2)'

# Select adults
echo '[{"age":10},{"age":25},{"age":17}]' | jq 'map(select(.age >= 18))'

# Get keys
echo '{"a":1,"b":2}' | jq 'keys'

# Unique and sort
echo '[3,1,2,3,1]' | jq 'unique | sort'

Quiz

String Functions

+        Concatenate strings
*        Repeat string
split(S)  Split into array
join(S)   Join array with separator
ascii_downcase  lowercase
ascii_upcase    UPPERCASE
ltrimstr(S)   Remove prefix
rtrimstr(S)   Remove suffix

Examples

# Concatenate
echo '"Hello"' | jq '. + " World"'

# Uppercase
echo '"hello"' | jq 'ascii_upcase'

# Split
echo '"a,b,c"' | jq 'split(",")'

# String interpolation with +
echo '{"name":"Alice"}' | jq '"Hello, " + .name'

Quiz

curl + jq = Power

Query APIs and extract exactly what you need with curl piped to jq.

Examples

# GitHub API - get repo stars
curl -s api.github.com/repos/torvalds/linux | jq '.stargazers_count'

# Get all repo names
curl -s api.github.com/users/torvalds/repos | jq '.[].name'

# Get user's followers
curl -s api.github.com/users/torvalds | jq '{login, followers, public_repos}'

Practical Examples

# Weather API (example)
curl -s wttr.in/London.json | jq '{temp: .current_condition.temp_C, condition: .current_condition.weatherDesc[0].value}'

# Docker stats JSON
docker stats --no-stream --format '{{json .}}' | jq '{name: .Name, cpu: .CPUPerc, mem: .MemPerc}'

Quiz

Variable Binding

# Assign to variable
echo '{"name":"Alice"}' | jq '.name as $name | "Hello \($name)"'

# Multiple variables
echo '{"first":"John","last":"Doe"}' | jq '.first as $f | .last as $l | "\($f) \($l)"'

let keyword (jq 1.7+)

# Define constant
echo '5' | jq 'let $x = 10; . + $x'

Quiz

Bash Script Examples

#!/bin/bash
# Fetch and parse API data
USERNAME="torvalds"
REPOS=$(curl -s "https://api.github.com/users/$USERNAME/repos" | jq -r '.[].name')

echo "Repos for $USERNAME:"
for repo in $REPOS; do
    stars=$(curl -s "https://api.github.com/repos/$USERNAME/$repo" | jq '.stargazers_count')
    echo "  $repo: $stars stars"
done

Process JSON Logs

#!/bin/bash
# Extract errors from JSON logs
jq -r 'select(.level=="error") | "\(.timestamp) \(.message)"' app.log.json > errors.txt

# Generate summary
jq '{errors: map(select(.level=="error")) | length, warnings: map(select(.level=="warning")) | length}' app.log.json

Congratulations!

You've mastered jq! You now understand:

• Basic filters and object/array access
• Pipe chaining
• Object and array construction
• Conditionals and comparisons
• Built-in functions (map, select, keys)
• String operations
• API integration with curl
• Variables and string interpolation