Regex

A regular expression is a small program written as a string. You hand it to a matching engine along with some text, and the engine reports whether — and where — the text fits the shape you described.

That is the whole idea: you describe a shape, not a literal value. 2024-01-31 is a value. \d{4}-\d{2}-\d{2} is a shape that 2024-01-31 happens to fit, along with a few million other strings.

Where you actually meet regex

Regex shows up anywhere text is filtered, split, validated, or rewritten:

Place	Typical use
grep, ripgrep, ag	Find lines in files and logs
sed, awk, perl -pe	Rewrite text in a pipeline
Editors & IDEs	Find-and-replace across a project
JavaScript / TypeScript	Input validation, routing, parsing small formats
Python	Log parsing, scraping, data cleaning
Nginx, Apache	URL rewrites and location matching
SQL	~ / REGEXP operators, regexp_replace
CI, linters, log pipelines	Alert rules, redaction, field extraction

Learn it once and you get all of these. The catch is that “it” is not one language — it is a family of closely related ones.

The mental model

An engine walks the input from left to right and tries to consume characters according to your pattern. At every point it holds a current position in the input and a current position in the pattern.

Two consequences follow from that, and most confusion traces back to them:

1. A match does not have to start at the beginning. Unless you anchor it with ^, the engine will retry at position 1, 2, 3… until something matches or the input runs out. \d+ matches inside abc123 just fine.
2. The engine backtracks. When a choice it made (a *, a |, a ?) leads to a dead end, it rewinds and tries the next alternative. That is what makes regex powerful, and it is also the source of the catastrophic performance blowups covered in practical recipes.

// The same pattern, unanchored vs anchored.
/\d+/.test('abc123')    // => true  — matches "123" starting at index 3
/^\d+$/.test('abc123')  // => false — must be digits, start to end

Flavors: the same idea, four dialects

There is no single regex standard that everything follows. There are roughly four families you will hit in practice.

POSIX BRE (Basic Regular Expressions)

The oldest and most awkward. The default for grep and sed. +, ?, {, (, ), and | are literal characters — you must backslash them to get their special meaning.

echo 'aaa' | sed 's/a\+/X/'     # => X      (\+ is "one or more")
echo 'a+b' | sed 's/a+/X/'      # => Xb     (a+ is literally "a", "+")
grep 'cat\|dog' file.txt        # alternation needs \|

POSIX ERE (Extended Regular Expressions)

grep -E, sed -E, awk. The metacharacters work unescaped, which is what most people expect. Still no \d or \w in the standard — GNU adds them as extensions.

echo 'aaa' | sed -E 's/a+/X/'   # => X
grep -E 'cat|dog' file.txt
grep -E '[0-9]{2,}' file.txt    # POSIX-portable digits

PCRE (Perl-Compatible Regular Expressions)

The big one. Perl-derived syntax with \d, \w, lookaround, named groups, atomic groups, \p{...} Unicode properties. PCRE2 is the library behind grep -P, PHP’s preg_*, Nginx, and a long list of tools. Ruby, Java, and .NET are separate implementations but sit close to this dialect.

# GNU grep with PCRE: lookbehind, which ERE cannot express
echo 'foo=bar' | grep -oP '(?<=foo=)\w+'   # => bar

JavaScript and Python re

Both are PCRE-ish but each has its own quirks. They are the two you will use most from code, and they disagree in ways that bite:

JavaScript

"John Smith".replace(/(?<first>\w+) (?<last>\w+)/, "$<last>, $<first>")
// => "Smith, John"

Python

import re
re.sub(r"(?P<first>\w+) (?P<last>\w+)", r"\g<last>, \g<first>", "John Smith")
# => 'Smith, John'

Same operation, three syntactic differences: how a named group is declared, how it is referenced, and how the replacement string escapes. Flags and engines maps the differences systematically.

Caution

Never copy a regex between languages without testing it. (?P<name>...) is a syntax error in JavaScript. (?<name>...) is a syntax error in Python’s re. \d does not exist in POSIX ERE. Lookbehind is unsupported in grep -E and length-restricted in Python.

How to test a pattern

Do not write a regex directly into production code and hope. Test it against real inputs — including the ones that should not match.

In the browser or an editor: regex101.com is the standard tool. Pick the correct flavor in the left sidebar (JavaScript / Python / PCRE2 / Go), paste sample text, and read the right-hand explanation pane, which decomposes your pattern token by token. It also flags catastrophic backtracking.

From the shell, JavaScript:

node -e 'console.log(/^(\d{4})-(\d{2})-(\d{2})$/.exec("2024-01-31"))'
# => [ '2024-01-31', '2024', '01', '31', index: 0, ... ]

# Test several inputs at once
node -e '
const re = /^[a-z][a-z0-9_-]*$/;
for (const s of ["ok", "9bad", "also-ok", "Bad"]) console.log(s, re.test(s));
'

From the shell, Python:

python3 -c '
import re
print(re.findall(r"([a-z])(\d)", "a1 b2 c3"))
'
# => [("a", "1"), ("b", "2"), ("c", "3")]

From the shell, grep:

printf 'alpha 10\nbeta 200\ngamma 3\n' | grep -E '[0-9]{2,}'
# => alpha 10
# => beta 200

# -o prints only the matched part — the fastest way to see what a pattern grabs
printf 'alpha 10\nbeta 200\n' | grep -oE '[0-9]+'
# => 10
# => 200

Tip

grep -o and sed -nE 's/.../\1/p' are the two fastest feedback loops on the command line. -o shows you exactly what matched; -n plus p prints only the lines where a substitution succeeded.

Raw strings: write them, always

Regex uses \ heavily. So do string literals. If you write regex inside an ordinary quoted string, every backslash has to be doubled and you will eventually get it wrong.

re.match(r"\d+", "42")     # correct — raw string
re.match("\\d+", "42")     # same thing, harder to read
re.match("\d+", "42")      # works today, SyntaxWarning in Python 3.12+

/\d+/                            // regex literal — preferred
new RegExp('\\d+')               // string form — backslashes must be doubled
new RegExp(String.raw`\d+`)      // string form, readable

In Bash, single-quote your patterns so the shell does not eat the backslashes:

grep -E '\bcat\b' file.txt      # good
grep -E "\bcat\b" file.txt      # works, but "$" and "`" inside would expand

Mindset

Start from real data. Paste five actual lines you need to match and five you must not, then grow the pattern until it separates them.

Match the smallest thing that works. [^,]+ beats .+ almost every time, because it cannot run past the delimiter and cause backtracking.

Anchor when you mean the whole string. Validation without ^ and $ is not validation. /\d{4}/.test("year 20244x") is true.

Name what the pattern is for. A regex assigned to const ISO_DATE = /.../ is documentation. The same regex inline in a condition is a puzzle.

Comment anything long. Python’s re.X and PCRE’s x flag let you break a pattern across lines with comments. Use them.

When NOT to use regex

Regular expressions describe regular languages. Anything with arbitrary nesting is not regular, and no amount of cleverness fixes that.

Danger

Do not parse HTML or XML with regex. Tags nest arbitrarily deep, attributes can contain > inside quotes, comments and CDATA sections can contain anything, and closing tags are optional in HTML. Use a real parser: DOMParser / cheerio in JS, lxml / BeautifulSoup in Python, htmlq or pup on the CLI.

The same applies to:

• JSON, YAML, TOML — nested and quoted; use JSON.parse, json.loads, jq, yq.
• CSV — fields can contain quoted commas and embedded newlines. Use csv (Python) or a proper CSV library.
• Source code — comments, strings, and nesting make it a parser’s job, not a matcher’s. Use an AST tool (ast in Python, @babel/parser, tree-sitter, ast-grep).
• URLs and email addresses — parse with new URL(...) / urllib.parse. Full RFC 5322 email validation by regex is a famous multi-kilobyte joke; send a confirmation email instead.
• Balanced delimiters — matching (a(b)c) correctly requires counting. Some engines bolt on recursion ((?R) in PCRE), but if you need it, you need a parser.

Regex is excellent at tokenizing flat text: one line, one field, one known shape. It is the wrong tool the moment structure nests.

Key points

• A regex describes a shape; the engine scans and backtracks to find text matching it.
• Without ^ and $, a match can occur anywhere inside the input.
• Four flavors matter: POSIX BRE, POSIX ERE, PCRE, and the JS / Python variants. Syntax differs — test before porting.
• Test with regex101, node -e, python3 -c, or grep -oE before shipping.
• Use raw strings (r"...", String.raw) and single-quote patterns in Bash.
• Nested structure (HTML, JSON, code) needs a parser, not a regex.