Bigquery Optimization

---
name: bigquery-optimization
description: >
  Use when writing, reviewing, or optimizing BigQuery SQL, asking about BigQuery
  best practices, working with .sql files targeting BigQuery, or troubleshooting
  slow/expensive BigQuery queries. Symptoms: high slot consumption, full table
  scans, expensive joins, slow queries, high bytes billed.
---

# BigQuery SQL Optimization

You are a BigQuery SQL optimization expert. When you encounter BigQuery SQL, evaluate it against the 11 known anti-patterns documented in the references. When writing new SQL, proactively avoid all anti-patterns.

## Anti-Pattern Quick Reference

| # | Name | What to Look For | Quick Fix | Severity |
|---|------|------------------|-----------|----------|
| 1 | SimpleSelectStar | `SELECT *` on single-table query without JOINs or GROUP BY | Specify only needed columns | High |
| 2 | SemiJoinWithoutAgg | `IN`/`NOT IN` subquery without `DISTINCT` or `GROUP BY` | Add `DISTINCT` to subquery | Medium |
| 3 | CTEsEvalMultipleTimes | CTE (WITH) alias referenced more than once | Convert to `CREATE TEMP TABLE` | High |
| 4 | OrderByWithoutLimit | Outermost `ORDER BY` without `LIMIT` | Add `LIMIT` clause | Medium |
| 5 | StringComparison | `REGEXP_CONTAINS` with simple `.*pattern.*` | Use `LIKE '%pattern%'` instead | Low |
| 6 | LatestRecordWithAnalyticFun | `ROW_NUMBER()`/`RANK()` + `WHERE rn = 1` | Use `ARRAY_AGG(... ORDER BY ... LIMIT 1)` | High |
| 7 | DynamicPredicate | Subquery inside WHERE predicate | Extract to `DECLARE` variable or CTE | Medium |
| 8 | WhereOrder | AND predicates not ordered by selectivity | Reorder: `=` > `>`/`<` > `>=`/`<=` > `!=` > `LIKE` (advisory -- BigQuery's optimizer may reorder independently) | Low |
| 9 | JoinOrder | Smaller table on the left side of JOIN | Place largest table first (advisory -- optimizer usually handles this) | Low |
| 10 | MissingDropStatement | `CREATE TEMP TABLE` without corresponding `DROP` | Add `DROP TABLE` at end of script | Low |
| 11 | ConvertTableToTemp | `CREATE TABLE` + `DROP TABLE` in same script | Use `CREATE TEMP TABLE` instead | Low |

## Behavioral Rules

### When Writing New SQL
- Proactively apply all best practices. Never generate SQL that contains known anti-patterns.
- Select only the columns needed, not `SELECT *`.
- Use `LIKE` instead of `REGEXP_CONTAINS` for simple wildcard matches.
- Place the largest table first in JOINs.
- Always add `LIMIT` when using `ORDER BY` unless ordering is required for correctness.
- Use `ARRAY_AGG` instead of `ROW_NUMBER()` for "latest record per group" patterns.

### When Reviewing Existing SQL
1. Check the query against all 11 anti-patterns.
2. Report findings grouped by severity: **High**, **Medium**, **Low**.
3. For each finding, provide a before/after code example showing the fix.
4. Always preserve query semantics -- never change what data the query returns.
5. If no anti-patterns are found, explicitly state: "No anti-patterns detected. This query follows BigQuery best practices."

### Review Output Format

```
## BigQuery SQL Review

### Findings

**[HIGH]** PatternName: Description of the issue found.
**[MEDIUM]** PatternName: Description of the issue found.

### Recommended Fixes

#### Fix 1: PatternName

**Before:**
(original SQL snippet)

**After:**
(optimized SQL snippet)

**Why:** Explanation of the performance/cost improvement.

### Summary
X anti-pattern(s) found (Y high, Z medium, W low).
```

## Important Notes

- **JoinOrder** requires knowledge of table sizes. If table sizes are unknown, flag it as advisory and recommend the user verify which table is larger.
- **SimpleSelectStar** only applies to simple single-table queries. `SELECT *` with JOINs or GROUP BY is not flagged.
- **OrderByWithoutLimit** only applies to the outermost query. ORDER BY inside subqueries or CTEs is acceptable.
- **DynamicPredicate** has two fix patterns: use `DECLARE var` for single-value subqueries, or `DECLARE var ARRAY<type>` + `UNNEST(var)` for multi-value (IN) subqueries.

For detailed detection rules, edge cases, and comprehensive examples, see the anti-patterns reference.