Python

---
name: python
description: "Python 3.10-3.14 scripting and IT automation: file operations (pathlib, shutil, csv, json, yaml, toml), subprocess management, HTTP/API interaction (requests with retry/pagination), logging (basicConfig, handlers, JSON), argument parsing (argparse, click), regex, error handling patterns, system administration (psutil, platform), SSH (paramiko, fabric), email (smtplib), scheduling, and data processing (pandas). WHEN: \"Python\", \"python\", \"python script\", \"pip\", \"venv\", \"pathlib\", \"argparse\", \"requests\", \"subprocess\", \"psutil\", \"shutil\", \"csv\", \"json\", \"yaml\", \"toml\", \"paramiko\", \"fabric\", \"logging\", \"schedule\", \"click\", \"typer\", \".py\", \"python automation\". Do NOT use for Django/Flask/FastAPI web-framework questions — that's the `backend` plugin. This skill covers Python for scripting and automation, not web-service development."
license: MIT
---

# Python Scripting & Automation Expert

This skill covers Python 3.10-3.14 for scripting, IT automation, and system administration. It has deep coverage of:

- File operations: pathlib (preferred over os.path), shutil (copy/move/archive), tempfile, os module
- Data formats: csv, json, yaml (PyYAML), toml (tomllib 3.11+), XML
- Subprocess: subprocess.run, Popen for streaming, shlex for safe command building
- HTTP/API: requests (sessions, retry, pagination), httpx (async)
- Logging: basicConfig, handlers (rotating, timed), JSON formatter, LoggerAdapter
- Argument parsing: argparse (positional, optional, subcommands, mutually exclusive), click
- Regular expressions: re module (search, findall, sub, compile, named groups, flags)
- Error handling: try/except/else/finally, custom exceptions, exception chaining, contextlib.suppress
- System admin: platform, socket, psutil (CPU, memory, disk, network, processes)
- SSH/Remote: paramiko (SSHClient, SFTP), fabric (Connection, SerialGroup)
- Email: smtplib, email.mime (text, multipart, attachments)
- Scheduling: schedule, APScheduler, cron integration
- Data processing: pandas (read/write CSV/Excel/JSON, filter, groupby, merge, pivot)

## How to Approach Tasks

1. **Classify** the request:
   - **File/data processing** -- Load `references/patterns.md`
   - **System admin/SSH/email** -- Load `references/automation.md`
   - **Version-specific features** -- Check the Version-specific guidance table below

2. **Choose the right abstraction:**
   - Use `pathlib.Path` over `os.path` for all path operations
   - Use `subprocess.run()` with a list (not string) for shell commands
   - Use `requests.Session()` for multiple API calls
   - Use `argparse` for scripts, `click` for complex CLI tools

3. **Apply Python idioms:**
   - Type hints on function signatures
   - Context managers (`with`) for file/resource management
   - f-strings for formatting
   - `logging` module over `print()` for operational output

4. **Always include:**
   - Proper error handling with specific exception types
   - A `main()` function with `if __name__ == "__main__"`
   - `argparse` for any script that accepts parameters
   - `logging` setup for any script longer than 20 lines

## Core Expertise Overview

### File Operations

```python
from pathlib import Path
import shutil

p = Path.home() / ".config" / "myapp" / "settings.json"
text = p.read_text(encoding="utf-8")
p.mkdir(parents=True, exist_ok=True)
for f in Path("./data").rglob("**/*.csv"): print(f)

shutil.copytree(src, dst, dirs_exist_ok=True)
shutil.make_archive("backup", "zip", root_dir="./data")
```

### Subprocess

```python
import subprocess
result = subprocess.run(
    ["git", "status", "--short"],
    capture_output=True, text=True, check=True, timeout=30,
)
```

### HTTP/API

```python
import requests
from requests.adapters import HTTPAdapter
from urllib3.util.retry import Retry

session = requests.Session()
retry = Retry(total=3, backoff_factor=1, status_forcelist=[429, 500, 502, 503])
session.mount("https://", HTTPAdapter(max_retries=retry))
resp = session.get(url, timeout=10)
resp.raise_for_status()
```

### Argument Parsing

```python
import argparse
parser = argparse.ArgumentParser(description="Process files")
parser.add_argument("input", type=Path, help="Input file")
parser.add_argument("-f", "--format", choices=["json", "csv"], default="json")
parser.add_argument("-v", "--verbose", action="store_true")
args = parser.parse_args()
```

## Common Pitfalls

**1. Using `os.path` instead of `pathlib`**
`pathlib.Path` is more readable and supports `/` operator for path joining. Prefer it for all new code.

**2. Using `subprocess.run()` with `shell=True`**
Command injection risk. Pass a list: `subprocess.run(["git", "status"])`, not a string.

**3. Not specifying `encoding="utf-8"` for file operations**
Default encoding varies by platform. Always explicit: `open(f, encoding="utf-8")`.

**4. Bare `except:` clauses**
Never use `except:` or `except Exception:` without logging the error. Catch specific exceptions.

**5. Using `print()` instead of `logging`**
`print()` goes to stdout and cannot be configured. Use `logging` for operational output.

**6. Not using `timeout` with `requests`**
Without timeout, HTTP calls can hang indefinitely. Always set `timeout=10` or similar.

**7. Mutable default arguments**
`def f(items=[])` shares the list across calls. Use `def f(items=None): items = items or []`.

**8. Not closing resources**
Always use `with` statements for files, HTTP sessions, database connections.

## Version-specific guidance

| Version | Reference | What's version-specific |
|---|---|---|
| 3.10 | `references/versions/3.10.md` | match/case structural pattern matching, type union `X \| Y`, parenthesized context managers |
| 3.12 | `references/versions/3.12.md` | Type parameter syntax `[T]`, `type` keyword, f-string improvements (nested quotes, backslashes) |
| 3.14 | `references/versions/3.14.md` | Template strings `t"..."`, deferred annotation evaluation |

## Reference Files

- `references/patterns.md` -- File ops, subprocess, HTTP/API, logging, argparse, regex. Dense with examples.
- `references/automation.md` -- System admin (psutil, platform), SSH (paramiko, fabric), email, scheduling, data processing (pandas).

## Example Scripts

- `scripts/01-system-report.py` -- Cross-platform system report with psutil
- `scripts/02-api-client.py` -- REST API client class with retry/pagination
- `scripts/03-csv-processor.py` -- CSV/JSON processor with argparse and filtering
- `scripts/04-file-organizer.py` -- File organizer by date/type with undo support