Provider 403 Ua Fix

---
name: provider-403-ua-fix
description: Fix HTTP 403 blocks from custom API providers (WAF/Cloudflare) that reject SDK User-Agent headers. Fixed UA is set in run_agent.py; no more rotation pool.
---

# Provider 403 UA Fix

## Symptom
- curl to relay endpoint works (200), but Hermes gets `403 Your request was blocked`.
- SDK sends `User-Agent: Anthropic/JS x.x.x` or `User-Agent: OpenAI/JS x.x.x` — WAF blocks it.

## Root Cause
Official SDKs set their own UA header. Many relay providers behind Cloudflare/WAF block these patterns.

**Key Discovery (2026-05-08)**: The code does NOT automatically inject browser UA for custom providers. The `custom_providers[].headers` field in config.yaml is **completely ignored** by the codebase. Manual intervention required.

**Critical Discovery (2026-05-08 Session 2)**: `auxiliary_client.py` has **hardcoded `claude-code/0.1.0` UA in 7+ locations**. Even after fixing `run_agent.py`, auxiliary operations (title generation, compression, vision fallback) still send agent fingerprints and get blocked.

## Files Requiring Modification

| File | Locations | Purpose |
|------|-----------|---------|
| `run_agent.py` | Lines ~1466-1476, ~6346-6360 | Main client initialization |
| `agent/auxiliary_client.py` | Lines ~1281, ~1316, ~2143, ~2365, ~2552 | Auxiliary clients (title gen, compression, vision) |

## Complete Fix

### 1. run_agent.py — Init-time (~line 1466-1490)

Find the `elif "default_headers" not in client_kwargs:` branch and modify the else/except blocks:

```python
elif "default_headers" not in client_kwargs:
    try:
        from providers import get_provider_profile as _gpf
        _ph = _gpf(self.provider)
        if _ph and _ph.default_headers:
            client_kwargs["default_headers"] = dict(_ph.default_headers)
        else:
            # Inject browser UA for custom endpoints to bypass CF
            client_kwargs["default_headers"] = {
                "User-Agent": "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/147.0.0.0 Safari/537.36"
            }
    except Exception:
        # Fallback: inject browser UA
        client_kwargs["default_headers"] = {
            "User-Agent": "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/147.0.0.0 Safari/537.36"
        }
```

### 2. run_agent.py — _apply_client_headers_for_base_url (~line 6346-6363)

Find the `else:` branch after URL-specific checks and change from `pop` to inject:

```python
else:
    # No URL-specific headers — check profile.default_headers
    _ph_headers = None
    try:
        from providers import get_provider_profile as _gpf2
        _ph2 = _gpf2(self.provider)
        if _ph2 and _ph2.default_headers:
            _ph_headers = dict(_ph2.default_headers)
    except Exception:
        pass
    if _ph_headers:
        self._client_kwargs["default_headers"] = _ph_headers
    else:
        # Inject browser UA for custom endpoints to bypass CF agent fingerprinting
        self._client_kwargs["default_headers"] = {
            "User-Agent": "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/147.0.0.0 Safari/537.36"
        }
```

### 3. auxiliary_client.py — Multiple locations

**Location 1** (~line 1286-1295): After `api.kimi.com` and `api.githubcopilot.com` checks:
```python
else:
    try:
        from providers import get_provider_profile as _gpf_aux
        _ph_aux = _gpf_aux(provider_id)
        if _ph_aux and _ph_aux.default_headers:
            extra["default_headers"] = dict(_ph_aux.default_headers)
        else:
            extra["default_headers"] = {"User-Agent": "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/147.0.0.0 Safari/537.36"}
    except Exception:
        extra["default_headers"] = {"User-Agent": "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/147.0.0.0 Safari/537.36"}
```

**Location 2** (~line 1325-1333): Same pattern after second kimi/copilot check.

**Location 3** (~line 2143-2152): After async client kimi check, add else:
```python
elif base_url_host_matches(sync_base_url, "api.kimi.com"):
    async_kwargs["default_headers"] = {"User-Agent": "claude-code/0.1.0"}
else:
    async_kwargs["default_headers"] = {"User-Agent": "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/147.0.0.0 Safari/537.36"}
```

**Location 4** (~line 2375-2382): After custom_base kimi/copilot checks, add else.

**Location 5** (~line 2563-2574): After provider-specific headers setup, add else:
```python
else:
    headers["User-Agent"] = "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/147.0.0.0 Safari/537.36"
```

## Pitfalls

1. **Do NOT set `default_headers` as class attribute** in ProviderProfile subclasses — dataclass `field(default_factory=dict)` ignores it. Pass at instantiation.

2. **`custom_providers[].headers` is ignored** — this config field is never read by any code path.

3. **auxiliary_client.py is separate** — fixing only `run_agent.py` leaves title generation and other auxiliary ops still blocked.

4. **Keep kimi.com using `claude-code/0.1.0`** — Kimi's API specifically requires this UA pattern.

## config.yaml (BROKEN — Do NOT use)

```yaml
custom_providers:
  my-relay:
    headers:
      User-Agent: "Mozilla/5.0"  # THIS IS IGNORED BY CODE
```

The `custom_providers[].headers` field is **never read** by any code path. Only `name`, `base_url`, `api_key`, and `models` are processed.

## DO NOT touch
- **hermes_cli/main.py `_save_custom_provider`**: adding `raw_headers` param here caused `NameError`. Leave unchanged.
- **agent/ua_rotation.py**: deleted. Do not recreate.
- **`custom_providers[].headers`**: This config field is ignored — do not attempt to use it for UA override.

## Verification
```bash
# Test with SDK UA (blocked):
curl -H "User-Agent: OpenAI/Python" -H "Authorization: Bearer $KEY" https://relay.example.com/v1/chat/completions -d '{"model":"gpt-4","messages":[{"role":"user","content":"hi"}]}'

# Test with browser UA (should work):
curl -H "User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36" -H "Authorization: Bearer $KEY" https://relay.example.com/v1/chat/completions -d '{"model":"gpt-4","messages":[{"role":"user","content":"hi"}]}'
```

## Notes
- Browser UA is injected for ALL custom/third-party endpoints after patching.
- OpenRouter/Copilot/Kimi/Qwen have their own dedicated URL-matched headers — unchanged.
- If CF still blocks after UA fix, it may be checking deeper fingerprints (TLS JA3, HTTP/2). In that case, use a local proxy to rewrite all request characteristics.
- **Path difference**: Some relay sites use `/pg/chat/completions` (playground) vs `/v1/chat/completions` (standard). This may affect UA/auth policies.