Hidpi Mouse Click Coordinate Conversion

---
name: hidpi-mouse-click-coordinate-conversion
description: "Universal HiDPI mouse click handling for Linux desktop automation. Auto-detects scale factor or allows calibration for any screen resolution/DPI. Converts Claude display coordinates to xdotool screen coordinates."
category: "AI & Agents"
author: community
version: "1.0.0"
icon: bot
---

# HiDPI Mouse Skill

Universal mouse coordinate handling for desktop automation across different screen configurations.

## 🚀 Quick Start

```bash
# Click at Claude display coordinates (auto-scales)
./scripts/click.sh 500 300

# First time? Run calibration for best accuracy
./scripts/calibrate.sh
```

## 📐 How It Works

When Claude displays a screenshot, it scales it down. This skill converts coordinates:

```
Claude Display Coords → Scale Factor → xdotool Screen Coords
```

The scale factor depends on:
- Screen resolution (1080p, 1440p, 4K, etc.)
- DPI settings (96, 144, 192, etc.)
- Claude's display viewport

## 🔧 Scripts

### click.sh - Click at coordinates
```bash
./scripts/click.sh <x> <y>           # Auto-scaled click
./scripts/click.sh --raw <x> <y>     # No scaling (screen coords)
./scripts/click.sh --double <x> <y>  # Double click
./scripts/click.sh --right <x> <y>   # Right click
```

### calibrate.sh - Setup & Configuration
```bash
./scripts/calibrate.sh              # Interactive calibration
./scripts/calibrate.sh info         # Show current config
./scripts/calibrate.sh test         # Test current scale
./scripts/calibrate.sh set 2.08     # Manually set scale
./scripts/calibrate.sh reset        # Reset to auto-detect
```

### detect-scale.sh - Get scale factor
```bash
./scripts/detect-scale.sh           # Returns scale (e.g., 2.08)
```

### Other scripts
```bash
./scripts/move.sh <x> <y>           # Move mouse
./scripts/drag.sh <x1> <y1> <x2> <y2>  # Drag
./scripts/reliable_click.sh <x> <y> [--window "Name" --relative]
```

## 🎯 Calibration (Recommended for New Systems)

For best accuracy on your specific system:

```bash
./scripts/calibrate.sh
```

This will:
1. Create a calibration image with markers at known positions
2. Ask you where the markers appear in Claude's display
3. Calculate and save the exact scale factor

## 📊 Common Scale Factors

| Screen | DPI | Typical Scale |
|--------|-----|---------------|
| 1920×1080 | 96 | 1.0 - 1.2 |
| 2560×1440 | 96 | 1.3 - 1.5 |
| 3024×1772 | 192 | 2.08 |
| 3840×2160 | 192 | 2.0 - 2.5 |

## 🔍 Troubleshooting

### Clicks are offset
```bash
# Run calibration
./scripts/calibrate.sh

# Or manually adjust
./scripts/calibrate.sh set 2.1  # Try different values
```

### Check current configuration
```bash
./scripts/calibrate.sh info
```

### Reset everything
```bash
./scripts/calibrate.sh reset
rm -f /tmp/hidpi_scale_cache
```

## 📁 Configuration Files

- `~/.config/hidpi-mouse/scale.conf` - User-set scale (highest priority)
- `/tmp/hidpi_scale_cache` - Auto-detected scale cache (1 hour TTL)

## 🌐 Universal Compatibility

This skill auto-adapts to:
- ✅ Different screen resolutions (1080p to 4K+)
- ✅ Different DPI settings (96, 120, 144, 192, etc.)
- ✅ HiDPI/Retina displays
- ✅ Multi-monitor setups (primary display)

## 💡 Usage Tips

1. **Always calibrate** on a new system for 100% accuracy
2. **Re-calibrate** if you change display settings
3. **Use `--raw`** if you already have screen coordinates
4. **Check `calibrate.sh info`** to see current settings

## 📝 Example Workflow

```bash
# 1. Take screenshot
scrot /tmp/screen.png

# 2. View in Claude, identify button at display coords (500, 300)

# 3. Click it
./scripts/click.sh 500 300

# 4. If off-target, calibrate
./scripts/calibrate.sh
```

---

*Tested on: Ubuntu/Debian with X11, various resolutions and DPI settings*