896 Feature Pdf To Epub Conversion 4078e23f

---
phase: planning
title: PDF to EPUB Conversion - Planning
description: Task breakdown and implementation plan
feature: pdf-to-epub-conversion
---

# Implementation Plan

## Overview
This plan covers Phase 5 of the Product Vision: implementing the conversion pipeline that transforms extracted PDF structure into valid EPUB files.

## Prerequisites
**Completed Dependencies:**
- ✅ Phase 1-4: PDFExtractor, TextSegmenter, reading order sorters, structure detectors
- ✅ Validation components: CompletenessChecker, OrderChecker, TextCanonicalizer
- ✅ Test fixtures and integration test framework

**Required Before Start:**
- Review Product Vision Phase 5 section
- Review design document (feature-pdf-to-epub-conversion.md)
- Verify all existing tests pass (56/56)

## Task Breakdown

### Phase 1: Foundation (2-3 hours)

#### Task 1.1: Implement BaseStrategy
**File:** `pdf_to_epub/conversion/strategies/base_strategy.py`
**Description:** Create abstract base class defining strategy interface
**Details:**
- Define template method `convert(pdf_path, config) -> StructuredContent`
- Add hook methods: `extract()`, `order_blocks()`, `detect_structure()`
- Add utility method: `_validate_config(config)`
- Include comprehensive docstrings

**Acceptance:**
- [x] BaseStrategy is abstract (cannot be instantiated)
- [x] All hook methods raise NotImplementedError
- [x] Type hints for all parameters and return values

**Status:** ✅ Completed - Dec 30, 2025

#### Task 1.2: Create StructuredContent data models
**File:** `pdf_to_epub/conversion/models.py` (new)
**Description:** Define data classes for structured content representation
**Details:**
- `StructuredContent` (chapters, metadata, confidence)
- `Chapter` (title, level, content, footnotes)
- `Footnote` (marker, id, text)
- `ConversionResult` (epub_path, status, confidence, log)
- `ConversionLog` (timestamp, strategy, config, steps, warnings, errors)

**Acceptance:**
- [x] All models use `@dataclass` decorator
- [x] Include `from_dict()` and `to_dict()` methods
- [x] Add validation in `__post_init__()` if needed

**Status:** ✅ Completed - Dec 30, 2025 (12 dataclasses created)

### Phase 2: Simple Strategy (4-5 hours)

#### Task 2.1: Implement SimpleStrategy.extract()
**File:** `pdf_to_epub/conversion/strategies/simple_strategy.py` (new)
**Description:** Extract text blocks, images, and metadata from PDF
**Details:**
- Call `PDFExtractor` with config parameters
- Apply `exclude_regions` from config
- Extract images using PyMuPDF image extraction
- Extract metadata from PDF info dict (title, author, subject, keywords, etc.)
- Return list of `TextBlock` objects, images, and metadata

**Acceptance:**
- [x] Uses existing `PDFExtractor` from core/
- [x] Applies top/bottom/left/right exclusions correctly
- [x] Extracts images in PNG/JPEG/GIF formats
- [x] Reads metadata from PDF info (fallback to "Unknown" if missing)
- [x] Returns blocks with bbox, text, font info

**Status:** ✅ Completed - Dec 30, 2025

#### Task 2.2: Implement SimpleStrategy.order_blocks()
**File:** Same as 2.1
**Description:** Sort blocks into reading order
**Details:**
- Use `YSorter` from detectors/reading_order/
- Return ordered blocks + confidence score
- Confidence = 1.0 for Y-sort (always deterministic)

**Acceptance:**
- [x] Blocks are sorted top-to-bottom
- [x] Returns tuple: (ordered_blocks, 1.0)
- [x] No blocks are dropped or duplicated

**Status:** ✅ Completed - Dec 30, 2025

#### Task 2.3: Implement SimpleStrategy.detect_structure()
**File:** Same as 2.1
**Description:** Classify blocks and build chapter structure with metadata
**Details:**
- Use `FontAnalyzer` to identify heading fonts
- Use `StructureClassifier` to mark headings vs paragraphs
- Use `StructureBuilder` to group into chapters
- Create `StructuredContent` with chapters, metadata, and images

**Acceptance:**
- [x] Chapters have titles extracted from H1 blocks
- [x] Paragraphs are concatenated as HTML `<p>` tags
- [x] Empty chapters are handled (e.g., books without chapter markers)
- [x] Metadata (title, author, language) included in StructuredContent
- [x] Images list included in StructuredContent
- [x] Returns valid `StructuredContent` object

**Status:** ✅ Completed - Dec 30, 2025

### Phase 3: EPUB Builder (5-6 hours)

#### Task 3.1: Implement EPUBBuilder.build() - Core Structure
**File:** `pdf_to_epub/core/epub_builder.py`
**Description:** Create EPUB3 directory structure and package with metadata
**Details:**
- Create temp directory with EPUB structure
- Write `mimetype` file (no compression)
- Write `META-INF/container.xml`
- Write `OEBPS/content.opf` with metadata (title, author, language, ISBN) and spine

**Acceptance:**
- [x] `mimetype` is first file, uncompressed, contains "application/epub+zip"
- [x] `container.xml` points to `OEBPS/content.opf`
- [x] `content.opf` includes all XHTML files in spine
- [x] `content.opf` includes extracted metadata (title, author, language, publisher, ISBN)

**Status:** ✅ Completed - Dec 30, 2025

#### Task 3.2: Implement EPUBBuilder - Content and Images
**File:** Same as 3.1
**Description:** Generate XHTML chapter files and embed images
**Details:**
- Create `chapterN.xhtml` for each chapter
- Use XHTML5 template with proper DOCTYPE
- Include CSS stylesheet link
- Escape HTML entities in content
- Save images to `OEBPS/images/` folder
- Add `<img>` tags in XHTML where images appear
- Update content.opf manifest with image items

**Acceptance:**
- [x] XHTML files are well-formed (validate with lxml)
- [x] Chapter titles are in `<h1>` tags
- [x] Paragraphs are in `<p>` tags
- [x] Special characters are escaped (e.g., &, <, >)
- [x] Images saved as `OEBPS/images/img-page-N-M.{png|jpg|gif}`
- [x] XHTML includes `<img src="images/img-page-N-M.png"/>` tags
- [x] content.opf manifest includes all image items

**Status:** ✅ Completed - Dec 30, 2025

#### Task 3.3: Implement EPUBBuilder - TOC and Packaging
**File:** Same as 3.1
**Description:** Create navigation and ZIP the EPUB
**Details:**
- Generate `toc.ncx` with chapter hierarchy
- Create basic `stylesheet.css`
- ZIP all files maintaining structure
- Rename to `.epub` extension

**Acceptance:**
- [x] TOC includes all chapters with correct titles
- [x] ZIP uses appropriate compression (store for mimetype, deflate for rest)
- [x] Final file has `.epub` extension
- [ ] File validates with epubcheck (manual check) - Pending testing

**Status:** ✅ Completed - Dec 30, 2025 (epubcheck validation pending)

### Phase 4: Converter Orchestrator (3-4 hours)

#### Task 4.1: Implement Converter.convert() - Core Logic
**File:** `pdf_to_epub/conversion/converter.py`
**Description:** Orchestrate conversion workflow
**Details:**
- Load config from JSON or use defaults
- Select strategy by name ("simple", "academic", etc.)
- Call `strategy.convert()`
- Call `epub_builder.build()`
- Generate `ConversionLog`

**Acceptance:**
- [x] Handles missing config gracefully (use defaults)
- [x] Raises clear error for unknown strategy name
- [x] Returns `ConversionResult` with all fields populated

**Status:** ✅ Completed - Dec 30, 2025

#### Task 4.2: Implement Converter - Error Handling
**File:** Same as 4.1
**Description:** Handle errors and generate logs
**Details:**
- Catch exceptions at each step
- Log errors with context (step name, file path)
- Mark `ConversionResult.status` as "failed" on error
- Always write log, even on failure

**Acceptance:**
- [x] Encrypted PDFs fail with clear message
- [x] Missing files fail with path in error
- [x] Logs include timestamp, strategy, config
- [x] Failed conversions still return `ConversionResult` (not exception)

**Status:** ✅ Completed - Dec 30, 2025

#### Task 4.3: Implement Config Loading and Validation
**File:** `pdf_to_epub/conversion/converter.py`
**Description:** Load and validate conversion config (fail-fast)
**Details:**
- Load from JSON file if provided
- Merge with default config
- Validate required fields immediately (metadata.title, metadata.author)
- Validate types (exclude_regions 0.0-1.0, page_ranges are ints)
- Validate enums (reading_order_strategy in allowed values)
- Raise ValueError with specific field name if invalid

**Acceptance:**
- [x] Invalid JSON raises clear error
- [x] Missing optional fields use defaults
- [x] Config validation checks types and ranges before conversion starts
- [x] Validation errors specify which field is invalid

**Status:** ✅ Completed - Dec 30, 2025

### Phase 5: CLI Integration (2-3 hours)

#### Task 5.1: Implement analyze.py CLI
**File:** `pdf_to_epub/scripts/analyze.py`
**Description:** CLI to analyze PDF structure without converting
**Details:**
- Parse arguments: `--pdf`, `--config`, `--output`
- Run extraction + structure detection
- Output structured JSON report

**Acceptance:**
- [x] `python -m pdf_to_epub.scripts.analyze --pdf book.pdf` works
- [x] Output includes chapter titles, block count, confidence
- [x] Handles errors gracefully

**Status:** ✅ Completed - Dec 30, 2025

#### Task 5.2: Implement convert.py CLI
**File:** `pdf_to_epub/scripts/convert.py`
**Description:** CLI to convert PDF to EPUB
**Details:**
- Parse arguments: `--pdf`, `--output`, `--config`, `--strategy`
- Call `Converter.convert()`
- Print progress and result

**Acceptance:**
- [x] `python -m pdf_to_epub.scripts.convert --pdf book.pdf --output book.epub` works
- [x] Shows progress messages (extracting, ordering, detecting, building)
- [x] Prints conversion log on completion

**Status:** ✅ Completed - Dec 30, 2025

#### Task 5.3: Implement validate.py CLI
**File:** `pdf_to_epub/scripts/validate.py`
**Description:** CLI to validate EPUB against PDF
**Details:**
- Parse arguments: `--pdf`, `--epub`, `--config`
- Run completeness + order checks
- Print validation report

**Acceptance:**
- [x] Uses existing `CompletenessChecker` and `OrderChecker`
- [x] Prints pass/fail for each check
- [x] Returns exit code 0 for pass, 1 for fail

**Status:** ✅ Completed - Dec 30, 2025

## Dependencies

### External Dependencies
- PyMuPDF (fitz) - already installed
- lxml - already installed
- None new required

### Internal Dependencies
```
SimpleStrategy depends on:
  ├─ PDFExtractor (core/)
  ├─ YSorter (detectors/reading_order/)
  ├─ FontAnalyzer (detectors/)
  ├─ StructureClassifier (detectors/)
  └─ StructureBuilder (detectors/)

EPUBBuilder depends on:
  ├─ lxml (for XML generation)
  └─ zipfile (for packaging)

Converter depends on:
  ├─ BaseStrategy (strategies/)
  ├─ SimpleStrategy (strategies/)
  └─ EPUBBuilder (core/)

Scripts depend on:
  ├─ Converter (conversion/)
  ├─ CompletenessChecker (validation/)
  └─ OrderChecker (validation/)
```

## Testing Strategy

### Unit Tests
- `test_base_strategy.py` - Test abstract interface
- `test_simple_strategy.py` - Test each method independently
- `test_epub_builder.py` - Test EPUB structure and packaging
- `test_converter.py` - Test orchestration and error handling

### Integration Tests
- `test_end_to_end_conversion.py` - PDF → EPUB with fixture
- `test_validation_integration.py` - PDF → EPUB → Validate

### Test Coverage Target
- Minimum: 80% line coverage
- Critical paths: 100% coverage (error handling, config validation)

## Milestones

### M1: Foundation Complete ✅ (Dec 30, 2025)
- [x] BaseStrategy implemented
- [x] Data models defined (12 dataclasses)
- [ ] Unit tests passing - **Pending**

### M2: Simple Strategy Complete ✅ (Dec 30, 2025)
- [x] SimpleStrategy implemented
- [ ] Integration test with fixture passes - **Pending**
- [x] Can convert simple PDF to structured content

### M3: EPUB Builder Complete ✅ (Dec 30, 2025)
- [x] EPUBBuilder creates valid EPUB3
- [ ] Manual epubcheck validation passes - **Pending**
- [ ] Can open EPUB in reader - **Pending**

### M4: Full Pipeline Complete ✅ (Dec 30, 2025)
- [x] Converter orchestrates full workflow
- [x] CLI scripts functional (convert, analyze, validate)
- [ ] All tests passing (target: 80+ tests) - **Next Phase**

## Risks & Mitigation

### Risk 1: EPUB Validation Failures
**Likelihood:** High
**Impact:** Medium
**Mitigation:** Test with epubcheck early and often, use reference EPUB as template

### Risk 2: Reading Order Errors
**Likelihood:** Medium
**Impact:** High
**Mitigation:** Reuse proven YSorter, add reading order confidence to result, validate with OrderChecker

### Risk 3: Character Encoding Issues
**Likelihood:** Medium
**Impact:** Low
**Mitigation:** Always use UTF-8, escape HTML entities, test with non-ASCII text

## Open Questions
- [x] Should EPUBBuilder support EPUB2 or only EPUB3?
  - **Decision:** EPUB3 only - implemented in EPUBBuilder
- [x] How to handle images in PDF?
  - **Decision:** Implemented image extraction using PyMuPDF, saved to OEBPS/images/
- [x] Should conversion log be JSON or text?
  - **Decision:** Structured ConversionLog dataclass, CLI outputs human-readable format