---
name: pdf
description: Use this skill whenever the user wants to do anything with PDF files. This includes reading or extracting text/tables from PDFs, combining or merging multiple PDFs into one, splitting PDFs apart, rotating pages, adding watermarks, creating new PDFs, filling PDF forms, encrypting/decrypting PDFs, extracting images, and OCR on scanned PDFs to make them searchable. If the user mentions a .pdf file or asks to produce one, use this skill.
license: Proprietary. LICENSE.txt has complete terms
---

# PDF Processing Guide

## Overview

This guide covers essential PDF processing operations using Python libraries and command-line tools. For advanced features, JavaScript libraries, and detailed examples, see REFERENCE.md. If you need to fill out a PDF form, read FORMS.md and follow its instructions.

## Quick Start

```python
from pypdf import PdfReader, PdfWriter

# Read a PDF
reader = PdfReader("document.pdf")
print(f"Pages: {len(reader.pages)}")

# Extract text
text = ""
for page in reader.pages:
    text += page.extract_text()
```

## Python Libraries

### pypdf - Basic Operations

#### Merge PDFs
```python
from pypdf import PdfWriter, PdfReader

writer = PdfWriter()
for pdf_file in ["doc1.pdf", "doc2.pdf", "doc3.pdf"]:
    reader = PdfReader(pdf_file)
    for page in reader.pages:
        writer.add_page(page)

with open("merged.pdf", "wb") as output:
    writer.write(output)
```

#### Split PDF
```python
reader = PdfReader("input.pdf")
for i, page in enumerate(reader.pages):
    writer = PdfWriter()
    writer.add_page(page)
    with open(f"page_{i+1}.pdf", "wb") as output:
        writer.write(output)
```

#### Extract Metadata
```python
reader = PdfReader("document.pdf")
meta = reader.metadata
print(f"Title: {meta.title}")
print(f"Author: {meta.author}")
print(f"Subject: {meta.subject}")
print(f"Creator: {meta.creator}")
```

#### Rotate Pages
```python
reader = PdfReader("input.pdf")
writer = PdfWriter()

page = reader.pages[0]
page.rotate(90)  # Rotate 90 degrees clockwise
writer.add_page(page)

with open("rotated.pdf", "wb") as output:
    writer.write(output)
```

### pdfplumber - Text and Table Extraction

#### Extract Text with Layout
```python
import pdfplumber

with pdfplumber.open("document.pdf") as pdf:
    for page in pdf.pages:
        text = page.extract_text()
        print(text)
```

#### Extract Tables
```python
with pdfplumber.open("document.pdf") as pdf:
    for i, page in enumerate(pdf.pages):
        tables = page.extract_tables()
        for j, table in enumerate(tables):
            print(f"Table {j+1} on page {i+1}:")
            for row in table:
                print(row)
```

#### Advanced Table Extraction
```python
import pandas as pd

with pdfplumber.open("document.pdf") as pdf:
    all_tables = []
    for page in pdf.pages:
        tables = page.extract_tables()
        for table in tables:
            if table:  # Check if table is not empty
                df = pd.DataFrame(table[1:], columns=table[0])
                all_tables.append(df)

# Combine all tables
if all_tables:
    combined_df = pd.concat(all_tables, ignore_index=True)
    combined_df.to_excel("extracted_tables.xlsx", index=False)
```

### reportlab - Create PDFs

#### Basic PDF Creation
```python
from reportlab.lib.pagesizes import letter
from reportlab.pdfgen import canvas

c = canvas.Canvas("hello.pdf", pagesize=letter)
width, height = letter

# Add text
c.drawString(100, height - 100, "Hello World!")
c.drawString(100, height - 120, "This is a PDF created with reportlab")

# Add a line
c.line(100, height - 140, 400, height - 140)

# Save
c.save()
```

#### Create PDF with Multiple Pages
```python
from reportlab.lib.pagesizes import letter
from reportlab.platypus import SimpleDocTemplate, Paragraph, Spacer, PageBreak
from reportlab.lib.styles import getSampleStyleSheet

doc = SimpleDocTemplate("report.pdf", pagesize=letter)
styles = getSampleStyleSheet()
story = []

# Add content
title = Paragraph("Report Title", styles['Title'])
story.append(title)
story.append(Spacer(1, 12))

body = Paragraph("This is the body of the report. " * 20, styles['Normal'])
story.append(body)
story.append(PageBreak())

# Page 2
story.append(Paragraph("Page 2", styles['Heading1']))
story.append(Paragraph("Content for page 2", styles['Normal']))

# Build PDF
doc.build(story)
```

#### Subscripts and Superscripts

**IMPORTANT**: Never use Unicode subscript/superscript characters (₀₁₂₃₄₅₆₇₈₉, ⁰¹²³⁴⁵⁶⁷⁸⁹) in ReportLab PDFs. The built-in fonts do not include these glyphs, causing them to render as solid black boxes.

Instead, use ReportLab's XML markup tags in Paragraph objects:
```python
from reportlab.platypus import Paragraph
from reportlab.lib.styles import getSampleStyleSheet

styles = getSampleStyleSheet()

# Subscripts: use <sub> tag
chemical = Paragraph("H<sub>2</sub>O", styles['Normal'])

# Superscripts: use <super> tag
squared = Paragraph("x<super>2</super> + y<super>2</super>", styles['Normal'])
```

For canvas-drawn text (not Paragraph objects), manually adjust font the size and position rather than using Unicode subscripts/superscripts.

## Command-Line Tools

### pdftotext (poppler-utils)
```bash
# Extract text
pdftotext input.pdf output.txt

# Extract text preserving layout
pdftotext -layout input.pdf output.txt

# Extract specific pages
pdftotext -f 1 -l 5 input.pdf output.txt  # Pages 1-5
```

### qpdf
```bash
# Merge PDFs
qpdf --empty --pages file1.pdf file2.pdf -- merged.pdf

# Split pages
qpdf input.pdf --pages . 1-5 -- pages1-5.pdf
qpdf input.pdf --pages . 6-10 -- pages6-10.pdf

# Rotate pages
qpdf input.pdf output.pdf --rotate=+90:1  # Rotate page 1 by 90 degrees

# Remove password
qpdf --password=mypassword --decrypt encrypted.pdf decrypted.pdf
```

### pdftk (if available)
```bash
# Merge
pdftk file1.pdf file2.pdf cat output merged.pdf

# Split
pdftk input.pdf burst

# Rotate
pdftk input.pdf rotate 1east output rotated.pdf
```

## Common Tasks

### Extract Text from Scanned PDFs
```python
# Requires: pip install pytesseract pdf2image
import pytesseract
from pdf2image import convert_from_path

# Convert PDF to images
images = convert_from_path('scanned.pdf')

# OCR each page
text = ""
for i, image in enumerate(images):
    text += f"Page {i+1}:\n"
    text += pytesseract.image_to_string(image)
    text += "\n\n"

print(text)
```

### Add Watermark
```python
from pypdf import PdfReader, PdfWriter

# Create watermark (or load existing)
watermark = PdfReader("watermark.pdf").pages[0]

# Apply to all pages
reader = PdfReader("document.pdf")
writer = PdfWriter()

for page in reader.pages:
    page.merge_page(watermark)
    writer.add_page(page)

with open("watermarked.pdf", "wb") as output:
    writer.write(output)
```

### Extract Images
```bash
# Using pdfimages (poppler-utils)
pdfimages -j input.pdf output_prefix

# This extracts all images as output_prefix-000.jpg, output_prefix-001.jpg, etc.
```

### Password Protection
```python
from pypdf import PdfReader, PdfWriter

reader = PdfReader("input.pdf")
writer = PdfWriter()

for page in reader.pages:
    writer.add_page(page)

# Add password
writer.encrypt("userpassword", "ownerpassword")

with open("encrypted.pdf", "wb") as output:
    writer.write(output)
```

## Quick Reference

| Task | Best Tool | Command/Code |
|------|-----------|--------------|
| Merge PDFs | pypdf | `writer.add_page(page)` |
| Split PDFs | pypdf | One page per file |
| Extract text | pdfplumber | `page.extract_text()` |
| Extract tables | pdfplumber | `page.extract_tables()` |
| Create PDFs | reportlab | Canvas or Platypus |
| Command line merge | qpdf | `qpdf --empty --pages ...` |
| OCR scanned PDFs | pytesseract | Convert to image first |
| Fill PDF forms | pdf-lib or pypdf (see FORMS.md) | See FORMS.md |

## Next Steps

- For advanced pypdfium2 usage, see REFERENCE.md
- For JavaScript libraries (pdf-lib), see REFERENCE.md
- If you need to fill out a PDF form, follow the instructions in FORMS.md
- For troubleshooting guides, see REFERENCE.md

---

## Document Understanding / Extraction (OpenDataLoader PDF)

### Overview

OpenDataLoader PDF(ODL)는 Java 코어 + Python SDK 기반의 PDF 문서 이해/추출 도구입니다. 텍스트, 테이블, OCR, 구조 분석에 최적화되어 있으며, 선택적으로 docling AI 백엔드(hybrid 모드)와 연동하여 정확도를 대폭 높일 수 있습니다.

### When to Use ODL vs Existing Tools

작업 유형에 따라 아래 라우팅 가이드를 참고하세요:

| 작업 유형 | 권장 도구 |
|-----------|-----------|
| 폼 작성 / 필드 추출 | 기존 스크립트 (pypdf, pdfplumber) — scripts/ 디렉토리의 기존 8개 스크립트 참조 |
| 텍스트 추출 / 마크다운 변환 | ODL `extract_text_odl.py` |
| 테이블 추출 | ODL `extract_table_odl.py` (hybrid 모드 권장) |
| OCR (스캔 PDF) | ODL `ocr_pdf_odl.py` (hybrid 필수) |
| 범용 변환 (모든 옵션) | ODL `convert_pdf_odl.py` |
| PDF 합치기/나누기/회전/워터마크 | 기존 도구 (pypdf, qpdf) |

### Quick Start (ODL)

**1. 텍스트 추출 (마크다운):**
```bash
python3 scripts/extract_text_odl.py document.pdf
python3 scripts/extract_text_odl.py document.pdf --format text  # 순수 텍스트
python3 scripts/extract_text_odl.py document.pdf --pages "1-5"   # 특정 페이지
```

**2. 테이블 추출:**
```bash
python3 scripts/extract_table_odl.py document.pdf
python3 scripts/extract_table_odl.py document.pdf --hybrid docling-fast  # AI 향상
```

**3. 범용 변환:**
```bash
python3 scripts/convert_pdf_odl.py document.pdf --format json
python3 scripts/convert_pdf_odl.py document.pdf --format markdown --reading-order xycut
python3 scripts/convert_pdf_odl.py document.pdf --format json --sanitize  # 개인정보 마스킹
```

### ODL Options Reference

| 옵션 | 값 | 설명 |
|------|----|------|
| `--format` | json, text, markdown, html, pdf | 출력 형식 (기본: markdown) |
| `--reading-order` | off \| xycut | 읽기 순서 (기본: xycut, 다단 레이아웃 순서 보존) |
| `--hybrid` | off \| docling-fast | AI 백엔드 (테이블/OCR 정확도 대폭 향상) |
| `--hybrid-url` | URL | 하이브리드 서버 URL (기본: http://localhost:5002) |
| `--sanitize` | 플래그 | 개인정보 자동 마스킹 (이메일, 전화, 카드번호) |
| `--pages` | "1,3,5-7" | 페이지 범위 지정 |
| `--ocr-lang` | "ko,en" | OCR 언어 (기본: "ko,en") |
| `--force-ocr` | 플래그 | 전체 페이지 강제 OCR |
| `--keep-line-breaks` | 플래그 | 줄바꿈 보존 |
| `--include-header-footer` | 플래그 | 헤더/푸터 포함 |
| `--use-struct-tree` | 플래그 | PDF 구조 태그 활용 (Tagged PDF용) |

### Hybrid Mode

- **로컬 모드**: Java만 사용 (빠름, 0.05s/page)
- **Hybrid 모드**: Java + docling AI (정확, 0.43s/page)
  - 테이블 정확도: 로컬 49% → hybrid 93%
  - OCR: hybrid 모드에서만 작동
- hybrid 서버 시작: `bash /home/jay/workspace/scripts/start-hybrid-server.sh`
- hybrid 서버 종료: `bash /home/jay/workspace/scripts/stop-hybrid-server.sh`

### OCR (Scanned PDFs)

OCR은 hybrid 모드가 필수입니다. hybrid 서버를 먼저 시작한 후 실행하세요.

```bash
python3 scripts/ocr_pdf_odl.py scanned.pdf                         # 기본 한/영
python3 scripts/ocr_pdf_odl.py scanned.pdf --ocr-lang "ko,en,ja"   # 다국어
python3 scripts/ocr_pdf_odl.py scanned.pdf --force-ocr              # 전체 페이지 강제 OCR
```

### Requirements

- **Java 11+**: `/home/jay/.local/jvm/jdk-11.0.2` (JAVA_HOME)
- **Python**: opendataloader-pdf >= 2.0.0
- **Hybrid (선택)**: docling, FastAPI, uvicorn