My Journey Building a PDF to Markdown Converter from Scratch with AI: Pitfalls, Optimizations, and Full Practice
Background: I wanted to parse PDF documents into Markdown format for my RAG (Retrieval-Augmented Generation) system. However, many RAG tools fail to recognize headers and footers, causing them to mix into the middle of documents, which severely degrades quality. Docling can intelligently identify and remove headers and footers, so I decided to use it to build this tool. Plus, it’s a perfect opportunity to practice AI-assisted programming—I used Zoo (AI code assistant) to complete the entire development process.
Part 1: Requirements Analysis – Why Do We Need This Tool?
1.1 Document Processing Pain Points in RAG Systems
When building a RAG system, we need to convert documents of various formats (PDF, Word, PPT, etc.) into plain text or Markdown format, then chunk them, vectorize them, and store them in a vector database.
But PDF parsing has several classic problems:
- Headers and footers: Many tools treat headers and footers as main content, causing retrieval to return大量 irrelevant information
- Lost tables: Complex table structures get completely messed up after conversion
- Image handling: Text in images (OCR) cannot be extracted
- Messy formatting: Title hierarchy, lists, quotes, and other formats get lost
1.2 Why Choose Docling?
Docling is an open-source document processing library developed by the DS4SD team. Its core advantages:
- Intelligent layout understanding: Can identify headers, footers, main text, tables, images, and other regions
- Automatic header/footer removal: This is exactly what I need
- Multi-format support: PDF, DOCX, HTML, PPTX, etc.
- Structured output: Outputs DoclingDocument, preserving the document’s semantic structure
Part 2: Technology Stack – Streamlit + Docling
2.1 Streamlit: Rapid Web Application Development
Streamlit is a Python framework that can rapidly build interactive web applications with minimal code. Its advantages:
- Pure Python: No need to write HTML/CSS/JavaScript
- Hot reload: Page auto-refreshes after code changes
- Built-in components: File upload, buttons, dropdowns, etc., ready to use out of the box
2.2 Docling: Powerful Document Conversion Engine
Docling’s core API:
from docling.document_converter import DocumentConverter
converter = DocumentConverter()
result = converter.convert("document.pdf")
markdown_content = result.document.export_to_markdown()
Part 3: Development Process – From Requirements to Deployment
3.1 Step 1: Create Project Structure
docling/
├── app.py # Main application
├── requirements.txt # Dependencies list
└── plans/
└── pdf-to-markdown-website.md # Implementation plan
3.2 Step 2: Write Core Code
The core logic of [`app.py`](../app.py):
import os
import tempfile
import streamlit as st
from docling.document_converter import DocumentConverter
# ... Page configuration, file upload, conversion logic ...
3.3 First Pitfall – torch.compile Error
Problem Description
After clicking the “Convert” button, the webpage kept showing “Converting document…” then errored:
torch._inductor.exc.InductorError: InvalidCxxCompiler: Compiler: cl is not found.
Problem Analysis
By examining Docling’s source code, I discovered:
- Docling enables `torch.compile` (PyTorch’s model compilation optimization) by default
torch.compilerequires the MSVC C++ compilercl.exeon Windows
3. My system didn’t have Visual Studio Build Tools installed
Solution
Option A: Disable torch.compile (Recommended)
# Must set before importing docling
os.environ["DOCLING_INFERENCE_COMPILE_TORCH_MODELS"] = "false"
# Second line of defense: Directly modify global settings
from docling.datamodel.settings import settings as docling_settings
docling_settings.inference.compile_torch_models = False
Option B: Install Visual Studio Build Tools (For full performance)
Download [Visual Studio Build Tools](https://visualstudio.microsoft.com/visual-cpp-build-tools/), check the “Desktop development with C++” workload.
3.4 Second Pitfall: Memory Issues – Challenges in Multi-User Scenarios
Problem Discovery
I tested Docling’s memory usage:
| Operation | Operation Server Memory Usage |
|---|---|
Import docling + Create DocumentConverter() | +356MB (18MB → 374MB) |
| After deleting converter | Memory not freed (models stay resident) |
This means: If deployed to a server, each user would load a new copy of the model (~350MB) on every click. 10 users = 3.5GB, 20 users = 7GB… Memory would quickly deplete.
Solution: Use `st.cache_resource` to Cache the Converter
@st.cache_resource(show_spinner="Loading conversion engine...")
def get_converter():
return DocumentConverter()
How it works:
@st.cache_resourceis Streamlit’s global resource cache decorator- All users share a single
DocumentConverterinstance - Models load only once (~350MB), then all users reuse it
- Memory no longer grows linearly with user count
Measured Results
| Scenario | Before Optimization | After Optimization |
|---|---|---|
| 1 user | ~700MB | ~350MB |
| 10 users | ~7GB | ~350MB |
| 20 users | ~14GB | ~350MB |
3.5 Third Pitfall: API Version Changes
Problem Description
'ConversionResult' object has no attribute 'render_as_markdown'
Problem Analysis
Tutorials online use the old Docling API: result.render_as_markdown()
But in the current version of Docling, the markdown export method has moved to the result.document object, and the method name changed to `export_to_markdown()`.
Solution
# Old (incorrect):
# markdown_content = result.render_as_markdown()
# New (correct):
markdown_content = result.document.export_to_markdown()
Part 4: Complete Code Walkthrough
4.1 [`app.py`](../app.py) Core Logic
import os
import tempfile
import streamlit as st
from docling.document_converter import DocumentConverter
# Disable torch.compile (solves cl.exe issue on Windows)
os.environ["DOCLING_INFERENCE_COMPILE_TORCH_MODELS"] = "false"
from docling.datamodel.settings import settings as docling_settings
docling_settings.inference.compile_torch_models = False
# Cache converter (solves multi-user memory issue)
@st.cache_resource(show_spinner="Loading conversion engine...")
def get_converter():
return DocumentConverter()
# Page configuration
st.set_page_config(page_title="PDF to Markdown", layout="wide")
st.title("📄 PDF to Markdown Converter")
# File upload (limited to 50MB)
MAX_FILE_SIZE_MB = 50
uploaded_file = st.file_uploader("Choose a PDF file", type=["pdf"])
if uploaded_file is not None:
# Check file size
file_size_mb = uploaded_file.size / (1024 * 1024)
if file_size_mb > MAX_FILE_SIZE_MB:
st.error(f"File is {file_size_mb:.1f} MB, exceeding limit.")
else:
# Save to temporary file
with tempfile.NamedTemporaryFile(delete=False, suffix=".pdf") as tmp_file:
tmp_file.write(uploaded_file.read())
tmp_path = tmp_file.name
# Conversion
if st.button("🚀 Convert to Markdown"):
try:
converter = get_converter() # Use cached converter
result = converter.convert(tmp_path)
markdown_content = result.document.export_to_markdown()
# Display preview
st.success("Conversion successful!")
st.markdown(markdown_content)
# Provide download
st.download_button(
label="📥 Download Markdown File",
data=markdown_content,
file_name=uploaded_file.name.replace(".pdf", ".md"),
mime="text/markdown"
)
except Exception as e:
st.error(f"Error: {e}")
finally:
os.remove(tmp_path) # Clean up temporary file
4.2 requirements.txt (../requirements.txt)
streamlit>=1.30.0
docling>=2.118.0
Part 5: Deployment Recommendations
5.1 Local Development
pip install -r requirements.txt
streamlit run app.py
5.2 Production Deployment
| Option | Use Case | Memory Estimate |
|---|---|---|
| Single instance | < 50 users | ~400MB |
| Multiple instances + Nginx load balancing | 50-200 users | ~800MB – 1.6GB |
| Docker containerization | Any scale | Per container count |
Important Notes:
- Don’t run bare `streamlit run` in production: Use Docker containers + reverse proxy instead
- Set file size limits: Prevent huge PDFs from dragging down the server (code already sets this to 50MB)
- Monitor memory usage: Each converter instance uses ~350MB; estimate instance count based on user count
Part 6: Lessons Learned
6.1 Pitfall Checklist
| Problem | Cause | Solution |
|---|---|---|
cl is not found | torch.compile needs MSVC compiler | Disable torch.compile |
| Multi-user memory explosion | Each user loads independent model | Use st.cache_resource to share converter |
render_as_markdown doesn’t exist | API version change | Use result.document.export_to_markdown() instead |
6.2 Best Practices for AI-Assisted Programming
- Describe problems clearly: Provide complete error information and context
- Verify solutions: Don’t blindly trust AI answers; test them yourself
- Understand the principles: Know why you’re changing something, not just how
- Iterative optimization: Start with MVP, then gradually improve
6.3 Future Improvement Directions
- Support batch conversion (multiple PDFs processed simultaneously)
- Support more formats (DOCX, HTML, PPTX)
- Add OCR option (for scanned PDFs)
- Custom output options (whether to keep images, tables, etc.)
- API mode (for other programs to call)
Part 7: Conclusion
This tool may seem simple, but it involves several important technical points:
- Document processing: How to intelligently parse PDFs with Docling
- Web development: How to rapidly build interactive interfaces with Streamlit
- Performance optimization: How to cache models and limit memory usage
- API version management: How to handle library API changes
Most importantly, this practice validated the value of AI-assisted programming:
- Efficiency boost: From requirements to deployment in less than 1 hour
- Knowledge acquisition: Through AI’s explanations, I deeply understood Docling’s internal mechanisms
- Problem solving: Quickly identified and resolved multiple tricky issues
I hope this blog helps you avoid similar pitfalls. Feel free to try using AI assistants to accelerate your development process!
Reference Links:
