My Journey Building a PDF to Markdown Converter from Scratch with AI: Pitfalls, Optimizations, and Full Practice

Background: I wanted to parse PDF documents into Markdown format for my RAG (Retrieval-Augmented Generation) system. However, many RAG tools fail to recognize headers and footers, causing them to mix into the middle of documents, which severely degrades quality. Docling can intelligently identify and remove headers and footers, so I decided to use it to build this tool. Plus, it’s a perfect opportunity to practice AI-assisted programming—I used Zoo (AI code assistant) to complete the entire development process.


Part 1: Requirements Analysis – Why Do We Need This Tool?

1.1 Document Processing Pain Points in RAG Systems

When building a RAG system, we need to convert documents of various formats (PDF, Word, PPT, etc.) into plain text or Markdown format, then chunk them, vectorize them, and store them in a vector database.

But PDF parsing has several classic problems:

  • Headers and footers: Many tools treat headers and footers as main content, causing retrieval to return大量 irrelevant information
  • Lost tables: Complex table structures get completely messed up after conversion
  • Image handling: Text in images (OCR) cannot be extracted
  • Messy formatting: Title hierarchy, lists, quotes, and other formats get lost

        1.2 Why Choose Docling?

        Docling is an open-source document processing library developed by the DS4SD team. Its core advantages:

        1. Intelligent layout understanding: Can identify headers, footers, main text, tables, images, and other regions
        2. Automatic header/footer removal: This is exactly what I need
        3. Multi-format support: PDF, DOCX, HTML, PPTX, etc.
        4. Structured output: Outputs DoclingDocument, preserving the document’s semantic structure

        Part 2: Technology Stack – Streamlit + Docling

        2.1 Streamlit: Rapid Web Application Development

        Streamlit is a Python framework that can rapidly build interactive web applications with minimal code. Its advantages:

        • Pure Python: No need to write HTML/CSS/JavaScript
        • Hot reload: Page auto-refreshes after code changes
        • Built-in components: File upload, buttons, dropdowns, etc., ready to use out of the box

        2.2 Docling: Powerful Document Conversion Engine

        Docling’s core API:

        from docling.document_converter import DocumentConverter
        
        converter = DocumentConverter()
        
        result = converter.convert("document.pdf")
        
        markdown_content = result.document.export_to_markdown()
        

        Part 3: Development Process – From Requirements to Deployment

        3.1 Step 1: Create Project Structure

        docling/

        ├── app.py                    # Main application

        ├── requirements.txt          # Dependencies list

        └── plans/

            └── pdf-to-markdown-website.md   # Implementation plan

        3.2 Step 2: Write Core Code

        The core logic of [`app.py`](../app.py):

        import os
        import tempfile
        import streamlit as st
        from docling.document_converter import DocumentConverter
        
        # ... Page configuration, file upload, conversion logic ...

        3.3 First Pitfall – torch.compile Error

        Problem Description

        After clicking the “Convert” button, the webpage kept showing “Converting document…” then errored:

        torch._inductor.exc.InductorError: InvalidCxxCompiler: Compiler: cl is not found.

        Problem Analysis

        By examining Docling’s source code, I discovered:

        1. Docling enables `torch.compile` (PyTorch’s model compilation optimization) by default
        2. torch.compile requires the MSVC C++ compiler cl.exe on Windows

        3. My system didn’t have Visual Studio Build Tools installed

        Solution

        Option A: Disable torch.compile (Recommended)

        # Must set before importing docling
        os.environ["DOCLING_INFERENCE_COMPILE_TORCH_MODELS"] = "false"
        
        # Second line of defense: Directly modify global settings
        from docling.datamodel.settings import settings as docling_settings
        
        docling_settings.inference.compile_torch_models = False

        Option B: Install Visual Studio Build Tools (For full performance)

        Download [Visual Studio Build Tools](https://visualstudio.microsoft.com/visual-cpp-build-tools/), check the “Desktop development with C++” workload.

        3.4 Second Pitfall: Memory Issues – Challenges in Multi-User Scenarios

        Problem Discovery

        I tested Docling’s memory usage:

        Operation Operation
        Server Memory Usage
        Import docling + Create DocumentConverter() +356MB (18MB → 374MB)
        After deleting converterMemory not freed (models stay resident)

        This means: If deployed to a server, each user would load a new copy of the model (~350MB) on every click. 10 users = 3.5GB, 20 users = 7GB… Memory would quickly deplete.

        Solution: Use `st.cache_resource` to Cache the Converter

        @st.cache_resource(show_spinner="Loading conversion engine...")
        
        def get_converter():
            return DocumentConverter()

        How it works:

        • @st.cache_resource is Streamlit’s global resource cache decorator
        • All users share a single DocumentConverter instance
        • Models load only once (~350MB), then all users reuse it
        • Memory no longer grows linearly with user count

        Measured Results

        Scenario Before OptimizationAfter Optimization
        1 user~700MB~350MB
        10 users~7GB~350MB
        20 users~14GB~350MB

        3.5 Third Pitfall: API Version Changes

        Problem Description

        'ConversionResult' object has no attribute 'render_as_markdown'

        Problem Analysis

        Tutorials online use the old Docling API: result.render_as_markdown()

        But in the current version of Docling, the markdown export method has moved to the result.document object, and the method name changed to `export_to_markdown()`.

        Solution

        # Old (incorrect):
        # markdown_content = result.render_as_markdown()
        # New (correct):
        markdown_content = result.document.export_to_markdown()

        Part 4: Complete Code Walkthrough

        4.1 [`app.py`](../app.py) Core Logic

        import os
        import tempfile
        import streamlit as st
        from docling.document_converter import DocumentConverter
        
        # Disable torch.compile (solves cl.exe issue on Windows)
        os.environ["DOCLING_INFERENCE_COMPILE_TORCH_MODELS"] = "false"
        
        from docling.datamodel.settings import settings as docling_settings
        
        docling_settings.inference.compile_torch_models = False
        
        # Cache converter (solves multi-user memory issue)
        
        @st.cache_resource(show_spinner="Loading conversion engine...")
        def get_converter():
            return DocumentConverter()
        
        # Page configuration
        st.set_page_config(page_title="PDF to Markdown", layout="wide")
        st.title("📄 PDF to Markdown Converter")
        
        # File upload (limited to 50MB)
        MAX_FILE_SIZE_MB = 50
        uploaded_file = st.file_uploader("Choose a PDF file", type=["pdf"])
        
        if uploaded_file is not None:
            # Check file size
            file_size_mb = uploaded_file.size / (1024 * 1024)
            if file_size_mb > MAX_FILE_SIZE_MB:
                st.error(f"File is {file_size_mb:.1f} MB, exceeding limit.")
            else:
                # Save to temporary file
                with tempfile.NamedTemporaryFile(delete=False, suffix=".pdf") as tmp_file:
                    tmp_file.write(uploaded_file.read())
                    tmp_path = tmp_file.name
        
                # Conversion
                if st.button("🚀 Convert to Markdown"):
                    try:
                        converter = get_converter()  # Use cached converter
                        result = converter.convert(tmp_path)
                        markdown_content = result.document.export_to_markdown()
                        # Display preview
                        st.success("Conversion successful!")
                        st.markdown(markdown_content)
                        # Provide download
                        st.download_button(
                            label="📥 Download Markdown File",
                            data=markdown_content,
                            file_name=uploaded_file.name.replace(".pdf", ".md"),
                            mime="text/markdown"
                        )
                    except Exception as e:
                        st.error(f"Error: {e}")
                    finally:
                        os.remove(tmp_path)  # Clean up temporary file

        4.2 requirements.txt (../requirements.txt)

        streamlit>=1.30.0
        docling>=2.118.0

        Part 5: Deployment Recommendations

        5.1 Local Development

        pip install -r requirements.txt
        streamlit run app.py

        5.2 Production Deployment

        OptionUse CaseMemory Estimate
        Single instance< 50 users~400MB
        Multiple instances + Nginx load balancing50-200 users ~800MB – 1.6GB
        Docker containerizationAny scalePer container count

        Important Notes:

        1. Don’t run bare `streamlit run` in production: Use Docker containers + reverse proxy instead
        2. Set file size limits: Prevent huge PDFs from dragging down the server (code already sets this to 50MB)
        3. Monitor memory usage: Each converter instance uses ~350MB; estimate instance count based on user count

        Part 6: Lessons Learned

        6.1 Pitfall Checklist

        Problem Cause Solution
        cl is not foundtorch.compile needs MSVC compiler Disable torch.compile
        Multi-user memory explosionEach user loads independent modelUse st.cache_resource to share converter
        render_as_markdown doesn’t existAPI version changeUse result.document.export_to_markdown() instead

        6.2 Best Practices for AI-Assisted Programming

        1. Describe problems clearly: Provide complete error information and context
        2. Verify solutions: Don’t blindly trust AI answers; test them yourself
        3. Understand the principles: Know why you’re changing something, not just how
        4. Iterative optimization: Start with MVP, then gradually improve

        6.3 Future Improvement Directions

        • Support batch conversion (multiple PDFs processed simultaneously)
        • Support more formats (DOCX, HTML, PPTX)
        • Add OCR option (for scanned PDFs)
        • Custom output options (whether to keep images, tables, etc.)
        • API mode (for other programs to call)

        Part 7: Conclusion

        This tool may seem simple, but it involves several important technical points:

        1. Document processing: How to intelligently parse PDFs with Docling
        2. Web development: How to rapidly build interactive interfaces with Streamlit
        3. Performance optimization: How to cache models and limit memory usage
        4. API version management: How to handle library API changes

        Most importantly, this practice validated the value of AI-assisted programming:

        • Efficiency boost: From requirements to deployment in less than 1 hour
        • Knowledge acquisition: Through AI’s explanations, I deeply understood Docling’s internal mechanisms
        • Problem solving: Quickly identified and resolved multiple tricky issues

        I hope this blog helps you avoid similar pitfalls. Feel free to try using AI assistants to accelerate your development process!


        Reference Links:

        Leave a Reply

        Your email address will not be published. Required fields are marked *

        This site uses Akismet to reduce spam. Learn how your comment data is processed.