How to prepare documents for a RAG pipeline

A retrieval system can only ground an answer in information that survived extraction. Converting a document to Markdown makes the structure inspectable, but it does not prove the structure is correct.

Before indexing, compare the Markdown with the source. Pay particular attention to tables, multi-column PDFs, footnotes, repeated headers, slide reading order, and merged spreadsheet cells. Those are common places where text can be present but attached to the wrong context.

Text-based PDFs and ordinary Office files are good candidates for local browser conversion. Scanned PDFs, dense tables, and visually complex layouts benefit from OCR or a layout-aware server-side path.

Repeated navigation, page numbers, confidentiality footers, and decorative text can dominate similarity search even though they add little meaning. Remove those patterns consistently after confirming they are not part of the document’s argument.

Keep headings, units, dates, table headers, captions, source links, and section identifiers. These details help a retrieved passage remain understandable after it is separated from the rest of the document.

Use headings and complete table sections as natural boundaries. Avoid splitting a heading from its first paragraph, a question from its answer, or a table header from its rows.

There is no universal chunk size. Test retrieval with the questions your users actually ask, then adjust chunk size and overlap based on missed context—not on a generic token target alone.

Store the source filename, document title, section heading, page or slide number when available, conversion date, and a stable source URL or identifier. Provenance makes citations possible and gives reviewers a path back to the original evidence.

If a source changes regularly, include a version or modified date so old chunks can be replaced instead of silently mixed with newer information.

Test questions that require a table lookup, a definition, a cross-section comparison, and an exact date or number. Also test questions the documents do not answer. A useful RAG pipeline should retrieve the right evidence when it exists and decline to invent evidence when it does not.