Preparing your documents for a RAG pipeline: clean Markdown and JSON

A RAG pipeline (retrieval-augmented generation) is only as good as the documents you feed it. If you ingest raw, heterogeneous files full of layout noise, you get incoherent chunks and shaky answers. Preparing your documents for a RAG pipeline therefore starts with one simple but decisive step: normalize any format into clean Markdown and JSON, before you even think about chunking or embeddings.

Why a raw document breaks a RAG pipeline

A PDF, a Word file and a web page are nothing alike under the hood. One carries a multi-column layout, another XML markup, the third HTML tags and scripts. When you chunk them without cleaning them first, several problems show up:

• Chunks cut in the wrong place, because the file's structure doesn't match the content's structure.
• Noise baked into the embeddings: headers, footers, artifacts, all diluting the signal.
• Formats handled differently, so search results that aren't comparable from one document to the next.

The final model can't fix this: it works with what it's given. Quality is decided upstream, at the entrance of the pipeline.

Normalize any format into a single output

The core idea is normalization: whatever the source, you aim for a single, clean, stable output. That's what a converter like Sygal does, turning 29 file types into readable Markdown and structured JSON, relying on MarkItDown for local conversion.

The point is not to save tokens everywhere, which would be false for an already-structured Word file. The point is to get the same output format for your whole corpus: a PDF, a Word note and a spreadsheet export all come out in the same shape, ready to split and index. You write your chunking logic once, not once per format.

Markdown or JSON: which one for your pipeline?

The two outputs serve different needs, and many pipelines use both.

Markdown preserves the reading structure: headings, lists, tables, hierarchy. It's ideal for semantic chunking, because headings mark natural boundaries, and for giving the model readable context.

Structured JSON exposes fields your code can parse: identified blocks, tables turned into data, metadata. It's what you need when your pipeline requires stable fields to parse rather than free text, for example to tie each chunk back to its source or to treat tables as data.

A simple rule: Markdown for the content to chunk and embed, JSON for the structure and metadata your code has to read.

A concrete example: a price table in a PDF. In Markdown, it comes out as a readable table, perfect for giving a model context. In JSON, the same rows become data, where each column is a field your code can read and compare. The same content, two uses: one for generation, one for processing.

Keeping a consistent output across a mixed corpus

A real corpus mixes formats and sources. Output consistency is what keeps the pipeline maintainable. Concretely, that means:

• Batch conversion that processes a whole folder and produces .md and .json for each file.
• The same output schema, whatever the input format.
• Simple retries when a file fails, without breaking the whole batch.

From the command line, this step automates inside a script or an agent orchestrator. Sygal CLI exposes the same conversion engine as the app, with strict JSON output and stable error codes, designed to drop into a pipeline without surprises.

Chunking cleanly: a few pointers

Once your documents are normalized into Markdown, chunking gets much easier, because the structure is explicit. A few pointers that avoid the most common mistakes:

• Split on headings. The H2s and H3s in Markdown mark natural semantic boundaries. A chunk that starts and ends on a section stays coherent, unlike blind splitting every N characters, which often cuts a sentence in half.
• Keep tables whole. A table cut in two loses its meaning. Structured JSON helps here: it isolates the table as a block you can handle separately.
• Tie each chunk to its source. The file name, the page, the section: this metadata, carried by the JSON, lets you cite the source in the answer and trace back to the original document.
• Stay consistent across the corpus. The same chunking strategy should apply everywhere. That's exactly what a normalized output enables: you tune chunking once, not per format.

None of these rules is tied to a specific tool. They simply become workable as soon as the input is clean and uniform, which always leads back to the same point: a RAG pipeline's quality is decided at conversion, before indexing.

Does everything have to run in the cloud?

No, and it's often a blocker for enterprise data. Text document conversion runs entirely locally: your files are not uploaded. Only two actions touch the network, and only if you trigger them: OCR on an image, sent to the provider you choose with your key, and importing a URL. For a corpus of sensitive documents, that boundary changes everything, a topic we detail in the guide on converting documents without the cloud.

In short

A solid RAG pipeline starts with clean inputs. Normalize first, index second: convert each source into consistent Markdown and JSON, keep the same output across the whole corpus, and keep the network out of the equation as much as possible. You can test conversion on your own documents with the free trial, or wire it straight into your scripts through the CLI.