"Which page is that on?"
That one question from the compliance team changed our entire approach to RAG citations. We had built a retrieval-augmented generation pipeline on Azure AI Search for a financial services client — 2.4 million documents, mostly contracts, regulatory filings, and technical specs. The search worked well. The answers were relevant. But when a compliance reviewer needed to verify a citation, they needed the exact page number, not just the document title. And we were not giving them that.
The chunking problem
RAG systems split documents into overlapping chunks for semantic search. Azure AI Search indexes those chunks, runs vector similarity, and returns the most relevant text. The problem is that chunk boundaries have nothing to do with page boundaries. A single chunk might span pages 14 and 15. A page break might land in the middle of a sentence. When we audited our citations against manually verified page numbers, accuracy was around 58%. That is not good enough when the output goes into a legal filing.
Why re-indexing was off the table
The obvious fix was to embed page metadata directly into the index — add a page_number field to every chunk at indexing time. But re-indexing 2.4 million documents took 8 to 12 hours per corpus update, and the upstream document pipelines would all need changes to emit that metadata. The corpus was also live — new documents arrived daily. Re-indexing was ruled out on day one.
We needed a solution that worked with the existing index, not against it.
The byte-offset insight
It turned out the answer was already in the data. Every chunk in our Azure AI Search index carried a chunk_offset field — the byte offset of that chunk within its source document. PDF page boundaries are deterministic: given the raw file, you can compute the exact byte offset where each page starts. If we stored those boundaries once at ingest time and looked them up at query time, we could map any chunk to its page without touching the index.
The extraction function was straightforward:
from dataclasses import dataclass
@dataclass
class Citation:
document_id: str
title: str
page_number: int
excerpt: str
score: float
def extract_citations(
search_results: list[dict],
page_boundaries: dict[str, list[int]],
) -> list[Citation]:
"""Map Azure AI Search chunk results to page-number citations."""
citations = []
for result in search_results:
doc_id = result["document_id"]
byte_offset = result.get("chunk_offset", 0)
boundaries = page_boundaries.get(doc_id, [])
page_number = _offset_to_page(byte_offset, boundaries)
citations.append(
Citation(
document_id=doc_id,
title=result.get("title", ""),
page_number=page_number,
excerpt=result["content"][:300],
score=result["@search.score"],
)
)
return sorted(citations, key=lambda c: c.score, reverse=True)Page boundaries were computed once per document at ingest using PyMuPDF and stored in a DynamoDB table keyed by document ID — just a list of byte offsets, one per page. At query time, a single batch_get_item call loaded the boundaries for every document in the search results. The lookup added about 4 milliseconds to each query, which was acceptable for compliance workflows that previously took hours.
The page resolution itself was a binary search:
def _offset_to_page(offset: int, boundaries: list[int]) -> int:
"""Binary search to find the 1-indexed page for a byte offset."""
lo, hi = 0, len(boundaries) - 1
while lo < hi:
mid = (lo + hi) // 2
if boundaries[mid] <= offset:
lo = mid + 1
else:
hi = mid
return lo + 1Results and caveats
We benchmarked against a manually audited set of 500 randomly sampled citations across 15 document types. Exact-page accuracy went from 58% to 94%. Compliance review time dropped from about four hours per document to 45 minutes — reviewers could jump directly to the cited page instead of scanning entire documents.
Most of the remaining misses came from scanned PDFs where OCR introduced byte-offset drift between the text layer and the visual layout. For those cases, we added a GPT-4o fallback that re-ranked candidate pages using layout cues from the chunk's surrounding context. It fired on about 12% of queries and added 250 milliseconds — a worthwhile trade for the extra accuracy.
The deeper lesson was about stability. Byte offsets are fixed properties of a document file. They do not change when you tweak your chunking strategy, update your prompts, or swap your embedding model. Building the citation layer on byte offsets instead of chunk text meant it was immune to every other change in the pipeline. That turned out to matter more than we expected.