Sanskrit in, Telugu out: a RAG over 17 Puranas

Part 1 built a question-answering system over the Bhagavad Gita in Telugu. That was the easy case, in one specific way: a public-domain Telugu tatparyam (meaning) existed for every verse, so the thing we searched and the thing the user typed were in the same language.

The Puranas are the hard case. They are the texts a Telugu devotee actually asks about — why Dhruva went to the forest, what the Garuda Purana says happens after death, whether a curse given in anger can be undone — and they exist, in open-licensed form, only in Sanskrit. Modern Telugu prose Puranas are all under copyright. The medieval Telugu poets' versions are public domain but are adaptations in classical verse, not translations, and they cover a fraction of the eighteen.

So the situation was:

• The corpus is Sanskrit: 17 Purana texts from GRETIL, 2,809 chapters, 140,873 verses, in Roman transliteration.
• The questions are Telugu, often typed in Latin letters ("dhruvudu enduku tapassu chesadu?").
• The answer must be Telugu, and every claim must point to a verse.
• There is no Telugu meaning to search, to show, or to quote.

Here is the finished pipeline. The rest of the article is about the one idea that made it work, and the numbers that say how well it works.

01Three ways to bridge two languages, and why two of them fail

When the corpus and the query are in different languages there are only three places to close the gap: translate the corpus, translate the query, or build a bridge in the middle.

Translate the corpus

Generate a Telugu tatparyam for every verse, then run the Gita pipeline unchanged. It is the obvious plan and it is wrong for this corpus. There are 141,000 verses in the texts we have, and roughly 400,000 across the eighteen Puranas. Verse-by-verse machine translation of Puranic Sanskrit is decent but not trustworthy, and once the Telugu exists people will quote it as scripture. Every one of those verses would need review before it could be shown. That is a multi-year project disguised as a batch job.

Embed the Sanskrit directly

Put the Sanskrit verses into a multilingual embedding model and search them with the Telugu question. Two things go wrong. Multilingual encoders learned what Sanskrit they know from Devanagari text, and our corpus is either Roman transliteration or Telugu script, both out of distribution. And even in the best case, a single sloka rarely contains the idea a devotee asks about. "Why did Dhruva go to the forest" is spread over forty verses of insults, tears and a mother's advice. Verse-level retrieval across languages is the hardest configuration you can choose.

Build a bridge at the level where questions live

A devotee never asks about verse 14 of chapter 8. They ask about a story, a rule, or a lesson. Stories, rules and lessons live at the chapter level, and there are only 2,809 chapters. If each chapter had a Telugu description of what it contains, written the way a devotee would ask about it, then a Telugu question could find a Sanskrit chapter by matching Telugu to Telugu. The Sanskrit would only be read after the chapter is found, by a model that reads Sanskrit well.

02Index the questions, not the verses: chapter cards

A chapter card is a structured Telugu description of one adhyaya, produced by a language model that reads the chapter's Sanskrit and writes:

• a title and a 6-to-10 sentence summary of what happens, in order;
• the key events;
• the dharmic lesson, phrased as general life guidance;
• 10 to 14 questions a devotee would ask that this chapter answers, in everyday Telugu, including at least three phrased as a personal dilemma without naming any character;
• every named character, deity, place and concept, each with its Telugu form and a canonical Sanskrit stem;
• anchors: 3 to 8 key events mapped to the exact verse ids that state them.

Here is a trimmed card for the chapter of the Bhagavata where Ashvatthama is captured and Draupadi asks for his life.

Three properties of the card matter for what comes later. The questions make retrieval a same-language, same-register problem. The entities carry canonical names, so "కర్ణుడు", "కర్ణ" and "karna" can be normalised to one id. The anchors mean the generation step does not have to read the whole chapter to know where the important verses are.

03Building 2,809 cards: model choice, prompt, cost

The card generator is one structured-output call per chapter: the chapter's Sanskrit verses in transliteration, the narrator markers, and a system prompt that says, in short, "you are a Telugu pandit; write plain modern Telugu a devotee reads without a dictionary; stay strictly within these verses; copy verse ids exactly." The output is validated against a schema, and every anchor id is checked against the chapter's real verse ids before the card is accepted.

Which model writes the cards?

I generated the first skandha of the Bhagavata (19 chapters) with two models and measured them with the same 35 hand-written Telugu questions, scoring whether the right chapter came first.

Two things fell out of this. Opus writes better cards, by about 12 points of hit@1, but both models put the right chapter in the top five essentially every time, and the top five is what the answer step reads. And the miss analysis showed the gap was concentrated on moral questions: the cards had story questions but not "should a wrongdoer be forgiven or punished" questions. Requiring three life-situation questions per card doubled Sonnet's moral hit@1 for one extra cent a chapter. That prompt change is worth more than the model change.

Sonnet with the stronger prompt did the bulk run.

Cost and scale

The Batches API is the right tool for this shape of job: 2,389 requests went in as six batches in the afternoon and every one came back within about an hour. Of the first 2,308 batch cards, 20 failed validation, under one percent. Twelve had written anchor ids in GRETIL's own reference style ("BrP_114.6") and were repaired by mapping; the rest had stray Latin letters or an empty question list and were regenerated. The parsing work was larger than the generation work: GRETIL's seventeen files use five different verse-reference conventions, including half-verse markers and a parenthesised form for prose passages.

04Retrieval: Telugu to Telugu, then coarse to fine

Each card is split into units: every generated question, the summary, the moral and the title. Each unit gets a BGE-M3 vector, 45,489 vectors in all. A chapter's dense score is the best score among its units, so a chapter with twelve questions gets twelve chances to match. In parallel, BM25 over character 4-grams of the whole card text handles names, which is where Telugu inflection defeats whole-word search (పరీక్షిత్తు, పరీక్షిత్, పరీక్షిత్తుకు). The two rankings are fused with reciprocal rank fusion, at chapter level.

Before search, a small model does two cheap things: transliterates romanised Telugu into Telugu script (the encoder cannot relate "dhruvudu" to ధ్రువుడు otherwise), and suggests which Puranas are likely to hold the answer.

A routing bug worth writing down

That Purana suggestion was first implemented as a hard filter. It was the single largest source of misses in the full evaluation. When the router guessed wrong, the right chapter was excluded outright: Bharata and the deer was sent to Markandeya, Rukmangada's Ekadashi story to Garuda. Changing the hint into a soft boost, by fusing the unfiltered ranking with the filtered one, lifted recall@10 from 0.82 to 0.90 with no other change.

No vector database

45,489 vectors at 1,024 dimensions is 93 MB in half precision. It lives in RAM as a numpy matrix, and a query is one matrix multiply: 11 ms. The whole retrieval stage, including the routing call and encoding the query, is about 4.6 seconds; search itself is a rounding error. A vector database would have replaced the cheapest step in the pipeline. The index is built once on a Hugging Face Space's GPU in about twenty minutes and saved to a dataset repo, so a restart loads it in seconds.

05Answering: the Sanskrit goes into the context, the Telugu comes out

This is the second half of the language solution, and it is where "no Telugu meaning exists" stops being a problem.

Once the top three chapters are known, their Sanskrit verses are loaded into the answer model's context, in Telugu script for quoting and in transliteration for reading, together with each chapter's card summary. The model is told to answer in modern Telugu, only from those verses, and to attach citations. Each citation is three things: the verse id, a verbatim quote of the Telugu-script sloka, and a one-sentence Telugu tatparyam of that verse written by the model.

Then the quote is checked. Whitespace and punctuation are stripped from both sides and the quoted string must be a substring of the real verse. A citation that fails is dropped. An answer with no surviving citations is not shown; the system refuses instead. This is the same guarantee as Part 1, applied to Sanskrit: the model may write the meaning, but it cannot invent the source.

Notice what happened to the translation problem. We never translated the corpus. The model translates the handful of verses it cites, at answer time, in the context of a specific question, and the translation is shown next to the verbatim Sanskrit it came from so a reader can check it. Every verified tatparyam is written to a cache marked unreviewed. The Telugu meaning layer that did not exist is now growing, verse by verse, driven by what people actually ask, and only those verses will ever need a reviewer.

An example, shortened. Question: "కోపంలో ఇచ్చిన శాపం వల్ల ఏ నష్టం జరుగుతుంది?" (what harm comes from a curse given in anger?). The system found Matsya Purana chapter 33, where Yayati curses the sons who refuse to take his old age, and answered with seven verified citations, one of which was:

06Evaluation

Two layers were measured separately, on the deployed system, not on a laptop copy: retrieval (does the right chapter come up?) and generation (is the answer faithful and correct?).

Retrieval

96 hand-written Telugu questions, each tagged with the chapter or chapters that answer it: 66 on the Bhagavata and 30 spread over twelve other Puranas, including eleven typed in Latin letters and eleven moral questions with no character named. Where a story appears in several Puranas, every valid chapter counts as correct. The system returns its top 20 chapters and we score where the gold chapter lands.

Read the second table before the first. For every kind of question that names what it is about, the right chapter is in the top ten at least nine times in ten, and in the top five, which is more than the answer model reads, 86 to 100 percent of the time. Romanised input, which scored zero before the transliteration step, is now within a few points of native Telugu. Stock BGE-M3, no fine-tuning, no reranker.

The moral row is the failure, and it is an honest one. "Should a guru's adharmic order be obeyed?" does not embed close to the summary of the chapter where Bali defies Shukracharya, not when there are 2,809 summaries to choose from. Several of those "misses" are arguably other valid chapters, but the number is bad either way and it is the next thing to work on.

Generation

For generation, a stratified sample of questions was run through the full pipeline and each answer was scored by a second model acting as judge. Faithfulness: split the answer into its factual claims and count how many are supported by the cited verses and chapter summaries. Correctness: compare the answer with a reference built from the gold chapters' cards, on a 1 to 5 scale.

Only five of the planned 24 answers were judged before the API credit for the day ran out. Five is a small sample and I am reporting it as such; the remaining nineteen are queued.

Two notes on reading these. Every quoted sloka in every answer was a verbatim substring of its verse; the verification step never had to drop one. And the faithfulness figure is conservative: the judge was shown only the cited verses, and the four claims it flagged as unsupported, such as Putana's fall flattening trees for twelve krosas, are in the chapter the model had read but had not cited. With the full context as evidence the score would be higher. The next run will do that.

Speed

A full answer took 76 seconds in the version measured above, of which retrieval was 4.6 seconds and search proper 11 milliseconds. The time is in generation: the model was reading up to three whole chapters and writing about eight citations, each with a quote, a meaning and a claim. Four changes have since been deployed and not yet re-measured: a cap of five citations and a 150-word answer, sending only the anchored verses of each chapter instead of the whole chapter, folding the "is this enough?" check into the generation call, and streaming the answer so the first words appear while the citations are still being written. In Part 1 the same kind of changes took 45 seconds to 10.

07What is not solved

• Moral questions. 0.18 recall@10. Candidates: a cross-encoder reranker over the top 20 (already cached, not yet measured), a dedicated index over the cards' moral fields, or routing moral questions to a judge that picks a story. Part 1 had the same weak spot under a different name.
• The generation sample is five answers. Retrieval numbers are solid at 96 questions; generation numbers are indicative.
• Coverage. Padma, Varaha, Bhavishya and Brahmavaivarta are not on GRETIL at all, and Skanda only as its Reva Khanda. Skanda matters most for temple and Kartika questions.
• Licence. GRETIL's e-texts are CC BY-NC-SA. The Sanskrit is public domain, but the digitisation is not free for commercial use. Verse ids follow the editions, so re-sourcing from Ambuda or Wikisource is a re-parse, not a rebuild.
• The tatparyams are unreviewed. They are shown next to the Sanskrit they translate, which is the safeguard, but a reviewer queue is the real fix.

08Takeaways

1. Close a language gap at build time, not query time. One offline pass per chapter turned a cross-lingual, cross-script search into a Telugu-to-Telugu one.
2. Index at the granularity of questions. Chapters, not verses. It is 50 times smaller and matches how people ask.
3. Search in the user's language; read in the source's. The model that answers is good at Sanskrit. The encoder does not need to be.
4. Do not translate what you cannot verify. Translate the cited verses, on demand, next to the original, and cache them.
5. The prompt that writes your index is a retrieval parameter. Asking for life-situation questions doubled moral-question accuracy for a cent a chapter.
6. Hints should boost, never filter. A wrong hard filter costs the answer; a wrong soft boost costs a rank.
7. Measure the deployed system, per question type. The aggregate hid a category at zero.

Corpus: GRETIL e-texts, CC BY-NC-SA 4.0. Chapter cards generated with Claude Sonnet 5 and Opus 5; answers with Sonnet 5; judge Sonnet 5. Retrieval: stock BAAI/bge-m3 plus BM25. Code: MIT.