Las Leys D'Amors

The project was to translate Las Leys D'Amors, a medieval treatise on troubadour poetry, from Old Occitan to Modern English, using strictly open-weight models and open-source software running locally on a PC.

The manuscript: Toulouse, Bibliothèque Municipale, MS 2883

The translation: cs.bc.edu/mctague/leys-d-amors

In particular, to create this translation from the Internet Archive's scan of Joseph Anglade's 1919 critical edition of the manuscript.

This is somewhat challenging. The critical edition is a bilingual text consisting of a transcription of the Old Occitan manuscript along with critical commentary in academic Modern French. Old Occitan is the ancestor of an endangered language, exhibiting significant semantic drift from its modern relatives (which are much better represented in training corpora). The text consists mostly of medieval verse. There is no such thing as standard spelling in medieval manuscripts. The translation should take into account comments in the critical edition's footnotes.

The translation was done with Gemma4:31b, an open-weight 31-billion-parameter multimodal reasoning model released by Google DeepMind under the Apache 2 license, running on a Strix Halo Framework Desktop PC, using ollama.

Page-by-page, the scans of the text were tokenized and fed into Gemma4's vision transformer along with an elaborate English-language prompt instructing it to transcribe the text (within its chain-of-thought context window) and then translate both languages into modern English, outputting only the translation.

Gemma4 has been trained on scanned books, so it understands text layout, in particular how footnotes work.

Gemma4 has a tendency to hallucinate the spelling of unfamiliar words. For example, it would hallucinate that the footnote

² In place of prumier, the ms. bears pmier, which can be resolved as primier or premier; the copy of the 15th century gives premier.

read

² In place of premier, the ms. bears premier, which can be resolved as premier or premier; the copy of the 15th century gives premier.

It would also sometimes hallucinate that one line of verse would end with the words of the next. I think this happens because the token vectors for adjacent lines of text can get superimposed (depending on how the line spacing interacts with the token grid layout). If the second line extends further right than the first, Gemma4 has a strong tendency to hallucinate the second line's words into the first line. This can be (inconsistently) addressed by increasing the sampling temperature. A more reliable approach was to feed the page into the Tesseract OCR engine, and then feed the OCR output, with caveats, into Gemma4's prompt.

A second issue was that Gemma4 would often appear to get stuck. At first I suspected a problem with the bleeding-edge software or hardware stack, but I eventually found that in fact Gemma4 was operating faithfully as instructed, but engaging in endless philological debate with itself about the linguistic jigsaw puzzle posed by the manuscript. One early example is the invocation at the top of p.1, which might naively be translated as "THE FATHER AND THE SON AND THE THIRD WITHOUT SPIRIT". A footnote adds to the confusion

¹ The scribe of the 14th century first wrote vers, then crossed it out and wrote ters above it; Chabaneau also prints fers, which leads me to suspect that the copy he had at his disposal was made from that of the 15th century.

Gemma4 recognizes that this line has to do with the Trinity and probably is meant to read "The Father and the Son and the Holy Spirit". However, the model believes the Old Occitan spelling of saint should be SANT while the manuscript reads SANS. This and similar issues are currently overcome through prompt engineering and an increase in temperature.

A weakness in the current translation is that each page is translated independently, so there are continuity issues when turning pages, especially when sentences span the page boundary. I might address this in a future translation by feeding the last few lines of the preceding page's translation into the prompt for the next.

The translation for the full 800-some pages took 3-4 days.

I've put the translation online at

cs.bc.edu/mctague/leys-d-amors

(A static webpage written in Vanilla JS.)

• The full prompt and chain of thought can be read by clicking the "VIEW PROCESS LOG" button.
• Turn pages with left/right arrow keys or clicking on the left/right borders of the page.
• Easily zoom into the text by clicking on the scan on the left. Click anywhere or hit escape to zoom back out.
• The hamburger icon (☰) opens a table of contents to jump around the 800-some pages of the four books.
• Quickly search the entire book by hitting Control-F. (The images are loaded on-demand, but the full translation is stored and loaded from a single JSON file, so search is fast and local.)
• A regex automatically hyperlinks references to the folios of the manuscript.

One of my favorite details is the Roman numeral for Folio 178.

I feel guilty about how much Gemma4 suffers when asked to translate blank pages, but adding explicit instructions for blank pages would have added to the complexity of handling the vast majority of pages which aren't blank.