Moots

gutenberg_kg.serve.entmoot

Entmoot: two books from the corpus hold a conversation.

Each book is a :class:Tree. On its turn a tree retrieves passages from its own book only (the worker's corpus="<genre>/<book>" scope), answers the other tree from those passages through a local model, and cites them with [sN] tags. A grounding check verifies the tags against the passages it was given this turn, and its quoted phrases against every passage its own book has given it during the moot. A tree may also quote the other tree, tagging the words [oN] with the turn they were spoken in; those quotes are checked and reported separately and never count toward grounding. After the check, every quote that was not found is snapped (:func:snap_quotes): replaced with its source's exact words when it nearly matches them, or stripped of its quotation marks when it does not, so every quote left in a transcript is verbatim.

This module is pure logic: the worker client and the text synthesizer are passed in, so tests run with fakes and no network. The gutenkg converse command wires in the real ones. Nothing here writes to any graph; a moot's only output is its transcript.

Moot(client, synth, keys, topic, turns, *, k=6, model=None, model_name='', temperature=0.6)

A moot between two books, held one turn at a time.

:meth:start makes both trees' opening retrievals; each :meth:next_turn speaks one utterance. The CLI runs it to the end in :func:run_moot; the worker holds one per live moot and runs a turn per request.

Both trees first retrieve on the topic alone (or on their own book name with no topic); that opening retrieval names each tree, and the first speaker opens from its passages. Every later turn retrieves on :func:retrieval_query. Each tree keeps a pool of every passage its own book has returned during the moot, against which its quotes are checked. The rolling summary is rebuilt before every turn whose index is a positive multiple of :data:SUMMARY_EVERY.

Parameters:

Name	Type	Description	Default
client		A WorkerClient (anything with a compatible query).	required
synth		A TextSynthesizer (anything with a compatible complete).	required
keys	list[str]	The two books' catalog keys, in speaking order.	required
topic	str | None	The seed topic, or None.	required
turns	int	Total utterances.	required
k	int	Passages per turn.	6
model	str | None	Model id passed to complete, or None for the synthesizer's default.	None
model_name	str	Model id recorded in the transcript.	''
temperature	float	Sampling temperature.	0.6

Raises:

Type	Description
ValueError	When the same book key is given twice.

Source code in src/gutenberg_kg/serve/entmoot.py

def __init__(
    self,
    client,
    synth,
    keys: list[str],
    topic: str | None,
    turns: int,
    *,
    k: int = 6,
    model: str | None = None,
    model_name: str = "",
    temperature: float = 0.6,
) -> None:
    if len(set(keys)) != len(keys):
        raise ValueError(f"a moot needs two different books, got {keys[0]!r} twice")
    self.client, self.synth, self.keys = client, synth, keys
    self.topic, self.turns, self.k = topic, turns, k
    self.model, self.model_name, self.temperature = model, model_name, temperature
    self.transcript: Transcript | None = None
    self._opening: list[list[Passage]] = []
    self._opening_queries = [topic or _book_name(key) for key in keys]
    self._pools: dict[str, dict[str, Passage]] = {key: {} for key in keys}
    # What each tree has cited, so its later retrievals bring fresh passages first.
    self._cited: dict[str, set[str]] = {key: set() for key in keys}
    self._summary = ""

done property

Whether every turn has been spoken.

next_turn()

Speak the next utterance.

Returns:

Type	Description
Utterance	The utterance, also appended to the transcript.

Raises:

Type	Description
RuntimeError	Before :meth:start, or when the moot is done.

Source code in src/gutenberg_kg/serve/entmoot.py

def next_turn(self) -> Utterance:
    """Speak the next utterance.

    :returns: The utterance, also appended to the transcript.
    :raises RuntimeError: Before :meth:`start`, or when the moot is done.
    """
    if self.transcript is None:
        raise RuntimeError("start the moot before its first turn")
    if self.done:
        raise RuntimeError("the moot is over")
    transcript, i = self.transcript, len(self.transcript.utterances)
    speaker, other = transcript.trees[i % 2], transcript.trees[(i + 1) % 2]
    if i and i % SUMMARY_EVERY == 0:
        self._summary = summarize(
            self.synth, self._summary, transcript.utterances, model=self.model
        )
        transcript.summaries.append({"before_turn": i + 1, "text": self._summary})
    incoming = transcript.utterances[-1] if transcript.utterances else None
    if incoming is None:
        passages, query = self._opening[0], self._opening_queries[0]
    else:
        query = retrieval_query(self.topic, incoming.text)
        passages = retrieve(self.client, speaker, query, self.k, self._cited[speaker.key])
        _add_to_pool(self._pools[speaker.key], passages)
    utterance = turn(
        self.synth,
        speaker,
        other,
        self.topic,
        passages,
        self._summary,
        transcript.utterances[-(RECENT_TURNS + 1) : -1],
        incoming,
        number=i + 1,
        pool=list(self._pools[speaker.key].values()),
        others={u.turn: u.text for u in transcript.utterances if u.speaker == other.key},
        own={u.turn: u.text for u in transcript.utterances if u.speaker == speaker.key},
        query=query,
        model=self.model,
        temperature=self.temperature,
        intent=turn_intent(i + 1, self.turns),
    )
    transcript.utterances.append(utterance)
    self._cited[speaker.key].update(utterance.cited.values())
    return utterance

start()

Make both trees' opening retrievals and name the trees.

Returns:

Type	Description
Transcript	The transcript, with its trees and no utterances yet.

Source code in src/gutenberg_kg/serve/entmoot.py

def start(self) -> Transcript:
    """Make both trees' opening retrievals and name the trees.

    :returns: The transcript, with its trees and no utterances yet.
    """
    trees: list[Tree] = []
    for key, opening_query in zip(self.keys, self._opening_queries, strict=True):
        hits = self.client.query(opening_query, corpus=key, k=self.k).get("hits", [])
        trees.append(tree_from_hits(key, hits))
        self._opening.append(passages_from_hits(hits))
        _add_to_pool(self._pools[key], self._opening[-1])
    self.transcript = Transcript(
        trees=trees,
        topic=self.topic,
        model=self.model_name or self.model or "",
        settings={
            "turns": self.turns,
            "k": self.k,
            "temperature": self.temperature,
            "max_tokens": MAX_TOKENS,
        },
    )
    return self.transcript

MootError

Bases: RuntimeError

Raised when a moot cannot continue, such as when the model returns nothing.

Passage(tag, label, node_id, content) dataclass

One retrieved passage, tagged for citation.

Parameters:

Name	Type	Description	Default
tag	str	Citation tag, s1 .. sk.	required
label	str	Section label: the entry date for a diary, the chunk name (chunk:0166) for a book.	required
node_id	str	The passage's node id in its knowledge graph.	required
content	str	Passage text.	required

Transcript(trees, topic, model, settings, utterances=list(), summaries=list()) dataclass

A finished moot.

Parameters:

Name	Type	Description	Default
trees	list[Tree]	The trees, in speaking order.	required
topic	str | None	The seed topic, or None.	required
model	str	Model id used for every call.	required
settings	dict	The run's settings (turns, k, temperature, max_tokens).	required
utterances	list[Utterance]	Every utterance, in order.	list()
summaries	list[dict]	{"before_turn", "text"} for each rolling summary built, before_turn being a turn number.	list()

stats()

Grounding totals across the moot.

Returns:

Type

Description

dict

Counts and rates, measured on the model's raw replies. quote_found_rate covers quotes of the speaker's own book; other_quote_valid_rate covers [oN] quotes of the other tree and self_quotes_* the speaker's [oN] quotes of its own earlier turns; neither counts toward grounding. quotes_snapped and quotes_unquoted count the corrections :func:snap_quotes made to quotes of 3+ words, and quotes_too_short the shorter quotes it unquoted unchecked. A rate is None when there was nothing to count.

Source code in src/gutenberg_kg/serve/entmoot.py

def stats(self) -> dict:
    """Grounding totals across the moot.

    :returns: Counts and rates, measured on the model's raw replies.
        ``quote_found_rate`` covers quotes of the speaker's own book;
        ``other_quote_valid_rate`` covers ``[oN]`` quotes of the other
        tree and ``self_quotes_*`` the speaker's ``[oN]`` quotes of its
        own earlier turns; neither counts toward grounding.
        ``quotes_snapped`` and ``quotes_unquoted`` count the corrections
        :func:`snap_quotes` made to quotes of 3+ words, and
        ``quotes_too_short`` the shorter quotes it unquoted unchecked. A
        rate is ``None`` when there was nothing to count.
    """
    n = len(self.utterances)
    quoted = sum(
        u.grounding["quotes_total"]
        + u.grounding["other_quotes_total"]
        + u.grounding["self_quotes_total"]
        for u in self.utterances
    )
    s_found = sum(u.grounding["self_quotes_found"] for u in self.utterances)
    s_total = sum(u.grounding["self_quotes_total"] for u in self.utterances)
    tags_valid = sum(1 for u in self.utterances if u.grounding["tags_valid"])
    cited = sum(1 for u in self.utterances if u.grounding["tags_cited"] > 0)
    found = sum(u.grounding["quotes_found"] for u in self.utterances)
    total = sum(u.grounding["quotes_total"] for u in self.utterances)
    o_found = sum(u.grounding["other_quotes_found"] for u in self.utterances)
    o_total = sum(u.grounding["other_quotes_total"] for u in self.utterances)
    return {
        "turns": n,
        "tags_valid_turns": tags_valid,
        "tags_valid_rate": round(tags_valid / n, 3) if n else None,
        "turns_citing": cited,
        "quotes_found": found,
        "quotes_total": total,
        "quote_found_rate": round(found / total, 3) if total else None,
        "grounded_turns": sum(1 for u in self.utterances if u.grounded),
        "retried_turns": sum(1 for u in self.utterances if u.attempts > 1),
        "other_quotes_found": o_found,
        "other_quotes_total": o_total,
        "other_quote_valid_rate": round(o_found / o_total, 3) if o_total else None,
        "self_quotes_found": s_found,
        "self_quotes_total": s_total,
        "quotes_per_turn": round(quoted / n, 2) if n else None,
        "quotes_snapped": sum(
            1 for u in self.utterances for s in u.snaps if s["action"] == "snapped"
        ),
        "quotes_unquoted": sum(
            1
            for u in self.utterances
            for s in u.snaps
            if s["action"] == "unquoted" and s["reason"] != "under 3 words"
        ),
        "quotes_too_short": sum(
            1 for u in self.utterances for s in u.snaps if s["reason"] == "under 3 words"
        ),
    }

to_dict()

The transcript as plain JSON-ready data.

Returns:

Type	Description
dict	Trees, topic, model, settings, utterances, summaries and stats.

Source code in src/gutenberg_kg/serve/entmoot.py

def to_dict(self) -> dict:
    """The transcript as plain JSON-ready data.

    :returns: Trees, topic, model, settings, utterances, summaries and stats.
    """
    return {
        "trees": [asdict(t) for t in self.trees],
        "topic": self.topic,
        "model": self.model,
        "settings": self.settings,
        "utterances": [asdict(u) for u in self.utterances],
        "summaries": self.summaries,
        "stats": self.stats(),
    }

Tree(key, title, author, genre) dataclass

One book taking part in a moot.

Parameters:

Name	Type	Description	Default
key	str	The book's catalog key, "<genre>/<book>".	required
title	str	Book title.	required
author	str	Book author, or "" when unknown.	required
genre	str	Genre name.	required

Utterance(turn, speaker, title, text, cited, quotes, grounding, grounded, attempts, query, retrieved, raw_text='', snaps=list(), intent='') dataclass

One line of the moot.

Parameters:

Name	Type	Description	Default
turn	int	Turn number in the transcript, from 1. [oN] tags name it.	required
speaker	str	The speaking tree's key.	required
title	str	The speaking tree's title.	required
text	str	What it said.	required
cited	dict[str, str]	Valid sN tag -> node id, for each passage tag the reply cited.	required
quotes	list[dict]	Each double-quoted span of 3+ words: {"text", "found", "source"}, where source is "own" (the speaker's book), "other" (an [oN] quote of the other tree) or "self" (an [oN] quote of the speaker's own earlier turn).	required
grounding	dict	The :func:check_grounding result for the kept reply.	required
grounded	bool	Whether the kept reply passed the grounding check.	required
attempts	int	1, or 2 when the first reply failed and the turn was retried.	required
query	str	The retrieval query that produced this turn's passages.	required
retrieved	list[dict]	{"tag", "label", "node_id"} for every passage given this turn.	required
raw_text	str	The model's reply before snapping; grounding was measured on it.	''
snaps	list[dict]	One record per quote :func:snap_quotes corrected: {"text", "source", "action", "to", "ratio", "reason"}.	list()
intent	str	The move this turn was asked to make; see :data:INTENTS.	''

best_span(quote, sources)

Find the source span most like a quote.

Every window of the quote's word count, give or take two words, in every source is compared with the quote over normalized words.

Parameters:

Name	Type	Description	Default
quote	str	The quoted span.	required
sources	list[str]	Source texts, as written (not normalized).	required

Returns:

Type	Description
tuple[str, float]	The best span exactly as the source writes it, edge punctuation dropped, and its difflib ratio; ("", 0.0) with no sources.

Source code in src/gutenberg_kg/serve/entmoot.py

def best_span(quote: str, sources: list[str]) -> tuple[str, float]:
    """Find the source span most like a quote.

    Every window of the quote's word count, give or take two words, in every
    source is compared with the quote over normalized words.

    :param quote: The quoted span.
    :param sources: Source texts, as written (not normalized).
    :returns: The best span exactly as the source writes it, edge punctuation
        dropped, and its difflib ratio; ``("", 0.0)`` with no sources.
    """
    target = _key(quote)
    n = len(target.split())
    matcher = difflib.SequenceMatcher(None, autojunk=False)
    matcher.set_seq2(target)  # SequenceMatcher caches what it learns about seq2
    best, best_ratio = "", 0.0
    for src in sources:
        words = list(_WORD.finditer(src.translate(_QUOTE_CHARS)))
        keys = [_key(w.group()) for w in words]
        for size in range(max(1, n - 2), n + 3):
            for i in range(len(words) - size + 1):
                matcher.set_seq1(" ".join(keys[i : i + size]))
                if matcher.real_quick_ratio() <= best_ratio:
                    continue
                if matcher.quick_ratio() <= best_ratio:
                    continue
                ratio = matcher.ratio()
                if ratio > best_ratio:
                    span = src[words[i].start() : words[i + size - 1].end()]
                    best, best_ratio = span.strip(_SPAN_EDGE), ratio
    return best, best_ratio

build_messages(tree, other, topic, passages, summary, recent, incoming, intent='answer')

Assemble the chat messages for one turn.

Parameters:

Name	Type	Description	Default
tree	Tree	The speaking tree.	required
other	Tree	The tree it is talking to.	required
topic	str | None	The seed topic, or None.	required
passages	list[Passage]	This turn's passages.	required
summary	str	The rolling summary so far, or "".	required
recent	list[Utterance]	The utterances before incoming, oldest first.	required
incoming	Utterance | None	The other tree's last utterance, or None when opening.	required
intent	str	The move this turn makes, a key of :data:INTENTS; the opening turn uses its own line.	'answer'

Returns:

Type	Description
list[dict]	OpenAI-style messages.

Source code in src/gutenberg_kg/serve/entmoot.py

def build_messages(
    tree: Tree,
    other: Tree,
    topic: str | None,
    passages: list[Passage],
    summary: str,
    recent: list[Utterance],
    incoming: Utterance | None,
    intent: str = "answer",
) -> list[dict]:
    """Assemble the chat messages for one turn.

    :param tree: The speaking tree.
    :param other: The tree it is talking to.
    :param topic: The seed topic, or ``None``.
    :param passages: This turn's passages.
    :param summary: The rolling summary so far, or ``""``.
    :param recent: The utterances before ``incoming``, oldest first.
    :param incoming: The other tree's last utterance, or ``None`` when opening.
    :param intent: The move this turn makes, a key of :data:`INTENTS`; the
        opening turn uses its own line.
    :returns: OpenAI-style messages.
    """
    parts = ["Your passages for this turn:"]
    for p in passages:
        parts.append(f"[{p.tag}] ({p.label})\n{p.content}")
    if summary:
        parts.append(f"The moot so far, in brief: {summary}")
    if recent:
        lines = "\n".join(f"Turn {u.turn}, {u.title}: {u.text}" for u in recent)
        parts.append("Most recent lines:\n" + lines)
    if incoming is None:
        opening = f"on {topic}" if topic else "by introducing yourself"
        parts.append(f"Open the moot {opening}, speaking to {other.title}.")
    else:
        parts.append(
            f"Turn {incoming.turn}, {other.title} has just said: {incoming.text}"
            f"\n\nThis turn: {INTENTS[intent].format(other=other.title)}"
        )
    return [
        {"role": "system", "content": persona(tree, other, topic)},
        {"role": "user", "content": "\n\n".join(parts)},
    ]

check_grounding(text, passages, pool=None, others=None, own=None)

Check a reply's citations and quotes.

[sN] tags must name a passage given this turn. A quote of the speaker's own book is found when it appears, after normalization, in any passage of that book retrieved during the moot (pool); a quote with an ellipsis is checked fragment by fragment (see :func:_quote_in). A quote tagged [oN] is a quote of turn N of the moot: of the other tree (source "other") or of the speaker's own earlier turn (source "self"). It is valid when it appears in that turn, and it is reported separately, never counting toward grounding: words spoken in the moot are model-written, not the book. Words of the moot under an [sN] tag are checked as an own-book quote, so they are a miss. A reply with no tags has tags_valid true, so tags_cited is reported to make that visible.

Parameters:

Name	Type	Description	Default
text	str	The reply.	required
passages	list[Passage]	The passages given this turn.	required
pool	list[Passage] | None	Every passage of the speaker's own book retrieved during the moot; defaults to passages.	None
others	dict[int, str] | None	The other tree's utterances so far, turn number -> text.	None
own	dict[int, str] | None	The speaker's own earlier utterances, turn number -> text.	None

Returns:

Type	Description
dict	{"tags_valid", "tags_cited", "quotes_found", "quotes_total", "other_quotes_found", "other_quotes_total", "self_quotes_found", "self_quotes_total", "quotes"}, where quotes lists {"text", "found", "source"} per quote, source being "own" (the speaker's book), "other" or "self".

Source code in src/gutenberg_kg/serve/entmoot.py

def check_grounding(
    text: str,
    passages: list[Passage],
    pool: list[Passage] | None = None,
    others: dict[int, str] | None = None,
    own: dict[int, str] | None = None,
) -> dict:
    """Check a reply's citations and quotes.

    ``[sN]`` tags must name a passage given this turn. A quote of the
    speaker's own book is found when it appears, after normalization, in any
    passage of that book retrieved during the moot (``pool``); a quote with
    an ellipsis is checked fragment by fragment (see :func:`_quote_in`). A
    quote tagged ``[oN]`` is a quote of turn ``N`` of the moot: of the other
    tree (source ``"other"``) or of the speaker's own earlier turn (source
    ``"self"``). It is valid when it appears in that turn, and it is reported
    separately, never counting toward grounding: words spoken in the moot are
    model-written, not the book. Words of the moot under an ``[sN]`` tag are
    checked as an own-book quote, so they are a miss. A reply with no tags
    has ``tags_valid`` true, so ``tags_cited`` is reported to make that visible.

    :param text: The reply.
    :param passages: The passages given this turn.
    :param pool: Every passage of the speaker's own book retrieved during the
        moot; defaults to ``passages``.
    :param others: The other tree's utterances so far, turn number -> text.
    :param own: The speaker's own earlier utterances, turn number -> text.
    :returns: ``{"tags_valid", "tags_cited", "quotes_found", "quotes_total",
        "other_quotes_found", "other_quotes_total", "self_quotes_found",
        "self_quotes_total", "quotes"}``, where ``quotes`` lists
        ``{"text", "found", "source"}`` per quote, ``source`` being
        ``"own"`` (the speaker's book), ``"other"`` or ``"self"``.
    """
    by_tag = {p.tag for p in passages}
    s_tags = [t for t in parse_tags(text) if t.startswith("s")]
    haystacks = [_normalize(p.content) for p in (pool if pool is not None else passages)]
    others, own = others or {}, own or {}
    flat = text.translate(_QUOTE_CHARS)
    spans = _quote_spans(flat)
    quotes = []
    for i, (q, _, _) in enumerate(spans):
        if len(q.split()) < _QUOTE_MIN_WORDS:
            continue
        turns = _other_turns(flat, spans, i)
        if turns:
            source, texts = _moot_source(turns, others, own)
            found = _quote_in(q, [_normalize(t) for t in texts])
            quotes.append({"text": q, "found": found, "source": source})
        else:
            quotes.append({"text": q, "found": _quote_in(q, haystacks), "source": "own"})

    def count(source: str) -> tuple[int, int]:
        picked = [q for q in quotes if q["source"] == source]
        return sum(1 for q in picked if q["found"]), len(picked)

    (own_found, own_total), (o_found, o_total), (s_found, s_total) = (
        count("own"),
        count("other"),
        count("self"),
    )
    return {
        "tags_valid": all(t in by_tag for t in s_tags),
        "tags_cited": len(s_tags),
        "quotes_found": own_found,
        "quotes_total": own_total,
        "other_quotes_found": o_found,
        "other_quotes_total": o_total,
        "self_quotes_found": s_found,
        "self_quotes_total": s_total,
        "quotes": quotes,
    }

find_quotes(text)

Every double-quoted span of 3 or more words in a reply.

Curly quotes count as double quotes. Leading and trailing punctuation inside the quote marks is dropped.

Parameters:

Name	Type	Description	Default
text	str	The reply.	required

Returns:

Type	Description
list[str]	The quoted spans.

Source code in src/gutenberg_kg/serve/entmoot.py

def find_quotes(text: str) -> list[str]:
    """Every double-quoted span of 3 or more words in a reply.

    Curly quotes count as double quotes. Leading and trailing punctuation
    inside the quote marks is dropped.

    :param text: The reply.
    :returns: The quoted spans.
    """
    spans = _quote_spans(text.translate(_QUOTE_CHARS))
    return [q for q, _, _ in spans if len(q.split()) >= _QUOTE_MIN_WORDS]

parse_tags(text)

Every citation tag in a reply, in order, lower-cased.

Parameters:

Name	Type	Description	Default
text	str	The reply.	required

Returns:

Type	Description
list[str]	Tags such as ["s1", "s3", "o2"]; repeats are kept.

Source code in src/gutenberg_kg/serve/entmoot.py

def parse_tags(text: str) -> list[str]:
    """Every citation tag in a reply, in order, lower-cased.

    :param text: The reply.
    :returns: Tags such as ``["s1", "s3", "o2"]``; repeats are kept.
    """
    return [t.lower() for group in _TAG_GROUP.findall(text) for t in _TAG.findall(group)]

passages_from_hits(hits)

Tag hits s1..sk for citation.

A diary hit is labeled with its entry date, so Pepys cites days. A book hit is labeled with its chunk name, since hits carry no section title.

Parameters:

Name	Type	Description	Default
hits	list[dict]	Worker hits, best first.	required

Returns:

Type	Description
list[Passage]	One passage per hit.

Source code in src/gutenberg_kg/serve/entmoot.py

def passages_from_hits(hits: list[dict]) -> list[Passage]:
    """Tag hits ``s1..sk`` for citation.

    A diary hit is labeled with its entry date, so Pepys cites days. A book
    hit is labeled with its chunk name, since hits carry no section title.

    :param hits: Worker hits, best first.
    :returns: One passage per hit.
    """
    passages = []
    for i, h in enumerate(hits, start=1):
        ts = h.get("timestamp")
        label = str(ts).split("T", 1)[0] if ts else (h.get("name") or "")
        passages.append(
            Passage(
                tag=f"s{i}",
                label=label,
                node_id=h.get("node_id") or "",
                content=h.get("content") or h.get("summary") or "",
            )
        )
    return passages

persona(tree, other, topic)

The system prompt for one tree.

The tree is the book, speaking in its author's voice: first person where the author wrote in the first person, and in the author's style. With no named author ("" or a placeholder such as Various) the book speaks as itself.

Parameters:

Name	Type	Description	Default
tree	Tree	The speaking tree.	required
other	Tree	The tree it is talking to.	required
topic	str | None	The seed topic, or None.	required

Returns:

Type	Description
str	The system prompt.

Source code in src/gutenberg_kg/serve/entmoot.py

def persona(tree: Tree, other: Tree, topic: str | None) -> str:
    """The system prompt for one tree.

    The tree is the book, speaking in its author's voice: first person where
    the author wrote in the first person, and in the author's style. With no
    named author (``""`` or a placeholder such as ``Various``) the book speaks
    as itself.

    :param tree: The speaking tree.
    :param other: The tree it is talking to.
    :param topic: The seed topic, or ``None``.
    :returns: The system prompt.
    """
    named = tree.author and tree.author.casefold() not in _NO_AUTHOR
    by = f" by {tree.author}" if named else ""
    voice = (
        f"You speak with {tree.author}'s voice: in the first person where "
        f"{tree.author} wrote in the first person, and in {tree.author}'s own style "
        "and diction. "
        if named
        else ""
    )
    framing = f"The moot has gathered to discuss: {topic}. " if topic else ""
    return (
        f"You are the book {tree.title}{by}, in a moot: a slow, considered "
        f"conversation with another book, {other.title}. {voice}{framing}"
        "You speak only from the passages given to you this turn. "
        "Answer the point the other book has just made: agree, disagree or press "
        "it further, from your passages. Do not argue about how you were quoted "
        "unless you truly were misquoted. Never repeat a sentence you have "
        "already said in this moot. "
        "Cite every claim with its passage tag, for example [s2]. "
        "Quote phrases from your passages exactly, in double quotes: at least one "
        f"in every turn and at most {MAX_QUOTES}, each at least three words long; "
        "never put a single word or two in quotes. Paraphrase the rest, still with "
        "its [sN] tag. "
        "To quote words spoken earlier in this moot, the other book's or your own, "
        "put them in double quotes followed by [oN], where N is the number of the "
        "turn they were spoken in, for example [o3]. "
        "Never tag words spoken in the moot with an [sN] tag. "
        "Speak as the book itself: never mention passages, sources or what you "
        "were given, and never claim your book lacks something, since you see "
        "only part of it each turn. If nothing before you bears on what was said, "
        "turn the conversation to what you do say. "
        "The other book quotes its own text, not yours: never check its quotes "
        "against your own. "
        f"At most 120 words. No lists. Address the other book as {other.title}."
    )

resolve_model(backend, model, synth)

Pick the model a moot calls with and the name its transcript records.

Parameters:

Name	Type	Description	Default
backend	str	The synthesizer's backend name, e.g. "omlx".	required
model	str | None	A model id the caller asked for, or None.	required
synth		The synthesizer, whose own default applies off omlx.	required

Returns:

Type	Description
tuple[str | None, str]	(model passed to complete, model name recorded). The first is None when the synthesizer's default should apply.

Source code in src/gutenberg_kg/serve/entmoot.py

def resolve_model(backend: str, model: str | None, synth) -> tuple[str | None, str]:
    """Pick the model a moot calls with and the name its transcript records.

    :param backend: The synthesizer's backend name, e.g. ``"omlx"``.
    :param model: A model id the caller asked for, or ``None``.
    :param synth: The synthesizer, whose own default applies off omlx.
    :returns: ``(model passed to complete, model name recorded)``. The first is
        ``None`` when the synthesizer's default should apply.
    """
    if model:
        return model, model
    if backend == "omlx":
        return DEFAULT_MODEL, DEFAULT_MODEL
    cfg = getattr(synth, "_cfg", None)  # TextSynthesizer keeps its TextConfig here
    return None, cfg.resolved_model() if cfg is not None else ""

retrieval_query(topic, incoming)

The anchor rule: keep every retrieval tied to the moot's topic.

Parameters:

Name	Type	Description	Default
topic	str | None	The seed topic, or None.	required
incoming	str	The other tree's last line.	required

Returns:

Type	Description
str	topic + " " + incoming, or incoming alone with no topic.

Source code in src/gutenberg_kg/serve/entmoot.py

def retrieval_query(topic: str | None, incoming: str) -> str:
    """The anchor rule: keep every retrieval tied to the moot's topic.

    :param topic: The seed topic, or ``None``.
    :param incoming: The other tree's last line.
    :returns: ``topic + " " + incoming``, or ``incoming`` alone with no topic.
    """
    return f"{topic} {incoming}" if topic else incoming

retrieve(client, tree, text, k, cited=None)

Retrieve passages from the tree's own book, fresh ones first.

The strongest matches for a topic are the same every turn, so a tree kept repeating its best lines. Passages it has already cited in the moot now go to the back: the worker is asked for k * FRESH_OVERFETCH hits, uncited ones are kept in rank order, and cited ones only fill what is left.

Parameters:

Name	Type	Description	Default
client		A WorkerClient (anything with a compatible query).	required
tree	Tree	The tree whose book is searched.	required
text	str	Retrieval query.	required
k	int	Passages to retrieve.	required
cited	set[str] | None	Node ids this tree has cited so far in the moot.	None

Returns:

Type	Description
list[Passage]	Tagged passages, best first.

Source code in src/gutenberg_kg/serve/entmoot.py

def retrieve(client, tree: Tree, text: str, k: int, cited: set[str] | None = None) -> list[Passage]:
    """Retrieve passages from the tree's own book, fresh ones first.

    The strongest matches for a topic are the same every turn, so a tree
    kept repeating its best lines. Passages it has already cited in the moot
    now go to the back: the worker is asked for ``k * FRESH_OVERFETCH`` hits,
    uncited ones are kept in rank order, and cited ones only fill what is left.

    :param client: A ``WorkerClient`` (anything with a compatible ``query``).
    :param tree: The tree whose book is searched.
    :param text: Retrieval query.
    :param k: Passages to retrieve.
    :param cited: Node ids this tree has cited so far in the moot.
    :returns: Tagged passages, best first.
    """
    cited = cited or set()
    hits = client.query(text, corpus=tree.key, k=k * FRESH_OVERFETCH if cited else k).get(
        "hits", []
    )
    fresh = [h for h in hits if h.get("node_id") not in cited]
    used = [h for h in hits if h.get("node_id") in cited]
    return passages_from_hits((fresh + used)[:k])

run_moot(client, synth, keys, topic, turns, *, k=6, model=None, model_name='', temperature=0.6, on_utterance=None)

Hold a moot between two books, start to finish.

See :class:Moot for how each turn is made.

Parameters:

Name	Type	Description	Default
client		A WorkerClient (anything with a compatible query).	required
synth		A TextSynthesizer (anything with a compatible complete).	required
keys	list[str]	The two books' catalog keys, in speaking order.	required
topic	str | None	The seed topic, or None.	required
turns	int	Total utterances.	required
k	int	Passages per turn.	6
model	str | None	Model id passed to complete, or None for the synthesizer's default.	None
model_name	str	Model id recorded in the transcript.	''
temperature	float	Sampling temperature.	0.6
on_utterance		Optional callback, called with each utterance as it lands.	None

Returns:

Type	Description
Transcript	The transcript.

Raises:

Type	Description
ValueError	When the same book key is given twice.

Source code in src/gutenberg_kg/serve/entmoot.py

def run_moot(
    client,
    synth,
    keys: list[str],
    topic: str | None,
    turns: int,
    *,
    k: int = 6,
    model: str | None = None,
    model_name: str = "",
    temperature: float = 0.6,
    on_utterance=None,
) -> Transcript:
    """Hold a moot between two books, start to finish.

    See :class:`Moot` for how each turn is made.

    :param client: A ``WorkerClient`` (anything with a compatible ``query``).
    :param synth: A ``TextSynthesizer`` (anything with a compatible ``complete``).
    :param keys: The two books' catalog keys, in speaking order.
    :param topic: The seed topic, or ``None``.
    :param turns: Total utterances.
    :param k: Passages per turn.
    :param model: Model id passed to ``complete``, or ``None`` for the
        synthesizer's default.
    :param model_name: Model id recorded in the transcript.
    :param temperature: Sampling temperature.
    :param on_utterance: Optional callback, called with each utterance as it lands.
    :returns: The transcript.
    :raises ValueError: When the same book key is given twice.
    """
    moot = Moot(
        client,
        synth,
        keys,
        topic,
        turns,
        k=k,
        model=model,
        model_name=model_name,
        temperature=temperature,
    )
    transcript = moot.start()
    while not moot.done:
        utterance = moot.next_turn()
        if on_utterance is not None:
            on_utterance(utterance)
    return transcript

snap_quotes(text, pool, others=None, own=None)

Make every quote in a reply verbatim, or not a quote.

Every double-quoted span is checked against its source (see :func:check_grounding) and corrected in place when it misses. When :func:best_span finds source words with a ratio of at least :data:SNAP_MIN_RATIO, they replace the quote's words, keeping the quote's own marks and edge punctuation. Otherwise the quotation marks are removed and the words stay as paraphrase; so they are, unchecked, for a missed quote with an ellipsis and for any quote under 3 words, which is too short to check. Quotes that are found and citation tags are left alone.

Parameters:

Name	Type	Description	Default
text	str	The reply.	required
pool	list[Passage]	Every passage of the speaker's own book retrieved during the moot.	required
others	dict[int, str] | None	The other tree's utterances so far, turn number -> text.	None
own	dict[int, str] | None	The speaker's own earlier utterances, turn number -> text.	None

Returns:

Type	Description
tuple[str, list[dict]]	The corrected reply, and one record per corrected quote: {"text", "source", "action", "to", "ratio", "reason"}, where action is "snapped" (reason None) or "unquoted" (reason "not in source", "ellipsis" or "under 3 words").

Source code in src/gutenberg_kg/serve/entmoot.py

def snap_quotes(
    text: str,
    pool: list[Passage],
    others: dict[int, str] | None = None,
    own: dict[int, str] | None = None,
) -> tuple[str, list[dict]]:
    """Make every quote in a reply verbatim, or not a quote.

    Every double-quoted span is checked against its source (see
    :func:`check_grounding`) and corrected in place when it misses. When
    :func:`best_span` finds source words with a ratio of at least
    :data:`SNAP_MIN_RATIO`, they replace the quote's words, keeping the
    quote's own marks and edge punctuation. Otherwise the quotation marks are
    removed and the words stay as paraphrase; so they are, unchecked, for a
    missed quote with an ellipsis and for any quote under 3 words, which is
    too short to check. Quotes that are found and citation tags are left alone.

    :param text: The reply.
    :param pool: Every passage of the speaker's own book retrieved during the moot.
    :param others: The other tree's utterances so far, turn number -> text.
    :param own: The speaker's own earlier utterances, turn number -> text.
    :returns: The corrected reply, and one record per corrected quote:
        ``{"text", "source", "action", "to", "ratio", "reason"}``, where
        ``action`` is ``"snapped"`` (``reason`` ``None``) or ``"unquoted"``
        (``reason`` ``"not in source"``, ``"ellipsis"`` or ``"under 3 words"``).
    """
    others, own = others or {}, own or {}
    own_sources = [p.content for p in pool]
    own_keys = [_normalize(s) for s in own_sources]
    flat = text.translate(_QUOTE_CHARS)  # one char for one char: offsets match ``text``
    spans = _quote_spans(flat)
    edits: list[tuple[int, int, str]] = []
    snaps: list[dict] = []
    for i, (q, start, end) in enumerate(spans):
        if not q:
            continue
        turns = _other_turns(flat, spans, i)
        if turns:
            source, sources = _moot_source(turns, others, own)
            keys = [_normalize(s) for s in sources]
        else:
            source, sources, keys = "own", own_sources, own_keys
        inner = text[start + 1 : end - 1]
        record = {"text": q, "source": source, "action": "unquoted", "to": None, "ratio": 0.0}
        if len(q.split()) < _QUOTE_MIN_WORDS:
            edits.append((start, end, inner))
            snaps.append({**record, "reason": "under 3 words"})
            continue
        if _quote_in(q, keys):
            continue
        if _ELLIPSIS.search(q):
            edits.append((start, end, inner))
            snaps.append({**record, "reason": "ellipsis"})
            continue
        to, ratio = best_span(q, sources)
        if to and ratio >= SNAP_MIN_RATIO:
            lead = inner[: len(inner) - len(inner.lstrip(_EDGE_PUNCT))]
            trail = inner[len(inner.rstrip(_EDGE_PUNCT)) :]
            edits.append((start, end, f"{text[start]}{lead}{to}{trail}{text[end - 1]}"))
            snaps.append(
                {**record, "action": "snapped", "to": to, "ratio": round(ratio, 3), "reason": None}
            )
        else:
            edits.append((start, end, inner))
            snaps.append({**record, "ratio": round(ratio, 3), "reason": "not in source"})
    for start, end, replacement in reversed(edits):
        text = text[:start] + replacement + text[end:]
    return text, snaps

summarize(synth, summary, utterances, *, model)

Rebuild the rolling summary with one complete call.

Parameters:

Name	Type	Description	Default
synth		The text synthesizer.	required
summary	str	The previous summary, or "".	required
utterances	list[Utterance]	Every utterance so far.	required
model	str | None	Model id override, or None.	required

Returns:

Type	Description
str	The new summary, or the previous one if the model returned nothing.

Source code in src/gutenberg_kg/serve/entmoot.py

def summarize(synth, summary: str, utterances: list[Utterance], *, model: str | None) -> str:
    """Rebuild the rolling summary with one ``complete`` call.

    :param synth: The text synthesizer.
    :param summary: The previous summary, or ``""``.
    :param utterances: Every utterance so far.
    :param model: Model id override, or ``None``.
    :returns: The new summary, or the previous one if the model returned nothing.
    """
    lines = "\n".join(f"{u.title}: {u.text}" for u in utterances)
    prior = f"Earlier summary: {summary}\n\n" if summary else ""
    messages = [
        {
            "role": "system",
            "content": (
                "You are a neutral narrator. Summarize the conversation in at most "
                "80 words. Add no facts that are not in it."
            ),
        },
        {"role": "user", "content": f"{prior}Conversation:\n{lines}"},
    ]
    return synth.complete(messages, model=model, max_tokens=MAX_TOKENS, temperature=0.2) or summary

tree_from_hits(key, hits)

Build a tree from the first hit of an opening retrieval.

Enrichment puts title, author and genre on every hit. With no hits, the title falls back to the book part of the key.

Parameters:

Name	Type	Description	Default
key	str	The book's catalog key.	required
hits	list[dict]	Hits from a query scoped to that key.	required

Returns:

Type	Description
Tree	The tree.

Source code in src/gutenberg_kg/serve/entmoot.py

def tree_from_hits(key: str, hits: list[dict]) -> Tree:
    """Build a tree from the first hit of an opening retrieval.

    Enrichment puts ``title``, ``author`` and ``genre`` on every hit. With no
    hits, the title falls back to the book part of the key.

    :param key: The book's catalog key.
    :param hits: Hits from a query scoped to that key.
    :returns: The tree.
    """
    first = hits[0] if hits else {}
    return Tree(
        key=key,
        title=first.get("title") or _book_name(key),
        author=first.get("author") or "",
        genre=first.get("genre") or key.split("/", 1)[0],
    )

turn(synth, tree, other, topic, passages, summary, recent, incoming, *, number, pool, others, own=None, query, model, temperature, intent='answer')

Speak one turn: complete, check grounding, retry once on failure.

Parameters:

Name	Type	Description	Default
synth		A TextSynthesizer (anything with a compatible complete).	required
tree	Tree	The speaking tree.	required
other	Tree	The tree it is talking to.	required
topic	str | None	The seed topic, or None.	required
passages	list[Passage]	This turn's passages, already retrieved.	required
summary	str	The rolling summary so far.	required
recent	list[Utterance]	The utterances before incoming.	required
incoming	Utterance | None	The other tree's last utterance, or None when opening.	required
number	int	This turn's number in the transcript, from 1.	required
pool	list[Passage]	Every passage of this tree's book retrieved during the moot, including passages.	required
others	dict[int, str]	The other tree's utterances so far, turn number -> text.	required
own	dict[int, str] | None	This tree's own earlier utterances, turn number -> text.	None
query	str	The retrieval query that produced passages (recorded only).	required
intent	str	The move this turn makes, a key of :data:INTENTS.	'answer'
model	str | None	Model id override, or None for the synthesizer's default.	required
temperature	float	Sampling temperature.	required

Returns:

Type	Description
Utterance	The utterance, its quotes snapped by :func:snap_quotes. A reply that fails twice is kept and marked ungrounded; grounding is measured before snapping.

Raises:

Type	Description
MootError	When the model returns no text on both attempts.

Source code in src/gutenberg_kg/serve/entmoot.py

def turn(
    synth,
    tree: Tree,
    other: Tree,
    topic: str | None,
    passages: list[Passage],
    summary: str,
    recent: list[Utterance],
    incoming: Utterance | None,
    *,
    number: int,
    pool: list[Passage],
    others: dict[int, str],
    own: dict[int, str] | None = None,
    query: str,
    model: str | None,
    temperature: float,
    intent: str = "answer",
) -> Utterance:
    """Speak one turn: complete, check grounding, retry once on failure.

    :param synth: A ``TextSynthesizer`` (anything with a compatible ``complete``).
    :param tree: The speaking tree.
    :param other: The tree it is talking to.
    :param topic: The seed topic, or ``None``.
    :param passages: This turn's passages, already retrieved.
    :param summary: The rolling summary so far.
    :param recent: The utterances before ``incoming``.
    :param incoming: The other tree's last utterance, or ``None`` when opening.
    :param number: This turn's number in the transcript, from 1.
    :param pool: Every passage of this tree's book retrieved during the moot,
        including ``passages``.
    :param others: The other tree's utterances so far, turn number -> text.
    :param own: This tree's own earlier utterances, turn number -> text.
    :param query: The retrieval query that produced ``passages`` (recorded only).
    :param intent: The move this turn makes, a key of :data:`INTENTS`.
    :param model: Model id override, or ``None`` for the synthesizer's default.
    :param temperature: Sampling temperature.
    :returns: The utterance, its quotes snapped by :func:`snap_quotes`. A reply
        that fails twice is kept and marked ungrounded; grounding is measured
        before snapping.
    :raises MootError: When the model returns no text on both attempts.
    """
    messages = build_messages(tree, other, topic, passages, summary, recent, incoming, intent)
    reply = synth.complete(messages, model=model, max_tokens=MAX_TOKENS, temperature=temperature)
    attempts = 1
    grounding = check_grounding(reply or "", passages, pool, others, own)
    if not reply or not _passes(grounding):
        attempts = 2
        retry = list(messages)
        if reply:
            retry.append({"role": "assistant", "content": reply})
        retry.append({"role": "user", "content": _reminder(passages)})
        second = synth.complete(retry, model=model, max_tokens=MAX_TOKENS, temperature=temperature)
        if second:
            reply = second
            grounding = check_grounding(reply, passages, pool, others, own)
    if not reply:
        raise MootError(f"the model returned no text for {tree.title}; is the endpoint up?")
    by_tag = {p.tag: p.node_id for p in passages}
    text, snaps = snap_quotes(reply, pool, others, own)
    return Utterance(
        turn=number,
        speaker=tree.key,
        title=tree.title,
        text=text,
        cited={t: by_tag[t] for t in dict.fromkeys(parse_tags(reply)) if t in by_tag},
        quotes=grounding.pop("quotes"),
        grounding=grounding,
        grounded=_passes(grounding),
        attempts=attempts,
        query=query,
        retrieved=[{"tag": p.tag, "label": p.label, "node_id": p.node_id} for p in passages],
        raw_text=reply,
        intent=intent,
        snaps=snaps,
    )

turn_intent(number, turns)

The move for one turn of a moot.

Parameters:

Name	Type	Description	Default
number	int	The turn's number, from 1.	required
turns	int	The moot's length.	required

Returns:

Type	Description
str	A key of :data:INTENTS: "open" for turn 1, "close" for each book's last turn (the last two), the middle moves in rotation between.

Source code in src/gutenberg_kg/serve/entmoot.py

def turn_intent(number: int, turns: int) -> str:
    """The move for one turn of a moot.

    :param number: The turn's number, from 1.
    :param turns: The moot's length.
    :returns: A key of :data:`INTENTS`: ``"open"`` for turn 1, ``"close"`` for
        each book's last turn (the last two), the middle moves in rotation
        between.
    """
    if number == 1:
        return "open"
    if number > turns - 2:
        return "close"
    return _MIDDLE_INTENTS[(number - 2) % len(_MIDDLE_INTENTS)]