Diary parser

gutenberg_kg.diary.parser

parser.py — Parse Project Gutenberg diary markdown into dated PSV entries.

Supports multiple diary date formats via pluggable parser subclasses:

• pepys — ALL-CAPS MONTH YEAR section headers + Month Nth. entries (Pepys, any standard Gutenberg diary)
• evelyn — Day-first inline year: 29th December, 1659.
• boswell — Weekday prefix, no inline year: Sunday, 15th August (year anchored at construction time, default 1773)

Use :func:get_parser to select a parser by format name, or instantiate directly. The module-level :func:parse and :func:write_psv use the Pepys format for backward compatibility.

Output: pipe-delimited lines compatible with DiaryTransformer::

TIMESTAMP | diary | prose | CONTENT

BaseDiaryParser

Bases: ABC

State-machine base for Gutenberg diary parsing.

Subclasses implement :meth:_match_section and :meth:_match_date. Set requires_section_header = True (Pepys) to skip date matching before the first section header fires; leave False (Evelyn, Boswell) to start accumulating from the first matched date line.

parse(md_path)

Yield dated entries from a Gutenberg diary markdown file.

:yields: :class:ParsedEntry in chronological order.

Parameters:

Name	Type	Description	Default
md_path	Path	Path to the downloaded markdown.	required

Source code in src/gutenberg_kg/diary/parser.py

def parse(self, md_path: Path) -> Iterator[ParsedEntry]:
    """Yield dated entries from a Gutenberg diary markdown file.

    :param md_path: Path to the downloaded markdown.
    :yields: :class:`ParsedEntry` in chronological order.
    """
    lines = md_path.read_text(encoding="utf-8").splitlines()

    in_diary = False
    current_year = 0
    current_month = 0
    current_day = 0
    current_lines: list[str] = []
    current_start = 0

    def _flush() -> ParsedEntry | None:
        if not current_lines or current_day == 0:
            return None
        content = _clean("\n".join(current_lines))
        if len(content) < 10:
            return None
        try:
            ts = datetime(current_year, current_month, current_day)
        except ValueError:
            return None
        return ParsedEntry(timestamp=ts, content=content, source_line=current_start)

    bracket_depth = 0

    for lineno, raw_line in enumerate(lines, start=1):
        line = raw_line.strip()

        if not line:
            bracket_depth = 0
        depth_at_start = bracket_depth
        bracket_depth = max(0, bracket_depth + line.count("[") - line.count("]"))

        # Back matter — stop before the editor's afterword, transcriber's
        # notes and appendices are accumulated into the final entry.
        if depth_at_start == 0 and in_diary and _END_OF_DIARY_RE.match(line):
            break

        # Section header (e.g. JANUARY 1660)
        if depth_at_start == 0:
            section = self._match_section(line)
            if section is not None:
                month, year = section
                if in_diary and current_day:
                    entry = _flush()
                    if entry:
                        yield entry
                    current_lines = []
                    current_day = 0
                in_diary = True
                current_month = month
                current_year = year
                continue

        # For Pepys-style parsers: skip preamble before first section header
        if not in_diary and self.requires_section_header:
            continue

        # Date matching
        if depth_at_start == 0:
            result = self._match_date(line, current_month, current_year)
            if result is not None:
                day, month, year, rest = result
                entry = _flush()
                if entry:
                    yield entry
                in_diary = True
                current_day = day
                current_month = month
                current_year = year
                current_lines = [rest] if rest.strip() else []
                current_start = lineno
                continue

        if in_diary and current_day > 0:
            current_lines.append(raw_line)

    if in_diary and current_day:
        entry = _flush()
        if entry:
            yield entry

BoswellParser(anchor_year=1773)

Bases: BaseDiaryParser

Boswell format: Sunday, 15th August — weekday prefix, no inline year.

The tour (1773) spans a single year, supplied via anchor_year.

Source code in src/gutenberg_kg/diary/parser.py

def __init__(self, anchor_year: int = 1773) -> None:
    self.anchor_year = anchor_year

EvelynParser

Bases: BaseDiaryParser

Evelyn format: day-first with inline year — 29th December, 1659.

Each entry carries its own year; no section headers. Handles the older ordinal d suffix used by Evelyn (3d, 22d).

ParsedEntry(timestamp, content, source_line) dataclass

One dated diary entry extracted from a Gutenberg diary markdown.

PepysParser

Bases: BaseDiaryParser

Pepys format: ALL-CAPS MONTH YEAR headers + Month Nth. entries.

get_parser(fmt)

Return the parser for fmt (default: pepys if unknown).

Parameters:

Name	Type	Description	Default
fmt	str	Format name — one of "pepys", "evelyn", "boswell".	required

Returns:

Type	Description
BaseDiaryParser	Shared :class:BaseDiaryParser instance.

Source code in src/gutenberg_kg/diary/parser.py

def get_parser(fmt: str) -> BaseDiaryParser:
    """Return the parser for *fmt* (default: ``pepys`` if unknown).

    :param fmt: Format name — one of ``"pepys"``, ``"evelyn"``, ``"boswell"``.
    :return: Shared :class:`BaseDiaryParser` instance.
    """
    return _PARSERS.get(fmt.lower().strip(), _PARSERS["pepys"])

parse(md_path)

Parse using the Pepys format (default). See :func:get_parser for others.

Source code in src/gutenberg_kg/diary/parser.py

def parse(md_path: Path) -> Iterator[ParsedEntry]:
    """Parse using the Pepys format (default).  See :func:`get_parser` for others."""
    return PepysParser().parse(md_path)

write_psv(entries, out_path)

Write parsed entries as a pipe-delimited file for DiaryTransformer.

Format per line::

TIMESTAMP | diary | prose | CONTENT

Pipe characters within content are replaced with em-dashes.

Parameters:

Name	Type	Description	Default
entries	Iterator[ParsedEntry]	Iterator of :class:ParsedEntry objects.	required
out_path	Path	Destination .psv file path.	required

Returns:

Type	Description
int	Number of lines written.

Source code in src/gutenberg_kg/diary/parser.py

def write_psv(entries: Iterator[ParsedEntry], out_path: Path) -> int:
    """Write parsed entries as a pipe-delimited file for DiaryTransformer.

    Format per line::

        TIMESTAMP | diary | prose | CONTENT

    Pipe characters within content are replaced with em-dashes.

    :param entries: Iterator of :class:`ParsedEntry` objects.
    :param out_path: Destination ``.psv`` file path.
    :return: Number of lines written.
    """
    count = 0
    with out_path.open("w", encoding="utf-8") as fh:
        for entry in entries:
            ts = entry.timestamp.strftime("%Y-%m-%dT%H:%M:%S")
            content = entry.content.replace("|", "—")
            fh.write(f"{ts} | diary | prose | {content}\n")
            count += 1
    return count