Neuropil meshes (connectomekg.neuropil_meshes)

connectomekg.neuropil_meshes

Neuropil surface meshes: fetched once from FlyWire's public bucket, cached per dataset.

FAFB v783 synapses are assigned to neuropils by volumes mapped from the JFRC2 template brain into FlyWire space (Ito et al. 2014, Neuron 81, 755-765). The surfaces of those volumes are published as legacy Neuroglancer precomputed meshes at gs://flywire_neuropil_meshes/neuropils/neuropil_mesh_v141_v6: one manifest and one binary fragment per integer id, with no names.

:data:FAFB_783_MESH_NAMES names them. It was derived by matching every v6 fragment, vertex set for vertex set, against the named PLY copies fafbseg packages from the same source (fafbseg/data/JFRC2NP.surf.fw.zip, whose README records the v6 origin): 78 of 78 exact, 78 distinct names. The one v783 neuropil without a mesh is UNASGD, the synapses no volume claimed. Do not substitute fafbseg/data/volume_name_dict.json: it numbers the synapse-assignment volume, not these meshes, and matches none of them.

Vertices are nanometres in FlyWire (FAFB14.1) space, the same frame as the graph's neuron coordinates, so a mesh goes through :meth:connectomekg.scene.WorldFrame.to_world unchanged.

decode_fragment(data)

Decode one legacy Neuroglancer precomputed mesh fragment.

The format is a little-endian uint32 vertex count, that many float32 x y z triples, then uint32 vertex-index triangles to the end.

Parameters:

Name	Type	Description	Default
data	bytes	The fragment's bytes.	required

Returns:

Type	Description
Mesh	(vertices, faces).

Raises:

Type	Description
ValueError	If the bytes are not a well-formed fragment.

Source code in src/connectomekg/neuropil_meshes.py

def decode_fragment(data: bytes) -> Mesh:
    """Decode one legacy Neuroglancer precomputed mesh fragment.

    The format is a little-endian ``uint32`` vertex count, that many ``float32``
    ``x y z`` triples, then ``uint32`` vertex-index triangles to the end.

    :param data: The fragment's bytes.
    :return: ``(vertices, faces)``.
    :raises ValueError: If the bytes are not a well-formed fragment.
    """
    if len(data) < 4:
        raise ValueError("mesh fragment is shorter than its header")
    (n,) = struct.unpack_from("<I", data)
    rest = len(data) - 4 - 12 * n
    if rest < 0 or rest % 12:
        raise ValueError(f"mesh fragment of {len(data)} bytes does not hold {n} vertices")
    vertices = np.frombuffer(data, "<f4", 3 * n, 4).reshape(n, 3)
    faces = np.frombuffer(data, "<u4", offset=4 + 12 * n).reshape(-1, 3)
    if faces.size and int(faces.max()) >= n:
        raise ValueError("mesh fragment has a face index beyond its vertices")
    return vertices, faces

fetch_neuropil_meshes(dataset_id, dest, *, fetch=_http_get, progress=None)

Download a dataset's neuropil meshes and cache them as one .npz.

Parameters:

Name	Type	Description	Default
dataset_id	str	Dataset id, e.g. "fafb783".	required
dest	str | Path	The cache file to write, usually <dataset dir>/.connectomekg/neuropil_meshes.npz.	required
fetch	Callable[[str], bytes]	Returns the bytes at a URL; the default is a plain HTTPS GET.	_http_get
progress	Callable[[str], None] | None	Called with a short message every 20 meshes.	None

Returns:

Type	Description
Path	dest.

Raises:

Type	Description
ValueError	If the dataset has no mesh source or a fragment is malformed.

Source code in src/connectomekg/neuropil_meshes.py

def fetch_neuropil_meshes(
    dataset_id: str,
    dest: str | Path,
    *,
    fetch: Callable[[str], bytes] = _http_get,
    progress: Callable[[str], None] | None = None,
) -> Path:
    """Download a dataset's neuropil meshes and cache them as one ``.npz``.

    :param dataset_id: Dataset id, e.g. ``"fafb783"``.
    :param dest: The cache file to write, usually
        ``<dataset dir>/.connectomekg/neuropil_meshes.npz``.
    :param fetch: Returns the bytes at a URL; the default is a plain HTTPS GET.
    :param progress: Called with a short message every 20 meshes.
    :return: ``dest``.
    :raises ValueError: If the dataset has no mesh source or a fragment is malformed.
    """
    if dataset_id not in _SOURCES:
        known = ", ".join(sorted(_SOURCES))
        raise ValueError(f"no neuropil meshes published for dataset {dataset_id!r}; have {known}")
    base, names = _SOURCES[dataset_id]
    arrays: dict[str, np.ndarray] = {}
    for i, (mesh_id, name) in enumerate(sorted(names.items()), 1):
        vertices, faces = decode_fragment(fetch(f"{base}/{mesh_id}:0:0"))
        arrays[f"{name}/vertices"] = vertices
        arrays[f"{name}/faces"] = faces
        if progress is not None and (i % 20 == 0 or i == len(names)):
            progress(f"fetched {i} of {len(names)} neuropil meshes")
    dest = Path(dest)
    dest.parent.mkdir(parents=True, exist_ok=True)
    # Written whole, then renamed, so an interrupted fetch leaves no partial cache.
    buffer = io.BytesIO()
    np.savez_compressed(buffer, **arrays)  # ty: ignore[invalid-argument-type]
    partial = dest.with_suffix(".part")
    partial.write_bytes(buffer.getvalue())
    partial.replace(dest)
    return dest

has_mesh_source(dataset_id)

Whether a dataset has published neuropil meshes to fetch.

Parameters:

Name	Type	Description	Default
dataset_id	str	Dataset id, e.g. "fafb783".	required

Returns:

Type	Description
bool	True if :func:fetch_neuropil_meshes accepts it.

Source code in src/connectomekg/neuropil_meshes.py

def has_mesh_source(dataset_id: str) -> bool:
    """Whether a dataset has published neuropil meshes to fetch.

    :param dataset_id: Dataset id, e.g. ``"fafb783"``.
    :return: ``True`` if :func:`fetch_neuropil_meshes` accepts it.
    """
    return dataset_id in _SOURCES

load_neuropil_meshes(path)

Read a cache written by :func:fetch_neuropil_meshes.

Parameters:

Name	Type	Description	Default
path	str | Path	The .npz cache.	required

Returns:

Type	Description
dict[str, Mesh]	{neuropil name: (vertices_nm, faces)}.

Source code in src/connectomekg/neuropil_meshes.py

def load_neuropil_meshes(path: str | Path) -> dict[str, Mesh]:
    """Read a cache written by :func:`fetch_neuropil_meshes`.

    :param path: The ``.npz`` cache.
    :return: ``{neuropil name: (vertices_nm, faces)}``.
    """
    with np.load(path) as npz:
        names = sorted({key.rsplit("/", 1)[0] for key in npz.files})
        return {n: (npz[f"{n}/vertices"], npz[f"{n}/faces"]) for n in names}

neuropil_mesh_path(db_path)

Where a dataset's neuropil mesh cache lives: beside its graph.

Parameters:

Name	Type	Description	Default
db_path	str | Path	The dataset's graph.sqlite.	required

Returns:

Type	Description
Path	<dataset dir>/.connectomekg/neuropil_meshes.npz, which may not exist.

Source code in src/connectomekg/neuropil_meshes.py

def neuropil_mesh_path(db_path: str | Path) -> Path:
    """Where a dataset's neuropil mesh cache lives: beside its graph.

    :param db_path: The dataset's ``graph.sqlite``.
    :return: ``<dataset dir>/.connectomekg/neuropil_meshes.npz``, which may not exist.
    """
    return Path(db_path).parent / MESH_CACHE