nebulous

NewsBlur MCP server and cutting-garden plugin — a local-first feed/story index that serves NewsBlur data to Claude over JSON-RPC stdio, and exposes that same index as a structured newsblur:// tree to the cutting-garden capture/ traversal framework.

One Go module, two surfaces over one local store:

1. NewsBlur MCP server (nebulous serve mcp) — serves feeds, stories, subscriptions, folders, and OPML import/export from a local persistent index rather than the live API.
2. cutting-garden newsblur plugin (nebulous-cg) — exposes the same local index as a structured tree under the newsblur:// URI scheme (feeds / stories / tags, with a per-story content + original leaf), implementing the cutting-garden plugin SDK's RootProvider / RootLister / LeafReader. Read-only by default; no token required. Optionally becomes read-write when NEWSBLUR_TOKEN is set, implementing NodeMutator (create_node/patch_node/delete_node — star/unstar, mark_read/mark_unread, unsubscribe, rename_feed/move_feed).

Built on go-mcp from amarbel-llc/purse-first. Operates in two distinct modes; sync and capture are now both phases of the same fetch command rather than separate processes:

• Sync (+ Capture) — nebulous fetch is the sole ingestion pipeline. It sequentially fetches feed metadata, starred stories, and original article text from the NewsBlur API, with adaptive backoff that learns from rate-limit bursts, and persists responses into the local store: a SHA256-keyed manifest ($XDG_DATA_HOME/nebulous/manifest.json) whose response bodies live in a dedicated nebulous madder blob store. It also runs a capture phase as a fourth step: a self-healing gap-filling scan that, for each configured format (--formats, default markdown-reader), drives cutting-garden capture <store-id> web:<permalink> (which drives chrest under the hood) for any starred story that doesn't yet have a recorded receipt for that format, writing a full-page archive independent of NewsBlur's own (often truncated) story_content. New stories only by default, via a persisted watermark. A failed capture simply has no receipt, so the next scan retries it automatically — no separate retry bookkeeping. Gated by its own interval (NEBULOUS_CAPTURE_INTERVAL, default 6h) rather than fetch's own sync cadence, since the corpus scan behind the capture loop is real cost tied to the manifest's mtime; soft-skips if cutting-garden isn't on PATH or --no-capture was passed. nebulous capture also remains a standalone subcommand for manual invocations and --backfill (captures the full existing corpus for one run, ignoring the interval gate, while still skipping pairs already captured).
• Serve — nebulous serve mcp reads exclusively from that local store; it never hits the API for reads. In-memory feed and story indices (word-search accelerated) rebuild themselves whenever the manifest file's mtime changes since the last check, so a concurrently-running nebulous fetch's new data becomes visible without restarting the server. Mutation tools (star/unstar, mark read/unread, subscribe/unsubscribe, folders) are the exception and call the NewsBlur API directly; star/unstar and mark read/unread also patch their local cache entry in place afterward so a read immediately reflects the change, instead of lagging until the next nebulous fetch.

Query surface: feed_query and story_query tools (structured filters by year/tag/feed/status plus word search), a facets resource (nebulous://stories/facets), and per-story resources (nebulous://story/{hash}, …/content, …/original).

nebulous-cg is a cutting-garden CLI with the newsblur:// scheme plugin baked in (internal/charlie/cgplugin, over the tools.ReadIndex façade). It serves the same local index as a content-addressable traversal tree:

newsblur://feeds                  the subscription list (→ feeds)
newsblur://stories                the starred-story corpus (→ stories)
newsblur://tags                   the tag dictionary (→ tags)
newsblur://feed/{id}              one feed's stories, plus its metadata leaf
newsblur://feed/{id}/metadata     cached feed subscription record (JSON)
newsblur://tag/{tag}              stories carrying a tag
newsblur://story/{hash}                    a story → its leaves below
newsblur://story/{hash}/content            cached story_content (HTML, stripped)
newsblur://story/{hash}/original           cached original article text (HTML)
newsblur://story/{hash}/metadata           bibliographic fields (authors, tags, date, …)
newsblur://story/{hash}/capture/{format}   a completed capture's receipt id + timestamp

The capture/{format} leaf only appears once nebulous capture has recorded a receipt for that (story, format) pair — it is not a placeholder for uncaptured formats. It exposes the receipt's markl-id and when it was captured; it does not walk the RFC 0002 receipt merkle tree itself — a madder-aware consumer resolves it further via madder://blobs/<digest>.

Discover and traverse it via the cutting-garden commands, e.g. nebulous-cg health or nebulous-cg list newsblur://feeds. Reads only the local cache — no NewsBlur token needed.

Feed and story nodes also carry facet dimensions (year, user_tag, story_tag, feed, read on stories; folder, active on feeds) via the cutting-garden FacetDescriber/FacetCounter/FacetLabeler capabilities — see internal/charlie/cgplugin/facets.go.

The flake exports producer modules (circus-host-integration(7)'s self-passing convention, mirroring cutting-garden's own module shape):

• nixosModules.default (services.nebulous) — installs nebulous + nebulous-cg, and runs nebulous fetch on a single periodic systemd timer (fetchInterval, default 1h) so a host's local cache stays warm without a manual invocation. environmentFile is the secret seam for NEWSBLUR_TOKEN (a path to VAR=value lines — the value never enters the Nix store). This module does not define an MCP-serving unit: nebulous serve mcp is a moxy stdio child on hosts that expose it (Path 1, per circus-host-integration(7)), wired host-side by the consumer, not by this module. Supplying chrestPackage and cuttingGardenPackage (both nullable types.package, unset by default) additionally enables the gap-filling capture scan described above as a fourth phase of the same nebulous fetch run — no second timer. It's gated by its own interval (captureInterval, default 6h, threaded in as NEBULOUS_CAPTURE_INTERVAL and checked against a persisted last-scan timestamp, since a no-op pass is cheap but the corpus scan behind it isn't) with captureFormats (default [ "markdown-reader" ]) and captureStoreId (default "nebulous") passed through as -formats/-store flags. Both packages are external to nebulous's own flake, so the consumer (circus) supplies them as option values from its own chrest/cutting-garden flake inputs — a host that only wants NewsBlur sync (no captures) leaves them unset and fetch runs sync-only.
• homeManagerModules.default (programs.nebulous) — installs the binaries for interactive/workstation use. No systemd unit; the periodic timer is a NixOS-host concern.

nebulous serve mcp                    Start the MCP server over stdio
nebulous fetch [--formats f1,f2] [--store id] [--no-capture]
                                      Sync feeds/starred stories/original text; also runs
                                      the gap-filling capture scan when its interval elapses
nebulous capture [--formats f1,f2] [--store id] [--backfill]
                                      Run the capture scan standalone, ignoring the interval gate
nebulous corpus-list / corpus-read    Starred-story corpus access (for maneater)
nebulous generate-plugin | hook | install-mcp
                                      Plugin/install plumbing (no token needed)
nebulous-cg <command>                 Drive the newsblur:// plugin (health, list, …)

• NEWSBLUR_TOKEN (NewsBlur session cookie) is required for serve mcp and fetch. Store it in .secrets.env (gitignored, loaded by direnv). The corpus, plugin, and nebulous-cg subcommands read only the local store and need no token. nebulous-cg optionally becomes read-write (mutations via create_node/patch_node/delete_node) when NEWSBLUR_TOKEN is set; without it, nebulous-cg stays read-only exactly as before.

The build entrypoint is the justfile:

just build          # build-go + build-nix
just build-go       # debug build → build/debug/{nebulous,migrate-cache,nebulous-cg}
just build-nix      # reproducible Nix build (buildGoApplication + gomod2nix)
just test           # go tests + bats lanes (zz-tests_bats/)
just install-dev    # nix build + install MCP server config
just cg ...         # run nebulous-cg against the local cache (e.g. just cg list newsblur://feeds)

After changing Go dependencies: go mod tidy && gomod2nix (the devShell's go-sync-wrap hook regenerates gomod2nix.toml automatically).