Back to home

How Openbird works

How accessibility context is collected, processed, and turned into the current daily summary.

Openbird's current summary is built from a local accessibility timeline, not screenshots or a remote backend. The pipeline is straightforward: collect the frontmost context, clean and group it, then generate a recap from those prepared activity chunks.

This is intentionally a narrow proof of concept. The point is to validate accessibility-first personal context before this capability is integrated natively into fastrepl/char.

Short version

1. Capture the frontmost app, window, URL when available, visible accessibility text, and timestamps.
2. Sanitize noisy UI text, merge repeated context, and group related events.
3. Turn those grouped events into evidence sections for the day.
4. Generate markdown with your configured model, or fall back to a built-in heuristic recap.

1. Data collection

• Openbird runs locally on macOS after you grant Accessibility permission.
• This repo is an experimental proof of concept focused on the accessibility pipeline that will later inform native personal-context features in fastrepl/char.
• It watches the frontmost app and focused window, capturing app name, bundle ID, window title, browser URL when available, visible accessibility text, and timestamps.
• Capture runs when the active app changes and on a short recurring interval while the collector is running.
• Pause rules and app or domain exclusions are checked before data is saved.
• Openbird does not capture raw key events, clipboard contents, passwords, secure text fields, hidden windows, minimized windows, or automatic screenshots.

All captured activity is stored locally at ~/Library/Application Support/Openbird/openbird.sqlite.

2. Local processing

• Before persistence, snapshots are sanitized to remove duplicate lines, low-signal browser or chat chrome, and generic titles when a better local label is available.
• If the visible context has not meaningfully changed, Openbird extends the current activity event instead of creating a duplicate entry.
• Raw events are grouped into more meaningful activity chunks by app, time proximity, titles, URLs, and cleaned text excerpts.
• Prepared activity chunks are cached locally per day so summary generation does not need to rebuild the evidence from scratch every time.

This processing step is what makes the summary readable. It reduces tab chrome, repeated labels, and noisy app switches before any generation happens.

3. Summary generation

• When you generate a daily review, Openbird loads that day's grouped events, drops excluded items, and compacts them into a smaller set of evidence sections.
• Each evidence section keeps a time window plus the strongest cues from the underlying activity: app names, titles, URL summaries, and cleaned excerpts.
• If you configured a provider, Openbird sends those prepared evidence sections to the model with instructions to write a concise markdown recap of the day.
• If no provider is configured, or the provider call fails, Openbird falls back to a local heuristic summary so the review still renders.
• The resulting journal is then saved back to the local database with links to the underlying source events.

The generated summary is meant to read like a grounded recap of your day, not a raw timeline dump. Openbird explicitly asks the model to synthesize the evidence into a few human sections and keep the output in markdown.

What leaves your Mac

By default, capture and storage stay local. If you use a local provider such as Ollama or LM Studio, generation stays local too.

If you configure a remote OpenAI-compatible endpoint, Openbird sends the prepared evidence for the selected day to that endpoint at generation time. It does not upload automatic screenshots because it does not take them in the first place.

The control surface stays the same either way: you can pause capture, exclude apps or domains, inspect the raw log, and delete the last hour, the last day, or everything.