data-ai MCP Server
Open-source macOS desktop AI agent
Discovered via agent-topic:ai-agent and last synced 3mo ago.
Install instructions not detected yet
Check the source repository for the latest setup steps.
[Electron](https://www.electronjs.org/) (Chromium + Node main process)
"translate this", "what does that say"
Limited
Erase the last turn from memory ("that wasn't me")
[Rive](https://rive.app)
GPT-4o Vision
Switch background / observe / ask / takeover
PKCE + built-in client IDs (Google, GitHub, Spotify)
GPT-5.5 visual desktop control — sees screen, operates any app via CGEvent
Switch normal / passive listening
Example
--- ## FAQ **What is Samuel?** An open-source voice AI desktop agent that continuously sees your screen and hears your audio, lets you control it by voice, browses the web like a human, and writes and repairs its own tools at runtime using GPT-5.5 with reasoning. **What can I use it for?** Language learning while watching content, hands-free web browsing ("show me my emails"), building custom AI tools by voice, live meeting interpretation, searching and summarizing anything on the web, ambient monitoring ("tell me when you hear X"), and general desktop automation. **How is this different from ChatGPT Voice?** ChatGPT can't see your screen continuously, can't browse the web as a real browser, can't write persistent tools, and can't auto-repair when things break. Samuel does all of these, and runs locally on your Mac. **What models does it use?** OpenAI Realtime API for voice, GPT-5.5 with reasoning for code generation and failure diagnosis, GPT-4o Vision for screen capture, GPT-4o-mini for code review, Whisper for transcription. **How does auto-repair work?** Every plugin declares a `validates()` function. If output fails validation, or the user says "that's wrong," GPT-5.5 diagnoses the failure (syntax bug? API change? structural issue?), picks a repair strategy (patch, rewrite, or ask user), generates a fix, and verifies it before deploying. Max 2 attempts, then clean escalation. **Can Samuel modify its existing tools?** Yes, via the wraps/middleware pattern. A plugin can wrap any existing tool — intercepting calls, modifying inputs/outputs, adding caching or logging — without replacing the original. **Does Samuel browse the web?** Three ways: (1) API-based search via SerpAPI, (2) AI-powered deep search via OpenAI, (3) real browser automation via Playwright where he opens a visible Chromium window, you sign in, and he reads/interacts with the page. **Is my data private?** Screen captures and audio are sent to OpenAI for processing. Memory, preferences, skills, plugins, and secrets are stored locally in `~/.samuel/`. Browser sessions run locally via Playwright. **Is it free?** The code is MIT-licensed. You pay OpenAI API costs directly. **Does it work on Windows or Linux?** Currently macOS only. Cross-platform is on the roadmap. --- ## Contributing Samuel is growing fast. Every contribution — code, skills, ideas, bug reports — shapes where this goes. ### What we need help with - **Windows + Linux ports** — ScreenCaptureKit alternatives (WASAPI, PipeWire/PulseAudio) - **One-click installer** — signed `.dmg` packaging via `electron-builder` - **Persistent browser sessions** — save Playwright cookies/profiles across launches - **Plugin sandboxing** — Web Worker isolation for plugin execution - **MCP integration** — `@openai/agents` + MCP servers for Notion, Slack, etc. - **Skill contributions** — write workflows you'd actually use - **Tool description tuning** — better descriptions = more reliable tool selection - **Documentation** — install walkthroughs and "what tripped me up" reports ### How to help, by time available
Activate ambient language tutoring
Technology
Approx. cost
Purpose
Search the internet (3 tiers) and read web pages
[OpenAI Realtime API](https://platform.openai.com/docs/guides/realtime) (WebRTC, SCTP-bounded payloads)
Cluely
What happens
What it does
Start/stop system audio capture
Secure API key storage
Persistent user facts and preferences
React 19 + Vite + TypeScript
**Samuel**
Plugin code review, trigger classification, screen text extraction
Self-modification — propose, write, **repair**, remove, list plugins
Vocabulary cards (manual/auto mode)
Voice-controlled UI changes
Speak correct pronunciation
[LRCLIB](https://lrclib.net) + [Genius](https://genius.com) + web search fallback
Example
When Samuel uses it
Recording transcription (high-fidelity)
Song segmentation with timestamps
Captures and analyzes what's on screen
Opens real browser, navigates, reads, clicks, types, screenshots
Save, search, and replay multi-step workflows
Play, pause, lyrics, refetch, correct
Register ambient triggers — keyword or classifier-based
Adjust Samuel's voice or macOS system volume
Zero-config OAuth for Google/GitHub/Spotify
Read, write, list files on disk
Behavioral feedback storage
Self-awareness — recall recent tool calls
Cluely
✅
Reads any macOS app via Accessibility Tree (Chrome, WeChat, Slack, Notes...)
Pull-model transcript-on-demand: "translate the last 30s", "what did they say?"
Float a styled HTML panel over the desktop ("show me a panel with…")
Active
Toggle the rolling system-audio buffer (consent popup the first time it flips on)
What you can do