MineContext

MineContext is an open-source proactive AI assistant designed for personal knowledge management and context engineering. It serves as a tool for users who want an AI that understands their digital activity without manual data entry, acting as an alternative to traditional manual note-taking apps or static AI chatbots that require constant prompting. By observing the user's digital world, it aims to bring clarity to fragmented workflows and study sessions.

The application is available for macOS and Windows. It operates by monitoring the user's screen to capture and comprehend visual context, which it then processes to provide automated insights and organization. Users configure the tool by providing an API key for a compatible model service to enable the backend processing of captured data. Once the screen sharing permissions are granted and recording starts, the system works in the background to synthesize information while the user focuses on other tasks.

Key features

• Automated collection of digital context via screen monitoring and screenshots
• Proactive delivery of daily and weekly summaries to the homepage
• Generation of automated to-do lists and activity records from captured data
• Intelligent resurfacing of relevant context during the creative process
• Support for local-first data storage to maintain user privacy
• Compatibility with OpenAI API protocol for use with local or third-party models
• Integration with Vision Language Models and embedding models for multimodal data
• Backend debugging interface for adjusting system prompts and task intervals

The software is built using Electron, React, and TypeScript for the frontend, with a Python-based backend. It utilizes a retrieval-augmented generation (RAG) approach and vector databases to manage the lifecycle of multimodal data, including capture, storage, and retrieval. The architecture separates the main process, preload scripts, and renderer process to handle system-level screen sharing permissions and IPC communication securely. Users can further enhance privacy by using tools like LMStudio to run fully local models, ensuring that no data leaves the local environment.

MineContext positions itself as a background utility that transforms raw screen activity into structured, actionable intelligence for students and creators.