Fullmoon

Fullmoon is an open-source web app and server that allows users to interact with large language models locally. It is deployed as an iOS application compatible with iPhone, iPad, and Mac devices, providing a dedicated interface for local AI inference on mobile and desktop Apple hardware.

The software enables offline AI interactions by running models directly on the device hardware. This approach removes the need for an internet connection during chat sessions and ensures that data does not leave the device. It focuses on privacy by saving all chat history locally in a database on the device rather than on a remote server managed by a third party.

Key features

• Local execution of large language models
• Offline functionality for chat interactions
• Support for Llama 3.2 1B
• Support for Llama 3.2 3B
• Support for DeepSeek-R1-Distill-Qwen-1.5B-4bit
• Local storage of chat history
• Customizable application appearance

The application is built using MLX Swift, which is an array framework developed by Apple for machine learning research. This architecture ensures the software is optimized specifically for Apple silicon, allowing the models to utilize the integrated GPU and Neural Engine for inference. By leveraging the MLX framework, the app can handle the computational requirements of the supported models while maintaining efficiency on portable devices. This makes the tool suitable for developers and privacy advocates who require a local environment for testing or interacting with small language models.

Fullmoon serves as a private, local interface for users who want to run specific small language models on Apple hardware without relying on cloud providers or external API subscriptions.