NightMean/OlliteRT

OlliteRT turns your Android smartphone into a local inference server compatible with the OpenAI API. It works completely offline, ensuring privacy and open-source code in Kotlin.

// KEY FEATURES

• Runs LLM inference directly on the device without needing the cloud.
• Provides an HTTP endpoint fully compatible with the OpenAI API (chat/completions).
• Supports Gemma and Anthropic models via LiteRT (TensorFlow Lite) in Kotlin.
• Easy integration with Home Assistant for voice control of a smart home.

New repositories every 30 minutes

REDDYX AI scans GitHub 24/7 and ships the best AI/ML/Web3 projects to Telegram.