All talks
Talk · MediaPipe · Vector DBs

Building Private Brain on Android: Offline RAG with Vector Databases & MediaPipe

Privacy-conscious users and business needs are pushing toward local-first AI. Running a large language model on the device is a good start; the real challenge is making that model smart with your own private data, without the cloud. This session sets up offline Retrieval-Augmented Generation (RAG) on Android.

It shows how to run local vector search with high-performance databases such as ObjectBox or Couchbase Lite to store and query embeddings, and how to build a pipeline that feeds real-time local context into open on-device models such as Gemma through the MediaPipe LLM Inference API. It also answers an architectural question: when should you invest in LoRA fine-tuning instead of the flexibility of RAG?

MediaPipeVector DBsSpeaker: Dinoy Raj

Key takeaways

Architecting local RAG

The building blocks of a retrieval pipeline that never leaves the phone.

Vector database implementation

Storing and querying embeddings with ObjectBox or Couchbase Lite.

Real-time context injection

Feeding local data into Gemma through the MediaPipe LLM Inference API.

LoRA vs RAG

When fine-tuning pays off and when retrieval is the better choice.

Performance optimisation

Keeping on-device retrieval and generation fast.

Presented at

View all events