All talks
Talk · MediaPipe · LoRA

Build Your Pocket Brain: Open On-Device LLMs via MediaPipe on Android

MediaPipe's LLM Inference API leads the way for Android developers. Beyond running pre-trained models such as Gemma on the device, this session dives into customisation with Low-Rank Adaptation (LoRA), a parameter-efficient fine-tuning (PEFT) technique.

It walks the end-to-end workflow for a specialised on-device LLM: fine-tuning a base model like Gemma or Phi-2 on your own dataset with the PEFT library in Python, converting the base model and LoRA weights to MediaPipe's FlatBuffer format, and integrating the result in an Android app by configuring LlmInferenceOptions in Kotlin. The payoff is personalised AI that is fast, works offline and stays private.

MediaPipeLoRASpeaker: Dinoy Raj

Key takeaways

Choosing the right tool

When to use MediaPipe LLM Inference and when Gemini Nano for on-device generative AI.

Setup and configuration

Configuring MediaPipe LLM Inference, its options and multimodal prompting.

LoRA fine-tuning

Adapting Gemma-2B or Phi-2 to a specific use case, cost-effectively.

Deployment and ethics

Deployment workflows, GPU-accelerated LoRA inference and responsible AI practice.

Presented at

View all events