PocketGPT

An ChatGPT-like iOS app that runs Multimodal LLM, Stable Diffusion and Whisper fully on mobile device. App Store Link.

Credits

This app integrates several open-source projects/models. Great thanks to these awesome projects!

Code:

• Inference engine: llama.cpp(forked) for the language and audio models, and CoreML for the Diffusion model.
• App interface: builds on top of LLMFarm.

Models:

• MLLM: MobileVLM, quantized using llama.cpp to 4 bits.
• Diffusion: SD Turbo, quantized/palettized using CoreML tools to 8 bits.
• Audio: Whisper Base.