Kuzco

Kuzco is a Swift package for integrating large language models (LLMs) directly into iOS and macOS apps. Built on top of llama.cpp, Kuzco enables you to run AI models entirely on-device, ensuring privacy and speed.

Key Features:

• Local LLM Execution: Runs models on-device using llama.cpp.
• Multiple Model Architectures: Supports LLaMA, Mistral, Phi, Gemma, Qwen, and more.
• Async/Await Native: Modern Swift concurrency with streaming responses.
• Cross-Platform: Works on iOS, macOS, and Mac Catalyst.
• Flexible Model Settings: Fine-tune context length, batch size, GPU layers, and CPU threads.
• Customizable Sampling: Control temperature, top-K, top-P, repetition penalties.
• Automatic Architecture Detection: Detects model architectures from filenames.

Use Cases:

• Building offline AI-powered features in iOS and macOS apps.
• Creating privacy-focused chat applications.
• Implementing on-device AI for resource-constrained devices.
• Experimenting with different LLM architectures and configurations.