Open Source Alternatives
  • Alternatives
  • Categories
    Categories by size1,302 projects
    • Developer Tools673
    • AI & Machine Learning549
    • Business & Productivity311
    • Web Development219
    • Security & Monitoring198
    • Communication & Collaboration168
    • IT Management165
    • Design & Creative145
    • LLMOps & AI Tooling142
    • DevOps & CI/CD141
    Browse all 21 categoriesCloud & Hosting, Product & Project Management and 9 more
  • Blog
  • Advertise
SearchNotion, Slack, password manager.../
Submit
Open Source Alternatives
  • Alternatives
    • Developer Tools673
    • AI & Machine Learning549
    • Business & Productivity311
    • Web Development219
    • Security & Monitoring198
    • Communication & Collaboration168
    • Browse all 21 categories
  • Blog
  • Advertise
Submit your project

New open source projects by email

The projects we add and every new guide. Free, unsubscribe any time.

Open Source Alternatives LogoOpen Source Alternatives

Handpicked Open Source Alternatives to Paid Softwares

Add Open Source Alternatives as a preferred source on Google

Product

  • Categories
  • Tag
  • Advertise

Resources

  • Blog
  • Collection
  • Submit

Company

  • Privacy Policy
  • Terms of Service
  • Refund Policy
  • Sitemap

Alternatives

  • Claude Code
  • Jira
  • Notion
  • Slack
  • Linear
  • Wispr Flow
  • All alternatives
© 2026 Open Source Alternatives. All rights reserved.
Open SourceAlternatives
Home/Categories/AI & Machine Learning/Smolmaxxing
icon of Smolmaxxing

Smolmaxxing

  • 969 stars
  • MIT
  • Active this week
  • Last commit 4 days ago

Last verified Nov 18, 2026

Run the biggest language model your constrained hardware can handle, with chip-specific kernels and flash streaming for TV boxes, tablets, and phones.

Visit websiteGitHub969 starsDeploy on Hostinger
image of Smolmaxxing
TL;DR

Smolmaxxing is an open source patch set on llama.cpp that runs large language models on devices with 1 GB of RAM or less: TV boxes, old tablets, and mid-range phones. It combines chip-specific kernels, vocabulary pruning, and mixture-of-experts flash paging to reach 5.1 tokens per second for a 16B model on a 1 GB TV box, and 2.6 to 4.5 tokens per second on a 2013 iPad Air. MIT licensed. All inference runs locally with no cloud dependency.

Read the full breakdownHide the full breakdownWho it is for, how it works, trade-offs, the quick start and the FAQ
who it's for

Who Smolmaxxing is for#

Profiling edge inference on ARM hardware

Smolmaxxing gives real benchmark numbers on TV boxes and tablets: speed per model size class, failures included, across 165 documented experiments. If you need actual data on what ARM hardware with 1 GB of RAM can run at what speed, the experiment log and device docs are a primary source.

Skip if:

You need a production-ready inference service. This is a research and experimentation framework with no stable API, no container image, and device coverage still in progress.

IoT and edge product builders targeting constrained ARM

If you are building a product around ARM hardware with 1 to 4 GB of RAM and want local LLM inference without a cloud dependency, the techniques here (expert paging, flash streaming, vocab pruning) are MIT-licensed and forkable from a patch set on llama.cpp. The model files work on any ARMv8.2 Android device.

Skip if:

Your target hardware has a GPU or 8 GB or more of RAM. Standard llama.cpp or Ollama covers those cases without the patch, and their communities are much larger.

Researchers studying quantization on mobile ARM chips

The repo documents 1-bit lookup-table kernels, 2-bit and ternary repacked formats, and their measured gains on Cortex-A55 and Apple A7. The experiment log covers 165 runs with failures included. This is one of the few public resources benchmarking these techniques on mobile ARM chips with detailed, reproducible methodology.

Skip if:

You need CUDA, Vulkan, or GPU backends. The focus is CPU inference on mobile ARM; GPU work is out of scope.

the problem

The problem it solves#

Running a language model locally is a solved problem on modern hardware. A machine with 8 GB or more RAM and a capable GPU runs a 7B or larger model without special work. The problem is that most of the world's installed base of small computing devices sits outside that range. TV boxes, single-board computers, and phones from a few years ago typically have 1 to 4 GB of RAM and no GPU, and standard inference tools either refuse to load models on them or produce output at unusable speeds.

The workaround most developers reach for is a cloud API, but that moves every request off-device, adds network latency, and introduces per-token costs that add up for anything beyond occasional use. For offline use cases, edge deployments, or devices with no internet connection, the cloud path is not available at all.

how Smolmaxxing solves it

How it solves it#

Chip-tuned kernels for Cortex-A55 and Apple A7

A 1-bit lookup-table kernel and repacked 2-bit and ternary formats for the Cortex-A55, with matching versions for the Apple A7, which has no dot-product instructions. These kernels are what let 1-bit and ternary models run at conversational speeds on hardware that standard llama.cpp optimizations do not reach.

Mixture-of-experts flash paging

A MoE model uses a few experts per token. The pager keeps the most-used experts in RAM and reads the rest from flash on demand, prefetching the next layer ahead of time. This is how 8B and 16B models run on 1 GB of RAM and 35B and 80B models run on a phone with 3 GB free: most of the model never loads into memory at once.

Vocabulary head pruning

Most of a small model's output vocabulary is used rarely. The model files drop those rarely-used rows from the head, reducing the RAM each model occupies. On the TV box, a headless mode also stops the launcher and background services while a model runs, freeing additional memory.

Dense model streaming at flash speed

A dense model bigger than available RAM keeps part of itself in memory and streams the rest from flash at the drive's full read speed. The TV box gets 278 MB/s from its flash; at that rate, a 9B dense model reaches 0.27 tokens per second where it would otherwise refuse to load. For mixture-of-experts models, selective expert reads make the same hardware run 27 times faster.

Pre-built GGUF files for each device

Nine GGUF model files are published on Hugging Face, matched to the TV box and iPad hardware. They are the best file of each size class, already quantized and tuned: Qwen3.5-2B at 7.7 tokens per second on the TV box, Bonsai-1.7B at 24 to 26 tokens per second on the iPad. The TV box and iPad files need the patched llama.cpp build to load.

strengths · trade-offs

Strengths and trade-offs#

Strengths

  • MIT license, fully offline after setupThe code is MIT licensed. All inference runs locally on the device with no network connection required after the model is downloaded and no API key. Nine model files on Hugging Face are ready to download; the phone's 35B and 80B files are built on-device by a script in the repo. No cloud account, no per-token billing.
  • Measured results on actual hardware, not estimatesEvery speed figure in the repo comes from benchmarks run on the device itself. The experiment log documents 165 experiments with failures included. The best-of-class model files on Hugging Face are the ones that reached the published speeds on the published hardware, not projected numbers.
  • 27x speed gap documented between dense streaming and expert pagingOn the TV box, an 8B mixture-of-experts model runs at 7.2 tokens per second. A dense 9B model streaming from the same flash storage reaches 0.27 tokens per second: a 27x difference on the same hardware, because MoE skips most weight reads per token. The repo documents the measured gain for every technique with before-and-after numbers.

Trade-offs

  • -Three devices with published results, early stageThe TV box, the 2013 iPad Air, and one Android phone are the only devices with fully published results and tuned model files. Single-board computers, routers, and additional phone models are listed as coming. The technique set works on any Linux or Android ARM device with more model than RAM, but patched builds and tuned files only exist for these three today.
  • -Engineering project, not a consumer appThe model files for the TV box and iPad need the patched llama.cpp; stock llama.cpp cannot load their pruned heads. Setting up the TV box requires network ADB; the iPad requires an iOS 12 jailbreak. The README explicitly notes this is an engineering project with no finished product layer on top.
  • -Dense models past RAM are very slowThe streaming approach makes large dense models accessible but not practical for chat. On the TV box, the 9B dense model peaks at 0.27 tokens per second: a proof of concept, not usable in a conversation. Mixture-of-experts models at the same RAM budget run 27 times faster, which is the practical path on 1 GB devices.
versus alternatives

Smolmaxxing vs alternatives#

Smolmaxxing vs Cloud LLM APIs

When a device has too little RAM to run an LLM locally, most developers turn to a cloud API. Services like the OpenAI API give access to capable models over HTTP, billed per token, with no hardware requirements on the client side.

FeatureSmolmaxxingCloud LLM API
LicenseMITProprietary
Inference locationOn-deviceRemote server
Network requiredSetup onlyEvery request
Per-token costNoneYes
PrivacyPrompts stay on-deviceData sent to provider
Model size on phoneUp to 80B MoEFrontier models
LatencyDevice-localAPI round-trip

Smolmaxxing is the right path when offline operation is a requirement, when per-token API costs are a constraint at high volumes, or when data privacy demands that prompts never leave the device. A TV box running a 2B model at 7.7 tokens per second, or an iPad Air running a 16B model at 2.6 tokens per second, replace a cloud API for tasks where local output at acceptable speed is sufficient.

Cloud LLM APIs are still the better fit when response quality matters more than cost or privacy. A frontier model's reasoning and coding capability are beyond what any sub-100B model running on 1 to 3 GB of RAM can match. For tasks where the prompt needs frontier reasoning or broad world knowledge, local inference on constrained hardware is not the right trade. Smolmaxxing's niche is cases where local, offline output at a fraction of the cost is sufficient.

install · quick start

Quick start#

bash
Setup requires cloning the repository to get the patched llama.cpp build scripts and device-specific guides.
```bash
git clone https://github.com/hyperionaidev/smolmaxxing
```
tech stack · detected from GitHub

What it's built on#

Languages
CC++Python
frequently asked

FAQ#

What hardware is supported?

Three devices have published results: a Xiaomi TV Box S 3rd Gen (1 GB usable RAM, Android 14), a 2013 iPad Air (700 MB usable, iOS 12.5.7, jailbroken), and a OnePlus Nord CE 2 (3 GB free, Android 13). The core techniques work on any Linux or Android ARM device with more model than RAM, but patched builds and tuned model files only exist for these three so far. Single-board computers and more phones are listed as coming.

Does setup require root or jailbreak?

It depends on the device. The TV box and the Android phone do not require root; they use network ADB for file transfer and standard Android userspace. The iPad Air requires an iOS 12 jailbreak to get shell access and install the patched build. Future device targets may differ based on their platform constraints.

Can I run this on a Raspberry Pi or other single-board computer?

Single-board computers are listed as a future target but have no published results yet. The 1-bit NEON kernel works on any ARMv8 chip and the flash streaming runs on any Linux device, so the fundamental techniques apply. There are no tuned model files or documented setup paths for SBCs today.

How does this relate to llama.cpp or Ollama?

Smolmaxxing is a patch set on llama.cpp, not a fork or replacement. It adds kernels, pruned model file formats, and an expert pager that the upstream project does not include. Ollama uses stock llama.cpp and cannot load the pruned model files smolmaxxing produces. The TV box and iPad files specifically require the patched build; the phone files load on stock llama.cpp but need the patch to reach the published speeds.

Do models run offline after setup?

Yes. After the initial setup and model download from Hugging Face, all inference runs locally on the device with no network connection required. The only network dependency is during setup: downloading the model file and, for the TV box, ADB over the local network.

also worth a look

Similar open-source tools#

Hermes Agent

Featured

Alternative toGrok Bot

Self-hosted AI agent: persistent memory, multi-channel chat, model choice.

  • 263,992 GitHub stars
  • MITLicence MIT

Agentrun

InAI & Machine Learning

Self-learning agent control plane for Claude Code, Codex and Pi.

  • 1,210 GitHub stars
  • Apache-2.0Licence Apache-2.0

Ninfer

InAI & Machine Learning

Native C++/CUDA inference for Qwen3 and GLM models, self-hosted.

  • 3,356 GitHub stars
  • Apache-2.0Licence Apache-2.0

AutoRAG

Alternative toRagie AI

Local-first AI librarian for files, messengers, and cloud drives

  • 5,899 GitHub stars
  • Apache-2.0Licence Apache-2.0

SkillEvaluator

InAI & Machine Learning

Validation, deduplication, and live sandbox eval for AI agent skills.

  • 1,698 GitHub stars
  • Apache-2.0Licence Apache-2.0

Llama.cpp

Alternative toOpenAI API

C/C++ LLM inference engine with OpenAI-compatible API, no cloud.

  • 135,212 GitHub stars
  • MITLicence MIT
by category

Open source tools in these categories#

549

AI & Machine Learning

REA (Reverse Engineer Anything), freeCodeCamp, OpenClaw
142

LLMOps & AI Tooling

Hermes Agent, OpenCode, MarkItDown
673

Developer Tools

Skills for Real Engineers, REA (Reverse Engineer Anything), superpowers
Hostinger

Self-host Smolmaxxing on a Hostinger VPS

Keep it running 24/7, even when your laptop is off. A Hostinger VPS gives it a server of its own.

  1. 01

    Choose a plan

    Starts from $6.49/mo

  2. 02

    Apply

    Extra 10% off yearly plans

  3. 03

    Install Smolmaxxing

    On a server that stays on

  4. 04

    Start building

    Focus on the app, not the server

Deploy on Hostinger

Sponsors

Inbox Zero

Alternative toSuperhuman

AI email assistant that drafts replies and clears your inbox

  • 13,058 GitHub stars
  • AGPL-3.0Licence AGPL-3.0
  • #Email

rakazo

Alternative toGrok Bot

AI teammates you own: your keys, your model, your machine.

  • 8,913 GitHub stars
  • Apache-2.0Licence Apache-2.0
  • #AI Agents

SnitchFeed

Live monitoring on LinkedIn, X, Reddit & more. Agentic social listening for startups.

Marketing & Growth

Advertise your product

New open source projects by email

The projects we add and every new guide. Free, unsubscribe any time.

Find open source alternatives to paid software

/
  • Claude Code to OpenCode
  • Jira to Windshift
  • Notion to AppFlowy
  • Slack to Mattermost
  • Linear to GitLab
  • Wispr Flow to Handy
  • Arc to Zen Browser
  • CapCut to Concat
  • Grok Bot to rakazo
  • Superhuman to Inbox Zero
  • Discord to Stoat
  • Airtable to AITable
  • Datadog to SigNoz
  • Muse to OpenClaw
  • Railway to Coolify
  • GitHub Copilot to Cline
  • Confluence to Wiki.js
  • Microsoft Teams to Rocket.Chat
  • 1Password to KeePassXC
  • Firebase to Supabase
  • Pinecone to Qdrant
  • Zapier to n8n
  • Devin AI to OpenHands
  • Asana to Leantime
  • Claude Code to OpenCode
  • Jira to Windshift
  • Notion to AppFlowy
  • Slack to Mattermost
  • Linear to GitLab
  • Wispr Flow to Handy
  • Arc to Zen Browser
  • CapCut to Concat
  • Grok Bot to rakazo
  • Superhuman to Inbox Zero
  • Discord to Stoat
  • Airtable to AITable
  • Datadog to SigNoz
  • Muse to OpenClaw
  • Railway to Coolify
  • GitHub Copilot to Cline
  • Confluence to Wiki.js
  • Microsoft Teams to Rocket.Chat
  • 1Password to KeePassXC
  • Firebase to Supabase
  • Pinecone to Qdrant
  • Zapier to n8n
  • Devin AI to OpenHands
  • Asana to Leantime