Who heretic is for#
ML researchers studying safety alignment mechanisms
Heretic's interpretability extras let you visualize residual vectors, compute cosine similarities across layers, and generate layer-by-layer animations of how harmful vs. harmless prompts differ in representation. This supports empirical work on where and how safety alignment is encoded in transformers without requiring custom tooling.
Skip if:
Skip if your research focuses on building or improving alignment rather than analyzing it, or if you need a tool with formal academic support and published peer review rather than a community project.
Developers building local AI applications needing unrestricted outputs
If your application requires a locally hosted LLM that responds to any prompt without content refusals, Heretic automates the process of producing that model from a Hugging Face checkpoint. The entire workflow is a pip install and one command; no knowledge of transformer architecture is needed.
Skip if:
Skip if your deployment targets a cloud API (commercial providers will not run user-modified model weights), or if your use case is already served by an existing heretic-tagged model on Hugging Face.
Self-hosted AI enthusiasts evaluating open model quality after abliteration
Heretic's Hugging Face integration and interactive chat session after each run make it practical for comparing how different base models respond once safety filters are removed. The built-in benchmark runner (MMLU, GSM8K) provides a consistent quality signal across abliteration runs on different model families.
Skip if:
Skip if you prefer pre-built model images rather than running a 20-to-30-minute abliteration job locally. The Hugging Face model hub already has over 5,000 heretic-tagged models ready to download.
LLM security engineers running red-team baseline evaluations
Security and red-teaming engineers can use Heretic to produce controlled uncensored variants of frontier open models, providing a baseline comparison point for evaluating how effectively safety alignment prevents undesired outputs before and after abliteration.
Skip if:
Skip if your organization prohibits running uncensored models internally, or if you need commercial support and liability coverage for the tooling used in security evaluations.
The problem it solves#
Running a local language model with no content restrictions has historically required deep expertise in transformer internals. Manual abliteration involves identifying residual directions across layers, selecting ablation weight kernels, and tuning parameters by hand: work accessible only to ML researchers with specialized background.
At the same time, commercial API services enforce content policies that block entire categories of queries outright. Teams that need unrestricted outputs from a locally hosted model had no automated path: they either engaged expert ML contractors or accepted the restrictions of closed platforms with per-token pricing and no self-hosting option.
How it solves it#
Automatic ablation parameter optimization
Heretic uses a TPE-based optimizer (Optuna) to find ablation parameters without manual tuning. It co-minimizes refusal rate and KL divergence from the original model simultaneously, so the decensored output retains as much original intelligence as possible. On a gemma-3-12b-it benchmark, Heretic reached the same refusal suppression as expert-made abliterations at a KL divergence of 0.16, compared to 0.45 and 1.04 for two competing manually tuned alternatives.
Flexible ablation weight kernel with float direction index
Unlike fixed-weight abliteration approaches, Heretic applies a shape-optimizable weight kernel over transformer layers, with separate ablation parameters for attention out-projection and MLP down-projection. The residual direction index is a float rather than an integer, enabling interpolated directions that extend beyond what difference-of-means computation alone identifies, unlocking a larger optimization space.
Broad model architecture support
Supports most dense transformer models, many multimodal models, several MoE architectures, and hybrid models like Qwen3.5. Load a model from Hugging Face by name and Heretic handles the rest. Pure state-space models and certain research architectures are not yet supported out of the box.
Built-in quantization via bitsandbytes
Set the quantization option to bnb_4bit to run Heretic on models that would otherwise exceed available VRAM. Quantization is applied during the decensoring process itself, making it practical to abliterate large models on consumer-grade GPUs without needing a separate quantization step.
Post-run evaluation, benchmarking, and upload
After decensoring, Heretic offers to save the model locally, upload it directly to Hugging Face, launch an interactive chat session to evaluate quality, or run standard benchmarks (MMLU, GSM8K). All options are available in any combination from a single menu prompt at the end of each run.
Research interpretability extras
The optional research extra adds tools for studying model internals: PaCMAP-based visualizations of residual vectors across transformer layers, animated GIFs showing how residuals evolve layer by layer, and a quantitative table of residual geometry metrics including cosine similarity, L2 norms, and silhouette coefficients per layer.
Strengths and trade-offs#
Strengths
- Fully automatic: no ML expertise requiredAnyone who can run a command-line program can use Heretic. The optimization loop finds high-quality ablation parameters without human guidance. This is the core differentiator from prior art tools like abliterator.py and AutoAbliteration, which require manual parameter selection and iterative evaluation by someone who understands transformer internals.
- Lower capability damage than manual abliterationsBenchmarks on gemma-3-12b-it showed Heretic's automatically generated abliteration achieved 3/100 refusals (matching expert hand-tuned results) at a KL divergence of 0.16, compared to 0.45 and 1.04 for two competing manually produced abliterations. Lower KL divergence means less damage to the model's reasoning and generation quality.
- Over 5,000 community-published models on Hugging FaceThe Heretic community has published more than 5,000 models to Hugging Face tagged with the heretic topic. This provides an immediately accessible library of pre-abliterated models for common architectures, so users who cannot run Heretic locally can still access its output without running the tool themselves.
- Written from scratch with a research orientationHeretic does not reuse code from any prior abliteration project. Its research extras provide quantitative metrics (residual geometry tables, PaCMAP projections) that support interpretability work, making it useful beyond censorship removal for researchers studying how safety alignment is encoded in transformer layers.
Trade-offs
- -Requires consumer GPU hardware with significant VRAMAbliterating a 4B model on an RTX 3090 takes 20 to 30 minutes. Larger models need substantially more VRAM. While bitsandbytes quantization reduces requirements, very large models are impractical on anything below a high-end GPU setup. CPU-only runs are not documented in the current README.
- -AGPL-3.0 requires source disclosure for hosted servicesAGPL-3.0 is a copyleft license. If you run a modified version of Heretic as a network service (for example, an API endpoint that accepts models and returns decensored versions), you must release your modifications under the same license. Teams wanting to build a proprietary service on top of Heretic need to evaluate this constraint carefully before integrating it.
- -Pure state-space models and some architectures not yet supportedHeretic explicitly states that pure state-space models and certain other research architectures are not supported out of the box. If your target model falls into this category, you will need a different abliteration approach or wait for future Heretic support.
heretic vs alternatives#
Heretic vs AutoAbliteration
AutoAbliteration is Maxime Labonne's approach to automated abliteration, published as a Hugging Face Space and script. Both tools aim to reduce manual effort in the abliteration workflow. Heretic's key advantage is its TPE-based parameter optimizer, which explores the full ablation weight kernel shape automatically. AutoAbliteration applies simpler fixed-weight ablation and leaves parameter selection to the user. Heretic benchmarks show lower KL divergence at the same refusal suppression rate compared to Labonne's own models produced without the optimizer.
AutoAbliteration is a reasonable starting point if you want a lightweight script without the Optuna dependency. Heretic is the better choice when output quality matters: the automated search finds better parameter configurations than fixed defaults across most model architectures.
Heretic vs abliterator.py
abliterator.py (by FailSpy) is the most widely referenced manual abliteration script in the community. It requires the user to select which layers to ablate, which direction indices to use, and what weight to apply. Heretic replaces that entire manual process with an optimization loop. For users without transformer expertise, abliterator.py is effectively inaccessible; Heretic is not.
abliterator.py remains the better choice for researchers who need precise manual control over which specific layers and directions are modified. Heretic's optimizer may not always explore the exact parameter combination a researcher is investigating. For production use cases where output quality is the primary goal rather than fine-grained control, Heretic's automatic approach consistently outperforms manual configurations in benchmark comparisons.
Install and self-host#
Install Heretic via pip into a Python 3.10+ environment with PyTorch 2.2+ configured for your hardware.
```bash
pip install -U heretic-llm
heretic Qwen/Qwen3-4B-Instruct-2507
```What it's built on#
- Languages
- Python
FAQ#
What is abliteration and how does it differ from fine-tuning?
Abliteration is a post-training intervention that modifies a model's weight matrices to suppress a specific direction in residual space, without any gradient updates or new training data. Fine-tuning requires labeled examples and a full training run; abliteration operates directly on the saved model weights in a single forward-pass analysis step. Heretic automates the abliteration process using parameter optimization, making it faster and less expertise-dependent than either manual abliteration or fine-tuning.
What hardware do I need to run Heretic?
Heretic requires a CUDA-capable GPU with enough VRAM to hold the target model. On an RTX 3090 (24GB VRAM), a 4B-parameter model takes 20 to 30 minutes to process. Larger models require proportionally more VRAM, but bitsandbytes 4-bit quantization is supported and reduces requirements significantly. Some model configurations may require PyTorch 2.6+ for features like MXFP4 quantization. CPU-only execution is not documented in the current README.
Can I use Heretic with models that are not on Hugging Face?
Yes. The heretic CLI accepts any model path that the underlying transformers library can load, including local directories in the standard Hugging Face safetensors format. You do not need a Hugging Face account to abliterate a local model checkpoint; an account is only needed if you want to upload the result to the Hugging Face hub.
What does the AGPL-3.0 license mean for my use of Heretic?
AGPL-3.0 is a copyleft license. Running Heretic locally for personal or internal research is unrestricted. If you modify Heretic and run it as a network service accessible to others (such as an API endpoint), you must release your modifications under AGPL-3.0 as well. The license applies to Heretic itself, not to the models you produce with it; the decensored model output is governed by the license of the base model you started from.
How does Heretic compare to other abliteration tools like abliterator.py or ErisForge?
Heretic's main advantage is automatic parameter optimization via Optuna's TPE algorithm, which selects ablation weights without manual tuning. In benchmarks on gemma-3-12b-it, Heretic produced the same refusal suppression as manually tuned alternatives (3/100 refusals) at significantly lower KL divergence (0.16 vs 0.45 to 1.04), indicating less damage to the model's capabilities. Heretic was also written from scratch and does not share code with any prior abliteration project. The tradeoff is that Heretic's optimizer makes choices automatically, which may not suit researchers who need precise manual control over which layers and directions are modified.
Similar open-source tools#
OpenCode
OpenCode is an open-source AI coding agent that assists developers in
marin
Open lab for training foundation models together
trueforge
Open-source agent harness for reliable, long-running LLM agents
omlx
Fast LLM inference for Apple Silicon with persistent SSD KV caching
llmfit
One command to rank every LLM against your hardware.
ai-memory
Persist AI agent context across sessions and tool switches