Who Comet is for#
Developers running multi-session tasks on Claude Code or Codex
Comet writes phase state, decisions, and evidence to the project directory at every stage of a task. After an interruption, context compression, or device change, `/comet` resumes from the saved state without rereading documents or reconstructing progress manually. Setup takes one `comet init` in the project directory.
Skip if:
If your tasks complete in a single session with no interruptions, Comet's state management adds setup overhead without proportional benefit for short one-shot work.
Teams building and distributing reusable AI coding Skills
`comet eval` runs Skills on real tasks in isolation and scores them with Rubric, Pass@k, and Pass^k. Failure attribution identifies whether the issue is in the workflow design, task definition, or model. Results can flow to LangSmith or Langfuse for cross-run comparison. `/comet-any` packages the finished Skill for distribution to all 37 supported platforms.
Skip if:
If you are building a one-time Skill for personal use with no intent to distribute or iterate, the evaluation and packaging infrastructure adds more ceremony than is warranted.
Engineers coordinating complex deliverables across multiple agents
Supervisor Change accepts a complex goal, splits it into dependency-mapped child changes, and coordinates independent Codex sessions or Claude Code Agent Teams in isolated worktrees. The Runtime handles worktree creation, verification ordering, integration, and the parent change's final acceptance. Define the goal; Comet manages the coordination and state tracking.
Skip if:
If a single agent can implement and verify the full deliverable in one pass, multi-agent coordination adds overhead without benefit.
The problem it solves#
Long-running AI coding tasks have a reliability problem. Agents lose context after session interruptions, skip verification steps when they feel close to done, and produce output with no objective quality measurement. Teams re-run the same task from scratch after a bad session, review every output manually with no systematic rubric, or abandon multi-step automation after one failure.
Evaluation is the second layer of the problem. There is no standard way to know whether a coding Skill performs better than writing the prompt by hand. Without reproducible scoring, Skills never improve systematically, and every iteration is guesswork with no measurable baseline.
How it solves it#
Phase-Guarded Native and Classic Workflows
Comet provides two independent workflow modes: Native, which uses a structured brief and automated coding loop for strong models that can plan and verify autonomously, and Classic, a five-phase Open to Archive workflow with explicit confirmation points for tasks needing stronger constraints. Both share the same phase guard system: cross-phase writes are intercepted, confirmation points cannot be skipped, and workflow state is written to disk at every stage.
Recoverable State Across Interruptions and Devices
Comet writes the current phase, decisions, verification evidence, and next step to the project directory in each change's `.comet.yaml`. Work resumes after a session interruption, context compression, or device switch without rereading documents or manually reconstructing progress. The CLI output always identifies the current state and the next required action.
Dual-Agent Skill Evaluation
`comet eval` runs any local Skill on real tasks in an isolated environment. A user-simulation agent auto-replies at decision points and resumes the same session via `--resume`, so multi-phase workflows complete automatically. Output includes Rubric multi-dimensional scores, Pass@k (does it pass at all), and Pass^k (does it pass consistently), with failure attribution across four layers: tool configuration, workflow design, task definition, and model.
Skill Creator and Distribution
`/comet-any` composes existing Skills into new reusable packages. Confirm the workflow contract first, then evaluate and publish. Published Skills can be distributed to all 37 supported AI coding platforms with `comet init`, turning one-time workflows into shareable capabilities with reproducible evaluation history.
Supervisor Change for Multi-Agent Delivery
Complex goals are split into dependency-aware child changes. Independent Codex sessions or Claude Code Agent Teams implement and verify each child change in isolated worktrees. The Runtime manages worktree creation, dependency ordering, integration, and the parent change's final Verify acceptance. Available when a requirement has deliverables that can be implemented and tested independently.
Self-Evolving Personal Memory and Project Knowledge
Personal Memory retains your collaboration preferences and verified experiences across sessions, supplying only the context relevant to the current task. Project Knowledge builds a model of the codebase from code, configuration, and review history, and automatically deactivates stale entries when source files change. Both are inspectable, correctable, and pausable.
Strengths and trade-offs#
Strengths
- Runs Inside Your Existing AI Coding ToolComet initializes into any project via `comet init`, writing Skills, rules, and phase guard hooks for each detected platform. It works with 37 AI coding tools including Claude Code, Codex, GitHub Copilot, and Cursor. You keep your current workflow; Comet adds the state machine and evaluation layer without replacing the AI tool or requiring a new account.
- MIT License With No Cloud DependencyThe CLI runs on your machine as a pure Node.js process with no cloud account, no hosted service, and no Docker container required. The MIT license permits commercial use without restriction. Installation is a single npm command; the only runtime requirement is Node.js 22.16 or later.
- Evaluation Grounded in Reproducible MetricsRubric scoring plus Pass@k and Pass^k metrics produce measurable evidence for whether a Skill version improves outcomes. A 16-task, 48-run benchmark comparing Native mode to the prior approach showed 76.8% fewer total tokens, 57.4% fewer agent rounds, and 47.4% less time. Native reached 87.5% pass^3 versus 75.0% for the prior approach. Results are reproducible and traceable to specific workflow layers.
- LangSmith and Langfuse IntegrationEvaluation results connect to LangSmith or Langfuse for cross-run observability, dataset management, and scoring comparison. Traces, datasets, and scores flow to your existing observability stack, so Skill evaluation is not siloed from the rest of your production tooling.
Trade-offs
- -Node.js 22.16 or Later RequiredComet requires Node.js 22.16+ (22.x or 24+) and npm. Teams on older Node.js runtimes need to upgrade before using it. The pure Node.js runtime works on Windows, macOS, and Linux without Bash or WSL, but the version constraint is a hard requirement that cannot be worked around.
- -Early-Stage API and Configuration SurfaceComet launched in May 2026 and is at version 0.4.x with 30 open GitHub issues. The configuration format and workflow API are still evolving. The `/comet-any` Skill composer is explicitly labeled experimental in the docs. Teams with low tolerance for breaking changes in tooling should review the changelog before adopting it.
- -Classic Mode Designed for OpenSpec UsersClassic Spec mode is built around the OpenSpec and Superpowers five-phase methodology. Teams not already using these tools get limited value from Classic mode and may find the confirmation-point model adds more friction than it removes. Native mode is the more accessible entry point for most developers.
Comet vs alternatives#
Comet vs Devin
Devin, from Cognition, is a fully autonomous AI software engineer that operates in its own cloud environment, taking a GitHub issue and working through implementation, testing, and deployment without requiring the developer's local tooling. Comet takes a different approach, running inside your existing AI coding tool as a workflow layer and adding phase governance and evaluation to tools like Claude Code, Codex, or Cursor.
| Feature | Comet | Devin |
|---|---|---|
| License | MIT | Proprietary |
| Self-hosting | Yes (Node.js CLI) | No (cloud-only) |
| Works with existing tools | Yes (37 platforms) | No (replaces them) |
| Skill evaluation | Rubric + Pass@k/Pass^k | Not available |
| Pricing | Free | Subscription |
Comet is the better choice when you want workflow discipline and reproducible evaluation inside the tools you already use, without migrating to a separate platform or paying per-task fees. Devin is worth considering when you need a fully autonomous agent that handles an entire GitHub issue end-to-end with minimal developer input.
Comet vs GitHub Copilot Workspace
GitHub Copilot Workspace is a task-centric AI coding environment built into GitHub, helping developers plan and implement changes from a GitHub issue. Comet is an open source CLI that adds persistent state, multi-session recovery, and Skill evaluation to any AI coding tool, independent of the code hosting platform.
| Feature | Comet | Copilot Workspace |
|---|---|---|
| License | MIT | Proprietary |
| Self-hosting | Yes | No (GitHub-hosted) |
| Multi-session state | Yes | No (task-scoped) |
| Skill evaluation | Yes (Rubric, Pass@k) | No |
| Platform coverage | 37 AI coding tools | GitHub Copilot only |
Copilot Workspace is the natural choice for GitHub-centric teams who want AI-assisted task planning and implementation without additional tooling. Comet is the better choice for developers who need resumable multi-session workflows, reproducible Skill evaluation across runs, or support for AI coding tools outside the GitHub ecosystem.
Quick start#
Install Comet globally via npm, then initialize it in your project directory.
```bash
npm install -g @rpamis/comet
cd your-project
comet init
```What it's built on#
- Languages
- JavaScriptPythonTypeScript
- Frameworks
- React
- Tooling
- esbuild
FAQ#
Does Comet replace my existing AI coding tool?
No. Comet installs inside your existing tool by writing Skills, rules, and phase guard hooks into the project directory. It supports 37 platforms including Claude Code, Codex, Cursor, and GitHub Copilot. You keep your current AI tool; Comet adds the workflow state machine and evaluation layer on top of it.
What is the difference between Native mode and Classic mode?
Native mode is designed for strong models that can plan and verify autonomously. After you confirm the target spec, the model drives implementation, testing, and review; Comet handles state checks and acceptance. Classic mode is a five-phase workflow (Open, Design, Build, Verify, Archive) with explicit confirmation points at each stage, suited for larger changes or models that benefit from stronger constraints. Both modes share the same phase guard and state recovery system.
How does Comet evaluate AI coding Skills?
comet eval uses a dual-agent architecture: the evaluated agent runs the Skill in an isolated environment while a user-simulation agent auto-replies at decision points and resumes the session via --resume. Output includes Rubric multi-dimensional scores, Pass@k (does it pass at all), and Pass^k (does it pass consistently across runs), with failure attribution identifying whether the issue is in the workflow design, task definition, or model.
Is Comet free and open source?
Yes. Comet is MIT licensed and available at github.com/rpamis/comet. It installs via npm install -g @rpamis/comet and runs entirely on your local machine with no managed cloud service, no per-use billing, and no account required.
Which AI coding platforms does Comet support?
Comet supports 37 AI coding platforms as of version 0.4.x, including Claude Code, Codex, GitHub Copilot, Cursor, Devin Desktop, Amazon Q Developer, Gemini CLI, RooCode, Cline, and Continue. The comet init wizard detects installed platforms and configures Skills and phase guard hooks for each in one step.
Similar open-source tools#
Hermes Agent
254,611 GitHub starsSelf-hosted AI agent: persistent memory, multi-channel chat, model choice.
Alternative to
Grok Bot
Orgtree
146 GitHub starsDesktop canvas for persistent multi-agent AI teams, Windows-only
Alternative to
Conductor
Agent Cache Web
11 GitHub starsConvert docs URLs into Markdown bundles for AI coding agents
Alternative to
Context7
iFixAi
29,142 GitHub starsSelf-hosted AI agent auditing across 60 trust and safety inspections.
Alternative to
Zenity
WeKnora
34,526 GitHub starsSelf-hosted knowledge base with RAG search, AI agent, and wiki
Alternative to
Glean
Marker
40,517 GitHub starsConvert documents to clean markdown and JSON with open source AI.
Alternative to
Unstructured.io