needle

Dataset & Model

cactus-compute/needle

14MB foundation model for on-device AI enabling tool calling and structured extraction.

Overview

Needle 2 is a highly compressed 45M-parameter model packaged as a single 14MB binary for phones, wearables, and robots. It performs tool calling, device use, and structured extraction with confidence gating and a 28MB memory footprint. The Python package offers inference, LoRA fine-tuning, and export.

README Preview

\n\n# Needle 2\n\nNeedle 2 is an open 45M-parameter model for tool calling, device use and structured extraction. The whole model is a single 14MB binary that runs a full session in about 28MB of RAM. It is built on our Simple Attention Network findings, compressed to CQ2-bit with Cactus Quants, and baked into its own engine. On the benchmarks below, Needle 2 trades wins with other small models like FunctionGemma 270M, LFM2.5 230M and Apple FM, at 5x to 70x smaller, and 2 bits against their f16.\n\nThis repository is the Python package: inference, LoRA fine-tuning, and export. `pip install cactus-needle`, describe your tools, and call them from Python. The inference engine is fetched once from Hugging Face and cached; there is nothing else to build.\n\n- **Self-contained**: weights baked into a single 14MB engine; no separate model files to manage, and inference does no network.\n- **Simple contract**: tool calls come back as structured data, text in, JSON out; a byte-level grammar compiled from your schemas constrains every token.\n- **Confidence-gated**: every response carries a calibrated confidence score from a learned head; set a threshold, act above it, escalate below it.\n- **Tool retrieval**: declare a large catalogue and a built-in retrieval head renders only the top five tools per turn, with the grammar constrained to that subset.\n- **Bounded memory**: a 256-token sliding window with the tools pinned as KV sinks, so total memory stays near 28MB no matter how long the conversation runs.\n\nWeights: [huggingface.co/Cactus-Compute/needle2](https://huggingface.co/Cactus-Compute/needle2) · source: [github.com/cactus-compute/needle](https://github.com/cactus-compute/needle).\n\n\n\n## Simple Attention Network\n\nNeedle 2 is a Simple Attention Network, our dense small-model recipe: a Hadamard MLP in place of the FFN, GQA attention, engram key-value memory, and multi-lane hyper-connections. See the paper for the design and ablations: [arXiv:2607.18363](htt

FAQ (5)

Troubleshooting
How do I fix the 'needle fetch' 404 for cactus_needle-3.0.2 wheel after installing cactus-needle 3.0.5?

The failure occurs because cactus-needle 3.0.5 requests engine wheel 3.0.2, but the Hugging Face repository only has 3.0.0 and 3.0.1. Immediate workaround: downgrade to 3.0.4, which pins engine 3.0.1:

pip install --force-reinstall cactus-needle==3.0.4
needle fetch

The code fix changes ENGINE_VERSIONS[3] in needle/agent/fetch.py from 3.0.2 back to the published 3.0.1 wheel, or installs a release containing that fix. Alternatively, the 3.0.2 wheels must be uploaded to the Cactus-Compute/needle3 Hugging Face repository.

GitHub Issue #142
Troubleshooting
How to fix "unrecognized arguments: --layers" when running needle finetune?

Do not pass --layers to needle finetune; the option is only advertised by stale help text and is not supported by the finetune parser. Run fine-tuning without it, for example needle finetune data.jsonl --checkpoint <path>. Rung selection belongs to the build step: use needle build --layers N. Local fine-tuning runs at the full 20 layers. Update to a release containing the CLI help fix if the outdated hint still appears.

GitHub Issue #139
Troubleshooting
How to fix empty `function_calls` on alternating sequential requests when reusing a Needle generation=2 instance?

This happens because complete()/_complete() treats each call as a turn in the same conversation. Unrelated prompts accumulate in the history, causing confidence to drop and tool calls to alternate with function_calls: [].

For independent prompts, reset the conversation between calls:

agent = needle.Needle(tools=[get_weather, send_message], generation=2)

for prompt in prompts:
    resp = agent._complete(prompt)
    print(resp["function_calls"], resp["confidence"])
    agent.reset()

agent.close()

Alternatively, use the stateless mode if your installed version includes it:

agent = needle.Needle(
    tools=[get_weather, send_message],
    generation=2,
    stateless=True,
)

stateless=True automatically resets before each query. Creating and closing a fresh Needle instance for each independent request is also a valid workaround. Only reuse the instance without resetting when the prompts are actually part of one conversation.

GitHub Issue #138
Troubleshooting
How do I fix Needle initialization failing with a 404 for cactus_needle-3.0.2 wheel on Hugging Face?

The failure occurs because needle requests engine wheel cactus_needle-3.0.2-py3-none-manylinux2014_x86_64.whl, but the Cactus-Compute/needle3 Hugging Face repo only publishes 3.0.0 and 3.0.1. Upgrade cactus-needle to a release containing the fetch fix. If no fixed release is installed yet, pin the engine to 3.0.1 locally:

import pathlib, needle
p = pathlib.Path(needle.__file__).parent / "agent" / "fetch.py"
s = p.read_text()
p.write_text(s.replace("3.0.2", "3.0.1"))

Then restart the Python process/Colab runtime and initialize needle.Needle() again. Alternatively, use a package version that still references the available 3.0.1 wheel.

GitHub Issue #146
Troubleshooting
How do I fix Needle parsing the hour in "5 June 19:30" as a year and marking the date ungrounded?

This is a regex bug in needle/__init__.py in _source_years. The day/month pattern uses (\d{1,4})(?![0-9A-Za-z]), so 19 in 19:30, 14 in 14:00, and 7 in 7 pm are incorrectly treated as years. Upgrade to a release containing the issue #153 fix. If you need to patch it immediately, change the trailing year lookahead to reject time suffixes, for example: (\d{1,4})(?!\s*:\d{2}|\s*[ap]\.?m\.?(?![A-Za-z])|[0-9A-Za-z]) with case-insensitive matching. This still allows short years such as "5th September 42". As a temporary workaround, include an explicit four-digit year, e.g. "5 June 2026 19:30". Verify with needle._source_years("Book a table on 5 June 19:30"), which should return set(), then run pytest -q -m "not slow".

GitHub Issue #153