TensorFold
What it is
TensorFold is a Python library and CLI tool that serves text models on Apple Silicon (MLX) and NVIDIA GPUs via an OpenAI-compatible API. It implements exact speculative decoding where draft tokens are only accepted if they match the serial engine's output, ensuring deterministic results while improving speed.
The problem it solves
Standard speculative decoding implementations often produce non-deterministic or approximate outputs, and local LLM serving on Apple Silicon lacks efficient, exact decoding pipelines for modern model families.
What you could use it for
- Serving LLMs locally on Apple Silicon Macs with OpenAI API compatibility
- Running speculative decoding for models like Qwen3.8, Nemotron, and GLM-5.3
- Deploying quantized (4-bit/EXL3) models with exact arithmetic guarantees
Read the source ↗Back to the library →
Summary written by an AI model from the source; check the source before you rely on it.