GreenBoost Windows
What it is
A Windows port of the GreenBoost project that provides a transparent VRAM extension for LLM inference. It works by intercepting `cudaMalloc` calls and routing them through CUDA Unified Memory or a kernel-driver-mapped page system, allowing GPUs to use system RAM and NVMe as VRAM.
The problem it solves
The limitation of physical VRAM on consumer GPUs, which prevents running large models that require more memory than is physically available on the graphics card.
What you could use it for
- Running Large Language Models (LLMs) that exceed the physical VRAM of a 12GB or 16GB NVIDIA GPU.
- Inference on Windows without modifying the source code of the inference engine.
- Expanding addressable GPU memory for multi-GPU or single-GPU setups on Windows.
Read the source ↗Back to the library →
Summary written by an AI model from the source; check the source before you rely on it.