Cactus: Accelerating Auto-Regressive Decoding with Constrained Acceptance Speculative Sampling

Create a high-performance C++/CUDA implementation of constrained acceptance speculative sampling for common open-source LLMs. This helps developers achieve higher throughput without sacrificing distribution accuracy.

Suggested repo: cactus-decode

"Accelerate your LLM inference with flexible, distribution-aware speculative sampling."

Estimated effort: 80h