Belgian startup Corbenic AI wants to stop graphics processing units from repeatedly reading identical text documents during AI operations. The company introduced its Galahad software in beta, utilizing standard Linux environments, Nvidia GPUs, and local NVMe solid-state drives without requiring exotic new hardware.
The software works by saving a language model's key-value cache—the attention state generated while processing a prompt—directly to encrypted local storage. When the identical text appears again after a process restart or when GPU memory is cleared, the system reloads the saved state rather than recomputing it from scratch. Founder and CEO Sietse Schelpe noted that the method effectively integrates the server's existing SSD into the artificial intelligence working memory layer.
The system relies on a component called Taliesin, which writes each block of text state to disk using a precise fingerprint of the input bytes, tenant, and model. However, performance data indicates that achieving high performance involves more than just loading saved work from storage. A secondary text-retrieval component named Blaise supplies the model with specific relevant sections rather than entire corpuses, driving substantial speed increases in certain testing scenarios alongside the core storage reuse features.
Corbenic enters a competitive landscape already populated by alternative persistence methods, including open-source projects like LMCache as well as enterprise offerings from Nvidia, Phison, and WEKA. While industry competitors pursue various strategies to alleviate high-end memory bottlenecks in data centers, Corbenic focuses on exact, encrypted reuse delivered through a compact library designed to maximize efficiency on existing server hardware.
Comments