DS-Serve

A Framework for Efficient and Scalable Neural Retrieval

We present DS-Serve, a framework that transforms largescale text datasets—comprising half a trillion tokens—into a high-performance neural retrieval system. DS-Serve offers both a web interface and API endpoints, achieving low latency with modest memory overhead on a single node. The framework also supports inference-time tradeoffs between latency, accuracy, and result diversity. We anticipate that DS-Serve will be broadly useful for a range of applications such as large-scale retrieval-augmented generation (RAG), training data attribution, training a search agent, and beyond.

---