A New RAG State of the Art, with Second-Scale Retrieval Across 500 Million Records

Recently, 512,000 lines of Claude Code's TypeScript source code were accidentally leaked.

It seemed that Anthropic had exposed the inner workings of Claude Code. After examining the code, the AI community found that the most interesting part was the operating system around the coding agent, rather than the model itself.

Much of the code handled when to read files, call tools, compress context, and proceed to the next step.

This is the increasingly prominent agent harness.

The accident offered a reminder: in the agent era, model intelligence is only part of the story. Context organization, tool calling, and task loops matter too.

RAG, the infrastructure connecting models with external knowledge, must evolve alongside them.

Why?

Suppose someone asks, “Who influenced this product direction?” The answer might follow a chain:

Company A acquired Company B. Company B's CTO later joined Project C. Project C then influenced the product direction.

Individually, these three facts may not look especially similar to the question. Connecting them produces the answer.

Traditional RAG quickly finds passages that appear similar to a query, but may struggle to identify their relationships.

For agents, this matters greatly. They continue reasoning, calling tools, and making decisions based on retrieved material. An incorrect first retrieval can send the entire workflow in the wrong direction.

How serious can this become? One study of clinical text generation found that traditional RAG increased the hallucination rate from a 5.0% baseline to 43.6%.

It retrieved seemingly relevant material rather than the evidence actually required.

It is time to reconsider RAG.

Graphs are not a universal solution

Approaches such as Microsoft's GraphRAG are an important response to traditional RAG's limitations.

For the product-direction question, GraphRAG first extracts entities such as Company A, Company B, the CTO, Project C, and the product direction, together with their relationships, into a knowledge graph.

It can then organize an answer by examining connected entities, events with a common theme, and information supporting the same conclusion.

This is a substantial step from vector similarity toward structured relational reasoning. GraphRAG is particularly effective for global understanding and thematic summaries.

A knowledge graph provides structure, but construction and maintenance can be burdensome.

Extracting triples, merging entities, normalizing relationships, building a global graph, and summarizing communities all incur costs and introduce potential errors.

Even after construction, many systems do not fully follow graph relationships when answering queries. They return to retrieving a few similar nodes or summaries.

Meanwhile, the world keeps changing: project leaders change, customer requirements evolve, and product directions are revised. Rebuilding a precomputed graph every day is impractical.

Another strong approach, HippoRAG 2, arrived later.

Inspired by hippocampal memory, it aims to recall information by starting from a clue and spreading activation through graph relationships.

For the product-direction question, it identifies entities such as Company A, Company B, and the CTO, activates relevant graph nodes, and uses algorithms such as Personalized PageRank to expand along relationships—from Company B to its CTO, a person, Project C, and eventually the product direction. An LLM then uses these clues to answer.

By advancing structured memory and multi-hop retrieval, HippoRAG 2 addresses some of traditional RAG's weaknesses in multi-hop reasoning and long-term memory.

But substantial engineering challenges remain.

Like GraphRAG, it relies on a global graph constructed offline. Queries also require graph ranking such as PageRank or Personalized PageRank.

The approach is strong at benchmark scale. In real agent applications, maintaining and ranking a global graph becomes expensive as new documents, entities, aliases, and relationships arrive continuously.

Could we retain structure without building an elaborate global graph first? Support multi-hop retrieval without complex global ranking on every query? Enable long-term agent use without rebuilding everything for each new batch of data?

This is where SAG—SQL-Retrieval Augmented Generation—from Guangzhou-based Zleap AI comes in.

SAG: rebuilding the agent data foundation with hyperedges

The name signals its approach: SQL retrieval.

Offline, SAG organizes source text into a database of events and entities. When a query arrives, SQL dynamically assembles a local network of relevant clues.

For example, consider these source chunks discussing the film Love Letters to Grandma.

Traditional triples might split the event chain into statements such as:

• Overseas Chinese remittance letters — have — family-letter properties.
• Overseas Chinese remittance letters — have — remittance-receipt properties.
• Shenzhen company — invests in — Love Letters to Grandma.
• Film — uses — dialect.

Yet a passage often describes a complete event rather than a single relationship. Splitting it into many triples can fragment its meaning; an incorrectly extracted relationship can break the evidence chain.

SAG instead uses this structure:

Each chunk corresponds to a complete event. One event connects to multiple entities, and one entity can appear in multiple events.

An event connecting multiple entities resembles a many-to-many hyperedge.

The information is written into SQL and vector indexes. At query time, shared entities temporarily connect related events.

SAG structure: offline ingestion.

Suppose the question is: Why would someone invest in Love Letters to Grandma?

SAG first uses an LLM to identify query entities, such as investors, Shenzhen companies, funding sources, and investment decisions. Retrieval then follows two paths.

The first is structural.

SQL finds event records associated with those entities. It may first retrieve the event “A Shenzhen company invests in Love Letters to Grandma.”

That event explains the film's potential for social reach and market circulation but does not fully explain the investment case.

SAG therefore reads its entities—Shenzhen company, Chaoshan, overseas Chinese hometown economy, Chinese diaspora sentiment, family audiences, and social circulation—and uses SQL to find other events containing them.

This can retrieve an event about the cultural value of overseas remittance letters, then follow remittance letters, regional culture, overseas Chinese communities, and public cultural value to another event about the filmmakers' experience and modest production costs reducing investment risk.

The process is essentially SQL joins, rather than global graph reasoning. Information scattered across chunks becomes an evidence chain.

SAG structure: online retrieval.

The second path is semantic.

SAG retains vector retrieval, using the query embedding to find semantically similar source chunks.

It therefore obtains two groups of candidates. Similarity filtering narrows them, and an LLM selects the key events from the smaller set.

Those events map back to source chunks and merge with directly retrieved vector results to form the final evidence supplied to the LLM.

The resulting answer might explain that investors backed the film despite apparent risks—dialect dialogue, actors without large celebrity followings, and a less obviously commercial genre—because it combined distinctive cultural identity, controlled production costs, and emotional resonance with potential to spread.

A new RAG benchmark result

Does SAG work? Zleap AI tested it on three established multi-hop question-answering datasets:

HotpotQA, 2WikiMultiHopQA, and MuSiQue.

All require following connections. MuSiQue is especially demanding, with up to four reasoning hops.

HippoRAG 2 is a strong baseline. Under the same configuration:

• SAG achieved average Recall@2 / Recall@5 of 79.3% / 88.2%.
• HippoRAG 2 achieved 68.2% / 83.3%.

SAG's ability to retrieve key evidence in the first two results was 11.1 percentage points higher. Earlier retrieval can save tokens, reduce latency, and limit divergence in subsequent reasoning.

On MuSiQue, SAG's Recall@5 was 80.0%, compared with 65.1% for HippoRAG 2—almost 15 percentage points higher.

The event-and-entity structure with SQL expansion appears especially useful where more hops are required.

Ablation experiments further support the importance of the structure:

• On MuSiQue, a triple-based SAG variant achieved 77.1% Recall@5, versus 80.0% for hyperedges.
• Disabling query-time expansion reduced Recall@5 from 80.0% to 69.4%.
• Replacing Qwen3.6-Flash with a lightweight reranker for final selection reduced Recall@5 from 80.0% to 62.2%.

The paper also tested sensitivity to embedding models.

With the stronger NV-Embed-v2:

• SAG's MuSiQue Recall@5 changed only modestly, from 80.0% to 81.7%.
• HippoRAG 2 was more sensitive, rising from 65.1% with BGE to 74.6% with NV-Embed-v2.

The underlying structural change, rather than a stronger embedding model alone, accounts for the main improvement.

From benchmarks to production scale

According to Zleap AI, SAG is deployed in a production environment containing approximately 500 million records, with the dataset continuing to grow and online retrieval latency remaining within seconds.

Combining benchmark performance with deployment at that scale is notable.

SAG's low latency at scale comes from dividing the work.

Expensive work happens offline: an LLM extracts structure, converting chunks into events and entities.

Fast work happens online: SQL, vector indexes, and full-text indexes retrieve candidates, and the LLM assesses only a small subset.

SAG is also designed for incremental growth.

Chunks are natural units of parallel processing, and each can be processed independently.

New pages, documents, and projects become new events and entities within the index system without recomputing global relationships.

This supports incremental processing and continuous expansion without repeatedly rebuilding a global knowledge graph.

What happens as the entity count grows? Does merging become difficult?

It can, but SAG does not make perfect entity merging part of its critical path.

In GraphRAG, entities are core graph nodes. Incorrect merging can contaminate the graph; failing to merge can break paths. Entity disambiguation therefore requires substantial and growing engineering effort.

SAG can tolerate some imperfect merging. Entities guide retrieval; events retain complete descriptions of what happened.

If one company appears under several names, the system need not resolve every identity during ingestion. It can first normalize strings and check SQL conservatively: reuse an entity if its source, type, and name match; otherwise insert a new one.

Vector retrieval, full-text search, and LLM reranking can recover related evidence later.

Zleap AI also built a Wikipedia search demo. We asked for films with themes similar to Love Letters to Grandma.

It quickly produced a summary based on more than a dozen results, with evidence cards including To My Dear Granny, Forever Love, and Love Letter.

Try it: https://wiki.zleap.com/search

Opening To My Dear Granny shows why the result was retrieved and the corresponding source evidence.

Results appear on the left; explanations of why each was retrieved appear on the right.

This is SAG's traceability. Agents need to know where answers come from and which clues to follow next.

The View Graph feature is particularly interesting.

Original screenshot.

This is a local evidence network expanded for the current question, rather than a prebuilt knowledge graph.

Nodes represent retrieved events. A query about family films expands around family bonds, letters, family, and memories.

A question such as “Which university did the founder of the company that acquired Instagram attend?” expands around Instagram, Facebook, founders, universities, and their relationships instead.

SAG activates only the local relationships needed when a question arrives, instead of precomputing every relationship in advance. This is central to its scalability and its distinction from conventional RAG and GraphRAG.

Knowledge and memory

An agent data foundation must also support memory.

Beyond external knowledge, agents need to know a user's preferred communication style, a project's current stage, the conclusion of a previous task, and which earlier judgments were superseded by new information.

Storing these as ordinary RAG chunks may retrieve a similar conversation without distinguishing historical background, current state, and obsolete information.

SAG provides a more natural organization.

Each memory can be an event: who did what, when, to which object, and what changed. Related people, projects, tasks, and preferences become connecting entities.

Memory then becomes a continuously writable collection of events that can be retrieved through clues and dynamically expanded for each question, instead of loosely organized conversation history.

The paper acknowledges that long-term agent memory still requires additional versioning and temporal awareness. This is also a promising direction for SAG as an agent data foundation.

From this perspective, SAG points toward a data organization paradigm where knowledge can be continuously added, memories retrieved through relationships, and state changes tracked over time.

That may be an important missing capability in the next generation of agent infrastructure.

References

1. Open-source project: https://github.com/Zleap-AI/SAG
2. Paper: https://arxiv.org/abs/2606.15971
3. Medical AI hallucination paper: Representation Before Retrieval: Structured Patient Artifacts Reduce Hallucination in Clinical AI Systems — https://www.medrxiv.org/content/10.64898/2026.02.13.26346256v1.full.pdf