In January 2025, Aditi Singh and coauthors published a survey titled Agentic Retrieval-Augmented Generation: A Survey on Agentic RAG, revised in April 2026. They describe agentic retrieval augmented generation, or agentic RAG, as a search system with an AI agent inside it. The agent can plan its next step, check its own results, and use other programs, such as a calculator or a search box, while it looks for information. The same survey lists evaluation and efficiency among the open challenges. That is the part I wanted to measure.

Retrieval augmented generation, or RAG, means that a language model, an AI system that writes text, answers a question using text it first retrieved from a collection of documents. RAG with plain search looks the question up once, as written. Agentic RAG adds an agent. A language model reads the first results and decides whether to stop or to write a new search.

More searches look like they should find better evidence, meaning the paper that actually answers the question. Each extra search also needs another language model call, and whoever runs the system pays for it. Each one also makes the user wait longer and gives the agent another chance to write a worse search than the one it started with.

So I ran one test. I designed it and used an AI coding assistant, Claude Code, a tool that writes and runs code from instructions, to build and run it. I tried four ways of searching, with the same 298 questions over the same 2,000 recent arXiv papers. The four ways were these.

• Plain search looks the question up once, as written.

• A reranker makes the same search, then uses a small scoring model to reorder the first 50 results.

• A question rewriter asks a language model to rewrite the question three ways, searches with all four versions, and merges the results.

• An agent makes a search, then lets a language model read the first results. The model decides whether to stop or to write a new search, up to three searches in all.

All four are ways of doing the first half of RAG. RAG has two halves. The first half retrieves text that is relevant to the question. The second half gives that text to a language model, which writes the answer. I tested only the first half, so none of these four writes an answer.

Here is how each one fits. Plain search is the retrieval half of ordinary RAG in its simplest form. The reranker is the same, with a common extra step that reorders the results. The question rewriter is the retrieval half of RAG with question rewriting, another common technique, and it follows the same fixed steps every time. Only the agent is agentic, because it reads the results and chooses whether to stop or to search again.

For each way, I counted how often it put the right paper among the first five results it returned, its top 5. The question rewriter and the agent use a language model, so they do not return exactly the same answer twice. I ran each of them three times.

What the run showed

• The agent put the right paper in the top 5 for 96.87 percent of questions, against 95.64 percent for plain search. That gap is +1.2 percentage points, and the range of plausible values runs from -0.8 to +3.5. A range that includes zero means this test cannot tell the two apart.

• Plain search left the right paper out of its top 5 on 13 questions. The agent found the right paper for 7 to 8 of them in each run, and lost 3 to 5 papers that plain search had found.

• A reranker reorders the first 50 results with a second model and needs no language model call. It put the right paper higher in the list by a clear margin, with a score of 0.959 against 0.895 on the rank measure explained below. It cost nothing in model fees.

• Rewriting the question into three variants and merging the results did worse than plain search, at 93.6 against 95.6 percent, though that gap also sits inside its range.

The short answer to the title is that, on this test, giving the agent more searches did not clearly find better evidence. It found some papers plain search missed and lost some plain search had found, and the gain it left was too small to tell from chance. The one clear improvement came from the reranker, a cheap step that reordered the first 50 results and moved the right paper higher.

Everything below shows how I got there, what I changed along the way, and where the result stops being reliable.

How should you test whether a search agent finds better evidence?

I needed to measure the searching alone, so the quality of a final written answer could not hide it. The test therefore stops one step before the answer. For each question, there is exactly one paper whose abstract, its short summary, answers it. A way of searching succeeds if that paper appears in the first five results. From here on I call each way of searching a design.

Here are the four designs, named dense, rerank, multiquery, and agent. The first one, dense, is the plain search and the baseline, which is the version the others are compared with. Each of the others adds one thing. The one that rewrites the question first is multiquery, and the one that can search again is agent.

Dense is the plain search and the baseline. It turns the question into an embedding, which is a list of numbers that represents what the text means. Every abstract has its own embedding too, made beforehand by the same model. The search returns the abstracts whose numbers sit closest to the question’s, much like finding the nearest points on a map.

It makes one search and calls no language model. I used the embedding model BAAI/bge-small-en-v1.5, which is released under the MIT license.

Rerank is the same search with one extra step. The dense search first returns its top 50 abstracts. A second model, called a cross encoder, then reads the question and each abstract together and gives each pair a score for how well they fit. The 50 are reordered by that score, so the abstract that fits best moves to the top.

It reads each pair in full, so it runs only on the 50 and not on all 2,000 abstracts. I used cross-encoder/ms-marco-MiniLM-L-6-v2, released under Apache 2.0. It is a small scoring model that never writes text.

Multiquery is the search with rewriting. Claude Haiku 5.5, a small and fast model from Anthropic, writes three different versions of the question. The system searches with the original and the three versions, which gives four ranked lists. It merges the four lists into one with reciprocal rank fusion, or RRF.

RRF gives each paper 1 divided by (60 plus its position) for every list it appears in, and adds the scores up. A paper in position 1 of a list earns 1/61, and a paper in position 10 earns 1/70. A paper that sits high in several lists therefore ends up on top. Cormack, Clarke, and Büttcher introduced it in 2009 and fixed the constant at 60.

Agent is the search that can try again. The first search uses the question as written. Claude Haiku 5.5 then sees the titles of the top 5 results and replies either “stop” or with a new query, which is the words to search for. It can search at most three times. The lists from all its searches are merged with the same RRF setting, so it ends with one ranked list like the others.

A librarian is a fair picture of the agent. Ask a plain question and a plain search returns what sits nearest on the shelf. A librarian who rephrases it, perhaps using the words researchers in that area actually use, may find the right book. The same librarian may also guess words from the wrong area and walk away from the right shelf. Which of those happens more often is what I measured.

One fairness point matters here. The agent and multiquery search more than once, and dense searches once. The results table below prints the average number of searches next to every design, so more searching is visible and not hidden.

What exactly were the questions and the papers?

The test is only as good as its questions and its collection, so here is how I built both, and what I checked.

Where did the papers come from?

I collected 2,000 papers from arXiv, the open server where researchers post papers before peer review. They come from four categories. cs.AI is for artificial intelligence, cs.CL for computation and language, cs.IR for information retrieval, and cs.LG for machine learning.

The papers were submitted on 92 different days between 1 July and 30 September 2026. I chose that window because Anthropic’s model overview lists a June 2026 training data cutoff for the models I used. That is the last date their training data covers, so the papers should not be in it. The title and abstract of each paper are the only text any design could see.

arXiv’s terms of use for its programming interface allow its descriptive metadata to be used under the Creative Commons CC0 1.0 public domain declaration. The terms describe descriptive metadata as including fields such as title, abstract, authors, identifiers, and classification terms. That covers the titles and abstracts I used. The papers’ full texts carry their authors’ own licenses, and I did not use them.

How were the questions written?

Before running any search, I picked 300 papers at random to write questions about. I call them source papers. For each one, Claude Sonnet 5.5 read the title and abstract and wrote one question that the paper answers. The instructions told it never to copy the title or a stretch of four or more words from it.

The questions rotate through three styles. A specific question names the method or setting in Claude Sonnet 5.5’s own words. A problem question describes the problem without naming the method. A vague question is short and imprecise, like something half remembered.

The table shows three of the 298 questions, one of each style. Each is the first of its style in my list, so I picked them by a fixed rule and not because they looked good. For each one, it gives the paper the question was written from and the place that paper came in when plain search ran, where 1 means first. All 298 questions are in the repository.

I set the rules, and Claude Sonnet 5.5 wrote each question from a paper’s abstract. Each question was then checked against that abstract, as the section on checking below describes. The questions here are long and detailed. Even the one labeled vague is 28 words and names specific ideas, such as “two separate measures” and “ROC curves”. A person typing into a search box might write only a few words, such as “clustering evaluation ROC”, which gives a search much less to work with.

What are the lookalike papers?

In a small trial run first, which I call the pilot and describe below, plain search was too good, so I added a harder condition. For each source paper, I put its five most similar papers into the collection, measured by shared words (term frequency inverse document frequency, or TF IDF). I used shared words and not the embedding model, so the lookalike papers were not chosen by the same model that does the searching.

These neighbors are only loosely similar. The median top neighbor had a similarity of 0.15 on a scale where 1 means identical. That means the neighbors share some topic words but are far from copies. This test is therefore harder than a random collection and still resembles ordinary lookalike papers on the same topic.

How were the questions checked?

A question can be unfair if the abstract does not actually answer it, or if another paper answers it just as well. The coding assistant first read every question against its source abstract and the three nearest other papers and marked each one pass or fail. I then spot checked 30 of the passes against their abstracts and found no unsupported answers. That check covers 30 questions, and the other 268 rest on the assistant’s pass alone.

Two questions failed because they asked for a result the abstract does not report, which left 298. For example, one asked how much labeled data is needed, which the abstract never states. Their IDs are q064 and q088 in the repository.

What did the pilot change?

I ran a small pilot first, because a test is useless if the plain search already gets everything right. The pilot used 300 papers and 20 questions, of which 17 passed review.

Plain search put the right paper at rank 1, which is first place, for all 17 of them. No design could have looked better, so the pilot could not separate anything. That is why I added the lookalike papers and grew the collection to 2,000.

The pilot also exposed a mistake in my first agent. It merged its searches by keeping each paper’s best raw similarity score. On one pilot question, a rewritten query pushed the right paper from rank 1 down to rank 4. The reason is that a score of 0.8 for one query does not mean the same as 0.8 for another. I changed the agent to merge by position with RRF, the same setting multiquery uses.

I did not tune the constant of 60, and I did not try other values.

Here is everything that changed along the way, and what triggered each change.

• The collection grew and gained lookalike papers, because the pilot was too easy.

• The agent’s merge changed from best score to RRF, because of the pilot question above.

• Three questions changed after review and before any search that calls a language model ran. Two repeated four or more words of their paper’s title, so I regenerated them. One question’s source paper had also been a pilot source, so I replaced it with another paper.

• Five questions initially came back without a question because Claude Sonnet 5.5 replied with plain text, not the format I expected. I accepted the plain text, so all 300 got a question.

After I saw the plain search and reranker results, I read again all 13 questions that plain search missed, along with the papers it returned instead. I changed no question, and I removed none because of a result. I also did not drop the questions that plain search already ranked first. Removing them would have meant choosing the test after seeing the answer.

How do I score a design, and how much should I trust a number?

Two measures carry the results, and both are higher when the design is better.

Recall at 5 (recall@5) is the share of questions where the right paper appears in the top 5. If it is 95 percent, then for 95 of 100 questions a reader looking at five results would see the right paper.

Mean reciprocal rank at 10 (MRR@10) rewards putting the right paper first. A paper at rank 1 scores 1, rank 2 scores one half, rank 5 scores one fifth, and a paper outside the top 10 scores 0. The score is the average over questions.

A single average hides how much it could move with a different set of questions. So I report a range for each difference. Imagine drawing 298 questions from the pile again and again, putting each one back after it is drawn, and recomputing the difference each time. I did that 2,000 times and took the middle 95 percent of the results. This is a bootstrap range, and it shows how much the answer moves when the set of questions changes.

The difference is paired, meaning each design is compared with plain search on the same questions. When a range includes zero, the data are compatible with no real difference.

The two designs that call a language model, multiquery and agent, ran three times each. Dense and rerank return the same result every time, so they ran once. Claude Haiku 5.5 does not let me set the temperature. That setting is the control that makes a model repeat itself more or less exactly, so its replies vary a little from run to run.

I turned off its thinking option, where the model reasons at length before it answers. That way the reported cost and the delay, which is the time from the question to the ranked list, reflect the retrieval design alone, with no hidden reasoning added. Claude Sonnet 5.5, which wrote the questions, does not allow thinking to be turned off, so I used its lowest thinking setting instead.

What did the four designs score?

The table below answers the question the whole test was built for, which is whether more searching finds the right paper more often. The multiquery and agent rows show the mean over three runs, and the brackets show the lowest and highest run. Differences in this article are computed from the unrounded values, so a gap can differ by 0.1 from subtracting the rounded numbers shown.

Image by author. Each dot is one design’s score on the 298 questions, which were fixed before the multiquery and agent runs, from the saved result files. Higher is better on both charts, and the short lines mark the lowest and highest of three runs for multiquery and agent. The reranker and the agent sit above plain search on both measures, and multiquery sits below.

Plain search already finds the right paper in 95.6 percent of questions, and it ranks it first for 253 of 298. That leaves 13 questions where an extra search could help. Recall can rise by at most 13 of 298 questions, which is 4.4 points, so no design can gain more than that on recall. I come back to this under the limits.

Which differences are real?

This section matters because a higher number in the table does not mean a better design. The chart shows each design’s change from plain search, with its range.

Image by author. Each dot is the average change from plain search over the three runs, on the same questions, and each line is the 95 percent bootstrap range. A line that crosses the dashed zero line means the test cannot separate that design from plain search. Only the reranker’s gain in MRR@10 clears zero.

• The reranker gained +1.3 points on recall@5 (range -0.7 to +3.7), which includes zero. On rank quality, which is MRR@10, it gained +0.064 (range +0.037 to +0.093), which clears zero. It is the only clear improvement in the test. It moves the right paper higher, and that costs one extra scoring pass and no model fees. It also adds about 1.5 seconds of delay on my laptop.

• The agent gained +1.2 points on recall (range -0.8 to +3.5). On rank quality it gained +0.011 (range -0.006 to +0.029). Both ranges include zero. The honest reading is that the agent may be a little better, may be no different, and may be slightly worse, and this test cannot say which.

• Multiquery changed recall by -2.0 points (range -4.6 to +0.4), and on rank quality by -0.026 (range -0.057 to +0.002). Both ranges include zero, but in every run it lost 11 to 13 of the 285 papers plain search had found, which also suggests it is worse.

The three runs differ by 0.3 points for agent and 1.3 points for multiquery on recall. Both are smaller than the 4 to 5 points of width in the ranges above. That width comes from having 298 questions, so repeating the runs more would not change the picture. More questions would.

Where did the agent help, and where did it hurt?

The averages hide what actually happened inside the searches, so this section looks at specific questions. I saved every query the agent wrote, and the queries quoted below are exactly as it wrote them, including the technical shorthand.

Where did the agent help?

The agent turned plain wording into the words researchers in that area actually use. One question asked how to send incoming requests, in real time, to several servers that generate text, each with limited capacity, while balancing response time against total output. Plain search did not return the right paper in its top 10. The agent’s second query was SLO-aware request routing across decode replicas with KV cache capacity constraints optimization, and the right paper moved to rank 1 in all three runs.

Another question asked for a way to make a byte pair encoding vocabulary more efficient by hiding rarely used intermediate pieces. That vocabulary is the list of word pieces a model reads text in. Plain search put the right paper at rank 6. The agent rewrote it as BPE vocabulary pruning removing unused merge tokens reallocating vocabulary slots, and the paper moved to rank 1 or 2.

Where did the agent hurt?

The agent also added words that were not in the question, guessed from its memory of the field. One question asked for a language model design that feeds its own last internal state back in as its next input. Plain search ranked the right paper first. The agent’s second query added the words “continuous thought,” and its third added the name “Coconut,” which the question never used.

The results for those queries were other papers about models that reason in internal numbers and not in words, and they were merged above the right paper. It slipped from rank 1 to rank 5 and rank 4 in two runs, which is still inside the top 5. In the third run it fell out of the top 10, and that is the only run where it counts as lost.

Another question asked how to test whether the internal memory of a learned reinforcement learning update rule helps or hurts. Plain search had the paper at rank 5. The agent’s third query added “LSTM” and “learned optimizer.” These are words from other research areas, and the question never used them. The paper ended outside the top 5 in all three runs.

This is my reading of the 16 questions where the result changed, not a measured effect. Rewrites that borrowed words from the question’s own research area helped. Rewrites that guessed words from the agent’s memory of the area often named something else. The merge then placed those other papers above a paper plain search had already found.

Only 2 questions were lost in all three runs. The other 6 were lost in just one of the three runs, which is ordinary variation between runs. That is 12 losses in total, split 4, 3, and 5 across the three runs.

How often did the agent search?

The agent stopped after one search on about 59 percent of questions, after a single model call to decide. It used all three searches on about 26 percent. On 7 to 8 questions per run it used all three and still missed. Five questions were never found by plain search or by any agent run.

What does this not show?

A result without its limits is easy to overread, so here are the limits, in the order I would worry about them.

There is little room to improve. Plain search is right on 285 of 298 questions, so only 13 questions can show a gain on recall. The ranges above reach about 2 to 3 points on either side of each estimate, so a smaller real gain would not show. A harder collection, or many more questions, would be needed to detect a small real benefit.

It is one collection and one kind of question. The papers are recent AI preprints from one quarter, and a model wrote each question from the abstract. Questions written from an abstract share its wording, which helps any search that matches words. Every question here has between 14 and 54 words, with a median of 32. A real user’s questions would be messier and may be much shorter, which gives a search less to work with. I make no claim about enterprise documents, other languages, or longer texts.

The agent is one model with one set of instructions. I wrote the agent’s instructions once and did not tune them. A stronger model, better instructions, or a different merge could change the result in either direction.

The review covered only part of the questions. The coding assistant did the first pass over all 300 questions, and I checked 30 of them. Some unfair questions may remain. The questions plain search misses are the ones that matter most here, and I read those 13 again by hand.

There are a few smaller issues. One replacement question has no lookalike papers planted next to it and may be easier. Of the pilot’s source papers, 12 also appear in the full collection, so the pilot and the full run overlap. The test also measures retrieval only, so it says nothing about whether the final written answer is better.

What would I do with this?

The finding is “no clear improvement on this collection, at the cost of extra model calls and delay”. That is the useful result, because it tells you where not to spend first.

If you run a RAG system, count first. Take a few hundred real questions, run plain search, and see how many it misses. If it misses very few, an agent has little to fix. Plain search missed 13 of my 298 questions.

Add a reranker before you add an agent. It was the only change here with a clear gain, and it needs no language model calls, though it adds about 1.5 seconds of delay on my laptop. Then, if you add an agent, read the queries it writes. Cap the number of searches, merge by position and not by raw score, and watch for new terms that were not in the user’s question. In the examples I read, those added words were where papers were lost.

I left one question open. The reranker gave the clearer gain for free. What would have to be true of your questions for an agent’s extra searches to be worth their delay and their failures?

Can you run the test yourself?

The code, the saved result files, and the question set are in the companion repository on GitHub. Everything below runs from the top folder of that repository in a terminal. I ran it with Python 3.9.

Setup installs four packages and needs an Anthropic API key. The key is used by Claude Sonnet 5.5, which wrote the questions, and by Claude Haiku 5.5, which runs the question rewriter and the agent. Plain search and the reranker need no key and ran on my laptop.

python3 -m venv .venv && source .venv/bin/activatepip install anthropic arxiv numpy sentence-transformersexport ANTHROPIC_API_KEY=your_key_hereMODE=full python run_eval.py --designs dense,rerankMODE=full python run_eval.py --designs multiquery,agent --out results/full_run1.jsonlMODE=full python summarize_runs.py

These commands use the saved papers and my reviewed question file, so you score exactly the 298 questions in this article. To build a new collection and question set, run MODE=full python collect_papers.py and MODE=full python make_questions.py first, then mark each question 1 or 0 in the ok (1/0) column of data_full/questions_review.csv, because only questions marked 1 are scored. A new set will differ from mine. The repository already contains my saved results, and run_eval.py skips rows it already has, so delete results/full.jsonl first if you want the commands above to recompute them.

Two pieces of code carry the main idea. The first is the merge step.

def rrf(rankings: List[List[str]], k: int = config.RRF_K) -> List[str]:    """Reciprocal-rank fusion of several ranked id lists."""    score: Dict[str, float] = {}    for ranking in rankings:        for rank, doc in enumerate(ranking, start=1):            score[doc] = score.get(doc, 0.0) + 1.0 / (k + rank)    return [d for d, _ in sorted(score.items(), key=lambda t: -t[1])]

The second is the agent loop. It makes the first search with the question as written and shows the model the top 5 titles. The model then either stops or writes a new query, up to the search limit.

def retrieve(self, question: str) -> Result:    t0 = time.time()    res = Result([], searches=0)    rankings: List[List[str]] = []      # one ranked id list per search, fused with RRF at the end    history: List[str] = []    query = question    while res.searches < config.AGENT_MAX_SEARCHES:        hits = self.c.search(query, config.RERANK_CANDIDATES)   # same depth as multiquery        res.searches += 1        rankings.append([d for d, _ in hits])        shown = hits[:config.K_FINAL]                    # the model sees the top 5 titles        history.append(f'Search {res.searches}: "{query}"\n' +                       "\n".join(f"  - {self.c.title[d]}" for d, _ in shown))        res.trace.append({"query": query, "top": [d for d, _ in hits[:config.RANK_DEPTH]]})        if res.searches >= config.AGENT_MAX_SEARCHES:            break        system = ("You are a research assistant finding the single arXiv paper that answers a "                  "question. After each search decide whether the results contain it. "                  'Reply with only JSON: {"decision": "stop"} or '                  '{"decision": "search", "query": "<a new, different query>"}.')        user = f"Question: {question}\n\n" + "\n\n".join(history)        text, usage = llm.complete(config.AGENT_MODEL, system, user, max_tokens=150)        _add(res, usage)        parsed = llm.extract_json(text) or {}        nxt = parsed.get("query") if parsed.get("decision") == "search" else None        if not isinstance(nxt, str) or not nxt.strip() or nxt.strip() == query:            break        query = nxt.strip()    # Scores from different queries are not comparable (pilot q005: a rewrite pushed the right    # paper from rank 1 to rank 4 under a best-score merge), so searches are fused by rank,    # with the same RRF setting as multiquery.    res.ranked_ids = rrf(rankings)[:config.RANK_DEPTH]    res.latency_s = time.time() - t0    return res

The prices I used are list prices copied from the provider’s pricing page on 15 October 2026. Claude Sonnet 5.5 cost about $0.60 for writing all questions, retries, and fixes. The three runs of multiquery and agent cost $0.14 together, and the pilot cost under $0.05. The whole project cost under $0.80.

Selected sources

[1] A. Singh, A. Ehtesham, S. Kumar, T. Talaei Khoei, A. V. Vasilakos, Agentic Retrieval-Augmented Generation: A Survey on Agentic RAG (2025, revised 2026), arXiv. Source for the definition of agentic RAG and the open challenges named in the opening.

[2] G. V. Cormack, C. L. A. Clarke, S. Büttcher, Reciprocal rank fusion outperforms Condorcet and individual rank learning methods (2009), Proceedings of the 32nd International ACM SIGIR Conference on Research and Development in Information Retrieval. Source for the merge method and its constant of 60.

[3] S. Xiao, Z. Liu, P. Zhang, N. Muennighoff, C-Pack: Packaged Resources To Advance General Chinese Embedding (2023), arXiv, and the model card for BAAI/bge-small-en-v1.5, released under the MIT license. The embedding model used by every design.

[4] cross-encoder/ms-marco-MiniLM-L-6-v2, Hugging Face model card, released under Apache 2.0. The reranking model.

[5] arXiv, Terms of use for arXiv APIs. The terms allow descriptive metadata, including title and abstract, to be used under CC0 1.0, and ask for no more than one request every three seconds. Used for the titles and abstracts in the collection.

[6] Anthropic, Models overview and thinking documentation. Source for the June 2026 training data cutoff, the prices, and the thinking and temperature settings of Claude Sonnet 5.5 and Claude Haiku 5.5.

[7] agentic-rag-evidence-test, GitHub repository for this article (2026). The code, the 298 reviewed questions, and the saved results for every run in this article.