ZeroSearch: Enhancing LLM Reasoning Abilities Through Simulated Search

ZeroSearch: A New Approach to Improving the Search Capabilities of LLMs

Large language models (LLMs) have made impressive progress in recent years in areas such as text generation and translation. However, for more complex tasks that require in-depth knowledge and logical reasoning, access to external information sources is essential. The integration of search functionalities into LLMs is therefore an active research area. A promising approach is the use of Reinforcement Learning (RL) to improve the search capabilities of LLMs through interaction with search engines.

However, previous RL-based methods encounter two central challenges: The quality of documents returned by search engines is often unpredictable, and the API costs for the numerous search queries during RL training are substantial. These limitations hinder the scalability and broad application of such approaches.

ZeroSearch, a novel RL framework, offers a solution to these problems. Instead of relying on real search engines, ZeroSearch trains LLMs to generate their own "search results." The process begins with supervised fine-tuning to transform the LLM into a query module that can generate both relevant and irrelevant documents. Subsequently, curriculum-based RL training is employed. The quality of the generated documents is gradually decreased to confront the LLM with increasingly difficult scenarios and promote its ability to reason logically.

How ZeroSearch Works in Detail

The core principle of ZeroSearch is to replace the interaction with real search engines by simulating search results within the LLM itself. This is done in two phases:

1. Supervised Fine-tuning: The LLM is trained to generate both relevant and "noisy" documents for a given query. This step lays the foundation for the subsequent RL phase.

2. Curriculum-based RL Training: In this phase, the LLM learns to use the generated documents effectively. The difficulty is increased by gradually reducing the quality of the generated documents. The LLM is thus trained to extract relevant information and draw sound conclusions even under unfavorable conditions.

Results and Advantages of ZeroSearch

Experiments show that ZeroSearch significantly improves the search capabilities of LLMs. Even a 3B LLM as a query module achieves remarkable results. A 7B module achieves performance comparable to real search engines, while a 14B module even surpasses them. Moreover, ZeroSearch shows good generalizability across different model sizes and RL algorithms and is compatible with both base and instruction-tuned models.

The advantages of ZeroSearch can be summarized as follows:

- Elimination of dependence on external search engines and associated API costs - Control over the quality of training data by generating documents within the model - Scalability by avoiding external API calls - Improved robustness and generalizability of the trained LLM

ZeroSearch represents a promising approach to improving the search capabilities of LLMs. By avoiding dependence on real search engines and precisely controlling the training data, ZeroSearch enables efficient and scalable optimization of LLMs for more complex tasks that require in-depth knowledge and logical reasoning.

Bibliography: arxiv.org/abs/2505.04588 paperreading.club/page?id=304020 huggingface.co/papers www.chatpaper.ai/papers sunhaopku.github.io/ deeplearn.org/ chatpaper.com/chatpaper/?id=3&date=1746633600&page=1 huggingface.co/Tommy930/activity/all arxiv.org/html/2503.05592v1