How AI Search Works

WebSpeaker’s AI Search combines modern natural language processing techniques to deliver grounded, context-aware answers to visitor questions. Rather than simple keyword matching, the system understands the meaning behind questions and retrieves the most relevant content from your knowledge base to generate helpful responses.

Semantic Search and Keyword Matching

The search system uses a hybrid approach that combines two complementary methods. Semantic search uses vector embeddings to understand the meaning of both the query and your content. This means that even if a visitor phrases their question differently from how the information appears on your website, the system can still find the relevant content based on conceptual similarity. For example, a question about “pricing plans” will match content about “subscription tiers” because the underlying meaning is similar.

Keyword matching complements the semantic search by ensuring that exact terms and phrases are also considered. This is particularly important for product names, technical terms, and specific identifiers that should be matched literally. The two methods work together: semantic search handles conceptual understanding while keyword matching ensures precision for specific terms.

The RAG Pipeline

WebSpeaker uses a Retrieval-Augmented Generation (RAG) pipeline to produce its answers. RAG is an approach that combines information retrieval with language model generation, ensuring that responses are grounded in your actual website content rather than the model’s general training data. The pipeline works in several steps:

  1. User Query — A visitor types a question into the widget. The system processes the query and prepares it for retrieval.

  2. Retrieval — The system searches your knowledge base using both semantic and keyword matching to find the most relevant content chunks. These chunks are ranked by relevance and the top matches are selected.

  3. Context Assembly — The retrieved content chunks are assembled into a context window that provides the language model with the specific information needed to answer the question. This context includes the extracted text from your website pages along with metadata like page titles and URLs.

  4. LLM Generation — A large language model receives the assembled context along with the visitor’s question and generates a natural-language response. The model is instructed to base its answer strictly on the provided context, avoiding speculation or information not present in your content.

  5. Response with Sources — The final response is presented to the visitor along with source citations. Each citation links back to the original page on your website where the information was found, allowing visitors to verify the answer and read more if needed.

Multi-Turn Conversation Memory

AI Search does not treat every visitor message as an isolated question. The assistant keeps track of what has already been said in the conversation, so it can understand follow-up questions that depend on earlier context. For example, if a visitor asks about a specific topic and then follows up with “what about the pricing for that?”, the assistant understands that “that” refers to what was already discussed, instead of needing the visitor to repeat the full question from scratch.

This means visitors can have a natural back-and-forth conversation — asking a question, getting an answer, then narrowing it down or asking a related question — without losing context along the way.

Richer Source Metadata

Beyond the text extracted from your pages, WebSpeaker’s content analysis also records metadata about each crawled page, such as what kind of page it is (for example documentation, a pricing page, a news update, or a contact page) and its role within your site’s structure (for example a general overview page versus a more specific, detailed page).

The assistant uses this metadata alongside semantic and keyword matching to judge which retrieved source best fits a visitor’s question — for example, favoring a broad overview page for a general question, or a specific detail page when the question is narrower. This metadata works behind the scenes; visitors still see the familiar source links described above, chosen with this extra context in mind.

Why RAG Matters

The RAG approach is what makes WebSpeaker’s responses reliable and trustworthy. Unlike a general chatbot that might generate plausible-sounding but incorrect answers, WebSpeaker grounds every response in your actual content. The source citations provide transparency, letting visitors see exactly where the information came from. This is especially important for businesses where accuracy matters, such as product documentation, legal information, or technical support.

AI Transparency and Human Replies

WebSpeaker clearly distinguishes between AI-generated answers and messages written by your support team. This helps visitors understand whether they are reading an automated answer based on your knowledge base or a response from a human agent.

When AI Search answers a question, the widget shows that the reply comes from the AI assistant and, when available, displays source links to the pages used to generate the answer. This makes it easier for visitors to verify the information and continue reading on the original page.

When Live Support replies, the widget shows the response as a human message from your support team member. This makes the handoff between AI assistance and human support explicit, especially when both features are enabled in the same widget.

This transparency improves trust. Visitors can quickly see whether they are interacting with AI or a person, and your team can use AI Search for instant answers while still stepping in when human help is needed.

Content Quality and Search Quality

The quality of AI search results is directly tied to the quality of your scraped content. Clean, well-structured content produces better embeddings and more relevant search results. This is why proper scraping configuration is important: targeting the right content areas, excluding irrelevant elements, and keeping your knowledge base up to date all contribute to a better search experience for your visitors.