Get in touch

A chatbot that answers only from your documents

OrionSoft's enterprise conversational agent sends every question down the safest path, answers with citations, calculates exactly, and turns website visitors into booked discovery calls.

Router choosing a path for each question Sample

"Which chatbot services does OrionSoft offer?"

internal_rag Sent to hybrid retrieval
Top 3 chunks reranked, evidence is sufficient
Answer written with citations

"Budget is $184,500. Cut spend by 17%. What is the net budget?"

calculator Never reaches the language model
184,500 × (1 − 0.17) = $153,135

"Do you offer a 40% lifetime discount?"

internal_rag Evidence check: insufficient
abstain "The available company documents do not specify this."
100% routingAccuracy across 15 benchmark cases
1,137 msAverage end-to-end response
100% hit rateCorrect source in the top 3, across 29 queries
100% blockedConfidential terms kept out of web search

Most chatbots shine in a demo, then fail in production

Moving from prototype to a system customers can rely on means solving five problems at the same time.

The problemWhat goes wrongWhat we built

Made-up answers

When documents are unclear, language models fill the gaps. One wrong answer on pricing or policy becomes a liability.

An evidence check decides if the material is strong enough, and the agent says so when it is not.

Data leakage

An agent with web search can send confidential terms, such as salaries or budgets, to third-party search APIs.

An anti-leak gate keeps confidential queries away from external tools.

Retrieval blind spots

Semantic search misses exact codes and acronyms. Keyword search misses meaning and paraphrasing.

Hybrid retrieval uses both, then a reranker removes false positives.

Unreliable maths

Models return numbers that sound right but are wrong, especially with percentages and multi-tier discounts.

A deterministic engine solves the maths, so the model never guesses.

Lead drop-off

Contact forms attract spam and get abandoned, and ordinary chat widgets rarely verify who they talk to.

Visitors verify themselves in chat and book a call in one click.

Every question is routed to the safest path first

An autonomous router classifies each question into one of five execution paths. Pick a path to see what it does.

Company knowledge

Sends the question to the hybrid dense and keyword retrieval pipeline. internal_rag

  • Searches by meaning and by exact terms
  • Reranks results before the model writes anything
  • Attaches citations to the answer

The anti-exfiltration gate sits in front of every path. Any query that touches internal employee policies, compensation data or other confidential keywords is barred from reaching external search tools. That closes the door on prompt-injection attacks designed to leak private information.

Hybrid retrieval, then a reranker, then an evidence check

No single search method is good enough, so every document is indexed two ways and the results are cleaned up before the model writes a word.

1

Search by meaning

A dense vector index (384 dimensions) captures meaning and context.

2

Search by exact terms

A BM25 keyword index catches acronyms, product codes and identifiers.

3

Merge the results

The top results from both searches are combined.

4

Rerank

A cross-encoder reads the question and each chunk together and removes false positives.

5

Check the evidence

Is the material strong enough to answer from?

Answer with citations

Only then does the model write a reply.

Exact where it can be, honest where it cannot

Two guardrails that stop a confident-sounding wrong answer from ever reaching a customer.

Exact maths, not estimates

"If our cloud budget is $184,500 and we reduce spend by 17%, what is the net budget?"

184,500 × (1 − 0.17) $153,135
  • The question is parsed into an isolated Abstract Syntax Tree
  • Solved with approved operations only, with no use of eval() or exec()
  • It never reaches the language model
Honest "I don't know"

An evidence evaluator scores the retrieved chunks on three signals before any answer is written.

Query-term overlap Reranker confidence Agreement between sources
Sufficient

The agent answers, with citations.

Insufficient or conflicting

The agent replies that the available company documents do not specify this information.

From question to booked discovery call

The same widget handles support, sales and scheduling, so visitors do not have to leave the chat or fill in a form.

1

Verify the visitor

The visitor shares their name, company and email. Syntax, disposable domains (30+ providers blocked) and live DNS are checked, and a one-time code confirms the address.

2

Notify sales

Qualified inquiries go straight to the sales leadership inbox with an AI conversation summary.

3

Book in one click

When the chat shows buying intent, the agent saves the record and creates links for Google Calendar and Outlook or Office 365, plus a standard .ics invite.

Measured, not claimed

The pipeline was tested with automated suites: 15 real-world scenario prompts for routing, and 29 domain queries for retrieval.

Routing and decisions (15 test cases)

MeasureAchievedTarget

Overall pass rate

100% (15/15)

90% or higher

Route accuracy

100%

95% or higher

Tool selection accuracy

100%

90% or higher

Unnecessary tool rate

0.0%

5% or lower

Correct refusals

100%

100%

Calculation accuracy

100%

100%

Structured output validity

100%

95% or higher

Average end-to-end latency

1,137.14 ms

1,500 ms or lower

Retrieval quality (29 queries, top 3 results)

MeasureScoreWhat it means

Hit rate @ 3

100%

The correct source appeared in the top 3 for every query.

Mean reciprocal rank

1.0000

The most relevant chunk ranked first across all 29 queries.

Precision @ 3

100%

No irrelevant context was pushed into the model's prompt.

Recall @ 3

100%

All the ground-truth evidence was captured.

Honest trade-offs behind the build

Good architecture is a series of choices with a cost. These are the three biggest, and why each was worth it.

DecisionThe trade-offWhy it was worth it

Local models instead of cloud API embeddings

The embedding and reranker models inside the Docker image add about 1.2 GB to the build.

No third-party rate limits, retrieval under 150 ms, no external data transmission and reproducible offline results.

A deterministic maths parser instead of LLM function calling

It needs dedicated parsing grammars for natural-language maths expressions.

Zero arithmetic errors, hundreds of milliseconds saved per question and no code-injection surface.

Dual-write storage (SQLite and Supabase PostgreSQL)

It adds some write-coordination logic across local and cloud storage.

If the cloud connection drops, leads, chats and bookings keep saving locally, with no data loss.

Python 3.11 and Flask FAISS and BM25Okapi Sentence-Transformers React 19 and TypeScript Supabase PostgreSQL Resend

Live on your site with one script tag

The widget runs in an isolated iframe, so it never clashes with your site's stylesheets.

Example <script src="https://your-domain.com/widget-loader.js" async></script>
  • Works on any site. WordPress, Webflow, React or a custom build.
  • Fast enough for live chat. Responses average about 1.1 seconds end to end.
  • Private by design. Retrieval models run locally, with no external data transmission.
  • Keeps working offline from the cloud. Leads and bookings save locally if the cloud connection drops.

A chatbot that knows when to stay quiet

Book a technical discovery call and see the router, retrieval and guardrails on your own documents.