- Requirement analysis, system design, hybrid retrieval implementation, evidence-control logic, structured output, citation validation and automated testing.
- RAG · Knowledge Base · Enterprise Search · Troubleshooting
- 368 tests passed · 85.08% test coverage · ~60 frontline users
Problem
A knowledge-base assistant that lets frontline staff query equipment manuals, commissioning documents and fault records in plain language.
The system is designed to reduce the cost of finding equipment information, not to replace on-site engineering judgment.
Equipment commissioning on site depends on experienced technicians. Their knowledge is valuable and hard to transfer.
The goal was a single entry point where frontline staff can ask questions about equipment in plain language — for example, “The equipment shows alarm XX after start-up — what should I do?”
- A shortage of experienced technicians on site.
- Limited training for new staff.
- Equipment documentation scattered across many sources.
- Senior staff experience that is hard to capture.
- High cost of looking up a fault when it happens.
End-to-end ownership of requirement analysis, solution design, RAG pipeline implementation, evidence-control logic and automated testing for the equipment knowledge assistant.
Build
The assistant combines vector retrieval and keyword retrieval to search equipment knowledge from different perspectives. FAISS handles semantic similarity while BM25 preserves exact technical terms, fault codes and equipment-specific keywords.
Retrieval results are fused with Reciprocal Rank Fusion (RRF) before being passed through an evidence-control layer. The system only generates an answer when the retrieved evidence is sufficient.
Structured output and citation validation are used to keep answers traceable to the underlying equipment documents.
Knowledge sources
- Equipment Manuals设备说明书
- Commissioning Documents调试文档
- Fault Records故障记录
- Operating Procedures操作规范
- User Query
- Query Processing
- FAISS Vector RetrievalBM25 Keyword Retrieval
- RRF Fusion
- Retrieved Evidence
- Evidence Gate
- LLM Generation
- Structured Output + Citations
- Answer / Fail-closed
The assistant searches across equipment manuals, commissioning documents, fault records and operating procedures to assemble relevant evidence for each query.
- Query
- Hybrid Retrieval
- RRF
- Evidence Gate
- Answer
- 02› FAISS semantic search + BM25 keywords (fault codes, models)
- 03› RRF fuses both result lists
- 04› Evidence sufficient → generate
- 05› Structured answer with citations
Query
Q: alarm XX after start-up — what should I do?
Hybrid Retrieval
FAISS semantic search + BM25 keywords (fault codes, models)
RRF
RRF fuses both result lists
Evidence Gate
Evidence sufficient → generate
Answer
Structured answer with citations
Q ›The equipment shows alarm XX after start-up — what should I do?
Retrieved evidence
- Equipment manual
- Fault records
- Commissioning docs
answerable→ LLM, answer with citations
- 01
Hybrid Retrieval
Semantic similarity alone is not enough for equipment data. Fault codes, model numbers and technical terminology also require exact keyword matching.
- 02
Fuse Before Generation
FAISS and BM25 results are combined with RRF before evidence is passed to the model.
- 03
No Evidence, No Confident Answer
When retrieved evidence is insufficient, the system follows a fail-closed path instead of forcing the LLM to generate an answer.
- 04
Traceable Output
Structured output and citation validation make it possible to trace the answer back to retrieved knowledge.
Engineering
- Hybrid retrieval with FAISS and BM25
- Reciprocal Rank Fusion (RRF)
- Evidence Gate before generation
- Structured Output
- Citation Validation
- Fail-closed handling for insufficient evidence
- Automated regression testing
- tests passed · 1 skipped
- 368
- test coverage
- 85.08%
- frontline users
- ~60
Delivered a working equipment knowledge assistant for frontline information retrieval, with hybrid search, evidence control and citation-aware responses.
Used as a knowledge-query tool for approximately 60 frontline employees.
Notes
In enterprise RAG, the knowledge sources matter more than the model. Deciding what goes in is most of the work.
The real user is someone on site with limited time. The interface has to respect that.
- RAG
- FAISS
- BM25
- RRF
- Knowledge Base
- Enterprise Search
- LLM
- Troubleshooting
- Knowledge Management