Answers need a traceable context
A departmental knowledge assistant needs to find relevant material and recognize when that material does not support an answer. I built Course Intelligence RAG around that retrieval-to-answer boundary.
My contribution
I developed a modular retrieval and LLM orchestration architecture using Llama 3.1 (8B), SBERT embeddings and FAISS vector search. Hierarchical semantic chunking prepares the source material for retrieval; filtering and structured context control shape what the language model receives.
A PyQt6 desktop interface runs retrieval and generation on a background thread, so the window stays responsive while the local model, served through Ollama, writes its answer.
Evaluation
I evaluated the assistant on a quantitative question set and with trap questions that check unsupported answers, using retrieval filtering and strict system prompting.
These checks describe this project's own evaluation. The sample size, question distribution and independent replication are not reported here, so they are not evidence of universal accuracy or a guarantee that hallucinations cannot occur.
Design decisions
Separating embeddings, retrieval, context assembly and generation makes the pipeline easier to inspect. The system can handle knowledge across departments while keeping retrieved context distinct from model-generated text.
Limitations
Missing or outdated source material limits answer quality. Retrieval can fail even when the source contains the answer. Broader evaluation should distinguish retrieval failures, grounded-answer quality and refusal behavior rather than collapsing them into one score.