Accuracy Auditing, Behavioral Fixes, and What It Means to Test a Chatbot Honestly

Introduction

This paper documents the post-Week 10 accuracy audit of the Hypotify Clinical Chatbot. After running 100 questions against the live Streamlit engine, the initial pass rate was 86%. The 14 failures fell into three categories: non-deterministic responses (7), wrong intent routing (5), and inaccurate expected answers (2). Root cause analysis traced each failure through the chatbot's _build_response() method. Five targeted code fixes — explicit help/farewell intercepts, non-hex UUID handling, expanded population insights vocabulary, and NER capability routing — achieved a 100% final pass rate with 20 documentation corrections.