Generative AI Chatbot for Telecom Analytics Company
A telecom analytics company wanted a chatbot that could answer questions for their live agents about a sophisticated mobile routing product. The project started out as primarily a deterministic AI chatbot and evolved into one that heavily utilized generative AI to pull answers from source files.
I was in charge of creating the deterministic AI chatbot in Sicura (NOHOLD’s proprietary chat system) and collaborating with employees of the telecom analytics company to identify issues with the generative AI and create training documents to remedy these issues.
Deterministic AI Chatbot
I organized approximately 100 user questions provided by the telecom company in a logical manner in the Sicura knowledge base.
They also provided a troubleshooting flow that they wanted turned into a chat flow. This flow was contained on a multi-page document. I worked with employees to understand how the flows contained in the document related to each other and then translated it into this chat flow. Green boxes represent the start of a subflow and red boxes represent the end of a flow. I built this chat flow in Sicura as well.
Once the chatbot was live for testing, there was an opportunity for agents to give feedback on the answers they were receiving from the bot. They could rate an answer with a thumbs up or thumbs down. Any thumbs down would give them the opportunity to write out what answer they would have expected to get. I organized this feedback and collaborated with employees of the telecom company to edit or draft new content based on the feedback.
Adding Generative AI
The telecom company sent us multiple files containing hundred-page guides to train our generative AI on. I organized these files, then sent them to the engineering team to feed into the LLM.
When the generative AI side of the bot went live, I pulled chat logs on a regular basis so that they could be analyzed for accuracy by an employee of the telecom company. If the LLM answered a question incorrectly, the employee would draft a correct response to the question. I would draft alternative forms of the question and then send these questions along with the answer to engineering to feed back into the LLM.