Los Gatos Rowing Club
Los Gatos Rowing Club (LGRC) wanted to reduce the number of emails their volunteers had to answer. It was decided that we would build a generative AI bot trained on those questions. I was brought on to do the initial content and conversation design, UI design, and testing.
Content and Conversation Design
I discussed with LGRC what kind of content they wanted. They provided me with a list of frequently asked questions. I also looked over their website and came up with a list of suggested content to feed into the LLM, which they agreed to.
Persona
Since this was to be primarily a generative AI bot, I would not have full control over what the bot said. I checked with engineering to see if I could write prompts in order to give the generative AI a persona. The engineering team confirmed that this was not a feature that was currently available, but may be available in the future. So, I put a hold on developing a persona for this bot.
Deterministic AI
Although this was to be a primarily generative AI bot, there were certain questions where we needed the bot to give users links to particular pages in the club’s website. In these cases, I created deterministic intents so users would get the same response each time.
UI Design
I took inspiration from the colors included in the website and utilized club logos. I came up with several different designs and they chose the first one.
Testing
I wrote up instructions to test the bot on deterministic and generative responses and sent them out to the team for testing. I also tested the bot.
Results from testing were mixed. I noticed a dip in the bot’s ability to answer the questions defined in the FAQ document as well as some hallucinations around prices. I made the recommendation that we lean more heavily into deterministic AI for this particular bot. I suggested adding intents for all FAQs and any question a user might have about prices.