Bronze medal on Developing LLM/RAG system for Kaggle competition.
The Challenge
Kaggle competition Eedi - Mining Misconceptions in Mathematics asked for the design and development of an AI system for autonomous finding of misconceptions when grading multiple choice Math exams.
My Solution
I developed an LLM System that indexed all the possible misconceptions in a vector database (Qdrant) and when grading a multiple choice question, used a series of LLM prompts to translate each wrong answer into a search query that could find the top 10 most likely misconceptions.
The Outcome
The accuracy of my system was on par with other competitors’ baselines, which was my goal since this was a baseline system. The value proposition of my solution was the use of Stanford’s DSPY Framework for prompt optimization. My notebook reached Bronze medal on Kaggle and allowed me to develop a better understanding of DSPY framework. I shared my findings in a tweet where the Author of the framework replied.