Enhancing Math Education with AI
Fibo: AI Math Tutor is our internal R&D project aimed at exploring the potential of integrating AI into education. Fibo is designed to assist secondary and high school students in understanding fundamental mathematics concepts and practicing them effectively.
Mathematics, along with physics and chemistry, is often considered one of the most challenging subjects in schools. For many students, learning and doing homework without a teacher's assistance can hinder or even diminish the effectiveness of their learning. Private lessons can also be costly.
We wanted to explore the possibility of substituting a real teacher with an AI agent, tweaked to assist in learning and explaining mathematical concepts. We aim to make private tutoring more accessible to a greater number of students without the high cost.
By integrating an AI chat feature into our app, which focuses on teaching rather than simply providing straight answers, we made Fibo interactive and engaging. Users can learn and practice their skills using an in-app knowledge base that includes learning videos and quizzes.
Scope of Work
Fibo is a proof of concept (PoC) built entirely from scratch internally. We began by identifying the problems AI could solve, then moved to idea validation, and finally developed the PoC. This allowed us to test our idea with real users and begin outreach to find partners who could help validate the idea or who might need such a solution.
Idea validation
Our main concern was the issue of AI hallucinations, particularly if they were so severe that they could create more problems than solutions. To test this, we used AI to solve several school-leaving exams, checking if it could handle basic mathematical exercises without hallucinating. Even without advanced prompt engineering, the AI performed adequately, which gave us hope that with further prompt engineering, we could achieve very good results.
Cross-Platform App Development
We decided to use Flutter to build a cross-platform app with a single codebase. With Flutter, we quickly developed a high-quality MVP for both Android and iOS with just a team of two programmers.
AI Integration And Backend Development
The true heart and soul of the project lies in AI integration on the backend. Creating appropriate prompts for AI was crucial to achieve conversations that are focused on learning and exploring problems without immediately providing solutions.
Quality evaluation
In developing intelligent apps, iterating is crucial. We are constantly testing our solution with real students. Plus, we’re also solving real-world exams and exercises using Fibo, to validate and improve the product.
Solution
Validating AI for Educational Applications
Sometimes it's difficult to determine if AI is a suitable solution for our problems. At first glance, solutions like ChatGPT appear capable of solving many issues. However, this initial impression can be misleading, especially when tackling advanced problems, where we often encounter AI hallucinations.
To create a solution capable of solving mathematical problems at the secondary or high school level, we selected a few school-leaving exams and attempted to solve them using ChatGPT, employing various methods such as uploading problem photos or rewriting their content into chat format. Our results were promising, with few instances of hallucinations, indicating potential for further improvements through advanced prompt engineering in later development stages.
Rapid Mobile App development with Flutter
We decided to develop our app using Flutter, a cross-platform framework. This enabled us to deliver a high-quality app with a small team of just two developers.
One of the most unique front-end challenges in developing applications for learning mathematics is displaying math formulas. To simplify and ease implementation, we decided to use TeX, a widely used typesetting system renowned for its ability to handle mathematical and scientific documentation with precise control over layout and typography.
AI engineering and Backend development
The project employs a robust stack of modern technologies to ensure optimal performance and scalability. At its core, the backend is powered by NestJS, a progressive Node.js framework that primarily serves as an API, enabling efficient, scalable server-side applications.
NestJS is paired with PostgreSQL, a powerful open-source relational database, to handle data management and storage with reliability and integrity.
To enhance the project's capabilities in natural language processing and artificial intelligence, we integrated LangChain and OpenAI technologies. This combination allows the application to leverage advanced AI models for text understanding and generation, providing intelligent and context-aware interactions.
To increase quality, efficiency, and accuracy, we introduce a chain of prompts based on the Tree of Thought method. According to our tests, this approach dramatically improves accuracy, ensuring more precise and relevant responses in AI-driven interactions.
Additionally, Redis is utilized for handling performance-heavy tasks in a background thread. This approach effectively separates time-consuming processes from the main application flow, reducing response times and increasing the overall efficiency of the system.
Together, these technologies form a comprehensive and powerful platform that supports a wide range of features and scales efficiently to meet the demands of growing user bases.
PoC Validation
If we want to validate our app that integrates with AI, a good idea is to divide the process into two separate validation streams:
- independent of AI
- dependent on AI
The AI-independent validation is quite straightforward, similar to standard software testing; we conduct it once, and the outcomes are usually clear — things either work or they don't.
AI-dependent validation can be more complex. AI solutions are not always deterministic: sometimes they perform brilliantly, but other times they may produce hallucinations that render our system useless. Because of this, we often introduce small changes to our prompts, which can significantly improve results. That's why iterating over different prompts and AI solutions is crucial for achieving the best results, and this can be a never-ending cycle.
During our validation process, we regularly simulated real-world usage by using our app to solve mathematical exercises through the chat. We experimented with different prompts to achieve the best results. Our iterations focused on how the AI created math formulas so that they were properly displayed on the screen. Initially, the performance was suboptimal, but through continuous improvements, we developed an AI agent that meets our needs.
Fibo Helps Students Learn on their own effectively
By focusing on creating an interactive AI math tutor that not only gives correct answers to questions but also helps students understand how to find the answer themselves, we proved that AI can be effectively used to support students in learning mathematics.
