Our Journey
From a simple idea to a powerful platform
While inspecting our college website in Developer Mode, we discovered a shocking security loophole. It was possible to reset anyone's password with any value and then log back into the system using that newly set password. This vulnerability opened up an opportunity to extract necessary data for our project.
Choosing the Right Web Scraping Tool
Building the Automation Script
With the data collection automated, we moved forward with developing the application.
Tech Stack
After setting up the architecture, we successfully built a full-stack application that could fetch, process, and visualize results in an interactive way.
Once the data was stored in MongoDB, we wanted to build an AI-powered chatbot that could answer user queries, analyze results, and generate reports based on the available data.
Exploring AI Solutions
We initially attempted to fine-tune a model for better query understanding. However, due to hardware limitations (insufficient CPU/GPU performance), this approach was not feasible.
This method involved:
- Vectorizing the entire dataset and storing it in a vector database.
- Using similarity search to extract relevant context.
- Providing responses based on the retrieved context.
This solution showed promising results and became one of the best working approaches.
We also experimented with an agent-based approach where:
- The database schema and structure were provided as context.
- A user query was processed to generate a MongoDB query dynamically.
- The query was executed, and the retrieved response was fed back to the AI model for final output.
However, this method failed due to accuracy issues, making it unreliable for our use case.
We are now looking for AI enthusiasts and developers to contribute and improve our system. If you are passionate about AI, data extraction, and automation, you can help us build a sustainable solution and earn a spot in our contributor list.
Join us in solving this challenge and making something impactful! 🚀✨