🏆 Mobile-MMLU Challenge

About the Competition

The Mobile-MMLU Benchmark Competition is a premier challenge designed to evaluate and advance mobile-optimized Large Language Models (LLMs). This competition is an excellent opportunity to showcase your model's ability to handle real-world scenarios and excel in mobile intelligence.

With a dataset spanning 80 distinct fields and featuring 16,186 questions, the competition emphasizes practical applications, from education and healthcare to technology and daily life.

Why Compete?

Participating in this competition allows you to:

• 🌟 Showcase your expertise in developing and optimizing LLMs for mobile platforms.
• 🚀 Benchmark your model’s performance against others in a highly competitive environment.
• 📈 Contribute to advancements in mobile AI, shaping the future of user-centric AI systems.

How It Works

1. 1️⃣ Download the Dataset: Access the dataset and detailed instructions on the GitHub page. Follow the steps to ensure your environment is set up correctly.
2. 2️⃣ Generate Predictions: Use the provided script in the GitHub repository to generate answers. Ensure the output file matches the format in the github
3. 3️⃣ Submit Predictions: Upload your CSV file to the Submission Page on this platform.
4. 4️⃣ Evaluation: Your submission will be scored based on accuracy. The results will include overall accuracy metric.
5. 5️⃣ Leaderboard: Optionally, add your results to the real-time leaderboard to compare your model's performance with others.

Resources

• 📂 GitHub Repository: Contains the dataset, scripts, and detailed instructions.
• 📊 Dataset Link: Direct access to the competition dataset.
• ❓ Support Page: Use this for queries or issues during participation.

Submit Your Predictions

Upload your prediction file and provide your model name to evaluate and optionally submit your results to the leaderboard.

Submit Your Predictions

Upload your prediction file and provide your model name to evaluate and optionally submit your results to the leaderboard.