Artificial intelligence - Smarter than we think (MMLU increases for GPT models) [FIXED]

---

---

AI Summary

• Summary of AI Visualization for 2024:
  • Background:
    • Presenter has expertise in human intelligence research and AI since 2001.
    • Focus on AI development, particularly since the last 4 years.
  • Large Language Models Analysis:
    • Title: “Large Language Models: Smarter Than We Think”
    • Utilized the Massive Multitask Language Understanding (MMLU) Benchmark.
    • MMLU designed to challenge the most advanced AI models.
  • Performance of GPT Models:
    • Excluded GPT-1 (limited training data).
    • GPT-2:
      • Trained on popular web content.
      • Scored 32.4 on MMLU, just below human average of 34.5.
    • GPT-3:
      • Trained on significantly more data.
      • Scored 43.9, surpassing human average.
    • Instruct GPT/GPT-3.5:
      • Achieved a score of 70 on MMLU.
    • GPT-4:
      • Hidden in development since August 2022.
      • Scored 86.4, close to human expert average of 89.8.
    • Score Increases:
      • GPT-2 to GPT-3: 35% increase.
      • GPT-3 to GPT-3.5: 59% increase.
      • GPT-3.5 to GPT-4: 23% increase.
  • Future Expectations:
    • Anticipation for GPT-4.5 release in January 2024.
    • GPT-5 expected within the year, predicted to surpass human expert scores in all 57 subjects.
  • Engagement and Outreach:
    • Keynotes planned for major companies, Fortune 500s, and governments.
    • YouTube presence and invitations to join “the memo.”
  • The Memo:
    • Provides updates on AI developments in plain English.
    • Available to subscribers from Fortune 500s, governments, and individuals.
    • Covers AI advancements, humanoid integration, AI IQ progression, and global use cases.