📝 Reviews for system description papers are out! View reviews → ✅ Camera-ready due Sep 10 🏆 Leaderboard
ImageEval 2026

Cultural Grounding in Arabic Multimodal Generation and Understanding

Welcome to ImageEval 2026

ImageEval 2026: Cultural Grounding in Arabic Multimodal Generation and Understanding is a shared task on culturally grounded Arabic multimodal AI. The task suite evaluates visual question answering, image-grounded hallucination detection, and cultural accuracy evaluation for text-to-image generation.

The shared task focuses on Arabic language varieties and culturally specific visual content from the MENA region, with dedicated tracks for participants to build, evaluate, and compare multimodal systems.

Tasks

  • Task 1: AynVQA - Culturally Grounded Arabic Visual Question Answering and Hallucination Detection.
  • Task 2: CRAI-Bench - Cultural Accuracy Evaluation for Text-to-Image Generation.

Recent Updates

  • August 31, 2026: 📝 Reviews for system description papers are released on OpenReview, together with acceptance notifications. Camera-ready papers are due September 10, 2026. Please revise your paper to address the reviewers’ comments: incorporating the requested changes (or doing your best where a request is not feasible) is expected for the final version. Camera-ready papers may use up to 5 content pages (one extra page over the submission limit, to accommodate revisions), with unlimited references and appendix, and must pass ACL PubCheck.
  • August 17, 2026: System description papers are submitted on OpenReview and are due August 19. ArabicNLP also ran a webinar on writing a strong system description paper, presented by Dr Usman Naseem: watch the recording. See the submission guidelines.
  • August 11, 2026: All gold labels are released, including the blind test splits for both tasks. The dataset is consolidated in one repository, ImageEval-ArabicNLP26, now including the full Task 2 (CRAI-Bench) data with human gold scores, and complete metadata (country, category, subcategory) for every Task 1 split.
  • August 10, 2026: 👁️ The official leaderboard is released! Final test-phase results for all tasks are on the Leaderboard page. Every score was independently re-verified with the official scorers before release. Congratulations to all teams!
  • August 10, 2026: Important: all participating teams must fill in the system description form by August 15. We use it to describe your system in the shared task overview paper. System description papers remain due August 19 via OpenReview.
  • August 6, 2026: The test phase deadline is extended to August 9 (AoE), which is Monday August 10, 3 PM Qatar time. Each team may now make up to 16 submissions in total, with a maximum of 10 per day. The official leaderboard will be released later the same day, August 10. See the updated Important Dates.
  • August 6, 2026: System description papers will be submitted through OpenReview. Please create your account early, as verification can take a few days. There will also be special recognition for the Best System Description Paper.
  • August 3, 2026: The test phase is open. The blind Task 1 test set is live on Hugging Face; the Task 2 test set is shared separately. Gold labels for the devtest split are also being released. Leaderboards stay hidden until the phase closes on August 7 (AoE).
  • August 2, 2026: Dev-phase standings are now published on the Leaderboard page. In the test phase each team may make a limited number of submissions and the best one counts. Codabench leaderboards will be hidden while the test phase runs, and results released after it closes.
  • July 30, 2026: Submission details have been updated.
  • July 15, 2026: Deadlines extended. The development phase now closes August 2 (AoE) and the test phase closes August 7 (AoE), with shared task papers due August 19. See the updated Important Dates.
  • June 8, 2026: Task 1 information, Git repository, and Codabench links have been updated.
  • May 22, 2026: Task website, training/development data, and evaluation scripts released.
  • Join our Slack: Hop into the ImageEval Slack channel for announcements, deadlines, and discussion.
  • Submission guidelines: Paper and submission guidance is available for participating teams.

Important Dates

  • May 22, 2026: Release of task website, training/dev data, and evaluation scripts
  • July 20, 2026 → August 2, 2026 (AoE): Development phase closes; test set released
  • August 7, 2026 → August 9, 2026 (AoE): Test phase closes (run submission)
  • July 30, 2026 August 9, 2026 → August 10, 2026: Leaderboard release
  • August 15, 2026: System description form due (all teams)
  • August 15, 2026 → August 19, 2026: Shared task papers due on OpenReview
  • August 25, 2026 → August 30, 2026: Notification of acceptance
  • September 5, 2026 → September 10, 2026: Camera-ready papers due
  • October 24-29, 2026: ArabicNLP conference

Contact

For questions related to ImageEval 2026, contact the organizers at imageeval2026@gmail.com.

Organizers

View the official leaderboard for the final test-phase results across all tasks.