VATEX

Welcome to VATEX Captioning Challenge 2020!

We are pleased to announce VATEX Captioning Challenge 2020! This year, we release an additional private test set with 6,278 new videos for evaluation! The challenge will be hosted at the Workshop on Language & Vision with applications to Video Understanding, CVPR 2020.

Please stay tuned for more information!

Challenge Launch: ~~April 12th, 2020.~~
Results Submission Deadline: June 1st, 2020.
Challenge Paper Submission Deadline: June 8th, 2020.
The winners will be announced at the LVVU workshop, CVPR 2020 on June 19th, 2020.

Challenge Paper Requirements

To be eligible for result archives and consideration for awards, we kindly request you to send the following information to vatex.org@gmail.com using your main contact email:

Team name.
Team members.
The username used in CodaLab submissions.
An arXiv link to a 2-4 page paper, which describes your systems (including data processing, methods, experimental results, etc.) using the CVPR 2020 paper template.

Note that the challenge paper will be accepted to our CVPR 2020 LVVU workshop as a non-archival paper.

The VATEX dataset is a new large-scale multilingual video description dataset, which contains over 41,250 videos and 825,000 captions in both English and Chinese. Among the captions, there are over 206,000 English-Chinese parallel translation pairs. Compared to the widely-used MSRVTT dataset, VATEX is multilingual, larger, linguistically complex, and more diverse in terms of both video and natural language descriptions. Please refer to our ICCV paper for more details. This VATEX Captioning Challenge aims to benchmark progress towards models that can describe the videos in various languages such as English and Chinese.

Dataset Download

Please refer to the details at the Download page. You can download English/Chinese captions and video features from the page.

Submission

The challenge is hosted at the CodaLab. Please go to the Challenge page to submit your models.

Challenge Phases

Public Test (English/Chinese): This phase evaluates algorithms on the VATEX public test set where the English references are available. A submission needs to consist of results on the entire public test set to be considered as a valid submission.
Private Test (English/Chinese): This phase evaluates algorithms on the VATEX private test set with all references heldout. A submission needs to consist of results on the entire test set to be considered as a valid submission. This phase is aimed at the final evaluation of the model and one is not allowed to create multiple submissions using multiple teams.