EMC2 Challenge 2020 FAQs
Latest Update: 06/16/2020
Q : Who can participate the EMC2 challenge?
A : Anyone is welcomed
Q : What are the metrics to rank the submissions ?
A : Accuracy and Model Size
Q : When can I submit?
A :June 22, 2020
Q : What the results will be announced?
A : We will announce the winners in the same time as NeurIPS 2020.
Q : What resources are available to get familiar with model compression?
A1 : Review article: https://arxiv.org/abs/2006.03669
A2 : List of papers with code: https://paperswithcode.com/task/model-compression
A3 : TensorFlow model optimization toolkit: https://github.com/tensorflow/model-optimization
A4 : AI Model Efficiency Toolkit (AIMET): https://github.com/quic/aimet
A5 : Neural network distiller: https://github.com/NervanaSystems/distiller
Q : Is Transformer model allowed in the challenge?
A : All models are allowed in this challenge.
Q : How is the model size evaluated?
A : We only look at the physical bytes of the model file. We don't consider the runtime memory footprint.
Q : Which NLP tasks are evaluated by the model? Is per task model allowed?
A : We will evaluate the NLP model on all GLUE tasks (https://gluebenchmark.com/leaderboard) and will weight all the tasks equally. Any model or strategies are welcome!