Dynabench is a research platform for dynamic data collection and benchmarking. In the first part of this workshop, we introduce how we built the dynamic model evaluation pipeline for Dynabench using a collection of open source Python toolkits, e.g. pytorch, torchserve, boto3, etc, and how anyone in the community can easily submit their models into the pipeline using our open source toolkit, Dynalab. We believe that the evaluation cloud of Dynabench will transform how the community thinks of models and benchmarks, and continuously push the community to innovate beyond the current status quo from a completely new angle of thinking.
The two key components in a multi-task Reinforcement Learning codebase are (i) Multi-task RL algorithms and (ii) Multi-task RL environments. Facebook AI developed open-source libraries for both components. Part 2 of this workshop describes how one can use these two libraries to get started with multi-task reinforcement learning. By the end of the talk, the audience should be able to design their own multi-task RL agent using the components from the MTRL library (https://github.com/facebookresearch/mtrl) and run them on environments provided by the MTEnv library (https://github.com/facebookresearch/mtenv).