DORS Tutorial: Introduction to reinforcement learning

Place: Online via Zoom (link will be sent to the registered participants)

Time: 17/12/2020 13:00-17:00

About the tutorial:

Reinforcement learning provides a mathematical formalism for learning-based control. By utilizing reinforcement learning, we can automatically acquire near-optimal behavioral skills, represented by policies, for optimizing user-specified reward functions. It contrasts with the traditional optimization toolbox by considering a stochastic environment whose dynamics are not know apriori. By interacting with the environment and observing delayed rewards, reinforcement learning agents have shown outstanding results at solving complex sequential decision-making problems such as playing Go and videogames at super-human-level performance, autonomous driving, smart grid optimization, etc. This tutorial will cover the basics of reinforcement learning, including terminology and mathematical formalism, Markov Decision Processes, Q-learning and Deep Q-learning. Given the limited time, it will prioritise "breadthness"  over depth, giving pointers to where to learn about certain aspects in more detail. It will be split in multiple theory "blocks" interleaved with practical "blocks", where you get the chance to try out some of the concepts in practice. The practical blocks will be based on a Jupyter notebook and Python.

Instructor: Filipe Rodrigues

Filipe Rodrigues is an associate professor at the Technical University of Denmark (DTU) in the Machine Learning for Smart Mobility (MLSM) group. He works on machine learning models for understanding urban mobility and the behaviour of crowds. Previously, he was a H.C. Ørsted / Marie-Skłodowska Curie Actions (COFUND) postdoctoral fellow, also at DTU, working on spatio-temporal models of mobility demand with emphasis on modelling uncertainty and the effect of special events. Filipe holds a PhD from University of Coimbra (Portugal), during which he worked on probabilistic models for learning from crowdsourced data.

Prerequisites:

• Attendants are expected to be familiar with Python programming and know basic concepts from statistics (random variable, conditional probability, expectation, etc.), linear algebra and calculus (derivative, integral, etc.).
• Attendants are also expected to have a working Python and Jupyter notebooks installation on their computers (see installation instructions here: https://test-jupyter.readthedocs.io/en/latest/install.html). Alternatively, attendants can use the Google Colab platform (free; only requires a Google account - https://colab.research.google.com/) to upload the tutorial notebook and work on it. If you never used Jupyter notebooks before, please watch the following short introductory video: https://youtu.be/jZ952vChhuI

Agenda:

PART 1 (THEORY):

• Welcome
• Introduction to reinforcement learning
• Terminology and setup
• The reinforcement learning problem

PART 2 (THEORY):

• Policy optimization

PART 3 (THEORY):

• Actor-critic algorithms
• Q-learning

PART 4 (PRACTICE): 

• Jupyter notebook

PART 5 (THEORY):

• Deep Q-learning

PART 6 (PRACTICE): 

• Jupyter notebook

Price: 

- DORS members: Free

- Students: 100,- DKK

- Others: 300,- DKK

Payment is accepted through bank transfer. Payment information will be provided upon registration.