REINFORCEMENT LEARNING

AIM

This course aims to equip students with a comprehensive understanding of reinforcement learning (RL), encompassing both foundational theories and modern algorithmic approaches. It focuses on developing students' ability to design, analyze, and implement RL agents capable of learning optimal behavior through interaction with dynamic environments. Key topics such as Markov decision processes (MDPs), dynamic programming, Monte Carlo methods, temporal-difference learning, and policy gradient techniques will be introduced. Additionally, students will gain hands-on experience with deep reinforcement learning algorithms and frameworks, including Deep Q-Networks (DQN) and Actor-Critic methods. By the end of the course, students will be prepared to apply RL techniques to real-world problems, critically evaluate algorithm performance, and pursue further study or research in autonomous systems and intelligent decision-making.

CONTENT

This course contains; Introduction to Reinforcement Learning,Multi-Armed Bandits, Exploration,Markov Decision Processes,Dynamic Programming,Monte Carlo Methods,Temporal-Difference Learning,Eligibility Traces and TD(λ),Planning and Learning ,Function Approximation,Deep Q-Learning (DQN),Policy Gradient Methods,Actor-Critic Methods,Advanced Policy Optimization (PPO, TRPO),Exploration Strategies,Reinforcement Learning Applications.

1. 1

   Understand key reinforcement learning concepts, including agents, environments, rewards, and value functions

   Taught by: Problem Solving Method, Project Based Learning Model, Lecture Method · Assessed by: Traditional Written Exam, Homework, Project Task

2. 2

   Formulate decision-making problems as Markov Decision Processes

   Taught by: Problem Solving Method, Project Based Learning Model, Lecture Method · Assessed by: Traditional Written Exam, Homework, Project Task

3. 3

   Apply dynamic programming, Monte Carlo, and temporal-difference learning techniques

   Taught by: Problem Solving Method, Project Based Learning Model, Lecture Method · Assessed by: Traditional Written Exam, Homework, Project Task

4. 4

   Implement tabular and approximate reinforcement learning algorithms

   Taught by: Problem Solving Method, Project Based Learning Model, Lecture Method · Assessed by: Traditional Written Exam, Project Task

5. 5

   Design and evaluate policy-based and value-based learning agents

   Taught by: Problem Solving Method, Project Based Learning Model, Lecture Method · Assessed by: Traditional Written Exam, Homework, Project Task

6. 6

   Utilize deep reinforcement learning architectures such as DQN and Actor-Critic

   Taught by: Problem Solving Method, Project Based Learning Model, Lecture Method · Assessed by: Traditional Written Exam, Homework, Project Task

7. 7

   Analyze the impact of exploration strategies and function approximation

   Taught by: Problem Solving Method, Project Based Learning Model, Lecture Method · Assessed by: Traditional Written Exam, Homework, Project Task

8. 8

   Develop and test reinforcement learning solutions in simulated environments

   Taught by: Problem Solving Method, Project Based Learning Model, Lecture Method · Assessed by: Traditional Written Exam, Homework, Project Task

9. 9

   Critically assess the strengths and limitations of various RL algorithms

   Taught by: Problem Solving Method, Project Based Learning Model, Lecture Method · Assessed by: Traditional Written Exam, Homework, Project Task

10. 10

    Apply reinforcement learning to real-world problems and research scenarios

    Taught by: Problem Solving Method, Project Based Learning Model, Lecture Method · Assessed by: Traditional Written Exam, Homework, Project Task

WEEKLY PLAN

1. WEEK 1

   Introduction to Reinforcement Learning

2. WEEK 2

   Multi-Armed Bandits, Exploration

3. WEEK 3

   Markov Decision Processes

4. WEEK 4

   Dynamic Programming

5. WEEK 5

   Monte Carlo Methods

6. WEEK 6

   Temporal-Difference Learning

7. WEEK 7

   Eligibility Traces and TD(λ)

8. WEEK 8

   Planning and Learning

9. WEEK 9

   Function Approximation

10. WEEK 10

    Deep Q-Learning (DQN)

11. WEEK 11

    Policy Gradient Methods

12. WEEK 12

    Actor-Critic Methods

13. WEEK 13

    Advanced Policy Optimization (PPO, TRPO)

14. WEEK 14

    Exploration Strategies

15. WEEK 15

    Reinforcement Learning Applications

ASSESSMENT

• Rate of Midterm Exam to Success50%
• Rate of Final Exam to Success50%

READING

• Reinforcement Learning: An Introduction, 2018, 2nd edition, MIT Press, Sutton and Barto; Deep Reinforcement Learning, 2022, Springer, Aske Plaat

TEACHING STAFF

• Assist.Prof. İbrahim KARLIAĞACOORDINATOR
• Assist.Prof. İbrahim KARLIAĞA