Syllabus

Time and Place:

5:00-7:30 Tuesdays, HNB 100

Course Description:

This seminar/lecture style course provides a comprehensive introduction to reinforcement learning, an approach of how to make sequences of decisions to achieve goals in a stochastic environment. Knowledge from this course will be useful for general decision making in artificial intellgence problems, for learning to control processes (e.g., in factory automization, autonomous robotics, networking and routing system), and for understanding learning in biology (e.g., conditioning and motor learning). Initially, topics of this course focus on the core topics of reinforcement learning, including Markov decision processes, dynamic programming, temporal-difference learning, Monte Carlo learning methods, eligibility traces, the role of neural networks, the integration of learning and planning. At a later stage of the course, advanced topics will be discussed, such as: partially observable problems, policy gradient methods, function approximation, alternate definitions of return, advanced methods for exploitation/exploration tradeoff, methods for integrating prior knowledge, abstraction and hierarchies, multi-criterion and multiagent reinforcement learning, biological connections.

Class Format:

The class will be a mixture of seminar style and lectures.

Grading:

• 33% active participation in class
• 33% paper presentation
• 33% project

Either one paper presentation and one project, or two projects are required of each participant

Prerequisites:

Ideally CS542 or CS561, basic knowledge in linear algebra and statistics, or permission by instructor.

Readings:

Primary readings:

Sutton & Barto (1998). Reinforcement Learning: An Introduction. Cambridge, MA: MIT Press, 1998. There is an online version of this book.

Additional readings:

Introduction to the Theory of Neural Computation (Hertz, J. ,Krogh, A., Palmer, R. G., 1991, Addision Wesley) (optional)

Instructors:

Dr. Sethu Vijayakumar
Dr. Auke Ijspeert
Dr. Stefan Schaal
Associate Professor
University of Southern California
Ronald Tutor Hall RTH-417
Los Angeles, CA 90089-2520
phone: (213) 740 9418
email: {$ \textrm{sethu@usc.edu} $} email: {$ \textrm{ijspeert@rana.usc.edu} $} email: {$ \textrm{sschaal@usc.edu} $}

Office Hours:

According to email arrangement with instructors

Academic Integrity:

All students are required to abide by the USC code of Academic Integrity. Violation of that Code will be dealt with as described in SCAMPUS. If you have any questions about the responsibilities of either students, faculty, or graders under this policy, contact the instructor or the Office of Student Conduct.

Disabilities and Academic Accomodations:

Students requesting academic accomodations based on a disability are required to register with Disability Services and Programs (DSP) each semester. A letter of verification for approved accomodations can be obtained from DSP when adequate documentaion is filed. Please be sure the letter is delivered to the instructor (or TA) as early in the semester as possible. DSP is open Monday-Friday, 8:30-5:00. The office is in Student Union 301 and their phone number is (213) 740-0776.