2021 Abstract from Sophia and Ben

Telescope Scheduling with Reinforcement Learning Techniques.

Student Researchers: Benjamin Cohen (Montgomery Blair High School)

Sophia Zhou (Massachusetts Academy of Math and Science)

Scientist Mentors: Brian Nord, Yuanyuan Zhang

We present first results from our work on automated telescope scheduling with reinforcement learning techniques. With the increasing size of optical astronomical surveys, automated observation scheduling tools are becoming necessary for the operation of large space and ground-based telescopes in an efficient manner. These scheduling methods need to have the capacity for rapid adjustment to stochastic elements (such as the weather). We frame the astronomical survey-scheduling problem as a finite Markov Decision Process solvable with reinforcement learning (RL) techniques. Using this framework, we assess the results of the application of modern RL techniques, such as Proximal Policy Optimization (PPO) and a Deep Q-Network (DQN), on the scheduling problem and compare against modern scheduling algorithms. Specifically, we show that PPO’s clipping is necessary for avoiding rapid fluctuations in the agent’s training. To extend this work, we plan to design additional reward functions for specific scientific objectives and apply and assess them on exposure scheduling.