2019 Abstract from Eric

Reaching for All the Stars: Sky Survey Scheduling Using Deep Reinforcement Learning

Student Researcher: Eric Chen (Naperville North High School)

Scientist Mentor: Dr. Brian Nord

In order to optimize telescope scheduling for sky surveys, we implemented a deep reinforcement learning agent that outperforms conventional algorithms, promising more efficient use of telescopic resources in the future.

Modern sky survey projects, such as DES and LSST, have to account for many factors such as weather, the movement of the sky, object brightness, and others, when scheduling telescope movements. Since longer exposure times increase image quality, researchers must find a balance between taking only a few higher quality pictures or taking many lower quality pictures. Currently, conventional telescope scheduling strategies are simulated algorithmically with the aid of computer software, such as Astroplan, and then hand-tuned by researchers, a lengthy process that may not always lead to the highest data acquisition. In the future, conventional algorithms will not be able to keep up with more complex sky surveys. Our solution was to create a reinforcement learning agent that can find the optimal schedule given information about a certain night. Reinforcement learning (RL) is a type of machine learning where an agent acts on and gradually learns from an environment.

Tabular-based RL methods are memory limiting because they explicitly store values for each action given each state. As a result, we created a Deep Q Network (DQN) reinforcement learning agent as an approximation of the tabular method that will scale better with larger data requirements. We built the DQN agent in Python with Google’s open-sourced TensorFlow library. Afterwords, we trained the agent in a gym environment that simulates the night sky. As a first investigation into the effectiveness of neural network agents, we discovered that the agent outperforms conventional algorithms, but it will require further optimization before being used on real sky surveys. In the future, we will need to test the agent in an environment with more targets that better models a real sky survey and in an environment that simulates multiple consecutive nights.