Project:	An Operational Metrics Dashboard for the Keck Observatory Archive.
Disciplines:	Astronomy, Computer Science
Mentor:	Bruce Bertiman, Senior Scientist/MPS, (PMA), gbb@ipac.caltech.edu, Phone: 6262604966
Mentor URL:	N/A  (opens in new window)
AO Contact:	As above, As above
Background:	The W. M. Keck Observatory archive (KOA; https://ipac.caltech.edu) curates and serves all observations acquired at the WM Keck Observatory in Hawaii.The W Observatory operates 13 instruments on two 10 m class telescopes, which are among the largest and was productive telescopes in the world. KOA makes these data freely available to researchers cross the world: only an internet connection is required to gain access. The archive ihas been operating for 20 years and provides a unique collection of data to researchers, encompassing visual and infrared observations of Solar system objects, exoplanets, the Galactic Center, the intergalactic medium, and more. It offers a through a web-based interface and a Python API services to discover, explore, visualize and download public data sets. KOA is undergoing a major modernization to position itself to support new and exciting avenues of research, such as discovering the nature of transient sources that will be discovered by new surveys such as those that will be made by the Rubin Observatory. KOA data have already been used to probe the nature of mysterious Fast Radio Bursts.
Description:	The modernization is already underway. KOA now ingests data in real time, generally 10 seconds after acquisition, and makes these data immediately available to the observers, after ingesting data at the end of the night as was done for many years, and has developed a very fast Data Discovery Service, which searches the archive in seconds. The student will join KOA to build out a successful prototype service, develipoed, for gathering metrics that monitor the content and performance of the archive. The prototype, a web-based dashboard, is built on the Plotly-Dash framework and records every 7 seconds the number of files in the archive. The prototype must be expanded to report a comprehensive set of metrics for the archive, such as as umber of files, data volume, number of queries, volume of data downloaded and so on, for each instrument. The service must be integrated into the archive architecture, and the functionality expanded as needed; e.g. plots that overlay values for multiple instruments. The work offers the possibility of authorship a technical professional paper describing the service.
References:	The Keck Observatory Archive web site is at https://koa.ipac.caltech.edu/cgi-bin/KOA/nph-KOAlogin Video showing the operation of the metrics dashboard: https://youtu.be/QlJ-67CJNfw A New Python Based Archive for the Keck Observatory Archive, Moseley et al. 2024 https://arxiv.org/pdf/2412.12356 Integration of Data Reduction and Near Real-Time Archiving into the Keck Observing Model. 2022, Brodheim et al. https://arxiv.org/pdf/2406.04510
Student Requirements:	The work requires proficiency in Python, a working knowledge of Unix/Linux, GitHub and relational databases.The student will work as part of the archive team, and must have good communication skills and the ability to work in a small team. The following videos are recommended as an introduction to KOA (the numbers are out of date, but the overall description of the archive remains correct). https://youtu.be/qRWHShMxwxs https://www.youtube.com/watch?v=AC4V9nHze7E&;t=521s This video provides an introduction to the operation of the Keck Observatory: https://www.youtube.com/watch?v=fyzsvpnRwzQ
Programs:	This AO can be done under the following programs:   Program    Available To        Amgen Scholars    Non-Caltech students only   Click on a program name for program info and application requirements.