Home  /  Programs  /  SURF  /  Announcements of Opportunity

Announcements of Opportunity

SURF: Announcements of Opportunity

Below are Announcements of Opportunity posted by Caltech faculty and JPL technical staff for the SURF program.

Each AO indicates whether or not it is open to non-Caltech students. If an AO is NOT open to non-Caltech students, please DO NOT contact the mentor.

Announcements of Opportunity are posted as they are received. Please check back regularly for new AO submissions! Remember: This is just one way that you can go about identifying a suitable project and/or mentor. Click here for more tips on finding a mentor.

Announcements for external summer programs are listed here.

New for 2021: Students applying for JPL projects should complete a SURF@JPL application instead of a "regular" SURF application.

Students pursuing opportunities at JPL must be
U.S. citizens or U.S. permanent residents.

  << Prev    Record 17 of 69    Next >>           Back To List


Project:  Using Artificial Intelligence to Explore and Discover the Ocean
Disciplines:  Computer Science, robotics
Mentor:  Kakani Katija, Visiting Researcher, (EAS), kakani@caltech.edu
Mentor URL:  www.bioinspirationlab.org  (opens in new window)
Background:  In order to fully explore our ocean and discover the life that lives there, we need to scale up our observational capabilities both in time and space. Given that the ocean represents the largest habitable ecosystem on our planet, and that less than 10% of that volume has been explored, automated observations via distributed networks of underwater sensors and vehicles are required. As we collect more data, the oceanographic community faces a data analysis backlog that researchers are only beginning to tackle using artificial intelligence and machine learning. This situation is especially challenging for underwater imaging, as robotic vehicles use this sensing modality to sample the environment as well as provide navigation and control. How can we leverage novel computer and data science tools to address this challenge of automated image and video analysis in the ocean (or Ocean AI)?
Description:  Researchers at the Monterey Bay Aquarium Research Institute’s Bioinspiration Lab (along with collaborators at MIT Media Lab and CVision AI) have been mining a 30+ year, curated, expertly annotated underwater image and video database to build an open-source image training set called FathomNet that can be used for machine learning algorithm development. FathomNet is built on the ideas of other image training sets like Pascal VOC, Caltech 101, ImageNet, and COCO. Our goal is to create an image training set that represents >200k marine species in Animalia, with more than 1000 instances of iconic and non-iconic imagery, from around the world. The database will be released sometime in 2021, and there are many opportunities for potential SURF students to contribute to the project that include:

1. Developing computer vision and/or machine learning workflows for automated analysis of robotic video transects in the midwaters of the ocean,
2. Applying generative adversarial networks on simulated 3D imagery to validate vehicle control algorithms for automated tracking of visual targets,
3. Developing computer vision and machine learning workflows to rapidly generate training data from underwater imagery and video for FathomNet,
4. Developing a FathomNet demonstration that can be displayed at local aquariums to share the value of Ocean AI,
5. Developing and testing optical flow algorithms that can be used on robotic underwater vehicles for accurate speed estimation,
6. Apply computer vision and machine learning algorithms for rapid 3D surface reconstructions of deep sea animals using novel imaging technologies (DeepPIV and EyeRIS).
References:  1. FathomNet: https://arxiv.org/abs/2007.00114 and presentation: https://youtu.be/STMek4QEQzA
2. Integrating machine learning into underwater vehicle control algorithms: https://bit.ly/3cFvN6c
3. ImageNet: http://www.image-net.org and publication: http://www.image-net.org/papers/imagenet_cvpr09.pdf
4. Microsoft’s COCO: https://cocodataset.org/ and publication: https://arxiv.org/abs/1405.0312
5. Monterey Bay Aquarium Research Institute: www.mbari.org
6. Bioinspiration Lab: www.bioinspirationlab.org
7. DeepPIV: https://www.nature.com/articles/s41586-020-2345-2
Student Requirements:  Required background and skills:
Demonstrable experience in computer vision (e.g., image segmentation, 3D pose estimation, optical flow)
Demonstrable experience in machine learning (e.g., weakly supervised and supervised deep learning, convolutional neural networks)
Programs:  This AO can be done under the following programs:

  Program    Available To
       SURF    Caltech students only 

Click on a program name for program info and application requirements.


  << Prev    Record 17 of 69    Next >>           Back To List