Simulations

Simulations

The ROS (Robot Operating System) and Simulations team uses the ROS software libraries and tools to enable communication between the Perception, AI, and Controls code. Members spend a lot of time thinking about the big picture of our codebase, optimizing its design and usability, troubleshooting integration issues, and ensuring the rest of the team’s code is compatible. Additionally, this team is responsible for creating a simulations framework for the AI and controls team to test their code in isolation.

Synthetic Image Data

We use cameras to identify obstacles in the water. However, our location limits our ability to test our vision models throughout the year, so we need another way to gather realistic data.

  • Use Blender's Python bindings to create scripts that generate realistic scenes on which to train our YOLO model
  • Use Blender to hand-create 3D models of the buoys, docks, color indicators, and boats
  • Create realistic scenes and water in Blender
  • Technologies: Python, Blender, YOLO

Next steps: The Sim-to-Real gap is a major open question in robotics research. We hope to leverage new research in this field to apply it to the maritime domain, which remains mostly unexplored.

Reinforcement Learning Sim

As part of our efforts to improve our boat's pathfinding abilities, we are implementing Reinforcement Learning to control the boat's decision-making to complete each task.

  • Create simulated environments to represent the different tasks we perform at competition
  • Accurately and cheaply model obstacle collision
  • Implement stochastic model for perception data
  • Technologies: Python, NumPy, Shapely, MatPlot, Pytorch

3D Simulation

Currently, we use a Unity-based 3D simulation for testing our boat. However, the buoyancy physics for the model are relatively simplistic.

  • Implement the Fossen 6-DOF Marine Craft model for accurately representing boat buoyancy physics
  • Improve physical modeling of the Racquetball Launcher and Water Gun to better match their real-life counterparts
  • Technologies: Linear Algebra, Physics, Unity, C#