Six teams of Naval Undersea Warfare Center Division Newport engineers and scientists gathered in a warehouse on Naval Station Newport for the “Spot Robot Warehouse Challenge Innovation Event” culminating competition on June 21. The month-long challenge provided an opportunity to demonstrate an effective end- to-end machine learning pipeline.
The process allowed teams to develop technological solutions to some core problems such as identification and tracking of items, detecting misplaced items, and identifying missing or changed items in a warehouse environment. Teams, assisted by Division Newport photographers, gathered imagery, conducted pre-process planning with the appropriate tools, labeled hundreds of images, built/implemented the right machine learning model architecture, verified performance, and then deployed that solution “in production” on Spot , to robot dog.
A product of Boston Dynamics, Spot is a four-legged robot designed to automate routine inspection tasks and data capture safely, accurately, and frequently. Engineers Gary Huntress and Eugene Chabot of the Undersea Warfare (USW) Platforms and Payload Integration Department acquired Spot so that Division Newport scientists and engineers can develop their skills in autonomy. For this challenge, teams used Spot to identify objects of varying sizes, shapes and visual characteristics.
The US Navy contains many complex and dynamic environments from detecting mines in cluttered landscapes to identifying changes to our critical infrastructure, the event organizers noted. Navy logistics is an example of a critical function to supply the necessary materials to keep the fleet enabled in its mission.
“NAVSEA Logistic Command’s warehouse located on Naval Station Newport is a complex and dynamic environment requiring a lot of coordination to complete this challenge,” Huntress said. “Autonomy and artificial intelligence provide new opportunities for reducing the labor demands and potential human error.”
The event successfully showcased teams’ ability to work on algorithms that are the basis of artificial intelligence and machine learning. Support from Naval Surface Warfare Center Philadelphia Division (NSWC PD), University of Massachusetts Lowell, and Brown University — all of whom brought their own Spot robots to the challenge — provided collaboration beyond Division Newport’s science and technology community. At some point, the warfare center teams will collaborate to explore the idea of leveraging mutual strengths. NSWC PD would use their advanced autonomy as a sensing platform while Division Newport would provide a machine learning classifier via a docker container suitable for deployment on their Spot.
Adam Sherman of the USW Platforms and Payload Integration Department, organized the event as part of his three-month NAVSEA Journey-Level Leadership (JLL) rotation.
“When I was approached to organize a ‘Spot Robot Warehouse Challenge Innovation Event’ as part of my JLL rotation, I had no idea what I was getting myself into, but it sounded interesting and different,” Sherman said. “I never could have imagined how much work and detail would need to go into this effort to make this event successful. I was very fortunate to have a dedicated team that was always available. Each member of the team had different attributes that were monumental in making sure that everything that needed to be accomplished was completed.”
Each participant was required to have some skills in Python programming, imaging and autonomy-based machine learning. Each participant also had to commit 32 hours to the event, which included training, help sessions and the challenge event. Teams had to work within the allotted time for the challenge and had limited access to the warehouse.
The core of the challenge was based on identifying objects in different ways. There was a fixed set of exactly 30 known objects and a well-defined autonomy task. The teams were told what to submit after each run and they were scored as uniformly as possible. Some tasks were harder than others and more points were awarded accordingly. Because the Innovation Event was also a competition, event organizers devised a challenge and scoring rubric.
“We challenged 30 people to take a robot that they had never used, into an unfamiliar environment, and teach it how to search for and identify objects using a collection of machine learning techniques that most had never tried,” Huntress said. “The simplest approach would have been to use the built-in autonomy of Spot to record a predefined path, and to use the baseline pre-trained (but non-optimized) machine learning model provided to them. No team did this. Every team worked hard to extend the autonomy and build a better machine learning classifier and every team succeeded.
“This is important because the pipeline is then very easy to modify,” Huntress said. “If a new object of interest were identified, it would then be easy to obtain/label new imagery and rerun the model training with no modification. That is, the pipeline is more important than the machine learning itself.”
Ultimately, having the teams better understand the end-to-end machine learning pipeline made the event worthwhile. As far as possible transition to use by the US Navy, this event showed that Spot could assist with shipyard maintenance tasks in addition to warehousing.
“The outcome was everything I hoped for. I had that ‘aha!’ moment when I saw how much more valuable that pipeline was,” Huntress said. “There are nuances of the autonomy aspect. You can have a lot of tools in the toolbox but until you put them in the pipeline, you don’t have anything.”
NUWC Newport is the oldest warfare center in the country, tracing its heritage to the Naval Torpedo Station established on Goat Island in Newport Harbor in 1869. Commanded by Capt. Chad Hennings, NUWC Newport maintains major detachments in West Palm Beach, Florida, and Andros Island in the Bahamas, as well as test facilities at Seneca Lake and Fisher’s Island, New York, Leesburg, Florida, and Dodge Pond, Connecticut.
Join our team! NUWC Division Newport, one of the 20 largest employers in Rhode Island, employs a diverse, highly trained, educated, and skilled workforce. We are continuously looking for engineers, scientists, and other STEM professionals, as well as talented business, finance, logistics and other support experts who wish to be at the forefront of undersea research and development. Please connect with NUWC Division Newport Recruiting at this site- https://www.navsea.navy.mil/Home/Warfare-Centers/NUWC-Newport/Career-Opportunities/ and follow us on LinkedIn @NUWC-Newport and on Facebook @NUWCNewport.