Computer science professor Nikolaus Correll and his lab at ŷ���ڱ���Ƶ Boulder have been awarded $1.8 million by the U.S. Department of Energy Advanced Research Projects Agency-Energy (ARPA-E) to help establish a circular supply chain for domestic electric vehicle (EV) batteries.

The percentage of EV passenger vehicles on the road is  to 28% by 2030 and 58% by 2040, globally.

The existing supply chain for EV batteries relies mostly on recycling to recover critical minerals such as cobalt, nickel or copper.

However, conventional battery recycling methods are energy-intensive, produce significant quantities of greenhouse gases, and lead to large volumes of waste deposited in landfills.

ŷ���ڱ���Ƶ Boulder joins 12 other projects around the country working to change this dynamic through ARPA-E's  program.

Correll's project focuses on autonomous robotic disassembly of EV lithium-ion battery packs. Humanoid robots will work together with robotic arms to manipulate wire harnesses and remove screws and other components before dismantling commercial battery packs with a heavy-duty industrial arm.

Correll explained that people are interested in using robots for the task due to the hazardous nature of the work.

"The batteries are quite dangerous to handle due to the risk of electrocution and spontaneous ignition," Correll said.

The Correll Lab's project will use state-of-the-art perception models and large-language models to consider the physics and context of each battery.

By advancing the efficiency and ability of battery disassembly systems, component recycling could be done at a commercial scale more safely and cost-effectively, leading to less waste in landfills and more material available for new EV batteries.

The director of ARPA-E, Evelyn N. Wang, said, "I look forward to seeing how these CIRŷ���ڱ���ƵLAR projects develop regeneration, repair, reuse, and remanufacture technologies to create a sustainable EV battery supply chain." 

Off

William Xie, a first-year PhD student in computer science, is teaching a robot to reason how gently it should grasp previously unknown objects by using large language models (LLMs). 

, Xie's project, is an intriguing step beyond the custom, piecemeal solutions currently used to avoid pinching or crushing novel objects. 

In addition, Deligrasp helps the robot translate what it can 'touch' into meaningful information for people. 

"William has gotten some neat results by leveraging common sense information from large language models. For example, the robot can estimate and explain the ripeness of various fruits after touching them." Said his advisor, Professor Nikolaus Correll. 

Let's learn more about DeliGrasp, Xie's journey to robotics, and his plans for the conference Japan and beyond. 

[video:https://www.youtube.com/watch?v=OMzTgY1gxLw]

How would you describe this research? 

As humans, we’re able to quickly intuit how exactly we need to pick up a variety of objects, including delicate produce or unwieldy, heavy objects. We’re informed by the visual appearance of an object, what prior knowledge we may have about it, and most importantly, how it feels to the touch when we initially grasp it. 

Robots don’t have this all-encompassing intuition though, and they don’t have end-effectors (grippers/hands) as effective as human hands. So solutions are piecemeal: the community has researched “hands” across the spectrum of mechanical construction, sensing capabilities (tactile, force, vibration, velocity), material (soft, rigid, hybrid, woven, etc…). And then the corresponding machine learning models and/or control methods to enable “appropriately forceful” gripping are bespoke for each of these architectures.

Embedded in LLMs, which are trained on an internet’s worth of data, is common sense physical-reasoning that crudely approximates a human’s (as the saying goes: “all models are wrong, some are useful”). We use the LLM-estimated mass and friction to simplify the grasp controller and deploy it on a two-finger gripper, a prevalent and relatively simple architecture. Key to the controller working is the force feedback sensed by the gripper as it grasps an object, and knowing at what force threshold to stop—the LLM-estimated values directly determine this threshold for any arbitrary object, and our initial results are quite promising.

How did you get inspired to pursue this research?

I wouldn’t say that I was inspired to pursue this specific project. I think, like a lot of robotics research, I had been working away at a big problem for a while, and stumbled into a solution for a much smaller problem. My goal since I arrived here has been to research techniques for assistive robots and devices that restore agency for the elderly and/or mobility-impaired in their everyday lives. I’m particularly interested in shopping (but eventually generalist) robots—one problem we found is that it is really hard to determine, let alone pick ripe fruits and produce with a typical robot gripper and just a camera. In early February, I took a day to try out picking up variably sized objects via hand-tuning our MAGPIE gripper’s force sensing (an affordable, open-source gripper developed by the Correll Lab). It worked well; I let ChatGPT calibrate the gripper which worked even better, and it evolved very quickly into DeliGrasp.

What would you say is one of your most interesting findings so far?

LLMs do a reasonable job of estimating an arbitrary object’s mass (friction, not as well) from just a text description. This isn’t in the paper, but when paired with a picture, they can extend this reasoning for oddballs—gigantic paper airplanes, or miniature (plastic) fruits and vegetables.

With our grasping method, we can sense the contact forces on the gripper as it closes around an object—this is a really good measure of ripeness, it turns out. We can then further employ LLMs to reason about these contact forces to pick out ripe fruit and vegetables!

What does the day-to-day of this research look like?

Leading up to submission, I was running experiments on the robot and picking up different objects with different strategies pretty much every day. A little repetitive, but also exciting. Prior to that, and now that I’m trying to improve the project for the next conference, I spend most of my time reading papers, thinking/coming up with ideas, and setting up small, one-off experiments to try out those ideas.

How did you come to study at ŷ���ڱ���Ƶ Boulder? 

For a few years, I’ve known that I really wanted to build robots that could directly, immediately help my loved ones and community. I had a very positive first research experience in my last year of undergrad and learned what it felt like to have true personal agency in pursuing work that I cared about. At the same time I knew I’d be relocating to Boulder after graduation. I was very fortunate that Nikolaus accepted me and let me keep pursuing this goal of mine.

It’d be unfathomable if I could keep doing this research in academia or industry, though of course that would be ideal. But I’m biased toward academia, particularly teaching. I’ve been teaching high school robotics for 5 years now, and now teaching/mentoring undergrads at ŷ���ڱ���Ƶ—each day is as fulfilling as the first. I have great mentors across the robotics faculty and senior PhD students we work in ECES 111, a giant, well-equipped space that 3 robotics labs share, and it’s great for collaboration and brainstorming. 

What are your hopes for this international conference (and what conference is it?)

The venue is a workshop at the 2024 International Conference on Robotics and Automation (ICRA 2024), happening in Yokohama, Japan from May 13-17. The name of the workshop is a mouthful: Vision-Language Models for Navigation and Manipulation (VLMNM).

A workshop is detached from the main conference, and kind of is its own little bubble (like a big supermarket—the conference—hosting a pop-up food tasting event—the workshop). I'm really excited to meet other researchers and pick their brains. As a first-year, I’ve spent the past year reading papers from practically everyone on the workshop panel, and from their students. I’ll probably also spend half my time exploring (eating) around the Tokyo area.
 

Off

Off

Off

Off

Off

Off

Prosthetics with a realistic sense of touch. Bridges that detect and repair their own damage. Vehicles with camouflaging capabilities.

Advances in materials science, distributed algorithms and manufacturing processes are bringing all of these things closer to reality every day, says a  by Nikolaus Correll, assistant professor of computer science, and research assistant Michael McEvoy, both of the University of ŷ���ڱ���Ƶ Boulder.

The "robotic materials" being developed by Correll Lab and others are often inspired by nature, Correll said.

"We looked at organisms like the cuttlefish, which change their appearance depending on their environment, and the banyan tree, which grows above-ground roots to support the increasing weight of the trunk," Correll said. "We asked what it would take to engineer such systems."

Robotic materials require tight integration between sensing, computation and actually changing the materials properties of the underlying material. While materials can already be programmed to change some of their properties in response to specific stimuli, robotic materials can sense stimuli and determine how to respond on their own.

Correll and McEvoy use the example of artificial skin equipped with microphones that would analyze the sounds of a texture rubbing the skin and route information back to the central computer only when important events occurred.

"The human sensory system automatically filters out things like the feeling of clothing rubbing on the skin," Correll said. "An artificial skin with possibly thousands of sensors could do the same thing, and only report to a central 'brain' if it touches something new."

While all of these materials are possible, the authors caution that manufacturing techniques remain a challenge.

"Right now, we're able to make these things in the lab on a much larger scale, but we can't scale them down," Correll said. "The same is true for nano- and microscale manufacturing, which can't be scaled up to things like a building façade."

The field also faces an education gap, the authors say. Developing robotic materials requires interdisciplinary knowledge that currently isn't provided by materials science, computer science or robotics curricula alone.

At ŷ���ڱ���Ƶ-Boulder, Correll is addressing that gap with a freshman-level engineering projects class called "Materials That Think."

"We expose engineering students to both materials and computing, no matter what their background is," he said.

In the long run, Correll believes robotic materials will be used in everyday items, like shoe insoles that could sense pressure and adapt their stiffness to adjust to walking or running.

"While we can imagine such a material consisting of little patches that each include an actuator, sensor and little computer, we have a hard time imagining such a complex piece of technology could ever be affordable," he said. "I think the last 10 years of advances in smartphones have demonstrated the opposite."

Off

Off

Off