Robots perform many tasks that humans can&rsquo;t or don&#39;t want to perform, getting around on intricately designed wheels and limbs. If they tip over, however, they are rendered almost useless. A team of University of Illinois mechanical engineers and entomologists are looking to click beetles, who can right themselves without the use of their legs, to solve this robotics challenge.The researchers presented their findings at Living Machines 2017: The 6th International Conference on Biomimetic and Biohybrid Systems at Stanford University, and later won second place in a student and faculty research competition at the international BIOMinnovate Challenge, in Paris, France &ndash; a research expo that showcases biologically-inspired design in engineering, medicine and architecture.<iframe src="https://www.youtube.com/embed/9ozz2jRfBUk?&wmode=opaque" allowfullscreen="" class="fr-draggable" style="width: 766px; height: 406px;" frameborder="0"></iframe> &ldquo;This idea came to life when a group of insect physiology students decided to take a closer look at what makes click beetles jump as part of a class project,&rdquo; said <a href="https://www.life.illinois.edu/entomology/index.html" target="_blank">department of entomology</a> research scientist and study co-author <a href="https://www.life.illinois.edu/entomology/faculty/alleyne.html" target="_blank">Marianne Alleyne.</a>The beetles have a unique hinge-like mechanism between their heads and abdomens that makes a clicking sound when initiated and allows them to flip into the air and back onto their feet when they are knocked over, Alleyne said.&ldquo;Very little research had been performed on these beetles, and I thought this legless jumping mechanism would be a perfect candidate for further exploration in the field of bioinspiration,&rdquo; said Alleyne, who teaches a bioinspiration design course with <a href="https://mechanical.illinois.edu/" target="_blank">mechanical science and engineering</a> professor, co-author and lead investigator <a href="https://mechanical.illinois.edu/directory/faculty/awissa" target="_blank">Aimy Wissa</a>.The researchers looked at several species of click beetles, ranging in size from a few just few millimeters to several centimeters in length.&ldquo;Each insect goes through an assembly line of analyses that involve basic characterization, high-speed filming to observe the jump and measurements in the <a href="https://publish.illinois.edu/tribology/" target="_blank">Materials Tribiology Lab</a> with co-author and mechanical science and engineering professor <a href="http://mechanical.illinois.edu/directory/profile/acd" target="_blank">Alison Dunn</a>, to determine how much force it takes to overcome the friction of the hinge within an individual beetles jumping mechanism,&rdquo; Wissa said. &ldquo;We observe, model and validate each stage of the jump with the hopes that we can later integrate them into a self-righting robot.&rdquo;<img src="https://s3-us-west-1.amazonaws.com/wevolver-project-images/froala%2F1568998642025-118939.jpg" style="width: 766px;" class="fr-fic fr-dib">The University of Illinois click beetle robotics team. Front row, from left: Professor Aimy Wissa, Aidan Garrett, Luis Urrutia, Ophelia Bolmin and professor Alison Dunn. Back row, from left: Isandro Malik, Xander Hazel, Robert Chapa, Harmen Alleyne, Chengfang Duan and research scientist Marianne Alleyne. Garrett, Malik and H. Alleyne are University of Illinois Laboratory High School students. Photo by L. Brian StaufferThe group has already built several prototypes of a hinge-like, spring-loaded device that will eventually be incorporated into a robot, the researchers said.&ldquo;This study is a two-way street &ndash; engineers are informing the biologists and vice versa,&rdquo; Wissa said. &ldquo;We look forward to seeing where this research will take us and are very proud of the attention it received at the BIOMinnovate Challenge expo.&rdquo;

Click beetles can jump without the aid of their limbs when they are tipped onto their backsides. A team of University of Illinois researchers are examining this mechanism to engineer self-righting robots.  Photo by L. Brian Stauffer

Robots perform many tasks that humans can’t or don't want to perform, getting around on intricately designed wheels and limbs. If they tip over, however, they are rendered almost useless.

Click beetles inspire design of self-righting robots

Compression therapy is a standard form of treatment for patients who suffer from venous ulcers and other conditions in which veins struggle to return blood from the lower extremities. Compression stockings and bandages, wrapped tightly around the affected limb, can help to stimulate blood flow. But there is currently no clear way to gauge whether a bandage is applying an optimal pressure for a given condition.Now engineers at MIT have developed pressure-sensing photonic fibers that they have woven into a typical compression bandage. As the bandage is stretched, the fibers change color. Using a color chart, a caregiver can stretch a bandage until it matches the color for a desired pressure, before, say, wrapping it around a patient&rsquo;s leg.The photonic fibers can then serve as a continuous pressure sensor &mdash; if their color changes, caregivers or patients can use the color chart to determine whether and to what degree the bandage needs loosening or tightening.&ldquo;Getting the pressure right is critical in treating many medical conditions including venous ulcers, which affect several hundred thousand patients in the U.S. each year,&rdquo; says Mathias Kolle, assistant professor of mechanical engineering at MIT. &ldquo;These fibers can provide information about the pressure that the bandage exerts. We can design them so that for a specific desired pressure, the fibers reflect an easily distinguished color.&rdquo;Kolle and his colleagues have published their results in the journal Advanced Healthcare Materials. Co-authors from MIT include first author Joseph Sandt, Marie Moudio, and Christian Argenti, along with J. Kenji Clark of the Univeristy of Tokyo, James Hardin of the United States Air Force Research Laboratory, Matthew Carty of Brigham and Women&rsquo;s Hospital-Harvard Medical School, and Jennifer Lewis of Harvard University.<img src="https://s3-us-west-1.amazonaws.com/wevolver-project-images/froala%2F1568994301001-single-fiber-stretch.gif" style="width: 766px;" class="fr-fic fr-dib">Natural inspirationThe color of the photonic fibers arises not from any intrinsic pigmentation, but from their carefully designed structural configuration. Each fiber is about 10 times the diameter of a human hair. The researchers fabricated the fiber from ultrathin layers of transparent rubber materials, which they rolled up to create a jelly-roll-type structure. Each layer within the roll is only a few hundred nanometers thick.In this rolled-up configuration, light reflects off each interface between individual layers. With enough layers of consistent thickness, these reflections interact to strengthen some colors in the visible spectrum, for instance red, while diminishing the brightness of other colors. This makes the fiber appear a certain color, depending on the thickness of the layers within the fiber.&ldquo;Structural color is really neat, because you can get brighter, stronger colors than with inks or dyes just by using particular arrangements of transparent materials,&rdquo; Sandt says. &ldquo;These colors persist as long as the structure is maintained.&rdquo;The fibers&rsquo; design relies upon an optical phenomenon known as &ldquo;interference,&rdquo; in which light, reflected from a periodic stack of thin, transparent layers, can produce vibrant colors that depend on the stack&rsquo;s geometric parameters and material composition. Optical interference is what produces colorful swirls in oily puddles and soap bubbles. It&rsquo;s also what gives peacocks and butterflies their dazzling, shifting shades, as their feathers and wings are made from similarly periodic structures.&ldquo;My interest has always been in taking interesting structural elements that lie at the origin of nature&rsquo;s most dazzling light manipulation strategies, to try recreating and employing them in useful applications,&rdquo; Kolle says.<img src="blob:https://www.wevolver.com/6c22c5d1-56bb-4ed0-a5ce-9b2933c079ca" style="width: 766px;" class="fr-fic fr-dib">A multilayered approachThe team&rsquo;s approach combines known optical design concepts with soft materials, to create dynamic photonic materials.While a postdoc at Harvard in the group of Professor Joanna Aizenberg, Kolle was inspired by the work of Pete Vukusic, professor of biophotonics at the University of Exeter in the U.K., on&nbsp;Margaritaria nobilis,&nbsp;a tropical plant that produces extremely shiny blue berries. The fruits&rsquo; skin is made up of cells with a periodic cellulose structure, through which light can reflect to give the fruit its signature metallic blue color.Together, Kolle and Vukusic sought ways to translate the fruit&rsquo;s photonic architecture into a useful synthetic material. Ultimately, they fashioned multilayered fibers from stretchable materials, and assumed that stretching the fibers would change the individual layers&rsquo; thicknesses, enabling them to tune the fibers&rsquo; color. The results of these first efforts were published in&nbsp;Advanced Materials in 2013.When Kolle joined the MIT faculty in the same year, he and his group, including Sandt, improved on the photonic fiber&rsquo;s design and fabrication. In their current form, the fibers are made from layers of commonly used and widely available transparent rubbers, wrapped around highly stretchable fiber cores. Sandt fabricated each layer using spin-coating, a technique in which a rubber, dissolved into solution, is poured onto a spinning wheel. Excess material is flung off the wheel, leaving a thin, uniform coating, the thickness of which can be determined by the wheel&rsquo;s speed.For fiber fabrication, Sandt formed these two layers on top of a water-soluble film on a silicon wafer. He then submerged the wafer, with all three layers, in water to dissolve the water-soluble layer, leaving the two rubbery layers floating on the water&rsquo;s surface. Finally, he carefully rolled the two transparent layers around a black rubber fiber, to produce the final colorful photonic fiber.&shy;&shy;Reflecting pressureThe team can tune the thickness of the fibers&rsquo; layers to produce any desired color tuning, using standard optical modeling approaches customized for their fiber design.&ldquo;If you want a fiber to go from yellow to green, or blue, we can say, &lsquo;This is how we have to lay out the fiber to give us this kind of [color] trajectory,&rsquo;&rdquo; Kolle says. &ldquo;This is powerful because you might want to have something that reflects red to show a dangerously high strain, or green for &lsquo;ok.&rsquo; We have that capacity.&rdquo;The team fabricated color-changing fibers with a tailored, strain-dependent color variation using the theoretical model, and then stitched them along the length of a conventional compression bandage, which they previously characterized to determine the pressure that the bandage generates when it&rsquo;s stretched by a certain amount.The team used the relationship between bandage stretch and pressure, and the correlation between fiber color and strain, to draw up a color chart, matching a fiber&rsquo;s color (produced by a certain amount of stretching) to the pressure that is generated by the bandage.<img src="https://s3-us-west-1.amazonaws.com/wevolver-project-images/froala%2F1568995401582-MIT-Bandage-Pressure-02.jpg" style="width: 766px;" class="fr-fic fr-dib">As the bandage is stretched, the fibers change color. Courtesy of the researchersTo test the bandage&rsquo;s effectiveness, Sandt and Moudio enlisted over a dozen student volunteers, who worked in pairs to apply three different compression bandages to each other&rsquo;s legs: a plain bandage, a bandage threaded with photonic fibers, and a commercially-available bandage printed with rectangular patterns. This bandage is designed so that when it is applying an optimal pressure, users should see that the rectangles become squares.Overall, the bandage woven with photonic fibers gave the clearest pressure feedback. Students were able to interpret the color of the fibers, and based on the color chart, apply a corresponding optimal pressure more accurately than either of the other bandages.The researchers are now looking for ways to scale up the fiber fabrication process. Currently, they are able to make fibers that are several inches long. Ideally, they would like to produce meters or even kilometers of such fibers at a time.&ldquo;Currently, the fibers are costly, mostly because of the labor that goes into making them,&rdquo; Kolle says. &ldquo;The materials themselves are not worth much. If we could reel out kilometers of these fibers with relatively little work, then they would be dirt cheap.&rdquo;Then, such fibers could be threaded into bandages, along with textiles such as athletic apparel and shoes as color indicators for, say, muscle strain during workouts. Kolle envisions that they may also be used as remotely readable strain gauges for infrastructure and machinery.&ldquo;Of course, they could also be a scientific tool that could be used in a broader context, which we want to explore,&rdquo; Kolle says.

Engineers at MIT have developed pressure-sensing photonic fibers that they have woven into a typical compression bandage  Courtesy of the researchers

Bandage is threaded with photonic fibers that change color to signal pressure level.

Engineers design color-changing compression bandage

The work blurs the boundary between materials and robots. In the self-propelled devices, the material itself makes the machine function. "Our examples show that we can use structured materials that deform in response to environmental cues, to control and propel robots," says Daraio, professor of mechanical engineering and applied physics in Caltech's Division of Engineering and Applied Science, and corresponding author of a paper unveiling the robots that appears in the Proceedings of the National Academy of Sciences&nbsp;on May 15.The new propulsion system relies on strips of a flexible polymer that is curled when cold and stretches out when warm. The polymer is positioned to activate a switch inside the robot's body, that is in turn attached to a paddle that rows it forward like a rowboat.The switch is made from a bistable element, which is a component that can be stable in two distinct geometries. In this case, it is built from strips of an elastic material that, when pushed on by the polymer, snaps from one position to another.<iframe src="https://www.youtube.com/embed/tBxb1A6boTI?&amp;wmode=opaque" allowfullscreen="" class="fr-draggable" style="width: 767px; height: 439px;" frameborder="0"></iframe>When the cold robot is placed in warm water, the polymer stretches out, activates the switch, and the resulting sudden release of energy paddles the robot forward. The polymer strips can also be "tuned" to give specific responses at different times: that is, a thicker strip will take longer to warm up, stretch out, and ultimately activate its paddle than a thinner strip. This tunability allows the team to design robots capable of turning and moving at different speeds.The research builds on previous work by Daraio and Dennis Kochmann, professor of aerospace at Caltech. They used chains of <a href="http://www.caltech.edu/news/utility-instability-51629" target="_blank">bistable elements to transmit signals and build computer-like logic gates</a>.<img src="https://s3-us-west-1.amazonaws.com/wevolver-project-images/froala%2F1564499883153-Screenshot_2019-07-30+Harnessing+bistability+for+directional+propulsion+of+soft%2C+untethered+robots+-+5698+full+pdf%281%29.png" style="width: 766px;" class="fr-fic fr-dib">In the latest iteration of the design, Daraio's team and collaborators were able to link up the polymer elements and switches in such a way to make a four-paddled robot propel itself forward, drop off a small payload (in this case, a token with a Caltech seal emblazoned on it), and then paddle backward.&nbsp;"Combining simple motions together, we were able to embed programming into the material to carry out a sequence of complex behaviors," says Caltech postdoctoral scholar Osama R. Bilal, co-first author of the PNASpaper. In the future, more functionalities and responsivities can be added, for example using polymers that respond to other environmental cues, like pH or salinity. Future versions of the robots could contain chemical spills or, on a smaller scale, deliver drugs, the researchers say.<img src="https://s3-us-west-1.amazonaws.com/wevolver-project-images/froala%2F1564499825448-1.jpg" style="width: 766px;" class="fr-fic fr-dib">Currently, when the bistable elements snap and release their energy, they must be manually reset in order to work again. Next, the team plans to explore ways to redesign the bistable elements so that they are self-resetting when water temperature shifts again—making them potentially capable of swimming on indefinitely, so long as water temperature keeps fluctuating.The PNAS&nbsp;paper is titled <a href="http://resolver.caltech.edu/CaltechAUTHORS:20180515-101708023" target="_blank">"Harnessing bistability for directional propulsion of soft, untethered robots."</a> Daraio and Bilal collaborated with Tian Chen and Kristina Shea from ETH in Zurich. This research was supported by the Army Research Office and an ETH postdoctoral fellowship to Bilal.

 An artist's rendering of the new design for a robot that uses material deformation to propel itself through water. 

Engineers have developed robots capable of self-propulsion without using any motors, servos, or power supply. Instead, these first-of-their-kind devices paddle through water as the material they are constructed from deforms with temperature changes.

Self propelling robots

The android called Kannon Mindar. Photo by Richard Atrero de Guzman 

As icons and rituals adapt to newer technologies, the rise of robotics and AI can change the way we practice and experience spirituality.

Robots and Religion: Mediating the Divine.

You finally have it. The most awesome robot, with all the sensors and actuators you’ve ever dreamed of. LIDAR, RADAR, vision, far-field microphone array, pressure-sensitive touch, precision GPS, and odometry. Speakers, screens, legs, arms, hands, and maybe even a flamethrower. This robot has everything. Maybe it’s something you’ve put together in your garage in your spare time. Whichever, many years of work, research, and rework have gone into building your dream robot, which you’re confident can sense and do anything.So now the real work begins — you need to somehow get this robot to do what you want.<img src="https://s3-us-west-1.amazonaws.com/wevolver-project-images/froala%2F1562343304603-1+u2VI1R_l3BbOS0F8E0Tn5A.jpeg" style="width: 766px;" class="fr-fic fr-dib">Let’s say you’ve got a task that you want this robot to perform. It could be anything, from juggling 6 flaming chainsaws to helping a stroke victim do their exercises to baking an apple pie. It could be the key to your money-making robot company concept, or just a useful behavior to help you around the house. Whatever it is, inside your brain, latent in your mind, is a policy that, if transferred to the robot, would make it perform the task just the way you want it. The problem of getting the task from your mind to the robot is what we call Human-Robot Policy Transfer (HRPT). And there are a few different ways to approach it.Teleoperation The first, and generally simplest, way to get a robot to do something is direct teleoperation. That is, having a human directly control the robot to perform the task. You’ll need an appropriate method of controlling the robot, which could be anything from a set of joysticks to a full-body telemetry suit. You’ll also need a human, possibly you (or a paid expert), to control the robot, or maybe even a whole team of humans, each controlling a different part of the robot in different ways. Whatever the control system, the idea is that humans implement the control policy themselves. That is, a human observes the robot’s state either directly or through sensors, then tells the robot how to use its actuators (such as manipulators, cutters, lifters, etc.) to achieve the task. With teleoperation, even though some low-level systems (for example, balance) might be automated, all high-level decision-making is done by a human.There are many advantages to teleoperation. First and foremost, it will save you the trouble of figuring out how to state your goals explicitly; that is, the policy can remain latent in your mind. This feature is a big help if the policy is fairly complex. When you consider the effort it could take to program a robot to perform a complicated, multi-step task, it can be clear that simply controlling the robot yourself might get the job done more quickly.Further, with a human in the loop, adapting to a changing environment can be done immediately. Additionally, since a human is making the decisions, legal and ethical questions of liability and responsibility are easier to answer. An excellent example of teleoperation for HRPT is the use of surgical robots, where the precision and scale of the robot are well-used by the complex — and unexpressed — control policy inside the human surgeon’s head.<img src="https://s3-us-west-1.amazonaws.com/wevolver-project-images/froala%2F1562343339019-1+Pa9G1zlq4JA9ysd6driv0g.png" style="width: 766px;" class="fr-fic fr-dib"><figcaption>Intuitive Surgical’s DaVinci robotic surgery system.</figcaption>There are, however, also many drawbacks to teleoperation as an HRPT approach. The main one is that, since humans have to be involved with teleoperated robots at every moment, it doesn’t really achieve the dream of developing autonomous robots. A teleoperated robot doesn’t free humans from doing work or enable one human to multiply their effectiveness and become as productive as 10 people. (It does, however, allow humans to remove themselves from dangerous situations, which is sometimes the entire point.)Similarly, since a human is, in effect, still doing the work with teleoperation, all the faults and issues of humans come through. For example, teleoperators get tired and distracted, show up late, or just make mistakes — all things that we expect (perhaps incorrectly) our robots not to do.As a first-pass tool, teleoperation is great, in that it proves that the task you want your robot to perform is actually performable by the robot. It’s good to know that the robot’s hardware is physically capable of doing the task you want it to do, before you try and develop an autonomous control policy for it.<img src="https://s3-us-west-1.amazonaws.com/wevolver-project-images/froala%2F1562343368819-1+PTPiPsT1Umcvd3Qf7d_DVA.png" style="width: 766px;" class="fr-fic fr-dib"><figcaption>Real-time teleoperation of the Mahru robot developed at the Korea Institute of Science and Technology.</figcaption>Further, if you limit yourself to only using the robot’s sensor information during teleoperation, you can prove that the task is decideable by the robot. That is, that the robot has access to enough information to actually determine what to do in response to its sensor stream. Again, this is highly useful to know before you pour hours/weeks/months/years into trying to get the robot to do the task on its own.So let’s say you’ve proved that the task you want your robot to accomplish is actually decideable and performable by this awesome robot body, and now you want to make the robot perform the task by itself. Great! How’re you going to do that? Probably with the second major approach to HRPT.Explicit ProgrammingThat right. You’re going to sit down and try to — explicitly and in great detail, in your favorite programming language — write out exactly what the robot needs to do, how it needs to react to its environment, in order to accomplish the task.<img src="https://s3-us-west-1.amazonaws.com/wevolver-project-images/froala%2F1562343408817-1+sPNDae_0Ib0kK4TjnwbFpQ.jpeg" style="width: 766px;" class="fr-fic fr-dib"><figcaption>Programming the LEGO(R) MINDSTORMS(R) EV3.</figcaption>Honestly, to consider every possible situation that could potentially happen while your robot goes about its business would take too long (there are an infinity of them, after all, and even people paid to come up with odd situations miss a bunch). So, you’ll likely make some assumptions about what situations are unlikely to arise and ignore them in your code. Do you really need to consider what should happen if your robot encounters a <a data-href="http://www.siliconbeat.com/2016/04/05/google-self-driving-car-brakes-for-wheelchair-lady-chasing-duck-with-broom/" href="http://www.siliconbeat.com/2016/04/05/google-self-driving-car-brakes-for-wheelchair-lady-chasing-duck-with-broom/" rel="nofollow noopener" target="_blank">wheelchair/duck/broom combo</a>?Even ignoring the more unlikely scenarios, there’s a lot of work required to program a robot to perform anything other than the most basic tasks in a wide variety of different contexts. In fact, for a given level of programming effort (say, a month), as the complexity of the task goes up, the variance of the environment that it can be programmed to operate in goes down.Sure, it’s easy to code a simple task (drive in a square) to be performed in a wide variety of environments (different floor surfaces/rooms). It’s also just about as easy to program a complex task (set the table) in one particular environment (all plates/utensils and tables/chairs in fixed, known locations). But programming a complex task for varying environments (setting the table in any kitchen) is much, MUCH harder. All of this effort (designing, architecting, coding, debugging, and testing) can in fact be a lot more work than just doing the task yourself.<img src="https://s3-us-west-1.amazonaws.com/wevolver-project-images/froala%2F1562343435763-1+sifPS1Kt67_XzFI5hdnezw.png" style="width: 766px;" class="fr-fic fr-dib"><figcaption>Simulation and offline programming software by Staubli Robotics.</figcaption>So why bother trying to get a robot to do complex tasks autonomously? Because once your programming works — if it works — the robot can perform the task over and over and over again. With luck, you’ll recoup your effort in the long run.Almost all autonomous robots you’re likely to have interacted with up to now (outside of research lab demos) have gone through this process. They’ve been painstakingly, explicitly coded by a team of developers to perform specific tasks in a set of constrained environments. Change one parameter (the height of a step, for example) beyond the constrained range, and the whole system comes tumbling down like, well, a robot trying to walk.LearningThe third main approach to HRPT is to use learning: having a robot figure out how to perform the task by analyzing data.Now, there are many different types of machine learning using many different technologies, but they all try and get at the same goal: to find patterns in data that are useful for accomplishing a desired task. The task might be identifying a cat in an image, predicting the stock market, or determining a control policy for a robot. When it comes to HRPT, this last one is exactly what we need.A conceptually simple way to have a robot learn to perform a task is to tell the robot what the goal of the task is, and let the robot experiment to figure out how to achieve this goal. This is reinforcement learning (RL), where the robot gets a reward when the task is accomplished (or pays a penalty when the task is NOT accomplished) and tries to figure out how to use its capabilities to get to the goal. The reward itself is generated by a reward function that maps the robot’s state (or robot state-action pairs) to how good (or bad) it is for the robot to be in that state (or perform that action in that state).<img src="https://s3-us-west-1.amazonaws.com/wevolver-project-images/froala%2F1562343607363-1+FawajzcHQaLKocy5aNYDTQ.png" style="width: 766px;" class="fr-fic fr-dib"><figcaption>Rewarding robots does not usually look like this.</figcaption>On one hand, RL can be really easy to use, as the end-state, or desired goal, is often simple to describe. When a simple end-state is all that matters, the reward function is easy to write: If there is an omelette on my plate, +1000, else 0.On the other hand, to find its way to the desired end-state, a robot might have to explore all of the possible configurations it and the world around it could be in (its state space) and all of the possible actions it could perform (its action space) in each of those states. If the state and action spaces are large, it can take a very, very long time for the robot to find ANY path to the goal, let alone an optimal one. But, for restricted situations with small state-spaces and limited action-spaces (such as simulated systems in <a data-href="http://www.prismmodelchecker.org/casestudies/robot.php" href="http://www.prismmodelchecker.org/casestudies/robot.php" rel="nofollow noopener" target="_blank">a grid world</a>), RL can often lead to robust and optimal control policies.One way to limit RL issues in larger spaces is to supply additional rewards, say, for when the robot accomplishes intermediate states on the way to the goal. Or, a developer can define heuristics to guide the robot toward more task-appropriate behavior. Unfortunately, providing these types of information make the reward function more complicated and difficult to write down and can approach the complexity of explicit programming.Learning… from DemonstrationAnother learning approach bypasses the tricky parts of RL by just letting humans provide the robot with helpful examples, or demonstrations, of what it should do to accomplish the desired task. This approach is called Learning from Demonstration (LfD) or, sometimes, Programming by Demonstration (PbD).<img src="https://s3-us-west-1.amazonaws.com/wevolver-project-images/froala%2F1562343653832-1+fLwb1fb_wDXc43LttIfhig.jpeg" style="width: 766px;" class="fr-fic fr-dib"><figcaption>Part of the Learning from Demonstration Challenge at the Association for the Advancement of Artificial Intelligence 2011 conference.</figcaption>LfD only assumes that you, the human, can perform the task you want the robot to do. Interestingly enough, one of the most common forms that demonstrations take are teleoperations of the robot through the task. Using teleoperation in service of robotic learning nicely leverages the benefits of that method of HRPT, namely proving the performability of the task by the robot.Another form of generating demonstrations is direct kinesthetic manipulation, where the robot is physically guided through the task, again proving performability. Unfortunately, this approach (and teleoperation as well), can prove difficult when a robot has many degrees-of-freedom (think 2 arms, each with 7 joints, as well as legs, a neck, etc).<img src="https://s3-us-west-1.amazonaws.com/wevolver-project-images/froala%2F1562343688526-1+EdrY_RpaRODV9RHiy428Fg.jpeg" style="width: 766px;" class="fr-fic fr-dib"><figcaption>When Rethink Robotics was still developing robots, they were very clear — their robots are “trained, not programmed”, in this case by direct kinesthetic manipulation.</figcaption>A much-sought-after LfD approach is to have the robot merely observe the human perform the task themselves. If you can solve the correspondence problem (which parts of the human correspond to which parts of the robot), observation can be an extremely powerful way to collect demonstration data.<img src="https://s3-us-west-1.amazonaws.com/wevolver-project-images/froala%2F1562343743988-1+IwXJOsXnrS4Ts0z1VZxSSg.jpeg" style="width: 766px;" class="fr-fic fr-dib"><figcaption>Sylvain Calinon demonstrates robotic learning <a data-href="http://calinon.ch/research_prev.php" href="http://calinon.ch/research_prev.php" rel="nofollow noopener" target="_blank">via observation</a>.</figcaption>So let’s say you’ve got some nice demonstrations of how the task should go. What’s next?Simply having the robot play back the demonstration is not enough, as the world is not generally guaranteed to behave exactly as it did before. So, instead, many LfD techniques build a model of the demonstration data, capturing both the expected behavior and allowable variance around it. One common approach is to take the demonstrations, warp them all to be the same length in time, and then build a dynamical model of how the robot’s state evolves over time. You can even guarantee stability of the system, by imposing different constraints on it. These systems can be used to do some pretty cool things, like <a data-href="https://www.researchgate.net/publication/228975860_Learning_to_Play_Mini-Golf_from_Human_Demonstration_using_Autonomous_Dynamical_Systems" href="https://www.researchgate.net/publication/228975860_Learning_to_Play_Mini-Golf_from_Human_Demonstration_using_Autonomous_Dynamical_Systems" rel="nofollow noopener" target="_blank">play mini-golf</a>.Still, a common issue with LfD is that the learned control policy is often limited to being as good as the human demonstrator. A different approach, termed Inverse Reinforcement Learning (IRL) is designed to combine the demonstration-based approach of LfD with the optimization powers of RL.In this family of techniques, instead of relying just on a control policy derived directly from demonstrations, the system first tries to estimate the reward function underlying them. By using both the reward function and an initial control policy based on the demonstrations, IRL systems can optimize for task performance. Eventually, it’s even possible for such a system to perform a task better than the humans themselves! A classic example of this is the <a data-href="http://heli.stanford.edu/" href="http://heli.stanford.edu/" rel="noopener nofollow" target="_blank">Stanford Autonomous Helicopter project</a>.<img src="https://s3-us-west-1.amazonaws.com/wevolver-project-images/froala%2F1562343771925-1+wY-4vOkvEwnjRyS_WJTGYQ.jpeg" style="width: 766px;" class="fr-fic fr-dib"><figcaption>The goal of the Stanford University project was to push the state of the art in autonomous helicopter flight: extreme aerobatics under computer control.</figcaption>I could go on and on, as there is so much interesting work in the field of LfD. There’s work that looks at discovering reusable sub-skills from demonstrations, so new, different tasks can be accomplished. And there’s work that looks at giving feedback to the human during the training process so they can generate more useful training data. And then there’s my own work that has looked at how to determine when there are multiple correct ways to achieve a goal, or how to extract useful information from failed demonstration attempts, instead of assuming the human can always perform the task correctly.As exciting as it is, learning as a solution to HRPT still has drawbacks. The major issue, as with any learning system, is how to guarantee limits on the robot’s behavior. With direct teleoperation, or explicit programming, you can say with some degree of confidence that the robot butler will never put mustard in your coffee (barring perceptual issues, or a very tired or malicious human teleoperator). However, with a learned system, you’re never quite certain what the robot will do in any given situation.And that’s where the real fun begins.

Capio Upper Body Exoskeleton for Teleoperation by the DFKI GmbH Robotics Innovation Center.

Because your robot’s not going to program itself (yet)

How to Get a Robot to Do What You Want

This is the first article, in a series, exploring the vast landscape of artificial muscle actuators. In the subsequent articles, the quest to create and harness artificial muscle technology will be explored.Until recently, conventional robots have utilized rigid materials and fabrication methods. Ironically, traditional robots are relatively weak and brittle, when compared with biological organisms, which utilize soft adaptive tissues to produce movement. A new approach to robotics has taken shape in the form of soft robotics and biomimicry.So, what does “soft” robotics mean? If you ever stayed up late as a child and remember watching old science fiction movies, as I did, your immediate concept of a “Robot” is typically a large, metal creature, lumbering about. In sharp contrast to this classic robot concept, soft robots are flexible and agile; think of a ballerina or an athlete. Obviously, modern robots have advanced from this stereotype, however our best robot is still not capable of the nuanced movements and physical strength of a human.A key component to producing a robot that more closely replicates the movement of a human, starts with the actuation system. An actuator is any mechanism that can convert some form of energy into physical movement, similar to how our muscles work. Typically, servo motors are used to produce movement in most robotic systems. However, servo motors are bulky, have a low power to weight ratio(when compared to muscle tissue) and are non-compliant. Essentially, compliance of a material means that it has some give to it, like a rubber band. You may not realize it, but the human body is capable of withstanding incredible forces.As demonstrated at the 2015 DARPA Robotics Challenge, a humanoid robot struggled to accomplish physical tasks that any average human would be able to do with ease. Common bumps and bruises that humans experience, would be catastrophic to most modern robots. Our ability to adapt to different physical environments is the template that will be used to create future robots.Artificial Muscle Research Engineers have sought a multitude of materials and fabrication methods in order to replicate the attributes of biological muscle tissue. I have also spent close to 20 years attempting to create an artificial muscle actuator, with limited success. So, what makes muscle tissue so special and replication of its attributes so elusive? Well, for starters, nature has had a 560 million year head start. Additionally, each of our muscles is a compact and powerful actuator that is made of soft, self-healing tissue. Our muscle tissue affords us dynamic movement and uses an efficient chemical fuel as a power source, glucose. Muscle tissue is composed of thousands of contractile fibers-within each fiber are up to 100,000 molecular units that synchronously shorten in length with a tremendous amount of force and velocity. This force and velocity enables basic movement and locomotion.Extensive research has been conducted to create an artificial muscle actuator which can replicate or exceed biological muscle tissue, with varying levels of success. Several types of technologies that have been explored include: Electroactive polymers (EAPs), Shape-memory metal alloy wires/liquid crystalline elastomers (SMAs), the Mckibben air muscle (Pneumatic artificial muscles), nylon fiber (torsional actuators) and Carbon fiber (PDMS rubber/torsional actuators). While all of these types of artificial muscle actuators, possess some of the sought after attributes of biological muscle, none of them are a complete solution in and of themselves. In order to fully appreciate the vast landscape of artificial muscle research and current innovations-each of these technologies will be further discussed; including history, fabrication techniques, advantages, drawbacks and future applications.<img src="https://s3-us-west-1.amazonaws.com/wevolver-project-images/froala%2F1561300296387-materiability2.jpg" style="width: 752px;" class="fr-fic fr-dib">EAP(Electroactive Polymer) ActuatorElectroactive polymers (EAPs) EAP technology is based upon the principle of generating a motive force in polymeric materials via an applied electrostatic field. One of the basic principles behind this technology is static electricity, if you have ever heard the crackle when you remove laundry from the dryer or accidentally, “shocked” your friend when walking, you have experienced static electricity. The electroactive effect occurs when a high voltage is applied to a flexible material, the electric static field that is generated is strong enough to produce movement in the material.Another technique used for actuation is based on elastomeric, meta-materials that exhibit the piezoelectric effect. Piezoelectric materials are actually very common, they are the technology that enables, haptic feedback in your cell phone. Typically, Piezoelectric materials are in the form of quartz crystals, which have the incredible ability to convert physical stress into electricity and vice versa. Moreover, certain polymer compounds can be, “doped”, with certain metals to produce the piezoelectric effect.&nbsp;While EAPs can embody, several permutations, the main focus of this discussion will be on the most commercially viable configurations which include, dielectric elastomers and ionic polymer-metal composites. EAP research started as early as the 1800s, however it wasn't until the turn of the century that EAP technology was realized for practical applications.Physicist, Dr. Yoseph Bar-Cohen, of the Jet Propulsion Laboratory, further advanced EAP technology by inspiring a global research initiative which challenged research groups around the world to compete in the world's first, “Arm wrestling Match of EAP Robotic Arm against Human challenge”. In 2002, Japanese tech company Eamex, released one of the first commercial products-a realistic robotic fish toy, utilizing a viable EAP, artificial muscle. Continued research, funded by DARPA, resulted in the tech start-up; Artificial Muscle, Inc-which specializes in development and manufacture of products based on EAP technology.<img src="https://s3-us-west-1.amazonaws.com/wevolver-project-images/froala%2F1561300403609-image022.jpg" style="width: 748px;" class="fr-fic fr-dib">Arm wrestling Match of EAP Robotic Arm against Human challenge(human opponent, Panna Felsen)Dielectric Elastomeric Actuator Dielectric elastomeric architecture involves an elastomeric film or cross sectional area of polymeric material, coated with an electrically conductive layer on either side. This configuration yields a compliant capacitor, where in, the conductive coating serves as a pair of electrodes. When an electrical field is applied to the DEA, changes in the electric field distribution of the electrodes generate a mechanical strain via Coulombic charge interaction of the compliant electrodes. Essentially, the conductive electrodes become electrostatically attracted and repelled in specific regions, causing the material to compress and expand. Additionally, the strain deformation state can be held in a static position via a constant DC voltage; analogous to an isometric muscle contraction.Composition of a DEA is relegated to a suitable elastomeric material with high elastic modulus; typically silicone or acrylic elastomers are used. During the fabrication process, elastomeric film of a predetermined thickness, is stretched upon a rigid frame in order to provide bidirectional static prestrain(stretch) of the elastomeric film. This prestrain fabrication technique has been shown to significantly increase the &nbsp;mechanical efficiency and responsiveness of DEAs. The electrically conductive coating, typically comprises conductive carbon grease or paste, applied to either side of the elastomeric film to provide diametrically opposed electrodes. This fabrication process can also be repeated several times, with each elastomeric film/electrode layer, “stacked” onto the subsequent layer in order to provide an actuator capable of exerting much larger forces.Ionic Polymer-Metal CompositesIPMC actuators are characterized by synthetic polymers engineered with positively charged metal ions, which are randomly arranged in an aqueous, polymer matrix. Typically, a noble metal coating is applied to the polymer-metal matrix in order to provide positive/negative electrodes(orientation of cathode/anode is based upon the terminal electrical wires attachment). If you remember from Chemistry class, noble elements are those that do not play well with other elements or resist chemical interaction. The positive metal ions partially entangle with water molecules and any inherent, negatively charged ions within the polymer matrix. When an electrical field is applied, the conjugated metal ions are attracted to the negatively charged metal electrode. This ionic migration of entangled water molecules provides an osmotic pressure shift resulting in mechanical strain across the polymer matrix membrane. Conversely, mechanically bending or deformation of the polymer matrix can produce electrical potential and output voltage, which can be advantageous for energy harvesting and biomedical sensors.Edge Over Other ActuatorsPossessing many biomimetic attributes has warranted continued research into EAPs development. Similar to biological muscle tissue, Dielectric Elastomers have an inherent ability to store large amounts of mechanical energy and contract with a large amount of usable force. Another appealing attribute is measured by how well an actuator can operate under continued stress and adaptability under varied conditions. A crucial element when considering building a humanoid robot that can work along side humans. With this in mind, DEAs have been shown to operate within standard room conditions, without the need for additional considerations. A crucial element for any biomimetic actuator is the ability to respond quickly when activation stimuli is applied, DEAs continue to excel with rapid response activation times measuring in the milliseconds.Additionally, Ionic Polymer-Metal Composites bring a unique set of properties to the table, including bi-directional actuation. Dependent upon the polarity of the activation voltage applied, IPMCs can reverse actuation direction, contributing further to adaptable mobility. Due to the unique chemistry of IPMCs, the mechanism of movement is attributed mostly to molecular motion, which allows for low activation voltages. An attribute that is extremely appealing in the quest to create a sustainable, autonomous robot. Configuration of IPMCs is not only constrained to simple membranes. There complaint nature allows the material to be rolled up into compact cylinders, with the ability to bend or extend. A quality that is important in order to approximate various muscle shapes and sizes, inherent in biological organisms.<img src="https://s3-us-west-1.amazonaws.com/wevolver-project-images/froala%2F1561300566769-a-Optical-image-of-the-bending-actuation-of-the-i-EAP-actuator-under-AC-electric.jpg" style="width: 300px;" class="fr-fic fr-dib">(a) Optical image of the bending actuation of the EAP actuator under AC electric signals.(b) Schematic of bending actuation mechanism.Challenges In The FieldWhile EAPs do posses several remarkable properties, we must also evaluate the engineering challenges faced in the development of EAPs for practical applications. One such hurdle to overcome for any robotic system is power consumption, especially within the context of realizing a self-contained humanoid robot. Unfortunately, this is a major con for DEAs, which require thousands of volts of electricity to operate. With the current state of battery technology this is far from ideal. EAP longevity is also limited due to the high voltage field, often exceeding the breakdown threshold of the polymer material. While IPMCs require minimal activation voltage, additional considerations are required for typical usage. Ironically, not unlike a biological muscle, IPMCs also require encapsulation in an impermeable membrane to prevent dehydration of the polymer matrix. In addition to IPMCs low actuation force, the most critical disadvantage remains the current inability to commercially, mass produce, a consistent ionic polymer. Until these challenges can be overcome, a viable EAP artificial muscle actuator remains elusive.Looking AheadWhat's in store for EAP technology? While an EAP based artificial muscle actuator seems unlikely in the immediate future. EAP technology is currently in development towards, biomechanical energy harvesting. Specifically, IPMC films can be attached to key flexing areas of the human body in order to harvest kinetic energy. So, next time you go to the gym, you can burn calories and charge your cell phone. This technology could also turn out to be invaluable for Soldiers in the midst of combat with little to no recharge options. The structure and mechanism of DEAs has found a practical application within the realm of virtual reality systems. An array of actuator “pixels” can be utilized for graphic displays for the visually impaired and possibly for theme park applications, such as virtual-haptic, hybrid environments. The National Science Foundation is continuing research towards an artificial muscle application, however. With the aid of 3-D printing technology, IPMC technology may prove to one day be a suitable artificial muscle solution.<img src="https://s3-us-west-1.amazonaws.com/wevolver-project-images/froala%2F1561300675742-3D_printed_IPMC_hand.jpg" style="width: 750px;" class="fr-fic fr-dib">A small 3-D printed ionic polymer-metal composite soft robotic hand. National Science Foundation researchers are working to transform this material into artificial muscles. This is the first article in a series&nbsp;about artificial muscles, created by&nbsp;<a href="https://www.wevolver.com/profile/samuel.manriquez" rel="noopener noreferrer" target="_blank">Samuel Manriquez</a>.

Skeleton with an artificial bicep. Credit: Aslan Miriyev - Columbia Engineering

The current state of robotic actuators and mechanical control systems are failing to keep pace with advancing AI software, exploring novel actuator technologies may bridge the gap between man and machine.

Humanoids

Latest Posts

Click beetles inspire design of self-righting robots

Engineers design color-changing compression bandage

Self propelling robots

Robots and Religion: Mediating the Divine.

How to Get a Robot to Do What You Want

Biomechanical marvel-creating an artificial muscle actuator