Automatic speech recognition and translation systems such as the Lecture Translator of the Karlsruhe Institute of Technology (KIT) can convert the spoken word in lectures into a text in several languages in real time. Such systems improve access to information for students with disabilities and foreign students. They also speed up work and learning in general through intelligent post-processing and archiving of spoken texts. KIT researchers have added new functions to the Lecture Translator. Automatic recognition of the spoken word in several languages at the same time, text segmentation and title generation in real time, summaries and links to technical terms as well as queries on what has been heard now make it easier to understand and efficiently process lectures."With Lecture Translator's automatic simultaneous translation, we have brought spoken lectures closer to an international audience. However, this usually represents only 15 percent of the audience. With the new AI tools, we want to break down not only language barriers, but also comprehension barriers," says Alexander Waibel, Professor of Computer Science at KIT. "Automatically transcribed texts of spoken language are often difficult to read, because they appear too quickly as a long text without paragraphs and subheadings - just as the lecture was delivered orally". Processing the lecture is also time-consuming, because you have to search the lecture for gaps in understanding," says Waibel.<h2>Better overview of documents</h2>The further development of the Lecture Translator provides a remedy. The researchers have developed several new automatic functions such as "Smart Chaptering", "Summarization", "Q&amp;A" and "Auto-Links". A new type of artificial intelligence (AI) that automatically recognizes language converts the spoken text into a transcript in multiple languages and automatically identifies paragraphs, chapter headings and key points. It also creates an acoustic rendition in which the user can select one of 18 languages. The program also automatically displays links as cross-references to relevant sources in lecture notes or Wikipedia, which students can use to better process the lecture. "With our new AI models, conversations and lectures can be better structured and even videos can be divided into easily navigable chapters," says Waibel. This enables better understanding not only during the lecture, but also after the lecture.<h2>Lecture Translator translates into 18 languages</h2>The research team has integrated the work into the Lecture Translator, which is used at KIT to automatically transcribe lectures in real time. Chapter division, title generation, paragraph layout, summaries with links - which can also be used online and offline - now extend the Lecture Translator service and make it easier to work with the material. Translation is available in 18 languages. The technology has specific applications for content creators, students, teachers and podcasters who can structure their audio and video content for the first time. "Users can navigate through videos and lectures more efficiently, find relevant sections more quickly, and capture important core content more compactly and efficiently - they have a much better overall view and faster access to details," says Waibel.The research was carried out as part of the project "How is AI Changing Science?" and was funded for four years by the Volkswagen Foundation. In addition to KIT, the University of Bonn and the University of Vienna were also involved in the project.<a href="https://www.kcist.kit.edu/" target="_blank">Details on the KIT Center Information - Systems - Technologies</a>

With new functions such as automatic chapter division and summaries, the Lecture Translator offers improved speech recognition. (Photo: Markus Breig, KIT)

IT Researchers Further Develop Intelligent Speech Recognition and Translation System: Automatic Text Segmentation, Summarization, and Intelligent Query Possible

Automatic Speech Recognition and Translation: Faster Working and Learning

<h1 id="isPasted"> </h1><section>Electronic design thrives on innovation, but keeping pace can be a challenge. Electrical engineers are constantly pressured to develop groundbreaking products quickly. Traditionally, translating these ideas into ˚physical layouts has relied heavily on the assistance of PCB designers. While this collaboration is vital, it can be time-consuming.This is where Artificial Intelligence (AI) steps in. AI-powered PCB design assistants are shaking things up by offering a powerful tool: the freedom to explore.<h2>Identifying Alternate Design Avenues, Fast</h2>Imagine an experienced electrical engineer tasked with creating a prototype for a novel smart device with various sensors. Traditionally, such a complex layout utilizing multiple components in a restricted space would involve collaborating with a PCB designer to create a single layout.However, with AI design tools, the engineer can explore many possibilities. These tools can generate several variations of the same PCBA, each with different dimensions, connector placements, and design constraints.This ability to rapidly explore diverse options is a game-changer. It empowers engineers to experiment with various design concepts before settling on a final layout. This translates into several advantages:<ul><li>Sharper Design Choices: Visualizing multiple layouts allows engineers to make more informed decisions about size, performance, and manufacturability.</li><li>Faster Time to Market: Exploring design options upfront can significantly reduce the time needed to finalize a layout, accelerating product development cycles.</li><li>Enhanced Innovation: AI design tools liberate engineers from the constraints of traditional methods, allowing them to explore more creative and innovative solutions.</li></ul><h2>Beyond Exploration: Freedom from Constraints</h2>AI design tools offer engineers a key benefit: freedom from limitations. Traditionally, requesting modifications from a PCB designer can be a time-consuming process. While collaboration is crucial, AI design tools offer additional flexibility.<img src="https://s3-us-west-1.amazonaws.com/wevolver-project-images/froala%2F1724255800655-1724255800655.png" alt="1iuoshi4o8837458sdj" class="fr-fic fr-dii">This newfound freedom empowers engineers to be more creative and efficient in their design process. Take <a href="https://www.quilter.ai/" target="_blank">a company like Quilter</a>, for instance. Their software utilizes reinforcement learning to generate multiple PCB layout options that meet an engineer's specifications. This allows engineers to experiment with different design iterations and explore a wider range of possibilities, ultimately leading to a more innovative final product.Engineers can then focus on their core strengths – innovation and exploration. Quilter's software facilitates a more iterative design flow, enabling engineers to explore various configurations and arrive at an optimal solution faster. As a result of this streamlined workflow, PCB designers get pre-tested configurations from which to build the final design, thereby saving time for both parties.<h2>The Power of Machine Learning</h2>At the core of these AI design tools lies a powerful technology called machine learning (ML). Machine learning algorithms are essential computer programs that can learn from data, and they typically learn in one of two ways: supervised learning from existing layouts, or from reinforcement learning using simulation within a controlled environment.In simulation learning, ML algorithms are trained on massive datasets of past successful PCB designs. These datasets include existing board layouts, design rules, and performance parameters. By analyzing these examples, the AI learns the relationships between component placement, routing strategies, and desired outcomes like signal integrity and thermal management.In reinforcement learning (the kind Quilter uses), the AI acts like a grad student in a virtual PCB design lab. It gets to experiment with countless layouts, but its grade depends on following real-world rules. These rules consider things like physics (think keeping electromagnetic interference under control) and how easy it would be to build the board using standard practices while keeping costs down. By rewarding layouts that excel in these areas and penalizing designs with issues, the AI can explore new possibilities and potentially discover innovative solutions that traditional methods might miss. This process of experimentation and feedback allows the AI to continually improve its design capabilities.Those of you who were raised on 1980s movies may recall the last scene in WarGames where Joshua (certainly an AI) takes over NORAD's systems and begins playing every possible variant of thermonuclear war until he realizes there is no winning move. This kind of repeated simulation and analysis is similar to the kind of reinforcement learning many AI tools use today.<h2>The Future of PCB Design with Faster Manufacturing</h2>The potential of AI-powered design tools is clear. By offering engineers the freedom to explore and experiment, these tools can significantly improve the efficiency and creativity of the PCB design process.<img src="https://s3-us-west-1.amazonaws.com/wevolver-project-images/froala%2F1724255800106-1724255800106.png" alt="2j8y94579yushigd" class="fr-fic fr-dii">Further, AI can act as a bridge between design and manufacturing. By incorporating knowledge of different manufacturers' supported stack-ups and design rule sets, AI tools can propose layouts that are inherently manufacturable with CM services like MacroFab. This eliminates the need for rework later in the process, streamlining the transition from design to fabrication and accelerating the entire journey from idea to finished product.As AI technology matures and machine learning algorithms become even more sophisticated, we can expect more advanced tools that will further streamline the design workflow and empower engineers to bring their ideas to life in a much shorter timeframe.Once you've finalized the perfect design with the help of AI exploration tools, services like MacroFab can handle your PCB assembly with incredible speed and precision. Their online platform allows for instant quotes, real-time DFM checks, and a streamlined ordering process, ensuring a smooth transition from design to high-quality production. The combination of AI design and MacroFab allows your innovative ideas to reach the market faster than ever before.</section><h4>Ready to Get Started?</h4><a href="https://factory.macrofab.com/quick_quote/?utm_source=direct&utm_medium=none" target="_blank">Get an instant quote now</a><section><h5> </h5></section>

AI design tools offer engineers a key benefit: freedom from limitations.

Unleashing Creativity: AI-Powered Exploration in PCB Design

The whitepaper titled "Learning how to learn: Neuromorphic AI inference at the edge," published by BrainChip, explores the transformative potential of neuromorphic computing for artificial intelligence (AI) applications, particularly at the edge.&nbsp;As traditional computing architectures face limitations, this document outlines how neuromorphic silicon can provide a more efficient, intelligent, and sustainable approach to AI inference.<h2 dir="ltr">The Challenges of Conventional Computing</h2>The semiconductor industry is at a crossroads. Traditional computing, especially those based on the Von Neumann architecture, is increasingly struggling to keep up with the demands of modern AI applications.BrainChip’s whitepaper emphasizes that the current AI paradigm is not sustainable for the next generation of intelligent applications. It outlines several critical issues:<ul><li dir="ltr">Von Neumann Bottleneck: This refers to the inefficiency arising from separating the memory and processing units in traditional architectures. This leads to delays in data transfer and processing.</li><li dir="ltr">Moore’s Law and Dennard Scaling:&nbsp;While Moore’s Law has historically predicted the doubling of transistors on a chip every few years, this trend is slowing. Similarly, Dennard Scaling states that as transistors get smaller, the power density remains constant. However, this is no longer true due to limitations like leakage currents and heat dissipation. This limits the performance improvements that can be achieved through conventional means.</li><li dir="ltr">Cloud-Centric Limitations: AI applications increasingly rely on cloud computing, yet they face challenges such as latency, bandwidth constraints, and security vulnerabilities. The growing reliance on cloud data centers for AI training and inference is not sustainable, especially with the rising demand for real-time processing and the need for privacy in data handling.</li></ul><h2 dir="ltr">Neuromorphic Computing: A Paradigm Shift</h2>Neuromorphic computing represents a shift in how we approach AI. Unlike traditional models, which rely heavily on centralized processing and extensive data transfers, neuromorphic systems are designed to mimic the human brain's architecture and processing capabilities.&nbsp; <img src="https://s3-us-west-1.amazonaws.com/wevolver-project-images/froala%2F1724156635047-1724156635047.png" width="624" height="301" class="fr-fic fr-dii">The whitepaper highlights several advantages of this approach:<ul><li dir="ltr">Efficiency: Neuromorphic chips can process information in parallel, reducing latency and power consumption. This is particularly beneficial for edge applications where resources are limited.</li><li dir="ltr">Event-Driven Processing: Neuromorphic systems operate on an event-driven basis. This means they only process data when changes occur rather than continuously. This leads to more efficient use of computational resources and lower energy consumption.</li><li dir="ltr">Scalability: As AI applications evolve, neuromorphic computing offers a scalable solution that can adapt to increasing demands without the same limitations faced by traditional architectures.</li></ul><h2 dir="ltr">The Power of "Learning how to Learn"</h2>One of the most compelling concepts explored in the whitepaper is "learning how to learn." This idea revolves around the ability of AI systems to go beyond static learning models to embrace a more dynamic, continuous learning process. Traditional AI models are typically trained on a fixed dataset and then deployed.&nbsp;However, with time, they don’t adapt to new data unless they are retrained. This process is not only inefficient but also impractical for many real-world applications.<img src="https://s3-us-west-1.amazonaws.com/wevolver-project-images/froala%2F1724156634645-1724156634645.png" width="624" height="264" class="fr-fic fr-dii">Presenting a solution that supports on-device learning, <a href="https://brainchip.com/designing-smarter-and-safer-cars-with-essential-ai-2/" target="_blank">BrainChip’s Akida</a> neuromorphic processor is designed to continue to learn and adapt even after deployment. This capability is crucial for applications like autonomous vehicles, where the environment is constantly changing, and the AI needs to adjust its behavior in real-time.<h2 dir="ltr">Real-World Applications and Impact</h2>The whitepaper emphasizes the growing importance of edge AI, where data is processed closer to its source rather than being sent to the cloud. This shift is driven by the need for real-time insights and reduced latency. Neuromorphic computing is particularly well-suited for edge applications due to its efficiency and ability to handle complex tasks with limited resources.Key applications discussed in the whitepaper include:<ul><li dir="ltr">Autonomous Systems: Neuromorphic chips can enhance the capabilities of autonomous vehicles and drones by enabling real-time decision-making and processing of sensory data.</li></ul><img src="https://s3-us-west-1.amazonaws.com/wevolver-project-images/froala%2F1724156634797-1724156634797.png" width="624" height="471" class="fr-fic fr-dii"><ul><li dir="ltr">Smart Devices: From wearable technology to smart home devices, neuromorphic computing can improve performance and battery life. This will make these devices more user-friendly and efficient.</li><li dir="ltr">Healthcare: In medical applications, neuromorphic systems can facilitate faster diagnosis and monitoring by processing data from various sensors in real time.</li></ul><h2 dir="ltr">Towards a Sustainable Future</h2>One of the most compelling arguments presented in the whitepaper is the potential for neuromorphic computing to contribute to a more sustainable future. By reducing the energy consumption associated with AI processing, these systems can help mitigate the environmental impact of data centers and cloud computing.<img src="https://s3-us-west-1.amazonaws.com/wevolver-project-images/froala%2F1724156635656-1724156635656.png" width="624" height="533" class="fr-fic fr-dii">The whitepaper posits that as the demand for AI continues to grow, transitioning to neuromorphic architectures will be important to maintain performance while minimizing energy use. This aligns with global efforts to reduce carbon footprints and promote sustainable technology practices.<h2 dir="ltr">Final Words&nbsp;</h2>"Learning How to Learn: Neuromorphic AI Inference at the Edge" provides an overview of the limitations of traditional computing models and the transformative potential of neuromorphic architectures. By addressing critical challenges such as latency, energy consumption, and security, neuromorphic computing offers a promising path forward for AI applications, particularly at the edge.The evolution of computing towards more brain-like architectures not only enhances performance but also paves the way for a more intelligent and sustainable technological future.&nbsp;For more insights, read and download the full <a href="https://brainchip.com/wp-content/uploads/2022/08/BrainChip-Learning-how-to-Learn.pdf" target="_blank" data-lf-fd-inspected-kn9eq4roawkarlvp="true">whitepaper here</a>.

Q&A with Peter Van Der Made, BrainChip Founder and Chief Technology Officer

Learning How to Learn: Neuromorphic AI Inference at the Edge

A new cross-disciplinary study by WashU researchers has uncovered an unexpected psychological phenomenon at the intersection of human behavior and artificial intelligence: When told they were training AI to play a bargaining game, participants actively adjusted their own behavior to appear more fair and just, an impulse with potentially important implications for real-world AI developers.“The participants seemed to have a motivation to train AI for fairness, which is encouraging, but other people might have different agendas,” said Lauren Treiman, a PhD student in the Division of Computational &amp; Data Sciences (DCDS) and lead author of the study. “Developers should know that people will intentionally change their behavior when they know it will be used to train AI.”The study, published in PNAS, was supported by a seed grant from the&nbsp;<a href="https://triads.wustl.edu/" target="_blank">Transdisciplinary Institute in Applied Data Sciences (TRIADS)</a>, a signature initiative of the Arts &amp; Sciences Strategic Plan. The co-authors are&nbsp;<a href="https://psych.wustl.edu/people/wouter-kool" target="_blank">Wouter Kool</a>, assistant professor of psychological and brain sciences in Arts &amp; Sciences, and&nbsp;<a href="https://engineering.wustl.edu/faculty/Chien-Ju-Ho.html" target="_blank">Chien-Ju Ho</a>, assistant professor of computer science and engineering in the McKelvey School of Engineering. Kool and Ho are Treiman’s graduate advisors.The study included five experiments, each with roughly 200-300 participants. Subjects were asked to play the “Ultimatum Game,” a challenge that requires them to negotiate small cash payouts (just $1 to $6) with other human players or a computer. In some cases, they were told their decisions would be used to teach an AI bot how to play the game.The players who thought they were training AI were consistently more likely to seek a fair share of the payout, even if such fairness cost them a few bucks. Interestingly, that behavior change persisted even after they were told their decisions were no longer being used to train AI, suggesting the experience of shaping technology had a lasting impact on decision-making. &nbsp;“As cognitive scientists, we’re interested in habit formation,” Kool said. “This is a cool example because the behavior continued even when it was not called for anymore.”Still, the impulse behind the behavior isn’t entirely clear. Researchers didn’t ask about specific motivations and strategies, and Kool explained that participants may not have felt a strong obligation to make AI more ethical. It’s possible, he said, that the experiment simply brought out their natural tendencies to reject offers that seemed unfair. “They may not really be thinking about the future consequences,” he said. “They could just be taking the easy way out.”&nbsp;“The study underscores the important human element in the training of AI,” said Ho, a computer scientist who studies the relationships between human behaviors and machine learning algorithms. “A lot of AI training is based on human decisions,” he said. “If human biases during AI training aren’t taken into account, the resulting AI will also be biased. In the last few years, we’ve seen a lot of issues arising from this sort of mismatch between AI training and deployment.”Some facial recognition software, for example, is less accurate at identifying people of color, Ho said. “That’s partly because the data used to train AI is biased and unrepresentative,” he said.Treiman is now conducting follow-up experiments to get a better sense of the motivations and strategies of people training AI. “It’s very important to consider the psychological aspects of computer science,” she said.&nbsp;<hr>Treiman LS, Ho CJ, Kool W. The consequences of AI training on human decision-making. Proceedings of the National Academy of Sciences (PNAS) Aug. 6, 2024.DOI:&nbsp;<a href="https://doi.org/10.1073/pnas.2408731121" target="_blank">https://doi.org/10.1073/pnas.2408731121</a>&nbsp;

WashU researchers have found humans adjust their own behavior to be more fair when training AI.

McKelvey’s Chien-Ju Ho working with Art & Sciences’ Wouter Kool, DCDS PhD student Lauren Treiman to understand how human behavior changes in training AI

Humans change their own behavior when training AI

This article was discussed in our <a href="https://www.wevolver.com/profile/the.next.byte" rel="noopener noreferrer" target="_blank">Next Byte podcast</a>.&nbsp;<iframe height="200px" width="100%" frameborder="no" scrolling="no" seamless="" src="https://player.simplecast.com/1b5ec4d2-94f9-4c45-8203-e2ce4f6cd565?dark=false" class="fr-draggable" data-lf-form-tracking-inspected-kn9eq4roawkarlvp="true" data-lf-yt-playback-inspected-kn9eq4roawkarlvp="true" data-lf-vimeo-playback-inspected-kn9eq4roawkarlvp="true"></iframe> The full article will continue below.<hr>Hurricane season is here, with Hurricane Beryl already leaving a trail of damage after hitting Texas’ shorelines. The National Oceanic and Atmospheric Administration (NOAA) is predicting a record number of named storms in the Atlantic this year. As these events continue to increase, it’s imperative to find solutions to help manage the inevitable hazards.Dr. Ali Mostafavi, associate professor in the Zachry Department of Civil and Environmental Engineering at Texas A&amp;M University, and his research team have been utilizing artificial intelligence (AI) to address disaster management through each phase of a hazard event. They want to bring cutting-edge AI to the forefront of community resilience and disaster management efforts.AI has become a very popular evolving technology, especially in research. When given large amounts of data, images or text, it can identify and analyze key features much quicker than a whole team of people.<h2>Mitigation and Preparedness</h2>A crucial step of the mitigation phase of disaster management is assessing risk to communities. Hurricanes bring significant flooding risks, but current flood risk assessments can be time-intensive and costly.Mostafavi’s team developed a machine-learning model called Flood Genome that determines flood risk in different communities based on factors like hydrological, topographic and built environment features. Just like genetic features can indicate health risks for people, Flood Genome analyzes different features of communities to assign flood risk levels.“We used historical flood damages from the National Flood Insurance Program to test this model to determine how well it can predict high-risk areas, and the model performed beautifully,” Mostafavi said. “It can inform flood mitigation and community resilience efforts by understanding the flood risk level of different areas.”Another AI tool developed by the Mostafavi lab, called Elev-vision, determines the lowest floor elevation of structures using Google Street View images.For a city like Houston, with hundreds of thousands of properties, surveying each property with traditional methods is not feasible. This new tool significantly cuts labor and expenses and provides updated vital information on flood risk to properties.<img src="https://images.wevolver.com/eyJidWNrZXQiOiJ3ZXZvbHZlci1wcm9qZWN0LWltYWdlcyIsImtleSI6ImZyb2FsYS8xNzI0MDQ5NjEyMzM0LU5ld3MtQ1ZFTi1JbnRlcmlvckFJLTE2SnVseTIwMjQuanBnIiwiZWRpdHMiOnsicmVzaXplIjp7IndpZHRoIjo5NTAsImZpdCI6ImNvdmVyIn19fQ==" style="width: 100%px;" class="fr-fic fr-dib">AI benefits for all four phases of disaster management: mitigation, preparedness, response and recovery. | Image: <cite id="isPasted">Courtesy of Ali Mostafavi.</cite><h2 id="isPasted">Response and Recovery</h2>Response efforts would ideally start before an event develops, but the fast-paced nature of a hurricane makes evacuations and supply distribution difficult to manage. The research team has developed solutions to this problem as well.“We leverage location-based data and graph analytic methods to determine the preparedness and evacuation patterns of communities before a hurricane or storm arrives and provide that in near-real-time to decision-makers to understand what areas have or have not evacuated,” Mostafavi said.He added that not only can this new technology tell which areas have evacuated, but it also monitors which routes and destinations are being used the most. Getting accurate and timely evacuation information to decision-makers would greatly improve public safety before and during disasters.This tool can also determine a community’s preparedness before a storm arrives. By analyzing patterns of human mobility and visitations to grocery stores, it can gauge how prepared a community is to weather the storm.Quick response efforts can also impact recovery after events.Rapid damage assessments in the wake of disasters are vital to making more resilient communities. This can lead to faster local and federal aid. The team’s AI research tackles this issue by using satellite images and computer vision techniques to classify properties based on the extent of damage.Many of the models and tools Mostafavi’s team developed were tested during or after previous events like hurricanes in Florida and Louisiana, and all performed very well.“AI technologies can improve resilience capabilities of communities at different stages,” Mostafavi said. “We see AI as the next defense barrier against disasters because it improves risk mapping, situational awareness and impact assessment.”There may not be a long wait for these improvements since the team has already begun a start-up company and developed prototypes to commercialize these technologies for public use.Mostafavi and his team have been working extensively on AI disaster management for many years, resulting in numerous publications and innovations.

Texas A&M researchers are using AI to improve the management of disasters like hurricanes before, during and after the events.

AI For All Phases of Disaster Management

“Nanomaterials” is a broad term used to describe chemical substances or materials in which a single unit is sized between 1 and 100 nanometers (a nanometer is a billionth of a meter). They include exotic materials such as carbon nanotubes, silver nanoparticles (used as antimicrobials), nanoporous materials, and many types of catalysts used for efficiently driving chemical reactions.Nanomaterials are currently used in a wide range of fields, from medicine to electronics, which means that the ability to determine their exact chemical composition is essential. Nonetheless, this proves challenging, because traditional methods for analyzing nanomaterials tend to be susceptible to low signal-to-noise ratios.For example, one extensively used method is energy-dispersive X-ray spectroscopy (EDX), combined with scanning transmission electron microscopy. This technique provides detailed maps of where different elements are located within a sample, but it often produces noisy data, especially on such small objects, and mixed signals when different materials overlap, making precise chemical analysis difficult.The noisy data are usually “cleaned up” with various techniques, from simple spatial filtering to more sophisticated machine learning approaches like principal component analysis, that separate the signals from the noise, but they too have their drawbacks. For example, they can introduce errors, or struggle to distinguish between chemical signals when they are very similar.Now, three scientists at EPFL, Hui Chen, Duncan Alexander, and Cécile Hébert have developed a machine learning-based method called PSNMF (“non-negative matrix factorization-based pan-sharpening”) that enhances the clarity and accuracy of EDX data, making it easier to identify and quantify different chemical elements in nanomaterials.The team started by leveraging a special characteristic of their data called “Poisson noise”. This type of noise occurs because the detection of X-ray photons is random. When the electron beam hits the sample, it produces X-ray photons, but the number detected varies randomly each time, creating a noisy, grainy pattern known as Poisson noise.To improve the clarity of their data, the researchers combined data from nearby pixels, enhancing the signal-to-noise ratio in the spectrum at the cost of the spatial resolution.They then applied a machine learning method called "non-negative matrix factorization" (NMF) to this clearer dataset. NMF is a mathematical technique that breaks down a large dataset into simpler, smaller parts, ensuring all parts are non-negative, which helps identify patterns in the data. This approach gave them good spectral data at the cost of having blurry images with large pixels.Next, they repeated the NMF process on the original high-resolution dataset to preserve detailed spatial information, but initializing the factorization with the previously identified spectral components. Finally, they combined the results from both steps to produce a high-quality dataset, that has both high spectral fidelity and high spatial resolution.The researchers validated PSNMF using synthetic data, computed thanks to a modelling algorithm developed in the lab. Those data mimicked real-world challenges, such as analyzing mineral samples formed under extreme conditions. The method proved highly effective, accurately identifying and separating different materials, even those in tiny amounts.When applied to actual samples, including a nanomineral and a nanocatalyst, PSNMF successfully separated and quantified overlapping materials. This precise analysis is crucial for understanding and developing new technologies that rely on these complex nanostructures.PSNMF is a significant improvement in nanoscale chemical analysis. By providing accurate results despite noisy data and overlapping signals, this method enhances our ability to study and utilize nanomaterials in various fields, from advanced electronics to medical devices.<hr>ReferencesHui Chen, Duncan T.L. Alexander, Cécile Hébert. Leveraging machine learning for advanced nanoscale X-ray analysis: Unmixing multicomponent signals and enhancing chemical quantification. NanoLetters 06 August 2024. DOI:&nbsp;<a href="https://doi.org/10.1021/acs.nanolett.4c02446" title="DOI URL" target="_blank" rel="noopener">10.1021/acs.nanolett.4c02446</a>

EPFL scientists have developed an AI-based technique to improve chemical analysis of nanomaterials, overcoming challenges of noisy data and mixed signals.

AI enhances chemical analysis at the nanoscale

Learn about some of the trends driving the use of technology in agriculture. 

Powering Precision Agriculture: Five Ways PCBAs Drive Ag Tech

<section id="isPasted">Plastic pollution poses a significant threat to our waterways and oceans.&nbsp;<a href="http://engineering.columbia.edu/" rel="nofollow" target="_blank">Columbia Engineering</a> and&nbsp;<a href="https://www.climate.columbia.edu/" rel="nofollow" target="_blank">Columbia Climate School</a> are taking decisive action to address this problem.&nbsp;Collaborators&nbsp;<a href="https://lamont.columbia.edu/directory/beizhan-yan" rel="nofollow" target="_blank">Beizhan Yan</a>, Lamont Research Professor at Lamont-Doherty Earth Observatory (LDEO) at Columbia Climate School, and&nbsp;<a href="https://www.ee.columbia.edu/zoran-kostic" rel="nofollow" target="_blank">Zoran Kostic</a>, professor of professional practice in Columbia Engineering’s&nbsp;<a href="https://www.ee.columbia.edu/" rel="nofollow" target="_blank">Department of Electrical Engineering</a>, have been awarded a $2.7M grant from the&nbsp;<a href="https://seagrant.noaa.gov/" rel="nofollow" target="_blank">National Oceanic and Atmospheric Administration's (NOAA) Sea Grant program</a> for their latest project, "Precision Plastic Waste Cleanup and Monitoring: AI-Enhanced Solution for Sustainable Waterways and Ocean Health."&nbsp;The award underscores the federal commitment to addressing the marine debris crisis. As part of NOAA’s efforts to protect and preserve America's natural resources, this investment supports innovative projects that promise sustainable, long-term solutions to the pervasive problem of plastic pollution. The research was selected for its potential to make a significant impact on both local and global scales.Their project combines expertise in the domains of environmental science, applications of artificial intelligence (AI), mechanical design, low-power signal processing and communications, image and video processing, data analysis, and internet of things (IoT).&nbsp;“The project exemplifies the power of multidisciplinary collaboration in addressing plastic pollution,” says Yan. “The current plastic collection approaches have encountered challenges in efficiency, leading to unsustainability. The urgent need for collaboration is also the exact reason why Prof.&nbsp;<a href="https://www.eee.columbia.edu/faculty/athanasios-bourtsalas" rel="nofollow" target="_blank">Thanos Bourtsalas</a> and I formed the&nbsp;<a href="https://people.climate.columbia.edu/networks/profile/plastic-pollution" rel="nofollow" target="_blank">Plastic Pollution Analysis and Suitability Solution network</a> about three years ago, where we have gathered expertise from engineering, science, policy, environmental health, and more to tackle this global issue.”Kostic adds, “It is immensely satisfying to be able to collaborate with experts in environmental sciences and use my scientific and engineering experience in AI and signal processing to benefit the cause of engineering for a healthy humanity.”</section><section><h2>Plastic waste collection with AI-supported automation</h2>The project addresses a critical environmental challenge: the vast amounts of plastic waste entering our oceans from rivers and coastlines. Traditional waste collectors often struggle to differentiate between plastic and natural debris, leading to inefficiencies and increased maintenance costs.&nbsp;Led by Kostic, who also is a member of Columbia’s&nbsp;<a href="https://datascience.columbia.edu/" rel="nofollow" target="_blank">Data Science Institute</a>, the researchers bring a wealth of expertise in deep learning and AI applications to this work. The Kostic lab,&nbsp;<a href="https://www.aidl.ee.columbia.edu/research-opportunities" rel="nofollow" target="_blank">Applications of AI and Deep Learning Lab</a> (AIDL), known for exploring AI-driven solutions in diverse fields, now leverages these advanced technologies to address the pressing issue of plastic pollution. Kostic is committed to integrating cutting-edge AI research with real-world applications. He is involved in the&nbsp;<a href="https://cait.engineering.columbia.edu/" rel="nofollow" target="_blank">Columbia Center of Artificial Intelligence Technology</a> and his recent research, presented at the&nbsp;<a href="https://cvpr.thecvf.com/" rel="nofollow" target="_blank">CVPR 2024 conference</a>, focuses on a data-driven<a href="https://www.ee.columbia.edu/news/ee-research-team-showcases-ai-based-urban-traffic-applications-cvpr-2024" rel="nofollow" target="_blank">&nbsp;simulation environment</a>s for modeling traffic in metropolitan street intersections using real-world tracking data, designing low-latency high-bandwidth deep-learning systems for&nbsp;<a href="https://www.aidl.ee.columbia.edu/" rel="nofollow" target="_blank">applications</a> in smart cities, and applications of AI in surgery and speech analysis.</section><section><h2>An advanced monitoring and collection system</h2>The novel device will leverage advanced spectral and consumer color camera imaging to accurately distinguish plastics from natural debris such as leaves and branches. This selective approach, say the researchers, enhances the efficiency of waste collection and also minimizes ecological disruption by preserving natural debris in its ecosystem.The prototypes, once developed, will undergo rigorous testing in the Hudson and Bronx Rivers. This ensures the system's adaptability to various hydrological conditions and its effectiveness in real-world scenarios. The researchers plan to create a cost-effective version for deployment in developing countries.</section><section><h2>Sustainable outcomes and global impact</h2>The system's on-board and in-situ processing capabilities will significantly reduce sorting and transport costs by accurately identifying and collecting plastic waste. The technology's adaptability promises broad applications, say the researchers, in commercial waste collectors, and could lead to widespread adoption and substantial cost savings.Environmental and economic analyses will underscore the benefits of this innovative solution. By preventing plastic waste from entering the oceans, the project will mitigate the generation of microplastics and nanoplastics, improving waterway health and protecting aquatic life. Additionally, recycling efforts will be enhanced through the depolymerization of collected plastics into reusable monomers using engineered enzymes</section><section><h2>A focus on community engagement and education</h2>A crucial aspect of this initiative is community involvement. The New York Sea Grant will develop educational materials and hands-on activities to raise awareness and foster engagement, particularly in underserved communities.&nbsp;<a href="https://cleanoceanaction.org/about-coa" rel="nofollow" target="_blank">Clean Ocean Action</a> will extend these efforts in New Jersey and Pennsylvania, further amplifying the project's reach and impact.Other investigators on this project include Thanos Bourtsalas (Columbia Engineering),&nbsp;<a href="https://lamont.columbia.edu/directory/nicholas-p-frearson" rel="nofollow" target="_blank">Nick Frearson</a> and&nbsp;<a href="https://lamont.columbia.edu/directory/joaquim-i-goes" rel="nofollow" target="_blank">Joaquim Goes</a> (LDEO), and Katherine Bunting-Howarth and Catherine Prunella (New York Sea Grant).</section>

Columbia Engineering wins a new grant from the National Oceanic and Atmospheric Administration to develop AI-system for sustainable waterways and ocean health

Combatting Microplastics with AI Real-Time Monitoring

Research Facility 2.0 (RF 2.0) is an EU-funded research project aimed at making the operation of particle accelerators and other large-scale facilities more resource-efficient. Since the beginning of the year, ten European project partners coordinated by Karlsruhe Institute of Technology (KIT), have been working on optimized components and digital solutions to reduce the energy consumption of accelerator systems. The European Commission (Horizon Europe program), and the Swiss State Secretariat for Education, Research and Innovation are funding the project with a total of EUR 5.6 million over three years. The KITTEN test center at KIT plays a central role in the RF 2.0 project: It hosts researchers that are working on the analysis of important parameters and the development of realistic demonstrators.&nbsp;Particle accelerators not only play an important role in numerous fields of research. Thousands of such accelerators are in use worldwide for industrial and medical applications, for example in material testing or radiotherapy. These high-performance facilities are real energy guzzlers: The annual energy consumption of large plants often amounts to several hundred gigawatt hours (GWh). This compares to the energy consumption of small- or medium-sized European cities. Previous approaches targeted reducing energy consumption and the carbon footprint of particle accelerators or using more sustainable materials, but were limited to specific technical issues. The RF 2.0 project, in contrast, uses a comprehensive approach to making particle accelerators more energy-efficient.&nbsp;<h2>The Objective: Operating Accelerators with Renewable Energies</h2>“We bring expertise in physics and energy technology together. It is our vision to operate particle accelerators safely, stably, and with nothing but renewable energy, to make them as independent as possible from the public power grid and create less impact on the environment,” says Professor Giovanni De Carne from the Institute for Technical Physics at KIT. De Carne is the RF 2.0 project coordinator and initiated the project together with Professor Anke-Susanne Müller, Head of the Institute for Beam Physics and Technology (IBPT) at KIT. A total of ten partners are working together on RF 2.0, including five of the largest particle accelerator facilities in Europe – ALBA Synchrotron (Spain), CERN, Deutsches Elektronen-Synchrotron DESY in Hamburg, Helmholtz-Zentrum Berlin, and MAX IV Laboratory (Sweden) – supported by specialized technology companies.&nbsp;<h2>Realistic Tests with a View to Energy Efficiency and Grid Stability</h2>A key element of the project is the KITTEN test center. The acronym stands for “KIT Testfeld für Energieeffizienz und Netzstabilität in großen Forschungsinfrastrukturen” (KIT test field for energy efficiency and network stability in large-scale research infrastructures). KITTEN integrates the KARA research accelerator and the Energy Lab at KIT. “This infrastructure enables us to create a real-life environment for investigating and testing energy efficiency and network stability in large-scale research infrastructures,” says De Carne. The analysis phase is currently underway. “Particle accelerators always require a very high, constant power supply that must be free from interference. This challenge could not be met in terms of voltage qualities so far due to a lack of flexibility. We use our tests to get a deeper understanding of correlations, create mitigation strategies (i.e. approaches to reduce negative effects and lower the projected costs), analyze the CO2 footprint of the equipment, and create a sustainability matrix.”&nbsp;Building on the results, the elaboration of solutions for more energy-saving technologies with better efficiency is on the agenda. The focus is on four areas: new, highly efficient components; implementation of digital solutions for optimized operating periods, faster start-up and shutdown processes for improved design and operation; integration of low-carbon technologies by using renewable energies and energy storage systems as well as higher energy consumption flexibility through dynamic monitoring and control systems. &nbsp;<h2>More Efficiency Thanks to Artificial Intelligence (AI) and Digital Twins </h2>Demonstrator projects conducted in cooperation with the project partners help the research team to test and validate specific approaches under realistic conditions. “For example, we are working on AI-based solutions for more efficient and flexible operation of accelerators. We are further developing a digital twin that will enable us to validate new energy technologies faster in the future,” De Carne explains. One of the projects will tackle the flexibilization of data centers to better accommodate the fluctuation in the yields from renewable energies. Another project is aimed at testing measurement systems that detect disruptions in the grid and enable faster reactions to such fluctuations.“All of these research activities are clearly transfer-oriented,” says De Carne. “The involved technology companies seek to develop market-ready products from the solutions we are working on together.” The range of possible applications is not limited to particle accelerators for research purposes. “Other areas that consume much energy will also benefit from our concepts – be it MRT equipment in hospitals, heavy industry plants for steel and cement production, or data centers that can be operated with more energy efficiency.”<h2>The Next Milestone: KARA Dashboard Makes Results Freely Accessible &nbsp;</h2>Currently, De Carne and his team are developing an open dashboard for the Karlsruhe research accelerator. Its goal is to make the data and results obtained accessible to external researchers and developers in the best possible way. They will not only see the energy consumption of the accelerator and the CO2 footprint in real-time, but also the effects of the alternative solutions tested.&nbsp;

One example of the KIT research on resource-effective technologies for future accelerators is the linear accelerator FLUTE. (Photo: Markus Breig, KIT)

Research Facility 2.0 (RF 2.0) is an EU-funded research project aimed at making the operation of particle accelerators and other large-scale facilities more resource-efficient. 

Project RF 2.0: Research for More Sustainability in Particle Accelerators

Accenture is at the forefront of technological innovation with a focus on developing next-generation computing technologies. In a recent podcast, Dr. Eric Gallo, a senior principal at Accenture Labs, shared insights into the promising field of neuromorphic computing. This technology mimics how the human brain processes information and offers significant advantages in power efficiency and real-time data processing.&nbsp;As industries increasingly rely on edge computing, neuromorphic systems present a unique solution to the challenges of energy consumption and integration in smart devices. This episode of the BrainChip podcast explores Accenture's initiatives in neuromorphic computing. They also explore its applications in various sectors and the potential it holds for the future of edge intelligence.<h2 dir="ltr">The Rise of Edge Computing and the Need for Heterogeneous Computing</h2>Edge computing is rapidly gaining traction as industries increasingly rely on real-time data processing and intelligence at the edge of the network. As the industry matures, there is a growing understanding that a heterogeneous set of computing devices is necessary to achieve optimal results.Traditional computing architectures will continue to play a role, but they will make way for other specialized architectures that excel in different situations and locations. This will create a continuum where edge architectures, cloud architectures, and other specialized architectures work together to provide the right amount of compute power where it's needed most.Companies with edge devices are actively examining their AI strategies to avoid being left behind. While computing power is readily available, the ability to perform inference very close to the edge is a key focus for many organizations. This is where neuromorphic computing shines, offering a low-power solution for real-time data processing at the edge. This energy efficiency is not just a minor improvement; it’s a game-changer.&nbsp;Dr. Gallo highlighted how neuromorphic technology could potentially achieve power savings of up to 100,000 times compared to current methods. This opens up new possibilities for developing intelligent devices that can operate in environments where power is limited, such as in remote locations, wearable devices, or even deep space missions. Neuromorphic computing is still in its early stages, but its potential to transform industries is immense.<h2 dir="ltr">The Power of Neuromorphic Technology</h2>One of the most significant advantages of neuromorphic technology is its remarkable energy efficiency. Traditional computing systems often require a large amount of power to perform complex tasks. However, neuromorphic systems can achieve the same results while using only a fraction of that energy. This makes them ideal for applications where power is a critical concern.Dr. Eric Gallo explained that this technology could be a game-changer in various fields. For example, in defense, neuromorphic chips could be used to create advanced situational awareness systems for soldiers. These systems could process vast amounts of data in real time, helping soldiers make better decisions in the field without the need for bulky, power-hungry equipment.Neuromorphic technology could also enhance the intelligence of factory equipment in industrial settings. Machines equipped with neuromorphic chips could adapt to changing conditions on the fly, improving efficiency and reducing downtime. These chips can also be powered by small batteries or energy harvesters, making them suitable for environments where access to power is limited.As this technology continues to develop, its impact on different sectors will also grow. This will lead to more efficient and intelligent solutions.<h2 dir="ltr">Accenture's Neuromorphic Computing Initiatives in Space</h2>Eric Gallo believes that neuromorphic architectures can enable smart satellites and other space devices without the significant power and thermal constraints of traditional computing systems.Accenture is working towards demonstrating real-time neuromorphic computing in space. By leveraging the low-power capabilities of neuromorphic chips, Accenture aims to make space devices more intelligent and responsive.The space industry has traditionally relied on less advanced computing technologies due to the challenges of power and heat dissipation. However, with the emergence of neuromorphic computing, there is a sudden realization that space systems can be made much smarter without the usual constraints.<h2 dir="ltr">Accenture's Partnership with BrainChip</h2>Accenture has formed a strong partnership with BrainChip, a leader in neuromorphic computing technology. This collaboration uses the neuromorphic chip to explore practical applications in various industries. Dr. Eric Gallo, who leads Accenture's neuromorphic initiatives, has shared valuable insights from their work with this advanced technology.The chip stands out for its ability to save power compared to traditional computing systems. In practical tests, Accenture observed that systems using the chip consumed only a fraction of the power—about one-fifth—compared to conventional setups. This efficiency is important for applications where power resources are limited, such as edge devices and satellites.Working with BrainChip has allowed Accenture to access cutting-edge neuromorphic technology and gain practical experience in real-world environments. The partnership has been characterized by strong support and collaboration that enables both teams to tackle challenges and continuously improve their systems.<h2 dir="ltr">The Future of Neuromorphic Computing: Spanning Material Spaces and Scales</h2>One of the most exciting aspects of neuromorphic computing is its potential to span a wide range of material spaces and scales. Dr. Gallo envisions the possibility of creating tiny, biodegradable neuromorphic sensors that can be used in applications like water quality monitoring. These sensors could transmit data to small neural networks, which could then determine if there is a need for concern.At the other end of the spectrum, neuromorphic architectures are enabling large-scale neural networks like Spike GPT. These advanced systems demonstrate the versatility of neuromorphic computing, which can be applied from the smallest sensors to the most powerful artificial intelligence systems.Dr. Gallo emphasizes that anyone, even those without expertise in computing architectures, can get involved and contribute to the advancement of neuromorphic computing. The field is open to new ideas and innovations, and there are many opportunities for individuals to make meaningful contributions.<h2 dir="ltr">Final Words</h2>Looking ahead, the future of neuromorphic computing appears bright. Its ability to span a wide range of materials and scales opens doors for anyone interested in contributing to this exciting field. As organizations like Accenture lead the charge, we can expect to see more practical implementations that harness the power of neuromorphic technology, ultimately making our world smarter and more efficient. To learn more about the future of neuromorphic computing watch the full podcast above!

In this episode of the “This is Our Mission” podcast, Sean Hehir interviews Dr. Eric Gallo, a Senior Principal at Accenture Labs. They discuss the advantages of neuromorphic technology and its impact on edge computing, as well as advancements in SpaceTech. 

BrainChip Podcast Epi 34: The State of Neuromorphic Computing

With the hype around machine learning (ML) and the rush to transform businesses with it, unsurprisingly, not all ML projects have been a success. Often, a “solution before the problem” mentality leads to poorly defined requirements and goals for the use of ML. Failing to understand why ML should be used, as well as how business metrics will be impacted, may lead to proof of concept (POC) work that takes up valuable time without delivering results. To avoid that, companies seeking to introduce ML into their processes or products should aim to understand their overall goal in doing so and then link that goal to relevant business metrics used to measure a current baseline, setting the stage for assessing ML performance as the POC work progresses. Finally, connecting those indicators to appropriate ML metrics based on the task or use case for the POC and developing a short roadmap for the research and development direction will increase the likelihood of success.<h2>Distilling Goals into Business Metrics</h2>Businesses usually decide to introduce ML into their processes for one of several overarching reasons. Typically, these reasons include improving revenue by increasing the productivity of the team, elevating the success rate of some particular aspect of the business, improving customer outcomes by reducing the time to respond to incoming information, or reducing costs associated with errors or other waste. These reasons tie into more fine-grained business metrics that should be identified when starting an ML project. For example, if the goal is to increase revenue by enhancing the team or company’s capabilities, then relevant metrics for measuring the current state of productivity might relate to sales and marketing metrics such as quota attainment rate, cost per lead, or net sales revenue, among others. For improving customer success, relevant metrics may include churn, customer satisfaction scores, or rate of lost deals. Most importantly, identifying these business needs via correct metrics is crucial when developing projects rather than identifying solutions without understanding the concrete “why” behind their necessity. Deciding to implement an ML process for triaging support tickets or summarizing long documents won’t make sense unless the impact of doing so is effectively measured as a baseline.<h2>Matching Business Metrics to ML Equivalents</h2>Once appropriate business metrics have been chosen for the project based on the overall goals, they should be matched to one or more ML metrics based on the tasks identified for ML to tackle. Typically, a cluster of metrics for the ML side will be needed to capture an improvement on the business side. For example, suppose the goal is to increase productivity by accelerating a team working on a particular process, perhaps quality assurance of software pre-deployment. In that case, the business metrics might be the time from start to end of testing, the number of elements tested, and the number of tests run per session. However, adjacent and equally important metrics include the number of bugs caught pre-deployment and the number of previously passing tests that now fail. When establishing an ML process to handle some of the quality assurance team’s work, the metrics are not just latency or how fast the model can assess expected versus actual output for discrepancies but also model accuracy—specifically, the number of false positives and negatives. Because mistakes that the model makes may need to be assessed by a human, too many flagged issues will add to the team’s workload, and too many missed issues will result in work upstream for the development team. Alongside having a benchmark of the team’s current performance, based on the business metrics discussed, the baseline accuracy is also useful for assessing whether ML performs at least as well to be considered valid.Sometimes, depending on the use case or task and the type of output required, the ML metrics may not be as straightforward as accuracy or error rate. However, some way of measuring how well an ML process is performing usually exists, as long as one can measure how well things work without it. Natural language processing (NLP) is an example of one domain with less direct metrics. For instance, tasks like summarizing text or generating content appear to be difficult to assess on the surface. Still, if some time is spent to build a dataset of input text and examples of the desired output, then metrics such as Recall-Oriented Understudy for Gisting Evaluation (ROUGE) can be used. This and other metrics measure things like the overlap of words between desired and actual output to account for the fact that no single “correct” answer exists; rather, answers vary according to levels of correctness.<h2>Building the ML POC Roadmap</h2>Finally, with the ML metrics defined, the process of building a brief but detailed roadmap of the POC implementation and experimentation approaches can start. This should involve dividing the whole task into smaller, simpler pieces that are quick to validate in terms of success with an ML model, which could include triaging urgent versus non-urgent support tickets instead of a more complicated set of classifications. Additionally, a preliminary literature review or prior art search should be conducted to identify a series of approaches with increasing complexity in case the simpler options fail at the task. Implementing this strategy starts with off-the-shelf models or those available through a third-party application programming interface (if allowable in the business context), then moves to architectures that can be adjusted by fine-tuning or retraining for the particular task, and finishes with those that need to be implemented from scratch with perhaps more complicated training regimes. The costs for each approach can be approximated to provide a fair assessment of the return on investment (ROI) for the ML project based on the expected value. Doing so highlights whether some POC directions will be prohibitively expensive to try and helps set the cutoff point for abandoning the project if the best performance achieved is insufficient.<h2>Conclusion</h2>Introducing ML into a company’s processes or products helps achieve overarching business goals like improving productivity or reducing costs and errors. However, to ensure success, these ML projects need to start on the right foot. Measuring impact via the appropriate business metrics is key, followed by identifying the right ML metrics to show that developed models will provide the expected value because they work as well as required. Once these details are established, building a roadmap to capture the direction of POC implementation will keep the project on track. This roadmap will also ensure that ROI versus effort is tracked so that a sufficient point of abandonment can be identified that balances the two. These best practices are the first of several required to transition from business goals to ML POC to production.Follow this AI blog series to learn the other important stages involved in developing an ML POC in an agile but robust fashion and how to put the resulting outputs into production. Future blogs will cover identifying and establishing a dataset for the project, setting up experimentation tooling, developing resources and approaches for POC-building, including open-source models, creating guidelines and focus points when extending to a production-ready version, as well as considering what to anticipate and monitor post-deployment.

Moving Your ML Proof of Concept to Production Part 1: From Business Goals to ML Metrics

Machine Learning Proof of Concept to Production: Part One

This article was discussed in our <a href="https://www.wevolver.com/profile/the.next.byte" rel="noopener noreferrer" target="_blank">Next Byte podcast</a>.&nbsp;<iframe height="200px" width="100%" frameborder="no" scrolling="no" seamless="" src="https://player.simplecast.com/270862db-fef1-42ce-8710-bb98f6e896b3?dark=false" class="fr-draggable" data-lf-form-tracking-inspected-kn9eq4roawkarlvp="true" data-lf-yt-playback-inspected-kn9eq4roawkarlvp="true" data-lf-vimeo-playback-inspected-kn9eq4roawkarlvp="true"></iframe> The full article will continue below.<hr><h2>Tl;DR</h2>Fraunhofer Institute's Robuscope provides a comprehensive framework for evaluating the robustness and safety of AI models, ensuring dependable AI applications.<h2>Overview</h2>The integration of AI models into critical applications—from autonomous driving to medical diagnostics—necessitates thorough validation to ensure their robustness and safety.&nbsp;Traditional testing methods are often inadequate to capture the full range of real-world conditions these models will face. The Fraunhofer Institute has introduced Robuscope, a tool designed to rigorously evaluate AI models. This development is crucial for enhancing the reliability of AI systems, which in turn can prevent failures and improve safety in various high-stakes environments.<h2>Fraunhofer Institute</h2>The Fraunhofer Institute for Cognitive Systems IKS is a leading research organization dedicated to advancing AI and cognitive systems.&nbsp;Robuscope is led by key figures including Karsten Roscher, the Department Head of Perception and Imaging, and Anna Guderitz, the Head of Business Development. Their leadership and expertise underscore the institute's dedication to advancing AI safety and reliability. <iframe src="https://c8056756.sibforms.com/serve/MUIFALkfH6sHm0PhgjOdf-A3a5im2wlA4nqaKD9YAAUIzVeXNxunIDvjpxguE0chxmsTCpStIc50Yt6yNbfyjS2pH4r0runZJkLimSEF68pQl2k36X5NK7ob4s89UaGdy192tfsSLKChucCwhS8KSN9_LCHx4aLzkteU_nutPS9IB1r1-o5COxrGtS9fAA4lNCwJPIsbUCE47fCT" frameborder="0" scrolling="no" allowfullscreen="" style="display: block; margin-left: auto; margin-right: auto; max-width: 100%; width: 830px; height: 402px;" class="fr-draggable" data-lf-form-tracking-inspected-kn9eq4roawkarlvp="true" data-lf-yt-playback-inspected-kn9eq4roawkarlvp="true" data-lf-vimeo-playback-inspected-kn9eq4roawkarlvp="true">&nbsp;</iframe><h2>Meet Robuscope</h2>Robuscope is an innovative tool that focuses on the rigorous evaluation of AI models to gauge their robustness. The tool encompasses a comprehensive testing framework that allows users to input test data - with as little as 30 data points - for testing their model without requiring sensitive data. By employing these methods, Robuscope identifies potential weaknesses and failure points that may not be evident in standard testing procedures while offloading the tedious task of manual testing. The tool also offers an online interface for users to submit their AI models and receive detailed reports on their robustness. This allows developers to gain insights into how their models perform under different scenarios and make necessary adjustments to enhance their reliability. The Fraunhofer Institute's approach with Robuscope ensures that AI models are thoroughly vetted before deployment in critical applications like analyzing CAT/MRI scans or autonomous driving.<h2>Impact</h2>The introduction of Robuscope has significant implications for the development and deployment of AI technologies.Positive Impact:<ul><li>Enhanced Safety: Rigorous testing ensures that AI models used in critical applications, such as autonomous vehicles and medical devices, are reliable and less prone to failures, thereby improving safety.</li><li>Increased Trust: Providing detailed evaluations of AI model performance builds trust in AI systems, encouraging broader adoption across various industries.</li><li>Proactive Improvement: Developers can identify and address potential issues early, leading to continuous improvements in AI model robustness.</li></ul>Negative Impact:<ul><li>Online: Users do not have to input sensitive data for testing their models but certain models might require sensitive data for thorough testing and the online nature of this tool would likely be a limiting factor for those use cases.</li></ul>Overall, Robuscope represents a significant step forward in AI testing, offering a robust solution to ensure the reliability and safety of AI applications. This innovation not only improves the performance of AI systems but also builds confidence in their use, facilitating their integration into critical sectors.

Researchers at Fraunhofer Institute have developed a free, online tool for testing the robustness of AI models

AI Testing Reimagined with Robuscope

Picture a robot arriving in a warehouse. It comes packaged with the skills it was trained on, like placing an object, and now it needs to pick items from a shelf it’s not familiar with. At first, the machine struggles with this, since it needs to get acquainted with its new surroundings. To improve, the robot will need to understand which skills within an overall task it needs improvement on, then specialize (or parameterize) that action.A human onsite could program the robot to optimize its performance, but researchers from MIT’s Computer Science and Artificial Intelligence Laboratory (CSAIL) and The AI Institute have developed a more effective alternative. Presented at the Robotics: Science and Systems Conference last month, their “Estimate, Extrapolate, and Situate” (EES) algorithm enables these machines to practice on their own, potentially helping them improve at useful tasks in factories, households, and hospitals.&nbsp;<h2>Sizing up the situation</h2>To help robots get better at activities like sweeping floors, EES works with a vision system that locates and tracks the machine’s surroundings. Then, the algorithm estimates how reliably the robot executes an action (like sweeping) and whether it would be worthwhile to practice more. EES forecasts how well the robot could perform the overall task if it refines that particular skill, and finally, it practices. The vision system subsequently checks whether that skill was done correctly after each attempt.EES could come in handy in places like a hospital, factory, house, or coffee shop. For example, if you wanted a robot to clean up your living room, it would need help practicing skills like sweeping. According to Nishanth Kumar SM ’24 and his colleagues, though, EES could help that robot improve without human intervention, using only a few practice trials.“Going into this project, we wondered if this specialization would be possible in a reasonable amount of samples on a real robot,” says Kumar, co-lead author of a&nbsp;<a href="https://arxiv.org/pdf/2402.15025.pdf" target="_blank" data-lf-fd-inspected-kn9eq4roawkarlvp="true">paper</a> describing the work, PhD student in electrical engineering and computer science, and a CSAIL affiliate. “Now, we have an algorithm that enables robots to get meaningfully better at specific skills in a reasonable amount of time with tens or hundreds of data points, an upgrade from the thousands or millions of samples that a standard reinforcement learning algorithm requires.”<h2>See Spot sweep</h2>EES’s knack for efficient learning was evident when implemented on Boston Dynamics’ Spot quadruped during research trials at The AI Institute. The robot, which has an arm attached to its back, completed manipulation tasks after practicing for a few hours. In one demonstration, the robot learned how to securely place a ball and ring on a slanted table in roughly three hours. In another, the algorithm guided the machine to improve at sweeping toys into a bin within about two hours. Both results appear to be an upgrade from previous frameworks, which would have likely taken more than 10 hours per task. “We aimed to have the robot collect its own experience so it can better choose which strategies will work well in its deployment,” says co-lead author Tom Silver SM ’20, PhD ’24, an electrical engineering and computer science (EECS) alumnus and CSAIL affiliate who is now an assistant professor at Princeton University. “By focusing on what the robot knows, we sought to answer a key question: In the library of skills that the robot has, which is the one that would be most useful to practice right now?”EES could eventually help streamline autonomous practice for robots in new deployment environments, but for now, it comes with a few limitations. For starters, they used tables that were low to the ground, which made it easier for the robot to see its objects. Kumar and Silver also 3D printed an attachable handle that made the brush easier for Spot to grab. The robot didn’t detect some items and identified objects in the wrong places, so the researchers counted those errors as failures.<h2>Giving robots homework</h2>The researchers note that the practice speeds from the physical experiments could be accelerated further with the help of a simulator. Instead of physically working at each skill autonomously, the robot could eventually combine real and virtual practice. They hope to make their system faster with less latency, engineering EES to overcome the imaging delays the researchers experienced. In the future, they may investigate an algorithm that reasons over sequences of practice attempts instead of planning which skills to refine. “Enabling robots to learn on their own is both incredibly useful and extremely challenging,” says Danfei Xu, an assistant professor in the School of Interactive Computing at Georgia Tech and a research scientist at NVIDIA AI, who was not involved with this work. “In the future, home robots will be sold to all sorts of households and expected to perform a wide range of tasks. We can't possibly program everything they need to know beforehand, so it’s essential that they can learn on the job. However, letting robots loose to explore and learn without guidance can be very slow and might lead to unintended consequences. The research by Silver and his colleagues introduces an algorithm that allows robots to practice their skills autonomously in a structured way. This is a big step towards creating home robots that can continuously evolve and improve on their own.”“This paper tackles a very challenging and interesting problem: how to enable a robot to improve a repertoire of low-level skills based on interactions with the environment, while sequencing these low-level behaviors into long-horizon plans,” says Florian Shkurti, University of Toronto assistant professor in robotics who was also not involved with the paper. “At the heart of this work is a new method for probabilistically modeling the competence of a skill as a function of the number of data collected, and prioritizing the practice of skills that will lead to the biggest expected improvement in a distribution of tasks. Every active learning algorithm crucially depends on these two design choices, and the paper shows that a simple probabilistic competence model and the expected improvement criterion lead to effective, targeted, and fast exploration.” Silver and Kumar’s co-authors are The AI Institute researchers Stephen Proulx and Jennifer Barry, plus four CSAIL members: Northeastern University PhD student and visiting researcher Linfeng Zhao, MIT EECS PhD student Willie McClinton, and MIT EECS professors Leslie Pack Kaelbling and Tomás Lozano-Pérez. Their work was supported, in part, by The AI Institute, the U.S. National Science Foundation, the U.S. Air Force Office of Scientific Research, the U.S. Office of Naval Research, the U.S. Army Research Office, and MIT Quest for Intelligence, with high-performance computing resources from the MIT SuperCloud and Lincoln Laboratory Supercomputing Center.

The phrase “practice makes perfect” is usually reserved for humans, but it’s also a great maxim for robots newly deployed in unfamiliar environments.

Helping robots practice skills independently to adapt to unfamiliar environments

EPFL researchers have published a programmable framework that overcomes a key computational bottleneck of optics-based artificial intelligence systems. In a series of image classification experiments, they used scattered light from a low-power laser to perform accurate, scalable computations using a fraction of the energy of electronics.As digital artificial intelligence systems grow in size and impact, so does the energy required to train and deploy them – not to mention the associated <a href="https://www.technologyreview.com/2023/12/05/1084417/ais-carbon-footprint-is-bigger-than-you-think/" target="_blank" rel="noopener">carbon emissions</a>. Recent research suggests that if current AI server production continues at its current pace, their annual energy consumption could outstrip <a href="https://www.sciencedirect.com/science/article/pii/S2542435123003653" target="_blank" rel="noopener">that of a small country</a> by 2027. Deep neural networks, inspired by the architecture of the human brain, are especially power-hungry due to the millions or even billions of connections between multiple layers of neuron-like processors.To counteract this mushrooming energy demand, researchers have doubled down on efforts to implement optical computing systems, which have existed experimentally since the 1980s. These systems rely on photons to process data, and although light can theoretically be used to perform computations much faster and more efficiently than electrons, a key challenge has hindered optical systems’ ability to surpass the electronic state-of-the art.<blockquote>Our method is up to 1,000 times more power-efficient than state-of-the-art deep digital networks, making it a promising platform for realizing optical neural networks.<footer>Demetri Psaltis, Optics Laboratory</footer></blockquote>“In order to classify data in a neural network, each node, or ‘neuron’, must make a ‘decision’ to fire or not based on weighted input data. This decision leads to what is known as a nonlinear transformation of the data, meaning the output is not directly proportional to the input,” says Christophe Moser, head of the <a href="https://www.epfl.ch/labs/lapd/" target="_blank">Laboratory of Applied Photonics Devices</a> in EPFL’s School of Engineering.<img src="https://s3-us-west-1.amazonaws.com/wevolver-project-images/froala%2F1723249459280-c685037f.jpg" style="width: 100%px;" class="fr-fic fr-dib">Postdoctoral researcher Niyazi Ulas Dinc and PhD student Mustafa Yildrim © Alain HerzogMoser explains that while digital neural networks can easily perform nonlinear transformations with transistors, in optical systems, this step requires very powerful lasers. Moser worked with students Mustafa Yildirim, Niyazi Ulas Dinc, and Ilker Oguz, as well as <a href="https://www.epfl.ch/labs/lo/people-2/demetri-psaltis/" target="_blank">Optics Laboratory</a> head Demetri Psaltis, to develop an energy-efficient method for performing these nonlinear computations optically. Their new approach involves encoding data, like the pixels of an image, in the spatial modulation of a low-power laser beam. The beam reflects back on itself several times, leading to a nonlinear multiplication of the pixels."Our image classification experiments on three different datasets showed that our method is scalable, and up to 1,000 times more power-efficient than state-of-the-art deep digital networks, making it a promising platform for realizing optical neural networks,” says Psaltis.The research, supported by a <a href="https://www.snf.ch/en/HzVMPWm96mz69ZJ8/funding/programmes/sinergia" target="_blank" rel="noopener">Sinergia</a> grant from the Swiss National Science Foundation, has recently been published in <a href="https://www.nature.com/articles/s41566-024-01494-z" target="_blank" rel="noopener">Nature Photonics</a>.<h2>A simple structural solution</h2>In nature, photons do not directly interact with each other the way charged electrons do. To achieve nonlinear transformations in optical systems, scientists have therefore had to ‘force’ photons to interact indirectly, for example by using a light intense enough to modify the optical properties of the glass or other material it passes through.<img src="https://s3-us-west-1.amazonaws.com/wevolver-project-images/froala%2F1723249488379-af235dfc.jpg" style="width: 100%px;" class="fr-fic fr-dib">The system was tested in a series of image classification experiments © Alain Herzog/Jih-Liang HsiehThe scientists worked around this need for a high-power laser with an elegantly simple solution: they encoded the pixels of an image spatially on the surface of a low-power laser beam. By performing this encoding twice, via adjustment of the trajectory of the beam in the encoder, the pixels are multiplied by themselves, i.e., squared. Since squaring is a non-linear transformation, this structural modification achieves the non-linearity essential to neural network calculations, at a fraction of the energy cost. This encoding can be carried out two, three or even ten times, increasing the non-linearity of the transformation and the precision of the calculation.“We estimate that using our system, the energy required to optically compute a multiplication is eight orders of magnitude less than that required for an electronic system,” Psaltis says.Moser and Psaltis emphasize that the scalability of their low-energy approach is a major advantage, as the ultimate goal would be to use hybrid electronic-optical systems to mitigate the energy consumption of digital neural networks. However, further engineering research is needed to achieve such scale-up. For example, because optical systems use different hardware than electronic systems, a next step that the researchers are already working on is developing a compiler to translate digital data into code that optical systems can use.<hr>ReferencesYildirim, M., Dinc, N. U., Oguz, I., Psaltis, D., &amp; Moser, C. (2023). Nonlinear Processing with Linear Optics. arXiv preprint arXiv:2307.08533.The researchers note that structural nonlinearity has also recently been reported, using a different implementation method, by&nbsp;<a href="https://arxiv.org/abs/2208.08906" target="_blank" rel="noopener">Cao et al</a> at Yale University.

EPFL researchers have published a programmable framework that overcomes a key computational bottleneck of optics-based artificial intelligence systems.

Engineers bring efficient optical neural networks into focus

A team of computer scientists, engineers, mathematicians and cognitive scientists, led by the University of Cambridge, developed an open-source evaluation platform called CheckMate, which allows human users to interact with and evaluate the performance of large language models (LLMs).The researchers tested CheckMate in an experiment where human participants used three LLMs – InstructGPT, ChatGPT and GPT-4 – as assistants for solving undergraduate-level mathematics problems.The team studied how well LLMs can assist participants in solving problems. Despite a generally positive correlation between a chatbot’s correctness and perceived helpfulness, the researchers also found instances where the LLMs were incorrect, but still useful for the participants. However, certain incorrect LLM outputs were thought to be correct by participants. This was most notable in LLMs optimised for chat.The researchers suggest models that communicate uncertainty, respond well to user corrections, and can provide a concise rationale for their recommendations, make better assistants. Human users of LLMs should verify their outputs carefully, given their current shortcomings.The&nbsp;<a href="https://www.pnas.org/doi/10.1073/pnas.2318124121" target="_blank">results</a>, reported in the&nbsp;Proceedings of the National Academy of Sciences (PNAS), could be useful in both informing AI literacy training, and help developers improve LLMs for a wider range of uses.While LLMs are becoming increasingly powerful, they can also make mistakes and provide incorrect information, which could have negative consequences as these systems become more integrated into our everyday lives.“LLMs have become wildly popular, and evaluating their performance in a quantitative way is important, but we also need to evaluate how well these systems work with and can support people,” said co-first author Albert Jiang, from Cambridge’s Department of Computer Science and Technology. “We don’t yet have comprehensive ways of evaluating an LLM’s performance when interacting with humans.”The standard way to evaluate LLMs relies on static pairs of inputs and outputs, which disregards the interactive nature of chatbots, and how that changes their usefulness in different scenarios. The researchers developed CheckMate to help answer these questions, designed for but not limited to applications in mathematics.“When talking to mathematicians about LLMs, many of them fall into one of two main camps: either they think that LLMs can produce complex mathematical proofs on their own, or that LLMs are incapable of simple arithmetic,” said co-first author&nbsp;<a href="https://www.eng.cam.ac.uk/profiles/kmc61" target="_blank">Katie Collins</a>, PhD student&nbsp;from the Department of Engineering. “Of course, the truth is probably somewhere in between, but we wanted to find a way of evaluating which tasks LLMs are suitable for and which they aren’t.”The researchers recruited 25 mathematicians, from undergraduate students to senior professors, to interact with three different LLMs (InstructGPT, ChatGPT, and GPT-4) and evaluate their performance using CheckMate. Participants worked through undergraduate-level mathematical theorems with the assistance of an LLM and were asked to rate each individual LLM response for correctness and helpfulness. Participants did not know which LLM they were interacting with.The researchers recorded the sorts of questions asked by participants, how participants reacted when they were presented with a fully or partially incorrect answer, whether and how they attempted to correct the LLM, or if they asked for clarification. Participants had varying levels of experience with writing effective prompts for LLMs, and this often affected the quality of responses that the LLMs provided.An example of an effective prompt is “what is the definition of X” (X being a concept in the problem) as chatbots can be very good at retrieving concepts they know of and explaining it to the user.“One of the things we found is the surprising fallibility of these models,” said Katie Collins. “Sometimes, these LLMs will be really good at higher-level mathematics, and then they’ll fail at something far simpler. It shows that it’s vital to think carefully about how to use LLMs effectively and appropriately.”However, like the LLMs, the human participants also made mistakes. The researchers asked participants to rate how confident they were in their own ability to solve the problem they were using the LLM for. In cases where the participant was less confident in their own abilities, they were more likely to rate incorrect generations by LLM as correct.“This kind of gets to a big challenge of evaluating LLMs, because they’re getting so good at generating nice, seemingly correct natural language, that it’s easy to be fooled by their responses,” said Albert Jiang. “It also shows that while human evaluation is useful and important, it’s nuanced, and sometimes it’s wrong. Anyone using an LLM, for any application, should always pay attention to the output and verify it themselves.”Based on the results from CheckMate, the researchers say that newer generations of LLMs are increasingly able to collaborate helpfully and correctly with human users on undergraduate-level maths problems, as long as the user can assess the correctness of LLM-generated responses. Even if the answers may be memorised and can be found somewhere on the internet, LLMs have the advantage of being flexible in their inputs and outputs over traditional search engines (though should not replace search engines in their current form).While CheckMate was tested on mathematical problems, the researchers say their platform could be adapted to a wide range of fields. In the future, this type of feedback could be incorporated into the LLMs themselves, although none of the CheckMate feedback from the current study has been fed back into the models.“These kinds of tools can help the research community to have a better understanding of the strengths and weaknesses of these models,” said Katie Collins. “We wouldn’t use them as tools to solve complex mathematical problems on their own, but they can be useful assistants, if the users know how to take advantage of them.”The research was supported in part by the Marshall Commission, the Cambridge Trust, Peterhouse, Cambridge, The Alan Turing Institute, the European Research Council, and the Engineering and Physical Sciences Research Council (EPSRC), part of UK Research and Innovation (UKRI).<hr>Reference: Katherine M. Collins, Albert Q. Jiang, et al. ‘<a href="https://www.pnas.org/doi/10.1073/pnas.2318124121" target="_blank">Evaluating Language Models for Mathematics through Interactions</a>.’ Proceedings of the National Academy of Sciences (2024). DOI: 10.1073/pnas.2318124121

Chatbot concept. Credit: wutzkoh – stock.adobe.com

artificial intelligence

Latest Posts