Article #4 of the Enabling IoT Series. Industrial enterprises are moving to a new approach to asset management, namely predictive maintenance by leveraging powerful new sensors and data. 


Predictive Maintenance: Using Smart Sensors to get the most out of your assets

This is the second of a&nbsp;<a href="https://www.wevolver.com/article/reducing-ais-vulnerable-attack-surface-with-edge-computing" rel="noopener noreferrer" target="_blank">series of articles</a>&nbsp;exploring the benefits of Edge AI for a variety of applications.&nbsp;<style type="text/css" id="isPasted">
p.p1 {margin: 0.0px 0.0px 0.0px 0.0px; font: 11.0px Arial; color: #000000} p.p2 {margin: 0.0px 0.0px 0.0px 0.0px; font: 11.0px Arial; color: #000000; min-height: 12.0px} p.p3 {margin: 0.0px 0.0px 18.0px 0.0px; font: 11.0px Arial; color: #000000; min-height: 12.0px} p.p4 {margin: 0.0px 0.0px 6.0px 0.0px; font: 16.0px Arial; color: #000000} p.p5 {margin: 0.0px 0.0px 18.0px 0.0px; font: 11.0px Arial; color: #000000} p.p6 {margin: 0.0px 0.0px 0.0px 0.0px; text-align: justify; font: 11.0px Arial; color: #000000} p.p7 {margin: 0.0px 0.0px 0.0px 0.0px; text-align: justify; font: 11.0px Arial; color: #000000; min-height: 12.0px} p.p8 {margin: 0.0px 0.0px 0.0px 36.0px; font: 11.0px Arial; color: #000000; min-height: 12.0px} li.li1 {margin: 0.0px 0.0px 0.0px 0.0px; font: 11.0px Arial; color: #000000} li.li6 {margin: 0.0px 0.0px 0.0px 0.0px; text-align: justify; font: 11.0px Arial; color: #000000} span.s1 {font: 7.3px Arial} span.s2 {color: #103cc0} ul.ul1 {list-style-type: disc}
</style>Wearable technology, or &quot;wearables&quot;, refers to electronic devices that are worn as accessories such as glasses or watches, contain processing and computational capabilities, and some form of connectivity[1]. They also include devices embedded in clothing, implanted in a consumer&rsquo;s body, or even tattooed onto their skin.Wearables have experienced an incredibly rapid growth in recent years. Between 2016 and 2022, the number of connected wearable devices worldwide skyrocketed 240% from 325 million to 1.1 billion[2].However, a lot of the data and information processing these devices mediate is still done centrally on the cloud, or on a connected smartphone. This leads to high computing power consumptions, increased emissions, decreased battery life, and privacy concerns.In this article, we discuss how Edge AI can address these issues by making decisions not in the cloud, but instead on the device itself - or on the &ldquo;edge&rdquo; of our analog and digital worlds. Edge AI helps wearables&#39; battery life last longer, reduces latency and cloud communication delays, and facilitates a two-way interactive relationship between device and user, where it was previously more one-sided.<h2>What is Edge AI?&nbsp;</h2>Traditionally, artificial intelligence is often associated with the cloud, a centralized location where data is collected and AI algorithms are developed and run for processing or making decisions based on the data. By contrast, edge AI refers to the application of AI as close to the end user as possible - where the computation is done on the &quot;edge&quot; of the network, close to where the data is located and captured, rather than centrally in a cloud computing facility[3].The reach of the internet is global, meaning data can be captured anywhere. This makes it theoretically possible for the &quot;edge&quot; of a network to be any location on the planet. Depending on the data being captured, edge AI might be deployed in a hospital, a 5G cell tower, a retail store, on traffic lights, and on personal devices like phones, laptops, or wearables.Edge AI is a powerful tool that can bring digital transformation practices that are commonplace in the cloud close to the physical world[4]. It can be used to improve efficiency, performance, management, and security of personal devices using cloud best practices.&nbsp;<h2>How does Edge AI Work?&nbsp;</h2>Edge AI is an exciting development because it allows traditional technologies to run more efficiently, with higher performance, and on less power. It relies on neural networks and inference algorithms to bring AI development outside of the cloud[5].Neural networks are often used to deploy the most complex of machine learning algorithms, known as deep learning, and require a lot of data and computational power to be adequately trained[6]. Furthermore, the data provided to train this network has to cover examples of positive and negative versions of what you&#39;re trying to teach your network to decipher between. For example, if you&#39;re trying to teach a wearable heart rate monitor such as a FitBit to detect when your heart rate has risen to a dangerous high, you need to feed it data on healthy heart elevation rates as well as dangerous rates, so it learns to distinguish. This training - or deep learning as it is known - often takes place in the cloud or powerful GPUs due to the amount of data needed to train a model.Once the neural network is sufficiently trained, it can be taught to do inferencing. Inferencing is the act of using a trained neural network to provide insights independently without intervention from a human[7]. It isn&#39;t dissimilar to a student going from training to be a teacher to working in a classroom in the real world. The &quot;graduated&quot; trained models can now answer real-world questions relating to the data they were trained on.Training the network takes a lot of computational power but inferencing only requires a fraction of it. This means that it should be possible to inference on the edge, with devices that are purposefully designed to offer the right performance, while consuming very little energy so they can be deployed in a wearable device. For example, neural decision processors offer more than 100x efficiency improvement while providing over a 10x increase in throughput compared to typical MCUs, making them ideal for inferencing on the edge.<h2>Why Use Edge AI for Wearables?</h2>A rapidly growing application is the use of edge AI on wearable devices, specifically wearables targeted towards health and fitness. 75% of users agree that wearables help them keep track of their health[8].&nbsp;Using edge AI on wearables provides a host of benefits compared to relying on traditional cloud-based AI.<ul><li>Low power: A self-learning AI can update its edge AI framework without access to the cloud, requiring much less power to operate[9]. This is because deploying a trained neural network on the edge and allowing it to inference there, rather than in the cloud, means it requires significantly less computational power. This is great for companies and users, since there is no need for expensive, power-hungry hardware, and the device&#39;s battery life is extended considerably[10].</li><li>Less latency, more real-time results:&nbsp;Inferencing locally eliminates delays that could arise with cloud communication, or the possibility of central servers being down. Using a low-power computer and inference accelerator close to the source of data means much faster response time. Cloud-based response time is usually a few seconds. Edge AI takes this down to milliseconds[11].&nbsp;</li><li>Increased reliability: Since edge AI does not rely on communication back and forth from the cloud, it is able to work autonomously despite technical issues that might happen on the network or the cloud</li><li>Stronger security and data protection: EdgeAI applications minimize the amount of information transferred between the device and the cloud. This reduces the attack surface of the AI system, which leads to stronger security and fewer opportunities for data breaches. By reducing data beaches, EdgeAI provides a clear value proposition to wearables applications that handle sensitive data in areas such as smart cities and healthcare.</li></ul><img src="https://images.wevolver.com/eyJidWNrZXQiOiJ3ZXZvbHZlci1wcm9qZWN0LWltYWdlcyIsImtleSI6ImZyb2FsYS8xNjY1NDIwNzQzMDUwLWltYWdlMS5qcGVnIiwiZWRpdHMiOnsicmVzaXplIjp7IndpZHRoIjo5NTAsImZpdCI6ImNvdmVyIn19fQ==" style="width: 300px;" class="fr-fic fr-dib">Edge AI can provide secure conditions for training and maintenance with mixed reality wearables.<style type="text/css" id="isPasted">
p.p1 {margin: 0.0px 0.0px 6.0px 0.0px; font: 16.0px Arial; color: #000000} p.p2 {margin: 0.0px 0.0px 20.0px 0.0px; text-align: justify; font: 11.0px Arial; color: #000000} p.p3 {margin: 0.0px 0.0px 6.0px 0.0px; font: 20.0px Arial; color: #000000} p.p4 {margin: 0.0px 0.0px 0.0px 0.0px; text-align: justify; font: 11.0px Arial; color: #000000} p.p5 {margin: 0.0px 0.0px 0.0px 0.0px; text-align: justify; font: 11.0px Arial; color: #000000; min-height: 12.0px} p.p6 {margin: 0.0px 0.0px 18.0px 0.0px; text-align: justify; font: 11.0px Arial; color: #000000; min-height: 12.0px} p.p7 {margin: 0.0px 0.0px 12.0px 0.0px; font: 11.0px Arial; color: #000000; min-height: 12.0px} li.li8 {margin: 0.0px 0.0px 0.0px 0.0px; font: 11.0px Arial; color: #000000} span.s1 {text-decoration: underline ; color: #103cc0} span.s2 {font: 7.3px Arial} ol.ol1 {list-style-type: decimal}
</style><h2>Conclusion</h2>Edge AI brings hyper-personalized, ultra-low-power solutions to end users by bypassing the cloud and carrying out all inference locally at the edge. It enables pervasive wearable devices to access the compute resources they need, while ensuring strong security and real-time performance. Moreover, local inference allows for a smoother user/device interaction, for example through voice-activated and touch-free wearables.&nbsp;<h2>Designing Edge AI wearables with Syntiant</h2>California-based edge AI company <a href="https://www.syntiant.com/applications" target="_blank">Syntiant</a> makes ready-to-use technology that allows users to interface naturally with their wearable devices[12]. For example, Syntiant empowers users to interact with their devices through a speech interface, or gestures, without having to navigate a menu on a small screen.Syntiant&rsquo;s NDPs technology can enable any device with always-on neural processing capabilities, offering unparalleled power and performance. Specifically, Syntiant&rsquo;s end-to-end solutions bring a new level of machine learning to wearables, which was previously only available at 10x the power and at least twice the die size.Syntiant technology enables the integration and execution of low-power, performance savvy applications into smartwatches, fitness trackers, health monitoring wristbands and smart textile devices. As a prominent example, Syntiant NDP technology empowers wearable devices OEMs (Original Equipment Manufacturers) to integrate robust speech processing capabilities for effective voice commands. Likewise, it can also enable the integration of advanced motion detection functionalities that can greatly enhance the intelligence capabilities of the wearable device.&nbsp;Overall, Syntiant facilitates the implementation of small-size, low-power wearable solutions that provide unique user interfaces. The latter enable solutions that are ergonomic, extremely easy to use, and stand out in the market. Syntiant&rsquo;s technology is ready to integrate and use within real-life wearables applications.The first article discussed <a href="https://www.wevolver.com/article/reducing-ais-vulnerable-attack-surface-with-edge-computing" rel="noopener noreferrer" target="_blank">about r</a><a href="https://www.wevolver.com/article/reducing-ais-vulnerable-attack-surface-with-edge-computing" rel="noopener noreferrer" target="_blank">educing AI&#39;s Vulnerable Attack Surface with Edge Computing.</a>The second article talked about <a href="https://www.wevolver.com/article/why-edge-ai-for-wearables" rel="noopener noreferrer" target="_blank">Edge AI in wearables.</a>The third article explored <a href="https://www.wevolver.com/article/how-edge-ai-is-enabling-cutting-edge-advances-in-sustainability" rel="noopener noreferrer" target="_blank">h</a><a href="https://www.wevolver.com/article/how-edge-ai-is-enabling-cutting-edge-advances-in-sustainability" rel="noopener noreferrer" target="_blank">ow edge AI is enabling cutting-edge advances in sustainability.</a>The forth article explained <a href="https://www.wevolver.com/article/why-edge-ai-is-a-win-for-automotive" rel="noopener noreferrer" target="_blank">why Edge AI is a win for automotive.</a>The fifth article analyzed <a href="https://www.wevolver.com/article/computer-vision-on-compute-constrained-embedded-devices-high-performance-and-increased-security-at-a-lower-cost" rel="noopener noreferrer" target="_blank">c</a><a href="https://www.wevolver.com/article/computer-vision-on-compute-constrained-embedded-devices-high-performance-and-increased-security-at-a-lower-cost" rel="noopener noreferrer" target="_blank">omputer Vision on Compute-Constrained Embedded Devices.</a>The sixth article explained <a href="https://www.wevolver.com/article/why-edge-ai-for-ev-battery-management" rel="noopener noreferrer" target="_blank">why e</a><a href="https://www.wevolver.com/article/why-edge-ai-for-ev-battery-management" rel="noopener noreferrer" target="_blank">dge AI is essential for EV Battery Management.</a><h2 id="isPasted">About the sponsor: Syntiant</h2>Syntiant Corp. is a leader in delivering end-to-end deep learning solutions for always-on applications by combining purpose-built silicon with an edge-optimized data platform and training pipeline. The company&rsquo;s advanced chip solutions merge deep learning with semiconductor design to produce ultra-low-power, high-performance, deep neural network processors for edge AI applications across a wide range of consumer and industrial use cases, from earbuds to automobiles. Syntiant&rsquo;s Neural Decision Processors&trade; typically offer more than 100x efficiency improvement, while providing a greater than 10x increase in throughput over current low-power MCU-based solutions, and subsequently, enabling larger networks at significantly lower power.<img src="https://images.wevolver.com/eyJidWNrZXQiOiJ3ZXZvbHZlci1wcm9qZWN0LWltYWdlcyIsImtleSI6ImZyb2FsYS8xNjY1NDIwOTA3MzQ2LVVudGl0bGVkIGRlc2lnbi02ICgxKS5wbmciLCJlZGl0cyI6eyJyZXNpemUiOnsid2lkdGgiOjk1MCwiZml0IjoiY292ZXIifX19" style="width: 300px;" class="fr-fic fr-dib"> <h2>References</h2>1. Hayes A. The Ins and outs of Wearable Technology [Internet]. Brock T, editor. Investopedia. Investopedia; 2022 [cited 2022Oct6]. Available from: <a href="https://www.investopedia.com/terms/w/wearable-technology.asp" target="_blank">https://www.investopedia.com/terms/w/wearable-technology.asp</a>&nbsp;2. Laricchia F. Global Connected Wearable Devices 2016-2022 [Internet]. Statista. 2022 [cited 2022Oct6]. Available from: <a href="https://www.statista.com/statistics/487291/global-connected-wearable-devices" target="_blank">https://www.statista.com/statistics/487291/global-connected-wearable-devices</a> &nbsp;3. Yeung T. What is Edge Ai and How does it Work?&amp;nbsp; [Internet]. NVIDIA Blog. 2022 [cited 2022Oct6]. Available from: <a href="https://blogs.nvidia.com/blog/2022/02/17/what-is-edge-ai/" target="_blank">https://blogs.nvidia.com/blog/2022/02/17/what-is-edge-ai/</a>&nbsp;4. Lawton G. What is Edge AI? [Internet]. SearchEnterpriseAI. TechTarget; 2021 [cited 2022Oct6]. Available from: <a href="https://www.techtarget.com/searchenterpriseai/definition/edge-AI" target="_blank">https://www.techtarget.com/searchenterpriseai/definition/edge-AI</a>&nbsp;5. Ibid6. Ceron R. AI Today: Data, Training and Inferencing [Internet]. Servers &amp;amp; Storage. 2020 [cited 2022Oct6]. Available from: <a href="https://www.ibm.com/blogs/systems/ai-today-data-training-and-inferencing/" target="_blank">https://www.ibm.com/blogs/systems/ai-today-data-training-and-inferencing/</a>&nbsp;7. Yeung T. What is Edge Ai and How does it Work?&amp;nbsp; [Internet]. NVIDIA Blog. 2022 [cited 2022Oct6]. Available from: <a href="https://blogs.nvidia.com/blog/2022/02/17/what-is-edge-ai/" target="_blank">https://blogs.nvidia.com/blog/2022/02/17/what-is-edge-ai/</a>&nbsp;8. Samet A. Wearable technology trends that will shape healthcare in 2022 [Internet]. Insider Intelligence. 2022 [cited 2022Oct6]. Available from: <a href="https://www.insiderintelligence.com/insights/top-healthcare-wearable-technology-trends/" target="_blank">https://www.insiderintelligence.com/insights/top-healthcare-wearable-technology-trends/</a>&nbsp;9. Flaherty N. First self-learning edge AI sensor for Wearables - Video [Internet]. EENewsEurope. 2020 [cited 2022Oct6]. Available from: <a href="https://www.eenewsanalog.com/en/first-self-learning-edge-ai-sensor-for-wearables-video/" target="_blank">https://www.eenewsanalog.com/en/first-self-learning-edge-ai-sensor-for-wearables-video/</a>&nbsp;10. Woolley A. What is AI Inference at the Edge?: Insights: Steatite [Internet]. Steatite Embedded. 2020 [cited 2022Oct6]. Available from: <a href="https://www.steatite-embedded.co.uk/what-is-ai-inference-at-the-edge/" target="_blank">https://www.steatite-embedded.co.uk/what-is-ai-inference-at-the-edge/</a>&nbsp;

Article #2 of the "Why Edge?" Series. Edge AI increases battery life, reduces latency, and in many cases, eliminates the need for cloud communication, making wearable devices more user-friendly and secure. 


Why Edge AI for Wearables?

This is the first of a series of articles exploring the benefits of Edge AI for a variety of applications.&nbsp;<style type="text/css" id="isPasted">
p.p1 {margin: 0.0px 0.0px 0.0px 0.0px; text-align: justify; font: 11.0px Arial; color: #000000} p.p2 {margin: 0.0px 0.0px 0.0px 0.0px; text-align: justify; font: 11.0px Arial; color: #000000; min-height: 12.0px} p.p3 {margin: 0.0px 0.0px 0.0px 0.0px; font: 11.0px Arial; color: #000000; min-height: 12.0px} p.p4 {margin: 0.0px 0.0px 0.0px 0.0px; font: 12.0px Arial; color: #000000} p.p5 {margin: 0.0px 0.0px 0.0px 0.0px; font: 14.0px Arial; color: #000000; min-height: 16.0px} p.p6 {margin: 0.0px 0.0px 0.0px 0.0px; text-align: justify; font: 12.0px Arial; color: #000000} p.p7 {margin: 0.0px 0.0px 18.0px 0.0px; text-align: justify; font: 11.0px Arial; color: #000000} p.p8 {margin: 0.0px 0.0px 6.0px 0.0px; font: 12.0px Arial; color: #000000} li.li1 {margin: 0.0px 0.0px 0.0px 0.0px; text-align: justify; font: 11.0px Arial; color: #000000} span.s1 {text-decoration: underline ; color: #0000ff} span.s2 {font: 11.0px Symbol} ul.ul1 {list-style-type: disc}
</style>The proliferation of digital transactions, smart phones, cyber-physical systems, and other internet connected objects in modern enterprises has led to generation of unprecedented volumes of data. Nowadays, companies are increasingly taking advantage of this data to improve the efficacy of their business processes and make better decisions via analytics. Machine learning (ML) and Artificial Intelligence (AI) algorithms play an instrumental role in collecting and analyzing enterprise data in fast, automated, and cost-efficient ways.&nbsp;Most enterprise-scale ML/AI systems are deployed in conjunction with cloud computing infrastructures. This is because cloud computing eases the storage and management of large datasets, as well as the processing of many data points. Nevertheless, cloud-based AI systems are not the best choice when it comes to processing data close to the field and providing real-time performance. Likewise, cloud systems are vulnerable to significant cyber-security threats as large volumes of data are transferred from different field systems to a single cloud infrastructure.&nbsp;The edge computing paradigm alleviates the latency and security limitations of cloud computing. Edge computing brings data collection and decision-making closer to the field where data are produced. In this way, it also moves computing power close to the data sources, which reduces latency. Moreover, it limits the amount of data that are transferred from the field to the cloud, which foundational for reducing data breaches and stronger data protection.&nbsp;Reduced latency and increased security is the reason behind the ongoing shift of AI/ML applications <a href="https://www.idc.com/getdoc.jsp?containerId=prUS49532322" target="_blank">from the cloud to the edge</a>. In recent years edge AI deployments are gaining traction across various verticals. For instance, connected and autonomous vehicles leverage edge AI to reduce latency as split-second decision-making can make a huge impact on passenger safety. Similarly, across the manufacturing sectors, companies are taking advantage of ML algorithms on the edge to provide near real-time detection of machine failures and of defective products.&nbsp;<h2>The Security Benefits of Edge AI</h2>In general, state-of-the-art AI/ML applications needs strong security for a variety of reasons, including:<ul><li>Their data intensive nature, which makes them susceptible to costly breaches. This can be critical especially in applications that handle sensitive information such as healthcare and personal data management.&nbsp;</li><li>Machine learning itself has security vulnerabilities due to its reliance on suitable training data for proper operation. For example, in instances known as poisoning attacks adversaries compromise the operation of an ML system through by its training data with malicious datasets. In another example, evasion attacks confuse ML/AI systems by providing them with adversarial examples e.g., perturbed malicious input that looks like an untampered copy, yet it side-tracks the operation of the ML algorithms. &nbsp;&nbsp;</li></ul>Edge computing is recognized for its ability to provide stronger security and data protection than conventional cloud deployments. Specifically, edge AI provides the following security benefits over cloud AI deployments:<ul><li>Operation based on safer (local) data. In edge AI data need not travel over a wide area network, which reduces the possibility of cybersecurity attacks. AI algorithms operate based on local data, which reside within edge servers or on the device itself. This makes them much more difficult to tamper with than data that are transferred across networks.</li><li>Almost impossible for malicious party to access all data. Edge AI datasets are distributed across various on-premises data centers or devices, which means that there is no single point of vulnerability. Edge AI systems use distributed data that reside at the edge of the network, which makes it impossible for an adversary to gain access to all the data by attacking one point of the network. &nbsp;</li><li>Much fewer breach points. Edge AI deployments limit drastically the number of connections and the number of data transfers from edge to the cloud. This means that they provide much fewer breach points than conventional cloud AI deployments. Moreover, the fewer number of connections facilitates the deployment of a set of carefully encrypted connections between the edge devices and the cloud data centers. In this way, the vulnerable attack surface of edge AI applications is significantly reduced.</li><li>Flexible and cost-effective regulatory compliance. Edge AI facilitates enterprises in their efforts to comply with mandatory privacy and data protection regulations such as the GDPR (General Data Protection Regulation) in Europe. In particular, edge computing enables end-users and data providers to control their personal data rather than relying on the services of a vendor or infrastructure provider. Moreover, it enables the implementation of decentralized security policies that tend to be more resilient. In this way, the risk of non-compliance for AI/ML services providers is significantly reduced.&nbsp;</li></ul><h2>Increasing Resilience with Different Edge AI Deployment Paradigms&nbsp;</h2>Edge AI applications are usually deployed in conjunction with cloud infrastructures, as some algorithms run in the cloud, where data points from various distributed sources and edge computing processes are aggregated. Moreover, edge AI applications come at varying deployment configurations based on the placement of ML/AI processes. For instance, AI analytics functions can be run inside a microsystem (e.g., a sensor), within an edge computing cluster or gateway, or even in the cloud. &nbsp;Different ML paradigms enable a wide set of edge AI deployment configurations. These ML paradigms include:&nbsp;<ul><li>Embedded Machine Learning, which means executing ML algorithms within embedded devices. This paradigm enables data processing and data analytics within devices embedded with powerful processors ranging from CPUs and GPUs to neural native AI processing chips. Set-top boxes, smartphones, connected cars OBUs (On-Board Units), and many other types of fog nodes and internet-connected devices make use of this paradigm.&nbsp;</li><li>TinyML, which refers to the execution of full stack machine learning systems (e.g., deep neural networks) within very small processors such as microcontrollers and AI Accelerators amongst others. In most cases, TinyML systems serve applications that do not require complex computations. TinyML delivers the security benefits of edge AI at the highest possible degree. This is because data are virtually processed inside the data source and there is absolutely no need for data I/O (input/output) operations.</li><li>Federated Machine Learning, is a novel ML technique that trains an algorithm across multiple decentralized edge devices. This technique is very powerful as it offers the benefits of a consolidated data lake while allowing data owners have full control of their data locally, offering great security and performance.</li></ul>In non-trivial edge AI applications, it is possible to combine more than one of the above paradigms in a solution configuration. This provides increased versatility in specifying deployment configurations that meet stringent security and data protection needs.&nbsp;<h2>Case Study: Security and Data Protection for Healthcare Applications&nbsp;</h2>The future of healthcare relies on the collection and processing of massive amounts of data from patients, including for example, data from medical and consumer devices (e.g., smart watches), clinical data, laboratory data, diagnostic devices (e.g., medical imaging), genomic databases, data about the patient&rsquo;s medical history, and many more.&nbsp;The analysis of this data though AI/ML algorithms is already driving significant improvements in many healthcare sectors, including prevention, diagnosis, prognosis, and treatment. For example, the analysis of clinical, genomic, imaging, and lifestyle data provides cardiologists with predictive insights on risk factors for various cardiovascular diseases. However, the regulatory approval and consumer adoption of AI/ML enhances services hinges on strong security and data protection measures.Many state-of-the-art ML applications in the healthcare domain rely on cloud infrastructures, which creates considerable risks for data breaches and other cyber attacks. This is because large data volumes are transferred from the patients to the cloud, which makes it easier for malicious parties to tamper with healthcare data. In this context, the deployment of ML systems and algorithms at the edge of the network is becoming a good practice that alleviates such security concerns.&nbsp;In an edge AI deployment, patient data is initially processed locally within edge devices like IoT (Internet of Things) gateways. Hence, patient data do not travel outside the perimeter of the network of the hospital or home care infrastructure. Nevertheless, select AI-based insights can be properly encrypted and transferred to the cloud infrastructure to enable further processing for statistical or clinical purposes (e.g., clinical trials). For example, if a patient&rsquo;s respiratory sounds are to be analyzed, instead of transferring the entire recording to the cloud, edge AI may be used to infer significant events: number of coughs, sneezes, wheezes, and so on. In this way, both the volume of data that needs to be transferred to the clouds is reduced, reducing bandwidth requirements, and the patient&rsquo;s data, which may include conversations is protected on the premise or within a device.&nbsp;Healthcare data processing infrastructures must be compliant to applicable regulations like HIIPA (Health Insurance Portability and Accountability) in US and GDPR in Europe. This can be quite challenging given that patient information comprises personally identifiable data. Fortunately, Edge AI can help reduce the risks of data tampering and boosts regulatory compliance. Most importantly, patients and healthcare providers can choose when to share their data and for what purpose. This eases the implementation of informed consent processes that are fundamental for compliance to data protection regulations.&nbsp;<h2>Conclusion&nbsp;</h2>Edge computing can provide stronger security and data protection than conventional cloud deployments making it the right choice for security-sensitive applications such as Ip protection, healthcare, and many more. This is the first in a series of articles exploring the benefits of Edge AI across applications, industries, and products.&nbsp;The first article discussed <a href="https://www.wevolver.com/article/reducing-ais-vulnerable-attack-surface-with-edge-computing" rel="noopener noreferrer" target="_blank">about r</a><a href="https://www.wevolver.com/article/reducing-ais-vulnerable-attack-surface-with-edge-computing" rel="noopener noreferrer" target="_blank">educing AI&#39;s Vulnerable Attack Surface with Edge Computing.</a>The second article talked about <a href="https://www.wevolver.com/article/why-edge-ai-for-wearables" rel="noopener noreferrer" target="_blank">Edge AI in wearables.</a>The third article explored <a href="https://www.wevolver.com/article/how-edge-ai-is-enabling-cutting-edge-advances-in-sustainability" rel="noopener noreferrer" target="_blank">h</a><a href="https://www.wevolver.com/article/how-edge-ai-is-enabling-cutting-edge-advances-in-sustainability" rel="noopener noreferrer" target="_blank">ow edge AI is enabling cutting-edge advances in sustainability.</a>The forth article explained <a href="https://www.wevolver.com/article/why-edge-ai-is-a-win-for-automotive" rel="noopener noreferrer" target="_blank">why Edge AI is a win for automotive.</a>The fifth article analyzed <a href="https://www.wevolver.com/article/computer-vision-on-compute-constrained-embedded-devices-high-performance-and-increased-security-at-a-lower-cost" rel="noopener noreferrer" target="_blank">c</a><a href="https://www.wevolver.com/article/computer-vision-on-compute-constrained-embedded-devices-high-performance-and-increased-security-at-a-lower-cost" rel="noopener noreferrer" target="_blank">omputer Vision on Compute-Constrained Embedded Devices.</a>The sixth article explained <a href="https://www.wevolver.com/article/why-edge-ai-for-ev-battery-management" rel="noopener noreferrer" target="_blank">why e</a><a href="https://www.wevolver.com/article/why-edge-ai-for-ev-battery-management" rel="noopener noreferrer" target="_blank">dge AI is essential for EV Battery Management.</a><h2>Security with Syntiant</h2>Syntiant combines advanced silicon solutions and deep learning models to provide ultra-low-power, high performance, deep neural network processing for edge AI applications across a wide range of consumer and industrial use cases, from earbuds to automobiles. The company&rsquo;s Neural Decision Processors (NDP) are optimally designed to deploy deep (machine) learning models on the edge, where power and area are often constrained. Syntiant&rsquo;s NDP solutions can equip, every device, from earbuds and doorbells to automobiles and healthcare wearables, with powerful deep learning capabilities, enabling real-time data processing and decision making, with near zero latency, enabling secure and private artificial intelligent solutions.<img src="https://images.wevolver.com/eyJidWNrZXQiOiJ3ZXZvbHZlci1wcm9qZWN0LWltYWdlcyIsImtleSI6ImZyb2FsYS8xNjY0NzMzNDk2NDc3LVVudGl0bGVkIGRlc2lnbi02ICgxKS5wbmciLCJlZGl0cyI6eyJyZXNpemUiOnsid2lkdGgiOjk1MCwiZml0IjoiY292ZXIifX19" style="width: 300px;" class="fr-fic fr-dib">&nbsp;

Article #1 of the "Why Edge?" Series. Edge AI presents a multitude of benefits for security across industry. 


Reducing AI's Vulnerable Attack Surface with Edge Computing

A short guide to understanding the edge neural network design. 


Neural Network Architectures at the Edge: Modelling for Energy Efficiency and Machine Learning Performance

This article is based on two blogs by Rouzbeh Shirvani and Sean McGregor, with additions and editing by John Soldatos.&nbsp;&nbsp;<style type="text/css" id="isPasted">
p.p1 {margin: 0.0px 0.0px 0.0px 0.0px; text-align: justify; font: 11.0px Arial; color: #000000} p.p2 {margin: 0.0px 0.0px 0.0px 0.0px; text-align: justify; font: 11.0px Arial; color: #000000; min-height: 12.0px} p.p3 {margin: 0.0px 0.0px 0.0px 0.0px; font: 14.0px Arial; color: #000000; min-height: 16.0px} p.p4 {margin: 0.0px 0.0px 0.0px 0.0px; font: 12.0px Arial; color: #000000; min-height: 14.0px} p.p5 {margin: 0.0px 0.0px 0.0px 0.0px; font: 12.0px Arial; color: #000000} p.p6 {margin: 0.0px 0.0px 0.0px 0.0px; font: 11.0px Arial; color: #000000; min-height: 12.0px} li.li1 {margin: 0.0px 0.0px 0.0px 0.0px; text-align: justify; font: 11.0px Arial; color: #000000} span.s1 {font: 11.0px Symbol} span.s2 {text-decoration: underline ; color: #0000ff} span.s3 {text-decoration: underline} span.s4 {font: 7.3px Arial} ul.ul1 {list-style-type: disc}
</style>In recent years, there is a surge of interest in Machine Learning (ML) and Artificial Intelligence (AI) solutions. This interest is driven by advances in parallel hardware, scalable software systems, advanced machine learning frameworks, as well as cloud/edge computing. Furthermore, the explosion of digital data collected makes it possible to develop more effective and accurate ML systems based on deep neural networks. This is because deep learning (DL) outperforms classical ML when large volumes of training data are available.&nbsp;State-of-the-art ML/DL systems span both cloud computing and edge computing infrastructures, and can have varying characteristics, such as ML model performance (e.g., classification accuracy), training speed, and energy efficiency. These characteristics determine whether a system can meet specific application requirements such as latency and environmental performance. In most cases it is not possible to optimize all these parameters simultaneously. Therefore ML/DL developers often have to determine the relevant trade-offs and optimize the performance according to their specific application needs.&nbsp;The choice of processors is key in setting the performance characteristics of a model. Larger, more power-hungry processors may appear to be the higher performing option, but specifically designed neural processors, or accelerators, often provide the better power/performance tradeoff. Complicating matters, deep learning systems are statistical and non-deterministic, which means that there is rarely a single optimal system configuration that yields the best performance and accuracy. Rather, it is possible to deploy alternative configurations of hardware and algorithms with comparable performance and efficiency.<h2>Introducing ML/DL and Neural Architectures Benchmarks</h2>The variety of available neural network architectures, configurations, and performance-related parameters makes it challenging to compare alternative ML/DL products and services. To facilitate this, engineers often turn to proven benchmarks for compute intense and data intense systems. For example:<ul><li>The <a href="http://www.spec.org/spec/faq/#01SPEC.General.10whatis" target="_blank">Standard Performance Evaluation Corporation (SPEC</a>) family of benchmarks focuses on computational workloads over different computing architectures including cloud-based systems.</li><li>The <a href="https://netlib.org/utk/people/JackDongarra/faq-linpack.html" target="_blank">LINPACK Benchmarks</a> provide measures of a computer&rsquo;s floating-point rate of execution i.e., they focus on floating-point computing power.&nbsp;</li><li>The <a href="https://www.tpc.org/" target="_blank">TPC consortium</a> includes industry leaders in computing systems and datasets, which have provided a suite of benchmarks for stress testing the transaction processing and data warehousing capabilities of data intensive systems.</li><li>The <a href="https://hpcg-benchmark.org/" target="_blank">High Performance Conjugate Gradients (HPCG)</a> benchmark provides novel metrics that enable the evaluation and ranking of High Performance Computing (HPC) systems.</li></ul>These benchmarks aid in evaluating different components of ML/DL systems, which are typically deployed over computationally intensive platforms. However, they are overly focused on conventional computational workloads and hence not sufficient for training and executing deep learning tasks that are typically more complex. &nbsp;To provide proper benchmarks for ML systems an end-to-end approach is needed. It must consider all the different steps that are involved in the development and execution of an ML/DL pipeline i.e., from the training of an accurate model to the execution of different implementation and computation options towards optimizing system performance (e.g., execution speed).&nbsp;Independent benchmarks provide a credible and impartial evaluation of alternative solutions, providing stakeholders with invaluable insights on the available market offerings. Moreover, they provide guidance for the co-design of ML/DL solutions by ML modelers, data scientists and embedded systems engineers.&nbsp;Specifically, the benchmarks provide these teams with tangible metrics that can help them evaluate alternative system designs for a variety of popular use cases. Furthermore, ML benchmarking suites enable market players to compete against the same set of metrics, fostering innovation and healthy competition.&nbsp;<h2>The MLPerf Benchmarking Suite</h2><a href="https://mlcommons.org/en/" target="_blank">MLPerf is one of the most popular benchmarking suites</a> for ML/DL systems. It is comprised of a rich set of system tests that span ML models, software, hardware, and the power efficiency characteristics of an ML/DL system. MLPerf was established in 2018 by several organizations including Google, Harvard University, and Stanford University. Since then, it has greatly boosted innovation in the ML space and has helped vendors deliver high-performance and energy efficient ML/DL systems.The suite includes several benchmarks for different ML tasks. Each benchmark comprises the following characteristics:<ul><li>The name of the task (e.g., Image Classification), which indicates the benchmarking context. For instance, the Image Classification task aims at selecting the class that best describes the content of an image.</li><li>The dataset, which is typically a popular dataset for the task at hand. For instance, the <a href="https://www.image-net.org/" target="_blank">ImageNet database</a> is commonly used for benchmarking Image Classification tasks.</li><li>The model to be used, which identifies the ML/DL model that will be used to gauge the computational performance of the ML hardware and software system. As an example, several variations of the Residual Network (ResNet) are commonly used to benchmark hardware performance in computer vision tasks like Image Classification.</li><li>The Quality Threshold, which denotes a level of acceptable quality for the systems that are benchmarked. For instance, a 74.9% classification accuracy can be used as the threshold for placing an ML/DL system in the Top-1 accuracy.</li></ul>Along with the above-listed parameters, MLPerf standardizes and specifies other aspects of the benchmarking process such as the initialization of parameters, the optimizer schedule to be used, and data augmentation processes. In this way, it ensures that the benchmarked systems are comparable and can be ranked in an unbiased way.&nbsp;The results of the MLPerf benchmarking process are published in a transparent way. This makes it easy for market stakeholders to understand how different systems perform in the various tasks and how they compare to each other. The publication of the results for each system includes information about the system&rsquo;s benchmark scores, its market availability, the models used (e.g., <a href="https://github.com/tensorflow/models/blob/master/research/slim/nets/mobilenet_v1.md" target="_blank">MobileNet-v1</a> for mobile phones and <a href="https://en.wikipedia.org/wiki/Residual_neural_network" target="_blank">ResNet-50</a> v1.5 for bigger accelerators), as well as the scenarios and configurations of the benchmarking process.&nbsp;The scenarios are destined to cover different deployment configurations, including inference at the edge and inference at cloud data centers. Moreover, there is information about the number of accelerators used by the systems under comparison, as the MLPerf benchmark is designed to rank entire systems and not individual components.In practice, the process of running the MLPerf benchmarking is quite straightforward. It starts with cloning the MLPerf repository and setting up the configuration parameters. Accordingly, the system under benchmarking is added to the list of available system and a proper Docker image for the tests is built. The latter enables data preprocessing, as well as execution of validation models and tests. &nbsp;<h2>Syntiant&rsquo;s Performance in the MLPerf Benchmarking</h2><a href="https://www.eetasia.com/syntiant-leads-tinyml-benchmark-results/" target="_blank">Syntiant is strong leader in tinyML keyword spotting</a> according to the respective MLPerf benchmarking process.&nbsp;&nbsp;The company&rsquo;s performance in the benchmark is summarized as follows:<ul><li>Achieved Performance: Syntiant&rsquo;s NDP120 ran the tinyML keyword spotting benchmark in 1.80 ms. This is a remarkably fast inference, making the company the clear winner in this category.&nbsp;</li><li>Energy Usage: The benchmarked system used 49.59 &mu;J of energy at 1.1V/100 MHz. Energy efficiency was further improved by setting supply voltage down to 0.9 V and reducing clock frequency to 30 MHz, resulting in only 35.29 &mu;J of energy being consumed. Of course, reducing the clock frequency increases the latency to 4.30 ms, but this highlights both the energy and latency floor of the device, leaving it to the system designer to optimize power and performnce based on their specific application needs.&nbsp;</li></ul>The NDP120 is based on Syntiant&rsquo;s second generation AI accelerator core, Syntiant Core 2&reg;, alongside a Tensilica HiFi DSP (Digital Signal Processor) for feature extraction and an Arm Cortex-M0 core for system management. Syntiant did not enter any other benchmark scores for the NDP120, as the chip was architected for time-series data inferences, such as speech and audio events.&nbsp;Syntiant Core 2&reg;, is a highly flexible, ultra-low-power deep neural network inference engine with a highly configurable audio front-end interface. Thanks to this front-end interface, NDP120 is very easily programmable. The ease of deployment is further aided by its native support for all major neural network architectures, allowing for the direct execution of neural network layers without intermediary processing steps.&nbsp;NDP120 can run multiple models, such as a keyword spotting model and an audio event detection model, concurrently and supports flexible feature extraction and acoustic far-field processing with the included DSP. Using multiple neural network layers, the system provides accurate high-performance inference and classification, which supports use cases in a variety of application. For instance, it can accurately capture voice commands (e.g., Alexa prompts) in very noise environments.The device is ideal for ultra-low-power, always-on, edge applications, where an inference based on time series audio or sensor data is required. The deployed solutions using the NDP120 span market &nbsp;verticals in consumer and industrial, such as security devices detecting acoustic events, mobile phones, tablets, smart speakers, smart displays, as well as smart home appliances and security devices. As with all Syntiant Core 2 family of solutions, it provides optimized memory access and at-memory compute, ensuring high efficiency and extremely low power consumption. The latter is clearly evident in the benchmarking of energy consumption across its peers.For more information about Syntiant NDP120 visit: <a href="https://www.syntiant.com/ndp120" rel="noopener noreferrer" target="_blank">https://www.syntiant.com/ndp120</a><h2 id="isPasted">About the sponsor: Syntiant</h2>Syntiant Corp. is a leader in delivering end-to-end deep learning solutions for always-on applications by combining purpose-built silicon with an edge-optimized data platform and training pipeline. The company&rsquo;s advanced chip solutions merge deep learning with semiconductor design to produce ultra-low-power, high-performance, deep neural network processors for edge AI applications across a wide range of consumer and industrial use cases, from earbuds to automobiles. Syntiant&rsquo;s Neural Decision Processors&trade; typically offer more than 100x efficiency improvement, while providing a greater than 10x increase in throughput over current low-power MCU-based solutions, and subsequently, enabling larger networks at significantly lower power.<img src="https://images.wevolver.com/eyJidWNrZXQiOiJ3ZXZvbHZlci1wcm9qZWN0LWltYWdlcyIsImtleSI6ImZyb2FsYS8xNjYzMzU5OTg2NjI5LVVudGl0bGVkIGRlc2lnbi02ICgxKS5wbmciLCJlZGl0cyI6eyJyZXNpemUiOnsid2lkdGgiOjk1MCwiZml0IjoiY292ZXIifX19" style="width: 300px;" class="fr-fic fr-dib">

Proven benchmarks provide a structured method for comparing ML/DL products and services. 


The Art and Science of Benchmarking Neural Network Architectures

Over the last decade, deep neural networks have emerged as the solution to several AI complex applications from speech recognition and object detection to autonomous vehicular systems. In essence, deep learning isn’t a single algorithm, but a family of algorithms with many layers that lack transparency, making it hard to understand the error, especially a downside for areas where these errors have a high cost. Most of the complexity in these deep neural networks comes from the input dataset which is used to make the training and inference faster and more efficient. Measuring input simplicity has been one of the lines of research that have caught the attention of AI architects to devise a framework that not only improves performance but also focuses on interpretability.&nbsp;The removal of certain parameters from the input data was an essential part of the research work to understand the correlation of a particular input region or frequency on the output performance. However, most of the existing work depended on random removal or exploiting the domain knowledge that made the inputs simpler on which network performance was analyzed. A team of researchers from the University Medical Center Freiburg and Google Brain [1] proposed a technique for the model to synthesize the inputs that retrain prediction-relevant information. Through this, without relying on prior knowledge, the model can achieve a state of intuition for simplifying the input. Some of the findings reported [2,3] that in density-based anomaly detection, the generative image models usually assign higher probability densities and lower bits for visually simpler inputs.<h2 dir="ltr">What does simplification of input mean?</h2>In the paper, “When less is more: Simplifying inputs aids neural network understanding,” the team designed SimpleBits, a framework that synthesizes the inputs to contain less information without affecting the output performance. The approach measures simplicity with the encoding bit size that is given to the pre-trained generative model and reduces the bit size to simplify the inputs. Additionally, the framework can be used to analyze the trade-off between the information content and the task performance. After training neural networks, SimpleBits can act as an analysis tool to understand what information was used by the model to make an optimized informed decision.&nbsp;<img src="https://s3-us-west-1.amazonaws.com/wevolver-project-images/froala%2F1645493691498-Overview+of+Simplification+of+Inputs+for+Deep+Neural+Networks.png" style="width: 800px;" class="fr-fic fr-dib" alt="Overview of Simplification of Inputs for Deep Neural Networks">Overview of Simplification of Inputs for Deep Neural Networks [Image Credit: Research Paper]<div style='pointer-events: none; overflow: hidden; padding-bottom: 2px; z-index: auto; position: absolute; left: 0px; top: 477.5px; height: 39px; width: 766px; font-style: normal; font-variant: normal; font-weight: 400; font-stretch: 100%; font-size: 14px; font-family: "PT Serif", serif; text-align: center; text-transform: none; letter-spacing: normal; word-spacing: 0px; tab-size: 8;'> </div> The evaluation of simplification of the inputs was performed in different settings: conventional training, dataset condensation and post-hoc analysis. The image above depicts the tasks covered in the paper, starting with SimpleBits acting as a per-instance simplifier during the training, where each image in the dataset is simplified but the total number of images remains unchanged. In simplifying the input, the framework removes superfluous information for tasks with injected distractors, while on a natural image dataset, the framework highlights task-relevant information. The chosen dataset contained both useful and redundant information for task learning, helping the researchers to evaluate the model accuracy over different datasets. By increasing the simplification process, the researchers noticed an accuracy drop reporting of the trade-off between input simplification and task performance for different setups.&nbsp;For dataset simplifications with condensation, the dataset is compressed to a few samples per class which are simplified at the same time. In this setting, the training data are converted into a smaller set of synthetic images which are further simplified to reduce the encoding size without affecting the task performance. Finally, for the post-training setting, SimpleBits can act as a post-hoc analysis tool used for auditing processes. “Our exploration of SimpleBits guided audits suggests it can provide intuition into model behavior in individual examples, including finding features that may contribute to misclassifications,” the team notes.&nbsp;<h2 dir="ltr">Summary of simplified inputs that aid neural network understanding&nbsp;</h2><ol><li dir="ltr">An information-based method to simplify the inputs while maintaining the output performance and neural network accuracy. This can be done without any prior knowledge or expertise.</li><li dir="ltr">The framework is able to remove injected distractors, retaining accuracy while reducing the complexity of the input dataset images.</li><li dir="ltr">It can be used as an analysis tool to monitor the trade-off between task performance and input simplifications to check if any essential information is not removed.&nbsp;</li></ol><h2 dir="ltr">References</h2>[1] Robin Tibor Schirrmeister, Rosanne Liu, Sara Hooker, Tonio Ball: When less is more: Simplifying inputs aids neural network understanding. DOI <a href="https://arxiv.org/abs/2201.05610" target="_blank">arXiv.2201.05610</a> [cs.LG][2] Kirichenko, P., Izmailov, P., and Wilson, A. G. Why normalizing flows fail to detect out-of-distribution data. In Larochelle, H., Ranzato, M., Hadsell, R., Balcan, M., and Lin, H. (eds.), Advances in Neural Information Processing Systems 33: Annual Conference on Neural Information Processing Systems 2020, NeurIPS 2020, December 6-12, 2020, virtual, 2020.&nbsp;[3] Schirrmeister, R., Zhou, Y., Ball, T., and Zhang, D. Understanding anomaly detection with deep invertible networks through hierarchies of distributions and features. In Larochelle, H., Ranzato, M., Hadsell, R., Balcan, M., and Lin, H. (eds.), Advances in Neural Information Processing Systems 33: Annual Conference on Neural Information Processing Systems 2020, NeurIPS 2020, December 6-12, 2020, virtual, 2020.&nbsp;

SimpleBits framework for Simplifying of Inputs for Deep Neural Networks

Over the last decade, deep neural networks have emerged as the solution to several AI complex applications from speech recognition and object detection to autonomous vehicular systems.

How to simplify the inputs for better neural network performance?

We have already covered a detailed article on <a href="https://www.wevolver.com/article/smart-face-mask-platform-in-response-to-covid-19-pandemic" target="_blank">the smart face mask platform</a> integrating software and hardware to offer a user-friendly platform– that displays physiological signals [1]. With Covid-19 rising in several countries, face masks have become a part of day-to-day life– prolonged use of these face masks has adverse effects, primarily caused due to CO2 rebreathing. There have been reports concluding a significant drop in blood oxygen saturation and increase in heart and respiratory rate due to the increased CO2 concentration in the dead space volume (DSV). DSV is defined as the volume of ventilated air in which there is no significant gas exchange of oxygen and carbon dioxide.&nbsp;The established mechanism to monitor the concentration or partial pressure of carbon dioxide in respiratory gases– capnography and other portable hardware to measure the gas inside the mask DSV have created complex issues for the required components and its bulky structure. Even though the proposed solutions are reliable with high resolution and accuracy, they are limited to laboratory experiments due to their size and power requirements. To solve the challenges in the space of wearability and conformability, utilizing the recent technological advancements and exploiting the developed research reports, a team of researchers, affiliated with the University of Granada, Spain, have proposed a system with wearable hardware that interacts with the custom smartphone application via a near-field communication link (NFC) [2].<h2 dir="ltr">NFC tag for Wireless CO2&nbsp;readout</h2>In the paper, “Smart facemask for wireless CO2 monitoring” the team has developed a system that consists of an opto-chemical sensor combined with a battery-free NFC tag with a resolution of 103 ppm for measuring the gas concentration in the DSV of the facemask. The sensor circuitry is fabricated in a tiny 60x45 mm2 sized polymeric substrate with the NFC link generated for the interaction with the custom-developed software application. The smartphone application is responsible for data processing, alert management, result displaying and sharing purposes. “Through performance tests during daily activity and exercise monitoring, we demonstrate its utility for non-invasive, wearable health assessment and its potential applicability for preclinical research and diagnostics,” the team explains in the research article.<img src="https://images.wevolver.com/eyJidWNrZXQiOiJ3ZXZvbHZlci1wcm9qZWN0LWltYWdlcyIsImtleSI6ImZyb2FsYS8xNjQ1MzM2MjExNDIyLUNpcmN1aXQgYmxvY2sgZGlhZ3JhbSBvZiB0aGUgTkZDIHRhZy5wbmciLCJlZGl0cyI6eyJyZXNpemUiOnsid2lkdGgiOjk1MCwiZml0IjoiY292ZXIifX19" style="width: 800px;" class="fr-fic fr-dib" alt="Circuit block diagram of the NFC tag">Circuit block diagram of the NFC tag [Image Credit: Research Paper]<div style='pointer-events: none; overflow: hidden; padding-bottom: 2px; z-index: auto; position: absolute; left: 0px; top: 477.5px; height: 39px; width: 766px; font-style: normal; font-variant: normal; font-weight: 400; font-stretch: 100%; font-size: 14px; font-family: "PT Serif", serif; text-align: center; text-transform: none; letter-spacing: normal; word-spacing: 0px; tab-size: 8;'> </div>While designing the portable hardware, the general design criteria were based on the IoT design flow– sensing, connectivity, compactness, integration, scalability, low-power consumption and affordability. Keeping all this into consideration, the proposed system with an NFC tag printed on a transparent flexible substrate is fixed on the inner part of the FFP2-type facemask. The goal is to read out the CO2 concentration through the deposited chemical sensor with the fluorescent response of which results will be wirelessly transferred onto the software applications. Based on stable inorganic phosphor, the opto-chemical CO2 sensor is the most essential hardware component in the whole hardware circuitry. Being an NFC-based system, the monitoring is completely passive with no requirement of battery– whereas the power supply is obtained from the near electromagnetic field induced by the external NFC reader (in this case the smartphone).&nbsp;In the process for CO2 monitoring, the NFC tag has an attached ultraviolet light-emitting diode, which when powered on, the sensing substrate is active. The luminescence intensity is measured to determine the CO2 concentration through the red coordinate of the RGB color space. The incoming luminescence intensity is collected by the digital color detector then sent to the microcontroller unit. The final results will be sent to the mobile application for further processing.&nbsp;<h2 dir="ltr">Did performance prove its utility?</h2>For evaluation purposes, the smart facemask was tested on short-term and long-term usage in static conditions. For the initial test case, the CO2 concentration was monitored every 30 seconds through the proposed smart face mask as well as a reference instrument such as MultiRAE Lite, RAE Systems. By comparing the results, the curves show to be in good agreement with close results. In the early stage of the monitoring, the face smart curve is slightly in a delayed fashion in comparison to the reference standard– this is because of the time required to fill the mask’s DSV. Three apnoeas can be seen when the holding of breath during the test was performed to see the varying results and accuracy.<img src="https://images.wevolver.com/eyJidWNrZXQiOiJ3ZXZvbHZlci1wcm9qZWN0LWltYWdlcyIsImtleSI6ImZyb2FsYS8xNjQ1MzM2NDI0MDA0LVNtYXJ0IGZhY2VtYXNrIHBlcmZvcm1hbmNlLnBuZyIsImVkaXRzIjp7InJlc2l6ZSI6eyJ3aWR0aCI6OTUwLCJmaXQiOiJjb3ZlciJ9fX0=" style="width: 800px;" class="fr-fic fr-dib" alt="Smart Face Mask Performance">Smart Face Mask Performance: Short Term (left) and Long Term (right) [Image Credit: Research Paper]<div style='pointer-events: none; overflow: hidden; padding-bottom: 2px; z-index: auto; position: absolute; left: 0px; top: 286.5px; height: 39px; width: 766px; font-style: normal; font-variant: normal; font-weight: 400; font-stretch: 100%; font-size: 14px; font-family: "PT Serif", serif; text-align: center; text-transform: none; letter-spacing: normal; word-spacing: 0px; tab-size: 8;'> </div>For the long-term test, the face mask was worn for over 2.5 hours and the measurements were taken every 15 minutes. The monitoring is performed with temperature values into consideration to check the stability of the opto-chemical sensor. Two apnoeas at the stamp time of 80 and 110 minutes show a sharp drop in CO2 concentration. Overall, the CO2 accumulation for common conditions ranged from 1.5% to up to 3.5% which is well beyond the atmospheric air of approximately 0.04%. “Our results are in line with the reviewed literature, with CO2 values around 2% during low work-rate activities and peak values up to ~5% during high-intensity exercise,” the team notes.&nbsp;In summary, the developed platform for CO2 monitoring with batteryless hardware structure enabling NFC-based wireless communication and scalable hardware fitted inside the FFP2 face mask makes the system versatile and robust for real-life deployment. For future work, the researchers expect to combine several optical chemical sensors that can be integrated into a single flexible passive NFC tag to monitor several gases' concentrations.&nbsp;<h2 dir="ltr">References</h2>[1] Alexander Curtiss, Blaine Rothrock, Abu Bakar, Nivedita Arora, Jason Huang, Zachary Englhardt, Aaron-Patrick Empedrado, Chixiang Wang, Saad Ahmed, Yang Zhang, Nabil Alshurafa, and Josiah Hester. 2022. FaceBit: Smart Face Masks Platform. Proc. ACM Interact. Mob. Wearable Ubiquitous Technol. 5, 4, Article 151 (Dec 2021), 44 pages. DOI: <a href="https://doi.org/10.1145/3494991" target="_blank">https://doi.org/10.1145/3494991</a>[2] P. Escobedo, M. D. Fernández-Ramos, N. López-Ruiz, O. Moyano-Rodríguez, A. Martínez-Olmos, I. M. Pérez de Vargas-Sansalvador, M. A. Carvajal, L. F. Capitán-Vallvey, and A. J. Palma. Smart facemask for wireless CO2 monitoring. Nature Communication 13, Article 72 (Jan 2022). DOI: <a href="https://doi.org/10.1038/s41467-021-27733-3" target="_blank">https://doi.org/10.1038/s41467-021-27733-3</a>&nbsp;

Smart Face Mask for CO2 monitoring through an opto-chemical sensor; results displayed on Smartphone application

A system with an opto-chemical sensor combined with a battery-free NFC tag of resolution 103 ppm for measuring the CO2 gas concentration in the dead space volume of the facemask

Wearable and Intelligent Face Mask for CO2 monitoring

Mutations in the virus have given birth to multiple research streams for the treatment of viral infection. COVID-19 caused because of the SARS-CoV-2 virus, has changed the daily routine of human life– making it more conscious to protect itself from the global pandemic. Most of the day-to-day safety protocols in place are in the form of passive devices; limit the scope of improvement and cannot alert the user for potential risk. Wearing masks has been the most used protection against the SARS-CoV-2 virus infection– but lacks the technology to signal the user for mask degradation of improper fit. The innovation towards building smart devices that are scalable, lightweight and energy-efficient has been on the rise after realizing the need for active devices.In the space of the maker community, there have been efforts in designing hardware platforms that can extract physiological signals from facial sensing points. Smart glasses [2] for measuring the heart rate to smart filter [3] for PPE to monitor the air filter quality in the detection of COVID-19 have played a significant role in wearable physiological sensing. Most of these solutions lacked scalability and on-device computing capabilities– in response to the COVID-19 spread. All these resource-constrained computing platforms and software/hardware ecosystem helped the researchers to develop an efficient and user-friendly smart device.A team of researchers from Northwestern University, Georgia Institute of Technology, and the University of California at Los Angeles have proposed an open-source hardware platform for smart face mask applications. The increasing new highs of COVID-19 positive cases due to new infectious variants, and analyzing the slow global vaccination rate along– make this smart hardware platform a way to keep monitoring health parameters. Thanks to edge computing and AI inference advancements, the smart face masks platform creates a new research area for developing sustainable and convenient forms of health management, even for the COVID-19 pandemic.<h2 dir="ltr">What is FaceBit?&nbsp;</h2>In the paper, “FaceBit: Smart Face Masks Platform”, the team proposes the design of a hardware and software ecosystem consisting of FaceBit circuit board along with a custom-built mobile application that can provide data collection, visualization and alert. Along with alertness, the hardware interacts with physiological changes– monitoring heart rate (through FaceBeat algorithm) without skin contact via ballistocardiography and respiration rate through temperature changes. Additionally, the hardware also checks the mask fitting and wear time from pressure signals; leveraging the always on-device with an energy-efficient system. All the data collected from the onboard sensors are wirelessly transmitted to the custom phone application for analysis.&nbsp;Scaling such a product involves several factors that are to be considered– FaceBit aims to be low cost, energy-efficient and easy-to-integrate hardware with rich sensing functionalities. The hardware offers harvesting energy from breathing, motion and sunlight to supplement the tiny primary cell battery that alone delivers a battery lifetime of 11 days or more. With the help of a small magnet, the tiny form factor device fits on top of your existing masks without any modification. &nbsp;Moreover, the inclusion of a flexible interface has made it more convenient for the users to monitor the health parameters and keep track of the COVID-19 infection risk.<img src="https://images.wevolver.com/eyJidWNrZXQiOiJ3ZXZvbHZlci1wcm9qZWN0LWltYWdlcyIsImtleSI6ImZyb2FsYS8xNjQ0MzExMDA5ODg2LU92ZXJ2aWV3IG9mIEZhY2VCaXQucG5nIiwiZWRpdHMiOnsicmVzaXplIjp7IndpZHRoIjo5NTAsImZpdCI6ImNvdmVyIn19fQ==" style="width: 800px;" class="fr-fic fr-dib" alt="Overview of Facebit Smart Face Mask Platform">Overview of Facebit Smart Face Mask Platform [Image Credit: Research Paper]<div style='pointer-events: none; overflow: hidden; padding-bottom: 2px; z-index: auto; position: absolute; left: 0px; top: 253.1px; height: 39px; width: 766px; font-style: normal; font-variant: normal; font-weight: 400; font-stretch: 100%; font-size: 14px; font-family: "PT Serif", serif; text-align: center; text-transform: none; letter-spacing: normal; word-spacing: 0px; tab-size: 8;'> </div>The FaceBit block diagram shows the essential parts that make up the whole ecosystem consisting of three energy sources– solar, thermal and battery, while three wireless sensors (temperature sensor, IMU and pressure sensor) for monitoring physiological signals. All the incoming data from the remote sensors is then digitally processed and categorized under respiratory rate, heart rate and mask fitting. Wirelessly sent to user-controlled mobile applications to complete the design flow of the hardware/software smart mask platform.On the other side of the hardware platform is the FaceBit Companion application for phones and desktops that interacts with the FaceBit circuit board via Bluetooth Low Energy using a custom GATT profile. Built on iOS and macOS using Swift Programming Language, the user interface is written within the SwiftUI framework and uses Mac Catalyst that supports iOS applications deployed on macOS. “The application handles both high-frequency time-series data streams for debugging, as well as low frequency computed metrics from the sensor board for actual deployment,” the researchers explain. “The vision is that a user will be able to customize the home screen with desired widgets to receive timely information on the signals of interest.”<img src="https://images.wevolver.com/eyJidWNrZXQiOiJ3ZXZvbHZlci1wcm9qZWN0LWltYWdlcyIsImtleSI6ImZyb2FsYS8xNjQ0MzExMTIwNTM0LUZhY2VCaXQgUGhvbmUgQXBwbGljYXRpb24ucG5nIiwiZWRpdHMiOnsicmVzaXplIjp7IndpZHRoIjo5NTAsImZpdCI6ImNvdmVyIn19fQ==" style="width: 800px;" class="fr-fic fr-dib" alt="FaceBit Phone Application">FaceBit Phone Application [Image Credit: Research Paper]<div style='pointer-events: none; overflow: hidden; padding-bottom: 2px; z-index: auto; position: absolute; left: 0px; top: 467.15px; height: 39px; width: 766px; font-style: normal; font-variant: normal; font-weight: 400; font-stretch: 100%; font-size: 14px; font-family: "PT Serif", serif; text-align: center; text-transform: none; letter-spacing: normal; word-spacing: 0px; tab-size: 8;'> </div>For future studies, the developers aim to optimize the signal pipeline to tune the FaceBeat algorithm to reduce the number of falsely detected heart rates. A battery-free system is something that can improve the sustainability of the device by integrating low-power modules like Ambiq Apollo3. As an open-source contribution, the innovators encourage the community to modify the hardware and software to bring more functionalities– available at <a href="https://facebit.health/" target="_blank">facebit.health</a>.&nbsp;<h2 dir="ltr">Reference</h2>[1] Alexander Curtiss, Blaine Rothrock, Abu Bakar, Nivedita Arora, Jason Huang, Zachary Englhardt, Aaron-Patrick Empedrado, Chixiang Wang, Saad Ahmed, Yang Zhang, Nabil Alshurafa, and Josiah Hester. 2022. FaceBit: Smart Face Masks Platform. Proc. ACM Interact. Mob. Wearable Ubiquitous Technol. 5, 4, Article 151 (Dec 2021), 44 pages. DOI: <a href="https://doi.org/10.1145/3494991" target="_blank">https://doi.org/10.1145/3494991</a>[2] Hernandez, Javier et al. "Wearable Motion-Based Heart Rate at Rest: A Workplace Evaluation." IEEE Journal of Biomedical and Health Informatics 23, 5 (September 2019): 1920 - 1927[3] Lut, J.-J. &amp; Goddijn, M. Smart Personal Protective Equipment: Sensing and Control: The Future of Face Masks (Delft University of Technology, Delft, 2020).

FaceBit aims to be low-cost, energy-efficient, and easy-to-integrate hardware with rich sensing functionalities– monitors physiological signals

Smart Face Mask Platform in Response to COVID-19 Pandemic

The <a href="https://www.linleygroup.com/events/agenda.php?num=52&day=1" target="_blank">Linley Processor Conference</a> is an annual event providing an overview of recent achievements and forecasting trends and challenges. The Linley Group hosted its Fall Processor Conference in person on October 20-21. &nbsp;Its virtual event featured live-streamed presentations on October 27-29 and November 3-5. The full conference can be accessed on-demand&nbsp;<a href="https://gateway.on24.com/wcc/eh/3377544/linley-fall-processor-conference-2021" rel="noopener noreferrer" target="_blank">here</a>.The recent conference featured a full program of talks and panel discussions covering a broad range of topics such as:<ul><li dir="ltr">High-Performance Processors</li><li dir="ltr">Server Acceleration</li><li dir="ltr">SoC Design</li><li dir="ltr">Edge-AI Processing &amp; Software</li><li dir="ltr">Edge IP</li><li dir="ltr">Low-Power AI</li><li dir="ltr">DSP &amp; IoT</li><li dir="ltr">FPGAs for AI</li></ul>In this article, we summarize a few of the featured keynote presentations.&nbsp;<h2>AI Moves from the cloud to the edge</h2>The conference kicked off with a keynote from Linley Gwennap, principal analyst for The Linley Group. He spoke about the latest trends in AI silicon as the technology moves from the cloud to the edge. AI has firmly evolved from a product to a feature, with AI acceleration appearing in server processors, PC processors, FPGAs, and a vast array of edge devices. These chips are making an impact across the sector. In the context of data centers, new chips focus on improving performance per watt. Architecture innovation is particularly rampant at the edge, where power limitations are stringent, and software stacks are smaller. Edge chips target a range of applications from battery-powered sensors to powerful camera-based systems. So-called ‘Conscious cameras’ use edge processing to make decisions based on object detection and facial recognition. These cameras can learn what they need to be seeing and send triggers when something is unrecognized or deemed a threat. Applications range from smart home monitoring to industrial quality control.&nbsp;<img src="https://s3-us-west-1.amazonaws.com/wevolver-project-images/froala%2F1636049063870-Triton-Edge-SoC-all-sides1.png" style="width: 788px;" class="fr-fic fr-dib">Image 1: Edge processing in cameras will have a massive impact on security and industry. Image credit: Triton. &nbsp;<h2>Core Edge AI Processors: From In-Memory to At-Memory</h2><a href="https://www.wevolver.com/profile/syntiant" target="_blank">Syntiant</a> co-founder and chief technology officer Stephen Bailey, Ph.D., presented a talk as part of the conference’s Low-Power Sensing and AI” session. Dr. Bailey demonstrated how Syntiant’s custom at-memory, deep learning processor architecture provides the required cost, performance, and efficiency necessary for deploying ultra-low-power, always-on applications.&nbsp;“Syntiant was founded to deliver edge AI solutions with chips based on in-memory, analog deep-learning processing technology,” Bailey explains. &nbsp;“We discovered early in our journey that demand for these solutions was more immediate and diverse than anticipated, and that a custom at-memory, digital deep-learning processor architecture would provide the required cost, performance, and efficiency while providing chip-process portability.” His lecture described the Syntiant journey so far and shared architectural insights and performance comparisons of the Syntiant Core 2 processor found in their Neural Decision Processor (NDP) chips.&nbsp;<img data-fr-image-pasted="true" src="https://lh5.googleusercontent.com/eJSpociG9qyBCOguEoEmTux_dQ7JYI4zCrMGKKxBIsVICZoUXaabh2UffgRws4Eij7xuiTqXlMZNdJTqAtNxY0k6mdY95Iuh2MZM2t8AmDAWDLP2C16k6aPp3j_QyhhHiN8-pOa2" width="602" class="fr-fic fr-dii" style="width: 756px; height: 707.104px;">Image 2: The tiny and powerful Syntiant chips. Image credit: Syntiant.&nbsp;<h2 dir="ltr">The SystemReady Standards Program: Enabling Systems Where Software ‘Just Works’</h2><a href="https://www.arm.com/#:~:text=Arm%20is%20the%20leading%20technology,%2C%20Neoverse%2C%20Ethos%20and%20SecurCore." target="_blank">Arm</a> aims to bring together software and hardware to accelerate IoT design and fuel a new IoT economy. Arm enables specialized processing built on the economics, design freedom, and accessibility of general-purpose compute. Arm lead standards architect and fellow, Dong Wei presented a talk titled “SystemReady Standards Program: Enabling Systems Where Software ‘Just Works’.” The lecture outlines the standards-based approach of Arm System Ready, which provides requirements that can be applied to a broad range of devices from cloud data centers to edge and IoT. It reduces or eliminates the cost of customization for the system firmware and kernel build.&nbsp;<h2>Achieving Real-Time High-Resolution Video Inference at Ultra-Low Power</h2><a href="https://roviero.com/" rel="noopener noreferrer" target="_blank">Roviero</a> is a deep-tech startup founded in early 2019 that has developed the CortiOne™ NPE (Neural Processing Engine) and integrated CortiSoft™ compiler technology. The company has a vision to enable AI in all edge devices through a hybrid hardware-software design. Despite massive leaps, AI implementation in edge devices is still very limited due to the power consumed by the compute and associated memory.&nbsp;Models with good accuracy are still large in size and contribute to high memory and power consumption. &nbsp;Ravi Setty, SVP of Engineering and Cofounder of Roviero, discussed their solution to this problem; Roviero’s CortiOne NPE and compiler. The CortiSoft compiler optimizes memory access such that the model memory is in a sleep state 99% of the time, thereby dramatically reducing power consumption while maintaining full accuracy. Watch a demo of the compiler below.&nbsp;<iframe src="https://www.youtube.com/embed/8QOA_3T_JjY?&amp;wmode=opaque&amp;rel=0" frameborder="0" allowfullscreen="" class="fr-draggable" style="width: 740px; height: 401px;"></iframe> <h2>Summary</h2>The Linley Fall Processor Conference 2021 provided a robust look at the current state of processors. The significant focus on edge processing and AI demonstrates that the next few years will see significant growth and change in this area. The full conference is available to watch on-demand <a href="https://gateway.on24.com/wcc/eh/3377544/linley-fall-processor-conference-2021" rel="noopener noreferrer" target="_blank">here.</a><h2>About the sponsor: &nbsp;The Linley Group</h2>The Linley Group is the industry's leading source for independent technology analysis of semiconductors for a broad range of applications including AI, networking, communications, data center, and embedded. The company provides strategic consulting services, in-depth analytical reports, and conferences focused on advanced technologies for chip and system design.&nbsp;<img src="https://s3-us-west-1.amazonaws.com/wevolver-project-images/froala%2F1636051457879-linley+dl.jpg" style="width: 300px;" class="fr-fic fr-dib">

A rich line-up of speakers discuss the trends and challenges of the industry. 


From AI and beyond: Highlights of the Linley Fall Processor Conference 2021

Utilizing audio edge processors for efficient execution of  Machine Learning Tasks.

An engineer's guide to deploying machine learning in smart devices using audio edge processors

A closer look at the recent market trends and audio technologies that enhance the functionalities of TWS devices.

TWS Evolution: What is next in True Wireless Audio?

Wevolver and&nbsp;<a href="https://www.wevolver.com/profile/syntiant" target="_blank">Syntiant</a> are creating a series that explore the work of innovators and the future of pervasive AI. Syntiant is developing ultra-low-power AI processors. Because they believe in the importance of innovation, Syntiant is engaging in these fireside chats with engineers and designers who are on the cutting edge of their field.The fourth conversation in our Voice Of Innovation series was held between Syntiant CEO Kurt Busch and Andreas Urschitz, Division President Power &amp; Sensor Systems at Infineon. Infineon is one of the largest semiconductor companies in the world, they have partnered with Syntiant for many years with the shared goal of innovating artificial intelligence at the edge. In this wide-ranging conversation, Kurt and Andreas discuss the potential of deep learning, strategies to improve efficiency, and the importance of system innovation.&nbsp;<h1 dir="ltr">Building the five senses into semiconductors</h1><blockquote>“Infineon is pursuing its vision of emulating the five human senses, eventually even the sixth human sense, into each and every kind of electronics.” - Andreas Urschitz.</blockquote>To make electronics understand in a meaningful way and without latency, such as understanding voice through a microphone and interpreting a specific command or making electronics understand a particular gas combination in the environment or make equipment smell, requires artificial intelligence at the edge through machine learning.&nbsp;We have always tried to move the real world into the visual into the virtual world in some way, says Kurt, and with the advent of deep learning, we were able to do things very very quickly. For example, in a very short period of time speech recognition was developed, and within a couple of years, devices such as Amazon Echoes have appeared. This is the sort of thing that deep learning can bring to the table - both rapid innovation and entirely new types of interfaces.&nbsp;<h1 dir="ltr">Prioritizing sustainability&nbsp;</h1>With processing at the edge, no matter how good a device is there is going to be a demand for more, continues Kurt. What we are creating is a never-ending cycle of increased intelligence inside these devices and it's actually truly exciting because a solution that you have today that works well, you know that next year you are going to have 10 times the performance and that will generate seamless user interfaces.&nbsp;Of course, this increase in compute has power requirements. According to Kurt and Andreas, both Syntiant and Infineon are dedicated to prioritizing ways to increase the efficiency of their solutions in order to strive for better environmental credentials and clear sustainability. Syntiant has ramped up its research and investment into power flow solutions, but just energy saving is not enough. What’s crucial is the data that is collected through the millions of sensors we are now surrounded by is analyzed and pre-filtered so that the actual amount of data that needs to be managed in the data center is reduced to the bare minimum. The solution to this is edge processing.<h1 dir="ltr">The importance of system innovation</h1>Digitalization is changing so many things in our environment, says Andreas. One of the things we have observed over the last couple of years is that formerly we are confronted with a world with so much information and technology available but often quite spread out over the planet.&nbsp;Andreas: “We must ask ourselves how could you combine the different technologies - &nbsp;software, &nbsp;hardware - to create meaningful use cases that are helping humanity? We see systemic proceedings such as bringing partners together who are owners of different bits and pieces of technologies to come up with things that either solve customers’ problems or create opportunities as an essential way to make meaning of these technological advances. I believe there are a lot of chances for different companies to team up jointly to strive for system innovation to truly make an impact that goes beyond what is possible alone.”&nbsp;<hr>This video is part of a series of fireside chats in which Wevolver and Syntiant partner to engage with global innovators. In <a href="https://www.wevolver.com/tag/fireside-chat" rel="noopener noreferrer" target="_blank">previous discussions </a>we spoke with Arduino CEO Fabio Violante, Star Wars Animatronic Designer Gustav Hoegen and Fashiontech Designer Anouk Wipprecht. &nbsp;I encourage you to stay tuned and to follow <a href="https://www.wevolver.com/profile/syntiant" target="_blank">Syntiant's profile</a> for more of these conversations.

#4 of our 'Voice of Innovation' fireside chat series, in which Syntiant CEO Kurt Busch talks with Andreas Urschitz Division President Power & Sensor Systems at Infineon.

Fireside Chat With Andreas Urschitz from Infineon

This article was co-authored by Raj Senguttuvan and Vikram Shrivastava.&nbsp;&nbsp;<h1 dir="ltr">Introduction</h1>The last decade, but particularly 2020, saw a massive increase in video calling and engagement with voice-enabled devices primarily through the work-from-home trend created by the pandemic. However, our use of video calls isn&rsquo;t restricted to just work Zoom meetings. We are now using video calling for teaching, exercise classes, to experience live music, as a tool for interacting at conferences, and <a href="https://www.eatwith.com/" target="_blank">even to go to restaurants.</a> This virtual engagement occurs via laptops, smartphones, tablets, and other IoT devices such as Amazon Echo Show, Facebook Portal, Peloton, <a href="https://imp.i277339.net/c/1141873/823006/11505?u=https%3A%2F%2Ftempo.fit%2Fpdp%2Ftempo-studio&subId2=[]st[p]cjv2u8o050017o5yeyn1o93c5[i]H2ac18[t]w[r]google.com[d]D[z]m" target="_blank">Tempo Studio</a>, etc. We have also embraced the use of voice-controlled devices such as home assistants and wireless earphones, and other smart home applications.&nbsp;The core point that limits enjoyable and engaging interactive audio, video call, or home assistant experiences is consistent sound quality in the presence of noise and other distractors. The ability of your device to intelligently manage sound is what makes or breaks your ability to communicate.&nbsp;<h2 dir="ltr">Increasing audio expectations</h2>Dubbed &lsquo;smart sound&rsquo; by product makers, audio intelligence is the ability of a device to process sound to deliver the best user experience. As the increase in voice-led device use for communication, entertainment, and health management increases, so does the demand for a seamless, low-barrier experience with increased functionality. Users now expect devices to understand more than simple wake or keywords (such as Alexa) and seek the ability to move between devices and applications with superior sound quality to enable an immersive and seamless experience, whether for professional conferencing or personal entertainment.&nbsp;Devices should be able to combine your voice and/or voice command with your personal preferences and environmental data to adapt sound processing to your specific context. This is known as contextual awareness.&nbsp;<h1 dir="ltr">What is Contextual Awareness?</h1>Contextual-aware devices combine user-specific information, such as location, preferences, and ambient sensor data, to better understand what the user is asking for and more accurately execute functions in response to a certain command or trigger.&nbsp;Always-listening devices use signal processing technology combined with <a data-lf-fd-inspected-kn9eq4roawkarlvp="true" href="https://mediatum.ub.tum.de/doc/1463108/document.pdf" rel="noopener noreferrer" target="_blank">machine learning</a> to differentiate between sound types such as natural sounds, voices, background interference, etc. These sounds are often divided into &ldquo;scenes&#39;&#39; and &ldquo;events&rdquo;. A scene is the users setting such as a noisy airport terminal or a quiet working space. The events are someone speaking, a glass breaking, or a dog barking. A context-aware device can process these groups of sounds to ensure the intent of the action, whether it is a video call or a voice command, and is processed for optimal experience.&nbsp; <div class="fr-img-space-wrap"><img src="https://images.wevolver.com/eyJidWNrZXQiOiJ3ZXZvbHZlci1wcm9qZWN0LWltYWdlcyIsImtleSI6ImZyb2FsYS8xNjE0NzkxMjg1ODE2LUlNQUdFIDQ2LTAxLmpwZyIsImVkaXRzIjp7InJlc2l6ZSI6eyJ3aWR0aCI6OTUwLCJmaXQiOiJjb3ZlciJ9fX0=" style="width: 788px;" class="fr-fic fr-dib">Figure 1: Context-aware system schematic.&nbsp;Dedicated audio edge processors with a focus on audio fidelity and with machine learning optimized cores are the key to contextually aware and high-quality audio communication devices. These processors can deliver enough compute power to process complex audio using just a small percentage of the energy of a generic Digital Signal Processing (DSP) implementation. These audio edge processors use ML algorithms for superior sound processing in addition to providing a &nbsp;more contextual audio experience.&nbsp;</div> <h2 dir="ltr">&ldquo;I think you are on mute&rdquo;</h2>Video conference calls have been a focal point in 2020 and continue to dominate work and social lives in 2021. The gaps in technology are revealed quickly when devices, people, and environments fail to keep up with the demands of the calls - to the point where the phrase &lsquo;I think you&#39;re on mute&rsquo; was dubbed the<a href="https://www.humanresourcesonline.net/the-most-used-business-phrases-of-2020-i-think-you-re-on-mute" rel="noopener noreferrer" target="_blank">&nbsp;most used business term</a> in 2020. Dedicated audio processors with ML is the necessary approach to solve some of the common video conferencing pain points.&nbsp;ML on dedicated audio processors are already addressing some common video conference pain points:<ul><li dir="ltr">Selective noise canceling. By understanding a range of scenes and action noises, certain background elements such as a fan, heater, or street noise can be removed, thereby, improving overall call quality and the ability to work in a wider variety of environments.&nbsp;</li><li dir="ltr">Hi-fi audio reconstruction. Analogous to the way budget smartphones use smart software to increase the image quality from low-quality cameras, ML in real-time audio processing can improve the audio quality from low-quality microphones.&nbsp;</li><li dir="ltr">Spatial audio for improved user experience. Already available on advanced gaming headphones and some TWS devices, advanced digital sound optimization techniques provide users a spatial audio experience. Spatial audio can create a natural listening experience for the user, which can help mitigate &ldquo;<a href="https://news.stanford.edu/2021/02/23/four-causes-zoom-fatigue-solutions/" target="_blank">Zoom Fatigue</a>&rdquo;.</li></ul>In the future, ML on audio processors will have even more advanced capabilities such as real-time speech processing that would enable automatic language translations. Additionally, ML can solve the &lsquo;on mute&rsquo; problem by becoming more aware of the user&rsquo;s context and offer prompts to unmute.<h1 dir="ltr">A look at specialized audio edge processors</h1>There are several areas audio edge processors get right to deliver exceptional audio and voice command experiences.&nbsp;<h2 dir="ltr">Noise and Distance</h2>Beamforming is the process where the device can engage its ability to focus the listening or output sound in a particular direction. Noise suppression makes conversations in everyday scenarios possible. A listening device determines the direction(s) of the voice source(s) and that of noise sources. ML classification techniques are then used to determine which beams have voice or noise in them. &nbsp;The DSP then focuses on beams with voice content only for further voice UI processing. For example, in a conferencing system, the device must identify the direction of arrival of sound, and multiple speakers must be tracked at all times in a 360-degree fashion. Noise sources can also be classified further for audio event detection like glass break, fire alarms, and gunshots, etc for further expanding the role into audio aware smart home systems.&nbsp;Proximity detection is also essential for dynamic listening and speaking experience. &nbsp;The device detects user proximity to the microphone and adjusts the microphone&rsquo;s gain. This feature enables active video conferencing for presentations, workouts, and learning environments. These features are central to the design of advanced video conferencing devices such as the new Amazon Show which features a screen that rotates to where it senses the user.&nbsp;<iframe src="https://www.youtube.com/embed/fxUSSgrHL-o?&wmode=opaque&rel=0&enablejsapi=1&origin=https://www.wevolver.com" frameborder="0" allowfullscreen="" class="fr-draggable" style="width: 789px; height: 504px;" data-gtm-yt-inspected-8="true" data-lf-form-tracking-inspected-kn9eq4roawkarlvp="true" data-lf-yt-playback-inspected-kn9eq4roawkarlvp="true" data-lf-vimeo-playback-inspected-kn9eq4roawkarlvp="true"></iframe><h2 dir="ltr">Latency</h2>Humans can <a href="https://towardsdatascience.com/real-time-noise-suppression-using-deep-learning-38719819e051" rel="noopener noreferrer" target="_blank">generally tolerate</a> up to 200ms of end-to-end latency on a call before we begin to talk over each other. Low latency processing in edge processors is, therefore, a critical requirement for ensuring high-quality voice communication.<h2 dir="ltr">Power consumption</h2>Audio edge processors designed with proprietary architectures, hardware accelerators, and special instruction sets can optimally run audio and ML algorithms. These optimizations help with reducing power consumption in audio-heavy use cases such as video conferencing.<h2 dir="ltr">Integration</h2>Audio edge processors that open up their architectures and development environments accelerate innovation by providing audio application developers the tools and support to create new devices and applications. Future audio devices will be a collaborative effort.<h2 dir="ltr">Security</h2>Edge processing can minimize the need for cloud processing providing many benefits, including increased data security. For example, most consumers are not comfortable with data from personal smart home devices being transferred continuously to the cloud for processing. These fears have proven true with several <a href="https://www.theguardian.com/technology/2020/dec/23/amazon-ring-camera-hack-lawsuit-threats" target="_blank">major breaches</a> by leading device manufacturers in the last years. Peace of mind can be achieved when personal data is processed for analysis or inference at the device. An excellent example of this is a smart home security device that has been trained to hear certain event sounds, such as glass breaking, that act as a trigger to alert the homeowner. As the processing of the sound and the alert occurs on the device, it does not need to be in continuous connection to the cloud, which enhances the security of the system.<h1 dir="ltr">Delivering Audio Intelligence at the Edge</h1><a href="https://www.knowles.com/products/audio-processors" target="_blank">Knowles AISonic&trade; IA8201: Dual Core</a> is an audio edge processor specifically designed for advanced audio and machine learning applications, enabling power-efficient intelligence and privacy at the edge. It offers robust voice activation and multi-microphone audio processing with sufficient compute power to perform advanced voice processing and audio output, context awareness, and gesture control for today&rsquo;s most advanced consumer electronics. Processing voice and audio voice with an audio edge processor like IA8201 can keep the power-hungry host processor off as long as possible to extend battery life. &nbsp;In addition, it also enables new use cases for enhanced user experience by having the capability to run ML algorithms on multiple sensor inputs to deliver a more natural and contextual user experience.Audio edge processors like IA8201 have a place in a variety of audio and voice-enabled applications including smart speakers, video conferencing devices, tablets, Bluetooth headphones, and smart home devices.Recommended reading:&nbsp;<a href="https://www.wevolver.com/article/enhancing-the-utility-of-audio-ai-in-everyday-devices-using-audio-source-classification" rel="noopener noreferrer" target="_blank">Enhancing the utility of Audio AI in everyday devices using Audio Source Classification</a><h1 dir="ltr">Conclusion</h1>Dedicated audio edge processors will define the next generation of audio and voice-enabled devices creating more context-aware, immersive, and seamless audio communication experiences. Their ability to enable efficient processing for low power and low latency voice communications, noise reduction, context awareness, and accelerated ML inferencing of sensor inputs opens the possibilities to an explosion of new user experiences for the human-machine interfaces.&nbsp;<hr><h1>References and further reading:<a href="https://blog.adobe.com/en/publish/2019/04/09/contextual-ai-the-next-frontier-of-artificial-intelligence.html#gs.tuw2rt" target="_blank"></a></h1><a href="https://blog.adobe.com/en/publish/2019/04/09/contextual-ai-the-next-frontier-of-artificial-intelligence.html#gs.tuw2rt" target="_blank">https://blog.adobe.com/en/publish/2019/04/09/contextual-ai-the-next-frontier-of-artificial-intelligence.html#gs.tuw2rt</a>Antonini, Mattia &amp; Vecchio, Massimo &amp; Antonelli, Fabio &amp; Ducange, Pietro &amp; Perera, Charith. (2018). Smart Audio Sensors in the Internet of Things Edge for Anomaly Detection. IEEE Access. PP. 1-1. 10.1109/ACCESS.2018.2877523.&nbsp;Do, Ha &amp; Sheng, Weihua &amp; Liu, Meiqin &amp; Zhang, Senlin. (2016). Context-aware sound event recognition for home service robots. 739-744. 10.1109/COASE.2016.7743476.&nbsp;<a data-lf-fd-inspected-kn9eq4roawkarlvp="true" href="https://www.knowles.com/docs/default-source/default-document-library/knowles-ia8201-product-brief-final1b6a11c731dff6ddbb37cff0000940c19.pdf" target="_blank">https://www.knowles.com/docs/default-source/default-document-library/knowles-ia8201-product-brief-final1b6a11c731dff6ddbb37cff0000940c19.pdf</a><a href="https://voicebot.ai/2019/06/01/dedicated-audio-processors-at-the-edge-are-the-future-here-are-the-reasons-why/" target="_blank">https://voicebot.ai/2019/06/01/dedicated-audio-processors-at-the-edge-are-the-future-here-are-the-reasons-why/</a><a href="https://www.networkworld.com/article/3262105/how-edge-computing-makes-voice-assistants-faster-and-more-powerful.html" target="_blank">https://www.networkworld.com/article/3262105/how-edge-computing-makes-voice-assistants-faster-and-more-powerful.html</a>

edge processors

Latest Posts