<h1>Introduction</h1>Groq is on a mission to build the computer for the next generation of high-performance machine learning. Founded by former Google engineer, Johnathan Ross the company has doubled in size since its inception and is on track to double its headcount again by the end of 2021. In late September, Ross presented as part of the online AI summit 2020. His keynote gave key insights on the industry, the market, and the Groq architecture capable of producing the world’s first PetaOp chip. We’ve pulled out some of the highlights from the presentation.<h1>An answer to the end of Moore’s Law</h1>In 1971, Intel co-founder Gordon Moore predicted that the number of transistors we can pack into a microchip would double every 18-24 months. A statement that has proved itself true for a time, but as Moore himself noted “the nature of exponentials is that you push them out and eventually disaster happens”. Computing now finds itself on a precipice where<a href="https://www.wevolver.com/article/what.a.little.more.computing.power.can.do" rel="noopener noreferrer" target="_blank">&nbsp;cost, power, and efficiency&nbsp;</a>compete for priority to deliver the computing ambitions of the next generation.Rock’s Law or Moore’s second law states that the cost of a semiconductor chip fabrication plant doubles every four years. In line with this prediction, next-generation factories are estimated to cost more than $20 billion.Realizing the enormity of what these two laws mean for the future of computing placed Jonathon Ross, <a href="https://groq.com/" rel="noopener noreferrer" target="_blank">CEO, and Founder of Groq</a>, on his current path.Ross recounts when working for Google back in 2012, Jeff Dean gave a presentation to the leadership team, containing just two slides. The first slide was good news; machine learning finally works. The second slide was bad news; It would require another 20 to 40 data centers to be able to afford it - and that was just for speech recognition. At the end of the presentation, Ross started the TPU (Tensor Processing Unit) at Google.Fast forward several years and Ross, now the head of Groq, unveils the world’s first and only PetaOp chip. The chip’s performance is equivalent to one quadrillion operations per second, or 1e15 ops/s. Groq's architecture is also capable of up to 250 trillion floating-point operations per second (FLOPS).<img src="https://s3-us-west-1.amazonaws.com/wevolver-project-images/froala%2F1603820876791-Groq+banner.jpg" style="width: 300px;" class="fr-fic fr-dib"><h2>Architecture over speed</h2>“The industry often focuses on faster speeds,” begins Ross. “That’s what so many chip makers are focused on today. Everyone in this field is trying to make a faster chip. That approach of throwing more transistors onto a die means cost increases with incremental performance improvements. Following a trend that is running out of steam isn't a win for customers who are planning for the future. What we've done is come up with something revolutionary. I call this deterministic or predictable computing. A radically simplified approach which is scalable for customers. This is what enabled us to create the first and only PetaOp processor.”<h2>The shortest path to performance</h2>“How did we get there?” Ross reflects at the start of his keynote address. &nbsp;“We got there by doing things differently. A good example of this, we built on a 14-nanometer process when others were building on a 7-nanometer process. We simply didn't need the bleeding edge lithography to produce the fastest chip. Instead of adding more transistors, our performance came from our architecture. Everyone here believes Moore's law will run out. We talk about it and no one is acting any differently than they did before.”Ross asserts that the real question is not the end of Moore’s law but what happens if ML accelerator hardware doesn't get any better. The answer, Ross believes, is in designing and delivering a new architecture not based on the familiar systems of FPGA, CPU, or GPU. “It’s something completely different,” says Ross, “we call it the TSP or the tensor streaming processor.”<h2>Compute as power</h2>“Compute is a utility like power,” states Ross. “The country that controls compute will be in an advantaged position.” The future of computing is likely to need a careful mix of private enterprise, government, and academic input, research, and funding to ensure both its <a href="https://www.wevolver.com/article/a-global-collaboration-to-move-artificial-intelligence-principles-to-practice" rel="noopener noreferrer" target="_blank">continued development and shared access</a>. It isn’t only owning or controlling the technology that will give a nation an advantage but rather the overall deliberate direction and ambition around legislation, use, access, and dominance.“The future isn't guaranteed,” says Ross, “but you do get the future that you stand up for.” For Ross being deliberate about your approach is key to success. But he also understands this needs to happen across the industry to be effective for as many as possible.&nbsp;“At Groq we believe in the shortest path to performance,” continues Ross. “We have to stand up for the future we want by creating new trends with our approach. Trends like:<ul><li>Inference being different than training and being bigger than training.</li><li>Batch Size 1</li><li>Determinism and predictability, and</li><li>Performance and software first, which to us means the complexity and intelligence of a task is managed in the software stack.</li></ul>“Defining these trends originated with what we’re accomplishing at Groq but more importantly, these are going to enable customers' AI applications in production, that today are simply too costly.”<h1>Hopes and Challenges for the Future of AI</h1>Key investors and stakeholders in the AI field joined Ross in the presentation to share their hopes and challenges for the future. &nbsp;Each speaker pointed to where their priorities are for the development of the technology and gave insights into where their organization is directing resources. Speakers included Nicolas Savage, the Managing Director of TDK ventures the, corporate VC arm of TDK, CEO of Canadian quantum computing company, &nbsp;1QBit, &nbsp;Andrew Fursman, &nbsp;Teddy Gleeser, &nbsp;a partner at D1 Capita and CTO of the ASIC business unit at Marvell, Igor Arsovski. Their full remarks can be viewed in the video excerpts below:<h2>Predictability and reliability</h2>“I’m personally excited about future advancements like bioengineering, finance, government applications, and especially autonomous vehicles because it will save lives,” shared Ross. &nbsp;“My cousin was a passenger in a car that got into an accident. The driver of the other car had been drinking and he slammed into the side of my cousin's car. This didn't have to happen and if we had autonomous cars it wouldn't have happened. I want our chip in those cars because we need the level of predictability that I know our chip can bring but there's something else that AI brings you. When cars can make deliveries to you when auto-editing makes you a better photographer, when auto-sorting your inbox or when speech recognition keeps an EMT's hands-free so they can focus on their patient in an emergency.”“One major challenge in today's AI's systems is when you look at AI accelerators which typically push both performance but at the same time try to drop the voltage as low as possible so they can achieve really high energy efficiency. This tops per watt metric is kind of critical. Furthermore, these accelerators typically get integrated into data centers and automotive applications that have critical uptime and reliability requirements. I know reliability is not as sexy as the tops per watt metric and because of that it's taken a backseat to many AI discussions and many AI conferences but reliability is really critical to deliver useful systems.”<h2>Time is a Limited Resource</h2>“Silicon Valley has gotten a lot of criticism for focusing on convenience but I’m not going to apologize for that whether it's a safety issue or convenience time is a limited resource and we only get so much of it. What are the obstacles keeping us from that new reality? You’re probably thinking about speed but that's only part of the equation. Performance is a big challenge but reliability, predictability, and certainty in the real world is key. Chip makers like us need to deliver that performance not just in the lab but also in the real world,” details Ross.“Training cost isn't an issue. Train the model once and use it as much as you want. With inference, each response has to be real-time and it costs money and energy that's a scaling issue. Now instead of scaling with the number of ML researchers, you're scaling with the number of users and the number of queries. Inference at scale is harder than training.”“When I was at google roughly 80% of the ML models that were trained ended up having to be abandoned because they were too expensive to run in production. AI success is hinged on speed for inference as well as training but for the workloads of the future total cost of ownership is dependent on scalability. Just like when the team and I created the TPU at Google and it saved billions of dollars, we as an industry have just begun tackling these challenges.”<img src="https://s3-us-west-1.amazonaws.com/wevolver-project-images/froala%2F1603874293788-groq+in+body.jpg" style="width: 788px;" class="fr-fic fr-dib"><h1>The future isn't guaranteed</h1>One of the guiding mantras at Groq is that the ‘future isn’t guaranteed’. This is essentially a guiding principle that the future is both unknown yet directable. The intention within the development of technologies such as AI and computing must be deliberate and understood within a wider socio-economic context. Ross takes the responsibility that he and his company, and others like it have the power to change the future, very seriously. It's a way of thinking that is embedded in the company's methods and products.“At Google, the team and I created the TPU for existing workloads that we could make better as well as new and emerging workloads,” says Ross. &nbsp;“To plan for the future means understanding those workloads. Those were yesterday's challenges and we have new challenges today like power efficiency. To Groq those obstacles are the path and that's where we're going.”“Today, we're shipping to our customers both as individual PCIe cards and systems with eight cards each and there's even more on the roadmap to come. I know 2020 has been a crazy year. Together we're pushing through and making great strides. Groq is going to grow our <a href="https://groq.com/careers/" rel="noopener noreferrer" target="_blank">headcount by 4X by the end of next year.</a> We’re not going to lose who we are because it's scrappy maneuverability that lets us innovate faster and more affordably than everyone else. That’s how we built the first PetaOp chip and that's how we're going to build what's next.”<h1>Conclusion</h1>The power of computing is at a tipping point where major players are making decisions about the direction they will pursue that will have huge impacts on the future of technological innovation and potential. Groq is one of these companies that understands it has the potential to completely change the direction of computing and massively influence the potential of AI. With that potential comes responsibility - something Ross and his team have demonstrated they are ready to shoulder. Interested in learning more about Groq? Head to their website <a href="http://www.groq.com" target="_blank">www.groq.com</a> to learn more about the future of computing and review their <a href="https://www.linkedin.com/company/groq/jobs/" target="_blank">multiple career opportunities</a>.<hr><h1>About the sponsor: &nbsp;Groq</h1>Building the computer for the next generation of high performance machine learning. Groq hardware is designed to be both high performance and highly responsive. Groq’s new simplified architecture drives incredible performance at batch size 1. Whether you have one image or a million, Groq hardware responds faster. &nbsp;<img src="https://s3-us-west-1.amazonaws.com/wevolver-project-images/froala%2F1603879016114-groq++logo+banner.jpg" class="fr-fic fr-dib"> <div language="JavaScript"> </div>

Jonathan Ross, Founder, and CEO of Groq delivered a keynote speech as part of the AI Summit on September 29, 2020, describing insights on the industry, the market, and the unique Groq architecture capable of producing the world’s first PetaOp chip

The shortest path to performance requires a new approach

Today, artificial intelligence — and the computing systems that underlie it — are more than just matters of technology; they are matters of state and society, of governance and the public interest.&nbsp;The choices that technologists, policymakers, and communities make in the next few years will shape the relationship between machines and humans for decades to come.The rapidly increasing applicability of AI has prompted a number of organizations to develop high-level principles on social and ethical issues such as privacy, fairness, bias, transparency, and accountability. Building on those broader principles, the&nbsp;<a href="https://aipolicyforum.mit.edu/" target="_blank">AI Policy Forum</a>, a global effort convened by the MIT Stephen A. Schwarzman College of Computing, will provide an overarching policy framework and tools for governments and companies to implement in concrete ways.“Our goal is to help policymakers in making practical decisions about AI policy,” says Daniel Huttenlocher, dean of the MIT Schwarzman College of Computing. “We are not trying to develop another set of principles around AI, several of which already exist, but rather provide context and guidelines specific to a field of use of AI to help policymakers around the world with implementation.”“Moving beyond principles means understanding trade-offs and identifying the technical tools and the policy levers to address them. We created the college to examine and address these types of issues, but this can’t be a siloed effort. We need for this to be a global collaboration and engage scientists, technologists, policymakers, and business leaders,” says MIT Provost Martin Schmidt. “This is a challenging and complex process for which we need all hands-on deck.”The AI Policy Forum is designed as a yearlong process. Activities associated with this effort will be distinguished by their focus on tangible outcomes — their engagement with key government officials at the local, national, and international level charged with designing those public policies, and their deep technical grounding in the latest advances in the science of AI.&nbsp;The measure of success will be whether these efforts have bridged the gap between these communities, translated principled agreement into actionable outcomes, and helped create the conditions for deeper trust between humans and machines.The global collaboration will begin in late 2020 and early 2021 with a series of AI Policy Forum Task Forces, chaired by MIT researchers and bringing together the world’s leading technical and policy experts on some of the most pressing issues of AI policy, starting with AI in finance and mobility. Further task forces throughout 2021 will convene more communities of practice with the shared aim of designing the next chapter of AI: one that both delivers on AI’s innovative potential and responds to society’s needs.Each task force will produce results that inform concrete public policies and frameworks for the next chapter of AI, and help define the roles that the academic and business communities, civil society, and governments will need to play in making it a reality. Research from the task forces will feed into the development of the AI Policy Framework, a dynamic assessment tool that will help governments gauge their own progress on AI policy-making goals and guide application of best practices appropriate to their own national priorities.On May 6–7, 2021, MIT will host — most likely online — the first AI Policy Forum Summit, a two-day collaborative gathering to discuss the progress of the task forces towards equipping high-level decision-makers with a deeper understanding of the tools at their disposal — and trade-offs to be made — to produce better public policy around AI, and better AI systems with concern for public policy. Then, in fall 2021, a follow-on event at MIT will bring together leaders from across sectors and countries and, built atop the leading research from the task forces, the forum will provide a focal point for work to move from AI principles to AI practice, and serve as a springboard to global efforts to design the future of AI.

Convened by the MIT Schwarzman College of Computing, the AI Policy Forum will develop frameworks and tools for governments and companies to implement concrete policies.

A global collaboration to move artificial intelligence principles to practice

With the prevalence of engineering design and materials development at the forefront of additive manufacturing, there is still a lot to learn from nature when it comes to the design of advanced components. When accounting for strength-to-weight ratio, biological structures often vastly exceed man-made material impact resistance even when compared to the most cutting-edge research out there. The structural principles that define these biological “armors” are profound and groundbreaking. But there is a modeling limitation on these structures – how do you utilize current tools to define these structures in 3D for use? Up until now researchers have used computational modeling to evaluate certain biological structures, but usually this is for a one-off test or visualization. The use of such structures as applied to commercial applications has been limited by the current software and modeling technology. In my <a href="https://ntopology.com/design-for-additive-manufacturing-and-the-nfl-helmet-challenge/?utm_source=blog&utm_medium=event&utm_campaign=dfam-blog-promo&utm_content=20200330-nfl---blog:-matt-shomper" rel="noopener" target="_blank">upcoming webinar</a>, I step through several biological structure examples – the telson armor of the mantis shrimp, the specialized bones of the bighorn sheep – and analyze how they provide an unprecedented amount of impact resistance.&nbsp;<img src="https://images.wevolver.com/eyJidWNrZXQiOiAid2V2b2x2ZXItcHJvamVjdC1pbWFnZXMiLCAia2V5IjogImZyb2FsYS8xNjAxMjY1MTUxNTMzLW5Ub3BfRGZBTV9CbG9nU2hvbXBlckltYWdlMS5qcGciLCAiZWRpdHMiOiB7InJlc2l6ZSI6IHsid2lkdGgiOiA5NTAsICJmaXQiOiAiY292ZXIifX19">The analysis of these creatures’ biological structures offers us a glimpse into how we might borrow design elements to better resist impacts&nbsp;But understanding how these structures look is only the first (and easiest in some respects) part of the problem. I then move into the complex computational modeling of these structures as periodic (i.e., repeatable) implicit elements that can be readily used to fill 3-Dimensional spaces (such as football helmets, body armor, etc.) at any scale – without sacrificing speed of use or quality.&nbsp;<img src="https://images.wevolver.com/eyJidWNrZXQiOiAid2V2b2x2ZXItcHJvamVjdC1pbWFnZXMiLCAia2V5IjogImZyb2FsYS8xNjAxMjY1MTc0MzkyLW5Ub3BfRGZBTV9CbG9nU2hvbXBlckltYWdlMi5qcGciLCAiZWRpdHMiOiB7InJlc2l6ZSI6IHsid2lkdGgiOiA5NTAsICJmaXQiOiAiY292ZXIifX19">Computational modeling of the mantis shrimp’s telson armor allows for its application in impact-resistant&nbsp;The modeling of these structures also accounts for a wide range of user inputs so as to be infinitely controllable.<hr>This article was first published on the&nbsp;<a href="https://ntopology.com/blog/2020/04/06/impact-resistant-bio-structures/" rel="noopener noreferrer" target="_blank">nTopology blog</a>.

Computational modeling of the mantis shrimp’s telson armor allows for its application in impact-resistant

Taking a look at what nature can teach us about mitigating collisions. The third NFL Helmet Challenge DfAM eSeries Guest Blog

Impact Resistant Bio-Structures

For years, scientists have looked for ways to cool molecules down to ultracold temperatures, at which point the molecules should slow to a crawl, allowing scientists to precisely control their quantum behavior. This could enable researchers to use molecules as complex bits for quantum computing, tuning individual molecules like tiny knobs to carry out multiple streams of calculations at a time.While scientists have super-cooled atoms, doing the same for molecules, which are more complex in their behavior and structure, has proven to be a much bigger challenge.Now MIT physicists have found a way to cool molecules of sodium lithium down to 200 billionths of a Kelvin, just a hair above absolute zero. They did so by applying a technique called collisional cooling, in which they immersed molecules of cold sodium lithium in a cloud of even colder sodium atoms. The ultracold atoms acted as a refrigerant to cool the molecules even further.Collisional cooling is a standard technique used to cool down atoms using other, colder atoms. And for more than a decade, researchers have attempted to supercool a number of different molecules using collisional cooling, only to find that when molecules collided with atoms, they exchanged energy in such a way that the molecules were heated or destroyed in the process, called &ldquo;bad&rdquo; collisions.In their own experiments, the MIT researchers found that if sodium lithium molecules and sodium atoms were made to spin in the same way, they could avoid self-destructing, and instead engaged in &ldquo;good&rdquo; collisions, where the atoms took away the molecules&rsquo; energy, in the form of heat. The team used precise control of magnetic fields and an intricate system of lasers to choreograph the spin and the rotational motion of the molecules. As result, the atom-molecule mixture had a high ratio of good-to-bad collisions and was cooled down from 2 microkelvins to 220 nanokelvins.&ldquo;Collisional cooling has been the workhorse for cooling atoms,&rdquo; adds Nobel Prize laureate Wolfgang Ketterle, the John D. Arthur professor of physics at MIT.&nbsp;&ldquo;I wasn&rsquo;t convinced that our scheme would work, but since we didn&rsquo;t know for sure, we had to try it. We know now that it works for cooling sodium lithium molecules. Whether it will work for other classes of molecules remains to be seen.&rdquo;Their findings, published today in the journal Nature, mark the first time researchers have successfully used collisional cooling to cool molecules down to nanokelvin temperatures.Ketterle&rsquo;s coauthors on the paper are lead author Hyungmok Son, a graduate student in Harvard University&rsquo;s Department of Physics, along with MIT physics graduate student Juliana Park, and Alan Jamison, a professor of physics at the University of Waterloo and visiting scientist in MIT&rsquo;s Research Laboratory of Electronics.<img src="https://s3-us-west-1.amazonaws.com/wevolver-project-images/froala%2F1586440710384-MIT-Molecular-Fridge-02.jpg">A new refrigerator for molecules. Sodium atoms (yellow spheres) collide with sodium-lithium molecules (combined-yellow-red-spheres). The atom-molecule mixture is trapped in an optical trap whose effective edge is shown as a white rim. As the trap is loosened (depicted as a dimmer rim), the most energetic sodium atoms leave the trap, providing evaporative cooling. The cooling is transferred to the molecules via elastic collisions. The frost on the molecules indicates that they have reached a temperature of 200 billionths of a degree Kelvin. Figure credit: Pilsu Heo at Micropicture (South Korea)<h2>Reaching ultralow temperatures</h2>In the past, scientists found that when they tried to cool molecules down to ultracold temperatures by surrounding them with even colder atoms, the particles collided such that the atoms imparted extra energy or rotation to the molecules, sending them flying out of the trap, or self-destructing all together by chemical reactions.The MIT researchers wondered whether molecules and atoms, having the same spin, could avoid this effect, and remain ultracold and stable as a result. They looked to test their idea with sodium lithium, a &ldquo;diatomic&rdquo; molecule that Ketterle&rsquo;s group experiments with regularly, consisting of one lithium and one sodium atom.&ldquo;Sodium lithium molecules are quite different from other molecules people have tried,&rdquo; Jamison says. &ldquo;Many folks expected those differences would make cooling even less likely to work. However, we had a feeling these differences could be an advantage instead of a detriment.&rdquo;The researchers fine-tuned a system of more than 20 laser beams and various magnetic fields to trap and cool atoms of sodium and lithium in a vacuum chamber, down to about 2 microkelvins &mdash; a temperature Son says is optimal for the atoms to bond together as sodium lithium molecules.Once the researchers were able to produce enough molecules, they shone laser beams of specific frequencies and polarizations to control the quantum state of the molecules and carefully tuned microwave fields to make atoms spin in the same way as the molecules. &nbsp;&ldquo;Then we make the refrigerator colder and colder,&rdquo; says Son, referring to the sodium atoms that surround the cloud of the newly formed molecules. &ldquo;We lower the power of the trapping laser, making the optical trap looser and looser, which brings the temperature of sodium atoms down, and further cools the molecules, to 200 billionths of a kelvin.&rdquo;The group observed that the molecules were able to remain at these ultracold temperatures for up to one second. &ldquo;In our world, a second is very long,&rdquo; Ketterle says. &ldquo;What you want to do with these molecules is quantum computation and exploring new materials, which all can be done in small fractions of a second.&rdquo;If the team can get sodium lithium molecules to be about five times colder than what they have so far achieved, they will have reached a so-called quantum degenerate regime where individual molecules become indistinguishable and their collective behavior is controlled by quantum mechanics. Son and his colleagues have some ideas for how to achieve this, which will involve months of work in optimizing their setup, as well as acquiring a new laser to integrate into their setup.&nbsp;&ldquo;Our work will lead to discussion in our community why collisional cooling has worked for us but not for others,&rdquo; Son says &ldquo;Perhaps we will soon have predictions how other molecules could be cooled in this way.&rdquo;

 A new “refrigerator” super-cools molecules to nanokelvin temperatures. The technique may enable more complex, molecule-based quantum computing.  Image: José-Luis Olivares, MIT

Technique may enable molecule-based quantum computing.

New "refrigerator" super-cools molecules to nanokelvin temperatures

Large-scale software services fight the efficiency battle on two fronts &ndash; efficient software that is flexible to changing consumer demands, and efficient hardware that can keep these massive services running quickly even in the face of diminishing returns from CPUs. Together, these factors determine both the quality of the user experience and the performance, cost, and energy efficiency of modern data centers.A change on one front requires adjustments on the other, and a new software architecture growing in popularity has posed a challenge to the hardware solutions current in most data centers. Called microservices, this modular approach to designing big enterprise software has left something to be desired in its interactions with another major rising force in datacenter efficiency, hardware accelerators.To bring these two promising technologies together more effectively, CSE PhD student Akshitha Sriraman, working with researchers from Facebook, has designed a way to measure exactly how much a hardware accelerator would speed up a datacenter. Appropriately named Accelerometer, the analytical model can be applied in the early stages of an accelerator&rsquo;s design to predict its effectiveness before ever being installed.Still a somewhat new technology in general computing usage, the effectiveness of hardware accelerators isn&rsquo;t as easy to predict as CPUs, which have decades of experience behind them. Investing in this sort of diverse custom hardware presents a risk at scale, since it might not live up to its expectations.But the potential for a big impact is there. Designed to perform one type of function extremely quickly, accelerators could theoretically be called upon for all the redundant, repetitive tasks used in common by bigger applications.That includes microservices. This software architecture approach conceives of a larger application as a collection of modular, task-specific services that can each be improved upon in isolation. This allows for changes to be made to the larger application without needing to change one huge, central codebase. It also allows for more services to be added more easily.Sriraman demonstrated that as few as 18% of most microservices&rsquo; CPU cycles are spent executing instructions that are core to their functionality. The remaining 82% are spent on common operations that are ripe for accelerating.&ldquo;Accelerating these overheads we identified can indeed improve speedup to a significant extent,&rdquo; Sriraman says. Beyond speed, it would make all of the datacenter&rsquo;s functions cheaper and more energy efficient. &ldquo;Acceleration will allow us to pack more work for the same power constraints and improve resource utilization at scale, so data center energy and cost savings will improve greatly.&rdquo;The issue with microservices is that their designs can turn out to be quite dissimilar, particularly with regard to how they interact with hardware. For example, a microservice can communicate with an accelerator while continuing to run other instructions on a CPU, or it could bring all of its functions to a halt while it offloads to the accelerator. Both of these cases face different &ldquo;offload overheads&rdquo; (the time spent sending a task from one processor to another), which becomes lost time for the datacenter if it&rsquo;s not accounted for.<img src="https://s3-us-west-1.amazonaws.com/wevolver-project-images/froala%2F1586355215730-accelerometer-graph-content.png">Accelerometer-estimated speedup for key overheads identified by the researchers.&nbsp;&ldquo;Each of these software design choices can result in different overheads that affect the overall speedup from acceleration,&rdquo; says Sriraman. This overhead is left out of the picture in prior work, she continues, as is the impact of the different microservice designs on performance.Additionally, accelerators themselves have to be used judiciously to have a net positive effect.&ldquo;Throwing an accelerator at every problem is ridiculous because it takes a lot of time, cost, and effort to build, test, and deploy each one,&rdquo; she concludes. &ldquo;There is a real need to precisely understand what and how to accelerate.&rdquo;Accelerometer is an analytical model that measures exactly how much performance would be improved by installing a given processor, if at all, with all of these nuances taken into account. That means it measures the positive effect of acceleration as well as the negative effect of spending time shuffling instructions around between computing components. And its capabilities aren&rsquo;t limited to new accelerators &ndash; the model can be applied to any kind of hardware, ranging from a simple CPU optimization to an extremely specialized remote ASIC.The tool was validated in Facebook&rsquo;s production environment using three retrospective case studies, demonstrating that its real speedup estimates have less than 3.7% error.The model is sufficiently accurate to already be put to use by Facebook, with early interest from other companies.&ldquo;We have received word that several of the big cloud players have started using Accelerometer to quickly discard bad accelerator choices and identify the good ones, to make well-informed hardware investments,&rdquo; Sriraman says. Facebook is using the model to explore new accelerators, incorporating it as a first step to quickly sort out good and bad hardware choices.This project, titled &ldquo;<a href="https://research.fb.com/publications/accelerometer-understanding-acceleration-opportunities-for-data-center-overheads-at-hyperscale/" target="_blank">Accelerometer: Understanding Acceleration Opportunities for Data Center Overheads at Hyperscale</a>,&rdquo; was accepted by the 2020 Architectural Support for Programming Languages and Operating Systems (ASPLOS) Conference and presented virtually.<iframe src="https://www.youtube.com/embed/H1a6FPFKG4A?&wmode=opaque" frameborder="0" allowfullscreen="" class="fr-draggable" style="width: 788px; height: 436px;"></iframe>Akshitha Sriraman&#39;s presentation to ASPLOS 2020 delivered virtually

The analytical model, called Accelerometer, can be applied in the early stages of an accelerator’s design to predict its effectiveness before ever being installed.

Analytical model predicts exactly how much a piece of hardware will speed up data centers

<div data-wow-delay=".09s">A new type of neural network made with memristors can dramatically improve the efficiency of teaching machines to think like humans. The network, called a reservoir computing system, could predict words before they are said during conversation, and help predict future outcomes based on the present.The research team that created the reservoir computing system, led by Wei Lu, U-M professor of electrical engineering and computer science, recently <a href="https://www.nature.com/articles/s41467-017-02337-y" target="_blank">published their work</a> in Nature Communications.Reservoir computing systems, which improve on a typical neural network&rsquo;s capacity and reduce the required training time, have been created in the past with larger optical components. However, the U-M group created their system using memristors, which require less space and can be integrated more easily into existing silicon-based electronics.Memristors are a special type of resistive device that can both perform logic and store data. This contrasts with typical computer systems, where processors perform logic separate from memory modules. In this study, Lu&rsquo;s team used a special memristor that memorizes events only in the near history.Inspired by brains, neural networks are composed of neurons, or nodes, and synapses, the connections between nodes.</div><blockquote data-wow-delay=".5s" data-wow-iteration="2">A lot of times, it takes days or months to train a network. It is very expensive.</blockquote><blockquote data-wow-delay=".5s" data-wow-iteration="2"><cite>Wei Lu</cite></blockquote>To train a neural network for a task, a neural network takes in a large set of questions and the answers to those questions. In this process of what&rsquo;s called supervised learning, the connections between nodes are weighted more heavily or lightly to minimize the amount of error in achieving the correct answer.Once trained, a neural network can then be tested without knowing the answer. For example, a system can process a new photo and correctly identify a human face, because it has learned the features of human faces from other photos in its training set.&ldquo;A lot of times, it takes days or months to train a network,&rdquo; says Lu. &ldquo;It is very expensive.&rdquo;Image recognition is also a relatively simple problem, as it doesn&rsquo;t require any information apart from a static image. More complex tasks, such as speech recognition, can depend highly on context and require neural networks to have knowledge of what has just occurred, or what has just been said.&ldquo;When transcribing speech to text or translating languages, a word&rsquo;s meaning and even pronunciation will differ depending on the previous syllables,&rdquo; says Lu.This requires a recurrent neural network, which incorporates loops within the network that give the network a memory effect. However, training these recurrent neural networks is especially expensive, Lu says.Reservoir computing systems built with memristors, however, can skip most of the expensive training process and still provide the network the capability to remember. This is because the most critical component of the system &ndash; the reservoir&nbsp;&ndash; does not require training.When a set of data is inputted into the reservoir, the reservoir identifies important time-related features of the data, and hands it off in a simpler format to a second network. This second network then only needs training like simpler neural networks, changing weights of the features and outputs that the first network passed on until it achieves an acceptable level of error.<img src="https://s3-us-west-1.amazonaws.com/wevolver-project-images/froala%2F1582158436980-reservoir.jpg">Schematic of a reservoir computing system, showing the reservoir with internal dynamics and the simpler output. Only the simpler output needs to be trained, allowing for quicker and lower-cost training. Courtesy Wei Lu.&nbsp;&ldquo;The beauty of reservoir computing is that while we design it, we don&rsquo;t have to train it,&rdquo; says Lu.The team proved the reservoir computing concept using a test of handwriting recognition, a common benchmark among neural networks. Numerals were broken up into rows of pixels, and fed into the computer with voltages like Morse code, with zero volts for a dark pixel and a little over one volt for a white pixel.Using only 88 memristors as nodes to identify handwritten versions of numerals, compared to a conventional network that would require thousands of nodes for the task, the reservoir achieved 91% accuracy.<img src="https://s3-us-west-1.amazonaws.com/wevolver-project-images/froala%2F1582158527708-648579.png" style="width: 1114px;" class="fr-fic fr-dib">Reservoir computing systems are especially adept at handling data that varies with time, like a stream of data or words, or a function depending on past results.To demonstrate this, the team tested a complex function that depended on multiple past results, which is common in engineering fields. The reservoir computing system was able to model the complex function with minimal error.Lu plans on exploring two future paths with this research: speech recognition and predictive analysis.&ldquo;We can make predictions on natural spoken language, so you don&rsquo;t even have to say the full word,&rdquo; explains Lu.&ldquo;We could actually predict what you plan to say next.&rdquo;In predictive analysis, Lu hopes to use the system to take in signals with noise, like static from far-off radio stations, and produce a cleaner stream of data. &ldquo;It could also predict and generate an output signal even if the input stopped,&rdquo; he says.<img src="https://s3-us-west-1.amazonaws.com/wevolver-project-images/froala%2F1582158568994-wei-lu.jpg">Wei Lu, Professor of Electrical Engineering &amp; Computer Science at the University of Michigan holds a memristor he created. Photo: Marcin Szczepanski.&nbsp;The work was published in&nbsp;Nature Communications&nbsp;in the article, &ldquo;Reservoir computing using dynamic memristors for temporal information processing&rdquo;, with authors Chao Du, Fuxi Cai, Mohammed Zidan, Wen Ma, Seung Hwan Lee, and Prof. Wei Lu.The research is part of a $6.9 million DARPA project, called &ldquo;Sparse Adaptive Local Learning for Sensing and Analytics,&rdquo; that aims to build a computer chip based on self-organizing, adaptive neural networks. The memristor networks are fabricated at Michigan&rsquo;s <a href="https://lnf.umich.edu/" target="_blank">Lurie Nanofabrication Facility</a>.Lu and his team <a href="https://news.engin.umich.edu/2017/05/next-gen-computing/" target="_blank">previously used</a> memristors in implementing &ldquo;sparse coding,&rdquo; which used a 32-by-32 array of memristors to efficiently analyze and recreate images.

The memristor chip that powers the new reservoir computing system. Photo: Wei Lu.

U-M researchers created a reservoir computing system that reduces training time and improves capacity of similar neural networks.

New quick-learning neural network powered by memristors

Researchers at the University of Michigan have developed an ultra-energy-efficient processor to help autonomous micro aerial vehicles navigate their environment. They were able to take advantage of highly effective deep learning algorithms to realize visual simultaneous localization and mapping (SLAM) technology in a single chip measuring a little more than a centimeter.The goal of SLAM is to provide a continual estimate of where a moving, autonomous device is located is located along its posture within its environment, both vertically and horizontally, while also constructing a 3D map as the robot navigates its environment. SLAM, which is also used extensively in augmented/virtual reality applications, can use a variety of sensing and location techniques such as lidar, GPS, and cameras.Visual SLAM relies solely on cameras, and is the preferred method for autonomous aerial vehicles. However, cameras require extensive processing, which increases the amount of power needed and the overall weight for the battery. That added weight is incompatible with micro aerial vehicles &ndash; some of which can be mere centimeters.<img src="https://s3-us-west-1.amazonaws.com/wevolver-project-images/froala%2F1575288164203-slam-system-content.jpg" style="width: 1114px;" class="fr-fic fr-dib">SLAM system.&nbsp;A team of faculty and students, led by Professors Hun-Seok Kim, David Blaauw, and Dennis Sylvester, have created the first visual SLAM processor on a single chip that provides highly accurate, low-power, and real-time results.&ldquo;Our processor can be integrated into the camera so that as the camera moves, the processor continuously computes its location and posture, while accurately mapping the surrounding area,&rdquo; said Ziyun Li, a doctoral student working on the project.The researchers were able to do this using convolutional neural networks (CNNs), a deep learning algorithm that has proven to be highly successful in image analysis. However, since CNNs require significant resources for computation and memory, they first needed to develop hardware that could support a CNN while flying onboard a drone.Similar systems developed by others had resulted in an incomplete SLAM system, or one that was limited in quality and range. A more recent attempt required an off-chip high-precision inertial measurement unit to simplify the complexity that comes with using CNN, which increased both the cost and the power.&ldquo;We implemented a single chip system,&rdquo; said Li, &ldquo;that allowed us to use the computation-heavy and accurate CNN algorithm to identify features.&rdquo; The system also accounted for drift (i.e., small errors that accumulate when mapping a path), resulting in a highly accurate map of the surrounding area.<iframe src="https://www.youtube.com/embed/3xgf5qWfGgk?&wmode=opaque" frameborder="0" allowfullscreen="" class="fr-draggable" style="width: 1115px; height: 624px;"></iframe>This video shows the Michigan CNN-SLAM processor in action. The incoming image stream is processed and the trajectory is estimated in real-time. The estimated path (~1km range) can be used for localization and navigation of the car.&nbsp;The system was tested on the industrial standard KITTI benchmark, which is often used to compare the vision systems used in autonomous vehicles. It achieved 97.9% accuracy in translation and 99.34% in rotation on KITTI rendering automotive scenes over 1km. The 10.92mm2 chip consumed 243.6mW to process 80fps VGA at 3.6TOPS/W, marking a 15x improvement in performance and 1.44x in energy efficiency over existing systems.Future applications of the technology include facilitating human computer interactions, such as virtual reality scenarios where changing the position of your head signals a specific intention.

The SLAM system onboard a drone in the atrium of the EECS Building.

Researchers built the first visual SLAM processor on a single chip that provides highly accurate, low-power, and real-time results.

SLAM-ming good hardware for drone navigation

Researchers from Carnegie Mellon University&rsquo;s College of Engineering and the Institute of Systems and Robotics of the University of Coimbra, Portugal have developed a simple, efficient method to make robust, highly flexible, tattoo-like circuits for use in wearable computing. The low-cost process adds trace amounts of an electrically-conductive, liquid metal alloy to tattoo paper that adheres to human skin. These ultrathin tattoos can be applied easily with water, the same way one would apply a child&rsquo;s decorative tattoo with a damp sponge.Other tattoo-like electronics either require complex fabrication techniques inside a cleanroom or lack the material performance required for stretchable digital circuit functionality on skin.<img src="https://s3-us-west-1.amazonaws.com/wevolver-project-images/froala%2F1573729030390-1011-em-carmel-tattoo-1.png" style="width: 1114px;" class="fr-fic fr-dib">The tattoos can conform and adhere to highly curved 3D surfaces, like a model of the human brain.Carmel Majidi, an associate professor of mechanical engineering and the director of the&nbsp;<a href="http://sml.me.cmu.edu/" rel="noopener" target="_blank">Soft Machines Lab</a> at Carnegie Mellon, and Mahmoud Tavakoli, the director of the&nbsp;<a href="https://sites.google.com/site/fsrlabisr/" rel="noopener" target="_blank">Soft and Printed Microelectronics Laboratory</a> <a href="https://sites.google.com/site/fsrlabisr/" target="_blank"></a>at the University of Coimbra, teamed up to develop methods for direct printing of stretchable electronic circuits.Their collaboration succeeded.&nbsp;&ldquo;This is a breakthrough in the printed electronics area,&rdquo; said Tavakoli. &ldquo;We showed for the first time that inkjet-printed patterns of silver nano particles can be sintered at room temperature using the gallium indium alloy. Removing the need for high temperature sintering makes our technique compatible with thin-film and heat sensitive substrates.&rdquo;<blockquote>The technique is simple. The tattoos are ultrathin, very stretchable, and inexpensive to produce.</blockquote>In addition to low-cost processing, these tattoos provide other advantages. Because they have mechanical properties similar tolightweight fabrics, they remain functional under bending, folding, twisting, and strains above 30% (which is the typical stretchability of human skin). They can conform and adhere to highly curved 3D surfaces, like a model of a human brain or a lemon.Applications for ultrathin, compliant tattoos include epidermal biomonitoring, soft robotics, flexible displays, and 3D-transferable printed electronics.The findings were published in Advanced Materials&nbsp;in a paper titled &ldquo;<a href="https://doi.org/10.1002/adma.201801852" rel="noopener" target="_blank">EGaIn‐Assisted Room‐Temperature Sintering of Silver Nanoparticles for Stretchable, Inkjet‐Printed, Thin‐Film Electronics</a>.&rdquo; Carmel Majidi explains the technology behind the electronic tattoos:<iframe src="https://www.youtube.com/embed/OaITzyMptFQ?&wmode=opaque" frameborder="0" allowfullscreen="" class="fr-draggable" style="width: 1115px; height: 619px;"></iframe>Video credit: Institute of Systems and Robotics of the University of Coimbra, Portugal and College of Engineering, Carnegie Mellon University

The ultrathin tattoos can be applied and removed easily with water, similar to deorative, temporary tattoos.

These stretchable, inexpensive, tattoo-like circuits are made with an inkjet printer.

Electronic tattoos for wearable computing

FingerSound uses wearable ring technology to detect numbers and letters drawn on fingers.

Technology provides eyes-free way to interact with smart devices

Wearable Computing Ring Allows Users to Write Words and Numbers with Thumb

The first programmable memristor computer&mdash;not just a memristor array operated through an external computer&mdash;has been developed at the University of Michigan. It could lead to AI processing directly on small, energy-constrained devices such as smartphones and sensors.A smartphone AI processor would mean that voice commands would no longer have to be sent to the cloud for interpretation, speeding up response time.&ldquo;Everyone wants to put an AI processor on smartphones, but you don&rsquo;t want your cell phone battery to drain very quickly,&rdquo; said <a href="http://www-personal.umich.edu/~wluee/" target="_blank">Wei Lu</a>, a professor of electrical and computer engineering at U-M and senior author of the paper in Nature Electronics.In medical devices, the ability to run AI algorithms without the cloud would enable better security and privacy.<h4>Why memristors are good for machine learning</h4>The key to making this possible could be an advanced computer component called the memristor. This circuit element, an electrical resistor with a memory, has a variable resistance that can serve as a form of information storage. Because memristors store and process information in the same location, they can get around the biggest bottleneck for computing speed and power: the connection between memory and processor.This is especially important for machine-learning algorithms that deal with lots of data &nbsp;to do things like identify objects in photos and videos&mdash;or predict which hospital patients are at higher risk of infection. Already, programmers prefer to run these algorithms on graphical processing units (GPUs) rather than a computer&rsquo;s main processor, the central processing unit (CPU).&ldquo;GPUs and very customized and optimized digital circuits are considered to be about 10-100 times better than CPUs in terms of power and throughput. Memristor AI processors could be another 10-100 times better,&rdquo; said Lu.GPUs perform better at machine learning tasks because they have thousands of small cores for running calculations all at once, as opposed to the string of calculations waiting their turn on one of the few powerful cores in a CPU. A memristor array takes this even further. Each memristor is able to do its own calculation, allowing thousands of operations within a core to be performed at once. In this experimental-scale computer, there were more than 5,800 memristors. A commercial design could include millions of them.Memristor arrays are especially suited to machine learning problems. The reason for this is the way that machine learning algorithms turn data into vectors&mdash;essentially, lists of data points. In predicting a patient&rsquo;s risk of infection in a hospital, for instance, this vector might list numerical representations of a patient&rsquo;s risk factors. Then, machine learning algorithms compare these &ldquo;input&rdquo; vectors with &ldquo;feature&rdquo; vectors stored in memory.&nbsp;These feature vectors represent certain traits of the data (such as the presence of an underlying disease). If matched, the system knows that the input data has that trait. The vectors are stored in matrices, which are like the spreadsheets of mathematics, and these matrices can be mapped directly onto the memristor arrays.&nbsp;What&rsquo;s more, as data is fed through the array, the bulk of the mathematical processing occurs through the natural resistances in the memristors, eliminating the need to move feature vectors in and out of the memory to perform the computations. This makes the &nbsp;arrays highly efficient at complicated matrix calculations.&nbsp;Earlier studies demonstrated the potential of memristor arrays for speeding up machine learning, but they needed external computing elements to function.<img src="https://s3-us-west-1.amazonaws.com/wevolver-project-images/froala%2F1569831908719-memristors-9content.jpg" style="width: 594px;" class="fr-fic fr-dib">Wei Lu stands with Seung Hwan Lee, an electrical engineering PhD student and co-author, who holds the memristor array. Photo: Robert Coelius, Michigan Engineering Communications &amp; Marketing&nbsp;<h4>Building a programmable memristor computer</h4>To build the first programmable memristor computer, Lu&rsquo;s team worked with <a href="https://zhang.engin.umich.edu/" target="_blank">Zhengya Zhang</a> and <a href="https://www.mpflynngroup.com/michael-flynn.html" target="_blank">Michael Flynn</a>, an associate professor and a professor of electrical and computer engineering at U-M, to design a chip that could integrate the memristor array with all the other elements needed to program and run it. Those components included a conventional digital processor and communication channels, as well as digital/analog converters to serve as interpreters between the analog memristor array and the rest of the computer.&nbsp;Lu&rsquo;s team then integrated the memristor array directly on the chip at U-M&rsquo;s Lurie Nanofabrication Facility. They also developed software to map machine learning algorithms onto the matrix-like structure of the memristor array.The team demonstrated the device with three bread-and-butter machine learning algorithms:<ul><li>Perceptron, which is used to classify information. They were able to identify imperfect Greek letters with 100% accuracy.</li><li>Sparse coding, which compresses and categorizes data, particularly images. The computer was able to find the most efficient way to reconstruct images in a set and identified patterns with 100% accuracy.</li><li>Two-layer neural network, designed to find patterns in complex data. This two-layer network found commonalities and differentiating factors in breast cancer screening data and then classified each case as malignant or benign with 94.6% accuracy.</li></ul>There are challenges in scaling up for commercial use: Memristors can&rsquo;t yet be made as identical as they need to be, and the information stored in the array isn&rsquo;t entirely reliable because it runs on analog&rsquo;s continuum rather than the digital either/or. These are future directions of Lu&rsquo;s group.

The memristor array chip plugs into the custom computer chip, forming the first programmable memristor computer. The team demonstrated that it could run three standard types of machine learning algorithms. Photo: Robert Coelius, Michigan Engineering Communications & Marketing

Circuit elements that store information in their electrical resistances enable a brain-like form of computing, storing and processing information in the same place.

First programmable memristor computer aims to bring AI processing down from the cloud

computing

Photoresist, a light-sensitive material, is essential in photolithography for transferring intricate circuit patterns onto semiconductor wafers. This article explores positive vs. negative photoresists, detailing their chemical mechanisms, processing parameters, and performance traits.

Positive vs. Negative Photoresist: A Comprehensive Guide to Photolithography in Semiconductor Fabrication

CMOS and CCD sensors both convert light into electronic signals, with CMOS using pixel-level transistors for faster, energy-efficient processing, while CCD transfers charge across the chip for superior image quality but with slower speed and higher power consumption.

CCD vs CMOS Image Sensors: A Comprehensive Guide for Engineering Professionals

I3C is becoming a strong alternative to I2C. Learn about the main differences, benefits, and uses of these two important communication protocols. Explore the world of inter-integrated circuit communication and see which protocol is best for your next project.

I3C vs I2C: Unraveling the Battle of Communication Protocols

CUDA Cores and Tensor Cores are specialized units within NVIDIA GPUs; the former are designed for a wide range of general GPU tasks, while the latter are specifically optimized to accelerate AI and deep learning through efficient matrix operations.

Tensor Cores vs CUDA Cores: The Powerhouses of GPU Computing from Nvidia

Optical Micro-Electro-Mechanical Systems (MEMS) Sensor 

This article delves deeper into the microscopic world of MEMS sensors, exploring their operating principles, diverse applications, and the cutting-edge advancements propelling them into the future.

MEMS Sensors: Miniature Marvels Revolutionizing Modern Technology

Capacitors store electrical energy, similar to batteries, and are used in many electronic devices. Due to their voltage-storing nature, handling them can be dangerous. This article outlines various techniques and safety measures to safely discharge capacitors.

Computing

Technical Guides

Positive vs. Negative Photoresist: A Comprehensive Guide to Photolithography in Semiconductor Fabrication...

CCD vs CMOS Image Sensors: A Comprehensive Guide for Engineering Professionals...