AI Energy Consumption and Sustainability

The TDL webinar on Thursday, 30 April featured a fascinating and timely discussion on a key economic and technical topic with a panel of experts who addressed the issues of energy consumption and sustainability in AI, highlighting major challenges and proposed mitigations to the current AI-related energy issues.

AI is and has been for some time front and centre in economic growth, productivity gains and job market realignment. It has also been a key factor in growing energy consumption. Combined with slow efficiency gains, the skyrocketing energy needs for AI create major concerns. Some efficiency gains have been achieved, and more attention is paid to this aspect today; but, with growing model sizes, longer training and more versatile utilisation of AI, improvements lead to Jevons paradox^[1] where greater efficiency leads to greater utilisation. High power density and heat concentration in hardware begin to affect architecture choices and deployment. Infrastructure constraints, inefficient algorithms and faulty hardware add to the problem.

Summary

This TDL webinar discussed AI energy consumption and sustainability, featuring panellists Ro Cammarota, Sean Koehl, Rahima Mohammad, and Marcus Pan. The discussion focused on the challenges of AI’s growing energy demands, with panellists highlighting three key areas for improvement: operational strategies for workload scheduling, hardware and model efficiency, and governance mechanisms. The panellists highlighted both short-term mitigations like digital twins, power regulation technologies and heterogeneity in computing environments, as well as long-term innovations including microfluidic cooling, silicon photonics and beyond CMOS (complementary metal-oxide-semiconductor) technologies. Despite acknowledging significant challenges, all panellists expressed optimism about the potential for technical innovation and human-driven progress in addressing AI’s energy sustainability issues. They believed the necessity driven by energy bottlenecks can bring forward the golden age of computing technologies where difficult problems, in optimisation, utilisation models, security, transmission and related areas will finally have to be addressed.

Background

Each participant was asked to comment on their view on the most important issues in energy efficiency including economics, designs, deployment architectures, specific applications, like AI or homomorphic encryption, and regional differences.

One of the key questions was whether demand is continuing to outpace efficiency improvements and what the main drivers of demand are. If that is the case, it is vital to understand what mitigations, including technical, economic, organizational or regulatory, are available. One approach to efficiency gains could be to improve grid architectures; whereas new computational designs may be the way to offer immediate relief. Another possible viable option to reduce consumption would be via reducing model size. Looking outside of AI, energy efficiency could be achieved in, for example, security and cryptography related applications.

It is vital to understand what realistic mitigations are available immediately, that can be adopted quickly, at least in parts of the market. For example, a viable energy capping in TPUs and GPUs could generate immediate energy savings. Likewise, operational shifts such as better workload scheduling may prove efficient or advanced cooling which could produce immediate and sizable energy savings. It’s feasible that today’s approaches to model optimization are not efficient and we could look to non-AI applications to see what uses are available.

There are also questions relating to regulatory and economic challenges alongside the potential improvements. A conundrum is teasing out how the Jevons paradox might play out in this environment and what the outcomes of the huge CapEx needed to continue to support data centre propagation. There are obvious risks if the economic promise of AI and other resource intensive technologies don’t play out as expected such as stranded assets and many more.

It may not be realistic to expect that the tech firms, many of them very young, will be able to pay for grid improvements and electricity costs, and it’s worth investigating what operational improvements can play a significant role in resolving this issue: for example, aggregating workloads, better scheduling/distribution, etc. There could be regulatory and standardisation approaches that could help in the short term that are as yet unformulated.

Looking to longer term solutions, there could be specific innovations that should be part of AI-plus-hardware design and innovations that should enable energy efficient crypto applications. Some of the innovations that could change the picture may be among today’s hot topics, such as silicon photonics, neuromorphic designs et al.

The future could go in several contrasting directions. Power abundance may be realistic within the longer term e.g., after 2030 but equally computing needs could be reduced at source for data, algorithms etc within the same timeframe. Similarly, truly advanced cooling could be introduced into data centres and there are likely to be other economic, organisational and operational considerations that will play a significant role.

Finally, the panellists were asked to step back and consider whether they felt optimistic about the improvements in the short term as well as long term. Read on to find out!

Speakers

The distinguished panellists tacking these thorny issues were:

· Ro Cammarota, Associate Professor, UC Irvine Donald Bren School of Information and Computer Sciences; Chief Scientist, Advisor, Leader in Encrypted Computing & Privacy-Preserving AI

· Sean Koehl, Senior Director of Tech Leadership and Communities, Intel Labs (retired)

· Rahima Mohammad, Semiconductor Technical Advisor, Vinci4D.ai

· Marcus Pan, Chair, AI and Mixed-Signal Hardware Track, IEEE Design Automation Conference (DAC)

The session was moderated by TDL strategic adviser, Claire Vishik

Data Centre Energy Efficiency Discussion

The meeting opened with a discussion on the impact of AI and resource-intensive technologies like cryptography on data centres and energy consumption. It was noted that, while efficiency improvements have been made in energy-efficient encryption and component design, demand growth often outpaces these improvements. The discussion was set to explore current developments, important areas for focus and future directions in energy efficiency and sustainability in computing, with the four distinguished panellists scheduled to provide insights

AI Energy Consumption Challenges

The conversation followed the challenges of AI energy consumption, identifying three key areas: infrastructure constraints, software model inefficiency, and explosive demand. The importance of aligning incentives across the stack to ensure sustainable growth was emphasised, highlighting that operational strategies can be improved immediately, followed by model efficiency, hardware development and finally governance mechanisms. It was agreed that significant innovation is needed, particularly in creating more specialised, energy-efficient models, and noted that, while there is exuberance around AI adoption, there are also efforts to address energy demands through innovations in architecture and grid technology.

AI Energy Consumption Challenges

The panellists discussed challenges and potential solutions for AI energy consumption and data centre power demands. It was mentioned that AI agents may not reduce compute demand but could enable more personalised, localised models. The growing power consumption in data centres was highlighted, projecting a significant increase to 948 terawatts by 2030, and so emphasising the need for multi-layered approaches to address energy efficiency across component design, silicon validation, system deployment and AI applications. It was suggested that digital twins and improved workload scheduling could help mitigate power issues, while also advocating for standardised benchmarks to highlight inefficiencies in AI applications. The discussion concluded with a focus on the need for market-oriented solutions to drive change beyond efficiency gains.

Data Centre Energy Efficiency

The panellists discussed energy efficiency challenges and mitigations in AI and data centre computing. They identified several short-term and long-term solutions, including power regulation improvements, gallium nitride materials, voltage stacking techniques and fluidic cooling. The group agreed that heterogeneity in computing environments and better workload scheduling would be key to improving efficiency. Long-term advances could include beyond-CMOS technologies, microfluidic cooling and expanded use of silicon photonics. All the panellists expressed optimism about the potential for significant progress in energy efficiency, driven by both technical innovations and the current focus on sustainability in the AI era.

The energy efficiency landscape is moving fast and the global industry problem needs multi-layered solutions. Between in-chip microfluidic cooling, the synopsys-ansys consolidation, and the federal grid-flexibility rulemaking, the next 18 months will look quite different from what we see today, suggesting that we should plan a follow up webinar within the next six to twelve months.

Afterthoughts

One of the panellists considered that we are hitting some fundamental limits in what can be done at the transistor level, for example, in terms of reducing minimum voltages, that push the problems higher up the stack. A few promising tech companies looking to reduce energy expenditures in the near term are using voltage stacking and GaN (gallium nitride) materials for power delivery, wafer-level integration to reduce I/O overheads and immersion cooling to reduce data centre level cooling overhead.

If these are too incremental, it’s most likely the priority should be to attack the AI problem in software and algorithmically; for example, by shifting from primarily using large monolithic models to a hybrid approach using much smaller specialised models where possible.

In the short term, it may be that the Jevons paradox is inevitable given the massive investments already put into AI that need to show financial returns, but longer term, there is some promise for more much more efficient transistor architectures using spintronic and ferroelectric approaches and more efficient I/O through micro LED optical approaches, both of which interestingly require an unnatural shift to doing many things slowly i.e., many slow I/Os, many slow but super-efficient compute cores.

Overall, the panellists were at least cautiously optimistic and some very optimistic about the direction in which the energy issues can push design and innovation. With the dire need to develop more efficient components and systems and the technology readiness for new commercial solutions, there is a rare opportunity to usher in new classes of systems, solve the problems that seemed too difficult but now need to be solved and move computing to a new plane, not just energy efficiency, but also better security, improved optimisation and utilisation models.

A non-technical question is whether a society, where a very large amount of knowledge work is done by AI, including commercial art, is sustainable culturally and economically. A topic for another webinar perhaps!

Watch the full recording of the webinar on our YouTube channel!

[1] The Jevons paradox in AI occurs when improvements in AI efficiency (lower energy consumption or cost per task) lead to such a massive increase in demand and usage that total energy consumption and resource utilization rise, rather than fall. Cheaper, more accessible AI models are driving massive expansion in AI applications, triggering this phenomenon. The term was originally coined by William Stanley Jevons in 1865 in a different context.