Designing energy-efficient deep learning algorithms

Saying sustainable AI is important would be an understatement. Especially techniques like deep learning tend to consume lots of power due to the huge amount of required computations. And a lot of power means a lot of carbon emissions. Let’s dive into some techniques that help to reduce the power consumption associated with AI systems.

Let’s start with an example of the training process of a large language model. Don’t fall off your chair reading the rest of this sentence, but that process can produce up to five times the amount of carbon emissions of an average car over its entire lifetime. Yes, you read that right: up to 5 times! Take a moment to let that sink in because this issue will only grow over time as more and more data is being collected annually and more and larger (smarter) models are being trained on this data. So what can we do to deal with this growing problem?

The premise above might sound familiar if you’ve already read the blog by Jente or seen the talk by Bart De Vos and Niels on this topic. If not, we definitely recommend doing so first to get a general scope. In this perspective we dive deeper into this issue, shifting the scope to techniques focused on reducing the energy consumption of deep learning algorithms.

Decision-making AI

Let’s start by asking a very obvious question: “Do we really need a prediction that frequently?”. A prediction is not always needed, sometimes it’s perfectly fine to wait a bit or to even skip it completely. Reducing the number of inference tasks means a reduction of computations and subsequently also the energy consumption.
For connected devices, a second question comes up: ”Where should the inference be performed?”. Both cloud and edge have their advantages and disadvantages that need to be balanced against each other to determine the best choice.