Have you ever wondered why your smart home devices are rather dumb when you switch off the internet? Turns out, many of these devices are not as bright as you think. They borrow their intelligence from their ‘cloud’ overlords. Maybe you’re entertaining the idea of making your company’s product smart as well. You might wonder: “Why can’t my device think on its own?” Is this cloud approach always the best approach? Can’t we liberate these devices from said overlords? In this blog, we’ll dive into the world of central (cloud) vs local (edge) intelligence and share how we weigh these options when we sketch out a new AI design for smart home applications.
Don’t need an introduction to cloud and edge intelligence? Skip to the 4-step guide.
Smart home devices and appliances are almost always connected to the internet. This is not just to control them remotely. Suppose they have some level of intelligent decision-making capabilities, they typically use that connection to tap into a centralized brain at some server farm, far away from your home. This is called decentralized intelligence or ‘cloud intelligence’.
Edge vs cloud intelligence © Verhaert
Your Alexa smart assistant, smart lights, smart thermostat, or Google Home Assistant are only brilliant when connected to the internet. You can test this by using your Google Home Assistant on your smartphone: try giving it a command when your smartphone is connected to the internet. It will understand you pretty well. Now try the same with your phone in airplane mode. You will notice the device is struggling to understand you.
But, it need not be this way. For instance, a smart washing machine can predict the perfect amount of laundry detergent based on the number of clothes you put in, even when the Wi-Fi connection hampers. Luckily, your smart robot vacuum cleaner doesn’t repeatedly bump into your favorite chine vase when the Wi-Fi connection hampers. It finds its way in your house perfectly fine. These devices seem to have all the intelligence they need locally. They are said to have ’edge intelligence‘. So if this is possible, why don’t all devices have this edge intelligence?
The answer is in the hardware. When it comes to making intelligent devices, size matters. At least when it comes to computational power. In older machines, it was more difficult to add intelligence as the edge devices’ computational capabilities were not sufficient. The best option was to have the intelligence run in some high-end computer connected through the internet instead of locally. More recently, advances in hardware technologies have emerged to bring intelligence towards the device itself. They finally allow intelligent machines to be freed from their cloud overlords.
The hardware that is unlocking edge AI
The critical enabling hardware technologies for edge AI can be divided into AI boards, AI chips, and AI MEMS. Developer boards like the NVIDIA Jetson are small but full-fledged computers, even though they’re only 10 by 10 cm big. AI chips are significantly smaller. We see them in more and more smartphones, they’re GPUs and TPUs of about 1 by 1 cm that can be embedded in any custom PCB. And even smaller are the Micro-Electromechanical Systems, or MEMS for short. Even though applying AI in MEMS is still mainly being researched, some commercial options are already available.
Overview AI hardware options © Verhaert
The four levels of intelligence
There are levels to this game. Four, to be exact. First, we could keep using cloud intelligence as before, we would then need to rent or maintain a server where this intelligence will live. On the other hand, we could move the intelligence towards the edge. When the AI algorithms are deployed on the edge device itself, this is called extreme edge.
But, it is also possible to deploy the AI somewhere between the device and the extreme edge. In this case, AI is often deployed to a computer that resides at a network hub. Finally, a hybrid approach is possible, in which part of the AI resides in the device and part in the cloud. This is also called ’distributed intelligence‘. This hybrid solution combines the larger compute power of cloud intelligence with the autonomy of edge intelligence for critical tasks. For more information on this, read our blog on hybrid intelligence.
4 levels of intelligence © Verhaert
A 4-step plan for deciding between edge and cloud
Based on our experience, we have developed a 4-step plan to help you decide between edge, cloud, or hybrid solutions. Through 4 questions, we challenge you to consider different factors and ultimately help you choose between edge or cloud.
Question 1: What are the physical constraints of your product?
First, let’s consider the size, weight, and temperature constraints and power consumption. Does your machine run on a battery, or is it plugged into a power socket? Can your device handle a temperature of over 60℃, or does the user touch it?
More powerful edge hardware will typically be larger to allow more simultaneous computations at higher speeds. They’ll also consume more power. Some of the most potent edge hardware devices require up to 50 watts at peak conditions. Using all this power will produce heat, increasing the temperature of the electronics. And if the temperature gets too high, these electronics will need to be cooled. What we are now doing, is eliminating the most powerful edge hardware.
Question 2: What kind of intelligence do you envision your product to have?
The next step is to think about the intelligence you want your product to have. For this, we first need to define the actual task of the intelligence and how complex it is.
Complex tasks often require some level of cloud computation. For example, continuous learning – in which the model keeps learning even after deployment – requires some form of powerful cloud intelligence. In comparison, a frozen model that’s not retrained after deployment requires less computation and might be performed in the edge.
Caution! Does your device perform a critical task, and does it need to be as accurate as possible? Or, is it sometimes allowed to make a mistake? To get a higher performance, we typically need more powerful hardware. By asking these questions, we can uncover if the power of a server would be needed or not.
Another essential aspect is how fast the intelligence should be. Does the device need to react in real-time? Or, can it take a few seconds to process its inputs? When you need to perform the task as fast as possible, we cannot rely on cloud intelligence as network latency will seriously impact this speed. We need to bring the intelligence closer to the edge to make the intelligence faster.
Does your system have some critical tasks that need to work, even when the internet is down? In this case, these tasks need to be pushed closer to the edge.
Question 3: What approach would be the cheapest for you?
Let’s talk money. Using more powerful edge hardware will make the product more expensive. Meaning that we still might need to scrap the powerful edge hardware, even if we found it to be technically feasible. For cloud intelligence, you need to maintain a server, or rent cloud space. For this, we make a careful trade-off between the bill of material and running server costs. By keeping in mind the requirements, we can come to a final decision.
Question 4: Are there other conflicting requirements?
Some of your other business requirements can drastically change your ultimate decision. One example could be the central collection of data. As it can contain critical insights about your customers, usage data can be valuable for your business. In cases where the decision process would result in some form of edge intelligence, it might now be better to have some hybrid approach.
For instance, with an intelligent coffee machine, we might want to know which coffees are preferred by which people. The coffee machine will need to send data to the cloud, which could open up the possibility of a hybrid approach. Maybe your product needs to process sensitive data. Is it worth the risk of sending this data over the internet? In some cases, it might be better to process the data as close to the edge as possible to keep the sensitive data contained within your product. This last step aims to evaluate each product requirement compared to its impact on your choice between edge and cloud.
Example use cases
In conclusion, we want to provide you with some examples to make the 4-step plan more tangible.
Examples AI applications © Verhaert
In case one, we want a device fitted to a window to detect suspicious behavior, which could be burglars. In the second case, we want to create a baby monitor that uses a camera to give the parents smart alerts. A final possibility is a coffee machine that changes its settings to brew the perfect coffee.
Let’s go through the 4-step plan. We know that the burglary detection device should be small and can’t consume too much power in terms of physical constraints. So in terms of edge hardware, we are constrained. Then for intelligence, we know this should be continuously learning the behavior of the window or door. Continuous learning is computationally quite complex. Combined with the answer to the first question, we can already rule out the extreme edge approach. In terms of speed, we want this to be a system that reacts immediately to dissuade potential burglars. This pushes us away from the cloud, especially since the device would no longer function without an internet connection. This way, we end up with either an edge or a hybrid approach, depending on the cost. Going with the edge approach would allow us to connect multiple devices to a single hub in the home.
If we do the same for the smart baby monitor and the smart coffee machine, we find that the best approach should be an extreme edge or a hybrid approach, as seen in the visual.
Examples AI applications with approach © Verhaert
As you’ve seen, there are many things to consider throughout the edge vs cloud decision process. And selecting the right approach is not always as simple as it seems. So good luck, we can’t wait to see your smart product in the market!
Wondering how we can just split intelligence to create a hybrid system using both edge and cloud. Or if you are just hungry for more content. Don’t fret, we got you covered! The next part in this series is all about hybrid intelligence, which you can read here.
