Use simulation to design and build a better IoT system, faster and easier.

The Internet of Things (IoT) will have a major impact on the way business is conducted in the future. New business models will be made possible by capturing and analyzing sensor data in real-time. For many organizations, the number of data points necessary to provide, for example, a cost competitive usage-based industrial robot contract will be in the hundreds. For even a modest multi-national robot manufacturer turned service provider, the world-wide deployment of IoT sensors and systems will run into thousands of sensors, gateways and systems.

These massive “systems-of-systems” will present a major challenge for testers. Ensuring that the IoT application is ready for a reliable deployment will be crucial. The real challenge is how to live test a deployment of hundreds or thousands of geographically dispersed devices. Fitting them into a lab is not only physically impossible but it wouldn’t properly test the more variable elements of the system, such as wireless connectivity, environmental conditions and real-life use patterns. The only realistic alternative is to test the overall system by simulation.

Most IoT systems tend to follow a similar three-tiered approach, like that shown in Figure 1. Small groups of nodes connect to each other and then connect, via wireless gateways and networks, to cloud-based data storage and analytic servers. Typically the node devices will be sensors to measure temperature, electricity consumption, light sensors or control actuators, such as switches, pumps, and heating controls. The gateways or concentrators handle the connection to the outside world, and ensure security. The back-end server, which is often in the cloud, deals with the business and control aspects of the IoT system.

Click to enlarge

Being able to test a system that would normally be geographically dispersed over multiple sites, installations, or facilities would require a similar number of lab setups. Achieving a physical simulation of this would turn out to be more expensive that the cost of the nodes themselves.

Thankfully, using a simulator, the setting up of a large network is relatively straightforward. This is achieved by writing code within the simulator to virtually deploy and layout the nodes over a virtual wireless network. Some modeling of the wireless network is then required to simulate the variable aspects of the wireless link between the nodes. A single script can replace attempting to manage the testing of many hundreds of physical items.

An example of a simulation solution is Wind River’s Simics. Using this simulator, you can simulate all aspects of your IoT design such as the hardware setup of each node, processors, tiers, wireless networking, etc. The real operating system and target code applications, using the identical binaries used on the actual hardware, run the simulated nodes.

Using this approach, you can see how your wireless stack performs under poor link conditions and when network faults occur, as well as what happens with the sensors and actuators, hardware sleep modes/wake-up for the nodes and how well they conserve power. You can also test all the middleware that manages the network nodes and how software updates, especially any OTA updates, function. Security can also be tested in this way, ensuring that gateways and nodes can operate correctly and not become exposed to potential security threats or hacking attempts.

Simulation will allow the tester to dynamically vary the radio network conditions with ease. Changing the signal strengths between a pair of nodes and to randomly drop packets as a result is something that can be set up within the simulation rules. Such conditions can occur often in the real world, such as when a train passes through the line of sight between two nodes and disrupting communications for a minute or so. Attempting to create such simulation in the lab with a physical set up would be difficult to achieve.

Simulation is also a very good way to test how well an IoT application can scale. Hardware and software behavior can change as the number of active nodes increases and this can affect the system’s overall performance. Sometimes the implications of how a system can scale aren’t immediately obvious but again, simulation provides a method to test this correctly.

As shown in Figure 2, simulation provides the ability to build systems of any size, from small to very large. This means that the system’s behavior can be tested on a whole range of scales, from small unit tests or subsystem tests, all the way up to the largest setups imaginable. Often, each system scale will reveal different issues in the system. It’s not just about the very largest setups, but also about making sure things work efficiently at intermediate system sizes, too.

Click to enlarge

Testing can also scale out horizontally, as shown in Figure 3. It’s easy to build many network variants to test the software with different ways to deploy the same number of nodes. Different balances between gateways and sensor nodes can be tested, as well as different network topologies. Figure 3 also shows how simulation lets you run many different tests in parallel, which makes the total time to run a set of tests lower than if they had to be run serially on hardware.

Click to enlarge

Another interesting aspect of an IoT-based design is that despite there being many hundreds or thousands of nodes, the duty cycle might be rather low. Sensors aren’t constantly reporting data to the cloud and that fact greatly assists for those sensors that are powered by batteries. As a consequence, there tends to be a lot of idle time within a system under test. This time can be exploited to accelerate the simulation by using “hyper-simulation.” Rather than playing out idle time cycle by cycle, a simulation solution like Simics jumps right to the next interesting event that would wake up a sleeping node. Hence, a system that’s mostly idle can be simulated many times faster than real time, which is a property that’s exploited in large IoT simulations.

In the end, physical labs are needed to perform final system testing. You must test what you ship and ship what you test. However, using simulation to augment the physical test lab to cover more test cases and run more test variants is necessary to ensure that quality is maintained and that the system is robust across a wide variety of situations. With simulation, you’ll be able to design and build a better IoT system faster and easier.

Dr. Jakob Engblom is a Product Line Manager at Wind River Systems, leading the development of the Simics simulation tool. He’s been working with embedded systems programming, simulation, and virtual platforms since 2002. Jakob holds a PhD in real-time computer systems from Uppsala University and an MSc in Computer Science, also from Uppsala University.