The Best Approaches to Automating IoT-Enabled HVAC System Testing

January 12, 2021

Story

HVACs are now smarter than ever as they can be remotely controlled with just a simple click on our mobile device using a connected application. HVAC systems can also be configured with Alexa or Google Smart Home to make our life more convenient through the use of voice controls. IoT technology has also made HVAC systems smarter and more efficient. 

HVACs are now smarter than ever as they can be remotely controlled with just a simple click on our mobile device using a connected application. HVAC systems can also be configured with Alexa or Google Smart Home to make our life more convenient through the use of voice controls. IoT technology has also made HVAC systems smarter and more efficient.  

Major components of a Smart HVAC system consist of the HVAC device, the cloud for communicating with the device, and a mobile application that controls device functions. Multiple HVAC systems combine in zoned systems that achieve greater energy efficiency and independent and customizable heating and cooling needs in different areas. 

HVAC Zone System

Why Test Smart HVAC Systems?

Since HVAC systems control the environmental condition of a particular area or region, it’s extremely critical to ensure their overall performance. Also, when the system is smart enough to be controlled by a remote application, it becomes vital to verify the end-to-end integrity of the mobile application, cloud APIs, and communications protocols and device functionalities both ways (i.e., from the mobile application to the cloud to the HVAC system and back).

If any of these components fail to function properly, the entire system may fail. And that’s why testing HVAC systems is a complex but crucial part of the development and deployment lifecycle.

Controlling HVAC on-the-go.

An Approach to Testing Smart HVAC Systems

In order to ensure the overall system integrity of HVAC system, there are three different approaches to consider:

1.   Functional Feature Validation

2.   API Testing

3.   Performance/Load Testing

Functional Feature Validation

HVAC devices have scheduling features wherein the scheduled mode and temperature can be set throughout the day or on a weekly basis using time slots. Some devices may facilitate the vacation/one-touch away feature, which can put the thermostat in “away” mode.

All these features can be validated using either manual or automated testing, which includes the testing of mobile applications and its connection to HVAC devices via communications protocol and APIs. One of the most important aspects is to ensure that the thermostat is energy efficient and its power consumption is minimal, which can be verified through usage graphs that give an overall picture of energy usage versus consumption.

(Editor's note: Learn more about automated end-to-end system testing here).

(Functional Validation)

There are various tools and frameworks available in the market, but the best open-source tool, Appium, is widely used to test mobile applications and can be used with the Robot Framework for HVAC device validation via MQTT.

An HVAC device can connect to a Raspberry Pi and a Python script (that is Publish/Subscribed to request and response data) would read or write the values of thermostat device objects. Automation scripts can validate the functional aspects of a mobile application by updating the values that will be verified on the thermostat and reading the corresponding object values over an MQTT response topic it is subscribed, and vice versa.

API Testing

APIs are the building blocks developers use to establish communication between mobile applications and devices over communications protocols. Hence, it’s important to ensure that APIs give the expected response in the request sent to the server.

There are various tools available for testing Rest APIs, and Postman is one of them (Editor's note: Refer to this blog for details on API test automation using Postman and Continuous Integration testing using Jenkins).

(API Testing)

Performance/Load Testing

When it comes to the testing of a large number of devices and handling concurrent users simultaneously, performance/load testing must be executed in order to ensure the reliability, scalability, and load-handling capability of a system. While N number of users are using the HVAC system at a time, its performance may degrade or the response time may become delayed, which can crash the server during heavy loads.

This can be overcome by detecting the ability of a server to handle a maximum number of loads at a given time and distributing them over the different instances via  load testing. There are various tools available in the market, with Locust being one based on Python that can be used to develop performance scripts that execute over N number of HVAC devices to derive the tolerance/behavior of the system.

(Load Testing)

Automated Testing: Be Cool

Smart HVAC systems are now a part of most residential, industrial, and commercial buildings, which is why testing the right approach to testing is mandatory if you hope to discover unknown defects, ensure higher efficiency, and maximize performance prior to and after release.

Shraddha Shah has been a QA Automation Lead at VOLANSYS Technologies for more than 2 years. She has vast experience in Web, Mobile, and Cloud Automation testing domains.