It’s all about change.

“What are my options to maintain my existing tests when the source code has been modified?” This is a very common question I get when talking to my clients.

Some of my interlocutors are pointing out that they had to refactor their software, some others will talk about redesign efforts.

Firstly, I noticed that these two notions relating to software changes are not always clear in the minds of people and are sometimes used in the wrong context. These concepts might be crystal clear to you, but if they aren’t, here are some hints to help you understand the difference.

What is the difference between redesigning and refactoring software?

The main difference between these concepts is this: Redesigning implies that you modify your software to change what it does, whereas refactoring is an effort to modify how it does it.

• Redesigning efforts are conducted for multiple reasons. For instance, because of hardware changes the software needs to work on a different CPU or has to address new peripherals, thus the code needs to be modified or extended to address these physical modifications and deliver the new functionality. Redesigning can also occur when the software needs to interface with new or updated 3^rd-party libraries which provide new services that will benefit your application. You could probably find many other reasons for redesigning but for most cases, the software changes that are performed in this context have an impact on the general behavior or the functions offered by the modified application.
• As opposed to redesigning, refactoring is an effort to optimize the internal implementation of your code in order to increase its maintainability and reduce its overall cost of operation. Like many people, I believe that one of the best definitions of software Refactoring has been written by Martin Fowler in his “Refactoring Book”:

“A change made to the internal structure of software to make it easier to understand and cheaper to modify without changing its observable behavior.”

Given this definition, Refactoring is generally performed by developers when they:

• Need to control the technical debt to a tolerable level, i.e. below the line where it would appear more economical to rebuild the entire code from scratch again.
• Reduce the complexity and internal dependencies to make the software more modular, easier to extend, more readable and manageable for newcomers in the development team, etc.
• Ensure over time the original design stays understandable and clear, and its intended functions are preserved…

Given that we now have a clearer understanding of Redesigning vs Refactoring efforts,

Which circumstances require your software to be reverified?

Well, the essence of software tests is that they primarily check the code meets its purpose. In other words, they verify each software unit that makes up the system behaves as expected, according to the functional requirements of the application. Having said that, if you are attempting to redesign your code, you must test it to ensure the new functions are verified against the newly introduced requirements, and make sure at the same time that these new extensions are not introducing regressions in your existing passing tests.

You could argue that refactoring efforts only impact the software internal structure, and thus does not necessarily affect the code interfaces and the general services delivered as per the application requirements. Yes, but… Refactoring like any other development activity, is quite an easy way to introduce new bugs, so you must re-test your software. Maintaining a complete and exhaustive set of passing tests will ensure your Refactoring does not cause regression errors in your code to go undetected. Indeed, whenever you make a small change, you should re-execute the existing tests as  a safety net in order to check that you didn't modify the expected behavior. After a series of incremental changes, you will reach the refactored state you were originally targeting in a safe way.

Most organizations wish to retain the value of previous test investments by updating these tests as the source code changes. But this can result in high test maintenance costs. The solution is not as simple as just identifying a subset of impacted tests to re-run that are affected by a code change (sometimes called test impact analysis or change based testing). The expensive part of test maintenance is the developer effort spent identifying dependencies and updating corresponding tests to ensure they are synchronized with the modified software.

How then can a proper test automation lower these test maintenance costs?

1)      Through initial analysis of the code changes and test dependencies:

·       Understanding of changes to code being tested (by retaining information on the code when it was last tested and comparing that to the changed code)

·       Identifying which tests are affected by a code change

·       Identifying all the changes in the code which affect the test in a single view

·       Identifying code changes which may impact the code coverage achieved by the existing tests

2)      By providing developers with guided choices for automatic test updates in order to re-synchronize the source code and the tests:

•      For each code change, suggesting suitable updates to the test script and cases

•      Refactoring the test scripts automatically in order to be time and cost effective

3)      For code changes mostly affecting the internal structure of the software, automatically generating a safety-net or baseline of passing tests in order to:

•      Pinpoint failures during regression testing or continuous integration

•      Identify testability issues like unreachable code

As a professional software vendor, QA Systems is acutely aware of the importance of controlling test maintenance costs in the context of software modifications. To address this problem, we have developed as part of our testing solution Cantata, a Code Change Analysis and Management feature as well as an AutoTest generation framework which are unique technologies for automating unit & integration test maintenance throughout the lifecycle of your software projects. Redesigning or Refactoring your software should not be the (annoying) question anymore when you need to manage your tests!

If you are interested in trying the Cantata testing tool solution for your own software verification projects contact the QA Systems team by e-mail at [email protected].

About the Author

Having graduated with an MSc in Electronics and Business, Pierre-Henri started his professional career in 1998 at Attol Testware SA., one of the first Independent Software Vendors to offer professional testing solutions for safety-critical software. Subsequent years were spent contributing to the technical adoption and sales promotion of dynamic software testing solutions at Rational Software Inc., IBM Inc. and Wind River Inc. There, he enabled sales teams and participated in industrial testing solution deployments at major players of the Telecom, Aerospace, Defence and Automotive markets. In 2013, Pierre-Henri Stanek was appointed as Country Manager of QA Systems France and Cantata Product Manager.