Many of Datalight’s recent conversations with customers and industry folks have focused on concerns over the decreasing endurance of flash memory. At this point, some smaller lithography parts have program/erase (P/E) cycles counted in the hundreds. Such limited P/E cycles can restrict the effective lifetime of devices that control infrastructure, such as the power grid, transportation systems, and medical devices.

As we enter the age of the Industrial Internet of Things (IIoT), the amount of data IIoT devices create, process, and transmit continues to increase at an astounding rate, meaning that using flash parts with low endurance will limit the effectiveness of devices. Thus, we often hear the question – what techniques can be used to make sure writes (and hence erases) are held to a minimum while still providing the level of performance and reliability that embedded devices require? We had had some theories, but wanted proof.

When we talk about our transactional file system, Reliance Nitro, we emphasize its ability to be configured for a variety of use cases. Regardless of how transaction settings are defined, the data that is written by the file system is protected from corruption and loss due to power failure. Databases also use transactions but, without being able to know what the behavior of the file system beneath them is, often take a “belt and suspenders” approach which can result in excessive writes. Modern flash has so few P/E cycles available that it is critical to pay attention to write amplification of any kind. Would adjusting the behavior of a database to rely on Reliance Nitro’s transactions instead of writing redundant data reduce the write amplification and improve performance at the same time?

We chose Mentor Graphics Embedded Linux as the operating system to test this out, giving us a stable and tested platform that could be brought up quickly. We also contacted several database companies to work with us on this project. Raima stepped up and willingly modified their RDM database.

I’m happy to say that the results of this simple experiment were quite successful. In the course of this work, it was discovered that additional changes to the database application might provide even further improvement. The changes Raima made this time around were simple: just use the atomic nature of Reliance Nitro transactions to ensure all of the individual files necessary for proper database operation were updated as one.

What are your ideas for optimizing and preserving the lifetime of flash media?

Results of the Datalight, Mentor Graphics, and Raima file system experiment can be found in the white paper, “Optimizing Your Software Stack for Industrial Internet of Things Devices.”

Kerri McConnell is a vice president at Datalight. She has over two decades of technology marketing and product management experience and has learned the importance of listening to customers to build better products. Known as the “queen of analogies,” Kerri excels in translating technical topics into laymen’s terms and finding the business benefit to be gained from product features.