A software QA engineer walks into a bar. He orders a beer. Orders 0 beers. Orders 99999999999 beers. Orders a lizard. Orders -1 beers. Orders a ueicbksjdhd. First real customer walks in and asks where the bathroom is. The bar bursts into flames, killing everyone.
Unknown Redditor in r/Jokes
We all know that testing is a critical part of embedded firmware. Most problems can be avoided simply by using good coding practices, and, even if some managers don’t want to hear it, taking the time necessary. In most large companies, a human factor will be involved, a tester who will follow a procedure, but also overstep those boundaries to dig deeper when needed. They are a fascinating breed to watch; a good tester will have a “feeling” that something isn’t right, and will dig deeper until they find the problem.
Some problems just can’t be found by humans until it is too late. One company I’ve worked with had a problem years after the release of a product, undetectable by traditional means. Data was written to an EEPROM, but this particular EEPROM could only handle pages of 256 bytes. Anything more than that required a separate write into another page, otherwise the beginning of the page would be overwritten. It took a few years for the data structure to grow beyond 256 bytes, but when that happened, the problem was instant. How could this have happened? We’ll never know; the original team had long since moved on. Perhaps no one read the datasheet correctly and simply didn’t plan for this. The engineer may have known about the 256-byte limit, but at the time, it was considered that going above 256 would never happen. Maybe there were tests in place, but nobody ever pushed the tests past the limits. Whatever the reason, the consequence was dire. Devices out in the field were dying.
Test-Driven Development (TDD for short) is one way of programming. If you need to write a function, write some tests to show that it works. Let’s take something really simple:
uint8_t addTwo(uint8_t u32_ValA, uint8_t u32_ValB)
{
return(u32_ValA + u32_ValB);
}We all know what this does, and at first glance, it looks correct, but seasoned engineers will already have identified a problem – what happens when the addition overflows? Of course, no one in their right mind would ever write a function as simple as this, but you get the point. Writing tests for this function would be simple – assert that two plus two equals four. You might write another ten lines, asserting that five and three equals eight, but there isn’t much point. However, due to the size constraints, adding 255 and 255 will not produce 510; it will overflow. If your code will only ever handle numbers up to 16, then you might be tempted just to leave it as is, but that is probably what happened with the EEPROM driver back in the day.
Another interesting question is: where do you test? The previous function is easy enough to test in a simulator, any processor on any computer should be able to handle this, even old Pentiums with the FDIV bug. What about the EEPROM driver? That might not be easy to implement on a simulator, or maybe the implementation itself is prone to error.
Sometimes, you need to perform tests on the hardware itself.
An elegant solution
My preferred solution is to use an application from SEGGER called JRun. It requires a JLink or JTrace to function, but is included in the software package. It is a command-line utility that is heavily configurable. It can run tests directly on your target and even help automate tests. It doesn’t take much to add it to a GitLab runner to perform tests automatically on a target when you commit. A single command can upload the test firmware, configure the tests to be run, and return the result.
In this mini-series, I’ll show you how to get JRun up and running on your system and how to use it on your target. I’ll use an STM32H563ZI Nucleo board and a J-Link Ultra+, but the principle remains the same for any board or SEGGER debugger. On with the show! In the next article, I’ll show you how to get up and running with this solution.