Introduction
Throughout my career, I have been a software tester. I have tested all kinds of products from compilers, to specific algorithms, to complex software systems. Luckily I have also been a software creator, so those development skills have helped me a lot when my role switched from being a developer to being a tester.
Even though many of these activities have been unpaid endeavors (i.e., a volunteer software tester) I sometimes have gained more important and valuable experience doing this work than some of the “real” work I have done on my job. Being a volunteer software tester can be a great opportunity to expand your knowledge, skills and your professional network.
Sometimes we call ourselves “Beta testers”, but the truth is we are usually testing new functionality of an existing software system. Since software continues to evolve into more complicated paradigms, it is important for software companies to have good software testers.
Why Are Software Testers Needed?
Good software testers are needed because most development teams cannot possibly put the software through the multitude of operations that real users will give it. The diversity of input, the ways that the software will be used, and the ways it will be extended are found mostly in the minds of users of the software.
If you don’t understand what I mean by those statements, take a look at the diversity of innovative visualizations created in Tableau. I suspect that the founders of Tableau have been surprised that its drawing canvas and computational elements have allowed the data artisans to create such a wide array of functional art. Much of the same could be said for Alteryx. The data wranglers have twisted and contorted Alteryx in a nearly unimaginable web of workflows and analytic apps that will continue to grow long into the future. Software that has engaged users coupled with clever and expandable functionality will need good software testers. Products like these give us plenty of opportunities to be software testers.
One of My Early Lessons in Software Testing
In one software testing example from a long time ago, I completed a detailed testing example that allowed me to peer into a Microsoft software product. I identified an important memory limitation that their product development team denied existed.
It was a couple of months later that they finally relaxed their egos enough to understand what I was trying to tell them with my example. They finally admitted I was right, and they also told me: “If you can do that without being on the development team, you need to be working for us”.
What this means is that some smart software companies will find ways to include software testers from their user base. This makes a lot of sense since they get to distribute the testing load to those people that have the need and the passion for using the software. By coupling excellent development teams with their software user base, the successful software companies will benefit in many ways from this collaboration.
If you can do that without being on the development team, you need to be working for us. – Microsoft, 1987
Developer Benefits Received From Software Testers
As most software developers know, when you write a code that requires input, it seems logical to you that the software users of your code should know the types of allowable input. This seems logical to you as the developer because you have been writing and testing the code and you generally provide only valid input your code when you are doing your own testing.
What developers sometimes fail to realize, however, is that when software testers look at a data input input sheet or some sort of software configuration setting for the first time, they may have no idea what to put into the input fields or how to properly use the settings. In many cases, the users might enter information that is incompatible with the desired type of input.
In these cases, the software fails and the tester gets to report to the developer that the code failed. This failure might have occurred because the documentation was insufficient or some other type of issue occurred that you never thought about while writing the code. At times this type of “code-failure” experience might be a pain in the rear for the developer to address, but it will definitely make the code more robust and usable by forcing better documentation and/or the implementation of input validation methods.
By having an external team of testers that are familiar with your software, you don’t have to train them on how to use the program. You simply give them an overview of the new features and ask them to test these features. Usually what emerges from this testing are insights that will make the product better and will help the developer see how their software is being used.
Example 1 – A Recent Example of Software Testing
As a software tester, we are bound by confidentiality agreements for obvious reasons. Companies like to release new product updates/features on their own terms and when they are ready. New software capabilities are of huge importance and can provide the fuel that leads to new sales and customers. For this reason, it isn’t always possible to provide details of the testing that was conducted. Such is the case with this example.
In a recent example of software testing, I decided to do a thorough evaluation of a new, very important product feature. The nature of my testing was not necessarily outside the norm for using this product, but it did require a commitment in time and focus that went beyond the norm. In other words, most software testers would not engage themselves to the same degree that I did on this example.
My testing was designed to span a range of usage from normal data sets (up to a few million records of data) to some pretty big data sets (billions of records). By having the willingness to perform such detailed and time-consuming testing, both the software company and I learned a great deal about software capabilities, performance, and component interactions. Some of these findings would likely not have been identified if it weren’t for this style of intensive testing.
By being a software tester, I was able to help the development team learn at least two very important insights about their new product feature. That result makes me happy and gives me the motivation to continue to do this work even though it is essentially an undocumented component of my career.
One additional benefit of being a tester is that you get to see and use the new product features before other people do. You may even get to help shape the future of a product by providing your time and expertise to the software company. In many ways, you have the possibility of becoming an important visionary that can help shape the future of a software product.
By having the willingness to perform such detailed and time-consuming testing, both the software company and I learned a great deal about software capabilities, performance, and component interactions.
Example 2 – The Inaccurate Brake Pad Indicators
I bought a new truck in Sept 2019. It has a lot of features, many of which are software-driven. With each software-enabled function, there are potential problems that can occur. Each problem that does occur can be costly for the manufacturer to rectify. Some manufacturers can send wireless instructions to fix the problem, while in other cases the vehicle has to be returned to the dealership.
Within three months of owning the truck, I knew that there was a software problem. I detected the problem by looking at the brake pad wear indicator screen. Figure 1 shows several snapshots I took over time, beginning in late Dec 2019. Please ignore the fact that only true data dorks would take this many pictures of an issue, but I really wanted to create a model of when the brake pads were expected to fail!
I know from a lifetime of experience that brake pads last a long time. I’d say that they can last 60K to 100K miles or more. When I first noticed that the front pad indicator was at 45% and the rear at 66% with only 4K miles on the truck, I knew that there was a problem. For this reason, I snapped a series of pictures and watched the lifetime indicators drop down to 27%/56% within the next thousand miles of driving.
At that time, the only questions in my mind were these: “How long will it take the manufacturer to notice this problem, how will they fix it, and how long will it take to fix?”
Well, it took about 4 months for me to be notified and the fix to occur. I just got a letter saying that the brake pad indicators are inaccurate. The message indicates that the indicators are off by a factor of 16! The fix required a trip to the dealership.
Being a software tester, my brain immediately translated this written message in this way. The algorithm they used to estimate pad lifetime is based on pad thickness. This algorithm may have had some sort of conversion error, with a factor of sixteen either being missing or incorrectly being added to the lifetime estimation equation. I could speculate that the pad thicknesses are measured in 1/16th of an inch increments and the developers forgot about units being in 1/16th increments when they wrote the estimator. Of course, the reason could be something completely different from this, too.
This example demonstrates one of those unspoken benefits of being a software tester. You automatically think in terms of algorithms and you become really good at spotting errors when they arise.
In this case, the suspect algorithms was not sufficiently tested. This mistake will likely cause big financial losses for the manufacturer and frustration for the customers. These are just two reasons why software testing is so important.
Final Thoughts
By being a software tester, you will reap the benefits of this work in ways that you cannot imagine. I strongly encourage young data workers to get involved with the software companies that are building your favorite products.
Even if you are not a software developer, you will help the development team by sending them your insights about their product that you achieved during this type of testing. You will learn a lot and help the company develop better software.
Thanks for reading!
Pingback: Results of Big Data Experimentation With #Alteryx and #Tableau (Part 1) | Data Blends