I keep running into testers, and others, who like to categorize testing into functional and non-functional aspects. This needs to stop, in my humble opinion.
Functional testing is an approach, not a type. That means it has a very broad mandate because it’s a very broad term. It covers three main techniques of testing: unit, integration, and system. Within those techniques you will have various types of testing that you perform and that operate at the level of the technique.
Some testers will tend to lump types of testing, like compatibility testing or load testing, under non-functional testing. But let’s think about this for a second. Compatibility testing is about how your application functions in a different context. Load testing is about how your application performs its functions under a determined amount of load.
These are different types of testing you will employ within a functional approach.
A Type of Testing is Inherently Functional
To drive the point home, let’s consider security testing. This is about how your application functions in a secure manner. The idea of portability testing simply means I test how my app functions when I port it to a different platform.
If anything, types of testing like performance, security, and compatibility should be listed as qualities of a given application rather than lumped under “non-functional.” In the post Non-Functional Requirements: Do User Stories Really Help?, Rachel Davies seems to agree with this point even saying that “non-functional is probably a bad name” — yet she continues to use it. (And, by the way, there’s no “probably” about it. It is a bad name.)
The reason I harp on this is because the word “functional” simply speaks to how something works or operates in a given context. Thus everything is functional testing to some extent. Even usability or accessibility. For example, you can ask “how does the site function (work) for a user with vision difficulties?” In the world of mobile development, the breakpoints in my web application will dictate how the application functions on different devices that a user may be employing. Even if all that changes is the appearance, it’s still an aspect of functionality because the user still has to be able to find menus, use navigation, select products, and so forth.
Consider the article Functional vs Non Functional Requirements where Mark Debono says:
Simply put, the difference is that non-functional requirements describe how the system works, while functional requirements describe what the system should do.
Simply put, huh? Perhaps the author is trying to suggest a difference between intent and implementation? If so, then the above quote is a perfect example of where the term “non-functional” confuses the issue. Notice an example of a “non-functional requirement” given in that post is for a hard hat, with the idea that it “must not break under pressure of less than 10,000 PSI.” How is that non-functional? What that’s really saying is that “the hat must continue to function as a protective device with pressures up to 10,000 PSI.”
In this context, what we call “non-functional requirements” serve to place restrictions on a given application or product in terms of where and how it continues to function and where that functionality breaks down. If I perform stress testing on an application because the statement given is “it should accept up to 10,000 concurrent connections all searching for non-filtered data”, well then what I have there is a specific statement under which my application should remain functional for its users and I have a possible restriction on when that functionality is potentially going to break down.
A Testable Context is Inherently Functional
I think where “non-functional” got started is that people didn’t put numbers in a testable context that was functional in nature. So, for example, if I had been given the requirement “it should accept up to 10,000 concurrent connections”, that still is functional but it’s also somewhat divorced from a specific context except the broad. The qualifier I added (“all searching for non-filtered data”) provides a specific functional context.
A brief anecdote: I did some testing for game emulators that were being provided by Activision and others to play older games on modern PCs. I would often get what were called “non-functional requirements” for “efficiency”. These were basically stating how well the application should utilize resources, like CPU cycles, disk space, memory, bandwidth, etc. That can certainly seem “non-functional” when you consider it in the abstract like that. What I did, however, was get people to state their requirements more like this: “The application should use no more than 40,000 CPU cycles to emulate the approximate speed of a 486DX chip running at 40 MHz.” Thus when the profile for a given game said to use the “486 compatibility” I could check how the application functioned in terms of the CPU cycles. There was nothing non-functional about that.
What about availability?
This is often lumped under non-functional requirements. Availability generally refers to the uptime, meaning the time that an application and/or its environment is operational and thus available for use. Here the functional aspect is not necessarily your application as it is the functionality of the environment it runs within. But it is still a functional aspect. I wouldn’t bother calling it “a non-functional requirement.” Even from a testing standpoint, what I would doing is exercising the system continuously — using its functionality — and making sure that the system remains available.
How about scalability?
This is another aspect that’s treated as non-functional. The idea here is that your application (and its environment) has to scale based on changing conditions. Thus a so-called “non-functional requirement” might talk about the ways the system can scale up: such as adding hardware, increasing virtualization, etc. That’s in the abstract, though. Once you get into testing the scalability, you start exercising functionality and see if the extra hardware or virtualization allowed that functionality to keep working, whether that be in the context of load, volume, security, data integrity, etc.
What about robustness?
This is yet another non-functional aspect, according to many testers. Well, let’s consider what “robust” even means here. It tends to mean that the application can handle error conditions, such as invalid inputs or missing resources, in a graceful way. Sounds like functionality to me.
Have I made the point yet?
If none of the above sways you, look up the word non-functional. You’ll generally find some definition like “not having any particular purpose or function.” Is that really what we mean? If that’s not what we mean — and it isn’t! — then why use the term? Why re-purpose an existing term that has a clear meaning just to break it up into categories of testing?
So, modern testers, don’t just follow a pre-existing convention because you were taught it. Excise the term “non-functional” from your vocabulary. If you have project managers or technical business analysts that use the term, educate them as to why they shouldn’t.