There Is No “Non-Functional”

I keep running into testers, and others, who like to categorize testing into functional and non-functional aspects. This needs to stop, in my humble opinion.

Functional testing is an approach, not a type. That means it has a very broad mandate because it’s a very broad term. It covers three main techniques of testing: unit, integration, and system. Within those techniques you will have various types of testing that you perform and that operate at the level of the technique.

Some testers will tend to lump types of testing, like compatibility testing or load testing, under non-functional testing. But let’s think about this for a second. Compatibility testing is about how your application functions in a different context. Load testing is about how your application performs its functions under a determined amount of load.

These are different types of testing you will employ within a functional approach.

A Type of Testing is Inherently Functional

To drive the point home, let’s consider security testing. This is about how your application functions in a secure manner. The idea of portability testing simply means I test how my app functions when I port it to a different platform.

If anything, types of testing like performance, security, and compatibility should be listed as qualities of a given application rather than lumped under “non-functional.” In the post Non-Functional Requirements: Do User Stories Really Help?, Rachel Davies seems to agree with this point even saying that “non-functional is probably a bad name” — yet she continues to use it. (And, by the way, there’s no “probably” about it. It is a bad name.)

The reason I harp on this is because the word “functional” simply speaks to how something works or operates in a given context. Thus everything is functional testing to some extent. Even usability or accessibility. For example, you can ask “how does the site function (work) for a user with vision difficulties?” In the world of mobile development, the breakpoints in my web application will dictate how the application functions on different devices that a user may be employing. Even if all that changes is the appearance, it’s still an aspect of functionality because the user still has to be able to find menus, use navigation, select products, and so forth.

Consider the article Functional vs Non Functional Requirements where Mark Debono says:

Simply put, the difference is that non-functional requirements describe how the system works, while functional requirements describe what the system should do.

Simply put, huh? Perhaps the author is trying to suggest a difference between intent and implementation? If so, then the above quote is a perfect example of where the term “non-functional” confuses the issue. Notice an example of a “non-functional requirement” given in that post is for a hard hat, with the idea that it “must not break under pressure of less than 10,000 PSI.” How is that non-functional? What that’s really saying is that “the hat must continue to function as a protective device with pressures up to 10,000 PSI.”

In this context, what we call “non-functional requirements” serve to place restrictions on a given application or product in terms of where and how it continues to function and where that functionality breaks down. If I perform stress testing on an application because the statement given is “it should accept up to 10,000 concurrent connections all searching for non-filtered data”, well then what I have there is a specific statement under which my application should remain functional for its users and I have a possible restriction on when that functionality is potentially going to break down.

A Testable Context is Inherently Functional

I think where “non-functional” got started is that people didn’t put numbers in a testable context that was functional in nature. So, for example, if I had been given the requirement “it should accept up to 10,000 concurrent connections”, that still is functional but it’s also somewhat divorced from a specific context except the broad. The qualifier I added (“all searching for non-filtered data”) provides a specific functional context.

A brief anecdote: I did some testing for game emulators that were being provided by Activision and others to play older games on modern PCs. I would often get what were called “non-functional requirements” for “efficiency”. These were basically stating how well the application should utilize resources, like CPU cycles, disk space, memory, bandwidth, etc. That can certainly seem “non-functional” when you consider it in the abstract like that. What I did, however, was get people to state their requirements more like this: “The application should use no more than 40,000 CPU cycles to emulate the approximate speed of a 486DX chip running at 40 MHz.” Thus when the profile for a given game said to use the “486 compatibility” I could check how the application functioned in terms of the CPU cycles. There was nothing non-functional about that.

What about availability?

This is often lumped under non-functional requirements. Availability generally refers to the uptime, meaning the time that an application and/or its environment is operational and thus available for use. Here the functional aspect is not necessarily your application as it is the functionality of the environment it runs within. But it is still a functional aspect. I wouldn’t bother calling it “a non-functional requirement.” Even from a testing standpoint, what I would doing is exercising the system continuously — using its functionality — and making sure that the system remains available.

How about scalability?

This is another aspect that’s treated as non-functional. The idea here is that your application (and its environment) has to scale based on changing conditions. Thus a so-called “non-functional requirement” might talk about the ways the system can scale up: such as adding hardware, increasing virtualization, etc. That’s in the abstract, though. Once you get into testing the scalability, you start exercising functionality and see if the extra hardware or virtualization allowed that functionality to keep working, whether that be in the context of load, volume, security, data integrity, etc.

What about robustness?

This is yet another non-functional aspect, according to many testers. Well, let’s consider what “robust” even means here. It tends to mean that the application can handle error conditions, such as invalid inputs or missing resources, in a graceful way. Sounds like functionality to me.

Have I made the point yet?

If none of the above sways you, look up the word non-functional. You’ll generally find some definition like “not having any particular purpose or function.” Is that really what we mean? If that’s not what we mean — and it isn’t! — then why use the term? Why re-purpose an existing term that has a clear meaning just to break it up into categories of testing?

So, modern testers, don’t just follow a pre-existing convention because you were taught it. Excise the term “non-functional” from your vocabulary. If you have project managers or technical business analysts that use the term, educate them as to why they shouldn’t.

About Jeff Nyman

Anything I put here is an approximation of the truth. You're getting a particular view of myself ... and it's the view I'm choosing to present to you. If you've never met me before in person, please realize I'm not the same in person as I am in writing. That's because I can only put part of myself down into words. If you have met me before in person then I'd ask you to consider that the view you've formed that way and the view you come to by reading what I say here may, in fact, both be true. I'd advise that you not automatically discard either viewpoint when they conflict or accept either as truth when they agree.
This entry was posted in Communication, Testing. Bookmark the permalink.

3 Responses to There Is No “Non-Functional”

  1. richard says:

    I think your argument is fairly interesting but I’m also sure that there is a difference between compliance and intrinsic quality and non functional requirements really are concerned with intrinsic quality not compliance.  Data integrity, maintainability, security, performance, environment, human, cost, people are all good examples of non functional requirements – i.e. it must comply to the regulations but must do so in a way which is usable for the context, functionally a button can states a and b, but its how it transitions between those states, not the states themselves which sometimes requires further testing.

     

    In reality its very difficult to perform non functional testing effectively with the typical resources you have available in business and requires a different mind set, therefore, non-functional, or rather intrinsic quality often suffers as a result, which is why I felt the need to comment as in that sense to undermine non-functional requirements is actually quite detrimental to the promotion of the types  testing and constraints within this domain, which people typically refer to under the term

     

    • Jeff Nyman says:

      Thanks for your comment! Much appreciated. I would prefer to hear from people who disagree.

      Speaking to your points:

      My view is that if some aspect is concerned with “intrinsic quality” then call them “intrinsic quality requirements”, if you prefer. That would be still be better than “non-functional.” This is where I was speaking to qualities or constraints, as just one example, which can be stated as internal factors.

      Regarding this point, let me quote you in full:

      “Data integrity, maintainability, security, performance, environment, human, cost, people are all good examples of non functional requirements – i.e. it must comply to the regulations but must do so in a way which is usable for the context, functionally a button can states a and b, but its how it transitions between those states, not the states themselves which sometimes requires further testing.”

      When you say “usable for the context”, that implies functionality of some sort, whether that’s a system or a human. People and machines and applications function in a context. Going with your example, how something transitions between states is without question a functional aspect. It’s how the thing transitions (functions) between states. When I click the “Buy” button on Amazon and it transitions me to a shopping cart, that transition is functionality — as our the states that “Buy” button may have been in at various times.

      Performance is something I covered in the post: it’s absolutely a statement of functionality under various operating conditions. This is the case whether you are talking about a REST service under load or a jet turbine under pressure or car brakes on a slick road.

      Even human factors can be stated functionally in that how our business functions (such as getting releases out, continuing an assembly line) is a factor of the human elements within it.

      Regarding this point:

      “In reality its very difficult to perform non functional testing effectively with the typical resources you have available in business and requires a different mind set, …”

      I would argue this difficulty comes from the terms people use and the way those terms constrain their thinking. Consider back in the days of Competitive Engineering or even the PLanguage concepts of Tom Gilb and others. That entire movement was essentially predicated upon the idea of considering everything functional in some aspect or to some degree that can be quantified. The industry has moved away from that and, personally, I think that’s a bad thing.

      I have yet to see where anyone can unambiguously point to why “non-functional” is a term worth promoting — particularly given what the word means — rather than using other terms that are not only more indicative but also more situationally accurate.

  2. donal says:

    Yep, couldn’t agree more. I think it is a very poor use of English. The division between functional and non-functional that computer-geeks are attributing to the word functional is a mis-attribution. The usage is jargon, it’s difficult to explain, the division is hard to maintain, the use of ‘functional’ versus ‘non’-functional is misleading to anyone who speaks English. Plus, no-one who uses a software or system [and why build something if it can’t be used?] is going to consider it appropriate to describe that system as ‘functional’ when it’s so inefficient in returning a query to render it practically useless. (I think I just described the opposite of Google search.)

Leave a Reply

Your email address will not be published. Required fields are marked *