Category Archives: General Testing Questions

Negative Test Cases

Posted on by
Reply

What are negative test cases?

If you get faced by this software testing question during an interview, first define positive and negative test cases, and then give a few examples.

Positive test cases test that the application does what it should.

Negative test cases test that the application doesn’t do what it shouldn’t do.

Huh? Not too clear?

Basically, positive test cases means feeding in valid input into the application, and checking that the app computes the right result, or otherwise behaves in the correct way.

Negative test cases, on the other hand, feed the application invalid input, and generally try to trick the app in different ways. The point is that the app should not do the “correct” thing – the thing that it would do in the positive case.

Examples?

Negative Test CasesImagine a basic login screen. The login screen should only allow you to log in when you’ve supplied the right username and password.

The positive test case would check that the application allows you to log in when you give it a valid username and password.

A negative test case would check that the app doesn’t allow you to log in when you give an invalid username and password, or when you don’t supply any password, or when you supply a password containing characters that the system doesn’t accept as password characters.

Example 2:

Imagine a function written in a dynamically typed language that computes the square of a number.

The positive test cases would feed a number to the function, and check that the result is indeed the square of that number.

The negative test cases would depend on the specifications for how the function should behave for all input other than integers. Should the app throw an exception, return the input value, try to compute the square using operator overloading, return some constant, or is the behavior undefined? The specifications have to tell us this. However, at the least, we can test whether the function will crash or throw a fault or otherwise do something that is clearly undesirable. So the negative test case might feed the function a string, an object reference, a list, or any other non-number value.

How Do You Know When To Stop Testing?

Posted on by
Reply

How Do You Know When To Stop TestingHow do you know when to stop testing?

Here’s how you should answer in case you are faced with this software testing question during an interview.

It’s one of the basics of theoretical computer science that there is no algorithm for predicting how an arbitrary program will behave with a given input (aside from running that program with the input).

What does that mean for the above software testing question?

It means that there is no hard answer to the question for when you should stop testing. The answer will depend on the application you are testing, and probably, on a lot of other factors not directly having to do with the application. Basically, there might be heuristics (ways to approximate) for the right amount of testing, but there’s no way to know the true answer, except for very small toy programs.

What are some of these heuristics?

First of all, if the application comes with clearly specified requirements, we cannot stop testing until those requirements have been validated.

Secondly, a more complex application will always require more testing than a simple application. This means that even when the functionality has been validated on the surface, we should push to break the complex application harder, because the chances of it containing bugs are greater.

Third, we should consider what the application does. A guidance system for the space shuttle should be tested much more thoroughly than a new blogging platform. Can you imagine the catastrophic, horrifying failure that a bug in the space shuttle could cause?

Fourth, if there are additional, non-functional, requirements, such as performance or security, we cannot stop before validating those requirements also.

Fifth, if the application will run in an environment that is different from the development environment, we cannot stop testing before we have run the app in the deployment environment.

What about some business realities?

We should stop testing if tester and developer time can be used more productively elsewhere (new development, or testing of another company product that takes priority for some reason).

We should stop testing if the testing deadline has passed, and it’s time to deliver the product.

We should stop testing if the testing budget has been depleted.

In conclusion, if you are asked how you know when to stop testing, you should say that the answer depends on the application you are testing, and the business environment you are in. Both will dictate when you have found the application to be satisfactory.

Load Testing Metrics

Posted on by
Reply

Load Testing MetricsWhich load testing metrics would you consider and why? This is an advanced software testing question that you might come up agains in case your interview has been going well. There isn’t a single correct answer; instead, your interviewer will be looking to see if you have a good understanding of software engineering, from both the technical and the business side.

So, how should you answer?

First, explain that there are lots of possible load testing metrics. Some of them are standard, but which metrics you ultimately focus on will depend on the specific requirements of the application.

Second, list a number of metrics: requests per second; bytes per second; response time; errors returned; number of concurrent users; memory usage; number of database queries.

Finally, focus on a couple of these metrics, and explain in which situations you might consider using them.

Example 1: for a web application, the number of concurrent users is one of the key metrics from the business perspective: if the application supports fewer users than are actually trying to access the site, valuable business is being lost.

Example 2: the response time is a valuable metric to asses the user experience (if this is typically longer than a second or two, users will get frustrated and will stop using the application).

Example 3: the response time might be very short, which is good. However, this might be because the number of errors returned is very large (perhaps the application throws an exception immediately on a new request), which would again make for a bad user experience.

Example 4: in case the number of database queries is very large, performance might be improved through caching.

In summary, when asked to explain which load testing metrics you would focus on, explain that there are lots of relevant metrics, and that the specific metrics should be determined by the need of the system and of the application. Then, illustrate with examples.