What Software Tests Should Not Be Automated?

This is a slightly tricky and more advanced software testing interview question. The reason it’s tricky is that in the last 10 years there has been so much emphasis on the benefits of continuous, automated software testing. From that, it’s very easy to assume that automated testing is simply superior to manual testing in every way. However, this answer would indicate a lack of experience in the software testing industry.

So, how should you answer this question? First, you should acknowledge all the benefits of automated tests: they are easily repeatable; they test things that humans cannot test reliably (or cannot test at all); they can be run continuously with every change in the code.

However, it’s important to realize that automating tests has its own cost above a manual test that will be repeated only once (or a few times). Because of this, there may be situations in which it doesn’t make sense to automate. Here are a few examples:

  1. When you’re testing the GUI, but that’s changing all the time
  2. When you’re testing code, but the API or the implementation is changing all the time
  3. When you are under time pressure, and automating tests prevents you from carrying out important manual tests

Also, it’s important to mention that because test automation can be hard in many scenarios, tests sometimes become weakened so they can be automated. This is clearly a bad thing.

Finally, you can conclude by emphasizing that it’s always a good idea to formalize tests (for example, by writing test cases), but that’s different from necessarily automating those tests.