1. How do/will you you ensure no bugs?

The short answer to this question is that we can’t; and trust me, I really wish we could. If only it were possible.

The long answer is that to ensure something is to make certain of; and no amount of testing can create certainty that zero bugs exist in any product. In the domains of science and philosophy this is a well accepted concept and often referred to as “you can’t prove a negative” and relates to the problem of induction, also known as the black swan problem.

Through testing if I find a bug, I can easily demonstrate that a product doesn’t work. But if I perform a test and am unable to find a bug, this only shows that the product can work to some extent, at that point in time, under specific conditions. Change any of those conditions and I cannot know for sure what’s going to happen next or how the product will behave. This is one of the things that makes testing difficult. I could test forever, there an almost infinite number of variables to change with an infinite number of combinations. So I have to develop strategies to choose what to test and under what conditions so that I have the best chance of learning something important about the product. In the end, no matter how many functions I check, how many different combinations I try, and how many scenarios I test I can’t be sure the next thing I try will not reveal a bug.

At a certain point we will have achieved the project’s testing goals. This might be to mitigate the risks identified, checked all the acceptance criteria, used all the time we had, and/or created and ran the automated checks we planned to. At this point, we will stop testing as continuing will eventually become prohibitively expensive and offers a diminishing return on investment for the client.

2. How do you assure/ensure quality?

This is a tricky question; it’s often directed toward myself as a tester as if to say testers are responsible for ensuring quality. This assumption isn’t correct. No amount of testing can create a quality product; quality has to be built in - and that’s the mindset we embody at Inlight.

The answer also depends on whether I’m being asked to ensure (to make certain of) or assure (to make someone confident of) quality. As mentioned above, we can’t ensure a perfect product but we can take steps that will hopefully assure the person asking the question the product is of high quality. Inlight has a reputation for producing high quality products but a significant amount of the work to do this is not performed by a tester.

At Inlight we use an agile approach to producing software called AgencyAgile. This approach aims to set up a project in such a way that creates a universal understanding between the developers, testers, designers, project managers, product owner, and client stakeholders. This helps us to produce a product that looks and behaves in a way that the client and we intend.

In addition we follow a development workflow which emphasises quality through activities such as peer reviews and unit testing. It is largely through activities such as these that we produce a high quality product much of which is not performed by a tester.

A tester will also perform many testing activities including questioning, probing, experimenting, and checking in an effort to discover anything that might be improved or has managed to get through our other quality assurance measures.

3. How do you plan to do (user / acceptance / automated / security / load / penetration / stress / integration / e2e / x) testing on this project?

There are a few of things that make these types of questions difficult to answer. The first being it’s hard to know exactly what someone means when they say, for example, UAT (User acceptance testing). Everyone seems to have their own definitions for may of these terms so unless I’m able to follow up to get a better understanding of the types of activities and scope I’m being asked about it can be very difficult to provide an answer.

The second issue I have in answering this question is that the question has a built in assumption that we will be doing a certain type of testing activity. This may not be the case and I will often need to find out a lot more about the project in order to make that assessment. Having said this, there are often situations where my answer is that it’s unlikely that we’ll need to do any of x testing activity. Take load testing on a static website for example; most static websites we build will have no trouble handling hundreds of thousands of pageviews a day. If the most page views a site has previously ever received is under 100,000 on the busiest day of the year then it’s probably much more worthwhile spending testing time on identifying issues that are much more likely to affect their users.

Generally the testing activities that are performed on any specific project are decided once I have had the opportunity to model the product to discover and learn about all the different aspects of the product such as the structure, functions, data, interface(s), platform, operations, and timing. Then I would perform a risk analysis to identify what parts of the product to test under what conditions within the timeframe and resources available to me. At this point I will have a good understanding of the testing activities I would like to perform.

So there you have it. Some of the common testing questions I see and my responses. I hope you find this article useful and it helps provide a better understanding of testing and quality in software projects.