Software testing. What you need to know.

 

Testing a software product can be a daunting experience since there is a wide range of possible outcomes, many of which can be detrimental to the aspirations of your growing company. More daunting than this however is not testing at all or perhaps not testing well, resulting in issues surfacing when the product is already live and in the hands of your paying (and vocal) customers.

By the end of 2017, financial loss due to lapses in software quality assurance stood at USD1.7 trillion, according to TechRepublic, with impactful defects catching out even the big names like Yahoo, Tesla, and Airbus. Evidently, software testing can hurt a company both before and after release if not done right.

There are various ways to go about testing the myriad of diverse kinds of software products, made more complex when you consider the business goals and constraints of the project at hand, so let’s figure out where to start:

1. Develop a testing strategy

A test strategy can be viewed as a logical starting point to align all the stakeholders on the desired outcomes of any testing to be undertaken.

It is the first step on the path to capturing and communicating all the elements and factors of the company and its resources that will influence the strategy to be undertaken for software testing.

It should paint a picture of what the foundational aspects of the company’s product or solution are, its appetite for risk, balanced with the need for velocity and speed to market. A good test strategy will enable you to understand the magnitude of each factor’s contribution to the overall picture and therefore allow more effective resource and schedule and overall approach planning to occur.

With a solid strategy, you’ll know which areas may need more focus, time, and/or resources, and any over-arching questions that may occur around the “why” (we test this way) should be answered.

2. Draw up a testing plan

Once you have a test strategy in hand, you need to come up with a plan on how to run the actual test levels, phases, and cycles.

So how exactly is a test plan different from a test strategy? Well, while a test plan will cover the type of tests and the specific scope of product features and functionality at a project or program level, a Test Strategy should cover the over-arching organisational approach to testing (best practice).

A large part of the test plan is devoted to communicating WHAT will be tested and WHEN, creating a schedule with time slots allocated to different phases of testing (including planning and execution for each) along with the testers or teams who will run each phase. It also includes other procedural information such as the deliverable documents related to each test, the team responsibilities and escalation paths, and the guidelines for the definition of success for each test phase.

A clear test plan will help you plan the order of events, avoid repetition, and overlaps while assisting with any estimations and resource scheduling challenges.

3. Ascertain your test cases

Think of test cases as the results expected when specific actions are taken using the software. In this pre-testing phase, you need to note down every capability you’ve built into the product and establish the benchmark expected outcome that would occur when the user exercises each feature in a specific context.

For starters, test cases enable testers to know whether the results are a pass or fail, especially in cases where the result is multi-faceted or quantitative in nature. Having your expected results of the test cases clearly laid out will also help you get a clearer picture of what to prioritise.

Test cases allow consideration for the incorporation of end-user feedback when refining the product BEFORE the solution is delivered, which helps to make the entire process more value-driven.

For instance, the test team may describe in a test case a result that they expect (based on their interpretation of the requirements) whereas the end-user may have a different expectation. Test case and coverage reviews can drive out these ambiguities and inconsistencies before they have a chance to progress further down the development lifecycle.

Not only will you determine the most important functions, to begin with when testing, you’ll also iron out any of these disparities between the end-user and team’s expectations.

4. Prepare your test data

In various software tests, data must be fed into the system to be acted on according to a given set of rules. Test data can either be created (mocked) or taken from production, sanitised, and ‘washed’ of identifiable data and used carefully and appropriately. The approach here is dictated by the requirements of your project and governed by any laws or guidelines in your country regarding the use of customers’ data in this way.

Test data development will often run concurrently with test case development. Test data can help you to easily track and differentiate the samples you are applying varying actions to. For example, you may have a list of fictional customers with nationality, age, and other attributes included.

Say you apply different registration rules, with the age of eligibility varying from one country to another. This means you’ll need to have multiple fictional customers whose data meets different combinations of criteria. By doing so, you can see how two customers of the same age, but different nationality may be handled differently by the system.

The creation and use of test data need to be carefully thought out in alignment with the tests being undertaken. For example, using a customer name of ‘Dummy Dummy’ would not be advisable, since you would not be able to differentiate whether the first or last name is being used by the solution at various points. In many cases, it is an integral part of ensuring that the described logic is being adhered to.

5. Establish your testing environment

Many applications are built to be cross-platform, designed in such a way that users with different devices and operating systems are expected to have the same user experience. However, achieving this result takes careful planning and testing across many scenarios and is reliant upon the correct setup and availability of test systems, devices, and environments.

Every necessary piece of hardware and its accompanying operating systems and environment may need to made available during testing if you are to find out how well they operate under those circumstances. There are also tools (and vendors) that can assist you with these requirements, using appropriate testing techniques, virtual devices, or device farms for example.

A product might perform a certain function on one operating system properly, then fail on another OS. The issue could be caused by a stark difference in the way these operating systems make certain resources available, or it could just be a flaw in the product’s code.

Such concerns heighten when you have a product built using various languages, with some not being supported by a particular operating system. Some pieces of the code may work fine while others may not, bringing faults in the collective function they serve.

Conclusion

Testing is a continuous process and so are many of the initial steps. Test plans can be altered as new defects are discovered and environment setups could change too as the platforms that should be supported by the product evolve.

There is also the possibility of a change in stakeholders’ requirements or focus or in those of the customers too. These realities make testing particularly stressful if you are not confident in your approach, and the stakes nevertheless remain high for any business with grand ambitions for its products that rely on customer satisfaction.

All of these considerations come as second nature to experienced testing professionals, it is embedded into the way they think. Testers do not look to create issues, but instead, they seek to uncover and eradicate them from the end-product and ultimately increase your customer’s satisfaction with your software solution.

If you are looking for professional help in determining the ideal frameworks that will provide the maximum return on investment throughout the software development and testing of your solution, get in touch with the team via [email protected]