Validation Testing (QA)

These tests focus on assessing whether the software performs the intended functions correctly. It is checked that each of the requirements is well implemented.

If we focus on the set-top box scenario, it involves verifying that functionalities such as channel changes, subtitle activation, programme guide, etc., act as expected.

These tests highlight the importance of having a battery of test cases that perfectly meet the project requirements. Similarly, it is vital to update them periodically during the life cycle of the device.

These are tests that evaluate the interaction between the different modules or components of the software.
It is verified that the functional blocks communicate correctly and that none of them adds issues or regressions to the rest.

In the case of set-top boxes or TVs, modules such as power supply, HDMI, conditional access, user interface, etc. are verified.

We also include in this type of tests the interaction scenarios between applications. An example is the correct acquisition and release of resources, such as the audio module, for launching and closing the corresponding app.

These tests check the usability of the device and the user experience.

They check if the user interface is intuitive, if the navigation between the different menus and screens is clear and if the RCU actions are coherent.

In this type of testing there is a component of subjectivity (not everyone perceives the concept of usability or user experience in the same way). It is therefore necessary to find a good balance between the Marketing/Design teams, the testers with their accumulated experience and the developers.

Performance tests analyse how the software behaves under load or stress. Key Performance Indicators (KPIs) are established, usually from previous projects, and measurements are taken of different parameters: zapping time, memory consumption, CPU occupation percentage, app opening time, etc.

These tests are launched once the software has reached a certain level of development maturity and always prior to any deployment to the end user.

These regression tests are designed to ensure that issues fixed in past software releases do not recur.

They are part of the maintenance test suite of the device and are incremental as successive software releases are deployed.

They are relevant to show that the quality of the software is maintained and build trust between all parties involved in the project.

Finally, upgrade testing focuses on ensuring that new software versions are applied smoothly and do not cause errors in the operation of the device. It must be ensured that the update is as transparent as possible to the user, and if not, that there are the necessary messages and notifications so that the user is aware at all times of what is happening.

In this type of test, other maintenance operations such as reboots (useful for freeing up memory if the system is saturated) or factory resets (for scenarios where a reboot is not sufficient to return the device to its original operating state) are also verified.