How to test software well?

Question

Well, I was thinking how sites with great access users do tests on their software and test them for final release, why anything or error of code can lead to gross system failure is interruption at a general level, which would cause an ugly loss.

I read a little about test automation in PHP with Phpunit however I stayed open with the following general questions.

• What would be a good way to test the software?

• It would be interesting to use manual or automatic testing processes?

• There are criteria for this?

• How to define that that software is fit for the end user?

Answer 1

State of the art of the test

Mastering code and ensuring its reliability has become one of the top priorities for all developers facing complex, increasingly rich architectures.

Certain tools are able to help us test the Web Applications to meet all our expectations in terms quality development.

Integration test

Integration testing is the phase of software testing in which modules are combined and tested in a group. The purpose of the integration test is to verify the functional, performance and reliability requirements in system modeling. With it it is possible to discover interface errors between system components.

Unitary Test

Unit testing is the entire test application on the input and output signatures of a system. It consists of validating valid and invalid data via I/O (input/output) being applied by developers or test analysts.

A unit is the smallest testable part of a computer program. In procedural programming, a unit can be either an individual function or a procedure. Ideally, each unit test is independent of the others, which enables the programmer to test each module separately.

Tools

• Mocha
• Junit
• Jasmine

Functional Test

Ensures application stability by mimicking the user path in the browser. Allows checking if the application is working normally and can alert for possible regressions.

Tools

• Selenium
• Capybara
• Protractor

Multi-browser test

To ensure cross-browser compatibility of your application, it is necessary to run functional tests on multiple browsers and platforms.

Tools

• Saucelabs
• Browserstack
• Webdriverjs

Safety Test

Known as Security Testing, it allows you to assess vulnerabilities in applications and services against different types of security attacks - such as Denial of Service Attacks or Man-Attackin-the-Middle - and discover new vulnerabilities before they are exploited by attackers.

Tools

• Skipfish
• OWASP Zed Attack Proxy

Performance test

Performance Testing or Performance Testing consists of assessing the responsiveness, robustness, availability, reliability and scalability of an application, depending on the number of simultaneous connections, evaluating their performance in high workload and considering their behavior in normal circumstances.

Tools

• Agileload
• Webpagetest
• Google Page Speed Insight

Stress test

Stress testing consists of subjecting the software to extreme situations. Basically, the stress test is based on testing the software limits and evaluating its behavior. Thus, it is evaluated up to when the software may be required and what failures (if any) arise from the test.

Tools

• Gremlins.js
• Apache Jmeter
• http_load

Integrating everything

It is not enough simply to test everything and not integrate automatically, users cannot be without the system.

Zero Downtime Deployment in Practice

Zero Downtime Deployment ("Deploy Without Interruption") is a strategy that aims to make a deploy without interrupting the operation of the application so that the exchange becomes transparent to users.

How to put new app versions into production without impacting user experience?

The application of Zero Downtime Deployment is based on some standards and best practices.

Blue/Green Deployment

Blue/Green Deployment is the classic standard of Zero Downtime Deployment. It assumes that the application is in production on at least two sets of machines. The goal is to raise the N+1 version in one set (green below) while the service is maintained in another set (blue) in version N.

Canary Release

This standard combined with Blue/Green Deployment allows you to confront the N+1 version with a limited population of users while most users use the N version. The mechanisms involved are the same as for Blue/Green Deployment.

This pattern is used by Facebook, where its employees use the new version of the site for a day before making it accessible to all users if everything is correct.

The Dark Launch

This standard allows to invisibly put into production a functionality to progressively simulate a load test with the traffic that will exist in the actual use.

The goal of this standard is to validate the performance and scalability of the platform. Progressively simulating the expected traffic helps to prepare and optimize the systems so that everything happens well during the launch of the functionality on day D.

Implementation

The goal is to combine the load distribution mechanism (Load Balancing) with the deployment process:

1. Load Balancer takes out the set of machines where we will put the N+1 version,
2. after deploy, Load Balancer drives some users up to this set with version N+1,
3. the Load Balancer takes the (s) other (s) sets(s) of machines, incrementally if it has several, to rise from version to N+1 and puts again in the load distribution.

We conclude

That the test is essential for any company that needs to ensure that it never breaks your system and becomes unusable to all its users, and with that we also need to define strategies to release everything, it is not possible to drop everything for 5 minutes while we are climbing a new version, so we understand in a more comprehensive way how it works.

We may have more testing and release strategies, but I hope I’ve shed some light on your doubt, hug.

Answer 2

My answer aims to complement the @guiandmag, as some questions remain.

It is important to know the techniques to test a software, but it takes a good deal of wisdom to determine what is feasible and appropriate to implement.

What would be a good way to test the software?

Among all the available techniques, it will be necessary to determine those that will effectively ensure that the user is having the needs met.

Unit tests help the developer and end up being essential for a sound code base. Some integration tests can be interesting.

Depending on the maturity of the product, automated interface testing can be very interesting to avoid regressions. However, if the product is something new and is in accelerated development, probably the time to create and maintain such tests would be better spent developing the new functionalities.

Let us think of two extremes. On the one hand, a banking system based on legislation whose possibility of change is low. On the other hand, a Saas beta application. If we think about applying the same types of test in the same proportions for both scenarios, it means we’re doing something wrong.

Would it be interesting to use manual or automatic testing processes? There are criteria for this?

Some level of automation is always welcome, especially at low level.

At a high level we return to the previous topic: it depends on the type of system. A relatively uncomplicated system whose user is directly involved would benefit more from a manual User Acceptance Test than from a complex layer of interface automation that always tests the same scenarios almost always incompletely.

How to define that that software is fit for the end user?

If the user participates in the development process, directly or indirectly, he must take the Acceptance Test (UAT - User Acceptance Test), which is the final endorsement of the needs to be met or not.

When the user is unknown or distant, as in a cloud service, there may be a mix of automated tests that cover the most common application usage scenarios (which requires statistics and therefore some beta phase to collect them) and some responsible team analyze potential problems.

Considerations

It is important to understand that testing everything is simply one myth. You can better understand the reasons in this my other answer.

The way is to balance the quantity of each type of test, increasing and decreasing according to the proportion of productivity and quality if you want to achieve.

You can’t anticipate all the problems that can happen, so most of the time the tests serve only to avoid making the same mistakes again.

Finally, I can conclude that the quality of a system, although a very subjective criterion, can be improved by testing by decreasing the amount of times an end-user encounters a problem. However, quality is much more impacted by how the system is developed, from requirements to implementation. Good analysts and good developers will do a much better job for quality than all possible tests together.