How do system testing tools and techniques differ from other testing methods?

It's well known that a system is more than the sum of its parts. A system can do more than any of its components or the sum of those capabilities. A system is only a system when those parts are put together in a specific way. A system manifests emergence. Now, System testing is end to end testing of the entire system as a whole. While unit testing is verifying each component, and integration tests that those components have been put together correctly. Still, the system might not exhibit the expected behaviour, or not in all conditions. Remember emergence: that can be desired or undesired. When testing the system end to end as a black-box one can show that it is behaving as expected in all conditions within its operating environment.

One thing that I have found to be helpful is to view System Test & Evaluation (T&E) in 3 categories of T&E which happen during various phases of System Development 1. Developmental T&E (DT&E): which occur during Development 2. Acquisition T&E (AT&E): 'formal' testing that is conducted as per System user requirements to accept it from development agency. 3. Operational T&E (OT&E): conducted under realistic Operational conditions in order to validate that system delivers the Capabilities envisaged

Imagine a mobile app as a car. System testing would be like taking the car for a test drive and checking how it handles different situations and interacts with the environment (other cars, roads, traffic lights). Other testing methods would be like inspecting the car's engine (code analysis), checking the locks and windows (UI security), or trying to hotwire the car (penetration testing). Each approach plays a crucial role in ensuring the overall security and functionality of a mobile app. They work together to provide a comprehensive security assessment and identify potential vulnerabilities across different layers of the system.

When I hear "System testing" I tend think about it on the axis of system-test - integration-test - unit-test. Since I do more back-end work, my personal sweet spot is integration tests, which could also be thought of as system tests for the back end. I think where system tests shine is in testing actual usage of a system. That's not to say that integration or unit tests do not test actual usage, but it's much easier to write unit tests that don't provide enough value to justify the extra lines of code. I had an email conversation with James O Coplien, where I admitted to Cope that I tended to have unit tests in places where I thought the code was dodgy. By the way, I recommend Coplien's "Why Most Unit Testing is Waste".

Typically black-box testing methods. From boundary values to equivalence partitioning. To testing (most relevant) input combinations, testing never ends. What is important is to have a clear definition of what the system under test is (as it might be part of a super-system, or contain lower level systems in a SoS context). And the tests that are executed are (mostly) black-box and end to end relative to the system's boundary. Don't forget negative testing ang of course intuitive testing. More, robustness and reliability also needs testing. One could use techniques like smoke testing and stress testing. Then if the system is a physical one (depending what it is) one should consider Environmental and EMC testing too. Testing never complete

Test Environment System testing is done in the same environment as production server so that any environment specific issue could be identified before pushing to production. Other testing methods can be executed in test server which may or may not be same as production server.