Development with automated component tests (also known as automated unit tests - AUTs) enables a modern and accelerated project approach. This is largely due to the following three properties of component testing:
- for the test execution no transport is required in the Q-System
- on the granularity of components, test design methods can be applied, which enable complete test coverage
- it works with minimal simulation data and not with mass data of the system
The ideal project approach looks like this:
First meeting with the business department
In the two-hour meeting, the department for receivables management was introduced to the xGile test portal, which was still unknown to them. I explained the idea behind it and how a test cases get created. Surprisingly, errors in the stored expectations of the existing test cases were immediately identified by them. A key experience. Not only did the business department understand the test procedure, it also immediately recognized errors in the test case definitions of implementation phase. The findings were fundamental. The department is able to understand the test procedure without problems and to validate the test cases in detail. Issued based on misunderstandings are history from now on.
But it became better. When I instantly changed the test case expectation in the meeting and ran the test case again, it reported an error, due to the not meet expectation. I opened the SAP HANA procedure, quickly adjusted the code and ran the test case again. The test was now successful and the requirement was met. A thing that only took five minutes and which we carried out together in the meeting to show how the basic approach looks like. I'll never forget the comment from the head of department who didn't said any word till now and I thought to have bored all the time: "Who builds something like that? That's awesome!".
The next day, the number of test cases grew without me by another 6 test cases up to 17 test cases. We should have had carry out an initial test design, but I was more than satisfied that the department already jumped on the train and was eagerly testing by them self. We followed up with the test design session later to determine the optimal number of test cases. For the moment it was a pure success to have everyone on board.
“The department actively joined testing and trusted the test procedure and results.”xGile testing
Within a week everything was fully implemented and test automated. As a result, 138 lines of code and 22 xGile test cases on test data existed, automatically approved by the department! All tests were successful and the first transport to the Q system was done then. Here the business user looked manually for unexpected deviations. A few deviations were discovered and the root cause identified: credits which resulted in negative sales. This requirement was initially not considered and had to be implemented yet. For this purpose, an xGile test case was created in the development system and the problem was proven by that. We could reproduce the issue with 3 lines of test data. Shaka! This requirement was now stored and further development took place until this test case and thus the requirement was successfully approved. Finally, all test cases were executed to prove that no previous test cases / requirements were corrupted. Only then did the second transport into the Q-system take place and only now was the department again asked to carry out cross checks on real data. Conclusion from a business user afterwards:
“What impressed me most was that once a mistake was found, it could no longer occur.”Department of receivables management on xGile testing
After the code went live, no errors were identified until today (3 years later). A fact that speaks for the quality of the test procedure. The developments have already been passed on to other developers without any problems, because my entire knowledge and that of the business department lies in 22 automatic test cases. Subsequent changes can be made quickly and easily with this safety net.
The financial benefit
How can one calculate the financial benefit of the test automation? To answer this, we calculate the automation factor based on the automation degree. This was around 70%, everything done on the development system.
The automation factor is 70% devided by 30 % = a factor of 2,3. The factor is now multiplied with the 9 man days spend for the manual work in the Q-System. We also have to substract the 1,5 man days invested for creating the automated tests with xGile and we will get:
= 2,3 * 9 MD - 1,5MD = 19,2 MD saved !
This means, when ever the 22 xGile test are executed, it will save 19,2 MDs ! Assuming a MD of 700 $ that would result in savings of:
700 $ * 19,2 = 13.444,0 $ for for every test run !
Well, you might be surprised about the high number, but the truth is, because manual testing is that expensive, it is in practice not executed completely and just as spot checks.
The testing mainly took place on the development system. The xGile test portal is easy to use and understandable for business users. The business user involvement not only increases the level of quality and speed, but also reduces his final participation and time spend thanks to automated validation and approval. His confidence level in the test approach is strengthened and shadow testing no longer takes place. In the end, no more errors were identified in production. A safety net for future changes was created that saves 19.2 man days, approx. € 13,444.0 at every test run.