Archiv für den Monat: April 2016

ReTest vs. MagicTest

Two weeks ago I attended the JAX 2016 in Mainz. My main interests were testing or QA in general. One of the sessions was about MagicTest and it reminded me of another session from JUGF about ReTest. Both have a “new” method to test Java code.

In this blog I want to describe the two approaches in a general way. If there will be an opportunity to test the software by myself I will create more details reports about the tools in the future.

ReTest

At the beginning of 2016 I attended the Java User Group in Frankfurt. The topic was very promising for me: JUGF – Lasst die Affen testen (let the monkeys test) by Dr. Jeremias Rößler. The company ReTest is currently creating a software in Java to perform random GUI tests.

The basic idea behind the software is, that manual GUI testing is expensive, slow and boring to do. On the other hand automatic GUI tests are difficult to create and maintain. The software from ReTest is doing random tests on any Java GUI and is logging the feedback from the GUI.

The recorded actions and the logged feedback is than the baseline for further tests of the same GUI. With this approach nobody has to define in detail what is expected. The current feedback is the truth about the software. Any other feedback will be a mistake to report by the testing software.

The testing software can get some hints by the user how to test the software. As a simple example take the login dialog of an application. It might take some time for the testing monkey to find a matching combination of username and password to get into the application and start the actual testing. For this and more complex functions a kind of capture and replay can be integrated into the test runs.

MagicTest

At the JAX in May 2016 I attended the talk MagicTest – Visuelles Testen für Java (visual testing for Java) by Thomas Mauch.

The main purpose of MagicTest is to get rid of assert statements in unit testing. If a developer wants to write a test for a method he can create a test method for each test case as in standard JUnit or he can create one method with all test cases.

During execution of MagicTest byte code manipulation is the testing. Every statement is automatically surrounded with assert and fail statements. Even if in between of the tests an uncaught exception is thrown the generated code will be fetch the exception and deals with it.

Here is a sample code that was presented in a similar way at the JAX.

Code a developer has to write:

   1: public final void testDivision() {
2: MathUtil.divide(10, 10);
3: 
4: MathUtil.divide(10, 0);
5: 
6: MathUtil.divide(0, 10);
7: }

MagicTest byte code manipulator is creating something like this:

   1: public final void testDivision() {
2: assertEquals(1, MathUtil.divide(10, 10));
3: 
4: try {
5: MathUtil.divide(10, 0);
6: fail();
7: } catch(UnsupportedException e) {
8: asstertTrue(true);
9: }
10: 
11: assertEquals(0, MathUtil.divide(0, 10));
12: }

Additionally at the end a nice HTML page with the test results is created. If the test result gets different after a software change the user is presented the new test result in the HTML page. Here the he has two options:

  • Resolve the problem in the code
  • Define the new test result as valid

In my opinion both test strategies are very promising. MagicTest ca reduce the effort to create new unit tests. But this is only helpful if the developer already create unit tests. ReTest on the other hand can also help to create a safety net for a software that has user interface. This tool can be used additionally to other (existing) tests but also as a starting point to create any automatic tests.