Wiki source code of Compatibility Test Suite
Last modified by Vincent Massol on 2018/02/21 10:52
Show last authors
author | version | line-number | content |
---|---|---|---|
1 | {{box cssClass="floatinginfobox" title="**Contents**"}} | ||
2 | {{toc/}} | ||
3 | {{/box}} | ||
4 | |||
5 | This is a suite of tests that are available and that you can run on your Syntax Parser and Renderer to verify that they work just fine. | ||
6 | |||
7 | = Adding a new Test Suite = | ||
8 | |||
9 | Imagine that you've developed a new Syntax that is named ##mysyntax/1.0##. To test it you'd write the following unit test: | ||
10 | |||
11 | {{code language="java"}} | ||
12 | import org.junit.runner.RunWith; | ||
13 | import org.xwiki.rendering.test.cts.CompatibilityTestSuite; | ||
14 | import org.xwiki.rendering.test.cts.Syntax; | ||
15 | |||
16 | @RunWith(CompatibilityTestSuite.class) | ||
17 | @Syntax("mysyntax/1.0") | ||
18 | public class MySyntaxCompatibilityTest | ||
19 | { | ||
20 | } | ||
21 | {{/code}} | ||
22 | |||
23 | Running it the first time will generate the following type of JUnit results (example is based on XWiki Syntax 1.0, just replace it mentally with "My Syntax 1.0"): | ||
24 | |||
25 | {{image reference="result1.png"/}} | ||
26 | |||
27 | As you can see all the tests are marked as "Missing" since you haven't written them yet. | ||
28 | |||
29 | = Adding Tests = | ||
30 | |||
31 | The first thing to do is to create a ##src/test/resources/syntax10## directory that will hold your tests. | ||
32 | |||
33 | Then check the list of missing tests and for each missing test you need to write a test in the corresponding directory. For example for ##simple/bold/bold1## you'll need to add a test file in ##src/test/resources/syntax10/simple/bold/##. | ||
34 | |||
35 | There are 2 types of tests you can write: | ||
36 | |||
37 | * Tests for your Parser | ||
38 | * Tests for your Renderer | ||
39 | |||
40 | For Parser tests, the format of the files is ##<test name>.in.txt## (e.g. ##bold1.in.txt##) | ||
41 | For Renderer tests, the format of the files is ##<test name>.out.txt## (e.g. ##bold1.out.txt##) | ||
42 | |||
43 | If you have both a Parser and a Renderer and the content of their input and output files are the same then you can merge them into a single file named ##<test name>.inout.txt## (e.g. bold1.inout.txt##).## | ||
44 | |||
45 | For Parsers tests your goal is to write an input file that generates an XDOM that matches the output expected by the corresponding {{scm project="xwiki-rendering" path="xwiki-rendering-test/src/main/resources/cts"}}CTS Test{{/scm}} | ||
46 | For Renderer tests, a {{scm project="xwiki-rendering" path="xwiki-rendering-test/src/main/resources/cts"}}CTS input file{{/scm}} is Parsed and then the resulting XDOM is rendered using your Renderer and the results are compared. | ||
47 | |||
48 | For example here's what you could have (example based on XWiki Syntax 1.0 which only has a Parser and no Renderer): | ||
49 | |||
50 | {{image reference="tests.png"/}} | ||
51 | |||
52 | = Test Configuration = | ||
53 | |||
54 | Notice the ##config.properties## file. It contains configuration data for your tests. Namely it can contain the following: | ||
55 | |||
56 | * List of tests that are not applicable for your Syntax. These tests are not executed. Example:((( | ||
57 | {{code}} | ||
58 | notApplicableTests = .*bold2.* | ||
59 | notApplicableTests = .*list1.* | ||
60 | {{/code}} | ||
61 | ))) | ||
62 | * List of failing tests. These tests are ignored but marked as failing when executed. Example:((( | ||
63 | {{code}} | ||
64 | failingTests = .*bold2.* | ||
65 | failingTests = .*list1.* | ||
66 | {{/code}} | ||
67 | ))) | ||
68 | * Inherit Syntax. If your Syntax extends an existing syntax you might want to run all the tests for that other syntax too, in this case you can specify an inherit syntax like this (in this example we inherit from XWiki Syntax 2.0):((( | ||
69 | {{code}} | ||
70 | inheritSyntax = xwiki/2.0 | ||
71 | {{/code}} | ||
72 | ))) | ||
73 | |||
74 | Note that ##notApplicableTests## and ##failingTests## are specified as regexes against the name of the tests as you see when you execute them. For example: ##simple/bold/bold1 [xwiki/2.0, IN:bold1.in.txt, CTS:bold1.inout.xml]##. | ||
75 | |||
76 | = Generating a Report = | ||
77 | |||
78 | We're providing a [[CTS Report in the form of a code snippet>>snippets:Extension.Syntax Test Report]] that you can copy/paste in you own wiki instance. | ||
79 | |||
80 | Since we're running this Compatibility Test Suite on all the Syntaxes that we support you can checkout the [[resulting Test Report>>Main.SyntaxReport]]. | ||
81 | |||
82 | = Advanced = | ||
83 | |||
84 | == Pattern == | ||
85 | |||
86 | If you wish your test suite to only execute a single test (to debug only this test for example), you can add the ##pattern## value in the ##Scope## annotation. For example the following will only execute the ##**/table1.test**##** test:** | ||
87 | |||
88 | {{code language="java"}} | ||
89 | @RunWith(RenderingTestSuite.class) | ||
90 | @RenderingTestSuite.Scope(value = "latex10.specific", pattern="table1.test") | ||
91 | @AllComponents | ||
92 | public class LaTeXSpecificTest | ||
93 | { | ||
94 | } | ||
95 | {{/code}} | ||
96 | |||
97 | == Mocking == | ||
98 | |||
99 | You may wish to perform some initialization of mocks before running the tests. This can be achieved by adding a method annotated with ##@RenderingTestSuite.Initialized## and accepting a ##MockitoComponentManager## parameter, as in the following example: | ||
100 | |||
101 | {{code language="java"}} | ||
102 | @RunWith(RenderingTestSuite.class) | ||
103 | @RenderingTestSuite.Scope(value = "latex10.specific") | ||
104 | @AllComponents | ||
105 | public class LaTeXSpecificTest | ||
106 | { | ||
107 | @RenderingTestSuite.Initialized | ||
108 | public void initialize(MockitoComponentManager componentManager) throws Exception | ||
109 | { | ||
110 | MockSetup.setUp(componentManager); | ||
111 | |||
112 | componentManager.registerComponent(MockWikiModel.getComponentDescriptor()); | ||
113 | } | ||
114 | } | ||
115 | {{/code}} | ||
116 | |||
117 | Also notice the usage of the ##@AllComponents## annotation which tells the Rendering Test Site to automatically register all XWiki Components found in the ClassLoader used to load this test. You could also use the ##@ComponentList({...})## annotation to only register some components. For example: | ||
118 | |||
119 | {{code language="java"}} | ||
120 | @ComponentList({ | ||
121 | DefaultBeanManager.class, | ||
122 | DefaultConverterManager.class, | ||
123 | EnumConverter.class, | ||
124 | ConvertUtilsConverter.class | ||
125 | }) | ||
126 | {{/code}} | ||
127 | |||
128 | == MetaData == | ||
129 | |||
130 | If your Parser generates an XDOM with Macros that contain macros with content in a different Syntax that yours then you can specify this in the ##@Syntax## annotation you use. For example the XWiki Syntax 1.0 Parser converts XWiki Syntax 1.0 content into XWiki Syntax 2.0 content and thus its Test Suite is written like this: | ||
131 | |||
132 | ((( | ||
133 | {{code language="java"}} | ||
134 | @RunWith(CompatibilityTestSuite.class) | ||
135 | @Syntax(value = "xwiki/1.0", metadata = "xwiki/2.0") | ||
136 | public class XWiki10CompatibilityTest | ||
137 | { | ||
138 | } | ||
139 | {{/code}} | ||
140 | ))) |