1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
|
The output of running a problem suite is a next dictionary that can easily be converted into json:
--------
problem
--------
Normal case:
problem = {
name: name of problem <string>
cases: test case results <list(case)>
passed: <bool>
}
Critical error occured (Syntax, Indentation, FunctionNotFound):
problem = {
name: name of problem <string>
critical_error: details of error <exception_data>
passed: False <bool>
}
----
case
----
Normal:
case = {
name: name of test case <string>
parts: test part results <list(part)>
passed: all parts passed <bool>
}
Exception occured:
case = {
name: name of test case <string>
exception: details of exception <exception_data>
passed: False <bool>
}
----
part
----
Part passed:
part = {
description: <string>
passed: True <bool>
}
Part failed:
part = {
description: <string>
passed: False <bool>
error_message: <string>
}
--------------
exception_data
--------------
exception_data = {
name: class name of exception <string>
detail: extra info <string>
critical: does exception prevent all tests from being run <bool>
}
|