|
2198.5.2
by patrick crews
Added docs for randgen |
1 |
**********************************
|
2 |
randgen (random query generator)
|
|
3 |
**********************************
|
|
4 |
||
5 |
||
6 |
||
7 |
Description
|
|
8 |
===========
|
|
9 |
||
10 |
The randgen aka the random query generator is a database |
|
11 |
testing tool. It uses a grammar-based stochastic model to represent |
|
12 |
some desired set of queries (to exercise the optimizer, for example) |
|
13 |
and generates random queries as allowed by the grammar |
|
14 |
||
15 |
The primary documentation is here: http://forge.mysql.com/wiki/RandomQueryGenerator |
|
16 |
||
17 |
This document is intended to help the user set up their environment so that the tool |
|
18 |
may be used in conjunction with the dbqp.py test-runner. The forge documentation |
|
19 |
contains more information on the particulars of the tool itself. |
|
20 |
||
21 |
Requirements
|
|
22 |
============
|
|
23 |
||
24 |
DBD::drizzle
|
|
25 |
-------------
|
|
26 |
The DBD::drizzle module is required it can be found here http://launchpad.net/dbd-drizzle/ |
|
27 |
||
28 |
Additional information for installing the module::
|
|
29 |
||
30 |
Prerequisites
|
|
31 |
----------------
|
|
32 |
* Perl
|
|
33 |
* Drizzle (bzr branch lp:drizzle)
|
|
34 |
* libdrizzle (bzr branch lp:libdrizzle)
|
|
35 |
* C compiler
|
|
36 |
||
37 |
Installation
|
|
38 |
-------------
|
|
39 |
You should only have to run the following:
|
|
40 |
||
41 |
perl Makefile.PL --cflags=-I/usr/local/drizzle/include/ --libs=-"L/usr/local/drizzle/lib -ldrizzle"
|
|
42 |
||
43 |
||
44 |
Depending on where libdrizzle is installed. Also, you'll want to make
|
|
45 |
sure that ldconfig has configured libdrizzle to be in your library path
|
|
46 |
||
47 |
Additional information may be found here: http://forge.mysql.com/wiki/RandomQueryGeneratorQuickStart |
|
48 |
||
49 |
Installing the randgen
|
|
50 |
=======================
|
|
51 |
||
52 |
The code may be branched from launchpad: bzr branch lp:randgen |
|
53 |
||
54 |
it also may be downloaded from here http://launchpad.net/randgen/+download |
|
55 |
||
56 |
That is all there is : ) |
|
57 |
||
58 |
Randgen / dbqp tests
|
|
59 |
====================
|
|
60 |
||
61 |
These tests are simple .cnf files that can define a few basic variables |
|
62 |
that are needed to execute tests. The most interesting section is test_servers. It is a simple list of python lists |
|
63 |
Each sub-list contains a string of server options that are needed. Each sub-list represents a server that will be started. |
|
64 |
Using an empty sub-list will create a server with the default options::
|
|
65 |
||
66 |
[test_info]
|
|
67 |
comment = does NOT actually test the master-slave replication yet, but it will.
|
|
68 |
||
69 |
[test_command]
|
|
70 |
command = ./gentest.pl --gendata=conf/drizzle/drizzle.zz --grammar=conf/drizzle/optimizer_subquery_drizzle.yy --queries=10 --threads=1
|
|
71 |
||
72 |
[test_servers]
|
|
|
2210.4.2
by Andrew Hutchings
Merge with trunk |
73 |
servers = [[--innodb.replication-log],[--plugin-add=slave --slave.config-file=$MASTER_SERVER_SLAVE_CONFIG]]
|
|
2198.5.2
by patrick crews
Added docs for randgen |
74 |
|
75 |
Running tests
|
|
76 |
=========================
|
|
77 |
||
78 |
There are several different ways to run tests using :doc:`dbqp` 's randgen mode. |
|
79 |
||
80 |
It should be noted that unless :option:`--force` is used, the program will |
|
81 |
stop execution upon encountering the first failing test. |
|
82 |
:option:`--force` is recommended if you are running several tests - it will |
|
83 |
allow you to view all successes and failures in one run. |
|
84 |
||
85 |
Running individual tests
|
|
86 |
------------------------
|
|
87 |
If one only wants to run a few, specific tests, they may do so this way::
|
|
88 |
||
89 |
./dbqp --mode=randgen --randgen-path=/path/to/randgen [OPTIONS] test1 [test2 ... testN]
|
|
90 |
||
91 |
Running all tests within a suite
|
|
92 |
--------------------------------
|
|
93 |
Many of the tests supplied with Drizzle are organized into suites. |
|
94 |
||
95 |
The tests within drizzle/tests/randgen_tests/main are considered the 'main' suite. |
|
96 |
Other suites are also subdirectories of drizzle/tests/randgen_tests. |
|
97 |
||
98 |
To run the tests in a specific suite::
|
|
99 |
||
100 |
./dbqp --mode=randgen --randgen-path=/path/to/randgen [OPTIONS] --suite=SUITENAME
|
|
101 |
||
102 |
Running specific tests within a suite
|
|
103 |
--------------------------------------
|
|
104 |
To run a specific set of tests within a suite::
|
|
105 |
||
106 |
./dbqp --mode=randgen --randgen-path=/path/to/randgen [OPTIONS] --suite=SUITENAME TEST1 [TEST2..TESTN]
|
|
107 |
||
108 |
Calling tests using <suitename>.<testname> currently does not work. |
|
109 |
One must specify the test suite via the :option:`--suite` option. |
|
110 |
||
111 |
||
112 |
Running all available tests
|
|
113 |
---------------------------
|
|
114 |
One would currently have to name all suites, but the majority of the working tests live in the main suite |
|
115 |
Other suites utilize more exotic server combinations and we are currently tweaking them to better integrate with the |
|
116 |
dbqp system. The slave-plugin suite does currently have a good config file for setting up simple replication setups for testing. |
|
117 |
To execute several suites' worth of tests::
|
|
118 |
||
119 |
./dbqp --mode=randgen --randgen-path=/path/to/randgen [OPTIONS] --suite=SUITE1, SUITE2, ...SUITEN
|
|
120 |
||
121 |
Interpreting test results
|
|
122 |
=========================
|
|
123 |
The output of the test runner is quite simple. Every test should pass. |
|
124 |
In the event of a test failure, please take the time to file a bug here: |
|
125 |
*https://bugs.launchpad.net/drizzle*
|
|
126 |
||
127 |
During a run, the program will provide the user with: |
|
128 |
* test name (suite + name) |
|
129 |
* test status (pass/fail/skipped)
|
|
130 |
* time spent executing each test
|
|
131 |
||
132 |
At the end of a run, the program will provide the user with a listing of: |
|
133 |
* how many tests were run |
|
134 |
* how many tests failed
|
|
135 |
* percentage of passing tests
|
|
136 |
* a listing of failing tests
|
|
137 |
* total time spent executing the tests
|
|
138 |
||
139 |
Example output::
|
|
140 |
||
141 |
24 Feb 2011 17:27:36 : main.outer_join_portable [ pass ] 7019
|
|
142 |
24 Feb 2011 17:27:39 : main.repeatable_read [ pass ] 2764
|
|
143 |
24 Feb 2011 17:28:57 : main.select_stability_validator [ pass ] 77946
|
|
144 |
24 Feb 2011 17:29:01 : main.subquery [ pass ] 4474
|
|
145 |
24 Feb 2011 17:30:52 : main.subquery_semijoin [ pass ] 110355
|
|
146 |
24 Feb 2011 17:31:00 : main.subquery_semijoin_nested [ pass ] 8750
|
|
147 |
24 Feb 2011 17:31:03 : main.varchar [ pass ] 3048
|
|
148 |
24 Feb 2011 17:31:03 : ================================================================================
|
|
149 |
24 Feb 2011 17:31:03 INFO: Test execution complete in 288 seconds
|
|
150 |
24 Feb 2011 17:31:03 INFO: Summary report:
|
|
151 |
24 Feb 2011 17:31:03 INFO: Executed 18/18 test cases, 100.00 percent
|
|
152 |
24 Feb 2011 17:31:03 INFO: STATUS: PASS, 18/18 test cases, 100.00 percent executed
|
|
153 |
24 Feb 2011 17:31:03 INFO: Spent 287 / 288 seconds on: TEST(s)
|
|
154 |
24 Feb 2011 17:31:03 INFO: Test execution complete
|
|
155 |
24 Feb 2011 17:31:03 INFO: Stopping all running servers...
|
|
156 |
||
157 |
|
|
158 |
Additional uses
|
|
159 |
===============
|
|
160 |
Starting a server for manual testing and (optionally) populating it
|
|
161 |
--------------------------------------------------------------------
|
|
162 |
||
163 |
:doc:`dbqp` 's randgen mode allows a user to get a Drizzle server up and running quickly. This can be useful for fast ad-hoc testing. |
|
164 |
||
165 |
To do so call::
|
|
166 |
||
167 |
./dbqp --mode=randgen --randgen-path=/path/to/randgen --start-and-exit [*OPTIONS*]
|
|
168 |
||
169 |
This will start a Drizzle server that you can connect to and query |
|
170 |
||
171 |
With the addition of the --gendata option, a user may utilize the randgen's gendata (table creation and population) tool |
|
172 |
to populate a test server. In the following example, the test server is now populated by the 8 tables listed below::
|
|
173 |
||
174 |
./dbqp --mode=randgen --randgen-path=/randgen --start-and-exit --gendata=/randgen/conf/drizzle/drizzle.zz
|
|
175 |
<snip>
|
|
176 |
24 Feb 2011 17:48:48 INFO: NAME: server0
|
|
177 |
24 Feb 2011 17:48:48 INFO: MASTER_PORT: 9306
|
|
178 |
24 Feb 2011 17:48:48 INFO: DRIZZLE_TCP_PORT: 9307
|
|
179 |
24 Feb 2011 17:48:48 INFO: MC_PORT: 9308
|
|
180 |
24 Feb 2011 17:48:48 INFO: PBMS_PORT: 9309
|
|
181 |
24 Feb 2011 17:48:48 INFO: RABBITMQ_NODE_PORT: 9310
|
|
182 |
24 Feb 2011 17:48:48 INFO: VARDIR: /home/pcrews/bzr/work/dbqp_randgen_updates/tests/workdir/testbot0/server0/var
|
|
183 |
24 Feb 2011 17:48:48 INFO: STATUS: 1
|
|
184 |
# 2011-02-24T17:48:48 Default schema: test
|
|
185 |
# 2011-02-24T17:48:48 Executor initialized, id GenTest::Executor::Drizzle 2011.02.2198 ()
|
|
186 |
# 2011-02-24T17:48:48 # Creating Drizzle table: test.A; engine: ; rows: 0 .
|
|
187 |
# 2011-02-24T17:48:48 # Creating Drizzle table: test.B; engine: ; rows: 0 .
|
|
188 |
# 2011-02-24T17:48:48 # Creating Drizzle table: test.C; engine: ; rows: 1 .
|
|
189 |
# 2011-02-24T17:48:48 # Creating Drizzle table: test.D; engine: ; rows: 1 .
|
|
190 |
# 2011-02-24T17:48:48 # Creating Drizzle table: test.AA; engine: ; rows: 10 .
|
|
191 |
# 2011-02-24T17:48:48 # Creating Drizzle table: test.BB; engine: ; rows: 10 .
|
|
192 |
# 2011-02-24T17:48:48 # Creating Drizzle table: test.CC; engine: ; rows: 100 .
|
|
193 |
# 2011-02-24T17:48:49 # Creating Drizzle table: test.DD; engine: ; rows: 100 .
|
|
194 |
24 Feb 2011 17:48:49 INFO: User specified --start-and-exit. dbqp.py exiting and leaving servers running...
|
|
195 |
||
196 |
||
197 |