1
**********************************
2
randgen (random query generator)
3
**********************************
10
The randgen aka the random query generator is a database
11
testing tool. It uses a grammar-based stochastic model to represent
12
some desired set of queries (to exercise the optimizer, for example)
13
and generates random queries as allowed by the grammar
15
The primary documentation is here: http://forge.mysql.com/wiki/RandomQueryGenerator
17
This document is intended to help the user set up their environment so that the tool
18
may be used in conjunction with the dbqp.py test-runner. The forge documentation
19
contains more information on the particulars of the tool itself.
26
The DBD::drizzle module is required it can be found here http://launchpad.net/dbd-drizzle/
28
Additional information for installing the module::
33
* Drizzle (bzr branch lp:drizzle)
34
* libdrizzle (bzr branch lp:libdrizzle)
39
You should only have to run the following:
41
perl Makefile.PL --cflags=-I/usr/local/drizzle/include/ --libs=-"L/usr/local/drizzle/lib -ldrizzle"
44
Depending on where libdrizzle is installed. Also, you'll want to make
45
sure that ldconfig has configured libdrizzle to be in your library path
47
Additional information may be found here: http://forge.mysql.com/wiki/RandomQueryGeneratorQuickStart
49
Installing the randgen
50
=======================
52
The code may be branched from launchpad: bzr branch lp:randgen
54
it also may be downloaded from here http://launchpad.net/randgen/+download
56
That is all there is : )
61
These tests are simple .cnf files that can define a few basic variables
62
that are needed to execute tests. The most interesting section is test_servers. It is a simple list of python lists
63
Each sub-list contains a string of server options that are needed. Each sub-list represents a server that will be started.
64
Using an empty sub-list will create a server with the default options::
67
comment = does NOT actually test the master-slave replication yet, but it will.
70
command = ./gentest.pl --gendata=conf/drizzle/drizzle.zz --grammar=conf/drizzle/optimizer_subquery_drizzle.yy --queries=10 --threads=1
73
servers = [[--innodb.replication-log=true],[--plugin-add=slave --slave.config-file=$MASTER_SERVER_SLAVE_CONFIG]]
76
=========================
78
There are several different ways to run tests using :doc:`dbqp` 's randgen mode.
80
It should be noted that unless :option:`--force` is used, the program will
81
stop execution upon encountering the first failing test.
82
:option:`--force` is recommended if you are running several tests - it will
83
allow you to view all successes and failures in one run.
85
Running individual tests
86
------------------------
87
If one only wants to run a few, specific tests, they may do so this way::
89
./dbqp --mode=randgen --randgen-path=/path/to/randgen [OPTIONS] test1 [test2 ... testN]
91
Running all tests within a suite
92
--------------------------------
93
Many of the tests supplied with Drizzle are organized into suites.
95
The tests within drizzle/tests/randgen_tests/main are considered the 'main' suite.
96
Other suites are also subdirectories of drizzle/tests/randgen_tests.
98
To run the tests in a specific suite::
100
./dbqp --mode=randgen --randgen-path=/path/to/randgen [OPTIONS] --suite=SUITENAME
102
Running specific tests within a suite
103
--------------------------------------
104
To run a specific set of tests within a suite::
106
./dbqp --mode=randgen --randgen-path=/path/to/randgen [OPTIONS] --suite=SUITENAME TEST1 [TEST2..TESTN]
108
Calling tests using <suitename>.<testname> currently does not work.
109
One must specify the test suite via the :option:`--suite` option.
112
Running all available tests
113
---------------------------
114
One would currently have to name all suites, but the majority of the working tests live in the main suite
115
Other suites utilize more exotic server combinations and we are currently tweaking them to better integrate with the
116
dbqp system. The slave-plugin suite does currently have a good config file for setting up simple replication setups for testing.
117
To execute several suites' worth of tests::
119
./dbqp --mode=randgen --randgen-path=/path/to/randgen [OPTIONS] --suite=SUITE1, SUITE2, ...SUITEN
121
Interpreting test results
122
=========================
123
The output of the test runner is quite simple. Every test should pass.
124
In the event of a test failure, please take the time to file a bug here:
125
*https://bugs.launchpad.net/drizzle*
127
During a run, the program will provide the user with:
128
* test name (suite + name)
129
* test status (pass/fail/skipped)
130
* time spent executing each test
132
At the end of a run, the program will provide the user with a listing of:
133
* how many tests were run
134
* how many tests failed
135
* percentage of passing tests
136
* a listing of failing tests
137
* total time spent executing the tests
141
24 Feb 2011 17:27:36 : main.outer_join_portable [ pass ] 7019
142
24 Feb 2011 17:27:39 : main.repeatable_read [ pass ] 2764
143
24 Feb 2011 17:28:57 : main.select_stability_validator [ pass ] 77946
144
24 Feb 2011 17:29:01 : main.subquery [ pass ] 4474
145
24 Feb 2011 17:30:52 : main.subquery_semijoin [ pass ] 110355
146
24 Feb 2011 17:31:00 : main.subquery_semijoin_nested [ pass ] 8750
147
24 Feb 2011 17:31:03 : main.varchar [ pass ] 3048
148
24 Feb 2011 17:31:03 : ================================================================================
149
24 Feb 2011 17:31:03 INFO: Test execution complete in 288 seconds
150
24 Feb 2011 17:31:03 INFO: Summary report:
151
24 Feb 2011 17:31:03 INFO: Executed 18/18 test cases, 100.00 percent
152
24 Feb 2011 17:31:03 INFO: STATUS: PASS, 18/18 test cases, 100.00 percent executed
153
24 Feb 2011 17:31:03 INFO: Spent 287 / 288 seconds on: TEST(s)
154
24 Feb 2011 17:31:03 INFO: Test execution complete
155
24 Feb 2011 17:31:03 INFO: Stopping all running servers...
160
Starting a server for manual testing and (optionally) populating it
161
--------------------------------------------------------------------
163
:doc:`dbqp` 's randgen mode allows a user to get a Drizzle server up and running quickly. This can be useful for fast ad-hoc testing.
167
./dbqp --mode=randgen --randgen-path=/path/to/randgen --start-and-exit [*OPTIONS*]
169
This will start a Drizzle server that you can connect to and query
171
With the addition of the --gendata option, a user may utilize the randgen's gendata (table creation and population) tool
172
to populate a test server. In the following example, the test server is now populated by the 8 tables listed below::
174
./dbqp --mode=randgen --randgen-path=/randgen --start-and-exit --gendata=/randgen/conf/drizzle/drizzle.zz
176
24 Feb 2011 17:48:48 INFO: NAME: server0
177
24 Feb 2011 17:48:48 INFO: MASTER_PORT: 9306
178
24 Feb 2011 17:48:48 INFO: DRIZZLE_TCP_PORT: 9307
179
24 Feb 2011 17:48:48 INFO: MC_PORT: 9308
180
24 Feb 2011 17:48:48 INFO: PBMS_PORT: 9309
181
24 Feb 2011 17:48:48 INFO: RABBITMQ_NODE_PORT: 9310
182
24 Feb 2011 17:48:48 INFO: VARDIR: /home/pcrews/bzr/work/dbqp_randgen_updates/tests/workdir/testbot0/server0/var
183
24 Feb 2011 17:48:48 INFO: STATUS: 1
184
# 2011-02-24T17:48:48 Default schema: test
185
# 2011-02-24T17:48:48 Executor initialized, id GenTest::Executor::Drizzle 2011.02.2198 ()
186
# 2011-02-24T17:48:48 # Creating Drizzle table: test.A; engine: ; rows: 0 .
187
# 2011-02-24T17:48:48 # Creating Drizzle table: test.B; engine: ; rows: 0 .
188
# 2011-02-24T17:48:48 # Creating Drizzle table: test.C; engine: ; rows: 1 .
189
# 2011-02-24T17:48:48 # Creating Drizzle table: test.D; engine: ; rows: 1 .
190
# 2011-02-24T17:48:48 # Creating Drizzle table: test.AA; engine: ; rows: 10 .
191
# 2011-02-24T17:48:48 # Creating Drizzle table: test.BB; engine: ; rows: 10 .
192
# 2011-02-24T17:48:48 # Creating Drizzle table: test.CC; engine: ; rows: 100 .
193
# 2011-02-24T17:48:49 # Creating Drizzle table: test.DD; engine: ; rows: 100 .
194
24 Feb 2011 17:48:49 INFO: User specified --start-and-exit. dbqp.py exiting and leaving servers running...
added
removed
docs/testing/randgen.rst
