2144.1.1
by patrick crews
Overhaul of code. We can run rabbitmq : ) We now better encapsulate a per-executor working environment = one step closer to --parallel >: ) using subprocess goodness for server control |
1 |
#! /usr/bin/env python
|
2235.5.2
by Stewart Smith
dbqp source (and all its libs) should use emacs python mode, not emacs C mode |
2 |
# -*- mode: python; indent-tabs-mode: nil; -*-
|
2088.9.1
by patrick crews
Updated tree so that test-run.pl and test-run.py may live together in peace for a time |
3 |
# vim:expandtab:shiftwidth=2:tabstop=2:smarttab:
|
4 |
#
|
|
2194.2.1
by patrick crews
Integrated randgen with dbqp. We now have mode=randgen and a set of randgen test suites (very basic now). Output = same as dtr : ) We also have mode=cleanup to kill any servers we have started. Docs updates too. Gendata utility allows us to populate test servers |
5 |
# Copyright (C) 2010,2011 Patrick Crews
|
2088.9.1
by patrick crews
Updated tree so that test-run.pl and test-run.py may live together in peace for a time |
6 |
#
|
2121.3.2
by patrick crews
Updated license verbiage |
7 |
# This program is free software; you can redistribute it and/or modify
|
8 |
# it under the terms of the GNU General Public License as published by
|
|
9 |
# the Free Software Foundation; either version 2 of the License, or
|
|
10 |
# (at your option) any later version.
|
|
11 |
#
|
|
12 |
# This program is distributed in the hope that it will be useful,
|
|
13 |
# but WITHOUT ANY WARRANTY; without even the implied warranty of
|
|
14 |
# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
|
|
15 |
# GNU General Public License for more details.
|
|
16 |
#
|
|
17 |
# You should have received a copy of the GNU General Public License
|
|
18 |
# along with this program; if not, write to the Free Software
|
|
19 |
# Foundation, Inc., 51 Franklin St, Fifth Floor, Boston, MA 02110-1301 USA
|
|
2121.3.1
by patrick crews
Added licensing text to dbqp files |
20 |
|
2088.9.1
by patrick crews
Updated tree so that test-run.pl and test-run.py may live together in peace for a time |
21 |
|
22 |
""" drizzled.py: code to allow a serverManager
|
|
23 |
to provision and start up a drizzled server object
|
|
24 |
for test execution
|
|
25 |
||
26 |
"""
|
|
27 |
||
28 |
# imports
|
|
29 |
import os |
|
2194.2.1
by patrick crews
Integrated randgen with dbqp. We now have mode=randgen and a set of randgen test suites (very basic now). Output = same as dtr : ) We also have mode=cleanup to kill any servers we have started. Docs updates too. Gendata utility allows us to populate test servers |
30 |
from lib.server_mgmt.server import Server |
2088.9.1
by patrick crews
Updated tree so that test-run.pl and test-run.py may live together in peace for a time |
31 |
|
2194.2.1
by patrick crews
Integrated randgen with dbqp. We now have mode=randgen and a set of randgen test suites (very basic now). Output = same as dtr : ) We also have mode=cleanup to kill any servers we have started. Docs updates too. Gendata utility allows us to populate test servers |
32 |
class drizzleServer(Server): |
2088.9.1
by patrick crews
Updated tree so that test-run.pl and test-run.py may live together in peace for a time |
33 |
""" represents a drizzle server, its possessions
|
34 |
(datadir, ports, etc), and methods for controlling
|
|
35 |
and querying it
|
|
36 |
||
2144.1.1
by patrick crews
Overhaul of code. We can run rabbitmq : ) We now better encapsulate a per-executor working environment = one step closer to --parallel >: ) using subprocess goodness for server control |
37 |
TODO: create a base server class that contains
|
38 |
standard methods from which we can inherit
|
|
39 |
Currently there are definitely methods / attr
|
|
40 |
which are general
|
|
41 |
||
2088.9.1
by patrick crews
Updated tree so that test-run.pl and test-run.py may live together in peace for a time |
42 |
"""
|
43 |
||
2194.2.1
by patrick crews
Integrated randgen with dbqp. We now have mode=randgen and a set of randgen test suites (very basic now). Output = same as dtr : ) We also have mode=cleanup to kill any servers we have started. Docs updates too. Gendata utility allows us to populate test servers |
44 |
def __init__( self, name, server_manager, default_storage_engine |
45 |
, server_options, requester, workdir_root): |
|
46 |
super(drizzleServer, self).__init__( name, server_manager |
|
47 |
, default_storage_engine |
|
48 |
, server_options, requester |
|
49 |
, workdir_root) |
|
2088.9.1
by patrick crews
Updated tree so that test-run.pl and test-run.py may live together in peace for a time |
50 |
self.code_tree = self.system_manager.code_tree |
51 |
self.preferred_base_port = 9306 |
|
2240.4.2
by patrick crews
Moved client file declarations earlier in init function for drizzle server |
52 |
|
53 |
# client files
|
|
54 |
self.drizzledump = self.code_tree.drizzledump |
|
55 |
self.drizzle_client = self.code_tree.drizzle_client |
|
56 |
self.drizzleimport = self.code_tree.drizzleimport |
|
57 |
self.drizzleadmin = self.code_tree.drizzleadmin |
|
58 |
self.drizzleslap = self.code_tree.drizzleslap |
|
59 |
self.server_path = self.code_tree.drizzle_server |
|
60 |
self.drizzle_client_path = self.code_tree.drizzle_client |
|
61 |
self.schemawriter = self.code_tree.schemawriter |
|
62 |
||
2088.9.1
by patrick crews
Updated tree so that test-run.pl and test-run.py may live together in peace for a time |
63 |
# Get our ports
|
64 |
self.port_block = self.system_manager.port_manager.get_port_block( self.name |
|
65 |
, self.preferred_base_port |
|
2283.4.9
by Stewart Smith
add JSON_SERVER_PORT to test-run.pl and dbqp. Make the basic json_server test actually test something useful, such as interaction between SQL from normal protocol and SQL from http |
66 |
, 6 ) |
2088.9.1
by patrick crews
Updated tree so that test-run.pl and test-run.py may live together in peace for a time |
67 |
self.master_port = self.port_block[0] |
68 |
self.drizzle_tcp_port = self.port_block[1] |
|
69 |
self.mc_port = self.port_block[2] |
|
70 |
self.pbms_port = self.port_block[3] |
|
2144.1.1
by patrick crews
Overhaul of code. We can run rabbitmq : ) We now better encapsulate a per-executor working environment = one step closer to --parallel >: ) using subprocess goodness for server control |
71 |
self.rabbitmq_node_port = self.port_block[4] |
2283.4.9
by Stewart Smith
add JSON_SERVER_PORT to test-run.pl and dbqp. Make the basic json_server test actually test something useful, such as interaction between SQL from normal protocol and SQL from http |
72 |
self.json_server_port = self.port_block[5] |
2088.9.1
by patrick crews
Updated tree so that test-run.pl and test-run.py may live together in peace for a time |
73 |
|
74 |
# Generate our working directories
|
|
75 |
self.dirset = { self.name : { 'var': {'std_data_ln':( os.path.join(self.code_tree.testdir,'std_data')) |
|
76 |
,'log':None |
|
77 |
,'run':None |
|
78 |
,'tmp':None |
|
79 |
,'master-data': {'local': { 'test':None |
|
80 |
, 'mysql':None |
|
81 |
}
|
|
82 |
}
|
|
83 |
}
|
|
84 |
}
|
|
85 |
}
|
|
86 |
self.workdir = self.system_manager.create_dirset( workdir_root |
|
87 |
, self.dirset) |
|
88 |
self.vardir = os.path.join(self.workdir,'var') |
|
89 |
self.tmpdir = os.path.join(self.vardir,'tmp') |
|
90 |
self.rundir = os.path.join(self.vardir,'run') |
|
91 |
self.logdir = os.path.join(self.vardir,'log') |
|
92 |
self.datadir = os.path.join(self.vardir,'master-data') |
|
93 |
||
94 |
self.error_log = os.path.join(self.logdir,('%s.err' %(self.name))) |
|
95 |
self.pid_file = os.path.join(self.rundir,('%s.pid' %(self.name))) |
|
96 |
self.socket_file = os.path.join(self.vardir, ('%s.sock' %(self.name))) |
|
97 |
self.timer_file = os.path.join(self.logdir,('timer')) |
|
2194.2.1
by patrick crews
Integrated randgen with dbqp. We now have mode=randgen and a set of randgen test suites (very basic now). Output = same as dtr : ) We also have mode=cleanup to kill any servers we have started. Docs updates too. Gendata utility allows us to populate test servers |
98 |
|
99 |
# Do magic to create a config file for use with the slave
|
|
100 |
# plugin
|
|
101 |
self.slave_config_file = os.path.join(self.logdir,'slave.cnf') |
|
102 |
self.create_slave_config_file() |
|
103 |
||
2088.9.1
by patrick crews
Updated tree so that test-run.pl and test-run.py may live together in peace for a time |
104 |
self.snapshot_path = os.path.join(self.tmpdir,('snapshot_%s' %(self.master_port))) |
105 |
# We want to use --secure-file-priv = $vardir by default
|
|
106 |
# but there are times / tools when we need to shut this off
|
|
107 |
if self.no_secure_file_priv: |
|
108 |
self.secure_file_string = '' |
|
109 |
else: |
|
110 |
self.secure_file_string = "--secure-file-priv='%s'" %(self.vardir) |
|
111 |
self.user_string = '--user=root' |
|
112 |
||
113 |
self.initialize_databases() |
|
114 |
self.take_db_snapshot() |
|
115 |
||
116 |
if self.debug: |
|
117 |
self.logging.debug_class(self) |
|
118 |
||
119 |
def report(self): |
|
120 |
""" We print out some general useful info """
|
|
121 |
report_values = [ 'name' |
|
122 |
, 'master_port' |
|
123 |
, 'drizzle_tcp_port' |
|
124 |
, 'mc_port' |
|
125 |
, 'pbms_port' |
|
2144.1.1
by patrick crews
Overhaul of code. We can run rabbitmq : ) We now better encapsulate a per-executor working environment = one step closer to --parallel >: ) using subprocess goodness for server control |
126 |
, 'rabbitmq_node_port' |
2088.9.1
by patrick crews
Updated tree so that test-run.pl and test-run.py may live together in peace for a time |
127 |
, 'vardir' |
128 |
, 'status' |
|
129 |
]
|
|
2194.2.1
by patrick crews
Integrated randgen with dbqp. We now have mode=randgen and a set of randgen test suites (very basic now). Output = same as dtr : ) We also have mode=cleanup to kill any servers we have started. Docs updates too. Gendata utility allows us to populate test servers |
130 |
self.logging.info("%s server:" %(self.owner)) |
2088.9.1
by patrick crews
Updated tree so that test-run.pl and test-run.py may live together in peace for a time |
131 |
for key in report_values: |
132 |
value = vars(self)[key] |
|
133 |
self.logging.info("%s: %s" %(key.upper(), value)) |
|
134 |
||
2144.1.1
by patrick crews
Overhaul of code. We can run rabbitmq : ) We now better encapsulate a per-executor working environment = one step closer to --parallel >: ) using subprocess goodness for server control |
135 |
def get_start_cmd(self): |
136 |
""" Return the command string that will start up the server
|
|
137 |
as desired / intended
|
|
138 |
|
|
2088.9.1
by patrick crews
Updated tree so that test-run.pl and test-run.py may live together in peace for a time |
139 |
"""
|
2151.8.4
by patrick crews
gdb! : ) We try to set things up to launch the native Mac Terminal if we are running on Darwin |
140 |
|
141 |
server_args = [ self.process_server_options() |
|
142 |
, "--mysql-protocol.port=%d" %(self.master_port) |
|
143 |
, "--mysql-protocol.connect-timeout=60" |
|
144 |
, "--innodb.data-file-path=ibdata1:20M:autoextend" |
|
145 |
, "--sort-buffer-size=256K" |
|
146 |
, "--max-heap-table-size=1M" |
|
147 |
, "--mysql-unix-socket-protocol.path=%s" %(self.socket_file) |
|
148 |
, "--pid-file=%s" %(self.pid_file) |
|
149 |
, "--drizzle-protocol.port=%d" %(self.drizzle_tcp_port) |
|
2167.4.1
by patrick crews
Fixed default-storage-engine option, minor fixes + added repeat option, will run each testcase in a sequence times |
150 |
, "--default-storage-engine=%s" %(self.default_storage_engine) |
2151.8.4
by patrick crews
gdb! : ) We try to set things up to launch the native Mac Terminal if we are running on Darwin |
151 |
, "--datadir=%s" %(self.datadir) |
152 |
, "--tmpdir=%s" %(self.tmpdir) |
|
153 |
, self.secure_file_string |
|
154 |
, self.user_string |
|
155 |
]
|
|
156 |
||
157 |
if self.gdb: |
|
158 |
server_args.append('--gdb') |
|
159 |
return self.system_manager.handle_gdb_reqs(self, server_args) |
|
160 |
else: |
|
161 |
return "%s %s %s & " % ( self.cmd_prefix |
|
162 |
, self.server_path |
|
163 |
, " ".join(server_args) |
|
164 |
)
|
|
165 |
||
2144.1.1
by patrick crews
Overhaul of code. We can run rabbitmq : ) We now better encapsulate a per-executor working environment = one step closer to --parallel >: ) using subprocess goodness for server control |
166 |
|
167 |
def get_stop_cmd(self): |
|
168 |
""" Return the command that will shut us down """
|
|
2088.9.1
by patrick crews
Updated tree so that test-run.pl and test-run.py may live together in peace for a time |
169 |
|
2144.1.1
by patrick crews
Overhaul of code. We can run rabbitmq : ) We now better encapsulate a per-executor working environment = one step closer to --parallel >: ) using subprocess goodness for server control |
170 |
return "%s --user=root --port=%d --shutdown " %(self.drizzle_client_path, self.master_port) |
2088.9.1
by patrick crews
Updated tree so that test-run.pl and test-run.py may live together in peace for a time |
171 |
|
172 |
||
2144.1.1
by patrick crews
Overhaul of code. We can run rabbitmq : ) We now better encapsulate a per-executor working environment = one step closer to --parallel >: ) using subprocess goodness for server control |
173 |
def get_ping_cmd(self): |
174 |
"""Return the command string that will
|
|
175 |
ping / check if the server is alive
|
|
176 |
||
177 |
"""
|
|
178 |
||
179 |
return "%s --ping --port=%d --user=root" % (self.drizzle_client_path, self.master_port) |
|
2088.9.1
by patrick crews
Updated tree so that test-run.pl and test-run.py may live together in peace for a time |
180 |
|
2194.2.1
by patrick crews
Integrated randgen with dbqp. We now have mode=randgen and a set of randgen test suites (very basic now). Output = same as dtr : ) We also have mode=cleanup to kill any servers we have started. Docs updates too. Gendata utility allows us to populate test servers |
181 |
def is_started(self): |
182 |
""" Determine if the server is up and running -
|
|
183 |
this may vary from server type to server type
|
|
2088.9.1
by patrick crews
Updated tree so that test-run.pl and test-run.py may live together in peace for a time |
184 |
|
185 |
"""
|
|
2194.2.1
by patrick crews
Integrated randgen with dbqp. We now have mode=randgen and a set of randgen test suites (very basic now). Output = same as dtr : ) We also have mode=cleanup to kill any servers we have started. Docs updates too. Gendata utility allows us to populate test servers |
186 |
|
187 |
# We experiment with waiting for a pid file to be created vs. pinging
|
|
188 |
# This is what test-run.pl does and it helps us pass logging_stats tests
|
|
189 |
# while not self.ping_server(server, quiet=True) and timer != timeout:
|
|
190 |
||
191 |
return self.system_manager.find_path( [self.pid_file] |
|
192 |
, required=0) |
|
193 |
||
194 |
def create_slave_config_file(self): |
|
195 |
""" Create a config file suitable for use
|
|
196 |
with the slave-plugin. This allows
|
|
197 |
us to tie other servers in easily
|
|
198 |
||
199 |
"""
|
|
200 |
||
2260.1.1
by David Shrewsbury
Revert multi-master code |
201 |
config_data = [ "master-host=127.0.0.1" |
2194.2.1
by patrick crews
Integrated randgen with dbqp. We now have mode=randgen and a set of randgen test suites (very basic now). Output = same as dtr : ) We also have mode=cleanup to kill any servers we have started. Docs updates too. Gendata utility allows us to populate test servers |
202 |
, "master-port=%d" %self.master_port |
203 |
, "master-user=root" |
|
204 |
, "master-pass=''" |
|
205 |
, "max-reconnects=100" |
|
2239.7.4
by patrick crews
Tweaks to drizzled.py's slave.cnf file to play with multi-master rpl |
206 |
#, "seconds-between-reconnects=20"
|
2194.2.1
by patrick crews
Integrated randgen with dbqp. We now have mode=randgen and a set of randgen test suites (very basic now). Output = same as dtr : ) We also have mode=cleanup to kill any servers we have started. Docs updates too. Gendata utility allows us to populate test servers |
207 |
]
|
208 |
outfile = open(self.slave_config_file,'w') |
|
209 |
for line in config_data: |
|
210 |
outfile.write("%s\n" %(line)) |
|
211 |
outfile.close() |
|
212 |
||
213 |
||
214 |
||
215 |
||
2088.9.1
by patrick crews
Updated tree so that test-run.pl and test-run.py may live together in peace for a time |
216 |
|
2194.2.1
by patrick crews
Integrated randgen with dbqp. We now have mode=randgen and a set of randgen test suites (very basic now). Output = same as dtr : ) We also have mode=cleanup to kill any servers we have started. Docs updates too. Gendata utility allows us to populate test servers |
217 |
|
218 |
||
219 |
||
220 |
||
221 |
||
2088.9.1
by patrick crews
Updated tree so that test-run.pl and test-run.py may live together in peace for a time |
222 |
|
223 |
||
224 |
||
225 |
||
226 |