~drizzle-trunk/drizzle/development

2194.2.1 by patrick crews
Integrated randgen with dbqp. We now have mode=randgen and a set of randgen test suites (very basic now). Output = same as dtr : ) We also have mode=cleanup to kill any servers we have started. Docs updates too. Gendata utility allows us to populate test servers
1
#! /usr/bin/env python
2235.5.2 by Stewart Smith
dbqp source (and all its libs) should use emacs python mode, not emacs C mode
2
# -*- mode: python; indent-tabs-mode: nil; -*-
2194.2.1 by patrick crews
Integrated randgen with dbqp. We now have mode=randgen and a set of randgen test suites (very basic now). Output = same as dtr : ) We also have mode=cleanup to kill any servers we have started. Docs updates too. Gendata utility allows us to populate test servers
3
# vim:expandtab:shiftwidth=2:tabstop=2:smarttab:
4
#
5
# Copyright (C) 2010 Patrick Crews
6
#
7
#
8
# This program is free software; you can redistribute it and/or modify
9
# it under the terms of the GNU General Public License as published by
10
# the Free Software Foundation; either version 2 of the License, or
11
# (at your option) any later version.
12
#
13
# This program is distributed in the hope that it will be useful,
14
# but WITHOUT ANY WARRANTY; without even the implied warranty of
15
# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
16
# GNU General Public License for more details.
17
#
18
# You should have received a copy of the GNU General Public License
19
# along with this program; if not, write to the Free Software
20
# Foundation, Inc., 51 Franklin St, Fifth Floor, Boston, MA  02110-1301  USA
21
22
""" dtr_test_execution:
23
    code related to the execution of dtr test cases 
24
    
25
    We are provided access to a testManager with 
2317.1.1 by patrick crews
Updates to dbqp to add a sysbench mode. Created test suites to duplicate readonly / readwrite drizzle-automation tests. Still needs some work, but tests execute
26
    randgen-specific testCases.  We contact the executionManager
2194.2.1 by patrick crews
Integrated randgen with dbqp. We now have mode=randgen and a set of randgen test suites (very basic now). Output = same as dtr : ) We also have mode=cleanup to kill any servers we have started. Docs updates too. Gendata utility allows us to populate test servers
27
    to produce the system and server configurations we need
28
    to execute a test.
29
30
"""
31
32
# imports
33
import os
34
import sys
35
import subprocess
36
import commands
37
38
import lib.test_mgmt.test_execution as test_execution
39
40
class randgenTestExecutor(test_execution.testExecutor):
2317.1.1 by patrick crews
Updates to dbqp to add a sysbench mode. Created test suites to duplicate readonly / readwrite drizzle-automation tests. Still needs some work, but tests execute
41
    """ randgen-specific testExecutor 
2194.2.1 by patrick crews
Integrated randgen with dbqp. We now have mode=randgen and a set of randgen test suites (very basic now). Output = same as dtr : ) We also have mode=cleanup to kill any servers we have started. Docs updates too. Gendata utility allows us to populate test servers
42
43
    """
44
  
45
    def execute_testCase (self):
2317.1.1 by patrick crews
Updates to dbqp to add a sysbench mode. Created test suites to duplicate readonly / readwrite drizzle-automation tests. Still needs some work, but tests execute
46
        """ Execute a randgen testCase
2194.2.1 by patrick crews
Integrated randgen with dbqp. We now have mode=randgen and a set of randgen test suites (very basic now). Output = same as dtr : ) We also have mode=cleanup to kill any servers we have started. Docs updates too. Gendata utility allows us to populate test servers
47
48
        """
49
        test_execution.testExecutor.execute_testCase(self)
50
        self.status = 0
51
52
        # execute the randgen
53
        self.execute_randgen()
54
55
        # analyze results
56
        self.current_test_status = self.process_randgen_output()
57
        self.set_server_status(self.current_test_status)
2225.5.6 by patrick crews
Added cases for crash tests for innodb log + slave plugin
58
        self.server_manager.reset_servers(self.name)
2194.2.1 by patrick crews
Integrated randgen with dbqp. We now have mode=randgen and a set of randgen test suites (very basic now). Output = same as dtr : ) We also have mode=cleanup to kill any servers we have started. Docs updates too. Gendata utility allows us to populate test servers
59
 
60
61
    
62
63
    def execute_randgen(self):
64
        """ Execute the commandline and return the result.
65
            We use subprocess as we can pass os.environ dicts and whatnot 
66
67
        """
68
      
69
        testcase_name = self.current_testcase.fullname
70
        self.time_manager.start(testcase_name,'test')
71
        randgen_outfile = os.path.join(self.logdir,'randgen.out')
72
        randgen_output = open(randgen_outfile,'w')
73
        dsn = "--dsn=dbi:drizzle:host=localhost:port=%d:user=root:password="":database=test" %(self.master_server.master_port)
74
        randgen_cmd = " ".join([self.current_testcase.test_command, dsn])
75
        randgen_subproc = subprocess.Popen( randgen_cmd
76
                                         , shell=True
77
                                         , cwd=self.system_manager.randgen_path
78
                                         , env=self.working_environment
79
                                         , stdout = randgen_output
80
                                         , stderr = subprocess.STDOUT
81
                                         )
82
        randgen_subproc.wait()
83
        retcode = randgen_subproc.returncode     
84
        execution_time = int(self.time_manager.stop(testcase_name)*1000) # millisec
85
86
        randgen_output.close()
87
        randgen_file = open(randgen_outfile,'r')
88
        output = ''.join(randgen_file.readlines())
2225.5.6 by patrick crews
Added cases for crash tests for innodb log + slave plugin
89
        if self.debug:
90
            self.logging.debug(output)
2194.2.1 by patrick crews
Integrated randgen with dbqp. We now have mode=randgen and a set of randgen test suites (very basic now). Output = same as dtr : ) We also have mode=cleanup to kill any servers we have started. Docs updates too. Gendata utility allows us to populate test servers
91
        randgen_file.close()
92
93
        if self.debug:
94
            self.logging.debug("randgen_retcode: %d" %(retcode))
95
        self.current_test_retcode = retcode
96
        self.current_test_output = output
97
        self.current_test_exec_time = execution_time
98
99
    def process_randgen_output(self):
100
        """ randgen has run, we now check out what we have """
101
        retcode = self.current_test_retcode
102
        if retcode == 0:
103
            return 'pass'
104
        else:
105
            return 'fail'
106
107
    def handle_system_reqs(self):
108
        """ We check our test case and see what we need to do
109
            system-wise to get ready.  This is likely to be 
110
            mode-dependent and this is just a placeholder
111
            method
112
113
        """
114
115
        self.process_environment_reqs()
116
        self.process_symlink_reqs()
117
        self.process_master_sh()  
118
        return
119
120
    def process_master_sh(self):
121
        """ We do what we need to if we have a master.sh file """
122
        if self.current_testcase.master_sh:
123
            retcode, output = self.system_manager.execute_cmd("/bin/sh %s" %(self.current_testcase.master_sh))
124
            if self.debug:
125
                self.logging.info("retcode: %retcode")
126
                self.logging.info("%output")
127
128
    def process_environment_reqs(self):
129
        """ We generate the ENV vars we need set
130
            and then ask systemManager to do so
131
132
        """
2207.5.1 by patrick crews
Did what was needed to run our trx log / slave tests. We now communicate additional information via environment variables that the randgen can use to communicate with the proper server(s). Modified the randgen as well
133
        env_reqs = {  'DRIZZLETEST_VARDIR': self.master_server.vardir
2194.2.1 by patrick crews
Integrated randgen with dbqp. We now have mode=randgen and a set of randgen test suites (very basic now). Output = same as dtr : ) We also have mode=cleanup to kill any servers we have started. Docs updates too. Gendata utility allows us to populate test servers
134
                   ,  'DRIZZLE_TMP_DIR': self.master_server.tmpdir
135
                   ,  'MASTER_MYSOCK': self.master_server.socket_file
136
                   ,  'MASTER_MYPORT': str(self.master_server.master_port)
137
                   ,  'MC_PORT': str(self.master_server.mc_port)
138
                   ,  'PBMS_PORT': str(self.master_server.pbms_port)
139
                   ,  'RABBITMQ_NODE_PORT': str(self.master_server.rabbitmq_node_port)
140
                   ,  'DRIZZLE_TCP_PORT': str(self.master_server.drizzle_tcp_port)
141
                   ,  'EXE_DRIZZLE': self.master_server.drizzle_client
142
                   ,  'MASTER_SERVER_SLAVE_CONFIG' : self.master_server.slave_config_file
143
                   ,  'DRIZZLE_DUMP': "%s --no-defaults -uroot -p%d" %( self.master_server.drizzledump
144
                                                        , self.master_server.master_port)
145
                   ,  'DRIZZLE_SLAP': "%s -uroot -p%d" %( self.master_server.drizzleslap
146
                                                        , self.master_server.master_port)
147
                   ,  'DRIZZLE_IMPORT': "%s -uroot -p%d" %( self.master_server.drizzleimport
148
                                                          , self.master_server.master_port)
149
                   ,  'DRIZZLE': "%s -uroot -p%d" %( self.master_server.drizzle_client
150
                                                   , self.master_server.master_port)
2258.1.1 by patrick crews
Made the transaction_reader utility part of the codeTree. We now export the path an an env var that can be more easily called by drizzletest. Made adjustments to trx_log tests that need this
151
                   ,  'DRIZZLE_BASEDIR' : self.system_manager.code_tree.basedir
152
                   ,  'DRIZZLE_TRX_READER' : self.system_manager.code_tree.drizzle_trx_reader
2194.2.1 by patrick crews
Integrated randgen with dbqp. We now have mode=randgen and a set of randgen test suites (very basic now). Output = same as dtr : ) We also have mode=cleanup to kill any servers we have started. Docs updates too. Gendata utility allows us to populate test servers
153
                   }     
154
2207.5.1 by patrick crews
Did what was needed to run our trx log / slave tests. We now communicate additional information via environment variables that the randgen can use to communicate with the proper server(s). Modified the randgen as well
155
2194.2.1 by patrick crews
Integrated randgen with dbqp. We now have mode=randgen and a set of randgen test suites (very basic now). Output = same as dtr : ) We also have mode=cleanup to kill any servers we have started. Docs updates too. Gendata utility allows us to populate test servers
156
        self.working_environment = self.system_manager.create_working_environment(env_reqs)
157
2207.5.1 by patrick crews
Did what was needed to run our trx log / slave tests. We now communicate additional information via environment variables that the randgen can use to communicate with the proper server(s). Modified the randgen as well
158
2194.2.1 by patrick crews
Integrated randgen with dbqp. We now have mode=randgen and a set of randgen test suites (very basic now). Output = same as dtr : ) We also have mode=cleanup to kill any servers we have started. Docs updates too. Gendata utility allows us to populate test servers
159
    def process_symlink_reqs(self):
160
        """ Create any symlinks we may need """
161
        needed_symlinks = []
162
163
        self.system_manager.create_symlinks(needed_symlinks)
164
165
    
166
   
167
168