~drizzle-trunk/drizzle/development

2144.1.1 by patrick crews
Overhaul of code. We can run rabbitmq : ) We now better encapsulate a per-executor working environment = one step closer to --parallel >: ) using subprocess goodness for server control
1
#! /usr/bin/env python
2235.5.2 by Stewart Smith
dbqp source (and all its libs) should use emacs python mode, not emacs C mode
2
# -*- mode: python; indent-tabs-mode: nil; -*-
2088.9.1 by patrick crews
Updated tree so that test-run.pl and test-run.py may live together in peace for a time
3
# vim:expandtab:shiftwidth=2:tabstop=2:smarttab:
4
#
5
# Copyright (C) 2010 Patrick Crews
6
#
2121.3.1 by patrick crews
Added licensing text to dbqp files
7
#
2121.3.2 by patrick crews
Updated license verbiage
8
# This program is free software; you can redistribute it and/or modify
9
# it under the terms of the GNU General Public License as published by
10
# the Free Software Foundation; either version 2 of the License, or
11
# (at your option) any later version.
12
#
13
# This program is distributed in the hope that it will be useful,
14
# but WITHOUT ANY WARRANTY; without even the implied warranty of
15
# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
16
# GNU General Public License for more details.
17
#
18
# You should have received a copy of the GNU General Public License
19
# along with this program; if not, write to the Free Software
20
# Foundation, Inc., 51 Franklin St, Fifth Floor, Boston, MA  02110-1301  USA
2121.3.1 by patrick crews
Added licensing text to dbqp files
21
2088.9.1 by patrick crews
Updated tree so that test-run.pl and test-run.py may live together in peace for a time
22
""" dtr_test_execution:
23
    code related to the execution of dtr test cases 
24
    
25
    We are provided access to a testManager with 
26
    dtr-specific testCases.  We contact teh executionManager
27
    to produce the system and server configurations we need
28
    to execute a test.
29
30
"""
31
32
# imports
33
import os
34
import sys
35
import subprocess
36
import commands
37
2337.1.12 by patrick crews
Additional work on allowing dbqp to handle multiple server types / versions (even simultaneously)
38
import lib.test_mgmt.test_execution as test_execution
2088.9.1 by patrick crews
Updated tree so that test-run.pl and test-run.py may live together in peace for a time
39
2194.2.1 by patrick crews
Integrated randgen with dbqp. We now have mode=randgen and a set of randgen test suites (very basic now). Output = same as dtr : ) We also have mode=cleanup to kill any servers we have started. Docs updates too. Gendata utility allows us to populate test servers
40
class testExecutor(test_execution.testExecutor):
2088.9.1 by patrick crews
Updated tree so that test-run.pl and test-run.py may live together in peace for a time
41
    """ dtr-specific testExecutor 
42
        We currently execute by sending test-case
43
        data to client/drizzletest...for now
44
45
    """
46
  
2088.9.18 by patrick crews
Updates to allow for timing of test cases and reporting and whatnot
47
    def execute_testCase (self):
2088.9.1 by patrick crews
Updated tree so that test-run.pl and test-run.py may live together in peace for a time
48
        """ Execute a dtr testCase via calls to drizzletest (boo)
49
            Eventually, we will replace drizzletest with pythonic
50
            goodness, but we have these classes stored here for the moment
51
52
        """
2088.9.18 by patrick crews
Updates to allow for timing of test cases and reporting and whatnot
53
        test_execution.testExecutor.execute_testCase(self)
2088.9.1 by patrick crews
Updated tree so that test-run.pl and test-run.py may live together in peace for a time
54
        self.status = 0
2088.9.18 by patrick crews
Updates to allow for timing of test cases and reporting and whatnot
55
56
        # generate command line
57
        drizzletest_cmd = self.generate_drizzletest_call()
58
59
        # call drizzletest
60
        self.execute_drizzletest(drizzletest_cmd)
61
62
        # analyze results
63
        self.current_test_status = self.process_drizzletest_output()
64
        self.set_server_status(self.current_test_status)
2088.9.4 by patrick crews
Updates to server management and signalling a bad server start
65
 
2088.9.1 by patrick crews
Updated tree so that test-run.pl and test-run.py may live together in peace for a time
66
67
    def generate_drizzletest_call(self):
68
        """ Produce the command line we use to call drizzletest
69
            We have a healthy number of values, so we put this in a 
70
            nice function
71
72
        """
73
74
        drizzletest_arguments = [ '--no-defaults'
75
                                , '--silent'
76
                                , '--tmpdir=%s' %(self.master_server.tmpdir)
77
                                , '--logdir=%s' %(self.master_server.logdir)
78
                                , '--port=%d' %(self.master_server.master_port)
79
                                , '--database=test'
80
                                , '--user=root'
81
                                , '--password='
82
                                #, '--testdir=%s' %(self.test_manager.testdir)
83
                                , '--test-file=%s' %(self.current_testcase.testpath)
84
                                , '--tail-lines=20'
85
                                , '--timer-file=%s' %(self.master_server.timer_file)
86
                                , '--result-file=%s' %(self.current_testcase.resultpath)
87
                                ]
88
        if self.record_flag:
89
            # We want to record a new result
90
            drizzletest_arguments.append('--record')
2088.9.24 by patrick crews
Added code to handle valgrind. Currently not working - getting bad error on malloc-fill=DEADBEEF : (, checking in anyway as it is largely there
91
        drizzletest_cmd = "%s %s %s" %( self.cmd_prefix
2337.1.12 by patrick crews
Additional work on allowing dbqp to handle multiple server types / versions (even simultaneously)
92
                                      , self.master_server.code_tree.drizzletest
2088.9.24 by patrick crews
Added code to handle valgrind. Currently not working - getting bad error on malloc-fill=DEADBEEF : (, checking in anyway as it is largely there
93
                                      , " ".join(drizzletest_arguments))
2088.9.1 by patrick crews
Updated tree so that test-run.pl and test-run.py may live together in peace for a time
94
        return drizzletest_cmd
95
96
    def execute_drizzletest(self, drizzletest_cmd):
97
        """ Execute the commandline and return the result.
98
            We use subprocess as we can pass os.environ dicts and whatnot 
99
100
        """
2088.9.18 by patrick crews
Updates to allow for timing of test cases and reporting and whatnot
101
        testcase_name = self.current_testcase.fullname
102
        self.time_manager.start(testcase_name,'test')
2148.1.5 by patrick crews
Some code cleanup + bug fix + setting up for parallel - using subprocess module for drizzletest calls >: )
103
        #retcode, output = self.system_manager.execute_cmd( drizzletest_cmd
104
                                                 #         , must_pass = 0 )
105
        drizzletest_outfile = os.path.join(self.logdir,'drizzletest.out')
106
        drizzletest_output = open(drizzletest_outfile,'w')
107
        drizzletest_subproc = subprocess.Popen( drizzletest_cmd
108
                                         , shell=True
2373.2.1 by patrick crews
Minor tweak to dbqp's dtr_execution_manager for Brian's valgrind fixes
109
                                         , cwd=self.system_manager.testdir
2148.1.5 by patrick crews
Some code cleanup + bug fix + setting up for parallel - using subprocess module for drizzletest calls >: )
110
                                         , env=self.working_environment
111
                                         , stdout = drizzletest_output
112
                                         , stderr = subprocess.STDOUT
113
                                         )
114
        drizzletest_subproc.wait()
115
        retcode = drizzletest_subproc.returncode     
2088.9.18 by patrick crews
Updates to allow for timing of test cases and reporting and whatnot
116
        execution_time = int(self.time_manager.stop(testcase_name)*1000) # millisec
2088.9.1 by patrick crews
Updated tree so that test-run.pl and test-run.py may live together in peace for a time
117
2148.1.5 by patrick crews
Some code cleanup + bug fix + setting up for parallel - using subprocess module for drizzletest calls >: )
118
        drizzletest_output.close()
119
        drizzletest_file = open(drizzletest_outfile,'r')
2151.8.3 by patrick crews
Fix to drizzletest output handling
120
        output = ''.join(drizzletest_file.readlines())
2148.1.5 by patrick crews
Some code cleanup + bug fix + setting up for parallel - using subprocess module for drizzletest calls >: )
121
        drizzletest_file.close()
122
2337.1.4 by patrick crews
Tweak of logging behavior (no more 'if debug' checks...) and related code cleanup. Also groundwork for allowing multiple basedirs (server code bases) to the test runner)
123
        self.logging.debug("drizzletest_retcode: %d" %(retcode))
2088.9.18 by patrick crews
Updates to allow for timing of test cases and reporting and whatnot
124
        self.current_test_retcode = retcode
125
        self.current_test_output = output
126
        self.current_test_exec_time = execution_time
2088.9.1 by patrick crews
Updated tree so that test-run.pl and test-run.py may live together in peace for a time
127
2088.9.18 by patrick crews
Updates to allow for timing of test cases and reporting and whatnot
128
    def process_drizzletest_output(self):
2088.9.1 by patrick crews
Updated tree so that test-run.pl and test-run.py may live together in peace for a time
129
        """ Drizzletest has run, we now check out what we have """
2088.9.18 by patrick crews
Updates to allow for timing of test cases and reporting and whatnot
130
        retcode = self.current_test_retcode
2088.9.1 by patrick crews
Updated tree so that test-run.pl and test-run.py may live together in peace for a time
131
        if retcode == 0:
132
            return 'pass'
2088.9.3 by patrick crews
Update to properly catch skipped test from drizzletest
133
        elif retcode == 62 or retcode == 15872:
2088.9.1 by patrick crews
Updated tree so that test-run.pl and test-run.py may live together in peace for a time
134
            return 'skipped'
135
        elif retcode == 63 or retcode == 1:
136
            return 'fail'
137
        else:
138
            return 'fail'
139