1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
|
CVE IMPORT PROCESS
==================
Launchpad has a cronscript (in "cronscripts/update-cve.py") that will fetch
the latest CVE database, in XML format, from the web, and then ensure that
the CVE data in it is correctly reflected in the local Launchpad database.
It takes care to reflect the correct description, status (Entry, Candidate,
or Deprecated), as well as all the References for that CVE entry.
This documentation test will describe and test the behaviour of this
infrastructure. Note that it assumes that the XML format is not changing,
and it does not attempt to connect to the outside world to verify that is
the case.
>>> import subprocess, os.path, sys
>>> import transaction
>>> from lp.services.config import config
OK. So now lets import the first XML database. First, lets se how many CVE
entries are in the database.
>>> from lp.bugs.model.cve import Cve
>>> print Cve.select().count()
10
>>> script = os.path.join(config.root, 'cronscripts', 'update-cve.py')
>>> url_base = os.path.join(
... config.root, 'lib', 'lp', 'bugs', 'tests', 'data'
... )
Now run the cronscript.
>>> url = 'file://%s' % os.path.join(url_base, 'cvedb_init.xml.gz')
>>> process = subprocess.Popen(
... [sys.executable, script, '-u', url], stdin=subprocess.PIPE,
... stdout=subprocess.PIPE, stderr=subprocess.STDOUT,
... )
>>> (output, empty) = process.communicate()
>>> print output
INFO Creating lockfile: /var/lock/launchpad-updatecve.lock
...
INFO CVE-1999-0002 created
INFO Creating new SGI reference for 1999-0002
INFO Creating new BID reference for 1999-0002
INFO Creating new XF reference for 1999-0002
INFO CVE-1999-0003 created
INFO Creating new CERT reference for 1999-0003
INFO Creating new SGI reference for 1999-0003
INFO Creating new XF reference for 1999-0003
INFO Creating new XF reference for 1999-0003
INFO CVE-1999-0005 created
INFO Creating new CERT reference for 1999-0005
INFO Creating new BID reference for 1999-0005
INFO CVE-1999-0007 created
INFO Creating new CERT reference for 1999-0007
INFO Creating new XF reference for 1999-0007
INFO CVE-2005-2761 created
INFO Creating new MISC reference for 2005-2761
INFO Creating new DEBIAN reference for 2005-2761
INFO CVE-2005-2762 created
INFO CVE-2005-2763 created
INFO CVE-2005-2764 created
...
<BLANKLINE>
And lets make sure we got the right number of CVE entries.
>>> transaction.commit()
>>> print Cve.select().count()
18
We will make a note of the CVE modification time of 1999-0002. When we
update it later, we can use this modification time to check that its
modification time is being updated correctly.
>>> c = Cve.bySequence('2005-2734')
>>> mod_time = c.datemodified
And while we are here, make a note of the number of references for that CVE
entry.
>>> print c.references.count()
6
Now, let's run an import of the update db.
>>> url = 'file://%s' % os.path.join(url_base, 'cvedb_update.xml.gz')
>>> process = subprocess.Popen(
... [sys.executable, script, '-u', url], stdin=subprocess.PIPE,
... stdout=subprocess.PIPE, stderr=subprocess.STDOUT,
... )
>>> (output, empty) = process.communicate()
>>> print output
INFO Creating lockfile: /var/lock/launchpad-updatecve.lock
...
INFO Creating new CERT reference for 1999-0002
INFO Creating new CIAC reference for 1999-0002
INFO Creating new NAI reference for 1999-0003
INFO Creating new SGI reference for 1999-0003
INFO Creating new BID reference for 1999-0003
INFO Creating new SUN reference for 1999-0005
INFO Creating new XF reference for 1999-0005
INFO CVE-1999-0006 created
INFO Creating new CERT reference for 1999-0006
INFO Creating new SGI reference for 1999-0006
INFO Creating new AUSCERT reference for 1999-0006
INFO Creating new XF reference for 1999-0006
INFO Creating new BID reference for 1999-0006
INFO CVE-1999-0007 updated description
INFO Creating new SECUNIA reference for 2005-2761
INFO CVE-2005-2734 updated description
INFO Removing BID reference for 2005-2734
INFO Removing BUGTRAQ reference for 2005-2734
INFO Removing CONFIRM reference for 2005-2734
INFO Removing SECTRACK reference for 2005-2734
INFO Removing SECUNIA reference for 2005-2734
INFO Removing XF reference for 2005-2734
INFO CVE-2005-2764 updated description
INFO CVE-2005-2765 created
INFO Creating new MSKB reference for 2005-2765
INFO CVE-2005-2766 created
INFO Creating new BUGTRAQ reference for 2005-2766
...
<BLANKLINE>
Let's make sure we got the new CVE's.
>>> transaction.commit()
>>> print Cve.select().count()
21
And lets make sure the modification time of 2005-2734 was updated, as were
the number of comments.
>>> c.sync()
>>> print mod_time < c.datemodified
True
>>> print c.references.count()
0
Script class
------------
The class the cronscript uses to do its work is CVEUpdater.
>>> from lp.bugs.scripts.cveimport import CVEUpdater
The method that does all the processing of the CVE XML is
processCVEXML(). It accepts that XML as a string. If no CVEs are found
in the XML, a LaunchpadScriptFailure is raised. This helps us catch
issues that causes no CVEs to be processed, for example if the XML
schema is changed.
>>> cve_updater = CVEUpdater('cve-updater', test_args=[])
>>> cve_updater.processCVEXML('<some-xml />')
Traceback (most recent call last):
...
LaunchpadScriptFailure: No CVEs found in XML file.
|