1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
|
Migrating translations between distro series
============================================
When a new release series is created for a distribution it usually inherits
all settings and data from its parent (preceding) series. To facilitate this
for the translations the method copyTranslationsFromParent is called from a
script once after the release series is created.
>>> login("foo.bar@canonical.com")
>>> foobuntu = factory.makeDistribution('foobuntu', 'Foobuntu')
>>> barty = factory.makeDistroSeries(foobuntu, '99.0', name='barty')
>>> carty = factory.makeDistroSeries(
... foobuntu, '99.1', name='carty', previous_series=barty)
Functions to create source packages, templates and and translations.
>>> def makeSourcePackage(name):
... packagename = factory.getOrMakeSourcePackageName(name)
... return factory.makeDistributionSourcePackage(packagename,
... foobuntu)
>>> def makePOTemplateAndPOFiles(distroseries, package, name, languages):
... return factory.makePOTemplateAndPOFiles(languages,
... distroseries=distroseries,
... sourcepackagename=package.sourcepackagename,
... name=name, translation_domain=name+'-domain')
>>> def makeTranslation(template, msgid, translations, sequence=None):
... if sequence is None:
... sequence = factory.getUniqueInteger()
... msgset = factory.makePOTMsgSet(template, msgid, sequence=sequence)
... for language, translation in translations.items():
... pofile = template.getPOFileByLang(language)
... factory.makeCurrentTranslationMessage(
... pofile, msgset, translations=[translation],
... current_other=True)
... return msgset
>>> package1 = makeSourcePackage('package1')
>>> package2 = makeSourcePackage('package2')
>>> template1 = makePOTemplateAndPOFiles(barty, package1,
... 'template1', ['eo'])
>>> template2 = makePOTemplateAndPOFiles(barty, package1,
... 'template2', ['eo', 'de'])
>>> template3 = makePOTemplateAndPOFiles(barty, package1,
... 'template3', ['eo'])
>>> msgset11 = makeTranslation(template1, 'msgid11',
... {'eo': 'eo11'})
>>> msgset21 = makeTranslation(template2, 'msgid21',
... {'eo': 'eo21', 'de': 'de21'})
>>> msgset22 = makeTranslation(template2, 'msgid22',
... {'eo': 'eo22', 'de': 'de22'})
>>> msgset31 = makeTranslation(template3, 'msgid31', {'eo': 'eo31'})
The parent series may have obsolete POTMsgSets which will not be copied.
>>> msgset12 = makeTranslation(template1, 'msgid12', {'eo': 'eo12'}, 0)
Also, template3 happens to be deactivated.
>>> template3.iscurrent = False
We need a transaction manager (in this case a fake one) to make the copy work.
>>> from lp.testing.faketransaction import FakeTransaction
>>> transaction_stub = FakeTransaction()
Preconditions for migrating translations between distro series
==============================================================
Before we are able to migrate translations, there are a set of preconditions
that should be met:
First one is that we should keep the new distroseries's translations hidden
from the public, so the copying procedure is not confused by concurrent
updates:
>>> carty.hide_all_translations = False
>>> carty.copyTranslationsFromParent(transaction_stub)
Traceback (most recent call last):
...
AssertionError: hide_all_translations not set!...
# Set the field to TRUE so we meet this precondition for following
# tests.
>>> carty.hide_all_translations = True
The other one is that, for the same reason, the import queue should not be
accepting uploads for this distroseries.
>>> carty.defer_translation_imports = False
>>> carty.copyTranslationsFromParent(transaction_stub)
Traceback (most recent call last):
...
AssertionError: defer_translation_imports not set!...
# Set the field to TRUE so we meet this precondition for following
# tests.
>>> carty.defer_translation_imports = True
Performing the migration
========================
A pristine distroseries can be filled with copies of translation templates
and translation files from the parent. The actual translations, stored in
POTMsgSet and TranslationMessage object, are shared between the two series.
>>> carty.copyTranslationsFromParent(transaction_stub)
All current templates were copied from the parent series but the deactivated
template template3 was not copied.
>>> from lp.translations.interfaces.potemplate import IPOTemplateSet
>>> carty_templates = getUtility(
... IPOTemplateSet).getSubset(distroseries=carty)
>>> len(carty_templates)
2
>>> print sorted([template.name for template in carty_templates])
[u'template1', u'template2']
>>> carty_template1 = carty_templates.getPOTemplateByName('template1')
>>> carty_template2 = carty_templates.getPOTemplateByName('template2')
>>> carty_template1 == template1
False
>>> carty_template2 == template2
False
All POFiles for the copied POTemplates have also been copied.
>>> all_pofiles = sum(
... [list(template.pofiles) for template in carty_templates], [])
>>> print sorted([pofile.path for pofile in all_pofiles])
[u'template1-domain-eo.po',
u'template2-domain-de.po',
u'template2-domain-eo.po']
All POTMsgSets from the parent series that were not obsolete are now found
in the new series.
>>> potmsgsets = carty_template1.getPOTMsgSets()
>>> print potmsgsets.count()
1
>>> potmsgsets[0] == msgset11
True
>>> potmsgsets = carty_template2.getPOTMsgSets()
>>> print potmsgsets.count()
2
>>> potmsgsets[0] == msgset21
True
>>> potmsgsets[1] == msgset22
True
Once the migration is done, copyTranslationsFromParent must not be called
again as it only operates on distroseries without any translation templates.
Because of message sharing incremental copies are no longer needed.
>>> carty.copyTranslationsFromParent(transaction_stub)
Traceback (most recent call last):
...
AssertionError:
The child series must not yet have any translation templates.
Running the script
==================
Now, we execute the script that will do the migration using
copyTranslationsFromParent. For that we create a new child series to receive
those translations. For testing purposes this series has translation imports
enabled.
>>> darty = factory.makeDistroSeries(
... foobuntu, '99.2', name='darty', previous_series=barty)
>>> darty_id = darty.id
>>> darty.defer_translation_imports = False
The script starts its own transactions, so we need to commit here to be sure
the new series will be available in the script.
>>> import transaction
>>> transaction.commit()
The script fails as long as the defer_translation_imports flag is not
set.
>>> from canonical.launchpad.ftests.script import run_script
>>> returnvalue, output, error_output = run_script(
... 'scripts/copy-translations-from-parent.py',
... ['--distribution=foobuntu', '--series=darty'])
>>> returnvalue
1
>>> print error_output
INFO Creating lockfile:
/var/lock/launchpad-copy-missing-translations-foobuntu-darty.lock
ERROR Before this process starts, set the hide_all_translations and
defer_translation_imports flags for distribution foobuntu, series
darty; or use the --force option to make it happen
automatically.
<BLANKLINE>
>>> transaction.abort()
>>> from lp.registry.model.distroseries import DistroSeries
>>> darty = DistroSeries.get(darty_id)
>>> darty.defer_translation_imports
False
>>> darty.hide_all_translations
True
It succeeds, however, when we pass the --force option. The script then
sets the defer_translation_imports flag itself before copying.
>>> transaction.abort()
>>> darty = DistroSeries.get(darty_id)
>>> returnvalue, output, error_output = run_script(
... 'scripts/copy-translations-from-parent.py',
... ['--distribution=foobuntu', '--series=darty', '--force'])
>>> returnvalue
0
>>> print error_output
INFO Creating lockfile:
/var/lock/launchpad-copy-missing-translations-foobuntu-darty.lock
INFO Starting...
INFO Populating blank distroseries darty with translations from barty.
INFO Extracting from potemplate into
"temp_potemplate_holding_foobuntu_darty"...
INFO Extracting from translationtemplateitem into
"temp_translationtemplateitem_holding_foobuntu_darty"...
INFO Extracting from pofile into
"temp_pofile_holding_foobuntu_darty"...
INFO Extracting from pofiletranslator into
"temp_pofiletranslator_holding_foobuntu_darty"...
INFO Pouring "temp_potemplate_holding_foobuntu_darty"
back into potemplate...
INFO Pouring "temp_translationtemplateitem_holding_foobuntu_darty"
back into translationtemplateitem...
INFO Pouring "temp_pofile_holding_foobuntu_darty"
back into pofile...
INFO Pouring "temp_pofiletranslator_holding_foobuntu_darty"
back into pofiletranslator...
INFO Done.
<BLANKLINE>
After completing, the script restores the defer_translation_imports
flag to its previous value (off).
>>> transaction.abort()
>>> darty = DistroSeries.get(darty_id)
>>> darty.defer_translation_imports
False
>>> darty.hide_all_translations
True
Once the script has finished, the new distro series has all the active
templates of the parent series.
>>> dartempls = getUtility(IPOTemplateSet).getSubset(distroseries=darty)
>>> len(dartempls)
2
>>> print sorted([template.name for template in dartempls])
[u'template1', u'template2']
|