|
4284.4.1
by Tom Haddon
Added nightly.sh to cronscripts directory |
1 |
#!/bin/sh
|
|
8687.15.7
by Karl Fogel
Add the copyright header block to more files. |
2 |
#
|
|
7675.758.8
by Jeroen Vermeulen
Okay, okay, running it through garbo instead of cron. |
3 |
# Copyright 2009 Canonical Ltd. This software is licensed under the
|
|
8687.15.7
by Karl Fogel
Add the copyright header block to more files. |
4 |
# GNU Affero General Public License version 3 (see the file LICENSE).
|
|
4284.4.1
by Tom Haddon
Added nightly.sh to cronscripts directory |
5 |
|
|
10303.1.18
by Gary Poster
fix some remaining bis from the buildout change |
6 |
# This script performs nightly chores. It should be run from
|
|
4284.4.1
by Tom Haddon
Added nightly.sh to cronscripts directory |
7 |
# cron as the launchpad user once a day. Typically the output
|
8 |
# will be sent to an email address for inspection.
|
|
9 |
||
|
10303.1.18
by Gary Poster
fix some remaining bis from the buildout change |
10 |
# Note that http/ftp proxies are needed by the product
|
|
4284.4.2
by Tom Haddon
Adding comment about why the http_proxy and ftp_proxy variables are needed |
11 |
# release finder
|
12 |
||
|
7516.1.1
by Tom Haddon
Moving nightly.sh script to run on loganberry |
13 |
# Only run this script on loganberry
|
|
12314.1.6
by William Grant
Use $() instead of ``. |
14 |
THISHOST=$(uname -n) |
|
7516.1.1
by Tom Haddon
Moving nightly.sh script to run on loganberry |
15 |
if [ "loganberry" != "$THISHOST" ] |
|
4430.1.1
by Tom Haddon
Making nightly.sh executable, changing release to cdimage on prober |
16 |
then
|
|
7516.1.1
by Tom Haddon
Moving nightly.sh script to run on loganberry |
17 |
echo "This script must be run on loganberry." |
|
4430.1.1
by Tom Haddon
Making nightly.sh executable, changing release to cdimage on prober |
18 |
exit 1 |
19 |
fi
|
|
20 |
||
21 |
# Only run this as the launchpad user
|
|
|
12314.1.6
by William Grant
Use $() instead of ``. |
22 |
USER=$(whoami) |
|
4430.1.1
by Tom Haddon
Making nightly.sh executable, changing release to cdimage on prober |
23 |
if [ "launchpad" != "$USER" ] |
24 |
then
|
|
25 |
echo "Must be launchpad user to run this script." |
|
26 |
exit 1 |
|
27 |
fi
|
|
28 |
||
29 |
||
|
4759.3.1
by Tom Haddon
Mirror prober separated out from nightly.sh, some nightly.sh changes and rosetta config for forster |
30 |
export LPCONFIG=production |
|
4284.4.1
by Tom Haddon
Added nightly.sh to cronscripts directory |
31 |
export http_proxy=http://squid.internal:3128/ |
32 |
export ftp_proxy=http://squid.internal:3128/ |
|
33 |
||
|
12314.1.5
by William Grant
Log locking failures too. |
34 |
LOGDIR=/srv/launchpad.net/production-logs/nightly |
35 |
LOGFILE=$LOGDIR/nightly.log |
|
36 |
||
|
4284.4.1
by Tom Haddon
Added nightly.sh to cronscripts directory |
37 |
LOCK=/var/lock/launchpad_nightly.lock |
38 |
lockfile -r0 -l 259200 $LOCK |
|
39 |
if [ $? -ne 0 ]; then |
|
|
12314.1.6
by William Grant
Use $() instead of ``. |
40 |
echo $(date): Unable to grab $LOCK lock - aborting | tee -a $LOGFILE |
|
4284.4.1
by Tom Haddon
Added nightly.sh to cronscripts directory |
41 |
ps fuxwww |
42 |
exit 1 |
|
43 |
fi
|
|
44 |
||
45 |
cd /srv/launchpad.net/production/launchpad/cronscripts
|
|
46 |
||
|
12314.1.6
by William Grant
Use $() instead of ``. |
47 |
echo $(date): Grabbed lock >> $LOGFILE |
|
12314.1.4
by William Grant
Log when we start and finish. |
48 |
|
|
12314.1.6
by William Grant
Use $() instead of ``. |
49 |
echo $(date): Expiring memberships >> $LOGFILE |
|
12314.1.2
by William Grant
Make nightly.sh's subordinate scripts log to their own files. |
50 |
python -S flag-expired-memberships.py -q --log-file=DEBUG:$LOGDIR/flag-expired-memberships.log |
|
4284.4.1
by Tom Haddon
Added nightly.sh to cronscripts directory |
51 |
|
|
12314.1.6
by William Grant
Use $() instead of ``. |
52 |
echo $(date): Allocating revision karma >> $LOGFILE |
|
12314.1.2
by William Grant
Make nightly.sh's subordinate scripts log to their own files. |
53 |
python -S allocate-revision-karma.py -q --log-file=DEBUG:$LOGDIR/allocate-revision-karma.log |
|
6623.4.16
by Tim Penhey
Add a new cronscript to be run each night, and remove the creation of the historical karma revisions when a user validates an email address. This is now handled by the cronscript. |
54 |
|
|
12314.1.6
by William Grant
Use $() instead of ``. |
55 |
echo $(date): Recalculating karma >> $LOGFILE |
|
12314.1.2
by William Grant
Make nightly.sh's subordinate scripts log to their own files. |
56 |
python -S foaf-update-karma-cache.py -q --log-file=INFO:$LOGDIR/foaf-update-karma-cache.log |
|
4284.4.1
by Tom Haddon
Added nightly.sh to cronscripts directory |
57 |
|
|
12314.1.6
by William Grant
Use $() instead of ``. |
58 |
echo $(date): Updating cached statistics >> $LOGFILE |
|
12314.1.2
by William Grant
Make nightly.sh's subordinate scripts log to their own files. |
59 |
python -S update-stats.py -q --log-file=DEBUG:$LOGDIR/update-stats.log |
|
4284.4.1
by Tom Haddon
Added nightly.sh to cronscripts directory |
60 |
|
|
12314.1.6
by William Grant
Use $() instead of ``. |
61 |
echo $(date): Expiring questions >> $LOGFILE |
|
12314.1.2
by William Grant
Make nightly.sh's subordinate scripts log to their own files. |
62 |
python -S expire-questions.py -q --log-file=DEBUG:$LOGDIR/expire-questions.log |
|
4284.4.1
by Tom Haddon
Added nightly.sh to cronscripts directory |
63 |
|
|
12314.1.6
by William Grant
Use $() instead of ``. |
64 |
### echo == Expiring bugs $(date) ==
|
|
10637.3.7
by Guilherme Salgado
merge devel |
65 |
### python -S expire-bugtasks.py
|
|
4896.2.1
by Curtis Hovey
added expire-bugtasks.py to nightly.sh |
66 |
|
|
12314.1.1
by William Grant
Remove update-pkgcache.py from nightly.sh. It's taking 15 hours. |
67 |
# checkwatches.py is scheduled in lp-production-crontabs.
|
|
12314.1.6
by William Grant
Use $() instead of ``. |
68 |
### echo == Updating bug watches $(date) ==
|
|
10637.3.7
by Guilherme Salgado
merge devel |
69 |
### python -S checkwatches.py
|
|
4284.4.1
by Tom Haddon
Added nightly.sh to cronscripts directory |
70 |
|
|
12314.1.6
by William Grant
Use $() instead of ``. |
71 |
echo $(date): Updating bugtask target name caches >> $LOGFILE |
|
12314.1.2
by William Grant
Make nightly.sh's subordinate scripts log to their own files. |
72 |
python -S update-bugtask-targetnamecaches.py -q --log-file=DEBUG:$LOGDIR/update-bugtask-targetnamecaches.log |
|
5729.5.1
by Graham Binns
Moved the run of update-bugtask-targetnamecaches to the end of nightly.sh so that it doesn't block other scripts. |
73 |
|
|
12314.1.6
by William Grant
Use $() instead of ``. |
74 |
echo $(date): Updating personal standings >> $LOGFILE |
|
12314.1.2
by William Grant
Make nightly.sh's subordinate scripts log to their own files. |
75 |
python -S update-standing.py -q --log-file=DEBUG:$LOGDIR/update-standing.log |
|
6376.1.1
by Barry Warsaw
run update-standing.py once nightly |
76 |
|
|
12314.1.6
by William Grant
Use $() instead of ``. |
77 |
echo $(date): Updating CVE database >> $LOGFILE |
|
12314.1.2
by William Grant
Make nightly.sh's subordinate scripts log to their own files. |
78 |
python -S update-cve.py -q --log-file=DEBUG:$LOGDIR/update-cve.log |
|
6722.1.1
by Bjorn Tillenius
Rearrange nightly.sh, so that scripts that talk over the network, and |
79 |
|
|
12314.1.1
by William Grant
Remove update-pkgcache.py from nightly.sh. It's taking 15 hours. |
80 |
# update-pkgcache.py is scheduled in lp-production-crontabs.
|
|
12314.1.6
by William Grant
Use $() instead of ``. |
81 |
#echo == Updating package cache $(date) ==
|
|
12314.1.1
by William Grant
Remove update-pkgcache.py from nightly.sh. It's taking 15 hours. |
82 |
#python -S update-pkgcache.py -q
|
|
6722.1.1
by Bjorn Tillenius
Rearrange nightly.sh, so that scripts that talk over the network, and |
83 |
|
|
12314.1.1
by William Grant
Remove update-pkgcache.py from nightly.sh. It's taking 15 hours. |
84 |
# Release finder is scheduled in lp-production-crontabs.
|
|
12314.1.6
by William Grant
Use $() instead of ``. |
85 |
#echo == Product Release Finder $(date) ==
|
|
12225.3.1
by Curtis Hovey
removed product-release-finder.py from nightly.sh |
86 |
#python -S product-release-finder.py -q
|
|
6722.1.1
by Bjorn Tillenius
Rearrange nightly.sh, so that scripts that talk over the network, and |
87 |
|
|
12314.1.6
by William Grant
Use $() instead of ``. |
88 |
echo $(date): Removing lock >> $LOGFILE |
|
4284.4.1
by Tom Haddon
Added nightly.sh to cronscripts directory |
89 |
rm -f $LOCK
|