1
.. IVLE - Informatics Virtual Learning Environment
2
Copyright (C) 2007-2009 The University of Melbourne
4
.. This program is free software; you can redistribute it and/or modify
5
it under the terms of the GNU General Public License as published by
6
the Free Software Foundation; either version 2 of the License, or
7
(at your option) any later version.
9
.. This program is distributed in the hope that it will be useful,
10
but WITHOUT ANY WARRANTY; without even the implied warranty of
11
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
12
GNU General Public License for more details.
14
.. You should have received a copy of the GNU General Public License
15
along with this program; if not, write to the Free Software
16
Foundation, Inc., 51 Franklin St, Fifth Floor, Boston, MA 02110-1301 USA
22
IVLE is a complex piece of software that integrates closely with the
23
underlying system. It can be considered part web service and part local system
24
daemon. Due to the implementation of these parts it is tied to Apache Web
25
Server (mainly due to the use of mod_python) and Linux.
31
IVLE uses mod_python_ to allow Python scripts to be called from Apache. We
32
register the :mod:`ivle.dispatch` module as the ``PythonHandler`` in the
33
associated VirtualHost, allowing us to intercept all HTTP requests to the web
36
The :mod:`ivle.dispatch` module is responsible for mapping requests from the
37
client to the correct application plugin. Plugins can be specified by placing
38
a :file:`*.conf` file into the :file:`/etc/ivle/plugins.d/` directory
39
containing lines of the form :samp:`[{plugin_module}#{classname}]`.
41
.. TODO: Document Plugin Format and Routing Strings
43
In future, this may be ported to a WSGI (:pep:`333`) based dispatch to allow
44
IVLE to be run on web servers other than Apache.
46
.. _mod_python: http://www.modpython.org/
51
IVLE uses the Genshi_ XHTML template system to generate all HTML pages. We
52
have an inheritance-based "views" system. :class:`BaseView` is a class from
53
which all views derive.
55
There are 3 sub-types of :class:`BaseView` (more can be implemented if
59
* browser, console, debuginfo, diff, forum, groups, help, home, logout,
60
settings, subjects, svnlog, tos, tutorial
64
* consoleservice, fileservice, tutorialservice, userservice
66
The apps each derive from one of the above.
69
IVLE used to write its HTML output as a raw stream to an output file, until
70
it was refactored to use Genshi. All apps which haven't yet been refactored
71
properly were ported to use the "raw byte streaming" view.
73
.. _Genshi: http://genshi.edgewall.org/
78
One of the main features of IVLE is it's ability to execute user's code in a
79
customised environment that prevents access to other users files or underlying
80
file system as well as placing basic resource limits to prevent users from
81
accidentally exhausting shared resources such as CPU time and memory.
87
To each user, it appears that they have their own private Unix filesystem
88
containing software, libraries and a home directory to do with what they
89
please. This is mainly done by the setuid root program ``trampoline`` (See
90
:file:`bin/trampoline/trampoline.c`) which mounts the users home directory,
91
sets up the users environment, jumps into the user's jail using the
92
:manpage:`chroot(2)` system call and finally drops privileges to the desired
95
To prevent abuse, ``trampoline`` can only be used by root or one of the uids
96
specified when trampoline is built by ``setup.py build`` (defaults to UID 33,
97
www-data on Debian systems). Since it's one of two C programs involved in IVLE
98
and runs setuid root it is rather secuity sensative.
100
Base Image Generation
101
---------------------
103
All user jails share a common base image that contains the files required for
104
both IVLE's operation and for executing user code. This base image is
105
generated automatically by the ``ivle-buildjail`` script. This then calls the
106
distribution dependant details in :mod:`ivle.jailbuilder` module. At present
107
we only support building jails for Debian derived systems using
108
:program:`debootstrap`.
110
The contents of the base image contains a few core packages required for the
111
operation of IVLE - Python and the Python CJSON and SVN libraries. Other
112
options that can be configured in :file:`/etc/ivle/ivle.conf` are the file
113
mirror that debootstrap should use, the suite to build (such as hardy or
114
jaunty), extra apt-sources, extra apt keys and any additional packages to
117
To prevent users from altering files in the base image we change the
118
permissions of :file:`/tmp`, :file:`/var/tmp` and :file:`/var/lock` to not be
119
world writeable and check that no other files are world writeable.
121
Finally we make the user dependent :file:`/etc/passwd` and
122
:file:`/etc/ivle/ivle.conf` symlinks to files in the :file:`/home` directory
123
so that they will be used when we mount a user's home directory.
125
Mounting Home Directories
126
-------------------------
128
To give the appearance of a private file system we need to merge together a
129
user's local home directory with the base image.
130
To achieve this, IVLE uses the *bind mount* feature of Linux, which allows
131
directories to be accessible from another location in the file system. By
132
carefully bind-mounting the jail image as read-only and then bind-mounting the
133
user's :file:`/home` and :file:`/tmp` directory data over the top, we create a
134
jail with only three bind mounts and at virtually no filesystem overhead.
137
IVLE has historically used numerous solutions to this problem, which are
138
chronicled here to avoid the same mistakes being made again.
140
In the first release of IVLE this was done offline by hard-linking all the
141
files into the target directory, but for a large number of users, this
142
process can take several hours, and also runs the risk of exhausting
143
the number of inodes on the underlying file system.
145
The second solution was to use `AUFS <http://aufs.sourceforge.net/>`_ to
146
mount the user's home directory over a read-only version of the base on
147
demand. This was implemented as part of ``trampoline`` and used a secondary
148
program ``timount`` (see :file:`bin/timount/timount.c`), run at regular
149
intervals, to unmount unused jails. This used the :const:`MNT_EXPIRE` flag
150
for :manpage:`umount(2)` (available since Linux 2.6.8) that only unmounts a
151
directory if it hasn't been accessed since the previous call with
154
While quite effective, AUFS appeared to cause NFS caching issues when IVLE
155
was run as a cluster, and as its inclusion status in future Linux
156
distributions is questionable, the developers elected to use the much older
157
bind mount feature instead.
162
Before running the specified program in the users jail we need to
163
:manpage:`chroot(2)` into the users jail and update the processes environment
164
so that we have the correct environment variables and user/group ids.
166
At this stage we also may apply a number of resource limits (see
167
:manpage:`setrlimit`) to prevent run away processes (such as those containing
168
infinite loops or "fork bombs") from exhausting all system resources. The
169
default limits are on maximum address space (:const:`RLIMIT_AS`), process data
170
space (:const:`RLIMIT_DATA`), core dump size (:const:`RLIMIT_CORE`), CPU time
171
(:const:`RLIMIT_CPU`), file size (:const:`RLIMIT_FSIZE`) and number of
172
processes that may be spawned (:const:`RLIMIT_NPROC`).
174
Unfortunately due to glibc's :manpage:`malloc(2)` implementation being able to
175
allocate memory using :manpage:`mmap(2)`, :const:`RLIMIT_DATA` does not
176
provide an effective limit on the amount of memory that a process can allocate
177
(short of applying a kernel patch). Thus the only way to limit memory
178
allocations is by placing limits on the address space, but this can cause
179
problems with certain applications that allocate far larger address spaces
180
than the real memory used. For this reason :const:`RLIMIT_AS` is currently set
186
IVLE provides a web based programming console, exposing similar features to
187
Python's command line console. It is built around python script
188
:file:`services/python-console` which opens up a socket to which `JSON`_
189
encoded chat requests can be made. A new console is typically from launched on
190
demand by the web client to the HTTP API, which in turn calls the wrapper
191
class :class:`ivle.console.Console` to start a new console in the user's jail.
193
.. _JSON: http://json.org
195
User Management Server
196
======================
198
The **User Management Server** is a daemon responsible for handling privileged
199
actions on IVLE and should be launched along with IVLE. It is primarily
202
* Creating user jails, Subversion repositories, and Subversion authentication
204
* Creating group Subversion repositories.
205
* Rebuilding Subversion authorization files.
207
Communication with the Server is done using the `Chat Protocol`_. To prevent
208
unauthorized use, communication with the User Management Server requires that
209
a *shared secret* be used to communicate with the server. This secret is
210
stored in the `magic` variable in the `[usrmgt]` section of
211
:file:`/etc/ivle/ivle.conf`.
213
The User Management Server is called almost exclusively from the
214
:mod:`ivle.webapp.userservice` module.
216
See :file:`services/usrmgt-server` for details.
221
**Chat** is our JSON_-based client/server communication protocol used in
222
communicating to `Python Console`_ processes and `User Management Server`_.
223
Since it is JSON-based it can be called from either Python or JavaScript.
227
The protocol is a fairly simple client/server based one consisting of a single
228
JSON object. Before communication starts a shared secret :const:`MAGIC` must
229
be known by both parties. The shared secret is then used to form a
230
'keyed-Hash Message Authentication Code' to ensure that the content is valid
231
and not been modified in transit.
233
The client request takes the following form::
240
where :const:`DATA` is any valid JSON value and :const:`HASH` is an string
241
containing the MD5 hash of the :const:`DATA` appended to :const:`MAGIC` and
244
The server will respond with a JSON value corresponding to the request.
245
If an error occurs then a special JSON object will be returned of the
251
"traceback": TRACEBACK
254
where :const:`NAME` is a JSON string of the exception type (such as
255
'AttributeError'), :const:`VALUE` is the string value associated with the
256
exception and :const:`TRACEBACK` is a string of the traceback generated by the
257
server's exception handler.
259
See :file:`ivle/chat.py` for details.
271
.. TODO: Not yet merged
added
removed
doc/dev/architecture.rst
