1
.. IVLE - Informatics Virtual Learning Environment
2
Copyright (C) 2007-2009 The University of Melbourne
4
.. This program is free software; you can redistribute it and/or modify
5
it under the terms of the GNU General Public License as published by
6
the Free Software Foundation; either version 2 of the License, or
7
(at your option) any later version.
9
.. This program is distributed in the hope that it will be useful,
10
but WITHOUT ANY WARRANTY; without even the implied warranty of
11
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
12
GNU General Public License for more details.
14
.. You should have received a copy of the GNU General Public License
15
along with this program; if not, write to the Free Software
16
Foundation, Inc., 51 Franklin St, Fifth Floor, Boston, MA 02110-1301 USA
22
IVLE is a complex piece of software that integrates closely with the
23
underlying system. It can be considered part web service and part local system
24
daemon. Due to the implementation of these parts it is tied to Apache Web
25
Server (mainly due to the use of mod_python) and Linux.
31
IVLE uses mod_python_ to allow Python scripts to be called from Apache. We
32
register the :mod:`ivle.dispatch` module as the ``PythonHandler`` in the
33
associated VirtualHost, allowing us to intercept all HTTP requests to the web
36
The :mod:`ivle.dispatch` module is responsible for mapping requests from the
37
client to the correct application plugin. Plugins can be specified by placing
38
a :file:`*.conf` file into the :file:`/etc/ivle/plugins.d/` directory
39
containing lines of the form :samp:`[{plugin_module}#{classname}]`.
41
.. TODO: Document Plugin Format and Routing Strings
43
In future, this may be ported to a WSGI (:pep:`333`) based dispatch to allow
44
IVLE to be run on web servers other than Apache.
46
.. _mod_python: http://www.modpython.org/
51
IVLE uses the Genshi_ XHTML template system to generate all HTML pages. We
52
have an inheritance-based "views" system. :class:`BaseView` is a class from
53
which all views derive.
55
There are 3 sub-types of :class:`BaseView` (more can be implemented if
59
* browser, console, debuginfo, diff, forum, groups, help, home, logout,
60
settings, subjects, svnlog, tos, tutorial
64
* consoleservice, fileservice, tutorialservice, userservice
66
The apps each derive from one of the above.
69
IVLE used to write its HTML output as a raw stream to an output file, until
70
it was refactored to use Genshi. All apps which haven't yet been refactored
71
properly were ported to use the "raw byte streaming" view.
73
.. _Genshi: http://genshi.edgewall.org/
81
One of the main features of IVLE is it's ability to execute user's code in a
82
customised environment that prevents access to other users files or underlying
83
file system as well as placing basic resource limits to prevent users from
84
accidentally exhausting shared resources such as CPU time and memory.
90
To each user, it appears that they have their own private Unix filesystem
91
containing software, libraries and a home directory to do with what they
92
please. This is mainly done by the setuid root program ``trampoline`` which
93
mounts the users home directory, sets up the users environment, jumps into the
94
user's jail using the :manpage:`chroot(2)` system call and finally drops
95
privileges to the desired user and group.
97
To prevent abuse, ``trampoline`` can only be used by root or one of the uids
98
specified when trampoline is built by ``setup.py build`` (defaults to UID 33,
99
www-data on Debian systems). Since it's one of two C programs involved in IVLE
100
and runs setuid root it is rather security sensitive.
102
.. seealso:: Source code :file:`bin/trampoline/trampoline.c`
105
Base Image Generation
106
---------------------
108
All user jails share a common base image that contains the files required for
109
both IVLE's operation and for executing user code. This base image is
110
generated automatically by the ``ivle-buildjail`` script. This then calls the
111
distribution dependant details in :mod:`ivle.jailbuilder` module. At present
112
we only support building jails for Debian derived systems using
113
:program:`debootstrap`.
115
The contents of the base image contains a few core packages required for the
116
operation of IVLE - Python and the Python CJSON and SVN libraries. Other
117
options that can be configured in :file:`/etc/ivle/ivle.conf` are the file
118
mirror that debootstrap should use, the suite to build (such as hardy or
119
jaunty), extra apt-sources, extra apt keys and any additional packages to
122
To prevent users from altering files in the base image we change the
123
permissions of :file:`/tmp`, :file:`/var/tmp` and :file:`/var/lock` to not be
124
world writeable and check that no other files are world writeable.
126
Finally we make the user dependent :file:`/etc/passwd` and
127
:file:`/etc/ivle/ivle.conf` symlinks to files in the :file:`/home` directory
128
so that they will be used when we mount a user's home directory.
130
Mounting Home Directories
131
-------------------------
133
To give the appearance of a private file system we need to merge together a
134
user's local home directory with the base image.
135
To achieve this, IVLE uses the *bind mount* feature of Linux, which allows
136
directories to be accessible from another location in the file system. By
137
carefully bind-mounting the jail image as read-only and then bind-mounting the
138
user's :file:`/home` and :file:`/tmp` directory data over the top, we create a
139
jail with only three bind mounts and at virtually no file system overhead.
142
IVLE has historically used numerous solutions to this problem, which are
143
chronicled here to avoid the same mistakes being made again.
145
In the first release of IVLE this was done offline by hard-linking all the
146
files into the target directory, but for a large number of users, this
147
process can take several hours, and also runs the risk of exhausting
148
the number of inodes on the underlying file system.
150
The second solution was to use `AUFS <http://aufs.sourceforge.net/>`_ to
151
mount the user's home directory over a read-only version of the base on
152
demand. This was implemented as part of ``trampoline`` and used a secondary
153
program ``timount`` (see :file:`bin/timount/timount.c`), run at regular
154
intervals, to unmount unused jails. This used the :const:`MNT_EXPIRE` flag
155
for :manpage:`umount(2)` (available since Linux 2.6.8) that only unmounts a
156
directory if it hasn't been accessed since the previous call with
159
While quite effective, AUFS appeared to cause NFS caching issues when IVLE
160
was run as a cluster, and as its inclusion status in future Linux
161
distributions is questionable, the developers elected to use the much older
162
bind mount feature instead.
167
Before running the specified program in the users jail we need to
168
:manpage:`chroot(2)` into the users jail and update the processes environment
169
so that we have the correct environment variables and user/group ids.
171
At this stage we also may apply a number of resource limits (see
172
:manpage:`setrlimit`) to prevent run away processes (such as those containing
173
infinite loops or "fork bombs") from exhausting all system resources. The
174
default limits are on maximum address space (:const:`RLIMIT_AS`), process data
175
space (:const:`RLIMIT_DATA`), core dump size (:const:`RLIMIT_CORE`), CPU time
176
(:const:`RLIMIT_CPU`), file size (:const:`RLIMIT_FSIZE`) and number of
177
processes that may be spawned (:const:`RLIMIT_NPROC`).
179
Unfortunately due to glibc's :manpage:`malloc(2)` implementation being able to
180
allocate memory using :manpage:`mmap(2)`, :const:`RLIMIT_DATA` does not
181
provide an effective limit on the amount of memory that a process can allocate
182
(short of applying a kernel patch). Thus the only way to limit memory
183
allocations is by placing limits on the address space, but this can cause
184
problems with certain applications that allocate far larger address spaces
185
than the real memory used. For this reason :const:`RLIMIT_AS` is currently set
189
.. _ref-python-console:
194
IVLE provides a web based programming console, exposing similar features to
195
Python's command line console. It is built around the
196
:file:`services/python-console` script, which opens up a socket on a random
197
port to which `JSON`_ encoded chat requests can be made.
199
A new console is typically launched on demand by the web client to the HTTP
200
API, which in turn calls the wrapper class :class:`ivle.console.Console` to
201
start a new console in the user's jail.
203
Subsequent requests from the same in-browser console connect to the existing
204
console process. This is achieved by storing a string on the client which
205
identifies the server address and port. The client then makes requests
206
through the load balancer, sending this string through to an arbitrary slave
207
which forwards the request to the identified console.
209
This means that all slaves need access to all ports on every other slave.
211
.. _JSON: http://json.org
214
.. _ref-usrmgt-server:
216
User Management Server
217
======================
219
The **User Management Server** is a daemon responsible for handling privileged
220
actions on IVLE and should be launched along with IVLE. It is primarily
223
* Creating user jails, Subversion repositories, and Subversion authentication
225
* Creating group Subversion repositories.
226
* Rebuilding Subversion authorization files.
228
Communication with the Server is done using the :ref:`Chat Protocol
229
<ref-chat>`. To prevent unauthorized use, communication with the User
230
Management Server requires that a *shared secret* be used to communicate with
231
the server. This secret is stored in the `magic` variable in the `[usrmgt]`
232
section of :file:`/etc/ivle/ivle.conf`.
234
The User Management Server is called almost exclusively from the
235
:mod:`ivle.webapp.userservice` module.
237
.. seealso:: Source code :file:`services/usrmgt-server`
244
**Chat** is our JSON_-based client/server communication protocol used in
245
communicating to :ref:`Python Console <ref-python-console>` processes and
246
:ref:`User Management Server <ref-usrmgt-server>`. Since it is JSON-based it
247
can be called from either Python or JavaScript.
251
The protocol is a fairly simple client/server based one consisting of a single
252
JSON object. Before communication starts a shared secret :const:`MAGIC` must
253
be known by both parties. The shared secret is then used to form a
254
'keyed-Hash Message Authentication Code' to ensure that the content is valid
255
and not been modified in transit.
257
The client request takes the following form::
264
where :const:`DATA` is any valid JSON value and :const:`HASH` is an string
265
containing the MD5 hash of the :const:`DATA` appended to :const:`MAGIC` and
268
The server will respond with a JSON value corresponding to the request.
269
If an error occurs then a special JSON object will be returned of the
275
"traceback": TRACEBACK
278
where :const:`NAME` is a JSON string of the exception type (such as
279
'AttributeError'), :const:`VALUE` is the string value associated with the
280
exception and :const:`TRACEBACK` is a string of the traceback generated by the
281
server's exception handler.
283
.. seealso:: Source code :file:`ivle/chat.py`
289
Along with traditional file system access, IVLE allows users to version their
290
files using Subversion_. Much like how Subversion workspaces are used on a
291
standard desktop, workspaces are checked out into users home directories where
292
they can be manipulated through a series of AJAX requests to the
295
Like all other user file system actions, version control actions need to be
296
executed inside the user's :ref:`jail <ref-jail>`. Requests are made to the
297
``fileservice`` app in :mod:`ivle.webapp.fileservice` which then calls the
298
``fileservice`` CGI script using ``trampoline``. This script is simply a
299
wrapper around :mod:`ivle.fileservice_lib` which actually contains the code to
300
handle each of the actions.
302
Manipulation of the Subversion workspaces is done using the pysvn_ library.
304
.. _Subversion: http://subversion.tigris.org/
305
.. _pysvn: http://pysvn.tigris.org/
311
Each user is allocated a Subversion repository when their :ref:`Jail
312
<ref-jail>` is created by the :ref:`User Management Server
313
<ref-usrmgt-server>`. Repository are stored in the location specified by
314
``paths/svn/repo_path`` in :file:`/etc/ivle/ivle.conf` (by default
315
:file:`/var/lib/ivle/svn/repositories/`). User repositories are stored in the
316
:samp:`users/{USERNAME}/` subdirectory and group repositories in
317
:samp:`groups/{SUBJECT}_{YEAR}_{SEMESTER}_{GROUP}`.
321
While it would be possible to give users direct access to their repository
322
using Subversion's file backend, this would allow users to potentially
323
modify the history of any repository that they had access to. To ensure
324
repository integrity, all Subversion interaction must be done remotely.
330
These repositories are served by Apache using ``mod_dav_svn`` allowing access
331
over Subversion's WebDAV HTTP or HTTPS backends. Users are authenticated using
332
a randomly generated key which is stored in the database and is made available
333
to each user inside their jail (``svn_pass`` property inside
334
:file:`/home/.ivle.conf`). This key is automatically provided when doing
335
Subversion actions, but can be manually entered when accessing a user's
336
repository from an external Subversion client such as with :samp:`svn checkout
337
{svn_addr}/users/{USERNAME}/ workspace`.
339
Repository permissions for ``AuthzSVNAccessFile`` are automatically generated
340
and placed in the file specified by the ``paths/svn/conf`` config option
341
(usually ``/var/lib/ivle/svn/svn.conf``) for user repositories and the
342
``paths/svn/group_conf`` option for group repositories (usually
343
``/var/lib/ivle/svn/svn-group.conf``). User authentication keys for
344
``AuthUserFile`` are stored in the file specified by ``paths/svn/auth_ivle``,
345
usually ``/var/lib/ivle/svn/ivle.auth``. These will be regenerated each time
346
user or group repository settings change.
352
Worksheets provide a way for users to be able to attempt a set of coding
353
exercises along with accompanying instructions. In the past worksheets were
354
created directly using an XML format, but this has been deprecated in favour
355
of being generated automatically from reStructuredText.
357
Worksheets are now stored in the database as a :class:`Worksheet` object (see
358
:file:`ivle/database.py`). This allows them to be treated with the same
359
access permissions available to other objects and lays down the ground work
360
for providing versioned worksheets.
366
When users submit an exercise, the user's solution is tested against a series
367
of test cases which can be used to check if a solution is acceptable. Almost
368
all the behavior for exercises is contained within
369
:file:`ivle/webapp/tutorial/test/TestFramework.py`.
372
The TestFramework module is one of the oldest and most complicated in
373
IVLE, largely taken directly from the IVLE prototype. As such it has a
374
design that doesn't quite match the current architecture of IVLE, such as
375
using slightly different terminology and having a few testing facilities
376
that are untested or untested. It requires a substantial rewrite and
377
comprehensive test suite to be developed.
379
At the top level exists the :class:`Exercise` object (known as ``TestSuite``
380
in :file:`TestFramework.py`). This object encompasses the entire collection of
381
tests for a given exercise and details such as the exercise name, provided
382
solution and any "include code" (Python code available for all test cases, but
383
not the user's submission).
385
Each exercise may contain one or more :class:`TestSuite` objects (known as
386
``TestCase`` in :file:`TestFramework.py`. A test suite is a collection of
387
tests that run with some sort of common input - be that stdin contents, a
388
virtual file system configuration (presently disabled), inputs to particular
389
function or defining the contents of one or more variables. A test suite will
390
typically run until the first test case fails, but can be configured to
391
continue running test cases even after one has failed. Exceptions raised by
392
submitted code will typically cause the test to fail except if it is marked as
393
an "allowed exception".
395
Individual units to be tested (something that can pass or fail) are contained
396
within :class:`TestCase` objects (known as ``TestCaseParts`` in
397
:file:`TestFramework.py`). A test case can test the value of source code text,
398
the function return value (Will be ``None`` for scripts), stdout contents,
399
stderr contents, name of any raised exception and contents of the virtual file
400
system (presently disabled) of code submitted by users. These checks are
401
contained in a :class:`TestCasePart`. In addition, a normalisation function or
402
custom comparison function can be used instead of comparing the raw values
403
directly. By default, the value of each check will be ignored unless
404
overidden by a test case part.
413
URLs are resolved with a small IVLE-specific object publishing framework --
414
that is, resolution is implemented as traversal through an object graph. The
415
framework lives in :mod:`ivle.webapp.publisher`, and has an extensive test
418
This object graph is constructed by the dispatcher. Any plugin class deriving
419
from ViewPlugin will be searched for ``forward_routes``, ``reverse_routes``
420
and ``views`` sequences. Everything is class-based -- an object's routes
421
and views are determined by its class.
423
Forward routes handle resolution of URLs to objects. Given a source object
424
and some path segments, the route must calculate the next object.
425
A forward route is a tuple of ``(source class, intermediate path segments,
426
function, number of subsequent path segments to consume)``, or simply a
427
reference to a decorated function (see :mod:`ivle.webapp.admin.publishing`
428
for decoration examples). The function must return the next object in the
431
A reverse route handles URL generation for an object. Given just an object,
432
it must return a tuple of ``(previous object, intermediate path segments)``.
433
This creates a chain of objects and path segments until the root is reached.
434
Due to IVLE's lack of a utility framework, reverse routes at the root of the
435
URL space need to refer to the root object with the magical
436
:mod:`ivle.webapp.publisher.ROOT`.
438
Views are registered with a tuple of ``(source class, intermediate path segments,
441
In all of the above, "intermediate path segments" can either be a single
442
segment string, or a sequence of multiple strings representing multiple
446
While many applications prefer a pattern matching mechanism, this did not
447
work out well for IVLE. Our deep URL structure and multitude of nested
448
objects with lots of views meant that match patterns had to be repeated
449
tediously, and views required many lines of code to turn a match into a
450
context object. It also made URL generation very difficult.
452
The simple object publishing framework allows views to be registered with
453
just one line of code, getting their context object for free. URL
454
generation now comes at a cost of approximately one line of code per class,
455
and breadcrumbs are easy too. The reduced code duplication also improves
added
removed
doc/dev/architecture.rst
