87
90
To each user, it appears that they have their own private Unix filesystem
88
91
containing software, libraries and a home directory to do with what they
89
please. This is mainly done by the setuid root program ``trampoline`` (See
90
:file:`bin/trampoline/trampoline.c`) which mounts the users home directory,
91
sets up the users environment, jumps into the user's jail using the
92
:manpage:`chroot(2)` system call and finally drops privileges to the desired
92
please. This is mainly done by the setuid root program ``trampoline`` which
93
mounts the users home directory, sets up the users environment, jumps into the
94
user's jail using the :manpage:`chroot(2)` system call and finally drops
95
privileges to the desired user and group.
95
97
To prevent abuse, ``trampoline`` can only be used by root or one of the uids
96
98
specified when trampoline is built by ``setup.py build`` (defaults to UID 33,
97
99
www-data on Debian systems). Since it's one of two C programs involved in IVLE
98
and runs setuid root it is rather secuity sensative.
100
and runs setuid root it is rather security sensitive.
102
.. seealso:: Source code :file:`bin/trampoline/trampoline.c`
100
105
Base Image Generation
101
106
---------------------
180
185
than the real memory used. For this reason :const:`RLIMIT_AS` is currently set
189
.. _ref-python-console:
186
194
IVLE provides a web based programming console, exposing similar features to
187
Python's command line console. It is built around python script
188
:file:`services/python-console` which opens up a socket to which `JSON`_
189
encoded chat requests can be made. A new console is typically from launched on
190
demand by the web client to the HTTP API, which in turn calls the wrapper
191
class :class:`ivle.console.Console` to start a new console in the user's jail.
195
Python's command line console. It is built around the
196
:file:`services/python-console` script, which opens up a socket on a random
197
port to which `JSON`_ encoded chat requests can be made.
199
A new console is typically launched on demand by the web client to the HTTP
200
API, which in turn calls the wrapper class :class:`ivle.console.Console` to
201
start a new console in the user's jail.
203
Subsequent requests from the same in-browser console connect to the existing
204
console process. This is achieved by storing a string on the client which
205
identifies the server address and port. The client then makes requests
206
through the load balancer, sending this string through to an arbitrary slave
207
which forwards the request to the identified console.
209
This means that all slaves need access to all ports on every other slave.
193
211
.. _JSON: http://json.org
214
.. _ref-usrmgt-server:
195
216
User Management Server
196
217
======================
204
225
* Creating group Subversion repositories.
205
226
* Rebuilding Subversion authorization files.
207
Communication with the Server is done using the `Chat Protocol`_. To prevent
208
unauthorized use, communication with the User Management Server requires that
209
a *shared secret* be used to communicate with the server. This secret is
210
stored in the `magic` variable in the `[usrmgt]` section of
211
:file:`/etc/ivle/ivle.conf`.
228
Communication with the Server is done using the :ref:`Chat Protocol
229
<ref-chat>`. To prevent unauthorized use, communication with the User
230
Management Server requires that a *shared secret* be used to communicate with
231
the server. This secret is stored in the `magic` variable in the `[usrmgt]`
232
section of :file:`/etc/ivle/ivle.conf`.
213
234
The User Management Server is called almost exclusively from the
214
235
:mod:`ivle.webapp.userservice` module.
216
See :file:`services/usrmgt-server` for details.
237
.. seealso:: Source code :file:`services/usrmgt-server`
221
**Chat** is our JSON_-based client/server communication protocol used in
222
communicating to `Python Console`_ processes and `User Management Server`_.
223
Since it is JSON-based it can be called from either Python or JavaScript.
244
**Chat** is our JSON_-based client/server communication protocol used in
245
communicating to :ref:`Python Console <ref-python-console>` processes and
246
:ref:`User Management Server <ref-usrmgt-server>`. Since it is JSON-based it
247
can be called from either Python or JavaScript.
256
280
exception and :const:`TRACEBACK` is a string of the traceback generated by the
257
281
server's exception handler.
259
See :file:`ivle/chat.py` for details.
283
.. seealso:: Source code :file:`ivle/chat.py`
289
Along with traditional file system access, IVLE allows users to version their
290
files using Subversion_. Much like how Subversion workspaces are used on a
291
standard desktop, workspaces are checked out into users home directories where
292
they can be manipulated through a series of AJAX requests to the
295
Like all other user file system actions, version control actions need to be
296
executed inside the user's :ref:`jail <ref-jail>`. Requests are made to the
297
``fileservice`` app in :mod:`ivle.webapp.fileservice` which then calls the
298
``fileservice`` CGI script using ``trampoline``. This script is simply a
299
wrapper around :mod:`ivle.fileservice_lib` which actually contains the code to
300
handle each of the actions.
302
Manipulation of the Subversion workspaces is done using the pysvn_ library.
304
.. _Subversion: http://subversion.tigris.org/
305
.. _pysvn: http://pysvn.tigris.org/
311
Each user is allocated a Subversion repository when their :ref:`Jail
312
<ref-jail>` is created by the :ref:`User Management Server
313
<ref-usrmgt-server>`. Repository are stored in the location specified by
314
``paths/svn/repo_path`` in :file:`/etc/ivle/ivle.conf` (by default
315
:file:`/var/lib/ivle/svn/repositories/`). User repositories are stored in the
316
:samp:`users/{USERNAME}/` subdirectory and group repositories in
317
:samp:`groups/{SUBJECT}_{YEAR}_{SEMESTER}_{GROUP}`.
321
While it would be possible to give users direct access to their repository
322
using Subversion's file backend, this would allow users to potentially
323
modify the history of any repository that they had access to. To ensure
324
repository integrity, all Subversion interaction must be done remotely.
330
These repositories are served by Apache using ``mod_dav_svn`` allowing access
331
over Subversion's WebDAV HTTP or HTTPS backends. Users are authenticated using
332
a randomly generated key which is stored in the database and is made available
333
to each user inside their jail (``svn_pass`` property inside
334
:file:`/home/.ivle.conf`). This key is automatically provided when doing
335
Subversion actions, but can be manually entered when accessing a user's
336
repository from an external Subversion client such as with :samp:`svn checkout
337
{svn_addr}/users/{USERNAME}/ workspace`.
339
Repository permissions for ``AuthzSVNAccessFile`` are automatically generated
340
and placed in the file specified by the ``paths/svn/conf`` config option
341
(usually ``/var/lib/ivle/svn/svn.conf``) for user repositories and the
342
``paths/svn/group_conf`` option for group repositories (usually
343
``/var/lib/ivle/svn/svn-group.conf``). User authentication keys for
344
``AuthUserFile`` are stored in the file specified by ``paths/svn/auth_ivle``,
345
usually ``/var/lib/ivle/svn/ivle.auth``. These will be regenerated each time
346
user or group repository settings change.
271
.. TODO: Not yet merged
360
URLs are resolved with a small IVLE-specific object publishing framework --
361
that is, resolution is implemented as traversal through an object graph. The
362
framework lives in :mod:`ivle.webapp.publisher`, and has an extensive test
365
This object graph is constructed by the dispatcher. Any plugin class deriving
366
from ViewPlugin will be searched for ``forward_routes``, ``reverse_routes``
367
and ``views`` sequences. Everything is class-based -- an object's routes
368
and views are determined by its class.
370
Forward routes handle resolution of URLs to objects. Given a source object
371
and some path segments, the route must calculate the next object.
372
A forward route is a tuple of ``(source class, intermediate path segments,
373
function, number of subsequent path segments to consume)``, or simply a
374
reference to a decorated function (see :mod:`ivle.webapp.admin.publishing`
375
for decoration examples). The function must return the next object in the
378
A reverse route handles URL generation for an object. Given just an object,
379
it must return a tuple of ``(previous object, intermediate path segments)``.
380
This creates a chain of objects and path segments until the root is reached.
381
Due to IVLE's lack of a utility framework, reverse routes at the root of the
382
URL space need to refer to the root object with the magical
383
:mod:`ivle.webapp.publisher.ROOT`.
385
Views are registered with a tuple of ``(source class, intermediate path segments,
388
In all of the above, "intermediate path segments" can either be a single
389
segment string, or a sequence of multiple strings representing multiple
393
While many applications prefer a pattern matching mechanism, this did not
394
work out well for IVLE. Our deep URL structure and multitude of nested
395
objects with lots of views meant that match patterns had to be repeated
396
tediously, and views required many lines of code to turn a match into a
397
context object. It also made URL generation very difficult.
399
The simple object publishing framework allows views to be registered with
400
just one line of code, getting their context object for free. URL
401
generation now comes at a cost of approximately one line of code per class,
402
and breadcrumbs are easy too. The reduced code duplication also improves
added
removed
doc/dev/architecture.rst
