~azzar1/unity/add-show-desktop-key

.. IVLE - Informatics Virtual Learning Environment

   Copyright (C) 2007-2009 The University of Melbourne

.. This program is free software; you can redistribute it and/or modify

   it under the terms of the GNU General Public License as published by

   the Free Software Foundation; either version 2 of the License, or

   (at your option) any later version.

.. This program is distributed in the hope that it will be useful,

   but WITHOUT ANY WARRANTY; without even the implied warranty of

   MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the

   GNU General Public License for more details.

.. You should have received a copy of the GNU General Public License

   along with this program; if not, write to the Free Software

   Foundation, Inc., 51 Franklin St, Fifth Floor, Boston, MA  02110-1301  USA

.. _ref-install:

************

Installation

************

System requirements

===================

Given versions are those on which IVLE is known to work; earlier versions

might work too. Debian/Ubuntu package names are given after the name of the

software.

.. If this list changes, you also need to change the list below, and

   the list in bin/ivle-dev-setup.

* Ubuntu 8.04 LTS or later (other distros should work with some tweaking, but are untested)

* Apache 2.x (``apache2``) with modules:

   + mod_python (``libapache2-mod-python``)

   + mod_dav_svn and mod_authz_svn (``libapache2-svn``)

* Python 2.5 (``python2.5``) or 2.6 (``python2.6``) with modules:

   + ConfigObj (``python-configobj``)

   + docutils (``python-docutils``)

   + epydoc (``python-epydoc``)

   + FormEncode (``python-formencode``)

   + Genshi (``python-genshi``)

   + psycopg2 (``python-psycopg2``)

   + pysvn (``python-svn``)

   + Storm (``python-storm``)

   + simplejson (``python-simplejson``, for Python 2.5 only)

* jQuery (``libjs-jquery``)

* CodeMirror (``libjs-codemirror``)

* PostgreSQL 8.3 or later (``postgresql``)

* Subversion (``subversion``)

* debootstrap (``debootstrap``)

* rsync (``rsync``)

* GCC and related build machinery (``build-essential``)

Master versus slave servers

===========================

IVLE is normally deployed in a cluster of several machines, split into

two different roles: master and slave.

There must be exactly one master server per cluster. The master normally

runs the PostgreSQL database server, the Subversion server, and the IVLE User

Management Server (``ivle-usrmgt-server``). It might also export shared data

directories to the slaves over NFS.

There may be any number of slaves in a cluster. They run the IVLE web

application, which also starts console host processes. Each slave makes use

of the shared services on the master.

For a small instance a slave may be run on the same machine as the master.

This is the setup described in the source installation section, while the

Ubuntu package installation section describes a multi-node configuration.

Installing from source

======================

When setting up a development IVLE environment on Ubuntu 10.04 LTS or later,

there are scripts to automate most of the process. First get and extract

`a release tarball <https://launchpad.net/ivle/+download>`_, or check out

the latest code from the Bazaar branch: ::

   bzr get lp:ivle

This will create a new directory, ``ivle``, containing a pristine

source tree. The remaining steps assume that you are in this new

directory.

One of IVLE's dependencies (``libjs-codemirror``) does not yet have an

official Ubuntu package. To make this dependency package available, add

the production PPA, and update your local package cache: ::

   sudo add-apt-repository ppa:unimelb-ivle/production

   sudo apt-get update

Alternatively, manually obtain and install ``libjs-codemirror``.

Automated setup

---------------

The ``ivle-dev-setup`` script will configure PostgreSQL, Apache, IVLE

and the filesystem to cooperate, getting you most of the way to a

working system in just one step: ::

   bin/ivle-dev-setup

.. warning::

   This reconfigures parts of your system, and has the potential to

   break other applications using Apache or PostgreSQL. It may also

   fail to execute if you have existing incompatible configurations

   of those services.

This may take a few minutes, and will ask you to confirm installation

of the dependency packages.

Upon completion, you must build a self-contained jail in which to run

untrusted user code. ``ivle-dev-setup`` will have configured most of

the necessary settings, but you may wish to use a local Ubuntu mirror

to improve speed or minimise download costs. If you don't wish to use

a special mirror, you may omit the first step. ::

   sudo ivle-config --jail/mirror http://url.to.mirror/ubuntu

   sudo ivle-buildjail -r

.. warning::

   ``ivle-buildjail`` will download a large volume of package data --

   potentially some hundreds of megabytes.

``ivle-buildjail`` will download, unpack and install a minimal Ubuntu

system and configure it for IVLE usage. This could take a while.

Once the jail has been successfully built, IVLE is up and running,

but with no user accounts or other data in place. For development

or demonstration purposes, sample data (including fictitious users,

subjects, and projects) can be loaded.

For other environments, it may be more appropriate to start with an

empty database and just create users as required.

To load the sample data: ::

   sudo ivle-loadsampledata examples/db/sample.sql

.. warning::

   If you answer 'yes' to the ``ivle-loadsampledata`` prompt, any

   existing data in your IVLE database will be **permanently

   destroyed**.

... or to add a new admin user: ::

   sudo ivle-adduser --admin -p password username 'Full Name'

You should then be able to browse to http://ivle.localhost/, and

log in with username ``admin`` and password ``password``, or the

username and password that you gave to ``ivle-adduser``.

Manual steps

------------

If the automatic installation scripts do not work, or if you want more

control over the whole process, these manual steps are probably for

you. But you need not read this section at all if you were able to log

in after following the steps above.

.. If this list changes, you also need to change the list above, and

   the command in bin/ivle-dev-setup.

If you want to grab all of the required packages in one command, use::

    sudo apt-get install apache2 libapache2-mod-python libapache2-svn \

    python2.6 python-configobj python-docutils python-epydoc \

    python-formencode python-genshi python-psycopg2 python-svn python-storm \

    libjs-jquery libjs-codemirror postgresql subversion debootstrap rsync \

    build-essential

As IVLE needs to compile some binaries, you must first build, then

install it. From the source directory created earlier: ::

   ./setup.py build

   sudo ./setup.py install

.. _database-setup:

Setting up the database

~~~~~~~~~~~~~~~~~~~~~~~

First, it is recommended that you create a separate database user for IVLE.

You may use any name for the user. ::

   sudo -u postgres createuser ivleuser     # Answer 'n' to all questions

   sudo -u postgres psql -c "ALTER USER ivleuser WITH ENCRYPTED PASSWORD 'ivle-password';"

Now, you must create a PostgreSQL database, and populate it with the

IVLE schema. You may use any name for the database (here we use ``ivle``). ::

   sudo -u postgres createdb -O ivleuser ivle

   sudo -u postgres createlang plpgsql ivle

   psql -h localhost -W ivle ivleuser < userdb/users.sql

The configuration wizard - ``ivle-config`` - will ask you a series of

questions. You should give the database username and password as configured

above. Apart from database settings, the defaults should be correct

for a development system. If deploying IVLE properly - particularly on

multiple nodes - several options will need to be changed. Watching

carefully, run: ::

   sudo ivle-config

Creating the data tree

~~~~~~~~~~~~~~~~~~~~~~

IVLE needs a directory hierarchy in which to store filesystem data, which

by default lives in ``/var/lib/ivle``. Create it now. ::

   sudo ivle-createdatadirs

Configuring the jail environment

~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

You will require a self-contained jail environment in which to safely

execute student code. 

Before you can actually build the jail, a few configuration options are

needed. Open up ``/etc/ivle/ivle.conf``, and find the ``[jail]`` section

(**not** the ``[[jails]]`` section).

Add to it: ::

   devmode = True

   suite = jaunty # Replace this with the codename of your Ubuntu release.

   mirror = http://url.to.archive/mirror # Replace with a fast Ubuntu mirror.

.. TODO: Move this around a bit, as the config options required differ for

   the packaged version.

Now we can actually build the jail. The creation process basically downloads

a minimal Ubuntu system and installs it in ``/var/lib/ivle/jails/__base__``.

Note that this could download a couple of hundred megabytes. ::

   sudo ivle-buildjail -r

Configuring Apache

~~~~~~~~~~~~~~~~~~

IVLE makes use of two Apache virtual hosts: one for the application itself,

and one for the Subversion services. There are example configuration files

in ``examples/config/apache.conf`` and ``examples/config/apache-svn.conf``,

which will run IVLE at http://ivle.localhost/.

On a Debian or Ubuntu system, just copy those two files into

``/etc/apache2/sites-available`` under appropriate names (eg. ``ivle`` and

``ivle-svn``). Then you need to activate them: ::

   sudo a2ensite ivle

   sudo a2ensite ivle-svn

   sudo /etc/init.d/apache2 reload

Configuring hostname resolution

~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

All of IVLE's hostnames need to be resolvable from the local system. For a

production environment, this would be done in DNS. For a development system,

this is usually done in ``/etc/hosts``. Add this line to that file: ::

   127.0.1.1 ivle.localhost public.ivle.localhost svn.ivle.localhost

Code running inside the jail environment also needs to be able to resolve

those names. Add, to ``/var/lib/ivle/jails/__base_build__/etc/hosts``: ::

   127.0.1.1 svn.ivle.localhost

Then refresh the active copy of the jail: ::

   sudo ivle-buildjail

Configuring the user management server

~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

You need to have the IVLE user management server (``usrmgt-server``) running

for many parts of IVLE to operate properly, so it should be configured to

start on boot. There is an example init script in

``examples/config/usrmgt-server.init``. For Debian or Ubuntu, copy it to

``/etc/init.d/ivle-usrmgt-server``. Start it now, and set it to start

automatically: ::

   sudo /etc/init.d/ivle-usrmgt-server start

   sudo update-rc.d ivle-usrmgt-server defaults 99

Creating the initial user

~~~~~~~~~~~~~~~~~~~~~~~~~

The final step in getting a usable IVLE set up is creating a user. You'll

probably want admin privileges - if not, drop the ``--admin``. ::

   sudo ivle-adduser --admin -p PASSWORD USERNAME 'FULL NAME'

You should then be able to browse to http://ivle.localhost/, and

log in with that username and password.

*Alternatively*, you may wish to import the IVLE sample data, for a complete

working IVLE environment (not for production use). See :ref:`sample-data`.

.. note::

   For more advanced configuration, see :ref:`Configuring IVLE

   <ref-configuring-ivle>`.

Installing from an Ubuntu package

=================================

IVLE is packaged in `a Launchpad PPA <https://launchpad.net/~unimelb-ivle/+archive/production>`_,

providing a more managed installation and upgrade mechanism than a source

installation.

These instructions document the process to install a production-ready

multi-node IVLE cluster. They expect that you have three domain names:

one for the main IVLE web UI, one for the Subversion service, and one

for serving user files publicly.

.. warning::

   By design the public domain may have arbitrary user-generated content

   served. Because of this, it should not have any domain with sensitive

   cookies as a suffix, including the main IVLE web UI. Be very careful

   with your choice here.

Shared setup

------------

All master and slave nodes need to have access to the IVLE PPA.

`Visit it <https://launchpad.net/~unimelb-ivle/+archive/production>`_ and

follow the installation instructions on all involved systems.

Master setup

------------

Setting up the database server

~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

The master server runs the central IVLE PostgreSQL database. ::

   sudo apt-get install postgresql

Ubuntu's default PostgreSQL configuration doesn't permit remote access,

so we need to tweak it to allow password access from our slave. In

``/etc/postgresql/8.3/main/postgresql.conf``, find the ``listen_addresses``

option, and ensure it is set to ``*``. In

``/etc/postgresql/8.3/main/pg_hba.conf`` add a line like the following to the

end. This example will allow any host in the ``1.2.3.4/24`` subnet to

authenticate with a password as the ``ivle`` user to the ``ivle`` database. ::

   host    ivle        ivle        1.2.3.4/24      md5

Then restart PostgreSQL, and the slaves should be able to see the database. ::

   sudo /etc/init.d/postgresql-8.3 restart

Installing and configuring IVLE

~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

We can now install IVLE. The installation process will ask you a few questons.

Answer that this host is a **master**, let it generate a random usrmgt-server

secret, elect to manage the database with ``dbconfig-common``, and use a

random password. ::

   sudo apt-get install ivle

Once that's done, we have a couple of additional configuration items to set:

the URLs discussed earlier. Open up ``/etc/ivle/ivle.conf``, 

and replace ``public.ivle.localhost`` and ``svn.ivle.localhost`` with the

correct domain names.

Make sure you restart the ``usrmgt-server`` afterwards, or newly created users

may inherit the old domain names. ::

   sudo /etc/init.d/usrmgt-server restart

Sharing data between the servers

~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

As well as its relational database, IVLE has a data hierarchy on the

fileystem. Two part of this (``/var/lib/ivle/jails`` and

``/var/lib/ivle/sessions``) must be shared between the master and all of the

slaves. It doesn't matter how you achieve this, but a reasonable method is

described here: exporting over NFS from the master.

We'll first create a tree (``/export/ivle`` in this example, but it can be

whatever you want) to be exported to the slaves, move the existing data

directories into it, and symlink them back into place. ::

   sudo mkdir /export/ivle

   sudo mv /var/lib/ivle/{sessions,jails} /export/ivle

   sudo ln -s /export/ivle/{sessions,jails} /var/lib/ivle

Next install an NFS server. ::

   sudo apt-get install nfs-kernel-server

Now we can export the directory we created earlier across the network.

Add something like the following line to ``/etc/exports``. ``someslave``

should be replaced with the hostname, IP address, or subnet of your

slave(s). ::

   /export/ivle		someslave(rw,sync)

Make sure you inform the kernel of the new export. ::

   sudo exportfs -a

Configuring Apache

~~~~~~~~~~~~~~~~~~

The master serves Subversion repositories through Apache on the Subversion

domain name that was discussed above. ::

   sudo cp /usr/share/doc/ivle/apache-svn.conf /etc/apache2/sites-available/ivle-svn

   sudo a2ensite ivle-svn

Edit ``/etc/apache2/sites-available/ivle-svn``, ensuring that the

``ServerName`` matches your chosen domain name. Then reload Apache's

configuration. ::

   sudo /etc/init.d/apache2 reload

Setting up a jail environment

~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

IVLE requires that a base jail be provided, on top of which all of the

individual user jails are constructed in order to safely execute user code.

We need to change some configuration options before we can build a working

jail. First set the mirror and Ubuntu release -- make sure you replace the

URL and release code name with an Ubuntu mirror and your Ubuntu release. ::

   sudo ivle-config --jail/mirror http://url.to.mirror/ubuntu --jail/suite hardy

Now comes the ugly bit: we need to tell the jail builder where to get the

IVLE code that must be present in the jail. If you're using the production

PPA, the following ``/etc/ivle/ivle.conf`` snippet will work. If you're not,

you'll have to replace the ``extra_keys`` and ``extra_sources`` values ::

   [jail]

   extra_keys = '''

   -----BEGIN PGP PUBLIC KEY BLOCK-----

   Version: SKS 1.0.10

   mI0ES2pQKAEEANiscebT7+SFnvpN8nABcwT5nEV6psUOF8CcIIrz3iv6b6wA3lYd0DzbD7RD

   fs1MNriEHHgqPF6EUhGrkk1165Oqi+lULdjgL0Fzi3GFvLV9F8+BtL3wt3+MM7YC+aTS1nhF

   dQcPpnhNAJagW5gR4dIc4w87sNquxgCdJeJn/N3XABEBAAG0KkxhdW5jaHBhZCBVbml2ZXJz

   aXR5IG9mIE1lbGJvdXJuZSBJVkxFIFBQQYi2BBMBAgAgBQJLalAoAhsDBgsJCAcDAgQVAggD

   BBYCAwECHgECF4AACgkQVwp7ATtnautCMgP8C6PbLNyx9akgbiwhakFfGaEbxGFCo1EAUE7v

   FgdelJSEkeQLAn4WoANpixuojNi++PEDis22S4tz+ZC6G0dRU9Pcc1bb4xUgphR83QTcufH7

   5EagfTf5lLIWaLdg5f/NeuHHrKvwKvPVkNJ3ShQejFB/xWGpqe2Rr7Rscm9lT0Q=

   =TJYw

   -----END PGP PUBLIC KEY BLOCK-----

   '''

   extra_packages = ivle-services,

   extra_sources = deb http://ppa.launchpad.net/unimelb-ivle/production/ubuntu hardy main,

Now we can build the jail. This will download lots of packages, and install

a minimal Ubuntu system in ``/var/lib/ivle/jails/__base__``. ::

   sudo ivle-buildjail -r

You should now have a functional master.

Creating the initial user

~~~~~~~~~~~~~~~~~~~~~~~~~

The last master step for getting a usable IVLE set up is creating a user.

You'll probably want admin privileges - if not, drop the ``--admin``. ::

   sudo ivle-adduser --admin -p PASSWORD USERNAME 'FULL NAME'

You can then visit your IVLE web UI domain and login in with the username

and password.

Slave setup

-----------

Installing and configuring IVLE

~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

We need to tell the database configuration assistant that we want to connect

to a remote database. The second command will also ask you whether you want to

store administrative passwords: say no here. ::

   sudo apt-get install dbconfig-common

   sudo dpkg-reconfigure dbconfig-common

We are going to need some details from the master for authentication purposes.

Grab the ``password`` value from the ``database`` section, and the ``magic``

value from the ``usrmgt`` section of the master's ``/etc/ivle/ivle.conf``.

Now we can install IVLE. Advise the installer that this machine is not a

master, and use the details retrieved from the master to answer the rest of

the questions. ::

   sudo apt-get install ivle

Once the installation has completed, make the same configuration changes as on

the master: set the domain names in ``ivle.conf`` to real values.

For maximum performance, you should also set the ``version`` value in the

``media`` section. The exact string is not important, as long as the value is

identical on every slave, and changed on each upgrade. It is used to make

static file URLs unique, so clients can cache them indefinitely. The IVLE

version is conventionally used as this string.

Getting access to the shared data

~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

We need to mount the shared components of the IVLE data hierarchy from the

master. If you've used the suggested method, follow these instructions.

Otherwise you'll have to work it out for yourself.

First install the NFS common utilities, required for NFS mounts. ::

   sudo apt-get install nfs-common

Now we can add the mount to ``/etc/fstab``. Something like this should do: ::

  themaster:/export/ivle /export/ivle nfs defaults 0 0

Then mount the filesystem, and link the shared directories into the

hierarchy. ::

   mount -a

   ivle-createdatadirs

   rmdir /var/lib/ivle/{sessions,jails}

   ln -s /export/ivle/{sessions,jails} /var/lib/ivle

Configuring Apache

~~~~~~~~~~~~~~~~~~

The slaves use Apache to serve the main IVLE web UI and public user files.

Let's activate the configuration now. ::

   sudo cp /usr/share/doc/ivle/apache.conf /etc/apache2/sites-available/ivle

   sudo a2ensite ivle

Now edit ``/etc/apache2/sites-available/ivle``, and ensure that the

``ServerName`` matches your chosen IVLE web UI domain name, and

``ServerAlias`` your public name. Then reload Apache's configuration. ::

   sudo /etc/init.d/apache2 reload