Monday, September 20, 2010

Python : Virtual Python Environment builder

Virtual Python Environment builder
virtualenv is a tool to create isolated Python environments.
The basic problem being addressed is one of dependencies and versions, and indirectly permissions. Imagine you have an application that needs version 1 of LibFoo, but another application requires version 2. How can you use both these applications? If you install everything into /usr/lib/python2.4/site-packages (or whatever your platform's standard location is), it's easy to end up in a situation where you unintentionally upgrade an application that shouldn't be upgraded.
Or more generally, what if you want to install an application and leave it be? If an application works, any change in its libraries or the versions of those libraries can break the application.
Also, what if you can't install packages into the global site-packages directory? For instance, on a shared host.
In all these cases, virtualenv can help you. It creates an environment that has its own installation directories, that doesn't share libraries with other virtualenv environments (and optionally doesn't use the globally installed libraries either).

The basic usage is:
$ python virtualenv.py ENV

The --no-site-packages Option

If you build with virtualenv --no-site-packages ENV it will not inherit any packages from /usr/lib/python2.5/site-packages (or wherever your global site-packages directory is). This can be used if you don't have control over site-packages and don't want to depend on the packages there, or you just want more isolation from the global system.

Using Virtualenv without bin/python

Sometimes you can't or don't want to use the Python interpreter created by the virtualenv. For instance, in a mod_python or mod_wsgi environment, there is only one interpreter. Luckily, it's easy. You must use the custom Python interpreter to install libraries. But to use libraries, you just have to be sure the path is correct. A script is available to correct the path. You can setup the environment like:
activate_this = '/path/to/env/bin/activate_this.py'
execfile(activate_this, dict(__file__=activate_this))
This will change sys.path and even change sys.prefix, but also allow you to use an existing interpreter. Items in your environment will show up first on sys.path, before global items. However, this cannot undo the activation of other environments, or modules that have been imported. You shouldn't try to, for instance, activate an environment before a web request; you should activate one environment as early as possible, and not do it again in that process.
Download Link

Using with ModWSGI (Python WSGI adapter module for Apache.)

A virtual Python environment is useful where it is necessary to run multiple WSGI applications which have conflicting requirements as to what version of a Python module or package needs to be installed. They can also be used where Apache and daemon mode of mod_wsgi is used to host WSGI applications for different users and each user wants to be able to separately install their own Python modules and packages.

Baseline Environment

The first step in using virtual environments with mod_wsgi is to point mod_wsgi at a baseline Python environment. This step is actually optional and if not done the main Python installation for the system, usually that which mod_wsgi was compiled for, would be used as the baseline environment.
Although the main Python installation can be used, especially in a shared environment where daemon mode of mod_wsgi is used to host WSGI applications for different users, it is better to make the baseline environment a virgin environment with an effectively empty 'site-packages' directory. This way there is no possibility of conflicts between modules and packages in a users individual Python virtual environment and the baseline environment.
To create a virgin environment using the 'virtualenv' program, the '--no-site-packages' option should be supplied when creating the environment.
$ cd /usr/local/pythonenv
$ virtualenv --no-site-packages BASELINENew python executable in BASELINE/bin/pythonInstalling setuptools............done.

Once the baseline Python environment has been created, the WSGIPythonHome directive should be defined within the global part of the main Apache configuration files. The directive should refer to the top level directory for the baseline environment created by the 'virtualenv' script.

WSGIPythonHome /usr/local/pythonenv/BASELINE

This Python environment will now be used as the baseline environment for all WSGI applications running under mod_wsgi, whether they be run in embedded mode or daemon mode.
There is no need to set the WSGIPythonHome directive if you want to use the main Python installation as the baseline environment. 


For Applications

As an additional step, the WSGI script file described in the instructions would be modified to overlay the virtual environment for the application on top of the baseline environment. This would be done by adding at the very start of the WSGI script file the following:

import site
site.addsitedir('/usr/local/pythonenv/PYLONS-1/lib/python2.5/site-packages')

Note that in this case the full path to the 'site-packages' directory for the virtual environment needs to be specified and not just the root of the virtual environment.


No comments:

Post a Comment