10th January, 2014

It’ll Come Out in the Wash

Filed under: python — andy47 @ 3:36 pm

After reading this fine summary of the history of Python 3 by Nick Coghlan I was inspired to update as many of my half finished projects and miscellaneous scripts as possible. Then I looked up and I had lost several hours of my life. To save random internet strangers from the same pain as I experienced here is a catalogue of problems and how I solved them.

tl;dr – Python 3.3 on Ubuntu 12.04 LTS is possible but quite tricky. Unless you like system administration you are probably better off waiting until 14.04 LTS comes out in a few months.

I’ve settled on a default ‘stack’ of Python and related technologies that I use for most of the code that I write these days;

The good news is that all of the Python related components of these technologies have been ported to Python 3. The bad news is that Flask only runs on Python 3.3. Elementary OS is based on Ubuntu 12.04 LTS and that *only* has Python 3.2 in it’s repositories.

So, first things first, how do we get Python 3.3 on Elementary OS? Or Ubuntu 12.04 which are interchangeable for the purposes of the rest of this post. Quite easily thanks to the ‘deadsnakes PPA’ as explained in this askubuntu post.

I also don’t like to pollute my system Python or operating system packages with project specific Python modules so I always use virtualenv. Unfortunately this doesn’t work with the Python 3.3 acquired from the deadsnakes PPA as *that* doesn’t ship with easy_install.

So by this stage I have Python 3.3 running but no way to create or manage a virtual environment. I then remembered Nick’s talk at PyCon AU where he mentioned that a new lightweight virtual environment would be included in the standard library. Sure enough the venv module (and pyvenv script) were present on my system so I decided it was a good idea to use them.

The bad part for me is that (and it says this in the documentation) venv does not include any means of installing other Python packages. How then do you get a package installer/manager when you can’t install packages? The solution was in a c.l.p. post.

That link describes a process of creating a new virtual environment and then bootstrapping distribute. In case my link-fu fails me the simple shell based approach is;

$ pyvenv py3k ~/envs/py3k
$ cd py3k
$ source bin/activate
$ wget http://python-distribute.org/distribute_setup.py
$ python distribute_setup.py
$ ./local/bin/easy_install pip

One final little wrinkle though. The c.l.p. post and other references I found indicate that your package scripts (such as ‘pip’ and ‘python’) will be installed in $ENV_HOME/bin. In my environment they are installed in $ENV_HOME/local/bin. It’s probably something I have inadvertently done. But if you follow the instructions and can’t find something in bin have a look in local/bin and you should find it.

5 Comments

  1. ‘easy_install’ isn’t included with standard Python versions, because it’s part of setuptools. Please note that distribute (which was a fork of setuptools) has been merged back into setutools. So you should be really installing setuptools now again instead of distribute:

    wget https://bitbucket.org/pypa/setuptools/raw/bootstrap/ez_setup.py -O – | sudo python3.3

    Then go ahead and install pip and virtualenv (because IMHO it has a lot more to offer than pyenv):

    sudo easy_install pip
    sudo pip install virtualenv

    While you’re at it, install virtualenvwrapper too (see it’s PyPI page for additional setup instructions):

    sudo pip install virtualenvwrapper

    From then on you shouldn’t need to install any Python packages system-wide anymore. For each project you just create a new virtual environment with ‘mkvirtualenv’ (provided by virtualenvwrapper):

    mkvirtualenv myenv

    (myenv will be activated directly) and then install all the packages you need, e.g. with:

    pip install -e requirements.txt

    Should you want to share packes installed in the system-wide site-packages in several virtual environments, have a look ad my ‘addsitepackage2venv.py’ script:

    http://trac.chrisarndt.de/code/browser/projects/scripts/addsitepackage2venv.py

    Cheers, Chris

    Comment by Chris Arndt — 10/01/2014 @ 7:18 pm

  2. This worked for me, not sure if it’s easier though:

    # pip for python3.3 – once we upgrade past 12.04, this can all be replaced by adding python3-pip to packages.list.
    # also remove ppa:fkrull/deadsnakes from ppas.list and python3.3 from packages.list
    # make python3 symlink
    sudo rm /usr/bin/python3
    sudo ln -s /usr/bin/python3.3 /usr/bin/python3
    # install setuptools
    wget https://bitbucket.org/pypa/setuptools/raw/bootstrap/ez_setup.py
    sudo python3 ez_setup.py
    # install pip-3.3
    wget https://raw.github.com/pypa/pip/master/contrib/get-pip.py
    sudo python3 get-pip.py
    rm ez_setup.py get-pip.py *.gz
    # don’t know why but this is necessary
    sudo rm /usr/bin/pip
    sudo apt-get install –reinstall python-pip

    Comment by Henry Harrison — 10/01/2014 @ 11:24 pm

  3. I think the more canonical way of installing easy_install for Python3.3 is to bootstrap setuptools. So if you already have Python3.3 installed you would run:`wget https://bitbucket.org/pypa/setuptools/raw/bootstrap/ez_setup.py -O – | sudo python3.3`. This gives you easy_install-3.3 installed system-wide. From there you can run `sudo easy_install-3.3 pip` and `sudo pip-3.3 install virtualenv` and you’re off to the races.

    Comment by Ben Rousch — 10/01/2014 @ 11:24 pm

  4. And I forgot to mention, that may not even be necessary if you only want virtualenvs. You can use the –python-/usr/bin/python3.3 option even when running a python 2 version of virtualenv.

    Comment by Henry Harrison — 10/01/2014 @ 11:26 pm

  5. Hi. It is very easy to just build Python 3.3 from source on Ubuntu 12.04 LTS.

    Comment by Brian Neal — 12/01/2014 @ 6:35 am

RSS feed for comments on this post.

Sorry, the comment form is closed at this time.

Powered by WordPress