Forums

Hi everyone,

I wanted to show you all a work-in-progress, and solicit some feedback and suggestions, maybe even encourage some collaboration.

Background:

We're looking for ways to deal with the fact that, as we release new versions of PythonAnywhere, we want to be able to upgrade to the latest versions of packages in our batteries included. But, our users may not yet be ready for them - particularly with web apps, unexpected upgrades can break things. That's why we've had Django pinned to version 1.3 for a few months now.

So the solution, or a solution, is Virtualenv. If we encourage our users to use a virtualenv for their web apps, and maybe for other code too (scheduled tasks?), then they can isolate themselves from our upgrade path, and choose to upgrade their dependencies in their own time.

Virtualenvs aren't entirely simple to set up though, so I thought I'd have a go at creating a little helper script, whose purpose is to create a virtualenv for you out of an existing project. Here's the docopt:

This program attempts to "virtualenv-ify" an existing Python project, by:
- scanning its source tree for imports
- creating a virtualenv in its project root
- pip installing any detected dependencies into the virtualenv
- modifying the file at /var/www/wsgi.py to activate the virtualenv

Usage:
virtualenvify <target_directory> [--update-wsgi]
                                 [--always-copy-local]
                                 [--fake]
virtualenvify -h | --help

Options:
-h --help               show this screen
--fake                  preview only, do not write anything to disk.
                        [default: False]
--update-wsgi           for web apps, update /var/www/wsgi.py with
                        virtualenv activation code [default: False]
--always-copy-local     always copy locally installed version of
                        packages [default: False]

https://github.com/hjwp/virtualenvify

You can install virtualenvify by doing a

pip install --user virtualenvify

If you want to see the source code and the tests, do a checkout/fork of the repo from github.

So, your comments, feedback, suggestions and contributions are kindly solicited. Is this even a good idea? How could I make it better?

Also, I'd love to enlist your help with testing. So, if you have some code on PythonAnywhere, would you be able to try out virtualenvify on it? Make sure to do so on a test copy of your code first:

cp -r my_project /tmp/my_project
# try out some options that are like what you might use 
# for a PythonAnywhere web app but non-destructively!
virtualenvify.py /tmp/my_project --fake --update-wsgi
virtualenvify.py /tmp/my_project --always-copy-local

I should stress that this is very much a work-in-progress. But, they always say, if you aren't hideously embarassed by your first release, then you've left it too late!

Here are some todos:

• fix copy-local code to be less of a blatant hack
• handle version numbers more explicitly
• build some kind of mapping between pypi package names and importable module names
• implement a pip package cache on pythonanywhere
• <your suggestions here!>

Looks promising.

In my case I use a library from my Dropbox folder, which it detects as an external library and would presumably attempt to install it from PyPI (I've only run with --fake so far). I'm not pretending this is a particularly good practice, but I suspect it's likely to arise occasionally.

So, adding to your "todo" list above, I guess something which looks up packages in PyPI and skips anything it doesn't find would be useful, either as default behaviour of with a --skip-unrecognised option.

If I get time I'll take a look at adding something like that myself, but probably won't be for a few days at least. Might be useful to hang on for a few days and see whether I'm the only weirdo doing that or if anybody else is in the same boat, anyway.

Thanks Cartroo. I think that sounds like a perfectly fair use case actually. Good suggestion re looking stuff up on PyPi. Actually my next plan is to build a little web app that does a "reverse-pypi" index - basically giving the ability to look up a pypi package name from the name of a module it provides (since these aren't always the same, which is another problem my little script will run into)