Forums

Note: This question has also been posted on Stack Overflow, if you solve it here, please post their as well, so you can get credit.

http://stackoverflow.com/questions/13442109/python-dictionary-loses-value-when-virtualenv-active

I have a function that fetches data from a web page, and returns a simple list of dictionaries:

def get_users():
    # ...
    users = []
    for user in raw_users:
            name = user.get_text()
            url = BASE_URL + user['href']
            id_ = int(match(".*id=(\d*)", url).group(1))
            description = bs(str(user.next_sibling)).get_text().strip()
            new_user = dict(id=id_, name=name, url=url, description=description)

           users.append(new_user)

       return users

When using a virtualenv as the Python interpreter in Aptana/PyDev, the code returns the expected data:

[{'description': u'Lorem ipsum dolor sit amet, consectetur adipisicing elit ...',
  'id': 95,
  'name': u'Jane Doe',
  'url': 'http://example.com/detail.php?id=95'},
 {'description': u' Ut enim ad minim veniam, quis nostrud exercitation  ...',
  'id': 96,
  'name': u'John Doe',
  'url': 'http://example.com/detail.php?id=96'}]

In order to activate the exact same virtualenv under WSGI, I added the following code to the beginning of the module:

from os import path

user_home = path.expanduser("~")
activate_this = path.join(user_home, 'virtualenvs/myapp/bin/activate_this.py')
execfile(activate_this, dict(__file__=activate_this))

Now the exact same function returns the same dictionaries with empty strings for the description value, leaving everything else intact:

[{'description': u'',
  'id': 95,
  'name': u'Jane Doe',
  'url': 'http://example.com/detail.php?id=95'},
 {'description': u'',
  'id': 96,
  'name': u'John Doe',
  'url': 'http://example.com/detail.php?id=96'}]

I'm not sure why only the description values are affected. No change occurred after renaming the description key.

Can anyone explain what is causing this problem, and how to fix it?

To PythonAnywhere staff: I Think I've narowed the problem down to the way the virtualenv is activated. One Stack Ovrflow user mentioned that virtualenvs should not be activated via scipt, but rather by WSGI config. Why isn't a virtualenv configurable via the webapp dashboard?

Hi Sean -- may I take a look at the original code from inside your account? You just need to let me know yes or no, and I'll be able to go from there.

Re: the way the virtual env is set up -- it could be the cause, I guess, but it doesn't sound likely to me at this stage. Providing a setup option on in the webapp dashboard is a good idea, and is definitely on the list (I've upvoted it on your behalf) but ultimately it should only be a convenience -- things will be neater, and you won't need extra cruft in your wsgi.py, but I don't think it'll change behaviour.

Sure! I'm planning on open-sourcing the project once it's is a respectable state anyway. I'll send you an email to discribe the troubleshooting I've done thus far. Thanks for going above any beyond!

It turns out that Beautiful Soup was behaving diffrently in Aptana/PyDev than it actually was (which makes no sence, I know). This made me think my code was in working condition when it was not. I have since fixed my code. Sorry to have wasted your time.

No problem, thanks for confirming that!