Forums

I am trying to run a background process within web2py. It works fine on my local machine; and it works fine if I run the program directly from the console. However when I try to launch using subprocess it says no such file. Is that a deliberate restriction or have I missed something?

Example code:

task="%s %s"%(sys.executable, os.path.join(os.getcwd(), "test.py"))
p=subprocess.Popen(task)

That's probably not working because the current working directory returned by os.getcwd() is going to be different when it's running in the web server. You'd probably have to use the full path of test.py for Popen to find it.

Having said that, it's probably not a good idea to do this. We have some fairly loose limits on the number of processes/threads that each user can have at any time. If the process is properly cleaned up in some cases, it won't take very long before your web app stops responding.

The application only runs a single background process at startup so it will not slow anything down. I was using os.getcwd() so that it would work on both the server and on localhost. I also log the full path before it is called and it looks correct:

  /usr/bin/python /home/simoneva/trunk/applications/scraper/taskworker.py

If I run this from console then it works. But when it is run using subprocess I get error stack trace:

p=subprocess.Popen(task) File "/usr/lib/python2.6/subprocess.py", line 623, in init errread, errwrite) File "/usr/lib/python2.6/subprocess.py", line 1141, in _execute_child raise child_exception OSError: [Errno 2] No such file or directory

Doh! I should have spotted this immediately, because I've tripped over it recently, too. From the console, you can use the code ou have because it's in a shell. In the webapp, there is not shell, so you need to separate the pieces of the task into a list. Like this

task=[sys.executable, os.path.join(os.getcwd(), "test.py")]
p=subprocess.Popen(task)

That should work.

I still don't think it's a great idea to do it, but you know your app better than I do.

Brilliant. Thanks very much. What confused me is that windows accepts a string as args for subprocess.popen. However clearly linux requires a list.

Also a second trap is that if you use shlex to parse the string into a list then shlex does not recognise the windows path separator so you have to do:

task = shlex.split(task.replace('\\','/'))

Cool. Sorry it took so long for me to spot a problem that I was dealing with just a few weeks ago.

Popen can accept a string, but it needs to be in the right context. I've just defaulted to always using a list unless there's a really good reason.

I think shlex is specifically aimed at *nix-type systems, so it may produce some hilarious results on Windows, even ignoring the backslash path separator thing. Anyone want to see what happens to Windows when you have spaces in your filesnames and then escape them with a backslash?

** with John Lennon's Imagine playing in the background **

Imagine, there's no Windows...It's really easy if you try...