Forums

Hi all,

I have been speaking with Harry (PA dev) briefly about this issue but I thought I'd open it up to the community, hopefully so I can get input from others who have had the issue but also then to help those that might come across it.

Basically, when I run my code locally from my system it works perfectly, the problem is, when I run it on PA it just stops running half way through the process - this is only on large processes.

Essentially my script searches for links within files and adds them to a queue for processing. On PA, the script does work on small queues (I've had it completing on 600 links) but when I do larger queues it just stops.

Last night I started it through a scheduled task (on recommendation from Harry) on a large process (this completes in about 20 minutes of my local machine) and this morning it's just listed in the 'processes running' section, my CPU is static, I tested the scheduled task on the small process and as expected it worked fine.

Here is a simplified version of my code (just removing data processing etc.)...

import Queue
from threading import Thread
import csv

# queue
q = Queue.LifoQueue()

# parse the file to get the links and put them in the queue
def get_put_links(linkfile):
    # process file and get the links
    q.put(link)

# run the function
get_put_links(linkfile)

# start pulling from the queue doing GET requests on each one
def multi():
    while True:
        q.get()
        # do stuff with links (GET requests)
        q.task_done()
        if q.empty():
            break

# start 5 threads
t1 = Thread(target = multi)
t2 = Thread(target = multi)
t3 = Thread(target = multi)
t4 = Thread(target = multi)
t5 = Thread(target = multi)
t1.start()
t2.start()
t3.start()
t4.start()
t5.start()

q.join()

I'm starting to think there is a bug or restriction on PA for queue sizes? This is (hopefully) going to be the main processing logic to my app so It's important that it works.

Thanks in advance and hope you're having a nice weekend. :)

Hmm. We don't have any explicit limits on queue sizes -- and if your process was running out of memory it would be more likely to get killed rather than just lock up.

Perhaps you could add some logging to see exactly where it's blocking?

Hey Giles,

Not exactly logging but I've used print and it's while processing the queue. So adding to the queue is fine but then running through each queue item is where it runs into issues.

I've tried a dosen times locally and it works every time.

Where on running through the queue is it blocking? For example, is it blocking when trying to retrieve items from the queue? Or while processing an item it has retrieved?

I've wrote in some logging as suggested, I'll let you know how i get on.

thumbs up for working on a bank holiday :)

Looking forward to hearing more when you've had a chance to log more.

And it's a rainy bank holiday so working's not too painful ;-)

Well it's been a bloody long day!

First off, thanks Giles for introducing me to logging!!

After multiple, long tests, it was clear the issue was happening during the GET request using the requests library. The 'freezing' was due to there being no response and the timeout defaulting to None (meaning it will just wait forever).

I added the timeout parameter...

requests.get(url, timeout=5)

this will throw an error so to catch it I used the recommended exception...

except requests.exceptions.Timeout:
    # handle exception

this however wasn't working and the exception wasn't being caught, after further searching I found this thread which matched my error.

So I added the following and it has run in full for the first time!!

import socket
except socket.timeout:
    # handle exception

I will obviously testing this further by I'm confident the issue is now sorted. I doesn't explain why it was working locally though, the only thing I can think of is differing versions of Requests.

Awesome, glad you sorted it out! Perhaps it's some difference in the internet connection from home vs. the PythonAnywhere cluster? That would be odd, though -- PythonAnywhere is on Amazon EC2 so you'd expect it would be more reliable rather than less reliable.

If the timeouts are only happening for one particular site, perhaps it's because they've blocked us (or more likely, all of EC2)?