Forums

I have a script/task that pulls data from a third party API and performs a bulk_update in a Django database.  The script takes ~2-3 seconds to complete.

I'd like to run the script every 60 seconds or so.  What are the pros/cons of creating 60 scheduled tasks for this vs one always-on task that sleeps in between runs?

Also, is there any reason you don't support task scheduling by the minute (rather than having to create many scheduled tasks)?

The pros and cons are entirely based on what you're comfortable with. Personally, managing 60 scheduled tasks sounds like way more trouble that writing some code to do that in an always on task.

I guess the question is more about how each option consumes CPU minutes. In other words, if I use an always-on task with an infinite loop that implements time.sleep(60) after each run, would that consume the same CPU minutes as 60 separate scheduled tasks?

It would consume slightly less; starting up a Python interpreter uses up a small but non-zero amount of CPU time, and by having a single process running all of the time you would save that for each run through your loop.

Got it. Thanks! Out of curiosity, what criteria do you use to determine that a script is not consuming CPU minutes? Like is there some thread attribute in the OS that tells you that the CPU isn't being consumed?

Also, doesn't time.sleep(60) have to consume some kind of resource to know when to come back to life?

Linux has a concept of "control groups", aka cgroups for processes; each process belongs to one, and when we start a process for you, it, and any processes that it spawns, are put into a cgroup identified by your username. A cgroup keeps a log at kernel level of the number of CPU-nanoseconds that have been used by that group, based on the kernel's task scheduling records; we check it periodically, and store the value that it contains at that point then zero it out so that it starts counting again. (That in theory means that we might slightly undercount usage, because any CPU time used between when we check and when we zero it out will be ignored, but in practice that is small enough that it's a cost we're willing to take.) Then we aggregate those numbers across all of the servers where your code is running to come up with a total.

You're right that time.sleep(60) will use a small amount of CPU resources, but it will be of the order of a few milliseconds at the most.

I might be off-topic here, bust just gotta say, you guys have created an amazing product and are capable of awesome support. You have my respects(and my money :D).

Thanks! :)

I had the exact same question that pandichef had. Really appreciate your thorough and transparent answers, PythonAnywhere guys!

No problem, glad you found it useful :-)

how many cpu seconds do i get daily on always on task for a paid account

@kevin254 We don't distinguish CPU seconds for particular features, you get a total that you can be used by the code running on PythonAnywhere (except for the web apps which are currently not taken into account for the CPU count). The number of CPU seconds depends on the plan you choose. You can always upgrade an existing plan and add more CPU seconds, if you need.