Forums

Hey,

I use the console to populate an SQLite database every hour and this works really well (can do so in about 30 seconds). One thing I notice is that the console randomly shuts itself down meaning I lose a few hours of data (until I restart it).

I do know about always on tasks but these seem to run much slower (a few minutes versus 30 seconds). Guessing this is because I run the logs on the hour. So my preference would be to keep using the console (running UATV2.py if able to look).

Is there anyway (perhaps a seperate always on task that checks when console has stopped) to restart a console automatically or keep it on?

Hope this makes sense, thankyou for any help from anyone.

I think you mean you are running a scheduled task?

If so, I would suggest just running it not exactly on the hour but some minutes before or after.

Alternatively you could also setup an always on task that loops and repopulates say every 50min

No, I don't mean any scheduled task. I run a console for one of my data collection python scripts. I do this always on task in the console that repopulates every 60 minutes or 1 hour and it is working perfectly, but occasionally the consoles shut down (not sure why) which means it stays shut down until I go into the python file and click "Run" to restart it

edit: Just to clarify i'm using neither of the tasks in the "tasks" section of the website/dashboard as it runs a lot slower. I am literally running it in the "Consoles" section of website/dashboard and it is a lot more efficient

This has happened to me a few times before, and it's probably normal. If you really need your process to constantly be running, then you should use a scheduled task. I wish I had access to always-on tasks :(

Sadly I have found using always-on tasks and scheduled tasks to be much slower in terms of execution time when added there and I have no idea why. This is why I am sticking with Consoles for now.

Although I was thinking I could use a script (that actually is an always on task) to check when my Console has stopped running, would just need to know how to automatically restart it (for example, would os.startfile work?)

The PythonAnywhere API could be useful for this

Wow thankyou this looks like a great resource, however briefly looking over the console section the API allows you to create a console object but not actually start it. Seems you need to do this manually:

"(NB does not actually start the process. Only connecting to the console in a browser will do that)."

Yes, that's right -- the consoles API probably won't help in this instance.

It might be worth digging down into why always-on tasks aren't working well for you, because that's the feature we built to solve exactly this problem. While I'd expect a certain amount of slowdown on the hour for scheduled tasks, because a lot of people schedule stuff to run then, always-on tasks run on a different set of servers and don't have that periodic pattern.

One thing to look out for -- if you write an always-on task that does something, and then exits, then the always-on system will start regarding it as something that keeps crashing, and will slow down the rate at which it restarts it. Always-on tasks are meant to handle scripts that -- in general -- run all the time, with automated restarts being a rare event, for example if there's a hardware problem or if the script crashes unexpectedly.

So just make your script loop the task instead of running it once, and use that as the always-on task

Thanks giles - i'll give it a shot tomorrow and see how it goes / let you know

OK, please do :-)

Hey Giles, the always on task is actually working perfectly :D so I am definitely going to be using this - thanks. The only problem is I am not getting any print output to the log. I have set flush on all my print statements to True as specified to do here. Have waited for about 2 hours now with nothing but "Task preparing to start". It is definitely running though as my database is getting populated

Thanks for letting us know about that! It looks like there was a problem with our logging configuration for your account, which I've fixed -- stuff should be being logged now.

Hey Giles, thankyou so much I am getting full output and it's running swimmingly. My only concern is that there is quite a variance now in how long it takes to actually fully collect the data I need (under 1 minute versus 2-4 minutes each hour) but it is not a game changer.

Excellent, glad you got it working! The speed difference is interesting -- is there any particular pattern, like it's slower during the day but faster at night, or something like that?

Is it slower at 3 AM? ;)

No pattern that i'm seeing. It actually looks quite random. You will get the occasional 3-4 minute run but usually it's taking an average of 2 mins, 10 seconds every time it runs on the hour (give or take 30 seconds). This is compared to around 40-50 seconds using the actual console.

The alwayson servers use some resources to make sure everything's running well, so they will be a bit slower and the rest seems like variation that could just be caused by differing load and network conditions and perhaps because it's doing different things at different times.

Ahh thanks Glenn, that would explain the difference.

Therefore If I were to want to avoid that delay (and switch back to consoles). Would it be possible to have some always on task checking if my python script console is running, and if it isn't restart the console? Obviously I can't use the PythonAnywhere API dull mentioned, but I was thinking some other kind of method (if any).

For instance I have tried os.startfile("script.py") but it seems this method has been removed. I was thinking on running some bash command automatically had the script stopped but will that work in the domain of an always on task?

Thanks :)

There is no way to start and run consoles from always on tasks and we're probably never going to implement one. They have 2 different use cases - consoles are for interactive work that a person is doing and always on tasks are for continuous work that does not require the input of a human.

os.startfile has not been removed - it's a Windows-specific function that will not work on PythonAnywhere because our servers do not run Windows. I'm also not sure how it would help with a "stopped" script. If your script crashes, the always on task system will notice and restart it.

Hey should thankyou all for the help I know I have gone on lots for this topic. In the end I have gone back to consoles although will be manually restarting them. The trade off now is 5 minutes at times versus 30 seconds.

The only final question I would have is whether it is possible to get some kind of notification on when consoles are scheduled to go down so I can manually restart or plan ahead.

Again thankyou for all your help

+1 for the notification before console stops running

Unfortunately I don't think we can give advance notice of when console servers will need to be rebooted; because they're dealing with people doing interactive stuff, sometimes due to combinations of actions that people take, the WebSocket servers that they use get overloaded and need to be restarted. We've put in a lot of code to minimise the probability of that happening over the years, and have got it down to a pretty low minimum, but people still find ways to surprise us and blow things up :-(

Ahh thankyou giles, and worry no more about it. I'll probably try setup my own little email service to notify me when it has stopped. Thankyou for everyones help on this topic

No problem :-)

oof

Issue

I need to go to my Consoles tab every 4-5 hours and manually click on each of my consoles to reload them in case they've closed.

Goal

I want the always-on task in my plan to be able to automatically detect and restart closed consoles.

Background

I have 11 consoles running concurrently as part of my back-end generation.

Why not use Always-on tasks here?

Using always-on tasks has drawbacks:

• It forces me to use PythonAnywhere's timestamp format for messages.
• It reduces development efficiency as I can't watch the console live.
• Start-up/restart times are slow, making updates tedious.
• It would more than double my current costs.

Ideal solution

Allow us to start / restart consoles from the API.

Install some protection there so people can't use the API to spam-open 50 consoles at a time, or whatever else is preventing you from adding this.

Current workaround

Keeping a list of URLs and using Selenium to simulate reopening the console when it's closed. This seems overkill.

Adding a richer API for consoles is definitely on our top do list, though it's not on the near term roadmap. I'll add an upvote on your behalf to bump the priority up, though.

Thank you giles, I'm glad it's already on your "top to-do list", whatever that means.

I appreciate you adding my upvote here.

It means it's among the tickets highly requested / having high priority.

Greetings, its almost the end of 2024 and still the speed issue with always-on-task is not fixed. The login function that runs less than 5 seconds on console takes more than 15 seconds on always-on-task. Perhaps if you could let us start the console through API or any kind I really appreciate it. I have two question 1) Can i use selenium to scrape my own account and start the console then run my code? 2) If I run subprocess.run(["python", script_path, "arg1", "arg2"]) this from my always-on-task would it be using new consoles because consoles are way faster and my project is very much time sensitive?

Have a look at our Consoles API.