Forums

I'm designing a project that could really use a global dictionary and quad-tree stored in RAM. I'm wondering: do web workers run different instances of the python program, meaning they would store different values in their global variables? If so, any suggestions towards a workaround are appreciated, but not vital.

Yes, they do -- in fact, they could in theory be on completely different machines (though right now on PythonAnywhere they are on the same one. One of the purposes of using multiple web worker processes is to allow vertical scaling.

Most web apps tend to use the database or message queues to pass information between web worker processes. The database is the best option on PythonAnywhere right now. You could also use files.

Picking up 2 years later... In 2018 are web workers ever on different machines? If they were, how quickly would files be in sync? If I write json or pickle values out to a file, and a different machine's web worker tried to read it 1/2 second later, would it get the updated values?

Web workers for a particular web app are still always on the same machine. There is no file sync delay.

Hi Guys,

I've finally hit what appears to be the same problem. When calling the same page quickly before it's loaded the 2nd web worker takes part but doesn't appear to access the same global variable as the 1st WW as a key value is missing. It then leads to a mess.

As my app is still in it's very beginning, and whilst i later intend to move to PostGIS for storage, is there a way to artificially drop to one web worker so that i can avoid this mess until i'm ready to properly overcome it?

Thanks,

Tom

we could tweak that for you if you want, but keep in mind that that will artificially confine you to not being able to deal with high traffic, and later on when you have to deal with it it will be even harder.

are you using the database at all? using global variables is a pretty weird practice- why do you need it to sync across different workers?

Hi Conrad,

A tweak that is reversible?

I’m using a Postgres database for my GIS queries. As for user session data, I’m storing this as a dict and using Sessions in flask to create references (a Dict inside a Dict). A database is obviously the way forward, just need to get onto that bit!

Yes, the tweak is reversible -- what we'd do is just change the config for your website so that it has only one worker process available to it. Then all requests would go to the same process, of course, so your global variable would be fine. At a later point, when you've fixed the dependency on the global state, we'd just change the config back to the appropriate number of processes for your account type so that it could handle more than one request at a time.

Hi Giles. I think I need the same single-worker tweak on my Django site for basically the same reasons. I'm having problems with globals (which I can eventually write around - but this is just a demo site at this point) but also the intermittent "error loading dependancies" coming from plotly dash (there are 2 views loading in order to have state-less view - their code, not mine).
How difficult / time consuming would it be to limit me to a single worker? (I'll raise this request in a new topic too - but this thread is really the right context) Thank you, John

@johnpwolf12 It's done for you. Just reload our web app.