Forums

I have a strange issue that happens in PA but not when I run a local server. The project is built with Django and Python 3.10. The user is able to upload a .fit file containing data from a fitness device like Garmin Fenix. The code then parses the .fit file into lists, and dicts. They can be large: the .fit file contains data every second, and fitness episodes can last hours. However, the ones I use for examples are relatively short, 40-60sec. After uploading the file and displaying the data, the user can run an algorithm, defined in a Python class that will generate more data lists. Here is the flow of the program, and where the error occurs. 1. User uploads data, inspects it, then chooses to run the model. All this works fine, graphs created using the Plotly python package. 2. A FITMODEL object is created. I define this object in the settings.py file, to allow its access across different views in the views.py. 3. Step 1 of the modeling is run, and results displayed. So far, all is good. A new page is displayed with the original data and the fit to the data. 4. User is given option to run another routine that must use the FITMODEL object, with its data lists and routines that operate on them. This is where it sometimes fails. When testing if the FITMODEL objec is still not None and contains previously modeled data, the routinme finds FITMODLE is no longer defined and a 404 error page is displayed. Sometimes, this step works! 5. If step 4 works, user is given the chance to post the reults (contained in the FITMODEL object) to a postgresql DB. This step usually fails, as FITMODEL is now None. Is there something about the PA setup that limits the size of data objects? Why does FITMODEL go away on PA but not in other testing evnironments (Windows 11 and Ubuntu)?

The difference is probably that your local development server operates as a single process, while on PythonAnywhere, your account is set up to run with multiple processes (or web workers) to allow your website to handle more traffic. This means that the data might be available on web worker 1, but not on web worker 2. We explain this in more detail on this blog post: https://blog.pythonanywhere.com/200

Thanks for the response, this helps. I am investigating ways to do this (ZODB being one...), and the simplest to me seems to be to just use python's pickle to save my FITMODEL object after each view, then read it back in in the next view. I already have temporary files stored by user_id in my media folder, so I could use that to make the pickled object user-specific. Can you think of major issues with that approach? Security? Because the file storage would be keyed to the user, I dont think I would get issues with showing data to user B that actually comes from user A. Thoughts?

The pickler approach seemd to work fine: Here we create a FITMODEL object in the runModel view:

settings.FITMODEL = FitModel(admusr.id,trkItem[0],excllist,ptslst,mdlmthd)

After running the model, we pickle:

pcklPth = user_pickle_path(admusr, "fitmodel.pckl")
with open(pcklPth,'wb') as f:
    pickle.dump(settings.FITMODEL,f,pickle.HIGHEST_PROTOCOL)

Then in the next view (run_stepFunc) we un-pickle:

admusr = request.user
pcklPth = user_pickle_path(admusr, "fitmodel.pckl")
with open(pcklPth,'rb') as f:
    settings.FITMODEL = pickle.load(f)

At the end of which we pickle again, because the object has changed.

Then in the final view, do the un-pickle to save to the DB.

Thanks for your help, though I would appreciate anyone's thoughts on issues I am not seeing by doing this.

That sounds like a reasonable approach; the only concerns I'd raise would be if the same user hit multiple views at the same time, or out of the order that you expect. In the long run I'd recommend using a normal database like MySQL to store persistent data; we have a blog post with a simple tutorial on how that all fits together that you might want to run through -- the use case is much simpler than yours, but it might be a useful foundation and give you some ideas on how you might structure things.