Forums

I have a Flask-python web app that uploads a file, reads it into a pandas dataframe, performs a bunch of calculations on the data in the dataframe, then outputs an html table. On my computer, the pure python part of the code (the calculations on the dataframe data) took 0.15 seconds, but the same portion of the code on PythonAnywhere took 39.2 seconds (!). (Lapsed time was recorded using tic-toc print statements using time.clock.) That's almost 250x slower. What could be causing this and how can I fix it?

Just to make sure -- you're running this code inside your Flask app, right? It's likely that it's just contention for CPU on the web server. CPU-intensive stuff should really be done from consoles or scheduled tasks.

That said, it's worth noting that cloud servers are generally going to be slower than your local machine unless you pay quite a lot of money. A $200/month server on Amazon is four cores with 15GB RAM -- that is, it's basically the same as a normal laptop these days.

Yes, running from inside my Flask app. I just refactored my program and replaced pandas dataframes with lists of lists. That makes the program 15x faster on my computer than previously. I'll port the new code over to my Flask app and see how it does.

Would CPU-intensive code run faster on a paid account than a free/beginner account (which I am currently on)?

Using the code with lists of lists instead of pandas dataframes made a big difference when running the code in PythonAnywhere. Even the biggest data tables now only take about 30 seconds from clicking submit to getting the output, which is acceptable, and most will be less than 5 seconds.

Wow! That's a huge difference to get from a re-implementation. Glad you could fix it that way.

Any chance that also you could have gotten yourself in the tarpit? If you are running things on a free account, I've seen Pandas really eat my CPU allotment for the day, albeit with good sized dataframes. You can check CPU usage in the upper right corner of the "Dashboard" page.

That won't be the issue if it's in a web app -- the tarpit doesn't apply to them.

Oh, that is good to know.