Forums

How's it going guys,

Getting an error when attempting to send_file a generated xlsx from xlsxwriter through BytesIO

io.UnsupportedOperation: fileno

SystemError: <built-in function uwsgi_sendfile> returned a result with an error set

Did some digging and found some information on the issue:

https://github.com/unbit/uwsgi/issues/1126

https://github.com/jessicabp/CapstoneServer/issues/24

http://beenje.github.io/blog/posts/uwsgi-send_file-and-python35/

Adding the wsgi-disable-file-wrapper = true flag to the uwsgi.ini file seems to be the fix that has been proposed from what I have read.

Is it possible to add flags to the uswgi config through wsgi.py or any other means or should I just rewrite the app to output to csv ?

Hi there, we don't support sendfile I'm afraid. It's not just the uwsgi configuration that prevents it, our current load-balancer configuration won't allow it either...

Thanks for the response, appreciate the help.

Thinking of workarounds now, if I want to periodically download a generated excel document should I send it to Static and then redirect users from the download link to the static file route?

That sounds like it'll work just fine. As long as you don't need to restrict access to the file? Or maybe security-through-obscurity is good enough, if you generate random filenames?

Hmm I can't recall but I was hoping I'd be able to role restrict the static route using flask_user to support roles, but now that I think of it, it probably will be a pita to role require a static file won't it.

I think if you're using our static files framework, flask just won't be involved...

Hmm, still perusing through to see if I can get this accomplished.

Here's a forum post from 2015 where someone was looking to accomplish something similar and it was suggested that they cook up a flask app and use send_from_directory, which is a send_file call. This shouldn't work either I wouldn't think?

Really appreciate the help Harry, you guys are A1.

https://www.pythonanywhere.com/forums/topic/3113/

Nope, I'm afraid that won't work either. Static files will work fine, or just "normal" (non-streaming) file responses... Are these excel files particularly large?

Not really no, I'd expect no more than 20MB maximum, realistically more like 5ish on average.

I hadn't set up the PA static files, I only specified the directory and didn't bother providing URL because I figured , it's pretty low traffic and I am just putting everything through flask-assets/bundle, I am not 100% aware on how flask-assets handles things but I am pretty sure it just makes routes to the app directory "static" and I didn't have the will to look into how to get that to jive with PA static files, there's some options there to specify a CDN or an S3 bucket but I again I didn't look into much.

So that said I am serving through flask, and I was trying to think how I could coax that particular route to take a @login_required, @role_required decorator.

Ah, I'm out of my depth on the flask front now.

re our static files service, the way it works is that, you set up a URL prefix (eg /static/), and a path (eg ~/myproject/assets) then our front-end servers will intercepts any requests for /static/path/to/a/file, and go and check if ~/myproject/path/to/a/file exists. If it does, the frontend serve will serve it directly, bypassing flask altogether.

If the path does not exist, the frontend server passes the request onto flask, who can then do whatever it wants with it (usually, 404, but you could override that)

Gotcha, I'll put some more thought against a work around and if I come up with something I'll post it back here in the off chance it would be profitable for anyone else.

Just ad-libbing here, but if you wanted to add some sort of password-protection to the static files service, you could use the HTTP basic auth "password protection" we have at the bottom of the web tab. You wouldn't want that for your main site, but you could set up a second web app (say, assets.yourdomain.com), and use that just for static assets, and add password protection to it.

Then you can put links to said assets from your main site, with credentials built-in, using the https://username:password@assets.yourdomain.com syntax.

Hacky. Probably too hacky. But I thought I'd mention it anyways...

Here's what I am thinking, sort of based on your security-through-obfuscation thought.

Basically the file is generated on the fly anyway, I can set up the PA static folder along side of my static folder as opposed to sending it where all the web assets are so it won't interfere with anything webassets is doing, send the document with a uuid based name to it. Then delete the file post download.

So basically the download route which is behind user access will generate the single serve file and provide the route.

Only thing I have to think about is file clearing post download. I am thinking having the download expire based on time and then trigger the delete but I feel like that's somewhat annoying. There must be a way to capture that the user download has finished?

Working out when a file has been downloaded is a very thorny issue - how do you know the user got the file correctly? How do you know they didn't accidentally download it to a device where it's useless to them? etc.

I would think your best bet would be to serve the file for a set period and let your users know that the file will no longer be available after that period. Store the filename and a timestamp when you create a file and then run a scheduled task to clear up files that are too old to keep around.

Yea, you're right, a bit of a bummer in the sense that its not going to be an insta fix.

I'm going to go with that. Route with role requirement to page with a generator, sends the file to a static folder. A db model with the name, timestamp and user. If file for user exists then display link, if not display generator, expires and deletes the file/db entry after time limit etc.

Appreciate the help guys.

We have written a help page to help with this in future http://help.pythonanywhere.com/pages/FlaskSendFileBytesIO/