Forums

Robots.txt

Hi. I've tried submitting my flask app to Google and Bing but am encountering problems. I previously had a robots.txt in my routes.py to disallowing access to bits and indexing. I have now deleted the file and submitted the sites to be searched but Google and bing show that my site can't be indexed due to a robots.txt. a bing search if my site finds it but the description shows "We would like to show you a description here but the site won’t allow us.". Is there something I need to do or am I missing something? Any help would be appreciated.

Think I got it sorted so please ignore this, thanks

No problem, thanks for letting us know!

how do you solve it? I'm facing same issue

Could you give some more details about the problem you're seeing?

I also found the solution. The issue was that ChatGPT could not access my site to extract data even after making the robos.txt available. I solve by disable and enable the web app on PythonAnywhere dashboard.

Thanks for sharing