Forums

File "/home/teamip/mysite/zeroda_updation.py", line 50, in __init__ self.driver.get("https://kite.zerodha.com/") File "/home/teamip/mysite/env/lib/python3.9/site-packages/selenium/webdriver/remote/webdriver.py", line 442, in get self.execute(Command.GET, {'url': url}) File "/home/teamip/mysite/env/lib/python3.9/site-packages/selenium/webdriver/remote/webdriver.py", line 430, in execute self.error_handler.check_response(response) File "/home/teamip/mysite/env/lib/python3.9/site-packages/selenium/webdriver/remote/errorhandler.py", line 247, in check_response raise exception_class(message, screen, stacktrace) selenium.common.exceptions.WebDriverException: Message: unknown error: net::ERR_TUNNEL_CONNECTION_FAILED

this is my bash cosole error that I'm getting whenever i try to run selenium code. how can i resolve this?

That looks like your proxy settings may be incorrect. What do you get when you run this in that console?

echo $http_proxy

http://proxy.server:3128

do i have to upgrade my pythonanywhere account in order to run selenium code? And i have checked this example code from selenium import webdriver chrome_options = webdriver.ChromeOptions() chrome_options.add_argument("--no-sandbox") chrome_options.add_argument("--headless") chrome_options.add_argument("--disable-gpu") browser = webdriver.Chrome(options=chrome_options) try: browser.get("https://www.google.com") print("Page title was '{}'".format(browser.title)) finally: browser.quit() its running fine , I'm getting output for this code

Example code from selenium help page should work on the free account, as www.google.com is whitelisted. See https://www.pythonanywhere.com/whitelist/

can you whitelist this site it have all the proper documentation required

http://kite.zerodha.com

We can't whitelist that as it's not open. There is login required.

Hello, I am getting to know pythonanywhere and i stumbled on the same issue. Would it be possible to whitelist following domain?

https://www.biddit.be/

Thank you and kind regards, Jef

@jefs If you would like the above domain to be considered for the whitelist , please send us a link to the API documentation. Please note we can only whitelist sites with an official public API.

This website does not have a public API. I would like to move my webscraper to pythonanywhere. If only APIs are allowed i cannot use your services. Thank you for the quick response. Cheers, Jef

That's unfortunate.

Hello everyone! I'm currently stuck here. If I switch to a paid account, will I still be facing the same problem? Or will it be resolved with an upgraded account? (This is a private website with a login requirement, but I have the owner's permission for scraping).

Thank you so much for your help!

Edited 28.12.22: I've already switch the account and everything is working perfectly! Thanks!

Excellent, glad to hear you got it working!

Hi, I am getting the same error message like above

selenium.common.exceptions.WebDriverException: Message: unknown error: net::ERR_TUNNEL_CONNECTION_FAILED

with the following code from https://help.pythonanywhere.com/pages/selenium/, where I exchanged https://www.google.com with https://vemcount.app .

from selenium import webdriver
chrome_options = webdriver.ChromeOptions()
chrome_options.add_argument("--no-sandbox")
chrome_options.add_argument("--headless")
chrome_options.add_argument("--disable-gpu")
browser = webdriver.Chrome(options=chrome_options)
try:
    browser.get("https://vemcount.app")
    print("Page title was '{}'".format(browser.title))
finally:
   browser.quit()

I tried switching to a paid account like njaratwk above, but this did not resolve the issue. Is it possible to access this website with selenium?

Are you running the code in a console that you created after you upgraded? If you are running it in a console from before you upgraded, it will still be trying to use the proxy.

Hey, I had the same issue using the free tier. After upgrading to the $5/month subscription and reload my app from the file edit page. I don't have that error any more. Just like it states on the Whitelist page : "Paid accounts have unrestricted access" Good luck.

i am also facing this issue below is my code.

from selenium import webdriver
chrome_options = webdriver.ChromeOptions()
chrome_options.add_argument("--no-sandbox")
chrome_options.add_argument("--headless")
chrome_options.add_argument("--disable-gpu")
driver = webdriver.Chrome(options=chrome_options)
try:


    driver.get("https://tickets.rugbyworldcup.com")

    try:
        time.sleep(2.5)
        driver.find_element(By.XPATH, "//button[@type='button']").click()
    except:
        pass

    time.sleep(30)

    try:
        driver.find_element(By.XPATH, "//button[text()='Accept All Cookies']").click()
    except:
        pass

    # --------------- logging in -------------------
    time.sleep(1)
    log = driver.find_element(By.XPATH, "//a[contains(text(),'Log in')]")
    driver.execute_script("arguments[0].click()", log)

    print("logged in clicked")


finally:
    driver.quit()

can somebody help me about it. it will be a great pleasure

[edit by admin: formatting]

Free accounts have restricted Internet access and can only access the list of official public APIs on our allowlist. From the looks of it, that site isn't an API, so it's not something we can add to the list.

I am getting the same issue. I am scrapping https://www.sciencedirect.com this site. Can someone help me?

@TeamResearcher Free accounts have restricted Internet access and can only access the list of official public APIs on our allowlist. From the looks of it, that site isn't an API, so it's not something we can add to the list.

How about the search result page of google.com? For example, the page that appears when I search for "ABC" and the URL looks like "https://www.google.com/search?q=ABC&oq=ABC&gs_lcrp=EgZjaHJvbWUyBggAEEUYOTIKCAEQABixAxiABDIKCAIQABixAxiABDIHCAMQABiABDINCAQQABiDARixAxiABDIKCAUQABixAxiABDINCAYQABiDARixAxiABDIHCAcQABiABDINCAgQABiDARixAxiABDIHCAkQABiABNIBCDE5MDlqMGo5qAIAsAIA&sourceid=chrome&ie=UTF-8".

However, it seems that we are unable to use it. Has anyone been able to use the search result page?

It could be google preventing as they don't want to scraped.

hmm, i might just update my account then.

You could always upgrade to see if it works. If you downgrade in 30 days we'll refund the money. Just send a quick email to support@pythonanywhere.com asking for the refund.

I have the same issue and I have already switch to the Hack membership but still getting the same error, using selenium, accessing https://www.coes.org.pe/Portal/portalinformacion/demanda

net::ERR_TUNNEL_CONNECTION_FAILED

=========

# chromedriver veresion 122
# chorme version
from selenium import webdriver
from selenium.webdriver.common.by import By

chrome_options = webdriver.ChromeOptions()
chrome_options.add_argument("--no-sandbox")
chrome_options.add_argument("--headless")
chrome_options.add_argument("--disable-gpu")
driver = webdriver.Chrome(options=chrome_options)
try:

    webCOES = 'https://www.coes.org.pe/Portal/portalinformacion/demanda'
    driver.get(webCOES)

[edit by admin: formatting]

What happens if you run our exact sample code from the help page?

'https://www.set.or.th/en/listing/ipo/upcoming-ipo/set' can you whitelist this site it have all the proper documentation required

Use the request form on this page for whitelist additions - https://help.pythonanywhere.com/pages/RequestingAllowlistAdditions/

Hi I am trying to open the web page https://www.nseindia.com/market-data/pre-open-market-cm-and-emerge-market, but getting selenium.common.exceptions.WebDriverException: Message: unknown error: net::ERR_TUNNEL_CONNECTION_FAILED. Though I see that nseindia.com is whitelisted

Please help Thanks

I believe that that site blocks all incoming connections from cloud computing environments because they do not want people to be scraping their content.