Hello, i see that is possible to ask for enabling the use of headless chrome. Is it possible to have it enabled? many thanks

Sure! I've enabled the new feature; the next step is for you to upgrade Selenium for your account -- for example, if you're using Python 3.7, run this in Bash:

pip3.7 install --user --upgrade selenium

...and then you can run Selenium with Chrome using code like this:

chrome_options = webdriver.ChromeOptions()
chrome_options.add_argument("--no-sandbox")
chrome_options.add_argument("--headless")
chrome_options.add_argument("--disable-gpu")
browser = webdriver.Chrome(options=chrome_options)

try:
    browser.get("https://www.google.com")
    print("Page title was '{}'".format(browser.title))

finally:
    browser.quit()

many thanks! i am running it in a pipenv virtual enviroment how am i supposed to update selenium?

sorry, upgrade

I don't use pipenv myself, but for a normal virtualenv you would just activate it then run pip install --upgrade selenium

Hello, Could you please enable headless chrome for my account as well. Thank you

No problem. I have enabled it for your account.

Amazing, Thank you.

No problem :-)

Hi! Could you please enable headless Chrome for my account too. Thanks!

You have already new virtualization system enabled for your account. To use headless Chrome follow the steps mentioned above.

Can you please tell me what Chrome version is installed on my account? I don't know what chromedriver version I need to use.

The chrome and chromedriver that are already installed work together. You do not need to get a different chromedriver version.

I tried to run the code above but I got this two errors:

[Errno 2] No such file or directory: 'chromedriver': 'chromedriver'

selenium.common.exceptions.WebDriverException: Message: 'chromedriver' executable needs to be in PATH. Please see https://sites.google.com/a/chromium.org/chromedriver/home

Right, it appears that your system image has to be upgraded. One word of warning first -- because changing the system image upgrades a lot of the pre-installed Python packages, any code that you have that relies on those packages might break if it's not compatible with the new versions. Also, because the new image has newer versions of Python, if you have any virtualenvs, you may need to rebuild them. If you're happy to go ahead despite that, just let us know and we'll switch you over.

Yes, switch it on please.

OK, it's done. You're on the most recent system image now.

It works now, thank you very much!

Excellent, thanks for confirming!

Could I also this get enabled on my account? Running the code provided above (after including "from selenium import webdriver") produces the following exception:

selenium.common.exceptions.WebDriverException: Message: unknown error: Chrome failed to start: exited abnormally

No problem. It's done for you.

Hi, I'd also like headless chrome activated for my account. Is there any procedure I can follow here?

@mjgibbons3 Just confirm that you are aware of what I posted before in this thread.

@Pafk I'm aware. Can you please upgrade the system image?

No problem -- that's done now. You'll get the new system image and the new virtualization system in any new consoles you start; websites and scheduled/always-on task will pick them both up the next time they are restarted. For the next step for using headless Chrome, see my earlier post.

Thank you @giles - much appreciated!

Hi I followed the instructions but i have the following error: selenium.common.exceptions.WebDriverException: Message: unknown error: Chrome failed to start: exited abnormally

Ok. I have enabled the feature for your account.

Thank you @glenn

Hi, can I have selenium activated for me as well? I'm going to use Chrome driver. Thanks

Hi, can I have headless chrome enable for my account?

@wladzioo I enabled new virtualization for you.

@hamad290sca Yes, but we need to upgrade your system image first. One word of warning first -- because changing the system image upgrades a lot of the pre-installed Python packages, any code that you have that relies on those packages might break if it's not compatible with the new versions. Also, because the new image has newer versions of Python, if you have any virtualenvs, you may need to rebuild them. If you're happy to go ahead despite that, just let us know and we'll switch you over.

@pafk yes please go ahead

OK, I upgraded your system image to the most recent one and enabled beta virtualization feature for you.

Hi, could you please enable headless chrome for my account too? many thanks!

Yes, it's done for you.

Hi pafk, thank you for enabling headless chrome for me. However when I run the sample script supplied at the begining of this topic I got the following error:

browser = webdriver.Chrome(options=chrome_options)
TypeError: __init__() got an unexpected keyword argument 'options'

Could you please help me with this issue?

@ioanf did you upgrade Selenium first as it was advised?

@pafk yes I do.

Did you do that for the same Python version you are using to run the script? The error suggests that Selenium is not upgraded.

@pafk Huh, my bad. Obviously I upgraded Selenium to Python 3.7 not 3.8 I am currently using. Now everything works like charm :) Thank you once again!

Good to hear that you sorted it!

please enable for me

No problem. I have enabled it for your account.

Can you please enable for my account as well? Thank you!

No problem. I have enabled it for your account.

Hello, would it be possible to enable headless Chrome for my account please? Thanks!

ok, I have switched that on for you (ie. you will be able to run chrome). However, note that you are currently on a system image that may not have the most updated chrome binaries installed. If you want we can switch you over to the newest system image, but note that that will update python libraries and break your virtualenvs etc (you would have to reinstall them), as the python versions would have changed etc. Alternatively you can build and install the binaries yourself.

Hi Conrad, thanks - please go ahead and switch me over to the newest system image :) I don't mind reinstalling my virtualenvs

Ok. I have switched your system image for you.

Can you please enable Chrome headless for my account as well? Thank you! The code above gives me an error: selenium.common.exceptions.WebDriverException: Message: unknown error: Chrome failed to start: exited abnormally

@mouhabsnyeei no problem. It's done for you.

Can you please enable Chrome headless for my account as well? Thank you!

Sure, that's done now -- see my previous post for the next steps.

I would like to have Chrome headless enabled as well, thanks!

@TreeofPython. No problem, it's done for you

Awesome, this fixed my selenium issues. Thanks!

hey can anyone help i have used the above code but i am receiving the following exception.

WebDriverException: Message: unknown error: Chrome failed to start: exited abnormally (unknown error: DevToolsActivePort file doesn't exist) (The process started from chrome location /usr/bin/chromium-browser is no longer running, so ChromeDriver is assuming that Chrome has crashed.) (Driver info: chromedriver=2.42.591071 (0b695ff80972cc1a65a5cd643186d2ae582cd4ac),platform=Linux 5.4.0-1020-aws x86_64)

There's a beta feature that we have to activate for your account to make it work; I've done that, so if you try again in a newly-started console it should work now.

thank you. headless chrome is now working but selenium webdriver wait is not working properly i am getting timeoutException when trying to wait for an element to load. can you look into it ThankYou

That suggests just that the element that you are expecting to be present is not there; you can use the get_screenshot_as_file method on your WebDriver object to get a screenshot of the page and find out what it is displaying.

One thing to remember is that free accounts have restricted Internet access and can only access sites on our whitelist, so if you're trying to access a site that is not there, you might be getting an "Access Denied" page rather than the page you expect.

i am getting this error on my code was working fine before

WebDriverException: Message: unknown error: failed to write automation extension zip (Driver info: chromedriver=2.42.591071 (0b695ff80972cc1a65a5cd643186d2ae582cd4ac),platform=Linux 5.4.0-1020-aws x86_64)

You're over your disk quota, so there is no space for web driver to write the files that it needs to work.

Hi there, I am trying to run the above code but I am getting the following error:

"selenium.common.exceptions.WebDriverException: Message: unknown error: Chrome failed to start: exited abnormally"

I think my account needs to be enabled too? TIA, Clem.

hey there. if i upgrade my account will i be able to use the webdriver wait function

hi. can you enable the headless chrome on my account too i am currently getting the following error

WebDriverException: Message: unknown error: Chrome failed to start: exited abnormally (unknown error: DevToolsActivePort file doesn't exist) (The process started from chrome location /usr/bin/chromium-browser is no longer running, so ChromeDriver is assuming that Chrome has crashed.) (Driver info: chromedriver=2.42.591071 (0b695ff80972cc1a65a5cd643186d2ae582cd4ac),platform=Linux 5.4.0-1020-aws x86_64)

Сan you please activate chrome headless for my account? Thank you!

@clemlanoe and @MrGorsky -- no problem -- that's done now. You'll get the new system image and the new virtualization system in any new consoles you start; websites and scheduled/always-on task will pick them both up the next time they are restarted. For the next step for using headless Chrome, see my earlier post.

@tweets12 -- likewise, that's done for you, but one thing to remember is that free accounts have restricted Internet access and can only access sites on our whitelist.

@testpro1 -- you can use the webdriver wait function in a free account as well as in a paid account.

i am using selenium to take screenshot of tweet but twitter just loads the body of the site and tweet portion shows the try again button. tweet is not loading. anyone can help

Can you please upgrade my system image? i've the same problem with chrome

@tweets12 that sounds like you are running into the limitations of a free account. Free accounts have restricted Internet access and can only access sites on our whitelist.

@Abv1206 -- no problem -- that's done now. You'll get the new system image and the new virtualization system in any new consoles you start; websites and scheduled/always-on task will pick them both up the next time they are restarted. For the next step for using headless Chrome, see my earlier post.

twitter is on white list still not opening :/

@giles thank you! i'll try it :)

@tweets12 their API is, but their website as a whole is not. It is likely that the page that you are trying to scrape is using a hostname that is not part of their API.

@Abv1206 great!

if i upgrade my account will i be able to access twiiter like its working on localhost server?

@tweets12 yes

Can you please upgrade my system image? i've the same problem with chrome

No problem. I have updated your account.

thanks! :)

Hey, could you please activate selenium for my account

No problem, it's done for you.

Hello, would it be possible to enable headless Chrome for my account please? Thanks!

No problem -- that's done now. You'll get the new virtualization system in any new consoles you start; websites and scheduled/always-on tasks will pick it up the next time they are restarted. For the next step for using headless Chrome, see my earlier post.

Hello, could you please activate headless Chrome for my account? Thanks!

No problem, that's enabled for you @Monega.

Hello, could you please activate headless Chrome for my account? Thanks!

Sure -- that's done now.

Hi, even-though i have my chrome webdriver in the path , am getting below error No such file or directory: '/home/pythontrad/xxx.exe'

Could you show your code that you use to start Chrome?

Hello, could you please activate headless Chrome for my account? Thanks!

No problem. It's done for you.

Hello, would it be possible to have headless Chrome enabled on my account? Thank you

Sure! However, we would need to upgrade your system image as well as enabling that; one word of warning before we do that -- because of the changes to the point releases of Python, any virtualenvs you have might break -- and if you're not using virtualenvs, the pre-installed Python modules will be upgraded so that might break any code you have that relies on the old installed versions.

If you're happy for us to switch you over despite that, then let us know.

Thank you, Giles. Acknowledged - That's fine by me.

No problem -- that's done now. You'll get the new system image and the new virtualization system in any new consoles you start; websites and scheduled/always-on task will pick them both up the next time they are restarted. For the next step for using headless Chrome, see my earlier post.

Hello, could you please activate headless Chrome for my account? Thanks!

Hi @yonataliasu13 --- it's already enabled for you. Maybe you need to update selenium as Giles advised here: https://www.pythonanywhere.com/forums/topic/27202/#id_post_78144 ?

Hello, could you please activate headless Chrome for my account? Thanks! And how should I update selenium? I am new to this and I do not know where to run the following command: "pip3.7 install --user --upgrade selenium"

@Bernas99 -- it's done for you. You should run the command in the Bash console running on your PA account.

How should I call the webdriver? I am doing it like this:

from selenium import webdriver; driver = webdriver.Chrome()

because I can not upload de chromedriver.exe to my pa files. Thanks in advance, and sorry for being a noob

Do you get any errors?

Yes. Message: unknown error: Chrome failed to start: exited abnormally

I can not run my .py file. Please help me. I think that the headless chrome does not work for my scrape. I would like to use the regular chrome webdriver. However I can not do that

You have a free account, so most of the scraping won't work. See https://www.pythonanywhere.com/whitelist/

Also, the correct headless Chrome configuration is described here: https://www.pythonanywhere.com/forums/topic/27202/#id_post_78144

Hello, could you please activate headless Chrome for my account? Thanks!

No problem -- that's done now. You'll get the new virtualization system in any new consoles you start; websites and scheduled/always-on tasks will pick it up the next time they are restarted. For the next step for using headless Chrome, see my earlier post.

Hello, could you please activate headless Chrome for my account? Thanks very much.

Federico

@fedebenji it's done for you

Hello, could you please activate headless Chrome for my account? Thanks a lot.

No problem. I have enabled it for you account.

Hi, I have been having trouble running my .py file since I had headless chrome activated on my account. I followed the steps here to upgrade selenium: https://www.pythonanywhere.com/forums/topic/27202/#id_post_78144. I was able to get the sample script to run at that link, so it appears selenium with headless chrome is working correctly. However when I try to run my .py script on PA, I am getting the error output below. It looks like it is getting hung up trying to access the site? I am not having issues running this code on my computer:

Traceback (most recent call last):
  File "/usr/lib/python3.8/site-packages/urllib3/connection.py", line 158, in _new_conn
    conn = connection.create_connection(
  File "/usr/lib/python3.8/site-packages/urllib3/util/connection.py", line 80, in create_connection
    raise err
  File "/usr/lib/python3.8/site-packages/urllib3/util/connection.py", line 70, in create_connection
    sock.connect(sa)
ConnectionRefusedError: [Errno 111] Connection refused
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
  File "/usr/lib/python3.8/site-packages/urllib3/connectionpool.py", line 597, in urlopen
    httplib_response = self._make_request(conn, method, url,
  File "/usr/lib/python3.8/site-packages/urllib3/connectionpool.py", line 663, in urlopen
  File "/usr/lib/python3.8/site-packages/urllib3/connectionpool.py", line 354, in _make_request
    conn.request(method, url, **httplib_request_kw)
  File "/usr/lib/python3.8/http/client.py", line 1230, in request
    self._send_request(method, url, body, headers, encode_chunked)
  File "/usr/lib/python3.8/http/client.py", line 1276, in _send_request
    self.endheaders(body, encode_chunked=encode_chunked)
  File "/usr/lib/python3.8/http/client.py", line 1225, in endheaders
    self._send_output(message_body, encode_chunked=encode_chunked)
  File "/usr/lib/python3.8/http/client.py", line 1004, in _send_output
    self.send(msg)
  File "/usr/lib/python3.8/http/client.py", line 944, in send
    self.connect()
  File "/usr/lib/python3.8/site-packages/urllib3/connection.py", line 181, in connect
    conn = self._new_conn()
  File "/usr/lib/python3.8/site-packages/urllib3/connection.py", line 167, in _new_conn
    raise NewConnectionError(
urllib3.exceptions.NewConnectionError: <urllib3.connection.HTTPConnection object at 0x7f623f9f9880>: Failed to establish a new connection: [Errno 111] Connection refused

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/home/lilo2k19/webbot.py", line 172, in <module>
    searchforammo()
  File "/home/lilo2k19/webbot.py", line 39, in searchforammo
    driver.get("https://www.targetsportsusa.com")
  File "/home/lilo2k19/.local/lib/python3.8/site-packages/selenium/webdriver/remote/webdriver.py", line 333, in get
    self.execute(Command.GET, {'url': url})
  File "/home/lilo2k19/.local/lib/python3.8/site-packages/selenium/webdriver/remote/webdriver.py", line 319, in execute
    response = self.command_executor.execute(driver_command, params)
  File "/home/lilo2k19/.local/lib/python3.8/site-packages/selenium/webdriver/remote/remote_connection.py", line 374, in execute
    return self._request(command_info[0], url, body=data)
  File "/home/lilo2k19/.local/lib/python3.8/site-packages/selenium/webdriver/remote/remote_connection.py", line 397, in _request
    resp = self._conn.request(method, url, body=body, headers=headers)
  File "/usr/lib/python3.8/site-packages/urllib3/request.py", line 70, in request
    return self.request_encode_body(method, url, fields=fields,
  File "/usr/lib/python3.8/site-packages/urllib3/request.py", line 150, in request_encode_body
    return self.urlopen(method, url, **extra_kw)
  File "/usr/lib/python3.8/site-packages/urllib3/poolmanager.py", line 324, in urlopen
    response = conn.urlopen(method, u.request_uri, **kw)
  File "/usr/lib/python3.8/site-packages/urllib3/connectionpool.py", line 663, in urlopen
    return self.urlopen(method, url, body, headers, retries,
  File "/usr/lib/python3.8/site-packages/urllib3/connectionpool.py", line 663, in urlopen
    return self.urlopen(method, url, body, headers, retries,
  File "/usr/lib/python3.8/site-packages/urllib3/connectionpool.py", line 663, in urlopen
    return self.urlopen(method, url, body, headers, retries,
  File "/usr/lib/python3.8/site-packages/urllib3/connectionpool.py", line 637, in urlopen
    retries = retries.increment(method, url, error=e, _pool=self,
  File "/usr/lib/python3.8/site-packages/urllib3/util/retry.py", line 399, in increment
    raise MaxRetryError(_pool, url, error or ResponseError(cause))
urllib3.exceptions.MaxRetryError: HTTPConnectionPool(host='127.0.0.1', port=49233): Max retries exceeded with url: /session/db7d2cd3b4c08635cb2714bf3e722059/url (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f623f9f9880>: Failed to establish a new connecti
on: [Errno 111] Connection refused'))

Free accounts can only connect out using http(s) to our whitelist of sites. To be considered for the whitelist, a site must have publicly available API documentation and be something that other free users might find useful. If the site has public API documentation, send us a link to it and wee'll consider it for the whitelist.

Hi, I am not on the Free account, I signed up/paid for a custom plan. Is there a different account I would need to sign up for to access sites not on the whitelist?

Thanks

ah no- if you have a paying account, then you should be restricted to the whitelist. However, I see that you just upgraded to a paying account- could it be that you were still using a console from before you upgraded? you would need to start new consoles for the changes to take effect.

Hi, could you please enable headless chrome for my account as well. Thanks in advance!

@osari It's enabled for you

Hello, would it be possible to enable headless Chrome for my account please? Please note that if it entails upgrading the system image first I understand and accept the consequences. Thanks!

Hi. Facing the below error while trying to invoke Chrome. FileNotFoundError: [Errno 2] No such file or directory: './chromedriver': './chromedriver'

Please let me know if my accounts needs to be enabled with any related option.

I agree with the consequences of upgrade, if any.

Thanks in advance, Raja

@ambext -- no problem, that's done now. We did need to switch over your system image, so you will need to rebuild any virtualenvs/update your code to use the newer system-wide installed packages. Once that's done, you can upgrade Selenium and use headless Chrome using the instructions above.

@Raja2908 -- I've switched the new virtualization system on for you too, so you can use headless Chrome if you upgrade Selenium as per the earlier post.

However, from that error message it looks like your code might be explicitly specifying the path to Chromedriver; you should remove that part of your code so that it can pick up the location of Chromedriver from the system environment.

Thanks for the quick response. Tried upgrading selenium but looks like requirement already satisfied. Facing this error now. Tried multiple times. Can you please let me know if i am missing something?

WebDriverException: Message: unknown error: Chrome failed to start: exited abnormally
  (unknown error: DevToolsActivePort file doesn't exist)
  (The process started from chrome location /usr/bin/chromium-browser is no longer running, so ChromeDriver is assuming that Chrome has crashed.)
  (Driver info: chromedriver=2.42.591071 (0b695ff80972cc1a65a5cd643186d2ae582cd4ac),platform=Linux 5.4.0-1020-aws x86_64)

Executed the below code,

chrome_options = webdriver.ChromeOptions()
chrome_options.add_argument("--headless")
chrome_options.add_argument("--disable-gpu")
browser = webdriver.Chrome(options=chrome_options)

try:
    browser.get("https://www.google.com")
except:
    print('Some error')
finally:
    browser.quit()

[edit by admin: formatting]

Are you running the code in a console that was opened before we applied the new settings to your account? If not, that might be the problem -- try running it from a new console.

Hi, I created a new Jupyter notebook, changed kernel to my virtual env and then executed the above code. Update: Working good when run in bash console after switching to virtual environment. Thanks for your help. Is there any restriction in executing chromedriver in jupyter notebook?

Hi, could you please enable headless chrome for my account as well. Thanks a lot

@Raja2908 that's still an experimental feature and it's not working in notebooks yet.

@MomenEssam9, no problem, it's done for you.

Could you please enable headless chrome for me too, thanks!

Sure, no problem -- that's done now. See my earlier post for the next steps.

Hello, Could you please enable headless chrome for me too, I am a free user, Thanks

No problem, @greenpine, it's done for you.

Hello, can u enable headless chrome for me? Thanks

@ibotsupp, you had it already enabled. If you still get errors, check Giles' previous post.

File "/home/ibotsupp/bot/ibotv2.py", line 51, in init self.driver = webdriver.Chrome(options=chrome_options)

File "/home/ibotsupp/.local/lib/python3.8/site-packages/selenium/webdriver/chrome/webdriver.py", line 76, in init RemoteWebDriver.init(

File "/home/ibotsupp/.local/lib/python3.8/site-packages/selenium/webdriver/remote/webdriver.py", line 157, in init self.start_session(capabilities, browser_profile)

File "/home/ibotsupp/.local/lib/python3.8/site-packages/selenium/webdriver/remote/webdriver.py", line 252, in start_session response = self.execute(Command.NEW_SESSION, parameters)

File "/home/ibotsupp/.local/lib/python3.8/site-packages/selenium/webdriver/remote/webdriver.py", line 321, in execute self.error_handler.check_response(response)

File "/home/ibotsupp/.local/lib/python3.8/site-packages/selenium/webdriver/remote/errorhandler.py", line 242, in check_response raise exception_class(message, screen, stacktrace)

selenium.common.exceptions.WebDriverException: Message: unknown error: Chrome failed to start: crashed (unknown error: unable to discover open pages)

(The process started from chrome location /usr/bin/chromium-browser is no longer running, so ChromeDriver is assuming that Chrome has crashed.) (Driver info: chromedriver=2.42.591071 (0b695ff80972cc1a65a5cd643186d2ae582cd4ac),platform=Linux 5.4.0-1020-aws x86_64)

What wrong? Someone understand what need to fix?

are you already using giles's code as linked? otherwise, what is the code you are using?

also- what page are you trying to access? as a free user, you would be limited to a whitelist

Yeah, i'm already using gile's code for selenium headless, i'm trying to get access for instagram page

Maybe the problem is that they gave me a browser firefox, not google chrome. All the errors that I receive say that the web driver cannot create a new browser session.

The exception you're getting is not from Firefox, it's from chromium. It is fairly likely that instagram have measures to prevent scraping of their site. They have an API, use that instead.

Hmm, in my local machine its works but without headless mode. Probably problem in this mode.

Hmm, in my local machine its works but without headless mode. Probably problem in this mode.

Perhaps. You could try using Selenium with pyvirtualdisplay like we do with Firefox on this help page.

Thanks for advice will try

Hi, could you lep me please

from selenium import webdriver
def start():
options = Options()
options.add_argument('--ignore-certificate-errors')
options.add_argument('--ignore-ssl-errors')
options.add_argument("--headless")
options.add_argument("--disable-gpu")
driver=webdriver.Chrome(chrome_options=options)
again(driver)

and i get error:

File "/home/r1dddy4sv/1x.py", line 189, in <module>
start()
File "/home/r1dddy4sv/1x.py", line 21, in start
driver=webdriver.Chrome(chrome_options=options)
File "/usr/lib/python3.8/site-packages/selenium/webdriver/chrome/webdriver.py", line 65, in __init__
RemoteWebDriver.__init__(
File "/usr/lib/python3.8/site-packages/selenium/webdriver/remote/webdriver.py", line 90, in __init__
self.start_session(desired_capabilities, browser_profile)
File "/usr/lib/python3.8/site-packages/selenium/webdriver/remote/webdriver.py", line 177, in start_session
response = self.execute(Command.NEW_SESSION, capabilities)
File "/usr/lib/python3.8/site-packages/selenium/webdriver/remote/webdriver.py", line 236, in execute
self.error_handler.check_response(response)
File "/usr/lib/python3.8/site-packages/selenium/webdriver/remote/errorhandler.py", line 192, in 
check_response
raise exception_class(message, screen, stacktrace)
selenium.common.exceptions.WebDriverException: Message: unknown error: Chrome failed to start: exited 
abnormally
(unknown error: DevToolsActivePort file doesn't exist)
(The process started from chrome location /usr/bin/chromium-browser is no longer running, so ChromeDriver 
is assuming that Chrome has crashed.)
(Driver info: chromedriver=2.42.591071 (0b695ff80972cc1a65a5cd643186d2ae582cd4ac),platform=Linux 
5.4.0-1020-aws x86_64)

please, help me

You need to be using our new virrtualisation system for headless chrome to work. I have enabled it for your account. New consoles will use the new system.

hey,could you please enable it for me too?

hey ,could you please enable it for me too?

Thank you!

I have one more question

from selenium import webdriver
URL='some url'
hrome_options = Options()
chrome_options.add_argument('--no-sandbox')
chrome_options.add_argument('--ignore-certificate-errors')
chrome_options.add_argument('--ignore-ssl-errors')
chrome_options.add_argument("--headless")
chrome_options.add_argument("--disable-gpu")
driver=webdriver.Chrome(options=chrome_options)
driver.get(URL)
print("Page title was '{}'".format(driver.title))

and i got:

Page title was ""

so i cant get on website

could you help me one more time, please :)

Did you paste the code you are using in your script? Didn't you get any errors while running it? (There is an import missing for Options, you have a typo in third line, you should rather use chrome_options keyword when declaring driver, URL is not a valid url so it will print an empty string, etc...) Anyway -- since you have a free account, you will not be able to scrap internet. Free accounts are restricted to whitelisted domains (public APIs).

hello, can you please enable it for me too? thanks

Hello, Could you please enable headless chrome for me too, I am a free user, Thanks

No problem. I have enabled it for your account.

hi, I followed the instructions, but I still get an error

    14:19 ~ $ python3 /home/vpoghosyan/Barbershop/web_scraper_barbershop.py                                                                                                
xdpyinfo was not found, X start can not be checked! Please install xdpyinfo!
Traceback (most recent call last):
  File "/home/vpoghosyan/Barbershop/web_scraper_barbershop.py", line 26, in <module>
    driver = webdriver.Chrome(options=chrome_options)
  File "/home/vpoghosyan/.local/lib/python3.8/site-packages/selenium/webdriver/chrome/webdriver.py", line 76, in __init__
    RemoteWebDriver.__init__(
  File "/home/vpoghosyan/.local/lib/python3.8/site-packages/selenium/webdriver/remote/webdriver.py", line 157, in __init__
    self.start_session(capabilities, browser_profile)
  File "/home/vpoghosyan/.local/lib/python3.8/site-packages/selenium/webdriver/remote/webdriver.py", line 252, in start_session
    response = self.execute(Command.NEW_SESSION, parameters)
  File "/home/vpoghosyan/.local/lib/python3.8/site-packages/selenium/webdriver/remote/webdriver.py", line 321, in execute
    self.error_handler.check_response(response)
  File "/home/vpoghosyan/.local/lib/python3.8/site-packages/selenium/webdriver/remote/errorhandler.py", line 242, in check_response
    raise exception_class(message, screen, stacktrace)
selenium.common.exceptions.WebDriverException: Message: unknown error: Chrome failed to start: exited abnormally
  (unknown error: DevToolsActivePort file doesn't exist)
  (The process started from chrome location /usr/bin/chromium-browser is no longer running, so ChromeDriver is assuming that Chrome has crashed.)
  (Driver info: chromedriver=2.42.591071 (0b695ff80972cc1a65a5cd643186d2ae582cd4ac),platform=Linux 5.4.0-1020-aws x86_64)

Python and selenium versions.

14:21 ~ $ python3 --version
Python 3.8.0
14:21 ~ $ python3 -c "import selenium; print(selenium.__version__)"                                                                                                    
3.141.0

That stuff relating to xdpyinfo suggests that Chrome is not running in headless mode. See my earlier post for example code showing how to do that.

That stuff relating to xdpyinfo suggests that Chrome is not running in headless mode. See my earlier post for example code showing how to do that.

Hi - I am having some issues with my chrome driver, sometimes it works, but is crashing a lot:

selenium.common.exceptions.WebDriverException: Message: unknown error: Chrome failed to start: crashed (unknown error: unable to discover open pages) (The process started from chrome location /usr/bin/chromium-browser is no longer running, so ChromeDriver is assuming that Chrome has crashed.) (Driver info: chromedriver=2.42.591071 (0b695ff80972cc1a65a5cd643186d2ae582cd4ac),platform=Linux 5.4.0-1020-aws x86_64)

looks like maybe there is a beta feature you can add to my account to help? Can you please add, thank you very much

Chrome uses a large number of processes and a lot of CPU time, so if you are in the tarpit, it may have trouble starting. It will also have trouble if you are not ensuring that you are always closing the browser when you're done with it (including when there is an exception). In the second case, you will keep adding processes and we have a limit on how many processes each user can have on each server.

Hello, could you please enable headless chrome for me too, I am also a free user, Thanks a ton

No problem. I have enabled it for your account.

Could you enable chrome for me as well? thanks! I am also a free user

Sure, we actually already have that enabled for you.

Can you enable chrome for my account?

sure, we've updated that for you

Hi, could you enable chrome for my account, please?

Edit: Thanks for doing it so quickly and on a Sunday!

Hi, could you please also enable it for my account also? Thanks

@Lookout & @dreid254 It's done for you both

Hello, Could you please enable headless chrome for me too. Thanks

No problem. I have enabled it for your account.

Hello Glenn, Kindly help enable headless chrome for me too. Thanks

Sure, no problem -- that's done now. See my earlier post for the next steps.

Hi, I just came across the same problem. Could you please enable the feature also for my account? Many thanks!

Sure, @sofm, it's done for you.

Hello; I have already asked to enable headless chrome, and it has been enabled for me, I run this test :

import selenium
from selenium import webdriver
chrome_options = webdriver.ChromeOptions()
chrome_options.add_argument("--headless")
chrome_options.add_argument("--disable-gpu"
browser = webdriver.Chrome(options=chrome_options)
 try:
 browser.get("https://www.google.com")
 print("Page title is: '{}'".format(browser.title))
 finally:
 browser.quit()

the test works and it shows the title, but when I run my code I get this error:

driver = webdriver.Chrome(ChromeDriverManager().install(), options=chrome_options)
File "/home/greenpine/.local/lib/python3.8/site-packages/webdriver_manager/chrome.py", line 23, in __init__
self.driver = ChromeDriver(name=name,
File "/home/greenpine/.local/lib/python3.8/site-packages/webdriver_manager/driver.py", line 54, in __init__
self.browser_version = chrome_version(chrome_type)
File "/home/greenpine/.local/lib/python3.8/site-packages/webdriver_manager/utils.py", line 142, in 
chrome_version
raise ValueError(f'Could not get version for Chrome with this command: {cmd}')
ValueError: Could not get version for Chrome with this command: google-chrome --version || google-chrome-stable --version

the first part of my code is to log in to my account on Pinterest,

prior to the selenium update, my code worked fine on pythonanywhere,

after the code has been updated to use selenium webdriver, it does not work on pythonanywhere, although it does work on my windows 7 machine at home.

can you please help me to solve this problem?
Thanks.

You do not need to install Chrome and chromedriver in your code - they are already installed for you.

Thanks gelnn for the reply, I have tried some code to install webdriver , I am not sure I have done it correctly, here is the code I have for the driver:

import selenium
from selenium import webdriver
from selenium.webdriver.common.keys import Keys
from selenium.webdriver import Chrome
from selenium.webdriver.chrome.options import Options
from selenium.webdriver.chrome.service import Service
from webdriver_manager.chrome import ChromeDriverManager
chrome_options = webdriver.ChromeOptions()
chrome_options.add_argument("--headless")
chrome_options.add_argument("--disable-gpu")
browser = webdriver.Chrome(options=chrome_options)

driver = webdriver.Chrome(executable_path='/home/greenpine/.local/lib/python3.8/site- 
packages/chromedriver_py/chromedriver_linux64')

I get this error now:

Traceback (most recent call last):
 on.py", line 31, in <module>
driver = webdriver.Chrome(executable_path='/home/greenpine/.local/lib/python3.8/site-packages/chromedriver_py/chromedriver_linux64')File "/home/greenpine/.local/lib/python3.8/site-packages/selenium/webdriver/chrome/webdriver.py", line 76, in 
 __init__ RemoteWebDriver.__init__(
 File "/home/greenpine/.local/lib/python3.8/site-packages/selenium/webdriver/remote/webdriver.py", line 157, 
 in __init__ self.start_session(capabilities, browser_profile)
 File "/home/greenpine/.local/lib/python3.8/site-packages/selenium/webdriver/remote/webdriver.py", line 252,  in start_session response = self.execute(Command.NEW_SESSION, parameters)
File "/home/greenpine/.local/lib/python3.8/site-packages/selenium/webdriver/remote/webdriver.py", line 321, in  execute self.error_handler.check_response(response)
 File "/home/greenpine/.local/lib/python3.8/site-packages/selenium/webdriver/remote/errorhandler.py", line  242, in check_response raise exception_class(message, screen, stacktrace)
selenium.common.exceptions.WebDriverException: Message: unknown error: Chrome failed to start: exited  abnormally. (unknown error: DevToolsActivePort file doesn't exist)
(The process started from chrome location /usr/bin/chromium-browser is no longer running, so ChromeDriver is assuming that Chrome has crashed.)

I run these codes separately:

chromedriver --version

 Chromium 78.0.3904.70 Built on Ubuntu , running on Ubuntu 16.04

and:

 chromium-browser --version
 Chromium 78.0.3904.70 Built on Ubuntu , running on Ubuntu 16.04

I also tried this code to make sure Selenium is updated:

 pip3.8 install --user --upgrade selenium

I also run the following and get correct google title:

chrome_options = webdriver.ChromeOptions()
chrome_options.add_argument("--headless")
chrome_options.add_argument("--disable-gpu")
browser = webdriver.Chrome(options=chrome_options)

try:
    browser.get("https://www.google.com")
    print("Page title was '{}'".format(browser.title))

finally:
    browser.quit()

Do I have to add a code to use pythonanywhere inbuilt chrome browser? if so, I appreciate the code.

Thank you for your help

Just use the example code as your starting point. That uses the chrome and chromedriver that we already have installed. ChromeDriverManager().install() caused your original problem, setting the executable_path is causing the current one.

OK, problem solved, I deleted the driver that has been installed into the "site-packages/chromedriver_py", by some code that I had entered, I don't know what, after removing it, I run my code, and it worked. I have now these codes at top of my code, I am not sure if all of them are needed or not,

import selenium
from selenium import webdriver
from selenium.webdriver.common.keys import Keys
from selenium.webdriver import Chrome
from selenium.webdriver.chrome.options import Options
from selenium.webdriver.chrome.service import Service
from chromedriver_py import binary_path
from webdriver_manager.utils import ChromeType
from webdriver_manager.chrome import ChromeDriverManager
chrome_options = webdriver.ChromeOptions()
chrome_options.add_argument("--headless")
chrome_options.add_argument("--disable-gpu")
browser = webdriver.Chrome(options=chrome_options)

The last 4 lines are definitely necessary. As for the imports, you only need the ones that you actually use.

thanks, glenn for the response, something strange I experienced. I scheduled to run a python file I have, but nothing happened, in task log it shows:

 2021-03-02 08:35:30 -- Completed task, took 3.74 seconds, return code was 1.

but the code actually did not run, I run the code myself manually to see if it works or not, and it did,

The time that the code takes to run is more than 4 sec. though,

I made another empty python file and copy and paste my code in it, saved it, and tried a new task, the same thing.

any ideas about what is wrong? thanks

Do you have any prints in your script? The log will show only things that are being actually logged. Also -- did you get any errors in the logs?

thanks for the reply; No, I don't use any print in my script, There were some error regarding:

chromedriver_py import binary_path
webdriver_manager.utils import ChromeType
from webdriver_manager.utils import ChromeType

I removed them, from my script, now there is this error:

File "/home/greenpine/pin-repin-json/pin-repin-2.py", line 10, in <module>
from webdriver_manager.chrome import ChromeDriverManager
ImportError: No module named webdriver_manager.chrome

I have installed it by:

pip3.8 install --user webdriver_manager

it seems that is installed already.

Thanks

You don't need the webdriver_manager - it is for ensuring that chrome and chromedriver are installed and they already are. Just don't use it.

thanks for the reply, I removed it, now I get another error:

 Traceback (most recent call last):
 File "/home/greenpine/pin-repin-json/pin-repin-2.py", line 16, in <module>
browser = webdriver.Chrome(options=chrome_options)
TypeError: __init__() got an unexpected keyword argument 'options'

2021-03-02 12:37:18 -- Completed task, took 2.09 seconds, return code was 1.

I have these on top of my code:

import selenium
from selenium import webdriver
from selenium.webdriver.common.keys import Keys
from selenium.webdriver.chrome.options import Options
chrome_options = webdriver.ChromeOptions()
chrome_options.add_argument("--headless")
chrome_options.add_argument("--disable-gpu")
browser = webdriver.Chrome(options=chrome_options)

thanks

Make sure you've installed the most recent selenium, as Giles suggested in the second post in this thread.

Thanks for the reply, Yes I did run the upgrade for selenium as well.

I did run the following, and webdriver has been updated from 3.2.2 to 3.3.0, after running these updates, I run the file manually, and it did work, but with the schedule does not run it, and get the same error.

pip3.8 install --user --upgrade selenium

pip3.8 install --user --upgrade webdriver_manager

 pip3.8 install --user --upgrade selenium
Looking in links: /usr/share/pip-wheels
Requirement already up-to-date: selenium in ./.local/lib/python3.8/site-packages (3.141.0)
Requirement already satisfied, skipping upgrade: urllib3 in /usr/lib/python3.8/site-packages (from selenium) (1.24.3)

thanks

Hi! Could you please enable headless Chrome for my account too. Thanks!

@galamundi It's done for you

@fjl Thanks! Tested via pythonanywhere generated Run console (browser) and worked. However, it failed to work when running via ssh/bash. Selenium was upgraded with correct python version (3.8). Not sure what I am doing different in bash vs. when it runs via browser in pythonanywhere. I used "python3.8 connect.py" from bash/ssh. The error:

Traceback (most recent call last):
  File "connect.py", line 10, in <module>
    browser = webdriver.Chrome(options=chrome_options)
...
selenium.common.exceptions.WebDriverException: Message: unknown error: Chrome failed to start: exited abnormally
  (unknown error: DevToolsActivePort file doesn't exist)
  (The process started from chrome location /usr/bin/chromium-browser is no longer running, so ChromeDriver is assuming that Chrome has crashed.)
  (Driver info: chromedriver=2.42.591071 (0b695ff80972cc1a65a5cd643186d2ae582cd4ac),platform=Linux 5.4.0-1020-aws x86_64)

Unfortunately the new virtualization system that makes headless Chrome work only works in consoles, websites, scheduled and always-on tasks -- it doesn't work in SSH or in Jupyter Notebooks yet :-(

@giles Thanks. I actually was testing via ssh thinking it would be a good test to see if it works via scheduled task but you just confirmed that it will work via scheduled task. Much appreciated!

No problem!

I am still waiting to get this problem sorted out, I have posted above that it does work using a schedule.

any reason why the python file runs OK manually but not when scheduled? as you suggested, I have upgraded both selenium and webdriver_manager, and it does not seem to have a missing update of selenium or webdriver.

thanks

Hi @greenpine, sorry for that. You don't have any tasks scheduled right now, but can you take a look at the logs for previous runs and check for errors?

Thanks for the reply, here is the older logs for 4 previous run: same error as I mentioned above, thanks;

====>>

 Traceback (most recent call last):
 File "/home/greenpine/pin-repin-json/pin-repin-2.py", line 16, in <module>
 browser = webdriver.Chrome(options=chrome_options)
 TypeError: __init__() got an unexpected keyword argument 'options'

 2021-03-03 19:12:14 -- Completed task, took 1.79 seconds, return code was 1.

====>>

 Traceback (most recent call last):
 File "/home/greenpine/pin-repin-json/pin-repin-2.py", line 16, in <module>
 browser = webdriver.Chrome(options=chrome_options)
TypeError: __init__() got an unexpected keyword argument 'options'

2021-03-03 12:31:31 -- Completed task, took 4.07 seconds, return code was 1.

====>>

 Traceback (most recent call last):
 File "/home/greenpine/pin-repin-json/pin-repin-2.py", line 16, in <module>
 browser = webdriver.Chrome(options=chrome_options)
TypeError: __init__() got an unexpected keyword argument 'options'

2021-03-02 19:51:12 -- Completed task, took 2.55 seconds, return code was 1.

====>>

Traceback (most recent call last):
File "/home/greenpine/pin-repin-json/pin-repin-2.py", line 16, in <module>
browser = webdriver.Chrome(options=chrome_options)
TypeError: __init__() got an unexpected keyword argument 'options'

2021-03-02 19:48:10 -- Completed task, took 2.55 seconds, return code was 1.

Do you run your code with the python version that you installed selenium for?

What command do you use for your task?

See Scheduled tasks | PythonAnywhere help

Thanks for the reply,

ya, that was the main problem with it, thanks for the hint, I searched and in the help area, found the solution here, I put the following code on top of the code:

1

#!/usr/bin/python3.8

I did not get the same error anymore, but now I get another error,

File "/usr/lib/python3.8/json/decoder.py", line 355, in raw_decode
 raise JSONDecodeError("Expecting value", s, err.value) from None
json.decoder.JSONDecodeError: Expecting value: line 3 column 14 (char 20)

I have some code that calls json file:

# define function
def get_json_metadata(json_fpath):
"""Read json file
  Args:
    json_fpath -- string (filepath)
  Returns:
    json_list -- list"""
  with open(json_fpath, mode='r') as json_file:
    json_str = json_file.read()
    json_list = json.loads(json_str)
  return json_list

dir_json = '/home/greenpine/pin-repin-json/csv-test-blue/'
fname_json = 'csvRow-Pin-Blue.json'

json_fpath = os.path.join(dir_json, fname_json)
path_id_map = get_json_metadata(json_fpath)

and this is the content of JSON file:

 [
    {
      "path": "/home/greenpine/pin-repin-json/csv-test-blue/1-rp.txt",
      "id": "611645261832044560",
      "nrow": 11
   }
]

Any solution for this error?

thanks

Are you sure that's the exact contents of the JSON file? It parsed properly when I tried just now, but the error you're getting is saying that there's an issue at line 3 (the one that defines the "path" value).

thanks for the reply, problem solved, it was a typo on the file path,

Hello! May I have headless chrome enabled for my account as well?

Hello, Could you please enable headless chrome for me too, I am a free user, Thanks

@a89210896052 It's done for you

Hello there, could you please enable headless chrome for me too? Thanks.

@HackmanGandalf -- we're currently in the process of enabling the new virtualization system for all accounts on PythonAnywhere, which unfortunately may take a week or two to complete. We'll let you know when that has been done for your account; I've made a note to make sure that you're in one of the next batches to be moved over to it.

@pafk Noted. Thank you so much.

Hi, could you please also enable it for my account also? Thanks. I really need it urgently.

Also using this below code gives an error

from selenium import webdriver

chrome_options = webdriver.ChromeOptions() chrome_options.add_argument("--headless") chrome_options.add_argument("--disable-gpu") browser = webdriver.Chrome(options=chrome_options)

try: browser.get("https://www.google.com") print(f'Page title was {browser.title}')

finally: browser.quit()

@medalion -- we will continue our convo with you over here.

we're currently in the process of enabling the new virtualization system for all accounts on PythonAnywhere, which unfortunately may take a week or two to complete. We'll let you know when that has been done for your account; I've made a note to make sure that you're in one of the next batches to be moved over to it.

@HackmanGandalf you have just been moved over. You will need to pip upgrade selenium to run the latest headless chrome.

Hey, it would be great if you add my account to the list too, I need the headless chrome enabled, thanks!

hi, you are already on the newest system image and the new system. You will need to pip upgrade selenium yourself though.

Oh awesome, thanks!

Oh awesome, thanks!

Hi, I am trying to use selenium on chrome browser. I have updated it as I saw in the tutorial but based on what I am reading I still need you to activate it for me?

I am a complete noob to python and selenium, do you recommend any tutorials to get me going with automating with selenium.

Thanks !

It is not necessary for us to enable chrome for you account anymore. See the example code that giles posted above for a basic start.

Thanks for your time, much appreciated.

hii , I am also getting this error, selenium.common.exceptions.WebDriverException: Message: 'chromedriver' executable needs to be in PATH. Please see https://sites.google.com/a/chromium.org/chromedriver/home

please help me

Did you upgrade Selenium? To do it for example, if you're using Python 3.7, run this in Bash:

pip3.7 install --user --upgrade selenium

Are you able to run the example code from Giles' post above?

Can I know how to get rid of this error

Traceback (most recent call last):
  File "track.py", line 8, in <module>
    browser = webdriver.Chrome(options = coptions)
  File "/home/darshanchaithram/.local/lib/python3.7/site-packages/selenium/webdriver/chrome/webdriver.py", line 81, in __init__
desired_capabilities=desired_capabilities)
  File "/home/darshanchaithram/.local/lib/python3.7/site-packages/selenium/webdriver/remote/webdriver.py", line 157, in __init__
self.start_session(capabilities, browser_profile)
  File "/home/darshanchaithram/.local/lib/python3.7/site-packages/selenium/webdriver/remote/webdriver.py", line 252, in start_session
response = self.execute(Command.NEW_SESSION, parameters)
  File "/home/darshanchaithram/.local/lib/python3.7/site-packages/selenium/webdriver/remote/webdriver.py", line 321, in execute
self.error_handler.check_response(response)
  File "/home/darshanchaithram/.local/lib/python3.7/site-packages/selenium/webdriver/remote/errorhandler.py", line 242, in check_response
raise exception_class(message, screen, stacktrace)
selenium.common.exceptions.WebDriverException: Message: unknown error: Chrome failed to start: crashed
  (unknown error: DevToolsActivePort file doesn't exist)
  (The process started from chrome location /usr/bin/chromium-browser is no longer running, so ChromeDriver is assuming that Chrome has crashed.)
  (Driver info: chromedriver=2.42.591071 (0b695ff80972cc1a65a5cd643186d2ae582cd4ac),platform=Linux 5.4.0-1038-aws x86_64)

Here is that part of my code:

coptions = webdriver.ChromeOptions() 
coptions.add_argument('--no-sandbox')
coptions.add_argument('headless')
coptions.add_argument('--disable-dev-shm-usage')
browser = webdriver.Chrome(options = coptions)

I'm using chromedriver for linux (version 92.0.4515.43)

See our example here on how to ensure that you're quitting the browser when you're done with it so you don't end up accumulating processes: https://help.pythonanywhere.com/pages/selenium/

Hello, I want to run scheduled headless selenium script but I am getting this error:

Traceback (most recent call last): File "/home/rtw/scrapers/astore_analytics.py", line 54, in <module> driver = webdriver.Chrome(options=chrome_options) File "/home/rtw/.local/lib/python3.9/site-packages/selenium/webdriver/chrome/webdriver.py", line 70, in __init__ super(WebDriver, self).__init__(DesiredCapabilities.CHROME['browserName'], "goog", File "/home/rtw/.local/lib/python3.9/site-packages/selenium/webdriver/chromium/webdriver.py", line 93, in __init__ RemoteWebDriver.__init__( File "/home/rtw/.local/lib/python3.9/site-packages/selenium/webdriver/remote/webdriver.py", line 268, in __init__ self.start_session(capabilities, browser_profile) File "/home/rtw/.local/lib/python3.9/site-packages/selenium/webdriver/remote/webdriver.py", line 359, in start_session response = self.execute(Command.NEW_SESSION, parameters) File "/home/rtw/.local/lib/python3.9/site-packages/selenium/webdriver/remote/webdriver.py", line 424, in execute self.error_handler.check_response(response) File "/home/rtw/.local/lib/python3.9/site-packages/selenium/webdriver/remote/errorhandler.py", line 247, in check_response raise exception_class(message, screen, stacktrace) selenium.common.exceptions.WebDriverException: Message: unknown error: Chrome failed to start: crashed. (chrome not reachable) (The process started from chrome location /usr/bin/chromium is no longer running, so ChromeDriver is assuming that Chrome has crashed.) Stacktrace: 0 0x56061db67e89 <unknown>

The system image I am using is "glastonbury". Please help me!

Are you able to run basic example code from https://help.pythonanywhere.com/pages/selenium ?

Can I use the undetected version of selenium? A quick response will highly be appreciated.

What do you mean by "undetected version of selenium"? If I understand correctly, it's detectability is not related to selenium version, but to how you craft your script, see: https://stackoverflow.com/a/56529616/9536161

Hello, Could you please enable headless chrome for my account as well. Thank you

It is already enabled for all accounts.

Hi, My selenium stopped working and throws the error below, could you please check what's gone wrong.

selenium.common.exceptions.WebDriverException: Message: unknown error: Chrome failed to start: crashed. (unknown error: DevToolsActivePort file doesn't exist) (The process started from chrome location /usr/bin/chromium is no longer running, so ChromeDriver is assuming that Chrome has crashed.)

I have run pip install --upgrade selenium but nothing changed.

PS: this has been working well for months now until yesterday.

It looks like you may be starting to many chrome processes, make sure you're cleaning up in your code, see: Cleaning up manually section.

Adding the chrome_options.add_argument("--no-sandbox") fixed it the problem . Thank you

Secondly, do you have support for Firefox browsers now, I'll like to set it up as a fall back in my app

Unfortunately not; while it does sort of work on our system, it crashes quite a lot so it's too unreliable to use in practice.

Hi team, I'm using headless chrome=90.0.4430.212 in selenium and session is crashing constantly. Any suggestion on what might be going on?

to be more precise I get

selenium.common.exceptions.WebDriverException: Message: unknown error: session deleted because of page crash

It seems to be solved now with

chrome_options.add_argument('--disable-dev-shm-usage')

Can someone please try to explain why this is the case? On local it was working fine without this option and seems like a drawback of this option is that it decreases speed

It's required by how our virtualization is implemented.

I am getting this error

ImportError: cannot import name 'TimeoutException' from 'selenium.common' (/usr/local/lib/python3.8/dist-packages/selenium/common/init.py)

2023-06-26 21:55:25 -- Completed task, took 16.18 seconds, return code was 1.

@SamuelN1998 Are you able to run basic example code from https://help.pythonanywhere.com/pages/selenium ? Are you following recommendations from that help page?

please switch me, i need it

@timurkaLOT https://www.pythonanywhere.com/forums/topic/27202/#id_post_102044

I have the same problem. Please tell me what to do?

Sorry there are so many problems in this thread, what are you have trouble with?

Traceback (most recent call last): File "/home/Vladislav1989/bot1/My_projekt_mzda.py", line 449, in <module> bot.polling(none_stop=True, interval=0) File "/home/Vladislav1989/.local/lib/python3.10/site-packages/telebot/init.py", line 1043, in polling self.threaded_polling(non_stop=non_stop, interval=interval, timeout=timeout, long_polling_timeout=long_polling_timeout, File "/home/Vladislav1989/.local/lib/python3.10/site-packages/telebot/__init.py", line 1118, in threaded_polling raise e File "/home/Vladislav1989/.local/lib/python3.10/site-packages/telebot/__init.py", line 1074, in threaded_polling self.worker_pool.raise_exceptions() File "/home/Vladislav1989/.local/lib/python3.10/site-packages/telebot/util.py", line 147, in raise_exceptions raise self.exception_info File "/home/Vladislav1989/.local/lib/python3.10/site-packages/telebot/util.py", line 90, in run task(args, *kwargs) File "/home/Vladislav1989/.local/lib/python3.10/site-packages/telebot/__init.py", line 6770, in _run_middlewares_and_handler result = handler'function' File "/home/Vladislav1989/bot1/My_projekt_mzda.py", line 237, in callback Final_func.result(call) File "/home/Vladislav1989/bot1/Final_func.py", line 159, in result fr_result_1.insert(1, ((config.parsing(str(fr_result[1])))) + int(fr_food_stamps[1])) File "/home/Vladislav1989/bot1/config.py", line 128, in parsing driver = webdriver.Chrome(options=option) File "/usr/local/lib/python3.10/site-packages/selenium/webdriver/chrome/webdriver.py", line 70, in init super(WebDriver, self).init(DesiredCapabilities.CHROME['browserName'], "goog", File "/usr/local/lib/python3.10/site-packages/selenium/webdriver/chromium/webdriver.py", line 92, in init RemoteWebDriver.init( File "/usr/local/lib/python3.10/site-packages/selenium/webdriver/remote/webdriver.py", line 275, in init self.start_session(capabilities, browser_profile) File "/usr/local/lib/python3.10/site-packages/selenium/webdriver/remote/webdriver.py", line 365, in start_session response = self.execute(Command.NEW_SESSION, parameters) File "/usr/local/lib/python3.10/site-packages/selenium/webdriver/remote/webdriver.py", line 430, in execute self.error_handler.check_response(response) File "/usr/local/lib/python3.10/site-packages/selenium/webdriver/remote/errorhandler.py", line 247, in check_response raise exception_class(message, screen, stacktrace) selenium.common.exceptions.WebDriverException: Message: unknown error: Chrome failed to start: exited abnormally. (unknown error: DevToolsActivePort file doesn't exist) (The process started from chrome location /usr/bin/chromium is no longer running, so ChromeDriver is assuming that Chrome has crashed.) Stacktrace:

0 0x56528eabce89 <unknown>

I installed telegram bot with selenium. when it comes to the selenium function, it throws an error. what needs to be done for correct work?

You need to provide specific arguments to selenium for it to work on PythonAnywhere: https://help.pythonanywhere.com/pages/selenium/