めちゃくちゃ苦労した。

動機

モナレッジから自分の記事のマークダウンをすべて取得したい

ログインできず記事のマークダウンを取得できなかった。

seleniumを使えばモナレッジにログインできるかと思い試した。

結論

Chromiumの指定Profileを使って起動することには成功した。

モナレッジにログインするにはブラウザ拡張機能mpurseをインストールしたProfileでseleniumを起動する必要がある。そのProfileを指定する必要がある。

ただし、ログインを手動でやったあと、どうやって記事番号の取得などを実行させるかはまだわからない。

インストール

sudo pip3 install selenium
sudo apt-get install chromium-chromedriver

パッケージ	役割
selenium	ブラウザを起動するやつ？
chromium-chromedriver	Profileを指定するやつ？

動作確認コード

指定したProfileでブラウザを起動する。

#!/usr/bin/env python3.7
import time
from selenium import webdriver
from selenium.webdriver.common.desired_capabilities import DesiredCapabilities
from selenium.webdriver.chrome import service as fs

url = "https://monaledge.com/mypage"
user_data_dir_path = '/home/pi/.config/chromium'
profile_dir_name = 'Profile 3'

options = webdriver.ChromeOptions()
options.add_argument(f'--user-data-dir={user_data_dir_path}')
options.add_argument(f'--profile-directory={profile_dir_name}')

chrome_service = fs.Service(executable_path='/usr/bin/chromedriver')
driver = webdriver.Chrome(options=options, service=chrome_service)
#driver.get(url)
driver.execute_script(`window.open('${url}');`)

time.sleep(30)
driver.close()

パラメータ

キー	意味	値の例
`--user-data-dir`	ユーザープロファイルの保存先を指定	`/home/pi/.config/chromium`
`--profile-directory`	使用するユーザープロファイルを指定	`Profile 3`

値は各自の環境にあわせること。

情報源

Python+Selenium+ChromeDriverでログオン認証状態を維持する方法

selenium

pythonのバージョン確認

まずはPythonの状態確認から。以前anyenv、pyenvでPythonをインストールした。

electrum-monaをラズパイ4にインストールできなかったログ

インストール済みPythonのバージョンを確認する。

$ pyenv versions
  system
* 3.10.5
  3.9.13

$ python -V
Python 3.10.5

$ pip -V
pip 22.1.2 ... /pip (python 3.10)

seleniumをインストールする

失敗1

sudo pip install selenium

Looking in indexes: https://pypi.org/simple, https://www.piwheels.org/simple
Collecting selenium
  Downloading https://files.pythonhosted.org/packages/80/d6/4294f0b4bce4de0abf13e17190289f9d0613b0a44e5dd6a7f5ca98459853/selenium-3.141.0-py2.py3-none-any.whl (904kB)
    100% |████████████████████████████████| 911kB 225kB/s 
Requirement already satisfied: urllib3 in /usr/lib/python2.7/dist-packages (from selenium) (1.24.1)
Installing collected packages: selenium
Successfully installed selenium-3.141.0

すんなり入ったっぽい。そんなバカな。Pythonやらpipはいつも何かしらトラブルるイメージが染み付いているので逆に不安になる。

よくみるとpython2.7が参照されている。なんでだよ……3.10.5参照できてたろ……。やはり失敗している。

失敗2

sudoを取っ払ってみる。さっきと違うログ。たぶんsudoがつくと参照しているpythonコマンドが変わるのだろう。意味不明だがそういうものなのだろう。でもエラーが出ている。

pip install selenium

ログ

$ pip install selenium
Looking in indexes: https://pypi.org/simple, https://www.piwheels.org/simple
Collecting selenium
  Downloading selenium-4.3.0-py3-none-any.whl (981 kB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 981.4/981.4 kB 1.4 MB/s eta 0:00:00
Collecting trio~=0.17
  Downloading https://www.piwheels.org/simple/trio/trio-0.21.0-py3-none-any.whl (358 kB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 359.0/359.0 kB 184.9 kB/s eta 0:00:00
Collecting urllib3[secure,socks]~=1.26
  Downloading https://www.piwheels.org/simple/urllib3/urllib3-1.26.10-py2.py3-none-any.whl (139 kB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 139.2/139.2 kB 224.7 kB/s eta 0:00:00
Collecting trio-websocket~=0.9
  Downloading https://www.piwheels.org/simple/trio-websocket/trio_websocket-0.9.2-py3-none-any.whl (16 kB)
Collecting outcome
  Downloading https://www.piwheels.org/simple/outcome/outcome-1.2.0-py2.py3-none-any.whl (9.7 kB)
Collecting sniffio
  Downloading https://www.piwheels.org/simple/sniffio/sniffio-1.2.0-py3-none-any.whl (10 kB)
Collecting idna
  Using cached https://www.piwheels.org/simple/idna/idna-3.3-py3-none-any.whl (64 kB)
Collecting sortedcontainers
  Downloading https://www.piwheels.org/simple/sortedcontainers/sortedcontainers-2.4.0-py2.py3-none-any.whl (29 kB)
Collecting async-generator>=1.9
  Downloading https://www.piwheels.org/simple/async-generator/async_generator-1.10-py3-none-any.whl (18 kB)
Collecting attrs>=19.2.0
  Using cached https://www.piwheels.org/simple/attrs/attrs-21.4.0-py2.py3-none-any.whl (60 kB)
Collecting wsproto>=0.14
  Downloading https://www.piwheels.org/simple/wsproto/wsproto-1.1.0-py3-none-any.whl (24 kB)
Collecting PySocks!=1.5.7,<2.0,>=1.5.6
  Downloading https://www.piwheels.org/simple/pysocks/PySocks-1.7.1-py3-none-any.whl (16 kB)
Collecting cryptography>=1.3.4
  Downloading cryptography-37.0.4.tar.gz (585 kB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 585.9/585.9 kB 197.0 kB/s eta 0:00:00
  Installing build dependencies ... error
  error: subprocess-exited-with-error
  
  × pip subprocess to install build dependencies did not run successfully.
  │ exit code: 1
  ╰─> [30 lines of output]
      Looking in indexes: https://pypi.org/simple, https://www.piwheels.org/simple, https://www.piwheels.org/simple
      Collecting setuptools!=60.9.0,>=40.6.0
        Downloading https://www.piwheels.org/simple/setuptools/setuptools-63.2.0-py3-none-any.whl (1.2 MB)
           ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 1.2/1.2 MB 105.4 kB/s eta 0:00:00
      Collecting wheel
        Downloading https://www.piwheels.org/simple/wheel/wheel-0.37.1-py2.py3-none-any.whl (35 kB)
      Collecting cffi>=1.12
        Downloading cffi-1.15.1.tar.gz (508 kB)
           ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 508.5/508.5 kB 1.5 MB/s eta 0:00:00
        Preparing metadata (setup.py): started
        Preparing metadata (setup.py): finished with status 'error'
        error: subprocess-exited-with-error
      
        × python setup.py egg_info did not run successfully.
        │ exit code: 1
        ╰─> [1 lines of output]
            ERROR: Can not execute `setup.py` since setuptools is not available in the build environment.
            [end of output]
      
        note: This error originates from a subprocess, and is likely not a problem with pip.
      error: metadata-generation-failed
      
      × Encountered error while generating package metadata.
      ╰─> See above for output.
      
      note: This is an issue with the package mentioned above, not pip.
      hint: See above for details.
      
      [notice] A new release of pip available: 22.1.2 -> 22.2
      [notice] To update, run: pip install --upgrade pip
      [end of output]
  
  note: This error originates from a subprocess, and is likely not a problem with pip.
error: subprocess-exited-with-error

× pip subprocess to install build dependencies did not run successfully.
│ exit code: 1
╰─> See above for output.

note: This error originates from a subprocess, and is likely not a problem with pip.

[notice] A new release of pip available: 22.1.2 -> 22.2
[notice] To update, run: pip install --upgrade pip

なんかエラー出たけど。

失敗3

pipをアップグレードして再試行。だが同じエラー。

pip install --upgrade pip

ログ

Looking in indexes: https://pypi.org/simple, https://www.piwheels.org/simple
Requirement already satisfied: pip in /home/pi/.anyenv/envs/pyenv/versions/3.10.5/lib/python3.10/site-packages (22.1.2)
Collecting pip
  Downloading https://www.piwheels.org/simple/pip/pip-22.2-py3-none-any.whl (2.0 MB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 2.0/2.0 MB 112.8 kB/s eta 0:00:00
Installing collected packages: pip
  Attempting uninstall: pip
    Found existing installation: pip 22.1.2
    Uninstalling pip-22.1.2:
      Successfully uninstalled pip-22.1.2
Successfully installed pip-22.2
pi@raspberrypi:/tmp/work $ pip install selenium
Looking in indexes: https://pypi.org/simple, https://www.piwheels.org/simple
Collecting selenium
  Using cached selenium-4.3.0-py3-none-any.whl (981 kB)
Collecting trio~=0.17
  Using cached https://www.piwheels.org/simple/trio/trio-0.21.0-py3-none-any.whl (358 kB)
Collecting trio-websocket~=0.9
  Using cached https://www.piwheels.org/simple/trio-websocket/trio_websocket-0.9.2-py3-none-any.whl (16 kB)
Collecting urllib3[secure,socks]~=1.26
  Using cached https://www.piwheels.org/simple/urllib3/urllib3-1.26.10-py2.py3-none-any.whl (139 kB)
Collecting async-generator>=1.9
  Using cached https://www.piwheels.org/simple/async-generator/async_generator-1.10-py3-none-any.whl (18 kB)
Collecting sortedcontainers
  Using cached https://www.piwheels.org/simple/sortedcontainers/sortedcontainers-2.4.0-py2.py3-none-any.whl (29 kB)
Collecting sniffio
  Using cached https://www.piwheels.org/simple/sniffio/sniffio-1.2.0-py3-none-any.whl (10 kB)
Collecting attrs>=19.2.0
  Using cached https://www.piwheels.org/simple/attrs/attrs-21.4.0-py2.py3-none-any.whl (60 kB)
Collecting idna
  Using cached https://www.piwheels.org/simple/idna/idna-3.3-py3-none-any.whl (64 kB)
Collecting outcome
  Using cached https://www.piwheels.org/simple/outcome/outcome-1.2.0-py2.py3-none-any.whl (9.7 kB)
Collecting wsproto>=0.14
  Using cached https://www.piwheels.org/simple/wsproto/wsproto-1.1.0-py3-none-any.whl (24 kB)
Collecting cryptography>=1.3.4
  Using cached cryptography-37.0.4.tar.gz (585 kB)
  Installing build dependencies ... error
  error: subprocess-exited-with-error
  
  × pip subprocess to install build dependencies did not run successfully.
  │ exit code: 1
  ╰─> [25 lines of output]
      Looking in indexes: https://pypi.org/simple, https://www.piwheels.org/simple, https://www.piwheels.org/simple
      Collecting setuptools!=60.9.0,>=40.6.0
        Using cached https://www.piwheels.org/simple/setuptools/setuptools-63.2.0-py3-none-any.whl (1.2 MB)
      Collecting wheel
        Using cached https://www.piwheels.org/simple/wheel/wheel-0.37.1-py2.py3-none-any.whl (35 kB)
      Collecting cffi>=1.12
        Using cached cffi-1.15.1.tar.gz (508 kB)
        Preparing metadata (setup.py): started
        Preparing metadata (setup.py): finished with status 'error'
        error: subprocess-exited-with-error
      
        × python setup.py egg_info did not run successfully.
        │ exit code: 1
        ╰─> [1 lines of output]
            ERROR: Can not execute `setup.py` since setuptools is not available in the build environment.
            [end of output]
      
        note: This error originates from a subprocess, and is likely not a problem with pip.
      error: metadata-generation-failed
      
      × Encountered error while generating package metadata.
      ╰─> See above for output.
      
      note: This is an issue with the package mentioned above, not pip.
      hint: See above for details.
      [end of output]
  
  note: This error originates from a subprocess, and is likely not a problem with pip.
error: subprocess-exited-with-error

× pip subprocess to install build dependencies did not run successfully.
│ exit code: 1
╰─> See above for output.

note: This error originates from a subprocess, and is likely not a problem with pip.

成功

今度はsudo pip3で実行してみる。たぶんこれでOSに最初からインストールされていたPython3.7.3が参照されるはず。

sudo pip3 install selenium

ログ

$ sudo pip3 install selenium
Looking in indexes: https://pypi.org/simple, https://www.piwheels.org/simple
Collecting selenium
  Downloading https://files.pythonhosted.org/packages/85/67/bea2994e161226ee09c3ee4a1a8ebdf2c033136228afdecab530910f158a/selenium-4.3.0-py3-none-any.whl (981kB)
    100% |████████████████████████████████| 983kB 282kB/s 
Collecting trio-websocket~=0.9 (from selenium)
  Downloading https://files.pythonhosted.org/packages/db/c5/b5e8bc1f40568a354f2a9cc296b8892605a9d2f22e725290fc33836dd2a3/trio_websocket-0.9.2-py3-none-any.whl
Collecting trio~=0.17 (from selenium)
  Downloading https://files.pythonhosted.org/packages/a9/bc/aef5a15725e95df49d41838dd816b95aad7df07de9f87e4ff453a3326615/trio-0.21.0-py3-none-any.whl (358kB)
    100% |████████████████████████████████| 368kB 560kB/s 
Collecting urllib3[secure,socks]~=1.26 (from selenium)
  Downloading https://files.pythonhosted.org/packages/68/47/93d3d28e97c7577f563903907912f4b3804054e4877a5ba6651f7182c53b/urllib3-1.26.10-py2.py3-none-any.whl (139kB)
    100% |████████████████████████████████| 143kB 1.1MB/s 
Collecting async-generator>=1.10 (from trio-websocket~=0.9->selenium)
  Downloading https://files.pythonhosted.org/packages/71/52/39d20e03abd0ac9159c162ec24b93fbcaa111e8400308f2465432495ca2b/async_generator-1.10-py3-none-any.whl
Collecting wsproto>=0.14 (from trio-websocket~=0.9->selenium)
  Downloading https://files.pythonhosted.org/packages/4b/6e/5f8c3e8418966f50d028e336f0c2c568f8522577183678923609d4d24924/wsproto-1.1.0-py3-none-any.whl
Collecting outcome (from trio~=0.17->selenium)
  Downloading https://files.pythonhosted.org/packages/e9/4f/2f2d3f65d851852712b4de3fd0cfdcec9c5e9a9c347430e004ba770ef4db/outcome-1.2.0-py2.py3-none-any.whl
Collecting attrs>=19.2.0 (from trio~=0.17->selenium)
  Downloading https://files.pythonhosted.org/packages/be/be/7abce643bfdf8ca01c48afa2ddf8308c2308b0c3b239a44e57d020afa0ef/attrs-21.4.0-py2.py3-none-any.whl (60kB)
    100% |████████████████████████████████| 61kB 845kB/s 
Requirement already satisfied: idna in /usr/lib/python3/dist-packages (from trio~=0.17->selenium) (2.6)
Collecting sniffio (from trio~=0.17->selenium)
  Downloading https://files.pythonhosted.org/packages/52/b0/7b2e028b63d092804b6794595871f936aafa5e9322dcaaad50ebf67445b3/sniffio-1.2.0-py3-none-any.whl
Collecting sortedcontainers (from trio~=0.17->selenium)
  Downloading https://files.pythonhosted.org/packages/32/46/9cb0e58b2deb7f82b84065f37f3bffeb12413f947f9388e4cac22c4621ce/sortedcontainers-2.4.0-py2.py3-none-any.whl
Requirement already satisfied: certifi; extra == "secure" in /usr/lib/python3/dist-packages (from urllib3[secure,socks]~=1.26->selenium) (2018.8.24)
Requirement already satisfied: cryptography>=1.3.4; extra == "secure" in /usr/lib/python3/dist-packages (from urllib3[secure,socks]~=1.26->selenium) (2.6.1)
Requirement already satisfied: pyOpenSSL>=0.14; extra == "secure" in /usr/lib/python3/dist-packages (from urllib3[secure,socks]~=1.26->selenium) (19.0.0)
Collecting PySocks!=1.5.7,<2.0,>=1.5.6; extra == "socks" (from urllib3[secure,socks]~=1.26->selenium)
  Downloading https://files.pythonhosted.org/packages/8d/59/b4572118e098ac8e46e399a1dd0f2d85403ce8bbaad9ec79373ed6badaf9/PySocks-1.7.1-py3-none-any.whl
Collecting h11<1,>=0.9.0 (from wsproto>=0.14->trio-websocket~=0.9->selenium)
  Downloading https://files.pythonhosted.org/packages/19/d2/32a15a4955be1b8114a1c570999eefd31279c7f9aa2d2a43d492a79b53c5/h11-0.13.0-py3-none-any.whl (58kB)
    100% |████████████████████████████████| 61kB 740kB/s 
Collecting typing-extensions; python_version < "3.8" (from h11<1,>=0.9.0->wsproto>=0.14->trio-websocket~=0.9->selenium)
  Downloading https://files.pythonhosted.org/packages/ed/d6/2afc375a8d55b8be879d6b4986d4f69f01115e795e36827fd3a40166028b/typing_extensions-4.3.0-py3-none-any.whl
Installing collected packages: async-generator, typing-extensions, h11, wsproto, attrs, outcome, sniffio, sortedcontainers, trio, trio-websocket, PySocks, urllib3, selenium
  Found existing installation: urllib3 1.24.1
    Not uninstalling urllib3 at /usr/lib/python3/dist-packages, outside environment /usr
    Can't uninstall 'urllib3'. No files were found to uninstall.
Successfully installed PySocks-1.7.1 async-generator-1.10 attrs-21.4.0 h11-0.13.0 outcome-1.2.0 selenium-4.3.0 sniffio-1.2.0 sortedcontainers-2.4.0 trio-0.21.0 trio-websocket-0.9.2 typing-extensions-4.3.0 urllib3-1.26.10 wsproto-1.1.0

インストールに成功したっぽい。もうわけわからん。

chromedriver-binaryパッケージをインストールできなかった

seleniumでプロファイルを使うために必要なパッケージっぽい。残念ながらここでつまづいた。

sudo pip3 install chromedriver-binary

Looking in indexes: https://pypi.org/simple, https://www.piwheels.org/simple
Collecting chromedriver-binary
  Downloading https://files.pythonhosted.org/packages/c0/8f/1ca15d38ac01148a5211162ebe056b209f9656d6ae014e0af682c6b108c8/chromedriver-binary-104.0.5112.29.0.tar.gz
Building wheels for collected packages: chromedriver-binary
  Running setup.py bdist_wheel for chromedriver-binary ... error
  Complete output from command /usr/bin/python3 -u -c "import setuptools, tokenize;__file__='/tmp/pip-install-65f0x72b/chromedriver-binary/setup.py';f=getattr(tokenize, 'open', open)(__file__);code=f.read().replace('\r\n', '\n');f.close();exec(compile(code, __file__, 'exec'))" bdist_wheel -d /tmp/pip-wheel-dvt6ei1_ --python-tag cp37:
  running bdist_wheel
  running build
  running build_py
  
  Downloading Chromedriver...
  
  Traceback (most recent call last):
    File "<string>", line 1, in <module>
    File "/tmp/pip-install-65f0x72b/chromedriver-binary/setup.py", line 84, in <module>
      cmdclass={'build_py': DownloadChromedriver}
    File "/usr/local/lib/python3.7/dist-packages/setuptools/__init__.py", line 87, in setup
      return distutils.core.setup(**attrs)
    File "/usr/local/lib/python3.7/dist-packages/setuptools/_distutils/core.py", line 177, in setup
      return run_commands(dist)
    File "/usr/local/lib/python3.7/dist-packages/setuptools/_distutils/core.py", line 193, in run_commands
      dist.run_commands()
    File "/usr/local/lib/python3.7/dist-packages/setuptools/_distutils/dist.py", line 968, in run_commands
      self.run_command(cmd)
    File "/usr/local/lib/python3.7/dist-packages/setuptools/dist.py", line 1229, in run_command
      super().run_command(command)
    File "/usr/local/lib/python3.7/dist-packages/setuptools/_distutils/dist.py", line 987, in run_command
      cmd_obj.run()
    File "/usr/lib/python3/dist-packages/wheel/bdist_wheel.py", line 188, in run
      self.run_command('build')
    File "/usr/local/lib/python3.7/dist-packages/setuptools/_distutils/cmd.py", line 317, in run_command
      self.distribution.run_command(command)
    File "/usr/local/lib/python3.7/dist-packages/setuptools/dist.py", line 1229, in run_command
      super().run_command(command)
    File "/usr/local/lib/python3.7/dist-packages/setuptools/_distutils/dist.py", line 987, in run_command
      cmd_obj.run()
    File "/usr/local/lib/python3.7/dist-packages/setuptools/command/build.py", line 24, in run
      super().run()
    File "/usr/local/lib/python3.7/dist-packages/setuptools/_distutils/command/build.py", line 131, in run
      self.run_command(cmd_name)
    File "/usr/local/lib/python3.7/dist-packages/setuptools/_distutils/cmd.py", line 317, in run_command
      self.distribution.run_command(command)
    File "/usr/local/lib/python3.7/dist-packages/setuptools/dist.py", line 1229, in run_command
      super().run_command(command)
    File "/usr/local/lib/python3.7/dist-packages/setuptools/_distutils/dist.py", line 987, in run_command
      cmd_obj.run()
    File "/tmp/pip-install-65f0x72b/chromedriver-binary/setup.py", line 42, in run
      url = get_chromedriver_url(version=chromedriver_version)
    File "/tmp/pip-install-65f0x72b/chromedriver-binary/chromedriver_binary/utils.py", line 57, in get_chromedriver_url
      raise RuntimeError('Could not determine chromedriver download URL for this platform.')
  RuntimeError: Could not determine chromedriver download URL for this platform.
  
  ----------------------------------------
  Failed building wheel for chromedriver-binary
  Running setup.py clean for chromedriver-binary
Failed to build chromedriver-binary
Installing collected packages: chromedriver-binary
  Running setup.py install for chromedriver-binary ... error
    Complete output from command /usr/bin/python3 -u -c "import setuptools, tokenize;__file__='/tmp/pip-install-65f0x72b/chromedriver-binary/setup.py';f=getattr(tokenize, 'open', open)(__file__);code=f.read().replace('\r\n', '\n');f.close();exec(compile(code, __file__, 'exec'))" install --record /tmp/pip-record-_1o3r8go/install-record.txt --single-version-externally-managed --compile:
    running install
    /usr/local/lib/python3.7/dist-packages/setuptools/command/install.py:37: SetuptoolsDeprecationWarning: setup.py install is deprecated. Use build and pip and other standards-based tools.
      setuptools.SetuptoolsDeprecationWarning,
    running build
    running build_py
    
    Downloading Chromedriver...
    
    Traceback (most recent call last):
      File "<string>", line 1, in <module>
      File "/tmp/pip-install-65f0x72b/chromedriver-binary/setup.py", line 84, in <module>
        cmdclass={'build_py': DownloadChromedriver}
      File "/usr/local/lib/python3.7/dist-packages/setuptools/__init__.py", line 87, in setup
        return distutils.core.setup(**attrs)
      File "/usr/local/lib/python3.7/dist-packages/setuptools/_distutils/core.py", line 177, in setup
        return run_commands(dist)
      File "/usr/local/lib/python3.7/dist-packages/setuptools/_distutils/core.py", line 193, in run_commands
        dist.run_commands()
      File "/usr/local/lib/python3.7/dist-packages/setuptools/_distutils/dist.py", line 968, in run_commands
        self.run_command(cmd)
      File "/usr/local/lib/python3.7/dist-packages/setuptools/dist.py", line 1229, in run_command
        super().run_command(command)
      File "/usr/local/lib/python3.7/dist-packages/setuptools/_distutils/dist.py", line 987, in run_command
        cmd_obj.run()
      File "/usr/local/lib/python3.7/dist-packages/setuptools/command/install.py", line 68, in run
        return orig.install.run(self)
      File "/usr/local/lib/python3.7/dist-packages/setuptools/_distutils/command/install.py", line 690, in run
        self.run_command('build')
      File "/usr/local/lib/python3.7/dist-packages/setuptools/_distutils/cmd.py", line 317, in run_command
        self.distribution.run_command(command)
      File "/usr/local/lib/python3.7/dist-packages/setuptools/dist.py", line 1229, in run_command
        super().run_command(command)
      File "/usr/local/lib/python3.7/dist-packages/setuptools/_distutils/dist.py", line 987, in run_command
        cmd_obj.run()
      File "/usr/local/lib/python3.7/dist-packages/setuptools/command/build.py", line 24, in run
        super().run()
      File "/usr/local/lib/python3.7/dist-packages/setuptools/_distutils/command/build.py", line 131, in run
        self.run_command(cmd_name)
      File "/usr/local/lib/python3.7/dist-packages/setuptools/_distutils/cmd.py", line 317, in run_command
        self.distribution.run_command(command)
      File "/usr/local/lib/python3.7/dist-packages/setuptools/dist.py", line 1229, in run_command
        super().run_command(command)
      File "/usr/local/lib/python3.7/dist-packages/setuptools/_distutils/dist.py", line 987, in run_command
        cmd_obj.run()
      File "/tmp/pip-install-65f0x72b/chromedriver-binary/setup.py", line 42, in run
        url = get_chromedriver_url(version=chromedriver_version)
      File "/tmp/pip-install-65f0x72b/chromedriver-binary/chromedriver_binary/utils.py", line 57, in get_chromedriver_url
        raise RuntimeError('Could not determine chromedriver download URL for this platform.')
    RuntimeError: Could not determine chromedriver download URL for this platform.
    
    ----------------------------------------
Command "/usr/bin/python3 -u -c "import setuptools, tokenize;__file__='/tmp/pip-install-65f0x72b/chromedriver-binary/setup.py';f=getattr(tokenize, 'open', open)(__file__);code=f.read().replace('\r\n', '\n');f.close();exec(compile(code, __file__, 'exec'))" install --record /tmp/pip-record-_1o3r8go/install-record.txt --single-version-externally-managed --compile" failed with error code 1 in /tmp/pip-install-65f0x72b/chromedriver-binary/

ビルドエラーになったっぽい。chromedriver_binaryが参照できない。

以下のように参照するコードを書くと、そこでエラーになる。

run.py

#!/usr/bin/env python3.7
from selenium import webdriver
import chromedriver_binary
from selenium.webdriver.common.desired_capabilities import DesiredCapabilities

python3.7 run.py

Traceback (most recent call last):
  File "/tmp/work/login_monaledge.py", line 3, in <module>
    import chromedriver_binary
ModuleNotFoundError: No module named 'chromedriver_binary'

electrum-monaをラズパイ4にインストールできなかったログのときもそうだけど、Pythonが絡むと高確率で環境構築に失敗する。勘弁してください。Pythonさん仕事して。無駄骨ばかり折らされてつらい。

ググってみる

Rasberry Pi 4でcron を用いてSeleniumを実行する

なんか知らないパッケージがあった。これも一応入れてみる。

`webdriver-manager`

sudo pip3 install webdriver-manager

  Downloading https://files.pythonhosted.org/packages/0d/92/6d990e73dbdfcdfcc06f09d52a1ec9a018f851c6c3bd1fe605c6842adf2b/webdriver_manager-3.8.2-py2.py3-none-any.whl
Collecting python-dotenv (from webdriver-manager)
  Downloading https://files.pythonhosted.org/packages/30/5f/2e5c564bd86349fe6b82ca840f46acf6f4bb76d79ba9057fce3d3e008864/python_dotenv-0.20.0-py3-none-any.whl
Requirement already satisfied: requests in /usr/lib/python3/dist-packages (from webdriver-manager) (2.21.0)
Installing collected packages: python-dotenv, webdriver-manager
Successfully installed python-dotenv-0.20.0 webdriver-manager-3.8.2

`chromedriver_binary`

URL先のパッケージ名、よくみると_だった。

pip3 install chromedriver_binary

さっき私が指定したパッケージ名は-。え、まさか。

sudo pip3 install chromedriver-binary

sudoを付与して_でやってみる。

sudo pip3 install chromedriver_binary

ダメだった。同じエラーっぽい。なんだよもう。

ログ

Looking in indexes: https://pypi.org/simple, https://www.piwheels.org/simple
Collecting chromedriver_binary
  Using cached https://files.pythonhosted.org/packages/c0/8f/1ca15d38ac01148a5211162ebe056b209f9656d6ae014e0af682c6b108c8/chromedriver-binary-104.0.5112.29.0.tar.gz
Building wheels for collected packages: chromedriver-binary
  Running setup.py bdist_wheel for chromedriver-binary ... error
  Complete output from command /usr/bin/python3 -u -c "import setuptools, tokenize;__file__='/tmp/pip-install-3afrkyr5/chromedriver-binary/setup.py';f=getattr(tokenize, 'open', open)(__file__);code=f.read().replace('\r\n', '\n');f.close();exec(compile(code, __file__, 'exec'))" bdist_wheel -d /tmp/pip-wheel-5021g_ua --python-tag cp37:
  running bdist_wheel
  running build
  running build_py
  
  Downloading Chromedriver...
  
  Traceback (most recent call last):
    File "<string>", line 1, in <module>
    File "/tmp/pip-install-3afrkyr5/chromedriver-binary/setup.py", line 84, in <module>
      cmdclass={'build_py': DownloadChromedriver}
    File "/usr/local/lib/python3.7/dist-packages/setuptools/__init__.py", line 87, in setup
      return distutils.core.setup(**attrs)
    File "/usr/local/lib/python3.7/dist-packages/setuptools/_distutils/core.py", line 177, in setup
      return run_commands(dist)
    File "/usr/local/lib/python3.7/dist-packages/setuptools/_distutils/core.py", line 193, in run_commands
      dist.run_commands()
    File "/usr/local/lib/python3.7/dist-packages/setuptools/_distutils/dist.py", line 968, in run_commands
      self.run_command(cmd)
    File "/usr/local/lib/python3.7/dist-packages/setuptools/dist.py", line 1229, in run_command
      super().run_command(command)
    File "/usr/local/lib/python3.7/dist-packages/setuptools/_distutils/dist.py", line 987, in run_command
      cmd_obj.run()
    File "/usr/lib/python3/dist-packages/wheel/bdist_wheel.py", line 188, in run
      self.run_command('build')
    File "/usr/local/lib/python3.7/dist-packages/setuptools/_distutils/cmd.py", line 317, in run_command
      self.distribution.run_command(command)
    File "/usr/local/lib/python3.7/dist-packages/setuptools/dist.py", line 1229, in run_command
      super().run_command(command)
    File "/usr/local/lib/python3.7/dist-packages/setuptools/_distutils/dist.py", line 987, in run_command
      cmd_obj.run()
    File "/usr/local/lib/python3.7/dist-packages/setuptools/command/build.py", line 24, in run
      super().run()
    File "/usr/local/lib/python3.7/dist-packages/setuptools/_distutils/command/build.py", line 131, in run
      self.run_command(cmd_name)
    File "/usr/local/lib/python3.7/dist-packages/setuptools/_distutils/cmd.py", line 317, in run_command
      self.distribution.run_command(command)
    File "/usr/local/lib/python3.7/dist-packages/setuptools/dist.py", line 1229, in run_command
      super().run_command(command)
    File "/usr/local/lib/python3.7/dist-packages/setuptools/_distutils/dist.py", line 987, in run_command
      cmd_obj.run()
    File "/tmp/pip-install-3afrkyr5/chromedriver-binary/setup.py", line 42, in run
      url = get_chromedriver_url(version=chromedriver_version)
    File "/tmp/pip-install-3afrkyr5/chromedriver-binary/chromedriver_binary/utils.py", line 57, in get_chromedriver_url
      raise RuntimeError('Could not determine chromedriver download URL for this platform.')
  RuntimeError: Could not determine chromedriver download URL for this platform.
  
  ----------------------------------------
  Failed building wheel for chromedriver-binary
  Running setup.py clean for chromedriver-binary
Failed to build chromedriver-binary
Installing collected packages: chromedriver-binary
  Running setup.py install for chromedriver-binary ... error
    Complete output from command /usr/bin/python3 -u -c "import setuptools, tokenize;__file__='/tmp/pip-install-3afrkyr5/chromedriver-binary/setup.py';f=getattr(tokenize, 'open', open)(__file__);code=f.read().replace('\r\n', '\n');f.close();exec(compile(code, __file__, 'exec'))" install --record /tmp/pip-record-qbkmdx5e/install-record.txt --single-version-externally-managed --compile:
    running install
    /usr/local/lib/python3.7/dist-packages/setuptools/command/install.py:37: SetuptoolsDeprecationWarning: setup.py install is deprecated. Use build and pip and other standards-based tools.
      setuptools.SetuptoolsDeprecationWarning,
    running build
    running build_py
    
    Downloading Chromedriver...
    
    Traceback (most recent call last):
      File "<string>", line 1, in <module>
      File "/tmp/pip-install-3afrkyr5/chromedriver-binary/setup.py", line 84, in <module>
        cmdclass={'build_py': DownloadChromedriver}
      File "/usr/local/lib/python3.7/dist-packages/setuptools/__init__.py", line 87, in setup
        return distutils.core.setup(**attrs)
      File "/usr/local/lib/python3.7/dist-packages/setuptools/_distutils/core.py", line 177, in setup
        return run_commands(dist)
      File "/usr/local/lib/python3.7/dist-packages/setuptools/_distutils/core.py", line 193, in run_commands
        dist.run_commands()
      File "/usr/local/lib/python3.7/dist-packages/setuptools/_distutils/dist.py", line 968, in run_commands
        self.run_command(cmd)
      File "/usr/local/lib/python3.7/dist-packages/setuptools/dist.py", line 1229, in run_command
        super().run_command(command)
      File "/usr/local/lib/python3.7/dist-packages/setuptools/_distutils/dist.py", line 987, in run_command
        cmd_obj.run()
      File "/usr/local/lib/python3.7/dist-packages/setuptools/command/install.py", line 68, in run
        return orig.install.run(self)
      File "/usr/local/lib/python3.7/dist-packages/setuptools/_distutils/command/install.py", line 690, in run
        self.run_command('build')
      File "/usr/local/lib/python3.7/dist-packages/setuptools/_distutils/cmd.py", line 317, in run_command
        self.distribution.run_command(command)
      File "/usr/local/lib/python3.7/dist-packages/setuptools/dist.py", line 1229, in run_command
        super().run_command(command)
      File "/usr/local/lib/python3.7/dist-packages/setuptools/_distutils/dist.py", line 987, in run_command
        cmd_obj.run()
      File "/usr/local/lib/python3.7/dist-packages/setuptools/command/build.py", line 24, in run
        super().run()
      File "/usr/local/lib/python3.7/dist-packages/setuptools/_distutils/command/build.py", line 131, in run
        self.run_command(cmd_name)
      File "/usr/local/lib/python3.7/dist-packages/setuptools/_distutils/cmd.py", line 317, in run_command
        self.distribution.run_command(command)
      File "/usr/local/lib/python3.7/dist-packages/setuptools/dist.py", line 1229, in run_command
        super().run_command(command)
      File "/usr/local/lib/python3.7/dist-packages/setuptools/_distutils/dist.py", line 987, in run_command
        cmd_obj.run()
      File "/tmp/pip-install-3afrkyr5/chromedriver-binary/setup.py", line 42, in run
        url = get_chromedriver_url(version=chromedriver_version)
      File "/tmp/pip-install-3afrkyr5/chromedriver-binary/chromedriver_binary/utils.py", line 57, in get_chromedriver_url
        raise RuntimeError('Could not determine chromedriver download URL for this platform.')
    RuntimeError: Could not determine chromedriver download URL for this platform.
    
    ----------------------------------------
Command "/usr/bin/python3 -u -c "import setuptools, tokenize;__file__='/tmp/pip-install-3afrkyr5/chromedriver-binary/setup.py';f=getattr(tokenize, 'open', open)(__file__);code=f.read().replace('\r\n', '\n');f.close();exec(compile(code, __file__, 'exec'))" install --record /tmp/pip-record-qbkmdx5e/install-record.txt --single-version-externally-managed --compile" failed with error code 1 in /tmp/pip-install-3afrkyr5/chromedriver-binary/

URL先のコードを実行してみたらエラーになった。

a.py

import time # スリープを使うために必要
from selenium import webdriver # Webブラウザを自動操作する（python -m pip install selenium) 
from selenium.webdriver.chrome.options import Options # オプションを使うために必要 
option = Options() # オプションを用意
option.add_argument('--incognito') # シークレットモードの設定を付与
# option.add_argument('--headless')  # ヘッドレスモード(画面表示させない)設定
 
# Sleniumでwebブラウザを起動する
browser = webdriver.Chrome(ChromeDriverManager().install(), options=option) # Chromeを準備(optionでシークレットモードにしている）
 
# 5秒間保持する
time.sleep(5)
 
# Sleniumでwebブラウザを停止する
browser.quit()

$ python3.7 a.py
Traceback (most recent call last):
  File "a.py", line 9, in <module>
    browser = webdriver.Chrome(ChromeDriverManager().install(), options=option) # Chromeを準備(optionでシークレットモードにしている）
NameError: name 'ChromeDriverManager' is not defined

何もかも使えない。

妥協

こうなったらプロフィールは使えなくてもいい。せめてseleniumの起動だけでも確認したい。それだとログインを残せなくて自分の記事を取得できないだろうけど、もういい。妥協するから、頼む。お願いします神様仏様。

Raspberry Pi4上でのWebスクレピング環境構築

import sys
sys.path.append('/home/pi/.local/lib/python3.5/site-packages/')
from selenium import webdriver
browser = webdriver.Chrome(executable_path="/usr/lib/chromium-browser/chromedriver")
browser.get('https://www.yahoo.co.jp')

b.py:4: DeprecationWarning: executable_path has been deprecated, please pass in a Service object
  browser = webdriver.Chrome(executable_path="/usr/lib/chromium-browser/chromedriver")
Traceback (most recent call last):
  File "/usr/local/lib/python3.7/dist-packages/selenium/webdriver/common/service.py", line 76, in start
    creationflags=self.creationflags)
  File "/usr/lib/python3.7/subprocess.py", line 775, in __init__
    restore_signals, start_new_session)
  File "/usr/lib/python3.7/subprocess.py", line 1522, in _execute_child
    raise child_exception_type(errno_num, err_msg, err_filename)
FileNotFoundError: [Errno 2] No such file or directory: '/usr/lib/chromium-browser/chromedriver': '/usr/lib/chromium-browser/chromedriver'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "b.py", line 4, in <module>
    browser = webdriver.Chrome(executable_path="/usr/lib/chromium-browser/chromedriver")
  File "/usr/local/lib/python3.7/dist-packages/selenium/webdriver/chrome/webdriver.py", line 72, in __init__
    service_log_path, service, keep_alive)
  File "/usr/local/lib/python3.7/dist-packages/selenium/webdriver/chromium/webdriver.py", line 89, in __init__
    self.service.start()
  File "/usr/local/lib/python3.7/dist-packages/selenium/webdriver/common/service.py", line 83, in start
    os.path.basename(self.path), self.start_error_message)
selenium.common.exceptions.WebDriverException: Message: 'chromedriver' executable needs to be in PATH. Please see https://chromedriver.chromium.org/home

ぐは！　またしてもエラー。

ん？　URLのインストールログ、ちょっとへんじゃないか？

sudo pip install selenium==4.0.0
sudo apt-get install chromium-chromedriver　#chromium-chromedriverのインストールを実施する。
chromedriver --version #正しくインストールできたか確認

なぜchromium-chromedriverだけapt-getで入れてるの？　pipじゃないの？

sudo apt-get install chromium-chromedriver　#chromium-chromedriverのインストールを実施する。

ググってみる。

selenium向け ChromeDriverをpipでインストールする方法（パス通し不要、バージョン指定可能）

普通にpip install chromedriver-binaryコマンドでインストールすると、ChromeとChromeDriverの互換性の問題でseleniumが使えないことが多々あります

なん、だと？

私のChromimumバージョンは以下。

バージョン: 92.0.4515.98（Official Build）Built on Raspbian , running on Raspbian 10 （32 ビット）

これに合わせてchromedriver-binaryをインストールするみたい。

sudo pip3 install chromedriver-binary==92.0.4515.98

Looking in indexes: https://pypi.org/simple, https://www.piwheels.org/simple
Collecting chromedriver-binary==92.0.4515.98
  Could not find a version that satisfies the requirement chromedriver-binary==92.0.4515.98 (from versions: 2.29.1, 2.31.1, 2.33.1, 2.34.0, 2.35.0, 2.35.1, 2.36.0, 2.37.0, 2.38.0, 2.39.0, 2.40.1, 2.41.0, 2.42.0, 2.43.0, 2.44.0, 2.45.0, 2.46.0, 70.0.3538.16.0, 70.0.3538.67.0, 70.0.3538.97.0, 71.0.3578.30.0, 71.0.3578.33.0, 71.0.3578.80.0, 71.0.3578.137.0, 72.0.3626.7.0, 72.0.3626.69.0, 73.0.3683.20.0, 73.0.3683.68.0, 74.0.3729.6.0, 75.0.3770.8.0, 75.0.3770.90.0, 75.0.3770.140.0, 76.0.3809.12.0, 76.0.3809.25.0, 76.0.3809.68.0, 76.0.3809.126.0, 77.0.3865.10.0, 77.0.3865.40.0, 78.0.3904.11.0, 78.0.3904.70.0, 78.0.3904.105.0, 79.0.3945.16.0, 79.0.3945.36.0, 80.0.3987.16.0, 80.0.3987.106.0, 81.0.4044.20.0, 81.0.4044.69.0, 81.0.4044.138.0, 83.0.4103.14.0, 83.0.4103.39.0, 84.0.4147.30.0, 85.0.4183.38.0, 85.0.4183.83.0, 85.0.4183.87.0, 86.0.4240.22.0, 87.0.4280.20.0, 87.0.4280.87.0, 87.0.4280.88.0, 88.0.4324.27.0, 88.0.4324.27.1, 88.0.4324.96.0, 89.0.4389.23.0, 90.0.4430.24.0, 91.0.4472.19.0, 91.0.4472.101.0, 92.0.4515.43.0, 92.0.4515.107.0, 93.0.4577.15.0, 93.0.4577.63.0, 94.0.4606.41.0, 94.0.4606.61.0, 94.0.4606.113.0, 95.0.4638.10.0, 95.0.4638.17.0, 95.0.4638.54.0, 95.0.4638.69.0, 96.0.4664.18.0, 96.0.4664.35.0, 96.0.4664.45.0, 97.0.4692.20.0, 97.0.4692.36.0, 97.0.4692.71.0, 98.0.4758.48.0, 98.0.4758.80.0, 98.0.4758.102.0, 99.0.4844.17.0, 99.0.4844.35.0, 99.0.4844.51.0, 100.0.4896.20.0, 100.0.4896.60.0, 101.0.4951.15.0, 101.0.4951.41.0, 102.0.5005.27.0, 102.0.5005.61.0, 103.0.5060.24.0, 103.0.5060.53.0, 103.0.5060.134.0, 104.0.5112.20.0, 104.0.5112.29.0)
No matching distribution found for chromedriver-binary==92.0.4515.98

怒られた。存在するバージョンのリストが返された。自分のバージョンより少し小さいやつを選んでもう一度やるらしい。

ええと私の環境のバージョンが92.0.4515.98だから、それより小さいやつは……

92.0.4515.43.0, 92.0.4515.107.0

はい、92.0.4515.43.0だね。それを指定して再試行。

sudo pip3 install chromedriver-binary==92.0.4515.43.0

はいまたビルドエラー。今までと同じ。

Looking in indexes: https://pypi.org/simple, https://www.piwheels.org/simple
Collecting chromedriver-binary==92.0.4515.43.0
  Downloading https://files.pythonhosted.org/packages/82/db/8c973c3189c1b7d8660352f767c3923426dac79b8fea0a7b819b8d213294/chromedriver-binary-92.0.4515.43.0.tar.gz
Building wheels for collected packages: chromedriver-binary
  Running setup.py bdist_wheel for chromedriver-binary ... error
  Complete output from command /usr/bin/python3 -u -c "import setuptools, tokenize;__file__='/tmp/pip-install-k5clgq61/chromedriver-binary/setup.py';f=getattr(tokenize, 'open', open)(__file__);code=f.read().replace('\r\n', '\n');f.close();exec(compile(code, __file__, 'exec'))" bdist_wheel -d /tmp/pip-wheel-ybed9n2q --python-tag cp37:
  running bdist_wheel
  running build
  running build_py
  
  Downloading Chromedriver...
  
  Traceback (most recent call last):
    File "<string>", line 1, in <module>
    File "/tmp/pip-install-k5clgq61/chromedriver-binary/setup.py", line 84, in <module>
      cmdclass={'build_py': DownloadChromedriver}
    File "/usr/local/lib/python3.7/dist-packages/setuptools/__init__.py", line 87, in setup
      return distutils.core.setup(**attrs)
    File "/usr/local/lib/python3.7/dist-packages/setuptools/_distutils/core.py", line 177, in setup
      return run_commands(dist)
    File "/usr/local/lib/python3.7/dist-packages/setuptools/_distutils/core.py", line 193, in run_commands
      dist.run_commands()
    File "/usr/local/lib/python3.7/dist-packages/setuptools/_distutils/dist.py", line 968, in run_commands
      self.run_command(cmd)
    File "/usr/local/lib/python3.7/dist-packages/setuptools/dist.py", line 1229, in run_command
      super().run_command(command)
    File "/usr/local/lib/python3.7/dist-packages/setuptools/_distutils/dist.py", line 987, in run_command
      cmd_obj.run()
    File "/usr/lib/python3/dist-packages/wheel/bdist_wheel.py", line 188, in run
      self.run_command('build')
    File "/usr/local/lib/python3.7/dist-packages/setuptools/_distutils/cmd.py", line 317, in run_command
      self.distribution.run_command(command)
    File "/usr/local/lib/python3.7/dist-packages/setuptools/dist.py", line 1229, in run_command
      super().run_command(command)
    File "/usr/local/lib/python3.7/dist-packages/setuptools/_distutils/dist.py", line 987, in run_command
      cmd_obj.run()
    File "/usr/local/lib/python3.7/dist-packages/setuptools/command/build.py", line 24, in run
      super().run()
    File "/usr/local/lib/python3.7/dist-packages/setuptools/_distutils/command/build.py", line 131, in run
      self.run_command(cmd_name)
    File "/usr/local/lib/python3.7/dist-packages/setuptools/_distutils/cmd.py", line 317, in run_command
      self.distribution.run_command(command)
    File "/usr/local/lib/python3.7/dist-packages/setuptools/dist.py", line 1229, in run_command
      super().run_command(command)
    File "/usr/local/lib/python3.7/dist-packages/setuptools/_distutils/dist.py", line 987, in run_command
      cmd_obj.run()
    File "/tmp/pip-install-k5clgq61/chromedriver-binary/setup.py", line 42, in run
      url = get_chromedriver_url(version=chromedriver_version)
    File "/tmp/pip-install-k5clgq61/chromedriver-binary/chromedriver_binary/utils.py", line 57, in get_chromedriver_url
      raise RuntimeError('Could not determine chromedriver download URL for this platform.')
  RuntimeError: Could not determine chromedriver download URL for this platform.
  
  ----------------------------------------
  Failed building wheel for chromedriver-binary
  Running setup.py clean for chromedriver-binary
Failed to build chromedriver-binary
Installing collected packages: chromedriver-binary
  Running setup.py install for chromedriver-binary ... error
    Complete output from command /usr/bin/python3 -u -c "import setuptools, tokenize;__file__='/tmp/pip-install-k5clgq61/chromedriver-binary/setup.py';f=getattr(tokenize, 'open', open)(__file__);code=f.read().replace('\r\n', '\n');f.close();exec(compile(code, __file__, 'exec'))" install --record /tmp/pip-record-jr05ngeq/install-record.txt --single-version-externally-managed --compile:
    running install
    /usr/local/lib/python3.7/dist-packages/setuptools/command/install.py:37: SetuptoolsDeprecationWarning: setup.py install is deprecated. Use build and pip and other standards-based tools.
      setuptools.SetuptoolsDeprecationWarning,
    running build
    running build_py
    
    Downloading Chromedriver...
    
    Traceback (most recent call last):
      File "<string>", line 1, in <module>
      File "/tmp/pip-install-k5clgq61/chromedriver-binary/setup.py", line 84, in <module>
        cmdclass={'build_py': DownloadChromedriver}
      File "/usr/local/lib/python3.7/dist-packages/setuptools/__init__.py", line 87, in setup
        return distutils.core.setup(**attrs)
      File "/usr/local/lib/python3.7/dist-packages/setuptools/_distutils/core.py", line 177, in setup
        return run_commands(dist)
      File "/usr/local/lib/python3.7/dist-packages/setuptools/_distutils/core.py", line 193, in run_commands
        dist.run_commands()
      File "/usr/local/lib/python3.7/dist-packages/setuptools/_distutils/dist.py", line 968, in run_commands
        self.run_command(cmd)
      File "/usr/local/lib/python3.7/dist-packages/setuptools/dist.py", line 1229, in run_command
        super().run_command(command)
      File "/usr/local/lib/python3.7/dist-packages/setuptools/_distutils/dist.py", line 987, in run_command
        cmd_obj.run()
      File "/usr/local/lib/python3.7/dist-packages/setuptools/command/install.py", line 68, in run
        return orig.install.run(self)
      File "/usr/local/lib/python3.7/dist-packages/setuptools/_distutils/command/install.py", line 690, in run
        self.run_command('build')
      File "/usr/local/lib/python3.7/dist-packages/setuptools/_distutils/cmd.py", line 317, in run_command
        self.distribution.run_command(command)
      File "/usr/local/lib/python3.7/dist-packages/setuptools/dist.py", line 1229, in run_command
        super().run_command(command)
      File "/usr/local/lib/python3.7/dist-packages/setuptools/_distutils/dist.py", line 987, in run_command
        cmd_obj.run()
      File "/usr/local/lib/python3.7/dist-packages/setuptools/command/build.py", line 24, in run
        super().run()
      File "/usr/local/lib/python3.7/dist-packages/setuptools/_distutils/command/build.py", line 131, in run
        self.run_command(cmd_name)
      File "/usr/local/lib/python3.7/dist-packages/setuptools/_distutils/cmd.py", line 317, in run_command
        self.distribution.run_command(command)
      File "/usr/local/lib/python3.7/dist-packages/setuptools/dist.py", line 1229, in run_command
        super().run_command(command)
      File "/usr/local/lib/python3.7/dist-packages/setuptools/_distutils/dist.py", line 987, in run_command
        cmd_obj.run()
      File "/tmp/pip-install-k5clgq61/chromedriver-binary/setup.py", line 42, in run
        url = get_chromedriver_url(version=chromedriver_version)
      File "/tmp/pip-install-k5clgq61/chromedriver-binary/chromedriver_binary/utils.py", line 57, in get_chromedriver_url
        raise RuntimeError('Could not determine chromedriver download URL for this platform.')
    RuntimeError: Could not determine chromedriver download URL for this platform.
    
    ----------------------------------------
Command "/usr/bin/python3 -u -c "import setuptools, tokenize;__file__='/tmp/pip-install-k5clgq61/chromedriver-binary/setup.py';f=getattr(tokenize, 'open', open)(__file__);code=f.read().replace('\r\n', '\n');f.close();exec(compile(code, __file__, 'exec'))" install --record /tmp/pip-record-jr05ngeq/install-record.txt --single-version-externally-managed --compile" failed with error code 1 in /tmp/pip-install-k5clgq61/chromedriver-binary/

思うんだけど、ARMv7 32bit版のバイナリが提供されていないってオチではなかろうか。でも、そのためにコードからビルドしているはず。あるいは依存関係のライブラリが64bitしか対応していないとか？

ここ最近32bit CPUに人権ないからなぁ。

さらにググる。

Raspberry PiにSeleniumとchromedriverでブラウザ操作

方法	問題
オフィシャルサイトよりダウンロード（一番推奨される方法）	ただし、Linux版は64Bitしか提供されていない。
apt sourceでのインストール	chromiumブラウザともバージョンが合い、まだ安定して動作しました。
pipでのインストール	あまり安定して動作しませんでした。

ということらしい。もうaptで入手するしかないか。

sudo apt-get install chromium-chromedriver

パッケージリストを読み込んでいます... 完了
依存関係ツリーを作成しています                
状態情報を読み取っています... 完了
以下のパッケージが自動でインストールされましたが、もう必要とされていません:
  gyp libc-ares2 libjs-inherits libjs-is-typedarray libnode-dev libnode64 libuv1 libuv1-dev nodejs-doc python-colorzero rlwrap
これを削除するには 'sudo apt autoremove' を利用してください。
以下のパッケージが新たにインストールされます:
  chromium-chromedriver
アップグレード: 0 個、新規インストール: 1 個、削除: 0 個、保留: 0 個。
4,946 kB のアーカイブを取得する必要があります。
この操作後に追加で 13.5 MB のディスク容量が消費されます。
取得:1 http://archive.raspberrypi.org/debian buster/main armhf chromium-chromedriver armhf 92.0.4515.98~buster-rpt2 [4,946 kB]
4,946 kB を 16秒 で取得しました (302 kB/s)                                                                                            
以前に未選択のパッケージ chromium-chromedriver を選択しています。
(データベースを読み込んでいます ... 現在 134257 個のファイルとディレクトリがインストールされています。)
.../chromium-chromedriver_92.0.4515.98~buster-rpt2_armhf.deb を展開する準備をしています ...
chromium-chromedriver (92.0.4515.98~buster-rpt2) を展開しています...
chromium-chromedriver (92.0.4515.98~buster-rpt2) を設定しています ...

以下コードは動作した。

b.py

import sys
sys.path.append('/home/pi/.local/lib/python3.5/site-packages/')
from selenium import webdriver
browser = webdriver.Chrome(executable_path="/usr/lib/chromium-browser/chromedriver")
browser.get('https://www.yahoo.co.jp')

が、以下コードはエラーになった。

import time # スリープを使うために必要
from selenium import webdriver # Webブラウザを自動操作する（python -m pip install selenium) 
from selenium.webdriver.chrome.options import Options # オプションを使うために必要 
option = Options() # オプションを用意
option.add_argument('--incognito') # シークレットモードの設定を付与
# option.add_argument('--headless')  # ヘッドレスモード(画面表示させない)設定
 
# Sleniumでwebブラウザを起動する
browser = webdriver.Chrome(ChromeDriverManager().install(), options=option) # Chromeを準備(optionでシークレットモードにしている）
 
# 5秒間保持する
time.sleep(5)
 
# Sleniumでwebブラウザを停止する
browser.quit()

Traceback (most recent call last):
  File "a.py", line 9, in <module>
    browser = webdriver.Chrome(ChromeDriverManager().install(), options=option) # Chromeを準備(optionでシークレットモードにしている）
NameError: name 'ChromeDriverManager' is not defined

以下コードもダメ

0.py

#!/usr/bin/env python3.7
from selenium import webdriver
import chromedriver_binary
from selenium.webdriver.common.desired_capabilities import DesiredCapabilities

url = "https://monaledge.com/mypage"
user_profile = 'Profile 3'

# Optionでユーザプロファイルの場所を指定する
options = webdriver.ChromeOptions()
#options.add_argument('--user-data-dir=' + user_profile)
options.add_argument(f'--user-data-dir="{user_profile}"')

driver = webdriver.Chrome(options=options)
driver.get(url)

time.sleep(30)

driver.close()

Traceback (most recent call last):
  File "/tmp/work/0.py", line 3, in <module>
    import chromedriver_binary
ModuleNotFoundError: No module named 'chromedriver_binary'

import chromedriver_binaryをコメントアウトすれば起動するが、Profileが使われず、mpurseなどのアドオンもない状態。なのでログインもできない。それじゃダメなんだよなぁ。

$ chromedriver --version
ChromeDriver 92.0.4515.98 (564abd8de2c05f45308eec14f9110a10aff40ad9-refs/branch-heads/4515@{#1501})

$ which chromedriver
/usr/bin/chromedriver

Pythonのコードを以下のように一部修正してみる。

driver = webdriver.Chrome(options=options, executable_path='/usr/bin/chromedriver')

実行するとなんかいっぱい出た。

0.py:17: DeprecationWarning: executable_path has been deprecated, please pass in a Service object
  driver = webdriver.Chrome(options=options, executable_path='/usr/bin/chromedriver')
Traceback (most recent call last):
  File "0.py", line 17, in <module>
    driver = webdriver.Chrome(options=options, executable_path='/usr/bin/chromedriver')
  File "/usr/local/lib/python3.7/dist-packages/selenium/webdriver/chrome/webdriver.py", line 72, in __init__
    service_log_path, service, keep_alive)
  File "/usr/local/lib/python3.7/dist-packages/selenium/webdriver/chromium/webdriver.py", line 97, in __init__
    options=options)
  File "/usr/local/lib/python3.7/dist-packages/selenium/webdriver/remote/webdriver.py", line 277, in __init__
    self.start_session(capabilities, browser_profile)
  File "/usr/local/lib/python3.7/dist-packages/selenium/webdriver/remote/webdriver.py", line 370, in start_session
    response = self.execute(Command.NEW_SESSION, parameters)
  File "/usr/local/lib/python3.7/dist-packages/selenium/webdriver/remote/webdriver.py", line 435, in execute
    self.error_handler.check_response(response)
  File "/usr/local/lib/python3.7/dist-packages/selenium/webdriver/remote/errorhandler.py", line 247, in check_response
    raise exception_class(message, screen, stacktrace)
selenium.common.exceptions.InvalidArgumentException: Message: invalid argument: user data directory is already in use, please specify a unique value for --user-data-dir argument, or don't use --user-data-dir
Stacktrace:
#0 0x0000009da618 <unknown>
#1 0x0000007409f4 <unknown>
#2 0x000000762328 <unknown>
#3 0x00000075ec04 <unknown>
#4 0x00000079528c <unknown>
#5 0x00000078fa14 <unknown>
#6 0x000000768478 <unknown>
#7 0x0000007693a4 <unknown>
#8 0x000000a07944 <unknown>
#9 0x000000a1990c <unknown>
#10 0x000000a19550 <unknown>
#11 0x000000a1ad40 <unknown>
#12 0x000000a08944 <unknown>
#13 0x000000a1afec <unknown>
#14 0x0000009ff3ac <unknown>
#15 0x000000a326e8 <unknown>
#16 0x000000a32858 <unknown>
#17 0x000000a4cf4c <unknown>
#18 0x0000b6f60494 start_thread
#19 0x000000000000 <unknown>

ググってみる。

Selenium 4で「DeprecationWarning」が出る場合の対策

from selenium import webdriver
from selenium.webdriver.common.by import By
from selenium.webdriver.common.keys import Keys
from selenium.webdriver.chrome import service as fs
 
CHROMEDRIVER = "C:\data\etc\chromedriver.exe"
 
# ドライバー指定でChromeブラウザを開く
chrome_service = fs.Service(executable_path=CHROMEDRIVER)
browser = webdriver.Chrome(service=chrome_service)
 
# Googleアクセス
browser.get('https://www.google.com/')
 
# 検索ボックスを特定
elem = browser.find_element(By.NAME, 'q')
# 「Selenium」と入力して、「Enter」を押す
elem.send_keys('Selenium' + Keys.RETURN)
 
# ブラウザを閉じる
# browser.quit()

上記をもとに以下のように変更してみる。

#!/usr/bin/env python3.7
import time
from selenium import webdriver
#import chromedriver_binary
from selenium.webdriver.common.desired_capabilities import DesiredCapabilities
from selenium.webdriver.chrome import service as fs

url = "https://monaledge.com/mypage"
user_profile = 'Profile 3'
#user_profile = 'Default'

# Optionでユーザプロファイルの場所を指定する
options = webdriver.ChromeOptions()
#options.add_argument('--user-data-dir=' + user_profile)
#options.add_argument(f'--user-data-dir="{user_profile}"')
options.add_argument(f'--user-data-dir="{user_profile}"')

chrome_service = fs.Service(executable_path='/usr/bin/chromedriver')
#driver = webdriver.Chrome(options=options)
driver = webdriver.Chrome(options=options, service=chrome_service)
driver.get(url)

time.sleep(30)

driver.close()

起動した。が、Profileが使われていない。起動したブラウザを閉じると以下のようなエラーが出た。

Traceback (most recent call last):
  File "1.py", line 20, in <module>
    driver = webdriver.Chrome(options=options, service=chrome_service)
  File "/usr/local/lib/python3.7/dist-packages/selenium/webdriver/chrome/webdriver.py", line 72, in __init__
    service_log_path, service, keep_alive)
  File "/usr/local/lib/python3.7/dist-packages/selenium/webdriver/chromium/webdriver.py", line 97, in __init__
    options=options)
  File "/usr/local/lib/python3.7/dist-packages/selenium/webdriver/remote/webdriver.py", line 277, in __init__
    self.start_session(capabilities, browser_profile)
  File "/usr/local/lib/python3.7/dist-packages/selenium/webdriver/remote/webdriver.py", line 370, in start_session
    response = self.execute(Command.NEW_SESSION, parameters)
  File "/usr/local/lib/python3.7/dist-packages/selenium/webdriver/remote/webdriver.py", line 435, in execute
    self.error_handler.check_response(response)
  File "/usr/local/lib/python3.7/dist-packages/selenium/webdriver/remote/errorhandler.py", line 247, in check_response
    raise exception_class(message, screen, stacktrace)
selenium.common.exceptions.InvalidArgumentException: Message: invalid argument: user data directory is already in use, please specify a unique value for --user-data-dir argument, or don't use --user-data-dir
Stacktrace:
#0 0x000000a2a618 <unknown>
#1 0x0000007909f4 <unknown>
#2 0x0000007b2328 <unknown>
#3 0x0000007aec04 <unknown>
#4 0x0000007e528c <unknown>
#5 0x0000007dfa14 <unknown>
#6 0x0000007b8478 <unknown>
#7 0x0000007b93a4 <unknown>
#8 0x000000a57944 <unknown>
#9 0x000000a6990c <unknown>
#10 0x000000a69550 <unknown>
#11 0x000000a6ad40 <unknown>
#12 0x000000a58944 <unknown>
#13 0x000000a6afec <unknown>
#14 0x000000a4f3ac <unknown>
#15 0x000000a826e8 <unknown>
#16 0x000000a82858 <unknown>
#17 0x000000a9cf4c <unknown>
#18 0x0000b6f50494 start_thread
#19 0x000000000000 <unknown>

Traceback (most recent call last):
  File "1.py", line 25, in <module>
    driver.close()
  File "/usr/local/lib/python3.7/dist-packages/selenium/webdriver/remote/webdriver.py", line 552, in close
    self.execute(Command.CLOSE)
  File "/usr/local/lib/python3.7/dist-packages/selenium/webdriver/remote/webdriver.py", line 435, in execute
    self.error_handler.check_response(response)
  File "/usr/local/lib/python3.7/dist-packages/selenium/webdriver/remote/errorhandler.py", line 247, in check_response
    raise exception_class(message, screen, stacktrace)
selenium.common.exceptions.WebDriverException: Message: chrome not reachable
  (Session info: chrome=92.0.4515.98)
Stacktrace:
#0 0x000000aba618 <unknown>
#1 0x000000820894 <unknown>
#2 0x000000810c00 <unknown>
#3 0x000000803300 <unknown>
#4 0x000000876eb8 <unknown>
#5 0x00000086fb24 <unknown>
#6 0x000000848478 <unknown>
#7 0x0000008493a4 <unknown>
#8 0x000000ae7944 <unknown>
#9 0x000000af990c <unknown>
#10 0x000000af9550 <unknown>
#11 0x000000afad40 <unknown>
#12 0x000000ae8944 <unknown>
#13 0x000000afafec <unknown>
#14 0x000000adf3ac <unknown>
#15 0x000000b126e8 <unknown>
#16 0x000000b12858 <unknown>
#17 0x000000b2cf4c <unknown>
#18 0x0000b6ec7494 start_thread
#19 0x000000000000 <unknown>

どうやら、すでにブラウザが立ち上がっているとこういうエラーが出るらしい。ブラウザで調べながらコードを書いているのに、閉じろというのか。仕方ない。

はい、閉じても同じ状態、同じエラーでした。残念。

Profile指定方法をググる

SeleniumでChromeのプロファイルを使う

キー	意味	値の例
`--user-data-dir`	ユーザープロファイルの保存先を指定	`/home/pi/.config/chromium`
`--profile-directory`	使用するユーザープロファイルを指定	`Profile 3`

#!/usr/bin/env python3.7
import time
from selenium import webdriver
from selenium.webdriver.common.desired_capabilities import DesiredCapabilities
from selenium.webdriver.chrome import service as fs

url = "https://monaledge.com/mypage"
user_data_dir_path = '/home/pi/.config/chromium'
profile_dir_name = 'Profile 3'

options = webdriver.ChromeOptions()
options.add_argument(f'--user-data-dir={user_data_dir_path}')
options.add_argument(f'--profile-directory={profile_dir_name}')

chrome_service = fs.Service(executable_path='/usr/bin/chromedriver')
driver = webdriver.Chrome(options=options, service=chrome_service)
#driver.get(url)
driver.execute_script(`window.open('${url}');`)

time.sleep(30)
driver.close()

これで起動した。たぶんバージョンによって引数が変わったのだろう。

mpurseをインストールしたProfileを指定して起動すればログインできるはず。確認は次回にする。

ラズパイ4にseleniumとchromedriver-binaryをインストールする

動機

結論