Home>
I'm new to Python. I use selenium to get operation information from the Tokyo Metro homepage, but because it takes time, I wanted to use multiprocessing for parallel processing. I tried to write the code referring to various sites, but I got an error. I searched from the situation and error text on my own, but I couldn't find a solution. . . I would be happy if someone could tell me.
Codefrom bs4 import BeautifulSoup
from selenium import webdriver
from selenium.webdriver.chrome.options import Options
import time
from multiprocessing import Pool
import chromedriver_binary
def fetchclass (url):
driver.get (url)
time.sleep (3)
html = driver.page_source
soup = BeautifulSoup (html, 'lxml')
text = soup.find (class _ = 'v2_unkouReportInfo'). text
return print (text.strip ())
# Start browser
options = Options ()
options.add_argument ('-headless')
driver = webdriver.Chrome (options = options)
urls = ['https://www.tokyometro.jp/unkou/history/ginza.html','https://www.tokyometro.jp/unkou/history/marunouchi.html',
'https://www.tokyometro.jp/unkou/history/hibiya.html','https://www.tokyometro.jp/unkou/history/touzai.html',
'https://www.tokyometro.jp/unkou/history/chiyoda.html','https://www.tokyometro.jp/unkou/history/yurakucho.html',
'https://www.tokyometro.jp/unkou/history/hanzoumon.html','https://www.tokyometro.jp/unkou/history/nanboku.html',
if __name__ == "__main__":
p = Pool (4)
result = p.map (fetchclass, urls)
print (result)
# Close browser
driver.quit ()
Error text excerpt
multiprocessing.pool.RemoteTraceback:
ConnectionRefusedError: [WinError 10061] The connection could not be made because it was rejected by the target computer.
During handling of the above exception, another exception occurred:
urllib3.exceptions.NewConnectionError:<urllib3.connection.HTTPConnection object at 0x00000206A123B688>: Failed to establish a new connection: [WinError 10061] The connection could not be made because it was rejected by the target computer.
The above exception was the direct cause of the following exception:
-
Answer # 1
-
Answer # 2
ConnectionRefusedError: [WinError 10061] The connection could not be made because it was rejected by the target computer.
I can't connect, so what can I do on this side?
-
Answer # 3
If you get status here, you only need it once
There is Japanese at the bottom.
import requests import json r = requests.get (url) r.raise_for_status () # convert jsonp to json data_json = r.text.split ("(", 1) [1] .strip (")") result = json.loads (data_json) for line in result ["jp"] ["lines"]: print (f "{line ['line_name']}: {line ['contents']}")
Related articles
- python - selenium: element specification an error occurs in the code that can be executed once
- python - login automation error in selenium [error: device_event_log_implcc (211)]
- python - error when scraping with selenium and firefox
- python - i get the error shapes (1,1) and (2,) not aligned in the process of leave one out cross variation (loocv)
- python - tuple error does not resolve
- python - tkinter parallel processing
- python - an error occurs in the if statement program that compares the size of numbers
- python - sklearn, svm error
- python - when using the cv2adaptivethreshold function in the binarization process of an image, an error occurs in medianblur and
- python - error in image binarization using cv2adaptivethreshold function
- python - in raspberry pi, the error occurs only in the case of the automatic start program using systemd
- python - categorical_crossentoropy error does not resolve
- python - i want to display an image with pysimplegui, but an error occurs
- readcsv error in python
- python - idle cannot resolve the "rootgeometry" error
- indent error doesn't go away, python
- [python] graphviz output format error
- python - i get an error when connecting to a voice channel with discordpy
- python 3x - best estimator: i get an error with no syntax, so please tell me what to do
- [python] i don't know how to solve the error
Related questions
- changing the time zone of the Сhrome Selenium python web driver
- python : Error while deploying "git push heroku master"
- python selenium how to use proxy
- python : The button "next" in Instagram on SELENIUM is not pressed
- python : Selenium does not work correctly with Firefox
- Is it possible to get the values of variables of a web page in Selenium (Python)
- python : Beautifulsoup + Selenium parser does not parse elements from the entire scrolling page (infinite scroll)
- Python Selenium Firefox popup
- python : Parsing a dynamic table
- python : I don't know how to specify a filename when downloading a file from Selenium using Chrome