Nice to meet you
Using python requests, selenium and BeautifulSoup
I tried to get the name
Could not get any element.
How can I specify the css selector to receive the element properly?
This is the result image of the terminal.Applicable source code
What I tried
from bs4 import BeautifulSoup import requests from selenium import webdriver import time URL = "https://disclosure.edinet-fsa.go.jp/E01EW/BLMainController.jsp?uji.verb=W00Z1010initialize&uji.bean=ek.bean.EKW00Z1010Bean&PID=W1E63031&TID=W00Z1010&SESSIONKEY=1592144727926&stype=0&dchSelect&350h =&syoruiKanriNo=S100GG5L&keyword1=7203&keyword2=&keyword3=&keyword4=&keyword5=&lgKbn=2&pkbn=0&skbn=1&dskb=&askb=&dflg=0&iflg=0&preId=1&chr=7203&hbn=true&spf5=2&otcd=350&keys =1&iec=&icc=&inm=&spf3=1&fdc=&fnm=&spf4=1&cal=1&era=H&yer=&mon=&psr=1&pid=4&row=100&idx=0&str=&kbn=1&flg=" page = requests.get(URL) driver = webdriver.Chrome() driver.get(URL) time.sleep(3) soup = BeautifulSoup(driver.page_source, "html.parser") el = soup.select("div#diffBlock00_4") print(el) time.sleep(5)
At first, I got the site with requests and analyzed it with Beatiful Soup.
Not compatible browser
Since it was displayed, I started a browser with selenium and tried analysis.
I tried various css selectors by referring to this site, but the result was the same.
Supplemental information (FW/tool version, etc.)
IDE is visual stdio code
The web browser is Chrome Driver and the version is properly used.
I think that html is a poor text because I just started python because I was attracted to machine learning and automation of work around progate, thank you professor.
Answer # 1
Neither requests nor bs are needed.
from selenium import webdriver from time import sleep URL = "https://di" # Abbreviation driver = webdriver.Chrome() driver.get(URL) sleep(3) driver.switch_to.frame(driver.find_element_by_name('viewFrame')) driver.switch_to.frame(driver.find_element_by_name('mainFrame')) el = driver.find_element_by_xpath('//td[contains(text(), "[name or name]")]/following-sibling::td').text print(el)
Answer # 2
I don't know which element of HTML I want to take,Addendum
Since it means that the element is specified incorrectly, I can only say that I should review the HTML carefully.
It's like a page with a frame, but you're not looking inside the frame.
- html - about width specification of each element that is responsive (fixed version 2)
- python 3x - about random number seed specification by random_state
- python - please tell me about scraping
- python - i want to get a specific element by scraping
- python 3x - questions about python3 tkinter attributes()
- python - about uploading youtube using google-auth
- python - about plotting data
- python 3x - about line break method of list
- python 3x - cannot specify element with selenium
- about python value error
- about file specification program
- python - about pandas attribute error
- about csv duplication detection (python, for statement)
- python - ask about simultaneous progress
- about deleting strings (python)
- [python] regular expression: about the output of finder()
- please tell me about sorting python alphabetically!
- python 3x - about arranging duplicate elements in multiple lists
- python - about matrix arrangement when plotting matplotlib
- python - about spline interpolation by scipy's splprep
- web operation using python selenium
- get scraping html tag with python+selenium
- python selenium instagram i would like to follow
- i want to save pdf with name with python selenium
- html - i want to use [waitnotelement] of selenium basic
- python - different login screen appears when trying to open a spreadsheet with selenium headless