Yahoo! Auctions using Python! I want to get the shipping cost.
Since I have just started learning programming recently, please forgive me if there is something wrong with the words.
This is a part of HTML including the shipping cost of the target page.
I wanted to scrape the shipping cost using the CSS selector, but it's missing.Error message
Applicable source code
[Execution result] IndexError: list index out of range
import requests import lxml.html response = requests.get (URL) HTML = lxml.html.fromstring (response.text) SOURYOU = HTML.cssselect ('# method0>div')  .text.strip () print (SOURYOU)
Since the prompt decision price "4,350 yen" in the same page was stored in almost the same structure, the following code was executed as a test, and "4,350 yen" was obtained.
[HTML including prompt decision price]
- Prompt decision price
4,350 yen "
(0 yen tax)<\ span>
import requests import lxml.html response = requests.get (URL) HTML = lxml.html.fromstring (response.text) KAKAKU = HTML.cssselect ('# l-sub>div.ProductInformation>ul>li.ProductInformation__item.js-stickyNavigation-start>div>dl>dd.Price__value')  .text.strip () print (KAKAKU)
Since the structure is almost the same for the shipping cost and the prompt decision price, it was impossible to understand why the prompt decision price of "4,350 yen" could be obtained and only the shipping cost of "2,980 yen" could not be obtained.
Thanks for your support.Supplemental information (FW/tool version etc.)
Microsoft Windows 10
Please provide more information here.
Answer # 1
Because it is the text of the div element
import lxml.html HTML = lxml.html.fromstring ("" " "1,280 yen" (tax included) "" ") y = HTML.cssselect ('# method0>div')  .text.strip ()
>>>import lxml.html >>>HTML = lxml.html.fromstring ("" " ... ... ... ... "1,280 yen" ...(tax included) ... ... "" ") >>>y = HTML.cssselect ('# method0>div')  .text.strip () >>>print (y) "1,280 yen" >>>print (y [1: -1]) ¥ 1,280
Answer # 2
Click on the "Details" section on the page → The shipping cost will be displayed
It seems to be possible to get the HTML that appeared after running Googlescript by running Google Chrome in Python using a library called Selenium.
- python - elements cannot be acquired after screen transition in selenium
- python - regular expression: sub () cannot replace the matched part
- i can't get child elements by scraping python
- python: i want to check the match of the elements of the data frame
- python - regression analysis with statsmodels cannot resolve the error that it contains infinity or nan
- python - ngrok cannot be executed
- python - cannot convert csv file to excel file (contents are not copied )
- python - can the characteristics of pivot elements be `7n/10`?
- python - icrawler cannot save images
- python - cuda cannot be downgraded by rebuilding anaconda's gpu environment
- python - i want to get multiple elements with find_elements and iterate with a for statement
- python 3x - pillow cannot be installed on macos
- python - cannot read csv file with google colaboratory
- python - image cannot be read by opencv
- python : Django How to display data from db into multiple pages?