Home>

python. Get the value of the Phone field

Taking a questionnaire https://vedushi.ru/vedushi/tretyakov-roman/The task is to get a phone number.

  1. Tried selenium + python

    def get_phone():
     link= 'https://vedushi.ru/vedushi/tretyakov-roman/'
     chrome_options= Options()
     chrome_options.headless= False
     driver= webdriver.Chrome(chrome_options=chrome_options)
     driver.get(link)
     driver.execute_script("window.scrollBy(0,500)", "")
     button= driver.find_element(By.TAG_NAME, 'div.big.dark.phone')
     button.click()
     time.sleep(1)
     phone= driver.find_element(By.CSS_SELECTOR, 'div.big.dark.phone').text
     print(phone)
     time.sleep(5)
     driver.quit()

Result: after button click, actions like driver.page_source > save the page code to a file, shows that the page structure, where the block with the phone

<div class="big dark phone">    <a>+7 926 ***-**-**
         <span class="show_phone">Show phone</span>    </a></div>

remained unchanged, but when viewed through an open autobrowser > inspect > ... a couple more A tags appear in the code, which cannot be found .....

  1. I tried to make a requestom, but even then I got into trouble, due to inexperience)

    headers= {'user-agent': FAKE_USER.random}
 payload= {'artist_id': 1}
 r= requests.post("https://vedushi.ru/inc/ajax/contacts_click.php", headers=headers, params=payload)
 print(r.json())

I beg you to help and at least explain minimally where I'm messing up and how to do it right! To all who responded, many thanks in advance!

python requests beautiful-soup
  • Answer # 1

    Point 2 -almost correctly done. Only payload had to be passed as json.

    import requests
payload= {"artist_id": 1}
r= requests.post("https://vedushi.ru/inc/ajax/contacts_click.php", json=payload)
print(r.json())

    Result:

    {'result': 'success',
 'phone': '<a href="tel:+79266662244">+7 926 666-22-44</a>'}
  • Answer # 2

    Point 2 -almost correctly done. Only payload had to be passed as json.

    import requests
payload= {"artist_id": 1}
r= requests.post("https://vedushi.ru/inc/ajax/contacts_click.php", json=payload)
print(r.json())

    Result:

    {'result': 'success',
 'phone': '<a href="tel:+79266662244">+7 926 666-22-44</a>'}
  • Answer # 3

    Your mistake with requests is that you need to explicitly state that you are passing json in the post request. Then requests will change the Content-Type header to application/json.

    import requests
payload= {'artist_id': 1}
r= requests.post("https://vedushi.ru/inc/ajax/contacts_click.php", json=payload)
print(r.json())

    The response also arrives in json:

    {'result': 'success', 'phone': '<a href="tel:+79266662244">+7 926 666-22-44</a>'}

    If you need to get the artist id, it is stored here:

    <input name="artist_id" value="2119" type="hidden">
  • Answer # 4

    Your mistake with requests is that you need to explicitly state that you are passing json in the post request. Then requests will change the Content-Type header to application/json.

    import requests
payload= {'artist_id': 1}
r= requests.post("https://vedushi.ru/inc/ajax/contacts_click.php", json=payload)
print(r.json())

    The response also arrives in json:

    {'result': 'success', 'phone': '<a href="tel:+79266662244">+7 926 666-22-44</a>'}

    If you need to get the artist id, it is stored here:

    <input name="artist_id" value="2119" type="hidden">
  • Answer # 5

    Try this. Gives only a number.

    url= "https://vedushi.ru/vedushi/tretyakov-roman/"
r= requests.get(url=url, headers=headers, timeout=10))
soup= BeautifulSoup(r.text, "html.parser")
phone= soup.find('div', class_="big dark phone")
phone.span.decompose()
current_phone= phone.get_text(strip=True)
print(current_phone)
  • Answer # 6

    Try this. Gives only a number.

    url= "https://vedushi.ru/vedushi/tretyakov-roman/"
r= requests.get(url=url, headers=headers, timeout=10))
soup= BeautifulSoup(r.text, "html.parser")
phone= soup.find('div', class_="big dark phone")
phone.span.decompose()
current_phone= phone.get_text(strip=True)
print(current_phone)