Home>

Hello. New to Python.
I was able to use Django to display scraped data from a dynamic website on my localhost, but there are two parts ( tag andtag part). I want to get all the data by turning the scraped data with a for statement, but it doesn't work.

The problem i am experiencing

I want to get the tag with the get_date() function in coll/views.py below, and the contents of thetag with the get() function, but only one of them works. It will feel like you haven't. Does this mean that only one function works because the two functions are trying to get scraping different elements? ?

What I want to realize is, if I can get all the tags andtags of the specified website, I want to create an empty list and store the scraped data, but what kind of code can I write? Is it good?
If possible, I want to get both tag andtag only by get_date().
Would you please teach me? ? Thank you.

Applicable source code

coll/views.py

from django.shortcuts import render
from selenium import webdriver
import chromedriver_binary
from selenium.webdriver.common.keys import Keys
from selenium.webdriver.chrome.options import Options
import time
import urllib.request
options = Options()
options.headless = True
options.add_argument('--headless')

driver = webdriver.Chrome(options=options)
driver.get("dynamic website URL")
def get_date(requests):
    selector =' tag element'
    element = driver.find_elements_by_css_selector(selector)
    element = [a.get_attribute("href") for a in element]
    context = {'links': element}
    return render(requests,'coll/index.html', context)
def get(requests):
    date_element = "element of<span>"
    date = driver.find_elements_by_css_selector(date_element)
    context = {'dates': date}
    return render(requests,'coll/index.html', context)
Applicable source code

templates/coll/index.html

<!DOCTYPE html>
<head>
  <meta charset="UTF-8">
  <title>index</title>
</head>
<body>
{% for link in links %}
  {{ link }}
{% endfor %}
{% for datess in dates %}
    {{ datess }}
{% endfor %}
</body>
What I tried

If i write only one of get_date() or get() in views.py (the contents of index.html are changed a bit like variable names), both will be displayed normally on the local host. .. It seems that you can get the desired data with each function.

Actually, I want to write the code to get the andtags only in get_date(), so I tried the following but only the data of the tag could be obtained. How can I improve this?

def get_date(requests):
    selector =' tag element'
    element = driver.find_elements_by_css_selector(selector)
    element = [a.get_attribute("href") for a in element]
    context = {'links': element}
    date_element = "element of<span>"
    date = driver.find_elements_by_css_selector(date_element)
    conte = {'datess': date}
    return render(requests,'coll/index.html', context, conte)
Supplemental information (FW/tool ​​version, etc.)

mac10.15.3
python3.8

  • Answer # 1

    def get_date(requests):
        selector =' tag element'
        element = driver.find_elements_by_css_selector(selector)
        element = [a.get_attribute("href") for a in element]
        date_element = "element of<span>"
        date = driver.find_elements_by_css_selector(date_element)
        context = {'datess': date,'links': element}
        return render(requests,'coll/index.html', context)

    like,contextPass once to