It is required to parse information from a site whose new posts are loaded dynamically, that is, when you scroll down the page, the posts are loaded in batches. How can you get all these records? Is there a way to do without Selenium? I implemented this project in Python, used Selenium to scroll the page, but it slows down the parser a lot.

Please clarify your specific problem or provide more detailed information on what exactly you need. As it stands, it is difficult to know exactly what you are asking.

Дух сообщества2021-11-25 12:45:53

Open the browser's network log, look for server responses with the information you need, and send HTTP requests directly past the page.

aepot2021-11-25 12:50:35