next page
#1
by
TarasHu
- opened
How should i use it if there are many pages?
Hi! I would say that there are some possibilities. If want to fully automate the data extraction the best approach should be to take the Python function generated and create an algorithm that uses selenium and a headless browser to iterate through the different pages and save the HTML.
Finally, for each page apply the function generated.
It might be necessary to check if the element was already in the headless browser, due to some of the pages loading their data asynchronously.
A less straightforward approach might be to extract the HTML code manually and paste all the code into the input assigned.
In case you want to go for the first approach, I'll be glad to help in case you find it tricky.