You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Is your feature request related to a problem? Please describe.
Hello, I am trying to scrap a SPA page where "domcontentloaded" or "networkidle" event does not guarantee content readiness.
Is your feature request related to a problem? Please describe.
Hello, I am trying to scrap a SPA page where "domcontentloaded" or "networkidle" event does not guarantee content readiness.
Describe the solution you'd like
One solution would be adding a way to pass a hook function which gets called right after opening a web page and before getting content (https://github.com/ScrapeGraphAI/Scrapegraph-ai/blob/main/scrapegraphai/docloaders/chromium.py#L88). If the playwright.Page is passed to this hook, a caller can customize how to determine readiness of the page content.
Describe alternatives you've considered
I am currently trying to workaround this limitation by monkey patching playwright functions.
Additional context
N/A
The text was updated successfully, but these errors were encountered: