Skip to content

Commit

Permalink
limit to 15 pages
Browse files Browse the repository at this point in the history
Signed-off-by: brotholomew <bartoszek.blach@gmail.com>
Brotholomew committed Jun 15, 2024
1 parent a5de090 commit 8acbde2
Showing 1 changed file with 1 addition and 1 deletion.
2 changes: 1 addition & 1 deletion scrapers/impl/scraper_oevw.py
Original file line number Diff line number Diff line change
@@ -146,7 +146,7 @@ def oevw_scraper(timerObj: func.TimerRequest, q: func.Out[str]) -> None:
# if req.url != req_url:
# break
# azure timeout limitations, scan only first 20 pages
if page > 20:
if page > 15:
break

logging.info(f'scraping page no #{page}')

0 comments on commit 8acbde2

Please sign in to comment.