Skip to content

Commit

Permalink
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
limit to 15 pages
Browse files Browse the repository at this point in the history
Signed-off-by: brotholomew <bartoszek.blach@gmail.com>
Brotholomew committed Jun 15, 2024
1 parent 62d2a95 commit cb68556
Showing 1 changed file with 1 addition and 1 deletion.
2 changes: 1 addition & 1 deletion scrapers/impl/scraper_oevw.py
Original file line number Diff line number Diff line change
@@ -146,7 +146,7 @@ def oevw_scraper(timerObj: func.TimerRequest, q: func.Out[str]) -> None:
# if req.url != req_url:
# break
# azure timeout limitations, scan only first 20 pages
if page > 20:
if page > 15:
break

logging.info(f'scraping page no #{page}')

0 comments on commit cb68556

Please sign in to comment.