-
Notifications
You must be signed in to change notification settings - Fork 39
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
When scrape.py is run , It throws errors like Error opening url!! #1
Comments
Hello Any response ? |
The script is executing correctly. Could you be more specific about the problem? "Error Opening URL!!" (Line 40 in the script) is mainly thrown due to timeout/connection issues. Also, it has been tested on python3 only, older versions of requests library might be an issue. |
Thanks for responding.. I was debugging from my end.. I could zero in that error is because of the signal module in windows platform. Which does not have attributes such as SIGALRM etc.? Please tell me when u ran this did u ran on Unix platform or windows? |
one more question, |
It was tested on unix platform. Time function is there to handle request timeouts automatically. You can comment out the lines if required. The function could be simplified to:
|
Will try this in windows and let you know. Btw are you aware the legality of this code ? I mean is scraping data from Moneycontrol legal ? |
The robot.txt file disallows crawling certain endpoints, none of which this script accesses. |
Hi, Does it work in Google chrome also? As, I am running the script, I am facing an error 'Access is denied' |
This script is browser independent. There is no browser automation required to run it. So Google chrome is irrelevant to running the script. This will run in the terminal |
Hi KeerthanKumar Were you able to run it on Windows? I am getting the same "Error Opening URL" on spyder IDE on win 10. Please assist |
Hi, This was created a while back, maybe the endpoints have changed. Let me know the features required, maybe I can still patch it free time. |
If you could please patch it, it would be awesome |
For each company, it is saying "Data on ABC company doesnt exist anymore" where ABC is every company. |
Line 268 in fca06a3
Is starting from index 8. Thus it's always starting with H. |
Thanks, Didn't see it before. |
For all the companies, it says data doesn't exist anymore. Please take a look when you can and update |
Data on 'G K P Printing' doesn't exist anymore. even If I make a position change for the same! |
Is it still possible to scrape data from moneycontrol through this script in April 2018 or something changed in a website that script doesn't valid anymore?
The text was updated successfully, but these errors were encountered: