-
Notifications
You must be signed in to change notification settings - Fork 1
/
Copy pathTODO.txt
68 lines (41 loc) · 2.96 KB
/
TODO.txt
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
TODO:
1.) DONE! finish google reviews scraping
1.5) DONE! process reviews into meta data (score, star breakdown, etc)
1.75) look into getting updated data from different places page:
http://www.google.com/mapmaker?ui_type=pc&cid=17751536374062546769&src=1&continue=https://plus.google.com/100271274097694780455/about?gl%3DUS%26hl%3Den-US
2.) DONE! add review gathering and storgae for yelp (same as googles)
3.) DONE!! setup Klout model to hold klout users, ids, scores, etc
3.5) DONE! get retweeter data for retweeted tweets
4.) DONE! setup klout data gathering via API
5.?) DONE! setup foursquare tips to be gathered and stored as well (same as reviews for yelp and google)?
5.5) DONE! add a cron job the detects duplicate twitter handles and checks those id's for updated screen names
6.) SCRAPED (until later)! look into business plus page using admin user id
7.) DONE! Setup in-depth logging system for... everything!
7.a) DONE! add fallbacks and logs for errors/changes in data scraping to notify if something has changed
IF HERE LOOK BACK AND REFER TO 5.5!
8.) DONE!setup headers for web scraping to resemble browser
10.) Yelp claim business link and directions in guide
24.) Need to look into anonymous proxies for web scraping
10.) nodejitsu deploy and test
10.1) Move solicaite out from modules folder
10.2) create nodejitsu database connection
11.) DONE! Alert system THEN TODO:10
11.1) Clean up and testing
11.2) DONE! Add admin section to input new Google places 'at:' hash [http://127.0.0.1:3000/admin/google-hash]
11.3) attach a dev user account to Twitter for connections.updates harvest to increase API rate limit (using app only for duplicates detection fixing in harvester)
11.4) MAYBE DONE - TEST: work on Twitter retweeters method to be more ffiecient (todo listed in method)
11.5) DONE - NEEDS TESTING! Setup twitter search harvesting for users (use dummy search for now)
11.7) DONE - SEE IF WE CAN LOAD MORE ENGAGERS FROM INSIGHTS HARVESTING Setup Facebook to load in users to Conenctions page
11.8) Cleanup: commenting; and setup production Util variable for quickly switching between dev and production modes
12) Expand Google plus activities harvesting
12.1) Make sure we are using user based calls and not app bearer tokens (this might involve Auth overhaul [see 13])
12.2) DONE - get Followers count!
12.3) DONE - setup G+ to look for updates in last 100ish posts
12.4) DONE - Setup Google+ user Engagers loading from reviewers and activity!
12.5) Google+ Engagers lookup using API
12.6) Look into the openId connection call with G+ API (https://developers.google.com/+/api/latest/people/getOpenIdConnect) for Engagers collection
13.) Overhaul Auth file to not use instances anymore
14.) Sentiment
15.) Notifications setup
16.) Graphing (graphing.js to be called from socket.io to produce graph data)
00.) setup request abstraction layer to handle poxies and compressed file returns! [http://nickfishman.com/post/49533681471/nodejs-http-requests-with-gzip-deflate-compression]