Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Create storage crawling graph implementation for headless crawler #13

Open
Ice3man543 opened this issue Jun 30, 2022 · 3 comments · May be fixed by #174
Open

Create storage crawling graph implementation for headless crawler #13

Ice3man543 opened this issue Jun 30, 2022 · 3 comments · May be fixed by #174
Assignees
@Mzack9999
Labels
headless Priority: Medium This issue may be useful, and needs some attention. Status: In Progress This issue is being worked on, and has someone assigned.

Comments

@Ice3man543
Copy link
Member

  • Implement a crawl graph with relevant method for navigation and events storage and retrieval
@Ice3man543
Copy link
Member Author

Ice3man543 commented Jun 30, 2022 •
edited
Loading

  • Either we can have a graph with States as Nodes and Events as Edges
  • Or we can have a graph with States as Edges containing event information in them. The first one sounds better

Create an interface for the storage crawl graph, decide on methods/algorithms to implement + choose an implementation in golang.

@Ice3man543 Ice3man543 changed the title Create storage crawling graph implementation Create storage crawling graph implementation for headless crawler Jun 30, 2022
@ehsandeep ehsandeep added the Priority: Medium This issue may be useful, and needs some attention. label Jun 30, 2022
@Mzack9999 Mzack9999 self-assigned this Jul 4, 2022
@Mzack9999 Mzack9999 added the Status: In Progress This issue is being worked on, and has someone assigned. label Jul 4, 2022
@Mzack9999 Mzack9999 linked a pull request Jul 12, 2022 that will close this issue
Navigation Graph #19
Closed
@ehsandeep ehsandeep added Status: Review Needed The issue has a PR attached to it which needs to be reviewed and removed Status: In Progress This issue is being worked on, and has someone assigned. labels Jul 13, 2022
@ehsandeep ehsandeep added Status: In Progress This issue is being worked on, and has someone assigned. and removed Status: Review Needed The issue has a PR attached to it which needs to be reviewed labels Aug 31, 2022
@Mzack9999 Mzack9999 removed the Status: In Progress This issue is being worked on, and has someone assigned. label Sep 26, 2022
@brenocss
Copy link

brenocss commented Nov 8, 2022

call graph would be amazing for visualization as well

@Mzack9999 Mzack9999 mentioned this issue Nov 8, 2022
Open
@Mzack9999 Mzack9999 added the Status: In Progress This issue is being worked on, and has someone assigned. label Nov 9, 2022
@derekperkins
Copy link

Is the plan to make a pluggable storage layer? A vote for Vitess (sharded MySQL) from me :)

@Mzack9999 Mzack9999 linked a pull request Nov 13, 2022 that will close this issue
Adding support for graphdb #174
Draft
4 tasks
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@Mzack9999 Mzack9999

Labels
headless Priority: Medium This issue may be useful, and needs some attention. Status: In Progress This issue is being worked on, and has someone assigned.
Projects
None yet
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

Adding support for graphdb projectdiscovery/katana
Navigation Graph projectdiscovery/katana
5 participants
@derekperkins @ehsandeep @Mzack9999 @Ice3man543 @brenocss