Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Enhancement: customers will use the rest api for the project entity. #220

Open
wants to merge 48 commits into
base: master
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
48 commits
Select commit Hold shift + click to select a range
a5803bd
feat: added project v1 to rest conversion class
davidalexandru7013 Aug 9, 2024
5cee33b
feat: added method to check if a path is project v1
davidalexandru7013 Aug 9, 2024
39d9486
fix: updated delete project tests to mock rest route
davidalexandru7013 Aug 11, 2024
62e3c3b
chore: black formatting of client, managers and models files
davidalexandru7013 Aug 11, 2024
0ee912d
feat: moved get project by id from v1 to rest
davidalexandru7013 Aug 12, 2024
8fc8a36
fix: replace old route with new route for projects in tests
davidalexandru7013 Aug 12, 2024
cfe46fe
feat: patch method in client
davidalexandru7013 Aug 12, 2024
eeddafe
chore: removed debug from examples
davidalexandru7013 Aug 12, 2024
956394e
chore: removed ununsed class
davidalexandru7013 Aug 12, 2024
659827e
chore: replaced old api version with new one
davidalexandru7013 Aug 12, 2024
5c21f1d
feat: added patch update project test
davidalexandru7013 Aug 12, 2024
3b76531
feat: tests for patch update project
davidalexandru7013 Aug 12, 2024
f0a05e2
chore: added get project by id and tests back
davidalexandru7013 Aug 12, 2024
d6db244
chore: use rest boolean for clients and tests
davidalexandru7013 Aug 12, 2024
a04902a
chore: send query params
davidalexandru7013 Aug 12, 2024
2bc763f
feat: tests to avoid side-effects for api headers
davidalexandru7013 Aug 13, 2024
90f67bc
feat: clients can use rest path for delete and tests
davidalexandru7013 Aug 13, 2024
a823df4
chore: removed example file
davidalexandru7013 Aug 13, 2024
b849478
chore: rename test from new project to updated project
davidalexandru7013 Aug 13, 2024
16b51e3
feat: migration get project by id to rest api
davidalexandru7013 Aug 14, 2024
466df46
feat: get project by id manager uses rest api
davidalexandru7013 Aug 19, 2024
cff879b
fix: replaced project mock with project rest response and fix tests
davidalexandru7013 Aug 19, 2024
69063ea
feat: project rest response
davidalexandru7013 Aug 19, 2024
857cf5e
fix: get project's id
davidalexandru7013 Aug 19, 2024
657d4ec
chore: replace dict with Dict and extract latest version in a variable
davidalexandru7013 Aug 20, 2024
45d2ec8
chore: replace type from any to dict
davidalexandru7013 Aug 20, 2024
de0ffdc
fix: filter project query params
davidalexandru7013 Aug 20, 2024
0998780
fix: added rest content type for post requests when calling rest route
davidalexandru7013 Aug 20, 2024
d4fe020
fix: added rest content type for put method and tests
davidalexandru7013 Aug 20, 2024
69a5c34
fix: project rest mock response
davidalexandru7013 Aug 20, 2024
ef38878
fix: update tests for project
davidalexandru7013 Aug 21, 2024
e20d143
fix: project package manager type in issues and tests
davidalexandru7013 Aug 21, 2024
fe6b27f
Merge branch 'feat/client-rest' of github.com:davidalexandru7013/pysn…
davidalexandru7013 Aug 21, 2024
fa9968e
chore: added default value of 0 for total dependencies in a project
davidalexandru7013 Aug 21, 2024
2995c7a
fix: updated projects example and tests for query params
davidalexandru7013 Aug 22, 2024
f19cb14
fix: tests to check query params on project filtering
davidalexandru7013 Aug 22, 2024
bb3afed
fix: example 10 project attributes and test for get all project
davidalexandru7013 Aug 22, 2024
d083449
fix: new project attributes
davidalexandru7013 Aug 22, 2024
30dfac1
chore: removed mock example
davidalexandru7013 Aug 22, 2024
b152c0c
feat: update project method in project manager
davidalexandru7013 Aug 23, 2024
459c3ba
fix: updated documention with new project fields
davidalexandru7013 Aug 23, 2024
b0b9a51
fix: tests for update project method in project manager
davidalexandru7013 Aug 23, 2024
8e86708
fix: docs example of update project
davidalexandru7013 Aug 26, 2024
881b999
fix: mark old issue examples with deprecated
davidalexandru7013 Aug 26, 2024
110fcd6
fix: avoid altering tags if next url exists
davidalexandru7013 Aug 26, 2024
21ed6bf
fix: update project owner id
davidalexandru7013 Aug 26, 2024
8ef891c
fix: moved delete project path into manager
davidalexandru7013 Aug 26, 2024
1711125
fix: increase projects limit to 100
davidalexandru7013 Aug 26, 2024
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
25 changes: 25 additions & 0 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -274,3 +274,28 @@ response = rest_client.post(f"orgs/{snyk_org}/invites", body={"email": "some.bod
```

For backwards compatibility the get_rest_pages method has an alternative name of get_v3_pages to not break code already rewritten replatformed to the 0.9.0 pysnyk module.

## Migrating from old project to new project entity

The rest API introduces a new representation for the project class, which offers more information. There are some changes necessary to keep the same functionality
of the application:
- The old project attributes can be accessed using the attributes class. The below example
```python
project.name -> project.attributes.name
project.created -> project.attributes.created
project.origin -> project.attributes.origin
project.type -> project.attributes.type
project.readOnly -> project.attributes.read_only
project.attributes.criticality -> project.attributes.business_criticality
project.totalDependencies -> project.meta.latest_dependency_total.total
project.issueCountsBySeverity -> project.meta.latest_issue_counts
project.lastTestedDate -> project.meta.cli_monitored_at
project.importingUser.id -> project.relationships.importer.data.id
project.isMonitored -> project.attributes.status
project.targetReference -> project.attributes.target_reference
```

To update an existing project, the update method from `ProjectManager` can be used, as in the following example:
```python
client.organizations.get(org_id).projects.update(project_id, tags=[{"key":"added_tag_key2", "value":"added_tag_value2"}], environment=[], business_criticality=["critical","low","medium"], lifecycle=["development","production"], test_frequency="daily")
```
8 changes: 4 additions & 4 deletions examples/api-demo-1-list-projects.py
Original file line number Diff line number Diff line change
Expand Up @@ -19,8 +19,8 @@ def parse_command_line_args():

client = SnykClient(token=snyk_token)
for proj in client.organizations.get(org_id).projects.all():
print("\nProject name: %s" % proj.name)
print("\nProject name: %s" % proj.attributes.name)
print(" Issues Found:")
print(" High : %s" % proj.issueCountsBySeverity.high)
print(" Medium: %s" % proj.issueCountsBySeverity.medium)
print(" Low : %s" % proj.issueCountsBySeverity.low)
print(" High : %s" % proj.meta.latest_issue_counts.high)
print(" Medium: %s" % proj.meta.latest_issue_counts.medium)
print(" Low : %s" % proj.meta.latest_issue_counts.low)
5 changes: 3 additions & 2 deletions examples/api-demo-10-project-deps-licenses-report.py
Original file line number Diff line number Diff line change
Expand Up @@ -100,8 +100,8 @@ def get_flat_dependencies(dep_list):
client = SnykClient(snyk_token)
projects = client.organizations.get(org_id).projects.all()
for proj in projects:
if proj.origin in allowed_origins:
all_projects_list.append({"project_id": proj.id, "project_name": proj.name})
if proj.attributes.origin in allowed_origins:
all_projects_list.append({"project_id": proj.id, "project_name": proj.attributes.name})

project_trees = []
flattened_project_dependencies_lists = []
Expand All @@ -112,6 +112,7 @@ def get_flat_dependencies(dep_list):
next_project_id = next_project["project_id"]

if args.projectId == "all" or next_project_id == args.projectId:
# Deprecated, check api-demo-2c-list-issues-aggregated
next_project_tree = ProjectDependenciesReport.get_project_tree(
snyk_token, org_id, next_project_id
)
Expand Down
4 changes: 2 additions & 2 deletions examples/api-demo-11-update-github-checks.py
Original file line number Diff line number Diff line change
Expand Up @@ -37,9 +37,9 @@ def parse_command_line_args():
projects = client.organizations.get(org_id).projects.all()

github_projects = [
{"id": p.id, "name": p.name}
{"id": p.id, "name": p.attributes.name}
for p in projects
if p.origin == "github"
if p.attributes.origin == "github"
]

def get_project_by_id(projects, project_id):
Expand Down
1 change: 1 addition & 0 deletions examples/api-demo-2-list-issues.py
Original file line number Diff line number Diff line change
Expand Up @@ -57,6 +57,7 @@ def output_excel(vulns, output_path):


client = SnykClient(snyk_token)
#Deprecated, check api-demo-2c-list-issues-aggregated
issue_set = client.organizations.get(org_id).projects.get(project_id).issueset.all()

lst_output = []
Expand Down
4 changes: 2 additions & 2 deletions examples/api-demo-8-delete-projects-by-name.py
Original file line number Diff line number Diff line change
Expand Up @@ -42,8 +42,8 @@ def parse_command_line_args():

client = SnykClient(snyk_token)
for project in client.organizations.get(org_id).projects.all():
if project_name == project.name and (
project.origin == project_origin or not project_origin
if project_name == project.attributes.name and (
project.attributes.origin == project_origin or not project_origin
):
if project.delete():
print("Project ID %s deleted" % project.id)
2 changes: 1 addition & 1 deletion examples/api-demo-9-list-ignores.py
Original file line number Diff line number Diff line change
Expand Up @@ -23,7 +23,7 @@ def parse_command_line_args():

for proj in projects:
project_id = proj.id
project_name = proj.name
project_name = proj.attributes.name
ignores = proj.ignores.all()

if len(ignores) > 0:
Expand Down
10 changes: 5 additions & 5 deletions examples/api-demo-9b-ignore-vulns-by-id.py
Original file line number Diff line number Diff line change
Expand Up @@ -66,12 +66,12 @@ def parse_command_line_args():
# API call to collect every project in all of a customers orgs
http = urllib3.PoolManager()
for proj in client.organizations.get(org_id).projects.all():
print("\nProject name: %s" % proj.name)
print("\nProject name: %s" % proj.attributes.name)
print(" Issues Found:")
print(" High : %s" % proj.issueCountsBySeverity.high)
print(" Medium: %s" % proj.issueCountsBySeverity.medium)
print(" Low : %s" % proj.issueCountsBySeverity.low)

print(" High : %s" % proj.meta.latest_issue_counts.high)
print(" Medium: %s" % proj.meta.latest_issue_counts.medium)
print(" Low : %s" % proj.meta.latest_issue_counts.low)
# Deprecated, check api-demo-2c-list-issues-aggregated
url = "org/" + org_id + "/project/" + proj.id + "/issues"

print(url)
Expand Down
9 changes: 5 additions & 4 deletions examples/api-demo-9c-bulk-ignore-vulns-by-issueIdList.py
Original file line number Diff line number Diff line change
Expand Up @@ -57,11 +57,12 @@ def parse_command_line_args():
# API call to collect every project in all of a customers orgs

for proj in client.organizations.get(org_id).projects.all():
print("\nProject name: %s" % proj.name)
print("\nProject name: %s" % proj.attributes.name)
print(" Issues Found:")
print(" High : %s" % proj.issueCountsBySeverity.high)
print(" Medium: %s" % proj.issueCountsBySeverity.medium)
print(" Low : %s" % proj.issueCountsBySeverity.low)
print(" High : %s" % proj.meta.latest_issue_counts.high)
print(" Medium: %s" % proj.meta.latest_issue_counts.medium)
print(" Low : %s" % proj.meta.latest_issue_counts.low)
#Deprecated, check api-demo-2c-list-issues-aggregated
url = "org/" + org_id + "/project/" + proj.id + "/issues"
print(url)
# API call to grab all of the issue
Expand Down
111 changes: 97 additions & 14 deletions snyk/client.py
Original file line number Diff line number Diff line change
@@ -1,6 +1,8 @@
import copy
import logging
import re
import urllib.parse
from typing import Any, List, Optional
from typing import Any, Dict, List, Optional, Pattern
from urllib.parse import parse_qs, urlparse

import requests
Expand Down Expand Up @@ -40,13 +42,16 @@ def __init__(
"Authorization": "token %s" % self.api_token,
"User-Agent": user_agent,
}
self.api_post_headers = self.api_headers
self.api_post_headers = dict(self.api_headers)
self.api_post_headers["Content-Type"] = "application/json"
self.api_patch_headers = dict(self.api_headers)
self.api_patch_headers["Content-Type"] = "application/vnd.api+json"
self.tries = tries
self.backoff = backoff
self.delay = delay
self.verify = verify
self.version = version
self.__latest_version = "2024-06-21"

# Ensure we don't have a trailing /
if self.api_url[-1] == "/":
Expand Down Expand Up @@ -82,14 +87,31 @@ def request(
raise SnykHTTPError(resp)
return resp

def post(self, path: str, body: Any, headers: dict = {}) -> requests.Response:
url = f"{self.api_url}/{path}"
def post(
self,
path: str,
body: Dict[str, Any],
headers: Dict = {},
params: Dict[str, Any] = {},
use_rest: bool = False,
) -> requests.Response:
url = f"{self.rest_api_url if use_rest else self.api_url}/{path}"
logger.debug(f"POST: {url}")

request_headers = {**self.api_post_headers, **headers}
if use_rest:
if "version" not in params:
params["version"] = self.version or self.__latest_version
request_headers["Content-Type"] = "application/vnd.api+json"

resp = retry_call(
self.request,
fargs=[requests.post, url],
fkwargs={"json": body, "headers": {**self.api_post_headers, **headers}},
fkwargs={
"json": body,
"headers": request_headers,
"params": params,
},
tries=self.tries,
delay=self.delay,
backoff=self.backoff,
Expand All @@ -103,14 +125,68 @@ def post(self, path: str, body: Any, headers: dict = {}) -> requests.Response:

return resp

def put(self, path: str, body: Any, headers: dict = {}) -> requests.Response:
url = "%s/%s" % (self.api_url, path)
def patch(
self,
path: str,
body: Dict[str, Any],
headers: Dict[str, str] = {},
params: Dict[str, Any] = {},
) -> requests.Response:
url = f"{self.rest_api_url}/{path}"
logger.debug(f"PATCH: {url}")

if "version" not in params:
params["version"] = self.version or self.__latest_version

resp = retry_call(
self.request,
fargs=[requests.patch, url],
fkwargs={
"json": body,
"headers": {**self.api_patch_headers, **headers},
"params": params,
},
tries=self.tries,
delay=self.delay,
backoff=self.backoff,
logger=logger,
)

if not resp.ok:
logger.error(resp.text)
raise SnykHTTPError(resp)

return resp

def put(
self,
path: str,
body: Any,
headers: Dict = {},
params: Dict[str, Any] = {},
use_rest: bool = False,
) -> requests.Response:
url = "%s/%s" % (
self.rest_api_url if use_rest else self.api_url,
path,
)
logger.debug("PUT: %s" % url)

request_headers = {**self.api_post_headers, **headers}

if use_rest:
if "version" not in params:
params["version"] = self.version or self.__latest_version
request_headers["Content-Type"] = "application/vnd.api+json"

resp = retry_call(
self.request,
fargs=[requests.put, url],
fkwargs={"json": body, "headers": {**self.api_post_headers, **headers}},
fkwargs={
"json": body,
"headers": request_headers,
"params": params,
},
tries=self.tries,
delay=self.delay,
backoff=self.backoff,
Expand All @@ -125,7 +201,7 @@ def put(self, path: str, body: Any, headers: dict = {}) -> requests.Response:
def get(
self,
path: str,
params: dict = None,
params: Dict = None,
version: str = None,
exclude_version: bool = False,
exclude_params: bool = False,
Expand Down Expand Up @@ -163,10 +239,12 @@ def get(

# Python Bools are True/False, JS Bools are true/false
# Snyk REST API is strictly case sensitive at the moment

# List elements are separated using comma
for k, v in params.items():
if isinstance(v, bool):
params[k] = str(v).lower()
elif isinstance(v, list):
params[k] = ",".join(v)

# the limit is returned in the url, and if two limits are passed
# the API interprets as an array and throws an error
Expand Down Expand Up @@ -196,14 +274,19 @@ def get(

return resp

def delete(self, path: str) -> requests.Response:
url = f"{self.api_url}/{path}"
def delete(self, path: str, use_rest: bool = False) -> requests.Response:
url = f"{self.rest_api_url if use_rest else self.api_url}/{path}"

params = {}
if use_rest:
params["version"] = self.version or self.__latest_version

logger.debug(f"DELETE: {url}")

resp = retry_call(
self.request,
fargs=[requests.delete, url],
fkwargs={"headers": self.api_headers},
fkwargs={"headers": self.api_headers, "params": params},
tries=self.tries,
delay=self.delay,
backoff=self.backoff,
Expand All @@ -215,7 +298,7 @@ def delete(self, path: str) -> requests.Response:

return resp

def get_rest_pages(self, path: str, params: dict = {}) -> List:
def get_rest_pages(self, path: str, params: Dict = {}) -> List:
"""
Helper function to collect paginated responses from the rest API into a single
list.
Expand Down
Loading