Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

DMC: Support Visual Inspection Scenarios #90

Draft
wants to merge 8 commits into
base: main
Choose a base branch
from

Conversation

Valentin-Mueller
Copy link

@Valentin-Mueller Valentin-Mueller commented Nov 11, 2021

Supports retrieval of active scenarios and inspection logs from the DMC AIML API. Allows export of InspectionLogSets as DataFrames usable for general ML, binary classification, multilabel classification and object detection based on available information along with dictionaries containing the file content mapped to the file id.

Description

Supported the following use case for the DMC API:

  1. Search for Active Scenario
  2. Get Analytical Data Set (Inspection Logs) from a Scenario
  3. Get Images and Inspection results from
    a. as_ml_input(),
    b. as_binary_classification_input()
    c. as_multilabel_classification_input()
    d. as_object_detection_input()

Checklist

Mark "not applicable" if item on the list does not apply to this pull request.

  • I have created/adapted unit tests for new code
    • not applicable
  • I have added new dependencies as described at the Contributing page
    • not applicable
  • I have made corresponding changes to the documentation (incl. tutorial, etc.)
    • not applicable
  • I have provided release notes (in description or as comment) if this PR is a new feature OR a change worth mentioning for next release
    • not applicable
  • I have obtained API approvals if a new SAP API or endpoint is used
    • not applicable

Supports retrieval of active scenarios and inspection logs from the DMC AIML API. Allows export of InspectionLogSets as DataFrames usable for general ML, binary classification, multilabel classification and object detection based on available information along with dictionaries containing the file content mapped to the file id.
Reduced duplications in inspection logs by implementing functions commonly used by all 'as_ml_input()' methods. Implemented some additional tests for some edge cases that were overlooked initially. Fixed code smells with bounding box conditions, order of and spacing between inputs and refactored all non-compliant docstrings.
Copy link
Member

@cloudyday cloudyday left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think another round of discussion is needed on some of the issues.

And furthermore this change warrants a README / Docu / Tutorial change.

sailor/dmc/scenario.py Show resolved Hide resolved
sailor/dmc/scenario.py Show resolved Hide resolved
sailor/dmc/scenario.py Outdated Show resolved Hide resolved
sailor/dmc/inspection_log.py Outdated Show resolved Hide resolved
Comment on lines +39 to +40
'from_date': 'fromDate',
'to_date': 'toDate',
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this expects a very clearly defined format (yyyy-MM-dd HH:mm:ss). milliseconds can also be appended after a .. (e.g. 2021-11-18 00:00:00.999) not adhering to this format results in a HTTP 400 with no hint to what went wrong.

we have a problem in general with this "homebrew" list of filter fields and the corresponding "non-odata conform" filter language. we're not able to add query transformers.

This comment was marked as outdated.

sailor/dmc/inspection_log.py Outdated Show resolved Hide resolved
sailor/dmc/inspection_log.py Outdated Show resolved Hide resolved
sailor/dmc/inspection_log.py Outdated Show resolved Hide resolved
sailor/dmc/inspection_log.py Outdated Show resolved Hide resolved
sailor/dmc/inspection_log.py Outdated Show resolved Hide resolved
sailor/dmc/scenario.py Show resolved Hide resolved
sailor/dmc/inspection_log.py Outdated Show resolved Hide resolved
sailor/dmc/inspection_log.py Outdated Show resolved Hide resolved
sailor/dmc/inspection_log.py Outdated Show resolved Hide resolved
Comment on lines +39 to +40
'from_date': 'fromDate',
'to_date': 'toDate',

This comment was marked as outdated.

sailor/dmc/inspection_log.py Outdated Show resolved Hide resolved
sailor/dmc/inspection_log.py Outdated Show resolved Hide resolved
sailor/dmc/scenario.py Show resolved Hide resolved
sailor/dmc/scenario.py Outdated Show resolved Hide resolved
sailor/dmc/inspection_log.py Show resolved Hide resolved
small changes according to review for Scenarios
inspectionLog has a unique ID now.
timestamp now maps to inspectionLogTime

tests adapted and simplified
inspectionLogTime is now a pd.Timestamp

Consolidate data model (no tests)

store decoded images in InspectionLogSet with duplicate warning

small code and test adaptations

fix Inspection Log tests

Update comments

Rename find_scenarios and update docstring
@cloudyday cloudyday marked this pull request as draft December 20, 2021 16:32
@sonarcloud
Copy link

sonarcloud bot commented Dec 21, 2021

SonarCloud Quality Gate failed.    Quality Gate failed

Bug A 0 Bugs
Vulnerability A 0 Vulnerabilities
Security Hotspot A 0 Security Hotspots
Code Smell A 8 Code Smells

0.0% 0.0% Coverage
0.0% 0.0% Duplication

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants