manifest_path is mostly empty for packages

[pombredanne]

Description

with:

wget https://files.pythonhosted.org/packages/de/6b/65b4a2ee355a1ea1b78bdc858e314e767cedcc44e8aaf9cd46e4da41674b/license_expression-0.999-py2.py3-none-any.whl
extractcode license_expression-0.999-py2.py3-none-any.whl
scancode --package --json-pp - license_expression-0.999-py2.py3-none-any.whl-extract/ --verbose --only-findings

I get this error:

Setup plugins...
Collect file inventory...
Scan files for: packages with 1 process(es)...
[####################] 8                              
ERROR: failed to run scan plugin: packages:
Traceback (most recent call last):
  File "/home/pombreda/w421/scancode-toolkit/src/scancode/cli.py", line 1033, in run_codebase_plugins
    plugin.process_codebase(codebase, **kwargs)
  File "/home/pombreda/w421/scancode-toolkit/src/packagedcode/plugin_package.py", line 165, in process_codebase
    new_manifest_path = os.path.join(parent_directory(new_package_root.path), resource.name)
NameError: name 'os' is not defined

{
  "headers": [
    {
      "tool_name": "scancode-toolkit",
      "tool_version": "3.1.1",
      "options": {
        "input": [
          "license_expression-0.999-py2.py3-none-any.whl-extract/"
        ],
        "--json-pp": "-",
        "--only-findings": true,
        "--package": true
      },
      "notice": "Generated with ScanCode and provided on an \"AS IS\" BASIS, WITHOUT WARRANTIES\nOR CONDITIONS OF ANY KIND, either express or implied. No content created from\nScanCode should be considered or used as legal advice. Consult an Attorney\nfor any legal advice.\nScanCode is a free software code scanning tool from nexB Inc. and others.\nVisit https://github.com/nexB/scancode-toolkit/ for support and download.",
      "start_timestamp": "2019-09-09T064201.232217",
      "end_timestamp": "2019-09-09T064202.026168",
      "message": null,
      "errors": [
        "ERROR: failed to run scan plugin: packages:\nTraceback (most recent call last):\n  File \"/home/pombreda/w421/scancode-toolkit/src/scancode/cli.py\", line 1033, in run_codebase_plugins\n    plugin.process_codebase(codebase, **kwargs)\n  File \"/home/pombreda/w421/scancode-toolkit/src/packagedcode/plugin_package.py\", line 165, in process_codebase\n    new_manifest_path = os.path.join(parent_directory(new_package_root.path), resource.name)\nNameError: name 'os' is not defined\n"
      ],
      "extra_data": {
        "files_count": 1
      }
    }
  ],
  "files": [
    {
      "path": "license_expression-0.999-py2.py3-none-any.whl-extract/license_expression/_pyahocorasick.ABOUT",
      "type": "file",
      "packages": [
        {
          "type": "about",
          "namespace": null,
          "name": "pyahocorasick-python",
          "version": "ec2fb9",
          "qualifiers": null,
          "subpath": null,
          "primary_language": null,
          "description": null,
          "release_date": null,
          "parties": [
            {
              "type": "person",
              "role": "owner",
              "name": "nexB Inc.",
              "email": null,
              "url": null
            }
          ],
          "keywords": [],
          "homepage_url": "https://github.com/WojciechMula/pyahocorasick",
          "download_url": "https://github.com/WojciechMula/pyahocorasick/tree/ec2fb9cb393f571fd4316ea98ed7b65992f16127/py",
          "size": null,
          "sha1": null,
          "md5": null,
          "sha256": null,
          "sha512": null,
          "bug_tracking_url": null,
          "code_view_url": null,
          "vcs_url": null,
          "copyright": "originally authored by Wojciech Mula, modified by the license_expression authors.",
          "license_expression": "public-domain",
          "declared_license": "public-domain",
          "notice_text": null,
          "manifest_path": null,
          "dependencies": [],
          "contains_source_code": null,
          "source_packages": [],
          "purl": "pkg:about/pyahocorasick-python@ec2fb9",
          "repository_homepage_url": null,
          "repository_download_url": null,
          "api_data_url": null
        }
      ],
      "scan_errors": []
    }
  ]
}
Scanning done.
Some files failed to scan properly:
ERROR: failed to run scan plugin: packages:
Traceback (most recent call last):
  File "/home/pombreda/w421/scancode-toolkit/src/scancode/cli.py", line 1033, in run_codebase_plugins
    plugin.process_codebase(codebase, **kwargs)
  File "/home/pombreda/w421/scancode-toolkit/src/packagedcode/plugin_package.py", line 165, in process_codebase
    new_manifest_path = os.path.join(parent_directory(new_package_root.path), resource.name)
NameError: name 'os' is not defined
Summary:        packages with 1 process(es)
Errors count:   1
Scan Speed:     68.09 files/sec. 
Initial counts: 11 resource(s): 8 file(s) and 3 directorie(s) 
Final counts:   1 resource(s): 1 file(s) and 0 directorie(s) 
Timings:
  scan_start: 2019-09-09T064201.232217
  scan_end:   2019-09-09T064202.026168
  setup_scan:licenses: 0.67s
  setup: 0.67s
  scan: 0.12s
  total: 0.80s
Removing temporary files...done.

System configuration

For bug reports, it really helps us to know:

• What OS are you running on? Linux Python 2.7
• What version of scancode-toolkit was used to generate the scan file? v3.1.1 pre release
• What installation method was used to install/run scancode? standard archive

[pombredanne]

The base fix has been contributed by @adityaviki (Thanks! )
The follow on related work is tracked in #1237 (comment)