Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

HTTP 403 Forbidden Error During Model Prediction Due to File Access Denial when using with Cloud Storage Sync #660

Open
SabolSocare opened this issue Nov 4, 2024 · 4 comments
Open

HTTP 403 Forbidden Error During Model Prediction Due to File Access Denial when using with Cloud Storage Sync #660

SabolSocare opened this issue Nov 4, 2024 · 4 comments

Comments

@SabolSocare
Copy link

SabolSocare commented Nov 4, 2024 •
edited by makseq
Loading

I am unable to use the Yolo ML Backend model to annotate image data from Cloud Storage.

/label-studio/media/upload, upload_dir=/Users/admin/Library/Application Support/label-studio/media/upload
[2024-11-04 16:16:50,467] [INFO] [label_studio_sdk._extensions.label_studio_tools.core.utils.io::get_local_path::144] Resolving url using hostname [http://localhost:8089/]: http://localhost:8089/data/local-files/?d=input/63db415b7391984875aee6a7/workspaces/admin/756/files/categories/images/originals/1f742291bf201cfc4c95d5ab7ce3bfc5.jpg
[2024-11-04 16:16:50,468] [INFO] [label_studio_sdk._extensions.label_studio_tools.core.utils.io::download_and_cache::205] Download http://localhost:8089/data/local-files/?d=input/63db415b7391984875aee6a7/workspaces/admin/756/files/categories/images/originals/1f742291bf201cfc4c95d5ab7ce3bfc5.jpg to /Users/admin/Library/Caches/label-studio/24791863__1f742291bf201cfc4c95d5ab7ce3bfc5.jpg
[2024-11-04 16:16:50,468] [DEBUG] [label_studio_sdk._extensions.label_studio_tools.core.utils.io::download_and_cache::218] Authorization token is used for download_and_cache
[2024-11-04 16:16:50,586] [ERROR] [label_studio_ml.exceptions::exception_f::53] Traceback (most recent call last):
  File "/Users/admin/Work/X-Brain/New_ML_Release/label-studio-ml-backend/label_studio_ml/examples/yolo/venv/lib/python3.12/site-packages/label_studio_ml/exceptions.py", line 39, in exception_f
    return f(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^
  File "/Users/admin/Work/X-Brain/New_ML_Release/label-studio-ml-backend/label_studio_ml/examples/yolo/venv/lib/python3.12/site-packages/label_studio_ml/api.py", line 69, in _predict
    response = model.predict(tasks, context=context, **params)
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/admin/Work/X-Brain/New_ML_Release/yolo/model.py", line 120, in predict
    path = model.get_path(task)
           ^^^^^^^^^^^^^^^^^^^^
  File "/Users/admin/Work/X-Brain/New_ML_Release/yolo/control_models/base.py", line 187, in get_path
    else get_local_path(task_path, task_id=task.get("id"))
         ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/admin/Work/X-Brain/New_ML_Release/label-studio-ml-backend/label_studio_ml/examples/yolo/venv/lib/python3.12/site-packages/label_studio_sdk/_extensions/label_studio_tools/core/utils/io.py", line 166, in get_local_path
    filepath = download_and_cache(
               ^^^^^^^^^^^^^^^^^^^
  File "/Users/admin/Work/X-Brain/New_ML_Release/label-studio-ml-backend/label_studio_ml/examples/yolo/venv/lib/python3.12/site-packages/label_studio_sdk/_extensions/label_studio_tools/core/utils/io.py", line 221, in download_and_cache
    r.raise_for_status()
  File "/Users/admin/Work/X-Brain/New_ML_Release/label-studio-ml-backend/label_studio_ml/examples/yolo/venv/lib/python3.12/site-packages/requests/models.py", line 1024, in raise_for_status
    raise HTTPError(http_error_msg, response=self)
requests.exceptions.HTTPError: 403 Client Error: Forbidden for url: http://localhost:8089/data/local-files/?d=input/63db415b7391984875aee6a7/workspaces/admin/756/files/categories/images/originals/1f742291bf201cfc4c95d5ab7ce3bfc5.jpg

Traceback (most recent call last):
  File "/Users/admin/Work/X-Brain/New_ML_Release/label-studio-ml-backend/label_studio_ml/examples/yolo/venv/lib/python3.12/site-packages/label_studio_ml/exceptions.py", line 39, in exception_f
    return f(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^
  File "/Users/admin/Work/X-Brain/New_ML_Release/label-studio-ml-backend/label_studio_ml/examples/yolo/venv/lib/python3.12/site-packages/label_studio_ml/api.py", line 69, in _predict
    response = model.predict(tasks, context=context, **params)
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/admin/Work/X-Brain/New_ML_Release/yolo/model.py", line 120, in predict
    path = model.get_path(task)
           ^^^^^^^^^^^^^^^^^^^^
  File "/Users/admin/Work/X-Brain/New_ML_Release/yolo/control_models/base.py", line 187, in get_path
    else get_local_path(task_path, task_id=task.get("id"))
         ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/admin/Work/X-Brain/New_ML_Release/label-studio-ml-backend/label_studio_ml/examples/yolo/venv/lib/python3.12/site-packages/label_studio_sdk/_extensions/label_studio_tools/core/utils/io.py", line 166, in get_local_path
    filepath = download_and_cache(
               ^^^^^^^^^^^^^^^^^^^
  File "/Users/admin/Work/X-Brain/New_ML_Release/label-studio-ml-backend/label_studio_ml/examples/yolo/venv/lib/python3.12/site-packages/label_studio_sdk/_extensions/label_studio_tools/core/utils/io.py", line 221, in download_and_cache
    r.raise_for_status()
  File "/Users/admin/Work/X-Brain/New_ML_Release/label-studio-ml-backend/label_studio_ml/examples/yolo/venv/lib/python3.12/site-packages/requests/models.py", line 1024, in raise_for_status
    raise HTTPError(http_error_msg, response=self)
requests.exceptions.HTTPError: 403 Client Error: Forbidden for url: http://localhost:8089/data/local-files/?d=input/63db415b7391984875aee6a7/workspaces/admin/756/files/categories/images/originals/1f742291bf201cfc4c95d5ab7ce3bfc5.jpg

[2024-11-04 16:16:50,589] [DEBUG] [label_studio_ml.api::log_response_info::191] Response status: 500 INTERNAL SERVER ERROR
[2024-11-04 16:16:50,589] [DEBUG] [label_studio_ml.api::log_response_info::192] Response headers: Content-Type: application/json
Content-Length: 2389

[2024-11-04 16:16:50,589] [DEBUG] [label_studio_ml.api::log_response_info::193] Response body: b'{\n  "detail": "HTTPError: 403 Client Error: Forbidden for url: http://localhost:8089/data/local-files/?d=input/63db415b7391984875aee6a7/workspaces/admin/756/files/categories/images/originals/1f742291bf201cfc4c95d5ab7ce3bfc5.jpg",\n  "request": {},\n  "result": {\n    "traceback": "Traceback (most recent call last):\\n  File \\"/Users/admin/Work/X-Brain/New_ML_Release/label-studio-ml-backend/label_studio_ml/examples/yolo/venv/lib/python3.12/site-packages/label_studio_ml/exceptions.py\\", line 39, in exception_f\\n    return f(*args, **kwargs)\\n           ^^^^^^^^^^^^^^^^^^\\n  File \\"/Users/admin/Work/X-Brain/New_ML_Release/label-studio-ml-backend/label_studio_ml/examples/yolo/venv/lib/python3.12/site-packages/label_studio_ml/api.py\\", line 69, in _predict\\n    response = model.predict(tasks, context=context, **params)\\n               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^\\n  File \\"/Users/admin/Work/X-Brain/New_ML_Release/yolo/model.py\\", line 120, in predict\\n    path = model.get_path(task)\\n           ^^^^^^^^^^^^^^^^^^^^\\n  File \\"/Users/admin/Work/X-Brain/New_ML_Release/yolo/control_models/base.py\\", line 187, in get_path\\n    else get_local_path(task_path, task_id=task.get(\\"id\\"))\\n         ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^\\n  File \\"/Users/admin/Work/X-Brain/New_ML_Release/label-studio-ml-backend/label_studio_ml/examples/yolo/venv/lib/python3.12/site-packages/label_studio_sdk/_extensions/label_studio_tools/core/utils/io.py\\", line 166, in get_local_path\\n    filepath = download_and_cache(\\n               ^^^^^^^^^^^^^^^^^^^\\n  File \\"/Users/admin/Work/X-Brain/New_ML_Release/label-studio-ml-backend/label_studio_ml/examples/yolo/venv/lib/python3.12/site-packages/label_studio_sdk/_extensions/label_studio_tools/core/utils/io.py\\", line 221, in download_and_cache\\n    r.raise_for_status()\\n  File \\"/Users/admin/Work/X-Brain/New_ML_Release/label-studio-ml-backend/label_studio_ml/examples/yolo/venv/lib/python3.12/site-packages/requests/models.py\\", line 1024, in raise_for_status\\n    raise HTTPError(http_error_msg, response=self)\\nrequests.exceptions.HTTPError: 403 Client Error: Forbidden for url: http://localhost:8089/data/local-files/?d=input/63db415b7391984875aee6a7/workspaces/admin/756/files/categories/images/originals/1f742291bf201cfc4c95d5ab7ce3bfc5.jpg\\n"\n  },\n  "status": 500\n}\n'
[2024-11-04 16:16:50,589] [INFO] [werkzeug::_log::97] 127.0.0.1 - - [04/Nov/2024 16:16:50] "POST /predict HTTP/1.1" 500 -
@makseq
Copy link
Member

makseq commented Nov 5, 2024

How does your docker-compose.yml look?
Did you follow step #1 from https://github.com/HumanSignal/label-studio-ml-backend/blob/master/label_studio_ml/examples/yolo/README.md#quick-start ?

@SabolSocare
Copy link
Author

SabolSocare commented Nov 5, 2024 •
edited
Loading

Here's my Docker Compose configuration for the YOLO service:

It works when the image is uploaded from the browser, but it does not work with images synced from cloud storage.

version: "3.8"

services:
  yolo:
    container_name: yolo
    image: humansignal/yolo:v0
    build:
      context: .
      args:
        TEST_ENV: ${TEST_ENV}
    environment:
      # Optional basic auth for the model server
      - BASIC_AUTH_USER=
      - BASIC_AUTH_PASS=
      # Logging and server configuration
      - LOG_LEVEL=DEBUG
      - WORKERS=1
      - THREADS=8
      - MODEL_DIR=/data/models
      - PYTHONPATH=/app

      # Label Studio integration
      - LABEL_STUDIO_URL=http://127.0.0.1:8089
      - LABEL_STUDIO_API_KEY=9e06b3eca8f7edfc0f9b9XXXXXXXXXX

      # YOLO-specific configuration
      - ALLOW_CUSTOM_MODEL_PATH=true
      - DEBUG_PLOT=false
      - MODEL_SCORE_THRESHOLD=0.5
      - MODEL_ROOT=/app/models
    ports:
      - "9090:9090"
    volumes:
      - "./data/server:/data"
      - "./models:/app/models"
      - "./cache_dir:/app/cache_dir"

@makseq
Copy link
Member

makseq commented Nov 9, 2024 •
edited
Loading

LABEL_STUDIO_URL=http://127.0.0.1:8089

This line is the same as the use of localhost. When you are inside of the docker, it won't work. In the example docker compose configuration it was a comment about label studio url, please read it carefully. https://github.com/HumanSignal/label-studio-ml-backend/blob/master/label_studio_ml/examples/yolo/docker-compose.yml#L26

@SabolSocare
Copy link
Author

Thank you for the guidance. The URL http://127.0.0.1:8089 was just an example; I’m actually using my own domain, which works (auto-annotation) as expected with data under /data/upload. However, I'm encountering issues specifically with data paths prefixed by /data/local-files/? (Cloud Storage). It seems that these paths aren't accessible in the same way

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
2 participants
@makseq @SabolSocare