Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

mc mirror --remove --watch removes a file when a non-current version of the object is removed #5139

Open
oh2fih opened this issue Feb 13, 2025 · 0 comments · May be fixed by #5149
Open

mc mirror --remove --watch removes a file when a non-current version of the object is removed #5139

oh2fih opened this issue Feb 13, 2025 · 0 comments · May be fixed by #5149

Comments

@oh2fih
Copy link

oh2fih commented Feb 13, 2025

Expected behavior

  1. A MinIO bucket has versioning.
  2. An older (non-current) version of an object is deleted from a MinIO bucket. (Either manually or by ILMExpiry.)
  3. mc mirror --remove --watch watching for events from the bucket receives an ObjectRemoved:Delete event.
  4. As the event is not related to the current version of the object, the object is not removed from the destination directory.

Actual behavior

  1. A MinIO bucket has versioning.
  2. An older (non-current) version of an object is deleted from a MinIO bucket. (Either manually or by ILMExpiry.)
  3. mc mirror --remove --watch watching for events from the bucket receives an ObjectRemoved:Delete event.
  4. mc mirror does not validate the object is actually removed from the source and deletes it from the destination.

Steps to reproduce the behavior

  1. Add alias for the MinIO server:

    export MC_HOST_alias=...
    
  2. Enable versioning on your test bucket:

    mc version enable alias/bucket
    
  3. Create a local test file:

    echo "version 1" > test.txt`
    
  4. Copy it to the bucket:

    mc cp test.txt alias/bucket/test.txt
    
  5. You could alter the test file for clarity:

    echo "version 2" > test.txt
    
  6. Copy it again to have two versions of the object:

    mc cp test.txt alias/bucket/test.txt
    
  7. Leave mc mirror --remove --watch running (this is where the bug is):

    mc mirror --overwrite --remove --watch alias/bucket /path/to/local/directory
    
  8. List versions of the test file:

    mc ls --versions alias/bucket/test.txt
    
  9. Remove the older (non-current) version of the object:

    mc rm --version=UUID alias/bucket/test.txt
    
  10. Observe how the /path/to/local/directory/test.txt gets removed despite the current version of the object still exists.

Possible fix

After receiving an ObjectRemoved:Delete event, mc mirror could validate with a HEAD request whether the object still exist of not. This should work both with and without versioning and only takes one extra round-trip.

mc --version

mc version 2025-02-08T19-14-21Z
Runtime: go1.23.2 linux/amd64

System information

Ubuntu 22.04; 6.8.0-51-generic.

@dormanze dormanze linked a pull request Feb 24, 2025 that will close this issue
8 tasks
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging a pull request may close this issue.

1 participant