Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix: Walk shouldn't return empty baseName (i.e. object representing the directory itself) #19

Open
wants to merge 1 commit into
base: develop
Choose a base branch
from

Conversation

gldeng
Copy link

@gldeng gldeng commented Jan 10, 2025

Fix to the following issue:

Issue: Empty baseName causes Merger, Relayer and Firehose services to fail.
Root Cause: When walking GCP bucket directory, it returns the directory as the first object

panic: wrong filename format: ""

goroutine 101 [running]:
github.com/streamingfast/bstream.MustNewOneBlockFile(...)
        /home/runner/go/pkg/mod/github.com/streamingfast/[email protected]/oneblockfile.go:107
github.com/streamingfast/firehose-core/merger.(*ForkAwareDStoreIO).DeleteForkedBlocksAsync.func1({0xc00114b5f0?, 0xc00114b5f0?})
        /home/runner/go/pkg/mod/github.com/streamingfast/[email protected]/merger/merger_io.go:317 +0x13c
github.com/streamingfast/dstore.(*GSStore).WalkFrom(0xc000702ba0, {0x3144838, 0x4aef260}, {0x0, 0x0}, {0x0, 0x0}, 0xc0023fefa8)
        /home/runner/go/pkg/mod/github.com/streamingfast/[email protected]/gsstore.go:291 +0x727
github.com/streamingfast/firehose-core/merger.(*ForkAwareDStoreIO).DeleteForkedBlocksAsync(0xc00093f500, 0x1, 0x0)
        /home/runner/go/pkg/mod/github.com/streamingfast/[email protected]/merger/merger_io.go:313 +0xdb
github.com/streamingfast/firehose-core/merger.(*Merger).startForkedBlocksPruner.func1()
        /home/runner/go/pkg/mod/github.com/streamingfast/[email protected]/merger/merger.go:104 +0xa2
created by github.com/streamingfast/firehose-core/merger.(*Merger).startForkedBlocksPruner in goroutine 97
        /home/runner/go/pkg/mod/github.com/streamingfast/[email protected]/merger/merger.go:97 +0x225

…e as the first item when listing the directory
@maoueh
Copy link
Contributor

maoueh commented Jan 10, 2025

Is this happening because you are using a GCP bucket directly without any "directory"?

@gldeng gldeng changed the title Fix: Walk shouldn't return empty baseName (i.e. object representing the directory itself0 Fix: Walk shouldn't return empty baseName (i.e. object representing the directory itself) Jan 11, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants