Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[BUG] inconsistent yaml support from file:// vs s3:// #1512

Open
austindrenski opened this issue Jan 15, 2025 · 1 comment · May be fixed by #1522
Open

[BUG] inconsistent yaml support from file:// vs s3:// #1512

austindrenski opened this issue Jan 15, 2025 · 1 comment · May be fixed by #1522
Assignees
@beeme1mr
Labels
bug Something isn't working good first issue Good for newcomers help wanted Extra attention is needed

Comments

@austindrenski
Copy link
Member

Observed behavior

Hi folks,

I've run into some trouble trying to load YAML flag definitions from S3. Looking through the docs and repo, it seems like this should be supported, and indeed it is for local files, but when trying to load the YAML file from S3, it appears that the JSON evaluator is being invoked without a call to something like yamlToJSON(...).

docker run --rm -it --mount type=bind,source=$(pwd)/flags.flagd.yaml,target=/flags.flagd.yaml ghcr.io/open-feature/flagd:v0.11.5 start --log-format json --uri file:/flags.flagd.yaml
{"level":"info","ts":"2025-01-14T23:57:48.739Z","caller":"file/filepath_sync.go:62","msg":"Starting filepath sync notifier","component":"sync","sync":"fileinfo"}
{"level":"info","ts":"2025-01-14T23:57:48.741Z","caller":"file/filepath_sync.go:101","msg":"watching filepath: /flags.flagd.yaml","component":"sync","sync":"fileinfo"}
docker run --rm -it ghcr.io/open-feature/flagd:v0.11.5 start --log-format json --uri s3://${BUCKET_NAME}/flags.flagd.yaml
{"level":"info","ts":"2025-01-14T22:54:49.922Z","caller":"blob/blob_sync.go:51","msg":"starting sync from s3://***//flags.flagd.yaml with interval 5s","component":"sync","sync":"s3"}
{"level":"warn","ts":"2025-01-14T22:54:54.923Z","caller":"flag-sync/sync_service.go:82","msg":"timeout while waiting for all sync sources to complete their initial sync. continuing sync service","component":"FlagSyncService"}
{"level":"warn","ts":"2025-01-14T22:54:55.011Z","caller":"evaluator/json.go:112","msg":"failed to execute JSON schema validation: invalid character '$' looking for beginning of value"}
{"level":"error","ts":"2025-01-14T22:54:55.013Z","caller":"runtime/runtime.go:146","msg":"transposing evaluators: unmarshal: invalid character '$' looking for beginning of value","component":"runtime","stacktrace":"github.com/open-feature/flagd/flagd/pkg/runtime.(*Runtime).updateAndEmit\n\t/src/flagd/pkg/runtime/runtime.go:146\ngithub.com/open-feature/flagd/flagd/pkg/runtime.(*Runtime).Start.func1\n\t/src/flagd/pkg/runtime/runtime.go:56\ngolang.org/x/sync/errgroup.(*Group).Go.func1\n\t/go/pkg/mod/golang.org/x/[email protected]/errgroup/errgroup.go:78"}

Here's a stripped down version of flags.flagd.yaml as it appeared at the time of these^ logs. The specific error messages relating to $ are obviously complaining about the $schema, but removing $schema still fails, just with different this-isnt-json errors. Also, just to reiterate, this file works when used via flagd start --uri file:flags.flagd.yaml

$schema: https://flagd.dev/schema/v0/flags.json

flags:
  some_flag:
    defaultVariant: "true"
    state: ENABLED
    variants:
      "true": "true"
      "false": "false"

Related: #180, #206

Expected Behavior

No response

Steps to reproduce

No response
@austindrenski austindrenski added bug Something isn't working Needs Triage This issue needs to be investigated by a maintainer labels Jan 15, 2025
@beeme1mr
Copy link
Member

Hey @austindrenski, yeah, I think you're correct. I remember testing JSON configures but not YAML. This was an oversight on my part and I agree it should behave like the file sync.

@beeme1mr beeme1mr added help wanted Extra attention is needed good first issue Good for newcomers and removed Needs Triage This issue needs to be investigated by a maintainer labels Jan 15, 2025
@beeme1mr beeme1mr self-assigned this Jan 17, 2025
@beeme1mr beeme1mr linked a pull request Jan 17, 2025 that will close this issue
Open
@beeme1mr beeme1mr linked a pull request Jan 17, 2025 that will close this issue
fix: support yaml in blob syncs #1522
Open
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@beeme1mr beeme1mr

Labels
bug Something isn't working good first issue Good for newcomers help wanted Extra attention is needed
Projects
None yet
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

fix: support yaml in blob syncs open-feature/flagd
2 participants
@beeme1mr @austindrenski