Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

docs(thanos): add migration doc for thanos storage clients #16133

Open
wants to merge 9 commits into
base: main
Choose a base branch
from

Conversation

ashwanthgoli
Copy link
Contributor

What this PR does / why we need it:

Adds a migration doc to enable Thanos storage clients. These will replace the existing storage clients in a future release.
I have documented the configuration changes required to migrate clients for commonly used storage backends such as gcs, s3 and azure.

Which issue(s) this PR fixes:
Fixes #

Special notes for your reviewer:

Checklist

  • Reviewed the CONTRIBUTING.md guide (required)
  • Documentation added
  • Tests updated
  • Title matches the required conventional commits format, see here
    • Note that Promtail is considered to be feature complete, and future development for logs collection will be in Grafana Alloy. As such, feat PRs are unlikely to be accepted unless a case can be made for the feature actually being a bug fix to existing behavior.
  • Changes that require user attention or interaction to upgrade are documented in docs/sources/setup/upgrade/_index.md
  • If the change is deprecating or removing a configuration option, update the deprecated-config.yaml and deleted-config.yaml files respectively in the tools/deprecated-config-checker directory. Example PR

@ashwanthgoli ashwanthgoli requested a review from a team as a code owner February 6, 2025 13:50
@github-actions github-actions bot added the type/docs Issues related to technical documentation; the Docs Squad uses this label across many repositories label Feb 6, 2025
Copy link
Contributor

github-actions bot commented Feb 6, 2025

Copy link
Contributor

@JStickler JStickler left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[docs team] Nice job. I added a couple of links, and made some suggestions for revisions, but overall this looks good.


Loki 3.4 release introduces new object storage clients based on [Thanos Object Storage Client Go module](https://github.com/thanos-io/objstore).

One of the reasons for making this change is to have a consistent storage configuration across LGTM+ stack. If you are already using Grafana Mimir or Pyroscope, you can reuse the storage configuration for setting up Loki.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
One of the reasons for making this change is to have a consistent storage configuration across LGTM+ stack. If you are already using Grafana Mimir or Pyroscope, you can reuse the storage configuration for setting up Loki.
One of the reasons for making this change is to have a consistent storage configuration across the entire Loki, Grafana, Tempo, and Loki stack (LGTM+). If you are already using Grafana Mimir or Pyroscope, you can reuse the same storage configuration for setting up Loki.

Copy link
Contributor

@JStickler JStickler left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[docs team] Thanks for adding examples, users are always asking for more examples.

docs/sources/configure/examples/thanos-storage-configs.md Outdated Show resolved Hide resolved
gcs:
bucket_name: my-gcs-bucket

# JSON either from a Google Developers Console client_credentials.json file,
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This comment is confusing to me. Does this whole comment apply to the service_account? Shouldn't the input be a file.json of some sort?
Is there a typo at line 30? Should it read service_account: service-account.json instead?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

JSON either from a Google Developers Console client_credentials.json file,
or a Google Developers service account key. Needs to be valid JSON, not a
filesystem path.

only this portion of the comment applies to the service_account field. It expects the json contents, not a file path.

Rest of the comment mentions how the google SDK picks up credentials either from env or known file locations.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

i took this content from our configuration reference, didn't trim it down for the sake of completeness. we could rewrite this in both places if its confusing :)
https://grafana.com/docs/loki/next/configure/#thanos_object_store_config

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think I read this over half a dozen times, and while the commented text made sense to me, I was having trouble match it to the parameter. But reading it over again, I think the commented text is clear enough, i was just stuck trying to match it to the example parameter.

If you don't want to rewrite the comment, maybe change line 31 to read
service_account: your-service-account.json ?

docs/sources/configure/examples/thanos-storage-configs.md Outdated Show resolved Hide resolved
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
size/L type/docs Issues related to technical documentation; the Docs Squad uses this label across many repositories
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants