AWS Secrets Manager - How to get secret and use it in a template

[air3ijai]

Hello,

We are starting to use gomplate with data stored in AWS Secrets Manager, but looks that we need more information how to use it.

We have a secret test created in Virginia region and we have AWS CLI default profile with credentials.

Create secret

aws secretsmanager create-secret \
  --name test \
  --secret-string "{\"key1_name\":\"key1_value\",\"key2_name\":\"key2_value\"}"

aws cli

aws secretsmanager get-secret-value --secret-id test | jq -r .SecretString

{"key1_name":"key1_value","key2_name":"key2_value"}

gomplate

echo '{{ (ds "test") }}' | gomplate -d test=aws+sm:test

11:05:31 ERR  error="failed to render template -: template: -:1:4: executing \"-\" at <ds \"test\">: error calling ds: Couldn't read datasource 'test': Error reading aws+sm from AWS using GetSecretValue with input {\n  SecretId: \"test\"\n}: RequestError: send request failed\ncaused by: Post \"https://secretsmanager.us-east-1.amazonaws.com/\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"

1. How we can authenticate on AWS? Didn't find such information in the documentation - using-aws-sm-datasource
2. How we can get specific key name value from AWS Secrets Manager?
3. How we can reference to both keys in the template using a single in-line command?

[hairyhenderson]

Hi @air3ijai,

First off, the error you're getting looks like #1234. Try setting the AWS_TIMEOUT environment variable to a value like 2000 (milliseconds) to see if that works around the issue.

1. How we can authenticate on AWS? Didn't find such information in the documentation - using-aws-sm-datasource

Looks like that doc section is missing a link to https://docs.gomplate.ca/functions/aws/#configuring-aws - it's linked from the aws+smp docs, but I must've missed aws+sm!

2. How we can get specific key name value from AWS Secrets Manager?

I'm not sure I totally understand this question - are you referring to the key1_name JSON field names encoded in the secret's value? If so, there are two ways you could do this:

1. force the content type in the URL (see Overriding MIME Types), like aws+sm:test?type=application/json, at which point you should be able to reference the key directly (like (ds "test").key1_name), or...
2. parse the value in-line with the data.JSON function, like (data.JSON (ds "test")).key1_name

The first approach is obviously much simpler. And, if you use the context instead, it can be even simpler (gomplate -c test=aws+sm:test?type=application/json -i '{{ .test.key1_name }}').

3. How we can reference to both keys in the template using a single in-line command?

I'm not entirely sure I understand this question either, but if I was correct in my assumption above, that should answer this too...

[air3ijai]

@hairyhenderson, thank you for the hints - now it is more clear.

Variable solved the issue with the timeout:

# Timeout
export AWS_TIMEOUT=2000

# Authentication
export AWS_REGION=us-east-1
export AWS_PROFILE=staging


echo '{{ (data.JSON (ds "test")).key1_name }}' | gomplate -d test=aws+sm:test

key1_value

And we can get value from the JSON and refer in the template to the both keys using a single datasource

cat test.yaml

Test AWS Secrets: {{ (ds "test").key1_name }} - {{ (ds "test").key2_name }}

gomplate -d 'test=aws+sm:test?type=application/json' -f test.yaml

Test AWS Secrets: key1_value - key2_value

[hairyhenderson]

Great! I'm glad the issue is solved!