Enable vault name mapping and error suppression (dapr#1231)

* Added documentation detailing how serialization works using the DataContract serialization framework. (dapr#1222) Signed-off-by: Whit Waldo <[email protected]> Signed-off-by: James Croft <[email protected]> * Weakly typed actor polymorphic and null responses (dapr#1214) Signed-off-by: Remco Blok <[email protected]> Co-authored-by: Remco Blok <[email protected]> Co-authored-by: Phillip Hoff <[email protected]> Signed-off-by: James Croft <[email protected]> * Enable vault name mapping and error suppression Signed-off-by: Yash Nisar <[email protected]> Signed-off-by: James Croft <[email protected]> * Add additional secret descriptor constructor for required without key map Signed-off-by: James Croft <[email protected]> * Update configuration load exception rethrow to match rules Signed-off-by: James Croft <[email protected]> * Add tests for required/not required exception handling Signed-off-by: James Croft <[email protected]> * Implementing Cryptography building block in .NET (dapr#1217) * Added method to DaprClient and GRPC implementation to call cryptography proto endpoints Signed-off-by: Whit Waldo <[email protected]> * First pass at implementing all exposed Cryptography methods on Go interface Signed-off-by: Whit Waldo <[email protected]> * Added examples for Cryptography block Signed-off-by: Whit Waldo <[email protected]> * Added missing copyright statements Signed-off-by: Whit Waldo <[email protected]> * Updated to properly support Crypto API this time Signed-off-by: Whit Waldo <[email protected]> * Added copyright statements Signed-off-by: Whit Waldo <[email protected]> * Removed deprecated examples as the subtle APIs are presently disabled Signed-off-by: Whit Waldo <[email protected]> * Updated example to reflect new API shape Signed-off-by: Whit Waldo <[email protected]> * Updated example and readme Signed-off-by: Whit Waldo <[email protected]> * Added overloads for encrypting/decrypting streams instead of just fixed byte arrays. Added example demonstrating the same encrypting a file via a FileStream and decrypting from a MemoryStream. Signed-off-by: Whit Waldo <[email protected]> * Added some unit tests to pair with the implementation Signed-off-by: Whit Waldo <[email protected]> * Added null check for the stream argument Signed-off-by: Whit Waldo <[email protected]> * Changed case of the arguments as they should read "plaintext" and not "plainText" Signed-off-by: Whit Waldo <[email protected]> * Reduced number of encryption implementations by just wrapping byte array into memory stream Signed-off-by: Whit Waldo <[email protected]> * Constrainted returned member types per review suggestion Signed-off-by: Whit Waldo <[email protected]> * Updated methods to use ReadOnlyMemory<byte> instead of byte[] - updated implementations to use low-allocation spans where possible (though ToArray is necessary to wrap with MemoryStream). Signed-off-by: Whit Waldo <[email protected]> * Updated to use encryption/decryption options instead of lots of method overload variations. Simplified gRPC implementation to use fewer methods. Applied argument name updates applied previously (plainText -> plaintext). Signed-off-by: Whit Waldo <[email protected]> * Updated tests Signed-off-by: Whit Waldo <[email protected]> * Removed unused reference Signed-off-by: Whit Waldo <[email protected]> * Updated examples to reflect new method shapes. Downgraded package to .net 6 instead of .net 8 per review suggestion. Signed-off-by: Whit Waldo <[email protected]> * Updated to reflect non-aliased values per review suggestion Signed-off-by: Whit Waldo <[email protected]> * Update to ensure that both send/receive streams run at the same time instead of sequentially. Signed-off-by: Whit Waldo <[email protected]> * Updated to support streamed results in addition to fixed byte arrays. Refactored implementation to minimize duplicative code. Signed-off-by: Whit Waldo <[email protected]> * Updated example to fix compile issue Signed-off-by: Whit Waldo <[email protected]> * Removed encrypt/decrypt methods that accepted streams and returned ReadOnlyMemory<byte>. Marked implementations that use this on the gRPC class as private instead. Signed-off-by: Whit Waldo <[email protected]> * Added missing Obsolete attributes on Encrypt/Decrypt methods. Added overloads on decrypt methods that do not require a DecryptionOptions to be passed in. Signed-off-by: Whit Waldo <[email protected]> * Updated encrypt/decrypt options so the streaming block size no longer uses a uint. Added validation in its place to ensure the value provided is never less than or equal to 0. Signed-off-by: Whit Waldo <[email protected]> * Updated how validation works in the options to accommodate lack of the shorter variation in .NET 6 Signed-off-by: Whit Waldo <[email protected]> * Updated names of encrypt/decrypt streaming methods so everything uses just EncryptAsync or DecryptAsync Signed-off-by: Whit Waldo <[email protected]> * Fixed regression that would have prevented data from being sent entirely to the sidecar. Also simplified operation per suggestion in review. Signed-off-by: Whit Waldo <[email protected]> * Updated examples to reflect changed API Signed-off-by: Whit Waldo <[email protected]> * Updated so IAsyncEnumerable methods (encrypt and decrypt) return IAsyncEnumerable<ReadOnlyMemory<byte>> instead of IAsyncEnumerable<byte[]>. Signed-off-by: Whit Waldo <[email protected]> * Updated example to reflect change from IAsyncEnumerable<byte> to IAsyncEnumerable<ReadOnlyMemory<byte>> Signed-off-by: Whit Waldo <[email protected]> * Avoiding allocation by using MemoryMarshal instead of .ToArray() to create MemoryStream from ReadOnlyMemory<byte>. Signed-off-by: Whit Waldo <[email protected]> * Performance updates to minimize unnecessary byte array copies and eliminate unnecessary allocations. Signed-off-by: Whit Waldo <[email protected]> * Removed unnecessary return from SendPlaintextStreamAsync and SendCiphertextStreamAsync methods Signed-off-by: Whit Waldo <[email protected]> * Updated exception text to be more specific as to what's wrong with the input value. Signed-off-by: Whit Waldo <[email protected]> * Minor tweak to prefer using using a Memory Signed-off-by: Whit Waldo <[email protected]> * Deduplicated some of the Decrypt methods, simplifying the implementation Signed-off-by: Whit Waldo <[email protected]> * Eliminated duplicate encryption method, simplifying implementation Signed-off-by: Whit Waldo <[email protected]> * Updated to eliminate an unnecessary `await` and `async foreach`. Signed-off-by: Whit Waldo <[email protected]> * Updated stream example to reflect the changes to the API shape Signed-off-by: Whit Waldo <[email protected]> * Added notes about operations with stream-based data Signed-off-by: Whit Waldo <[email protected]> --------- Signed-off-by: Whit Waldo <[email protected]> Signed-off-by: James Croft <[email protected]> * Update DaprSecretDescriptor constructors and documentation Signed-off-by: James Croft Signed-off-by: James Croft <[email protected]> * Remove DaprSecretStoreConfigurationProvider Console.WriteLine Signed-off-by: James Croft <[email protected]> --------- Signed-off-by: Whit Waldo <[email protected]> Signed-off-by: James Croft <[email protected]> Signed-off-by: Remco Blok <[email protected]> Signed-off-by: Yash Nisar <[email protected]> Signed-off-by: James Croft Co-authored-by: Whit Waldo <[email protected]> Co-authored-by: Remco Blok <[email protected]> Co-authored-by: Remco Blok <[email protected]> Co-authored-by: Phillip Hoff <[email protected]> Co-authored-by: Yash Nisar <[email protected]> Signed-off-by: Divya Perumal <[email protected]>
divzi-p · Dec 10, 2024 · 7bf1fae · 7bf1fae
1 parent 38bc34b
commit 7bf1fae
Show file tree

Hide file tree

Showing 4 changed files with 181 additions and 40 deletions.
diff --git a/src/Dapr.Extensions.Configuration/DaprSecretDescriptor.cs b/src/Dapr.Extensions.Configuration/DaprSecretDescriptor.cs
@@ -1,4 +1,4 @@
-// ------------------------------------------------------------------------
+// ------------------------------------------------------------------------
 // Copyright 2021 The Dapr Authors
 // Licensed under the Apache License, Version 2.0 (the "License");
 // you may not use this file except in compliance with the License.
@@ -21,8 +21,11 @@ namespace Dapr.Extensions.Configuration
     public class DaprSecretDescriptor
     {
         /// <summary>
-        /// Gets or sets the secret name.
+        /// The name of the secret to retrieve from the Dapr secret store.
         /// </summary>
+        /// <remarks>
+        /// If the <see cref="SecretKey"/> is not specified, this value will also be used as the key to retrieve the secret from the associated source secret store.
+        /// </remarks>
         public string SecretName { get; }
 
         /// <summary>
@@ -31,20 +34,39 @@ public class DaprSecretDescriptor
         public IReadOnlyDictionary<string, string> Metadata { get; }
 
         /// <summary>
-        /// Secret Descriptor Construcutor
+        /// A value indicating whether to throw an exception if the secret is not found in the source secret store.
+        /// </summary>
+        /// <remarks>
+        /// Setting this value to <see langword="false"/> will suppress the exception; otherwise, <see langword="true"/> will not.
+        /// </remarks>
+        public bool IsRequired { get; }
+
+        /// <summary>
+        /// The secret key that maps to the <see cref="SecretName"/> to retrieve from the source secret store.
+        /// </summary>
+        /// <remarks>
+        /// Use this property when the <see cref="SecretName"/> does not match the key used to retrieve the secret from the source secret store.
+        /// </remarks>
+        public string SecretKey { get; }
+
+        /// <summary>
+        /// Secret Descriptor Constructor
         /// </summary>
-        public DaprSecretDescriptor(string secretName) : this(secretName, new Dictionary<string, string>())
+        public DaprSecretDescriptor(string secretName, bool isRequired = true, string secretKey = "") 
+            : this(secretName, new Dictionary<string, string>(), isRequired, secretKey)
         {
 
         }
 
         /// <summary>
-        /// Secret Descriptor Construcutor
+        /// Secret Descriptor Constructor
         /// </summary>
-        public DaprSecretDescriptor(string secretName, IReadOnlyDictionary<string, string> metadata)
+        public DaprSecretDescriptor(string secretName, IReadOnlyDictionary<string, string> metadata, bool isRequired = true, string secretKey = "")
         {
             SecretName = secretName;
             Metadata = metadata;
+            IsRequired = isRequired;
+            SecretKey = string.IsNullOrEmpty(secretKey) ? secretName : secretKey;
         }
     }
-}
+}
diff --git a/src/Dapr.Extensions.Configuration/DaprSecretStoreConfigurationProvider.cs b/src/Dapr.Extensions.Configuration/DaprSecretStoreConfigurationProvider.cs
@@ -1,4 +1,4 @@
-// ------------------------------------------------------------------------
+// ------------------------------------------------------------------------
 // Copyright 2021 The Dapr Authors
 // Licensed under the Apache License, Version 2.0 (the "License");
 // you may not use this file except in compliance with the License.
@@ -54,7 +54,7 @@ public DaprSecretStoreConfigurationProvider(
             bool normalizeKey,
             IEnumerable<DaprSecretDescriptor> secretDescriptors,
             DaprClient client) : this(store, normalizeKey, null, secretDescriptors, client, DefaultSidecarWaitTimeout)
-        {            
+        {
         }
 
         /// <summary>
@@ -181,6 +181,10 @@ private string NormalizeKey(string key)
             return key;
         }
 
+        /// <summary>
+        /// Loads the configuration by calling the asynchronous LoadAsync method and blocking the calling
+        /// thread until the operation is completed.
+        /// </summary>
         public override void Load() => LoadAsync().ConfigureAwait(false).GetAwaiter().GetResult();
 
         private async Task LoadAsync()
@@ -197,16 +201,34 @@ private async Task LoadAsync()
             {
                 foreach (var secretDescriptor in secretDescriptors)
                 {
-                    var result = await client.GetSecretAsync(store, secretDescriptor.SecretName, secretDescriptor.Metadata).ConfigureAwait(false);
+
+                    Dictionary<string, string> result;
+
+                    try
+                    {
+                        result = await client
+                            .GetSecretAsync(store, secretDescriptor.SecretKey, secretDescriptor.Metadata)
+                            .ConfigureAwait(false);
+                    }
+                    catch (DaprException)
+                    {
+                        if (secretDescriptor.IsRequired)
+                        {
+                            throw;
+                        }
+                        result = new Dictionary<string, string>();
+                    }
 
                     foreach (var key in result.Keys)
                     {
                         if (data.ContainsKey(key))
                         {
-                            throw new InvalidOperationException($"A duplicate key '{key}' was found in the secret store '{store}'. Please remove any duplicates from your secret store.");
+                            throw new InvalidOperationException(
+                                $"A duplicate key '{key}' was found in the secret store '{store}'. Please remove any duplicates from your secret store.");
                         }
 
-                        data.Add(normalizeKey ? NormalizeKey(key) : key, result[key]);
+                        data.Add(normalizeKey ? NormalizeKey(secretDescriptor.SecretName) : secretDescriptor.SecretName,
+                            result[key]);
                     }
                 }
 

diff --git a/test/Dapr.Extensions.Configuration.Test/Dapr.Extensions.Configuration.Test.csproj b/test/Dapr.Extensions.Configuration.Test/Dapr.Extensions.Configuration.Test.csproj
@@ -7,6 +7,7 @@
     </PackageReference>
     <PackageReference Include="FluentAssertions" Version="5.9.0" />
     <PackageReference Include="Microsoft.NET.Test.Sdk" Version="16.8.3" />
+    <PackageReference Include="Moq" Version="4.18.4" />
     <PackageReference Include="xunit" Version="2.4.1" />
     <PackageReference Include="xunit.runner.visualstudio" Version="2.4.1">
       <PrivateAssets>all</PrivateAssets>

diff --git a/test/Dapr.Extensions.Configuration.Test/DaprSecretStoreConfigurationProviderTest.cs b/test/Dapr.Extensions.Configuration.Test/DaprSecretStoreConfigurationProviderTest.cs
@@ -1,4 +1,4 @@
-// ------------------------------------------------------------------------
+// ------------------------------------------------------------------------
 // Copyright 2021 The Dapr Authors
 // Licensed under the Apache License, Version 2.0 (the "License");
 // you may not use this file except in compliance with the License.
@@ -19,6 +19,7 @@
 using FluentAssertions;
 using Grpc.Net.Client;
 using Microsoft.Extensions.Configuration;
+using Moq;
 using Xunit;
 using Autogenerated = Dapr.Client.Autogen.Grpc.v1;
 
@@ -142,23 +143,23 @@ public void AddDaprSecretStore_UsingDescriptors_DuplicateSecret_ReportsError()
         [Fact]
         public void LoadSecrets_FromSecretStoreThatReturnsOneValue()
         {
-            // Configure Client
-            var httpClient = new TestHttpClient()
+            var storeName = "store";
+            var secretKey = "secretName";
+            var secretValue = "secret";
+
+            var secretDescriptors = new[]
             {
-                Handler = async (entry) =>
-                   {
-                       var secrets = new Dictionary<string, string>() { { "secretName", "secret" } };
-                       await SendResponseWithSecrets(secrets, entry);
-                   }
+                new DaprSecretDescriptor(secretKey),
             };
 
-            var daprClient = new DaprClientBuilder()
-                .UseHttpClientFactory(() => httpClient)
-                .UseGrpcChannelOptions(new GrpcChannelOptions { HttpClient = httpClient })
-                .Build();
+            var daprClient = new Mock<DaprClient>();
+
+            daprClient.Setup(c => c.GetSecretAsync(storeName, secretKey,
+                    It.IsAny<Dictionary<string, string>>(), default))
+                .ReturnsAsync(new Dictionary<string, string> { { secretKey, secretValue } });
 
             var config = CreateBuilder()
-                    .AddDaprSecretStore("store", new DaprSecretDescriptor[] { new DaprSecretDescriptor("secretName") }, daprClient)
+                    .AddDaprSecretStore(storeName, secretDescriptors, daprClient.Object)
                     .Build();
 
             config["secretName"].Should().Be("secret");
@@ -167,32 +168,127 @@ public void LoadSecrets_FromSecretStoreThatReturnsOneValue()
         [Fact]
         public void LoadSecrets_FromSecretStoreThatCanReturnsMultipleValues()
         {
-            // Configure Client
-            var httpClient = new TestHttpClient()
+            var storeName = "store";
+            var firstSecretKey = "first_secret";
+            var secondSecretKey = "second_secret";
+            var firstSecretValue = "secret1";
+            var secondSecretValue = "secret2";
+
+            var secretDescriptors = new[]
             {
-                Handler = async (entry) =>
-                   {
-                       var secrets = new Dictionary<string, string>() {
-                           { "first_secret", "secret1" },
-                           { "second_secret", "secret2" }};
-                       await SendResponseWithSecrets(secrets, entry);
-                   }
+                new DaprSecretDescriptor(firstSecretKey),
+                new DaprSecretDescriptor(secondSecretKey),
+            };
+
+            var daprClient = new Mock<DaprClient>();
+
+            daprClient.Setup(c => c.GetSecretAsync(storeName, firstSecretKey,
+                    It.IsAny<Dictionary<string, string>>(), default))
+                .ReturnsAsync(new Dictionary<string, string> { { firstSecretKey, firstSecretValue } });
+
+            daprClient.Setup(c => c.GetSecretAsync(storeName, secondSecretKey,
+                    It.IsAny<Dictionary<string, string>>(), default))
+                .ReturnsAsync(new Dictionary<string, string> { { secondSecretKey, secondSecretValue } });
+
+            var config = CreateBuilder()
+                .AddDaprSecretStore(storeName, secretDescriptors, daprClient.Object)
+                .Build();
+
+            config[firstSecretKey].Should().Be(firstSecretValue);
+            config[secondSecretKey].Should().Be(secondSecretValue);
+        }
+
+        [Fact]
+        public void LoadSecrets_FromSecretStoreWithADifferentSecretKeyAndName()
+        {
+            var storeName = "store";
+            var secretKey = "Microsservice-DatabaseConnStr";
+            var secretName = "ConnectionStrings:DatabaseConnStr";
+            var secretValue = "secret1";
+
+            var secretDescriptors = new[]
+            {
+                new DaprSecretDescriptor(secretName, new Dictionary<string, string>(), true,
+                    secretKey)
+            };
+
+            var daprClient = new Mock<DaprClient>();
+
+            daprClient.Setup(c => c.GetSecretAsync(storeName, secretKey,
+                    It.IsAny<Dictionary<string, string>>(), default))
+                .ReturnsAsync(new Dictionary<string, string> { { secretKey, secretValue } });
+
+            var config = CreateBuilder()
+                .AddDaprSecretStore(storeName, secretDescriptors, daprClient.Object)
+                .Build();
+
+            config[secretName].Should().Be(secretValue);
+        }
+
+        [Fact]
+        public void LoadSecrets_FromSecretStoreNotRequiredAndDoesNotExist_ShouldNotThrowException()
+        {
+            var storeName = "store";
+            var secretName = "ConnectionStrings:DatabaseConnStr";
+
+            var secretDescriptors = new[]
+            {
+                new DaprSecretDescriptor(secretName, new Dictionary<string, string>(), false)
+            };
+
+            var httpClient = new TestHttpClient
+            {
+                Handler = async entry =>
+                {
+                    await SendEmptyResponse(entry);
+                }
             };
 
             var daprClient = new DaprClientBuilder()
                 .UseHttpClientFactory(() => httpClient)
-                .UseGrpcChannelOptions(new GrpcChannelOptions { HttpClient = httpClient })
                 .Build();
 
             var config = CreateBuilder()
-                    .AddDaprSecretStore("store", new DaprSecretDescriptor[] { new DaprSecretDescriptor("secretName") }, daprClient)
+                .AddDaprSecretStore(storeName, secretDescriptors, daprClient)
+                .Build();
+
+            config[secretName].Should().BeNull();
+        }
+
+        [Fact]
+        public void LoadSecrets_FromSecretStoreRequiredAndDoesNotExist_ShouldThrowException()
+        {
+            var storeName = "store";
+            var secretName = "ConnectionStrings:DatabaseConnStr";
+
+            var secretDescriptors = new[]
+            {
+                new DaprSecretDescriptor(secretName, new Dictionary<string, string>(), true)
+            };
+
+            var httpClient = new TestHttpClient
+            {
+                Handler = async entry =>
+                {
+                    await SendEmptyResponse(entry);
+                }
+            };
+
+            var daprClient = new DaprClientBuilder()
+                .UseHttpClientFactory(() => httpClient)
+                .Build();
+
+            var ex = Assert.Throws<DaprException>(() =>
+            {
+                var config = CreateBuilder()
+                    .AddDaprSecretStore(storeName, secretDescriptors, daprClient)
                     .Build();
+            });
 
-            config["first_secret"].Should().Be("secret1");
-            config["second_secret"].Should().Be("secret2");
+            Assert.Contains("Secret", ex.Message);
         }
 
-        //Here
+
         [Fact]
         public void AddDaprSecretStore_WithoutStore_ReportsError()
         {
@@ -565,7 +661,7 @@ await SendResponseWithSecrets(new Dictionary<string, string>()
                                 ["otherSecretName≡value"] = "secret",
                             }, entry);
                         }
-                    }                    
+                    }
                 }
             };