Add support for decimal logical type (#151)

* Add support for decimal logical type

https://avro.apache.org/docs/1.11.1/specification/#decimal

* DecimalType to use precision and scale from schema

* Make decimal logical type conditional by Avro::VERSION

* Set value class to Numeric

* Exclude strings from supported inputs

* Remove support for Rational

* Better method naming to follow convention for boolean methods

* Update README.md with decimal type

* Restrict decimal type to BigDecimal, Float and Integer

* Fix input and value classes for DecimalType

CHANGELOG.md

@@ -1,5 +1,8 @@
 # avromatic changelog
 
+## 4.3.0
+- Add support for decimal logical type
+
 ## 4.2.0
 - Add an `Avromatic.eager_load_models` attribute reader method.
 - Remove unnecessary files from the gem distribution.

README.md

@@ -8,7 +8,7 @@
 `Avromatic` generates Ruby models from [Avro](http://avro.apache.org/) schemas
 and provides utilities to encode and decode them.
 
-**This README reflects Avromatic 2.0. Please see the 
+**This README reflects Avromatic 2.0. Please see the
 [1-0-stable](https://github.com/salsify/avromatic/blob/1-0-stable/README.md) branch for Avromatic 1.0.**
 
 ## Installation
@@ -41,7 +41,7 @@ Avromatic with unreleased Avro features.
 * **schema_store**: A schema store is required to load Avro schemas from the filesystem.
   It should be an object that responds to `find(name, namespace = nil)` and
   returns an `Avro::Schema` object. An `AvroTurf::SchemaStore` can be used.
-  The `schema_store` is unnecessary if models are generated directly from 
+  The `schema_store` is unnecessary if models are generated directly from
   `Avro::Schema` objects. See [Models](#models).
 * **nested_models**: An optional [ModelRegistry](https://github.com/salsify/avromatic/blob/master/lib/avromatic/model_registry.rb)
   that is used to store, by full schema name, the generated models that are
@@ -53,28 +53,28 @@ Avromatic with unreleased Avro features.
   option is useful for defining models that will be extended when the load order
   is important.
 * **allow_unknown_attributes**: Optionally allow model constructors to silently
-  ignore unknown attributes. Defaults to `false`. WARNING: Setting this to `true` 
-  will result in incorrect union member coercions if an earlier union member is 
+  ignore unknown attributes. Defaults to `false`. WARNING: Setting this to `true`
+  will result in incorrect union member coercions if an earlier union member is
   satisfied by a subset of the latter union member's attributes.
 
 #### Custom Types
 
 See the section below on configuring [Custom Types](#custom-type-configuration).
 
 #### Using a Schema Registry/Messaging API
- 
-The configuration options below are required when using a schema registry 
+
+The configuration options below are required when using a schema registry
 (see [Confluent Schema Registry](http://docs.confluent.io/2.0.1/schema-registry/docs/intro.html))
 and the [Messaging API](#messaging-api).
-  
+
 * **schema_registry**: An `AvroSchemaRegistry::Client` or
   `AvroTurf::ConfluentSchemaRegistry` object used to store Avro schemas
   so that they can be referenced by id. Either `schema_registry` or
   `registry_url` must be configured. If using `build_schema_registry!`, only
   `registry_url` is required. See example below.
 * **registry_url**: URL for the schema registry. This must be configured when using
-  `build_schema_registry!`. The `build_schema_registry!` method may 
-  be used to create a caching schema registry client instance based on other 
+  `build_schema_registry!`. The `build_schema_registry!` method may
+  be used to create a caching schema registry client instance based on other
   configuration values.
 * **use_schema_fingerprint_lookup**: Avromatic supports a Schema Registry
   [extension](https://github.com/salsify/avro-schema-registry#extensions) that
@@ -113,9 +113,9 @@ to call both.
 
 #### Encoding
 * **use_custom_datum_writer**: `Avromatic` includes a modified subclass of
-  `Avro::IO::DatumWriter`. This subclass supports caching avro encodings for 
-  immutable models and uses additional information about the index of union 
-  members to optimize the encoding of Avro messages. By default this 
+  `Avro::IO::DatumWriter`. This subclass supports caching avro encodings for
+  immutable models and uses additional information about the index of union
+  members to optimize the encoding of Avro messages. By default this
   information is included in the hash passed to the encoder but can be omitted
   by setting this option to `false`.
 
@@ -138,7 +138,7 @@ instance = MyModel.new(id: 123, name: 'Tesla Model 3', enabled: true)
 instance.name # => "Tesla Model 3"
 
 # Models are immutable by default
-instance.name = 'Tesla Model X' # => NoMethodError (private method `name=' called for #<MyModel:0x00007ff711e64e60>) 
+instance.name = 'Tesla Model X' # => NoMethodError (private method `name=' called for #<MyModel:0x00007ff711e64e60>)
 
 # Booleans can also be accessed by '?' readers that coerce nil to false
 instance.enabled? # => true
@@ -309,14 +309,14 @@ end
 ```
 
 The full name of the type and an optional class may be specified. When a class is
-provided then values for attributes of that type are defined using the specified 
+provided then values for attributes of that type are defined using the specified
 class.
 
 If the provided class responds to the class methods `from_avro` and `to_avro`
-then those methods are used to convert values when assigning to the model and 
+then those methods are used to convert values when assigning to the model and
 before encoding using Avro respectively.
 
-`from_avro` and `to_avro` methods may be also be specified as Procs when 
+`from_avro` and `to_avro` methods may be also be specified as Procs when
 registering the type:
 
 ```ruby
@@ -331,7 +331,7 @@ end
 Nil handling is not required as the conversion methods are not be called if the
 inbound or outbound value is nil.
 
-If a custom type is registered for a record-type field, then any `to_avro` 
+If a custom type is registered for a record-type field, then any `to_avro`
 method/Proc should return a Hash with string keys for encoding using Avro.
 
 ### Encoding and Decoding
@@ -364,7 +364,7 @@ MyModel.avro_raw_decode(key: encoded_key, value: encoded_value)
 ```
 
 If the attributes where encoded using a different version of the model's schemas,
-then a new model instance can be created by also providing the schemas used to 
+then a new model instance can be created by also providing the schemas used to
 encode the data:
 
 ```ruby
@@ -417,7 +417,7 @@ MyTopic.register_schemas!
 #### Avromatic::Model::MessageDecoder
 
 A stream of messages encoded from various models using the messaging approach
-can be decoded using `Avromatic::Model::MessageDecoder`. The decoder must be 
+can be decoded using `Avromatic::Model::MessageDecoder`. The decoder must be
 initialized with the list of models to decode:
 
 ```ruby
@@ -447,12 +447,13 @@ The following coercions are supported:
 | Date, Time, DateTime | date |
 | Time, DateTime | timestamp-millis |
 | Time, DateTime | timestamp-micros |
+| Float, Integer, BigDecimal | decimal |
 | TrueClass, FalseClass | boolean |
 | NilClass | null |
 | Hash | record |
 
 Validation of required fields is done automatically when serializing a model to Avro. It can also be done
-explicitly by calling the `valid?` or `invalid?` methods from the 
+explicitly by calling the `valid?` or `invalid?` methods from the
 [ActiveModel::Validations](https://edgeapi.rubyonrails.org/classes/ActiveModel/Validations.html) interface.
 
 ### RSpec Support

lib/avromatic.rb

@@ -90,6 +90,10 @@ def self.eager_load_models!
     @eager_load_model_names&.each { |model_name| model_name.constantize.register! }
   end
   private_class_method :eager_load_models!
+
+  def self.allow_decimal_logical_type?
+    ::Gem::Requirement.new('>= 1.11.0').satisfied_by?(::Gem::Version.new(::Avro::VERSION))
+  end
 end
 
 require 'avromatic/railtie' if defined?(Rails)

lib/avromatic/model/types/decimal_type.rb

@@ -0,0 +1,63 @@
+# frozen_string_literal: true
+
+require 'bigdecimal'
+require 'bigdecimal/util'
+require 'avromatic/model/types/abstract_type'
+
+module Avromatic
+  module Model
+    module Types
+      class DecimalType < AbstractType
+        VALUE_CLASSES = [::BigDecimal].freeze
+        INPUT_CLASSES = [::BigDecimal, ::Float, ::Integer].freeze
+
+        attr_reader :precision, :scale
+
+        def initialize(precision:, scale: 0)
+          super()
+          @precision = precision
+          @scale = scale
+        end
+
+        def value_classes
+          VALUE_CLASSES
+        end
+
+        def input_classes
+          INPUT_CLASSES
+        end
+
+        def name
+          "decimal(#{precision}, #{scale})"
+        end
+
+        def coerce(input)
+          case input
+          when ::NilClass, ::BigDecimal
+            input
+          when ::Float, ::Integer
+            input.to_d
+          else
+            raise ArgumentError.new("Could not coerce '#{input.inspect}' to #{name}")
+          end
+        end
+
+        def coercible?(input)
+          input.nil? || input_classes.any? { |input_class| input.is_a?(input_class) }
+        end
+
+        def coerced?(value)
+          value.nil? || value_classes.any? { |value_class| value.is_a?(value_class) }
+        end
+
+        def serialize(value, _strict)
+          value
+        end
+
+        def referenced_model_classes
+          EMPTY_ARRAY
+        end
+      end
+    end
+  end
+end

lib/avromatic/model/types/type_factory.rb

@@ -4,6 +4,7 @@
 require 'avromatic/model/types/boolean_type'
 require 'avromatic/model/types/custom_type'
 require 'avromatic/model/types/date_type'
+require 'avromatic/model/types/decimal_type'
 require 'avromatic/model/types/enum_type'
 require 'avromatic/model/types/fixed_type'
 require 'avromatic/model/types/float_type'
@@ -50,6 +51,9 @@ def create(schema:, nested_models:, use_custom_types: true)
             )
           elsif schema.respond_to?(:logical_type) && SINGLETON_TYPES.include?(schema.logical_type)
             SINGLETON_TYPES.fetch(schema.logical_type)
+          elsif schema.respond_to?(:logical_type) && schema.logical_type == 'decimal' &&
+                Avromatic.allow_decimal_logical_type?
+            Avromatic::Model::Types::DecimalType.new(precision: schema.precision, scale: schema.scale || 0)
           elsif SINGLETON_TYPES.include?(schema.type)
             SINGLETON_TYPES.fetch(schema.type)
           else

spec/avro/dsl/test/logical_types.rb

@@ -4,5 +4,6 @@
   required :date, :int, logical_type: 'date'
   required :ts_msec, :long, logical_type: 'timestamp-millis'
   required :ts_usec, :long, logical_type: 'timestamp-micros'
+  required :decimal, :bytes, logical_type: 'decimal', precision: 4, scale: 2
   required :unknown, :int, logical_type: 'foobar'
 end

spec/avro/schema/test/logical_types_with_decimal.avsc

@@ -0,0 +1,44 @@
+{
+  "type": "record",
+  "name": "logical_types_with_decimal",
+  "namespace": "test",
+  "fields": [
+    {
+      "name": "date",
+      "type": {
+        "type": "int",
+        "logicalType": "date"
+      }
+    },
+    {
+      "name": "ts_msec",
+      "type": {
+        "type": "long",
+        "logicalType": "timestamp-millis"
+      }
+    },
+    {
+      "name": "ts_usec",
+      "type": {
+        "type": "long",
+        "logicalType": "timestamp-micros"
+      }
+    },
+    {
+      "name": "decimal",
+      "type": {
+        "type": "bytes",
+        "logicalType": "decimal",
+        "precision": 4,
+        "scale": 2
+      }
+    },
+    {
+      "name": "unknown",
+      "type": {
+        "type": "int",
+        "logicalType": "foobar"
+      }
+    }
+  ]
+}

spec/avromatic/model/builder_spec.rb

@@ -348,7 +348,9 @@
     end
 
     context "logical types" do
-      let(:schema_name) { 'test.logical_types' }
+      let(:schema_name) do
+        Avromatic.allow_decimal_logical_type? ? 'test.logical_types_with_decimal' : 'test.logical_types'
+      end
 
       it_behaves_like "a generated model"
 
@@ -430,6 +432,25 @@
           expect { test_class.new(date: 'today') }.to raise_error(Avromatic::Model::CoercionError)
         end
       end
+
+      context "decimal", skip: !Avromatic.allow_decimal_logical_type? do
+        it "accepts a BigDecimal" do
+          decimal = BigDecimal('3.4562')
+          instance = test_class.new(decimal: decimal)
+          expect(instance.decimal).to eq(decimal)
+        end
+
+        it "accepts an Integer" do
+          instance = test_class.new(decimal: 42)
+          expect(instance.decimal).to eq(42.to_d)
+        end
+
+        it "accepts a Float" do
+          float = 5.23
+          instance = test_class.new(decimal: float)
+          expect(instance.decimal).to eq(float.to_d)
+        end
+      end
     end
 
     context "recursive models" do
@@ -1141,6 +1162,22 @@ def initialize
           expect(instance.u).to eq(now)
         end
       end
+
+      context "union with a decimal", skip: !Avromatic.allow_decimal_logical_type? do
+        let(:schema) do
+          Avro::Builder.build_schema do
+            record :with_decimal_union do
+              required :u, :union, types: [:string, bytes(logical_type: 'decimal', precision: 4)]
+            end
+          end
+        end
+
+        it "coerces numeric to a union member" do
+          numeric = 42.42
+          instance = test_class.new(u: numeric)
+          expect(instance.u).to eq(numeric.to_d)
+        end
+      end
     end
 
     context "unsupported" do

spec/support/contexts/logical_types_serialization.rb

@@ -4,7 +4,9 @@
 #   decoded: a model instance based on the encoded_value
 shared_examples_for "logical type encoding and decoding" do
   context "logical types" do
-    let(:schema_name) { 'test.logical_types' }
+    let(:schema_name) do
+      Avromatic.allow_decimal_logical_type? ? 'test.logical_types_with_decimal' : 'test.logical_types'
+    end
     let(:test_class) do
       Avromatic::Model.model(schema_name: schema_name)
     end
@@ -26,7 +28,7 @@
             ts_msec: now,
             ts_usec: now,
             unknown: 42
-          }
+          }.tap { _1[:decimal] = BigDecimal('5.2') if Avromatic.allow_decimal_logical_type? }
         end
 
         it "encodes and decodes instances" do
@@ -44,7 +46,7 @@
             ts_msec: now.to_i + now.usec / 1000 * 1000,
             ts_usec: now.to_i * 1_000_000 + now.usec,
             unknown: 42
-          }
+          }.tap { _1[:decimal] = BigDecimal('1.5432') if Avromatic.allow_decimal_logical_type? }
         end
 
         it "encodes and decodes instances" do