Allows optional config for reader and ingeseter

[afred]

Adds 'reader_options' and 'ingester_options' as optional config options. These
values are then accessible on IngestTypeConfig object via #reader_options and
#ingester_options accessors.

TODO: The BatchRunner (and whichever other objects are responsible for
intantiating Reader and Ingester objects) need to pass these options along to
construtors for those classes.

[cjcolvar]

This looks pretty good. I'm curious what the use case for a single, non-keyed config string might be. I'm not quite sure how to pass that to the reader/ingester without it being confusing with opts sometimes being a hash and sometimes something else. I'd really prefer to be consistent and only allowing hashes.

[afred]

@cjcolvar you can always do something like this...

# in BatchRunner#read, current code does this...
# reader = config.reader.new(batch.source_location)
# instead, change it to this...
reader_params = [batch.source_location, config.reader_options]
reader = config.reader.new(*reader_params)

I don't have strong opinions though.

[afred]

As for use case, I was thinking more of flags, e.g.

class MyCustomIngester
  def initialize(run_derivatives=false)
    @run_derivatives = ['true', "TRUE", "1", true, 1].include? run_derivatives
  end
end

[cjcolvar]

I think I'd really like to keep everything as hashes for consistency, for keys, and for easier support for ENV variables (for docker/cloud deployment).

[afred]

Right on. Stand by for update.

[afred]

@cjcolvar the change doesn't really change anything, other than the dummy value passed in for the optional ingester_options value.

I started to do some validation on the the reader_options and ingester_options config values and realized the whole thing would be a lot cleaner if we use ActiveModel::Validatons instead of the existing ad-hoc validation going on, so I created #104 for doing that.

[cjcolvar]

👍