Amazon Kinesis Agent 1.1.3

• Add AggregationSplitter which enables record aggregation
  • Agent now supports aggregating multiple small records into a big record before putting to a Kinesis Data Firehose delivery stream when "aggregatedRecordSizeBytes" is specified in a delivery stream file flow.
• Allow CW metrics to include instance tag
  • Agent now supports to emit CloudWatch metrics with a new dimension of EC2 instance so that troubleshooting across a fleet of agents gets easier.
• Add a new pre-processor for addition of arbitrary metadata in JSON format
  • The metadata processor allows user to define any arbitrary metadata in JSON and merges the metadata into the processed JSON record in the data processing chain.
• Add EC2 metadata processor
  • EC2 metadata processor retrieves metadata from the EC2 instance metadata service and retrieves the instance tags via a EC2 describe tags request. The metadata is then merged with the processed JSON record in the data processing chain.
• Add separate config directory for flow configurations
  • The agent config directory placed at /etc/aws-kinesis/agent.d/ allows configuration management systems to add additional file flow configurations in separate files dynamically without having to edit the main configuration in place at /etc/aws-kinesis/agent.json.
• Add fileFooterPattern to give up tailing if not necessary
  • Allow agent to stop parsing a file when a specified footer pattern is reached in the file being tailed. This would save resources (CPU/memory consumption) by reducing unnecessary tailing and parsing.
• Bug fixes
  • Fix a bug in checkpointing when pre-processing is enabled
  • Minor fix in logging and exception handling