AutoSpotting Changelog - Community vs. Commercial Edition

[cristim]

This page documents the differences between the Community Edition and the Commercial Edition available on the AWS Marketplace.

Changes only available in the Commercial Edition

1.3.0-4 - 11 Apr 2024

New Features

1. Improvements in instance replacement logic to avoid reduced capacity during replacements, enhancing the reliability and performance of AutoSpotting.
2. Introduction of configurable minimum Spot diversification within each Availability Zone to increase reliability in case of concurrent Spot terminations.
3. Enhanced ECS task draining reliability for terminating Spot instances, ensuring smoother transitions without scheduling new ECS tasks on terminating instances.
4. Awareness of Beanstalk deployments and Autoscaling Instance Refresh actions, improving compatibility, reliability and reducing the need for deployment workarounds.
5. Automatic migration of stateful resources like EBS volumes and Elastic IPs to replacement instances.
6. Reduction of SQS message TimeToLive (TTL) to 15 minutes from 24 hours to prevent mass-processing of older events and improve reliability.
7. Disabling of Capacity Rebalancing for ASGs managed by AutoSpotting to reduce instance churn caused by Spot rebalancing events.
8. Reaping of unattached OnDemand instances to avoid unnecessary costs due to instances left outside ASGs after interruptions in the replacement logic.
9. Reuse of Launch Templates converted from Launch Configurations within the first minute and reaping after a week to manage stale configurations.
10. Automatic conversion of small EBS volumes to GP3 for Launch Configurations, achieving cost savings and performance benefits.
11. Updated instance type information to include all types released up to the end of February 2024, expanding the coverage of supported instances.
12. Conversion to SDK for Go v2 for improved performance, reduced memory consumption, and future-proofing of the AutoSpotting codebase.
13. Decreased memory consumption through optimizations like passing data structures by reference, improving scalability for larger customers.
14. Updated to the latest Go version for increased security and better performance.
15. Addressed about a dozen crash scenarios and implemented numerous code quality enhancements for improved reliability.
16. Implementation of numerous improvements required to pass all gosec static code checks, for improved security and code correctness.

Bugfixes

1. Billing bugfix included in both 1.2.3 and 1.3.0 releases, addressing an issue where customers were overcharged for Spot instances not launched by AutoSpotting.
2. Fixed support for EC2 Classic configuration style with Security Groups given by name, ensuring compatibility with older configurations.
3. Improved concurrent Lambda execution for handling Spot Termination events, allowing for parallel processing of other events.

Pricing change

This version increases pricing from 5% to 10% of the savings.

1.2.3-2 - 11 Apr 2024

Bugfix release

• fix billing issue - AutoSpotting used to bill any Spot instances, even if something else was used to launch them.

1.2.2-0 - 3 July 2023 - experimental release for MixedInstancesPolicy support

New Feature
This release adds experimental support for Autoscaling groups configured with MixedInstancesPolicy, which were previously silently ignored by AutoSpotting.

These groups are becoming very common and are the default when using the AWS Console to create new groups, so many customers ran into this issue and had to change the ASG configuration.
Going forward AutoSpotting will handle these groups automatically, without any additional configuration needed.

It works by launching Spot instances and protecting them against scale-in, otherwise the ASG would replace them to match the ASG's MixedInstancesPolicy.

AutoSpotting's cron logic now also manages the Scale-in protection of the Spot instance launched by AutoSpotting, clearing it automatically after the group desired capacity decreases to a few of the instances can be terminated.

Bugfixes

• fix the cron replacement logic, which had a regression since the previous version and failed to replace existing OnDemand instances with Spot instances, breaking the onboarding experience for all new AutoSpotting users.
• fix the cron Timezone configuration, which previously only supported the UTC timezone. Going forward all timezones are supported.

1.2.1-6 - 28 Apr 2023

This release improves on the early load balancer draining for Target Groups and load balancers released in the previous versions.

We now also drain ECS containers as early as possible, to ensure that no new containers are scheduled on terminating Spot instances.

ECS also implements such functionality but it is off by default, and only enabled when setting the ECS_ENABLE_SPOT_INSTANCE_DRAINING=true in the ECS configuration file, which requires customers to change their userdata scripts.

AutoSpotting now handles this seamlessly for new Spot instances in ECS clusters without any additional userdata configuration changes, and additionally also covering the Auto Scaling Capacity Rebalancing events(when AutoSpotting is configured to handle them) that are not covered by the ECS draining logic.

1.2.1-2 - 17 Apr 2023

This release improves on the early load balancer draining for Target Groups released in the previous version.

The previous version used to only consider the load balancer's listener port also for Target Group instances.

We now drain traffic on all the ports when terminating instances with other ports and now we also handle correctly instances with multiple ports such as in case of ECS environments.

1.2.1-0 - 15 Apr 2023

• Early connection draining from load balancers, especially helpful on ECS environments.

You can read more about it here.

1.2.0-2 - 11 Apr 2023

Major changes:

• Proactively launch instances when receiving Spot termination events, so we skip running the temporary OnDemand instance type launched by ASGs in such situations. (Note: this is still happening when scaling out)
• Failover to multiple OnDemand instance types when launching Spot instances as a result of Spot termination events, making AutoSpotting quite resilient to ICEs
• Reuse the Launch Template configured on ASGs, to avoid a class of bugs caused by imperfect conversion of the template data.
• Still Convert Launch Configurations to Launch templates(it's much easier than when converting Launch Templates) but no longer delete the converted templates to facilitate conversion of ASGs to Launch Templates now that Launch Configurations are deprecated.
• Drop the incomplete GP2 to GP3 and IO1 to IO2 conversion logic, which is now much better supported by EBS Optimizer.
• Drop the InstanceTerminationMethod configuration flag, which is no longer relevant.

Other changes

• Support installing AutoSpotting in any AWS region as its main region, previously it only supported Virginia
• Determine the Spot product automatically, and persist it as an ASG tag (this tag will need to be deleted when changing the AMI to another OS). We now use it to have correct instance type selection and billing based on the price.
• Consider UPDATE_ROLLBACK_FAILED as a valid steady CloudFormation stack state.
• Simplify regional event forwarding in both CloudFormation and Terraform as explained here, basically deleting the regional Lambdas that invoked the main Lambda and replaced them with EventBridge+SQS.
• Increase InService timeouts to up to 13min for slow to start instances.
• Reap unattached Spot instances running for more than 15min which could have resulted from such timeouts.
• Perform billing and SSM API calls to the main region where the Lambda function is installed, previously was hardcoded to us-east-1.
• Clean up the Beanstalk Userdata patching functionality which is no longer needed now that we attach Spot instances immediately after launch
• Avoid deleting the log group of the AutoSpotting Lambda function when uninstalling or updating the CloudFormation template

Bugfixes

• fixes on Security Groups: we previously duplicated them in some conditions, which resulted in errors on instances with more than 2 SGs, and completely missed them in other situations, getting the Default VPC security group.
• fix null-pointer-exception in the billing code that triggered when the Lambda was installed in other regions but Virginia
• fix crashes in the code handling the Spot termination notification events

Additionally, during this release cycle we also did the following changes in our other related projects:

AutoSpotting Terraform code

• brought in sync with CloudFormation, including EventBridge/SQS architecture
• support using an existing IAM role and VPC subnets for environments with strict landing zones that can't create IAM roles and VPCs.
• support for using trial builds from our public ECR.
• support for passing a IAM permissions boundary to our IAM roles.
• avoid the need for double runs for new users.
• update dependencies.

Savings estimator

• first public release, with support for estimating ASG cost savings and integration with AutoSpotting.
• Info box with next steps when applying AutoSpotting tags.
• CloudFormation template for deploying a IAM role with minimal permissions.
• Documentation updates.

Already released

1.1.2-4 - 24 Feb 2023

• parallel instance type replacements per ASG
• updated instance type coverage, up to M7g and R7g
• auto-update instance type information

1.1.1-0 - 25 Nov 2022 - Black Friday release

• Organize and group CloudFormation Stack parameters so they appear in a meaningful manner when installing/updating AutoSpotting
• Send savings reports by email on a daily/weekly/monthly basis.
• Wait for Beanstalk deployments to complete when replacing instances.
• Log retention for most growing logs.
• Installation fix when ECS Service Linked Role exists.

1.1.0-rc7 - 6 Oct 2022

• Rebalancing Recommendation events disabled by default.
• EBS bandwidth comparisons optional and off by default, for further diversification.
• updated instance type coverage up to R6a.
• Fix for the Disallowed instance types, which used to be ignored in the default configuration.
• Fix CloudFormation deployment on environments that already have the ECS Service Linked Role.