Skip to content

aws-ecs: Cloud-init script for EC2 fails when using AWS Linux 2023 #28518

@juinquok

Description

@juinquok

Describe the bug

When using EC2 as the capacity provider in the addAsgCapacityProvider method, the user is expected to specify the machineImageType for the capcity provider. In doing so, it will add a user data script that will inject additional data that is relevant to the user for the /etc/ecs/ecs.config file

One of the commands that is injected is:

autoScalingGroup.addUserData('sudo iptables --insert FORWARD 1 --in-interface docker+ --destination 169.254.169.254/32 --jump DROP');
autoScalingGroup.addUserData('sudo service iptables save');

On AWS Linux 2023 running the Linux 6.1.66-91.160.amzn2023.x86_64 kernel, this command results in an error when executing the cloud-init file.

+ sudo iptables --insert FORWARD 1 --in-interface docker+ --destination 169.254.169.254/32 --jump DROP
+ sudo service iptables save
The service command supports only basic LSB actions (start, stop, restart, try-restart, reload, reload-or-restart, try-reload-or-restart, force-reload, status, condrestart). For other actions, please try to use systemctl.
2023-12-29 06:11:05,181 - cc_scripts_user.py[WARNING]: Failed to run module scripts-user (scripts in /var/lib/cloud/instance/scripts)
2023-12-29 06:11:05,184 - util.py[WARNING]: Running module scripts-user (<module 'cloudinit.config.cc_scripts_user' from '/usr/lib/python3.9/site-packages/cloudinit/config/cc_scripts_user.py'>) failed

This results in the line after

echo ECS_AWSVPC_BLOCK_IMDS=true >> /etc/ecs/ecs.config

to not get run which is not ideal.

Expected Behavior

It should successfully run the required ECS setup configs when the EC2 instance starts.

Current Behavior

The cloud-init script will fail with the error message The service command supports only basic LSB actions (start, stop, restart, try-restart, reload, reload-or-restart, try-reload-or-restart, force-reload, status, condrestart). For other actions, please try to use systemctl.

Reproduction Steps

Start an ECS Cluster with an EC2 capacity provider and the AMI in the launch template for the autoscaling group to be the latest AWS Linux 2023 AMI (ecs.EcsOptimizedImage.amazonLinux2023(AmiHardwareType.STANDARD)). The error will occur when the instance starts up and the logs can be found in /var/log/cloud-init-output.log

Possible Solution

Introduce a new machineImageType in the addAsgCapacityProvider method and name it AMAZON_LINUX_2023. In the configureAutoScalingGroup method in cluster.ts, add in a new switch condition to render different user data for the ECS Optimized AMI for AWS Linux 2023. In particular, the sudo service iptables save will be changed to sudo iptables-save > /etc/sysconfig/iptables which will not throw the same error as above.

Additional Information/Context

No response

CDK CLI Version

2.114.1 (build 02bbb1d)

Framework Version

No response

Node.js Version

v18.17.0

OS

macOS 14.2

Language

TypeScript

Language Version

No response

Other information

No response

Metadata

Metadata

Assignees

Labels

@aws-cdk/aws-ecsRelated to Amazon Elastic ContainerbugThis issue is a bug.effort/mediumMedium work item – several days of effortp1

Type

No type

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions