[go: up one dir, main page]

Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add option to use custom AMI for AWS EKS nodes #2604

Closed
joneszc opened this issue Jul 31, 2024 · 4 comments
Closed

Add option to use custom AMI for AWS EKS nodes #2604

joneszc opened this issue Jul 31, 2024 · 4 comments

Comments

@joneszc
Copy link
Contributor
joneszc commented Jul 31, 2024

Feature description

Enable ability to specify a custom Amazon Machine Image to utilize for EKS cluster nodes in lieu of the default image. This feature would require an aws_launch_template terraform resource, possibly dependent on or incorporated with #2603, to run the /etc/eks/bootsrap.sh command as necessary when the ami_type is "CUSTOM". When specifying a custom AMI ID, an additional switch would be necessary to ensure that ami_type "CUSTOM" replaces "AL2_x86_64_GPU" or "AL2_x86_64", and onus is on the user to ensure the custom AMI is or isn't GPU-enabled.

Value and/or benefit

Nebari users would have the option (e.g. amazon_web_services.node_groups.custom_ami)to utilize customized/optimized ec2 AMI to accommodate customer requirements to ensure networking/security/performance compliance.

For example:
image

Anything else?

No response

@viniciusdc
Copy link
Contributor

Hi @joneszc, thanks for opening the issue, That was an amazing catch; I completely agree that there should be an optional setting in the node_group settings in our nebari-config.

I also don't see a problem with compatibility as the majority of the users would use the default AMI options by default, and this config while present will only be modified by users "looking for it", though appropriate docs will be required to guide users to avoid silly mistakes like per-zone AMIs etc..

@joneszc
Copy link
Contributor Author
joneszc commented Aug 2, 2024

Hello @viniciusdc

The option to specify an ami id could definitely cause some confusion to users not wanting to get into the weeds of EKS.
There is also added risk of users experiencing nodes failing to join the cluster if they are required to manually input or tamper with the /etc/eks/bootstrap.sh command. I feel it would be appropriate to hard code the bootstrap.sh command into the terraform, with proper logic/directives for triggering, and spare the user the responsibility of formulating the command and of potentially troubleshooting faulty nodes. Perhaps, down the road it would be beneficial as a separate use case to enable specific pointed overrides for arguments like --kubelet-extra-args.
I've combined testing of the custom ami option and #2603 at this fork

@tylergraff
Copy link
Contributor

@viniciusdc will review this as part of the PR linked above.

@tylergraff
Copy link
Contributor

This feature was introduced in #2668

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
Status: Done 💪🏾
4 participants