Lost access to your AWS EC2 instance?

Losing the key to your EC2 instance can be a frustrating experience, especially when you realize you can no longer connect to it. Whether you've lost the EC2 instance SSH key or can't connect for another reason, AWS Systems Manager (SSM) provides a secure and modern alternative for regaining access to your instance. Unlike traditional SSH key-based access, which poses challenges for lifecycle management and security, Systems Manager allows you to centrally manage and audit your connections without relying on keys. In this guide, we'll walk you through the step-by-step process to recover access to an EC2 instance using SSM, eliminating the need for SSH keys and aligning with best practices for secure infrastructure management.

Step 1: Shut Down the EC2 Instance

Note, before shutting down the instance make sure the termination behaviour is not set to terminate. Then shutting down the instance will cause you to lose the instance.

1. Log in to the AWS Management Console.
2. Navigate to the EC2 Dashboard and locate the instance you want to recover.
3. Stop the instance:
   • Select the instance.
   • Click on Instance State → Stop Instance.

Step 2: Create a New IAM Role for EC2

1. Go to the IAM Management Console.
2. Create a new IAM role:
   • Choose AWS service as the trusted entity.
   • Select EC2 as the use case.
3. Attach the AWS managed policy: AmazonSSMManagedInstanceCore.
4. Provide a name for the role, such as EC2-SSM-Role.
5. Complete the process and save the role.

Step 3: Attach the IAM Role to the EC2 Instance

1. Go back to the EC2 Dashboard.
2. Select the stopped instance.
3. Click on Actions → Security → Modify IAM Role.
4. Attach the newly created role (EC2-SSM-Role) to the instance.
5. Save the changes.

Step 4: Start the EC2 Instance

1. Select the stopped instance.
2. Click Instance State → Start Instance.

Step 5: Connect to the Instance Using Systems Manager

1. Ensure Systems Manager Agent (SSM Agent) is installed and running on the instance. Most recent AMIs have it pre-installed.
   • If it’s not installed, you may need to troubleshoot further (consider reattaching the root volume to another instance to install it manually).
2. Go to the Systems Manager Console.
3. Navigate to Session Manager → Start Session.
4. Select the instance and click Start Session to open a terminal.

Step 6: Gain Root Access and Generate a New SSH Key

1. In the terminal session, switch to the root user:

   sudo -i

2. Generate a new SSH key:

   ssh-keygen -t rsa -b 4096 -f /root/new_key -C "recovery-key"

   • Press Enter to save the key pair (you can name the key file new_key).
   • Leave the passphrase empty or set one as per your security requirements.
3. Copy the public key to the instance’s authorized keys (if needed for recovery):

   cat /root/new_key.pub >> ~/.ssh/authorized_keys

   Note: While this process temporarily restores SSH-based access, consider switching to SSM-only access and avoid long-term reliance on SSH.

Step 7: Replace the Compromised Key and Implement Best Practices

1. Replace any compromised keys in /home/<username>/.ssh/authorized_keys if applicable.
2. Remove all unnecessary keys from the instance.
3. Enforce keyless, centralized access using Systems Manager for long-term management.

Why Avoid SSH Keys?

SSH keys are tightly coupled to user machines, making lifecycle management challenging:

• Offboarding users is difficult since keys persist even after a user leaves.
• If a machine is compromised, all associated keys are at risk.
• Systems Manager provides centralized, auditable, and secure access without the risks associated with distributing and managing SSH keys.

Our blog post You Do Not Need That Bastion Host outlines the limitations of SSH keys and highlights how Systems Manager is a superior alternative.

By following this approach, you can recover and secure your EC2 instance while transitioning to a modern and secure access model.

Final note: EC2 needs access to Systems Manager

Additional Note: For AWS Systems Manager to work, the EC2 instance must have either a public IP address or access to the internet via a NAT gateway to reach the SSM endpoints. Alternatively, you can set up a VPC endpoint for Systems Manager to allow private connectivity without requiring internet access.