A team migrated off bastion hosts to Session Manager for a “more secure” access model, then spent an afternoon unable to connect to a single instance in a newly created private subnet. Every other instance in the account connected fine. The difference: the new subnet’s route table didn’t have a path to any of the three SSM VPC endpoints the rest of the VPC relied on, because the subnet was created after the endpoints and nobody updated the endpoint’s subnet associations. Session Manager wasn’t broken — that one subnet just had no way to reach it.

Session Manager removes the bastion host, but it doesn’t remove the network and IAM requirements underneath it — it just relocates them to different places that are less obvious when they’re missing.

The Problem

Connections fail with a handful of characteristic errors:

Symptom What It Means
TargetNotConnected SSM Agent not running, not registered, or can’t reach the SSM service endpoints
Instance doesn’t appear in Fleet Manager Missing or incorrect IAM instance profile, or agent not installed/outdated
Connects from the console but not via CLI Local AWS CLI missing the Session Manager plugin
Works in one subnet, fails in another Missing VPC endpoints (or route/security group blocking) for a private subnet with no NAT gateway
Session starts then immediately closes KMS key policy blocking session encryption, or an SCP denying the API call

Every one of these is a plumbing problem between the agent and the SSM service — not a bug in Session Manager itself.

Why Does This Happen?

  • SSM Agent not installed or not running: Most current AMIs (Amazon Linux 2/2023, recent Ubuntu/Windows AMIs) ship with the agent preinstalled, but older or custom AMIs may not have it, or it may be stopped.
  • Missing or misconfigured IAM instance profile: The instance needs a role with AmazonSSMManagedInstanceCore (or equivalent least-privilege policy) attached as an instance profile — not just an IAM role that exists somewhere, but one actually attached to the instance.
  • No network path to SSM endpoints from a private subnet: Instances without direct internet access (no NAT gateway, no public IP) need VPC interface endpoints for ssm, ssmmessages, and ec2messages — missing any one of the three causes the agent to fail to register or fail to establish a session channel.
  • Security group on the VPC endpoint blocking the instance: The interface endpoint’s security group must allow inbound HTTPS (443) from the instance’s security group or subnet CIDR — a default-deny endpoint security group silently blocks all agent traffic.
  • Local machine missing the Session Manager plugin: aws ssm start-session from the CLI requires the separately installed session-manager-plugin binary — without it, the AWS CLI itself errors out before ever reaching AWS.
  • KMS key policy blocking the session encryption key: If session data encryption is enabled with a customer-managed KMS key, the caller’s IAM principal needs kms:GenerateDataKey and kms:Decrypt on that key — a restrictive key policy causes sessions to fail immediately after starting.

The Fix

Step 1: Confirm the Instance Is Registered With SSM

aws ssm describe-instance-information \
  --filters "Key=InstanceIds,Values=i-0abc123def456" \
  --query "InstanceInformationList[0].{PingStatus:PingStatus,AgentVersion:PlatformType,LastPing:LastPingDateTime}"

If the instance doesn’t appear in the output at all, it has never successfully registered — that’s an agent, IAM, or network issue, not a Session Manager issue specifically.

Step 2: Verify the IAM Instance Profile

aws ec2 describe-instances \
  --instance-ids i-0abc123def456 \
  --query "Reservations[0].Instances[0].IamInstanceProfile"

Confirm the attached role includes the managed policy:

aws iam list-attached-role-policies \
  --role-name ec2-ssm-role \
  --query "AttachedPolicies[?PolicyName=='AmazonSSMManagedInstanceCore']"

If missing, attach it:

aws iam attach-role-policy \
  --role-name ec2-ssm-role \
  --policy-arn arn:aws:iam::aws:policy/AmazonSSMManagedInstanceCore

Step 3: Check Agent Status on the Instance

If you have any other access path (console-based EC2 Instance Connect, or the instance is reachable), check directly:

sudo systemctl status amazon-ssm-agent

sudo systemctl restart amazon-ssm-agent

Step 4: Verify Required VPC Endpoints Exist and Are Reachable From the Subnet

aws ec2 describe-vpc-endpoints \
  --filters "Name=vpc-id,Values=vpc-0abc123def" \
  --query "VpcEndpoints[*].{Service:ServiceName,State:State,Subnets:SubnetIds}" \
  --output table

Confirm all three are present: com.amazonaws.us-east-1.ssm, com.amazonaws.us-east-1.ssmmessages, com.amazonaws.us-east-1.ec2messages. If the affected subnet isn’t in the endpoint’s subnet list, add it:

aws ec2 modify-vpc-endpoint \
  --vpc-endpoint-id vpce-0abc123def \
  --add-subnet-ids subnet-0newsubnet456

Step 5: Check the VPC Endpoint’s Security Group

aws ec2 describe-security-groups \
  --group-ids sg-0endpointsg123 \
  --query "SecurityGroups[0].IpPermissions"

Should allow inbound 443 from the instance’s security group or subnet CIDR:

aws ec2 authorize-security-group-ingress \
  --group-id sg-0endpointsg123 \
  --protocol tcp \
  --port 443 \
  --source-group sg-0instancesg456

Step 6: Install the Session Manager Plugin Locally

aws ssm start-session --target i-0abc123def456

If this fails with a plugin-not-found error, install it (macOS example):

curl "https://s3.amazonaws.com/session-manager-downloads/plugin/latest/mac/sessionmanager-bundle.zip" -o "sessionmanager-bundle.zip"
unzip sessionmanager-bundle.zip
sudo ./sessionmanager-bundle/install -i /usr/local/sessionmanagerplugin -b /usr/local/bin/session-manager-plugin

Step 7: Check KMS Key Permissions if Session Encryption Is Enabled

aws ssm get-document \
  --name SSM-SessionManagerRunShell \
  --query "Content" | python3 -m json.tool | grep -i kms

Verify the caller’s role has permission on the referenced key:

aws kms get-key-policy \
  --key-id arn:aws:kms:us-east-1:111122223333:key/abc-123 \
  --policy-name default

Is This Safe?

The describe and get commands are read-only. Attaching the AmazonSSMManagedInstanceCore policy is additive and low-risk. Modifying a VPC endpoint’s subnet associations and security group rules affects all traffic using that endpoint — review what else depends on it before changing its security group broadly, though adding subnet associations is generally safe and non-disruptive to existing connections.

Key Takeaway

Session Manager failures are network and IAM plumbing issues wearing a connectivity error message. Work through it in order: confirm the instance registers with SSM at all, confirm the instance profile has the right managed policy, confirm the three required VPC endpoints exist and are reachable from the specific subnet in question, and confirm the local machine has the plugin installed. Private subnets added after the original VPC endpoint setup are the single most common gap — endpoints don’t retroactively cover new subnets unless you explicitly associate them.

Have questions or ran into a different Systems Manager issue? Connect with me on LinkedIn or X.