Computer Notes/Security Services Platform/Accessing the K8 Cluster

Accessing Security Services Platform (SSP) Kubernetes Cluster Nodes from SSPI using Key Pair Authentication is a operational procedure used by system administrators to establish secure, passwordless SSH sessions between a VMware Security Services Platform Installer (SSPI) appliance and its managed Kubernetes (K8s) cluster nodes.

Utilizing public-key cryptography eliminates the reliance on static passwords, simplifies automated node administration, and aligns with security best practices for infrastructure management.

Prerequisites

Before configuring key pair authentication, ensure the following requirements are met:

• Administrative shell access (`sysadmin` or `root`) to the deployment SSPI appliance.
• Operational access to the `kubectl` command-line utility configured on the SSPI host.
• Network connectivity on TCP port 22 (SSH) between the SSPI appliance and all target Kubernetes nodes (Control Plane and Worker nodes).

Step 1: Generate an SSH Key Pair on SSPI

If an SSH key pair does not already exist on the SSPI appliance, generate a high-entropy key pair (such as Ed25519) from the SSPI terminal shell.

1. Log into the SSPI host via SSH:

   ssh sysadmin@<SSPI_IP_ADDRESS>

2. Switch to the root account if required for system-level node administration:

   sudo -i

3. Generate a new Ed25519 key pair:

   ssh-keygen -t ed25519 -C "sspi-k8s-admin"

4. Press ENTER to accept the default file location (~/.ssh/id_ed25519).
5. Optionally enter a passphrase to secure the private key on disk.

Step 2: Retrieve Target Kubernetes Node IP Addresses

Query the Kubernetes API server from the SSPI command line to identify the internal IP addresses of the cluster nodes:

kubectl get nodes -o wide

Note the values in the INTERNAL-IP column for the target Control Plane or Worker nodes.

Step 3: Distribute the Public Key to Cluster Nodes

To establish trust, copy the generated public key (id_ed25519.pub) to the ~/.ssh/authorized_keys file on the target node.

Method A: Using ssh-copy-id (Automated)

If password authentication is temporarily enabled on the nodes, run:

ssh-copy-id -i ~/.ssh/id_ed25519.pub root@<NODE_INTERNAL_IP>

Enter the node's password when prompted to append the key automatically.

Method B: Manual Injection (Pre-deployment or Cluster Spec)

If direct password access to the nodes is disabled, inject the public key content directly into the cluster specification or pre-seed file during initial provision:

1. Display the public key:

   cat ~/.ssh/id_ed25519.pub

2. Copy the string output and append it to /root/.ssh/authorized_keys on the target node file system.
3. Ensure correct file permissions on the target node:

   chmod 700 /root/.ssh && chmod 600 /root/.ssh/authorized_keys

Step 4: Establish Passwordless SSH Connection

Verify the key pair authentication setup by opening a new SSH connection from the SSPI appliance to the target node:

ssh -i ~/.ssh/id_ed25519 root@<NODE_INTERNAL_IP>

Upon successful authentication, the terminal prompt will update to reflect the target node's hostname without prompting for a password.

Security Best Practices

• Restrict Root File Permissions: Verify that the private key on the SSPI appliance maintains strict permissions (chmod 600 ~/.ssh/id_ed25519) to prevent unauthorized read access by non-privileged local users.
• Audit Authorized Keys: Periodically audit the /root/.ssh/authorized_keys file on cluster nodes to remove obsolete or unauthorized public keys.
• Least Privilege: Limit direct node SSH access to emergency troubleshooting and diagnostic operations. Standard cluster administrative operations should be performed via `kubectl` or API endpoints.

Troubleshooting

Permission Denied (publickey)

If connection fails with a Permission denied (publickey) error:

• Verify key permissions on the SSPI host: chmod 600 ~/.ssh/id_ed25519.
• Check remote permissions on the node: chmod 700 /root/.ssh and chmod 600 /root/.ssh/authorized_keys.
• Verbose SSH output can be reviewed by appending -vvv to the SSH command:

  ssh -vvv -i ~/.ssh/id_ed25519 root@<NODE_INTERNAL_IP>