TL;DR — The right model for human SSH access to a Linux fleet on Azure is: generate the keypair inside Terraform, store the private key in Key Vault, grant humans Key Vault Secrets User via RBAC, and have them connect through Azure Bastion. This post walks through the CDKTF wiring and shares azkv-ssh-fetch — a CLI that turns the operator side of this pattern into a single command.

The “upload my public key” antipattern

The right model in one diagram

Minimal CDKTF snippet

*<!– TODO: ~30 lines of TypeScript showing:

• generating tls_private_key
• writing public key to vmss admin_ssh_key
• writing private key to KV as a secret with content_type ‘application/x-pem-file’
• role_assignment block granting a group ‘Key Vault Secrets User’ Keep it copy-paste-able. Real names, masked subs. –>*

// TODO: paste production-shaped snippet here.
// Replace with your actual stack name + group object ID.

What this buys you

• No key sprawl — one private key per VMSS, lives in one place.
• Auditable access — every get-secret call shows up in KV diagnostic logs with the user identity.
• Rotation is terraform apply — bump the tls_private_key resource, re-run, done.
• PIM-compatible — the role assignment can be marked PIM-eligible so humans elevate to access the key just-in-time.
• No shared keys in puppet/ansible — the only thing in config management is the public key fingerprint, if anything.

The operator side: akf

The operator workflow described above used to be a six-line shell snippet that every engineer re-typed (and got wrong) every time. I wrote azkv-ssh-fetch to remove the boilerplate:

pipx install azkv-ssh-fetch

# Find the SSH-shaped secrets in a vault
akf list --vault prod-myteam-kv

# Pull a key to ~/.ssh/<name> with mode 0600 (atomic)
akf fetch --vault prod-myteam-kv my-vmss-ssh

# Fetch + Bastion SSH in one command (key is shredded on disconnect by default)
akf connect \
  --vault prod-myteam-kv \
  --secret my-vmss-ssh \
  --bastion my-bastion --bastion-rg my-bastion-rg \
  --target-id "/subscriptions/.../virtualMachineScaleSets/my-vmss/virtualMachines/0"

Source: github.com/NaeemH/azkv-ssh-fetch · License: MIT.

Gotchas

*<!– TODO: bulleted list. Drafts:

• KV name length cap (24 chars) bites when you template names
• Bastion Standard SKU minimum for native-client tunneling
• VMSS image must have password auth disabled or this whole model is theater
• Don’t store the private key with content_type other than ‘application/x-pem-file’ — KV will let you, but tools may not filter it
• SIG image generation mismatch (Gen1 vs Gen2) — separate post material –>*

Further reading

• Azure Bastion native client
• Key Vault RBAC guide
• Terraform tls_private_key
• CDKTF docs

Found a mistake or want to discuss? Open an issue at NaeemH/azkv-ssh-fetch or ping me on LinkedIn.