Allow shared key access only for one user in SSH.
Disable the password authentication for one user in your SSH config. Edit /etc/ssh/sshd_config.
Match User username PasswordAuthentication no
Restart the SSH service.
service ssh restart
Copy user’s public key to the destination server’s authorized file in ~/.ssh/authorized_keys.
ssh-rsa AAAAxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx <-- public key goes on for miles
Back on your client, login via SSH. User will not be prompted for password since public key is already authorized on server.