How Agencies Can Address Weaknesses in SSH Key Management

The secure shell (SSH) data-in-transit protocol has been used as a secure method to move data from machine to machine and to provide remote administrator access for the better part of two decades. A version of SSH is shipped with every edition of Linux, Unix and Mac OS, and its use is growing rapidly in the Windows universe as well: About half of the world’s websites use some form of SSH.

Since its creation over 17 years ago, SSH has secured billions of data transactions without suffering any major security breaches caused by vulnerabilities in the protocol itself. While the protocol is highly secure, today’s rapidly changing threat landscape forces organizations to reconsider how they manage their SSH environments.

Recognize the Evolving Threat

Traditionally, SSH has been used to transfer massive amounts of sensitive information, including classified intelligence and healthcare records. From the perspective of an attacker or a malicious insider, SSH is an artery that carries vital organizational data.

Yet since the protocol itself is secure, how would a bad guy get access to sensitive information protected by SSH? In this case, the keys are critically important.

When users connect via SSH, a trust relationship between a computer and the server is established using a cryptographic key pair. These trust relationships are created and managed internally. None of the systems involved has the ability to search for where the company’s trust relationships exist. As a result, tracking trust relationships must be done manually.

When a network potentially has hundreds of thousands of keys, trust relationships are inevitably lost. If a malicious actor gains access to a key, he or she can mimic an authorized user with impunity. Therefore, improper management of SSH keys, presents a prominent vulnerability available for exploitation by attackers.

A study performed on the management operations of some of the largest organizations in the world revealed a disturbing trend:

• About 10 percent of all SSH user keys provide root access, creating a serious security and compliance issue.
• Organizations often share the same SSH host keys across thousands of computers, leaving the network vulnerable to attacks.
• Organizations rarely know what each key is used for, presenting not only a security risk but also a business continuity risk.
• Many SSH keys that grant access to critical servers are orphaned and no longer in use.
• Some organizations permit administrators to create or delete SSH user keys at will — without approvals or control — essentially granting unfettered, permanent access to systems and people.
• Very few organizations ever rotate SSH user keys, or even remove them when a user leaves or an application is decommissioned.
• Key-based access grants are essentially permanent, in direct violation of SOX, PCI and FISMA requirements for proper termination of access, leaving the network vulnerable to attack.

With advanced threat vectors becoming more commonplace, the risks faced by organizations without proper SSH key management protocols in place are very real. The greater the variance from a best practices approach to SSH key management, the greater the risk to the organization.

To get a handle on SSH key management problems, agencies must take steps to discover all existing users as well as public and private keys, and map trust between machines and users. In addition, IT departments should monitor the environment to determine what keys are actually used and remove keys no longer in use.

Identify Poor Key Management Practices

Problems with access control in secure shell environments are not a result of vulnerabilities or flaws in the SSH protocol itself. Rather, the security and compliance risks identified are caused by:

• Lack of clear guidelines or policies relating to SSH key management.
• Lack of understanding of the scope and implications of the problem.
• Insufficient time and resources to understand the issue and develop solutions.
• Lack of good tools and guidelines for solving key management issues.
• Reluctance on the part of auditors to flag issues for which they don’t have effective solutions.
• Focus of the access management field on interactive users without addressing automated access.

These problems have remained unrecognized largely because SSH key management is deeply technical, keeping the topic in the domain of system administrators. Most system administrators typically see only a small corner of the IT environment, not the full picture. And many administrators are so busy that they may not recognize the problem. If management is several steps removed from the problem and its potentially devastating consequences, no action will be taken.

But the risk remains. To mitigate this risk, agencies should enforce proper approvals for all key setups.

Remediate SSH Key Management Weaknesses

Because the vulnerability is usually found in Unix/Linux servers and many Windows servers, the steps needed to fix the issue will involve several teams within IT operations. The potential liability and compliance issues demand awareness and buy-in from executive management as well.

Steps to remedy the problem include the following:

• Automate key setups and key removals, thereby eliminating manual work and human errors. This step slashes the number of administrators needed for key setups from potentially several hundred to only a few highly trusted administrators.
• Rotate keys regularly, so that copied keys cease to work and proper termination of access can be ensured.
• Restrict where each key has access and what commands can be executed using the key.

To further reduce risk, proper key management should involve the establishment of internal boundaries within the organization. The organization should strictly control which key-based trust relationships can cross which boundaries and must enforce iron-clad IP address and “forced command” restrictions for all authorized keys involving trust relationships crossing such boundaries.

While SSH is widely considered the benchmark for data-in-transit security, the current threat landscape requires organizations to rethink how they are managing access to their encrypted networks. The SSH protocol has done a great job in protecting data-in-transit at a tactical level, but an ever-increasing number of threat vectors means effective management of the SSH environment is critical to secure network operations. Best security practices like the ones identified above will position agencies to prepare for security threats and new compliance mandates before they occur.