Smart card over RDP overview
Smartcards provide a very strong cryptographically based authentication mechanism. There are many components to providing a secure smartcard authentication within the Microsoft environment. To provide a secure process to validate certificates, there are a number of steps that are required in a Microsoft domain. To ensure secure communication between the smartcard hardware device and the client many of the steps are run in a local context. A remote desktop by its very nature is not local so the remote system needs a way to emulate the local environment securely. Network level authentication (NLA) protocol was introduced to help resolve this issue. It requires that the user be authenticated before a remote session is established. NLA uses the Credential Security Support Provider (CredSSP) Protocol to securely delegate a user's credentials from a client to a remote server.
The CredSSP Protocol is a composite protocol that relies on other standards-based protocols. It first uses the Transport Layer Security (TLS) Protocol to establish an encrypted channel between the CredSSP client and the CredSSP server. All subsequent messages are sent over this channel. The CredSSP Protocol then uses the Simple and Protected Generic Security Service Application Program Interface Negotiation Mechanism (SPNEGO) to authenticate the user and server in the encrypted TLS session.
All subsequent data that is sent between the client and server application by using the CredSSP Protocol is encrypted under TLS. At this point, RDP sessions can emulate the local client to interface with the smart card.
Given the process, it is not uncommon to see latency to the authentication process. The latency, in some cases, could cause the authentication process to fail. There are some settings that can help prevent timeouts and improve lag time.
Troubleshooting Latency
Troubleshooting latency issues can be challenging. The error messages are not always clear. Additionally latency issues can occur on some RDP sessions while not on others. Latency is highly dependent on network topology. if the latency in the network cannot be improved, the best options are to increase the timeout duration for the following settings.
Increase logon timeout
The login timeout is controlled in the registry key HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Control\Terminal Server\WinStations\RDP-Tcp. Add a new DWORD value for LogonTimeout. Set the timeout value, in seconds, and then restart Terminal Services. You will need to test different values to determine what works best for your environment. It is recommended to test with a value of 60 seconds and adjust from there.
This registry setting needs to be included on every RDP server that is being logged into. If there are many RDP sessions that the user hops through, it is best to start with the last server accessed as part of your troubleshooting steps to determine the appropriate values.
Increase transaction timeout
If the above setting doesn’t resolve your problems, try increasing the transaction time with the following registry settings. Increase the values of both registry settings at the same time as part of your test as they are related. The registry settings should be adjusted on every remote server that the user authenticates to.
- Set TransactionTimeoutMilliseconds DWORD Decimal value range of 1500 to 5000 located in HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\Cryptography\Defaults\Provider\Microsoft Base Smart Card Crypto Provider
- The default value is 1500 ms. Increase the value by 1500 millisecond increments to see if this resolves the issue.
- Additionally, set the TransactionTimeoutDelay DWORD Decimal value range of 5-60 located in HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\Cryptography\Calais
- This value should be larger than the TransactionTimeoutMilliseconds value. The default TransactionTimeoutDelay value is 5 seconds. Increment the value by 5 seconds to see if this resolves the issue.
Visit Microsoft’s General RDP troubleshooting site for additional troubleshooting guidance.