As OCS 2007 R2 has officially supported running on a Windows Server 2008 R2 system for nearly a year now there are a growing number of deployments on this version. Additionally most Lync Server 2010 installations will be on Server 2008 R2 as well.
There is an important behavioral change introduced in Server 2008 R2 (and Windows 7) which can impact how clients and servers negotiate NTLM authentication: the minimum encryption level has been raised to 128-bit by default. For most Microsoft client and server operating systems and software this change is not even noticed as 128-bit is a supported level on both sides. (Additional details on this change can be found here in TechNet.)
But when attempting to authenticate third-party solutions this can sometimes cause problems as not all systems will currently negotiate above 40-bit or 56-bit levels.
Update: the Polycom HDX natively supports 128-bit encryption since version 3.0 was released and the configuration changes in this article are not required.
For example, when attempting to natively register some video conferencing systems to an OCS or Lync registrar which is running on a default configuration of Server 2008 R2 the following error will be displayed on the System Status window after configuration:
Upon further investigation the diagnostic logs will report the following error after the SIP registration attempts fail.
ms-diagnostics: 1000;reason=”Final handshake failed”;HRESULT=”0xC3E93EE4(SIP_E_AUTH_NTLMMISMATCH)”;source=”LYNC.schertz.local”
The 0xC3E93EE4 error code indicates an NTLM encryption level mismatch meaning that an unsupported setting is typically being enforced by one side of the conversation.
This can be resolved by lowering the minimum level required on the OCS/Lync registrar server.
- Open the Local Security Policy administrative tool (secpol.msc), expand Local Policies and select Security Options.
- Locate the policy Network Security: Minimum session security for NTLM SSP based (including secure RPC) servers and view the properties.
Notice the Security Setting is currently at Require 128-bit encryption. Also notice that there are two settings, one for servers and one for clients; the client value does not need to be modified as the server setting is what impacts the HDX client registration process.
- Clear the Require 128-bit encryption setting.
- Verify that the Security Setting is now displayed as No minimum.
- Restart the OCS or Lync Front-End service to immediately pick-up the policy change and then reattempt SIP registration of the endpoint. The following status screen from an HDX reports a successful registration.
With the so far small amount of technical data, officially released by Microsoft, this blog is saving me a huge amount of time.
This individual post, has just resolved and closed one of my most annoying cases, and has saved be a LOT of time, searching for knowledge, mailing with different HW providers and so.
Thank you so much for this, and keep the posts comming, as they are extreamly well written, and of highly relevance.
– Mads Jensen
Actuelly, I've got a question now.
Our conference equipment are from TANDBERG, and like mentioned above, the registration now finally works!
After doing some tests today, something odd turned up.
Whenever the Tandberg system calls the user, everything works flawlessly, both A/V is superb.
However, when a user calls the TANDBERG system, the calling starts correct at the system, and I'm able to accept the incomming call. I then get a "Call Connected" etc., however the A/V for any of the two, won't start. The call isn't disconnected, A/V just simply wont start.
Any ideas?
Sincerely,
Mads
Mads, the Tandberg endpoints are not supported for native registration with OCS as Polycom's devices are. In order to get that sceanrio to work you'd need to purchase ad addtional Advanced Media Gateway to bridge the calls, which is limited to something like 20 VGA or 10 HD sessions per gateway.
Hi, we are receiving the same error as above "The registrar has rejected the system's attempt at registration". HDX8000 running 3.0.0.1 registering against OCS 2007 R2 (on Windows 2003) and passing through a Cisco HLB (ACE-4710).
Would certainly appreciate any help.
Try telnetting to port 23 of the HDX and entering the command "config sipdebug 0 true" and then go to the HDX configuration and click Update to force a re-registration. Capture the output of the telnet window to a text file and search for any errors occuring during the registration attempt.
[…] Also, if you encounter this error “The registrar has rejected the system’s attempt at registration“, Jeff Schertz has a good blog write-up on why this could be happening and how to resolve it: http://blog.schertz.name/2011/01/hdx-sip-registration-with-server-2008-r2/. […]
I keep receiving the error “The Registrar has rejected the systems atempt at registration”. I have a consolidated lync server built from your instructions on 2008 R2. I modified the Registtry setting to allow NTLM v2 and I am running HDX 8000 3.0.1-10628. I even followed your guide to use another account to register since my room account is disabled. However I keep getting the above error the system logs say ms-diagnostics: 1000;reason=”Final handshake failed”;HRESULT=”0xC3E93EC3(SIP_E_AUTH_UNAUTHORIZED)” The Polycomm has a self-signed Cert and the lync server has a cert from our internal CA.
What am I doing wrong?
Mike, make sure that you have the correct username information populated in the SIP Server settings as well as the password. Test that account on a Lync client as well, as the AUTH_UNAUTHORIZED error simply means that the server rejected the authentication attempt.
I am integrating an HDX 7000 with OCS R2 and am also getting the 0xC3E93EC3(SIP_E_AUTH_UNAUTHORIZED) error. I have tried multiple combinations under 'user name' as well as 'domain user name'. Is there a magin combination i am unaware of to make it work?
Hi,
Can this be an issues while working with Lync 2013 (on windows 2012 )and Lifesize passport?
You'd need to contact Lifesize as I wouldn't be able to even make a guess there.
I know this is an old thread. But I stumbled across through a web search.
I am seeing the error pictured up top, with the Lync Server rejected registration.
With my Polycom units most of them have registered no problem but others have the error. They all are running 3.1 or newer software.
any ideas?
If you are registering to Lync 2013 then this approach will not work as I've explained in the updated section.