Lync Phone Edition and Public Certificates

October 28, 2014 by · 13 Comments 

This article aims to serve two purposes.  Firstly, as a reminder to always keep Lync Phone Edition (LPE) devices updated and not to grow lackadaisical regarding the routine administrative tasks of frequently downloading these periodic cumulative updates and applying them to the Lync servers.

Secondly, if the previous advice was not taken to heart then some Lync administrators may already be experiencing a specific problem directly related to running older device firmware versions in a current Lync environment.  There is also the potential for newly purchased devices coming from existing stockpiles to have older firmware preinstalled from the factory which can exhibit this same issue, by no fault of the Lync administrator.

The problem referred to is that Lync Phone Edition devices may fail to successfully sign-in to a Lync Server due to an untrusted certificate being presented by the server itself during the initial secure TLS network connection setup.  This scenario can occur when an LPE device is running on an older version of firmware which still contains an outdated public Certificate Authority (CA) store.  In the event that the Lync Server is using a server certificate from a public CA which was signed and issued by a newer CA than the phone is aware of it will fail to connect because it will treat the certificate presented by the Lync server as untrusted.

Be aware that this issue should be limited to the USB-tethered provisioning approach which uses a different method  to download root CA certificates then when using the PIN Authentication method.  Depending on the exact issue it can be possible to leverage the DHCP Option 43 configuration to allow a PIN Authentication attempt to download the proper public CA certificate directly from the Lync Server Certificate Provisioning Web Service.

But to support USB-tethered provisioning a workaround may be required to get the updated public CA certificate installed into an internally connected phone using an LDAP connection to Active Directory so it can then trust the certificate passed by the Lync server to establish a successful TLS connection.  There are two ways to accomplish this, the easy way and the hard way.

  • The preferred method is simply the guidance stated in the opening paragraph and is a preventative step.  If the issue in this article is not a current problem in an environment but the ingredients sound familiar (public certificates on internal servers and LPE not running recent firmware versions) then heed this warning and go download and update those devices today before it is too late.  As usual test the newer version out on a few test phone to validate that no unrelated problems appear before rolling it out to all phones in the environment.

  • On the other hand if this article came up as a result of an administrator frantically searching for a resolution to this problem then unfortunately the preventative route cannot be taken at this point as that ship has sailed.  Luckily there are a few options available to get past this ‘catch-22’ scenario of needing to update the firmware to get TLS working again but requiring TLS in order to facilitate an update.

Background

Because Lync utilizes TLS for all signaling traffic this secure connection must be established before any additional steps like the actual user registration or a device update can take place.

A typical Lync environment will most often utilize an internal Windows Enterprise CA for the Lync server certificates, while a public certificate from a well-known third party certificate authority would be used for only the external side of any Edge Servers.  While this still is the most popular topology in use today it is slowly becoming common to see public certificates used on the internal Lync server roles instead of private certificates for a myriad of reasons.

Environments using public certificates on the internal Lync server roles present a unique problem to LPE devices that many other clients are not subject to.  There is no mechanism in the Windows CE-based LPE firmware to automatically refresh any preinstalled public root and intermediate CA certificates like other Windows-based operation systems.  In the event that a specific public CA makes a change to one of their root or issuing CAs then the new certificate for that CA would need to be propagated down to any clients or servers which currently trust the previous certificate.  Standard Windows operating systems utilize Windows Updates to refresh these types of changes, but LPE does not.  The only way these phones can be made aware of these external changes is by being upgraded to a newer firmware release in which Microsoft has specifically updated the certificate store.  Note that many other 3PIP Qualified phones for Lync can have the same manufacturer-updated behavior so make sure to keep those current as well for the same reasons.

Trusted Certificate Authorities

For optimized Lync Phone Edition devices from all manufacturers Microsoft manages the firmware and updates the root CA store periodically as public CA server certificates expire, are revoked, or are simply replaced.  The TechNet article Certificates for Lync Phone Edition contains a list of the various CA certificates stored in the firmware’s Trusted Authorities Cache but this table is not kept up-to-date.  As of the posting of this article that page still reflects the October 2013 Cumulative Release 10 (4.0.7577.4414) which is over a year old.

The following table lists all of the trusted root CA certificates utilized by LPE, both old and new:

Vendor Friendly Name Expiration Thumbprint Algorithm Key Added
CyberTrust GlobalSign Root CA 1/28/2014 2F173F7DE99667AFA57AF80AA2D1B12FAC830338 md5RSA 2048 RTM
VeriSign Comodo 1/7/2010 4463C531D7CCC1006794612BB656D3BF8257846F md2RSA 1024 RTM
CyberTrust Baltimore aTrust Root 5/12/2025 D4DE20D05E66FC53FE1A50882C78DB2852CAE474 sha1RSA 2048 RTM
CyberTrust GTE CyberTrust Global Root 8/13/2018 97817950D81C9670CC34D809CF794431367EF474 md5RSA 1024 RTM
VeriSign Class 2 Public Primary Certification Authority 8/1/2028 6782AAE0EDEEE21A5839D3C0CD14680A4F60142A md2RSA 1024 RTM
VeriSign Thawte Premium Server CA 12/31/2020 627F8D7827656399D27D7F9044C9FEB3F33EFA9A md5RSA 1024 RTM
VeriSign Thawte Server CA 12/31/2020 23E594945195F2414803B4D564D2A3A3F5D88B8C md5RSA 1024 RTM
VeriSign Class 3 Public Primary Certification Authority 8/1/2028 742C3192E607E424EB4549542BE1BBC53E6174E2 md2RSA 1024 RTM
Entrust Entrust.net Certification Authority (2048) 12/24/2019 801D62D07B449D5C5C035C98EA61FA443C2A58FE sha1RSA 2048 RTM
Comodo AAA Certificate Services 12/31/2020 D1EB23A46D17D68FD92564C2F1F1601764D8E349 sha1RSA 2048 RTM
Comodo AddTrust External CA Root 5/30/2020 02FAF3E291435468607857694DF5E45B68851868 sha1RSA 2048 RTM
Entrust Entrust.net Secure Server Certification Authority 5/25/2019 99A69BE61AFE886B4D2B82007CB854FC317E1539 sha1RSA 1024 RTM
Equifax Equifax Secure Certificate Authority 8/22/2018 D23209AD23D314232174E40D7F9D62139786633A sha1RSA 1024 RTM
Geo Trust GeoTrust Global CA 5/20/2022 DE28F4A4FFE5B92FA3C503D1A349A7F9962A8212 sha1RSA 2048 RTM
Go Daddy Go Daddy Class 2 Certification Authority 6/29/2034 2796BAE63F1801E277261BA0D77770028F20EEE4 sha1RSA 2048 RTM
Go Daddy ValiCert Class 2 Policy Validation Authority 6/25/2019 317A2AD07F2B335EF5A1C34E4B57E8B7D8F1FCA6 sha1RSA 1024 RTM
Go Daddy Starfield Class 2 Certification Authority 6/29/2034 AD7E1C28B064EF8F6003402014C3D0E3370EB58A sha1RSA 2048 RTM
DigiCert DigiCert High Assurance EV Root CA 11/9/2031 5FB7EE0633E259DBAD0C4C9AE6D38F1A61C7DC25 sha1RSA 2048 CU10
Entrust Entrust Root Certification Authority 11/27/2026 B31EB1B740E36C8402DADC37D44DF5D4674952F9 sha1RSA 2048 CU11
Entrust Entrust.net Certification Authority (2048) 7/24/2029 503006091D97D4F5AE39F7CBE7927D7D652D3431 sha1RSA 2048 CU11
DigiCert DigiCert Assured ID Root CA 11/9/2031 0563B8630D62D75ABBC8AB1E4BDFB5A899B24D43 sha1RSA 2048 CU11
DigiCert DigiCert Global Root CA 11/9/2031 A8985D3A65E5E5C4B2D7D66D40C6DD2FB19C5436 sha1RSA 2048 CU11
GlobalSign GlobalSign Root CA 1/28/2028 B1BC968BD4F49D622AA89A81F2150152A41D829C sha1RSA 2048 CU11
Entrust Entrust Root Certification Authority – G2 12/7/2030 8CF427FD790C3AD166068DE81E57EFBB932272D4 sha256RSA 2048 CU11
GeoTrust GeoTrust Global CA 2 3/3/2019 A9E9780814375888F20519B06D2B0D2B6016907D sha1RSA 2048 CU11
GeoTrust GeoTrust Primary Certification Authority – G3 12/1/2037 039EEDB80BE7A03C6953893B20D2D9323A4C2AFD sha256RSA 2048 CU11
GeoTrust GeoTrust Primary Certification Authority 7/16/2036 323C118E1BF7B8B65254E2E2100DD6029037F096 sha1RSA 2048 CU11
Go Daddy Go Daddy Root Certificate Authority – G2 12/31/2037 47BEABC922EAE80E78783462A79F45C254FDE68B sha256RSA 2048 CU11
Starfield Starfield Root Certificate Authority – G2 12/31/2037 B51C067CEE2B0C3DF855AB2D92F4FE39D4E70F0E sha256RSA 2048 CU11
Thawte thawte Primary Root CA – G3 12/1/2037 F18B538D1BE903B6A6F056435B171589CAF36BF2 sha256RSA 2048 CU11
Thawte thawte Primary Root CA 7/16/2036 91C6D6EE3E8AC86384E548C299295C756C817B81 sha1RSA 2048 CU11
VeriSign VeriSign Class 3 Public Primary Certification Authority – G3 7/16/2036 132D0D45534B6997CDB2D5C339E25576609B5CC6 sha1RSA 2048 CU11
VeriSign VeriSign Class 3 Public Primary Certification Authority – G5 7/16/2036 4EB6D578499B1CCF5F581EAD56BE3D9B6744A5E5 sha1RSA 2048 CU11
VeriSign VeriSign Class 4 Public Primary Certification Authority – G3 7/16/2036 C8EC8C879269CB4BAB39E98D7E5767F31495739D sha1RSA 2048 CU11

 

From a brief glance at the above table a few things should be apparent:

  • In versions older than CU10 nearly half of the cached certificate are expired or revoked.
  • If the phone is not running at at least the January 2014 Cumulative Update 11 (4.0.7577.4420) then it is not aware of a host of new CA certificates.

Root Cause

Where the problem introduced earlier comes into play is when a Lync environment is using a public certificate issued from one of the vendors in the table above that have made changes to the originally trusted certificates.  Older server certificates assigned to Lync server might have been signed by the original CA, but as those certificates approach their expiration dates any good administrator will preventively update them prior to that date.  So when these new certificate requests are generated on the Lync server and sent out to the same public CA for signing they are coming back issued from a different CA chain, and one that the older phone firmware is not aware of.  When the servers are updated with these new certificates then phones running that older firmware will no longer be able to register to Lync.

If there are plans to update an public Lync server certificates in the near future and there are any Lync IP Phones in the environment then now would be a good time to get those phones on a newer release.

It is required to upgrade to at CU11 (4.0.7577.4420) at a minimum to avoid this problem, but it is always recommended to move to the most recent firmware currently available.


Workaround

If the Lync server certificate has already been changed, or new phones running older firmware have been introduced into the environment, and they cannot successfully register then the following workaround can be used to address this problem.

Note that the hardcoded pre-authentication download attempt against the ‘ucupdates-r2’ hostname which was carried over from the original Office Communications Server Tanjay devices is not applicable here.  The issue is not that the phone is failing to register to the Lync server, but that it cannot even get to that step due to the underlying TLS requirement not being available.  This out-of-the-box update process requires TLS for a portion of the communication, even though the internal URL provided by the device update service uses HTTP.  So without a working TLS connection the device update service is not reachable.

While the following approach is not new to Lync Phone Edition this guidance may not be something that newer Lync administrators are often aware of.  Prior to the introduction of the current Aries family of LPE devices designed for Lync 2010 the older Tanjay devices introduced with OCS 2007 did not support the simplified PIN Authentication based on DHCP Option 43 as that feature was introduced in Lync Server 2010.  In OCS if public certificates where assigned to internal server roles then Active Directory could be used as a central distribution point for the required public CA certificates.  Because the current LPE firmware is based on that older platform then this behavior is still active in all LPE versions.

The process of publishing third party certificates into Active Directory was originally covered by Rui Jose Silva and references to this process can be found in various other articles, some of which appear to have vanished over the years.

Throughout the directions in this article the example scenario is a Lync 2013 Server utilizing the following newer DigiCert SLL certificate for all three usages on the Front End Server:

image

To clarify this is not an issue related specifically to DigiCert certificates, but any public CA unknown to the phone’s older firmware.  DigiCert has a great program for active Microsoft MVPs which allows them to utilize public certificates in personal test labs for researching and writing articles like this.

Retrieve Root Certificates

Before any certificates can be imported into Active Directory they must first be saved into a Base-64 encoded X.509 file.

This can be accomplished by either locating and downloading the file(s) from the Internet or manually exporting them from the Lync server.  The fastest method can be to just get them directly from the vendor’s website, but most public CAs have multiple root and subordinate/issuing servers so finding the correct certificate chain can sometimes be a daunting task.

  • If already familiar with this process and the exact certificates have been identified then locate and download them from the vendor.  Make sure to get all files in the certificate chain in the event that the Lync server certificate was not issued directly by a root server but was instead signed by a subordinate server.  (This is most common as third party CAs rarely have the all-important root CA online and available to sign requests directly.)

Alternatively to ensure that the correct certificates are retrieved it is recommended to validate the current server certificate and then manually export the chain.

  • On the Lync Server use the Lync Server Management Shell to execute the follow cmdlet which will list the currently assigned server certificate(s).

PS C:\> Get-CsCertificate | fl Issuer,Use,Thumbprint

image

Focusing on the Default, WebServiceInternal, and WebServicesExternal usages (and ignoring the OAuthTokenIssuer certificate) a  typical configuration will have the same certificate assigned to all three client-facing usages.  This Lync Server is utilizing a single certificate issued by DigiCert as made evident by the same Thumbprint value being reported for each.

  • On the same Lync Server open a Microsoft Management Console window (mmc.exe) and then add the Certificates snap-in for the local Computer Account.

image

  • Browse to the Certificates (Local Computer) > Personal > Certificates store and then locate the certificate which is most likely the one referenced in the previous output.

image

  • To confirm that the correct certificate was selected switch to the Details tab to compare the Thumbprint value with what was reported by the Get-CsCertificate cmdlet (e.g. 3DFB6A83366BAD70DC4427565A64A07A0733EA49).

image

  • Change to the Certificate Path tab to view the certificate chain for this certificate.  Highlight the root server (e.g. DigiCert) at the top of the tree and click the View Certificate button.

image

  • In the next Certificate window switch to the Details tab and then click the Copy To File button to export the certificate to a text file.

image

  • In the Certificate Export Wizard select the Export File Format as Base-64 encoded X.509 (.CER).

image

  • Save the file (e.g. C:\Temp\DigiCertRootCA.cer) to the local disk and complete the wizard.

At this point the root CA certificate, which is essentially the public key of the certificate, is now in a format suitable for importing into Active Directory.  Because this example includes a 2-tier CA chain the same steps must be repeated for the subordinate certificate.

  • Return to the Certification Path tab of the original server certificate, highlight the subordinate CA entry (e.g. DigiCert SHA2 Secure Server CA), and click View Certificate.

image

  • On the Details tab use the Copy To File button to perform the same export procedure as before, saving the file with a unique name (e.g. C:\Temp\DigiCertSubCA.cer).

Import Root Certificates into Active Directory

The next step is to import these files into the proper Active Directory store location so that the Lync Phone Edition devices will attempt to download them the next time they are powered-on.

Prior to importing these though files first that a look at where they will be stored in Active Directory.

  • On the Lync server launch ADSI Edit (adsiedit.msc) and connect to the Configuration Naming Context and browse to Services > Public Key Services.

CN=Public Key Services,CN=Services,CN=Configuration, DC=schertz,DC=name

CN=AIA

image

CN=CertificationAuthorities

image

Notice that the only object currently stored in these container is for the existing Windows Enterprise Certificate Authority which is AD-integrated by default when deployed in the domain.  If there is no internal Enterprise CA in the environment then these stores will typically be empty.

To import the two DigiCert CA certificates into this store the certutil.exe Windows command can be used, which will place each certificate file in both the CN=CertificationAuthorities and CN=AIA stores located under the Public Key Services container.

  • From the same domain-joined Lync server open a Windows Command Prompt window and issue the following command.

certutil -f -dspublish c:\Temp\DigiCertRootCA.cer RootCA

image

If the action is successful then the CA certificate should be reported as added to both DS store locations in the AD Configuration container.

  • Repeat the step for any additional certificates in the chain.

certutil -f -dspublish c:\temp\DigiCertSubCA.cer RootCA

image

  • Refresh the ADSI Edit window to verify that the new certificates are displayed in the stores as reported.

image

image

The final step is to attempt to sign-in on the phone using the USB-tethering approach so that the AD credentials can be supplied to the phone.  If any older CX700 models are in the environment they are not required to be tethered as the AD credentials can be entered directly on the phone.

If the phone is still unable to register at this point then try one or both of the following tips:

  • Perform a soft reset on the phone to clear any cached credentials and certificates and reattempt signing in.

  • When signing-in enter the User Name in the odd-looking format of the DNS Domain instead of the NetBIOS name in this format: domain.com\username (e.g. schertz.name\jeff).

image

About Jeff Schertz
Site Administrator

Comments

13 Responses to “Lync Phone Edition and Public Certificates”
  1. jonathansteeman says:

    Another awesome post. Thanks!

  2. @GregSeeber says:

    Thanks for the post, Jeff. My biggest problem these days is Exchange integration. I have a CX600 at home, connected over the edge of course. When I tethered the phone and logged in – Exchange integration was present. However, after a couple of weeks – i noticed the notification about "exchange integration failed". For some reason, it just stops working … even though my userid/password does not change. Any ideas on that one?

    • Jeff Schertz says:

      That issue has plagued random versions of LPE and can be caused by a number of different things. Try a few different firmware versions to see if any one version work better than another to start. Also double-check your Exchange Autodiscover configuration.

  3. JaphethNolt says:

    I followed the steps in this article and this successfully allowed the phones to sign in. But then conferencing didn't work and I narrowed it down to this article. http://support.microsoft.com/kb/2795828 My question is, if I add the Intermediate certs to the Intermediate Store in AD using (certutil -dspublish -f c:tempDigiCertSubCA.cer SubCA), will the LPE phones still be able to sign in?

    • Jeff Schertz says:

      I haven't tried that yet but that may be a more appropriate container for subordinate certificates. Try it out an report back with the results.

  4. @MaxSanna says:

    HI Jeff,
    I had this same issue with a customer recently: their phone stockpile came with a pretty old firmware, and they were using public Globalsign certificates on the frontend pool (to simplify BYOD etc.).
    The globalsign root has expired in January this year, so the phones couldn't sign in if they carried the old firmware.
    The workaround though was slightly simpler: instead of loading the certificates in AD (which was also an option, but the AD team was very restrictive with making these changes), I loaded them in the certificate web service. By making sure the certificate web service was reachable both on port 80 and 443 the phones could go in and update their root store successfully. Also, in my case the phones couldn't login either via USB or PIN Auth. The workaround fixed both.

    The cmdlets I used were:
    $x = New-CsWebTrustedCACertificate -Thumbprint "D543DFF74FEEA425162FD25F342786F1AB453BB3" -CAStore TrustedRootCA
    Set-CsWebServiceConfiguration -Identity site:Redmond -TrustedCACerts @{Add=$x}

    (and again for the intermediates with the switch -CAStore IntermediateCA), until the whole globalsign chain was loaded, and then everything started working. Hope it helps..
    Max

  5. Jan Pettersson says:

    Great info.
    Do you know if this is still an issue?
    https://heavens-reach.com/tag/rsassa-pss/
    We have a CA structure with RSASS-PSS-certiifcates…. :-S
    Not to happy about rebuilding both ROOT-CA and intermediatie CA.

  6. Wes says:

    Hi Jeff, great article is always.

    How does this work in a situation where you have multiple edge pools? We have two SRV records that point to our two edge namespaces… sipeast.domain.com and sipwest.domain.com

    We do have sip.domain.com on our cert, but I realized that it does not actually exist in DNS, nor is it referenced by any SRV records. Our CX phones work just fine in this configuration. As our cert renewal is coming due, I’m trying to figure out if I need to keep the sip.domain.com name on it.

    How about in the same situation, but with multiple sip domains? Do we need to have sip.domain2.com, sip.domain3.com etc on the cert?

    Thanks!

  7. mickeypsu says:

    Do we know if sha256 is testing with CX700, the post seems to indicate cu11 is required however running cx700 with vs 7577.4494 and keep getting “cannot download the certificate because domain is not accessible”

    thanks

Speak Your Mind

Tell us what you're thinking...
and oh, if you want a pic to show with your comment, go get a gravatar!