Troubleshooting Lync Phone Edition Issues
This article serves as a follow-up to a few previous articles which will further explain some of the requirements, capabilities, and limitations of the Lync Phone Edition firmware which appear to still be unclear to some and seem to warrant further discussion. It does not cover complete feature functionality but instead focuses on provisioning and operation of the client as well as some of the most common issues.
Additionally the focus of this article is primarily on the Lync Phone Edition firmware which runs on the Aries family of devices (Polycom CX500/600/3000, Aastra 6721ip/6725ip, and HP/SNOM 4110ip/420ip) but also references the original Tanjay version (Polycom CX7000, LG-Nortel 8540) on a few occasions. The following list of topics are discussed in this article.
- Authentication Methods
- Exchange Integration
- External Usage
- Common Issues
- Firmware Updates
It is suggested to review the following blog articles as some of the concepts in this article are explained in much more depth and thus a solid understanding of how the devices function can often be key to interpreting the observed behavior of the device.
Lync supports a variety of different authentication methods and understanding which method is used, when, and why is important when troubleshooting any issues.
There are currently three different types of authentication supported by Lync Server 2010: NTLM, Kerberos, and Certificate.
By default Kerberos is not completely functional throughout Lync Server as some additional configuration steps must be completed first. But both NTLM and Certificate authentication methods are enabled and used by default for both the Windows Lync client and the Lync Phone Edition client.
Although NTLM authentication needs no introduction, this new-for-Lync certificate based method is still a mystery to many. It is more specifically known as TLS-DSK and for some background on what it is an how it works the following materials are recommended reading.
- MSDN SIP Protocol Overview: http://msdn.microsoft.com/en-us/library/dd905435.aspx
- Microsoft Lync 2010 Technology Explained: http://channel9.msdn.com/Events/TechEd/Europe/2010/UNC310
As the Windows Lync client supports both types of authentication what typically happens is that a user signs into the application using their Active Directory credentials for the first time and during this process the Lync Server will submit a client certificate to that user which the client application then stores in its local cache. This client certificate can be used for future authentication attempts against any Lync Server registrar (Front End, Director, Edge, SBA) and explains why the Lync client can still successfully sign-in even after a user’s AD account password has expired (or the account has even been disabled). This cached client certificate on endpoints can be valid in Lync Server for between 6 to 12 months, which is typically much longer than most environment’s AD password expiration policies. By default the Lync client requests a validity period using the minimum value (180 days) when a user signs into the client on a specific endpoint for the first time.
For more details on the certificate validity parameters see the Set-CsWebServiceConfiguration cmdlet documentation.
MaxValidityPeriodHours When using certificate authentication, clients can request the period of time (in hours) that the certificate remains valid. MaxValidityPeriodHours represents the maximum amount of time a client can request.
MaxValidityPeriodHours can be any integer value between 8 hours and 8760 hours (365 days). The default value is 8760.
MinValidityPeriodHours When using certificate authentication, clients can request the period of time (in hours) that the certificate remains valid. MinValidityPeriodHours represents the minimum amount of time a client can request.
MinValidityPeriodHours can be any integer value between 8 hours and 4320 hours (180 days). The default value is 8
The new client certificate is then replicated to all other Lync registrars so that if that same user later attempts registration to a different servers from a different client endpoint then the same issued certificate is available to be downloaded to the new endpoint. Although the client certificates cannot be viewed under Lync Phone Edition, there are a few different ways to view them on a Windows workstation.
- Run mmc.exe and then add the Certificates snap-in. Select My User Account (instead of the Computer account which is normally chosen when dealing with server certificates).
- Expand the Personal container and then the Certificate container to see all of the client certificate stored in the current Windows user’s profile.
In the example above it is evident that multiple different Lync users have signed into Lync within the same Windows user profile. Normally only one client certificate issued by ‘Communications Server’ would appear in a standard user’s store.
- Alternatively open the User Accounts control panel in Windows 7 and select Manage your credentials to see the same list of client certificates under the Generic Credentials header.
In this example one important distinction can be made in that the same set of credentials is sometimes listed twice (with either an OCS:1 or EWS:1 extension) and each showing a different username (SIP URI versus Domain\Username).
As covered extensively in previous articles, PIN authentication is only available on the Aries family of Lync Phone Edition devices and this method of signing into a phone using only the user’s phone number (or extension) leverages only the TLS-DSK certificate method. Standard NTLM simply cannot be used here as there is no way for a user to key-in their full AD credentials in a DOMAIN\username or User Principal Name format.
Alternatively the touch-screen equipped Tanjay devices (e.g. CX700) does support NTLM because a user is presented with a full keyboard in which a SIP address, AD credentials, and password can be entered directly from the device. Certificate-based PIN Authentication is not supported in the Tanjay firmware release.
Also know as ‘Better Together’ this approach is only applicable for devices which have a USB Type B connector. When a device is USB-tethered to a supported Windows PC running the Lync client (Mac OS is currently unsupported and this does not work with the Lync:mac client) then the user is prompted for additional login information, as shown in the following screenshot:
The sole purpose of this dialog box is to provide a way to get the user’s Active Directory credentials into a device which does not contain the ability for the user to enter it in directly on the device. When a Tanjay device is USB-tethered and this prompt appears it is simply just an easier way to do what can be done directly from the phone’s touch-panel interface: enter in the AD credentials. But for an Aries device this is the only way to get the AD credentials into the phone.
This USB-based approach to authentication offers two distinct advantages over the PIN Authentication method:
- Simpler sign-in process for users
- Exchange Integration capabilities
The key concept to understand here is that only Lync Server supports TLS-DSK certification authentication and Exchange Server does not. Only NTLM authentication can be used by any Lync clients to directly connect to the Exchange Client Access Server to access the account’s mailbox.
Any device which does not currently have valid AD credentials cached (e.g. PIN Authentication was used) will not be able to successfully authenticate to an Exchange Server. Thus only devices with USB-tethering capabilities (e.g. CX600) or the ability to enter AD credentials directly on the device (e.g. CX700) will ever be able to leverage Exchange integration features. Primarily the CX500 will never be able to connect to Exchange as it does not include a USB connection since it is designed as a Common Area Phone. Common Area devices do not support Exchange integration as they are primarily designed to be used with the specific Common Area Phone accounts in Lync which are Active Directory contacts that are SIP-enabled for Lync in a special way. Since AD contacts cannot be Mailbox-enabled in Exchange then these accounts will never have any Exchange data to integrate with in the first place (e.g. Calendar or Voice Mail).
When Exchange Integration is not currently available for the signed-in user then the Menu button icon will change to an alert icon and the notification “Microsoft Exchange integration unavailable” will be displayed:
When the integration is functioning on the device then standard Exchange-enabled mailboxes will provide basic integration features like Calendar data, but only UM-enabled mailbox will provide Visual Voice Mail information. The following screenshots show the various related menu options as a side-by-side comparison between an Aries device using PIN Authentication and one using USB-tethering.
- There is a known issue that if Exchange UM is not enabled on the account’s mailbox then the Call Log information also will not be displayed, even though the Conversation History folder in the user’s mailbox is not a Unified Messaging feature.
- Although Exchange authentication has failed on the device on the left, because the Lync account is UM-enabled in Exchange the option to ‘Call Voice Mail’ is still displayed, offering one-touch access to the Exchange Subscriber Access number. As seen on the right Visual Voice Mail providing one-touch retrieval of individual voice messages is provided.
- One important exception is when using Exchange Server 2010 the Voice Mail Message Waiting Indicator (MWI) status can actually be updated directly from the Lync Server (this functionality is not available in Exchange Server 2007). When an Exchange Unified Messaging user receives a new voice mail message the UM service will send a SIP notification message directly to the Lync Server, which in turn will send this on to the registered Lync users via a presence update. So even without any access to Exchange Web Services directly from the device the unread Voice Mail is updated on the main menu, as well as the physical MWI lamp on equipped devices (e.g. CX600). The screenshots below shows the Exchange integration warning icon on the main menu, yet the ‘1 New Voice Mail’ indicator is still shown, as well as the MWI lamp is illuminated.
- Another common misunderstanding is around the topic of Outlook Contact integration. It is often believed that Outlook Contacts are not available in Lync Phone Edition, but this is not correct. Any contacts which only exists in the user’s Outlook mailbox and are not also added to the Contact List in Lync are still available on the phone, but only via searching, not via browsing. The Contacts menu only displays objects on the Lync contact list, but the Search menu will return results from either the downloaded Lync address book device files or the user’s Outlook mailbox, of course assuming that Exchange integration is currently available and functioning.
The ability to provision a Lync Phone Edition device out-of-the-box is covered in detail in this previous blog article but this process only works on some devices, not all. Any of the Common Area Phone models which do not contain a USB port (e.g. CX500) should not be used externally.
These common area devices are incapable of being provisioned externally as even if the required PIN Authentication settings were manually configured in the external network’s DHCP services to point to an Edge Server, the device can neither connect to the Web Ticket service nor authenticate in this fashion.
It is possible although to first provision the device internally on a network where the DHCP options are configured correctly and the device can successfully download and cache a client certificate for authentication to Lync. Then that same device can be taken off-site (e.g. home office) and successfully sign-in to the Edge server using the cached user credentials. This will only work until the client certificate expires though, at which point the device would need to be brought back into the internal network to be re-provisioned. Clearly this is not a viable solution for remote workers, but it is possible to use for demonstration or temporary testing purposes. Officially Microsoft does not support use of Common Area Phone models externally.
Branch Appliance WAN Outage
In a typical Survivable Branch Appliance (SBA) scenario where Lync Phone Edition phones are deployed in the branch office it is important to understand that it is not possible to sign-in to a new or reset device during a WAN outage.
Devices configured to support PIN Authentication in the branch office would be directed towards the SBA registrar via DHCP Option 120, yet Option 43 would still be pointing back to a Front End Pool typically in main office or datacenter because the SBA does not contain any Lync Web Services, thus it does not host a Certificate Provisioning service.
What this means is that during a WAN outage scenario all Lync clients and devices which already are signed-in to the SBA using cached credentials can still authenticate directly to the SBA, so clients can be successfully signed-out and signed-in during this outage. Lync Phone Edition devices can also sign-in after a reboot using cached credentials as well, but if a device is manually signed-out, factory-reset, or is brand new out-of-the-box then it cannot be provisioned through either PIN Authentication or USB-tethering methods until the WAN is back online as the device will have no way of reaching the Front End Pool to authenticate and receiving a client certificate. (In this scenario if Internet access is still available in the branch office and there is an Edge deployment in Lync then the branch users could potentially use USB-tethering while signed-in to an Edge server to provision a phone.)
Backup Registrar Pool
When an associated backup registrar pool is assigned to a Front End pool in the Lync topology then the Lync Phone Edition clients can automatically failover to that secondary server or pool just like a standard Windows Lync client.
But in earlier versions of the Lync Phone Edition firmware (this has recently been resolved with Cumulative Update 5 4.0.7577.4066) there was a known issue which prevented devices signed-in using Common Area Phone accounts from automatically reconnecting to either a secondary registrar (during a failover) or to the primary register (during a brief outage). This issue was related to the configured HotdeskingTimeout parameter on the client policy assigned to the device. If the timeout interval elapsed before the device was able to successfully connect to a registrar then although the screen would show that it was still ‘retrying’ a connection it was in fact no longer attempting to connect to the server and it would need to be rebooted in order to connect again.
Get-CsClientPolicy | Select-Object Identity,HotdeskingTimeout
The issue would typically occur when the hotdesking timeout (5 minutes by default) was reached before either the failure detection interval elapsed (in the event of a failover) or if the primary registrar was offline for more than 5 minutes (in the event of a server reboot or extended maintenance window). Meanwhile all other device with standard Lync user accounts sign-in, as well as Windows Lync clients would all successfully failover or reconnect to the working registrar after any time interval (hours or even days). So in this scenario if the failover interval was measurably longer than the hotdesking timeout, or if the a Front End server was offline for longer then the devices with Common Area Phone accounts would never sign back in without manual intervention.
Below is a list of the issues which can often prevent a device from working as designed with a Lync Server deployment. This is by no means a comprehensive list, yet most of the common issues are addressed.
Lync Registration Issues
- Ethernet Required – This seems basic but quite often customers have attempted to use these devices as USB-only handsets and they simply will not function in this manner. The device must be connected to an Ethernet switch at minimum and receive a valid DHCP lease on the network in order to contact the Lync Server, even if USB-tethering is being used to provision the device.
- Enterprise Voice Required – The only CX device that completely functions without the user’s Telephony setting configured for Enterprise Voice is the CX100 USB. When the user account is still in the default PC-to-PC Calling mode then the CX200/CX300 USB devices will function as audio devices but the presence lights do not function. The IP-based devices (CX500/600/700/3000) will not work at all. PIN Authentication will fail, as will USB-tethering since the Lync client will never prompt for the user’s credentials. Note that actual PSTN integration is not required as it is still possible to simply set the Telephony mode to Enterprise Voice but still leave the Line URI field blank. (Update: as of the August 2014 Cumulative Release (.4451) Enterprise Voice Telephony Mode is no longer a requirement).
- Using Old Firmware – This one should be obvious but many of the most common issues seen in the field can be resolved by simply upgrading to newer firmware which has already addressed the root cause. One of the more common issues is when the preinstalled public root certificate on older versions is missing, has expired or has been revoked. More recent firmware versions contain new or updated root certificates but the phone would need to be registered to a different environment in order to apply the firmware update needed to register successfully to the original environment. This is just one example, but whenever troubleshooting any issues the first step should always be to identify the firmware version on the phone and get it upgraded to the latest release.
- Missing Automatic Configuration Records – Lync Phone Edition cannot utilize Manual Configuration and requires that Automatic Configuration records for Lync clients are properly defined in DNS. So if Windows Lync clients are manually pointed to a Lync server (either directly in the client or passed via GPO) this information is not passed to a tethered phone, the phone must perform it’s own automatic lookup. Thus is the proper SRV or Host (A) records are not available in the DNS server provided to the phones then they will fail to sign-in, even when tethered. If the proper SRV records are not deployed for Lync then the only other option is to use the sip.<sipdomain>, sipinternal.<sipdomain>, or sipexternal.<sipdomain> hostnames for the Lync server, pool, or Access Edge names.
- Unsupported Version – If a device ever displays a screen which simply states "The hardware version of your phone is no longer supported and is now unusable. Please contact your support team to get a replacement" then that device is essentially ‘bricked’. This only happens on pre-production hardware and in Cumulative Update 2 Microsoft no longer supported any beta devices, so if this happens contact the vendor directly to swap out the hardware in the event that a demo unit was left out in the wild somewhere after the Lync 2010 Server software was officially launched.
Exchange Integration Issues
- Expired Credentials – One of the most common reasons for Exchange integration issues is related to the AD credentials in the phone expiring. As previously mentioned authentication to the Lync server happens using a client certificate (TLS-DSK) but Exchange authentication happens using NTLM which requires the AD user’s credentials. If the AD user account’s password has expired then it will still be able to connect to Lync using the client certificate, but NTLM authentication to the Exchange server will fail triggering the "Microsoft Exchange integration is unavailable” error message. In this scenario the device will not prompt the user for new credentials, even when actively USB-tethered to a Lync Windows client. The current user account must manually be signed out of the device and back in to successfully update the password.
- Message Waiting Indicator – A number of devices include a Message Waiting Indicator (MWI) lamp which can be illuminated via Exchange integration (messages directly from the Exchange server) or from the Lync Server through SIP registration (via a SIP NOTIFY message). Some of the common area phone models which do not include a USB port still contain a MWI lamp (e.g. the HP 4110ip). Although these devices cannot authenticate to Exchange directly when the user has a new voice mail message the Exchange UM server will always notify the Lync server, thus the Lync client can receive the new status information from the Lync Server. The Exchange Team Blog has a detailed article which covers troubleshooting MWI status in great detail.
One of the more common causes for this can be identified by looking for an MSExchange Unified Messaging error event of 1344 on the Exchange UM server which usually is related to a certificate configuration issue between Lync and Exchange. During TLS communications between Lync and Exchange UM the Lync server only accepts the Common Name (aka Subject Name) value of the certificate assigned to the Exchange UM service and completely ignores the Subject Alternative Field (SAN) entries. Thus if the Exchange UM service is assigned to a SAN cert which includes the server FQDN in the SAN (e.g. cas01.contoso.local) but a different FQDN in the Subject Name field (e.g. mail.contoso.com) then the MWI update message from the UM server to Lync will fail, and the client will never receive the notification. It is not until the client polls Exchange again that the new voice mail will be known about. This scenario normally translates into a longer delay for unread and read voice mail counts to refresh on any Lync client. The recommended resolution is to issue a separate certificate with only the Exchange server FQDN as the Common Name (and no SAN field) to the Exchange server and assign it to only the UM role. Alternatively a SAN certificate for all Exchange roles can still be used but the Common Name must be the server FQDN which does not always match best practices (and is not how the Exchange certificate wizard will format the certificate request by default).
- Missing Exchange Autodiscover SRV Record – This issue is a bit odd as it does not seem to be a definitive requirement. In some deployments Exchange integration has failed on all phones until the _autodiscover._tcp.contoso.com DNS Service Locator (SRV) Record was created; it appeared that the devices would simply refused to use the fall-back Host (A) record to locate the Exchange Client Access Server. Yet in other environments Exchange integration would perform normally yet the autodiscover SRV record never existed. But as it is best practice to deploy that SRV record for a properly designed Exchange autodiscover configuration then it is one of the first things to check when troubleshooting Exchange integration issues.
- Untrusted Certificates – Sometimes the Exchange Client Access servers (or load balancer) will have a certificate issued by a different Certificate Authority than what is used on the Lync servers. Often this can be a scenario where the Exchange server is using a public certificate while the Lync Front End Server has a private certificate. But since Lync Phone Edition has a shorter list of pre-installed public trusted certificates than the Windows client does then it’s possible that the device does not trust the certificate in use on Exchange. This also applies to using private internal certificates issued by a completely different certificate authority chain then was used to issue certificates to the Lync Server. Lync Phone Edition can only download the root certificate chain used on the Lync Server and is unable to download or store any additional certificates from a different CA.
- Wildcard Certificates – As previously discussed in this blog article Lync Phone Edition devices do not currently like wildcard entries in the Lync Server certificates. This carries over into Exchange integration as well, so if the Exchange certificate contains wildcard entries then the phones will also fail to establish a TLS connection with the Exchange Client Access Server thus preventing a successful authentication attempt. This issue can be identified by the error code 0x80072f06 reported in the device client log files and Mike Stacy has highlighted this issue in a past blog article as well.
Begin by reading through the Updating Lync Phone Edition Devices article to get understand how the update process works. Whether a Test Device is being used or a new update has been approved for a specific device model and revision then the phones should just automatically update themselves and reboot into the new version when left idle. There is no way to force the update and the more that the process is messed with (or seemingly the longer one stares at the phone) the less of a chance it will successfully complete the update.
So when updating a device it is recommended to reboot the phone and just let it sign in with the currently cached user account and then immediately lock; do not touch the device. In a normal situation the device will check for a new update upon initial sign-in and then begin downloading the package about 1 minute later, rebooting into the new firmware roughly 7 minutes after that. These time estimates depend on network bandwidth as the packages may take longer to download in some environments. But a good rule of thumb is that if the device has not updated after sitting untouched for ~30 minutes then something is preventing a successful update and some troubleshooting steps are required.
Normal Update Process
To briefly revisit the firmware update processing originally discussed in this article when a package is approved for a specific phone model and revision (or a test device is defined via the MAC address) the easiest way to start troubleshooting is to verify if the device has even checked in yet to ask the server if a new software version is available. This process happens on each boot sequence and systematically throughout the day as the devices experience inactivity periods in between use, as it would not be desirable to have the phones rebooting themselves to update firmware while a user is on a call.
- To follow the device check-in process first look at the IIS log for the associated web services directory on the appropriate Front End server: W3SVC34577 for the Internal Web Site or W3SVC34578 for the External Web Site.
- Search for the “RequestHandler” to locate the latest POST log entries, comparing the device IP address or MAC address to identify the correct line items. The initial POST message shows the device connecting to the /RequestHandler/ucdevice.upx service.
2012-03-17 15:35:01 192.168.1.23 POST /RequestHandler/ucdevice.upx – 443 – 192.168.1.108
Microsoft+UCPhone+Device+(lcs_se_w14_main:985232:2011/11/09:16:08:29) 200 0 0 128
- Then move to the latest RequestHandlerAuditLog_lync_xxxxxxxx.log stored under the imageUpdate log folder on the Lync Server to see what the service logged about that client request. The following text coincides with the request timestamp, which happens 4 seconds after the request logged in IIS from the same host. (Pay close attention to the hour as the IIS logs and Update Service logs do not use the same format; IIS uses the local time zone while the update logs use UTC.)
The logged response shows that the update service has passed the internal and external URLs for the device to retrieve the update package from.
03/17/2012 10:35:05,firstname.lastname@example.org, 192.168.103.108,UCPhone,3/17/2012 8:34:59 AM,"0004F2953DFC","0004f2953dfc", "POLYCOM","CX500","Rev-4","ENU",cpe.nbt;4.0.7577.4047;11/9/2011 5:32:04 PM, http://lync.schertz.local/RequestHandler/Files/UCPhone/POLYCOM/CX500/Rev-4/ ENU/4.0.7577.4066/CPE/CPE.nbt; https://external.mslync.net/RequestHandlerExt/Files/UCPhone/POLYCOM/CX500/Rev-4/ENU/4.0.7577.4066/CPE/CPE.nbt; 4.0.7577.4066;2/18/2012 8:49:56 PM
For comparison, when a device has the latest approved firmware already installed then the response from the update service will not include any URLs, as demonstrated here:
03/17/2012 10:42:35,email@example.com,192.168.103.108,UCPhone,3/17/2012 8:42:34 AM,"0004F2953DFC","0004f2953dfc","POLYCOM","CX500","Rev-4","ENU",cpe.nbt;4.0.7577.4066;2/18/2012 8:49:56 PM,
- Once it has been confirmed that the device was given a path to download the update the IIS log can be checked again and sometime shortly after the original POST message there should be a GET message showing the actual device requesting the file from the web server. The values at the very end of the lines in the IIS logs is the time elapsed for that request, so the log shows that the download of the CPE.nbt firmware file (52MB) took 47685 milliseconds, or 47 seconds.
2012-03-17 15:35:53 192.168.103.23 GET /RequestHandler/Files/UCPhone/POLYCOM/CX500/Rev-4/ENU/4.0.7577.4066/CPE/CPE.nbt – 80 – 192.168.103.108 Microsoft+UCPhone+Device+(lcs_se_w14_main:985232:2011/11/09:16:08:29) 200 0 0 47685
The example below (click the screenshot for a full-resolution image) shows the entire process of a Polycom CX500 located at 192.168.1.23 requesting the CU5 update package (4.0.7577.4066).
Troubleshooting Failed Updates
In the event that the device neither requests an update or does but seems to never actually update itself then additional steps can be performed to narrow down the issue.
The first item to check is the Last Update Status code shown on the device’s System Information menu, which can be used to identify if the device encountered an error in the update process. There is also a Send Logs button on this screen but in reality these logs are not very helpful. Firstly the device must be able to successfully sign in to the server before it can actually send the logs by uploading them to a directory on the Front End server. But then the resulting file is a unusable .clg1 Windows CE error log which requires the nearly-impossible-to-find readlog.exe utility. from Microsoft. And even when the logs are converted into text with this tool they are still very difficult to read and are primarily meant only for Microsoft PSS to troubleshoot from.
Most often recoding the status update information will point to the root cause and the client logs are not needed. the status code is actually two individual codes, the first value displayed in hexadecimal format is the WinInetError code and the second value in decimal format is the HTTPError code. The WinInetError codes can be reported generically (e.g. 0x2, file not found) or more specifically (e.g. 2F0D, invalid certificate trust).
When an issue prevents the device from downloading the update then the code value will often contain a non-null value, but searching online for that error code will often return nothing helpful. This is because the error code needs to first be converted to decimal, then the resulting error code will be much more helpful.
- For example assume that the Last Update Status is reported for a device as 0x2ee7/0. The hexadecimal value of 0x2ee7, simplified as just 2EE7 converted into decimal is 12007.
With that decimal value the following Microsoft articles can be used to lookup the actual error code description. Each page is not a complete list so make sure to search both references as some error codes are only listed in one them.
- Using the table in one of the articles above error 12007 is described as “ERROR_INTERNET_NAME_NOT_RESOLVED the server name could not be resolved.” This would point to the device failing to resolve the FQDN used in the download URL passed by the update services.
Following is a list of common status message and resolutions:
- 0x0/200 – This status is a normal state of no WinInetError (0x0) and an HTTP 200 OK status message, meaning that no error occurred and that device is on the latest approved version and there is nothing to do. If this is not the case and the device should be updating to a newer version verify that the version is approved for the proper revision. For example the Polycom CX600 device has multiple hardware revisions that use unique firmware packages which are reflected by the Rev-3, Rev-4, or Rev-5 label in the Part Number field displayed in the System Information menu.
- 0x5/404 – This code is a basic HTTP 404 NOT FOUND error and indicates that the device was unable to connect to the device update web service in order to download the firmware package. This is a common error in deployments where a traffic is not properly forwarded to the Front End servers. Often times only HTTPS traffic over TCP:443 will be configured on a Hardware Load Balancer and HTTP traffic over TCP:80 is accidentally omitted. As most other Lync client functionality is unaffected it is not until attempting to get Lync Phone Edition devices to work that this missing configuration can become a problem. As indicted in the device log examples in the earlier section any internally-registered devices will be passed the internal device update URL with an HTTP header, so if the phone is unable to connect over port 80 then the update process will fail here. Insure that the Lync Front End server’s internal web services are available over both port 80 and 443 to resolve this.
- 0x7/0 – This generic code entitled ERROR_ARENA_THRASHED means that the storage control blocks were destroyed and that, more simply, the new firmware was unable to be written to the device due to some corruption in the file system, or that there is no free space remaining. The resolution is to perform a hard reset (aka factory reset) on the device and then the let the upgrade process perform again and the newly rewritten file system should no longer contain the corruption.
- 0x5/401 – This error indicates ERROR_ACCESS_DENIED in the first code followed by a common HTTP 401 Access Denied error as well, indicating a general authentication failure of the device accessing the IIS web service when attempting to download the new firmware package. In scenario there may be an account permissions issue or a general configuration issue in IIS preventing proper access to the web server directory.
- 0x2EE7/0 – As described in the example above this error code of 12007 is described as ERROR_INTERNET_NAME_NOT_RESOLVED and means that the device is failing to resolve the FQDN of the download URL passed by the update service. This is normally the Internal or External Web Services FQDN in Lync.
- 0x2F0D/0 – This error code converts to 12045 in decimal and is certificate related based on the descriptive name of ERROR_WINHTTP_SECURE_INVALID_CA. This typically happens when an untrusted public certificate is installed on the web service (or reverse proxy for external device updates) and it was not issued by one of the trusted Root certificate authorities that Lync Phone Edition supports be default. See the TechNet article Certificate in Lync 2010 Phone Edition for a complete list of the preinstalled CA certificates. Also be on the lookout for Hardware Load Balancer configuration using SSL Offloading as even though the internal Front End services may be using an internal private certificate that the phone trusts the load balanced web services sometime can be installed with a different certificate which the device does not support, effectively breaking only device updates.
Each device family has a different process for rolling back the firmware to a prior version but the behavior and results are nearly identical. Both versions contain two separate reset procedures. First there is a soft reset which only wipes the phone’s current configuration data (user credentials, certificates, etc.) but does not impact the firmware. The second is a hard reset (or factory reset) which not only wipes the configuration data but also rolls the firmware back to the previously installed version.
Calling this second process a ‘factory-reset’ is technically incorrect as only the previously installed firmware can be restored, there is no factory firmware image stored anywhere on the phone. There is also no specifically designed ‘factory’ version as depending on the model revision and manufacturing date it could have come with a beta, RTM, or any CU version pre-installed. Additionally, if a device ever crashes during bootup and begins to be stuck in a crash-loop due to some file corruption for example, then after the 5th successive reboot the device will automatically perform a firmware rollback. Take note that once the device is rolled back it will immediately attempt a firmware upgrade if the update services still have that newer version approved for that device model and revision (or if it is defined as a Test Device).
Both device families use the same upgrade process in that each device contains two separate ‘partitions’ where firmware is stored: an active partition and an inactive partition. Under normal operation the device is working on its active partition and the inactive partition will have the previously installed firmware, whatever version that might have been for that device. So as multiple versions are loaded on a device over its life it becomes apparent why it can never truly be reset back to factory as each new installation simply overwrites the older, inactive partition data and the device simply ‘flip’s’ the partitions after a successful upgrade. For example a device originally shipped with RTM firmware which was systematically upgraded over the past year to CU1, then CU2, then CU3, then CU4, and finally with the most recent CU5 is only able to move back one version. It is also not possible to force installation of an older version using the device update services. (That being said, there is an unsupported workaround which can be used to roll a Tanjay device from 4.0.x Lync firmware back to the 3.5.x OCS R2 version.)
The process to reset a Tanjay device is triggered by depressing a recessed reset button (most easily with a straightened paperclip) which is located on the bottom of the handset in between the USB and headset ports.
- To perform a soft reset on a Tanjay device simply depress the switch while the device is currently powered on and it will immediately reboot back to the initial sign-in screen with the current user data wiped.
- To perform a hard reset on a Tanjay device just repeat this process 4 more times (for a total of 5 resets) in rapid succession to trigger the built-in failsafe firmware rollback.
The process to reset an Aries device is triggered by holding down certain keys while powering on the device and is documented in more detail in this previous article.
- To perform a hard reset on a Aries device simply hold down both the * and # buttons while connecting the power. Continue to hold the keys for approximately 10 seconds until the following screen appears, indicating that all user data and configuration settings will be erased.
- To perform a factory reset on a Aries device simply hold down both the 4 and 6 buttons while connecting the power. Continue to hold the keys for approximately 10 seconds until the following screen appears, indicating that the phone will be rolled back to the previously installed firmware version as well as erasing all settings.