Troubleshooting Jabber

Like a TAC Engineer

Harmit Singh, Technical Leader - Services
Agenda

Troubleshooting Logins
Troubleshooting Jabber Features
Tools Used By Jabber
Which Jabber Client Generates the Highest Number
of Cases for TAC?
Jabber for Mac
Jabber for iOS
Jabber for Android
Jabber for Windows
Agenda

Troubleshooting Logins
Service Discovery
Logins
Certificates

Troubleshooting Jabber Features

Tools Used By Jabber
Service Discovery Overview
Enables clients to automatically detect and locate services
Queries network for DNS SRV records
Allows administrators to centrally configure Jabber with server side changes.
Removes the need to use bootstrap file or having to instruct users to manually
configure client
Goal of discovery is to find a primary source of authentication
Typical Service Discovery Questions Seen By TAC
The Jabber client is unable to discover our SRV record
Service Discovery Flow
User Launches Client Determines Client checks if network is inside or outside the
Client Services Domain firewall, and if Expressway is deployed

Client sends cloud HTTP request Client starts monitoring for network changes
to check for WebEx service

Yes
Connect to Cisco WebEx Messenger service Connect to available services on corporate network
No

Client looks Found?

Yes
Connect to available services on corporate network
for _cisco-uds
No

Client looks Found? Yes

Connect to available services on corporate network
for _cuplogin
No

Is Expressway Yes
Omitted
Available?
No
Prompt user to manually enter connection details
Jabber Client Showing Service Discovery Failed
Service Discovery Failure Code and Logs
All detailed discovery-related logs have [service-discovery] logger name in them

Jabber client csf-unified.log

Example: 2014-03-15 17:07:16,829 WARN [0x00000854] [vices\impl\DiscoveryHandlerImpl.cpp(414)]

[service-discovery] [DiscoveryHandlerImpl::callOnFailedDiscoveryResultOnDispatcherThread] - Discovery
Failure -> (id) name :: (1005) ServiceDiscoveryNoSRVRecordsFound
Service Discovery Failure Codes
ID Name
1001 ServiceDiscoveryFailure
1002 ServiceDiscoveryAuthenticationFailure
1003 ServiceDiscoveryCannotConnectToCucmServer
1004 ServiceDiscoveryNoCucmConfiguration
1005 ServiceDiscoveryNoSRVRecordsFound
1006 ServiceDiscoveryCannotConnectToEdge
1007 ServiceDiscoveryNoNetworkConnectivity
UI message
Failed to discover network services.
Your username or password is not correct.
Cannot communicate with the server.
Failed to discover network services.
Failed to discover network services.
Cannot communicate with the server.
Cannot communicate with the server.
Description
Unknown discovery failure
Failed to authenticate with CUCM (9.0+)
Cannot connect to CUCM (9.0+)
CUCM server is misconfigured
No SRV records are found
Cannot connect to EDGE server
No network connectivity
Failed nslookup of SRV record
Packet Capture of Service Discovery Failed
Configure SRV in DNS
Configure SRV in DNS
Configure SRV in DNS
Configure SRV in DNS
Successful nslookup of SRV record

10.10.10.51
Successful Service Discovery
Packet Capture of Successful Service Discovery
Debugging Service Discovery Cache File
Located in %appdata%\Cisco\Unified Communications\Jabber\CSF\Config\service-location.xml
<?xml version="1.0"?>
<UCServices>
<DomainName>ciscolive.com</DomainName>
<UCService>
<type>CUCM</type>
<connectionInformation>
<name>_cisco-uds</name>
<scope>UNKNOWN</scope>
<address>cucm9.ciscolive.com</address>
<protocol>_tcp</protocol>
<port>8443</port>
</connectionInformation>
</UCService>
</UCServices>
Reset Jabber to Install state
Jabber can be reset to Install state by removing both
ProgramData
roaming and local profiles.
Users
[USERNAME] Jabber will reinitialise on next startup
AppData
Local
Unified Communications
Local Jabber
CSF Contacts
Profile History
Logs
Roaming Photo Cache
Unified Communications
Roaming Jabber

Profile
CSF Config
Credentials
Erase Jabber folders
CustomEmoticons
Security
Agenda

Troubleshooting Logins
Service Discovery
Logins
Certificates

Troubleshooting Jabber Features

Tools Used By Jabber
Common Login Questions Seen By TAC
Users are unable to login for a new deployment.
Only this one user is unable to login.
All users suddenly can not login.
Incorrect Password Entered by End User
Check End User Page Login
Check End User Page Login
What Do the Logs Show?
User is not licensed
IM&P Server - EPASSoapXXXXX.log (Client Profile Agent Logs)

2014-03-23 14:21:54,169 INFO [http-bio-443-exec-13] handlers.LoginHandler -

prelogin:queryString=EXECUTE PROCEDURE
ucSOAPPreLogin('estaal','CUP9','10.10.10.52','CUP9.ciscolive.com');

2014-03-23 14:21:54,170 DEBUG [http-bio-443-exec-13] imdb.ImdbGeneralAccessorUtil -

getValidAppusersResultset -- query: SELECT * FROM validappusers WHERE userid='estaal';

2014-03-23 14:21:54,188 WARN [http-bio-443-exec-13] handlers.LoginHandler - preLogin:PRELOGIN

reasoncode=FAILURE. User either not CUP licensed or not found in database
Ensure User is Licensed For CUP 8

Users do not have proper licensing configured

In Communications Manager 8.x this is under System > Licensing > Capabilities Assignment
Verify in CUCM for the specific user:
Ensure User is Licensed For IM&P 9.x/10.x
In Communications Manager 9.x this is under User Management > End User
Verify in CUCM for the specific user:
Verify Connection To CUCM & Services Are Started
Ensure System > CUCM Publisher shows green checkmarks. If not correct issue accordingly.

Verify the Cisco UP Sync Agent is started on the IM&P Publisher Server.
Verify this on the serviceability page under Tools > Control Centre Feature Serivces
Verify User Licensing Has Synced From CUCM
Verify licensing has replicated to IM&P server (under User Management > End User):

If the information in IM&P does not match what is in CUCM, try restarting the Cisco UP Sync Agent to
force a sync.

Check Sync Status System (System > CUCM Publisher)

Check End User Page Login
What Do the Logs Show?
IM&P Server - EPASSoapXXXXX.log (Client Profile Agent Logs)

Wrong Password

<ns1:login client-version="9.6.1.18100" client-type="CUPC"

force="true"><ns1:username>estaal</ns1:username><ns1:password>...</ns1:password></ns1:login>

2014-03-23 14:26:10,417 INFO [http-bio-443-exec-17] handlers.LoginHandler -

prelogin:queryString=EXECUTE PROCEDURE
ucSOAPPreLogin('estaal','CUP9','10.10.10.52','CUP9.ciscolive.com');

2014-03-23 14:26:10,417 DEBUG [http-bio-443-exec-17] imdb.ImdbGeneralAccessorUtil -

getValidAppusersResultset -- query: SELECT * FROM validappusers WHERE userid='estaal';

2014-03-23 14:26:10,549 INFO [http-bio-443-exec-17] handlers.LoginHandler - Wrong credential for :

estaal| IMS result code:1
IMS Result Codes
1) Wrong Credentials

2014-03-23 14:26:10,549 INFO [http-bio-443-exec-17] handlers.LoginHandler - Wrong credential for : estaal| IMS result code:1

2) Account locked by Admin

2014-03-23 14:34:37,576 INFO [http-bio-443-exec-7] handlers.LoginHandler - Administratively locked for : estaal| IMS result code:2

3) Account Hack lock

2014-03-23 14:38:16,259 INFO [http-bio-443-exec-20] handlers.LoginHandler - Hack locked for : estaal| IMS result code:3

4) Account locked due to inactivity

7) User inactive in LDAP

2014-03-23 14:42:53,284 INFO [http-bio-443-exec-12] handlers.LoginHandler - End user status is INACTIVE in LDAP for : estaal| IMS
result code:7
Make Sure LDAP Authentication User Connects
CUCM > System > LDAP > LDAP Authentication
Jabber Unable to Communicate With Server
Make Sure DNS Works
Client PC must resolve what is configured under System > Cluster Topology
Verify Correct Services Started For Login
Verify the XCP Connection Manager and XCP Authentication Server are started
on the IM&P Publisher Server.
Verify this on the serviceability page under Tools > Control Centre Feature Services
Client Profile Agent Logs For Successful Login
IM&P Server - EPASSoapXXXXX.log (Client Profile Agent Logs)

Successful Login

<ns1:login client-version="9.6.1.18100" client-type="CUPC"

force="true"><ns1:username>estaal</ns1:username><ns1:password>...</ns1:password></ns1:login>

2014-03-23 14:10:01,564 INFO [http-bio-443-exec-12] handlers.LoginHandler -

prelogin:queryString=EXECUTE PROCEDURE
ucSOAPPreLogin('estaal','CUP9','10.10.10.52','CUP9.ciscolive.com');

2014-03-23 14:10:01,709 INFO [http-bio-443-exec-12] handlers.LoginHandler - doLogin:IMS login result is

success for estaal| IMS result code:0

2014-03-23 14:10:01,713 INFO [http-bio-443-exec-12] handlers.LoginHandler - SOAP Login was

successful
Agenda

Troubleshooting Logins
Service Discovery
Logins
Certificates

Troubleshooting Jabber Features

Tools Used By Jabber
Common Certificate Questions Seen By TAC
I got my certificate signed by a CA; why do I still get a prompt?
Certificate Error Message As Seen By End User
How to see it in the logs
Jabber client csf-unified.log

2014-03-27 11:35:50,729 DEBUG [0x00000ae4] [src\cert\common\CertificateData.cpp(130)] [csf.cert]

[cert::CertificateData::parseSubjectCNField] - Subject CN field : CUP9.ciscolive.com

2014-03-27 11:35:50,745 DEBUG [0x00000ae4] [ls\src\cert\utils\AltNameParser.cpp(331)] [csf.cert.utils]

[cert::AltNameParser::verify] - Looking for match with CUP9

2014-03-27 11:35:50,745 ERROR [0x00000ae4] [ls\src\cert\utils\AltNameParser.cpp(375)] [csf.cert.utils]

[cert::AltNameParser::verify] - No Match Found

2014-03-27 11:35:50,745 ERROR [0x00000ae4] [ls\src\cert\common\CertVerifier.cpp(267)] [csf.cert]

[cert::CertVerifier::buildCertResult] - Verification of identity: 'CUP9' failed.

2014-03-27 11:35:50,745 INFO [0x00000ae4] [mmon\PlatformVerificationHandler.cpp(42)] [csf.cert]

[cert::PlatformVerificationHandler::handlePlatformVerificationResultSynchronously] - Verification result :
FAILURE reason : [CN_NO_MATCH]
Login Certificate Flow
Port 8443

Tomcat
Certificate

(Client
Validates
Certificate)

Continue
Login Process
CUCM Tomcat Certificate
CUCM > System > Server
Login Certificate Flow
Port 8443

Tomcat
Certificate

(Client
Validates
Certificate)

Continue
Login Process
Instant Messaging & Presence Tomcat Certificate
CUCM User Management > User Settings > UC Service
Login Certificate Flow
Port 5222

cup-xmpp
Certificate

(Client
Validates
Certificate)

Continue
Login Process
Instant Messaging & Presence XMPP certificate
IM&P > System > Cluster Topology
Login Certificate Flow
Port 8443

Tomcat
Certificate

(Client
Validates
Certificate)

Complete
Connection
Unity Connection Tomcat Certificate
CUCM User Management > User Settings > UC Service
Agenda

Troubleshooting Logins
Troubleshooting Jabber Features
Directory Integration
Desk Phone Control and Presence
Jabber for iOS, Android and Mac
Tools Used By Jabber
Common Directory Questions Seen By TAC
Why am I unable to search for users?
Why is the IM address incorrect?
All my users show offline, even though they are online.
When I add a contact, they disappear immediately.
My contacts phone number does not show.
Directory Not Connecting
Explanation of jabber-config.xml file
Centrally configure parameters for Jabber
Allows administrators to make changes to client
Prevent the need of a COP file
Sample jabber-config.xml file
<?xml version="1.0" encoding="utf-8"?>
<config version="1.0">
<Directory>
<DirectoryServerType>EDI</DirectoryServerType>
<PrimaryServerName>10.10.10.10</PrimaryServerName>
<ServerPort1>3268</ServerPort1>
<UseWindowsCredentials>0</UseWindowsCredentials>
<ConnectionUsername>user@example.com</ConnectionUsername>
<ConnectionPassword>P@ssw0rd</ConnectionPassword>
<SearchBase1>dc=example, dc=com</SearchBase1>
</Directory>
</config>
Directory Search Working
Incorrect IM Address
Sample jabber-config.xml file

<?xml version="1.0" encoding="utf-8"?>

<config version="1.0">
<Directory>
<DirectoryServerType>EDI</DirectoryServerType>
<PrimaryServerName>10.10.10.10</PrimaryServerName>
<ServerPort1>3268</ServerPort1>
<UseWindowsCredentials>0</UseWindowsCredentials>
<ConnectionUsername>user@example.com</ConnectionUsername>
<ConnectionPassword>P@ssw0rd</ConnectionPassword>
<SearchBase1>dc=example, dc=com</SearchBase1>
<UserAccountName>telephoneNumber</UserAccountName>
</Directory>
</config>
Correct IM Address
Incorrect Phone Number
Sample jabber-config.xml file
<?xml version="1.0" encoding="utf-8"?>
<config version="1.0">
<Directory>
<DirectoryServerType>EDI</DirectoryServerType>
<PrimaryServerName>10.10.10.10</PrimaryServerName>
<ServerPort1>3268</ServerPort1>
<UseWindowsCredentials>0</UseWindowsCredentials>
<ConnectionUsername>user@example.com</ConnectionUsername>
<ConnectionPassword>P@ssw0rd</ConnectionPassword>
<SearchBase1>dc=example, dc=com</SearchBase1>
<BusinessPhone>ipPhone</BusinessPhone>
</Directory>
</config>
Correct Phone Number
Agenda

Troubleshooting Logins
Troubleshooting Jabber Features
Directory Integration
Desk Phone Control and Presence
Jabber for iOS, Android and Mac
Tools Used By Jabber
Typical Desk Phone Questions Seen By TAC
All users are unable to get phone control.
This one new user has no phone control.
We do not see phone presence for our users.
Jabber Client Showing No Phone Control
Jabber Desk Phone Control
CUCM > User Management > End User
Jabber Desk Phone Control
CUCM > Device > Phone
Jabber Controlling 99xx/89xx phone
CUCM Server - SDL001_200_XXXXX.txt (CTI Manager Logs)
AppInfo |[CTI-APP] [CTIHandler::OutputCtiMessage ] CTI FailureResponse ( seq#=3
result=2362179824 description=User not configured to access device that supports Connected
Transfer and Conference feature)

CUCM > User Management > End User

CTI Logs Showing Login Timeout
CUCM Server - SDL001_200_XXXXX.txt (CTI Manager Logs)
14:32:49.188 |200 |AppInfo |||CTIHandler(2,200,22,2606812)|||[CTI-APP]
[CTIHandler::OutputCtiMessage ] CTI ProviderOpenCompletedEvent
(seq#=370) provider id=36161244 CM Version=9.1.2.10000-28 error
code=2362179701 description=Directory login failed - timeout enableIpv6=0
NoOfDaysPwdToExp=4294967295
Check LDAP Authentication Port
System > LDAP > LDAP Authentication
Check LDAP Authentication Port
System > LDAP > LDAP Authentication
Jabber Client Showing Phone Control
Phone Presence Not Working

?
Check the SIP Trunk to IM&P
Ensure that the SIP Trunk to IM&P on CUCM is resolvable
CUCM > Device > Trunk
Is PublishEPA being created?
CUCM Server - SDL001_100_XXXXX.txt (CCM Manager Logs)
|SdlSig |PublishInd |restart0 |PublishEPA(1,100,106,2)
|PublishManager(1,100,105,1) |1,100,13,18.14^14.48.69.104^SEP006440B580CE |[R:N-
H:0,N:1,L:0,V:0,Z:0,D:0] users.size()=1 users=estaal; pattern=9121053 numPlanPkid=5fe08c87-d701-
ddb9-38f7-eaab51652ae0 devicePkid=681733d4-c469-4720-a870-781956e4552d mobileNumber=
model=Cisco-CP8961/9.3.2 state=2 isDnd=F firstRegisterDevice=F deviceMac=006440B580CE
If not, check the line association in CUCM

CUCM > User Management > End User

Phone Presence Working
Agenda

Troubleshooting Logins
Troubleshooting Jabber Features
Directory Integration
Desk Phone Control and Presence
Jabber for iOS, Android and Mac
Tools Used By Jabber
Common Questions Seen By TAC
Why is my mobile client unable to log in?
Why can I search with Jabber for Windows, but none of the other clients?
Jabber for iOS/Android Unable to Login
What The Logs Show
Jabber client Jabber-2014-05-07-16042250.log

INFO [3b09218c] - [JabberWerx][log] [LoginMgr]: #1, connectOnRetrievedServerInfo login,

cup:CUP9.ciscolive.com

INFO [3b09218c] - [JabberWerx][log] [CupSoapCli]: login cup async, server:CUP9.ciscolive.com, user:******,

ver:9.6.1.0

INFO [7dc0000] - [JabberWerx][log] [CupSoapCli]: CupSoapClientImpl::Login,

endpoint:https://CUP9.ciscolive.com:8443/EPASSoap/service/v80

INFO [7dc0000] - [JabberWerx][log] [CupSoapCli]: Login() result, prim:, backup:, reason:SOAP 1.2 fault:
SOAP-ENV:Sender[no subcode]
"Host not found
Detail: get host by name failed in tcp_connect()
Where Mobile Client Gets Server Information
Mobile device must resolve what is configured under System > Cluster Topology
User Logged In
Unable to Search Directory
Be Sure jabber-config.xml BDI Section Is Configured
<?xml version="1.0" encoding="utf-8"?>
<config version="1.0">
<!-- LDAP Directory configuration for non-windows platform clients -->
<Directory>
<DirectoryServerType>BDI</DirectoryServerType>
<BDIPrimaryServerName>XX.XX.XX.XX</BDIPrimaryServerName>
<BDIServerPort1>3268</BDIServerPort1>
<BDIConnectionUsername>user@example.com</BDIConnectionUsername>
<BDIConnectionPassword>P@ssw0rd</BDIConnectionPassword>
<BDIPresenceDomain>example.com</BDIPresenceDomain>
<BDISearchBase1>dc=example, dc=com</BDISearchBase1>
</Directory>
</config>
Working Directory Search
Which Information Will TAC Ask For When You Open
A Case?
a) Problem Report
b) Version numbers (server and client)
c) Time the problem occurred
d) UserID of the user who is having an issue.

All of the Above!

Agenda

Troubleshooting Logins
Troubleshooting Jabber Features
Tools Used By Jabber
Jabber-config.xml
Understanding the Problem Report
Check if jabber-config.xml is Formatted Properly
jabber-config.xml Correct Formatting
jabber-config.xml File Generator
jabber-config.xml File Generator
jabber-config-user.xml
This file takes precedence over the jabber-config.xml file
Is meant for testing purposes only
Agenda

Troubleshooting Logins
Troubleshooting Jabber Features
Tools Used By Jabber
Jabber-config.xml
Understanding the Problem Report
How To Create A PRT
Windows Android
How To Create A PRT
Mac iOS
Jabber for Windows PRT
Install Parameter
Jabber for Windows PRT
Local and
Downloaded
Config
Jabber for Windows PRT
Local and
Internal memory
Cache
Jabber for Windows PRT

Low level Library

logs
Jabber for Windows PRT

General System
Info
Jabber for Windows PRT

Main Jabber
Logs
Jabber for Windows PRT

Jabber Version
and logged in
User Info
Jabber for Windows PRT

Crash Dump
Jabber for Windows PRT

How the PRT

was created
Jabber for Windows PRT

End-users
Comments
Jabber for Mac PRT
Jabber for iOS PRT
Q&A
Complete Your Online Session Evaluation
Give us your feedback and receive a
Cisco 2016 T-Shirt by completing the
Overall Event Survey and 5 Session
Evaluations.
Directly from your mobile device on the Cisco Live
Mobile App
By visiting the Cisco Live Mobile Site
http://showcase.genie-connect.com/ciscolivemelbourne2016/
Visit any Cisco Live Internet Station located
throughout the venue
Learn online with Cisco Live!
T-Shirts can be collected Friday 11 March Visit us online after the conference
for full access to session videos and
at Registration presentations.
www.CiscoLiveAPAC.com
Thank you
Appendix
Troubleshooting Video
Common Video Questions Seen By TAC
I am unable to make/receive video calls.
Video works for everyone, except one user.
Video Call Not Working
Medianet Not Installed
Medianet Installed, Still No Video
Ensure computer is tethered to phone
If controlling 99xx/89xx, unplug camera
Check for CDP/CAST traffic via packet capture
Ensure video is enabled
Video Shows Connected, Still No Video
Only H.264 is supported
Ensure call is internal
Check location/region settings on CUCM
Video Call Working
Contact Photos
Common Contact Photo Questions Seen By TAC
Where does Jabber get the contact photo from?
Why cant I see contact photos?
Packet Capture of Contact Photos Acquired Via URL
Contact Photos via Web Server
<?xml version="1.0" encoding="utf-8"?>
<config version="1.0">
<Directory>
<PhotoUriSubstitutionEnabled>true</PhotoUriSubstitutionEnabled>
<PhotoUriSubstitutionToken>sAMAccountName</PhotoUriSubstitutionToken>
<PhotoUriWithToken>http://10.10.10.50/images/sAMAccountName.jpg</PhotoUriWithToken>
</Directory>
</config>
Contact Photos via jpegPhoto
Jabber client csf-unified.log
Working
2014-03-23 11:28:20,255 DEBUG [0x00000848] [2g-person\src\main\person\Person.cpp(48)] [csf.person]
[person::Person::Person] - Person (00F559A8) is created from record 03501E18
(Roster:asemin@ciscolive.com). Total(created/live): 2/2
2014-03-23 11:28:20,520 DEBUG [0x000004e0] [ices\impl\contacts\PersonWatcher.cpp(30)]
[personWatcherLogger] [PersonWatcher::onPersonDataChanged] - Person 00F559A8. hasPhoto?: 1
2014-03-23 11:28:20,520 DEBUG [0x00000848] [e\src\services\impl\ContactImpl.cpp(456)] [csf-contact-impl]
[ContactImpl::setHasPhoto] - Setting hasPhoto to true for asemin@ciscolive.com

Non-working
2014-03-23 11:41:18,226 DEBUG [0x00000a64] [2g-person\src\main\person\Person.cpp(48)] [csf.person]
[person::Person::Person] - Person (00F56128) is created from record 035BA1A8
(ActiveDirectory:jpeters@ciscolive.com). Total(created/live): 7/7
2014-03-23 11:41:18,226 DEBUG [0x00000848] [ce\src\services\impl\ContactImpl.cpp(89)] [csf-contact-impl]
[ContactImpl::setPerson] - Set to person 00F56128(Justin Peters). Has photo? 0
Soft Phone Control
Soft Phone Not Registering
TFTP server not set under Application > Legacy Clients > Settings in IM&P
TFTP Traffic is Blocked (port TCP 6970 and UDP 69)
Client is not able to resolve CUCM hostname in Config File
SIP traffic being blocked (port 5060 and 5061)
No CCMCIP profile
No device association
Jabber Soft Phone Call Fails Out To PSTN
Even though Never start calls with video is selected, video is only muted
Video capabilities are sent to Telco
Telco refuses the call
Transfer Capability = Unrestricted Digital
Need to add bearer-cap speech to the voice-port for outbound calls.