View AR - : Assistance Request 1-5877056

New AR:
View AR | AR without Proprietary

History | States | Assignment | Timetracking
Add Attachment
AR Text Search | More | CARES Home
Assistance Request 1-5877056

Customer Ticket N/A
Short
Loss of supervision on RNC235 and NodeBs under it
Description
Current
was
Aug 24 : Analysis provided from the available logs, Issue
Summary
replace
caused due to CP card crash from slot-1, recommended to

the board in maintenance window. Closure confirmation
received
to close this AR.
State
Closed : Solution/Service Provided
Outage
No
Severity
Priority
Service Request Remote Technical Support

Request Type
Support
Sub-Type
Diagnose
Category
Other
Sub-Category
Customer Dedicated Support
Internal
Yes
Assignment Events
Ask Alcatel-Lucent
Product
Product
9370 RNC
Version
LR13.3.W
Patch or SU
LR13.3.1
No Change Request is linked to this AR

Product Location
Site
ETISALAT ABU DHABI
Instance
9370RNC-Etisalat : In Service
Site Company
Emirates Telecommunications Corporation
City
Abu Dhabi
Country
United Arab Emirates
Contact
Name
Karim IBRAHIM ABD EL NABY
Company
Alcatel-Lucent : Open on behalf of Customer
Phone
971 56 354 4568
Karim.Ibrahim@alcatel-lucent.com
Request Method
Email
Email-Eng
Dates
Occurred
(GMT+4)
15-Aug-2015 09:58
Time now
29-Aug-2015 09:21
Reported
15-Aug-2015 09:58
AR Created
15-Aug-2015 10:01
Service Start
15-Aug-2015 09:58
Responded
15-Aug-2015 10:09
SA - Calculated
Next Customer Contact
27-Aug-2015 14:30
Respond Target
15-Aug-2015 10:28
Restored
15-Aug-2015 12:10
SA - Calculated
Restore Target
16-Aug-2015 13:36
Resolved
19-Aug-2015 08:10
SA - Calculated
Resolve Target
23-Sep-2015 04:27
Targets for Restore & Resolve were extended by 8 days 18 hours

to account for Pending time
Closed
24-Aug-2015 14:09
Last Modified
24-Aug-2015 14:09
Modified By
pthota
Entitlement
Agreement
242895 (OXIA 627085)
Covered Service TS 24x7 (Gold, Wireless Mobile Access)

Script
This request is entitled to service.
People
Owner
TSCd-WLS-WCDMA-IN : pthota
Assignee
CloQ-WLS-WCDMA-vGlobal : pthota
Referred 1
NorP-WLS-WCDMA-vGlobal : pthota
Resolve Group
NorP-WLS-WCDMA-vGlobal
Submitter
igawrych
Description
Attachments
Loss of supervision on RNC235 and NodeBs under it
Showing Investigation and Proprietary logs together in time order

separately
Investigation Log and Proprietary Log
1. 15-Aug-2015 12:30
pthota
Update to Current Summary: Aug 15 : Under Investigation
Show
2. 15-Aug-2015 12:31
pthota
Update to Current Summary: Aug 15 : Awaiting Receive Log Files / Technical

Info
3. 15-Aug-2015 12:34
pthota
ALCATEL-LUCENT PROPRIETARY
On 15.08.15 07:57, Karim (Karim)** CTR ** IBRAHIM ABD EL NABY wrote:

Hello
PROBLEM DESCRIPTION: Loss of supervision on RNC235 and NodeBs under it
PROBLEM IMPACT:
Loss of supervision on RNC235 and it's NodeB, the history as below
1about
While doing unlock for Fddcell under RNC235 it take a very long time
2 hours and not locked

2Stop/Start for the pmgt_cp_utran WMS process done, and all process
come up,
3After that we observe loss of supervision on both RNC235 and it's
NodeBs,
EXPECTED RESULTS:
RCA and Soln for this issue as the customer is escalating us due to this
issue,
Contact Surname
Contact Given Name
Karim
: Ibrahim
Contact Phone
Contact Company
:+971 56 354 4568

: Alcatel-Lucent
Company
Contract Number
Emirates Telecommunications Corporation
: 242895
Region
: EMEA
Country
: United Arab Emirates
City
: Abu Dhabi
Site
: ETISALAT ABU DHABI
Product Instance
: 9370 RNC
Severity
: 1
Priority
Product Version
: 1
: LR13.3.1
Outage
Internal/External:
: No
:Internal
Thanks,
BR,
Karim Ibrahim
LTE/3G Integration Professional
Alcatel-Lucent, Dubai, Etisalat Program
M: +971(0)563544568| ONNET: 22341798
Every Success has its network
From: SUPPORT@ALCATEL-LUCENT.COM [mailto:SUPPORT@ALCATEL-LUCENT.COM]

Sent: Saturday, August 15, 2015 10:24 AM
To: IBRAHIM ABD EL NABY, Karim (Karim)** CTR **
Cc: WAGDY MANSOUR, AYMAN (AYMAN)** CTR **; KHEDR, MAHMOUD (MAHMOUD); ALI,
YASSER
(YASSER)** CTR **; ABDEL-HALIM, SAYED (SAYED); MERAD, ABDESSAMAD
(ABDESSAMAD);
EL-MIDANY, AHMED (AHMED)
Subject: Re: Etisalat UAE//Loss of supervision on RNC235 and NodeBs under it
/
1-5877056
Dear Customer,
Thank you for contacting the Alcatel-Lucent Welcome Center.

Your request has been processed and the ticket number for your reference is
AR
1-5877056
Please reference that number when contacting Alcatel-Lucent for follow-up.
An Alcatel-Lucent representative will contact you regarding your request.
Kind Regards,
Izabela Gawrychowska
ALCATEL-LUCENT
GLOBAL WELCOME CENTER
Visit OnLine Customer Support for Contact details
From: owner-india-wcdma-utran-be@LIST.ALCATEL-LUCENT.COM
[mailto:owner-india-wcdma-utran-be@LIST.ALCATEL-LUCENT.COM] On Behalf Of
IBRAHIM
ABD EL NABY, Karim (Karim)** CTR **
Sent: Saturday, August 15, 2015 12:39 PM
To: INDIA-WCDMA-UTRAN-BE; ARORA, Gaurav (Gaurav)** CTR **
YASSER
(ABDESSAMAD);
EL-MIDANY, AHMED (AHMED)
Subject: RE: Etisalat UAE//Loss of supervision on RNC235 and NodeBs under it
/
1-5877056
Hi
Please update who is assigned to this AR.
Thanks,
BR,
Karim Ibrahim
M: +971(0)563544568| ONNET: 22341798
THOTA,
Praveen-K (Praveen-K)** CTR **

YASSER
(ABDESSAMAD);
EL-MIDANY, AHMED (AHMED); INDIA-WCDMA-UTRAN-BE; ARORA, Gaurav (Gaurav)** CTR
**
Subject: AR 1-5877056//Critical//9370 RNC//LR13.3.1//Etisalat UAE//Loss of
supervision on RNC235 and NodeBs under it
Hi Karim,
I am the assignee of this AR.
As discussed issue resolved now with CP switchover action performed from the
RNC.
Please confirm.
Please provide below details and logs for further investigation of this
issue.
CONFIGURATION DETAILS :
1. Hardware configuration:
2. Software configuration:
>OAM:
> RNC:
> NodeB:
LR 13.3.1
HISTORY:
1. Which & how many NEs impacted ? Only One RNC 235 and its NodeBs...Please
confirm.
2. When the issue has started?
3. What actions were performed before or in time when the issue begun
(feature
activation, upgrade, 3rd party actions...)
4. What recovery actions have been done & with what results: CP switchover
done and issue resolved
--- Please confirm.
PROBLEM IMPACT:
1.
Frequency of the problem (once, x times, sporadic, systematic):
2.
Impact (drop call, board reset, KPI degradation, redundancy, None):
LOGS REQUIREMENT:
1)
HFB of the impacted RNC before 5 days from issue to till time.
2)
RNC Log collection full for the impacted RNC
3)
RNC Health check of the impacted RNC.
4)
Please provide below commands output from the RNC.
os pcsShowCards
os excDumpCheck
d fs
d fs part/*
d sh ca/* spserv
d sh ca/* carduptime
Thanks and Regards,

Praveen Thota
+91 96899 18050
4. 15-Aug-2015 12:58
nrehan
Incoming E-mail from Praveen-K (Praveen-K)** CTR ** THOTA

Workflow Status: Handled by an agent
/
1-5877056
Contact: Praveen-K (Praveen-K)** CTR ** THOTA
From: "THOTA, Praveen-K (Praveen-K)** CTR **"
<praveen-k.thota@alcatel-lucent.com>
To: "GAWRYCHOWSKA, Izabela (Izabela)** CTR **"
<Izabela.Gawrychowska@alcatel-lucent.com>, ALU Support
<support@alcatel-lucent.com>
Cc: INDIA-WCDMA-UTRAN-BE <india-wcdma-utran-be@alcatel-lucent.com>,
"AGRAWAL,
Ritesh (Ritesh)** CTR **" <ritesh.agrawal@alcatel-lucent.com>, "ARORA,
Gaurav (Gaurav)** CTR **" <g.arora@alcatel-lucent.com>
Sent on: August 15, 2015 2:09:42 PM IST
Hi Welcome Center/Izabela,
Could you please let me know the reason for not calling me for this AR
assignment?
I have not received any call for this critical AR.
I was just going through my mails and found that one Critical AR was
assigned to
me without any information to me.
Thanks and Regards,
Praveen Thota
+91 96899 18050
5. 15-Aug-2015 13:01
sunitap
Incoming E-mail from Izabela (Izabela)** CTR ** GAWRYCHOWSKA

Workflow Status: Handled by an agent
/
1-5877056
Contact: Izabela (Izabela)** CTR ** GAWRYCHOWSKA
From: "GAWRYCHOWSKA, Izabela (Izabela)** CTR **"
<Izabela.Gawrychowska@alcatel-lucent.com>
To: "THOTA, Praveen-K (Praveen-K)** CTR **"
<praveen-k.thota@alcatel-lucent.com>, ALU Support
<support@alcatel-lucent.com>
Cc: INDIA-WCDMA-UTRAN-BE <india-wcdma-utran-be@alcatel-lucent.com>,
"AGRAWAL,
Ritesh (Ritesh)** CTR **" <ritesh.agrawal@alcatel-lucent.com>, "ARORA,
Gaurav (Gaurav)** CTR **" <g.arora@alcatel-lucent.com>
Sent on: August 15, 2015 2:22:46 PM GMT+05:30
Dear Sir,
I called you 3 times on the number which was provided both as your office
and
mobile number +91 9689918050.
I have followed all instructions for your workgroup, finally I reached Mr.
Babu
Sudhakar, the Workgroup Leader, who told me to assign this request to
primary from
schedule (you) and give you a call. Unfortunately I could not reach you, so
I
assigned it as for Mr. Babu approval.
If you need further investigation, please let me know so I can involve my

Team
Leader.
Izabela Gawrychowska
Global Welcome Center
COO Service Assurance Operations
Alcatel-Lucent
Visit OnLine Customer Support for contact details.
6. 15-Aug-2015 16:09
pthota
From: IBRAHIM ABD EL NABY, Karim (Karim)** CTR **

To: THOTA, Praveen-K (Praveen-K)** CTR **
YASSER
(ABDESSAMAD);
**
Subject: RE: AR 1-5877056//Critical//9370 RNC//LR13.3.1//Etisalat UAE//Loss
of
Hello Praveen
Please note the issue is recovered by doing lp/o switchover for the
but now
RNC ,
we need to check and find the RC urgently as the customer is pushing ,
The issue started as below
1Doing unlock
success
for one Fddcell on the RNC235 take about 2 hours without
2Stop/start for pm_CP_UTRAN process done on the WMS, and finish

successfully
3-
After that loss of supervision for RNC235 and all NodeBs under it
4-
Lock/Unlock for the OAM link not help to recover the issue
5-
Then we go for lp/0 switchover on the RNC
Logs @/home/wcdma/Etisalat_UAE/1-5877056
My comments below and waiting your feedback,
Thanks,
BR,
Karim Ibrahim
M: +971(0)563544568| ONNET: 22341798
From: THOTA, Praveen-K (Praveen-K)** CTR **

YASSER
(ABDESSAMAD);
**
Subject: AR 1-5877056//Critical//9370 RNC//LR13.3.1//Etisalat UAE//Loss of
Hi Karim,
I am the assignee of this AR.
As discussed issue resolved now with CP switchover action performed from the
RNC.
Please confirm.
Please provide below details and logs for further investigation of this
issue.
CONFIGURATION DETAILS :
1. Hardware configuration:
2. Software configuration:
>OAM:LR14.2.1
> RNC:
LR 13.3.1
HISTORY:
1. Which & how many NEs impacted ? Only One RNC 235 and its NodeBs...Please
confirm. Yes only RNC235 and it's NodeBs
2. When the issue has started? 6am this morning
3. What actions were performed before or in time when the issue begun
(feature
activation, upgrade, 3rd party actions...) mentioned above,
4. What recovery actions have been done & with what results: CP switchover
done and issue resolved
--- Please confirm. Mentioned above
PROBLEM IMPACT:
1.
once,
Frequency of the problem (once, x times, sporadic, systematic):
2.
loss
Impact (drop call, board reset, KPI degradation, redundancy, None):
of Supervision, during the lp/0 switchover we have about 5 min outage

reported
from core team,
LOGS REQUIREMENT:
1)
2)
RNC Log collection full for the impacted RNC
3)
RNC Health check of the impacted RNC.
4)
Please provide below commands output from the RNC.
os pcsShowCards
os excDumpCheck
d fs
d fs part/*
d sh ca/* spserv
Thanks and Regards,

Praveen Thota
+91 96899 18050
7. 15-Aug-2015 16:11
pthota
Update to Current Summary: Aug 15 : Logs download in progress from the

server.
8. 17-Aug-2015 09:11
pthota
Update to Current Summary: Aug 17 : Logs received from server, under

investigation.
9. 17-Aug-2015 09:11
pthota

YASSER
(ABDESSAMAD);
**
of
Hello Praveen
Below my comments, and attached the output of the commands, I think

something NOK
for all card!,
Wating your feedback,
Thanks,
BR,
Karim Ibrahim
M: +971(0)563544568| ONNET: 22341798

YASSER
(ABDESSAMAD);
**
of
Hi Karim,
I have started downloading the logs, Please note that logs download is
taking time
due to the slow connection of data card. I will let you know once RNC log
collection download completed.
[ KARIM] No issue take your time
From RNC health check we could see that --> 1140 fddCells Locked, Could you
please
let us know the reason for keeping these many FDDCells in locked state?
[ KARIM] this created dummy sites for prepartions,
Is there any planned event or any other actions done before Fddcell unlock
action?
What is the reason for doing fddcell unlock action?
[ KARIM] no any event, and it's normal action,
Did you also perform unlock on any other FDDCell from any other RNCs at that
time?
If performed what were the results?

[ KARIM] yes and It was working fine
Could you please let us know the exact time of CP switchover to understand
the KPI
degradation from KPI reports, in normal condition CP switchover should not
cause
any outage to call processing?
[ KARIM] about 10am, no KPIs degradation happened as it's 15min based from
core
see attached they observed 5 min outage at the same time after the lp/0
switchover,
Also if you can please share the communication from core for the 5 minutes
outage
to understand the impacted KPI related to this action or not?
[ KARIM] Attached,
Did you try any other actions from WMS point of view before performing CP
Switchover on RNC other than one pm_CP_UTRAN process stop/start?
Like all
processes stop/start on WMS?

[ KARIM] Yes we try also stop/start fro all,
Is there any recommendation from support to stop/start the WMS process

pm_CP_UTRAN
for recovering the Fddcell unlock command? We want to know the reason for
doing
this action since it caused the RNC and it's nodeBs to Loss of Supervision.
[ KARIM] No recommendation, we take the action to try recover the issue and
we
already faced on other NWs,
I just gone through the provided HFB, I think you have provided the entire
network
HFB, please note it will be very difficult to filter for this RNC in this
format.
Request you to extract the HFB for only this RNC from WMS HFB browser window
as
per below requirement and provide the same. Thanks for your help.
[ KARIM] I cannot provide per RNC due to the limitation of the number of
alarms,
so please use the file I send as it contains all types of alarms,
1)
4)
attached,
Please provide below commands output from the RNC.[ KARIM]
os excDumpCheck
Thanks and Regards,

Praveen Thota
+91 96899 18050

Sent: Sunday, August 16, 2015 1:10 PM
YASSER
(ABDESSAMAD);
**

of
Hello Praveen
Please update the progress here,
Thanks,
BR,
Karim Ibrahim
M: +971(0)563544568| ONNET: 22341798
THOTA,
Sent: Monday, August 17, 2015 10:32 AM
YASSER
(ABDESSAMAD);
**
of
Hi Karim,
I am going through the logs and will let you know the findings very soon.
Thanks.
Thanks and Regards,

Praveen Thota
+91 96899 18050
10. 17-Aug-2015 15:12

pthota
Update to Current Summary: Aug 17 : Analysis provided from the available

logs,
requested few more logs and details. waiting for those details.
11. 17-Aug-2015 15:12

pthota
THOTA,
Sent: Monday, August 17, 2015 3:25 PM
YASSER
(ABDESSAMAD);

**;
INDIA-WCDMA-WMS-NPO-BE
of
++
Hi Karim,
Please find below investigation results from the provided logs,
From the HFB I could see below LOS alarms for 14th and 15th 2015 ...
From the above it is clear that issue is not only with the RNC 235, the LOS
was
observed on 239,232 and 231 and also WCE RNCs 241 and 242 as well. I could
see
that LOS happened on almost all RNCs for two times.
From the above it is suspected that there is some communication issue for
the four
RNCs (235,231,232 and 239). which was observed on 14-08-2014 at 17:06 PM and
15-08-2015 at 09:35 AM.
The alarms observed on 15-08-2015 at 09:08 AM for all RNCs including WCE
RNCs may
be due to the processes stop and start in WMS -' Could you please confirm at
what
the WMS processes stop/start done?
LOS observed on RNC 235 on 15-08-2015 at 04:55 AM was due to TRAP with Disk
error
issue on shelf Card/1 as mentioned below.
LOS observed on RNC 235 on 15-08-2015 at 09:40 AM, may be due to manual CP
switchover done at site which need to confirmed from logs.
From the provided RNC 235 RNC Log collection, which was collected at 08:15
AM of
August 15th, I could see the alarms till 2015-08-15 06:13:09.61 only.
For the LOS happened between 09:00 AM 10:00 AM, there is no logs for the
investigation. So for further investigation I request you to provide the
below
logs. Kindly upload them to the server with today's date folder. I will let
you
know if any further logs required.
1)
RNC Log collection full for this RNC
upload it
' Please collect it again and
to the server.
2)
not
Also request you to provide HFB for this RNC for 15th August 2015. (if
please upload the HFB file for August 15th date to the server).
3)
/opt/nortel/data/cmXML/CMSessions/YYYYMMDD/Online+Commands/*
4)
/opt/nortel/logs/wtk/JobHistory-00114*
5)
Screenshot of the unlock command struck in WMS, if not available you
can take
it from WMS commands history.
Also as issue happened due to below actions and it is observed on almost 4

RNCs,
we may need to investigate it from WMS point of view as well. Adding WMS
support
colleagues to this mail chain for their understanding and support.
1Doing unlock
success
for one Fddcell on the RNC235 take about 2 hours without
2Stop/start for pm_CP_UTRAN process done on the WMS, and finish

successfully
3-
After that loss of supervision for RNC235 and all NodeBs under it
From below observations and Crash of the Card/1, we recommend you to replace
the
Shelf Card/1 with new card in maintenance window and observe the card for
next few
days.
I have not observed any KPI degradation from the provided KPI reports of the
RNC,
So please provide the specific KPI reports in which degradation is observed.
From the PPC Alarms below are the observations...
As per the alarms I could see that CP Outage happened at 2015-08-15 04:54:52
(CP
outage) due to which the RNC went to Loss of supervision and recovered at
2015-08-15 04:56:11.65
Please find below explanation with alarms...
CP outage happened at below time....
Lp/0 ; 2015-08-15 04:55:07.43

MSG warning
equipment
70120203
ADMIN: unlocked
OPER: enabled
USAGE: active
AVAIL:
PROC:
CNTRL:
ALARM:
STBY: notSet
UNKNW: false
Id: 15
Com: Active CP last time in service: 2015-08-15 04:54:52 (CP outage)
Int: 0/1/2/24554; pcsCpElector.cc; 2749; RID3730
After that shelf card/0 is trying to become active...as below..
Shelf Card/0 ; 2015-08-15 04:55:03.35

MSG warning
qualityOfService
70120102
ADMIN: unlocked
OPER: enabled
USAGE: active
AVAIL:
PROC:
CNTRL:
ALARM:
STBY: notSet
UNKNW: false
Id: 2
Com: Active CP starting up on card 0
Int: 0/1/2/24556; pcsCpStarter.cc; 1141; RID3730
OMU-AP reset happened due to CP-swact detected....
Lp/5 Ap/5 ; 2015-08-15 04:55:04.37

SET major
equipment
70701005
ADMIN: unlocked
OPER: disabled
USAGE: idle
AVAIL:
PROC: reporting
CNTRL:
ALARM: major
STBY: notSet
UNKNW: false
Id: 5000008
Com: OMU -- AP will be reset; CP swact detected. Resetting Active or
Uninitialized
OMU.
Int: 0/0/0/114; apcsApcMgr.cc; 2899; RID37300E1953
After this CP-0 has become active as below from RNC Health check...
###################################################################
###
RNC235 STATE CHECKER --- 15/08/2015 at 07:46:07
###
-------------------------------------------------
--- RNC Cards list ----> Passport's card list : |
CP : 2 | PSFP: 12 | OC3: 0 | 4pGe:
2 |
--> RNC overload status is ok
INSTANCE
CARD
CPU Util %
--------
---------------------0
Active
FSM STATE
VISIBILITY
LP
-----------------
----------
--
CPeE
RunningAsActive
OnLine
CPeE
RunningAsStandby
OnLine
1
1
Standby
TYPE
There is a TRAP with Disk Error observed on CP-1 at the same time, due to
which
this CP Outage has happened.
Shelf Card/1 ; 2015-08-15 04:56:10.59
MSG indeterminate equipment
70120101
ADMIN: unlocked
OPER: enabled
USAGE: active
AVAIL:
PROC:
CNTRL:
ALARM:
STBY: notSet
UNKNW: false
Id: 1000001
Com: TRAP DATA INFORMATION:
Unrecoverable error at line 1715 in file sfsFileSys.cc during task
"taccTestCronTask" (0x1b7f9ce0)
--- Reason --Disk error, switch over to standby
[sfs:disk test]
0
Error Info: (0.18)
*** An exception has occurred at task level ***
Offending task "taccTestCronTask" (0x1b7f9ce0) was at address 0x019cf508
(errno = 0x029a000a)
0
Description Info: (0.17)
Problem is diagnosed as a(n) System Call exception
No handler for exception specific data installed for vector 0xc00
0
CPU Utilization Info: (0.18)
3 percent busy.
CPU: PowerPC750GX D1.2 @ 990MHz (165Mhz 60x bus clk)
0
Register Info: (0.74)
r00:
019cf504
r01:
1b7f9810
r02:
00000000
Int: 1/0/2/13293; pcsCardAgt.cc; 3653; RID3730
r03:
00000032
Shelf Card/1 ; 2015-08-15 04:56:10.59

70120101
ADMIN: unlocked
OPER: enabled
USAGE: active
AVAIL:
PROC:
CNTRL:
ALARM:
STBY: notSet
UNKNW: false
Id: 1000002
r04:
1b7f983b
r05:
0000000f
r06:
00000000
r07:
0000005b
r08:
00000001
r09:
1b7f984a
r10:
00000001
r11:
00000003
r12:
40000048
r13:
00000000
r14:
00000000
r15:
00000000
r16:
00000000
r17:
00000000
r18:
00000000
r19:
00000000
r20:
00000000
r21:
1e7f0000
r22:
019f0000
r23:
019f0000
r24:
1e7f0000
r25:
01bb0000
r26:
01bb0000
r27:
00007530
r28:
00000000
r29:
01ba0000
r30:
01ba4d44
r31:
1e7e9bbc
msr:
0000b131
cr:
40000048
ctr:
00000000
pc:
019cf508
xer:
00000000
dar:
00000000
dsisr:00000000
fpcsr:00004000
Interrupt Enable Reg: 000000a0

Link Register: reportDiskError__13SfsFileSystem10DiskIdTypePc+f0
(0x019cf504)
Shelf Card/1 ; 2015-08-15 04:56:10.59

70120101
ADMIN: unlocked
OPER: enabled
USAGE: active
AVAIL:
PROC:
CNTRL:
ALARM:
STBY: notSet
UNKNW: false
Id: 1000003
MSR[EE]=1 interrupts unlocked

0
Code Disassembly Info: (67.87)
0x19cf4ec
38a54d6c
addi
r5,r5, 0x4d6c (0x01ba4d6c)
0x19cf4f0
7fc6f378
or
r6,r30,r30
0x19cf4f4
38610008
addi
r3,r1, 0x8
0x19cf4f8
38800200
li
r4, 0x200
0x19cf4fc
4cc63182
crxor
crb6,crb6,crb6
0x19cf500
480a010b
bla
snprintf
0x19cf504
44000002
sc
-> 0x19cf508
389d4ce8
addi
r4,r29, 0x4ce8
0x19cf50c
386006b3
li
r3, 0x6b3
0x19cf510
38a10008
addi
r5,r1, 0x8
0x19cf514
4cc63182
crxor
crb6,crb6,crb6
0x19cf518
4809487b
bla
Sys_unrecoverableErrorPpc_v
Shelf Card/1 ; 2015-08-15 04:56:10.59

70120101
ADMIN: unlocked
OPER: enabled
USAGE: active
AVAIL:
PROC:
CNTRL:
ALARM:
STBY: notSet
UNKNW: false
Id: 1000004
0
0x19cf51c
8001021c
lwz
r0, 0x21c(r1)
0x19cf520
83a1020c
lwz
r29, 0x20c(r1)
0x19cf524
7c0803a6
mtspr
LR,r0
Stack Trace Info: (3.82)

019cf508
reportDiskError__13SfsFileSystem10DiskIdTypePc+f4 : 019cf508
1b7f9810: 1b7f9a28 019cf51c
6b736944 72726520
*(.......Disk err*
1b7f9820: 202c726f 74697773
6f206863 20726576
*or, switch over *
1b7f9830: 73206f74 646e6174
5b0a7962 3a736673
*to standby.[sfs:*
1b7f9840: 6b736964 73657420
30005d74 63726420
*disk test].0 drc*
1b7f9850: 6420323d 3d636672
000a2031 00000000
*=2 drfc=1 ......*
1b7f9860: 00000000 00000000
1b7f9a50 1b7f9a58
*........P...X...*
Shelf Card/1 ; 2015-08-15 04:56:10.59

70120101
ADMIN: unlocked
OPER: enabled
USAGE: active
AVAIL:
PROC:
CNTRL:
ALARM:
STBY: notSet
UNKNW: false
Id: 1000005
1b7f9870: 1b7f9a80 1b7f9b38
1b7f9b58 1b7f9b88
*....8...X.......*
1b7f9880: 1b7f9bb8 1b7f9be8
1b7f9c10 1b7f9c18
*................*
019d1950
sfsHandleLocalDiskAccessError+3c0: reportDisk
1b7f9a20:
1b7f9c40 019d1950
@...P...*
1b7f9a30: 00154e2a 001f0000
00000000 00000000
**N..............*
1b7f9a40: 001e0000 0001b131
1b7f9a80 8538c299
*....1.........8.*
1b7f9a50: 00000005 002853c4
00154e2a 001f0000
*.....S(.*N......*
1b7f9a60: 00000000 1947a720
00000000 1947a720
*.... .G..... .G.*
1b7f9a70: 00000080 0015452b
001ed4d8 00000000
*....+E..........*
1b7f9a80: 1b7f9b38 00095620
00000001 0015452b
*8... V......+E..*
Shelf Card/1 ; 2015-08-15 04:56:10.59

70120101
ADMIN: unlocked
OPER: enabled
USAGE: active
AVAIL:
PROC:
CNTRL:
ALARM:
STBY: notSet
UNKNW: false
Id: 1000006
1b7f9a90: 1b7f9ab8 00000000
1b7f9ac0 00001000
1b7f9aa0: 1b7f9ac0 00001000
*................*
019cf25c
localDiskAccessTest__13SfsFileSystem+54 : sfsHandleL
1b7f9c40: 1b7f9c50 019cf25c
019e9f18
*........
1e7f0000 00007530
*P...\.......0u..*
sfsAccTestCronTask__Fv+cc : localDiskA
1b7f9c50: 1b7f9c88 019e9f18
eeeeeeee 00000000
*................*
1b7f9c60: 00000000 00000000
00000000 00000000
*................*
1b7f9c70: 00000000 00000000
00000000 00000000
*................*
1b7f9c80: 00000000 019e9e4c
000d6768
vxTaskEntry
*....L...
+60 : sfsAccTest
1b7f9c80:
1b7f9ca0 000d6768
....hg..*
Shelf Card/1 ; 2015-08-15 04:56:10.59

ADMIN: unlocked
OPER: enabled
70120101
USAGE: active
AVAIL:
PROC:
CNTRL:
ALARM:
STBY: notSet
UNKNW: false
Id: 1000007
1b7f9c90: 016cda40 1ea
Below are the observations from crash report of card CP-1...

Error Info: (0.18)

Reset
Info: (0.17)
No significant reboot reason available

Last boot type: (warm autoboot)
Disk Error Counters:

ERR-bit1:1 ERR-bit2:1 ERR-bit6:1
B_COR:1
semTake_timeouts:2
srrc=10 drc=2 drfc=1
Serial ID: 13290946F612

Product: Micron_M500_MTFDDAK120MAV
2 disk error record(s) found in Flight Recorder and latest ones shown below
DISK ERROR32 15/08/15 00:54:25, SN:13290946F612,

PI:Micron_M500_MTFDDAK120MAV ;
OP:e7 ALT:d0 ERR:0 LBA:a0000200 DAT:0 SC:0 IS:0; stTo:1;
DISK ERROR5 15/08/15 00:54:51, OP:c8 ALT:46 ERR:46 LBA:46464646 DAT:4346
SC:46
IS:0; stTo:2; drc:1; F:1/5367 -25014ms
From Dumpcheck also we could see that there is a trace back for the card-1,
which
is not normal.
2> os excDumpCheck
Software error PRESENT on Card 0 (Standby CP)
Traceback and Software error PRESENT on Card 1 (Active CP)
Software error PRESENT on Card 2 (FP)

After CP Outage restored in RNC, the locked PMP associations alarms raised
again... as below...
23016 34980/1 34985/41 ; 2015-08-15 04:55:06.73

SET major
communications
70790202
ADMIN: unlocked
OPER: disabled
USAGE: idle
AVAIL:
PROC:
CNTRL:
ALARM:
STBY: notSet
UNKNW: false
Id: 9
Com: Pmp is down
Int: 0/1/2/24425;
m3uaPmp.cc; 901; RID37300E1953
23016 34980/1 34985/42 ; 2015-08-15 04:55:06.73

SET major
communications
70790202
ADMIN: unlocked
OPER: disabled
USAGE: idle
AVAIL:
PROC:
CNTRL:
ALARM:
STBY: notSet
UNKNW: false
Id: A
Com: Pmp is down
Int: 0/1/2/24425;
23016 34980/1 34985/56 ; 2015-08-15 04:55:06.73

SET major
communications
70790202
ADMIN: unlocked
OPER: disabled
USAGE: idle
AVAIL:
PROC:
CNTRL:
ALARM:
STBY: notSet
UNKNW: false
Id: B
Com: Pmp is down
Int: 0/1/2/24425;
23016 34980/1 34985/67 ; 2015-08-15 04:55:06.73

SET major
communications
70790202
ADMIN: unlocked
OPER: disabled
USAGE: idle
AVAIL:
PROC:
CNTRL:
ALARM:
STBY: notSet
UNKNW: false
Id: C
Com: Pmp is down
Int: 0/1/2/24425;
23016 34980/1 34985/68 ; 2015-08-15 04:55:06.73

SET major
communications
70790202
ADMIN: unlocked
OPER: disabled
USAGE: idle
AVAIL:
PROC:
CNTRL:
ALARM:
STBY: notSet
UNKNW: false
Id: D
Com: Pmp is down
Int: 0/1/2/24425;
23016 34980/1 34985/69 ; 2015-08-15 04:55:06.73

SET major
communications
70790202
ADMIN: unlocked
OPER: disabled
USAGE: idle
AVAIL:
PROC:
CNTRL:
ALARM:
STBY: notSet
UNKNW: false
Id: E
Com: Pmp is down
Int: 0/1/2/24425;
23016 34980/1 34985/70 ; 2015-08-15 04:55:06.73

SET major
communications
70790202
ADMIN: unlocked
OPER: disabled
USAGE: idle
AVAIL:
PROC:
CNTRL:
ALARM:
STBY: notSet
UNKNW: false
Id: F
Com: Pmp is down
Int: 0/1/2/24425;
23016 34980/1 34985/71 ; 2015-08-15 04:55:06.73

SET major
communications
70790202
ADMIN: unlocked
OPER: disabled
USAGE: idle
AVAIL:
PROC:
CNTRL:
ALARM:
STBY: notSet
UNKNW: false
Id: 10
Com: Pmp is down
Int: 0/1/2/24425;
23016 34980/1 34982/7 ; 2015-08-15 04:55:06.73

SET critical
communications
70790200
ADMIN: unlocked
OPER: disabled
USAGE: idle
AVAIL:
PROC:
CNTRL:
ALARM:
STBY: notSet
UNKNW: false
Id: 11
Com: DestSp is inaccessible

Int: 0/1/2/24425;
m3uaDestSp.cc; 458; RID37300E1953
23016 34980/1 34982/14 ; 2015-08-15 04:55:06.73

SET critical
communications
70790200
ADMIN: unlocked
OPER: disabled
USAGE: idle
AVAIL:
PROC:
CNTRL:
ALARM:
STBY: notSet
UNKNW: false
Id: 12
Com: DestSp is inaccessible
Int: 0/1/2/24425;
m3uaDestSp.cc; 458; RID37300E1953
23016 34980/1 34986/7 ; 2015-08-15 04:55:06.73

SET critical
communications
70790201
ADMIN: unlocked
OPER: disabled
USAGE: idle
AVAIL:
PROC:
CNTRL:
ALARM:
STBY: notSet
UNKNW: false
Id: 13
Com: Pme is down
Int: 0/1/2/24425;
m3uaPme.cc; 631; RID37300E1953
23016 34980/1 34986/14 ; 2015-08-15 04:55:06.73

SET critical
communications
70790201
ADMIN: unlocked
OPER: disabled
USAGE: idle
AVAIL:
PROC:
CNTRL:
ALARM:
STBY: notSet
UNKNW: false
Id: 14
Com: Pme is down
Int: 0/1/2/24425;
m3uaPme.cc; 631; RID37300E1953
From the Latest RNC health check

active ...as
I could see that again card-1 is
below..
###################################################################
###
RNC235 STATE CHECKER --- 15/08/2015 at 14:00:17
###
-------------------------------------------------
--- RNC Cards list ----> Passport's card list : |
CP : 2 | PSFP: 12 | OC3: 0 | 4pGe:
2 |
--> RNC overload status is ok
INSTANCE
CARD
CPU Util %
--------
---------------------0
Standby
FSM STATE
VISIBILITY
LP
-----------------
----------
--
CPeE
RunningAsStandby
OnLine
CPeE
RunningAsActive
OnLine
3
1
Active
TYPE
From the card uptime also we can conclude that issue started with CP-1 TRAP
disk
error issue due to which FDDCell unlock command failed and was incomplete in
WMS
till you reset the process in WMS for recovering it. So request you to
provide the
job history logs to confirm the same.
1> d sh ca/* carduptime
Shelf Card/*
+====+-----------------------------------------------------|Card|
|
cardUpTime
days
hours
minute
second
usecon
+====+-----------------------------------------------------|
0|
0
7
40
48
674428
confirm from logs whether it is due to CP Switchover or not?
|
1|
0
12
happened due to TRAP issue.
42
52
315787
-' Need to
-' reset
Thanks and Regards,

Praveen Kumar Thota
Mob: +91 96899 18050
Desk: +91 80 66390212
12. 17-Aug-2015 15:13

pthota
Time Tracking Entry Added - IR01-TSA3
13. 18-Aug-2015 12:16

pthota

logs,
requested few more logs and details. waiting for those details.
14. 18-Aug-2015 12:16

pthota
From: MERAD, ABDESSAMAD (ABDESSAMAD)

To: THOTA, Praveen-K (Praveen-K)** CTR **; IBRAHIM ABD EL NABY, Karim
(Karim)**
CTR **
YASSER
(YASSER)** CTR **; ABDEL-HALIM, SAYED (SAYED); EL-MIDANY, AHMED (AHMED);
INDIA-WCDMA-UTRAN-BE; ARORA, Gaurav (Gaurav)** CTR **; INDIA-WCDMA-WMS-NPOBE
of
Importance: High
Please summarize the required actions
thanks
Best Regards

To: MERAD, ABDESSAMAD (ABDESSAMAD); THOTA, Praveen-K (Praveen-K)** CTR **
YASSER
INDIA-WCDMA-UTRAN-BE; ARORA, Gaurav (Gaurav)** CTR **; INDIA-WCDMA-WMS-NPOBE

of
Hello Praveen
Thanks for your detailed analysis, please help to the RCAs next steps also,
Thanks,
BR,
Karim Ibrahim
M: +971(0)563544568| ONNET: 22341798
THOTA,
To: IBRAHIM ABD EL NABY, Karim (Karim)** CTR **; MERAD, ABDESSAMAD
(ABDESSAMAD)
YASSER
INDIA-WCDMA-UTRAN-BE; ARORA, Gaurav (Gaurav)** CTR **; INDIA-WCDMA-WMS-NPOBE;
CHOWDARY, Ajay (Ajay)** CTR **
of
++
Hi Karim/Merad,
Below I am summarizing the issue...
As indicated previously the LOS issue on RNC 235 and NodeBs under it, is not
only
with RNC 235. I think there is some external issue, due to which all RNCs
are
having LOS at the same time.
From HFB the issue is observed on all 4 legacy RNCs on 14-08-2015 at 17:06 &
15-08-2015 at 09:35 and LOS observed for all 6 RNCs(including WCE RNCs) on
15-08-2015 at 09:08 has to be investigated separately with the help of WMS
support
team.
Adding Mr. Ajay who is investigating from WMS point of view. As requested by
him,
please provide below logs from WMS for further investigation of the issue.
/opt/nortel/logs/utran
o
cp_utran-20000.log
cp_utran-20000.tra
WOM_ADAPTER-20002.log
WOM_ADAPTER-20002.tra
WOM_COMM_SEPE_RNC-20010.tra
WOM_COMM_SEPE_RNC-20010.log
From the RNC Point of view LOS observed on 15-08-2015 at 04:55 was due to
Shelf
Card/1 TRAP with disk Error issue. From the card installation to till date
this
card crashed for two times, So it is advised to replace this card in
Maintenance
window to avoid further issues.
For further investigation of the issues and LOS observed for RNC 235 between
09:00
AM to 10:00 AM on 15-08-2015. I request below logs, thanks to provide the
same at
the earliest.
Note : previously provided RNC logs are having details till 2015-08-15
06:13:09.61
only.
1)
RNC Log collection full for this RNC
upload
' Please collect it again and
it to the server.
2)
not
Also request you to provide HFB for this RNC for 15th August 2015. (if
please upload the HFB file from WMS stability folder for August 15th date to
the
server).
3)
/opt/nortel/data/cmXML/CMSessions/YYYYMMDD/Online+Commands/*
files
-' All
from this path for 15th August 2015.

4)
/opt/nortel/logs/wtk/JobHistory-00114*
August
-' All Job History files of
15th 2015.
5)
can
take it from WMS commands history.
Thanks and Regards,

Praveen Kumar Thota
Mob: +91 96899 18050
Desk: +91 80 66390212
15. 19-Aug-2015 13:15

pthota

logs,
Issue was caused due to CP card crash from slot-1, recommended to replace
the
board in maintenance window.
16. 19-Aug-2015 13:15

pthota

Sent: Tuesday, August 18, 2015 2:41 PM
To: THOTA, Praveen-K (Praveen-K)** CTR **; CHOWDARY, Ajay (Ajay)** CTR **
YASSER
INDIA-WCDMA-UTRAN-BE; ARORA, Gaurav (Gaurav)** CTR **; MERAD, ABDESSAMAD
(ABDESSAMAD); INDIA-WCDMA-WMS-NPO-BE

of
Hello Praveen,Ajay,
Thanks for your feedback, Ramses FTP is available so please download the
required
logs directly from it,
For the RNClog collection may I ask why it required again , as the one taken
during the action not help you, so why requested again now,
Please note the lOS happened as we did WMS stop/start in the same mentioned
all
times as we have another issue with other.
Please provide final/clear reason for the issue so we can provide to the
customer
clear reason for this issue, or you can confirm that the CP card is the main
RCA?!.
Thanks,
BR,
Karim Ibrahim
M: +971(0)563544568| ONNET: 22341798

To: IBRAHIM ABD EL NABY, Karim (Karim)** CTR **; CHOWDARY, Ajay (Ajay)** CTR
**
YASSER
of
Hi Karim,
I think you missed to read the reason from below for asking RNC Log
collection
again, anyway please find it again...
Ramses FTP is available so please download the required logs directly from
it,
No problem, we will collect the ftp possible logs from WMS directly. Thanks
For the RNClog collection may I ask why it required again , as the one taken
during the action not help you, so why requested again now,
From the provided RNC 235 RNC Log collection, which was collected at 08:15
AM of
August 15th, I could see the alarms till 2015-08-15 06:13:09.61 only.
So request you to run the RNC Log collection and let me know the file to be
collected, but please note it will take longer time to download it from
Ramses
directly.
Please provide below logs, as it cannot be collected through Ramses....
1)
can
take it from WMS commands history.
Please note the lOS happened as we did WMS stop/start in the same mentioned
all
times as we have another issue with other.
you
-' I didn't understand, could
please explain in detailed....
Please provide final/clear reason for the issue so we can provide to the
customer
clear reason for this issue, or you can confirm that the CP card is the main
RCA?!.
After checking the below requested logs, we will try to provide you the
clear
reason for the LOS....!!!
But I just want to inform you again that the LOS issue not happened alone to
RNC
235 at that time as per below LOS alarms of 15th August 2015, do you agree?
Thanks and Regards,

Praveen Kumar Thota
Mob: +91 96899 18050
Desk: +91 80 66390212

To: THOTA, Praveen-K (Praveen-K)** CTR **; CHOWDARY, Ajay (Ajay)** CTR **
YASSER
of
Hello Praveen
OK, thanks , please as I mentioned to you this loss of supervision is due to

stop/start of the WMS, that's why you see it on all the RNCs,
I'll collect the rnclog again and share with you,
Thanks,
BR,
Karim Ibrahim

M: +971(0)563544568| ONNET: 22341798

YASSER
(ABDESSAMAD); INDIA-WCDMA-WMS-NPO-BE; CHOWDARY, Ajay (Ajay)** CTR **
of
Hi Karim,
The alarms observed on 15-08-2015 at 09:08 AM for all RNCs including WCE
RNCs may
be due to the processes stop and start in WMS -' Could you please confirm at
what
time the WMS processes stop/start done and for how many times?
The LOS observed for four RNCs only (235,231,232 and 239) on 14-08-2014 at
17:06
PM and 15-08-2015 at 09:35 AM also due to WMS stop/start activity? Please
confirm.
Thanks and Regards,

Praveen Kumar Thota
Mob: +91 96899 18050

Desk: +91 80 66390212

YASSER
of
Hello Praveen
Confirmed,
Thanks,
BR,
Karim Ibrahim
M: +971(0)563544568| ONNET: 22341798
Sent: Wednesday, August 19, 2015 9:38 AM

YASSER
of
Hi Karim,
I have gone through the Latest RNC Log collection of this RNC 235 and also
checked
the RNC Log collection of RNC 239 and nothing found abnormal.
In RNC235 I didn't find anything abnormal during or before CP switchover

time.
From the logs analysis the only issue found was CP-1 card crash at around
5:00 AM.
Since there is no other abnormal behavior observed, This issue happened in

this
RNC could be due to CP-1 card crash only and consequently it has not
recovered
automatically when one process stop/start performed from WMS and also was
not
recovered like other RNCs when WMS all processes stop/start done. Finally it
recovered with CP-Switchover action.
As already communicated below, For the Shelf Card/1 card crash issue. We
could see
this card crashed for two times as below from the card installation to till
date,
So it is advised to replace this card at the earliest possible time in
Maintenance
window to avoid further issues.
Please let me know if you need any further analysis for the same.
Old Crash info :
Unrecoverable error at line 2047 in file sfsFileSys.cc during interrupt 236

(0x000000ec), currenttask "tCardReset" (0x1c015598)
Standby CP is reset by active CP due to a problem detected by SFS.
Error Info: (0.19)

*** An exception has occurred at interrupt level ***
Offending interrupt 236 (0xec) was near address 0x019cf91c, current task
"tCardReset" (0x1c015598)
(errno = 0x029c0000)
Card Info: (0.25)

PEC Code
: NTPN08AG-10
Serial number : NNTMPJ00WFK2

PM Rev Code
Reset
: NTPN2632-12
Info: (0.17)
Reboot Reason value is 400 [hardware watchdog reset]
Last boot type: (cold autoboot)
Latest Crash :

[sfs:disk test]
Error Info: (0.18)

Card Info: (0.26)

PEC Code
: NTPN08AG-10
Serial number : NNTMPJ00WFK2

PM Rev Code
Reset
: NTPN2632-12
Info: (0.17)
No significant reboot reason available

Last boot type: (warm autoboot)
Thanks and Regards,

Praveen Kumar Thota
Mob: +91 96899 18050
Desk: +91 80 66390212

Sent: Wednesday, August 19, 2015 12:03 PM
YASSER
of
Hello Praveen
Thanks for your detailed invistagation,
So finally that issue with CP1 and happened due to the crash of it,
Please what about the below errors we shared at the beginning of the AR? Ar
there
any issue or not?
2> os excDumpCheck
Software error PRESENT on Card 0 (Standby CP)
Traceback and Software error PRESENT on Card 1 (Active CP)

Thanks,
BR,
Karim Ibrahim
M: +971(0)563544568| ONNET: 22341798
THOTA,
YASSER

of
Hi Karim,
The below os excDumpCheck command display the information related to the

trace
back and software errors which were present on these cards previously.
From RNC Log collection I could see the crash report for card-1 only in this
RNC,
I didn't find it for any other cards. All the other cards which are having
software errors could be recoverable hence there is no crash reports for
them. If
any card had unrecoverable error one crash report will be created for that
card
for each error.
If you want to diagnose more on these software errors of other cards, you
may open
a new AR for this. Thanks for understanding.
Thanks and Regards,

Praveen Kumar Thota
Mob: +91 96899 18050
Desk: +91 80 66390212
17. 19-Aug-2015 13:15

pthota
18. 19-Aug-2015 18:32

nchatter
19. 20-Aug-2015 08:56

pthota

logs,
the
20. 20-Aug-2015 08:56

pthota

To: THOTA, Praveen-K (Praveen-K)** CTR **; IBRAHIM ABD EL NABY, Karim
(Karim)**
CTR **
YASSER
of
Hello Praveen
Kindly note that we will plan for replacement for CP1 shortly
Please share the latest update MoP for current SW.
Thanks for your support,
Karim
+97156 3544568
21. 20-Aug-2015 10:03

pthota
THOTA,
Sent: Thursday, August 20, 2015 11:05 AM
YASSER
of
Hi Karim,
PFA Maintenance guide for RNC 9370. Please refer section "Replacement of a
CP in a
two-CP node" from page No : 108 in Maintenance activities.
Thanks and Regards,

Praveen Kumar Thota
Mob: +91 96899 18050
Desk: +91 80 66390212
22. 20-Aug-2015 10:03

pthota
23. 21-Aug-2015 12:18

pthota
24. 24-Aug-2015 09:27

gauravarora
25. 24-Aug-2015 09:28

gauravarora
26. 24-Aug-2015 14:09

pthota

logs,
the
board in maintenance window. Closure confirmation received to close this AR.
27. 24-Aug-2015 14:09

pthota

YASSER
of
Hi Karim,
If no other query on this issue, can we proceed to close this AR?

confirm.
Thanks to
Thanks and Regards,

Praveen Kumar Thota
Mob: +91 96899 18050
Desk: +91 80 66390212

YASSER
of
Hello Praveen
Thanks you can close the AR, and please update the final RCA on cares if not
updated,
Thanks,
BR,
Karim Ibrahim
M: +971(0)563544568| ONNET: 22341798
THOTA,
YASSER
of
Thanks Karim for closure confirmation. Sure, I will update the AR with RCA
details
before closing it in cares.
Thanks and Regards,

Praveen Kumar Thota
Mob: +91 96899 18050
Desk: +91 80 66390212
Add to log
Resolution
the
End of Assistance Request 1-5877056

New AR:
View AR | AR without Proprietary | OLCS

History | States | Assignment | Timetracking
View Attachments | Add Attachment
AR Text Search | More | CARES Home

View AR - : Assistance Request 1-5877056

Diunggah oleh

Informasi Dokumen

Deskripsi Asli:

Judul Asli

Hak Cipta

Format Tersedia

Bagikan dokumen Ini

Bagikan atau Tanam Dokumen

Opsi Berbagi

Apakah menurut Anda dokumen ini bermanfaat?

Apakah konten ini tidak pantas?

Hak Cipta:

Format Tersedia

View AR - : Assistance Request 1-5877056

Diunggah oleh

Hak Cipta:

Format Tersedia

New AR:

View AR | AR without Proprietary

Assistance Request 1-5877056

Loss of supervision on RNC235 and NodeBs under it

Aug 24 : Analysis provided from the available logs, Issue

caused due to CP card crash from slot-1, recommended to

Closed : Solution/Service Provided

Service Request Remote Technical Support

Customer Dedicated Support

No Change Request is linked to this AR

ETISALAT ABU DHABI

Emirates Telecommunications Corporation

United Arab Emirates

Karim IBRAHIM ABD EL NABY

Alcatel-Lucent : Open on behalf of Customer

Next Customer Contact

Targets for Restore & Resolve were extended by 8 days 18 hours

242895 (OXIA 627085)

Covered Service TS 24x7 (Gold, Wireless Mobile Access)

This request is entitled to service.

Loss of supervision on RNC235 and NodeBs under it

Showing Investigation and Proprietary logs together in time order

Update to Current Summary: Aug 15 : Under Investigation

Update to Current Summary: Aug 15 : Awaiting Receive Log Files / Technical

On 15.08.15 07:57, Karim (Karim)** CTR ** IBRAHIM ABD EL NABY wrote:

PROBLEM DESCRIPTION: Loss of supervision on RNC235 and NodeBs under it

Loss of supervision on RNC235 and it's NodeB, the history as below

2 hours and not locked

:+971 56 354 4568

Emirates Telecommunications Corporation

: United Arab Emirates

: ETISALAT ABU DHABI

From: SUPPORT@ALCATEL-LUCENT.COM [mailto:SUPPORT@ALCATEL-LUCENT.COM]

Thank you for contacting the Alcatel-Lucent Welcome Center.

Please update who is assigned to this AR.

Praveen-K (Praveen-K)** CTR **

I am the assignee of this AR.

--- Please confirm.

Frequency of the problem (once, x times, sporadic, systematic):

Impact (drop call, board reset, KPI degradation, redundancy, None):

RNC Log collection full for the impacted RNC

RNC Health check of the impacted RNC.

Please provide below commands output from the RNC.

Thanks and Regards,

+91 96899 18050

Incoming E-mail from Praveen-K (Praveen-K)** CTR ** THOTA

me without any information to me.

Thanks and Regards,

+91 96899 18050

Incoming E-mail from Izabela (Izabela)** CTR ** GAWRYCHOWSKA

If you need further investigation, please let me know so I can involve my

Global Welcome Center

COO Service Assurance Operations

Visit OnLine Customer Support for contact details.

From: IBRAHIM ABD EL NABY, Karim (Karim)** CTR **

Sent: Saturday, August 15, 2015 4:21 PM

we need to check and find the RC urgently as the customer is pushing ,

The issue started as below

for one Fddcell on the RNC235 take about 2 hours without

2Stop/start for pm_CP_UTRAN process done on the WMS, and finish

Then we go for lp/0 switchover on the RNC

My comments below and waiting your feedback,

On 15.08.15 07:57, Karim (Karim) CTR IBRAHIM ABD EL NABY wrote:

Praveen-K (Praveen-K) CTR

Incoming E-mail from Praveen-K (Praveen-K) CTR THOTA

Incoming E-mail from Izabela (Izabela) CTR GAWRYCHOWSKA

From: IBRAHIM ABD EL NABY, Karim (Karim) CTR

From: THOTA, Praveen-K (Praveen-K) CTR

From: IBRAHIM ABD EL NABY, Karim (Karim) CTR

From: THOTA, Praveen-K (Praveen-K) CTR

From: IBRAHIM ABD EL NABY, Karim (Karim) CTR

or, switch over

disk test].0 drc

=2 drfc=1 ......