Anda di halaman 1dari 28

View AR | AR without Proprietary

New AR:
History | States | Assignment | Timetracking
AR Text Search | More | CARES Home
Assistance Request 1-5642423
Customer Ticket N/A

Short RCA for 50 Sites on RNC231 suffered from Unavailability

Description (CellFail or IubFail) at one Interval & restored again

without any action from our side.

Current Mar 24 : RCA provided, As per the NodeB alarms raised at


that

Summary time this issue is pointing to IP backhaul fluctuations due


to

which IUB user plane and NodeB control ports were down.

State Closed : Solution/Service Provided

Outage No

Severity 1

Priority 1

Service Request Remote Technical Support

Request Type Support

Sub-Type RCA

Category Other

Sub-Category Customer Dedicated Support

Internal Yes

Assignment Events
Product
Product 9370 RNC

Version LR13.3.W

No Change Request is linked to this AR

Product Location
Site Abu Dhabi AE - Alcatel-Lucent MENA JLT-

Instance 9370RNC-AUH : In Service

Site Company Emirates Telecommunications Corporation

City Abu Dhabi

Country United Arab Emirates

Contact
Name Yasser ALI

Company Alcatel-Lucent : Open on behalf of Customer

Phone 20 23539555 ext. 1008 Email


Yasser.Ali@alcatel-lucent.com

Request Method Email-Eng

Dates
Occurred 20-Mar-2015 21:01 Time now 29-Aug-2015 11:44
(GMT+4)

Reported 20-Mar-2015 21:01 AR Created 20-Mar-2015 21:04

Service Start 20-Mar-2015 21:01

Next Customer Contact 26-Mar-2015 14:30

Responded 20-Mar-2015 21:02 Respond Target 20-Mar-2015 21:31


SA - Calculated

Restored 20-Mar-2015 21:02 Restore Target 21-Mar-2015 21:01


SA - Calculated

Resolved 22-Mar-2015 06:40 Resolve Target 23-Apr-2015 10:20


SA - Calculated

Targets for Restore & Resolve were extended by 3 days 13 hours

to account for Pending time

Closed 24-Mar-2015 11:05


Last Modified 22-Jul-2015 16:06 Modified By archive

Entitlement
Agreement 242895 (OXIA 627085)

Covered Service TS 24x7 (Gold, Wireless Mobile Access)

Script This request is entitled to service.

People
Owner TSCd-WLS-WCDMA-IN : pthota

Assignee CloQ-WLS-WCDMA-vGlobal : pthota

Referred 1 NorP-WLS-WCDMA-vGlobal : pthota

Resolve Group NorP-WLS-WCDMA-vGlobal

Submitter glipski

Description
Attachments

Incoming E-mail from YASSER (YASSER)** CTR ** ALI

Workflow Status: Handled by an agent

Subject: Unavailability on 50 Sites on RNC231

Contact: YASSER (YASSER)** CTR ** ALI

From: "ALI, YASSER (YASSER)** CTR **" <yasser.ali@alcatel-lucent.com>

To: ALU Support <support@alcatel-lucent.com>

Cc: "KHEDR, MAHMOUD (MAHMOUD)" <Mahmoud.Khedr@alcatel-lucent.com>, "MERAD,

ABDESSAMAD (ABDESSAMAD)" <abdessamad.merad@alcatel-lucent.com>, "EL-


MIDANY,

AHMED (AHMED)" <Ahmed.El-Midany@alcatel-lucent.com>, "ABDEL-HALIM, SAYED

(SAYED)" <Sayed.Abdel-Halim@alcatel-lucent.com>, 3G_ALU_Integ_UAE

<3G_ALU_Integ_UAE@exchange.alcatel-lucent.com>, "HAZEM, EL SHERIF (EL

SHERIF)" <El_Sherif.Hazem@alcatel-lucent.com>

Sent on: March 20, 2015 6:01:14 PM CET


Contact Surname : Yasser

Contact Given Name : Ali

Contact Phone :+971566735920

Contact Company : Alcatel-Lucent

Company : Emirates Telecommunications Corporation

Contract Number : OXIA 627085

Region : EMEA

Country : United Arab Emirates

City : Abu Dhabi

Site : ETISALAT ABU DHABI

Product Instance : 9370 RNC (WCDMA)

Severity : 1

Priority : 1

Software Version : RNC-9.1.0-RID37300E1953s


Outage : No

Internal/External: :Internal

Short Description : RCA for 50 Sites on RNC231 suffered from

Unavailability (CellFail or IubFail) at one Interval & restored again

without any action from our side.

Current Summary :

Below table of the sites impacted by Unavailability today 20/3/2015 at


09:00.

Showing Investigation and Proprietary logs together in time order Show


separately

Investigation Log and Proprietary Log


1. 20-Mar-2015 21:32
pthota

Update to Current Summary: Mar 20 : Under Investigation

2. 20-Mar-2015 21:47
pthota

Update to Current Summary: Mar 20 : Awaiting Receive Log Files / Technical


Info

3. 20-Mar-2015 22:23 ALCATEL-LUCENT PROPRIETARY


pthota
From: THOTA, Praveen-K (Praveen-K)** CTR **

Sent: Friday, March 20, 2015 11:47 PM

To: ALI, YASSER (YASSER)** CTR **

Cc: KHEDR, MAHMOUD (MAHMOUD); MERAD, ABDESSAMAD (ABDESSAMAD); EL-MIDANY,


AHMED

(AHMED); ABDEL-HALIM, SAYED (SAYED); 3G_ALU_Integ_UAE; HAZEM, EL SHERIF (EL

SHERIF); INDIA-WCDMA-CdTSC-TEAM; INDIA-WCDMA-BE-TEAM

Subject: AR 1-5642423/Critical/9370 RNC/Emirates Telecommunications

Corporation/RCA for 50 Sites on RNC231 suffered from Unavailability


(CellFail or

IubFail) at one Interval & restored again

Hi Yasser,

We have received your AR for RCA of the 50 sites outage for one interval.

Please answer below queries and provide below requested logs for further

investigation.

Concerned NB or RNC name:

Type of alarm:

CONFIGURATION:

1. Hardware configuration:

2. Software configuration:

>OAM:

> RNC:

> NodeB:

IMPACT:

1. Frequency of the problem (once, x times, sporadic, systematic...):


2. Features impacted (iMCTA, CAC, Rate Adaptation, HSxPA over IUR,
other...):

HISTORY:

1. Which & how many NEs impacted (2 RNC, 3 NB...):

2. How the issue was detected by customer or local ALU support (via KPIs

degradation, customer complaint...):

3. When the issue has started exact time :

4. when the issue has restored exact time :

5. What actions were performed before or in time when the issue begun
(feature

activation, upgrade, 3rd party actions...):

6. What recovery actions have been done & with what results:

1) HFB for this RNC from 2 days before the issue to till time.

2) Full RNC Log collection.

3) RNC health check.

4) Network snapshot before and after the issue.

5) IMT logs for any affected Two NodeBs.

6) HFB for any two affected NodeBs from 2 days before the issue to till
time.

7) KPI reports for this RNC for 15 Minutes interval from 2 days before
the issue

to till time.

8) KPI reports for any affected two NodeBs for 15 Minutes interval from 2
days

before the issue to till time.

Please upload all the logs to the server and provide the server path
details to

download the logs.


Thanks and Regards,

Praveen Thota

+91 96899 18050

4. 23-Mar-2015 09:59 ALCATEL-LUCENT PROPRIETARY


pthota

Time Tracking Entry Added - IR01-TSA3

5. 23-Mar-2015 10:00 ALCATEL-LUCENT PROPRIETARY


pthota

Time Tracking Entry Added - IR01-TSA3

6. 23-Mar-2015 10:00 ALCATEL-LUCENT PROPRIETARY


pthota

Time Tracking Entry Added - IR01-TSA3

7. 23-Mar-2015 10:01 ALCATEL-LUCENT PROPRIETARY


pthota

Time Tracking Entry Added - IR01-TSA3

8. 23-Mar-2015 10:02 ALCATEL-LUCENT PROPRIETARY


pthota

Time Tracking Entry Added - IR01-TSA3

9. 24-Mar-2015 11:05
pthota
Update to Current Summary: Mar 24 : RCA provided, As per the NodeB alarms
raised

at that time this issue is pointing to IP backhaul fluctuations due to


which IUB

user plane and NodeB control ports were down.

10. 24-Mar-2015 11:05 ALCATEL-LUCENT PROPRIETARY


pthota

From: THOTA, Praveen-K (Praveen-K)** CTR **

Sent: Friday, March 20, 2015 10:17 PM

To: ALI, YASSER (YASSER)** CTR **

Cc: KHEDR, MAHMOUD (MAHMOUD); MERAD, ABDESSAMAD (ABDESSAMAD); EL-MIDANY,


AHMED

(AHMED); ABDEL-HALIM, SAYED (SAYED); 3G_ALU_Integ_UAE; HAZEM, EL SHERIF (EL

SHERIF); INDIA-WCDMA-CdTSC-TEAM; INDIA-WCDMA-BE-TEAM

Subject: AR 1-5642423/Critical/9370 RNC/Emirates Telecommunications

Corporation/RCA for 50 Sites on RNC231 suffered from Unavailability


(CellFail or

IubFail) at one Interval & restored again

Hi Yasser,

We have received your AR for RCA of the 50 sites outage for one interval.

Please answer below queries and provide below requested logs for further

investigation.

Concerned NB or RNC name:


Type of alarm:

CONFIGURATION:

1. Hardware configuration:

2. Software configuration:

>OAM: LR13.3

> RNC: RNC-9.1.0- RID37300E1953s

> NodeB: V 13D3.0 EBC.01

IMPACT:

1. Frequency of the problem (once, x times, sporadic, systematic...): ONCE

2. Features impacted (iMCTA, CAC, Rate Adaptation, HSxPA over IUR,


other...):

HISTORY:

1. Which & how many NEs impacted (2 RNC, 3 NB...): (One RNC "RNC231" ,
50

NBs)

2. How the issue was detected by customer or local ALU support (via KPIs

degradation, customer complaint...): By ALU RF Team

3. When the issue has started exact time : from 20 Mar2015 @ 09:00

4. when the issue has restored exact time : 20 Mar2015 @ 10:00

5. What actions were performed before or in time when the issue begun
(feature

activation, upgrade, 3rd party actions...): NO Actions

6. What recovery actions have been done & with what results: NO Actions

9) HFB for this RNC from 2 days before the issue to till time.

10) Full RNC Log collection.

11) RNC health check.

12) Network snapshot before and after the issue.

13) IMT logs for any affected Two NodeBs.


14) HFB for any two affected NodeBs from 2 days before the issue to till
time.

15) KPI reports for this RNC for 15 Minutes interval from 2 days before
the

issue to till time.

16) KPI reports for any affected two NodeBs for 15 Minutes interval from
2 days

before the issue to till time.

Please upload all the logs to the server and provide the server path
details to

download the logs.

" Data uploaded on : Server: 172.25.80.47

/srv/data202256/data/server/Traces/from_france/ETISALAT_UAE/AR 1-5642423

" IMT logs & NPO report will be provided soon.

Thanks and Regards,

Praveen Thota

+91 96899 18050

<UAE_48Sites_Impacted.xls>

From: ALI, YASSER (YASSER)** CTR **

Sent: Saturday, March 21, 2015 12:43 AM

To: THOTA, Praveen-K (Praveen-K)** CTR **

Cc: KHEDR, MAHMOUD (MAHMOUD); MERAD, ABDESSAMAD (ABDESSAMAD); EL-MIDANY,


AHMED

(AHMED); ABDEL-HALIM, SAYED (SAYED); 3G_ALU_Integ_UAE; HAZEM, EL SHERIF (EL

SHERIF); INDIA-WCDMA-CdTSC-TEAM; INDIA-WCDMA-BE-TEAM


Subject: RE: AR 1-5642423/Critical/9370 RNC/Emirates Telecommunications

Corporation/RCA for 50 Sites on RNC231 suffered from Unavailability


(CellFail or

IubFail) at one Interval & restored again

Dear Praveen,

Kindly find Answers below & data is available on:

Server: 172.25.80.47

/srv/data202256/data/server/Traces/from_france/ETISALAT_UAE/AR 1-5642423

Thanks & BR,

Yasser Ali

+971566735920

From: ARORA, Gaurav (Gaurav)** CTR **

Sent: Saturday, March 21, 2015 12:55 AM

To: ALI, YASSER (YASSER)** CTR **; THOTA, Praveen-K (Praveen-K)** CTR **

Cc: KHEDR, MAHMOUD (MAHMOUD); MERAD, ABDESSAMAD (ABDESSAMAD); EL-MIDANY,


AHMED

(AHMED); ABDEL-HALIM, SAYED (SAYED); 3G_ALU_Integ_UAE; HAZEM, EL SHERIF (EL

SHERIF); INDIA-WCDMA-CdTSC-TEAM; INDIA-WCDMA-BE-TEAM

Subject: RE: AR 1-5642423/Critical/9370 RNC/Emirates Telecommunications

Corporation/RCA for 50 Sites on RNC231 suffered from Unavailability


(CellFail or

IubFail) at one Interval & restored again


Hi Yasser ,

We will download the logs , analyze and respond. As you mentioned that
no

action was taken and it was automatically restored , did you also verify
the

transport media. We will check the same in HFB and other logs.

Thanks and regards

Gaurav Arora

+91 9503055691

E-mail-g.arora@alcatel-lucent.com

From: THOTA, Praveen-K (Praveen-K)** CTR **

Sent: Saturday, March 21, 2015 1:43 AM

To: ARORA, Gaurav (Gaurav)** CTR **; ALI, YASSER (YASSER)** CTR **

Cc: KHEDR, MAHMOUD (MAHMOUD); MERAD, ABDESSAMAD (ABDESSAMAD); EL-MIDANY,


AHMED

(AHMED); ABDEL-HALIM, SAYED (SAYED); 3G_ALU_Integ_UAE; HAZEM, EL SHERIF (EL

SHERIF); INDIA-WCDMA-CdTSC-TEAM; INDIA-WCDMA-BE-TEAM

Subject: RE: AR 1-5642423/Critical/9370 RNC/Emirates Telecommunications

Corporation/RCA for 50 Sites on RNC231 suffered from Unavailability


(CellFail or

IubFail) at one Interval & restored again

Hi Yasser,

We have received below logs only, Kindly provide all other logs as
requested

below. Also address below queries.


1) When the issue has started exact time : from 20 Mar2015 @ 09:00 AM ?

2) when the issue has restored exact time : 20 Mar2015 @ 10:00 AM ?

3) Kindly provide the list of sites down with their cell ids & and their

transmission connectivity topology.

4) All the sites down are from same location with respect transmission
media?

5) Kindly check with customer for the transmission media outage at the same
time?

6) The command output from RNC " d rncin tmu/* cellist ".

7) The command output from RNC " d -p ipif/* "

1) HFB for this RNC from 2 days before the issue to till time. -
received

2) Full RNC Log collection. - download started ' 303MB it will take
another 2

hours to download.

3) RNC health check. - received

4) Network snapshot before and after the issue. --- received.

5) IMT logs for any affected Two NodeBs. ---- Not Received

6) HFB for any two affected NodeBs from 2 days before the issue to
till time.

--- Not Received

7) KPI reports for this RNC for 15 Minutes interval from 2 days before
the

issue to till time. ---- Not Received

8) KPI reports for any affected two NodeBs for 15 Minutes interval
from 2

days before the issue to till time. ---- Not Received

Thanks and Regards,


Praveen Thota

+91 96899 18050

On Mar 21, 2015, at 7:07 AM, THOTA, Praveen-K (Praveen-K)** CTR **

<praveen-k.thota@alcatel-lucent.com> wrote:

Hi Yasser,

As per the HFB, below are my observations.

From the below alarm analysis all the 48 impacted sites FDD cells are
degraded due

to User plane Failure observed in the IUB backhaul due to Loss of Signal.

1) Total 48 sites are impacted between 09:30 AM to 10:00 AM for two


times,

one at 09:35 AM and next at 09:45 AM. Sites details, two times impacted
sites

details are in attached excel sheet. From the alarms it is confirm that
same 48

sites impacted again in 10 minutes time difference.

2) Sites impacted at 09:35 are for less than one minute.

3) Sites impacted at 09:45 are for less than 3 minutes.

4) All sites impacted due to User Plane Failure followed the


corresponding

nodeb cells went to enabled degraded or disabled failed status, Please find
below

one example for the same.

Full Component DN Event Time Severity Clear Time


Additional Text

NW:UTRAN RNC:RNC231 NodeB:M_AR_U_2622_BainonahMzdRoad


FDDCell:AR_2622_26223_F1_C
2015-03-20T09:35:02 +0400 Critical 2015-03-20T09:35:20 +0400
State change to

Enable Degraded Associated BTS:M_AR_U_2622_BainonahMzdRoad

NW:UTRAN RNC:RNC231 NodeB:M_AR_U_2622_BainonahMzdRoad


FDDCell:AR_2622_4695_F4_A

2015-03-20T09:35:02 +0400 Critical 2015-03-20T09:35:19 +0400


State change to

Enable Degraded Associated BTS:M_AR_U_2622_BainonahMzdRoad

NW:UTRAN RNC:RNC231 NodeB:M_AR_U_2622_BainonahMzdRoad


FDDCell:AR_2622_4696_F4_B

2015-03-20T09:35:02 +0400 Critical 2015-03-20T09:35:18 +0400


State change to

Enable Degraded Associated BTS:M_AR_U_2622_BainonahMzdRoad

NW:UTRAN RNC:RNC231 NodeB:M_AR_U_2622_BainonahMzdRoad


FDDCell:AR_2622_4692_F3_B

2015-03-20T09:35:02 +0400 Critical 2015-03-20T09:35:19 +0400


State change to

Enable Degraded Associated BTS:M_AR_U_2622_BainonahMzdRoad

NW:UTRAN RNC:RNC231 NodeB:M_AR_U_2622_BainonahMzdRoad


FDDCell:AR_2622_4691_F3_A

2015-03-20T09:35:02 +0400 Critical 2015-03-20T09:35:19 +0400


State change to

Enable Degraded Associated BTS:M_AR_U_2622_BainonahMzdRoad

NW:UTRAN RNC:RNC231 NodeB:M_AR_U_2622_BainonahMzdRoad


FDDCell:AR_2622_26226_F2_B

2015-03-20T09:35:02 +0400 Critical 2015-03-20T09:35:19 +0400


State change to

Enable Degraded Associated BTS:M_AR_U_2622_BainonahMzdRoad

NW:UTRAN RNC:RNC231 NodeB:M_AR_U_2622_BainonahMzdRoad


FDDCell:AR_2622_26225_F2_A

2015-03-20T09:35:02 +0400 Critical 2015-03-20T09:35:20 +0400


State change to

Enable Degraded Associated BTS:M_AR_U_2622_BainonahMzdRoad

NW:UTRAN RNC:RNC231 NodeB:M_AR_U_2622_BainonahMzdRoad


FDDCell:AR_2622_26222_F1_B
2015-03-20T09:35:02 +0400 Critical 2015-03-20T09:35:20 +0400
State change to

Enable Degraded Associated BTS:M_AR_U_2622_BainonahMzdRoad

NW:UTRAN RNC:RNC231 NodeB:M_AR_U_2622_BainonahMzdRoad


FDDCell:AR_2622_26221_F1_A

2015-03-20T09:35:02 +0400 Critical 2015-03-20T09:35:20 +0400


State change to

Enable Degraded Associated BTS:M_AR_U_2622_BainonahMzdRoad

NW:UTRAN RNC:RNC231 NodeB:M_AR_U_2622_BainonahMzdRoad


FDDCell:AR_2622_4693_F3_C

2015-03-20T09:35:02 +0400 Critical 2015-03-20T09:35:19 +0400


State change to

Enable Degraded Associated BTS:M_AR_U_2622_BainonahMzdRoad

NW:UTRAN RNC:RNC231 NodeB:M_AR_U_2622_BainonahMzdRoad


FDDCell:AR_2622_4697_F4_C

2015-03-20T09:35:02 +0400 Critical 2015-03-20T09:35:18 +0400


State change to

Enable Degraded Associated BTS:M_AR_U_2622_BainonahMzdRoad

NW:UTRAN RNC:RNC231 NodeB:M_AR_U_2622_BainonahMzdRoad


FDDCell:AR_2622_26227_F2_C

2015-03-20T09:35:02 +0400 Critical 2015-03-20T09:35:19 +0400


State change to

Enable Degraded Associated BTS:M_AR_U_2622_BainonahMzdRoad

NW:UTRAN RNC:RNC231 NodeB:M_AR_U_2622_BainonahMzdRoad


FDDCell:AR_2622_26224_F1_D

2015-03-20T09:35:02 +0400 Critical 2015-03-20T09:35:19 +0400


State change to

Enable Degraded Associated BTS:M_AR_U_2622_BainonahMzdRoad

NW:UTRAN RNC:RNC231 NodeB:M_AR_U_2622_BainonahMzdRoad


FDDCell:AR_2622_26228_F2_D

2015-03-20T09:35:02 +0400 Critical 2015-03-20T09:35:19 +0400


State change to

Enable Degraded Associated BTS:M_AR_U_2622_BainonahMzdRoad

NW:UTRAN RNC:RNC231 NodeB:M_AR_U_2622_BainonahMzdRoad


FDDCell:AR_2622_4694_F3_D
2015-03-20T09:35:02 +0400 Critical 2015-03-20T09:35:18 +0400
State change to

Enable Degraded Associated BTS:M_AR_U_2622_BainonahMzdRoad

NW:UTRAN RNC:RNC231 NodeB:M_AR_U_2622_BainonahMzdRoad


FDDCell:AR_2622_4698_F4_D

2015-03-20T09:35:02 +0400 Critical 2015-03-20T09:35:18 +0400


State change to

Enable Degraded Associated BTS:M_AR_U_2622_BainonahMzdRoad

NW:UTRAN RNC:RNC231 NodeB:M_AR_U_2622_BainonahMzdRoad 2015-03-


20T09:35:02 +0400

Major 2015-03-20T09:35:14 +0400 > Local value :


ANO_NODEB_IP_USER_PLANE_FAILURE

Specific Problem:> Local value : ANO_NODEB_IP_USER_PLANE_FAILURE,

Please find below alarm details for the User Plane Failure.

RNC_3034_29699 - ANO NODEB IP USER PLANE FAILURE

Description : If NodeB has been configured to support IP Transport as an


option to

carry User Plane traffic, this alarm is raised when RNC detects loss if IP

connectivity to the NodeB. All the Iub User Plane traffic carried on the IP

transport is lost and new traffic cannot be setup with IP transport The
alarm is

cleared when IP connectivity to the NodeB is established.

Alarm Code : RNC_3034_29699

Specific Problem : ANO_NODEB_IP_USER_PLANE_FAILURE

Object Type : Nodeb

Probable Cause : lossOfSignal

Event Type : communications

Thanks and Regards,

Praveen Thota
+91 96899 18050

From: KHEDR, MAHMOUD (MAHMOUD)

Sent: Saturday, March 21, 2015 8:49 AM

To: THOTA, Praveen-K (Praveen-K)** CTR **

Cc: ALI, YASSER (YASSER)** CTR **; MERAD, ABDESSAMAD (ABDESSAMAD); EL-
MIDANY,

AHMED (AHMED); ABDEL-HALIM, SAYED (SAYED); 3G_ALU_Integ_UAE; HAZEM, EL


SHERIF (EL

SHERIF); INDIA-WCDMA-CdTSC-TEAM; INDIA-WCDMA-BE-TEAM; ARORA, Gaurav


(Gaurav)** CTR

**

Subject: Re: AR 1-5642423/Critical/9370 RNC/Emirates Telecommunications

Corporation/RCA for 50 Sites on RNC231 suffered from Unavailability


(CellFail or

IubFail) at one Interval & restored again

Hello Praveen,

Thanks for the analysis

I have the following queries please

Are we sure that Nothing abnormal in RNC and it is IP network issue?

Is there any reset of TMUs can lead to this?

Why we have in some cases cell fail only and in other cases IuB fail?

Are all the sites impacted twice one at 9:35 and one at 9:45?

Best regards,

Mahmoud Khedr

Sent from my iPhone


From: THOTA, Praveen-K (Praveen-K)** CTR **

Sent: Saturday, March 21, 2015 8:07 AM

To: KHEDR, MAHMOUD (MAHMOUD)

Cc: ALI, YASSER (YASSER)** CTR **; MERAD, ABDESSAMAD (ABDESSAMAD); EL-
MIDANY,

AHMED (AHMED); ABDEL-HALIM, SAYED (SAYED); 3G_ALU_Integ_UAE; HAZEM, EL


SHERIF (EL

SHERIF); INDIA-WCDMA-CdTSC-TEAM; INDIA-WCDMA-BE-TEAM; ARORA, Gaurav


(Gaurav)** CTR

**

Subject: RE: AR 1-5642423/Critical/9370 RNC/Emirates Telecommunications

Corporation/RCA for 50 Sites on RNC231 suffered from Unavailability


(CellFail or

IubFail) at one Interval & restored again

Hi Mahmoud,

From the RNC Log collection, I have not found anything abnormal. So I am
not

suspecting anything from RNC. From HFB it can be concluded that the issue
is due

to IP Backhaul.

To find the reset TMUs -' I have requested some logs as below. From RNC
log

collection nothing abnormal with respect TMUs.

1) Kindly provide the list of sites down with their cell ids & and their

transmission connectivity topology.

2) The command output from RNC " d rncin tmu/* cellist ".

3) The command output from RNC " d -p ipif/* "


From the HFB we have alarms two times in between 09:00AM and 10:00AM, one
at

09:35AM and the other at 09:45AM. There is no other alarms raised in that
period.

For the cell failures only -'

All the cell failure alarms raised along with the below QOS alarms raised
for all

control ports of the NodeB at the same time when IUB alarms raised for
other

NodeBs, which is also pointing to backhaul issue.

Hence please check with concerned IP backhaul team for the issues observed
in the

network at 09:35 AM and 09:45AM.

RNC_0085_00036 - State change to Disable Failed

Description : This alarm is raised when the RNC sends to WMS a state change
to

Disable/Failed about the object Control Port State change to Disable


Failed.

Alarm Code RNC_0085_00036

NE Type 9370 RNC

Object Class : RNC/NodeB/ControlPort

Object Type RNC

Event Type : quality of service

Probable Cause : underlyingResourceUnavailable

Severity MAJOR

Impact on system operation : According the ControlPort type (nodeBCP,CCP),


the
service is no more available for the FddCell associated to the faulty CCP,
or for

the whole NodeB associated to the faulty nodeBCP.

Reason : This alarm is raised when the RNC sends to WMS a state change to

Disable/Failed about the object ControlPort.

Remedial Action : Check the transmission alarms and the NodeB alarms to
find the

root cause.

Yes, all the sites impacted at 09:35 AM are repeated at 09:45 AM. I have
provided

the sites details in the excel sheet attached to my last mail.

Thanks and Regards,

Praveen Thota

+91 96899 18050

From: KHEDR, MAHMOUD (MAHMOUD)

Sent: Saturday, March 21, 2015 10:39 AM

To: THOTA, Praveen-K (Praveen-K)** CTR **

Cc: ALI, YASSER (YASSER)** CTR **; MERAD, ABDESSAMAD (ABDESSAMAD); EL-
MIDANY,

AHMED (AHMED); ABDEL-HALIM, SAYED (SAYED); 3G_ALU_Integ_UAE; HAZEM, EL


SHERIF (EL

SHERIF); INDIA-WCDMA-CdTSC-TEAM; INDIA-WCDMA-BE-TEAM; ARORA, Gaurav


(Gaurav)** CTR

**

Subject: RE: AR 1-5642423/Critical/9370 RNC/Emirates Telecommunications

Corporation/RCA for 50 Sites on RNC231 suffered from Unavailability


(CellFail or

IubFail) at one Interval & restored again


Thanks a lot

From: HAZEM, EL SHERIF (EL SHERIF)

Sent: Saturday, March 21, 2015 12:48 PM

To: KHEDR, MAHMOUD (MAHMOUD); THOTA, Praveen-K (Praveen-K)** CTR **

Cc: ALI, YASSER (YASSER)** CTR **; MERAD, ABDESSAMAD (ABDESSAMAD); EL-
MIDANY,

AHMED (AHMED); ABDEL-HALIM, SAYED (SAYED); 3G_ALU_Integ_UAE;

INDIA-WCDMA-CdTSC-TEAM; INDIA-WCDMA-BE-TEAM; ARORA, Gaurav (Gaurav)** CTR


**

Subject: RE: AR 1-5642423/Critical/9370 RNC/Emirates Telecommunications

Corporation/RCA for 50 Sites on RNC231 suffered from Unavailability


(CellFail or

IubFail) at one Interval & restored again

Hi Praveen,

Kindly find attached the list of sites that were down with their cell
lists, and

the output from the RNC requested commands

Thanks

Regards,

Hazem El-Sherif

3G Integration Professional

3G Wireless Services.

Alcatel-Lucent
M: +971 56 610 4472

el_sherif.hazem@alcatel-lucent.com

Every sucess has its network

From: THOTA, Praveen-K (Praveen-K)** CTR **

Sent: Sunday, March 22, 2015 8:10 AM

To: HAZEM, EL SHERIF (EL SHERIF); KHEDR, MAHMOUD (MAHMOUD)

Cc: ALI, YASSER (YASSER)** CTR **; MERAD, ABDESSAMAD (ABDESSAMAD); EL-
MIDANY,

AHMED (AHMED); ABDEL-HALIM, SAYED (SAYED); 3G_ALU_Integ_UAE;

INDIA-WCDMA-CdTSC-TEAM; INDIA-WCDMA-BE-TEAM; ARORA, Gaurav (Gaurav)** CTR


**

Subject: RE: AR 1-5642423/Critical/9370 RNC/Emirates Telecommunications

Corporation/RCA for 50 Sites on RNC231 suffered from Unavailability


(CellFail or

IubFail) at one Interval & restored again

Hi Hazem and All,

Thanks for the Logs.

I have verified the TMUs and PCs. As per the attached sheets all affected
Cells

and IUBs are distributed among all the TMUs and PCs of the RNC, So this
issue is

not related to any particular TMUs or PCs.

From RNC Log collection PPC_alarm logs, I didn't notice any abnormal
messages

from any RNC boards or LPs during the outage period.


So As mentioned below please check with the concerned IP backhaul team for
the

root cause, sine the alarms raised on NodeBs are pointing to IP backhaul
issue.

If you have transmission connectivity topology, please check the IP path of


all

these sites to conclude the same.

Please let me know if you need any further assistance regarding this issue.

Thanks and Regards,

Praveen Thota

+91 96899 18050

From: THOTA, Praveen-K (Praveen-K)** CTR **

Sent: Tuesday, March 24, 2015 7:47 AM

To: HAZEM, EL SHERIF (EL SHERIF); KHEDR, MAHMOUD (MAHMOUD)

Cc: ALI, YASSER (YASSER)** CTR **; MERAD, ABDESSAMAD (ABDESSAMAD); EL-
MIDANY,

AHMED (AHMED); ABDEL-HALIM, SAYED (SAYED); 3G_ALU_Integ_UAE;

INDIA-WCDMA-CdTSC-TEAM; INDIA-WCDMA-BE-TEAM; ARORA, Gaurav (Gaurav)** CTR


**

Subject: RE: AR 1-5642423/Critical/9370 RNC/Emirates Telecommunications

Corporation/RCA for 50 Sites on RNC231 suffered from Unavailability


(CellFail or

IubFail) at one Interval & restored again

Hi Hazem,
If there is no other query, we would like to close this AR. Thanks to
confirm the

same.

Thanks and Regards,

Praveen Thota

+91 96899 18050

From: HAZEM, EL SHERIF (EL SHERIF)

Sent: Tuesday, March 24, 2015 12:03 PM

To: THOTA, Praveen-K (Praveen-K)** CTR **; KHEDR, MAHMOUD (MAHMOUD)

Cc: ALI, YASSER (YASSER)** CTR **; MERAD, ABDESSAMAD (ABDESSAMAD); EL-
MIDANY,

AHMED (AHMED); ABDEL-HALIM, SAYED (SAYED); 3G_ALU_Integ_UAE;

INDIA-WCDMA-CdTSC-TEAM; INDIA-WCDMA-BE-TEAM; ARORA, Gaurav (Gaurav)** CTR


**

Subject: RE: AR 1-5642423/Critical/9370 RNC/Emirates Telecommunications

Corporation/RCA for 50 Sites on RNC231 suffered from Unavailability


(CellFail or

IubFail) at one Interval & restored again

Hi Praveen,

Please close this AR.

As usual, thanks for your support

Regards,

Hazem El-Sherif
3G Integration Professional

3G Wireless Services.

Alcatel-Lucent

M: +971 56 610 4472

el_sherif.hazem@alcatel-lucent.com

Every sucess has its network

From: THOTA, Praveen-K (Praveen-K)** CTR **

Sent: Tuesday, March 24, 2015 12:24 PM

To: HAZEM, EL SHERIF (EL SHERIF); KHEDR, MAHMOUD (MAHMOUD)

Cc: ALI, YASSER (YASSER)** CTR **; MERAD, ABDESSAMAD (ABDESSAMAD); EL-
MIDANY,

AHMED (AHMED); ABDEL-HALIM, SAYED (SAYED); 3G_ALU_Integ_UAE;

INDIA-WCDMA-CdTSC-TEAM; INDIA-WCDMA-BE-TEAM; ARORA, Gaurav (Gaurav)** CTR


**

Subject: RE: AR 1-5642423/Critical/9370 RNC/Emirates Telecommunications

Corporation/RCA for 50 Sites on RNC231 suffered from Unavailability


(CellFail or

IubFail) at one Interval & restored again

Hi Hazem,

Thanks for the AR closure confirmation. This AR is closed Now.

Thanks and Regards,

Praveen Thota

+91 96899 18050

Time Tracking Entry Added - IR01-TSA1


Resolution
As per the NodeB alarms raised at that time this issue is pointing to IP
backhaul

fluctuations due to which IUB user plane and NodeB control ports were down.

End of Assistance Request 1-5642423


View AR | AR without Proprietary | OLCS
New AR:
History | States | Assignment | Timetracking
AR Text Search | More | CARES Home