2012 Feb EBS Concurrent Processing Troubleshooting

1
Copyright 2012, Oracle and/or its affiliates. All rights reserved.
My Oracle Support Advisor Webcast Program
EBS Concurrent Processing Troubleshooting Techniques

Wednesday, 15 February, 2012 Upcoming Webcasts in EBS Technology area : 06:00 PM CET (GMT +2) o March 2012 - EBS Patching Overview & Troubleshooting Teleconference Access: North America: 1866 966 9439 International (UK): +44 (0) 1452 562 665
Conference ID : 39485824
Do you have any requests for future EBS - Technology Webcast Events? Please email your suggestions to ruediger.ziegler@oracle.com,subject: Topics of Interest.
AGENDA
Presentation and Demo approximately 60 - 75 minutes Q&A Session maximum 15 minutes
Web attendees can ask questions via Q&A panel Phone attendees can ask questions via Q&A panel or phone
3 Copyright 2012, Oracle and/or its affiliates. All rights reserved.
Who to ask?
3 Send
your question
Q&A panel
type your question here
your question pops-up here
ATTENTION AUDIO INFORMATION

Voice streaming/Audio broadcast is available. For full audio access, please join the telephone conference.
Teleconference Connect details:
1. Conference ID: 39485824 2. International dial in: +44 (0) 1452 562 665 3. List with national toll free numbers is available in Note 1148600.1 Note: You can view this info anytime using WebEx menu from your WebEx-Session : Select Communicate --> Join Teleconference.
6
Safe Harbor Statement

The following is intended to outline our general product direction.
It is intended for information purposes only, and may not be

incorporated into any contract. It is not a commitment to deliver any material, code, or functionality, and should not be relied upon in making purchasing decisions. The development, release, and timing of any features or functionality described for Oracles products remains at the sole discretion of Oracle.
EBS Concurrent Processing Troubleshooting Techniques

Presenters: Kandasamy Nainamalai, Kalaivani Murugesan and Rakesh Chithuluri Q&A Panel Experts : Amira Abdelkader , Vishwanatha Rao,Thiagarajan, Martin Fritz
Program Agenda
Concurrent Manager - Troubleshooting.
1. Manager Not Running (inactive NoManager) 2. How to run cmclean(Pending Requests) 3. ICM Running , Other Managers are down 4. Only OPP & Workflow services Down 5. DBMS_LOCK Error - ICM Log 6. After CLONE Managers Down/FNDSM Errors/CONC-SM TNS FAIL 7. Managers down - Status "System Hold, Fix Manager" 8. Managers down - "Target node/queue unavailable 9. How to Enable diag=y
Concurrent Request Troubleshooting

1. ORA-20100 Error Troubleshooting 2. Rep-3000 Error troubleshooting 3. Unable to View output/log files 4. How to Collect a Trace and Tkprof
XML Publisher Reports - Troubleshooting

1. XML Publisher Introduction 2. How to Obtain OPP Log 3. How to enable & collect XDO Log 4. BI Publisher - Troubleshooting

1. Manager Not Running (inactive NoManager) 2. How to run cmclean(Pending Requests)
10
Concurrent Manager Troubleshooting

Concurrent Request Stays Inactive NoManager
11
Manager Not Running (Inactive NoManager)
System Administrator Responsibility -> Concurrent -> Manager ->
Administer Form
-> Verify the Concurrent Managers Status
Login into Backend CP Node
Verify ps ef | FNDLIBR process Start Concurrent Manager (adcmctl.sh) Ensure All Managers are running now.
12
How to Run CMCLEAN
Ensure Concurrent Managers are down. Collect cmclean.sql script from below document. Execute the cmclean.sql script. Start the Concurrent Manager (adcmctl.sh)
Note.134007.1 Ext/Pub CMCLEAN.SQL - Non Destructive Script to Clean Concurrent Manager Tables
13
Concurrent Manager Troubleshooting Demonstration part1
14

1. ICM Running , Other Managers are down 2. Only OPP & Workflow services Down 3. DBMS_LOCK Error - ICM Log 4. After CLONE Managers Down/ FNDSM Errors/CONC-SM TNS FAIL 5. Managers down - Status "System Hold, Fix Manager" 6. Managers down - "Target node/queue unavailable 7. How to Enable diag=y
15
ICM Running , Other Managers are down
16
Solution :
1. Check listener Status (adalnctl.sh status) 2. Stop Concurrent Manager (adcmctl.sh stop) 3. Start Listener (adalnctl.sh start) 4. Start Concurrent Manager (adcmctl.sh start)
17
Concurrent Manager Troubleshooting Demonstration part2
18
Only OPP & Workflow services Down
19
Solution :
Ensure Concurrent:GSM Enabled profile is set to Y
Verify Service Manager status in Administer Form.
Verify Service Manager Definition.

Ensure FNDSM Entries available in FND_CONCURRENT_QUEUES Table FNDSM entry should be correct in Tnsnames.ora file and tnsping FNDSM_hostname should work fine. Then Bounce the Services.
20
DBMS_LOCK Error Internal Concurrent Manager Log
21
Solution :
Ensure Concurrent Managers are down. Ensure no FNDLIBR process running ps ef|grep FNDLIBR Run Cmclea.sql ( commit is mandatory)
Then Start the services adstrtal.sh apps/apps

Now you can see Managers Up and running.
22
After CLONE Managers Down

After CLONE Concurrent Manager not Starting ICM logfile shows the following errors: Error#1: CONC-SM TNS FAIL Call to StopProcess failed for FNDCPGSC CONC-SM TNS FAIL Call to PingProcess failed for FNDOPP Error#2: Could not contact Service Manager FNDSM_DHSRVMDS518_SDBA. The TNS alias could not be located, the listener process on DHSRVMDS518 could not be contacted, or the listener failed to spawn the Service Manager process. Error#3:
Cause: cleanup_node failed due to ORA-01427: single-row subquery returns more than one row ORA-06512: at "APPS.FND_CP_FNDSM", line 29 ORA-06512: at line 1.
23
Solution:
Ensure Database is running Ensure Managers are down. Connect SQLPLUS as APPS user and run the following :
EXEC FND_CONC_CLONE.SETUP_CLEAN;
COMMIT; EXIT; Run Autoconfig on DB node and then Apps Node Run $FND_TOP/patch/115/sql/afdcm037.sql script Start Services (adstrtal.sh)
24
References:
Concurrent Processing - After Cloning all the Concurrent Managers do not start for the cloned Instance (Doc ID 555081.1)
ICM log file shows 'CONC-SM TNS FAIL', 'Call to PingProcess failed', and/or 'Call to StopProcess failed' for FNDCPGSC/FNDOPP (Doc ID 1312632.1)
25
Managers down - Status "System Hold, Fix Manager"
26
Solution:
Ensure Concurrent :GSM Enabled profile is set to Y Run $FND_TOP/patch/115/sql/afdcm037.sql Go to $FND_TOP/bin
adrelink.sh force=y fnd FNDLIBR

adrelink.sh force=y fnd FNDSM adrelink.sh force=y fnd FNDFS
adrelink.sh force=y fnd FNDCRM

Run cmclean.sql Start Application Service (adstrtal.sh)
27
Managers down Target node/queue unavailable
28
Solution:
Ensure Database is Running and Middle Tier Services are down. o Connect SQLPLUS as APPS user and run the following : EXEC FND_CONC_CLONE.SETUP_CLEAN; COMMIT; EXIT; o Run AutoConfig on all tiers, firstly on the DB tier and then the APPS tiers and webtier to repopulate the required system tables o Run the CMCLEAN.SQL script from the referenced note below (don't forget to commit). Note.134007.1 CMCLEAN.SQL - Non Destructive Script to Clean Concurrent Manager Tables
o Start the middle tier services including your concurrent manager.

o Retest the issue.
29
How to Enable diag=y

Stop Concurrent Manager Take backup of adcmctl.sh
Edit adcmctl.sh to append diag=y

$FND_TOP/bin/startmgr sysmgr="$unpw@$DB_NAME" $@ mgrname=$MGR_NAME diag=y else
$FND_TOP/bin/startmgr sysmgr="$unpw@$DB_NAME" mgrname=$MGR_NAME diag=y

fi;
Start Concurrent Manager
30
Concurrent Request Troubleshooting

1. ORA-20100 Error Troubleshooting 2. Rep-3000 Error troubleshooting 3. Unable to View output/log files 4. How to Collect a Trace and Tkprof
31
Concurrent Request Troubleshooting Demonstration Part1
32
ORA-20100 Error Troubleshooting
33
Solution:
Ensure Tmp Directory is valid. ($APPLPTMP)
Tmp Directory should have enough space.
Applmgr user does not have valid permission (Read/Write/Execute)

Different instances use same Tmp Directory
UTL_FILE_DIR Database Parameter does not have APPLPTMP Entry
34
Rep-3000 Error troubleshooting

Any Bitmap Reports Failing with following Error:
APPLLCSP Environment Variable set to : XML_REPORTS_XENVIRONMENT is : /u01/app/protoora/8.0.6/guicommon6/tk60/admin/Tk2Motif_UTF8.rgb XENVIRONMENT is set to: /u01/app/protoora/8.0.6/guicommon6/tk60/admin/Tk2Motif_UTF8.rgb
Current NLS_LANG and NLS_NUMERIC_CHARACTERS Environment Variables are : American_America.UTF8 '' REP-3000: Internal error starting Oracle Toolkit. REP-3000: Internal error starting Oracle Toolkit.
35
Solution:
Ensure an Xserver configured for CP server Ensure DISPLAY is set to valid Xserver Set DISPLAY in Xserver as root user and confirm Xclock is working Run xhost + From Backend CP Node Set DISPLAY as applmgr User
Run Xclock from CP Node to verify connection between CP Node and Xserver
Go to Front end and Verify DISPLAY Value If DISPLAY is incorrect in Front end then, Go to Oracle Application Manager and Change the s_display. Then Run Autoconfig Bounce Concurrent Manager.
36
Unable to View output/log files
Ensure RRA: Enabled profile is Set to Yes
37
Unable to View output/log files
Ensure Listener is Running fine. adalnctl.sh status

How to Collect a Trace and Tkprof

Program Level Trace (Not Recommended)
Go to Concurrent Program -> Define form -> Query for Concurrent Program Enable Trace
Request Level Trace (Recommended)

Ensure Concurrent: Allow Debugging profile is Set to Yes On Submission Request Form -> Click on Debug Options Button Select the Required Trace Level -> Click Ok.
39
Get Trace File Name

Run the following SQL to find out the Raw trace name and location for the concurrent program. The SQL prompts the user for the request id prompt accept request prompt 'Please enter the concurrent request id for the appropriate concurrent program:' prompt column traceid format a8 column tracename format a80 column user_concurrent_program_name format a40 column execname format a15 column enable_trace format a12 set lines 80 set pages 22 set head off SELECT 'Request id: '||request_id , 'Trace id: '||oracle_Process_id, 'Trace Flag: '||req.enable_trace, 'Trace Name: '||dest.value||'/'||lower(dbnm.value)||'_ora_'||oracle_process_id||'.trc', 'Prog. Name: '||prog.user_concurrent_program_name, 'File Name: '||execname.execution_file_name|| execname.subroutine_name , 'Status : '||decode(phase_code,'R','Running') ||'-'||decode(status_code,'R','Normal'), 'SID Serial: '||ses.sid||','|| ses.serial#, 'Module : '||ses.module from fnd_concurrent_requests req, v$session ses, v$process proc, v$parameter dest, v$parameter dbnm, fnd_concurrent_programs_vl prog, fnd_executables execname where req.request_id = &request and req.oracle_process_id=proc.spid(+) and proc.addr = ses.paddr(+) and dest.name='user_dump_dest' and dbnm.name='db_name' and req.concurrent_program_id = prog.concurrent_program_id and req.program_application_id = prog.application_id --- and prog.application_id = execname.application_id and prog.executable_application_id = execname.application_id and prog.executable_id=execname.executable_id;
40
Get TkProf File

Syntax : $ tkprof <Trace Filename> <Tkprof Filename> explain=apps/password sort=(prsela,exeela,fchela)
41
XML Publisher Reports

Introduction to XML Publisher Reports OPP log file XDO log file Troubleshooting
42
XML Publisher - Troubleshooting
Demonstration
43
OPP Log File

Possible ways to obtain the OPP log file: 1. Directly from the file system based after identifying the corresponding OPP log file name using the following SQL statement: SELECT fcpp.concurrent_request_id req_id, fcp.node_name, fcp.logfile_name FROM fnd_conc_pp_actions fcpp, fnd_concurrent_processes fcp WHERE fcpp.processor_id = fcp.concurrent_process_id AND fcpp.action_type = 6 AND fcpp.concurrent_request_id = &&request_id Note: The SQL statement will only return data for those Concurrent Requests having a Layout specified (Upon Completion... window).
2. Via the Forms application: 1.Login to the application as SYSADMIN 2. Responsibility: System Administrator 3. Function: Concurrent --> Manager --> Administration 4. Select the Output Post Processor 5. Click on the Processes button 6. Select the Concurrent Process which was active during the time that the request ran 7. Click on the Manager Log button to open the Output Post Processor log file
44
3. the Oracle Application Manager (OAM) 1. Login to the application as SYSADMIN 2. Responsibility: System Administration 3. Function: Oracle Applications Manager --> Concurrent Managers 4. Select the Output Post Processor Service and click on View Details 5. Click on View Processes 6. Select the Concurrent Process which was active during the time that the request ran 7. Click on the Log button to open the Output Post Processor log file
45
XML Publisher Core Engine Debug logs

The XDO log consist of complete debug information generated by XML Publisher Core engine. Follow the steps given below to enable debug
1. Connect to the concurrent manager server as Application Owner(applmgr). 2. Create an $XDO_TOP/temp and an $XDO_TOP/resource directory. 3. Create an xdodebug.cfg file in the $XDO_TOP/resource directory, containing the following 2 lines: LogLevel=STATEMENT LogDir=[full XDO_TOP]/temp 4. Restart the concurrent managers in case of single step publishing method (not required for two step publishing method). 5. Reproduce the problem.
Note: The xdodebug.cfg file can also be created in the $AF_JRE_TOP/jre/lib OR $AF_JRE_TOP/lib directory. With the above steps several debug files will be created under the $XDO_TOP/temp directory after reproducing the Problem xdo.log : XML Publisher Core engine debug log file xdo_...xsl : XSL-FO version of the RTF template xdo_...xml : XML data file xdo_...fo : temporary file created by the FO Engine (only RTF / XSL templates) xdo_...out : output file (may not exist in case of complete failure).
46
Troubleshooting BI Publisher Issues

Out Of Memory Error :
Symptoms: The Report completes with an Error The log file includes a message similar to the following: +------------- 1) PUBLISH -------------+ Beginning post-processing of request 744134 on node R12P660 at 10-JAN-2012 14:50:47. Post-processing of request 744134 failed at 10-JAN-2012 14:51:15 with the Error message: One or more post-processing actions failed. Consult the OPP service log for details. OR '****Warning!!! Due to high volume of data, got out of memory exception...*** '****Please retry with scalable option or modify the Data template to run in scalable mode...*** OR java.lang.OutOfMemoryError: Java heap space
Memory Issues can be at 2 levels:

1)Xml Data generation 2)Report Generation from XML data
47
Xml Data Generation

If the XML data generation is not proper, follow the steps given below
Steps:
Increase the memory from CP Definition Go to "System Administrator" responsibility. Navigate to Concurrent - > Program - > Define. Search for the report in the Concurrent Program (CP) window. Set the Options field to Xmx2048M (Note: Dont miss the dash before the X) Press Save Bounce the Concurrent Manager (optional).
48
Report Generation from XML data

XML Generation is fine but OPP fails with java.lang.OutOfMemoryError exception. 1.Check the current OPP JVM Heap size using following SQL query
SQL> select DEVELOPER_PARAMETERS from FND_CP_SERVICES where SERVICE_ID = (select MANAGER_TYPE from FND_CONCURRENT_QUEUES where CONCURRENT_QUEUE_NAME = 'FNDCPOPP'
2. The default should be: J:oracle.apps.fnd.cp.gsf.GSMServiceController:-mx512m 3. Increase the Heap Space per Process to 1024: SQL> update FND_CP_SERVICES set DEVELOPER_PARAMETERS = 'J:oracle.apps.fnd.cp.gsf.GSMServiceController:-mx1024m' where SERVICE_ID = (select MANAGER_TYPE from FND_CONCURRENT_QUEUES where CONCURRENT_QUEUE_NAME = 'FNDCPOPP'); SQL>Commit;
You need to restart the Concurrent Manager to make it effective. The maximim heap size should not exceed 2048.
49
Bi Publisher Enhancement Parameters

Temporary Directory : Set this to a valid path, make sure that the temp directory is at least 5GB or 20x larger than largest data file running. The application user should have read & write permissions to this directory
Navigation Path : XML Publisher Administrator responsibility -> Choose Configuration tab ->General ->Temporary directory
There are few FoProcessing Parameters which can be fine tuned for better performance
Use XML Publisher's XSLT processor : Set this value to True Enable scalable feature of XSLT processor : Set this value to False at Site level. For large reports set this value to true at data definition level and template level. Enable XSLT runtime optimization : Set this value to True
50
Specific Situations
There are situations for which the concurrent manager process (FNDLIBR) does produce an XML data file but the post-processing of the request fails due to a different issue encountered, some of which the OPP does not even get invoked in the first place:
1. The Output Post Processor is not running or does not pick up the request: ... +------------- 1) PUBLISH -------------+ Unable to find an Output Post Processor service to post-process request 260006. Check that the Output Post Processor service is running. +--------------------------------------+ ... The OPP log file cannot be determined using the SQL statement described earlier, as the OPP did not perform any action of the given request. Cause: OPP Manger is Down/Deactivated. Action: Verify that the Output Post Processor is enabled and active 1. 2. 3. 4. 5. 6. 7. Profile Option "Concurrent: GSM Enabled" must be set to Y Navigate to: Concurrent -> Managers -> Define Query for Manager = 'Output Post Processor or Short Name = FNDCPOPP Check the checkbox "Enable", save the changes. Navigate to: Concurrent -> Managers -> Administer Make sure the Output Post Processor Status is not 'Deactivated',Set Status to 'Activate' if necessary. Bounce the concurrent manager for the changes to get reflected.
51
2. The Output Post Processor is running but has not picked up the request: ... +------------- 1) PUBLISH -------------+ The Output Post-processor is running but has not picked up this request. No further attempts will be made to post-process this request, and the request will be marked with Warning status. +--------------------------------------+ The OPP log file cannot be determined using the SQL statement described earlier, as the OPP did not perform any action of the given request. Cause: The number of concurrent requests that the Output Post Processor can handle in parallel depends upon the number of Processes and the number of Threads Per Process. The default values are 2 Processes and 5 Threads Per Process so a total of 10 reports can be processed in parallel. In case there are other concurrent requests running which have already invoked the OPP then it might happen that no additional requests can be picked up for a period of time. Action: 1.Logon to Applications with "System Administrator" responsibility 2.Navigate to Concurrent -> Manager -> Define 3.Query for "Output Post Processor" 4.Click on "Work Shifts" and Increase the number of increase the number threads from 5 to 10 oracle.apps.fnd.cp.opp.OPPServiceThread:2:0:max_threads=5 to oracle.apps.fnd.cp.opp.OPPServiceThread:2:0:max_threads=10 5.Bounce the concurrent Manager for the changes to get reflected. Reference document for Process Vs Thread Doc ID 1399454.1- Tuning Output Post Processor (OPP) to Improve Performance
52
3. The concurrent manager has timed out waiting for the Output Post-processor to finish this request. ... +------------- 1) PUBLISH -------------+ The concurrent manager has timed out waiting for the Output Post-processor to finish this request. Check that there are enough Output Post-processor service processes running. More information may be found in the service process logfile. +--------------------------------------+ The concurrent manager process has successfully invoked the OPP but it a timeout is encountered as the OPP takes too long to complete the job
Cause: Once the Output Post Processor picks up the request, the BI Publisher engine is invoked to generate the final output file. The time that this takes will depends on various elements such as size of the XML data file, complexity of the template, performance of the server.By default a timeout will occur if it takes longer then 300 seconds (5 min.) for the BI Publisher engine to generate the output file. The concurrent request will complete with status Warning and the request log file will contain above error Message.
Action Increase the value of the Concurrent:OPP Process Timeout as follows:
1.Logon to Applications with "System Administrator" responsibility Navigate to Profile -> System 2.Query for "Concurrent:OPP Process Timeout". Set the value to 10800 sec (3 hours) 3. Query for "Concurrent:OPP Response Timeout". If the value of the Concurrent:OPP Response Timeout is 120 increase it to 240. (Value * 2)
53
Please provide the following information while raising SR with oracle support for XML Publisher Reports 1.Request log file of the failed concurrent program. 2.Out Post Processer log File
54
E-Business Suite Technology Group - Webcasts

March 2012 : EBS Patching Overview & Troubleshooting
o o EMEA-Session : 13th March 2012 at 10:00 CET (GMT +2) US-Session : 14th March 2012 at 18:00 CET (GMT +2)
For complete details on all upcoming Oracle Advisor Webcast Events, please see Note 740966.1, Oracle Advisor Webcast Schedule. For EBS Technology Specific Webcasts please check Note 1186338.1. Do you have any requests for future ATG Advisor Webcast Events ? Please email your suggestions to me :
ruediger.ziegler@oracle.com, subject: Topics of Interest.

Communities in My Oracle Support
56
EBS Technology Communities in MOS available

Following Communities are available in My Oracle Support
Oracle E-Business Suite ... BI Publisher - Business Intelligence Products (not only EBS) Core Concurrent Processing - Anything around Concurrent Processing and Concurrent Managers Core Workflow - Any Workflow isse not only E-Business Suite Diagnostic Tools - Anything around EBS Diagnostics E-Business Customizations - Your Customizations Installation - Fresh Install of the E-Business Suite Patch Review EBS - Review of Patches around the E-Business Suite Performance - EBS Performance Upgrade - EBS Upgrade User Produktivity Kit - User Productivity Kit (UPK) available for the E-Business Suite Utilities - Utilities / generic EBS DBA issues ... This is the current list for the E-Business Suite Applications Technology Group
57
EBS ATG Product Information Center Note 1160285.1
58
Q&A
59
THANK YOU
for attending our Advisor Webcast!
60
61
62

2012 Feb EBS Concurrent Processing Troubleshooting

Diunggah oleh

Informasi Dokumen

Deskripsi Asli:

Hak Cipta

Format Tersedia

Bagikan dokumen Ini

Bagikan atau Tanam Dokumen

Opsi Berbagi

Apakah menurut Anda dokumen ini bermanfaat?

Apakah konten ini tidak pantas?

Hak Cipta:

Format Tersedia

2012 Feb EBS Concurrent Processing Troubleshooting

Diunggah oleh

Hak Cipta:

Format Tersedia

1

Copyright 2012, Oracle and/or its affiliates. All rights reserved.

My Oracle Support Advisor Webcast Program

EBS Concurrent Processing Troubleshooting Techniques

Copyright 2012, Oracle and/or its affiliates. All rights reserved.

type your question here

Copyright 2012, Oracle and/or its affiliates. All rights reserved.

your question pops-up here

Copyright 2012, Oracle and/or its affiliates. All rights reserved.

ATTENTION AUDIO INFORMATION

Copyright 2012, Oracle and/or its affiliates. All rights reserved.

Safe Harbor Statement

It is intended for information purposes only, and may not be

Copyright 2012, Oracle and/or its affiliates. All rights reserved.

EBS Concurrent Processing Troubleshooting Techniques

Concurrent Request Troubleshooting

XML Publisher Reports - Troubleshooting

Copyright 2012, Oracle and/or its affiliates. All rights reserved.

Concurrent Manager - Troubleshooting.

Copyright 2012, Oracle and/or its affiliates. All rights reserved.

Concurrent Manager Troubleshooting

Copyright 2012, Oracle and/or its affiliates. All rights reserved.

Manager Not Running (Inactive NoManager)

System Administrator Responsibility -> Concurrent -> Manager ->

-> Verify the Concurrent Managers Status

Login into Backend CP Node

Copyright 2012, Oracle and/or its affiliates. All rights reserved.

How to Run CMCLEAN

Copyright 2012, Oracle and/or its affiliates. All rights reserved.

Concurrent Manager Troubleshooting Demonstration part1

Copyright 2012, Oracle and/or its affiliates. All rights reserved.

Concurrent Manager - Troubleshooting.

Copyright 2012, Oracle and/or its affiliates. All rights reserved.

ICM Running , Other Managers are down

Copyright 2012, Oracle and/or its affiliates. All rights reserved.

Copyright 2012, Oracle and/or its affiliates. All rights reserved.

Concurrent Manager Troubleshooting Demonstration part2

Copyright 2012, Oracle and/or its affiliates. All rights reserved.

Only OPP & Workflow services Down

Copyright 2012, Oracle and/or its affiliates. All rights reserved.

Verify Service Manager Definition.

Copyright 2012, Oracle and/or its affiliates. All rights reserved.

DBMS_LOCK Error Internal Concurrent Manager Log

Copyright 2012, Oracle and/or its affiliates. All rights reserved.

Then Start the services adstrtal.sh apps/apps

Copyright 2012, Oracle and/or its affiliates. All rights reserved.

After CLONE Managers Down

Copyright 2012, Oracle and/or its affiliates. All rights reserved.

Copyright 2012, Oracle and/or its affiliates. All rights reserved.

Copyright 2012, Oracle and/or its affiliates. All rights reserved.

Managers down - Status "System Hold, Fix Manager"

Copyright 2012, Oracle and/or its affiliates. All rights reserved.

adrelink.sh force=y fnd FNDLIBR

adrelink.sh force=y fnd FNDCRM

Copyright 2012, Oracle and/or its affiliates. All rights reserved.

Managers down Target node/queue unavailable

Copyright 2012, Oracle and/or its affiliates. All rights reserved.

o Start the middle tier services including your concurrent manager.

Copyright 2012, Oracle and/or its affiliates. All rights reserved.

How to Enable diag=y

Edit adcmctl.sh to append diag=y

$FND_TOP/bin/startmgr sysmgr="$unpw@$DB_NAME" mgrname=$MGR_NAME diag=y

Start Concurrent Manager