Using R12 adctrl utility for managing jobs

The Oracle E-Business Suite provides a comprehensive suite of DBA utilities that every Apps DBA needs to master to be an effective EBS Applications DBA. In this blog, I will cover the useful and often misunderstood utility called ADCTRL. The adctrl utility is a key utility in the toolkit of every veteran Oracle Applications DBA professional. We will use 12.0.6 release for R12 Oracle E-Business Suite.

ADCTRL is used to manage worker processes that perform a pivotal role in the function
of batch processsing for patching and many other Applications activities.

ADCTRL is located under $AD_TOP/bin directory with Oracle E-Business Suite.

$ cd $AD_TOP/bin

To begin a new adctrl session, you enter the adctrl at a command window prompt:

$ adctrl

Copyright (c) 2002 Oracle Corporation
Redwood Shores, California, USA

AD Controller

Version 12.0.0

NOTE: You may not use this utility for custom development
unless you have written permission from Oracle Corporation.

Your default directory is '/d1/bin/apps/apps_st/appl'.
Is this the correct APPL_TOP [Yes] ?

AD Controller records your AD Controller session in a text file
you specify. Enter your AD Controller log file name or press [Return]
to accept the default file name shown in brackets.

Filename [adctrl.log] :

************* Start of AD Controller session *************
AD Controller version: 12.0.0
AD Controller started at: Wed Sep 23 2009 22:25:12

APPL_TOP is set to /EBS121/bin/apps/apps_st/appl

Reading product information from file...

Reading language and territory information from file...

Reading language information from applUS.txt ...


Enter the ORACLE username of Application Object Library [APPLSYS] : apps

Enter the ORACLE password of Application Object Library [APPS] :

AD Controller is verifying your username/password.
The status of various features in this run of AD Controller is:

<-Feature version in->
Feature Active? APPLTOP Data model Flags
------------------------------ ------- -------- ----------- -----------
CHECKFILE No 1 -1 Y N N Y N N
PREREQ No 6 -1 Y N N Y N N
CONCURRENT_SESSIONS No 2 -1 Y Y N Y Y N
PATCH_TIMING No 2 -1 Y N N Y N N
PATCH_HIST_IN_DB No 6 -1 Y N N Y N N
SCHEMA_SWAP No 1 -1 Y N N Y Y N
JAVA_WORKER No 1 -1 Y N N Y N N
CODELEVEL No 1 -1 Y N N Y N N



AD Controller Menu
---------------------------------------------------

1. Show worker status

2. Tell worker to restart a failed job

3. Tell worker to quit

4. Tell manager that a worker failed its job

5. Tell manager that a worker acknowledges quit

6. Restart a worker on the current machine

7. Exit

After you have entered the APPLMGR username and password the above menu will open for ADCTRL.

Here we have 7 options.

You can start, stop and monitor the workers. Lets monitor status of workers with option 1:


Enter your choice [1] :


Control
Worker Code Context Filename Status
------ -------- ----------------- -------------------------- --------------
1 Run Generic R120 BOMFMCPY.pll Running
2 Run Generic R120 BOMFMMDE.pll Running
3 Run Generic R120 BOMFMOPR.pll Running
4 Run Generic R120 BOMFMREV.pll Running
5 Run Generic R120 BOMFMCON.pll Running
6 Run Generic R120 BOMFMBM2.pll Running
7 Run Generic R120 BOMFMBM3.pll Running
8 Run Generic R120 BOMFMBM1.pll Running

Undocumented Option 8:


This will allow you to continue processing a job and kill the adworker that failed.


Back at the main adctrl menu choose option 8:

Enter your choice [1] : 8

Enter the worker number(s)/range(s) or 'all' for all workers,
or press [Return] to go back to the menu :

To select the "Skip worker" option, type the number 8 and
press Return.

This can be useful in testing a patch and you need to let the patch or job run to completion.



ADWORKER LOGFILES for R12 EBS

How to find the log files for these workers? It is confusing to the newbie.

cd $APPL_TOP/admin/{ORACLE_SID}

Example:

apps@sandiego:/d1/bin/apps/apps_st/appl/admin
$ cd $APPL_TOP/admin/VIS/log

to find the adworker log files we do a ls command:

apps@sandiego:/d1/bin/apps/apps_st/appl/admin/VIS/log
$ ls -ls adw*.log
45871 -rw-r--r-- 1 apps dba 46934789 Sep 23 22:19 adwork001.log
25974 -rw-r--r-- 1 apps dba 26560214 Sep 23 22:19 adwork002.log
26037 -rw-r--r-- 1 apps dba 26637070 Sep 23 22:19 adwork003.log
25251 -rw-r--r-- 1 apps dba 25824196 Sep 23 22:18 adwork004.log
16379 -rw-r--r-- 1 apps dba 16750699 Sep 23 22:18 adwork005.log
17120 -rw-r--r-- 1 apps dba 17510037 Sep 23 22:18 adwork006.log
15612 -rw-r--r-- 1 apps dba 15965759 Sep 23 22:18 adwork007.log
16177 -rw-r--r-- 1 apps dba 16543829 Sep 23 22:18 adwork008.log

For each adworker process, a log file is generated that shows activity and errors if any. Lets examine one of these log files:


************* Start of AD Worker session *************
AD Worker version: 12.0.0
AD Worker started at: Fri Sep 18 2009 16:33:10

APPL_TOP is set to /d1/bin/apps/apps_st/appl

Worker id = 1
Started by: adpatch

NLS_LANG value from the environment is : AMERICAN_AMERICA.UTF8
NLS_LANG value for this AD utility run is : AMERICAN_AMERICA.UTF8

Connecting to SYSTEM......Connected successfully.

Already asked for the ORACLE username and password
of Application Object Library.
(The given username/password was: APPLSYS/*****)

Connecting to APPLSYS......Connected successfully.

The status of various features in this run of AD Worker is:

<-Feature version in->
Feature Active? APPLTOP Data model Flags
------------------------------ ------- -------- ----------- -----------
CHECKFILE Yes 1 1 Y N N Y N Y
PREREQ Yes 6 6 Y N N Y N Y
CONCURRENT_SESSIONS No 2 2 Y Y N Y Y N
PATCH_TIMING Yes 2 2 Y N N Y N Y
PATCH_HIST_IN_DB Yes 6 6 Y N N Y N Y
SCHEMA_SWAP Yes 1 1 Y N N Y Y Y
JAVA_WORKER Yes 1 1 Y N N Y N Y
CODELEVEL Yes 1 1 Y N N Y N Y


By understanding where to find the adworker log files you can troubleshoot E-Business issues quickly and become a better DBA.

Stay tuned as we cover more Apps DBA tips in future posts!

Cheers,
Ben

Tips and Tricks for Installing Oracle R12 E-Business

Dear readers,

I will present a brief overview of tips for monitoring the installation of an R12 E-Business environment for Oracle Applications. For our example, we installed a fresh new Vision instance of Oracle Applications (12.0.1) running under Oracle Enterprise Linux 5.2 (32 bit). During an installation, Oracle shows a basic splash screen:



Ok thats nice but still hides much of the actual work being performed under the covers. How do we troubleshoot the installation process? Oracle writes multiple log files during each step of the process for R12 to several directories:

Database Logfile ($ORACLE_HOME):
/d01/oracle/VIS/db/tech_st/10.2.0/appsutil/log

Apps Tier Logfile ($APPL_TOP):

/d01/oracle/VIS/inst/apps/VIS_bensr12/logs



R12 has several new changes from 11i Applications, one is the new instance directory
or INST_TOP directory.







We had a file access permissions problem during the install as reviewed by the output of the logfile:



Once we fixed it and granted correct permissions to it, we were able to complete the installation. Once the installation completes, R12 completes a series of post-install checks to verify all is correct. If anything fails, the Rapid Wizard installer throws an error with a red checkmark on items that have errors.



So, how do we fix this error? We have to bounce the apps tier and it solves the login page error. To do so, we use the scripts adstpall.sh and adstrtal.sh



Once the apps middle tier is back online, we can now access the login page for R12:



and once we sign in as sysadmin we are ready for training and testing with our new R12 Vision instance:



Stay tuned for upcoming tips with patching and concurrent manager tips with R12.

Fun and Games with OEL 5 Linux missing packages

Dear readers,

Recently as of yesterday I decided to do a new R12 EBS install this time with a Vision Demo instance to have a new EBS environment with fresh seed data to test our new features and for GRC (Govern Risk Control). I ran up2date oracle-validated to install the missing rpms and most of the R12 install went fine and took 9 hours!

However, after the install, I noticed that the listener and install did not succeed and found root cause was that some of the OEL Linux rpms failed to all download and install earlier. As shown below this is caused by kernel dependency problems:




Unfortunately in my enthusiasm and also due to the wee hours to get R12 setup, I did not realize my oversight. Then when R12 failed to start with listener errors and library missing errors and after search on Metalink, it dawned on me the issue was indeed these missing Redhat 5 packages. Oracle Enterprise Linux really is just Red Hat Linux kernel.

To resolve problems with the RPMs that failed to download and install correctly from the Oracle Unbreakable Linux Network (ULN), I had to run the up2date-nox --configure command and choose option 19 to bypass the kernel dependency warning.




We need to clear these warning messages out so that we can re-run up2date and download and install the missing Linux rpms.




And voila! Just like magic, we were able to download these key packages for Linux
so we can complete the R12 EBS installation.

Oracle 11g Release 2 Now Available

Oracle 11g Release 2 has been announced and available for download:

http://www.oracle.com/technology/software/products/database/index.html

As a beta tester for the Oracle 11gR2 team, I am excited and look forward to implementing many of the new features for 11gR2 with customers. Many new whitepapers are available at following sites:

http://www.oracledatabase11g.com/Authentication/Login_w.html?url=/?src=6811170&Act=28

I will be blogging on the many new features for 11gR2 in addition to Fusion Middleware and R12 EBS topics this year.

Cheers,
Ben