| Agent unreachable [message #284390] |
Thu, 29 November 2007 09:01  |
monto
Messages: 60
Registered: November 2007
Location: DALLAS
|
Member |
|
|
Hi,
I have installed agent 10g on unix machine(HPUX-11.1) agent runs for 6-8 hrs
after that it dies out and when i see the instance on that server through grid control it says agent unreachable , iam not sure whats happening here.
[oracle:leopard:NOSID] /oracle/product/agent10g/sysman/log > ps -ef|grep agen>
root 2661 1 0 May 20 ? 0:00 /etc/opt/resmon/lbin/emsagent
root 2617 1 0 May 20 ? 2:38 /usr/sbin/swagentd -r
oracle 23822 23690 0 Nov 20 ? 13:24 /oracle/product/agent10g/bin/emagent
oracle 23690 1 0 Nov 20 ? 2:54 /oracle/product/agent10g/perl/bin/perl /oracle/product/agent10g
aduaslq 16850 22944 0 Nov 23 ? 4:58 /amuaslq01s/app/amuaslqdb/10.2.0/bin/emagent---->agent is for OAM
[oracle:leopard:NOSID] /oracle/product/agent10g/bin > emctl status agent
Oracle Enterprise Manager 10g Release 3 Grid Control 10.2.0.3.0.
Copyright (c) 1996, 2007 Oracle Corporation. All rights reserved.
---------------------------------------------------------------
Agent is Not Running
[oracle:leopard:NOSID] /oracle/product/agent10g/bin > emctl start agent
Oracle Enterprise Manager 10g Release 3 Grid Control 10.2.0.3.0.
Copyright (c) 1996, 2007 Oracle Corporation. All rights reserved.
Starting agent ....... failed.
Failed to start HTTP listener.
Consult the log files in: /oracle/product/agent10g/sysman/log
[oracle:leopard:NOSID] /oracle/product/agent10g/bin > ps -ef|grep agent
root 2661 1 0 May 20 ? 0:00 /etc/opt/resmon/lbin/emsagent
root 2617 1 0 May 20 ? 2:38 /usr/sbin/swagentd -r
oracle 23822 23690 0 Nov 20 ? 13:23 /oracle/product/agent10g/bin/emagent
oracle 3913 15344 0 21:42:27 pts/7 0:00 grep agent
oracle 23690 1 0 Nov 20 ? 2:53 /oracle/product/agent10g/perl/bin/perl /oracle/product/agent10g
aduaslq 16850 22944 0 Nov 23 ? 4:57 /amuaslq01s/app/amuaslqdb/10.2.0/bin/emagent
/oracle/product/agent10g/sysman/log :vi emagent.trc
2007-11-27 17:48:44 Thread-189993 ERROR util.files: ERROR: nmeufos_new: failed i
n lfiopn on file: /oracle/product/agent10g/sysman/emd/agntstmp.txt.error = 24 (T
oo many open files)
2007-11-27 17:48:44 Thread-189993 ERROR pingManager: Error in updating the agent
time stamp file
2007-11-27 17:48:48 Thread-189994 ERROR util.fileops: ERROR: snmeuf_dirlist can'
t list directory: /oracle/product/agent10g/sysman/emd/upload: Too many open file
s (errno=24)
2007-11-27 17:48:51 Thread-189995 ERROR engine: Failed when generating a new ECI
D.
2007-11-27 17:48:51 Thread-189995 ERROR fetchlets.healthCheck: GIM-00104: file n
ot found
LEM-00031: file not found; arguments: [lempgmh] [lmserr]
LEM-00033: file not found; arguments: [lempgfm] [Couldn't open message file]
LEM-00031: file not found; arguments: [lempgmh] [lmserr]
2007-11-27 17:48:51 Thread-189995 ERROR engine: [oracle_database,leopard-amuaslq
.am,health_check] : nmeegd_GetMetricData failed : Instance Health Check initiali
zation failed due to one of the following causes: the owner of the EM agent proc
ess is not same as the owner of the Oracle instance processes; the owner of the
EM agent process is not part of the dba group; or the database version is not 10
g (10.1.0.2) and above.
The main error is this
2007-11-27 17:48:44 Thread-189993 ERROR util.files: ERROR: nmeufos_new: failed i
n lfiopn on file: /oracle/product/agent10g/sysman/emd/agntstmp.txt.error = 24 (T
oo many open files)
2007-11-27 17:48:44 Thread-189993 ERROR pingManager: Error in updating the agent
time stamp file
and after some gooling i found that It is giving too many files open error. It seems like kernel parameter settings for open files is low.I need to increase the kernel parameter maxfiles and current value is below but i have agent on another server which has exact same number of maxfiles(2048)runs
with no issues why is that this (server)has an issue with it,is it got to do with the number of instances running on the box or memory or different versions of oracle?
/etc/sysdef | grep maxfiles
maxfiles 2048 - 30-2048 -
maxfiles_lim 2048 - 30-2048 -
|
|
|
|
| Re: Agent unreachable [message #284440 is a reply to message #284390] |
Thu, 29 November 2007 12:23  |
DreamzZ
Messages: 1666
Registered: May 2007
Location: Dreamzland
|
Senior Member |
|
|
Check the permissions and ownership of sysman/log files
my box:/opt/oracle/10.2.0AGENT/agent10g/sysman/log$ ls -lt
total 42498
-rw-r----- 1 oracle dba 3145189 Nov 29 18:22 emagent.trc
-rw-r----- 1 oracle dba 5095273 Nov 29 18:15 emctl.log
-rw-r----- 1 oracle dba 4653920 Nov 28 23:39 emagent_perl.trc
-rw-r----- 1 oracle dba 4194524 Nov 28 09:45 emagent.trc.1
-rw-r----- 1 oracle dba 4194337 Nov 17 02:37 emagent.trc.2
-rw-r----- 1 oracle dba 937 Oct 26 20:29 nmcfsga.log
-rw-r----- 1 oracle dba 856 Oct 26 20:29 nmcfsga.err
-rw-r----- 1 oracle dba 8276 Oct 25 23:28 emagent.nohup
-rw-r----- 1 oracle dba 3299 Oct 24 23:28 emagent.log
-rw-r----- 1 oracle dba 2087 Oct 24 23:28 emdctl.trc
-rw-r----- 1 oracle dba 126 Oct 24 23:27 agabend.log
-rw-r----- 1 oracle dba 0 Sep 25 20:53 nfsPatchPlug.log
-rw-r----- 1 oracle dba 0 Sep 25 20:52 emagentfetchlet.trc
-rw-r----- 1 oracle dba 0 Sep 25 20:52 emagentfetchlet.log
-rw-r----- 1 oracle dba 1176 Sep 25 20:52 secure.log
-rw-r----- 1 oracle dba 0 Sep 25 20:51 emdctl.log
[Updated on: Thu, 29 November 2007 12:23] Report message to a moderator |
|
|
|
| Re: Agent unreachable [message #284460 is a reply to message #284390] |
Thu, 29 November 2007 14:38 |
monto
Messages: 60
Registered: November 2007
Location: DALLAS
|
Member |
|
|
Here is the ouput
-rw-r----- 1 oracle dba 1280 Nov 28 18:50 secure.log
-rw-rw---- 1 oracle dba 9244 Nov 28 18:51 emtgtctl.log
-rw-rw---- 1 oracle dba 79500 Nov 28 18:51 emtgtctl.trc
-rw-r----- 1 oracle dba 9244 Nov 28 18:51 emdctl.log
-rw-r----- 1 oracle dba 42 Nov 28 18:51 agabend.log
-rw-r----- 1 oracle dba 0 Nov 28 18:52 nfsPatchPlug.log
-rw-r----- 1 oracle dba 2719 Nov 28 18:54 emagent.log
-rw-r----- 1 oracle dba 19425 Nov 28 22:39 emagent_perl.trc
-rw-r----- 1 oracle dba 4194336 Nov 29 04:19 emagent.trc.3
-rw-r----- 1 oracle dba 4194452 Nov 29 11:04 emagent.trc.2
-rw-r----- 1 oracle dba 8691 Nov 29 16:56 emctl.log
-rw-r----- 1 oracle dba 4194426 Nov 29 17:48 emagent.trc.1
-rw-r----- 1 oracle dba 267666 Nov 29 20:35 emagentfetchlet.log
-rw-r----- 1 oracle dba 267666 Nov 29 20:35 emagentfetchlet.trc
-rw-r----- 1 oracle dba 148577 Nov 29 20:35 emagent.nohup
-rw-r----- 1 oracle dba 767467 Nov 29 20:35 emdctl.trc
-rw-r----- 1 oracle dba 1779313 Nov 29 20:35 emagent.trc
|
|
|
|
] 
Blog
Search
Help
Members
Register
Login
Home
