HQ Agent Crash on AIX 5.3

Agent version 4.2.0
OS AIX 5.3

Agent has been running for 8 days, crashed today. javacore file attached.

Here is the output from the wrapper.

ERROR | wrapper | 2010/03/02 00:24:47 | JVM appears hung: Timed out waiting for signal from JVM.
ERROR | wrapper | 2010/03/02 00:24:47 | JVM did not exit on request, terminated
STATUS | wrapper | 2010/03/02 00:24:52 | Reloading Wrapper configuration...
STATUS | wrapper | 2010/03/02 00:24:53 | Launching a JVM...
INFO   | jvm 7    | 2010/03/02 00:25:18 | Wrapper (Version 3.2.3) http://wrapper.tanukisoftware.org
INFO   | jvm 7    | 2010/03/02 00:25:18 |   Copyright 1999-2006 Tanuki Software, Inc. All Rights Reserved.
INFO   | jvm 7    | 2010/03/02 00:25:18 |
INFO   | jvm 7    | 2010/03/02 00:28:18 | - Invoking agent
INFO   | jvm 7    | 2010/03/02 00:28:18 | - Agent thread running
INFO   | jvm 7    | 2010/03/02 00:28:18 | - Verifying if agent is running...
INFO   | jvm 7    | 2010/03/02 00:31:20 | Error invoking agent: Agent reported an error while starting up
WARN   | wrapper | 2010/03/02 00:35:05 | JVM process was still running after receiving a SIGCHLD signal.
STATUS | wrapper | 2010/03/02 00:35:07 | JVM requested a restart.
INFO   | jvm 7    | 2010/03/02 00:35:08 | Stopping agent ...
INFO   | jvm 7    | 2010/03/02 00:35:08 | Success -- agent is stopped!
STATUS | wrapper | 2010/03/02 00:35:14 | Reloading Wrapper configuration...
STATUS | wrapper | 2010/03/02 00:35:14 | Launching a JVM...
INFO   | jvm 8    | 2010/03/02 00:35:24 | Wrapper (Version 3.2.3) http://wrapper.tanukisoftware.org
INFO   | jvm 8    | 2010/03/02 00:35:24 |   Copyright 1999-2006 Tanuki Software, Inc. All Rights Reserved.
INFO   | jvm 8    | 2010/03/02 00:35:24 |
INFO   | jvm 8    | 2010/03/02 00:37:11 | - Invoking agent
INFO   | jvm 8    | 2010/03/02 00:37:11 | - Agent thread running
INFO   | jvm 8    | 2010/03/02 00:37:11 | - Verifying if agent is running...
INFO   | jvm 8    | 2010/03/02 00:40:09 | - Agent is running
INFO   | jvm 8    | 2010/03/02 00:40:13 | Agent successfully started
INFO   | jvm 8    | 2010/03/02 00:46:53 | Unhandled exception
INFO   | jvm 8    | 2010/03/02 00:46:53 | Type=Segmentation error vmState=0x00000000
INFO   | jvm 8    | 2010/03/02 00:46:53 | J9Generic_Signal_Number=00000004 Signal_Number=0000000b Error_Value=00000000 Signal_Code=00000033
INFO   | jvm 8    | 2010/03/02 00:46:53 | Handler1=F0B2C110 Handler2=F0B2614C
INFO   | jvm 8    | 2010/03/02 00:46:53 | R0=00000000 R1=31390AC0 R2=F0B2D6D0 R3=30F7AC4C
INFO   | jvm 8    | 2010/03/02 00:46:53 | R4=00000008 R5=E83C3810 R6=00000000 R7=00000000
INFO   | jvm 8    | 2010/03/02 00:46:53 | R8=00000000 R9=00000000 R10=00000000 R11=00000000
INFO   | jvm 8    | 2010/03/02 00:46:53 | R12=D1A7FA58 R13=308B7C00 R14=312F9C10 R15=F0B32774
INFO   | jvm 8    | 2010/03/02 00:46:53 | R16=00000003 R17=00000000 R18=F0B2DF00 R19=0000007B
INFO   | jvm 8    | 2010/03/02 00:46:53 | R20=302D4850 R21=312F9C24 R22=00000000 R23=302D46E8
INFO   | jvm 8    | 2010/03/02 00:46:53 | R24=302D46E8 R25=F0B1E1E0 R26=F0B2CC5C R27=30585240
INFO   | jvm 8    | 2010/03/02 00:46:53 | R28=00000000 R29=30F7AC50 R30=00000005 R31=3002B2C8
INFO   | jvm 8    | 2010/03/02 00:46:53 | IAR=D1A7FA90 LR=D1A7FA7C MSR=0000D032 CTR=D1A7EF30
INFO   | jvm 8    | 2010/03/02 00:46:53 | CR=48000288 FPSCR=82004000 XER=00000004 TID=00000000
INFO   | jvm 8    | 2010/03/02 00:46:53 | MQ=00000000
INFO   | jvm 8    | 2010/03/02 00:46:53 | FPR0 427271c56a507000 (f: 1783656448.000000, d: 1.267491e+12)
INFO   | jvm 8    | 2010/03/02 00:46:53 | FPR1 c1ec752b5f200000 (f: 1595932672.000000, d: -3.819527e+09)
INFO   | jvm 8    | 2010/03/02 00:46:53 | FPR2 433000001c56a507 (f: 475440384.000000, d: 4.503600e+15)
INFO   | jvm 8    | 2010/03/02 00:46:53 | FPR3 4530000080000127 (f: 2147483904.000000, d: 1.934282e+25)
INFO   | jvm 8    | 2010/03/02 00:46:53 | FPR4 4330000100000000 (f: 0.000000, d: 4.503604e+15)
INFO   | jvm 8    | 2010/03/02 00:46:53 | FPR5 4330000100000000 (f: 0.000000, d: 4.503604e+15)
INFO   | jvm 8    | 2010/03/02 00:46:53 | FPR6 0000000000000000 (f: 0.000000, d: 0.000000e+00)
INFO   | jvm 8    | 2010/03/02 00:46:53 | FPR7 0000000000000000 (f: 0.000000, d: 0.000000e+00)
INFO   | jvm 8    | 2010/03/02 00:46:53 | FPR8 0000000000000000 (f: 0.000000, d: 0.000000e+00)
INFO   | jvm 8    | 2010/03/02 00:46:53 | FPR9 0000000000000000 (f: 0.000000, d: 0.000000e+00)
INFO   | jvm 8    | 2010/03/02 00:46:53 | FPR10 0000000000000000 (f: 0.000000, d: 0.000000e+00)
INFO   | jvm 8    | 2010/03/02 00:46:53 | FPR11 7e37e43c8800759c (f: 2281731584.000000, d: 1.000000e+300)
INFO   | jvm 8    | 2010/03/02 00:46:53 | FPR12 c0025de41aa2d008 (f: 446877696.000000, d: -2.295845e+00)
INFO   | jvm 8    | 2010/03/02 00:46:53 | FPR13 7e37e43c8800759c (f: 2281731584.000000, d: 1.000000e+300)
INFO   | jvm 8    | 2010/03/02 00:46:53 | FPR14 0000000000000000 (f: 0.000000, d: 0.000000e+00)
INFO   | jvm 8    | 2010/03/02 00:46:53 | FPR15 0000000000000000 (f: 0.000000, d: 0.000000e+00)
INFO   | jvm 8    | 2010/03/02 00:46:53 | FPR16 0000000000000000 (f: 0.000000, d: 0.000000e+00)
INFO   | jvm 8    | 2010/03/02 00:46:53 | FPR17 0000000000000000 (f: 0.000000, d: 0.000000e+00)
INFO   | jvm 8    | 2010/03/02 00:46:53 | FPR18 0000000000000000 (f: 0.000000, d: 0.000000e+00)
INFO   | jvm 8    | 2010/03/02 00:46:53 | FPR19 0000000000000000 (f: 0.000000, d: 0.000000e+00)
INFO   | jvm 8    | 2010/03/02 00:46:53 | FPR20 0000000000000000 (f: 0.000000, d: 0.000000e+00)
INFO   | jvm 8    | 2010/03/02 00:46:53 | FPR21 0000000000000000 (f: 0.000000, d: 0.000000e+00)
INFO   | jvm 8    | 2010/03/02 00:46:53 | FPR22 0000000000000000 (f: 0.000000, d: 0.000000e+00)
INFO   | jvm 8    | 2010/03/02 00:46:53 | FPR23 0000000000000000 (f: 0.000000, d: 0.000000e+00)
INFO   | jvm 8    | 2010/03/02 00:46:53 | FPR24 0000000000000000 (f: 0.000000, d: 0.000000e+00)
INFO   | jvm 8    | 2010/03/02 00:46:53 | FPR25 0000000000000000 (f: 0.000000, d: 0.000000e+00)
INFO   | jvm 8    | 2010/03/02 00:46:53 | FPR26 0000000000000000 (f: 0.000000, d: 0.000000e+00)
INFO   | jvm 8    | 2010/03/02 00:46:53 | FPR27 0000000000000000 (f: 0.000000, d: 0.000000e+00)
INFO   | jvm 8    | 2010/03/02 00:46:53 | FPR28 0000000000000000 (f: 0.000000, d: 0.000000e+00)
INFO   | jvm 8    | 2010/03/02 00:46:53 | FPR29 0000000000000000 (f: 0.000000, d: 0.000000e+00)
INFO   | jvm 8    | 2010/03/02 00:46:53 | FPR30 0000000000000000 (f: 0.000000, d: 0.000000e+00)
INFO   | jvm 8    | 2010/03/02 00:46:53 | FPR31 0000000000000000 (f: 0.000000, d: 0.000000e+00)
INFO   | jvm 8    | 2010/03/02 00:46:53 | Module=/opt/home/hquser/hyperic-hq-agent-4.2.0/jre/bin/libj9vm23.so
INFO   | jvm 8    | 2010/03/02 00:46:53 | Module_base_address=D1A50000
INFO   | jvm 8    | 2010/03/02 00:46:53 | Target=2_30_20071004_14218_bHdSMR (AIX 5.3)
INFO   | jvm 8    | 2010/03/02 00:46:53 | CPU=ppc (2 logical CPUs) (0x1e4000000 RAM)
INFO   | jvm 8    | 2010/03/02 00:46:53 | JVMDUMP006I Processing Dump Event "gpf", detail "" - Please Wait.
INFO   | jvm 8    | 2010/03/02 00:46:53 | JVMDUMP007I JVM Requesting System Dump using '/opt/home/hquser/hyperic-hq-agent-4.2.0/wrapper/sbin/core.20100302.004653.782370.dm
p'
INFO   | jvm 8    | 2010/03/02 00:47:02 | JVMDUMP010I System Dump written to /opt/home/hquser/hyperic-hq-agent-4.2.0/wrapper/sbin/core.20100302.004653.782370.dmp
INFO   | jvm 8    | 2010/03/02 00:47:02 | JVMDUMP007I JVM Requesting Snap Dump using '/opt/home/hquser/hyperic-hq-agent-4.2.0/wrapper/sbin/Snap0002.20100302.004653.782370.
trc'
INFO   | jvm 8    | 2010/03/02 00:47:09 | JVMDUMP010I Snap Dump written to /opt/home/hquser/hyperic-hq-agent-4.2.0/wrapper/sbin/Snap0002.20100302.004653.782370.trc
INFO   | jvm 8    | 2010/03/02 00:47:09 | JVMDUMP007I JVM Requesting Java Dump using '/opt/home/hquser/hyperic-hq-agent-4.2.0/wrapper/sbin/javacore.20100302.004653.782370.
txt'
INFO   | jvm 8    | 2010/03/02 00:47:44 | JVMDUMP010I Java Dump written to /opt/home/hquser/hyperic-hq-agent-4.2.0/wrapper/sbin/javacore.20100302.004653.782370.txt
INFO   | jvm 8    | 2010/03/02 00:47:44 | JVMDUMP013I Processed Dump Event "gpf", detail "".
ERROR | wrapper | 2010/03/02 00:47:45 | JVM exited unexpectedly.
STATUS | wrapper | 2010/03/02 00:47:49 | Reloading Wrapper configuration...
STATUS | wrapper | 2010/03/02 00:47:49 | Launching a JVM...
INFO   | jvm 9    | 2010/03/02 00:47:55 | Wrapper (Version 3.2.3) http://wrapper.tanukisoftware.org
INFO   | jvm 9    | 2010/03/02 00:47:55 |   Copyright 1999-2006 Tanuki Software, Inc. All Rights Reserved.
INFO   | jvm 9    | 2010/03/02 00:47:55 |
INFO   | jvm 9    | 2010/03/02 00:49:25 | - Invoking agent
INFO   | jvm 9    | 2010/03/02 00:49:25 | - Agent thread running
INFO   | jvm 9    | 2010/03/02 00:49:25 | - Verifying if agent is running...
INFO   | jvm 9    | 2010/03/02 00:51:04 | - Agent is running
INFO   | jvm 9    | 2010/03/02 00:51:06 | Agent successfully started

Latest Images

Trending Articles

Latest Images