1. 현재 프로세스 상태 조회
=> 문제가 되는 프로세스의 정보 조회
kars2141:/kaprod> ps -ef | grep hth
eworks 655576 729320 0 10:15:35 pts/1 0:00 grep hth
eworks 802880 1 0 6월 12일 - 0:00 hth -I jeuservice_565484 -b 565484
=> 해당 계정이 실행한 프로세스의 정보조회
kars2141:/kaprod> ps -ef | grep eworks
eworks 639160 499778 0 10:15:28 pts/2 0:00 -ksh
eworks 655578 729320 1 10:15:43 pts/1 0:00 ps -ef
eworks 688174 729320 1 10:15:43 pts/1 0:00 grep eworks
eworks 729320 811046 0 10:15:18 pts/1 0:00 -ksh
eworks 802880 1 0 6월 12일 - 0:00 hth -I jeuservice_565484 -b 565484
2. truss 를 통한 프로세스 상태 조회
=> aix5L에 truss명령어가 생겼더군요...
-p 옵션에 pid를 주면 됩니다. :)
truss -e -f -o truss_test.txt -p 745480
---- -- --------------- --------
환경변수, fork, outputfile, pid
kars2141:/kaprod> truss -p 802880
truss: 0915-023 Cannot control process #802880
kars2141:/kaprod>
<- 일단 현재상황에서는 truss를 통해 프로세스를 제어하지 못함
3. dbx를 통한 프로세스 상태 조회
=> dbx를 통해 동일하게 상태를 조회할 수 있습니다.
-a의 값으로 pid를 주고 다음값으로 문제의 프로세스의 파일의 절대경로를 지정해줌
kars2141:/kaprod> dbx -a 802880 /kaprod/jeus33/WebtoB/bin/hth
Waiting to attach to process 802880 ...
^C
kars2141:/kaprod>
<- dbx로도 프로세스에 접근할 수 없는 상황임
4. 해당 프로세스를 강제로 종료해봅니다.
=> 해당 계정의 권한 뿐만 아니라 root의 권한으로 동일하게 실행해봄
kars2141:/kaprod> kill -SEGV 802880
kars2141:/kaprod> ps -ef | grep hth
eworks 802880 1 0 Jun 12 - 0:00 hth -I jeuservice_565484 -b 565484
root 852110 606268 1 10:57:16 pts/3 0:00 grep hth
kars2141:/kaprod> kill -9 802880
kars2141:/kaprod> ps -ef | grep hth
eworks 802880 1 0 Jun 12 - 0:00 hth -I jeuservice_565484 -b 565484
root 852112 606268 0 10:57:38 pts/3 0:00 grep hth
kars2141:/kaprod>
<- 역시 종료가 않되는 상태
5. kdb를 통한 시스템상태 조회
=> 데이타량이 많으므로 미리 script명령을 실행해서 로그를 남김.
kdb실행전에 문제의 프로세스의 pid를 확인해둠
kars2141:/kaprod> script wlog.txt
Script command is started. The file is wlog.txt.
kars2141:/kaprod>
=> kdb를 실행
kars2141:/kaprod> kdb
The specified kernel file is a 64-bit kernel
Preserving 945581 bytes of symbol table
First symbol __mulh
START END <name>
0000000000003500 0000000002403068 _system_configuration+000020
F00000002FF3A600 F00000002FFCF8C8 __ublock+000000
000000002FF22FF4 000000002FF22FF8 environ+000000
000000002FF22FF8 000000002FF22FFC errno+000000
F100008780000000 F100008790000000 pvproc+000000
F100008790000000 F100008794000000 pvthread+000000
F100000040000000 F10000004028D4E8 vmmdseg+000000
F1000013B0000000 F1000083B5400000 vmmswpft+000000
F100000BB0000000 F1000013B0000000 vmmswhat+000000
F100000050000000 F100000060000000 ptaseg+000000
F100000070000000 F1000000B0000000 ameseg+000000
F100009E10000000 F100009E20000000 KERN_heap+000000
F1000089C0000000 F1000089D0000000 MBUF_heap+000000
F100009C00000000 F100009C10000000 lkwseg+000000
PFT:
PVT:
id....................0008
raddr.....00000000013BC000 eaddr.....0000000000000000
size..............00400000 align.............00001000
valid..1 ros....0 holes..0 io.....0 seg....1 wimg...2
(0)>
<-- 일단의 정보가 보여지고 프로프트가 (0)> 의 형태로 변경됨
6. conv명령을 이용하여 문제프로세스의 pid를 변환(pid값과 변환된 pid값을 메모해둠)
(0)> conv 802880
Binary : 0000000000000000000000000000000000000000000011000100000001000000
Octal : 0000000000000003040100
Decimal: 802880
Hex : 00000000000C4040
(0)>
7. tpid 명령을 이용하여 해당 프로세스의 슬롯값을 조회합니다.(슬롯값을 메모해둠)
(0)> tpid 00000000000C4040
SLOT NAME STATE TID PRI RQ CPUID CL WCHAN
pvthread+028E80 1309 hth SLEEP A3A053 03C 0 65 KERN_heap+538A160 slist_table+000010
(0)>
<-- 여기서는 1309 입니다.
8. set명령을 통해 kdb의 상태를 확인하고 다시 scroll 속성값을 false로 줍니다.
=> set명령의 인자로 No값을 주면 토글형태로 속성값이 변경됨
(0)> set
No toggle name current value
1 no_symbol false
2 mst_wanted true
3 screen_size 24
4 power_pc_syntax true
5 origin 00000000
6 Unix symbols start from 3500
7 hexadecimal_wanted true
8 screen_previous false
9 display_stack_frames false
10 display_stacked_regs false
11 64_bit true
12 ldr_segs_wanted false
18 trace_back_lookup false
20 scroll true
21 edit
22 logfile
23 loglevel 2
(0)> set 20
scroll is false
(0)>
9. stat명령과 status 명령으로 시스템정보를 조회합니다.
(0)> stat
SYSTEM_CONFIGURATION:
CHRP_SMP_PCI POWER_PC POWER_4 machine with 2 cpu(s) (64-bit registers)
SYSTEM STATUS:
sysname... AIX
nodename.. kars2141
release... 1
version... 5
machine... 003830DA4C00
nid....... 3830DA4C
age of system: 5 day, 16 hr., 33 min., 45 sec.
xmalloc debug: disabled
(0)> status
CPU TID TSLOT PID PSLOT PROC_NAME
0 16B205D 2905 BA060 93 kdb_64
1 6007 3 6006 3 wait
(0)>
10. proc명령으로 해당 프로세스의 상태를 조회합니다.(proc에서 나온 pid값과 슬롯값 역시 메모)
=> proc * 이란 명령을 사용하여 모든 프로세스의 상태를 조회한 후
문제의 프로세스의 상태를 확인합니다.
(0)> proc *
SLOT NAME STATE PID PPID ADSPACE CL #THS
pvproc+000000 0 swapper ACTIVE 0000000 0000000 0000000020B70000 0 0001
pvproc+000400 1 init ACTIVE 0000001 0000000 0000000024B92000 65 0001
pvproc+000800 2 wait ACTIVE 0004004 0000000 0000000024B72000 0 0001
pvproc+000C00 3 wait ACTIVE 0006006 0000000 0000000028B74000 0 0001
pvproc+001000 4 reaper ACTIVE 0008008 0000000 000000002CB76000 65 0001
pvproc+001400 5 lrud ACTIVE 000A00A 0000000 0000000030B78000 65 0001
pvproc+001800 6 xmgc ACTIVE 000C00C 0000000 0000000012B89000 65 0001
pvproc+001C00 7 netm ACTIVE 000E00E 0000000 0000000016B8B000 65 0001
pvproc+002000 8 gil ACTIVE 0010010 0000000 000000001AB8D000 65 0005
pvproc+002400 9 wlmsched ACTIVE 0012012 0000000 0000000020B90000 65 0001
pvproc+003000 12 syncd ACTIVE 0018044 0000001 0000000004C62400 65 0001
pvproc+003400 13 errdemon ACTIVE 001A060 0000001 0000000022C71400 65 0001
pvproc+003800 14 srcmstr ACTIVE 001C078 0000001 0000000038C7C400 65 0001
pvproc+003C00 15 j2pg ACTIVE 001E024 0000000 000000002CBF6000 65 000A
pvproc+004400 17 sendmail ACTIVE 0022062 001C078 0000000026CB3400 65 0001
pvproc+004800 18 dog ACTIVE 0024030 0000000 000000002AC35000 65 0002
pvproc+005000 20 IBM.CSMA ACTIVE 0028088 001C078 000000002EDD7400 65 000D
pvproc+005400 21 jfsz ACTIVE 002A078 0000000 0000000016C8B000 65 0001
pvproc+005800 22 lvmbb ACTIVE 002C03C 0000000 0000000010BE8000 65 0001
pvproc+006000 24 syslogd ACTIVE 0030068 001C078 000000000ACA5400 65 0001
pvproc+006400 25 portmap ACTIVE 0032038 001C078 0000000034CBA400 65 0001
pvproc+006800 26 shlap64 ACTIVE 0034052 0000001 0000000020C70400 65 0001
pvproc+006C00 27 inetd ACTIVE 0036036 001C078 0000000024CB2400 65 0001
pvproc+007000 28 xntpd ACTIVE 0038038 001C078 0000000032CB9400 65 0001
pvproc+007400 29 snmpd64 ACTIVE 003A03A 001C078 0000000012CC9400 65 0001
pvproc+007800 30 dpid2 ACTIVE 003C03C 001C078 0000000038CDC400 65 0001
pvproc+007C00 31 hostmibd ACTIVE 003E03E 001C078 000000000ACE5400 65 0001
pvproc+008000 32 rtcmd ACTIVE 0040042 0000000 0000000014CEA000 65 0001
pvproc+008400 33 kbiod ACTIVE 0042042 0000000 000000001CCEE000 65 0002
pvproc+009000 36 rpc.stat ACTIVE 0048050 001C078 0000000000D00400 65 0003
pvproc+009400 37 biod ACTIVE 004A04C 001C078 0000000010CE8400 65 0001
pvproc+009800 38 rpc.lock ACTIVE 004C050 001C078 000000000CD06400 65 0001
pvproc+009C00 39 rpc.lock ACTIVE 004E04E 0000001 0000000016D0B000 65 000A
pvproc+00A000 40 cron ACTIVE 0050050 0000001 000000002ECF7400 65 0001
pvproc+00A400 41 uprintfd ACTIVE 0052058 0000001 000000001AD0D400 65 0001
pvproc+00A800 42 writesrv ACTIVE 005405A 001C078 000000002ED17400 65 0001
pvproc+00AC00 43 qdaemon ACTIVE 0056058 001C078 0000000024D12400 65 0001
pvproc+00BC00 47 httpdlit ACTIVE 005E060 0000001 000000001AD2D400 65 0001
pvproc+00C400 49 diagd ACTIVE 0062064 0000001 0000000010D28400 65 0001
pvproc+00CC00 51 ksh ACTIVE 006606A 0000001 0000000036D3B400 65 0001
pvproc+00D000 52 rmcd ACTIVE 0068088 001C078 000000003CD7E400 65 0003
pvproc+00D400 53 itesmdem ACTIVE 006A06E 0000001 0000000004D62400 65 0001
pvproc+00D800 54 ctcasd ACTIVE 006C08E 001C078 0000000014D2A400 65 0007
pvproc+00DC00 55 IBM.Serv ACTIVE 006E09A 001C078 000000002CE16400 65 0007
pvproc+00EC00 59 dtlogin ACTIVE 0076088 001C078 0000000024D92400 65 0001
pvproc+00F800 62 sadc ACTIVE 007C0FC 00D006C 000000002AEB5400 65 0001
pvproc+00FC00 63 java ACTIVE 007E0F0 0000001 000000003B7BD400 65 008B
pvproc+010000 64 IBM.ERrm ACTIVE 008008A 001C078 0000000010E68400 65 0008
pvproc+010400 65 IBM.Audi ACTIVE 0082082 001C078 000000002AE15400 65 000A
pvproc+010C00 67 java ACTIVE 0086042 0000001 000000001B7AD400 65 008B
pvproc+011C00 71 sso_engi ACTIVE 008E090 0000001 0000000026A32400 65 0011
pvproc+012800 74 ksh ACTIVE 009403C 00CA036 000000000D047400 65 0001
pvproc+013C00 79 java ACTIVE 009E024 0000001 000000003B73D400 65 005B
pvproc+014000 80 script ACTIVE 00A002E 00A8082 000000000AB04400 65 0001
pvproc+015000 84 ksh ACTIVE 00A8082 00B20EA 000000003D03F400 65 0001
pvproc+016400 89 ksh ACTIVE 00B20EA 00C6028 0000000010AC9400 65 0001
pvproc+017400 93*kdb_64 ACTIVE 00BA060 009403C 0000000032AD8400 65 0001
pvproc+018800 98 hth ACTIVE 00C4040 0000001 00000000057A2400 65 0001
pvproc+018C00 99 telnetd ACTIVE 00C6028 0036036 0000000005702400 65 0001
pvproc+019400 101 script ACTIVE 00CA036 00A002E 0000000005043400 65 0001
pvproc+01A000 104 bsh ACTIVE 00D006C 0000001 0000000007042400 65 0001
pvproc+002800 10 NONE 0014016 0000001 0000000000000000 65 0000
pvproc+002C00 11 NONE 0016062 0000001 0000000000000000 65 0000
pvproc+004000 16 NONE 00200B4 00AA02E 0000000000000000 65 0000
pvproc+004C00 19 NONE 0026034 0000001 0000000000000000 65 0000
pvproc+005C00 23 NONE 002E052 0000001 0000000000000000 65 0000
pvproc+008800 34 NONE 0044028 0000001 0000000000000000 65 0000
pvproc+008C00 35 NONE 0046016 0000001 0000000000000000 65 0000
pvproc+00B000 44 NONE 0058060 0000001 0000000000000000 65 0000
pvproc+00B400 45 NONE 005A060 0000001 0000000000000000 65 0000
pvproc+00B800 46 NONE 005C0D2 00B20C0 0000000000000000 65 0000
pvproc+00C000 48 NONE 0060060 0000001 0000000000000000 65 0000
pvproc+00C800 50 NONE 0064000 00D203A 0000000000000000 65 0000
pvproc+00E000 56 NONE 00700C8 00AA02E 0000000000000000 65 0000
pvproc+00E400 57 NONE 0072034 00AA02E 0000000000000000 65 0000
pvproc+00E800 58 NONE 00740FE 0000001 0000000000000000 65 0000
pvproc+00F000 60 <zombie> ZOMB 00780B2 0076088 000000FFFFFFF000 65 0001
pvproc+00F400 61 NONE 007A042 0036036 0000000000000000 65 0000
pvproc+010800 66 NONE 008403A 00AA02E 0000000000000000 65 0000
pvproc+011000 68 NONE 00880B4 00AA02E 0000000000000000 65 0000
pvproc+011400 69 NONE 008A0EC 005C0D2 0000000000000000 65 0000
pvproc+011800 70 NONE 008C0B2 00880AA 0000000000000000 65 0000
pvproc+012000 72 <zombie> ZOMB 0090094 0076088 000000FFFFFFF000 65 0001
pvproc+012400 73 NONE 00920FA 00AA02E 0000000000000000 65 0000
pvproc+012C00 75 NONE 009606E 00AA02E 0000000000000000 65 0000
pvproc+013000 76 NONE 00980F0 0000001 0000000000000000 65 0000
pvproc+013400 77 NONE 009A05C 0000001 0000000000000000 65 0000
pvproc+013800 78 NONE 009C0B8 007A042 0000000000000000 65 0000
pvproc+014400 81 NONE 00A2062 0000001 0000000000000000 65 0000
pvproc+014800 82 NONE 00A4076 0000001 0000000000000000 65 0000
pvproc+014C00 83 NONE 00A600E 0000001 0000000000000000 65 0000
pvproc+015400 85 NONE 00AA08A 00A6008 0000000000000000 65 0000
pvproc+015800 86 NONE 00AC02E 00C2074 0000000000000000 65 0000
pvproc+015C00 87 NONE 00AE080 0000001 0000000000000000 65 0000
pvproc+016000 88 NONE 00B0030 0000001 0000000000000000 65 0000
pvproc+016800 90 NONE 00B400A 005C0D2 0000000000000000 65 0000
pvproc+016C00 91 NONE 00B602E 0000001 0000000000000000 65 0000
pvproc+017000 92 NONE 00B8036 00C2074 0000000000000000 65 0000
pvproc+017800 94 NONE 00BC05E 0000001 0000000000000000 65 0000
pvproc+017C00 95 NONE 00BE0A0 0000001 0000000000000000 65 0000
pvproc+018000 96 NONE 00C0094 0000001 0000000000000000 65 0000
pvproc+018400 97 NONE 00C20E2 0000001 0000000000000000 65 0000
pvproc+019000 100 NONE 00C8046 00A40D2 0000000000000000 65 0000
pvproc+019800 102 NONE 00CC070 00640EE 0000000000000000 65 0000
pvproc+019C00 103 NONE 00CE020 0000001 0000000000000000 65 0000
pvproc+01A400 105 NONE 00D203A 0050050 0000000000000000 65 0000
pvproc+01A800 106 NONE 00D40F6 00D80DC 0000000000000000 65 0000
pvproc+01AC00 107 NONE 00D60EA 0000001 0000000000000000 65 0000
pvproc+01B000 108 NONE 00D80EC 00C60F0 0000000000000000 65 0000
(0)>
<-- 문제가 된 hth프로세스는
pvproc+018800 98 hth ACTIVE 00C4040 0000001 00000000057A2400 65 0001
처럼 active상태로 정상처럼 보임
11. proc명령으로 문제 프로세스만의 상태를 조회(메모해둔 slot값을 인자로 사용)
(0)> proc 98
SLOT NAME STATE PID PPID ADSPACE CL #THS
pvproc+018800 98 hth ACTIVE 00C4040 0000001 00000000057A2400 65 0001
NAME....... hth
STATE...... stat :07 .... xstat :0000
FLAGS...... flag :00200001 LOAD EXECED
........... flag2 :00000000
........... atomic :00040000 ORPHANPGRP
LINKS...... child :0000000000000000
........... siblings :F10000878000CC00 <pvproc+00CC00>
........... uidinfo :F100009E2C3CF640
........... ganchor :0000000000000000
THREAD..... threadlist :F100008790028E80 <pvthread+028E80>
DISPATCH... synch :FFFFFFFFFFFFFFFF
WLM........ class/wlm :41/0000
IDENTIFIER. uid :000000C9 ........... suid :000000C9
........... pid :000C4040 ........... ppid :00000001
........... sid :0005C0D2 ........... pgrp :0008A0EC
MISC...... lock @ F100008780018930 0000000000000000
.......... lock_d @ F1000087800189E8 0000000000000000
..... parent_lock @ F1000087800189E0 0000000000000000
..... session_lock @ F1000087800189D8 0000000000000000
........... pgrpl :0000000000000000
........... pgrpb :0000000000000000 ... ttyl :0000000000000000
........... ipc :0000000000000000
........... dblist :0000000000000000 ... dbnext :0000000000000000
STATISTICS. nframes :0000000000000016 ... npsblks :0000000000000000
........... nvpages :0000000000000016 ... auditmask :00000000
SCHEDULER.. sched_next :0000000000000000 ... sched_back :0000000000000000
........... mempools @ F100008780018A00 8000000000000000
......... usched_lock @ F100008780018950 0000000000000000
........... uschedp :0000000000000000
........... asyncio :0000000000000000
CHECKPOINT. crid :00000000 ........... crid_token :FFFFFFFF
........... cridnext :0000000000000000 ... chksynch :FFFFFFFF
........... vpid :00000000 ........... vppid :00000000
........... vsid :00000000 ........... vpgrp :00000000
PROCFS..... procfsvn :0000000000000000
NUMA....... rset :0000000000000000
PROC....... procp :F100009E2C75D400 ... size :000001C0
FLAGS...... flag :00000000
........... flag2 :00000000
........... int :00000000
........... atomic:00000000
THREAD..... threadcount:00000001 ........... active :00000001
........... suspended :00000000 ........... terminating:00000000
........... local :00000000
SCHEDULE... nice : 60 ........... sched_pri : 255
DISPATCH... pevent :0000000000000000
IDENTIFIER. pid :000C4040
MISC....... adspace :00000000057A2400
........... adtable :FFFFFFFFFFFFFFFF ... adspace_ldr:000000FFFFFFF000
SIGNAL..... infoq :0000000000000000
........... pending :[3] 0000000000000000
........................[2] 0000000000000000
........................[1] 0000000000000000
........................[0] 0000000000000110 TRAP KILL
........... sigignore :[3] 0000000000000000
........................[2] 0000000000000000
........................[1] 0000000000000000
........................[0] 0400000000000000 CPUFAIL
........... sigcatch :[3] 0000000000000000
........................[2] 0000000000000000
........................[1] 0000000000000000
........................[0] 00000000F9FEF63F HUP INT QUIT ILL TRAP ABRT BUS
........................ SEGV PIPE ALRM TERM URG TSTP CONT
........................ CHLD TTIN TTOU IO XCPU XFSZ WINCH
........................ PWR USR1 USR2 PROF
........... siginfo :[3] 0000000000000000
........................[2] 0000000000000000
........................[1] 0000000000000000
........................[0] 0000000000000000
STATISTICS. page size :00000000000004AB ... minflt :0000000000000483
........... majflt :0000000000000000 ... pctcpu :00000000
SCHEDULER.. repage :0000000000000000 ... sched_count:00000000
........... cpticks :0000.... ........... msgcnt :0000
........... majfltsec :00000000
........... rs_attinfo :0000000000000000 ........... sradassign :0000
........... rs_rss :0000000000000476 ........... boundcount :0000
CHECKPOINT. chkblock :00000000 ........... chkfile :0000000000000000
PROCFS..... prtrcset :F100009E2C462500
PVPROC..... pvprocp :F100008780018800 ... size :00000400
(0)>
12. th명령으로 쓰레드상태를 조회(tpid에서 나온 슬롯값을 인자로 사용)
(0)> th 1309
SLOT NAME STATE TID PRI RQ CPUID CL WCHAN
pvthread+028E80 1309 hth SLEEP A3A053 03C 0 65 KERN_heap+538A160 slist_table+000010
NAME................ hth
WTYPE............... WLOCK
.................tid :0000000000A3A053 ......tsleep :FFFFFFFFFFFFFFFF
...............flags :00000000 ..............flags2 :00000000
DATA.........pvprocp :F100008780018800 <pvproc+018800>
LINKS.....prevthread :F100008790028E80 <pvthread+028E80>
..........nextthread :F100008790028E80 <pvthread+028E80>
DISPATCH.......synch :FFFFFFFFFFFFFFFF
SCHEDULER...affinity :00000000 .................pri :0000003C
.............boosted :00000000 ...............wchan :0000000000000000
...............state :00000003 ...............wtype :00000002
CHECKPOINT......vtid :00000000 .............chkfile :0000000000000000
LOCK........ lock_d @ F100008790028E90 0000000000000000
PROCFS......procfsvn :0000000000000000
THREAD.......threadp :F100009E2C3CDC00 ........size :00000080
FLAGS............... SUSP SIGAVAIL PROCFS SIGSLIH SIGINTR
.................tid :0000000000A3A053 ......stackp :000000002FF22470
.................scp :0000000000000000 .......ulock :0000000000000000
...............uchan :0000000000000000 ....userdata :0000000000000000
..................cv :0000000000000000 .......flags :0000000000000086
..............atomic :0000000000000000 ......flags2 :0000000000000000
DATA...........procp :F100009E2C75D400 <F100009E2C75D400>
...........pvthreadp :F100008790028E80 <pvthread+028E80>
...............userp :F00000002FF3B000 <__ublock+000A00>
............uthreadp :F00000002FF3A600 <__ublock+000000>
SLEEP/LOCK......usid :0000000000000000 ......wchan1 :F100009E1538A160
..............wchan2 :0000000000000000 ......swchan :000000000023C3C0
...........eventlist :0000000000000000 ......result :00000001
.............polevel :000014BB ..............pevent :0000000000000000
..............wevent :0000000000000000 .......slist :F100009E2C77AA00
...........wchan1sid :000000000003C9DE wchan1offset :0538A160
...........lockcount :00000001 ..........adsp_flags :0000
DISPATCH.......ticks :00000000 ...............prior :F100009E2C786C00
................next :F100009E2C55B600 ......dispct :00000000000004D0
...............fpuct :0000000000000000
MISC........graphics :0000000000000000 ...pmcontext :0000000000000000
...........lockowner :0000000000000000 ..kthreadseg :000000FFFFFFF000
..........time_start :0000000000000000 ..........wlm_charge :0
SIGNAL........sigproc:00000000 ..............cursig :00000000
......(pending) sig :[3] 0000000000000000
......................[2] 0000000000000000
......................[1] 0000000000000000
......................[0] 0000000000000000
............sigmask :[3] 0000000000000000
......................[2] 0000000000000000
......................[1] 0000000000000000
......................[0] 0000000000000000
SCHEDULER......cpuid :FFFFFFFF ..............scpuid :FFFFFFFF
.........affinity_ts :00ECB862 ..............policy :00000000
.................cpu :00000000 .............lockpri :0000003C
.............wakepri :000000FF ................time :000000FF
.............sav_pri :0000003C ...........run_queue :F100009E1404E000
................cpu2 :0000003A
.............suspend :00000001 .............fsflags :00000000
..........norun_secs :00000000
CHECKPOINT..chkerror :0000 ............chkblock :00000000
PROCFS.......whystop :00000001 ............whatstop :00000000
............pri_band :00 ........allowed_cpus :C000000000000000
.......prefunnel_cpu :00000000
......threadcontrolp :0000000000000000
PVTHREAD...pvthreadp :F100008790028E80 ........size :00000080
(0)>
13. f명령을 이용하여 스택정보를 조회(인자로 역시 tpid명령의 슬롯값을 사용)
(0)> f 1309
pvthread+028E80 STACK:
[00337370]slock+0003D8 (0000000000009910, 80000000000090B2 [??])
[00009558].simple_lock+000058 ()
[00110734]uipc_usrreq+0001F0 (??, ??, ??, ??, ??)
[00121200]soclose2+0005F8 (??, ??)
[00126D44]soclose+000010 (??)
[00126BEC]soo_close+0000C8 (??)
[004AE068]closef+000078 (??)
[004530A0]closefd+0000A0 (??, ??)
[00453334]close+000230 (??)
[0000379C]sc_msr_2_point+000028 ()
(0)>
14. set명령을 이용해서 display_stack_frames, display_stacked_regs,
trace_back_lookup 속성을 true로 변경
(0)> set
No toggle name current value
1 no_symbol false
2 mst_wanted true
3 screen_size 24
4 power_pc_syntax true
5 origin 00000000
6 Unix symbols start from 3500
7 hexadecimal_wanted true
8 screen_previous false
9 display_stack_frames false
10 display_stacked_regs false
11 64_bit true
12 ldr_segs_wanted false
18 trace_back_lookup false
20 scroll false
21 edit
22 logfile
23 loglevel 2
(0)> set 9
display_stack_frames is true
(0)> set 10
display_stacked_regs is true
(0)> set 18
trace_back_lookup is true
(0)> set
No toggle name current value
1 no_symbol false
2 mst_wanted true
3 screen_size 24
4 power_pc_syntax true
5 origin 00000000
6 Unix symbols start from 3500
7 hexadecimal_wanted true
8 screen_previous false
9 display_stack_frames true
10 display_stacked_regs true
11 64_bit true
12 ldr_segs_wanted false
18 trace_back_lookup true
20 scroll false
21 edit
22 logfile
23 loglevel 2
(0)>
15. 다시 f명령을 사용해서 스택정보를 조회합니다.
(0)> f 1309
pvthread+028E80 STACK:
[00337370]slock+0003D8 (0000000000009910, 80000000000090B2 [??])
r31 : 0000000000000000 r30 : F1000089C09C6D00 r29 : 0000000000000000
r28 : 0000000000000000 r27 : F1000089C0902C00 r26 : 0000000000000000
r25 : F1000089C0902C00 r24 : 0000000000000000 r23 : 0000000000000000
r22 : F100009D00011FF0 r21 : 0000000020015AC0 r20 : 00000000200155E0
r19 : 0000000000000000 r18 : 0000000000000000 r17 : 000000000000000C
r16 : 0000000000000008 r15 : 0000000000000000 r14 : 0000000000000010
=======================================================================
F00000002FF3A080: F000 0000 2FF3 A0F0 0000 0000 0000 0000 ..../...........
F00000002FF3A090: 0000 0000 0000 955C 0000 0000 0000 000C .......\........
F00000002FF3A0A0: 0000 0000 0000 0000 0000 0000 0059 9000 .............Y..
F00000002FF3A0B0: 0000 0000 2001 55E0 0000 0000 2001 5AC0 .... .U..... .Z.
=======================================================================
[00009558].simple_lock+000058 ()
=======================================================================
F00000002FF3A0F0: F000 0000 2FF3 A1B0 F100 0089 C028 F240 ..../........(.@
F00000002FF3A100: 0000 0000 0011 0738 F100 0089 C00A 5800 .......8......X.
F00000002FF3A110: F000 0000 2FF3 A1B0 2022 0124 C00A 5800 ..../... ".$..X.
F00000002FF3A120: 0000 0000 000E F54C 0000 0000 0000 0001 .......L........
=======================================================================
[00110734]uipc_usrreq+0001F0 (??, ??, ??, ??, ??)
r31 : F1000089C0902C00 r30 : 00000000000034C8 r29 : 0000000000000001
r28 : 0000000000000000 r27 : 0000000000000000 r26 : F1000089C028F240
r25 : 0000000000000000 r24 : 0000000000000000 r23 : 800000102520002C
=======================================================================
F00000002FF3A1B0: F000 0000 2FF3 A290 8442 2120 2C40 4240 ..../....B! ,@B@
F00000002FF3A1C0: 0000 0000 0012 1204 0000 0000 0000 0000 ................
F00000002FF3A1D0: 0000 0000 0000 0010 0000 0000 0059 9000 .............Y..
F00000002FF3A1E0: 0000 0000 0000 0008 0000 0000 0000 000C ................
=======================================================================
[00121200]soclose2+0005F8 (??, ??)
r31 : F100009D00011FF0 r30 : 0000000000000000 r29 : 0000000000000000
r28 : 00000000000034C8 r27 : F100009D00011FF0 r26 : F1000089C0902C00
r25 : 0000000000000060 r24 : 0000000000000006 r23 : 0000000000000000
=======================================================================
F00000002FF3A290: F000 0000 2FF3 A300 2422 2224 2FF3 A600 ..../...$""$/...
F00000002FF3A2A0: 0000 0000 0012 6D48 0000 0000 0000 0000 ......mH........
F00000002FF3A2B0: 0000 0000 0000 0008 0000 0000 0000 000C ................
F00000002FF3A2C0: 0000 0000 0000 0000 0000 0000 0000 0000 ................
=======================================================================
[00126D44]soclose+000010 (??)
=======================================================================
F00000002FF3A300: F000 0000 2FF3 A3A0 F100 009D 0001 E020 ..../..........
F00000002FF3A310: 0000 0000 0012 6BF0 0000 0000 0000 0000 ......k.........
F00000002FF3A320: 0000 0000 0000 0001 F100 009D 0001 E020 ...............
F00000002FF3A330: F000 0000 2FF3 A3A0 0000 0000 0000 0001 ..../...........
=======================================================================
[00126BEC]soo_close+0000C8 (??)
r31 : F100009D00011FF0 r30 : 0000000000000000 r29 : 0000000000000000
r28 : F00000002FF3A660 r27 : F100009D00011FF0 r26 : 0000000000000006
=======================================================================
F00000002FF3A3A0: F000 0000 2FF3 A430 F000 0000 2FF3 A490 ..../..0..../...
F00000002FF3A3B0: 0000 0000 004A E06C 0000 0000 0000 0005 .....J.l........
F00000002FF3A3C0: 0000 0010 0000 0000 0000 0000 0059 9000 .............Y..
F00000002FF3A3D0: 0000 0000 0000 0000 0000 0000 0000 0000 ................
=======================================================================
[004AE068]closef+000078 (??)
r31 : F00000002FF3A600 r30 : 0000000000000001 r29 : 0000000000000000
=======================================================================
F00000002FF3A430: F000 0000 2FF3 A4D0 0000 0000 0000 0004 ..../...........
F00000002FF3A440: 0000 0000 0045 30A4 2FF2 2618 0000 0001 .....E0./.&.....
F00000002FF3A450: 0000 0000 0053 629C 0000 0000 0059 9000 .....Sb......Y..
F00000002FF3A460: F000 0000 2FF3 A410 0000 0000 0000 0000 ..../...........
=======================================================================
[004530A0]closefd+0000A0 (??, ??)
r31 : 0000000000000000 r30 : 0000000000000006 r29 : 0000000000000006
r28 : 0000000000000000 r27 : F00000002FF3A600 r26 : 0000000000000000
=======================================================================
F00000002FF3A4D0: F000 0000 2FF3 A590 2222 2224 0000 0005 ..../..."""$....
F00000002FF3A4E0: 0000 0000 0045 3338 0000 0000 0000 0028 .....E38.......(
F00000002FF3A4F0: 0000 0000 0000 0000 0000 0000 0059 9000 .............Y..
F00000002FF3A500: 0000 0000 2FF2 2FF8 0000 0000 0000 0000 ...././.........
=======================================================================
[00453334]close+000230 (??)
r31 : 0000000000000000 r30 : 000000001000F30C r29 : 0000000000000007
r28 : 0000000020014B28 r27 : 0000000020015600 r26 : 000000000000016C
r25 : 0000000020000460 r24 : 0000000020015608 r23 : 0000000000000000
r22 : 0000000020015630
=======================================================================
F00000002FF3A590: 0000 0000 0000 0000 2222 2224 0000 0000 ........"""$....
F00000002FF3A5A0: 0000 0000 0000 37A0 0000 0000 0000 0000 ......7.........
F00000002FF3A5B0: 0000 0000 0000 0000 0000 0000 0000 0000 ................
F00000002FF3A5C0: 0000 0000 2FF2 2420 0000 0000 2000 49B8 ..../.$ .... .I.
=======================================================================
[0000379C]sc_msr_2_point+000028 ()
=======================================================================
2FF22470: 2FF2 25D0 0000 0000 1000 F230 0000 0004 /.%........0....
2FF22480: 0000 07E0 2000 A068 2045 6420 F008 D720 .... ..h Ed ...
2FF22490: 2FF2 2570 F008 6E00 0000 0000 0000 0000 /.%p..n.........
2FF224A0: 0000 0000 F008 6E00 3231 3431 0043 3030 ......n.2141.C00
=======================================================================
(0)>
16. kdb상태에서 netstat 정보를 조회합니다.
(0)> netstat
Active Internet connections
Proto Recv-Q Send-Q Local Address Foreign Address (state)
tcp4 0 0 172.80.177.102.37974 172.80.177.101.9900 ESTABLISHED
tcp4 0 0 172.80.177.102.37975 172.80.177.101.9900 ESTABLISHED
tcp4 0 0 172.80.177.102.37976 172.80.177.101.9900 ESTABLISHED
tcp4 0 0 172.80.177.102.37977 172.80.177.101.9900 ESTABLISHED
tcp4 0 0 172.80.177.102.37979 172.80.177.101.9900 ESTABLISHED
.
.
.
F1000089C0A5C000 stream 0 0 0 0 0 0
F1000089C0227000
(0)>
17. ipc정보 조회
(0)> ipc
<-- 먼저 ipc라고 실행후
IPC info
Select the display:
1) Message Queues
2) Shared Memory
3) Semaphores
<-- SharedMemory문제이므로 2번
Enter your choice: 2
1) all shmid_ds
2) select shmid_ds
<-- 모든 sharedmemory 를 조회
Enter your choice: 1
In kdb_pr_shmid_ds_all, shm_mark = 14
Shared memory ID 00000000 @ F10000AF50000000
uid............. 00000000 gid............. 00000000
cuid............ 00000000 cgid............ 00000000
mode............ 000081B6 seq............. 0000
key............. 000000000D01A845
.
.
.
vhat_next....... 0000000000000000
vhat_prev....... 0000000000000000
(0)>
18. sock명령으로 소켓정보를 조회합니다.
(0)> sock
---- TCP ----(inpcb: @ F1000089C027FA50)----
---- SOCKET INFO ----(@ F1000089C027F800)----
type........ 0001 (STREAM)
opts........ 0006 (ACCEPTCONN|REUSEADDR)
linger...... 0000 state....... 0080 (PRIV)
pcb..... F1000089C027FA50 proto... 062A63B8 lock.... F1000089C014EA40 head.... 00000000
q0...... 00000000 q....... 00000000 dq...... 00000000 q0len....... 0000
qlen........ 0000 qlimit...... 03E8 dqlen....... 0000 timeo....... 0000
error....... 0000 special..... 0A08 pgid.... 00000000 oobmark. 00000000
snd:cc...... 00000000 hiwat... 00004000 mbcnt... 00000000 mbmax... 00010000
lowat... 00001000 mb...... 00000000 sel..... 00000000 events...... 0000
iodone.. 00000000 ioargs.. 00000000 lastpkt. 00000000 wakeone. FFFFFFFF
timer... 00000000 timeo... 00000000 flags....... 0000 ()
wakeup.. 00000000 wakearg. 00000000 lock.... FFFFFFFFF1000089
rcv:cc...... 00000000 hiwat... 00004000 mbcnt... 00000000 mbmax... 00010000
lowat... 00000001 mb...... 00000000 sel..... 00000000 events...... 0001
iodone.. 00000000 ioargs.. 00000000 lastpkt. 00000000 wakeone. FFFFFFFF
timer... 00000000 timeo... 00000000 flags....... 0008 (SEL)
wakeup.. 00000000 wakearg. 00000000 lock.... FFFFFFFFF1000089
tpcb.... 00000000 fdev_ch. F100009E2C3CF680 sec_info 00000000 qos..... 00000000
gidlist. 00000000 private. 00000000 uid..... 00000000 bufsize. 00000000
threadcnt00000000 nextfree 00000000 siguid.. 00000000 sigeuid. 00000000
sigpriv. 00000000
sndtime. 00000000 sec 00000000 usec rcvtime. 00000000 sec 00000000 usec
proc/fd: 27/12
---- TCP ----(inpcb: @ F1000089C0283250)----
---- SOCKET INFO ----(@ F1000089C0283000)----
type........ 0001 (STREAM)
.
.
.
threadcnt00000000 nextfree 00000000 siguid.. 00000000 sigeuid. 00000000
sigpriv. 00000000
sndtime. 0000000A sec 00000000 usec rcvtime. 00000000 sec 00000000 usec
proc/fd:
(0)>
(0)>
(0)>
19. u명령으로 user thread를 조회합니다.
(0)> u
User thread context [F00000002FF3A600]:
save.... @ F00000002FF3A600 fpr..... @ F00000002FF3A808
Uthread System call state:
msr........800000000000D0B2
errnopp....0FFFFFFFFFFFFFE8 error......00000000
context...0000000000000000
sigssz....0000000000000000
stkb......0000000000000000
scsave[0]..BADC0FFEE0DDF00D scsave[1]..0000000000000008
scsave[2]..0000000000000004 scsave[3]..0900000000035894
scsave[4]..FFFFFFFFFFFFFFFF scsave[5]..00000000F0080CCC
scsave[6]..FFFFFFFFFFFFFFFF scsave[7]..0000000000000003
flags: ASSERTSIG CHKWAKE
kstack.....F00000002FF3A600 audsvc.....0000000000000000
.
.
.
SNODE... @ F00000002FF3B928
origin...F000000020000000
freeind..FFFFFFFF nextind..00000002
maxind...00071646 size.....00000120
UNODE... @ F00000002FF3B948
origin...F000000027F91000
freeind..FFFFFFFF nextind..00000011
maxind...000D6D93 size.....00000098
(0)>
20. oslevel명령과 lslpp명령으로 oslevel과 명령어 버전을 확인
=> oslevel -r 과 -g명령으로 oslevel상태를 체크함
kars1141:/kaprod> oslevel -r
5100-03
kars1141:/kaprod> oslevel -g
Fileset Actual Level Maintenance Level
-----------------------------------------------------------------------------
IMNSearch.bld.DBCS 2.3.1.15 2.3.1.0
IMNSearch.bld.SBCS 2.3.1.15 2.3.1.0
IMNSearch.rte.com 2.3.1.15 2.3.1.0
.
.
.
xlsmp.msg.en_US.rte 1.3.6.0 1.3.4.0
xlsmp.rte 1.3.6.0 1.3.4.0
kars1141:/kaprod>
=> lslpp 명령으로 실행가능한 명령어의 버전을 확인함
kars1141:/kaprod> lslpp -Lc
#Package Name:Fileset:Level:State:PTF Id:Fix State:Type:Description:Destination Dir.:Uninstaller:Message Catalog:Message Set:Message Number:Parent
IMNSearch.bld:IMNSearch.bld.DBCS:2.3.1.15: : :C: :NetQuestion DBCS Buildtime Modules : : : : : : :
IMNSearch.bld:IMNSearch.bld.SBCS:2.3.1.15: : :C: :NetQuestion SBCS Buildtime Modules : : : : : : :
.
.
.
cdrecord:cdrecord-1.9-4:1.9: : :C:R:A command line CD/DVD recording program.: :/bin/rpm -e cdrecord: : : : :
mkisofs:mkisofs-1.13-4:1.13: : :C:R:Creates an image of an ISO9660 filesystem.: :/bin/rpm -e mkisofs: : : : :
mtools:mtools-3.9.8-1:3.9.8: : :C:R:Programs for accessing MS-DOS disks without mounting the disks.: :/bin/rpm -e mtools: : : : :
kars1141:/kaprod>
21. q명령으로 kdb를 종료하고, exit명령으로 script모드를 종료합니다.
(0)> q
kars2141:/kaprod> ps -ef | grep hth
eworks 802880 1 0 Jun 12 - 0:00 hth -I jeuservice_565484 -b 565484
root 852098 606268 0 10:54:46 pts/3 0:00 grep hth
kars2141:/kaprod>
kars2141:/kaprod> exit
Script command is complete. The file is wlog.txt.
kars2141:/kaprod> ls -alrt wlog.txt
<- 해당 로그파일이 제대로 생성되었는지 확인
'OS' 카테고리의 다른 글
| split - 동일한 크기로 파일자르기 (0) | 2009.09.22 |
|---|---|
| [AIX] man netstat (0) | 2009.07.27 |
| [명령어]Diff (0) | 2009.05.18 |
| expr - 숫자계산 (0) | 2009.04.08 |
| 유닉스 쉘 간단한 팁 (0) | 2009.04.08 |