External links:
https://www.kernel.org/doc/Documentation/filesystems/proc.txt
http://brokestream.com/procstat.html
################################################################
Most of the time we need to know the different stat of a running process and we can find there information on the proc file system.
Lets say a java process is running at process id: 21879, and we need to know the stack size of that process, then we can use the following command. As per some reference, we can find the stack size stay on 23 field number.
so the command will be as per the example:
awk '{print $23}' /proc/21879/stat
heap=$(awk '{print $23}' /proc/21879/stat) ; echo $heap
You can also use codes from "man proc" about stats to format data in "ps" command, like:
ps axo stat,euid,ruid,tty,tpgid,sess,pgrp,ppid,pid,pcpu,comm | grep <PID number>
you can use pidof java to get the pid information of your java process.
################# /proc/pid/stat ########################3
Table 1-4: Contents of the stat files (as of 2.6.30-rc7)
..............................................................................
Field Content
pid process id
tcomm filename of the executable
state state (R is running, S is sleeping, D is sleeping in an
uninterruptible wait, Z is zombie, T is traced or stopped)
ppid process id of the parent process
pgrp pgrp of the process
sid session id
tty_nr tty the process uses
tty_pgrp pgrp of the tty
flags task flags
min_flt number of minor faults
cmin_flt number of minor faults with child's
maj_flt number of major faults
cmaj_flt number of major faults with child's
utime user mode jiffies
stime kernel mode jiffies
cutime user mode jiffies with child's
cstime kernel mode jiffies with child's
priority priority level
nice nice level
num_threads number of threads
it_real_value (obsolete, always 0)
start_time time the process started after system boot
vsize virtual memory size
rss resident set memory size
rsslim current limit in bytes on the rss
start_code address above which program text can run
end_code address below which program text can run
start_stack address of the start of the main process stack
esp current value of ESP
eip current value of EIP
pending bitmap of pending signals
blocked bitmap of blocked signals
sigign bitmap of ignored signals
sigcatch bitmap of catched signals
wchan address where process went to sleep
0 (place holder)
0 (place holder)
exit_signal signal to send to parent thread on exit
task_cpu which CPU the task is scheduled on
rt_priority realtime priority
policy scheduling policy (man sched_setscheduler)
blkio_ticks time spent waiting for block IO
gtime guest time of the task in jiffies
cgtime guest time of the task children in jiffies
start_data address above which program data+bss is placed
end_data address below which program data+bss is placed
start_brk address above which program heap can be expanded with brk()
arg_start address above which program command line is placed
arg_end address below which program command line is placed
env_start address above which program environment is placed
env_end address below which program environment is placed
exit_code the thread's exit_code in the form reported by the waitpid system call
##################### /proc/pid/status ############################
for i in `pidof java`; do grep Threads /proc/$i/status; done
Threads: 38
Threads: 15
Threads: 31
=======================================================================
sample output of status proc pid file:
cat /proc/21879/status
Name: java
State: S (sleeping)
Tgid: 21879
Pid: 21879
PPid: 1
TracerPid: 0
Uid: 1001 1001 1001 1001
Gid: 1001 1001 1001 1001
FDSize: 256
Groups: 1001
VmPeak: 1219112 kB
VmSize: 1210356 kB
VmLck: 0 kB
VmHWM: 167016 kB
VmRSS: 147272 kB
VmData: 1196816 kB
VmStk: 92 kB
VmExe: 40 kB
VmLib: 12944 kB
VmPTE: 400 kB
Threads: 38
SigQ: 0/16382
SigPnd: 0000000000000000
ShdPnd: 0000000000000000
SigBlk: 0000000000000000
SigIgn: 0000000000000003
SigCgt: 1000000181005ccc
CapInh: 0000000000000000
CapPrm: 0000000000000000
CapEff: 0000000000000000
CapBnd: ffffffffffffffff
Cpus_allowed: 3
Cpus_allowed_list: 0-1
Mems_allowed: 1
Mems_allowed_list: 0
voluntary_ctxt_switches: 20
nonvoluntary_ctxt_switches: 2
Table 1-2: Contents of the status files (as of 2.6.30-rc7)
..............................................................................
Field Content
Name filename of the executable
State state (R is running, S is sleeping, D is sleeping
in an uninterruptible wait, Z is zombie,
T is traced or stopped)
Tgid thread group ID
Pid process id
PPid process id of the parent process
TracerPid PID of process tracing this process (0 if not)
Uid Real, effective, saved set, and file system UIDs
Gid Real, effective, saved set, and file system GIDs
FDSize number of file descriptor slots currently allocated
Groups supplementary group list
VmPeak peak virtual memory size
VmSize total program size
VmLck locked memory size
VmHWM peak resident set size ("high water mark")
VmRSS size of memory portions
VmData size of data, stack, and text segments
VmStk size of data, stack, and text segments
VmExe size of text segment
VmLib size of shared library code
VmPTE size of page table entries
VmSwap size of swap usage (the number of referred swapents)
Threads number of threads
SigQ number of signals queued/max. number for queue
SigPnd bitmap of pending signals for the thread
ShdPnd bitmap of shared pending signals for the process
SigBlk bitmap of blocked signals
SigIgn bitmap of ignored signals
SigCgt bitmap of catched signals
CapInh bitmap of inheritable capabilities
CapPrm bitmap of permitted capabilities
CapEff bitmap of effective capabilities
CapBnd bitmap of capabilities bounding set
Seccomp seccomp mode, like prctl(PR_GET_SECCOMP, ...)
Cpus_allowed mask of CPUs on which this process may run
Cpus_allowed_list Same as previous, but in "list format"
Mems_allowed mask of memory nodes allowed to this process
Mems_allowed_list Same as previous, but in "list format"
voluntary_ctxt_switches number of voluntary context switches
nonvoluntary_ctxt_switches number of non voluntary context switches
######################################################################
cat /proc/21879/statm
302589 36966 2191 10 0 299227 0
Table 1-3: Contents of the statm files (as of 2.6.8-rc3)
..............................................................................
Field Content
size total program size (pages) (same as VmSize in status)
resident size of memory portions (pages) (same as VmRSS in status)
shared number of pages that are shared (i.e. backed by a file)
trs number of pages that are 'code' (not including libs; broken,
includes data segment)
lrs number of pages of library (always 0 on 2.6)
drs number of pages of data/stack (including libs; broken,
includes library text)
dt number of dirty pages (always 0 on 2.6)
#################################
/proc/21879# cat io
rchar: 79214336508
wchar: 79227167997
syscr: 20996240
syscw: 20059217
read_bytes: 48635904
write_bytes: 79561936896
cancelled_write_bytes: 1781272576
#################################
/proc/21879# cat limits
Limit Soft Limit Hard Limit Units
Max cpu time unlimited unlimited seconds
Max file size unlimited unlimited bytes
Max data size unlimited unlimited bytes
Max stack size 8388608 unlimited bytes
Max core file size 0 unlimited bytes
Max resident set unlimited unlimited bytes
Max processes unlimited unlimited processes
Max open files 1024 1024 files
Max locked memory 65536 65536 bytes
Max address space unlimited unlimited bytes
Max file locks unlimited unlimited locks
Max pending signals 16382 16382 signals
Max msgqueue size 819200 819200 bytes
Max nice priority 20 20
Max realtime priority 0 0
Max realtime timeout unlimited unlimited us
#################################
/proc/21879# cat sched
java (21879, #threads: 38)
---------------------------------------------------------
se.exec_start : 521472717.252865
se.vruntime : 943088.844576
se.sum_exec_runtime : 25.769130
se.avg_overlap : 0.234375
se.avg_wakeup : 2.000000
se.avg_running : 1.557718
se.wait_start : 0.000000
se.sleep_start : 521472717.252865
se.block_start : 0.000000
se.sleep_max : 104.997369
se.block_max : 0.000000
se.exec_max : 8.526763
se.slice_max : 3.511830
se.wait_max : 2.171300
se.wait_sum : 2.886877
se.wait_count : 22
se.iowait_sum : 1783.892404
se.iowait_count : 6247
sched_info.bkl_count : 0
se.nr_migrations : 0
se.nr_migrations_cold : 0
se.nr_failed_migrations_affine : 0
se.nr_failed_migrations_running : 0
se.nr_failed_migrations_hot : 0
se.nr_forced_migrations : 0
se.nr_forced2_migrations : 0
se.nr_wakeups : 19
se.nr_wakeups_sync : 19
se.nr_wakeups_migrate : 0
se.nr_wakeups_local : 19
se.nr_wakeups_remote : 0
se.nr_wakeups_affine : 0
se.nr_wakeups_affine_attempts : 0
se.nr_wakeups_passive : 0
se.nr_wakeups_idle : 0
avg_atom : 1.171324
avg_per_cpu : 0.000001
nr_switches : 22
nr_voluntary_switches : 20
nr_involuntary_switches : 2
se.load.weight : 1024
policy : 0
prio : 120
clock-delta : 255
#################################
/proc/21879# cat sessionid
4294967295
#################################
/proc/21879# cat stack
[<c015919c>] futex_wait_queue_me+0xac/0xd0
[<c0159ac4>] futex_wait+0xf4/0x220
[<c015bb16>] do_futex+0xe6/0x1f0
[<c015bc89>] sys_futex+0x69/0x110
[<c01047e0>] syscall_call+0x7/0xb
[<ffffffff>] 0xffffffff
#################################
NOTE: /proc/pid/task: [ each directory is for each thread ]
#################################
Table 1-5: Kernel info in /proc
..............................................................................
File Content
apm Advanced power management info
buddyinfo Kernel memory allocator information (see text) (2.5)
bus Directory containing bus specific information
cmdline Kernel command line
cpuinfo Info about the CPU
devices Available devices (block and character)
dma Used DMS channels
filesystems Supported filesystems
driver Various drivers grouped here, currently rtc (2.4)
execdomains Execdomains, related to security (2.4)
fb Frame Buffer devices (2.4)
fs File system parameters, currently nfs/exports (2.4)
ide Directory containing info about the IDE subsystem
interrupts Interrupt usage
iomem Memory map (2.4)
ioports I/O port usage
irq Masks for irq to cpu affinity (2.4)(smp?)
isapnp ISA PnP (Plug&Play) Info (2.4)
kcore Kernel core image (can be ELF or A.OUT(deprecated in 2.4))
kmsg Kernel messages
ksyms Kernel symbol table
loadavg Load average of last 1, 5 & 15 minutes
locks Kernel locks
meminfo Memory info
misc Miscellaneous
modules List of loaded modules
mounts Mounted filesystems
net Networking info (see text)
pagetypeinfo Additional page allocator information (see text) (2.5)
partitions Table of partitions known to the system
pci Deprecated info of PCI bus (new way -> /proc/bus/pci/,
decoupled by lspci (2.4)
rtc Real time clock
scsi SCSI info (see text)
slabinfo Slab pool info
softirqs softirq usage
stat Overall statistics
swaps Swap space utilization
sys See chapter 2
sysvipc Info of SysVIPC Resources (msg, sem, shm) (2.4)
tty Info of tty drivers
uptime Wall clock since boot, combined idle time of all cpus
version Kernel version
video bttv info of video resources (2.4)
vmallocinfo Show vmalloced areas
#################################
meminfo:
Provides information about distribution and utilization of memory. This
varies by architecture and compile options. The following is from a
16GB PIII, which has highmem enabled. You may not have all of these fields.
> cat /proc/meminfo
The "Locked" indicates whether the mapping is locked in memory or not.
MemTotal: 16344972 kB
MemFree: 13634064 kB
MemAvailable: 14836172 kB
Buffers: 3656 kB
Cached: 1195708 kB
SwapCached: 0 kB
Active: 891636 kB
Inactive: 1077224 kB
HighTotal: 15597528 kB
HighFree: 13629632 kB
LowTotal: 747444 kB
LowFree: 4432 kB
SwapTotal: 0 kB
SwapFree: 0 kB
Dirty: 968 kB
Writeback: 0 kB
AnonPages: 861800 kB
Mapped: 280372 kB
Slab: 284364 kB
SReclaimable: 159856 kB
SUnreclaim: 124508 kB
PageTables: 24448 kB
NFS_Unstable: 0 kB
Bounce: 0 kB
WritebackTmp: 0 kB
CommitLimit: 7669796 kB
Committed_AS: 100056 kB
VmallocTotal: 112216 kB
VmallocUsed: 428 kB
VmallocChunk: 111088 kB
AnonHugePages: 49152 kB
MemTotal: Total usable ram (i.e. physical ram minus a few reserved
bits and the kernel binary code)
MemFree: The sum of LowFree+HighFree
MemAvailable: An estimate of how much memory is available for starting new
applications, without swapping. Calculated from MemFree,
SReclaimable, the size of the file LRU lists, and the low
watermarks in each zone.
The estimate takes into account that the system needs some
page cache to function well, and that not all reclaimable
slab will be reclaimable, due to items being in use. The
impact of those factors will vary from system to system.
Buffers: Relatively temporary storage for raw disk blocks
shouldn't get tremendously large (20MB or so)
Cached: in-memory cache for files read from the disk (the
pagecache). Doesn't include SwapCached
SwapCached: Memory that once was swapped out, is swapped back in but
still also is in the swapfile (if memory is needed it
doesn't need to be swapped out AGAIN because it is already
in the swapfile. This saves I/O)
Active: Memory that has been used more recently and usually not
reclaimed unless absolutely necessary.
Inactive: Memory which has been less recently used. It is more
eligible to be reclaimed for other purposes
HighTotal:
HighFree: Highmem is all memory above ~860MB of physical memory
Highmem areas are for use by userspace programs, or
for the pagecache. The kernel must use tricks to access
this memory, making it slower to access than lowmem.
LowTotal:
LowFree: Lowmem is memory which can be used for everything that
highmem can be used for, but it is also available for the
kernel's use for its own data structures. Among many
other things, it is where everything from the Slab is
allocated. Bad things happen when you're out of lowmem.
SwapTotal: total amount of swap space available
SwapFree: Memory which has been evicted from RAM, and is temporarily
on the disk
Dirty: Memory which is waiting to get written back to the disk
Writeback: Memory which is actively being written back to the disk
AnonPages: Non-file backed pages mapped into userspace page tables
AnonHugePages: Non-file backed huge pages mapped into userspace page tables
Mapped: files which have been mmaped, such as libraries
Slab: in-kernel data structures cache
SReclaimable: Part of Slab, that might be reclaimed, such as caches
SUnreclaim: Part of Slab, that cannot be reclaimed on memory pressure
PageTables: amount of memory dedicated to the lowest level of page
tables.
NFS_Unstable: NFS pages sent to the server, but not yet committed to stable
storage
Bounce: Memory used for block device "bounce buffers"
WritebackTmp: Memory used by FUSE for temporary writeback buffers
CommitLimit: Based on the overcommit ratio ('vm.overcommit_ratio'),
this is the total amount of memory currently available to
be allocated on the system. This limit is only adhered to
if strict overcommit accounting is enabled (mode 2 in
'vm.overcommit_memory').
The CommitLimit is calculated with the following formula:
CommitLimit = ('vm.overcommit_ratio' * Physical RAM) + Swap
For example, on a system with 1G of physical RAM and 7G
of swap with a `vm.overcommit_ratio` of 30 it would
yield a CommitLimit of 7.3G.
For more details, see the memory overcommit documentation
in vm/overcommit-accounting.
Committed_AS: The amount of memory presently allocated on the system.
The committed memory is a sum of all of the memory which
has been allocated by processes, even if it has not been
"used" by them as of yet. A process which malloc()'s 1G
of memory, but only touches 300M of it will show up as
using 1G. This 1G is memory which has been "committed" to
by the VM and can be used at any time by the allocating
application. With strict overcommit enabled on the system
(mode 2 in 'vm.overcommit_memory'),allocations which would
exceed the CommitLimit (detailed above) will not be permitted.
This is useful if one needs to guarantee that processes will
not fail due to lack of memory once that memory has been
successfully allocated.
VmallocTotal: total size of vmalloc memory area
VmallocUsed: amount of vmalloc area which is used
VmallocChunk: largest contiguous block of vmalloc area which is free
#################################
3.3 /proc/<pid>/io - Display the IO accounting fields
-------------------------------------------------------
This file contains IO statistics for each running process
Example
-------
test:/tmp # dd if=/dev/zero of=/tmp/test.dat &
[1] 3828
test:/tmp # cat /proc/3828/io
rchar: 323934931
wchar: 323929600
syscr: 632687
syscw: 632675
read_bytes: 0
write_bytes: 323932160
cancelled_write_bytes: 0
Description
-----------
rchar
-----
I/O counter: chars read
The number of bytes which this task has caused to be read from storage. This
is simply the sum of bytes which this process passed to read() and pread().
It includes things like tty IO and it is unaffected by whether or not actual
physical disk IO was required (the read might have been satisfied from
pagecache)
wchar
-----
I/O counter: chars written
The number of bytes which this task has caused, or shall cause to be written
to disk. Similar caveats apply here as with rchar.
syscr
-----
I/O counter: read syscalls
Attempt to count the number of read I/O operations, i.e. syscalls like read()
and pread().
syscw
-----
I/O counter: write syscalls
Attempt to count the number of write I/O operations, i.e. syscalls like
write() and pwrite().
read_bytes
----------
I/O counter: bytes read
Attempt to count the number of bytes which this process really did cause to
be fetched from the storage layer. Done at the submit_bio() level, so it is
accurate for block-backed filesystems. <please add status regarding NFS and
CIFS at a later time>
write_bytes
-----------
I/O counter: bytes written
Attempt to count the number of bytes which this process caused to be sent to
the storage layer. This is done at page-dirtying time.
cancelled_write_bytes
---------------------
The big inaccuracy here is truncate. If a process writes 1MB to a file and
then deletes the file, it will in fact perform no writeout. But it will have
been accounted as having caused 1MB of write.
In other words: The number of bytes which this process caused to not happen,
by truncating pagecache. A task can cause "negative" IO too. If this task
truncates some dirty pagecache, some IO which another task has been accounted
for (in its write_bytes) will not be happening. We _could_ just subtract that
from the truncating task's write_bytes, but there is information loss in doing
that.
#################################
#################################
Friday, 25 April 2014
Monday, 21 April 2014
Saturday, 5 April 2014
lsof
#### lsof ####
lsof - list open files
In the absence of any options, lsof lists all open files belonging to all active processes.
$ sudo lsof -c apache2 :
[ -c c selects the listing of files for processes executing the command that begins with the characters of c. Multiple commands may be specified, using multiple -c options. They are joined in a sin‐
gle ORed set before participating in AND option selection. ]
$ lsof +D /etc :
-D D directs lsof's use of the device cache file. The use of this option is sometimes restricted. See the DEVICE CACHE FILE section and the sections that follow it for more information on this
option.
+|-L [l] enables (`+') or disables (`-') the listing of file link counts, where they are available - e.g., they aren't available for sockets, or most FIFOs and pipes.
When +L is specified without a following number, all link counts will be listed. When -L is specified (the default), no link counts will be listed.
When +L is followed by a number, only files having a link count less than that number will be listed. (No number may follow -L.) A specification of the form ``+L1'' will select open files that
have been unlinked. A specification of the form ``+aL1 <file_system>'' will select unlinked open files on the specified file system.
-N selects the listing of NFS files.
-p s excludes or selects the listing of files for the processes whose optional process IDentification (PID) numbers are in the comma-separated set s - e.g., ``123'' or ``123,^456''. (There should be
no spaces in the set.)
PID numbers that begin with `^' (negation) represent exclusions.
Multiple process ID numbers are joined in a single ORed set before participating in AND option selection. However, PID exclusions are applied without ORing or ANDing and take effect before
other selection criteria are applied.
EXAMPLES
For a more extensive set of examples, documented more fully, see the 00QUICKSTART file of the lsof di
stribution.
To list all open files, use:
lsof
To list all open Internet, x.25 (HP-UX), and UNIX domain files, use:
lsof -i -U
To list all open IPv4 network files in use by the process whose PID is 1234, use:
lsof -i 4 -a -p 1234
Presuming the UNIX dialect supports IPv6, to list only open IPv6 network files, use:
lsof -i 6
To list all files using any protocol on ports 513, 514, or 515 of host wonderland.cc.purdue.edu, use:
lsof -i @wonderland.cc.purdue.edu:513-515
To list all files using any protocol on any port of mace.cc.purdue.edu (cc.purdue.edu is the default
domain), use:
lsof -i @mace
To list all open files for login name ``abe'', or user ID 1234, or process 456, or process 123, or process 789, use:
lsof -p 456,123,789 -u 1234,abe
To list all open files on device /dev/hd4, use:
lsof /dev/hd4
To find the process that has /u/abe/foo open, use:
lsof /u/abe/foo
To send a SIGHUP to the processes that have /u/abe/bar open, use:
kill -HUP `lsof -t /u/abe/bar`
To find any open file, including an open UNIX domain socket file, with the name /dev/log, use:
lsof /dev/log
To find processes with open files on the NFS file system named /nfs/mount/point whose server is inacc
essible, and presuming your mount table supplies the device number for /nfs/mount/point, use:
lsof -b /nfs/mount/point
To do the preceding search with warning messages suppressed, use:
lsof -bw /nfs/mount/point
To ignore the device cache file, use:
lsof -Di
To obtain PID and command name field output for each process, file descriptor, file device number, and file inode number for each file of each process, use:
lsof -FpcfDi
To list the files at descriptors 1 and 3 of every process running the lsof command for login ID ``abe'' every 10 seconds, use:
lsof -c lsof -a -d 1 -d 3 -u abe -r10
To list the current working directory of processes running a command that is exactly four characters long and has an 'o' or 'O' in character three, use this regular expression form of the -c c option:
lsof -c /^..o.$/i -a -d cwd
To find an IP version 4 socket file by its associated numeric dot-form address, use:
lsof -i@128.210.15.17
To find an IP version 6 socket file (when the UNIX dialect supports IPv6) by its associated numeric colon-form address, use:
lsof -i@[0:1:2:3:4:5:6:7]
To find an IP version 6 socket file (when the UNIX dialect supports IPv6) by an associated numeric colon-form address that has a run of zeroes in it - e.g., the loop-back address - use:
lsof -i@[::1]
To obtain a repeat mode marker line that contains the current time, use:
lsof -rm====%T====
To add spaces to the previous marker line, use:
lsof -r "m==== %T ===="
lsof - list open files
In the absence of any options, lsof lists all open files belonging to all active processes.
$ sudo lsof -c apache2 :
[ -c c selects the listing of files for processes executing the command that begins with the characters of c. Multiple commands may be specified, using multiple -c options. They are joined in a sin‐
gle ORed set before participating in AND option selection. ]
$ lsof +D /etc :
-D D directs lsof's use of the device cache file. The use of this option is sometimes restricted. See the DEVICE CACHE FILE section and the sections that follow it for more information on this
option.
+|-L [l] enables (`+') or disables (`-') the listing of file link counts, where they are available - e.g., they aren't available for sockets, or most FIFOs and pipes.
When +L is specified without a following number, all link counts will be listed. When -L is specified (the default), no link counts will be listed.
When +L is followed by a number, only files having a link count less than that number will be listed. (No number may follow -L.) A specification of the form ``+L1'' will select open files that
have been unlinked. A specification of the form ``+aL1 <file_system>'' will select unlinked open files on the specified file system.
-N selects the listing of NFS files.
-p s excludes or selects the listing of files for the processes whose optional process IDentification (PID) numbers are in the comma-separated set s - e.g., ``123'' or ``123,^456''. (There should be
no spaces in the set.)
PID numbers that begin with `^' (negation) represent exclusions.
Multiple process ID numbers are joined in a single ORed set before participating in AND option selection. However, PID exclusions are applied without ORing or ANDing and take effect before
other selection criteria are applied.
EXAMPLES
For a more extensive set of examples, documented more fully, see the 00QUICKSTART file of the lsof di
stribution.
To list all open files, use:
lsof
To list all open Internet, x.25 (HP-UX), and UNIX domain files, use:
lsof -i -U
To list all open IPv4 network files in use by the process whose PID is 1234, use:
lsof -i 4 -a -p 1234
Presuming the UNIX dialect supports IPv6, to list only open IPv6 network files, use:
lsof -i 6
To list all files using any protocol on ports 513, 514, or 515 of host wonderland.cc.purdue.edu, use:
lsof -i @wonderland.cc.purdue.edu:513-515
To list all files using any protocol on any port of mace.cc.purdue.edu (cc.purdue.edu is the default
domain), use:
lsof -i @mace
To list all open files for login name ``abe'', or user ID 1234, or process 456, or process 123, or process 789, use:
lsof -p 456,123,789 -u 1234,abe
To list all open files on device /dev/hd4, use:
lsof /dev/hd4
To find the process that has /u/abe/foo open, use:
lsof /u/abe/foo
To send a SIGHUP to the processes that have /u/abe/bar open, use:
kill -HUP `lsof -t /u/abe/bar`
To find any open file, including an open UNIX domain socket file, with the name /dev/log, use:
lsof /dev/log
To find processes with open files on the NFS file system named /nfs/mount/point whose server is inacc
essible, and presuming your mount table supplies the device number for /nfs/mount/point, use:
lsof -b /nfs/mount/point
To do the preceding search with warning messages suppressed, use:
lsof -bw /nfs/mount/point
To ignore the device cache file, use:
lsof -Di
To obtain PID and command name field output for each process, file descriptor, file device number, and file inode number for each file of each process, use:
lsof -FpcfDi
To list the files at descriptors 1 and 3 of every process running the lsof command for login ID ``abe'' every 10 seconds, use:
lsof -c lsof -a -d 1 -d 3 -u abe -r10
To list the current working directory of processes running a command that is exactly four characters long and has an 'o' or 'O' in character three, use this regular expression form of the -c c option:
lsof -c /^..o.$/i -a -d cwd
To find an IP version 4 socket file by its associated numeric dot-form address, use:
lsof -i@128.210.15.17
To find an IP version 6 socket file (when the UNIX dialect supports IPv6) by its associated numeric colon-form address, use:
lsof -i@[0:1:2:3:4:5:6:7]
To find an IP version 6 socket file (when the UNIX dialect supports IPv6) by an associated numeric colon-form address that has a run of zeroes in it - e.g., the loop-back address - use:
lsof -i@[::1]
To obtain a repeat mode marker line that contains the current time, use:
lsof -rm====%T====
To add spaces to the previous marker line, use:
lsof -r "m==== %T ===="
super block - filesystem metadata - sort and hard link
Following command displays primary and backup superblock location on /dev/sda3:
# dumpe2fs /dev/hda3 | grep -i superblock
But what is in a File system?
Again file system divided into two categories:
User data - stores actual data contained in files
Metadata - stores file system structural information such as superblock, inodes, directories
Filesystem Failures:
Most of time fsck (front end to ext2/ext3 utility) can fix the problem, first simply run e2fsck - to check a Linux ext2/ext3 file system (assuming /home [/dev/sda3 partition] filesystem for demo purpose), first unmount /dev/sda3 then type following command :
# e2fsck -f /dev/sda3
Where,
-f : Force checking even if the file system seems clean.
Please note that If the superblock is not found, e2fsck will terminate with a fatal error. However Linux maintains multiple redundant copies of the superblock in every file system, so you can use -b {alternative-superblock} option to get rid of this problem. The location of the backup superblock is dependent on the filesystem's blocksize:
For filesystems with 1k blocksizes, a backup superblock can be found at block 8193
For filesystems with 2k blocksizes, at block 16384
For 4k blocksizes, at block 32768.
Tip you can also try any one of the following command(s) to determine alternative-superblock locations:
# mke2fs -n /dev/sda3
OR
# dumpe2fs /dev/sda3|grep -i superblock
To repair file system by alternative-superblock use command as follows:
# e2fsck -f -b 8193 /dev/sda3
However it is highly recommended that you make backup before you run fsck command on system, use dd command to create a backup (provided that you have spare space under /disk2)
# dd if=/dev/sda2 of=/disk2/backup-sda2.img
# Understanding UNIX / Linux filesystem Inodes
The inode (index node) is a fundamental concept in the Linux and UNIX filesystem. Each object in the filesystem is represented by an inode. But what are the objects? Let us try to understand it in simple words. Each and every file under Linux (and UNIX) has following attributes:
########## Inode stores ###############
=> File type (executable, block special etc)
=> Permissions (read, write etc)
=> Owner
=> Group
=> File Size
=> File access, change and modification time (remember UNIX or Linux never stores file creation time, this is favorite question asked in UNIX/Linux sys admin job interview)
=> File deletion time
=> Number of links (soft/hard)
=> Extended attribute such as append only or no one can delete file including root user (immutability)
=> Access Control List (ACLs)
###########################################
All the above information stored in an inode. In short the inode identifies the file and its attributes (as above) . Each inode is identified by a unique inode number within the file system. Inode is also know as index number.
Inode application
Many commands used by system administrators in UNIX / Linux operating systems often give inode numbers to designate a file. Let us see he practical application of inode number. Type the following commands:
$ cd /tmp
$ touch \"la*
$ ls -l
Now try to remove file "la*
You can't, to remove files having created with control characters or characters which are unable to be input on a keyboard or special character such as ?, * ^ etc. You have to use inode number to remove file.
# How to: Linux / UNIX Delete or Remove Files With Inode Number
An inode identifies the file and its attributes such as file size, owner, and so on. A unique inode number within the file system identifies each inode. But, why to delete file by an inode number? Sure, you can use rm command to delete file. Sometime accidentally you creates filename with control characters or characters which are unable to be input on a keyboard or special character such as ?, * ^ etc. Removing such special character filenames can be problem. Use following method to delete a file with strange characters in its name:
Please note that the procedure outlined below works with Solaris, FreeBSD, Linux, or any other Unixish oses out there:
Find out file inode
First find out file inode number with any one of the following command:
stat {file-name}
OR
ls -il {file-name}
Use find command to remove file:
Use find command as follows to find and remove a file:
find . -inum [inode-number] -exec rm -i {} \;
When prompted for confirmation, press Y to confirm removal of the file.
###
Understanding UNIX / Linux symbolic (soft) and hard links
Hard link vs. Soft link in Linux or UNIX
Hard links cannot link directories.
Cannot cross file system boundaries.
Soft or symbolic links are just like hard links. It allows to associate multiple filenames with a single file. However, symbolic links allows:
To create links between directories.
Can cross file system boundaries.
These links behave differently when the source of the link is moved or removed.
Symbolic links are not updated.
Hard links always refer to the source, even if moved or removed.
# Why isn’t it possible to create hard links across file system boundaries?
A single inode number use to represent file in each file system. All hard links based upon inode number.
So linking across file system will lead into confusing references for UNIX or Linux. For example, consider following scenario
* File system: /home
* Directory: /home/vivek
* Hard link: /home/vivek/file2
* Original file: /home/vivek/file1
Now you create a hard link as follows:
$ touch file1
$ ln file1 file2
$ ls -l
Output:
-rw-r--r-- 2 vivek vivek 0 2006-01-30 13:28 file1
-rw-r--r-- 2 vivek vivek 0 2006-01-30 13:28 file2
Now just see inode of both file1 and file2:
$ ls -i file1
782263
$ ls -i file2
782263
As you can see inode number is same for hard link file called file2 in inode table under /home file system. Now if you try to create a hard link for /tmp file system it will lead to confusing references for UNIX or Linux file system. Is that a link no. 782263 in the /home or /tmp file system? To avoid this problem UNIX or Linux does not allow creating hard links across file system boundaries.
######## What is a Superblock, Inode, Dentry and a File?
First and foremost, and I realize that it was not one of the terms from your question, you must understand metadata. Succinctly, and stolen from Wikipedia, metadata is data about data. That is to say that metadata contains information about a piece of data. For example, if I own a car then I have a set of information about the car but which is not part of the car itself. Information such as the registration number, make, model, year of manufacture, insurance information, and so on. All of that information is collectively referred to as the metadata. In Linux and UNIX file systems metadata exists at multiple levels of organization as you will see.
The superblock is essentially file system metadata and defines the file system type, size, status, and information about other metadata structures (metadata of metadata). The superblock is very critical to the file system and therefore is stored in multiple redundant copies for each file system. The superblock is a very "high level" metadata structure for the file system. For example, if the superblock of a partition, /var, becomes corrupt then the file system in question (/var) cannot be mounted by the operating system. Commonly in this event fsck is run and will automatically select an alternate, backup copy of the superblock and attempt to recover the file system. The backup copies themselves are stored in block groups spread through the file system with the first stored at a 1 block offset from the start of the partition. This is important in the event that a manual recovery is necessary. You may view information about superblock backups with the command dumpe2fs /dev/foo | grep -i superblock which is useful in the event of a manual recovery attempt. Let us suppose that the dumpe2fs command outputs the line Backup superblock at 163840, Group descriptors at 163841-163841. We can use this information, and additional knowledge about the file system structure, to attempt to use this superblock backup: /sbin/fsck.ext3 -b 163840 -B 1024 /dev/foo. Please note that I have assumed a block size of 1024 bytes for this example.
An inode exists in, or on, a file system and represents metadata about a file. For clarity, all objects in a Linux or UNIX system are files; actual files, directories, devices, and so on. Please note that, among the metadata contained in an inode, there is no file name as humans think of it, this will be important later. An inode contains essentially information about ownership (user, group), access mode (read, write, execute permissions) and file type.
A dentry is the glue that holds inodes and files together by relating inode numbers to file names. Dentries also play a role in directory caching which, ideally, keeps the most frequently used files on-hand for faster access. File system traversal is another aspect of the dentry as it maintains a relationship between directories and their files.
A file, in addition to being what humans typically think of when presented with the word, is really just a block of logically related arbitrary data. Comparatively very dull considering all of the work done (above) to keep track of them.
I fully realize that a few sentences do not provide a full explanation of any of these concepts so please feel free to ask for additional details when and where necessary.
File
A file just means a bunch of bytes arranged in a certain order. It's what normal people call the contents of a file. When Linux opens a file, it also creates a file object, which holds data about where the file is stored and what processes are using it. The file object (but not the file data itself) is thrown away when the file is closed.
Inode
An inode (short for "index node") is a bunch of attributes about a file that Linux stores. There is one inode for each file (though with some filesystems, Linux has to create its own inodes because the information is spread around the filesystem). The inode stores information like who owns the file, how big the file is, and who is allowed to open the file. Each inode also contains a number unique to the filesystem partition; it's like a serial number for the file described by that inode.
Dentry
A dentry (short for "directory entry") is what the Linux kernel uses to keep track of the hierarchy of files in directories. Each dentry maps an inode number to a file name and a parent directory.
Superblock
The superblock is a unique data structure in a filesystem (though multiple copies exist to guard against corruption). The superblock holds metadata about the filesystem, like which inode is the top-level directory and the type of filesystem used.
In simplicity, dentry and inode are the same thing, an abstraction of files and directories. The differences between dentry and inode are that dentry is used to facilitate directory-specific operations, inode is just a collection of metadata about files and directories. Superblock is the abstraction of filesystem.
###########
lsof:
The manpage of lsof on my Debian system says “When +L is followed by a number, only files having a link count less than that number will be listed.”
i.e: lsof +L1 [ will display all the files that is having 0 ref count => deleted files. ]
########
The attributes as handled by lsattr/chattr on Linux and some of which can be stored by quite a few file systems (ext2/3/4, reiserfs, JFS, OCFS2, btrfs, XFS, nilfs2, hfsplus...) and even queried over CIFS/SMB (when with POSIX extensions) are flags. Just bits than can be turned on or off to disable or enable an attribute (like immutable or archive...). How they are stored is file system specific, but generally as a 16/32/64 bit record in the inode.
The full list of flags is found on Linux native filesystems (ext2/3/4, btrfs...) though not all of the flags apply to all of FS, and for other non-native FS, Linux tries to map them to equivalent features in the corresponding file system. For instance the simmutable flag as stored by OSX on HFS+ file systems is mapped to the corresponding immutable flag in Linux chattr. What flag is supported by what file system is hardly documented at all. Often, reading the kernel source code is the only option.
Extended attributes on the other hand, as set with setfattr or attr on Linux store more than flags. They are attached to a file as well, and are key/value pairs that can be (both key and value) arbitrary arrays of bytes (though with limitation of size on some file systems).
The key can be for instance: system.posix_acl_access or user.rsync.%stat. The system namespace is reserved for the system (you wouldn't change the POSIX ACLs with setfattr, but more with setfacl, POSIX ACLs just happen to be stored as extended attributes at least on some file systems), while the user namespace can be used by applications (here rsync uses it for its --fake-super option, to store information about ownership or permissions when you're not superuser).
Again, how they are stored is filesystem specific. See WikiPedia for more information.
##########
http://unix.stackexchange.com/questions/117093/find-where-inodes-are-being-used
############
Try this with GNU find:
find /start/dir -L -samefile /file/to/check -exec ls -li {} \;
Example output:
1234704 -rw-r--r-- 2 user1 user1 1134 2009-09-11 11:12 ./x1
1234704 -rw-r--r-- 2 user1 user1 1134 2009-09-11 11:12 ./x2
1234983 lrwxrwxrwx 1 user1 user1 2 2009-10-31 16:56 ./testx -> x1
2345059 lrwxrwxrwx 1 user2 user2 2 2010-01-03 16:17 ./x3 -> x1
# dumpe2fs /dev/hda3 | grep -i superblock
But what is in a File system?
Again file system divided into two categories:
User data - stores actual data contained in files
Metadata - stores file system structural information such as superblock, inodes, directories
Filesystem Failures:
Most of time fsck (front end to ext2/ext3 utility) can fix the problem, first simply run e2fsck - to check a Linux ext2/ext3 file system (assuming /home [/dev/sda3 partition] filesystem for demo purpose), first unmount /dev/sda3 then type following command :
# e2fsck -f /dev/sda3
Where,
-f : Force checking even if the file system seems clean.
Please note that If the superblock is not found, e2fsck will terminate with a fatal error. However Linux maintains multiple redundant copies of the superblock in every file system, so you can use -b {alternative-superblock} option to get rid of this problem. The location of the backup superblock is dependent on the filesystem's blocksize:
For filesystems with 1k blocksizes, a backup superblock can be found at block 8193
For filesystems with 2k blocksizes, at block 16384
For 4k blocksizes, at block 32768.
Tip you can also try any one of the following command(s) to determine alternative-superblock locations:
# mke2fs -n /dev/sda3
OR
# dumpe2fs /dev/sda3|grep -i superblock
To repair file system by alternative-superblock use command as follows:
# e2fsck -f -b 8193 /dev/sda3
However it is highly recommended that you make backup before you run fsck command on system, use dd command to create a backup (provided that you have spare space under /disk2)
# dd if=/dev/sda2 of=/disk2/backup-sda2.img
# Understanding UNIX / Linux filesystem Inodes
The inode (index node) is a fundamental concept in the Linux and UNIX filesystem. Each object in the filesystem is represented by an inode. But what are the objects? Let us try to understand it in simple words. Each and every file under Linux (and UNIX) has following attributes:
########## Inode stores ###############
=> File type (executable, block special etc)
=> Permissions (read, write etc)
=> Owner
=> Group
=> File Size
=> File access, change and modification time (remember UNIX or Linux never stores file creation time, this is favorite question asked in UNIX/Linux sys admin job interview)
=> File deletion time
=> Number of links (soft/hard)
=> Extended attribute such as append only or no one can delete file including root user (immutability)
=> Access Control List (ACLs)
###########################################
All the above information stored in an inode. In short the inode identifies the file and its attributes (as above) . Each inode is identified by a unique inode number within the file system. Inode is also know as index number.
Inode application
Many commands used by system administrators in UNIX / Linux operating systems often give inode numbers to designate a file. Let us see he practical application of inode number. Type the following commands:
$ cd /tmp
$ touch \"la*
$ ls -l
Now try to remove file "la*
You can't, to remove files having created with control characters or characters which are unable to be input on a keyboard or special character such as ?, * ^ etc. You have to use inode number to remove file.
# How to: Linux / UNIX Delete or Remove Files With Inode Number
An inode identifies the file and its attributes such as file size, owner, and so on. A unique inode number within the file system identifies each inode. But, why to delete file by an inode number? Sure, you can use rm command to delete file. Sometime accidentally you creates filename with control characters or characters which are unable to be input on a keyboard or special character such as ?, * ^ etc. Removing such special character filenames can be problem. Use following method to delete a file with strange characters in its name:
Please note that the procedure outlined below works with Solaris, FreeBSD, Linux, or any other Unixish oses out there:
Find out file inode
First find out file inode number with any one of the following command:
stat {file-name}
OR
ls -il {file-name}
Use find command to remove file:
Use find command as follows to find and remove a file:
find . -inum [inode-number] -exec rm -i {} \;
When prompted for confirmation, press Y to confirm removal of the file.
###
Understanding UNIX / Linux symbolic (soft) and hard links
Hard link vs. Soft link in Linux or UNIX
Hard links cannot link directories.
Cannot cross file system boundaries.
Soft or symbolic links are just like hard links. It allows to associate multiple filenames with a single file. However, symbolic links allows:
To create links between directories.
Can cross file system boundaries.
These links behave differently when the source of the link is moved or removed.
Symbolic links are not updated.
Hard links always refer to the source, even if moved or removed.
# Why isn’t it possible to create hard links across file system boundaries?
A single inode number use to represent file in each file system. All hard links based upon inode number.
So linking across file system will lead into confusing references for UNIX or Linux. For example, consider following scenario
* File system: /home
* Directory: /home/vivek
* Hard link: /home/vivek/file2
* Original file: /home/vivek/file1
Now you create a hard link as follows:
$ touch file1
$ ln file1 file2
$ ls -l
Output:
-rw-r--r-- 2 vivek vivek 0 2006-01-30 13:28 file1
-rw-r--r-- 2 vivek vivek 0 2006-01-30 13:28 file2
Now just see inode of both file1 and file2:
$ ls -i file1
782263
$ ls -i file2
782263
As you can see inode number is same for hard link file called file2 in inode table under /home file system. Now if you try to create a hard link for /tmp file system it will lead to confusing references for UNIX or Linux file system. Is that a link no. 782263 in the /home or /tmp file system? To avoid this problem UNIX or Linux does not allow creating hard links across file system boundaries.
######## What is a Superblock, Inode, Dentry and a File?
First and foremost, and I realize that it was not one of the terms from your question, you must understand metadata. Succinctly, and stolen from Wikipedia, metadata is data about data. That is to say that metadata contains information about a piece of data. For example, if I own a car then I have a set of information about the car but which is not part of the car itself. Information such as the registration number, make, model, year of manufacture, insurance information, and so on. All of that information is collectively referred to as the metadata. In Linux and UNIX file systems metadata exists at multiple levels of organization as you will see.
The superblock is essentially file system metadata and defines the file system type, size, status, and information about other metadata structures (metadata of metadata). The superblock is very critical to the file system and therefore is stored in multiple redundant copies for each file system. The superblock is a very "high level" metadata structure for the file system. For example, if the superblock of a partition, /var, becomes corrupt then the file system in question (/var) cannot be mounted by the operating system. Commonly in this event fsck is run and will automatically select an alternate, backup copy of the superblock and attempt to recover the file system. The backup copies themselves are stored in block groups spread through the file system with the first stored at a 1 block offset from the start of the partition. This is important in the event that a manual recovery is necessary. You may view information about superblock backups with the command dumpe2fs /dev/foo | grep -i superblock which is useful in the event of a manual recovery attempt. Let us suppose that the dumpe2fs command outputs the line Backup superblock at 163840, Group descriptors at 163841-163841. We can use this information, and additional knowledge about the file system structure, to attempt to use this superblock backup: /sbin/fsck.ext3 -b 163840 -B 1024 /dev/foo. Please note that I have assumed a block size of 1024 bytes for this example.
An inode exists in, or on, a file system and represents metadata about a file. For clarity, all objects in a Linux or UNIX system are files; actual files, directories, devices, and so on. Please note that, among the metadata contained in an inode, there is no file name as humans think of it, this will be important later. An inode contains essentially information about ownership (user, group), access mode (read, write, execute permissions) and file type.
A dentry is the glue that holds inodes and files together by relating inode numbers to file names. Dentries also play a role in directory caching which, ideally, keeps the most frequently used files on-hand for faster access. File system traversal is another aspect of the dentry as it maintains a relationship between directories and their files.
A file, in addition to being what humans typically think of when presented with the word, is really just a block of logically related arbitrary data. Comparatively very dull considering all of the work done (above) to keep track of them.
I fully realize that a few sentences do not provide a full explanation of any of these concepts so please feel free to ask for additional details when and where necessary.
File
A file just means a bunch of bytes arranged in a certain order. It's what normal people call the contents of a file. When Linux opens a file, it also creates a file object, which holds data about where the file is stored and what processes are using it. The file object (but not the file data itself) is thrown away when the file is closed.
Inode
An inode (short for "index node") is a bunch of attributes about a file that Linux stores. There is one inode for each file (though with some filesystems, Linux has to create its own inodes because the information is spread around the filesystem). The inode stores information like who owns the file, how big the file is, and who is allowed to open the file. Each inode also contains a number unique to the filesystem partition; it's like a serial number for the file described by that inode.
Dentry
A dentry (short for "directory entry") is what the Linux kernel uses to keep track of the hierarchy of files in directories. Each dentry maps an inode number to a file name and a parent directory.
Superblock
The superblock is a unique data structure in a filesystem (though multiple copies exist to guard against corruption). The superblock holds metadata about the filesystem, like which inode is the top-level directory and the type of filesystem used.
In simplicity, dentry and inode are the same thing, an abstraction of files and directories. The differences between dentry and inode are that dentry is used to facilitate directory-specific operations, inode is just a collection of metadata about files and directories. Superblock is the abstraction of filesystem.
###########
lsof:
The manpage of lsof on my Debian system says “When +L is followed by a number, only files having a link count less than that number will be listed.”
i.e: lsof +L1 [ will display all the files that is having 0 ref count => deleted files. ]
########
The attributes as handled by lsattr/chattr on Linux and some of which can be stored by quite a few file systems (ext2/3/4, reiserfs, JFS, OCFS2, btrfs, XFS, nilfs2, hfsplus...) and even queried over CIFS/SMB (when with POSIX extensions) are flags. Just bits than can be turned on or off to disable or enable an attribute (like immutable or archive...). How they are stored is file system specific, but generally as a 16/32/64 bit record in the inode.
The full list of flags is found on Linux native filesystems (ext2/3/4, btrfs...) though not all of the flags apply to all of FS, and for other non-native FS, Linux tries to map them to equivalent features in the corresponding file system. For instance the simmutable flag as stored by OSX on HFS+ file systems is mapped to the corresponding immutable flag in Linux chattr. What flag is supported by what file system is hardly documented at all. Often, reading the kernel source code is the only option.
Extended attributes on the other hand, as set with setfattr or attr on Linux store more than flags. They are attached to a file as well, and are key/value pairs that can be (both key and value) arbitrary arrays of bytes (though with limitation of size on some file systems).
The key can be for instance: system.posix_acl_access or user.rsync.%stat. The system namespace is reserved for the system (you wouldn't change the POSIX ACLs with setfattr, but more with setfacl, POSIX ACLs just happen to be stored as extended attributes at least on some file systems), while the user namespace can be used by applications (here rsync uses it for its --fake-super option, to store information about ownership or permissions when you're not superuser).
Again, how they are stored is filesystem specific. See WikiPedia for more information.
##########
http://unix.stackexchange.com/questions/117093/find-where-inodes-are-being-used
############
Try this with GNU find:
find /start/dir -L -samefile /file/to/check -exec ls -li {} \;
Example output:
1234704 -rw-r--r-- 2 user1 user1 1134 2009-09-11 11:12 ./x1
1234704 -rw-r--r-- 2 user1 user1 1134 2009-09-11 11:12 ./x2
1234983 lrwxrwxrwx 1 user1 user1 2 2009-10-31 16:56 ./testx -> x1
2345059 lrwxrwxrwx 1 user2 user2 2 2010-01-03 16:17 ./x3 -> x1
Thursday, 3 April 2014
Log Analysis Tool
20 Free and Open Source Alternatives and Competitors of Splunk Log Analysis Tool
http://theprofessionalspoint.blogspot.in/2013/10/20-free-and-open-source-alternatives.html
Log Analysis Tool:
Log Analysis Tool
1. Scribe - Real time log aggregation used in Facebook
2. Logstash - Centralized log storage, indexing, and searching
3. Octopussy - Perl/XML Logs Analyzer, Alerter & Reporter
4. Awstats - Advanced web, streaming, ftp and mail server statistics
5. nxlog - Multi platform Log management
6. Graylog2 - Open Source Log Management
7. Fluentd - Data collector, Log Everything in JSON
8. Meniscus - The Python Event Logging Service
9. lucene-log4j - Log4j file rolling appender which indexes log with Lucene
10. Chainsaw - log viewer and analysis tool
11. Logsandra - log management using Cassandra
12. Clarity - Web interface for the grep
13. Webalizer - fast web server log file analysis
15. OtrosLogViewer - Log parser and Viewer
16. Kafka - A high-throughput distributed messaging system
17. Kibana - Web Interface for Logstash and ElasticSearch
18. Pylogdb - A Python-powered, column-oriented database suitable for web log analysis
19. Epylog - a Syslog parser
20. Indihiang - IIS and Apache log analyzing tool
http://theprofessionalspoint.blogspot.in/2013/10/20-free-and-open-source-alternatives.html
Log Analysis Tool:
Log Analysis Tool
1. Scribe - Real time log aggregation used in Facebook
2. Logstash - Centralized log storage, indexing, and searching
3. Octopussy - Perl/XML Logs Analyzer, Alerter & Reporter
4. Awstats - Advanced web, streaming, ftp and mail server statistics
5. nxlog - Multi platform Log management
6. Graylog2 - Open Source Log Management
7. Fluentd - Data collector, Log Everything in JSON
8. Meniscus - The Python Event Logging Service
9. lucene-log4j - Log4j file rolling appender which indexes log with Lucene
10. Chainsaw - log viewer and analysis tool
11. Logsandra - log management using Cassandra
12. Clarity - Web interface for the grep
13. Webalizer - fast web server log file analysis
15. OtrosLogViewer - Log parser and Viewer
16. Kafka - A high-throughput distributed messaging system
17. Kibana - Web Interface for Logstash and ElasticSearch
18. Pylogdb - A Python-powered, column-oriented database suitable for web log analysis
19. Epylog - a Syslog parser
20. Indihiang - IIS and Apache log analyzing tool
getconf - Query system configuration variables
getconf -a
LINK_MAX 65000
_POSIX_LINK_MAX 65000
MAX_CANON 255
_POSIX_MAX_CANON 255
MAX_INPUT 255
_POSIX_MAX_INPUT 255
NAME_MAX 255
_POSIX_NAME_MAX 255
PATH_MAX 4096
_POSIX_PATH_MAX 4096
PIPE_BUF 4096
_POSIX_PIPE_BUF 4096
SOCK_MAXBUF
_POSIX_ASYNC_IO
_POSIX_CHOWN_RESTRICTED 1
_POSIX_NO_TRUNC 1
_POSIX_PRIO_IO
_POSIX_SYNC_IO
_POSIX_VDISABLE 0
ARG_MAX 2097152
ATEXIT_MAX 2147483647
CHAR_BIT 8
CHAR_MAX 127
CHAR_MIN -128
CHILD_MAX 63014
CLK_TCK 100
INT_MAX 2147483647
INT_MIN -2147483648
IOV_MAX 1024
LOGNAME_MAX 256
LONG_BIT 64 #<------- System Arch [ 32 / 64 bit ]
MB_LEN_MAX 16
NGROUPS_MAX 65536
NL_ARGMAX 4096
NL_LANGMAX 2048
NL_MSGMAX 2147483647
NL_NMAX 2147483647
NL_SETMAX 2147483647
NL_TEXTMAX 2147483647
NSS_BUFLEN_GROUP 1024
NSS_BUFLEN_PASSWD 1024
NZERO 20
OPEN_MAX 1024
PAGESIZE 4096
PAGE_SIZE 4096
PASS_MAX 8192
PTHREAD_DESTRUCTOR_ITERATIONS 4
PTHREAD_KEYS_MAX 1024
PTHREAD_STACK_MIN 16384
PTHREAD_THREADS_MAX
SCHAR_MAX 127
SCHAR_MIN -128
SHRT_MAX 32767
SHRT_MIN -32768
SSIZE_MAX 32767
TTY_NAME_MAX 32
TZNAME_MAX 6
UCHAR_MAX 255
UINT_MAX 4294967295
UIO_MAXIOV 1024
ULONG_MAX 18446744073709551615
USHRT_MAX 65535
WORD_BIT 32
_AVPHYS_PAGES 812126
_NPROCESSORS_CONF 8
_NPROCESSORS_ONLN 8
_PHYS_PAGES 2021250
_POSIX_ARG_MAX 2097152
_POSIX_ASYNCHRONOUS_IO 200809
_POSIX_CHILD_MAX 63014
_POSIX_FSYNC 200809
_POSIX_JOB_CONTROL 1
_POSIX_MAPPED_FILES 200809
_POSIX_MEMLOCK 200809
_POSIX_MEMLOCK_RANGE 200809
_POSIX_MEMORY_PROTECTION 200809
_POSIX_MESSAGE_PASSING 200809
_POSIX_NGROUPS_MAX 65536
_POSIX_OPEN_MAX 1024
_POSIX_PII
_POSIX_PII_INTERNET
_POSIX_PII_INTERNET_DGRAM
_POSIX_PII_INTERNET_STREAM
_POSIX_PII_OSI
_POSIX_PII_OSI_CLTS
_POSIX_PII_OSI_COTS
_POSIX_PII_OSI_M
_POSIX_PII_SOCKET
_POSIX_PII_XTI
_POSIX_POLL
_POSIX_PRIORITIZED_IO 200809
_POSIX_PRIORITY_SCHEDULING 200809
_POSIX_REALTIME_SIGNALS 200809
_POSIX_SAVED_IDS 1
_POSIX_SELECT
_POSIX_SEMAPHORES 200809
_POSIX_SHARED_MEMORY_OBJECTS 200809
_POSIX_SSIZE_MAX 32767
_POSIX_STREAM_MAX 16
_POSIX_SYNCHRONIZED_IO 200809
_POSIX_THREADS 200809
_POSIX_THREAD_ATTR_STACKADDR 200809
_POSIX_THREAD_ATTR_STACKSIZE 200809
_POSIX_THREAD_PRIORITY_SCHEDULING 200809
_POSIX_THREAD_PRIO_INHERIT 200809
_POSIX_THREAD_PRIO_PROTECT 200809
_POSIX_THREAD_ROBUST_PRIO_INHERIT
_POSIX_THREAD_ROBUST_PRIO_PROTECT
_POSIX_THREAD_PROCESS_SHARED 200809
_POSIX_THREAD_SAFE_FUNCTIONS 200809
_POSIX_TIMERS 200809
TIMER_MAX
_POSIX_TZNAME_MAX 6
_POSIX_VERSION 200809
_T_IOV_MAX
_XOPEN_CRYPT 1
_XOPEN_ENH_I18N 1
_XOPEN_LEGACY 1
_XOPEN_REALTIME 1
_XOPEN_REALTIME_THREADS 1
_XOPEN_SHM 1
_XOPEN_UNIX 1
_XOPEN_VERSION 700
_XOPEN_XCU_VERSION 4
_XOPEN_XPG2 1
_XOPEN_XPG3 1
_XOPEN_XPG4 1
BC_BASE_MAX 99
BC_DIM_MAX 2048
BC_SCALE_MAX 99
BC_STRING_MAX 1000
CHARCLASS_NAME_MAX 2048
COLL_WEIGHTS_MAX 255
EQUIV_CLASS_MAX
EXPR_NEST_MAX 32
LINE_MAX 2048
POSIX2_BC_BASE_MAX 99
POSIX2_BC_DIM_MAX 2048
POSIX2_BC_SCALE_MAX 99
POSIX2_BC_STRING_MAX 1000
POSIX2_CHAR_TERM 200809
POSIX2_COLL_WEIGHTS_MAX 255
POSIX2_C_BIND 200809
POSIX2_C_DEV 200809
POSIX2_C_VERSION
POSIX2_EXPR_NEST_MAX 32
POSIX2_FORT_DEV
POSIX2_FORT_RUN
_POSIX2_LINE_MAX 2048
POSIX2_LINE_MAX 2048
POSIX2_LOCALEDEF 200809
POSIX2_RE_DUP_MAX 32767
POSIX2_SW_DEV 200809
POSIX2_UPE
POSIX2_VERSION 200809
RE_DUP_MAX 32767
PATH /bin:/usr/bin
CS_PATH /bin:/usr/bin
LFS_CFLAGS
LFS_LDFLAGS
LFS_LIBS
LFS_LINTFLAGS
LFS64_CFLAGS -D_LARGEFILE64_SOURCE
LFS64_LDFLAGS
LFS64_LIBS
LFS64_LINTFLAGS -D_LARGEFILE64_SOURCE
_XBS5_WIDTH_RESTRICTED_ENVS XBS5_LP64_OFF64
XBS5_WIDTH_RESTRICTED_ENVS XBS5_LP64_OFF64
_XBS5_ILP32_OFF32
XBS5_ILP32_OFF32_CFLAGS
XBS5_ILP32_OFF32_LDFLAGS
XBS5_ILP32_OFF32_LIBS
XBS5_ILP32_OFF32_LINTFLAGS
_XBS5_ILP32_OFFBIG
XBS5_ILP32_OFFBIG_CFLAGS
XBS5_ILP32_OFFBIG_LDFLAGS
XBS5_ILP32_OFFBIG_LIBS
XBS5_ILP32_OFFBIG_LINTFLAGS
_XBS5_LP64_OFF64 1
XBS5_LP64_OFF64_CFLAGS -m64
XBS5_LP64_OFF64_LDFLAGS -m64
XBS5_LP64_OFF64_LIBS
XBS5_LP64_OFF64_LINTFLAGS
_XBS5_LPBIG_OFFBIG
XBS5_LPBIG_OFFBIG_CFLAGS
XBS5_LPBIG_OFFBIG_LDFLAGS
XBS5_LPBIG_OFFBIG_LIBS
XBS5_LPBIG_OFFBIG_LINTFLAGS
_POSIX_V6_ILP32_OFF32
POSIX_V6_ILP32_OFF32_CFLAGS
POSIX_V6_ILP32_OFF32_LDFLAGS
POSIX_V6_ILP32_OFF32_LIBS
POSIX_V6_ILP32_OFF32_LINTFLAGS
_POSIX_V6_WIDTH_RESTRICTED_ENVS POSIX_V6_LP64_OFF64
POSIX_V6_WIDTH_RESTRICTED_ENVS POSIX_V6_LP64_OFF64
_POSIX_V6_ILP32_OFFBIG
POSIX_V6_ILP32_OFFBIG_CFLAGS
POSIX_V6_ILP32_OFFBIG_LDFLAGS
POSIX_V6_ILP32_OFFBIG_LIBS
POSIX_V6_ILP32_OFFBIG_LINTFLAGS
_POSIX_V6_LP64_OFF64 1
POSIX_V6_LP64_OFF64_CFLAGS -m64
POSIX_V6_LP64_OFF64_LDFLAGS -m64
POSIX_V6_LP64_OFF64_LIBS
POSIX_V6_LP64_OFF64_LINTFLAGS
_POSIX_V6_LPBIG_OFFBIG
POSIX_V6_LPBIG_OFFBIG_CFLAGS
POSIX_V6_LPBIG_OFFBIG_LDFLAGS
POSIX_V6_LPBIG_OFFBIG_LIBS
POSIX_V6_LPBIG_OFFBIG_LINTFLAGS
_POSIX_V7_ILP32_OFF32
POSIX_V7_ILP32_OFF32_CFLAGS
POSIX_V7_ILP32_OFF32_LDFLAGS
POSIX_V7_ILP32_OFF32_LIBS
POSIX_V7_ILP32_OFF32_LINTFLAGS
_POSIX_V7_WIDTH_RESTRICTED_ENVS POSIX_V7_LP64_OFF64
POSIX_V7_WIDTH_RESTRICTED_ENVS POSIX_V7_LP64_OFF64
_POSIX_V7_ILP32_OFFBIG
POSIX_V7_ILP32_OFFBIG_CFLAGS
POSIX_V7_ILP32_OFFBIG_LDFLAGS
POSIX_V7_ILP32_OFFBIG_LIBS
POSIX_V7_ILP32_OFFBIG_LINTFLAGS
_POSIX_V7_LP64_OFF64 1
POSIX_V7_LP64_OFF64_CFLAGS -m64
POSIX_V7_LP64_OFF64_LDFLAGS -m64
POSIX_V7_LP64_OFF64_LIBS
POSIX_V7_LP64_OFF64_LINTFLAGS
_POSIX_V7_LPBIG_OFFBIG
POSIX_V7_LPBIG_OFFBIG_CFLAGS
POSIX_V7_LPBIG_OFFBIG_LDFLAGS
POSIX_V7_LPBIG_OFFBIG_LIBS
POSIX_V7_LPBIG_OFFBIG_LINTFLAGS
_POSIX_ADVISORY_INFO 200809
_POSIX_BARRIERS 200809
_POSIX_BASE
_POSIX_C_LANG_SUPPORT
_POSIX_C_LANG_SUPPORT_R
_POSIX_CLOCK_SELECTION 200809
_POSIX_CPUTIME 200809
_POSIX_THREAD_CPUTIME 200809
_POSIX_DEVICE_SPECIFIC
_POSIX_DEVICE_SPECIFIC_R
_POSIX_FD_MGMT
_POSIX_FIFO
_POSIX_PIPE
_POSIX_FILE_ATTRIBUTES
_POSIX_FILE_LOCKING
_POSIX_FILE_SYSTEM
_POSIX_MONOTONIC_CLOCK 200809
_POSIX_MULTI_PROCESS
_POSIX_SINGLE_PROCESS
_POSIX_NETWORKING
_POSIX_READER_WRITER_LOCKS 200809
_POSIX_SPIN_LOCKS 200809
_POSIX_REGEXP 1
_REGEX_VERSION
_POSIX_SHELL 1
_POSIX_SIGNALS
_POSIX_SPAWN 200809
_POSIX_SPORADIC_SERVER
_POSIX_THREAD_SPORADIC_SERVER
_POSIX_SYSTEM_DATABASE
_POSIX_SYSTEM_DATABASE_R
_POSIX_TIMEOUTS 200809
_POSIX_TYPED_MEMORY_OBJECTS
_POSIX_USER_GROUPS
_POSIX_USER_GROUPS_R
POSIX2_PBS
POSIX2_PBS_ACCOUNTING
POSIX2_PBS_LOCATE
POSIX2_PBS_TRACK
POSIX2_PBS_MESSAGE
SYMLOOP_MAX
STREAM_MAX 16
AIO_LISTIO_MAX
AIO_MAX
AIO_PRIO_DELTA_MAX 20
DELAYTIMER_MAX 2147483647
HOST_NAME_MAX 64
LOGIN_NAME_MAX 256
MQ_OPEN_MAX
MQ_PRIO_MAX 32768
_POSIX_DEVICE_IO
_POSIX_TRACE
_POSIX_TRACE_EVENT_FILTER
_POSIX_TRACE_INHERIT
_POSIX_TRACE_LOG
RTSIG_MAX 32
SEM_NSEMS_MAX
SEM_VALUE_MAX 2147483647
SIGQUEUE_MAX 63014
FILESIZEBITS 64
POSIX_ALLOC_SIZE_MIN 4096
POSIX_REC_INCR_XFER_SIZE
POSIX_REC_MAX_XFER_SIZE
POSIX_REC_MIN_XFER_SIZE 4096
POSIX_REC_XFER_ALIGN 4096
SYMLINK_MAX
GNU_LIBC_VERSION glibc 2.17
GNU_LIBPTHREAD_VERSION NPTL 2.17
POSIX2_SYMLINKS 1
LEVEL1_ICACHE_SIZE 32768
LEVEL1_ICACHE_ASSOC 8
LEVEL1_ICACHE_LINESIZE 64
LEVEL1_DCACHE_SIZE 32768
LEVEL1_DCACHE_ASSOC 8
LEVEL1_DCACHE_LINESIZE 64
LEVEL2_CACHE_SIZE 262144
LEVEL2_CACHE_ASSOC 8
LEVEL2_CACHE_LINESIZE 64
LEVEL3_CACHE_SIZE 6291456
LEVEL3_CACHE_ASSOC 12
LEVEL3_CACHE_LINESIZE 64
LEVEL4_CACHE_SIZE 0
LEVEL4_CACHE_ASSOC 0
LEVEL4_CACHE_LINESIZE 0
IPV6 200809
RAW_SOCKETS 200809
Subscribe to:
Posts (Atom)