mysql Notes1:
001. To Login to mysql: By default there is no username and password require for normal login:
$ mysql [ This will give you the mysql prompt like: mysql> ]
$ mysql -u mysql [ login as mysql user, by default mysql user don't have any password ]
002. To list the databases;
$ show databases;
NOTE: Every mysql command need to finish with ; or with \G
NOTE: [ This will list the databases on your DataBase Management System -> DBMS ]
003. Log in as mysql root user:
$ mysql -u root
004. To change the root password: [ By default root don't have password. ] [ Login to mysql, first. ]
mysql> UPDATE mysql.user SET Password = PASSWORD('fooBar') WHERE User = 'root';
mysql> FLUSH PRIVILEGES;
NOTE:
a. mysql is the database name and user is the table where the user's informations are there.
b. PASSWORD is the function that encript the password.
c. There are more then one root by default: [ localhost, hostname and so on... like ipv4 and like that. ]
d. FLUSH PRIVILEGES [ To read the mysql permission. ]
005. Creating a user:
mysql> CREATE USER username@Hostname
mysql> CREATE USER apache@localhost;
mysql> CREATE USER rootBackup@localhost;
NOTE: This will allow that user only from that Host.
006. Granting the access:
mysql> GRANT SELECT, INSERT, UPDATE, DELETE, CREATE, DROP, FILE, INDEX, ALTER, CREATE TEMPORARY TABLES, CREATE VIEW, SHOW VIEW, CREATE ROUTINE, ALTER ROUTINE, EXECUTE ON *.* TO apache@localhost;
mysql> GRANT ALL ON *.* TO rootBackup@localhost WITH GRANT OPTION;
mysql> FLUSH PRIVILEGES;
007. Now setting the password:
mysql> UPDATE mysql.user SET Password = PASSWORD('fooBar') WHERE user = 'apache';
mysql> UPDATE mysql.user SET Password = PASSWORD('fooBar') WHERE user = 'rootBackup';
Sunday 26 January 2014
Monday 20 January 2014
Nagios nrpe with argument passing
Quick HOW TO STEPS:
On monitored Host:
1. On /etc/nagios/nrpe.cfg file check the value from "0" to "1" at dont_blame_nrpe=0
2. Define the command at [ nrpe.cfg ], such that it will except the arg values.
3. Restart the nrpe server services, after you do some change.
From Nagios Server:
1. Do a recheck with check_nrpe command and check all are working fine.
[ e.g ] check_nrpe -H MonitoredHost -c check_users1 -a 1 2
2. Define the service: [ having the ARG values ]
define service{
hostgroup_name all
service_description Current Users
check_command check_nrpe!check_users_with_ARG -a 10 15
use generic-service ; Name of service template to use
notification_interval 0 ; set > 0 if you want to be renotified
}
3. Restart the nagios server service.
### With Few more details:
########### Following is a check where we have not used npre or nsca ##########
# Define a service to check the load on the local machine.
define service{
use generic-service ; Name of service template to use
host_name amitAsus
service_description Current Load
check_command check_load!5.0!4.0!3.0!10.0!6.0!4.0
}
######### Following is a check where we have use nrpe ############
# Define a service to check the remote mysql partation information on the remote machine.
define service{
host_name mysqlDB1,mysqlDB2
service_description check_mysql_partition
check_command check_nrpe_1arg!check_mysql_partition
use generic-service-15 ; Name of service template to use
notification_interval 0 ; set > 0 if you want to be renotified
}
################################################
######## What is this check_nrpe_1arg ##########
# Default path where the defination is there:
/etc/nagios-plugins/config
$ cat check_nrpe.cfg
# this command runs a program $ARG1$ with arguments $ARG2$
define command {
command_name check_nrpe
command_line /usr/lib/nagios/plugins/check_nrpe -H $HOSTADDRESS$ -c $ARG1$ -a $ARG2$
}
# this command runs a program $ARG1$ with no arguments
define command {
command_name check_nrpe_1arg
command_line /usr/lib/nagios/plugins/check_nrpe -H $HOSTADDRESS$ -c $ARG1$
}
# NOTE: Original command stay: most of the time at the following location:
/usr/lib/nagios/plugins/check_nrpe
## check_nrpe -help
$ check_nrpe --help
NRPE Plugin for Nagios
Copyright (c) 1999-2008 Ethan Galstad (nagios@nagios.org)
Version: 2.12
Last Modified: 03-10-2008
License: GPL v2 with exemptions (-l for more info)
SSL/TLS Available: Anonymous DH Mode, OpenSSL 0.9.6 or higher required
Usage: check_nrpe -H <host> [-n] [-u] [-p <port>] [-t <timeout>] [-c <command>] [-a <arglist...>]
Options:
-n = Do no use SSL
-u = Make socket timeouts return an UNKNOWN state instead of CRITICAL
<host> = The address of the host running the NRPE daemon
[port] = The port on which the daemon is running (default=5666)
[timeout] = Number of seconds before connection times out (default=10)
[command] = The name of the command that the remote daemon should run
[arglist] = Optional arguments that should be passed to the command. Multiple
arguments should be separated by a space. If provided, this must be
the last option supplied on the command line.
-h,--help Print this short help.
-l,--license Print licensing information.
-n,--no-ssl Do not initial an ssl handshake with the server, talk in plaintext.
Note:
This plugin requires that you have the NRPE daemon running on the remote host.
You must also have configured the daemon to associate a specific plugin command
with the [command] option you are specifying here. Upon receipt of the
[command] argument, the NRPE daemon will run the appropriate plugin command and
send the plugin output and return code back to *this* plugin. This allows you
to execute plugins on remote hosts and 'fake' the results to make Nagios think
the plugin is being run locally.
### Following is an example of nrpe.cfg under /etc/nagios folder:
# Common Standard checks
command[check_ssh]=/usr/lib/nagios/plugins/check_ssh -H localhost -p 1717
command[check_users]=/usr/lib/nagios/plugins/check_users -w 10 -c 15
command[check_all_disks]=/usr/lib/nagios/plugins/check_disk -w 20% -c 10%
### Passing arguments user check_nrpe
# Following is the error you will get if you have not enable the argument passing at the monitored Host:
NagiosServer:/etc/nagios-plugins/config$ check_nrpe -H MonitoredHost -c check_disk -a 60 80 /dev/xvda1
CHECK_NRPE: Received 0 bytes from daemon. Check the remote server logs for error messages.
# from /var/log/syslog:
Jan 20 12:17:21 MonitoredHost nrpe[8527]: Error: Request contained command arguments, but argument option is not enabled!
Jan 20 12:17:21 MonitoredHost nrpe[8527]: Client request was invalid, bailing out...
### So, you have to enable the following at the Monitored Host:
### File: /etc/nagios/nrpe.cfg [ e.g. Example on ubuntu system, and following with line number example ]
So, at line number [94] you have to change dont_blame_nrpe, from "0" to "1" .
NOTE: Don't forget to restart your nrpe server services after changing the value and the command should been defined at the monitored host.
82 # COMMAND ARGUMENT PROCESSING
83 # This option determines whether or not the NRPE daemon will allow clients
84 # to specify arguments to commands that are executed. This option only works
85 # if the daemon was configured with the --enable-command-args configure script
86 # option.
87 #
88 # *** ENABLING THIS OPTION IS A SECURITY RISK! ***
89 # Read the SECURITY file for information on some of the security implications
90 # of enabling this variable.
91 #
92 # Values: 0=do not allow arguments, 1=allow command arguments
93
94 dont_blame_nrpe=0
NOTE: The above line should be configured as " dont_blame_nrpe=1 " to have arg enabled.
NOTE: The command at the monitored host's nrpe.cfg file should also configured to receive the arguments:
Following is an example that need to be there at the nrpe.cfg file at the monitored host:
command[check_users1]=/usr/lib/nagios/plugins/check_users -w $ARG1$ -c $ARG2$
command[check_total_procs1]=/usr/lib/nagios/plugins/check_procs -w $ARG1$ -c $ARG2$
And from Nagios server: you can run the following command. [ Note: only 1 (-a) and then ARG1 ARG2 and so on.. , no need to have another -a before ARG2, if you put more then 1 -a then you will get error. ]
user@nagiosServer:/etc/nagios-plugins/config$ check_nrpe -H MonitoredHost -c check_users1 -a 1 2
USERS OK - 1 users currently logged in |users=1;1;2;0
user@nagiosSever:/etc/nagios-plugins/config$ check_nrpe -H MonitoredHost -c check_total_procs1 -a 80 100
PROCS CRITICAL: 112 processes
user@nagiosSever:/etc/nagios-plugins/config$ check_nrpe -H MonitoredHost -c check_total_procs1 -a 200 300
PROCS OK: 110 processes
# The following examples use hardcoded command arguments...
# The following examples allow user-supplied arguments and can
# only be used if the NRPE daemon was compiled with support for
# command arguments *AND* the dont_blame_nrpe directive in this
# config file is set to '1'. This poses a potential security risk, so
# make sure you read the SECURITY file before doing this.
#command[check_users]=/usr/lib/nagios/plugins/check_users -w $ARG1$ -c $ARG2$
#command[check_load]=/usr/lib/nagios/plugins/check_load -w $ARG1$ -c $ARG2$
#command[check_disk]=/usr/lib/nagios/plugins/check_disk -w $ARG1$ -c $ARG2$ -p $ARG3$
#command[check_procs]=/usr/lib/nagios/plugins/check_procs -w $ARG1$ -c $ARG2$ -s $ARG3$
Hope, you liked the example over here, on how to enable $ARGn$ option at the Monitored Host's nrpe.cfg file and how to use the same on the check_npre command.
NOTE:
Following is one of the example, from your nagios server, where you can define your service along with ARG value, for the command that is defined on the monitored host.
define service{
hostgroup_name all
service_description Current Users
check_command check_nrpe!check_users_with_ARG -a 10 15
use generic-service ; Name of service template to use
notification_interval 0 ; set > 0 if you want to be renotified
}
In the monitored host you have the same command define as per the define server where the command also defined and that is tacking the arg values.
command[check_users_with_ARG]=/usr/lib/nagios/plugins/check_users -w $ARG1$ -c $ARG2$
Wednesday 15 January 2014
Apache Logging Module mod-log-firstbyte
How to install mod-log-firstbyte:
https://code.google.com/p/mod-log-firstbyte/
1. checkout the same project:
# Non-members may check out a read-only working copy anonymously over HTTP.
svn checkout http://mod-log-firstbyte.googlecode.com/svn/trunk/ mod-log-firstbyte-read-only
2. readme file:
Compile and install the module using apxs2, e.g.
apxs2 -c mod_log_firstbyte.c
apxs2 -i -a mod_log_firstbyte.la
This will add a line to your configuration file to load the module:
LoadModule log_firstbyte_module /path/to/mod_log_firstbyte.so
The module adds a new option to your log format string: '%F'. This is the request-to-first-byte time, in microseconds. Edit your configuration files to include a new LogFormat directive. I usually use:
LogFormat "%h %l %u %t \"%r\" %>s %b \"%{Referer}i\" \"%{User-Agent}i\" %D %F" combined-with-firstbyte
###### My custom with epoch time too ######
LogFormat "%{%s}t %D %F %h %l %u %t \"%r\" %>s %b \"%{Referer}i\" \"%{User-Agent}i\" " combined-with-firstbyte
%D The time taken to serve the request, in microseconds.
Sample output:
<epoch> <%D> <%F> => 1389809102 1471 1080
1389809102 1471 1080 127.0.0.1 - - [15/Jan/2014:18:05:02 +0000] "GET /server-status?auto HTTP/1.1" 200 373 "-" "libwww-perl/6.03"
######## End of My custom log #####
Finally, change the CustomLog directive:
CustomLog /var/log/apache2/access.log combined-with-firstbyte
3. Install the following cheks:
sudo apt-get install apache2-prefork-dev
sudo apt-get install apache2-threaded-dev
4.
sudo apxs2 -c mod_log_firstbyte.c
sudo apxs2 -i -a mod_log_firstbyte.la
5. After updating the config files [2] restart the apache.
Ref Links:
https://code.google.com/p/mod-log-firstbyte/
http://httpd.apache.org/docs/current/mod/mod_log_config.html
6. Installation output:
-----------------------
$ sudo apxs2 -c mod_log_firstbyte.c
/usr/share/apr-1.0/build/libtool --silent --mode=compile --tag=disable-static x86_64-linux-gnu-gcc -prefer-pic -DLINUX=2 -D_FORTIFY_SOURCE=2 -D_GNU_SOURCE -D_REENTRANT -I/usr/include/apr-1.0 -I/usr/include/openssl -I/usr/include/xmltok -pthread -I/usr/include/apache2 -I/usr/include/apr-1.0 -I/usr/include/apr-1.0 -c -o mod_log_firstbyte.lo mod_log_firstbyte.c && touch mod_log_firstbyte.slo
/usr/share/apr-1.0/build/libtool --silent --mode=link --tag=disable-static x86_64-linux-gnu-gcc -o mod_log_firstbyte.la -rpath /usr/lib/apache2/modules -module -avoid-version mod_log_firstbyte.lo
$ sudo apxs2 -i -a mod_log_firstbyte.la
/usr/share/apache2/build/instdso.sh SH_LIBTOOL='/usr/share/apr-1.0/build/libtool' mod_log_firstbyte.la /usr/lib/apache2/modules
/usr/share/apr-1.0/build/libtool --mode=install cp mod_log_firstbyte.la /usr/lib/apache2/modules/
libtool: install: cp .libs/mod_log_firstbyte.so /usr/lib/apache2/modules/mod_log_firstbyte.so
libtool: install: cp .libs/mod_log_firstbyte.lai /usr/lib/apache2/modules/mod_log_firstbyte.la
libtool: finish: PATH="/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/sbin" ldconfig -n /usr/lib/apache2/modules
----------------------------------------------------------------------
Libraries have been installed in:
/usr/lib/apache2/modules
If you ever happen to want to link against installed libraries
in a given directory, LIBDIR, you must either use libtool, and
specify the full pathname of the library, or use the `-LLIBDIR'
flag during linking and do at least one of the following:
- add LIBDIR to the `LD_LIBRARY_PATH' environment variable
during execution
- add LIBDIR to the `LD_RUN_PATH' environment variable
during linking
- use the `-Wl,-rpath -Wl,LIBDIR' linker flag
- have your system administrator add LIBDIR to `/etc/ld.so.conf'
See any operating system documentation about shared libraries for
more information, such as the ld(1) and ld.so(8) manual pages.
----------------------------------------------------------------------
chmod 644 /usr/lib/apache2/modules/mod_log_firstbyte.so
[preparing module `log_firstbyte' in /etc/apache2/mods-available/log_firstbyte.load]
Enabling module log_firstbyte.
To activate the new configuration, you need to run:
service apache2 restart
How to install other apache modules:
amit@amit:~$ sudo apt-get install libapache2-
Display all 116 possibilities? (y or n)
libapache2-authcassimple-perl libapache2-mod-php5filter
libapache2-authcookie-perl libapache2-mod-proxy-html
libapache2-authenntlm-perl libapache2-mod-python
libapache2-mod-apparmor libapache2-mod-python-doc
libapache2-mod-apreq2 libapache2-mod-qos
libapache2-mod-auth-cas libapache2-mod-random
libapache2-mod-auth-kerb libapache2-mod-removeip
libapache2-mod-auth-memcookie libapache2-mod-rivet
libapache2-mod-auth-mysql libapache2-mod-rivet-doc
libapache2-mod-authn-sasl libapache2-mod-rpaf
libapache2-mod-auth-ntlm-winbind libapache2-mod-ruby
libapache2-mod-authn-webid libapache2-mod-ruid2
libapache2-mod-authn-yubikey libapache2-mod-ruwsgi
libapache2-mod-authnz-external libapache2-mod-ruwsgi-dbg
libapache2-mod-auth-openid libapache2-mod-scgi
libapache2-mod-auth-pam libapache2-modsecurity
libapache2-mod-auth-pgsql libapache2-mod-shib2
libapache2-mod-auth-plain libapache2-mod-spamhaus
libapache2-mod-auth-pubtkt libapache2-mod-speedycgi
libapache2-mod-auth-radius libapache2-mod-suphp
libapache2-mod-auth-sys-group libapache2-mod-upload-progress
libapache2-mod-auth-tkt libapache2-mod-uwsgi
libapache2-mod-authz-unixgroup libapache2-mod-uwsgi-dbg
libapache2-mod-axis2c libapache2-mod-vhost-hash-alias
libapache2-mod-bw libapache2-mod-vhost-ldap
libapache2-mod-dacs libapache2-mod-wsgi
libapache2-mod-defensible libapache2-mod-wsgi-py3
libapache2-mod-dnssd libapache2-mod-xsendfile
libapache2-mod-encoding libapache2-reload-perl
https://help.ubuntu.com/13.04/serverguide/httpd.html
https://code.google.com/p/mod-log-firstbyte/
1. checkout the same project:
# Non-members may check out a read-only working copy anonymously over HTTP.
svn checkout http://mod-log-firstbyte.googlecode.com/svn/trunk/ mod-log-firstbyte-read-only
2. readme file:
Compile and install the module using apxs2, e.g.
apxs2 -c mod_log_firstbyte.c
apxs2 -i -a mod_log_firstbyte.la
This will add a line to your configuration file to load the module:
LoadModule log_firstbyte_module /path/to/mod_log_firstbyte.so
The module adds a new option to your log format string: '%F'. This is the request-to-first-byte time, in microseconds. Edit your configuration files to include a new LogFormat directive. I usually use:
LogFormat "%h %l %u %t \"%r\" %>s %b \"%{Referer}i\" \"%{User-Agent}i\" %D %F" combined-with-firstbyte
###### My custom with epoch time too ######
LogFormat "%{%s}t %D %F %h %l %u %t \"%r\" %>s %b \"%{Referer}i\" \"%{User-Agent}i\" " combined-with-firstbyte
%D The time taken to serve the request, in microseconds.
Sample output:
<epoch> <%D> <%F> => 1389809102 1471 1080
1389809102 1471 1080 127.0.0.1 - - [15/Jan/2014:18:05:02 +0000] "GET /server-status?auto HTTP/1.1" 200 373 "-" "libwww-perl/6.03"
######## End of My custom log #####
Finally, change the CustomLog directive:
CustomLog /var/log/apache2/access.log combined-with-firstbyte
3. Install the following cheks:
sudo apt-get install apache2-prefork-dev
sudo apt-get install apache2-threaded-dev
4.
sudo apxs2 -c mod_log_firstbyte.c
sudo apxs2 -i -a mod_log_firstbyte.la
5. After updating the config files [2] restart the apache.
Ref Links:
https://code.google.com/p/mod-log-firstbyte/
http://httpd.apache.org/docs/current/mod/mod_log_config.html
6. Installation output:
-----------------------
$ sudo apxs2 -c mod_log_firstbyte.c
/usr/share/apr-1.0/build/libtool --silent --mode=compile --tag=disable-static x86_64-linux-gnu-gcc -prefer-pic -DLINUX=2 -D_FORTIFY_SOURCE=2 -D_GNU_SOURCE -D_REENTRANT -I/usr/include/apr-1.0 -I/usr/include/openssl -I/usr/include/xmltok -pthread -I/usr/include/apache2 -I/usr/include/apr-1.0 -I/usr/include/apr-1.0 -c -o mod_log_firstbyte.lo mod_log_firstbyte.c && touch mod_log_firstbyte.slo
/usr/share/apr-1.0/build/libtool --silent --mode=link --tag=disable-static x86_64-linux-gnu-gcc -o mod_log_firstbyte.la -rpath /usr/lib/apache2/modules -module -avoid-version mod_log_firstbyte.lo
$ sudo apxs2 -i -a mod_log_firstbyte.la
/usr/share/apache2/build/instdso.sh SH_LIBTOOL='/usr/share/apr-1.0/build/libtool' mod_log_firstbyte.la /usr/lib/apache2/modules
/usr/share/apr-1.0/build/libtool --mode=install cp mod_log_firstbyte.la /usr/lib/apache2/modules/
libtool: install: cp .libs/mod_log_firstbyte.so /usr/lib/apache2/modules/mod_log_firstbyte.so
libtool: install: cp .libs/mod_log_firstbyte.lai /usr/lib/apache2/modules/mod_log_firstbyte.la
libtool: finish: PATH="/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/sbin" ldconfig -n /usr/lib/apache2/modules
----------------------------------------------------------------------
Libraries have been installed in:
/usr/lib/apache2/modules
If you ever happen to want to link against installed libraries
in a given directory, LIBDIR, you must either use libtool, and
specify the full pathname of the library, or use the `-LLIBDIR'
flag during linking and do at least one of the following:
- add LIBDIR to the `LD_LIBRARY_PATH' environment variable
during execution
- add LIBDIR to the `LD_RUN_PATH' environment variable
during linking
- use the `-Wl,-rpath -Wl,LIBDIR' linker flag
- have your system administrator add LIBDIR to `/etc/ld.so.conf'
See any operating system documentation about shared libraries for
more information, such as the ld(1) and ld.so(8) manual pages.
----------------------------------------------------------------------
chmod 644 /usr/lib/apache2/modules/mod_log_firstbyte.so
[preparing module `log_firstbyte' in /etc/apache2/mods-available/log_firstbyte.load]
Enabling module log_firstbyte.
To activate the new configuration, you need to run:
service apache2 restart
How to install other apache modules:
amit@amit:~$ sudo apt-get install libapache2-
Display all 116 possibilities? (y or n)
libapache2-authcassimple-perl libapache2-mod-php5filter
libapache2-authcookie-perl libapache2-mod-proxy-html
libapache2-authenntlm-perl libapache2-mod-python
libapache2-mod-apparmor libapache2-mod-python-doc
libapache2-mod-apreq2 libapache2-mod-qos
libapache2-mod-auth-cas libapache2-mod-random
libapache2-mod-auth-kerb libapache2-mod-removeip
libapache2-mod-auth-memcookie libapache2-mod-rivet
libapache2-mod-auth-mysql libapache2-mod-rivet-doc
libapache2-mod-authn-sasl libapache2-mod-rpaf
libapache2-mod-auth-ntlm-winbind libapache2-mod-ruby
libapache2-mod-authn-webid libapache2-mod-ruid2
libapache2-mod-authn-yubikey libapache2-mod-ruwsgi
libapache2-mod-authnz-external libapache2-mod-ruwsgi-dbg
libapache2-mod-auth-openid libapache2-mod-scgi
libapache2-mod-auth-pam libapache2-modsecurity
libapache2-mod-auth-pgsql libapache2-mod-shib2
libapache2-mod-auth-plain libapache2-mod-spamhaus
libapache2-mod-auth-pubtkt libapache2-mod-speedycgi
libapache2-mod-auth-radius libapache2-mod-suphp
libapache2-mod-auth-sys-group libapache2-mod-upload-progress
libapache2-mod-auth-tkt libapache2-mod-uwsgi
libapache2-mod-authz-unixgroup libapache2-mod-uwsgi-dbg
libapache2-mod-axis2c libapache2-mod-vhost-hash-alias
libapache2-mod-bw libapache2-mod-vhost-ldap
libapache2-mod-dacs libapache2-mod-wsgi
libapache2-mod-defensible libapache2-mod-wsgi-py3
libapache2-mod-dnssd libapache2-mod-xsendfile
libapache2-mod-encoding libapache2-reload-perl
https://help.ubuntu.com/13.04/serverguide/httpd.html
Monday 13 January 2014
Website Hardening
Some notes that we can think of fine tuning to keep our website faster or more reliable:
1:
The availability kernel sockets:
Linux Increase Local Port Range with net.ipv4.ip_local_port_range
If your Linux server is opening lots of outgoing network connection, you need to increase local port range. By default range is small. For example squid proxy server can come under fire if it runs out of ports.
You can use sysctl command to to modify kernel parameters at runtime. The parameters available are those listed under /proc/sys/. Please note that this hack is only useful for high bandwidth, busy Linux servers or large scale grid servers.
Set new local port range:
# echo 1024 65535 > /proc/sys/net/ipv4/ip_local_port_range
Or:
$ sudo sysctl -w net.ipv4.ip_local_port_range="1024 64000"
or edit : " /etc/sysctl.conf" file:
# increase system IP port limits
net.ipv4.ip_local_port_range = 1024 65535
You must restart your network for the change to take effect.
2:
Better manage your TCP/IP resources
This hack just make the time default values for TCP/IP connection lower so that more connections can be handled by time on your TCP/IP protocol. The following will decrease the amount of time your Linux box will try take to finish closing a connection and the amount of time before it will kill a stale connection. This will also turn off some IP extensions that aren't needed. The default setup for the TCP/IP parameters we'll change under Red Hat Linux are:
For the tcp_fin_timeout 180
For the tcp_keepalive_time 7200
For the tcp_window_scaling 1
For the tcp_sack 1
For the tcp_timestamps 1
To adjust the new TCP/IP values, type the following commands on your terminal:
Edit the /etc/sysctl.conf file and add the following lines:
# Decrease the time default value for tcp_fin_timeout connection
net.ipv4.tcp_fin_timeout = 30
# Decrease the time default value for tcp_keepalive_time connection
net.ipv4.tcp_keepalive_time = 1800
# Turn off the tcp_window_scaling
net.ipv4.tcp_window_scaling = 0
# Turn off the tcp_sack
net.ipv4.tcp_sack = 0
# Turn off the tcp_timestamps
net.ipv4.tcp_timestamps = 0
You must restart your network for the change to take effect.
Ref Links:
http://www.cyberciti.biz/tips/linux-increase-outgoing-network-sockets-range.html
http://www.faqs.org/docs/securing/chap6sec75.html
1:
The availability kernel sockets:
Linux Increase Local Port Range with net.ipv4.ip_local_port_range
If your Linux server is opening lots of outgoing network connection, you need to increase local port range. By default range is small. For example squid proxy server can come under fire if it runs out of ports.
You can use sysctl command to to modify kernel parameters at runtime. The parameters available are those listed under /proc/sys/. Please note that this hack is only useful for high bandwidth, busy Linux servers or large scale grid servers.
Set new local port range:
# echo 1024 65535 > /proc/sys/net/ipv4/ip_local_port_range
Or:
$ sudo sysctl -w net.ipv4.ip_local_port_range="1024 64000"
or edit : " /etc/sysctl.conf" file:
# increase system IP port limits
net.ipv4.ip_local_port_range = 1024 65535
You must restart your network for the change to take effect.
2:
Better manage your TCP/IP resources
This hack just make the time default values for TCP/IP connection lower so that more connections can be handled by time on your TCP/IP protocol. The following will decrease the amount of time your Linux box will try take to finish closing a connection and the amount of time before it will kill a stale connection. This will also turn off some IP extensions that aren't needed. The default setup for the TCP/IP parameters we'll change under Red Hat Linux are:
For the tcp_fin_timeout 180
For the tcp_keepalive_time 7200
For the tcp_window_scaling 1
For the tcp_sack 1
For the tcp_timestamps 1
To adjust the new TCP/IP values, type the following commands on your terminal:
Edit the /etc/sysctl.conf file and add the following lines:
# Decrease the time default value for tcp_fin_timeout connection
net.ipv4.tcp_fin_timeout = 30
# Decrease the time default value for tcp_keepalive_time connection
net.ipv4.tcp_keepalive_time = 1800
# Turn off the tcp_window_scaling
net.ipv4.tcp_window_scaling = 0
# Turn off the tcp_sack
net.ipv4.tcp_sack = 0
# Turn off the tcp_timestamps
net.ipv4.tcp_timestamps = 0
You must restart your network for the change to take effect.
Ref Links:
http://www.cyberciti.biz/tips/linux-increase-outgoing-network-sockets-range.html
http://www.faqs.org/docs/securing/chap6sec75.html
Friday 3 January 2014
Apache Per Second Request Count And Plotting That Data With GnuPlot
Over here I will explain how can get your Apache's http server's per second request count and plot the same data using the gnuplot.
While I was trying to get per second request count, parsing the apache httpd access log to plot with the standard apache log format, I found some difficulty, as the log was getting written to the log file in async format, So though of adding "epoch" time on the log and do the next things. Although you can see this status with server-status module [ http://amitmund.blogspot.in/2013/12/apachehttpservers-server-status.html ]
What I did to add the epoch time:
I am using the following LogFormat [ enabled at my /etc/apache2/sites-enabled/000-default ] at my site:
CustomLog "|/usr/sbin/rotatelogs /mnt/httpd/logs/access_log.%Y-%m-%d-%H_%M_%S 100M" combined
While I was trying to get per second request count, parsing the apache httpd access log to plot with the standard apache log format, I found some difficulty, as the log was getting written to the log file in async format, So though of adding "epoch" time on the log and do the next things. Although you can see this status with server-status module [ http://amitmund.blogspot.in/2013/12/apachehttpservers-server-status.html ]
What I did to add the epoch time:
I am using the following LogFormat [ enabled at my /etc/apache2/sites-enabled/000-default ] at my site:
CustomLog "|/usr/sbin/rotatelogs /mnt/httpd/logs/access_log.%Y-%m-%d-%H_%M_%S 100M" combined
So, I have to update the same "combined" format at my apache2.conf file, and I have added the following format at my LogFormat, so that it will record the epoch time.
NOTE: Following line %{s}t will add epoch time at the log
LogFormat "%{%s}t %h %l %u %t \"%r\" %>s %O \"%{Referer}i\" \"%{User-Agent}i\"" combined
Before that I had a log as:
10.210.193.192 - - [26/Dec/2013:10:16:25 +0000] "GET / HTTP/1.1" 200 517 "-" "check_http/v1.4.14 (nagios-plugins 1.4.14)"
Now I can see a epoch time added before the above log format:
1388744705 10.210.193.192 - - [26/Dec/2013:10:16:25 +0000] "GET / HTTP/1.1" 200 517 "-" "check_http/v1.4.14 (nagios-plugins 1.4.14)"
Further details about the log format:
http://httpd.apache.org/docs/2.2/mod/mod_log_config.html
Few of the useful options:
%{%s}t : epoch time
%{format}t : The time, in the form given by format, which should be in
strftime(3) format. (potentially localized)
%D : The time taken to serve the request, in microseconds.
%f : Filename
% : The request method
%s : Status. For requests that got internally redirected, this is
the status of the *original* request --- %>s
for the last.
%T : The time taken to serve the request, in seconds.
After updating the logFormat, I did a apache restart and cross checked to see the update format and found its working fine.
Getting Data:
After the log format got updated, I did a grep of the epoch time and redirected to a file.
$ awk '{print $1}' access-file > epochTime.dat
The above command will take the epoch time from the access-file and put the same at epochTime.dat file. [ Over here I have added epoch time at the 1st field, so I have used the $1 over here.
This is useful if you are doing a benchmark of your web server and want to know and plot how many apache request it is serving every second.
Using above command, you will find the data like following:
$ head epochTime.dat
1388745807
1388745807
1388745807
1388745807
1388745807
1388745807
1388745807
1388745807
Now you have to get the count of the same epoch time: so,
$ cat epochTime.dat | sort | uniq -c
By above command you will see the output as:
2 1388745779
400 1388745780
1016 1388745781
1048 1388745782
1057 1388745783
1063 1388745784
440 1388745785
Now I did:
awk '{print $2,$1}' epochTime.dat > epochTime1.dat
After the above command you can use the above data to plot graph easyly:
Now you get the data something like following:
$ head epochTime1.dat
1388745779 2
1388745780 400
1388745781 1016
1388745782 1048
1388745783 1057
1388745784 1063
1388745785 440
Now I have the data How I will plot the same:
I am using the gnuplot to plot the above data. [ other gnuplot example: http://amitmund.blogspot.in/2014/01/gnuplot.html ]
$ vi PerSecondReq.gplot: # the gnu plot code to plot the data
set xdata time
set timefmt "%s"
set xlabel "TIME"
set ylabel "Req"
plot "epochTime1.dat" using 1:2 title "Req" with lines
Now You have your plotted data.[ Following is an example of the same. ]
Thursday 2 January 2014
gnuplot
Here is a very basic notes on the gnuplot to plot your system sar information.
1: You can plot any data output but for example I am using "sar" command to plot sysstem's cpu information.
NOTE, that you can specify any column over here if your output data is not proper. But the data file must content the number values only.
2: If you don't have gnuplot install, then go ahead and install the same.
For sar: [ sudo apt-get install sysstat ]
For gnuplot: [ sudo apt-get install gnuplot ]
Over here I have updated the sar command with alias, so that you should see the time at 24 hour.
alias sar='LANG=C sar'
NOTE: You can have a gplot file that can carry all the commands that you want.
NOTE: The file extention must be " .gplot "
Example of the .gplot file:
vi cpuinfo.gplot
set xdata time
set timefmt "%H:%M:%S"
set xlabel "TIME"
set ylabel "CPU"
plot "cpuinfo.dat" using 1:3 title "User%" with lines
replot "cpuinfo.dat" using 1:5 title "System%" with lines
replot "cpuinfo.dat" using 1:8 title "Idle%" with lines
# On the above file: we are plotting column number 3, 5 and 8"
If you need to plot more then one value then you need to use "replot" and not the "plot" command.
NOTE: as part of example I have the "cpuinfo.dat" file and "cpuinfo.gplot" at the same location, and you can give any file name what ever you want but the plot script need to have ".gplot" is there.
To get the data, you just need to run the following command. [ NOTE: -persist option will keep the plot open untill you close the same. ]
gnuplot -persist cpuinfo.gplot
On the command line also you can run the same command under "gnuplot" prompt and get the data:
$ gnuplot
gnuplot> set xdata time
gnuplot> set timefmt "%H:%M:%S"
gnuplot> set xlabel "Time"
gnuplot> set ylabel "CPU"gnuplot> plot "cpuinfo.dat" using 1:3 title "User%" with lines
gnuplot> replot "cpuinfo.dat" using 1:5 title "System%" with lines
gnuplot> replot "cpuinfo.dat" using 1:8 title "Idle%" with lines
Very Imp:
You can not plot non numeric data, so you have to delete the lines that have nun numeric data.
Example the default output of sar might be as:
$ sar -u 1 5 > cpuinfo.dat
Linux 3.8.0-34-generic (amitAsus) 01/02/14 _x86_64_ (8 CPU)
19:46:33 CPU %user %nice %system %iowait %steal %idle
19:46:34 all 3.52 0.00 1.26 0.00 0.00 95.23
19:46:35 all 4.01 0.00 1.00 0.00 0.00 94.99
19:46:36 all 7.66 0.00 1.01 0.00 0.00 91.33
19:46:37 all 5.28 0.00 0.63 0.00 0.00 94.09
19:46:38 all 4.03 0.00 1.13 0.00 0.00 94.84
Average: all 4.90 0.00 1.00 0.00 0.00 94.10
In the above file you need to delete the first line, the last time and the line where we have the header info.
Example:
1: You can plot any data output but for example I am using "sar" command to plot sysstem's cpu information.
NOTE, that you can specify any column over here if your output data is not proper. But the data file must content the number values only.
2: If you don't have gnuplot install, then go ahead and install the same.
For sar: [ sudo apt-get install sysstat ]
For gnuplot: [ sudo apt-get install gnuplot ]
Over here I have updated the sar command with alias, so that you should see the time at 24 hour.
alias sar='LANG=C sar'
NOTE: You can have a gplot file that can carry all the commands that you want.
NOTE: The file extention must be " .gplot "
Example of the .gplot file:
vi cpuinfo.gplot
set xdata time
set timefmt "%H:%M:%S"
set xlabel "TIME"
set ylabel "CPU"
plot "cpuinfo.dat" using 1:3 title "User%" with lines
replot "cpuinfo.dat" using 1:5 title "System%" with lines
replot "cpuinfo.dat" using 1:8 title "Idle%" with lines
# On the above file: we are plotting column number 3, 5 and 8"
If you need to plot more then one value then you need to use "replot" and not the "plot" command.
NOTE: as part of example I have the "cpuinfo.dat" file and "cpuinfo.gplot" at the same location, and you can give any file name what ever you want but the plot script need to have ".gplot" is there.
To get the data, you just need to run the following command. [ NOTE: -persist option will keep the plot open untill you close the same. ]
gnuplot -persist cpuinfo.gplot
On the command line also you can run the same command under "gnuplot" prompt and get the data:
$ gnuplot
gnuplot> set xdata time
gnuplot> set timefmt "%H:%M:%S"
gnuplot> set xlabel "Time"
gnuplot> set ylabel "CPU"gnuplot> plot "cpuinfo.dat" using 1:3 title "User%" with lines
gnuplot> replot "cpuinfo.dat" using 1:5 title "System%" with lines
gnuplot> replot "cpuinfo.dat" using 1:8 title "Idle%" with lines
Very Imp:
You can not plot non numeric data, so you have to delete the lines that have nun numeric data.
Example the default output of sar might be as:
$ sar -u 1 5 > cpuinfo.dat
Linux 3.8.0-34-generic (amitAsus) 01/02/14 _x86_64_ (8 CPU)
19:46:33 CPU %user %nice %system %iowait %steal %idle
19:46:34 all 3.52 0.00 1.26 0.00 0.00 95.23
19:46:35 all 4.01 0.00 1.00 0.00 0.00 94.99
19:46:36 all 7.66 0.00 1.01 0.00 0.00 91.33
19:46:37 all 5.28 0.00 0.63 0.00 0.00 94.09
19:46:38 all 4.03 0.00 1.13 0.00 0.00 94.84
Average: all 4.90 0.00 1.00 0.00 0.00 94.10
In the above file you need to delete the first line, the last time and the line where we have the header info.
Example:
Subscribe to:
Posts (Atom)