Thursday 27 November 2014

reading a file using golang

Example 1:

$cat run readFile.go 

package main

import(
"io/ioutil"
)

func main() {
contents,_ := ioutil.ReadFile("readFile.go")
println(string(contents))
}




Example 2:

$ cat fileRead.go 

package main

import(
"io/ioutil"
)

func main() {
contents,_ := ioutil.ReadFile("/etc/passwd")
println(string(contents))
}

go installation and build with cross platform compilation

On this note:

1. We do the installation of go on the mac
2. Then create a simple hello world program
3. Execute and test
4. Build the code [ for this example its for mac]
5. Setup cross platform build



Installation of go [ for this example on mac ]

  • For detail information: link: http://golang.org/doc/install
  • installed from the tar.gz, but you can try from mac pkg file too.
  • created a directory "go" under "/usr/local"
  • downloaded go1.3.3.darwin-amd64-osx10.8.tar.gz from https://golang.org/dl/
  • copied the file to /usr/local folder.
  • change the ownership or give the full access to your current user for /usr/local/go folder
  • extract the your download file go1.3.3.darwin-amd64-osx10.8.tar.gz it will extract everything to your /usr/local/go folder
  • update your PATH with adding /usr/local/go/bin
  • type go and you should see the go usage output. [ it mean you have successfully install go :) ]

Create a simple hello world program


package main
import "fmt"
func main() {
fmt.Printf("hello, world\n")
}

 Execute and test

$ go run hello.go

hello, world


Build the code [ for this example its for mac]

$go build -o hello-mac hello.go

The above command will create an mac executable file.

$file hello-mac
hello-mac: Mach-O 64-bit executable x86_64

now you can run the code directly to the mac os:
$ ./hello-mac
hello, world


Setting up cross build

Further detail information: Link:
http://dave.cheney.net/2013/07/09/an-introduction-to-cross-compilation-with-go-1-1

$ cd /usr/local/go/scr
./all.bash [ if you get some error, try with sudo , just incase if you don't have write access. ]


https://github.com/davecheney/golang-crosscompile

$ git clone git://github.com/davecheney/golang-crosscompile.git
$ source golang-crosscompile/crosscompile.bash
[ if you get some error, try with sudo , just incase if you don't have write access. ]

go-crosscompile-build-all
[ if you get some error, try with sudo , just incase if you don't have write access. ]



This will compile the Go runtime and standard library for each platform. You can see these packages if you look in go/pkg.

% ls -1 go/pkg
darwin_386
darwin_amd64
freebsd_386
freebsd_amd64
linux_386
linux_amd64
linux_arm
obj
tool
windows_386
windows_amd64

And in your shell you can find all the following commands:

go                         go-darwin-386              go-linux-386               go-windows-386
go-all                     go-darwin-amd64            go-linux-amd64             go-windows-amd64
go-build-all               go-freebsd-386             go-linux-arm               godoc
go-crosscompile-build      go-freebsd-amd64           go-openbsd-386             gofmt
go-crosscompile-build-all  go-freebsd-arm             go-openbsd-amd64



go-build-all hello.go [ This will build your code for all the platform :) ]

and you can see all your build file:
hello-darwin-386 hello-freebsd-arm hello-mac hello-windows-amd64
hello-darwin-amd64 hello-linux-386 hello-openbsd-386 hello.go
hello-freebsd-386 hello-linux-amd64 hello-openbsd-amd64
hello-freebsd-amd64 hello-linux-arm hello-windows-386

I have tested and found, I can able to run the code in different arch.
NOTE: for windows build file, rename them to .exe before running them.


hello-darwin-386:    Mach-O executable i386
hello-darwin-amd64:  Mach-O 64-bit executable x86_64
hello-freebsd-386:   ELF 32-bit LSB executable, Intel 80386, version 1 (FreeBSD), statically linked, not stripped
hello-freebsd-amd64: ELF 64-bit LSB executable, x86-64, version 1 (FreeBSD), statically linked, not stripped
hello-freebsd-arm:   ELF 32-bit LSB executable, ARM, version 1 (FreeBSD), statically linked, not stripped
hello-linux-386:     ELF 32-bit LSB executable, Intel 80386, version 1 (SYSV), statically linked, not stripped
hello-linux-amd64:   ELF 64-bit LSB executable, x86-64, version 1 (SYSV), statically linked, not stripped
hello-linux-arm:     ELF 32-bit LSB executable, ARM, version 1 (SYSV), statically linked, not stripped
hello-openbsd-386:   ELF 32-bit LSB executable, Intel 80386, version 1 (OpenBSD), statically linked, for OpenBSD, not stripped
hello-openbsd-amd64: ELF 64-bit LSB executable, x86-64, version 1 (OpenBSD), statically linked, for OpenBSD, not stripped
hello-windows-386:   PE32 executable for MS Windows (console) Intel 80386 32-bit
hello-windows-amd64: PE32+ executable for MS Windows (console) Mono/.Net assembly
hello.go:            ASCII Java program text  <---- For me at mac it shows like this :)


Friday 21 November 2014

simple Dockerfile example1

1. cat Dockerfile

FROM cassandra
ADD start.sh /start.sh # to copy the file into
CMD ["bash", "./start.sh"]
EXPOSE 9160
EXPOSE 9042
EXPOSE 7000
EXPOSE 44478
EXPOSE 7199


2. cat start.sh
#!/bin/bash
service cassandra start
while true; do sleep 60; done  # Not sure any other best way for now

3. 
docker build -t="runcassandra" .     # this will create an image named runcassandra

4.
docker run -d -P --name cass1 runcassandra

NOTE: The cassandra will be up and will listen to rest of the port. But key thing is when you use EXPOSE, these ports get use for container to container communication ***ONLY***,  as per my current understanding, and this is what I found from the documentation too.


Wednesday 19 November 2014

copy file from host to the docker container

copy file from host to the docker container:

Steps:

1: Get container name or short container id :
  1a. docker ps
  1b. Get full container id
docker inspect -f '{{.Id}}' SHORT_CONTAINER_ID-or-CONTAINER_NAME

  1c. copy file :
  sudo cp path-file-host /var/lib/docker/aufs/mnt/FULL_CONTAINER_ID/PATH-NEW-FILE


EXAMPLE :

$docker ps

CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
450b4b6a87e7 amit/ssh:latest /usr/sbin/sshd -D cranky_pare

$docker inspect -f '{{.Id}}' cranky_pare

or

$docker inspect -f '{{.Id}}' 450b4b6a87e7
450b4b6a87e734ba68d1c077de5d3d9ac617b1b9b6e829fe28c917abf1dce44f

From your docker server:
$ sudo cp file.txt /var/lib/docker/aufs/mnt/450b4b6a87e734ba68d1c077de5d3d9ac617b1b9b6e829fe28c917abf1dce44f/root/file.txt


docker networking with hack part1

I found the easy way of having multiple container running on the same host, bind with the host IP, then assign multiple IP the same interface of the host.

1. How do I bind multiple IP to the same interface:

amund@prod-docker-app01:~$ cat /etc/network/interfaces
# This file describes the network interfaces available on your system
# and how to activate them. For more information, see interfaces(5).

# The loopback network interface
auto lo
iface lo inet loopback

# The primary network interface
auto eth0
iface eth0 inet static
address 172.22.19.75
netmask 255.255.255.0
network 172.22.19.0
broadcast 172.22.19.255
gateway 172.22.19.1
# dns-* options are implemented by the resolvconf package, if installed
dns-nameservers 172.17.0.10 172.17.0.14 172.27.2.20
dns-search example.com example.net example.in

# bind-ed one assigned IP
auto eth0:0
iface eth0:0 inet static
        address 172.22.19.76
        netmask 255.255.255.0
        network 172.22.19.0
        broadcast 172.22.19.255
        gateway 172.22.19.1
        # dns-* options are implemented by the resolvconf package, if installed
dns-nameservers 172.17.0.10 172.17.0.14 172.27.2.20
dns-search example.com example.net example.in

# bind-ed second assigned IP
auto eth0:1
iface eth0:1 inet static
        address 172.22.19.77
        netmask 255.255.255.0
        network 172.22.19.0
        broadcast 172.22.19.255
        gateway 172.22.19.1
        # dns-* options are implemented by the resolvconf package, if installed
dns-nameservers 172.17.0.10 172.17.0.14 172.27.2.20
dns-search example.com example.net example.in

# bind-ed thired assigned IP
auto eth0:2
iface eth0:2 inet static
        address 172.22.19.78
        netmask 255.255.255.0
        network 172.22.19.0
        broadcast 172.22.19.255
        gateway 172.22.19.1
        # dns-* options are implemented by the resolvconf package, if installed
dns-nameservers 172.17.0.10 172.17.0.14 172.27.2.20
dns-search example.com example.net example.in

2. If you are using remote host, good to reboot once and check your network update.


Thursday 30 October 2014

Docker design doc1


A sample docker design doc, how we can implement docker in the production environment. Will update other docker command later in the blog.

The ports are just for example. Note: we can run the keepalived and haproxy on the same server where docker is running, just for simplicity I draw at different server.

Following image's original pdf link:
https://drive.google.com/file/d/0B02LZ59YLdSdaTBlVGdmX0RlWU0/view?usp=sharing



Thursday 23 October 2014

vimrc

my quick ~/.vimrc

set hlsearch
syntax enable
set tabstop=4
set softtabstop=4
set expandtab
set showcmd
set cursorline
filetype indent on
set showmatch
set incsearch

Wednesday 8 October 2014

nc - [ netcat ] - unix command

nc

0)

Copy a complete directory [ lets say a mysql data " but you should not take a running mysql data " ]

Go to destination computer's destination EMPTY directory where you want to dump the data and run:[[[  nc -l 3333 | tar -zxvf -   ]]] - Note you can use any free port at the place of 3333

now go to the source computer and type: [[[ tar cvz <directory> | nc destination_hostname 3333 ]]]

1)

Destination host:

nc -lp [3333] > [file]     [ listen to port 3333 and what ever comes from that port put that to <<file>>


Sources host:
cat <<file>> | nc -w 1 IP <<3333>>   [ to transfer a <<file>> of course host with netcat with having timeout of 1 sec " -w 1 " and on port of 3333


2)
man nc:

-l      Used to specify that nc should listen for an incoming connection rather than initiate a connection to a remote host.  It is an error to use this option in conjunction
             with the -p, -s, or -z options.  Additionally, any timeouts specified with the -w option are ignored.

-p source_port
             Specifies the source port nc should use, subject to privilege restrictions and availability.  It is an error to use this option in conjunction with the -l option.


-w timeout
             If a connection and stdin are idle for more than timeout seconds, then the connection is silently closed.  The -w flag has no effect on the -l option, i.e. nc will
             listen forever for a connection, with or without the -w flag.  The default is no timeout.


3)
CLIENT/SERVER MODEL
     It is quite simple to build a very basic client/server model using nc.  On one console, start nc listening on a specific port for a connection.  For example:

           $ nc -l 1234

     nc is now listening on port 1234 for a connection.  On a second console (or a second machine), connect to the machine and port being listened on:

           $ nc 127.0.0.1 1234

     There should now be a connection between the ports.  Anything typed at the second console will be concatenated to the first, and vice-versa.  After the connection has been set
     up, nc does not really care which side is being used as a ‘server’ and which side is being used as a ‘client’.  The connection may be terminated using an EOF (‘^D’).


4)
DATA TRANSFER
     The example in the previous section can be expanded to build a basic data transfer model.  Any information input into one end of the connection will be output to the other
     end, and input and output can be easily captured in order to emulate file transfer.

     Start by using nc to listen on a specific port, with output captured into a file:

           $ nc -l 1234 > filename.out

     Using a second machine, connect to the listening nc process, feeding it the file which is to be transferred:

           $ nc host.example.com 1234 < filename.in

     After the file has been transferred, the connection will close automatically.


5)
TALKING TO SERVERS
     It is sometimes useful to talk to servers “by hand” rather than through a user interface.  It can aid in troubleshooting, when it might be necessary to verify what data a
     server is sending in response to commands issued by the client.  For example, to retrieve the home page of a web site:

           $ echo -n "GET / HTTP/1.0\r\n\r\n" | nc host.example.com 80

     Note that this also displays the headers sent by the web server.  They can be filtered, using a tool such as sed(1), if necessary.

     More complicated examples can be built up when the user knows the format of requests required by the server.  As another example, an email may be submitted to an SMTP server
     using:

           $ nc localhost 25 << EOF
           HELO host.example.com
           MAIL FROM: <user@host.example.com>
           RCPT TO: <user2@host.example.com>
           DATA
           Body of email.
           .
           QUIT
           EOF



6)
PORT SCANNING
     It may be useful to know which ports are open and running services on a target machine.  The -z flag can be used to tell nc to report open ports, rather than initiate a con-
     nection.  For example:

           $ nc -z host.example.com 20-30
           Connection to host.example.com 22 port [tcp/ssh] succeeded!
           Connection to host.example.com 25 port [tcp/smtp] succeeded!

     The port range was specified to limit the search to ports 20 - 30.

     Alternatively, it might be useful to know which server software is running, and which versions.  This information is often contained within the greeting banners.  In order to
     retrieve these, it is necessary to first make a connection, and then break the connection when the banner has been retrieved.  This can be accomplished by specifying a small
     timeout with the -w flag, or perhaps by issuing a "QUIT" command to the server:

           $ echo "QUIT" | nc host.example.com 20-30
           SSH-1.99-OpenSSH_3.6.1p2
           Protocol mismatch.
           220 host.example.com IMS SMTP Receiver Version 0.84 Ready

7)
EXAMPLES
     Open a TCP connection to port 42 of host.example.com, using port 31337 as the source port, with a timeout of 5 seconds:

           $ nc -p 31337 -w 5 host.example.com 42

     Open a UDP connection to port 53 of host.example.com:

           $ nc -u host.example.com 53

     Open a TCP connection to port 42 of host.example.com using 10.1.2.3 as the IP for the local end of the connection:

           $ nc -s 10.1.2.3 host.example.com 42

     Create and listen on a Unix Domain Socket:

           $ nc -lU /var/tmp/dsocket

     Connect to port 42 of host.example.com via an HTTP proxy at 10.2.3.4, port 8080.  
This example could also be used by ssh(1); see the ProxyCommand directive in ssh_config(5)
     for more information.

           $ nc -x10.2.3.4:8080 -Xconnect host.example.com 42


8) Minimal web-server example:
You can set up netcat to act as a very basic webserver which can just serve one file:
while `netcat -lp 8080 -c 'echo HTTP/1.0 200 OK';echo;cat file`;do;done


9) local port forwarding:
This command would forward every request on port 8080 to port 80:
while `netcat -lp 8080 -c 'netcat localhost 80'`;do;done


10) getting telnet command output using nc: [ example memcache ]

printf 'stats\n' | nc 127.0.0.1 11211 | grep limit_maxbytes

Tuesday 26 August 2014

quick perl note.

1.
#!/usr/bin/perl -w

print "Content-type: text/html\n\n";

# This will print "Hello, World"
print "Hello, world\n";

print("Hello, world\n");
print "Hello, world to all\n";



2. 

#!/usr/bin/perl
print "Content-type: text/html\n\n";

$a = 10;
print "Value of a = $a\n";
print 'Value of a = $a\n';



3.
#!/usr/bin/perl -w

print "Content-type: text/html\n\n";

$a = 10;
$var = <<"EOF";
This is the syntax for here document and it will continue
until it encounters a EOF in the first line.
This is case of double quote so variable value will be
interpolated. For example value of a = $a
EOF
print "$var\n";

$var = <<'EOF';
This is case of single quote so variable value will not be
interpolated. For example value of a = $a
EOF
print "$var\n";




4.
#!/usr/bin/perl -w

$result = "This is \"number\"\n";
print "$result",' adding...';
print "\n\$result\n";




5.

#!/usr/bin/perl

use strict;
use warnings;

my $name = "A useful tool in natural language processing is concordance. This allows a specific string to be displayed in its immediate context whereever it appears in a text. For example, a concordance program identifying the target string the might produce some of the following output. Notice how the occurrences of the target string line up vertically. ";
my @person = split(/ /, $name);

#print "$person[0]\n"

foreach my $val (@person) {
    print "$val\n"
}

exit 0;




6.
#!/usr/bin/perl -w

use strict;

# multi dimention array:

my @array1=(
#    ["1","2", ["Amit"]],
    ["1","2"],
    ["3","4"],
    ["5","6"]
    );

#print "$array1[0][2][0]\n";
print "$array1[0][0]\n";

# multi dimention array referance:

my $array1ref = [
["1","2"],
["3","4"],
["5","6"]
];

print "$array1ref->[0][0]\n";




7.

#!/usr/bin/perl
use strict;
use warnings;

open (fh2, "file1") || die "Error opening file1: $!";
#open (fh2, "/etc/passwd") || die "Error opening file1: $!";

my @array1;

while (<fh2>){
    push  @array1, [ split ]; # push function to push the data in array1 from $_
    #push  @array1, [ split/:/];
}

foreach (@array1){
    print "@$_[0]\n";
    print "@$_[1]\n";
    #print "@$_\n";
}




8.


#!/usr/bin/perl
use strict;
use warnings;

open (fh2, "passwd") || die "Error opening file1: $!";

my @array1;

while (<fh2>){
    push  @array1, [ split/:/];
}

foreach (@array1){
    print "@$_";
}

print "\nPrinting an extra line: $array1[0][0]\n";





9.

#!/usr/bin/perl
# Author: Amit Mund
# Date: 17.07.2014
# Purpose: Explain Lists, Slices, Ranges, etc.

use warnings;
use strict;

# qw : quote word

#Scalars(single values), Arrays(lists), Hashes(key/value pairs)

print (qw(Boston Charlotte newark miami));  # This is removing all the blank space.
print "\n";
print (qw(Boston Charlotte newark miami)[0,3]); # will print 0th and 3rd index value. [Bootonmiami]
print "\n";
print (qw(Boston Charlotte newark miami)[0..2]); # will print the index values from 0 to 3
print "\n\n";


##

my @uscities = qw(Boston Charlotte newark miami Austin Dallas Houston);
print "@uscities[1..$#uscities]\n";

my @eastcoastcities = @uscities[0..3,5];

print ("East Cost Cities: ","@eastcoastcities\n");
print ("Other Cities:", "@uscities[4..$#uscities]\n\n");



10.

 #!/usr/bin/perl -wl

use strict;

# you can use the same name in a different type of variables
#scalars = single values ($)
#Arrays = lists of scalars (@)
# Hashes = key/values pairs

# NOTE: $fname != @fname and we can have both at the same time.

my $carmanufactures = "This is scalar values";
my @carmanufactures = ("Honda","Toyota","Nissan", "Lexus","BMW"); # begining with 0

print "$carmanufactures";

print "$carmanufactures[0]";
print "$carmanufactures[1]";
print "$carmanufactures[2]";
print "$carmanufactures[3]";

# perl keep the working data into memory.

print "This arrays contents: 0 to $#carmanufactures, mean total $#carmanufactures+1";

# Note: leangth of arrays using $#arrayname

print "Here is the full arrays: @carmanufactures[0,1,2,3,4]";
# following also works
print "Here is the full arrays: @carmanufactures[0..4]";
# I believe the best way is following in these 3.
print "Here is the full arrays: @carmanufactures[0..$#carmanufactures]";



11.
#!/usr/bin/perl
use warnings;
use strict;


# NOTE: For interger use ==, <, >, <=, >=
if (1 < 2 && (5 < 4 || 5 > 4)) {

    print "I am at if block.\n";

}

## string test:

# NOTE: for string eq, ne
# if, elsif, else, unless  [elsif, to use within a sub-if block.]

my $fname = "amit";
my $name = "amit";

if ( $name ne $fname) {

    print "$fname is same as $name\n";
}
else {

    print "I am at else block.\n";
}


# unless is for negetation:
my $value1 = "1";
my $value2 = "2";

unless ($value1 == $value2) {

    print "$value1 is not equal to $value2\n";
}


# one line if test:

print "one liner if\n" if $value1 != $value2;

#END



12. fileio1.pl

#!/usr/bin/perl
use warnings;
use strict;

# open function for file handel.
# die function to cache any error.
# $! is the perl variable to keep the error.

my $min = 1;
my $max = 20;
my $fname = "Amit mund";
my $dob = "23/04/1982";

# defining OUTFILE as a fh.
open (OUTFILE, ">data1") || die "problems: $!"; # $! perl inbuilt variable for error return.
for ($min..$max) {
    print OUTFILE "$fname and DOB = ";
    print OUTFILE "$dob\n"
}


#END


13. fileio2.pl

#!/usr/bin/perl
use warnings;
use strict;

#my $OUT = "OUTFILE";
#my $filename = "/proc/cpuinfo";
my $filename = "data1";

open (INFILE, "$filename") || die "problems: $!";

#while (<INFILE>){
#    print "$_";
#}

my @data1contains = <INFILE>; # assoicate a file to an array.

foreach (@data1contains){
    print "$_";
}

#END



14. FileIO3.pl

#!/usr/bin/perl
use warnings;

$IN = "INFILE";
$OUT = "OUTFILE";
$filenamein = "/proc/cpuinfo";
$filenameout = "data2";

open ($IN, "$filenamein") || die "error while opening the file: $!";
open ($OUT, ">$filenameout") || die "error while writing the file: $!";

@data1contents = <$IN>;
foreach (@data1contents){
    if (/^processor/ || /^core id/ || /model name/){                # Nesting an if statement with regex
    #print $OUT "$_";
    s/model name/cpu/;                                                # Replace "model name with cpu"
    @cur_data=$_;
    print $OUT "@cur_data";
    }
}

#END




15. File04.pl

#!/usr/bin/perl

use warnings;

my $PROCESS;

open ($PROCESS, "ps aux | grep -i apache |") || die "process error $!";

@PROCESS_DATA = <$PROCESS>;
#print "@PROCESS_DATA"
foreach (@PROCESS_DATA){
    print "$_"
}

# END



16. fileIO04A.pl

#!/usr/bin/perl

use warnings;

my $PROCESS;

open ($PROCESS, "ps aux|") || die "process error $!"; # need to end with | pipe as per the input for $PROCESS.

@PROCESS_DATA = <$PROCESS>;
foreach (@PROCESS_DATA){
    if (/apache/){                # <----- Regex.
            print "$_"

    }
}



17. FileIO4_B.pl

#!/usr/bin/perl
use warnings;

$FH = "FileReadWrite";
$FILENAME = "data1";
$APPEND_OF_FILE = "Appended string\n";

# +< for read and append mode, and it must be in quote.
open ($FH, "+<$FILENAME") || die "file error: $!";
@dataOfFile = <$FH>;
foreach (@dataOfFile){
    #print $FH "$_";
}
print $FH "$APPEND_OF_FILE";




18. FileIO05.pl

#!/usr/bin/perl
use warnings;

#$DIR = "/home/amit/Documents/perl";
$DIR = "/etc";

$DH = "Handle";

opendir ($DH, "$DIR") || die "Error opening dir: $!";
@dirlist = readdir($DH); # list the first level of contents.

foreach (@dirlist){
    #if (/^[x-z]/){    # regex of start with x-z letters.
    #    print "$_\n";
    #}
    print "$_\n";
}
#closedir ($DH); # Good to close the handler.

#@dirlist = `ls -A $DIR`;
#foreach (@dirlist){
#    print "$_\n";
#}

#END




19.

#!/usr/bin/perl
use warnings;
#use strict;

#my $mem = perl -wln -e '@mem1 = split /:/, $_; if (/DirectMap2M/){print $mem1[1]} ;' /proc/meminfo
my $MEM_FH1 = "memory_FH";
my $mem_file = "/proc/meminfo";

open ($MEM_FH1, "$mem_file") || die "problem opening file: $!";

my @mem_data_contains = <$MEM_FH1>;

foreach(@mem_data_contains){
    if (/^MemFree/){
        #my @mem_value = split /:/, @mem_data_contains;
        my @mem_free_value = split /\s+/,$_;
        my @mem_free_value1 = $mem_free_value[1] * 1024; # in bite conversion
        print "Free Memory = @mem_free_value1\n";
    #print @mem_data_contains;
    }
}

foreach(@mem_data_contains){
    if (/^MemTotal/){
        #my @mem_value = split /:/, @mem_data_contains;
        my @mem_total_value = split /\s+/,$_;
        my @mem_total_value1 = $mem_total_value[1] * 1024; # in bite conversion
        print "Total Memory = @mem_total_value1\n";
    #print @mem_data_contains;
    }
}

#my $Free_percent =


20.

#!/bin/bash/perl
use strict;
use warnings;

#my $min = 1;
#my $max = 10;
#my $PROD_NAME = "Testing the perl foreach loop.";
my @array1 = ("Amit", "Kumar", "Mund", "Testing");

foreach (@array1) { # just need to pass the array value.
    # body...
    print "$_ ";    # $_ is an internal variable that keep the complete value.
}

$#array1 +=1;
print "\nTotal arrays element = $#array1";
print "\n";



21.

#!/usr/bin/perl -wl

use strict;

# Note: in hash it need to even number of objects.
my %make_model = (
    "Honda","city",
    "Maruti","swift"
    );


print $make_model{"Maruti"};

#### The nice way of writing the hash:

my $player = "Venus";

my %player_country = (
    Venus => "USA",
    Sharapova => "Russia",
    );

printf "$player represents: ";
print $player_country{"$player"};

###

my @num_of_players_keys = keys %player_country;
my @num_of_players_values = values %player_country;

print "The keys are: @num_of_players_keys[0..$#num_of_players_keys]";
print "The values are: @num_of_players_values[0..$#num_of_players_values]";



22.

#!/usr/bin/perl
  #
  # The traditional first program.

  # Strict and warnings are recommended.
  use strict;
  use warnings;

  # Print a message.
  print "Content-type: text/html\n\n";
  print "Hello, World!\n";

=begin comment
This is all part of multiline comment.
You can use as many lines as you like
These comments will be ignored by the
compiler until the next =cut is encountered.
=cut

=begin comment
for multiline comment it start with: " =begin" and end with "cut" after
the =
=cut

print "Hello, world\n";
print 'Hello, world\n';




23.

#!/usr/bin/perl

print "Content-type: text/html\n\n";

my $str = <<EOM;
<html>
<head>
<title>CGI Perl Example</title>
</head>
<body>
<h1>CGI Perl Example</h1>
<p>CGI Perl Example</p>
</body>
</html>
EOM

print $str



24. perldb.pl

#!/usr/bin/perl

use warnings;
use strict;
use DBI;

#Setp1 - creating connection object

my $dsn = 'DBI:mysql:thenewboston'; # module:db_engine:db_name
my $user = 'root';
my $pass = 'YourPassword';
my $host = 'localhost';

my $conn = DBI->connect($dsn,$user,$pass) || die "Error connecting" . DBI->errstr;

# step 2 - define query
my $query1 = $conn->prepare('SELECT id,name FROM user') || die "Error preparing query" . $conn->errstr;

# setp 3 - execute query
$query1->execute || die "Error executing query" . $conn->errstr;

# step 4 - return results
my @results;

while (@results = $query1->fetchrow_array()) {
    my $id = $results[0];
    my $name = $results[1];
    #my $name = $results[2];
    print "$id,$name\n"
}


# trying for another query too:

my $query2 = $conn->prepare('SHOW TABLES') || die "Error preparing query" . $conn->errstr;
$query2->execute || die "Error executing query" . $conn->errstr;

my @results2;

print "\nFollowing are the tables at thenewboston DB:\n";
while (@results2 = $query2->fetchrow_array()) {
    my $tables = $results2[0];
    print "$tables\n"
}




25.

#!/usr/bin/perl

use warnings;
use strict;

my @array1 = ("1","2","3","4");

print "@array1\n";

# pop function, to remove the last value.
my $ppop = pop @array1; # over here $ppop will be carring the removed value.
print "@array1\n";
print "$ppop\n"; # printing the remoed value.

# NOTE: if you don;t want you can just pop the @array1 and print the same again and it will remove the last one.
pop @array1;
print "@array1\n";


#my $ppop = pop @array1;
#print "@array1\n";
#print "$ppop\n";


#END




26.

#!/usr/bin/perl

use warnings;
use strict;

my @array1 = ("1","2","3","4");

print "Original array1 values: @array1\n";

# pop function, to remove the last value.
my $ppop = pop @array1; # over here $ppop will be carring the removed value.
print "Value after doing a pop: @array1\n";
print "What is ppop is carring now: $ppop\n"; # printing the remoed value.

# Now push the another value;

push @array1, $ppop; # adding back the removed one.
print "\nValue after doing a push of the ppop value: @array1\n";

# adding one more value.
push @array1, "5";
print "Pushing one more eliment: @array1\n";

# adding one more value.
push @array1, "Amit";
print "Pushing one more eliment: @array1\n\n";

# Adding the element(s) at the beginning of the array.

my $fname = "Amit";
my $mname = "Kumar";
my $lname = "Mund";

unshift (@array1, "$fname", "$mname", "$lname");
print "@array1\n";

# Removing the 1st element from the array
my $s_shift =shift @array1;
print "@array1\n";
print "$s_shift\n";


## "sort", For sorting the string.
my @array2 = ("Amit","kumar", "Mund", "Bitu");
my @sort_array2 = sort @array2;
print "@sort_array2\n";

# We can use the same array name after doing sort and assign the same.
@array2 = sort @array2;
print "@array2\n";


#END





27.
#!/usr/bin/perl

use warnings;
use strict;

# @ARGV variable [ array type spical variable ]
# print "$ARGV[0]\n";
# print "Trying to print all the input argument: @ARGV[0..$#ARGV]\n";

my $REQ = 3;
my $BADARGS = 165;
$#ARGV += 1;            # NOTE: No need to use my or our for ARGV. If you give, it will show error.

unless ($#ARGV == 3){
    print "$0 requires $REQ variables: ";
    print "ERROR! Insufficent argument.\n";
    exit "$BADARGS";
}

print "Correct number of argument $#ARGV has been passed.\n";

print "$ARGV[0] $ARGV[1] $ARGV[2]\n";
print "printing with array type : @ARGV[0..$#ARGV]\n"; # saw the same output at training too.

# perldoc -f exit; for more help.

#END




28.
#!/usr/bin/perl -wl

# w: option for the warning.
# l: option for the line group.

$fname = "Amit";
$lname = "Mund";
$fullname = "$fname $lname";

print "$fname $lname";
print "$fullname";

# NOTE: Must end with a semi-collon;




29.
#!/usr/bin/perl
# split, join and input files from shell.
use warnings;
use strict;

while (<>){ # reading by a stdin. [ file name as arg ]
    my @username = split /:/, $_;
    if(/false/){
    print "$username[0]\n";
    }
}


# perl -wln -e '@array1 = split /:/, $_; print $array1[0]' passwdFile
# perl -wln -e '@mem1 = split /:/, $_; if (/DirectMap2M/){print $mem1[1]} ;' /proc/meminfo
#     8226816 kB

#END



30.
#!/usr/bin/perl -w

use strict;


printf "Please enter your name: ";
my $fname = <STDIN>; # For standard input, you can declare the variable at the same time.

chomp($fname); # To remove the last newline character.

print "\n***************************************";
print "\nHello $fname, how are you doing today?\n";
print "***************************************\n\n";



31.
#!/usr/bin/perl -w
use strict;

testsub(); # This is how to call a sub-routines, Its the same as function
# and its not needed to be after declating this sub-routines.

sub testsub {             # Need to start with "sub"

    my $name = "Amit Mund";
    print "Hello $name\n";
}


# Note: you can pass the argument too in the sub-routines

testsub1("Amit Mund", "Bitu");

sub testsub1 {
    print "Following is an example of getting the value as argument from testsub2\n";
    print "Hello: $_[0]\n"; # $_ is the default place holder.
}


# Two argument in the "foreach" loop and printing Hello Amit and Hello Bitu.

testsub2 ("Amit","Bitu","Tutu");

sub testsub2 {
    foreach (@_){
        print "Hello $_\n";
    }
}

# NOTE: @_ is the for all the array value passed on the argument.

testsub3 ("Amit","Bitu","Tutu");

sub testsub3 {
    print "Hello @_\n";
}




32.
#!/usr/bin/perl
use warnings;
use strict;

# file handling:
open (fh1, ">>  sub2.log") || die "Error opening the file: $!";

my $etcdir = `ls -l /etc/passwd`;
chomp ($etcdir);    # to delete extra newline.

my $message = "Launching sub2.pl";

log_message("$message");
log_message("$etcdir");
log_message(`uptime; who`);


sub log_message{

    my $current_time = localtime;
    print "$current_time - $_[0]","\n";
    print fh1 "$current_time - $_[0]", "\n";
}


#perl -e '$decimal=10, print $decimal;'
#NOTE: perl -e always need to be in single quote, e.g:
# perl -l -e '$current_time = localtime, print $current_time;'




33.
#!/usr/bin/perl

use warnings;
use strict;

# Printing the arguments.
#print "$ARGV[0]\n";
#print "Trying to print all the input argument: @ARGV[0..$#ARGV]\n";

# printing value at hex.
#perl -wl -e '$hex = sprintf("0x%x",15), print $hex;'

#######################################
# NOTE: Try to always use the variable.
######################################

# Following is the "c" style for loop.

#my $min = 1;
#my $max = 100;
#my $i;
#
#for ($i=$min; $i<=$max; $i++){
#
#    print "$i\n";
#}

my $min = 1;
my $max = 100;
my $i=0;

for ($min..$max){
    print "For loop test\n";
}

# can put the same in a single line too:
for ($min..$max){$i++; print "$i\n";}


#END




34.
#!/usr/bin/perl
use warnings;
#use strict;

my $IN = INFILE;
my $infile = passwdFile;

open ($IN, "$infile") || die "problems: $!";

while (<$IN>){
    print "$_";
}
#END




35.
#!/usr/bin/perl
use warnings;

@mem = split /:/, $_; if (/DirectMap2M/){print "$mem1[1]";} ;' /proc/meminfo
print @mem;

@myls = `ls -l`;
print @myls;



36.
#!/usr/bin/perl
use warnings;
use strict;

print "Content-type: text/html\n\n";


# syntax:

#while (condition){

    #body;
#}

my $value1 = 1;
my $value2 = 20;

while ($value1 <= $value2){

    print "$value1,";  
    $value1 +=2;
}
print "\n";


# syntax:

#until (condition){

    #body;
#}
my $value11 = 1;
my $value22 = 20;

until ($value22 <= $value11){
    print "$value22,";
    $value22 -=2;
}
print "\n";

#END



Saturday 9 August 2014

html frame example

1. The following code will create a frame.

$vi frame.htm

<html>

<frameset cols="10%,*" border="7" bordercolor="red">
    <frame name="leftFrame" src="navigation.htm" marginheight=30 marginwidth="5">
    <frame name="mainFrame" src="mainHome.htm" marginheight="30" marginwidth="25">

</frameset>

</html>


NOTE: the frame name that you can do the target to for other pages.

2. navigation page:

$vi navigation.htm

<html>
<head>
   
</head>

<body>
   
<H5><u>Navigation</u></H5>
<a href="mainHome.htm" target="mainFrame"a>Home</a><br>
<a href="page1.htm" target="mainFrame"a> Page1<br>
<a href="page2.htm" target="mainFrame"a> Page2<br>

</body>

</html>

NOTE: e mainHome.htm, page1.htm and page2.htm all are target to the "mainFrame" that we have define on the frame.htm [above] code. So that when you click the above link, those pages will open on the mainFrame.

Different target:
http://www.w3schools.com/tags/att_a_target.asp


3. Now you create your pages as define on the Navigation.htm file:

example:
 $ vi mainHome.htm:

<html>
<head>
</head>

<body>
    This is the Home page.
</body>
</head>

example:
$ vi page1.htm

<html>
<head>
</head>

<body>
    This is page1.
</body>
</head>


example:
$vi page2.htm

<html>
<head>
</head>

<body>
    This is page2.
</body>
</head>


SnapShot:


Friday 18 July 2014

perl oneliner

1. Decimal to binary:
perl -wl -e '$num1 = sprintf("%b",15), print $num1;'

2. Decimal to Hexa:
perl -wl -e '$num1 = sprintf("%x",15), print $num1;'

3. Decimal to Octal:
perl -wl -e '$num1 = sprintf("%o",15), print $num1;'

* Further details: perldoc -f sprintf

4. Finding duplicate line:
perl -n -e 'print if $a{$_}++;' filename (or)
perl -wnl -e 'print if $_{$_}++;' filename

5. Removing blank line:
perl -wnl -e 'print unless /^$/;' filename

6. Numbering all the lines:
perl -wnl -e 'print $_= "$. $_";' passwd


7. Print the number of lines of file(s) that match a pattern:
 perl -wnl -e '$x++ if /amit/; END {print $x+0};' file1  file2

7. example:
perl -wnl -e '$x++ if /amit/; END {print $x+0};' test
3
$cat test
amit amit
amit amit
mund mund
amit mund

8 . printing the total number of a match:
perl -waln -e '$t += /amit/ for @F; END { print $t }' test
5
$cat test
amit amit
amit amit
mund mund
amit mund

 9. printing the unix time:
perl -wl -e 'print time;'
1405679195

10. printing localtime:
perl -wl -e 'print $curTime if $curTime=localtime;'
Fri Jul 18 15:57:34 2014

perl -wl -e 'print localtime;'
41561518611451980

l$ perl -le 'print scalar gmtime'
Fri Jul 18 10:29:56 2014

$ perl -le 'print scalar localtime'
Fri Jul 18 16:00:14 2014

11. Factorial of 6:
perl -MMath::BigInt -le 'print Math::BigInt->new(6)->bfac()'
720

12. print a..z:



perl -le 'print join ",", ("a".."z");'

13. print 1..1000:
perl -le 'print join ",", ("1".."1000");'

Wednesday 16 July 2014

perl split function example

#!/usr/bin/perl
use strict;
use warnings;

#open (fh2, "file1") || die "Error opening file1: $!";
open (fh2, "/etc/passwd") || die "Error opening file1: $!";

my @array1;

while (<fh2>){
    #push  @array1, [ split ]; # push function to push the data in array1 from $_
    push  @array1, [ split/:/];
}

foreach ( @array1){

   
    print "@$_[0],";
    print "@$_[5]\n";
    print "@$_\n";        # for printing all the field.
}

Sunday 13 July 2014

perl_mysql_connect_example


#!/usr/bin/perl

use warnings;
use strict;
use DBI;

#Setp1 - creating connection object

my $dsn = 'DBI:mysql:thenewboston'; # module:db_engine:db_name
my $user = 'Username';
my $pass = 'Password';
my $host = 'localhost';

my $conn = DBI->connect($dsn,$user,$pass) || die "Error connecting" . DBI->errstr;

# step 2 - define query
my $query1 = $conn->prepare('SELECT * FROM user') || die "Error preparing query" . $conn->errstr;

# setp 3 - execute query
$query1->execute || die "Error executing query" . $conn->errstr;

# step 4 - return results
my @results;

while (@results = $query1->fetchrow_array()) {
    my $id = $results[0];
    my $name = $results[1];
    print "$id,$name\n"
}

Wednesday 9 July 2014

perl cgi examples

Follow steps for perl-cgi:

1. I have all my example code at /home/amit/Documets folder.
2. And we have to point the same to the cgi-folder.

lrwxrwxrwx 1 root root 21 Jul  9 18:22 cgi-bin -> /home/amit/Documents/

amit@laptop:/usr/lib/cgi-bin$ ls -ld perl/
drwxrwxr-x 2 amit amit 4096 Jul  9 18:51 perl/

3. All the perl scripts should have executable permission.

4. On the browser, you can try the following link as example:
http://localhost/cgi-bin/perl/hello.pl


###################
Now few perl script example:


NOTE: The first 2 lines be:

#!/usr/bin/perl

print "Content-type: text/html\n\n";


example 1:

#!/usr/bin/perl
  #
  # The traditional first program.

  # Strict and warnings are recommended.
  use strict;
  use warnings;

  # Print a message.
  print "Content-type: text/html\n\n";
  print "Hello, World!\n";



example 2:

#!/usr/bin/perl

print "Content-type: text/html\n\n";

my $str = <<"EOM";
<html>
<head>
<title>CGI Perl Example</title>
</head>
<body>
<h1>CGI Perl Example</h1>
<p>CGI Perl Example</p>
</body>
</html>
EOM

print $str



example 3:

#!/usr/bin/perl

$a = 10;
$var = <<"EOF";
This is the syntax for here document and it will continue
until it encounters a EOF in the first line.
This is case of double quote so variable value will be
interpolated. For example value of a = $a
EOF
print "$var\n";

$var = <<'EOF';
This is case of single quote so variable value will not be
interpolated. For example value of a = $a
EOF
print "$var\n";



example 4:

#!/usr/bin/perl

use strict;
use warnings;

my $text = "A useful tool in natural language processing is concordance. This allows a specific string to be displayed in its immediate context whereever it appears in a text. For example, a concordance program identifying the target string the might produce some of the following output. Notice how the occurrences of the target string line up vertically. ";

my @values = split(/ /, $text);

#print "$values[0]\n"

foreach my $val (@values) {
    print "$val\n"
}

exit 0;


Tuesday 8 July 2014

puppet manifest.pp example

Example of the manifest.pp:

$ cat puppet/manifests/default.pp
Exec { path => [ "/bin/", "/sbin/" , "/usr/bin/", "/usr/sbin/" ] }

class system-update {
  exec { 'apt-get update':
    command => 'apt-get update',
  }

  $sysPackages = [ "build-essential" ]
  package { $sysPackages:
    ensure => "installed",
    require => Exec['apt-get update'],
  }
}

class apache {
  package { "apache2":
    ensure  => present,
    require => Class["system-update"],
  }

  service { "apache2":
    ensure  => "running",
    require => Package["apache2"],
  }

}

class iotop {
  package { "iotop":
    ensure  => present,
    require => Class["system-update"],
  }
}



include apache
include system-update
include iotop

ansible playbook example

Following is a sort example of how to have a playbook and how to run the same. Note that I have already define the hosts details in the inventory file.


amit@laptop:~/ansible-playbook$ cat playbook.yaml
---
- name: install and start nginx
  hosts: vgbox
  user: vagrant
  sudo: yes

  tasks:

  - name: install subversion
    apt: pkg=subversion state=installed


  - name: install nginx
    apt: pkg=nginx state=installed update_cache=true
    notify:
      -  start nginx

  handlers:
    - name: start nginx
      service: name=nginx state=started



## To run the play-book:

$ ansible-playbook playbook.yaml

PLAY [install and start nginx] ************************************************

GATHERING FACTS ***************************************************************
ok: [vg2]
ok: [vg3]
ok: [vg4]





TASK: [install subversion] ****************************************************
changed: [vg2]
changed: [vg4]
changed: [vg3]
 

 TASK: [install nginx] *********************************************************
changed: [vg2]
changed: [vg3]
changed: [vg4]


NOTIFIED: [start nginx] *******************************************************
changed: [vg2]
changed: [vg3]
changed: [vg4]

PLAY RECAP ********************************************************************
vg2                        : ok=4    changed=3    unreachable=0    failed=0  
vg3                        : ok=4    changed=3    unreachable=0    failed=0  
vg4                        : ok=4    changed=3    unreachable=0    failed=0  



NOTE: I have the ssh key in my config file, else you can update those details at the inventory file [ default: /etc/ansible/hosts: ]

[vgbox]
vg2
vg3
vg4



## ssh config example:

Host vg2
    HostName 192.168.0.32
    Port 22
    User vagrant
    IdentityFile ~/.vagrant.d/insecure_private_key

Host vg3
    HostName 192.168.0.33
    Port 22
    User vagrant
    IdentityFile ~/.vagrant.d/insecure_private_key

Host vg4
    HostName 192.168.0.34
    Port 22
    User vagrant
    IdentityFile ~/.vagrant.d/insecure_private_key

Saturday 5 July 2014

changing nagios default home page

Following page need to update to change the nagios default home page:

File: /usr/share/nagios3/htdocs/index.php


Here are a couple of paths that you may be interested in


my update# vi /usr/share/nagios3/htdocs/index.php

<head>
<meta name="ROBOTS" content="NOINDEX, NOFOLLOW">
<title>Nagios Core</title>
<link rel="shortcut icon" href="images/favicon.ico" type="image/ico">
</head>

<?php
 // allow specifying main window URL for permalinks, etc.
#$corewindow="main.php";
$corewindow="cgi-bin/status.cgi?host=all";
if(isset($_GET['corewindow'])){

...................................
...................................
...................................

 
"Tactical Summary": cgi-bin/tac.cgi
"Map"             : cgi-bin/statusmap.cgi?host=all
"Hosts"           : cgi-bin/status.cgi?hostgroup=all&style=hostdetail
"Services"        : cgi-bin/status.cgi?host=all
"Summary"         : cgi-bin/status.cgi?hostgroup=all&style=summary
"Problems"        : cgi-bin/status.cgi?host=all&servicestatustypes=28
"Availability"    : cgi-bin/avail.cgi
"Trends"          : cgi-bin/trends.cgi
"Summary"         : cgi-bin/summary.cgi





External page:
http://code.naishe.in/2012/07/change-nagios-default-home-page.html

mysql quick learning

Primary Keys:
  - all tables should have "primary keys"
  - column that is 100% unique
  - no rows can have some primary key.

SHOW databases;

USE <db_name>

SHOW tables;

SHOW COLUMNS FROM customers;

DESC customers;

SELECT * FROM customers LIMIT 0,30;

SELECT city FROM customers;


; is must at the end of the query.
white space & multiple line is allowed.
Its a company standard to use "standard CAPS"

SELECT name,city FROM customers;

SELECT * FROM customers;

SELECT DISTINCT state FROM customers;

SELECT COUNT(state) FROM customers;

SELECT DISTINCT state FROM customers LIMIT 5;

SELECT id,name FROM customers LIMIT 5;

[ its ordering using PRIMARY KEYS by default? ]

SELECT id,name FROM customers LIMIT 5,10

[ From id 6 to 15. (total 10 rows) ]
[In mysql db record start with 0]


Fully qualify name:

SELECT customers.address FROM customers;
SELECT name FROM customers ORDER BY name;
SELECT name,address FROM customers ORDER BY id;

SELECT state,city,name FROM customers ORDER BY state,name,

SELECT name,zip FROM customers ORDER BY zip DESC;

[ASC: for ascending, but this is by default. ]

SELECT name,id FROM customers ORDER BY id DESC LIMIT 1;

SELECT name FROM customers ORDER BY name LIMIT 1;


* BASIC DATA FILTERING:

SELECT id,name FROM customers WHERE id=54;
SELECT id,name FROM customers WHERE id!=54;
SELECT id,name FROM customers WHERE id<8;
SELECT id,name FROM customers WHERE id<=8;

SELECT id,name FROM customers WHERE id BETWEEN 25 AND 30;

[ in WHERE with char,use 'SINGLE QUOTATION' mark. ]

SELECT name,state FROM customers WHERE state='CA';

SELECT name,state,city FROM customers WHERE state='ca' AND city='Hollywood';

SELECT name,state,city FROM customers WHERE city='Boston' OR state='CA';

SELECT id,name,ciry FROM customers WHERE (id=1 OR id=2) AND city='Raleigh';


[ if you have multiple AND or OR statement please use () to group them. ]


*) IN or NOT IN:

SELECT name,state FROM customers WHERE state='CA' OR state='NC' OR state='NY';

(for bunch of OR...)

SELECT name,state FROM customers WHERE state IN('CA','NC','NY') ORDER BY state;

[ its a good idea to show your record in some order. ]

SELECT name,state FROM customers WHERE state NOT IN('CA','NC','NY') ORDER BY state;

% -> wild card -> any things.

SELECT name FROM items WHERE  name LIKE 'name%';

SELECT name FROM items WHERE name LIKE '%couputer%';

[ mysql is not case sensitive. ]

-   -> only a single character.


*) Regular Expression in mysql:


we can use the same regular expression.

SELECT name FROM items WHERE name REGEXP 'abc|xyz';

SELECT name FROM items WHERE name REGEXP '[0-9]boxes';



* when you want to include your own character in your sql output:

SELECT CONCAT(city,',',state) FROM customers;

SELECT CONCAT(city,','state) AS new_address FROM customers;

SELECT name,cost,cost-1 AS sale_proce FROM items;


* MYSQL FUNCTION:

SELECT name,UPPER(name) FROM customers;
SELECT AVG(cost) FROM items;
SELECT SUM(bids) FROM items;

SELECT COUNT(name)  FROM items WHERE seller_id=6;

SELECT seller_id,COUNT(NAME) FROM items GROUP BY seller_id ORDER BY seller_id;

SELECT AVG(cost) FROM items WHERE seller_id=6;

SELECT COUNT(*) AS item_count, MAX(cost) AS max, AVG(cost) AS avg FROM items WHERE seller_id=12;


* GROUP BY:

(instead of WHERE multiple time, we can use GROUP BY )

SELECT seller_id,COUNT(*) AS item_count FROM items GROUP BY seller_id;

[ HAVING is something like WHERE in GROUP BY, so we have to use HAVING in a GROUP BY query. ]

SELECT seller_id,COUNT(*) AS item_count FROM items GROUP BY seller_id HAVING COUNT(*) >=3;

[ So, in GROUP BY use HAVING ]

SELECT seller_id,COUNT(*) AS item_count FROM items GROUP BY seller_id HAVING COUNT(*) >=3 ORDER BY items_count DESC;


*) subquery:

Lets say list those items that's cost is more then the AVG(cost);

- Mysql work like inside-out, so first () and then rest;

SELECT name,cost FROM items WHERE cost > (SELECT AVG(cost)FROM items) ORDER By cost DESC;

query(subquery)


*) mysql join tables:

SELECT cousters.id,customers.name,items.name,item.cost FROM customers,items WHERE customers.id=items.seller_id ORDER BY customers.id;

* using AS we can also give a table 'nick name' not just column.

SELECT i.seller_id,i.name,c.id FROM customers AS C, items AS i WHERE i.seller_id=c.id


* Outer joins:

Inner join: when we have to column(from different table) and we want to match them together.

[ customers.id=items.seller_id => both should have values. ]

Outer joins:

example: I want to list the customers.name  even if they are not selling or list even items, that are not getting sold by any customers now.

SELECT customers.name, items.name FROM customers LEFT OUTER JOIN  items ON customers.id=items.seller_id;

[ will list coustomers even if its not there in items table. (because customers is in the left side of LEFT OUTER JOIN)

To include all the rows from the table in LEFT.

The other one:

SELECT customers.name, items.name FROM customers RIGHT OUTER JOIN items ON customers.id=seller_id;


*) UNION:

SELECT name,cost,bids FROM items WHERE bids > 190 OR cost>1000 ORDER BY cost;


But if its become more complex:

SELECT name,cost,bids FROM items WHERE bids>190
UNION
SELECT name,cost,bids FROM items WHERE cost>1000 ORDER BY cost;


taking multiple queries & getting them into one result set. (UNION):
for every UNION column's need to be same.
By default is removes the duplicate entries.
If you don't want to remove duplicate then use UNION ALL in place of UNION


* Fully-text searching:

ALTER TABLE items ADD FULLTEXT(name);

DESC items;  -> it you see KEY column for name row we have an INDEX having MUL.


SELECT name,cost FROM items WHERE MATCH(name) AGAINST('baby');

[no regexp or wild card over here. ]

IT do the ranking too & faster.

SELECT name,cost FROM items WHERE MATCH(name) AGAINST('+baby -coat' IN BOOLEAN MODE)




################################################

*) INSERT:

INSERT INTO items(id,name,cost,seller_id,bids) VALUES('102','fish','10','1',0');

*)  Multiple Row Insert:

INSERT INTO items(id,name,cost,seller_id,bids)
VALUES
('103','apple','1','1','0'),
('104','shoe','ro','1','0'),
('105','ring','100','1','0');

INSERT INTO items(id,name,cost,seller_id,bids) SELECT id,name,cost,seller_id,bids FROM AnotherTable;


*) UPDATE:

(Best to have LIMIT too) for to be safe.

UPDATE items SET name='applecake' WHERE id=103;
UPDATE items SET name='bananna',cost='2' WHERE id=103 LIMIT 1;

UPDATE user SET Password=PASSWORD('new-password') WHERE User='root';


DELETE FROM items WHERE id=103 LIMIT 1;

*) use PRIMARY KEY in UPDATE or DELETE to be safe.





#####################################

CREATE TABLE:

CREATE TABLE user(
id int,
Username varchar(30),
Password varchar(20),
PRIMARY KEY(id)
);



*)

CREATE TABLE classics(
auther varchar(128),
title varchar(128),
category varchar(16),
year smallINT,
isbn char(13),
INDEX (auther(20)),
INDEX (title(20)),
INDEX (category(4)),
INDEX (year),
PRIMARY KEY (isbn)
) ENGINE MyISAM;


#####################################


*) ALTER/DROP/RENAME Table:

ALTER TABLE user ADD address varchar(30);

ALTER TABLE user DROP  column address;


*) TO DROP A TABLE:

 DROP TABLE user_old1;

*) Too Rename A TABLE:

RENAME TABLE user1 to users;



###########################################

*) views:

  - temporary  table;
  - don't carry any own data, its a tempory table created by other tables;

CREATE VIEW mostbids AS
SELECT id,name,bids FROM items ORDER BY bids DESC LIMIT 10;

[ top 10 bids, dynamic tables. ]
[NOTE: After AS is query and the view name over here its before AS. ]


CREATE VIEW customers AS SELECT * FROM user;

[ Over here if you delete some thing from this view table, then it will update as original table too. like sym link? ]

CREATE VIEW address AS SELECT CONCAT(city,',',state) AS fulladdress FROM users;

[ view don't take any memory. ]


#################################################

Few more topics:

trigger, cursors, store procedures.


##################################################















Thursday 3 July 2014

Vagrantfile with Puppet config example

Following is a sample Vagrantfile that will start 4 vagrant box and assigning each an public ipaddress and puppet class, where we can define what all need to deployed on there box. Over here I have provided one vagrant_box image, but if you want you can provide each host with different vagrant_box_image.



$cat Vagrantfile


VAGRANTFILE_API_VERSION = "2"
Vagrant.configure(VAGRANTFILE_API_VERSION) do |config|

  config.vm.define "vg2" do |vg2|
    vg2.vm.box = "my_vagrant_image"
    vg2.vm.hostname = "vg2"
    vg2.vm.network "public_network", :bridge => 'wlan0', :ip => "192.168.0.32"
  end

  config.vm.define "vg3" do |vg3|
    vg3.vm.box = "my_vagrant_image"
    vg3.vm.hostname = "vg3"
    vg3.vm.network "public_network", :bridge => 'wlan0', :ip => "192.168.0.33"
  end


  config.vm.define "vg4" do |vg4|
    vg4.vm.box = "my_vagrant_image.box"
    vg4.vm.hostname = "vg4"
    vg4.vm.network "public_network", :bridge => 'wlan0', :ip => "192.168.0.34"
  end

  config.vm.define "vg5" do |vg5|
    vg5.vm.box = "my_vagrant_image.box"
    vg5.vm.hostname = "vg5"
    vg5.vm.network "public_network", :bridge => 'wlan0', :ip => "192.168.0.35"
  end

  config.vm.provision :puppet do |puppet|
    puppet.manifests_path = "puppet/manifests"
    puppet.manifest_file  = "default.pp"
    puppet.options = ['--verbose']
  end



explanation of puppet class:

Note: even if we have lots of different vagrant box defined in a single Vagrantfile, we just need to have only one puppet class, that should be enough for all of the vagrant box.

    puppet.manifests_path = "puppet/manifests"

The above line says what is the path where we have the manifests file. [ Relative to the location where we have the Vagrantfile. ]

     puppet.manifest_file  = "default.pp"

The above line says which is the manifest file.


Example of the manifest.pp:

$ cat puppet/manifests/default.pp
Exec { path => [ "/bin/", "/sbin/" , "/usr/bin/", "/usr/sbin/" ] }

class system-update {
  exec { 'apt-get update':
    command => 'apt-get update',
  }

  $sysPackages = [ "build-essential" ]
  package { $sysPackages:
    ensure => "installed",
    require => Exec['apt-get update'],
  }
}

class apache {
  package { "apache2":
    ensure  => present,
    require => Class["system-update"],
  }

  service { "apache2":
    ensure  => "running",
    require => Package["apache2"],
  }

}

class iotop {
  package { "iotop":
    ensure  => present,
    require => Class["system-update"],
  }
}



include apache
include system-update
include iotop


NOTE: you have to use the include lines for deploying those package to these instance. Note: Example the iotop class also have a require => Class, pointing to the system-update class. If you don't what to have the system-update class run at your system then, you can comment these line [with # ] and comment the following include line too.


vagrant box with percona-XtraDB centos image

For testing the mysql cluster I am using the percona-XtraDB, on centos server. I choose the basic centos6.5 image for that and later I have installed the percona-XtraDB on that and did a packaging of the same, so that we all can do a quick test of the same.

Step 1:
Install virtualbox on your system. [4.1 onwards or the latest would be good. ]
 [https://www.virtualbox.org/wiki/Downloads]

Step 2:
Install the vagrant on your system.
[https://www.vagrantup.com/downloads.html ]

Step 3: 
mkdir amit-percona && cd amit-percona

Step 4:
 Download the vagrantfile from the following share location:
https://drive.google.com/file/d/0B02LZ59YLdSddDRFTDdHaWJDQXc/edit?usp=sharing

 $ vagrant box add amit-percona-XtraDB amit-percona-XtraDB


output: [ something like following: ]

==> box: Adding box 'amit-percona-XtraDB' (v0) for provider:
    box: Downloading: file:///home/amit/vagrant-boxes/amit-percona/amit-percona-XtraDB
==> box: Successfully added box 'amit-percona-XtraDB' (v0) for 'virtualbox'!

if you type: $ vagrant box list
amit-percona-XtraDB  (virtualbox, 0)
[ you will see something like this. ]

Step 5: 
 5.a: $ mv ~/.vagrant.d/boxes/amit-percona-XtraDB/0/virtualbox/include/_Vagrantfile Vagrantfile

NOTE: This above command will copy custom vagrantfile and moved to your current folder with the name Vagrantfile. [ instead of doing vagrant init ]

 5.b: cat ~/.vagrant.d/boxes/amit-percona-XtraDB/0/virtualbox/include/ssh_config >> ~/.ssh/config

NOTE: This will add my ssh_config to your .ssh/config file.

Step 6:
 $ vagrant up

NOTE: it will start 3 vagrant instance as name cenos[2-4]

Step 7:
 $ vagrant global-status
[ You can see your box information ]

Step 8:
To login: ssh centos2
               ssh centos3
               ssh centos4

[this will work out-of-box because, we have added the ssh/config already, with a private key that comes with vagrant.

or you can do:
ssh vagrant@10.0.10.2 [ password: vagrant ]


NOTE:$ rpm -qa | grep -i percona
percona-release-0.0-1.x86_64
Percona-XtraDB-Cluster-galera-2-2.10-1.188.rhel6.x86_64
Percona-XtraDB-Cluster-server-55-5.5.37-25.10.756.el6.x86_64
Percona-XtraDB-Cluster-client-55-5.5.37-25.10.756.el6.x86_64
Percona-Server-shared-compat-5.1.68-rel14.6.551.rhel6.x86_64
percona-xtrabackup-2.2.3-4982.el6.x86_64


## Enjoy the 3 running vagrant instance that you can play your mysql cluster.






###############################################
###############################################

No need to follow the following section, if you are not interested on how I did this:

################# How I did this ####################

If you are still interested how I did all this then please follow the following notes:

Step 1:
Install virtualbox on your system. [4.1 onwards, the latest would be good. ]

Step 2:
Install the vagrant on your system.

Step 3:
$ vagrant box add centos https://github.com/2creatives/vagrant-centos/releases/download/v6.5.3/centos65-x86_64-20140116.box

  - This will download the base centos image. [ around 250MB ]
 - NOTE: Will update this blog, when I can share my vagrant box. [ that is with percona-XtraDB cluster for mysql. ]


Step 4:
$ vagrant init centos

 - This will create Vagrantfile at your present directory.

Step 5:

Over write your Vagrantfile with the following entry:

VAGRANTFILE_API_VERSION = "2"
Vagrant.configure(VAGRANTFILE_API_VERSION) do |config|
  config.vm.define "centos2" do |centos2|
    centos2.vm.box = "centos"
    centos2.vm.network :private_network, ip: "10.0.10.2"
  end

- Because this will give you a private_network ip of 10.0.10.2 at your system, and only from your system it can be accessable.

Step 6:
login to the vagrant system:
Option a: vagrant ssh centos2 # Need to be in the same directory where you have this Vagrantfile
or
Option b:  ssh vagrant@10.0.10.2  # password: vagrant

Step 7:
Installed the percona-XtraDB-Clustier over here:

  7a: sudo su -

  7b: # rpm -Uhv http://www.percona.com/downloads/percona-release/percona-release-0.0-1.x86_64.rpm
Retrieving http://www.percona.com/downloads/percona-release/percona-release-0.0-1.x86_64.rpm

  7c: yum -y install Percona-XtraDB-Cluster-server Percona-XtraDB-Cluster-client Percona-Server-shared-compat percona-xtrabackup

Step 8:
Once the installation complete I have update the VagrantFile to run the same instance 3 time.

VAGRANTFILE_API_VERSION = "2"
Vagrant.configure(VAGRANTFILE_API_VERSION) do |config|

  config.vm.define "centos4" do |centos4|
    centos4.vm.box = "centos"
    centos4.vm.network :private_network, ip: "10.0.10.4"
# centos4.vm.network "public_network", :bridge => 'wlan0', :ip => "192.168.0.30"
  end

  config.vm.define "centos2" do |centos2|
    centos2.vm.box = "centos"
    centos2.vm.network :private_network, ip: "10.0.10.2"
# centos4.vm.network "public_network", :bridge => 'wlan0', :ip => "192.168.0.31"
  end

  config.vm.define "centos3" do |centos3|
    centos3.vm.box = "centos"
    centos3.vm.network :private_network, ip: "10.0.10.3"
# centos4.vm.network "public_network", :bridge => 'wlan0', :ip => "192.168.0.32"
  end

end

NOTE: If you want your vagrant box need to get public_ip, you do the same using the following: [ as comment at above line ]
centos4.vm.network "public_network", :bridge => 'wlan0', :ip => "192.168.0.30"

Step 9: create a ssh_config file for you all that you can add the same at your ~/.ssh/config file

 Host centos2
    HostName 10.0.10.2
    Port 22
    User vagrant
    IdentityFile ~/.vagrant.d/insecure_private_key
Host centos3
    HostName 10.0.10.3
    Port 22
    User vagrant
    IdentityFile ~/.vagrant.d/insecure_private_key
Host centos4
    HostName 10.0.10.4
    Port 22
    User vagrant
    IdentityFile ~/.vagrant.d/insecure_private_key

Step 10: Repackage the same vagrant box, so that it can be re-used:

$ vagrant package centos2 --output amit-percona-XtraDB --vagrantfile Vagrantfile --include ssh_config
==> centos2: Exporting VM...
==> centos2: Compressing package to: /home/amit/vagrant-boxes/centos/amit-percona-XtraDB
==> centos2: Packaging additional file: ssh_config
==> centos2: Packaging additional file: Vagrantfile


Step 10: Now you can use my image:
 du -ksh amit-percona-XtraDB
365M    amit-percona-XtraDB

It having the base image where percona-XtraDB is install with the updated Vagrantfile that all the setting and with a ssh config file, that you can add to your ssh config so that you can use it.