Thursday, 3 July 2014

vagrant box with percona-XtraDB centos image

For testing the mysql cluster I am using the percona-XtraDB, on centos server. I choose the basic centos6.5 image for that and later I have installed the percona-XtraDB on that and did a packaging of the same, so that we all can do a quick test of the same.

Step 1:
Install virtualbox on your system. [4.1 onwards or the latest would be good. ]
 [https://www.virtualbox.org/wiki/Downloads]

Step 2:
Install the vagrant on your system.
[https://www.vagrantup.com/downloads.html ]

Step 3: 
mkdir amit-percona && cd amit-percona

Step 4:
 Download the vagrantfile from the following share location:
https://drive.google.com/file/d/0B02LZ59YLdSddDRFTDdHaWJDQXc/edit?usp=sharing

 $ vagrant box add amit-percona-XtraDB amit-percona-XtraDB


output: [ something like following: ]

==> box: Adding box 'amit-percona-XtraDB' (v0) for provider:
    box: Downloading: file:///home/amit/vagrant-boxes/amit-percona/amit-percona-XtraDB
==> box: Successfully added box 'amit-percona-XtraDB' (v0) for 'virtualbox'!

if you type: $ vagrant box list
amit-percona-XtraDB  (virtualbox, 0)
[ you will see something like this. ]

Step 5: 
 5.a: $ mv ~/.vagrant.d/boxes/amit-percona-XtraDB/0/virtualbox/include/_Vagrantfile Vagrantfile

NOTE: This above command will copy custom vagrantfile and moved to your current folder with the name Vagrantfile. [ instead of doing vagrant init ]

 5.b: cat ~/.vagrant.d/boxes/amit-percona-XtraDB/0/virtualbox/include/ssh_config >> ~/.ssh/config

NOTE: This will add my ssh_config to your .ssh/config file.

Step 6:
 $ vagrant up

NOTE: it will start 3 vagrant instance as name cenos[2-4]

Step 7:
 $ vagrant global-status
[ You can see your box information ]

Step 8:
To login: ssh centos2
               ssh centos3
               ssh centos4

[this will work out-of-box because, we have added the ssh/config already, with a private key that comes with vagrant.

or you can do:
ssh vagrant@10.0.10.2 [ password: vagrant ]


NOTE:$ rpm -qa | grep -i percona
percona-release-0.0-1.x86_64
Percona-XtraDB-Cluster-galera-2-2.10-1.188.rhel6.x86_64
Percona-XtraDB-Cluster-server-55-5.5.37-25.10.756.el6.x86_64
Percona-XtraDB-Cluster-client-55-5.5.37-25.10.756.el6.x86_64
Percona-Server-shared-compat-5.1.68-rel14.6.551.rhel6.x86_64
percona-xtrabackup-2.2.3-4982.el6.x86_64


## Enjoy the 3 running vagrant instance that you can play your mysql cluster.






###############################################
###############################################

No need to follow the following section, if you are not interested on how I did this:

################# How I did this ####################

If you are still interested how I did all this then please follow the following notes:

Step 1:
Install virtualbox on your system. [4.1 onwards, the latest would be good. ]

Step 2:
Install the vagrant on your system.

Step 3:
$ vagrant box add centos https://github.com/2creatives/vagrant-centos/releases/download/v6.5.3/centos65-x86_64-20140116.box

  - This will download the base centos image. [ around 250MB ]
 - NOTE: Will update this blog, when I can share my vagrant box. [ that is with percona-XtraDB cluster for mysql. ]


Step 4:
$ vagrant init centos

 - This will create Vagrantfile at your present directory.

Step 5:

Over write your Vagrantfile with the following entry:

VAGRANTFILE_API_VERSION = "2"
Vagrant.configure(VAGRANTFILE_API_VERSION) do |config|
  config.vm.define "centos2" do |centos2|
    centos2.vm.box = "centos"
    centos2.vm.network :private_network, ip: "10.0.10.2"
  end

- Because this will give you a private_network ip of 10.0.10.2 at your system, and only from your system it can be accessable.

Step 6:
login to the vagrant system:
Option a: vagrant ssh centos2 # Need to be in the same directory where you have this Vagrantfile
or
Option b:  ssh vagrant@10.0.10.2  # password: vagrant

Step 7:
Installed the percona-XtraDB-Clustier over here:

  7a: sudo su -

  7b: # rpm -Uhv http://www.percona.com/downloads/percona-release/percona-release-0.0-1.x86_64.rpm
Retrieving http://www.percona.com/downloads/percona-release/percona-release-0.0-1.x86_64.rpm

  7c: yum -y install Percona-XtraDB-Cluster-server Percona-XtraDB-Cluster-client Percona-Server-shared-compat percona-xtrabackup

Step 8:
Once the installation complete I have update the VagrantFile to run the same instance 3 time.

VAGRANTFILE_API_VERSION = "2"
Vagrant.configure(VAGRANTFILE_API_VERSION) do |config|

  config.vm.define "centos4" do |centos4|
    centos4.vm.box = "centos"
    centos4.vm.network :private_network, ip: "10.0.10.4"
# centos4.vm.network "public_network", :bridge => 'wlan0', :ip => "192.168.0.30"
  end

  config.vm.define "centos2" do |centos2|
    centos2.vm.box = "centos"
    centos2.vm.network :private_network, ip: "10.0.10.2"
# centos4.vm.network "public_network", :bridge => 'wlan0', :ip => "192.168.0.31"
  end

  config.vm.define "centos3" do |centos3|
    centos3.vm.box = "centos"
    centos3.vm.network :private_network, ip: "10.0.10.3"
# centos4.vm.network "public_network", :bridge => 'wlan0', :ip => "192.168.0.32"
  end

end

NOTE: If you want your vagrant box need to get public_ip, you do the same using the following: [ as comment at above line ]
centos4.vm.network "public_network", :bridge => 'wlan0', :ip => "192.168.0.30"

Step 9: create a ssh_config file for you all that you can add the same at your ~/.ssh/config file

 Host centos2
    HostName 10.0.10.2
    Port 22
    User vagrant
    IdentityFile ~/.vagrant.d/insecure_private_key
Host centos3
    HostName 10.0.10.3
    Port 22
    User vagrant
    IdentityFile ~/.vagrant.d/insecure_private_key
Host centos4
    HostName 10.0.10.4
    Port 22
    User vagrant
    IdentityFile ~/.vagrant.d/insecure_private_key

Step 10: Repackage the same vagrant box, so that it can be re-used:

$ vagrant package centos2 --output amit-percona-XtraDB --vagrantfile Vagrantfile --include ssh_config
==> centos2: Exporting VM...
==> centos2: Compressing package to: /home/amit/vagrant-boxes/centos/amit-percona-XtraDB
==> centos2: Packaging additional file: ssh_config
==> centos2: Packaging additional file: Vagrantfile


Step 10: Now you can use my image:
 du -ksh amit-percona-XtraDB
365M    amit-percona-XtraDB

It having the base image where percona-XtraDB is install with the updated Vagrantfile that all the setting and with a ssh config file, that you can add to your ssh config so that you can use it.


Wednesday, 2 July 2014

Vagrantfile multi-instance config exmaple

Few Vagrantfile config examples:


NOTE1:
VAGRANTFILE_API_VERSION = "2"
Vagrant.configure(VAGRANTFILE_API_VERSION) do |config|

  # Every Vagrant virtual environment requires a box to build off of.
  config.vm.box = "my_vagrant_image"


## NOTE: You can set the private ip of your vagrant box.

  # using a specific IP.
  config.vm.network :private_network, ip: "10.0.0.2"  # 0.1 is not a good idea.


 end

NOTE2:
Running multiple instance:

https://docs.vagrantup.com/v2/multi-machine/index.html

Note: In the following example we can have two instance running at the same time.

VAGRANTFILE_API_VERSION = "2"
Vagrant.configure(VAGRANTFILE_API_VERSION) do |config|

  config.vm.define "web" do |web|
    web.vm.box = "my_vagrant_image"
    web.vm.network :private_network, ip: "10.0.0.2"
  end

  config.vm.define "db" do |db|
    db.vm.box = "my_vagrant_image"
    db.vm.network :private_network, ip: "10.0.0.3"
  end

  config.vm.define "vg4" do |vg4|
    vg4.vm.box = "my_vagrant_image.box"
    vg4.vm.network :private_network, ip: "10.0.0.4"
  end

end



## Note there a private key, you can use:
~/.vagrant.d/insecure_private_key

and you can use this as your config file:
following is an example:

Host vg2
    HostName 10.0.0.2  ## using 1 is not a good idea.
    Port 22
    User vagrant
    IdentityFile ~/.vagrant.d/insecure_private_key
Host vg3
    HostName 10.0.0.3
    Port 22
    User vagrant
    IdentityFile ~/.vagrant.d/insecure_private_key

### and can directly use:
ssh vg2
ssh vg3
ssh vg4

Note: For 10.0.0.1, if you do ssh it don't work nicely...as this act as router ip address.

of go to the same directory from where you have the config file "Vagrantfile" and can do the following:

vagrant ssh web
vagrant ssh db
vagrant ssh vg4

### Where the file stay for very first time:

/home/amit/.vagrant.d/boxes/hashicorp-VAGRANTSLASH-precise32/0/virtualbox

amit@laptop:~/.vagrant.d/boxes/hashicorp-VAGRANTSLASH-precise32/0/virtualbox$ ls -l
total 288344
-rw------- 1 amit amit 295237632 Jun 28 18:07 box-disk1.vmdk
-rw------- 1 amit amit     14103 Jun 28 18:07 box.ovf
-rw-rw-r-- 1 amit amit        25 Jun 28 18:07 metadata.json
-rw-r--r-- 1 amit amit       505 Jun 28 18:07 Vagrantfile

Monday, 30 June 2014

sublime text adding custom build

Buid different programs with sublime text:

  Sublime text is one of the best IDE you can think of, using this you can have lots of build functionality to build your code from the IDE at the same time.

What all you have to do:

1. Install sublime text.

2. select "Tools" > "Build System" > "New Build System..."

And over write the:

{
    "shell_cmd": "make"
}

with the following for each of the different language and save them with there own language name at front for your reference. Although you can give what ever name you want.

This will create the file at the following folder:
/home/<user>/.config/sublime-text-3/Packages/User

example for me:
/home/amit/.config/sublime-text-3/Packages/User

3. Following are the config file for each of the language:

3a: For Bash
File name: bash.sublime-build

{
"cmd" : ["$file"],
"selector" : "source.shell",
"shell" : "bash"
}


3b: For mysql
File name: mysql.sublime-build

{
    "cmd": ["mysql", "-u", "root", "-pYourPassword", "-e", "source $file"],
    "file_regex": "sql$",
    "selector": "source.sql"
}


3c: For perl
File name: perl.sublime-build

{
    "cmd": ["perl", "-w", "$file"],
    "file_regex": "pl$",
    "selector": "source.perl"
}



3d: For PHP
File name: php.sublime-build

{
  "cmd": ["php", "$file"],
  "file_regex": "php$",
  "selector": "source.php"
}


On mac for go:
$ pwd
/Users/amund/Library/Application Support/Sublime Text 3/Packages/User
$ cat go.sublime-build
{
    "cmd":["/usr/local/go/bin/go", "run", "$file"],
    "file_regex": "go$",
    "selector": "source.go"
}


4. Few Notes:

  a.You can use ctrl-b to build and can see the output at the IDE.
  b. Other languages are there by default.
  c. I will suggest you to keep in Automatic mode, so that it can use use all the different language at the same time.

5. At the above config: "file_regex": "php$", mean the file end with php, so you might like to save the file with those names and then should try the build.

6. By default the bash script need to be executable permission, before doing the build at the bash script. What I did, I did a directory view of the IDE and created a small bash script that will change the file permission of the files inside a folder and keeping the bash script on those directory. [ chmod -R a+x /home/amit/Documents/bash ] and do a build of this script, when ever I add a bash script at the bash folder.


$ pwd
/home/amit/.config/sublime-text-3/Packages/User
$ l$ ls -1
bash.sublime-build
mysql.sublime-build
perl.sublime-build
php.sublime-build

Preferences.sublime-settings

Saturday, 28 June 2014

vagrant

vagrant installation:

1. Install the virtual box.
2. Download the vagrant code from website and install the same.
 

3.
  $ vagrant box add hashicorp/precise32 http://files.vagrantup.com/precise32.box
  $ vagrant init hashicorp/precise32
  $ vagrant up


4.$ vagrant box add hashicorp/precise32 http://files.vagrantup.com/precise32.box
==> box: Adding box 'hashicorp/precise32' (v0) for provider:
    box: Downloading: http://files.vagrantup.com/precise32.box
==> box: Successfully added box 'hashicorp/precise32' (v0) for 'virtualbox'!
 

amit@laptop:~$ vagrant init hashicorp/precise32
A `Vagrantfile` has been placed in this directory. You are now
ready to `vagrant up` your first virtual environment! Please read
the comments in the Vagrantfile as well as documentation on
`vagrantup.com` for more information on using Vagrant.
 

amit@laptop:~$ vagrant up
Bringing machine 'default' up with 'virtualbox' provider...
==> default: Importing base box 'hashicorp/precise32'...
==> default: Matching MAC address for NAT networking...
==> default: Setting the name of the VM: amit_default_1403959296506_47328
==> default: Clearing any previously set network interfaces...
==> default: Preparing network interfaces based on configuration...
    default: Adapter 1: nat
==> default: Forwarding ports...
    default: 22 => 2222 (adapter 1)
==> default: Booting VM...
==> default: Waiting for machine to boot. This may take a few minutes...
    default: SSH address: 127.0.0.1:2222
    default: SSH username: vagrant
    default: SSH auth method: private key
==> default: Machine booted and ready!
==> default: Checking for guest additions in VM...
    default: The guest additions on this VM do not match the installed version of
    default: VirtualBox! In most cases this is fine, but in rare cases it can
    default: prevent things such as shared folders from working properly. If you see
    default: shared folder errors, please make sure the guest additions within the
    default: virtual machine match the version of VirtualBox you have installed on
    default: your host and reload your VM.
    default:
    default: Guest Additions Version: 4.2.0
    default: VirtualBox Version: 4.1
==> default: Mounting shared folders...
    default: /vagrant => /home/amit
amit@laptop:~$


5: Now from few of the above point you can get that:

The virtual box is listening on port 2222 on 127.0.0.1 and default user is "vagrant" password is "vagrant".
You can create your PKI for this. [ if you see your virtual machine, you can see a vm is running. ]

6. How do I login: [ vagrant ssh ]

ssh -p 2222 vagrant@127.0.0.1  #password is vagrant.  [ this is old way. ]


7. vagrant suspend  [ saved current state of the instance. ]
8. vagrant resume   [ resume a previously suspended instance. ]
9. vagrant halt [ to stop the vm instance ]
10. vagrant up [ power on the vm machine. ]

11. start from scratch:
 a. vagrant destroy
 b. vagrant up

12. vagrant ssh [ to ssh into the vagrant box. ]


##
$ vagrant --help
Usage: vagrant [options] <command> [<args>]

    -v, --version                    Print the version and exit.
    -h, --help                       Print this help.

Common commands:
     box             manages boxes: installation, removal, etc.
     connect         connect to a remotely shared Vagrant environment
     destroy         stops and deletes all traces of the vagrant machine
     global-status   outputs status Vagrant environments for this user
     halt            stops the vagrant machine
     help            shows the help for a subcommand
     init            initializes a new Vagrant environment by creating a Vagrantfile
     login           log in to Vagrant Cloud
     package         packages a running vagrant environment into a box
     plugin          manages plugins: install, uninstall, update, etc.
     provision       provisions the vagrant machine
     rdp             connects to machine via RDP
     reload          restarts vagrant machine, loads new Vagrantfile configuration
     resume          resume a suspended vagrant machine
     share           share your Vagrant environment with anyone in the world
     ssh             connects to machine via SSH
     ssh-config      outputs OpenSSH valid configuration to connect to the machine
     status          outputs status of the vagrant machine
     suspend         suspends the machine
     up              starts and provisions the vagrant environment
     version         prints current and latest Vagrant version

For help on any individual command run `vagrant COMMAND -h`

Additional subcommands are available, but are either more advanced
or not commonly used. To see all subcommands, run the command
`vagrant list-commands`.

##

Wednesday, 25 June 2014

nagios service dependencies

Nagios service dependencies:

Some time we don't want to keep alerting for the child alert check which is dependent on another service:

Example:

we have a mysql slave running on host "db_slave_1" and it dependent on the mysql_master that is running on the "db_master".

But we don't want to get mysql slave alert if the mysql is not working on the db_master host.

To have this parent slave relationship we have "Nagios service dependency":


define servicedependency {
 
  host_name  db_master
  service_description  mysql_master

  dependent_host_name  db_slave_1
  dependent_service_description  db_slave_status

  execution_failure_criteria w,u,c
  notification_failure_criteria  w,u,c

}



Few more example: for logical understanding:

define servicedependency {

host_name                                      master_host
service_description                        master_service
dependent_host_name                    slave_host
dependent_service_description      slave_service
execution_failure_criteria w,u,c
notification_failure_criteria  w,u,c

}



Another example:

define servicedependency {
host_name                                                   ram_the_father's house
service_description                                     ram_the_father
dependent_host_name                                 ram_the_father's house
dependent_service_description                   kid
execution_failure_criteria                            w,u,c
notification_failure_criteria                          w,u,c

}




Further information:

http://nagios.sourceforge.net/docs/3_0/dependencies.html
http://nagios.sourceforge.net/docs/3_0/objectdefinitions.html

Tuesday, 17 June 2014

quick mysql master slave configuration

NOTE: I am using ubuntu as my OS with mysql 5.5.
On master server:
[1] sudo vi /etc/mysql/my.cnf
# under [mysqld] add the following line, considering you have the default lines:
log-bin = master-bin
log-bin-index = master-bin.index
server_id = 1
[2] # Update your bind-address so that any one can access the db. You can also update your firewall accordingly:
bind-address = 0.0.0.0         # NOTE: you can comment the bind-address line too.

[3] mysql> create user replicaiton_user
  grand replication slave on *.*
  to replication_user identified by 'your_password';

[4]: Restart the mysql:  " sudo service mysql restart "
[ You can have a look at /var/log/mysql/error/log mean while if you face any issue. ]

Configuring SLAVE:
[1]: sudo vi /etc/mysql/my.cnf
# under [mysqld] add the following line, considering you have the default lines:
relay_log = slave-relay-bin
relay_log_index = slave-relay-bin.index
server-id = 2

[2] Can comment the bind-address = 127.0.0.1 [ so that your application can access the slave system as a read only access. ]

[3] sudo service mysql restart

[4] mysql> change master to
  master_host = 'your_master_host',
  master_port = 'port_number_where_master_db_is_running',
  master_user = 'replication_user',
  master_password = 'your_replication_user's_password'
[5] > start slave;
[6] > show slave status;
[7]: Any further issue, check: /var/log/mysql/error.log

Saturday, 7 June 2014

python regex example1

Task: read a file and print match:


#!/usr/bin/env python

import re

amFile = open('file_name', 'r')
amRe = re.compile('\d{1,5}\s\w+\s\w+\.')   # -> your regex

for line in amFile:
    amOut = re.findall(amRe,line)
    if amOut > 0:
      print (amOut)
amFile.close()



NOTE:
\d{1,5} : any digit from 1 digit to 5 digits  => [0 to 99999 ]
\s : a single space
\w+ : any word at least 1 char or more.
\s : a single space again
\w+ : any word at least 1 char or more.
\. : end with a .

123 Main St.   <= This will match in the above example.



##############################################
 -P, --perl-regexp
              Interpret  PATTERN  as  a  Perl  regular  expression  (PCRE, see
              below).  This is highly experimental and grep  -P  may  warn  of
              unimplemented features.

 grep -P '\d{1,5}' filename



ifconfig | grep -o -P '(\d+\.){3}\d+'
192.168.0.110
192.168.0.255
255.255.255.0
127.0.0.1
255.0.0.0


=============================================


##############################################