Tuesday 25 February 2014

aws python api with boto note1

* How to install boto:
1. Need python.
2. download boto [ https://github.com/boto/boto/downloads/ ]
3. Install boto:
2a. tar xfz boto-2.1.tar.gz
2b. cd boto-2.1
2c. sudo python setup.py install

other way:
1. sudo apt-get install python-setuptools
2. sudo easy_install boto

NOTE: Some time few of the boto function don't work, e:g: boto 1.9 don't have instance.tags support, in that case you have to update the boto.

How to update boto: [ I am using easy_install: part of "python-setuptools: "
easy_install -U boto

To see your current boto version:
$python
>>> import boto
>>> boto.Version
'2.27.0'



example of setting the /etc/boto.cfg file:

[Credentials]
aws_access_key_id = ABCDEFGHIJK
aws_secret_access_key = ABCDEFASDFAS123ASF123


########### Example #########
import boto.ec2

# NOTE: if you don't put the key at conn then it will search for
# your profile file, if not /etc/boto.cfg if not your varible 'BOTO_CONFIG=/file/path'

conn=boto.ec2.connect_to_region('us-east-1')
#conn=boto.ec2.connect_to_region('us-east-1',aws_access_key_id='access_key' , aws_secret_access_key='secret_key')

# Following is the prod key: [ Full access ]
#AWS_ACCESS_KEY_ID = 'access_key'
#AWS_SECRET_ACCESS_KEY = 'secret_key'


reservations = conn.get_all_instances()
for res in reservations:
    for inst in res.instances:
            print "%s " % (inst.tags['Name'])
            print "%s  %s " % ("publicDnsName:", inst.public_dns_name)
            print "%s  %s " % ("internalDnsName:", inst.private_dns_name)
            print "%s  %s " % ("publicIP:", inst.ip_address)
            print "%s  %s " % ("internalIP:", inst.private_ip_address)
            print "%s  %s " % ("architecture:", inst.architecture)
            print "%s  %s " % ("image_id:", inst.image_id)
            print "%s    %s " % ("instance_type", inst.instance_type)
            print ""


########## ###################### ################

$ to see help with in boto:
>>>help(boto)

>>> dir(boto)
['BUCKET_NAME_RE', 'BotoConfigLocations', 'BucketStorageUri', 'Config', 'ENDPOINTS_PATH', 'FileStorageUri', 'GENERATION_RE', 'InvalidUriError', 'NullHandler', 'TOO_LONG_DNS_NAME_COMP', 'UserAgent', 'VERSION_RE', 'Version', '__builtins__', '__doc__', '__file__', '__name__', '__package__', '__path__', '__version__', 'auth', 'auth_handler', 'boto', 'cacerts', 'compat', 'config', 'connect_autoscale', 'connect_beanstalk', 'connect_cloudformation', 'connect_cloudfront', 'connect_cloudsearch', 'connect_cloudtrail', 'connect_cloudwatch', 'connect_directconnect', 'connect_dynamodb', 'connect_ec2', 'connect_ec2_endpoint', 'connect_elastictranscoder', 'connect_elb', 'connect_emr', 'connect_euca', 'connect_fps', 'connect_glacier', 'connect_gs', 'connect_ia', 'connect_iam', 'connect_kinesis', 'connect_mturk', 'connect_opsworks', 'connect_rds', 'connect_rds2', 'connect_redshift', 'connect_route53', 'connect_s3', 'connect_sdb', 'connect_ses', 'connect_sns', 'connect_sqs', 'connect_sts', 'connect_support', 'connect_swf', 'connect_vpc', 'connect_walrus', 'connection', 'datetime', 'ec2', 'exception', 'gs', 'handler', 'https_connection', 'init_logging', 'log', 'logging', 'os', 'perflog', 'platform', 'plugin', 'provider', 'pyami', 're', 'regioninfo', 'resultset', 's3', 'set_file_logger', 'set_stream_logger', 'storage_uri', 'storage_uri_for_key', 'sys', 'urlparse', 'utils']

>>> dir(boto.connect_ec2)
['__call__', '__class__', '__closure__', '__code__', '__defaults__', '__delattr__', '__dict__', '__doc__', '__format__', '__get__', '__getattribute__', '__globals__', '__hash__', '__init__', '__module__', '__name__', '__new__', '__reduce__', '__reduce_ex__', '__repr__', '__setattr__', '__sizeof__', '__str__', '__subclasshook__', 'func_closure', 'func_code', 'func_defaults', 'func_dict', 'func_doc', 'func_globals', 'func_name']

>>> dir(boto.connect_ec2.__get__)
['__call__', '__class__', '__cmp__', '__delattr__', '__doc__', '__format__', '__getattribute__', '__hash__', '__init__', '__name__', '__new__', '__objclass__', '__reduce__', '__reduce_ex__', '__repr__', '__self__', '__setattr__', '__sizeof__', '__str__', '__subclasshook__']

>>> dir(boto.ec2)
['EC2Connection', 'RegionData', 'RegionInfo', '__builtins__', '__doc__', '__file__', '__name__', '__package__', '__path__', 'address', 'attributes', 'blockdevicemapping', 'bundleinstance', 'connect_to_region', 'connection', 'ec2object', 'get_region', 'get_regions', 'group', 'image', 'instance', 'instanceinfo', 'instancestatus', 'instancetype', 'keypair', 'launchspecification', 'load_regions', 'networkinterface', 'placementgroup', 'regioninfo', 'regions', 'reservedinstance', 'securitygroup', 'snapshot', 'spotdatafeedsubscription', 'spotinstancerequest', 'spotpricehistory', 'tag', 'volume', 'volumestatus', 'zone']

>>> dir(boto.ec2.connect_to_region)
['__call__', '__class__', '__closure__', '__code__', '__defaults__', '__delattr__', '__dict__', '__doc__', '__format__', '__get__', '__getattribute__', '__globals__', '__hash__', '__init__', '__module__', '__name__', '__new__', '__reduce__', '__reduce_ex__', '__repr__', '__setattr__', '__sizeof__', '__str__', '__subclasshook__', 'func_closure', 'func_code', 'func_defaults', 'func_dict', 'func_doc', 'func_globals', 'func_name']


* Installing easy_install to install rest of the python package.
1.  sudo apt-get install python-setuptools
2. sudo easy_install paramiko [ ssh2 like tool. ]
3. sudo apt-get install -y euca2ools [ same as aws command line tools. ]

* aws credentials for boto:
1. at the code itself:  [example: ]
>>> import boto
>>> ec2 = boto.connect_ec2(aws_access_key_id='my_access_key',
aws_secret_access_key='my_secret_key')

2. at the system variable: as:
~/.bashrc:
AWS_ACCESS_KEY_ID
AWS_SECRET_ACCESS_KEY

3. Boto configuration files:
3a. ~/.boto
3b. /etc/boto.cfg
3c. set the varible 'BOTO_CONFIG=/file/path'
with following format:
[Credentials]
aws_access_key_id = your_access_key
aws_secret_access_key = your_secret_key

##### to test the above settings: ######
##### the /etc/boto.cfg file should have the read access for the Credentials ]
>>> import boto
>>> ec2 = boto.connect_ec2()
>>> ec2.get_all_zones()
[Zone:us-east-1a, Zone:us-east-1b, Zone:us-east-1c, Zone:us-east-1d, Zone:us-east-1e]

>>> ec3 = boto.connect_ec2()
>>> ec3.get_all_zones()
[Zone:us-east-1a, Zone:us-east-1b, Zone:us-east-1c, Zone:us-east-1d, Zone:us-east-1e]

>>> boto.ec2.regions()
[RegionInfo:eu-west-1, RegionInfo:sa-east-1, RegionInfo:us-east-1, RegionInfo:ap-northeast-1, RegionInfo:us-west-2, RegionInfo:us-west-1, RegionInfo:ap-southeast-1, RegionInfo:ap-southeast-2]

default region for boto is "us-east-1"


If you want to connect to a different regions, then:
import boto.ec2
ec_conn = boto.ec2.connect_to_region('eu-west-1')
>>> ec_conn.get_all_zones()
[Zone:eu-west-1a, Zone:eu-west-1b, Zone:eu-west-1c]

NOTE: If you want by default to connect to some other region then you can update your boto config file as:
[Boto]
ec2_region_name = eu-west-1


Q) If I have multiple aws account, then ?:
I can think of set the variable BOTO_CONFIG = /path/to/file/account1 [ on boto scripts. ]


############
[1] Following is one of the python code to get few of the param of the instances:

import boto.ec2
conn=boto.ec2.connect_to_region('us-east-1', aws_access_key_id='<access_key>', aws_secret_access_key='<secret key>')
reservations = conn.get_all_instances()
for res in reservations:
    for inst in res.instances:
        if 'Name' in inst.tags:

            print "%s (%s) [%s]" % (inst.tags['Name'], inst.id, inst.state)
        else:
            print "%s [%s]" % (inst.id, inst.state)



[2] In following example, the script is accessing the variable details from the profile file to get the access and the secret key:

import boto.ec2
conn=boto.ec2.connect_to_region("us-east-1")
reservations = conn.get_all_instances()
for res in reservations:
    for inst in res.instances:
        if 'Name' in inst.tags:

            print "%s (%s) [%s]" % (inst.tags['Name'], inst.id, inst.state)
        else:
            print "%s [%s]" % (inst.id, inst.state)



No comments:

Post a Comment