Lately I worked on configuring a cloud using Ubuntu 9.04 along with Xen and Eucalyptus. Started on a pretty novice note slowly it helped me in getting into myself as the days went by. Each new milestone gave a sense of self satisfaction and a proud feeling of making it my day. These things do a lot to make yourself feel special.
Anyways, lot has been said about how I feel but now lets talk about what I did and How I did....
I got a call from Ankit(my lead) that we need to do some RnD on how we can deploy Intellicus(Our Product) on a cloud and what all challenges we expect in doing so.
Suddenly, this lead me to hop to my NMG guys and check what I can expect them to help me with in this repect. My biggest concern was to take hold of machines that I would need to complete this task. And as usual my buddy from NMG Abhishek Sarkar helped me with all the logistics that I needed to get started.
I started reading articles and white papers related to the topic and initially could hardly found anything that could have helped me in getting the head start.
Even though little was getting materialized during my initial phases of RnD I tried to put in extra efforts to see how I can get this through. Poking Abhishek again and again I use to discuss him to get the clear understanding of the things like Hypervisor in Linux and its role in creating a cloud.
After two days of efforts, I decided to take help from out iLabs team (Mansij,Akash and Abhishek) who has already explored these areas.
These guys helped me to get through with what kind of platform should I use and the Hardware that I need to accomplish what I want.
They suggested me to go with Ubuntu 9.04 GA which includes support for Eucalyptus 1.5 and kvm for Hypervisor. The Machine that I was told to acquire should atleast have 4Gigs of RAM to avoid any road blocks later during my exercise.
I instantly forwarded the request to Ankit(My lead) and Rajesh(My Manager) to help me acquire these resources. With in next few days I had 2 Machines at my disposal one with 1 gigs of RAM for Cloud controller and another with 4 Gigs for Nodes.
I started with the help of Akash and Abhishek using their experience to get over this. But things are not as smooth as you expect them to be.
Now before I move forward let me talk a little about what cloud commuting and other relevant buzzwords are in the cloud computing jargon. Pardon me if you get to see these definitions on net as I kind of felt that it would be better to get them from Internet rather then explaining them on my own.
Cloud
cloud is a physical place, perhaps owned and controlled by some other entity, and it contains computing resources that are available pretty much on demand for a price.
Node
A machine which is the part of the cloud network and lends resources to that cloud.
Hypervisor
A technique similar to emulating one machine to act as multiple machines with heterogeneous environment. This is similar to VM ware that we all might be aware of. Two most popular ways to achieve this in Linux is by using XEN and KVM.
Eucalyptus
An open source framework provided to simulate the cloud setup in your own network similar to Amazon Ec2(Amazon Elastic Cloud Computing).
Cloud Controller
A module of Eucalyptus that helps managing the cloud
Node Controller
A module of Eucalyptus that helps managing the Nodes in a given cloud.
Kernel
kernel is the central component of most computer operating systems. Its responsibilities include managing the system's resources (the communication between hardware and software components).
Ramdisk
A RAM disk is a portion of RAM which is being used as if it were a disk drive. RAM disks have fixed sizes, and act like regular disk partitions. Access time is much faster for a RAM disk than for a real, physical disk. However, any data stored on a RAM disk is lost when the system is shut down or powered off. RAM disks can be a great place to store temporary data.
Machine Image
The file that image of the that contains the configuration and application that you want to deploy on the cloud.
Now as you are aware of these buzz words that I will refer again and again in my blog further lets move ahead.
Installing Ubuntu 9.04 along with Kvm support itself took around 2 days and yet we were not able to get it working.
Finally I had to work on it alone as I could not expect them to help me with more time as they were already busy with there own assignments. After 2 days I released that the hardware I have acquired does not support KVM. For KVM to work your processor needs to be of Intel-VT or AMD-V family. Not to my surprise, mine belonged to none :(.
Then I had to fall back to other option of hypervisor that is currently supported by Eucalyptus that is Xen. Using Amazon EC2 command line tools I started working on creating a machine image and deployed it on the configured cloud. But even that failed to work for me and Eucalyptus being relatively new I could hardly find anything that could have taken me through.
Even help from Akash and Abhishek could not get this working although they had already dirtied their hands using these tools. But when things don't have to work they don't. I hope everyone would agree to this.
Anyways, Finally i decided to take it head on and I started reading it from the scratch about the hypervisor Xen. I planned to take it step by step.
Now I will write the Steps that I followed to achieve my goal:
1. Install Ubuntu 9.04 and Set up Eucalyptus
- Install Ubuntu 9.04 afresh on CC machine and the Node Controller machine.
- Download eucalyptus-cc and eucalptus-cloud on the CC machine and install them. You can use apt-get install eucalyptus-cc eucalptus-cloud to install them.
- Download the Node Controller on the Node Controller Machine (the one with 4 gigs RAM) and install it using apt-get install eucalyptus-nc.
- Although you can install xen pretty much in the same manner as we did it for Node Controller using apt-get but the catch here is that It does not come with a kernel.
- Now the worl around for would be to install some other kernel for xen that i downloaded from the following location:
wget http://ftp.debian.org/debian/pool/main/l/\
linux-2.6/linux-modules-2.6.26-2-xen-686_2.6.26-15_i386.deb
wget http://ftp.debian.org/debian/pool/main/l/\
linux-2.6/linux-image-2.6.26-2-xen-686_2.6.26-15_i386.deb
sudo dpkg -i linux-modules-2.6.26-2-xen-686_2.6.26-15_i386.deb
sudo dpkg -i linux-image-2.6.26-2-xen-686_2.6.26-15_i386.deb - Once you do that proper entry will be present in /boot/grub/menu.lst file and you will be able to see the xen kernels in you /boot folder.
- Once this is done reboot the system in Xen kernel and see the list of domains by typing xm list command. This should list the Domain-0 by default.
- One you have successfully installed xen now its time for you to test an independent xen image using vmbuilder - a xen based image creation utility.
- This can be done by typing: vmbuilder xen ubuntu --firstboot ./firstboot \
--mirror=http://ipAddressofaptproxymachine:9999/ubuntu -o --tmpfs=- \
#--ip 192.168.152.211 \
#--mask 255.255.255.0 \
#--net 192.158.152.0 \
#--gw 192.168.152.1 \
#--dns 192.168.150.42 --dns 192.168.150.43 \
--addpkg vim --rootsize=10240 --arch i386 --verbose --debug \
- This command will install the vim package to the image and the size of the image will be 10 GB. Also I have commented few more parameters to show that they can be optinally provided while image creation. Else by default the image will use the default newtwork setting of your network.
- The firstboot file is used to execute some script that you want during the first boot of this image.
- In my case I wanted to install the openssl-server(apt-get install openssl-server) so that I can log in the image using ssl.
- Once this image is successfully created, It is always recommended to test this image independently without using the Eucalyptus. This is to ensure that the xen on the Node controller and the image creation utility is working as expected. Moreover, reducing the more control flows while you work on something new like this its always a good approch to 'work a little and test a little'. I have learned it the hard way anyways :(
- Before you can actually test this image some prerequesite steps needs to be performed. You need to download and install the bridge-utils package(apt-get install bridge-utils).
- After doing that you need to start the bridge. use network-script for doing that in the /etc/xen/scripts folder.
- Once bridge is up and you have created the image successfully, it could be another herculean effort get this working in a single go. It was for me atleast.
- Being new to Xen, Lot of things that I need to take of were missed.
- The default mount drives for root and swap space the image will look for needs to be corrected in the /etc/xen-tools/xen-tools.conf the disk-device needs to be set to sda and serial-device needs to be set to tty1. If in case you fail to do so, as I did in my initial days of this task, I use to successfully modify the conf file created while creating the image.
- One thing that I am still unable to figure out is that when I create an Image, My image does not include the modules from the /lib/modules/myKernel Folder. So , I have to copy this folder manually by mounting the image to a temporary folder. Now the command for this came after to me after a lot of efforts I had already put in but finally it came from Abhishek Sarkar who told me how we can get through this problem.
- Once these things are done, u can test your image using xm create and xm console commands. Use ssh to log in to the machine you launch.
- If everything goes as expected, now you can go ahead for deploying the image on the cloud but before that you will have to do some configuration on the cloud controller and the node controller.
- Go to the /etc/eucalyptus on the cloud controller and modify the eucalyptus.conf to provide the address of the node machine.
- Go to the /etc/eucalyptus on the cloud controller and modify the eucalyptus.conf to provide the Hypervisor as xen
- Once done, restart all three components the eucalyptus-cc eucalyptus-cloud and eucalyptus-nc.
- After restart log in to https://cloudControllermachineIp:8443/ and put in ur credentials to manage the cloud.
- Also configure the cloud space and ram settings to suite your need.
- Once you fill the requested details you will then be able to download the X.509 certificates that enable you to access you cloud from any linux terminal using Amazon Ec2 tools.
- Once you are done with the Cloud configuration, now you need to create kernel, ramdisk and machine image using the kernel, ramdisk and the image file.
- Configure your local machine to use the Amazon EC2 tools after downloading ec2-ami and ec2-api command line tools from the following location: http://developer.amazonwebservices.com/connect/kbcategory.jspa?categoryID=88
- Once you downlaoded them, unzip the downloaded X.509 certificates and move them to ~/.euca/eucarc folder on the lcoal machine
- execute source ~/.euca/eucarc to set the certificates as source for your Cloud communications.
- Once done with this unzip the ec2-tools and add the directories to your environment path using the following commands: export EC2_HOME=/root/eucalyptus/tools/ec2-api-tools-1.3-30349
export EC2_AMITOOLS_HOME=/root/eucalyptus/tools/ec2-ami-tools-1.3-26357
export PATH=$PATH:$EC2_HOME/bin
export PATH=$PATH:$EC2_AMITOOLS_HOME/bin - Now our environment is ready to deploy the create and deploy the bundles for the ramdisk, kernel and the machine image.
- Use the following commands for doing that:
mkdir intellikernel
ec2-bundle-image -i /boot/vmlinuz-2.6.28-13-server -d ./intellikernel --kernel true
ec2-upload-bundle -b intellikernel -m ./intellikernel/vmlinuz-2.6.28-13-server.manifest.xml
EKI=`ec2-register intellikernel/vmlinuz-2.6.28-13-server.manifest.xml | awk '{print $2}'`
echo $EKI
Creating the Ramdsik bundle
mkdir intelliramdisk
ec2-bundle-image -i /boot/initrd.img-2.6.28-13-server -d ./intelliramdisk --ramdisk true
ec2-upload-bundle -b intelliramdisk -m intelliramdisk/initrd.img-2.6.28-13-server.manifest.xml
ERI=`ec2-register intelliramdisk/initrd.img-2.6.28-13-server.manifest.xml | awk '{print $2}'`
echo $ERI
Creating the Image Bundle
mkdir intelliimage
ec2-bundle-image -i ubuntu-xen/root.img -d ./image --kernel $EKI --ramdisk $ERI
ec2-upload-bundle -b intelliimage -m ./intelliimage/root.img.manifest.xml
EMI=`ec2-register intelliimage/root.img.manifest.xml | awk '{print $2}'`
echo $EMI
Importing the keypair
ec2-add-keypair intellikey > ~/.euca/intellikey.priv
chmod 0600 ~/.euca/intellikey.priv
Launching the Deployed EMI
ec2-run-instances $EMI -k intellikey
Logging in into the launched EMI
ssh -i ~/.euca/intellikey.priv root@ipaddressofThemachine
Once done with all this now you are all set to use your application which has been deployed on a private cloud.
As I completed this with great satisfaction finally it also led me to provide a helping hand to the iLabs team while they were planning to create the cloud in their environment. They needed to do that as they had a presentation of it which they wanted to give to their ADoE.
Now we regularly exchange our ideas with each other to extend the horizons of out learning and help each other.
Next I am looking forward to bring in a load Balancer and work with multiple instances on my application to be a part of the same cloud. Hope that I will complete that task as well and come back with some more thoughts on it.
Until then keep clouding :D....
