Friday, June 3, 2011
Thesis Prototype Available
The EC2 broker prototype developed for the thesis is now available online, it can be downloaded here. The JavaDoc can be consulted here.
Tuesday, May 31, 2011
Thesis Document Finished
After a week of fun and networking in northern California earlier this month, where I attended Google IO, I came back with renewed energy to finish the writing process on my thesis document. I'm proud to present it here:
Monday, April 25, 2011
Recent EC2 related events and thesis update
I continue writing my thesis document, chapter 4 is finally taking form and will be finished this week. Then I first need to make a couple of additions and changes to my broker prototype, before I can start writing my final 5th chapter that handles about the evaluation of the performance (in terms of the achieved cost reduction) of my proposed broker. Will try my best to finish a draft before next Friday (May 6th) when I leave for San Fransisco to attend the Google IO conference.
Recently a couple of interesting EC2 related events happened.
With headlines as "Amazon Gets A Black Eye In The Cloud With Server Crash", "Amazon outage casts a cloud over cloud computing?", "Amazon’s EC2 Outage Proves Cloud Failure Recovery is a Myth!" and "Lessons From a Cloud Failure: It’s Not Amazon, It’s You". They all refer to the failures that began Thursday morning (US time) as the Amazon Cloud Crash. These failures were caused because of RDS databases that started to backup theirselves and connectivity problems in multiple availability zones in the US-East region. Popular websites such as Quara, Reddit, GroupMe and FourSquare were affected and weren't reachable for hours. Customers should foresee failure scenarios and do not rely on instances of single cloud provider in a single regions ... as always a single point of failure should be avoided. That's why AWS explicitly advises their developers to design their site’s architecture so that it is resilient to occasional failures and outages. This event brought cloud computing in the picture rather negatively, today the AWS website or AWS blog do not contain any articles telling their users what caused the outages. During the outages there was however a little update every hour or so on the AWS Health Dashboard, we see there that the recovery to bring everything back to normal took about 4 days.
Note: on Rightscale a good blogpost was found about the EC2 US-East outage.
On a different note on April 22th, AWS posted the public IP ranges used by the different geographical EC2 regions, accompanied by this introduction:
From this data we see that more than half of all the public IP addresses provided by Amazon EC2 are situated in the US-East region, it is followed by EU-West and US-West who account for about 15 percent of the IP addresses. The Asian regions only represent less than 10 percent of the IP addresses, but these regions are the newest introduced regions and are possibly growing faster than the other regions.
Recently a couple of interesting EC2 related events happened.
With headlines as "Amazon Gets A Black Eye In The Cloud With Server Crash", "Amazon outage casts a cloud over cloud computing?", "Amazon’s EC2 Outage Proves Cloud Failure Recovery is a Myth!" and "Lessons From a Cloud Failure: It’s Not Amazon, It’s You". They all refer to the failures that began Thursday morning (US time) as the Amazon Cloud Crash. These failures were caused because of RDS databases that started to backup theirselves and connectivity problems in multiple availability zones in the US-East region. Popular websites such as Quara, Reddit, GroupMe and FourSquare were affected and weren't reachable for hours. Customers should foresee failure scenarios and do not rely on instances of single cloud provider in a single regions ... as always a single point of failure should be avoided. That's why AWS explicitly advises their developers to design their site’s architecture so that it is resilient to occasional failures and outages. This event brought cloud computing in the picture rather negatively, today the AWS website or AWS blog do not contain any articles telling their users what caused the outages. During the outages there was however a little update every hour or so on the AWS Health Dashboard, we see there that the recovery to bring everything back to normal took about 4 days.
Note: on Rightscale a good blogpost was found about the EC2 US-East outage.
On a different note on April 22th, AWS posted the public IP ranges used by the different geographical EC2 regions, accompanied by this introduction:
We are pleased to announce that as part of our ongoing expansion, we have added a new public IP range (APAC-Tokyo).These IP ranges can be found here. They can be seen as an indication of the size of the different EC2 regions, it's interesting to have a look at how these number of IP addresses relate to each other:
From this data we see that more than half of all the public IP addresses provided by Amazon EC2 are situated in the US-East region, it is followed by EU-West and US-West who account for about 15 percent of the IP addresses. The Asian regions only represent less than 10 percent of the IP addresses, but these regions are the newest introduced regions and are possibly growing faster than the other regions.
Sunday, April 17, 2011
SpotWatch & Thesis Update
Amazon recently added even more instance types to the spot pricing market, namely the Cluster Compute and Cluster GPU instances in the US-East region! I updated the SpotWatch application with these two instance types, it is up and running on the website: spotwatch.eu.
Besides from that I've been busy writing my thesis, which seems to be a rather slow process. I just started writing the 4th chapter which presents the design of the proposed broker, this means that a first version of chapter 1 (Introduction), chapter 2 (Environmental Analysis) and chapter 3 (Resource Scheduling) are written. During the following two weeks I will also expand the implementation of my broker prototype and keep in mind that the next (and final) chapter handles about the evaluation of my brokers results.
Besides from that I've been busy writing my thesis, which seems to be a rather slow process. I just started writing the 4th chapter which presents the design of the proposed broker, this means that a first version of chapter 1 (Introduction), chapter 2 (Environmental Analysis) and chapter 3 (Resource Scheduling) are written. During the following two weeks I will also expand the implementation of my broker prototype and keep in mind that the next (and final) chapter handles about the evaluation of my brokers results.
Monday, March 28, 2011
On-Demand/Reserved EC2 Price vs Hardware Costs
During the last week I have been trying to continue writing my thesis, but it advances very slowly. I however did some interesting research about the relation between on-demand price reductions in EC2 (focus on US East region) and the hardware cost reduction that happened over time. A draft of the appendix containing this research is embedded here.
If this text appears as small as it does for me, you can download the PDF here as well.
If this text appears as small as it does for me, you can download the PDF here as well.
Saturday, March 5, 2011
New Asia Region & SpotWatch Update
Amazon released a new (5th) regions for AWS a couple of days ago, this was announced by Jeff Barr on the Amazon Web Services Blog like this:
I looked into this new region and made conclusions, some of them are listed below. The Excel file containing all of the tables and graphs can be found here. I delayed the pricing analysis that includes spot instances, since there is only data available from the first 4 days since the launch of this new Tokyo region. I added however support for this new region to my SpotWatch tool website, so we'll be able to follow the price evolution easily. I also made some little changes, check it out at the SpotWatch website. Thanks for the feedback I got about this little tool, it's fun to see people are interested in my work.
We've just opened up an AWS Region in Japan, Tokyo to be precise. The new region supports Amazon EC2 (including Elastic IP Addresses, Amazon CloudWatch, Elastic Block Storage, Elastic Load Balancing, VM Import, and Auto Scaling), Amazon S3, Amazon SimpleDB, the Amazon Relational Database Service, the Amazon Simple Queue Service, the Amazon Simple Notification Service, Amazon Route 53, and Amazon CloudFront. All of the usual EC2 instance types are available with the exception of the Cluster Compute and Cluster GPU. The page for each service includes full pricing information for the Region.
I looked into this new region and made conclusions, some of them are listed below. The Excel file containing all of the tables and graphs can be found here. I delayed the pricing analysis that includes spot instances, since there is only data available from the first 4 days since the launch of this new Tokyo region. I added however support for this new region to my SpotWatch tool website, so we'll be able to follow the price evolution easily. I also made some little changes, check it out at the SpotWatch website. Thanks for the feedback I got about this little tool, it's fun to see people are interested in my work.
-
It's the first time that the fixed upfront investment for a reserved instance is different for a region, all other regions hold the same fixed prices. These in Tokyo are a bit higher. -
The on-demand prices for linux instances are the most expensive in the Tokyo region, US-East is still the cheapest region, all others hold the same hourly price values that lay between the ones of these two regions. 
Reserved instances are more expensive in the new Notheast Asia region relative to on-demand prices than in the other regions (except for the Micro instance).
Data transfer costs are the same in the US and EU regions, they are a bit higher in Southeast Asia and even more expensive in the new Tokyo region.
Sunday, February 27, 2011
Intermediate Thesis Presentation
Long time no see! Sorry about that, I'm happy to be able to tell you my exam results were great!
Last week we had to give an intermediate presentation, which went rather good (I was nervous though). You can download this powerpoint presentation here (or you can find the pdf version here).
During this week I decided to work on a cloudexchange.org counterpart, that I entitled SpotWatch, which provides statistical data about the spot price history evolution. This website is hosted on Amazon EC2 itself and can (for the moment) be found on a subdomain of my website: http://spotwatch.kurtvermeersch.com Hope this will be a help for further research on the spot price analysis.
You'll here from me soon, I'll be finishing the 'Environmental Analysis' chapter of my thesis in the coming week (and I'll be switching to the Comp Latex theme). By the way I'll possibly attend the Cebit fair in Hannover on Friday, which has as main theme "Cloud Computing" this year.
UPDATE:
I've been working on the SpotWatch website, it now automatically updates its spot price history daily. And the service became more user friendly: e.g. the charts link to a bigger version of themselves. I also bought a domain, since the site seemed to attract a lot of people (over 50 the first day): http://spotwatch.eu.
Last week we had to give an intermediate presentation, which went rather good (I was nervous though). You can download this powerpoint presentation here (or you can find the pdf version here).
During this week I decided to work on a cloudexchange.org counterpart, that I entitled SpotWatch, which provides statistical data about the spot price history evolution. This website is hosted on Amazon EC2 itself and can (for the moment) be found on a subdomain of my website: http://spotwatch.kurtvermeersch.com Hope this will be a help for further research on the spot price analysis.
You'll here from me soon, I'll be finishing the 'Environmental Analysis' chapter of my thesis in the coming week (and I'll be switching to the Comp Latex theme). By the way I'll possibly attend the Cebit fair in Hannover on Friday, which has as main theme "Cloud Computing" this year.
UPDATE:
I've been working on the SpotWatch website, it now automatically updates its spot price history daily. And the service became more user friendly: e.g. the charts link to a bigger version of themselves. I also bought a domain, since the site seemed to attract a lot of people (over 50 the first day): http://spotwatch.eu.
Subscribe to:
Posts (Atom)