Data Transfer
If your dot com is based on high traffic levels and with heavy content in file size, then the data transfer costs might become a big bulk on your budget.
Of course we are not pretending to be able to provide a way to maintain a YouTube style site for cheap (they were allegedly pouring $1 million a month on Bandwidth![i]), but we will attempt to discuss the different ways to provide quality bandwidth to the website for as little as possible.
CDNs
The Content Distribution Networks are big big companies that provide the very specific service of giving highly scalable bandwidth that will be able to support any traffic surge you may have.
Advantages of using a CDN is that their bandwidth is of great quality, they usually have datacenters around the world (not all companies), so they will serve the content to the user from a server that is closest to their country of connection. CDNs also provide added value, many times in the form of reports, supporting multiple formats such as live windows media or flash. But the biggest advantage of all, as before, is that CDNs will always be able to support any amount of traffic your website may have as well as providing a set of tools that will empower your data serving.
The only disadvantage (albeit not minor at all) is their pricing. To discuss this in detail, we will be comparing prices with the price per TB (terabyte) that the services charge. Most CDNs charge between $400 to $3000 the TB. The less you pre-commit to, the more you will be charged. For a 1 TB monthly precommitment on a 12 month contract, CDNs will average $2000 per TB. This will scale down to about $400 per TB for high bandwidth usages and in the extreme usage cases, for instance YouTube, the prices come down considerably. For our framework purposes, we will very unlikely reach this level of usage for low cost, thus CDN prices will be too high and thus not a recommended solution.
Amazon Simple Storage Service (S3)
Amazon realized in 2006 that they have so many servers and so much excess capacity, that they should take advantage of this and commercialize it. This means that Amazon started selling their bandwidth (a.k.a becomes a CDN). Among other services. The biggest upside of Amazon S3 is their value. Their price/quality of bandwidth is unmatchable today.
Their charging scale:
Storage:
$0.15 per GB-Month of storage used
Data Transfer:
$0.10 per GB - all data transfer in
$0.17 per GB - first 10 TB / month data transfer out
$0.13 per GB - next 40 TB / month data transfer out
$0.10 per GB - data transfer out / month over 50 TB
Data transfer “in” and “out” refers to transfer into and out of Amazon S3. Data transferred between Amazon S3 and Amazon EC2 is free of charge.
Requests:
$0.01 per 1,000 PUT or LIST requests
$0.01 per 10,000 GET and all other requests
You can find here a nifty Amazon S3 price calculator.
Another great advantage of Amazon is their “SOA” architecture. Opening up their services so websites can automatically coordinate with Amazon services have enabled several websites to dramatically change the way they work. For instance Webmail.us has saved 75% of costs by switching to Amazon S3[ii] among many other success stories.
A setback for Amazon S3 is that is still in somewhat green, i.e. the service is not yet finished and as such may still contain errors. There have been minor “slow transfer times” when the quality of the bandwidth went down dramatically with only one 2 hour massive outage. Another possible disadvantage is that Amazon S3 may not have servers distributed around the world but centered in the
One other major disadvantage of Amazon S3 is that it provides a very poor Service Level Agreement (SLA). Many users still use the system though because it is widely assumed that a company such as Amazon won’t do anything to damage their reputation.
Data Transfer Tricks
The focus of the thesis is low cost and yes, we can go even cheaper than Amazon S3. If we are willing to do slightly more work and forgive at times the quality of the bandwidth, there are several neat tricks:
Use multiple cheap hosting accounts that provide a lot of transfer for little money. For instance Lunar Pages offers a $7.95 per month (with a 2 year contract) with 3.5 TBs of bandwidth per month! That is about 80 times cheaper than Amazon (around $600) and 800 times cheaper than a regular CDN (around $6000)! But wait! How can the hosting companies afford this, I am sure you are asking? Hosting companies reach the prices they advertise because they oversell. That means they *assume* in average their users won’t even consume 10% of what they offer, so they will be able to attend to the very few exceptions that do take advantage of their full package and there is always the chance, they won’t be able to grow as your service grows. So there is no catch? Not really, well the quality of the bandwidth is usually worst than the CDNs and the hosting companies put constrains on your accounts so you also do not overuse the requests per second (how many files are requested to the servers per second). Is there any way around this? Great you asked, well one way to try to avoid this issue (and this a tad more complicated), is to have less requests per second. To achieve this you can create one big image with smaller images. Then using Cascade Style Sheets (CSS) and JavaScript (Browser programming language) you can dynamically cut that image into smaller images. Now if your website has more transfer or requests than one host could provide, you can balance your content requests through different hosts.
Advantage of this solution is simply cost, there is no cheaper way than this. The disadvantage is the quality of the bandwidth and possible outages or banning by your provider.
These solutions are only recommended for startups looking to really cut costs and are willing to sacrifice possible downtimes. However it is still a valid approach.
In conclusion, if our data transfer needs are not that of a high volume and we need a reliable solution, Amazon S3 is the best solution. If we have a high volume and absolutely need very small costs, then we always have cheap tricks. I personally believe CDNs provide great service and values, however most startups will have a problem affording them or hopefully their pricing will go down.
[i] “Your Tube, Whose Dime?”, Dan Frommer, August 2008 http://www.forbes.com/home/intelligentinfrastructure/2006/04/27/video-youtube-myspace_cx_df_0428video.html
[ii] Amazon website, http://www.amazon.com/Success-Stories-AWS-home-page/b/ref=sc_fe_l_1/002-7513495-3639218?ie=UTF8&node=182241011&no=3435361&me=A36L942TSJ2AJA#amazon
2 responses so far ↓
1 Bert Armijo // Aug 8, 2007 at 2:38 am
Of course, this all depends on what you consider to be a successful company. If, like most investors, you define that as a profitable liquidity event then you need to consider a little more than just getting started cheap because your company will need to grow to a point where you can turn a profit.
While S3 and EC2 look cheap, in fact when you reach normal production resource usage levels they’re actually quite expensive. As much as double other solutions.
Plus, S3 and EC2 are unique in the way they operate. Building your service on them invariably will require you to adopt your application to them. In doing so you’ll lock yourself in. Moving to another service will be difficult and expensive.
Amazon took a great stride forward for the industry with both EC2 and S3 and they have many happy customers. However, there are services that don’t require lock-in and are also cheaper once you reach normal production levels.
2 Lucas // Aug 8, 2007 at 11:02 am
Hi Bert, there is no correct definition of a successful company however in this framework I defined it as:
1) Big Exit: 10 x investment under one year of labor and 20 x investment for more than one year of work.
2) Business Model: The start up will have proven to be a successful (functional and ongoing) business model. I’ll define a successful business model as a plan that provides monthly break even point at no longer than one year and total investment break even point no longer than two years after launch.
(This is all in a previous post)
—
Now regarding the “real cost” of S3 and EC2. The costs of S3 are linear as well as the other CDNs (albeit you do get discount the more you use, in all CDNs, including S3) so the usage mode of S3 vs Other CDNs is the same in the short run and the long run, except S3 costs less.
Now if you do have a lot of transfers there are cheaper ways to go about data transfer such as buying Mbits in a co-lo, but not through other CDNs, although the current price war seems very interesting.
What you are very correct about is the lock-in the AWS is imposing on its costumers, however, if your code structure is implemented correctly, you should be able to modify your modules to change providers.
BTW, I really like what you guys are doing at 3TERA, are you guys planning on providing low user end grid computing as well as virtual datacenters?
Leave a Comment