The Phoronix website posted an article on Amazon Elastic Cloud Computing (EC2) benchmarking using the Phoronix Test Suite. The Proronix Test Suite is an open-source test suite supporting Linux, OpenSolaris, Mac OS X, Windows and BSD operating Systems, normally used by Phoronix to benchmark hardware, but now used for benchmarking EC2.
For the benchmark the following configuration was used: Ubuntu Server 10.10 64-bit (ami-548c783d) on the m1.large (Large) and m1.xlarge (Extra Large) instances in Amazon’s east coast data center, these two configurations were compared with two physical systems. The tests included Apache, SQLite, PostMark, Bullet Physics, OpenSSL, Gcrypt, John The Ripper, POV-Ray, Parallel BZIP2 Compression, Himeno, MAFFT, NAS Parallel Benchmarks, x264, FFmpeg, and LAME MP3 encoding.
For now it’s obvious that currently the physical machines outperform both EC2 offerings, but it will be interesting to follow the benchmarks being made over time, which could eventually end up in a good overview of the performance of Infrastructure as a Service (IaaS) services. Besides Phoronix there are a number of startups also providing benchmarks for IaaS and Platform as a Service (PaaS) public clouds, including CloudHarmony, CloudSleuth and Bench the Cloud covered by cloud. Phoronix already announced that they will start with other benchmarks in the near future.
Another trend currently arising is benchmarking a whole datacenter infrastructure, like VMware is providing with its VMmark currently in version 2.0. which was released beginning this month. It’s already expected that VMware will provide benchmarking of IaaS in a next version of the VMmark product.