Networking

Performance

Following are network performance numbers for OpenSolaris 2009.06.

Uperf Micro Benchmark
Uperf is a micro benchmark used to heavily exercise the networking stack. The data acquired is based on out of box performance- no tuning of either system was done with the exception that the default SUSE driver for the Intel 10GbE chipset was not performing properly and was replaced by a driver from Intel.

The two graphs below show maximum aggregate TCP Receive and TCP Transmit data rates for different numbers of connections. The system under test is a Sun Fire X4270 Server with four Sun 10 GbE cards.

While micro benchmarks do not equate to real world workloads, they do help establish upper bounds for performance. OpenSolaris 2009.06 TCP receive rate maximum is an impressive 33.7 Gbps, 66% better than SUSE Linux Enterprise 11 which achieved 20.2 Gbps, and 59% better than Solaris 10 5/09 at 21.1 Gbps. Transmit numbers are much closer for the three operating systems although OpenSolaris has a consistent edge over Solaris 10. Maximum transmit numbers are very close- SUSE 11 36.9Gbps, OpenSolaris 36.7Gbps. Theoretical maximum for 4x10GbE interfaces is 40 Gbps.

As mentioned, none of the systems was tuned. Tuning can, of course, improve the numbers. As an example the following shows Solaris 10 tuned numbers plotted against untuned OpenSolaris.

While Solaris 10 numbers are much improved, the untuned OpenSolaris numbers are roughly equivalent for transmit and still consistently better for receive. For receive, tuned Solaris 10 maximum data rate jumps to 26.2 Gbps and does not drop below 20Gbps for larger number of connections.

The better receive numbers are explained by one facet of the crossbow design- dynamically shifting from interrupt to polling when the packet arrival rate exceeds a threshold. Polling has a key advantage for a busy system - one poll by the driver can potentially return dozens of packets in one operation, far more efficient that one packet per interrupt.

For more details on this test including the tuning parameters used for Solaris 10, see this blog.

Memcached real world benchmark
Memcached is a very popular open source project for Web 2.0 deployments. It is a distributed memory object caching system, typically used in speeding up dynamic web applications by alleviating the load on the MySQL database. With a proper workload, it can also be a good real world networking benchmark. The benchmark initially populates the server cache with objects of different sizes to simulate the types of data that real sites typically store in memcached:

• • Small objects (4-100 bytes) to represent locks and query results
• • Medium objects (1-2 KBytes) to represent thumbnails, database rows, result sets
• • Large objects (5-20 KBytes) to represent whole or partially generated pages

The benchmark then runs a mixture of operations (90% gets, 10% sets) and measures the throughput and response times when the system reaches steady-state.
Read more about setting up memcached as a benchmark and how the OpenSolaris numbers were obtained.
Read more about how the Red Hat numbers were obtained.

The results show that OpenSolaris 2009.06 posted 25% better numbers than Red Hat Enterprise Linux 5. These numbers were obtained on a slightly earlier OpenSolaris development release, build 109. The official OpenSolaris 2009.06 build was 111b2.