OpenSSH Speed Test

Summary

A comprehensive test of the various Ciphers & MACs included with OpenSSH 6.6.1p1 to determine which ciphers/MACs provide the highest speed encryption/decryption with openssh.

The test was done in Ubuntu 14.04.1 LTS:
OpenSSH_6.6.1p1 Ubuntu-2ubuntu2, OpenSSL 1.0.1f 6 Jan 2014

This was run on a machine with "Intel(R) Xeon(R) CPU E3-1231 v3 @ 3.40GHz" (4 core, hyperthreaded).  The CPU does have AES-NI extensions.

A 3.4GB (3,355,443,200-byte) file was copied via SCP over the loopback interface utilizing the SSH cipher & mac.  The 3GB file was on an SSD and the disk was not the bottleneck as 'dd' can read that file about 10x faster than the fastest SSH cipher.  The loopback interface was utilized to eliminate bottlenecks due to the ethernet interface.  The file tested was generated from /dev/urandom to eliminate any possible gains from compression (note: compression was disabled for this test).

The machine was idle for this -- but to make this test more rigorous and to eliminate any possible issues with "the machine got busy during this test", each Cipher & Mac was repeated 5 times (going through each cipher & mac value, then repeating 4 more times), and the median timing value was used in the report below. All data was validated from 'ssh -v' to make sure the intended cipher/mac was negotiated (it always was) and timing was also from 'ssh -v' (eg, Transferred: sent 587360, received 3359541856 bytes, in 4.4 seconds).

This was done with openssh instead of just looking at openssl benchmarks because I'm more interested in the end-to-end performance of the entire system.


SSH / SSHD Config

Here's my updated ssh_config and sshd_config Cipher/MACS lines now after having done this test:

  Ciphers aes128-gcm@openssh.com,aes256-gcm@openssh.com,aes128-ctr,aes192-ctr,aes256-ctr,chacha20-poly1305@openssh.com,arcfour128,arcfour256,arcfour
  MACs umac-128-etm@openssh.com,umac-128@openssh.com,hmac-sha1-etm@openssh.com,hmac-sha1,hmac-sha2-512-etm@openssh.com,hmac-sha2-512,hmac-sha2-256-et
m@openssh.com,hmac-ripemd160-etm@openssh.com,hmac-sha2-256,hmac-ripemd160@openssh.com,hmac-ripemd160

The "Ciphers" and "MACs" list in the sshd_config is an unordered list of the available Ciphers/MACs that SSHD will negotiate.
The "Ciphers" and "MACs" list in the ssh_config is an ordered list of the Ciphers/MACs that SSH will attempt to negotiate, in order of preference.

I'm also preferring the "ETM" (encrypt-then-mac) MACs as that allows verification that the encrypted packet hasn't been tampered with, prior to decryption.

"Insecure" Ciphers/Macs

For the purpose of this test (and what I'm doing), I'm treating CBC ciphers, 3des, as less secure than the other ciphers - so I'm not using them in my ssh/sshd config. I'm also treating MD5 MACs and MACs with keys <128 bits as "insecure".  I've configured both my client and my server to never attempt to negotiate these "insecure" ciphers/macs.


Discussion of Results

In short: the UMAC and AESxxx-GCM ciphers are amazingly fast - nearly as fast as a raw 'netcat'.

And avoid 3des!

Results

The values are a factor of the fastest test.  For example, arcfour with hmac-ripemd160 has a result of "4.6", this means it took 4.6x as long to copy the 3.6GB file over the loopback interface as the fastest cipher/mac.

Control result - cat the file over the loopback interface using netcat.  No encryption/integrity here.
cat /tmp/bigfile | nc -l 11223
time nc 127.0.0.1 11223 > /dev/null
Timing for this: 0.84x (3.44 sec) compared to the fastest result aes192-ctr/umac-64-etm@openssh.com of 1.0 (4.1 sec).

OpenSSH 6.6.1p1 Speed Test - Oct 2014


Appendix

Confirming AES-NI is being used by openssl:

$ openssl speed -elapsed -evp aes-128-ecb
You have chosen to measure elapsed time instead of user CPU time.
Doing aes-128-ecb for 3s on 16 size blocks: 147588093 aes-128-ecb's in 3.00s
Doing aes-128-ecb for 3s on 64 size blocks: 145084223 aes-128-ecb's in 3.00s
Doing aes-128-ecb for 3s on 256 size blocks: 65500952 aes-128-ecb's in 3.00s
Doing aes-128-ecb for 3s on 1024 size blocks: 16829298 aes-128-ecb's in 3.00s
Doing aes-128-ecb for 3s on 8192 size blocks: 2101166 aes-128-ecb's in 3.00s
OpenSSL 1.0.1f 6 Jan 2014
built on: Wed Oct 15 17:43:26 UTC 2014
options:bn(64,64) rc4(16x,int) des(idx,cisc,16,int) aes(partial) blowfish(idx)
compiler: cc -fPIC -DOPENSSL_PIC -DOPENSSL_THREADS -D_REENTRANT -DDSO_DLFCN -DHAVE_DLFCN_H -m64 -DL_ENDIAN -DTERMIO -g -O2 -fstack-protector --param=ssp-buffer-size=4 -Wformat -Werror=format-security -D_FORTIFY_SOURCE=2 -Wl,-Bsymbolic-functions -Wl,-z,relro -Wa,--noexecstack -Wall -DMD32_REG_T=int -DOPENSSL_IA32_SSE2 -DOPENSSL_BN_ASM_MONT -DOPENSSL_BN_ASM_MONT5 -DOPENSSL_BN_ASM_GF2m -DSHA1_ASM -DSHA256_ASM -DSHA512_ASM -DMD5_ASM -DAES_ASM -DVPAES_ASM -DBSAES_ASM -DWHIRLPOOL_ASM -DGHASH_ASM
The 'numbers' are in 1000s of bytes per second processed.
type             16 bytes     64 bytes    256 bytes   1024 bytes   8192 bytes
aes-128-ecb     787136.50k  3095130.09k  5589414.57k  5744400.38k  5737583.96k

Turning off AES-NI support, note aes-128-ecb is now slower.
$ export OPENSSL_ia32cap="~0x200000200000000"
$ openssl speed -elapsed -evp aes-128-ecb
You have chosen to measure elapsed time instead of user CPU time.
Doing aes-128-ecb for 3s on 16 size blocks: 68384125 aes-128-ecb's in 3.00s
Doing aes-128-ecb for 3s on 64 size blocks: 18748551 aes-128-ecb's in 3.00s
Doing aes-128-ecb for 3s on 256 size blocks: 4770284 aes-128-ecb's in 3.00s
Doing aes-128-ecb for 3s on 1024 size blocks: 1208960 aes-128-ecb's in 3.00s
Doing aes-128-ecb for 3s on 8192 size blocks: 151140 aes-128-ecb's in 3.00s


Ciphers tested:
$ ssh -Q cipher
3des-cbc
blowfish-cbc
cast128-cbc
arcfour
arcfour128
arcfour256
aes128-cbc
aes192-cbc
aes256-cbc
rijndael-cbc@lysator.liu.se
aes128-ctr
aes192-ctr
aes256-ctr
aes128-gcm@openssh.com
aes256-gcm@openssh.com
chacha20-poly1305@openssh.com

MACs tested:
$ ssh -Q mac
hmac-sha1
hmac-sha1-96
hmac-sha2-256
hmac-sha2-512
hmac-md5
hmac-md5-96
hmac-ripemd160
hmac-ripemd160@openssh.com
umac-64@openssh.com
umac-128@openssh.com
hmac-sha1-etm@openssh.com
hmac-sha1-96-etm@openssh.com
hmac-sha2-256-etm@openssh.com
hmac-sha2-512-etm@openssh.com
hmac-md5-etm@openssh.com
hmac-md5-96-etm@openssh.com
hmac-ripemd160-etm@openssh.com
umac-64-etm@openssh.com
umac-128-etm@openssh.com

Comments