I'm transferring large files between sites that have a 1Gbit uplink. Some times our clients need to upload somewhere along the 500GB range. I don't want to use FTPS because it's caused connection/firewalling problems for me and is nearly impossible to debug. mod_sftp is the way and the light.
We've had some performance issues. Some clients have reported speeds as low as 1MB/s and after extensive testing and debugging, I'm starting to presume the selected ciphers are at fault.
- Client is 'scp' command on a Debian 8 server.
- Server is ProFTPd with mod_ssl on a Debian 8 server. ProFTPD Version 1.3.5
- Test file is a 10GB file containing random data. generated using "dd if=/dev/urandom of=random-10G bs=10M count=1024"
- All tests had full use of 1 core on the Client (Xeon CPU in a server). All tests saturated the CPU to 100% usage, so I'm assuming that the transfer speed is CPU-bound.
- ProFTPd Server machine never reached 100% CPU usage
- ProFTPd Server machine is hosted on SSD SAN with 500MB/s+ write speeds available.
- There are other things running on the "client" machine. It is a KVM server. While there are 12 available threads, there's at least 11 virtual machines running on the same machine, however most of them are idle.
- There are other things happening on my network and the 1Gbit uplink to the data center is not always 100% available to me.
Further tests should control for these two variables, however I'd like to post my initial results.
cipher - speed - time
aes256-ctr 22.7MB/s 07:31
aes192-ctr 42.5MB/s 04:01
aes128-ctr 15.8MB/s 10:48
aes256-cbc 37.1MB/s 04:36
aes192-cbc 26.5MB/s 06:26
aes128-cbc 23.6MB/s 07:14
blowfish-cbc 43.0MB/s 03:58
cast128-cbc 41.3MB/s 04:08
3des-cbc 13.5MB/s 12:39
arcfour256 33.4MB/s 05:07
arcfour128 36.7MB/s 04:39
What surprised me the most is that aes192-ctr was faster than aes128-ctr. I'm blaming this on one of the variables listed under "mistakes" though.
When transferring the same file using OpenSSH as Server and using email@example.com
as cipher, I get transfer speeds upwards of 50-70MB/s, and the client is no longer using 100% CPU (hardware acceleration), so the bottleneck is moved to some other place.
My question is this:
- Are there any plans of supporting any extra ciphers in mod_sftp?
- Any chance that there will be hardware accelerated ciphers in the future?
- Are there any other ways I can improve transfer speeds, other than prioritizing the lower-bit aes ciphers in my SFTPCiphers configuration?
Thank you in advance.