Raymii.org

Table of Contents

• Introduction
• How many HSM's?
• Cluster setup
• Initialize the new HSM's
• Backup and restore the keys
• Raspberry Pi setup
• Apache setup
• Configure haproxy
• NGINX in TCP mode
• Charts and result interpretation
  • Siege result parsing
  • mod_ssl. no HSM
  • 1024 bit RSA key
  • 2048 bit RSA key
  • prime256v1 EC key
  • Different keysizes, 1 HSM
  • Burst mode
• Conclusion
• Chart creation
• Benchmark process
• Benchmarks without the HSM
  • Benchmarking 1024 bit RSA key without HSM
  • Benchmarking 2048 bit RSA key without HSM
  • Benchmarking 4096 bit RSA key without HSM
  • Benchmarking 8192 bit RSA key without HSM
  • Benchmarking prime256v1 (NIST curve P-256) EC key without HSM
• OpenSC benchmarks
  • 1 HSM (OpenSC)
    • Benchmarking 1024 bit RSA key with 1 HSM (OpenSC)
    • Benchmarking 2048 bit RSA key with 1 HSM (OpenSC)
    • Benchmarking EC prime256v1 key with 1 HSM (OpenSC)
  • 2 HSM's (OpenSC)
    • Benchmarking 1024 bit RSA key with 2 HSM's (OpenSC)
    • Benchmarking 2048 bit RSA key with 2 HSM's (OpenSC)
    • Benchmarking EC prime256v1 key with 2 HSM's (OpenSC)
  • 3 HSM's (OpenSC)
    • Benchmarking 1024 bit RSA key with 3 HSM's (OpenSC)
    • Benchmarking 2048 bit RSA key with 3 HSM's (OpenSC)
    • Benchmarking EC prime256v1 key with 3 HSM's (OpenSC)
• sc-hsm-embedded benchmarks
  • 1 HSM (sc-hsm-embedded)
    • Benchmarking 1024 bit RSA key with 1 HSM (sc-hsm-embedded)
    • Benchmarking 2048 bit RSA key with 1 HSM (sc-hsm-embedded)
    • Benchmarking EC prime256v1 key with 1 HSM (sc-hsm-embedded)
  • 2 HSM's (sc-hsm-embedded)
    • Benchmarking 1024 bit RSA key with 2 HSM's (sc-hsm-embedded)
    • Benchmarking 2048 bit RSA key with 2 HSM's (sc-hsm-embedded)
    • Benchmarking EC prime256v1 key with 2 HSM's (sc-hsm-embedded)
  • 3 HSM's (sc-hsm-embedded)
    • Benchmarking 1024 bit RSA key with 3 HSM's (sc-hsm-embedded)
    • Benchmarking 2048 bit RSA key with 3 HSM's (sc-hsm-embedded)
    • Benchmarking EC prime256v1 key with 3 HSM's (sc-hsm-embedded)

The 3-Pi HSM cluster to be used for the cluster articles

This article sets up a Nitrokey HSM/SmartCard-HSM web cluster and has a lot of benchmarks. This specific HSM is not a fast HSM since it's very inexpensive and targeted at secure key storage, not performance. But, what if you do want more performance? Then you scale horizontally, just add some more HSM's and a loadbalancer in front.

You want to put your private key material inside an HSM because it cannot be stolen that way. A HSM does not allow key material to be exported, so nobody can secretly copy the keys and use them without your knowledge. If the HSM is tampered with, it will also wipe itself, so brute forcing it will not work.

The cluster consists of Raspberry Pi's and Nitrokey HSM's and SmartCard-HSM's, softwarewise we use Apache, mod_nss and haproxy.

This is the first time I had an actual use case for Raspberry Pi's in a cluster, and I really enjoyed doing it. I might even, in the future, add some more Pi's on top and do some more benchmarks. But I already spent about two weeks working on this single article so three was just fine for now.

Recently I removed all Google Ads from this site due to their invasive tracking, as well as Google Analytics. Please, if you found this content useful, consider a small donation using any of the options below:

I'm developing an open source monitoring app called Leaf Node Monitoring, for windows, linux & android. Go check it out!

Consider sponsoring me on Github. It means the world to me if you show your appreciation and you'll help pay the server costs.

You can also sponsor me by getting a Digital Ocean VPS. With this referral link you'll get $200 credit for 60 days. Spend $25 after your credit expires and I'll get $25!

Do note that two devices were sponsored for this article.

We benchmark a small HTML file and a Wordpress site using:

• a 8192 bit RSA certificate without using the HSM's
• a 4096 bit RSA certificate without using the HSM's
• a 2048 bit RSA certificate without using the HSM's
• a 1024 bit RSA certificate without using the HSM's

• a prime256v1 EC certificate without using the HSM's

• a 2048 bit RSA certificate in the HSM

• a 1024 bit RSA certificate in the HSM

• a prime256v1 EC certificate in the HSM.

We do these benchmarks with the OpenSC module and with the sc-hsm-embedded module to see if that makes any difference.

The full raw results are provided at the end of the article. I first talk a bit more about the HSM's, the cluster setup and the issues I had with the Raspberry Pi's. Then we set up the three HSM devices and the load balancer. Finally, before the raw results, we have nice charts and interpretation of the charts, plus an unexpected conclusion.

Introduction

The Nitrokey HSM and the SmartCard-HSM

The Nitrokey HSM is an open hardware and open software device. It is a USB version of the SmartCard-HSM. Both the SmartCard-HSM as the Nitrokey HSM have sources available and are fully supported by the OpenSC project.

If you like this article, consider sponsoring me by trying out a Digital Ocean VPS. With this link you'll get a $5 VPS for 2 months free (as in, you get $10 credit). (referral link)

The SmartCard-HSM

If you are new to the NitroKey HSM/SmartCard HSM, please also read my getting started article. It explains what the HSM is, how to set it up and how to use it with OpenSSH for example.

I have multiple articles on this nice device, so make sure to read the others as well.

How many HSM's?

Three Nitrokey's in their bags

This guide uses three Nitrokey HSM devices. I've generated three keypairs on one of the HSM's, one RSA 1024, one RSA 2048 and one EC key, just as we did in the mod_nss tutorial. Please consult that article first, since the key generation and certificate loading part is not included in this guide.

I'll refer to the HSM where the DKEK was initialized and the keys were generated as HSM 1. The other two HSM's are referred two as HSM 2 and HSM 3.

If you like this article, consider sponsoring me by trying out a Digital Ocean VPS. With this link you'll get a $5 VPS for 2 months free (as in, you get $10 credit). (referral link)

This guide uses three Raspberry Pi 3 comuters, wired network with Raspbian testing (2016-07-29) and with the three HSM devices plugged in.

At first I tried to use the HSM's on the host computer, with different virtual hosts. That works when using the OpenSC module, but not with the sc-hsm-embedded module. That module doesn't support token labels, yet, so there was no way to distinguish between them. I did try to setup three different NSS databases where the HSM's were only enabled by ID but that still resulted in the three of them being used. To make all tests equal, I went for a second option.

That second option was using three different virtual machines on the same PC with VirtualBox and USB passthrough in VirtualBox:

That however gave all kinds of errors with the HSM's in use. At first it worked just fine with mod_nss but whenever I tried to do more than 5 concurrent actions on the HSM the connection was lost and it became unresponsive, to the part that even sc-hsm-tool did not recognize the HSM and a VM reboot was required. So that wasn't a viable solution either. On to the third option it was.

Cluster setup

THe final solution was using three different computers. Since I wanted to keep it simple, fair and not too expensive I decided to buy three Raspberry Pi 3's. I bought them from the dutch store Kiwi-Electronics including two stackable cases. (These links are not affiliate links, just had a great experience there. Their order confirmation lists the ordering IP, and in my case it was my IPv6 address. Yay +1 for them!). The case comes from ModMyPi and I might even just recreate it in Inkscape and use my lasercutter in the future for expansion.

If you like this article, consider sponsoring me by trying out a Digital Ocean VPS. With this link you'll get a $5 VPS for 2 months free (as in, you get $10 credit). (referral link)

The Raspberry Pi 3 Model B's have the following specs:

• 1.2GHz 64-bit quad-core ARMv8 CPU
• 1 GB LPDDR2 RAM 900 MHz
• 100 Mbit NIC
• 4x USB 2.0

They also have Wifi and bluetooth but I did not use that. All the benchmarks were done over the wired network. The OS is Raspbian Testing. Since the current normal Raspbian ships OpenSC 0.14 which doesn't support the SmartCard- HSM/Nitrokey HSM an upgrade was required. The current testing ships with OpenSC 0.16, which does work with the SmartCard-HSM/Nitrokey HSM.

To combine the three machines into one single service I used haproxy in TCP mode. haproxy is a very fast and scalable HTTP(s)/TCP load balancer. I've used it in production for many years now and have been happy with it ever since.

I did also try nginx since that also supports TCP and UDP load balancing. The results were very comparable, so I think the load balancer is not the limiting factor here.

The software stack used on Raspbian Testing is the following:

• Apache 2.4.23 (Raspbian)
• mpm_event
• PHP 7.0.8-5
• MySQL 5.6.30-1 (Raspbian)
• php-fpm / fcgi
• Wordpress 4.5.3
• mod_nss 1.0.12-2

I'm using mpm_event and php-fpm instead of mpm_prefork and mod-php because of issues with the initialization of the HSM by all the workers. See the mailinglist thread here for more information. Otherwise all kinds of errors like SSL Library Error: -8152 The key does not support the requested operation and SSL Library Error: -8023 Unknown and SSL Library Error: -12216 Attempt to write encrypted data to underlying socket failed and SSL input filter read failed. occur.

To install all the software and configure everything except for the HSM key generation I've created a set of Ansible playbooks. Because nobody wants to handcraft three special snowflakes. I've burned through 7 MicroSD cards before getting to a working setup. I also tried to use Arch and Ubuntu 16.04 but those images all had their own instabillites, so I just settled on Raspbian testing.

I might put the playbooks on here someday, but now they are intertwined to much with my personal playbooks to make sense without it.

How do you keep the HSM's apart? Well, I've used a very high-tech solution for that, namely using three different coloured key-cords:

Stickers might also be an option.

Initialize the new HSM's

Plug in HSM 2.

If you execute sc-hsm-tool it will notify you that the new HSM has never been initialized:

$ sc-hsm-tool 

Output:

Using reader with a card: Nitrokey Nitrokey HSM (010000000000000000000000) 02 00
Version              : 2.0
SmartCard-HSM has never been initialized. Please use --initialize to set SO-PIN and user PIN.

HSM 1 was initialized with one DKEK share. We initialize the new HSMs and import the DKEK share, to make sure the key backups work. We also give it a different label.

sc-hsm-tool --initialize --so-pin 3537363231383830 --pin 648219 --dkek-shares 1 --label 'hsm2'

Output:

Using reader with a card: Nitrokey Nitrokey HSM (010000000000000000000000) 02 00

Import the DKEK share:

sc-hsm-tool --import-dkek-share dkek-share-1.pbe

Output:

Using reader with a card: Nitrokey Nitrokey HSM (010000000000000000000000) 02 00
Enter password to decrypt DKEK share : 123456789

Deciphering DKEK share, please wait...
DKEK share imported
DKEK shares          : 1
DKEK key check value : 0FB85F69F6EBF256

Repeat the above process for HSM 3 and any more HSM's you have. Make sure to give them descriptive labels. Unplug the other HSM's when initializing one, to make sure you don't overwrite the wrong one.

The new HSM's are now initialized with the same DKEK as the old HSM. The next step is to securely backup the existing keys from the old HSM and import them into the new HSMs. This works because we're using the same DKEK.

Backup and restore the keys

Plug in HSM 1.

Wrap (export) the keys on HSM 1:

sc-hsm-tool --wrap-key wrap-key-1.bin --key-reference 1 --pin 648219
sc-hsm-tool --wrap-key wrap-key-2.bin --key-reference 2 --pin 648219
sc-hsm-tool --wrap-key wrap-key-3.bin --key-reference 3 --pin 648219

Output:

Using reader with a card: Nitrokey Nitrokey HSM (010000000000000000000000) 00 00

Plug in HSM 2.

Unwrap (import) the keys on the HSM 2:

sc-hsm-tool --unwrap-key wrap-key-1.bin --key-reference 1 --pin 648219
sc-hsm-tool --unwrap-key wrap-key-2.bin --key-reference 2 --pin 648219
sc-hsm-tool --unwrap-key wrap-key-3.bin --key-reference 3 --pin 648219

Output:

Using reader with a card: Nitrokey Nitrokey HSM (010000000000000000000000) 00 00
Wrapped key contains:
  Key blob
  Private Key Description (PRKD)
  Certificate
Key successfully imported

Repeat this for HSM 3.

With all the keys imported and three HSM's plugged in, pkcs11-tool gives some nice output:

pkcs11-tool --module opensc-pkcs11.so --login --pin 648219 --list-slots
Available slots:
Slot 0 (0x0): Nitrokey Nitrokey HSM (010000000000000000000000) 00 00
  token label        : hsm3 (UserPIN)
  token manufacturer : www.CardContact.de
  token model        : PKCS#15 emulated
  token flags        : rng, login required, PIN initialized, token initialized
  hardware version   : 24.13
  firmware version   : 2.0
  serial num         : DENK0100485
Slot 1 (0x4): Lenovo Integrated Smart Card Reader 01 00
  (empty)
Slot 2 (0x8): Nitrokey Nitrokey HSM (010000000000000000000000) 02 00
  token label        : hsm1 (UserPIN)
  token manufacturer : www.CardContact.de
  token model        : PKCS#15 emulated
  token flags        : rng, login required, PIN initialized, token initialized
  hardware version   : 24.13
  firmware version   : 2.0
  serial num         : DENK0100186
Slot 3 (0xc): Nitrokey Nitrokey HSM (010000000000000000000000) 03 00
  token label        : hsm2 (UserPIN)
  token manufacturer : www.CardContact.de
  token model        : PKCS#15 emulated
  token flags        : rng, login required, PIN initialized, token initialized
  hardware version   : 24.13
  firmware version   : 2.0
  serial num         : DENK0100436

If you want to do operations on a specific device you can add the --slot parameter to the pkcs11-tool command. For example, to generate a key just on HSM 2 (slot 3/c):

$ pkcs11-tool --module opensc-pkcs11.so --login --pin 648219 --keypairgen --key-type EC:prime256v1 --slot c --id 10 --label "ect"

Raspberry Pi setup

The Raspberry Pi's are on the network via a wired connection. They run Raspbian Testing because of the OpenSC version.

The following steps should be done on the three Pi's:

• HSM driver installation (OpenSC)
• Apache and mod_nss installation
• NSS Certificate database setup
• Apache and mod_nss configuration

Please consult the mod_nss opensc guide or the mod_nss with sc-hsm- embedded guide for the specific setup and repeat that on all the Raspberry Pi's. I'm not going to cover the setup here any further.

If you like this article, consider sponsoring me by trying out a Digital Ocean VPS. With this link you'll get a $5 VPS for 2 months free (as in, you get $10 credit). (referral link)

Apache setup

Remember to not use mod_php and mpm_prefork. I used mpm_event and php- fpm (PHP 7).

Here is the mpm_event configuration:

    StartServers             20
    MinSpareThreads          250
    MaxSpareThreads          500
    ThreadLimit              64
    ThreadsPerChild          25
    MaxRequestWorkers        500
    MaxConnectionsPerChild   150

I couldn't get fastcgi to run, but it seems Apache uses its own module now (proxy_fcgi).

Make sure to restart Apache after configuring.

Repeat this step on all the VM's.

Configure haproxy

Make sure you have haproxy installed:

apt-get install haproxy

I'm using version 1.6.6. I've got a few other articles on HAproxy if you're interested. HAproxy does not run on the Raspberry Pi's, but on my local host (A Lenovo Thinkpad x240, i5, 8GB RAM, Arch linux), so don't install this in the VM's.

HAproxy can do many things in http mode, but we're not using that. We will be using tcp mode with a roundrobin configuration. This means that when requests come in, haproxy proxy's the TCP connection to the backends. Request one goes to hsm1, request two to hsm 2, request 3 to hsm 3 and request 4 goes to hsm 1. This way the server can handle more concurrent requests. This is my basic haproxy configuration file, /etc/haproxy/haproxy.cfg:

global
    maxconn     20000
    log         hsmcluster.nl local0
    user        haproxy
    chroot      /usr/share/haproxy
    pidfile     /run/haproxy.pid
    daemon

frontend hsm
    bind *:443
    mode tcp
    default_backend hsm
    timeout client 1m

backend hsm
    mode tcp
    balance roundrobin
    timeout connect 10s
    timeout server 1m

    server hsm01 10.0.0.106:8443
    server hsm02 10.0.0.107:8443
    server hsm03 10.0.0.108:8443

My VM's have the 10.0.0.106, 107 and 108 addresses.

Restart haproxy after changing the config.

In my hosts file I've setup the domain hsmcluster.nl on localhost.

If you like this article, consider sponsoring me by trying out a Digital Ocean VPS. With this link you'll get a $5 VPS for 2 months free (as in, you get $10 credit). (referral link)

NGINX in TCP mode

If you prefer to use NGINX then you can use this example configuration:

worker_processes  1;
events {
    worker_connections  1024;
}

stream {
    server {
      listen 443;
      proxy_pass hsm_backend;
    }

    upstream hsm_backend {
      server 10.0.0.106:443;
      server 10.0.0.100:443;
      server 10.0.0.105:443;
    }
}

Charts and result interpretation

If you like this article, consider sponsoring me by trying out a Digital Ocean VPS. With this link you'll get a $5 VPS for 2 months free (as in, you get $10 credit). (referral link)

Here below I'll look into the various aspects and results of the benchmark. The last section of the article gives you the raw numbers/benchmark siege results to play around yourself. This is the more readable part if you're not into number crunching.

Siege result parsing

Siege gives you different types of metrics. We do the same benchmark over and over so we're interesed in some, not all. The time for example is 30 seconds everywhere. Here is, from the manual, the explanation of the metrics we use.

• Transactions is the number of server hits.
• Response time is the average time it took to respond to each simulated user's requests.
• Transaction rate is the average number of transactions the server was able to handle per second, in a nutshell: transactions divided by elapsed time.
• Concurrency is average number of simultaneous connections, a number which rises as server performance decreases.

mod_ssl. no HSM

As we can see here the 1024 bit RSA key is the fastest. 2048 bit RSA is just a tidbit slower, but still acceptable. 4096 bit RSA keys take a huge dump down in transactions and the response time and concurrency go up.

8192 bit RSA keys slow down to a grinding halt, huge response time, low transaction rate. It might be super secure, but super slow as well.

Same goes for the EC prime256v1 keys. Most of the time EC keys are faster, but it seems the Pi's have trouble with it, just as much as with the 8192 bit RSA keys.

1024 bit RSA key

We can see that the HSM is around 8 times slower than using regular mod_ssl. But, we knew that already. Adding a HSM to the loadbalancer doubles the performance, and adding two HSM's to the loadbalancer triples it, as we would expect. So, going down that route, if we have 8 HSM's, it would be just as fast as regular mod_ssl.

Something that caught my eye was that with one HSM the OpenSC module was faster. (Remember, best of three for the tests, every test is done three times, best result is kept.). When using multiple HSM's we see a small gain in transactions when using sc-hsm-embedded. I'm not quite sure why that is, but it's something that steps out.

But, as we all know, using an 1024 bit RSA key is considered insecure. So don't do that in production.

Here is the same data with only the HSM's so that you can see the differences better between OpenSC and sc-embedded-hsm:

2048 bit RSA key

2048 bit keys are way harder for the HSM, but not at all for regular mod_ssl. In this case it is about 26 times slower when using a HSM, comparing to no HSM. Here we see again that the transaction rate is doubled and trippled when adding HSM's to the cluster. The same thing here with sc-hsm-embedded, when used with 1 HSM it's slower, but when scaling up it becomes a little bit faster.

Here is the same data with only the HSM's so that you can see the differences better between OpenSC and sc-embedded-hsm:

prime256v1 EC key

I think the Pi's have trouble with the prime256v1 EC key. When using just one HSM, the performance almost doubles. When we use three HSM's the performance is almost as good as with the 1024 bit RSA key. In all cases OpenSC was slower than the sc-hsm-embedded module.

What suprises me the most is that EC algorithms are supposed to be faster then RSA. It might be that non-Pi hardware has the AES-NI extension or something.

Different keysizes, 1 HSM

This is an interesting one. We see that the EC prime256v1 key is almost as fast as the RSA 1024 bit key. Also, OpenSC is slower here with the EC keypair. Most browsers support prime256v1, also named NIST curve P-256. There however are some concerns since the NSA is involved. Keep that in mind. Still, the fastest modern algorithm in this HSM.

Burst mode

The burst mode benchmark is different than the above ones. The above tests take a random amount of seconds between 1 and 5 and delay each connection that amount. This gives you a more realistic test than when your just hammering every second. Hammering every second is a performance measure, because you can then measure how many connections your server could handle at peak times.

This test fires of 60 connections for half a minute and doesnt take any time between them. Just bang bang bang. The results differ from the 20 connection test because the HSM then has some time to recover, so to say, between connections. The more HSM's you add to the cluster, the more time each one will have to recover, the better your tests will be.

The burst benchmark doesn't hit the wordpress site but the small text file.

The chart above is for an 1024 bit RSA key. We see that without the HSM it is the fastest, topping a whopping 7000+ transactions. Here is the picture without that, so make it more clear what the differences with HSM are.

Same results as above, adding more HSM's gives better performance. Note that sc- hsm-embedded is only faster in the 3 HSM test.

This is the score for 2048 bit keys, including the test without the HSM:

Same as above, more HSM makes stuff faster, without the HSM we see an enourmus speed bump. Here's the graph without the last part:

Strange, sc-embedded-hsm is only faster here with 2 HSM's and OpenSC was significantly lower with 2 HSM's. I did rone this specific test again a few times but all gave comparable results.

Last but not least, the prime256v1 EC key. This is the only time where the HSM is faster, I suspect because it provides offloading (the Pi lacks crypto hardware?).

Using the HSM here almost doubles the performance, using three HSM's you get almost 6 times the performance of no HSM. I did not expect this at all, but I do find it awesome.

Conclusion

• Using this HSM with RSA is significantly slower than not using this HSM
• Using this HSM with EC is significantly faster than not using this HSM
• 8192 bit RSA is slower than trying to nail hammer jelly pudding to a wall
• Adding one HSM to the cluster doubles the performance
• Adding two HSM's to the cluster tripples the performance
• You need about 10 HSM's (for RSA 1024) or 26 HSM's (for RSA 2048) to have the same performance without a HSM
• You need about two Pi's without a HSM to have the same performance as with one HSM with prime256v1 EC.

In general this was what I expected, except for the EC part. As said at the start of the article, the Nitrokey/SmartCard-HSM is not built for this use case, although it works absolutely fine, just a bit slower.

I've browsed the wordpress site, did some searching, installed some plugins (slider, contact form etc) and created a few blog posts, which all works just fine. I have 500/500 mbit fiber at home so I do notice the delay, but the performance is comparable to a location with regular DSL and a 8/2 mbit speed. Which means I'm spoiled and most people will not notice the difference.

If you like this article, consider sponsoring me by trying out a Digital Ocean VPS. With this link you'll get a $5 VPS for 2 months free (as in, you get $10 credit). (referral link)

Below I'll talk more about something I tried first for the graphs and then give you more information on the benchmarks and the raw results.

Chart creation

I first tried to use GNUPlot to create charts based on the siege log file with the following gnuplot file:

set term png truecolor size 600,600
set output "data.png"
set title "2048 bit RSA key"
set boxwidth 1 relative
set grid
set key outside;
set key top;
set style fill transparent solid 0.5 border rgb"black"
set style data histogram
set style fill solid border
set style histogram clustered

plot for [COL=2:4] 'siegedata' using COL:xticlabels(1) title columnheader 

The siege log is transformed to a usable datafile using the following commands:

awk -F, '{print $2":"$5":"$8}' siege.log  | awk '{print NR-1"-PI:",$0}' | sed 's/\s\+//g'  | sed -e '1s/^...../Number /' -e 's/:/ /g'

This is the result of the command:

Number Trans RespTime Concurrent
1-HSM 9 0.83 0.25
2-HSM 23 1.06 0.82
3-HSM 124 2.34 9.68

The original siege log was:

      Date & Time,  Trans,  Elap Time,  Data Trans,  Resp Time,  TransRate,  Throughput,  Concurrent,    OKAY,   Failed
1 2016-07-31 08:35:21,      9,      29.86,           0,       0.83,        0.30,        0.00,        0.25,       9,       0
2 2016-07-31 08:36:39,     23,      29.60,           0,       1.06,        0.78,        0.00,        0.82,      23,      21
3 2016-07-31 08:40:15,    124,      29.99,           0,       2.34,        4.13,        0.00,        9.68,     124,       0

This would give me a graph like below:

But that doesn't scale very well for larger and smaller numbers and I didn't like the overall look. So I looked around for simple online charting services and via Opensource.com found Datawrapper. It's a nice service, lots of options while keeping it simple. I shoved them $12 to export the images because I'm to lazy to host it myself or take screenshots. People put effort into hosting and development, so let's reward them for it.

Benchmark process

All the benchmarks were done three times and the best result is used. The machine that runs haproxy wasn't doing anything else at the time, measured with nethogs. No spotify or skype skewing the benchmarks. All was done via a wired gigabit network, cat6 cabling.

I've also done benchmarks without the HSM, just regular apache with mod_ssl. Why not mod_nss you might ask? Well, because nobody will setup mod_nss when they can use mod_ssl. All the guides use mod_ssl.and the most sites online are using it. Only if you have special software or need PKCS#11 you need to use mod_nss.

Benchmarks without the HSM

The below benchmarks does not use the HSM, just regular mod_ssl and a certificate file. We're benchmarking one small page with only the contents 'Jeej it works!'. We're also benchmarking a Wordpress 4.5.3 install with the default content after install and the Hemmingway theme.

I'm not benchmarking multiple Pi's since the result of one Pi and the result of three Pi's was so comparable, I suspect they are fast enough and that mod_ssl doesn't have any bottlenecks there. Except for some tests, those just blew up the Pi's. (8192 bit keys).

The HSM does not support 4096 or 8192 bit RSA keys, which is why I didnot test those with the HSM's.

Here below are all the benchmarking results. Every benchmark was done three times, best result was kept.

Benchmarking 1024 bit RSA key without HSM

Self signed with OpenSSL:

#key + certificate
openssl req -nodes -x509 -sha256 -newkey rsa:1024 -keyout "pi1.hsmcluster.nl.key" -out "pi1.hsmcluster.nl.cert" -days 365 -subj "/C=NL/ST=Zuid Holland/L=Rotterdam/O=Sparkling Network/OU=IT Dept/CN=pi1.hsmcluster.nl"

A siege test with 5 concurrent users, 30 seconds:

siege -c5 -d5 -t30S https://hsmcluster.nl

Result:

Transactions:                     63 hits
Availability:                 100.00 %
Elapsed time:                  29.47 secs
Data transferred:               0.00 MB
Response time:                  0.04 secs
Transaction rate:               2.14 trans/sec
Throughput:                     0.00 MB/sec
Concurrency:                    0.08
Successful transactions:          63
Failed transactions:               0
Longest transaction:            0.06
Shortest transaction:           0.02

10 concurrent users:

siege -c10 -d5 -t30S https://hsmcluster.nl

Result:

Transactions:                    119 hits
Availability:                 100.00 %
Elapsed time:                  29.67 secs
Data transferred:               0.00 MB
Response time:                  0.04 secs
Transaction rate:               4.01 trans/sec
Throughput:                     0.00 MB/sec
Concurrency:                    0.16
Successful transactions:         119
Failed transactions:               0
Longest transaction:            0.10
Shortest transaction:           0.02

20 concurrent users:

siege -c20 -d5 -t30S https://hsmcluster.nl

Result:

Transactions:                    239 hits
Availability:                 100.00 %
Elapsed time:                  29.80 secs
Data transferred:               0.00 MB
Response time:                  0.04 secs
Transaction rate:               8.02 trans/sec
Throughput:                     0.00 MB/sec
Concurrency:                    0.30
Successful transactions:         239
Failed transactions:               0
Longest transaction:            0.13
Shortest transaction:           0.02

60 benchmark mode:

siege -c60 -b -t30S https://hsmcluster.nl

Result:

Transactions:                   7331 hits
Availability:                 100.00 %
Elapsed time:                  29.18 secs
Data transferred:               0.09 MB
Response time:                  0.23 secs
Transaction rate:             251.23 trans/sec
Throughput:                     0.00 MB/sec
Concurrency:                   56.54
Successful transactions:        7331
Failed transactions:               0
Longest transaction:            1.47
Shortest transaction:           0.03

Wordpress site with 10 concurrent users:

siege -c10 -d5 -t30S 'https://hsmcluster.nl/wordpress/'

Result:

Transactions:                    772 hits
Availability:                 100.00 %
Elapsed time:                  29.84 secs
Data transferred:               5.63 MB
Response time:                  0.06 secs
Transaction rate:              25.87 trans/sec
Throughput:                     0.19 MB/sec
Concurrency:                    1.67
Successful transactions:         772
Failed transactions:               0
Longest transaction:            0.75
Shortest transaction:           0.02

Wordpress site with 20 concurrent users:

siege -c20 -d5 -t30S 'https://hsmcluster.nl/wordpress/'

Result:

Transactions:                   1353 hits
Availability:                 100.00 %
Elapsed time:                  29.09 secs
Data transferred:               9.87 MB
Response time:                  0.09 secs
Transaction rate:              46.51 trans/sec
Throughput:                     0.34 MB/sec
Concurrency:                    4.23
Successful transactions:        1353
Failed transactions:               0
Longest transaction:            2.46
Shortest transaction:           0.02

Benchmarking 2048 bit RSA key without HSM

Self signed with OpenSSL:

#key + certificate
openssl req -nodes -x509 -sha256 -newkey rsa:2048 -keyout "pi1.hsmcluster.nl.key" -out "pi1.hsmcluster.nl.cert" -days 365 -subj "/C=NL/ST=Zuid Holland/L=Rotterdam/O=Sparkling Network/OU=IT Dept/CN=pi1.hsmcluster.nl"

A siege test with 5 concurrent users, 30 seconds:

siege -c5 -d5 -t30S https://hsmcluster.nl

Result:

Transactions:                     51 hits
Availability:                 100.00 %
Elapsed time:                  29.28 secs
Data transferred:               0.00 MB
Response time:                  0.06 secs
Transaction rate:               1.74 trans/sec
Throughput:                     0.00 MB/sec
Concurrency:                    0.10
Successful transactions:          51
Failed transactions:               0
Longest transaction:            0.10
Shortest transaction:           0.04

10 concurrent users:

siege -c10 -d5 -t30S https://hsmcluster.nl

Result:

Transactions:                    126 hits
Availability:                 100.00 %
Elapsed time:                  29.68 secs
Data transferred:               0.00 MB
Response time:                  0.07 secs
Transaction rate:               4.25 trans/sec
Throughput:                     0.00 MB/sec
Concurrency:                    0.28
Successful transactions:         126
Failed transactions:               0
Longest transaction:            0.19
Shortest transaction:           0.04

20 concurrent users:

siege -c20 -d5 -t30S https://hsmcluster.nl

Result:

Lifting the server siege...
Transactions:                    253 hits
Availability:                 100.00 %
Elapsed time:                  29.57 secs
Data transferred:               0.00 MB
Response time:                  0.07 secs
Transaction rate:               8.56 trans/sec
Throughput:                     0.00 MB/sec
Concurrency:                    0.60
Successful transactions:         253
Failed transactions:               0
Longest transaction:            0.28
Shortest transaction:           0.04

60 benchmark mode:

siege -c60 -b -t30S https://hsmcluster.nl

Result:

Transactions:                   2999 hits
Availability:                 100.00 %
Elapsed time:                  29.92 secs
Data transferred:               0.04 MB
Response time:                  0.58 secs
Transaction rate:             100.23 trans/sec
Throughput:                     0.00 MB/sec
Concurrency:                   58.55
Successful transactions:        2999
Failed transactions:               0
Longest transaction:            1.87
Shortest transaction:           0.14

60 benchmark mode with 3 Pi's in haproxy:

Transactions:                   4890 hits
Availability:                 100.00 %
Elapsed time:                  29.80 secs
Data transferred:               0.06 MB
Response time:                  0.36 secs
Transaction rate:             164.09 trans/sec
Throughput:                     0.00 MB/sec
Concurrency:                   58.74
Successful transactions:        4890
Failed transactions:               0
Longest transaction:            1.42
Shortest transaction:           0.04

Wordpress site with 10 concurrent users:

siege -c10 -d5 -t30S 'https://hsmcluster.nl/wordpress/'

Result:

Transactions:                    707 hits
Availability:                 100.00 %
Elapsed time:                  29.66 secs
Data transferred:               5.16 MB
Response time:                  0.10 secs
Transaction rate:              23.84 trans/sec
Throughput:                     0.17 MB/sec
Concurrency:                    2.29
Successful transactions:         707
Failed transactions:               0
Longest transaction:            1.86
Shortest transaction:           0.04

Wordpress site with 20 concurrent users:

siege -c20 -d5 -t30S 'https://hsmcluster.nl/wordpress/'

Result:

Transactions:                   1267 hits
Availability:                 100.00 %
Elapsed time:                  29.62 secs
Data transferred:               9.25 MB
Response time:                  0.12 secs
Transaction rate:              42.78 trans/sec
Throughput:                     0.31 MB/sec
Concurrency:                    5.26
Successful transactions:        1267
Failed transactions:               0
Longest transaction:            2.55
Shortest transaction:           0.04

Benchmarking 4096 bit RSA key without HSM

Self signed with OpenSSL:

#key + certificate
openssl req -nodes -x509 -sha256 -newkey rsa:4096 -keyout "pi1.hsmcluster.nl.key" -out "pi1.hsmcluster.nl.cert" -days 365 -subj "/C=NL/ST=Zuid Holland/L=Rotterdam/O=Sparkling Network/OU=IT Dept/CN=pi1.hsmcluster.nl"

A siege test with 5 concurrent users, 30 seconds:

siege -c5 -d5 -t30S https://hsmcluster.nl

Result:

Transactions:                     51 hits
Availability:                 100.00 %
Elapsed time:                  29.47 secs
Data transferred:               0.00 MB
Response time:                  0.22 secs
Transaction rate:               1.73 trans/sec
Throughput:                     0.00 MB/sec
Concurrency:                    0.38
Successful transactions:          51
Failed transactions:               0
Longest transaction:            0.45
Shortest transaction:           0.19

10 concurrent users:

siege -c10 -d5 -t30S https://hsmcluster.nl

Result:

Transactions:                    117 hits
Availability:                 100.00 %
Elapsed time:                  29.85 secs
Data transferred:               0.00 MB
Response time:                  0.29 secs
Transaction rate:               3.92 trans/sec
Throughput:                     0.00 MB/sec
Concurrency:                    1.14
Successful transactions:         117
Failed transactions:               0
Longest transaction:            0.87
Shortest transaction:           0.19  

20 concurrent users:

siege -c20 -d5 -t30S https://hsmcluster.nl

Result:

Transactions:                    234 hits
Availability:                 100.00 %
Elapsed time:                  29.30 secs
Data transferred:               0.00 MB
Response time:                  0.29 secs
Transaction rate:               7.99 trans/sec
Throughput:                     0.00 MB/sec
Concurrency:                    2.31
Successful transactions:         234
Failed transactions:               0
Longest transaction:            1.14
Shortest transaction:           0.19

60 benchmark mode:

siege -c60 -b -t30S https://hsmcluster.nl

Result:

Transactions:                    591 hits
Availability:                 100.00 %
Elapsed time:                  29.94 secs
Data transferred:               0.01 MB
Response time:                  2.91 secs
Transaction rate:              19.74 trans/sec
Throughput:                     0.00 MB/sec
Concurrency:                   57.35
Successful transactions:         592
Failed transactions:               0
Longest transaction:            3.82
Shortest transaction:           0.40

Wordpress site with 10 concurrent users:

siege -c10 -d5 -t30S 'https://hsmcluster.nl/wordpress/'

Result:

Transactions:                    483 hits
Availability:                 100.00 %
Elapsed time:                  29.22 secs
Data transferred:               3.53 MB
Response time:                  0.27 secs
Transaction rate:              16.53 trans/sec
Throughput:                     0.12 MB/sec
Concurrency:                    4.46
Successful transactions:         483
Failed transactions:               0
Longest transaction:            1.98
Shortest transaction:           0.04

Wordpress site with 20 concurrent users:

siege -c20 -d5 -t30S 'https://hsmcluster.nl/wordpress/'

Result:

Transactions:                    577 hits
Availability:                 100.00 %
Elapsed time:                  29.69 secs
Data transferred:               4.08 MB
Response time:                  0.67 secs
Transaction rate:              19.43 trans/sec
Throughput:                     0.14 MB/sec
Concurrency:                   13.10
Successful transactions:         577
Failed transactions:               0
Longest transaction:            2.37
Shortest transaction:           0.04

Benchmarking 8192 bit RSA key without HSM

Self signed with OpenSSL:

#key + certificate
openssl req -nodes -x509 -sha256 -newkey rsa:8192 -keyout "pi1.hsmcluster.nl.key" -out "pi1.hsmcluster.nl.cert" -days 365 -subj "/C=NL/ST=Zuid Holland/L=Rotterdam/O=Sparkling Network/OU=IT Dept/CN=pi1.hsmcluster.nl"

A siege test with 5 concurrent users, 30 seconds:

siege -c5 -d5 -t30S https://hsmcluster.nl

Result:

Transactions:                     39 hits
Availability:                 100.00 %
Elapsed time:                  29.84 secs
Data transferred:               0.00 MB
Response time:                  1.36 secs
Transaction rate:               1.31 trans/sec
Throughput:                     0.00 MB/sec
Concurrency:                    1.77
Successful transactions:          39
Failed transactions:               0
Longest transaction:            1.81
Shortest transaction:           1.30

10 concurrent users:

siege -c10 -d5 -t30S https://hsmcluster.nl

Result:

Transactions:                     70 hits
Availability:                 100.00 %
Elapsed time:                  29.94 secs
Data transferred:               0.00 MB
Response time:                  1.81 secs
Transaction rate:               2.34 trans/sec
Throughput:                     0.00 MB/sec
Concurrency:                    4.22
Successful transactions:          70
Failed transactions:               0
Longest transaction:            3.38
Shortest transaction:           1.30

20 concurrent users:

siege -c20 -d5 -t30S https://hsmcluster.nl

Result:

Transactions:                     71 hits
Availability:                 100.00 %
Elapsed time:                  29.25 secs
Data transferred:               0.00 MB
Response time:                  5.13 secs
Transaction rate:               2.43 trans/sec
Throughput:                     0.00 MB/sec
Concurrency:                   12.44
Successful transactions:          71
Failed transactions:               0
Longest transaction:            6.78
Shortest transaction:           1.36

60 benchmark mode:

siege -c60 -b -t30S https://hsmcluster.nl

This benchmark was done against THREE Pi's, one or two would fail and give the Pi a load of +100.

Result:

Transactions:                    176 hits
Availability:                 100.00 %
Elapsed time:                  29.75 secs
Data transferred:               0.00 MB
Response time:                  5.44 secs
Transaction rate:               5.92 trans/sec
Throughput:                     0.00 MB/sec
Concurrency:                   32.21
Successful transactions:         176
Failed transactions:               0
Longest transaction:           22.08
Shortest transaction:           1.30

Wordpress site with 10 concurrent users:

siege -c10 -d5 -t30S 'https://hsmcluster.nl/wordpress/'

Result:

Transactions:                     40 hits
Availability:                 100.00 %
Elapsed time:                  29.05 secs
Data transferred:               0.08 MB
Response time:                  6.76 secs
Transaction rate:               1.38 trans/sec
Throughput:                     0.00 MB/sec
Concurrency:                    9.31
Successful transactions:          40
Failed transactions:               0
Longest transaction:            7.89
Shortest transaction:           5.62

Wordpress site with 20 concurrent users:

siege -c20 -d5 -t30S 'https://hsmcluster.nl/wordpress/'

Result:

Transactions:                     62 hits
Availability:                 100.00 %
Elapsed time:                  29.39 secs
Data transferred:               0.10 MB
Response time:                  7.28 secs
Transaction rate:               2.11 trans/sec
Throughput:                     0.00 MB/sec
Concurrency:                   15.36
Successful transactions:          62
Failed transactions:               0
Longest transaction:            8.41
Shortest transaction:           5.33

Benchmarking prime256v1 (NIST curve P-256) EC key without HSM

Self signed with OpenSSL:

#key
openssl ecparam -out ec_key.pem -name pi1.hsmcluster.nl.key -name prime256v1 -genkey
#certificate
openssl req -new -key pi1.hsmcluster.nl.key -x509 -nodes -days 365 -out pi1.hsmcluster.nl.cert -subj "/C=NL/ST=Zuid Holland/L=Rotterdam/O=Sparkling Network/OU=IT Dept/CN=pi1.hsmcluster.nl"

A siege test with 5 concurrent users, 30 seconds:

siege -c5 -d5 -t30S https://hsmcluster.nl

Result:

Transactions:                     41 hits
Availability:                 100.00 %
Elapsed time:                  29.79 secs
Data transferred:               0.00 MB
Response time:                  1.37 secs
Transaction rate:               1.38 trans/sec
Throughput:                     0.00 MB/sec
Concurrency:                    1.89
Successful transactions:          41
Failed transactions:               0
Longest transaction:            2.12
Shortest transaction:           1.29

10 concurrent users:

siege -c10 -d5 -t30S https://hsmcluster.nl

Result:

Transactions:                     70 hits
Availability:                 100.00 %
Elapsed time:                  29.31 secs
Data transferred:               0.00 MB
Response time:                  1.72 secs
Transaction rate:               2.39 trans/sec
Throughput:                     0.00 MB/sec
Concurrency:                    4.11
Successful transactions:          70
Failed transactions:               0
Longest transaction:            3.46
Shortest transaction:           1.29

20 concurrent users:

siege -c20 -d5 -t30S https://hsmcluster.nl

Result:

Transactions:                     76 hits
Availability:                 100.00 %
Elapsed time:                  29.35 secs
Data transferred:               0.00 MB
Response time:                  5.10 secs
Transaction rate:               2.59 trans/sec
Throughput:                     0.00 MB/sec
Concurrency:                   13.21
Successful transactions:          76
Failed transactions:               0
Longest transaction:            6.59
Shortest transaction:           1.33

60 benchmark mode:

siege -c60 -b -t30S https://hsmcluster.nl

This benchmark was done against THREE Pi's, one or two would fail and give the Pi a load of +100.

Result:

Transactions:                     60 hits
Availability:                 100.00 %
Elapsed time:                  29.66 secs
Data transferred:               0.00 MB
Response time:                 23.60 secs
Transaction rate:               2.02 trans/sec
Throughput:                     0.00 MB/sec
Concurrency:                   47.74
Successful transactions:          60
Failed transactions:               0
Longest transaction:           26.21
Shortest transaction:          20.71

Wordpress site with 10 concurrent users:

siege -c10 -d5 -t30S 'https://hsmcluster.nl/wordpress/'

Result:

Transactions:                     86 hits
Availability:                 100.00 %
Elapsed time:                  29.18 secs
Data transferred:               0.55 MB
Response time:                  2.93 secs
Transaction rate:               2.95 trans/sec
Throughput:                     0.02 MB/sec
Concurrency:                    8.63
Successful transactions:          86
Failed transactions:               0
Longest transaction:            4.49
Shortest transaction:           0.05

Wordpress site with 20 concurrent users:

siege -c20 -d5 -t30S 'https://hsmcluster.nl/wordpress/'

Result:

Transactions:                     80 hits
Availability:                 100.00 %
Elapsed time:                  29.19 secs
Data transferred:               0.17 MB
Response time:                  6.75 secs
Transaction rate:               2.74 trans/sec
Throughput:                     0.01 MB/sec
Concurrency:                   18.50
Successful transactions:          80
Failed transactions:               0
Longest transaction:            8.17
Shortest transaction:           6.10

OpenSC benchmarks

The below benchmarks utilize the opensc-pkcs11 module with mod_nss. We're benchmarking one small page with only the contents 'Jeej it works!'. We're also benchmarking a Wordpress 4.5.3 install with the default content after install and the Hemmingway theme.

Note to self, the below command formats the siege output directly to space- seperated (instead of tab) markdown code output:

siege  -c10 -d5 -t30S 'https://hsmcluster.nl/' 2>&1 | grep -v '==> GET ' | expand | sed 's/^/    /'

1 HSM (OpenSC)

Benchmarking 1024 bit RSA key with 1 HSM (OpenSC)

A siege test with 5 concurrent users, 30 seconds:

siege -c5 -d5 -t30S https://hsmcluster.nl

Result:

Transactions:                     52 hits
Availability:                 100.00 %
Elapsed time:                  29.52 secs
Data transferred:               0.00 MB
Response time:                  0.33 secs
Transaction rate:               1.76 trans/sec
Throughput:                     0.00 MB/sec
Concurrency:                    0.57
Successful transactions:          52
Failed transactions:               0
Longest transaction:            1.06
Shortest transaction:           0.22

10 concurrent users:

siege -c10 -d5 -t30S https://hsmcluster.nl

Result:

Transactions:                    101 hits
Availability:                 100.00 %
Elapsed time:                  29.07 secs
Data transferred:               0.00 MB
Response time:                  0.52 secs
Transaction rate:               3.47 trans/sec
Throughput:                     0.00 MB/sec
Concurrency:                    1.80
Successful transactions:         101
Failed transactions:               0
Longest transaction:            2.06
Shortest transaction:           0.22

20 concurrent users:

siege -c20 -d5 -t30S https://hsmcluster.nl

Result:

Transactions:                    142 hits
Availability:                 100.00 %
Elapsed time:                  29.86 secs
Data transferred:               0.00 MB
Response time:                  1.78 secs
Transaction rate:               4.76 trans/sec
Throughput:                     0.00 MB/sec
Concurrency:                    8.47
Successful transactions:         142
Failed transactions:               0
Longest transaction:            5.07
Shortest transaction:           0.23

60 benchmark mode:

siege -c60 -b -t30S https://hsmcluster.nl

Result:

Transactions:                    126 hits
Availability:                 100.00 %
Elapsed time:                  29.37 secs
Data transferred:               0.00 MB
Response time:                  8.96 secs
Transaction rate:               4.29 trans/sec
Throughput:                     0.00 MB/sec
Concurrency:                   38.46
Successful transactions:         126
Failed transactions:               0
Longest transaction:           19.99
Shortest transaction:           0.60

Wordpress site with 10 concurrent users:

siege -c10 -d5 -t30S 'https://hsmcluster.nl/wordpress/'

Result:

Transactions:                    162 hits
Availability:                 100.00 %
Elapsed time:                  29.13 secs
Data transferred:               1.14 MB
Response time:                  1.50 secs
Transaction rate:               5.56 trans/sec
Throughput:                     0.04 MB/sec
Concurrency:                    8.35
Successful transactions:         162
Failed transactions:               0
Longest transaction:            8.85
Shortest transaction:           0.04

Wordpress site with 20 concurrent users:

siege -c20 -d5 -t30S 'https://hsmcluster.nl/wordpress/'

Result:

Transactions:                    165 hits
Availability:                 100.00 %
Elapsed time:                  29.77 secs
Data transferred:               1.07 MB
Response time:                  3.08 secs
Transaction rate:               5.54 trans/sec
Throughput:                     0.04 MB/sec
Concurrency:                   17.06
Successful transactions:         165
Failed transactions:               0
Longest transaction:           10.86
Shortest transaction:           0.05

Benchmarking 2048 bit RSA key with 1 HSM (OpenSC)

A siege test with 5 concurrent users, 30 seconds:

siege -c5 -d5 -t30S https://hsmcluster.nl

Result:

Transactions:                     35 hits
Availability:                 100.00 %
Elapsed time:                  29.01 secs
Data transferred:               0.00 MB
Response time:                  1.23 secs
Transaction rate:               1.21 trans/sec
Throughput:                     0.00 MB/sec
Concurrency:                    1.49
Successful transactions:          35
Failed transactions:               0
Longest transaction:            3.48
Shortest transaction:           0.66

10 concurrent users:

siege -c10 -d5 -t30S https://hsmcluster.nl

Result:

Transactions:                     45 hits
Availability:                 100.00 %
Elapsed time:                  29.92 secs
Data transferred:               0.00 MB
Response time:                  3.79 secs
Transaction rate:               1.50 trans/sec
Throughput:                     0.00 MB/sec
Concurrency:                    5.70
Successful transactions:          45
Failed transactions:               0
Longest transaction:           17.70
Shortest transaction:           0.68

20 concurrent users:

siege -c20 -d5 -t30S https://hsmcluster.nl

Result:

Transactions:                     44 hits
Availability:                 100.00 %
Elapsed time:                  29.15 secs
Data transferred:               0.00 MB
Response time:                  8.83 secs
Transaction rate:               1.51 trans/sec
Throughput:                     0.00 MB/sec
Concurrency:                   13.32
Successful transactions:          44
Failed transactions:               0
Longest transaction:           27.20
Shortest transaction:           0.69

60 benchmark mode:

siege -c60 -b -t30S https://hsmcluster.nl

Result:

Transactions:                     42 hits
Availability:                 100.00 %
Elapsed time:                  29.35 secs
Data transferred:               0.00 MB
Response time:                 12.53 secs
Transaction rate:               1.43 trans/sec
Throughput:                     0.00 MB/sec
Concurrency:                   17.94
Successful transactions:          42
Failed transactions:               0
Longest transaction:           28.97
Shortest transaction:           0.00

Wordpress site with 10 concurrent users:

siege -c10 -d5 -t30S 'https://hsmcluster.nl/wordpress/'

Result:

Transactions:                     49 hits
Availability:                 100.00 %
Elapsed time:                  29.24 secs
Data transferred:               0.24 MB
Response time:                  4.40 secs
Transaction rate:               1.68 trans/sec
Throughput:                     0.01 MB/sec
Concurrency:                    7.37
Successful transactions:          49
Failed transactions:               0
Longest transaction:           24.25
Shortest transaction:           0.04

Wordpress site with 20 concurrent users:

siege -c20 -d5 -t30S 'https://hsmcluster.nl/wordpress/'

Result:

Transactions:                     46 hits
Availability:                 100.00 %
Elapsed time:                  29.98 secs
Data transferred:               0.13 MB
Response time:                  9.63 secs
Transaction rate:               1.53 trans/sec
Throughput:                     0.00 MB/sec
Concurrency:                   14.77
Successful transactions:          46
Failed transactions:               0
Longest transaction:           29.81
Shortest transaction:           0.05

Benchmarking EC prime256v1 key with 1 HSM (OpenSC)

A siege test with 5 concurrent users, 30 seconds:

siege -c5 -d5 -t30S https://hsmcluster.nl

Result:

Transactions:                     52 hits
Availability:                 100.00 %
Elapsed time:                  29.95 secs
Data transferred:               0.00 MB
Response time:                  0.48 secs
Transaction rate:               1.74 trans/sec
Throughput:                     0.00 MB/sec
Concurrency:                    0.84
Successful transactions:          52
Failed transactions:               0
Longest transaction:            1.91
Shortest transaction:           0.27

10 concurrent users:

siege -c10 -d5 -t30S https://hsmcluster.nl

Result:

Transactions:                     92 hits
Availability:                 100.00 %
Elapsed time:                  29.49 secs
Data transferred:               0.00 MB
Response time:                  1.01 secs
Transaction rate:               3.12 trans/sec
Throughput:                     0.00 MB/sec
Concurrency:                    3.16
Successful transactions:          92
Failed transactions:               0
Longest transaction:            4.65
Shortest transaction:           0.27

20 concurrent users:

siege -c20 -d5 -t30S https://hsmcluster.nl

Result:

Transactions:                    103 hits
Availability:                 100.00 %
Elapsed time:                  29.96 secs
Data transferred:               0.00 MB
Response time:                  3.35 secs
Transaction rate:               3.44 trans/sec
Throughput:                     0.00 MB/sec
Concurrency:                   11.53
Successful transactions:         103
Failed transactions:               0
Longest transaction:            8.37
Shortest transaction:           0.28

60 benchmark mode:

siege -c60 -b -t30S https://hsmcluster.nl

Result:

Transactions:                     97 hits
Availability:                 100.00 %
Elapsed time:                  29.90 secs
Data transferred:               0.00 MB
Response time:                 10.76 secs
Transaction rate:               3.24 trans/sec
Throughput:                     0.00 MB/sec
Concurrency:                   34.89
Successful transactions:          97
Failed transactions:               0
Longest transaction:           29.19
Shortest transaction:           0.00

Wordpress site with 10 concurrent users:

siege -c10 -d5 -t30S 'https://hsmcluster.nl/wordpress/'

Result:

Transactions:                    120 hits
Availability:                 100.00 %
Elapsed time:                  29.84 secs
Data transferred:               0.80 MB
Response time:                  2.11 secs
Transaction rate:               4.02 trans/sec
Throughput:                     0.03 MB/sec
Concurrency:                    8.49
Successful transactions:         120
Failed transactions:               0
Longest transaction:            8.04
Shortest transaction:           0.04

Wordpress site with 20 concurrent users:

siege -c20 -d5 -t30S 'https://hsmcluster.nl/wordpress/'

Result:

Transactions:                    116 hits
Availability:                 100.00 %
Elapsed time:                  29.87 secs
Data transferred:               0.59 MB
Response time:                  4.25 secs
Transaction rate:               3.88 trans/sec
Throughput:                     0.02 MB/sec
Concurrency:                   16.52
Successful transactions:         116
Failed transactions:               0
Longest transaction:           14.15
Shortest transaction:           0.05

2 HSM's (OpenSC)

Benchmarking 1024 bit RSA key with 2 HSM's (OpenSC)

A siege test with 5 concurrent users, 30 seconds:

siege -c5 -d5 -t30S https://hsmcluster.nl

Result:

Transactions:                     51 hits
Availability:                 100.00 %
Elapsed time:                  29.17 secs
Data transferred:               0.00 MB
Response time:                  0.28 secs
Transaction rate:               1.75 trans/sec
Throughput:                     0.00 MB/sec
Concurrency:                    0.49
Successful transactions:          51
Failed transactions:               0
Longest transaction:            1.06
Shortest transaction:           0.22

10 concurrent users:

siege -c10 -d5 -t30S https://hsmcluster.nl

Result:

Transactions:                    107 hits
Availability:                 100.00 %
Elapsed time:                  29.23 secs
Data transferred:               0.00 MB
Response time:                  0.30 secs
Transaction rate:               3.66 trans/sec
Throughput:                     0.00 MB/sec
Concurrency:                    1.09
Successful transactions:         107
Failed transactions:               0
Longest transaction:            1.31
Shortest transaction:           0.22

20 concurrent users:

siege -c20 -d5 -t30S https://hsmcluster.nl

Result:

Transactions:                    217 hits
Availability:                 100.00 %
Elapsed time:                  29.96 secs
Data transferred:               0.00 MB
Response time:                  0.58 secs
Transaction rate:               7.24 trans/sec
Throughput:                     0.00 MB/sec
Concurrency:                    4.21
Successful transactions:         217
Failed transactions:               0
Longest transaction:            4.26
Shortest transaction:           0.22

60 benchmark mode:

siege -c60 -b -t30S https://hsmcluster.nl

Result:

Transactions:                    277 hits
Availability:                 100.00 %
Elapsed time:                  29.72 secs
Data transferred:               0.00 MB
Response time:                  5.66 secs
Transaction rate:               9.32 trans/sec
Throughput:                     0.00 MB/sec
Concurrency:                   52.77
Successful transactions:         277
Failed transactions:               0
Longest transaction:           18.27
Shortest transaction:           0.47

Wordpress site with 10 concurrent users:

siege -c10 -d5 -t30S 'https://hsmcluster.nl/wordpress/'

Result:

Transactions:                    286 hits
Availability:                 100.00 %
Elapsed time:                  29.76 secs
Data transferred:               2.04 MB
Response time:                  0.78 secs
Transaction rate:               9.61 trans/sec
Throughput:                     0.07 MB/sec
Concurrency:                    7.45
Successful transactions:         286
Failed transactions:               0
Longest transaction:            4.67
Shortest transaction:           0.04

Wordpress site with 20 concurrent users:

siege -c20 -d5 -t30S 'https://hsmcluster.nl/wordpress/'

Result:

Transactions:                    298 hits
Availability:                 100.00 %
Elapsed time:                  29.12 secs
Data transferred:               2.04 MB
Response time:                  1.51 secs
Transaction rate:              10.23 trans/sec
Throughput:                     0.07 MB/sec
Concurrency:                   15.49
Successful transactions:         298
Failed transactions:               0
Longest transaction:            9.48
Shortest transaction:           0.04

Benchmarking 2048 bit RSA key with 2 HSM's (OpenSC)

A siege test with 5 concurrent users, 30 seconds:

siege -c5 -d5 -t30S https://hsmcluster.nl

Result:

Transactions:                     47 hits
Availability:                 100.00 %
Elapsed time:                  29.00 secs
Data transferred:               0.00 MB
Response time:                  1.01 secs
Transaction rate:               1.62 trans/sec
Throughput:                     0.00 MB/sec
Concurrency:                    1.63
Successful transactions:          47
Failed transactions:               0
Longest transaction:            3.08
Shortest transaction:           0.66

10 concurrent users:

siege -c10 -d5 -t30S https://hsmcluster.nl

Result:

Transactions:                     75 hits
Availability:                 100.00 %
Elapsed time:                  29.05 secs
Data transferred:               0.00 MB
Response time:                  1.58 secs
Transaction rate:               2.58 trans/sec
Throughput:                     0.00 MB/sec
Concurrency:                    4.07
Successful transactions:          75
Failed transactions:               0
Longest transaction:            7.71
Shortest transaction:           0.66

20 concurrent users:

siege -c20 -d5 -t30S https://hsmcluster.nl

Result:

Transactions:                     82 hits
Availability:                 100.00 %
Elapsed time:                  29.74 secs
Data transferred:               0.00 MB
Response time:                  4.25 secs
Transaction rate:               2.76 trans/sec
Throughput:                     0.00 MB/sec
Concurrency:                   11.72
Successful transactions:          82
Failed transactions:               0
Longest transaction:           17.67
Shortest transaction:           0.67

60 benchmark mode:

siege -c60 -b -t30S https://hsmcluster.nl

Result:

Transactions:                     46 hits
Availability:                 100.00 %
Elapsed time:                  29.11 secs
Data transferred:               0.00 MB
Response time:                 15.96 secs
Transaction rate:               1.58 trans/sec
Throughput:                     0.00 MB/sec
Concurrency:                   25.22
Successful transactions:          46
Failed transactions:               0
Longest transaction:           28.88
Shortest transaction:           2.22

Wordpress site with 10 concurrent users:

siege -c10 -d5 -t30S 'https://hsmcluster.nl/wordpress/'

Result:

Transactions:                     92 hits
Availability:                 100.00 %
Elapsed time:                  29.44 secs
Data transferred:               0.61 MB
Response time:                  2.61 secs
Transaction rate:               3.12 trans/sec
Throughput:                     0.02 MB/sec
Concurrency:                    8.16
Successful transactions:          92
Failed transactions:               0
Longest transaction:           13.43
Shortest transaction:           0.05

Wordpress site with 20 concurrent users:

siege -c20 -d5 -t30S 'https://hsmcluster.nl/wordpress/'

Result:

Transactions:                     75 hits
Availability:                 100.00 %
Elapsed time:                  29.60 secs
Data transferred:               0.41 MB
Response time:                  5.35 secs
Transaction rate:               2.53 trans/sec
Throughput:                     0.01 MB/sec
Concurrency:                   13.56
Successful transactions:          75
Failed transactions:               0
Longest transaction:           21.47
Shortest transaction:           0.00

Benchmarking EC prime256v1 key with 2 HSM's (OpenSC)

A siege test with 5 concurrent users, 30 seconds:

siege -c5 -d5 -t30S https://hsmcluster.nl

Result:

Transactions:                     60 hits
Availability:                 100.00 %
Elapsed time:                  29.30 secs
Data transferred:               0.00 MB
Response time:                  0.31 secs
Transaction rate:               2.05 trans/sec
Throughput:                     0.00 MB/sec
Concurrency:                    0.64
Successful transactions:          60
Failed transactions:               0
Longest transaction:            1.09
Shortest transaction:           0.26

10 concurrent users:

siege -c10 -d5 -t30S https://hsmcluster.nl

Result:

Transactions:                    106 hits
Availability:                 100.00 %
Elapsed time:                  29.71 secs
Data transferred:               0.00 MB
Response time:                  0.42 secs
Transaction rate:               3.57 trans/sec
Throughput:                     0.00 MB/sec
Concurrency:                    1.50
Successful transactions:         106
Failed transactions:               0
Longest transaction:            1.58
Shortest transaction:           0.26

20 concurrent users:

siege -c20 -d5 -t30S https://hsmcluster.nl

Result:

Transactions:                    184 hits
Availability:                 100.00 %
Elapsed time:                  29.61 secs
Data transferred:               0.00 MB
Response time:                  0.80 secs
Transaction rate:               6.21 trans/sec
Throughput:                     0.00 MB/sec
Concurrency:                    4.99
Successful transactions:         184
Failed transactions:               0
Longest transaction:            4.12
Shortest transaction:           0.26  

60 benchmark mode:

siege -c60 -b -t30S https://hsmcluster.nl

Result:

Transactions:                    201 hits
Availability:                 100.00 %
Elapsed time:                  29.50 secs
Data transferred:               0.00 MB
Response time:                  5.56 secs
Transaction rate:               6.81 trans/sec
Throughput:                     0.00 MB/sec
Concurrency:                   37.90
Successful transactions:         201
Failed transactions:               0
Longest transaction:           27.49
Shortest transaction:           0.26

Wordpress site with 10 concurrent users:

siege -c10 -d5 -t30S 'https://hsmcluster.nl/wordpress/'

Result:

Transactions:                    237 hits
Availability:                 100.00 %
Elapsed time:                  29.56 secs
Data transferred:               1.69 MB
Response time:                  0.90 secs
Transaction rate:               8.02 trans/sec
Throughput:                     0.06 MB/sec
Concurrency:                    7.25
Successful transactions:         237
Failed transactions:               0
Longest transaction:            4.72
Shortest transaction:           0.04

Wordpress site with 20 concurrent users:

siege -c20 -d5 -t30S 'https://hsmcluster.nl/wordpress/'

Result:

Transactions:                    240 hits
Availability:                 100.00 %
Elapsed time:                  29.28 secs
Data transferred:               1.67 MB
Response time:                  2.08 secs
Transaction rate:               8.20 trans/sec
Throughput:                     0.06 MB/sec
Concurrency:                   17.03
Successful transactions:         240
Failed transactions:               0
Longest transaction:            7.64
Shortest transaction:           0.04

3 HSM's (OpenSC)

Benchmarking 1024 bit RSA key with 3 HSM's (OpenSC)

A siege test with 5 concurrent users, 30 seconds:

siege -c5 -d5 -t30S https://hsmcluster.nl

Result:

Transactions:                     54 hits
Availability:                 100.00 %
Elapsed time:                  29.88 secs
Data transferred:               0.00 MB
Response time:                  0.25 secs
Transaction rate:               1.81 trans/sec
Throughput:                     0.00 MB/sec
Concurrency:                    0.46
Successful transactions:          54
Failed transactions:               0
Longest transaction:            0.66
Shortest transaction:           0.22

10 concurrent users:

siege -c10 -d5 -t30S https://hsmcluster.nl

Result:

Transactions:                    118 hits
Availability:                 100.00 %
Elapsed time:                  29.72 secs
Data transferred:               0.00 MB
Response time:                  0.28 secs
Transaction rate:               3.97 trans/sec
Throughput:                     0.00 MB/sec
Concurrency:                    1.12
Successful transactions:         118
Failed transactions:               0
Longest transaction:            1.66
Shortest transaction:           0.22 

20 concurrent users:

siege -c20 -d5 -t30S https://hsmcluster.nl

Result:

Transactions:                    205 hits
Availability:                 100.00 %
Elapsed time:                  29.85 secs
Data transferred:               0.00 MB
Response time:                  0.32 secs
Transaction rate:               6.87 trans/sec
Throughput:                     0.00 MB/sec
Concurrency:                    2.19
Successful transactions:         206
Failed transactions:               0
Longest transaction:            1.69
Shortest transaction:           0.22

60 benchmark mode:

siege -c60 -b -t30S https://hsmcluster.nl

Result:

Transactions:                    407 hits
Availability:                 100.00 %
Elapsed time:                  29.07 secs
Data transferred:               0.01 MB
Response time:                  3.65 secs
Transaction rate:              14.00 trans/sec
Throughput:                     0.00 MB/sec
Concurrency:                   51.11
Successful transactions:         407
Failed transactions:               0
Longest transaction:           17.38
Shortest transaction:           0.25

Wordpress site with 10 concurrent users:

siege -c10 -d5 -t30S 'https://hsmcluster.nl/wordpress/'

Result:

Transactions:                    382 hits
Availability:                 100.00 %
Elapsed time:                  29.48 secs
Data transferred:               2.73 MB
Response time:                  0.40 secs
Transaction rate:              12.96 trans/sec
Throughput:                     0.09 MB/sec
Concurrency:                    5.22
Successful transactions:         382
Failed transactions:               0
Longest transaction:            1.99
Shortest transaction:           0.04

Wordpress site with 20 concurrent users:

siege -c20 -d5 -t30S 'https://hsmcluster.nl/wordpress/'

Result:

Transactions:                    449 hits
Availability:                 100.00 %
Elapsed time:                  29.31 secs
Data transferred:               3.18 MB
Response time:                  0.99 secs
Transaction rate:              15.32 trans/sec
Throughput:                     0.11 MB/sec
Concurrency:                   15.12
Successful transactions:         449
Failed transactions:               0
Longest transaction:            6.49
Shortest transaction:           0.04

Benchmarking 2048 bit RSA key with 3 HSM's (OpenSC)

A siege test with 5 concurrent users, 30 seconds:

siege -c5 -d5 -t30S https://hsmcluster.nl

Result:

Transactions:                     44 hits
Availability:                 100.00 %
Elapsed time:                  29.10 secs
Data transferred:               0.00 MB
Response time:                  0.75 secs
Transaction rate:               1.51 trans/sec
Throughput:                     0.00 MB/sec
Concurrency:                    1.13
Successful transactions:          44
Failed transactions:               0
Longest transaction:            1.50
Shortest transaction:           0.66

10 concurrent users:

siege -c10 -d5 -t30S https://hsmcluster.nl

Result:

Transactions:                     89 hits
Availability:                 100.00 %
Elapsed time:                  29.34 secs
Data transferred:               0.00 MB
Response time:                  0.90 secs
Transaction rate:               3.03 trans/sec
Throughput:                     0.00 MB/sec
Concurrency:                    2.72
Successful transactions:          89
Failed transactions:               0
Longest transaction:            2.67
Shortest transaction:           0.66

20 concurrent users:

siege -c20 -d5 -t30S https://hsmcluster.nl

Result:

Transactions:                    127 hits
Availability:                 100.00 %
Elapsed time:                  29.99 secs
Data transferred:               0.00 MB
Response time:                  2.05 secs
Transaction rate:               4.23 trans/sec
Throughput:                     0.00 MB/sec
Concurrency:                    8.69
Successful transactions:         127
Failed transactions:               0
Longest transaction:           11.20
Shortest transaction:           0.66

60 benchmark mode:

siege -c60 -b -t30S https://hsmcluster.nl

Result:

Transactions:                    128 hits
Availability:                 100.00 %
Elapsed time:                  29.53 secs
Data transferred:               0.00 MB
Response time:                  9.24 secs
Transaction rate:               4.33 trans/sec
Throughput:                     0.00 MB/sec
Concurrency:                   40.07
Successful transactions:         128
Failed transactions:               0
Longest transaction:           23.46
Shortest transaction:           0.66

Wordpress site with 10 concurrent users:

siege -c10 -d5 -t30S 'https://hsmcluster.nl/wordpress/'

Result:

Transactions:                    139 hits
Availability:                 100.00 %
Elapsed time:                  29.43 secs
Data transferred:               0.97 MB
Response time:                  1.79 secs
Transaction rate:               4.72 trans/sec
Throughput:                     0.03 MB/sec
Concurrency:                    8.48
Successful transactions:         139
Failed transactions:               0
Longest transaction:           11.38
Shortest transaction:           0.04

Wordpress site with 20 concurrent users:

siege -c20 -d5 -t30S 'https://hsmcluster.nl/wordpress/'

Result:

Transactions:                    140 hits
Availability:                 100.00 %
Elapsed time:                  29.39 secs
Data transferred:               0.89 MB
Response time:                  3.58 secs
Transaction rate:               4.76 trans/sec
Throughput:                     0.03 MB/sec
Concurrency:                   17.04
Successful transactions:         140
Failed transactions:               0
Longest transaction:           21.96
Shortest transaction:           0.05

Benchmarking EC prime256v1 key with 3 HSM's (OpenSC)

A siege test with 5 concurrent users, 30 seconds:

siege -c5 -d5 -t30S https://hsmcluster.nl

Result:

Transactions:                     51 hits
Availability:                 100.00 %
Elapsed time:                  29.73 secs
Data transferred:               0.00 MB
Response time:                  0.31 secs
Transaction rate:               1.72 trans/sec
Throughput:                     0.00 MB/sec
Concurrency:                    0.53
Successful transactions:          51
Failed transactions:               0
Longest transaction:            0.67
Shortest transaction:           0.25    

10 concurrent users:

siege -c10 -d5 -t30S https://hsmcluster.nl

Result:

Transactions:                    110 hits
Availability:                 100.00 %
Elapsed time:                  29.41 secs
Data transferred:               0.00 MB
Response time:                  0.34 secs
Transaction rate:               3.74 trans/sec
Throughput:                     0.00 MB/sec
Concurrency:                    1.26
Successful transactions:         110
Failed transactions:               0
Longest transaction:            1.29
Shortest transaction:           0.25

20 concurrent users:

siege -c20 -d5 -t30S https://hsmcluster.nl

Result:

Transactions:                    221 hits
Availability:                 100.00 %
Elapsed time:                  29.82 secs
Data transferred:               0.00 MB
Response time:                  0.44 secs
Transaction rate:               7.41 trans/sec
Throughput:                     0.00 MB/sec
Concurrency:                    3.24
Successful transactions:         221
Failed transactions:               0
Longest transaction:            2.12
Shortest transaction:           0.24

60 benchmark mode:

siege -c60 -b -t30S https://hsmcluster.nl

Result:

Transactions:                    331 hits
Availability:                 100.00 %
Elapsed time:                  29.64 secs
Data transferred:               0.00 MB
Response time:                  4.29 secs
Transaction rate:              11.17 trans/sec
Throughput:                     0.00 MB/sec
Concurrency:                   47.90
Successful transactions:         331
Failed transactions:               0
Longest transaction:           25.01
Shortest transaction:           0.25

Wordpress site with 10 concurrent users:

siege -c10 -d5 -t30S 'https://hsmcluster.nl/wordpress/'

Result:

Transactions:                    336 hits
Availability:                 100.00 %
Elapsed time:                  29.48 secs
Data transferred:               2.42 MB
Response time:                  0.54 secs
Transaction rate:              11.40 trans/sec
Throughput:                     0.08 MB/sec
Concurrency:                    6.10
Successful transactions:         336
Failed transactions:               0
Longest transaction:            2.70
Shortest transaction:           0.04

Wordpress site with 20 concurrent users:

siege -c20 -d5 -t30S 'https://hsmcluster.nl/wordpress/'

Result:

Transactions:                    377 hits
Availability:                 100.00 %
Elapsed time:                  29.86 secs
Data transferred:               2.59 MB
Response time:                  1.21 secs
Transaction rate:              12.63 trans/sec
Throughput:                     0.09 MB/sec
Concurrency:                   15.27
Successful transactions:         377
Failed transactions:               0
Longest transaction:            5.02
Shortest transaction:           0.04

sc-hsm-embedded benchmarks

The below benchmarks utilize the read only libsc-hsm-embedded module with mod_nss. Read more on the sc-hsm-embedded module here.

This module is targeted at embedded use in devices, but can also be used regularly. I recommend it over the OpenSC module, since in production you don't want to be able to write to the HSM. You should have a seperate, non-networked workstation for that.

1 HSM (sc-hsm-embedded)

Benchmarking 1024 bit RSA key with 1 HSM (sc-hsm-embedded)

A siege test with 5 concurrent users, 30 seconds:

siege -c5 -d5 -t30S https://hsmcluster.nl

Result:

Transactions:                     56 hits
Availability:                 100.00 %
Elapsed time:                  29.48 secs
Data transferred:               0.00 MB
Response time:                  0.31 secs
Transaction rate:               1.90 trans/sec
Throughput:                     0.00 MB/sec
Concurrency:                    0.60
Successful transactions:          56
Failed transactions:               0
Longest transaction:            1.13
Shortest transaction:           0.24

10 concurrent users:

siege -c10 -d5 -t30S https://hsmcluster.nl

Result:

Transactions:                    100 hits
Availability:                 100.00 %
Elapsed time:                  29.18 secs
Data transferred:               0.00 MB
Response time:                  0.52 secs
Transaction rate:               3.43 trans/sec
Throughput:                     0.00 MB/sec
Concurrency:                    1.78
Successful transactions:         100
Failed transactions:               0
Longest transaction:            2.24
Shortest transaction:           0.24

20 concurrent users:

siege -c20 -d5 -t30S https://hsmcluster.nl

Result:

Transactions:                    131 hits
Availability:                 100.00 %
Elapsed time:                  29.17 secs
Data transferred:               0.00 MB
Response time:                  2.04 secs
Transaction rate:               4.49 trans/sec
Throughput:                     0.00 MB/sec
Concurrency:                    9.17
Successful transactions:         131
Failed transactions:               0
Longest transaction:            6.56
Shortest transaction:           0.28

60 benchmark mode:

siege -c60 -b -t30S https://hsmcluster.nl

Result:

Transactions:                    124 hits
Availability:                 100.00 %
Elapsed time:                  29.50 secs
Data transferred:               0.00 MB
Response time:                 10.24 secs
Transaction rate:               4.20 trans/sec
Throughput:                     0.00 MB/sec
Concurrency:                   43.03
Successful transactions:         124
Failed transactions:               0
Longest transaction:           19.99
Shortest transaction:           0.94

Wordpress site with 10 concurrent users:

siege -c10 -d5 -t30S 'https://hsmcluster.nl/wordpress/'

Result:

Transactions:                    148 hits
Availability:                 100.00 %
Elapsed time:                  29.63 secs
Data transferred:               1.02 MB
Response time:                  1.68 secs
Transaction rate:               4.99 trans/sec
Throughput:                     0.03 MB/sec
Concurrency:                    8.39
Successful transactions:         148
Failed transactions:               0
Longest transaction:            5.48
Shortest transaction:           0.04

Wordpress site with 20 concurrent users:

siege -c20 -d5 -t30S 'https://hsmcluster.nl/wordpress/'

Result:

Transactions:                    144 hits
Availability:                 100.00 %
Elapsed time:                  29.12 secs
Data transferred:               0.98 MB
Response time:                  3.63 secs
Transaction rate:               4.95 trans/sec
Throughput:                     0.03 MB/sec
Concurrency:                   17.94
Successful transactions:         144
Failed transactions:               0
Longest transaction:           11.69
Shortest transaction:           0.04

Benchmarking 2048 bit RSA key with 1 HSM (sc-hsm-embedded)

A siege test with 5 concurrent users, 30 seconds:

siege -c5 -d5 -t30S https://hsmcluster.nl

Result:

Transactions:                     36 hits
Availability:                 100.00 %
Elapsed time:                  29.78 secs
Data transferred:               0.00 MB
Response time:                  1.59 secs
Transaction rate:               1.21 trans/sec
Throughput:                     0.00 MB/sec
Concurrency:                    1.93
Successful transactions:          36
Failed transactions:               0
Longest transaction:            3.89
Shortest transaction:           0.67

10 concurrent users:

siege -c10 -d5 -t30S https://hsmcluster.nl

Result:

Transactions:                     44 hits
Availability:                 100.00 %
Elapsed time:                  29.19 secs
Data transferred:               0.00 MB
Response time:                  4.21 secs
Transaction rate:               1.51 trans/sec
Throughput:                     0.00 MB/sec
Concurrency:                    6.34
Successful transactions:          44
Failed transactions:               0
Longest transaction:            9.43
Shortest transaction:           0.68

20 concurrent users:

siege -c20 -d5 -t30S https://hsmcluster.nl

Result:

Transactions:                     39 hits
Availability:                 100.00 %
Elapsed time:                  29.06 secs
Data transferred:               0.00 MB
Response time:                  9.92 secs
Transaction rate:               1.34 trans/sec
Throughput:                     0.00 MB/sec
Concurrency:                   13.32
Successful transactions:          39
Failed transactions:               0
Longest transaction:           16.25
Shortest transaction:           1.60

60 benchmark mode:

siege -c60 -b -t30S https://hsmcluster.nl

Result:

Transactions:                     23 hits
Availability:                 100.00 %
Elapsed time:                  29.82 secs
Data transferred:               0.00 MB
Response time:                 17.65 secs
Transaction rate:               0.77 trans/sec
Throughput:                     0.00 MB/sec
Concurrency:                   13.61
Successful transactions:          23
Failed transactions:               0
Longest transaction:           29.12
Shortest transaction:           0.00

Wordpress site with 10 concurrent users:

siege -c10 -d5 -t30S 'https://hsmcluster.nl/wordpress/'

Result:

Transactions:                     40 hits
Availability:                 100.00 %
Elapsed time:                  29.75 secs
Data transferred:               0.17 MB
Response time:                  6.87 secs
Transaction rate:               1.34 trans/sec
Throughput:                     0.01 MB/sec
Concurrency:                    9.23
Successful transactions:          40
Failed transactions:               0
Longest transaction:           16.12
Shortest transaction:           1.66

Wordpress site with 20 concurrent users:

siege -c20 -d5 -t30S 'https://hsmcluster.nl/wordpress/'

Result:

Transactions:                     33 hits
Availability:                 100.00 %
Elapsed time:                  29.01 secs
Data transferred:               0.07 MB
Response time:                 11.64 secs
Transaction rate:               1.14 trans/sec
Throughput:                     0.00 MB/sec
Concurrency:                   13.24
Successful transactions:          33
Failed transactions:               0
Longest transaction:           26.08
Shortest transaction:           3.18

Benchmarking EC prime256v1 key with 1 HSM (sc-hsm-embedded)

A siege test with 5 concurrent users, 30 seconds:

siege -c5 -d5 -t30S https://hsmcluster.nl

Result:

Transactions:                     59 hits
Availability:                 100.00 %
Elapsed time:                  29.15 secs
Data transferred:               0.00 MB
Response time:                  0.37 secs
Transaction rate:               2.02 trans/sec
Throughput:                     0.00 MB/sec
Concurrency:                    0.75
Successful transactions:          59
Failed transactions:               0
Longest transaction:            1.22
Shortest transaction:           0.25

10 concurrent users:

siege -c10 -d5 -t30S https://hsmcluster.nl

Result:

Transactions:                    103 hits
Availability:                 100.00 %
Elapsed time:                  29.67 secs
Data transferred:               0.00 MB
Response time:                  0.57 secs
Transaction rate:               3.47 trans/sec
Throughput:                     0.00 MB/sec
Concurrency:                    1.98
Successful transactions:         103
Failed transactions:               0
Longest transaction:            2.86
Shortest transaction:           0.25

20 concurrent users:

siege -c20 -d5 -t30S https://hsmcluster.nl

Result:

Transactions:                    122 hits
Availability:                 100.00 %
Elapsed time:                  29.85 secs
Data transferred:               0.00 MB
Response time:                  2.10 secs
Transaction rate:               4.09 trans/sec
Throughput:                     0.00 MB/sec
Concurrency:                    8.59
Successful transactions:         122
Failed transactions:               0
Longest transaction:            6.55
Shortest transaction:           0.30

60 benchmark mode:

siege -c60 -b -t30S https://hsmcluster.nl

Result:

Transactions:                    109 hits
Availability:                 100.00 %
Elapsed time:                  29.45 secs
Data transferred:               0.00 MB
Response time:                 11.17 secs
Transaction rate:               3.70 trans/sec
Throughput:                     0.00 MB/sec
Concurrency:                   41.33
Successful transactions:         109
Failed transactions:               0
Longest transaction:           23.97
Shortest transaction:           0.52

Wordpress site with 10 concurrent users:

siege -c10 -d5 -t30S 'https://hsmcluster.nl/wordpress/'

Result:

Transactions:                    137 hits
Availability:                 100.00 %
Elapsed time:                  29.04 secs
Data transferred:               0.93 MB
Response time:                  1.68 secs
Transaction rate:               4.72 trans/sec
Throughput:                     0.03 MB/sec
Concurrency:                    7.94
Successful transactions:         137
Failed transactions:               0
Longest transaction:            4.49
Shortest transaction:           0.04

Wordpress site with 20 concurrent users:

siege -c20 -d5 -t30S 'https://hsmcluster.nl/wordpress/'

Result:

Transactions:                    138 hits
Availability:                 100.00 %
Elapsed time:                  29.64 secs
Data transferred:               0.95 MB
Response time:                  3.96 secs
Transaction rate:               4.66 trans/sec
Throughput:                     0.03 MB/sec
Concurrency:                   18.44
Successful transactions:         138
Failed transactions:               0
Longest transaction:           12.19
Shortest transaction:           0.05

2 HSM's (sc-hsm-embedded)

Benchmarking 1024 bit RSA key with 2 HSM's (sc-hsm-embedded)

A siege test with 5 concurrent users, 30 seconds:

siege -c5 -d5 -t30S https://hsmcluster.nl

Result:

Transactions:                     59 hits
Availability:                 100.00 %
Elapsed time:                  29.34 secs
Data transferred:               0.00 MB
Response time:                  0.26 secs
Transaction rate:               2.01 trans/sec
Throughput:                     0.00 MB/sec
Concurrency:                    0.52
Successful transactions:          59
Failed transactions:               0
Longest transaction:            0.62
Shortest transaction:           0.22

10 concurrent users:

siege -c10 -d5 -t30S https://hsmcluster.nl

Result:

Transactions:                    109 hits
Availability:                 100.00 %
Elapsed time:                  29.04 secs
Data transferred:               0.00 MB
Response time:                  0.29 secs
Transaction rate:               3.75 trans/sec
Throughput:                     0.00 MB/sec
Concurrency:                    1.10
Successful transactions:         109
Failed transactions:               0
Longest transaction:            1.13
Shortest transaction:           0.22

20 concurrent users:

siege -c20 -d5 -t30S https://hsmcluster.nl

Result:

Transactions:                    211 hits
Availability:                 100.00 %
Elapsed time:                  29.36 secs
Data transferred:               0.00 MB
Response time:                  0.42 secs
Transaction rate:               7.19 trans/sec
Throughput:                     0.00 MB/sec
Concurrency:                    3.01
Successful transactions:         211
Failed transactions:               0
Longest transaction:            2.45
Shortest transaction:           0.22

60 benchmark mode:

siege -c60 -b -t30S https://hsmcluster.nl

Result:

Transactions:                    253 hits
Availability:                 100.00 %
Elapsed time:                  29.04 secs
Data transferred:               0.00 MB
Response time:                  5.24 secs
Transaction rate:               8.71 trans/sec
Throughput:                     0.00 MB/sec
Concurrency:                   45.62
Successful transactions:         253
Failed transactions:               0
Longest transaction:           18.60
Shortest transaction:           0.25

Wordpress site with 10 concurrent users:

siege -c10 -d5 -t30S 'https://hsmcluster.nl/wordpress/'

Result:

Transactions:                    308 hits
Availability:                 100.00 %
Elapsed time:                  29.22 secs
Data transferred:               2.20 MB
Response time:                  0.61 secs
Transaction rate:              10.54 trans/sec
Throughput:                     0.08 MB/sec
Concurrency:                    6.39
Successful transactions:         308
Failed transactions:               0
Longest transaction:            2.29
Shortest transaction:           0.04

Wordpress site with 20 concurrent users:

siege -c20 -d5 -t30S 'https://hsmcluster.nl/wordpress/'

Result:

Transactions:                    318 hits
Availability:                 100.00 %
Elapsed time:                  29.94 secs
Data transferred:               2.21 MB
Response time:                  1.53 secs
Transaction rate:              10.62 trans/sec
Throughput:                     0.07 MB/sec
Concurrency:                   16.23
Successful transactions:         318
Failed transactions:               0
Longest transaction:            7.03
Shortest transaction:           0.04

Benchmarking 2048 bit RSA key with 2 HSM's (sc-hsm-embedded)

A siege test with 5 concurrent users, 30 seconds:

siege -c5 -d5 -t30S https://hsmcluster.nl

Result:

Transactions:                     47 hits
Availability:                 100.00 %
Elapsed time:                  29.86 secs
Data transferred:               0.00 MB
Response time:                  0.82 secs
Transaction rate:               1.57 trans/sec
Throughput:                     0.00 MB/sec
Concurrency:                    1.29
Successful transactions:          47
Failed transactions:               0
Longest transaction:            1.93
Shortest transaction:           0.66

10 concurrent users:

siege -c10 -d5 -t30S https://hsmcluster.nl

Result:

Transactions:                     74 hits
Availability:                 100.00 %
Elapsed time:                  29.90 secs
Data transferred:               0.00 MB
Response time:                  1.50 secs
Transaction rate:               2.47 trans/sec
Throughput:                     0.00 MB/sec
Concurrency:                    3.71
Successful transactions:          74
Failed transactions:               0
Longest transaction:            4.43
Shortest transaction:           0.66

20 concurrent users:

siege -c20 -d5 -t30S https://hsmcluster.nl

Result:

Transactions:                     84 hits
Availability:                 100.00 %
Elapsed time:                  29.03 secs
Data transferred:               0.00 MB
Response time:                  4.29 secs
Transaction rate:               2.89 trans/sec
Throughput:                     0.00 MB/sec
Concurrency:                   12.42
Successful transactions:          84
Failed transactions:               0
Longest transaction:           14.70
Shortest transaction:           0.69

60 benchmark mode:

siege -c60 -b -t30S https://hsmcluster.nl

Result:

Transactions:                     72 hits
Availability:                 100.00 %
Elapsed time:                  29.87 secs
Data transferred:               0.00 MB
Response time:                 13.06 secs
Transaction rate:               2.41 trans/sec
Throughput:                     0.00 MB/sec
Concurrency:                   31.48
Successful transactions:          72
Failed transactions:               0
Longest transaction:           27.91
Shortest transaction:           2.27

Wordpress site with 10 concurrent users:

siege -c10 -d5 -t30S 'https://hsmcluster.nl/wordpress/'

Result:

Transactions:                     95 hits
Availability:                 100.00 %
Elapsed time:                  29.51 secs
Data transferred:               0.60 MB
Response time:                  2.75 secs
Transaction rate:               3.22 trans/sec
Throughput:                     0.02 MB/sec
Concurrency:                    8.86
Successful transactions:          95
Failed transactions:               0
Longest transaction:            8.09
Shortest transaction:           0.04

Wordpress site with 20 concurrent users:

siege -c20 -d5 -t30S 'https://hsmcluster.nl/wordpress/'

Result:

Transactions:                     87 hits
Availability:                 100.00 %
Elapsed time:                  29.89 secs
Data transferred:               0.41 MB
Response time:                  6.08 secs
Transaction rate:               2.91 trans/sec
Throughput:                     0.01 MB/sec
Concurrency:                   17.70
Successful transactions:          87
Failed transactions:               0
Longest transaction:           17.31
Shortest transaction:           0.05

Benchmarking EC prime256v1 key with 2 HSM's (sc-hsm-embedded)

A siege test with 5 concurrent users, 30 seconds:

siege -c5 -d5 -t30S https://hsmcluster.nl

Result:

Transactions:                     59 hits
Availability:                 100.00 %
Elapsed time:                  29.14 secs
Data transferred:               0.00 MB
Response time:                  0.29 secs
Transaction rate:               2.02 trans/sec
Throughput:                     0.00 MB/sec
Concurrency:                    0.59
Successful transactions:          59
Failed transactions:               0
Longest transaction:            0.71
Shortest transaction:           0.24

10 concurrent users:

siege -c10 -d5 -t30S https://hsmcluster.nl

Result:

Transactions:                     98 hits
Availability:                 100.00 %
Elapsed time:                  29.19 secs
Data transferred:               0.00 MB
Response time:                  0.34 secs
Transaction rate:               3.36 trans/sec
Throughput:                     0.00 MB/sec
Concurrency:                    1.13
Successful transactions:          98
Failed transactions:               0
Longest transaction:            1.20
Shortest transaction:           0.24

20 concurrent users:

siege -c20 -d5 -t30S https://hsmcluster.nl

Result:

Transactions:                    217 hits
Availability:                 100.00 %
Elapsed time:                  29.94 secs
Data transferred:               0.00 MB
Response time:                  0.52 secs
Transaction rate:               7.25 trans/sec
Throughput:                     0.00 MB/sec
Concurrency:                    3.78
Successful transactions:         217
Failed transactions:               0
Longest transaction:            2.39
Shortest transaction:           0.24

60 benchmark mode:

siege -c60 -b -t30S https://hsmcluster.nl

Result:

Transactions:                    232 hits
Availability:                 100.00 %
Elapsed time:                  29.22 secs
Data transferred:               0.00 MB
Response time:                  5.62 secs
Transaction rate:               7.94 trans/sec
Throughput:                     0.00 MB/sec
Concurrency:                   44.64
Successful transactions:         232
Failed transactions:               0
Longest transaction:           20.16
Shortest transaction:           0.27

Wordpress site with 10 concurrent users:

siege -c10 -d5 -t30S 'https://hsmcluster.nl/wordpress/'

Result:

Transactions:                    276 hits
Availability:                 100.00 %
Elapsed time:                  29.35 secs
Data transferred:               1.92 MB
Response time:                  0.72 secs
Transaction rate:               9.40 trans/sec
Throughput:                     0.07 MB/sec
Concurrency:                    6.74
Successful transactions:         276
Failed transactions:               0
Longest transaction:            2.42
Shortest transaction:           0.04

Wordpress site with 20 concurrent users:

siege -c20 -d5 -t30S 'https://hsmcluster.nl/wordpress/'

Result:

Transactions:                    288 hits
Availability:                 100.00 %
Elapsed time:                  29.86 secs
Data transferred:               1.94 MB
Response time:                  1.77 secs
Transaction rate:               9.65 trans/sec
Throughput:                     0.06 MB/sec
Concurrency:                   17.08
Successful transactions:         288
Failed transactions:               0
Longest transaction:            8.33
Shortest transaction:           0.04

3 HSM's (sc-hsm-embedded)

Benchmarking 1024 bit RSA key with 3 HSM's (sc-hsm-embedded)

A siege test with 5 concurrent users, 30 seconds:

siege -c5 -d5 -t30S https://hsmcluster.nl

Result:

Transactions:                     55 hits
Availability:                 100.00 %
Elapsed time:                  29.81 secs
Data transferred:               0.00 MB
Response time:                  0.24 secs
Transaction rate:               1.85 trans/sec
Throughput:                     0.00 MB/sec
Concurrency:                    0.45
Successful transactions:          55
Failed transactions:               0
Longest transaction:            0.47
Shortest transaction:           0.22

10 concurrent users:

siege -c10 -d5 -t30S https://hsmcluster.nl

Result:

Transactions:                    116 hits
Availability:                 100.00 %
Elapsed time:                  29.33 secs
Data transferred:               0.00 MB
Response time:                  0.26 secs
Transaction rate:               3.95 trans/sec
Throughput:                     0.00 MB/sec
Concurrency:                    1.05
Successful transactions:         116
Failed transactions:               0
Longest transaction:            0.81
Shortest transaction:           0.22

20 concurrent users:

siege -c20 -d5 -t30S https://hsmcluster.nl

Result:

Transactions:                    227 hits
Availability:                 100.00 %
Elapsed time:                  29.82 secs
Data transferred:               0.00 MB
Response time:                  0.31 secs
Transaction rate:               7.61 trans/sec
Throughput:                     0.00 MB/sec
Concurrency:                    2.39
Successful transactions:         227
Failed transactions:               0
Longest transaction:            1.58
Shortest transaction:           0.22

60 benchmark mode:

siege -c60 -b -t30S https://hsmcluster.nl

Result:

Transactions:                    420 hits
Availability:                 100.00 %
Elapsed time:                  29.90 secs
Data transferred:               0.01 MB
Response time:                  3.79 secs
Transaction rate:              14.05 trans/sec
Throughput:                     0.00 MB/sec
Concurrency:                   53.27
Successful transactions:         420
Failed transactions:               0
Longest transaction:           20.31
Shortest transaction:           0.27

Wordpress site with 10 concurrent users:

siege -c10 -d5 -t30S 'https://hsmcluster.nl/wordpress/'

Result:

Transactions:                    430 hits
Availability:                 100.00 %
Elapsed time:                  29.46 secs
Data transferred:               3.11 MB
Response time:                  0.37 secs
Transaction rate:              14.60 trans/sec
Throughput:                     0.11 MB/sec
Concurrency:                    5.38
Successful transactions:         430
Failed transactions:               0
Longest transaction:            1.96
Shortest transaction:           0.04

Wordpress site with 20 concurrent users:

siege -c20 -d5 -t30S 'https://hsmcluster.nl/wordpress/'

Result:

Transactions:                    494 hits
Availability:                 100.00 %
Elapsed time:                  29.72 secs
Data transferred:               3.49 MB
Response time:                  0.87 secs
Transaction rate:              16.62 trans/sec
Throughput:                     0.12 MB/sec
Concurrency:                   14.43
Successful transactions:         494
Failed transactions:               0
Longest transaction:            3.93
Shortest transaction:           0.04

Benchmarking 2048 bit RSA key with 3 HSM's (sc-hsm-embedded)

A siege test with 5 concurrent users, 30 seconds:

siege -c5 -d5 -t30S https://hsmcluster.nl

Result:

Transactions:                     50 hits
Availability:                 100.00 %
Elapsed time:                  29.58 secs
Data transferred:               0.00 MB
Response time:                  0.73 secs
Transaction rate:               1.69 trans/sec
Throughput:                     0.00 MB/sec
Concurrency:                    1.23
Successful transactions:          50
Failed transactions:               0
Longest transaction:            1.50
Shortest transaction:           0.66

10 concurrent users:

siege -c10 -d5 -t30S https://hsmcluster.nl

Result:

Transactions:                     99 hits
Availability:                 100.00 %
Elapsed time:                  29.82 secs
Data transferred:               0.00 MB
Response time:                  0.91 secs
Transaction rate:               3.32 trans/sec
Throughput:                     0.00 MB/sec
Concurrency:                    3.01
Successful transactions:          99
Failed transactions:               0
Longest transaction:            2.97
Shortest transaction:           0.66

20 concurrent users:

siege -c20 -d5 -t30S https://hsmcluster.nl

Result:

Transactions:                    126 hits
Availability:                 100.00 %
Elapsed time:                  29.85 secs
Data transferred:               0.00 MB
Response time:                  2.22 secs
Transaction rate:               4.22 trans/sec
Throughput:                     0.00 MB/sec
Concurrency:                    9.35
Successful transactions:         126
Failed transactions:               0
Longest transaction:            7.04
Shortest transaction:           0.66     

60 benchmark mode:

siege -c60 -b -t30S https://hsmcluster.nl

Result:

Transactions:                    118 hits
Availability:                 100.00 %
Elapsed time:                  29.28 secs
Data transferred:               0.00 MB
Response time:                 10.28 secs
Transaction rate:               4.03 trans/sec
Throughput:                     0.00 MB/sec
Concurrency:                   41.44
Successful transactions:         118
Failed transactions:               0
Longest transaction:           29.13
Shortest transaction:           0.66    

Wordpress site with 10 concurrent users:

siege -c10 -d5 -t30S 'https://hsmcluster.nl/wordpress/'

Result:

Transactions:                    141 hits
Availability:                 100.00 %
Elapsed time:                  29.05 secs
Data transferred:               1.00 MB
Response time:                  1.74 secs
Transaction rate:               4.85 trans/sec
Throughput:                     0.03 MB/sec
Concurrency:                    8.44
Successful transactions:         141
Failed transactions:               0
Longest transaction:            4.63
Shortest transaction:           0.04

Wordpress site with 20 concurrent users:

siege -c20 -d5 -t30S 'https://hsmcluster.nl/wordpress/'

Result:

Transactions:                    145 hits
Availability:                 100.00 %
Elapsed time:                  29.27 secs
Data transferred:               1.01 MB
Response time:                  3.57 secs
Transaction rate:               4.95 trans/sec
Throughput:                     0.03 MB/sec
Concurrency:                   17.70
Successful transactions:         145
Failed transactions:               0
Longest transaction:           12.44
Shortest transaction:           0.04

Benchmarking EC prime256v1 key with 3 HSM's (sc-hsm-embedded)

A siege test with 5 concurrent users, 30 seconds:

siege -c5 -d5 -t30S https://hsmcluster.nl

Result:

Transactions:                     57 hits
Availability:                 100.00 %
Elapsed time:                  29.41 secs
Data transferred:               0.00 MB
Response time:                  0.29 secs
Transaction rate:               1.94 trans/sec
Throughput:                     0.00 MB/sec
Concurrency:                    0.56
Successful transactions:          57
Failed transactions:               0
Longest transaction:            0.56
Shortest transaction:           0.24

10 concurrent users:

siege -c10 -d5 -t30S https://hsmcluster.nl

Result:

Transactions:                    104 hits
Availability:                 100.00 %
Elapsed time:                  29.02 secs
Data transferred:               0.00 MB
Response time:                  0.31 secs
Transaction rate:               3.58 trans/sec
Throughput:                     0.00 MB/sec
Concurrency:                    1.11
Successful transactions:         104
Failed transactions:               0
Longest transaction:            0.94
Shortest transaction:           0.25

20 concurrent users:

siege -c20 -d5 -t30S https://hsmcluster.nl

Result:

Transactions:                    220 hits
Availability:                 100.00 %
Elapsed time:                  29.82 secs
Data transferred:               0.00 MB
Response time:                  0.40 secs
Transaction rate:               7.38 trans/sec
Throughput:                     0.00 MB/sec
Concurrency:                    2.97
Successful transactions:         220
Failed transactions:               0
Longest transaction:            2.16
Shortest transaction:           0.25

60 benchmark mode:

siege -c60 -b -t30S https://hsmcluster.nl

Result:

Transactions:                    360 hits
Availability:                 100.00 %
Elapsed time:                  29.99 secs
Data transferred:               0.00 MB
Response time:                  4.34 secs
Transaction rate:              12.00 trans/sec
Throughput:                     0.00 MB/sec
Concurrency:                   52.12
Successful transactions:         360
Failed transactions:               0
Longest transaction:           18.87
Shortest transaction:           0.30    

Wordpress site with 10 concurrent users:

siege -c10 -d5 -t30S 'https://hsmcluster.nl/wordpress/'

Result:

Transactions:                    382 hits
Availability:                 100.00 %
Elapsed time:                  29.57 secs
Data transferred:               2.77 MB
Response time:                  0.45 secs
Transaction rate:              12.92 trans/sec
Throughput:                     0.09 MB/sec
Concurrency:                    5.86
Successful transactions:         382
Failed transactions:               0
Longest transaction:            2.32
Shortest transaction:           0.04

Wordpress site with 20 concurrent users:

siege -c20 -d5 -t30S 'https://hsmcluster.nl/wordpress/'

Result:

Transactions:                    427 hits
Availability:                 100.00 %
Elapsed time:                  29.96 secs
Data transferred:               2.90 MB
Response time:                  1.11 secs
Transaction rate:              14.25 trans/sec
Throughput:                     0.10 MB/sec
Concurrency:                   15.83
Successful transactions:         427
Failed transactions:               0
Longest transaction:            3.71
Shortest transaction:           0.04

Tags: apache , articles , cluster , cryptoki , haproxy , hsm , mod_nss , nginx , nitrokey , nitrokey-hsm , openssl , pkcs11 , safenet , smartcard , smartcard-hsm