Ubuntu server 12.04, apache2 - very slow response to first request

Question

The server is under pretty much load - few hundred requests per second. Vast majority of them is through SSL.

The problem is that first request through HTTPS to the server meets very slow response (like 10s) even if database is not involved. Next requests are realized in eye blink. At the same time without SSL it works fast constantly. Few days ago situation was inversed - most of the traffic was without SSL ant then it was fast without SSL and with SSL slow.

I want connections to go through SSL but the first reponse after some time of lack of activity at level of about 10 seconds is unacceptable. What could I change in probably apache2 config to avoid the first slow response?

Answer 1

It sounds like you are CPU bound on the key exchange. Common problem. When a client connects over SSL/TLS for the first time a (very computationally expensive) key exchange is made. After this key exchange has been completed the clients can reuse the keys obtained in the exchange in the following communication. This is why the first request over HTTPS takes so much time to complete.

There is very little you can do about this situation, other than adding resources. More CPU will speed up the calculations of the key exchange. More memory is always good. You can also configure Apache to keep generated keys in memory for longer, to avoid having clients redo the key exchange.

You could also get some marginal improvements to the performance by changing cipher suits or tweaking keylengths, but it is usually not worth the hassle.

If you are going to be running over this kind of load for a longer time it may be a good idea to offload the SSL/TLS calculations. You can get a separate box (using your favourite proxy), a SSL acceleration card or even a specialized SSL/TLS offloading box.

Answer 2

Looks like DNS issue. Check that DNS is configured on localhost and working properly, also, check HostnameLookups directive is set to off in apache configuration.

Answer 3

There are two scenarios I can think of:

• Check Apache's logs and make something clever out of it (errors, access logs timestamps, etc) - this is a longshot I think
• Or: I suggest you to Wireshark how the SSL handshake protocol goes. You should see where the 10s elapses - on server side or on client side. I guess opening the keystore or something like that takes up a lot of time, but at first you should sniff the network traffic for timestamps. Let us know how it went!

Answer 4

I have found a problem. I had to turn off the KeepAlive.

There were plenty of new connections every second (200-300/s, sometimes more). Most of them didn't need the connection to be kept alive for 5 seconds. As a result connection pool was quickly utilized and all new connections had to wait for the old one to terminate after the 5s.

It appears KeepAlive option is useful only when server is able to hold enough opened connections while most of users do sth for specified by KeepAliveTimeout parameter (like 5s in my case).

Other case it is actually useful to turn it off.