Calculating the maximum troughput of one TCP stream

Hi blog readers,

I've done a small project last week for a customer and the customer was complaining about not getting the speed that he actually ordered at the ISP where I am working for. A lot of network engineers are getting the question very often to measure the troughput/speed because a customer is just unsatisfied with the speeds he is getting.

Well I don't have to tell you that speedproblems can be the cause of a lot of things ... like for example an interface that is not configured correctly in terms of speed or duplex, a faulty cable, a lot of congestion in the LAN network, well I think I can go on for a little while, but what if all those things you checked are OK and the customer is still having speed/toughput problems?

It's important to first see what the customer is actually trying to do and what he is actualy testing ...

My customer for example had a SAN system of HP and he was trying to do a SAN SYNC session with another SAN system based on another loaction.

This SAN application only used 1 TCP session (stream) to do the actual syncronisation... If you don't know how TCP/IP is actually working you would say that this customer is using the full 200MB  fibre line for his SAN syncronisation session.

But nothing is less true ....

The facts are as following:

• The customer has 2 sites (SITE A and SITE B) (see drawing below)
• Between those sites the customer has a 200MB fibre connection
• The customer is trying to do a SAN Sync between those sites
• The SAN application is using a TCP Window Size of 32k
• The Round Trip Latency (RTT) is 8 milliseconds

When a TCP data transfer happens 2 elements are very important for the transfer, the TCP Window Size AND the round trip latency. If you know these 2 values (as we do) we can calculate the maximum troughput what you can get with 1 TCP stream regardless what speed line you have .

The theory behind this is that in our case here 32kb of data is sent to the other side (SITE B) and after each 32kb of data TCP has to send a ACK of 8 milliseconds, after this ACK is received another 32kb of data is send and after this is received an ACK is send again from SITE B to SITE A after 8 milliseconds ... this process is repeated over and over again untill the total data is sent. This 32kb is called the WINDOW SIZE.

We can see our TCP Window Size and our Round Trip Latency values above. With this we can now calculate the maximum troughput with the following formula:

TCP-Window-Size-in-bits / Latency-in-seconds = Bits-per-second-of-throughput

32kb = 32000 bytes

32000 bytes * 8 = 256000 bit (a conversion from bytyes to bits)

256000 / 0.008 (8 miliseconds) = 32000000 bits per second of troughput = 32Mbps maximum throughput per TCP session

So the maximum troughput that we can actually get on this link of 200MB is only 32Mbps.

Possible solutions for this is to either 1) Increase the Window Size that the system is using or 2) try to decrease the Round Trip Latency values. Decreasing the Round Trip Latency values is going to be a pain if these values are already good (in our case the distance between SITE A and SITE B is only 150 km and a RTT of 8 ms is just fine) so our only option is trying to increase the TCP Window Size of the system. This can only be done if the System/Application lets you change this and this is not always the case...

There is also a formule to calculate the ideal Window Size if you know the line speed :-)

Bandwidth-in-bits-per-second * Round-trip-latency-in-seconds = TCP window size in bits / 8 = TCP window size in bytes

The bandwidth of this line was 200MB as we've seen before.

200 Mb = 200 000 000 bits per second *  0.008 = 1600000 bits / 8 = 200000 Bytes = 200kb

Your ideal TCP Window Size is in this case 200kb (factor 6 of what the Window Size is now)

Just keep 1 thing in mind ... THIS IS ONLY FOR 1 TCP SESSION ... so if you would start 4 or 5 extra FTP sessions with a FTP file copy you will see the total bandwidth if being used of this line because all those sessions for example 5 will result in 5 * 32Mbps = 160Mbps wich will make the customer somehow happy with understanding the theory behind this.

There are also a some WAN accelerators available these days like Cisco WAAS for example wich can take these kinds of problems in terms of latency and appication data compression away for you but these technologies cost money and is a whole different BLOG story.

Hope I've teaced you guys something and if you have any questions just contact me :-)