Hello all,

I decided to do some experimenting a while back with my network by trying to have 2x1Gbps links between my server and my desktop PC. Now my switch allows me to do this with Link Aggregation, or "Port Trunking" as its also known, however what I didn't realise is the network card drivers also have to support this feature. Unfortunately in my case, they don't, so I wasn't able to try this out, but I'm going to explain how I thought it could be done...

The idea was to trunk each of the two ports on the server, and each of the two ports on my desktop together to make one "virtual" link of 2Gbps. Both the server and desktop PC then had their ports bridged into one so they had one IP each rather than one for each individual NIC. Sounds plausible so far doesn't it? That's what I thought. Now, while the setup this way works, it doesn't double the speed as you would expect, in fact it makes it slower...

I spoke to mrpijey about this and the only thing he could come up with was that each NIC was being sent random data from the trunking setup and it didn't know what to do with it immediately, so it had to decode it, and because it couldn't do this fast enough, the speed suffered. We (rather mrpijey) did some investigating into trunking, and it was discovered the cards drivers have to support it. Now it turns out Intel PRO/1000 desktop cards all support this, so my next idea was to get 4 of these cards... That was until I saw the cost. They cost £20 each. I guess if I want to do this I'll have to get saving!

It struck me as strange that Windows didn't have some sort of built in feature for this already. After reading wikipedia, it seems that *nix based systems can handle this a lot better than Windows systems can, and they can also use multiple brand NIC's too. I don't know, however, if they also need to support trunking or if *nix can virtualise this some how. Anyway, that's another story entirely. I want it to work on Windows, not *nix!

So, my next plan of action is to get 4 of these Intel PRO/1000 cards and give those a go. It will be a while though, probably will have to wait until I get a job given the price of them. It's something to look forward to though, and it was a nice experiment with what I already had in order to figure out the problems and restrictions with what I currently have.

Anyone else ever though of doing this, or does anyone actually have it working? Let me know!

I think it could work (in theory).

But also the speed of the harddrive is a big downpoint, most of the time between USB <-> Internal = 30mbps, and internal <-> internal = 50mbps (but sometimes faster, windows reports it as 250mb/s, but slows down to 50mbps on large files).

So you can get it to work, but I think it would not be faster then 1x 1000mb network card, hell, not even faster then 100mbps

Yes but the thing is networking that fast doesn't have to rely on the receiving hard disk being fast enough. It also comes in useful if you're doing clustered rendering, and lots of other things that don't necessarily use the hard drive, but does shift lots of data.

Also you really need to get your units correct. You state the speed in MB/s yet you use Mbps not MB/s as the unit. You confuse yourself AND others!

SATAII = 3Gbps (375MB/s)
USB2.0 = 480Mbps (60MB/s)

Most modern hard drives can hit 80-100MB/s with ease.

Do you have your TB drives set up in raid 0? Maybe you should get > 300MB/s

No because that would be suicide if one failed

In the cisco world a 'trunk' contrains multiple vlans... IE a trunk connection would be used to connect two switches together... think 802.1q.

Now a port 'channel' is bonding two connections together to get the aggrigate speed. However it's salesman speed, not real speed, in that if you were to 'bond' or 'channel' 4 connections you get 4GB WORTH of traffic, not a 4GB connection...

So a single user wouldn't notice any difference unless they were reading & writing a LOT of data at the same time.

Intel NIC's are the best for doing trunking & channeling in Windows... And yeah they are also the most expensive.. It *IS* just an option in the driver I recall some 'trick' being used to fool the driver to activate the features on the cheaper intel nic's but I *THINK* it required you to have at least one server "pro" card.....

I hope this helps.

Hi,

Link Aggregation (LACP, PAgP, Trunking, whatever) isn't that simple. The Switch balances packets on multiple links according to source/destination MAC Address.

This only works for servers which have traffic from many (at least the number of links) sources. Maximum throughput you can get from a single host to a single server is one of the links.

Of course this increases redundancy, if one of the cables get bad.

But for one server to one client this is useless.

For more information see: http://en.wikipedia.org/wiki/Link_aggregation

Not true. If you have several applications requiring bandwidth it can also use 2 or more links. Also, Intel based link teaming provides load balancing as well. Its not just the switch that controls this either, the NIC's have to support it as well.

Hi,

sorry to counter here, but http://www.intel.com/support/network/sb/cs-009747.htm#Features states that


You still have a single source/destination-MAC pair per connection. Even Intel Adaptive Loadbalancing only chooses the outgoing interface per destination. So connections to a single host still have one outgoing interface.

Most switches out there will distribute the load between links based on source- and destination-MAC, the more expensive gear balances based on tcp/udp ports.

(maybe I can come up with a more technical explanation later, if you want to)

Best,

Falk

From what I have read elsewhere, it also works with several connections from the same machine, since its possible to split and load balance those quite easily. A single connection can't have this done to it.

You can manually use two connections if you manually enter the IP of each NIC, but this basically does it for you over 1 IP. If this didn't work, then our co-admin mrpijey wouldn't be doing a similar thing with his workstation and server.

Bah, I hate it when this happens.

You are right and I am wrong. At least, when it comes to 802.3ad. This PDF http://www.ieee802.org/3/hssg/public/apr07/frazier_01_0407.pdf explains it in depth.

Of course your switch has to support it and also the network drivers.

What switch are you using?

(I should first read, then post)

Best,

Falk

It's a HP Procurve 1800-8G. It does support link aggregation etc. Sadly I don't have Intel NIC's so I can't do this now until I get a pair. I recently lost out on 3 Intel Pro 1000 Dual Port's on eBay for £6 each due to my own error in remembering what time the listing ended.

When you create a bonding on Linux, it set the same MAC on every network cards which are members of the bonding.

Playing with network is easier on *nix because the userspace tools are the same for every hardware.
And ebtables is very very interesting

Andy, have you thought about getting a card (or cards) with TCP/IP offload? That way the TCP/IP processing will be performed on the card(s) and your speeds won't suffer due to Windows working your CPU twice as hard to push that speed.
This is a very interesting read:
http://www.rtcmagazine.com/articles/view/100797

All my cards support it, but that's not what this article was about Oh and nice bump...

Oops...september! I didn't notice because it was so close to the top Plus I haven't signed in for so long that basically every post is marked as 'new' my bad!

All your cards support TCP/IP offload? Then windows should not be using any CPU to process the network traffic! Or maybe 1%

Yeah but where did you get the TCP/IP offload stuff from? It's not mentioned in any of my posts before yours, lol. We're talking about link aggregation, which is totally different.

From that I read that Windows (and in turn CPU) was having difficulties processing the traffic so had pointed out TCP/IP offload. Upon further inspection it seems like I may have misread your post.

Damn! Again, my bad
*hangs head in shame*