August 17th, 2019 by Blaise Robitaille
How To: Affordable 10Gb Ethernet
Whether you are a homelabber, home office power user or you want to bring your small/remote office up to snuff and share files at the speed of that SSD that you convinced your boss that you needed, now is the time to embrace 10GbE. Internet speed is getting faster and faster and we are working with exponentially more data daily. Providers around the country are scrambling to roll out fiber to homes and businesses to compete not only on speed but price. Not everything is dependent on the speed of the internet connection. Having a solid network allows for you to share files and media with very little latency.
NAS platforms from vendors such as QNAP, Synology, FreeNAS etc. all have the ability to be loaded with solid state drives. These drives can act as “Acceleration Cache” that plainly put, allows for commonly used files to be stored on the SSD and made available on the network at “high speed”. This “high speed” averages over 400 MB/s for many drives. A 500GB Samsung 860 EVO SSD as of July 2019 is less than $100.
With NAS boxes in the “Value” range – you can easily setup a simple setup that will have the ability to serve files faster than the standard gigabit ethernet speed that virtually all machines use today. Gigabit Ethernet maxes out at 125MB/second. A far cry slower than the 500 MB/s that a value SATA solid state drive can offer up files.
Bring your own Cloud
Many value-priced NAS enclosures offer 10Gb SFP+ ports built in QNAP TS-831x Series, Netgear ReadyNAS 520 series, Synology DS1819+, and many more that can easily be upgraded with a 10Gb NIC to enable 10Gb functionality. With a faster internet connection, you now have the privacy and availability to host your own files at a fraction of the cost. Whether you want to host a PLEX server for the family or if you would rather archive home movie, the cost has dropped under the $1,000 mark.
The most common 10GbE interfaces for modern networking are:
10Gbase-T Ethernet (RJ45)
If you spent the time to pull ethernet in the past 5 years, chances are you are running Cat6, if not you are likely running Cat5e. You can use your existing copper cables that you have running from point to point. If you pulled Cat6 you may achieve 10gbps if the run length is under 50m – if you are running on Cat5e you will still see a speed increase over your older 1gb connection however negotiated rates will drop down to 5 or even 2.5gbps as the Cat5e lacks the shielding to maintain higher throughput. If you can “Pullback” a line – replace the line with Cat6a or Cat7– this will let you move beyond 50m and achieve 10Gb.
QNAP has a great card (QNAP QXG-10G1T) - The cost of the 10Gbase-T adapter is less than $90.
Supported operating systems: QTS 4.3.4.0486 or newer, Windows, Linux
SFP+ Tranceivers w/ Fiber
Lower latency higher cost – there is a higher cost associated with fiber due to the high cost of the transceivers. With so much surplus fiber available on eBay – the costs of the cable can be reasonable. There are however a few things to consider when dealing with Fiber.
The transceiver that plugs into the SFP+ port can be expensive depending on the distance you need to run. Expect to pay around $25 per transceiver for a matched set of used transceivers (Yes, they must match). The transceiver will convert the signal into light to send down the glass fiber.
Fiber cable (LC/LC) is about the same price as ethernet cable on Amazon – keep in mind that this cable is far more delicate than ethernet as you can not kink, bend or stress the cable without breaking the filament inside. Keep this in mind when purchasing bulk used fiber.
Replacing the ends on a fiber cable requires special equipment.
There are definite benefits to Fiber – That being if you are running a line into your garage or attic, you can run fiber along side power lines and not suffer any interference.
Fiber is not susceptible to lightning strikes and is recommended when running a hard line to an outbuilding or across a yard or rooftop.
SFP+ DAC (Short Reach Copper)
As stated above – fiber can be expensive – One solution to an expensive fiber run would be to use an SFP+ card with a DAC cable for short runs (Less than 6m). You maintain the low latency and this is great for runs where the network equipment is all in the same room, however when things are spread out – this proves to be less than ideal.
The SFP+ DAC connector is a point to point cable that bundles the transceiver and connections into a single cable. These are very useful for patching NAS devices to network equipment as you will likely be presented with an SFP+ port on many modern NAS boxes.
Surplus SFP+ cards can be found on eBay in the $20 range – these cards permit for workstations and servers to connect relatively cheap. Dual SFP+ cards can be had for not much more.
I know what you are thinking – what if I just buy a few of these cheap SFP+ cards and some DAC connectors and “Daisy Chain” my server to my workstation to my router. Yes, that is possible, however you would still need something to switch the traffic passing through the cards. This turns out to be more trouble than its worth as if one section of the chain breaks – all fail.
Budget building requires thought. I have seen many builds where the builder picks an old enterprise switch that sounds like a jet engine is taking off beside them and uses more power than the server they connect it to. For example, a Cisco 4948 switch can be purchased relatively cheap but will consume 212 watts and generate 1023 BTU/hr of heat.
For this build we will be using a QNAP QSW-804-4C, the 8-port design allows for both SFP+ (DAC or Fiber) or to use 10Gbase-T Ethernet. It comes with “Rack Ears” so you could rack mount it in your home office or small business. Power consumption on this unit is 41 watts under load and 21W in Link-down standby. The unit is available for less than $500.
If budget is the deal breaker - there is also a brand new QSW-308-1C that QNAP has announced that is less expensive and is focused on the entry level user. The specs are promising with 3 SFP+ ports (One shared with a 10Gbase-T Ethernet port) and an 8-port gigabit switch. This model ships for under $200.
Your switch is the backbone of your network – With 8 ports – your switch can connect 8 devices at full 10Gb speed. Devices add up quickly. A NAS, workstation, test bench, iMac pro or MacBook (There are great thunderbolt 3 10Gb adapters available) – not to mention future Wi-Fi that is in excess of 1Gb.
Many new motherboards from ASUS as MSI have been including either 10Gb Ethernet cards or building it right on the main board of their higher end units for the past few years.
It comes down to future proofing your network. Admittedly the progression of 10Gb Ethernet has been rather slow in the home office / small business space, but when 5Gb internet becomes the norm in the next few years, we will see speeds well in excess of 1Gb being commonplace. A network switch is something that you purchase one and use for several years.
Let’s assume that you already have some CAT6 cables at home, (If you don’t – amazon basics work well)
A few QXG-10G1T cards I found to be the best solution for cheap and reliable connectivity.
Installing the drivers for the QNAP QXG-10G1T was simple – The controller on the card is a Aquantia AQC107 – Once installed the windows 10 x64 pro machine detected the NIC and it was available.
Once the setup was complete, I simply plug 10Gb switch into my existing router for standard network access (DHCP).
Attaching the SFP+ cable to my NAS – In this case I am using a QNAP TS-932X – This NAS includes TWO SFP + 10Gb ports and picks up an IP from my networks DHCP right away.
The NAS has one of those “cheaper” SSD drives that I made mention of earlier – and moving files from my local SSD to a windows network share on the NAS over the 10Gb connection jumps up to nearly 300MB/s.
Copying a media file from the NAS to my local machine was even faster. The QXG-10G1T does a great job keeping the load off the CPU.
You don’t need to have a massive media library or even a home lab that makes your dread your power bill every month to appreciate a fast network connection. The technology has finally dipped into the affordable range for many of us. If you are interested in experimenting with virtualization, iSCSI block storage, editing or even moving your gaming collection off your main drive – a high bandwidth connection to your other machines or servers just makes sense. We used a lot of QNAP gear in this project, there are of course other options. The ease of use and cost of the components used here though made for a great and cost effetive experience.