Early 2010 I build a beefy server for the time to handle virtualization and be a pretty comprehensive lab server in my house. Well… over the weekend it went dark. It would power up, the BMC would come up, but it wouldn’t post, wouldn’t generate post error codes, or anything, just black.
Here is the server when I built it in 2010:
The original specs were impressive for the day:
- 2x Xeon X5650 processors (6 core/12 thread per proc)
- SuperMicro 5520 chipset based SSI sized board w/ LSI SAS RAID onboard (shit-tastic RAID at that), and 4 Intel 1Gb NICs
- 48GB of ram
- Areca hardware RAID controller
- Lian Li all aluminum “server” case
- and a boat load of storage
Over the years, memory changed, storage changed around CPUs were even upgraded… but the original Supermicro motherboard stuck it out. I was wondering at what point motherboard would succumb to old age. The northbridge itself put off more heat than many desktop CPUs do! It idled at 65C most of the time.
This server idles @ 350w of power consumption doing nothing. Those 2 Xeons were very power hungry and hot.
So it wasn’t shocking when it failed… 9 years is a LONG time in the server world. Windows and Intel even had stopped supporting the storage controller onboard, the NIC were also barely supported. Here is the server after its downfall. Yeah wire management kind of went to shit over the years…
Note the brown area behind where the northbridge sits 
From all I can see the board seems in good shape, but there is a brown area behind the north bridge. I am guessing that controller that handles memory and PCIe interfaces finally fried.
The replacement:
The new repair job on the server is a much more modern deal with equally as much potential for upgrading over the years.
- Intel Xeon E5-2680v3 Processor – 12 core/24 thread (same core/thread count as my old setup, but 30+% less power usage, and 30% more speed) Picked it up on Ebay as a server pull for a song.
- Gigabyte MU70-SU0 motherboard – This thing can handle the v4 CPUs (That brings some nice performance improvements with faster ram speeds, etc…), and 768+GB of ram. It also has a BMC, and the ability to attach a 10Gb SPF board to a PCIe 4x connector made specially for it.
- 64GB of Ram in only 4 sticks. I had 48GB in the old setup… with 12 sticks!
- Onboard mSATA connector to move the OS disk out of the way.
- 4x 1Gb Intel Ethernet ports as well.
- Over all at full load the server uses under 200w of power, that’s a nice 30+% over all power and heat savings.
For a fraction of the price of what I paid for that original server build, I will get something that hopefully will get me another 9 years. I need to get an ATX power connector extension so I can run the power behind the board again like it was before. Amazingly the server worked out of the box with my old windows install… But next week once I get the mSATA drive, new TPM module, and a few other things, I am going to reinstall the server from scratch with Server 2019.
But… as I kind of expected with a new server build, you can’t escape hardware issues…
The BMC NIC on the board is broken, I can use the BMC if I force it to use one of the Intel NICs onboard, but the standalone dedicated NIC for the BMC is a non starter. Won’t grab DHCP, won’t work with static, etc… So I am returning this board and already ordered a new replacement. Fortunately Newegg is quick to turn this kind of return around.
Ingo
Regarding the BMC: it has the flaw that it negotiates a 1GBit connection, which it can’t handle. A workaround for this error: if you have a managed switch, limit the port for the BMC to 100MBit – the BMC will work fine, afterwards.