new builder E3-1670v6 / 64 / 2xNVME 450 / 2x2TB

News and Announcements related to GhostBSD
ASX
Posts: 988
Joined: Wed May 06, 2015 12:46 pm

new builder E3-1670v6 / 64 / 2xNVME 450 / 2x2TB

Post by ASX »

Image
:mrgreen:
ASX
Posts: 988
Joined: Wed May 06, 2015 12:46 pm

Re: new builder E3-1670v6 / 64 / 2xNVME 450 / 2x2TB

Post by ASX »

Performance looks reasonably good, but there is an unexpected high number of build failures, and as a consequence 6000 ports are skipped (not visible due to a ncurses/terminfo bug), but you can desume it from the difference between: total - built - left.

I'm going to stop this build to look at what is wrong.

Image
ASX
Posts: 988
Joined: Wed May 06, 2015 12:46 pm

Re: new builder E3-1670v6 / 64 / 2xNVME 450 / 2x2TB

Post by ASX »

OK, the failre was due to a ccache flag (run_second_cpp = false), that accordingly to the man page should have only affected the diagnostic output, clearly it is not true.

I took the chance to show the ccache effect, now built for the first 1000 pkgs ...

Image
ASX
Posts: 988
Joined: Wed May 06, 2015 12:46 pm

Re: new builder E3-1670v6 / 64 / 2xNVME 450 / 2x2TB

Post by ASX »

Something is not right yet ...
Image
ASX
Posts: 988
Joined: Wed May 06, 2015 12:46 pm

Re: new builder E3-1670v6 / 64 / 2xNVME 450 / 2x2TB

Post by ASX »

better, yet not enough ...
Image
ASX
Posts: 988
Joined: Wed May 06, 2015 12:46 pm

Re: new builder E3-1670v6 / 64 / 2xNVME 450 / 2x2TB

Post by ASX »

This machine has been a delusion.

A bit of info, for your own knowledge:

The operating system was installed on a single SATA disk, using ZFS. That leave 2 nvme disk and 1 SATA disk free.I have setup a new pool, made up from the two NVME disks, in RAID 0/stripe.

The NVME disk transfer rate is approx. 1000 MB/s (as reported from diskinfo), the aggregate transfer rate of the striped pool is approx 1500 MB/s. Impressive!

But that doesn't translate to a builder performance improvement, (ok, there is an improvement, but can be measured like +3% or+ 4%). Why so ?

a) ZFS, (and the 64 GB RAM installed), do an astonishing job in caching lot of things, as a result, the disk is really used only to write the built packages, the logs, and eventually the ccache. Additionally the write operations are of course asyncronous, in that synth "request" the write, the kernel/ZFS take in charge the job, and return an ACK to synth, before the files are effectively stored to the disk. The kernel will do the effective write sometimes later, by itself, in the meantime synth will continue with other packages.
The same "write offload" happens with SATA disks too, that's why there is no real difference by using NVME vs SATA. Probably swapping onto an NVME disk would be much faster, but here we don't swap at all.

b) The high speed of the nvme disks come to a price, in term of CPU usage, true you can achieve impressive write speed, but the CPU usage (sys in this case), will be relatively high. By doing that, those CPU cycles are detracted from the build processes. This situation could be eventually improved by using "intelligent/smart/cached/hw controllers, but that would be difficult to manage remotely. (and very likely would come at a cost too).

long story short: for builders choose the better CPU you can afford. :D

Now, the good news: we (me and ericbsd) already did it. :mrgreen:

SoYouStart renewed their offer, and we decided to grab two machines now that are available:
We have got two of these: https://www.soyoustart.com/ca/en/offers/173sys20.xml
These two machines (both) come at the same price of the E3-1670, but each of these perform better compared to the E3-1670, and we have two of them.

My personal thanks to the E3-1670, that I stressed a lot, :D but we will let it expire.

PS: ericbsd performed also a test using "poudriere" on this machine, the result was 440 pkg/hour.
I'm now doing a last round, just for comparison and will post the result here when completed.
ASX
Posts: 988
Joined: Wed May 06, 2015 12:46 pm

Re: new builder E3-1670v6 / 64 / 2xNVME 450 / 2x2TB

Post by ASX »

half repo .. and peaking at high speed

Image
ASX
Posts: 988
Joined: Wed May 06, 2015 12:46 pm

Re: new builder E3-1670v6 / 64 / 2xNVME 450 / 2x2TB

Post by ASX »

This build is nearly complete, the pkg rate is very similar to poudriere result, moreove I switched from a NVME based setup to a SATA setup. Once again a confirmation that the bottleneck is the CPU. ;)

Image

Stopping and goint to build the updated tree. ;)
ASX
Posts: 988
Joined: Wed May 06, 2015 12:46 pm

Re: new builder E3-1670v6 / 64 / 2xNVME 450 / 2x2TB

Post by ASX »

This server is going to expire in one week or so.

@ericbsd, let me know what you think, but IMO this should be dismissed.

If you want to perform furtther tests it is at your complete disposal.

~~~

I need a "script " to sync github.com/ghostbsd/freebsd-ports with upstream freebsd-ports; it is about time to start a next build.
User avatar
ericbsd
Developer
Posts: 2052
Joined: Mon Nov 19, 2012 7:54 pm

Re: new builder E3-1670v6 / 64 / 2xNVME 450 / 2x2TB

Post by ericbsd »

ASX wrote:This server is going to expire in one week or so.

@ericbsd, let me know what you think, but IMO this should be dismissed.

If you want to perform furtther tests it is at your complete disposal.
Nah we should dismiss that server.
I need a "script " to sync github.com/ghostbsd/freebsd-ports with upstream freebsd-ports; it is about time to start a next build.
I could maybe do simple shell script to sync the GitHub repository.
Post Reply