The "striped" setup seems to help:
This is our best result so far, lets hope it will continue.
BTW: ZFS stripe from 2 partition 1.7 + 1.7 TB, with ccache enabled and some tweaking.
Some further optimization is still possible, but for a test it is good enough.
building our source tree
Re: building our source tree
restarting with 9x5 instead of 7x6
- NevilleGoddard
- Developer
- Posts: 517
- Joined: Thu Dec 22, 2016 10:30 pm
- Location: Japan
Re: building our source tree
Very imptressive!
Re: building our source tree
9x5 doesn't seem to improve the rate, going to 8x6
- NevilleGoddard
- Developer
- Posts: 517
- Joined: Thu Dec 22, 2016 10:30 pm
- Location: Japan
Re: building our source tree
It just took me a little over 2 hours to do 220 packages on my machine.
Re: building our source tree
That's not bad.NevilleGoddard wrote:It just took me a little over 2 hours to do 220 packages on my machine.
- desktop packages on average are larger than whole ports/pkgs average.
- a single package may take hours, just think at libreoffice, chromium, webkit-xxx ...
- and of course a desktop machine usually is not a like a Xeon server
I started using synth on my desktop, when I though I was ready, started to build "everything" for GhostBSD, on a dedcated server.
Since then I discovered I was not ready, in fact building everything is a different story, you are going to have 26500 pkgs in one directory and this is going to affect overall performance. Similarly, when ccache grows up to 70~100 GB, with millions of files, the underlying filesystem play a big role in killing the performance.
Testing and measuring performance is difficult, because conditions changes nearly every times ... I was fooled many times by partial results ...
The end of the story is:
- UFS doesn't scale (related to ccache)
- FreeBSD kernel has a bottleneck while performing multiple 'umount' (we have to live with that).
I take as a reference the dragonflybsd results:
Dual Xeon E5-2620 (for a total of 16 cores / 32 threads), 128 GB RAM, NVME disks:
1200 pkgs/hour.
A FreeeBSD dev told me he build on a 40 cores system (40 threads really, 2 Ghz CPU), using poudriere and can build at 2800 pkgs/hour. He wanted me to switch to poudriere .... (in my mind I gave him the middle finger).
- NevilleGoddard
- Developer
- Posts: 517
- Joined: Thu Dec 22, 2016 10:30 pm
- Location: Japan
Re: building our source tree
A FreeeBSD dev told me he build on a 40 cores system (40 threads really, 2 Ghz CPU), using poudriere and can build at 2800 pkgs/hour. He wanted me to switch to poudriere .... (in my mind I gave him the middle finger). [/quote]
He he he
Thanks for all your efforts. I have to get busy on the GhostBSD wiki.
I thought ZFS would be better than UFS for this type of work or were you using Hammer on Dragonfly?
A bit off topic but....
I have been looking for a GTK3 theme for version 11 that looks consistent with all apps. I found a theme called Blackbird on GitHub that looks great. It's a dark theme. GTK2/GTK3.
I haven't found a problem with it yet.
He he he
Thanks for all your efforts. I have to get busy on the GhostBSD wiki.
I thought ZFS would be better than UFS for this type of work or were you using Hammer on Dragonfly?
A bit off topic but....
I have been looking for a GTK3 theme for version 11 that looks consistent with all apps. I found a theme called Blackbird on GitHub that looks great. It's a dark theme. GTK2/GTK3.
I haven't found a problem with it yet.
Re: building our source tree
No, I'm not using dragonfly right now, (other than a very short test), the report come from M. Dillon, dragonflybsd founder. (also evicted from FreeBSD devs, like Marino).NevilleGoddard wrote:I thought ZFS would be better than UFS for this type of work or were you using Hammer on Dragonfly?
Yeah, I read some other post from you about that, please continue.I have been looking for a GTK3 theme for version 11 that looks consistent with all apps. I found a theme called Blackbird on GitHub that looks great. It's a dark theme. GTK2/GTK3.
I haven't found a problem with it yet.
- NevilleGoddard
- Developer
- Posts: 517
- Joined: Thu Dec 22, 2016 10:30 pm
- Location: Japan
Re: building our source tree
Will do!
Do you know why they were evicted?the report come from M. Dillon, dragonflybsd founder. (also evicted from FreeBSD devs, like Marino).
Re: building our source tree
Do You know what is SPS ? Is the only reason I can think of.NevilleGoddard wrote:Will do!
Do you know why they were evicted?the report come from M. Dillon, dragonflybsd founder. (also evicted from FreeBSD devs, like Marino).
About Marino, the official motivation is about "unacceptable social behavior", which clearly is a lie.
https://forums.freebsd.org/threads/59705/#post-342578
https://archive.fosdem.org/2014/schedul ... hn_marino/
About Dillon don't know, happened much before, but I got it was something similar.
And with that I lost every trust on FreeBSD management. Literally shooting themselves in their feets.
Note: SPS (small p***s syndrome)