new build server
Posted: Thu Apr 27, 2017 8:29 am
Just a quick update, I realize that this thing was discussed only on IRC.
As of May 17, we rented a new build server: Xeon E5-1650-6c/12t, 64 GB, 2 x 3 TB,
The previous server Xeon E3-1245-4c/8t, 32 GB, 2 x 2 TB, will be dismissed today.
Reminder for ericbsd: we need to change the dns, pointing to the new builder IP address, and also to perform the associated jail setup on the new builder.
~~~
In the latest months I have tested several setup, looking for the best performance, the difficult thing is that each test require a lot of processing time (days), and I think that by now we have a good starting point setup;
32 GB is the minimum required RAM, we have no use of more than 64 GB RAM;
ccache is the most critical component when using synth, and it is the (only ?) component that would benefit on residing on an SSD/NVME disk.
Swap is not used at all on 64 GB system, and had a very limited use on 32 GB system.
I launched two builds on the two server, approx. 19 hours ago:
Xeon e3-1245: 4400 pkgs built, rate: 238 pkgs/hour
Xeon e5-1650: 5736 pkgs built, rate: 300 pkgs/hour
Note that this is a first run, where the ccache need be build up, subsequent run should double the output.
An interesting note is about UFS vs. ZFS:
ccache run definitely better on UFS, ZFS turned out to be slow when writing lot of small files, therefore we explicitly used UFS for ccache. (ada1)
The remaining component are on UFS for the Xeon 1245, and on ZFS for the Xeon 1650:
it turned out they perform comparably:
As you may see, the idle time is nearly the same on both system, the "system" is a bit higher in the case of the ZFS system, most likely due to the fact that there we used 12 builder in parallel (7 builder on UFS).
The use of ccache (ada1 for both), is of course higher for the system running more builder;
the disk activity for the other components (ada0) is higher for the UFS system, and lower for the ZFS one, that's where the ZFS ARC cache play a big role.
As of May 17, we rented a new build server: Xeon E5-1650-6c/12t, 64 GB, 2 x 3 TB,
The previous server Xeon E3-1245-4c/8t, 32 GB, 2 x 2 TB, will be dismissed today.
Reminder for ericbsd: we need to change the dns, pointing to the new builder IP address, and also to perform the associated jail setup on the new builder.
~~~
In the latest months I have tested several setup, looking for the best performance, the difficult thing is that each test require a lot of processing time (days), and I think that by now we have a good starting point setup;
32 GB is the minimum required RAM, we have no use of more than 64 GB RAM;
ccache is the most critical component when using synth, and it is the (only ?) component that would benefit on residing on an SSD/NVME disk.
Swap is not used at all on 64 GB system, and had a very limited use on 32 GB system.
I launched two builds on the two server, approx. 19 hours ago:
Xeon e3-1245: 4400 pkgs built, rate: 238 pkgs/hour
Xeon e5-1650: 5736 pkgs built, rate: 300 pkgs/hour
Note that this is a first run, where the ccache need be build up, subsequent run should double the output.
An interesting note is about UFS vs. ZFS:
ccache run definitely better on UFS, ZFS turned out to be slow when writing lot of small files, therefore we explicitly used UFS for ccache. (ada1)
The remaining component are on UFS for the Xeon 1245, and on ZFS for the Xeon 1650:
it turned out they perform comparably:
Code: Select all
Xeon 1245: # iostat
tty ada0 ada1 pass0 cpu
tin tout KB/t tps MB/s KB/t tps MB/s KB/t tps MB/s us ni sy in id
0 1053 35.29 91 3.12 58.50 81 4.64 0.00 0 0.00 78 0 12 0 10
Xeon 1650: # iostat
tty ada0 ada1 pass0 cpu
tin tout KB/t tps MB/s KB/t tps MB/s KB/t tps MB/s us ni sy in id
0 722 52.14 18 0.90 58.33 157 8.93 0.44 0 0.00 74 0 16 0 11
The use of ccache (ada1 for both), is of course higher for the system running more builder;
the disk activity for the other components (ada0) is higher for the UFS system, and lower for the ZFS one, that's where the ZFS ARC cache play a big role.