I've been using xlibre with no issues whatsoever. It seems to be faster than xorg but I could be wrong. I'm using amdgpu graphics.
The goods news is that with GhostBSD you can create a boot environment before you change so if something goes wrong you can boot into the previous boot environment and you should not have any problems.
I switched to XLibre a long time ago but I think this is how to do it.
Create boot environment
bectl create backup-xorg. (Or any name you want)
pkg delete xorg
pkg install xlibre (I think this is the pkg name)
To make sure of the package name for XLibre, run
pkg search xlibre
After installing XLibre, reboot and you should be good to go