GhostBSD performs poorly with large folders (Mate desktop)

BSDsEnjoyer

Hi, I generally like GhostBSD and I am currently actively using it in my project. I have many folders with large amounts of files, including some folders with many image files (tens of thousands of files).

It took Caja 10+ minutes to open a directory containing 25000 image files and then it took eom (eye of mate) about 7 minutes to open an image. ls command also takes a lot of time, several minutes.

This is an operating system/desktop environment issue as I do not experience such problems with the same hardware using different software (OS and DE).

Did anyone else experience something like this? What was your solution?

vimanuelt

On ZFS, listing a directory with 25,000 files should not normally take anywhere near that long. Check:

zpool status
zpool iostat -v 1
time ls -f >/dev/null
time find . -maxdepth 1 >/dev/null

If find and ls -f are also slow, the bottleneck is almost certainly below the desktop layer (ZFS, storage, or hardware), not Caja or ls itself.

How much RAM does the system have?

BSDsEnjoyer

vimanuelt Thanks for the reply, the system is currently busy, I will try this once it would become idle again. But it copies files at a speed of 30-40 MB/s which is rather normal for HDDs, I think.

vimanuelt

BSDsEnjoyer
What were the results when running the dirperf app?

vimanuelt

Here is a performance utility that will emit a report.
xcrsz/dirperf

BSDsEnjoyer

vimanuelt Apologies, the system is still busy at the moment (lots of file operations, all going smoothly) and I haven't yet started the performance testing because of that. What I can surely say is that zpool gave the healthy status for the ZFS pool and that file operations (copying, being copied to) are going well. I also have a hard drive encrypted with Veracrypt. Caja was slow with both storage types. I've tested the veracrypt drive on Linux Mint (Debian) and the 25000 files directory was opened by Nemo in 10 seconds, and a random image was opened in 5 seconds.

vimanuelt

The Linux Mint comparison is very useful because it rules out several possibilities.

If the same hardware and the same VeraCrypt volume can open a directory with 25,000 images in about 10 seconds under Nemo, but GhostBSD takes several minutes with both ZFS and the VeraCrypt drive, then the storage itself is probably not the limiting factor.

I'd still be interested in seeing the dirperf report once the file operations finish. In particular, if the report shows that plain directory enumeration is fast but Caja is still slow, that would point toward thumbnail generation, MIME detection, filesystem monitoring, or another desktop-layer component rather than the filesystem.

It would also be helpful to compare these after the system becomes idle:

time ls -f >/dev/null
time find . -maxdepth 1 >/dev/null

If those complete in a few seconds while Caja still takes minutes, then we can largely rule out ZFS and focus on the MATE/Caja stack instead.

BSDsEnjoyer

vimanuelt I did all the tests you've recommended and ~~attach the results in the log file~~ (ah, attaching .txt files is not allowed). dirperf is an interesting program, would be nice to have such software open sourced 😉
The cold pass was not very clean because I need the filesystem to be mounted at the moment.
Attaching the log file under spoiler proved to be inconvenient, so I uploaded it to https://privatebin.net/?2505227f40b4d845#Hs9v3ziQmXz2fYC8jLdabk38dB7ocSV28n7LbkD78uEQ
This link will expire in 3 days.

vimanuelt

BSDsEnjoyer

The measurements identify the root cause as application behavior rather than the filesystem. The ZFS pool is healthy, directory enumeration is fast, metadata performance is normal for a mirrored HDD pool, and sequential throughput matches the hardware. The only operation that reproduces the observed five-minute delay is opening every image and reading its first few kilobytes. That benchmark took 291.8 seconds, essentially identical to Caja's delay when opening the directory. Given that the directory contains roughly 25,000 PNG files and Caja is configured to display image thumbnails in icon view, the evidence indicates that thumbnail generation and image inspection dominate the startup time, with the remaining delay explained by the random I/O latency of HDDs rather than by ZFS itself.

BSDsEnjoyer

vimanuelt Thank you for the analysis! It confirms that this is Caja-related. I've changed 2 Caja settings:
In Views subwindow: Show icons to off (removed the checkmark)
In Preview subwindow: Show thumbnails for "Other Previewable FIles" to "Never"
But it still took Caja 8 minutes to list the directory. ZFS is being copied to, but Caja still shouldn't be that slow.

vimanuelt

BSDsEnjoyer

At this point, it would be best for you to submit a Caja bug report at: mate-desktop/caja80

The Caja developers can best advise you as they address the issue.

BSDsEnjoyer

vimanuelt I've created it at mate-desktop/caja1900
But I don't have high hopes to be honest.

Did anyone try to use directories with many files on Mate GhostBSD?

vimanuelt

BSDsEnjoyer

I use Caja as well, but my workflow is probably a bit different because I organize my files using a classification system rather than relying primarily on search.

I keep related files together under broad categories and projects with descriptive filenames. I find it much easier to browse when I don't remember the exact filename, and it also makes it easier to see all the documents related to a project in one place.

With ZFS, this organization has additional benefits. I can snapshot or replicate an entire project or category, apply different properties such as compression or quotas to separate datasets if needed, and keep backups organized without changing my directory structure. Search is still useful as a fallback, but I find a well-organized hierarchy works better for day-to-day use.