Benchmarks

[manocheese]

Here are the last two runs. I'm benchmarking on Steam Lite because work are too tight to upgrade my Pro license from 3.3.

I completely understand the difference between datasets and versions etc. I just want to make sure that I'm getting the best performance and help you make Zephyr better.

[Andrea Alessi]

Hi manocheese,

by looking at the log it looks like you're losing a lot of time when Zephyr has to load up images. When running the benchmark, zephyr creates a temporary directory where it dumps the benchmark images that will be then loaded for the actual benchmark testing.

Below, my reasoning:

Here is a phase where it simply reads the photos from disk:

[03:35:47] Preparing image : '0synth01.jpg'
[03:35:47] Preparing image : '0synth07.jpg'
[03:35:47] Preparing image : '0synth17.jpg'
[03:35:47] Preparing image : '0synth25.jpg'
[03:35:47] Preparing image : '0synth09.jpg'
[03:35:47] Preparing image : '0synth11.jpg'
[03:35:47] Preparing image : '0synth29.jpg'
[03:35:47] Preparing image : '0synth27.jpg'
[03:35:47] Preparing image : '0synth21.jpg'
[03:35:47] Preparing image : '0synth31.jpg'
[03:35:47] Preparing image : '0synth19.jpg'
[03:35:47] Preparing image : '0synth03.jpg'
[03:35:47] Preparing image : '0synth15.jpg'
[03:35:47] Preparing image : '0synth13.jpg'
[03:35:47] Preparing image : '0synth05.jpg'
[03:35:47] Preparing image : '0synth23.jpg'
[03:35:48] Preparing image : '0synth18.jpg'
[03:35:49] Preparing image : '0synth02.jpg'
[03:35:49] Preparing image : '0synth08.jpg'
[03:35:49] Preparing image : '0synth26.jpg'
[03:35:49] Preparing image : '0synth12.jpg'
[03:35:49] Preparing image : '0synth10.jpg'
[03:35:50] Preparing image : '0synth30.jpg'
[03:35:50] Preparing image : '0synth28.jpg'
[03:35:50] Preparing image : '0synth22.jpg'
[03:35:50] Preparing image : '0synth32.jpg'
[03:35:50] Preparing image : '0synth20.jpg'
[03:35:51] Preparing image : '0synth04.jpg'
[03:35:51] Preparing image : '0synth16.jpg'
[03:35:51] Preparing image : '0synth14.jpg'
[03:35:52] Preparing image : '0synth06.jpg'
[03:35:52] Preparing image : '0synth24.jpg'

here is the same step on one of our computers, a dell workstation that has a mechanical disk with SSD cache (so nothing super fancy)

[16:10:09] Preparing image : '0synth05.jpg'
[16:10:09] Preparing image : '0synth01.jpg'
[16:10:09] Preparing image : '0synth13.jpg'
[16:10:09] Preparing image : '0synth29.jpg'
[16:10:09] Preparing image : '0synth25.jpg'
[16:10:09] Preparing image : '0synth17.jpg'
[16:10:09] Preparing image : '0synth21.jpg'
[16:10:09] Preparing image : '0synth09.jpg'
[16:10:09] Preparing image : '0synth06.jpg'
[16:10:09] Preparing image : '0synth02.jpg'
[16:10:09] Preparing image : '0synth14.jpg'
[16:10:09] Preparing image : '0synth30.jpg'
[16:10:09] Preparing image : '0synth26.jpg'
[16:10:09] Preparing image : '0synth18.jpg'
[16:10:09] Preparing image : '0synth22.jpg'
[16:10:09] Preparing image : '0synth10.jpg'
[16:10:09] Preparing image : '0synth07.jpg'
[16:10:09] Preparing image : '0synth03.jpg'
[16:10:09] Preparing image : '0synth15.jpg'
[16:10:09] Preparing image : '0synth31.jpg'
[16:10:09] Preparing image : '0synth27.jpg'
[16:10:09] Preparing image : '0synth19.jpg'
[16:10:09] Preparing image : '0synth23.jpg'
[16:10:09] Preparing image : '0synth11.jpg'
[16:10:09] Preparing image : '0synth08.jpg'
[16:10:09] Preparing image : '0synth04.jpg'
[16:10:09] Preparing image : '0synth16.jpg'
[16:10:09] Preparing image : '0synth32.jpg'
[16:10:09] Preparing image : '0synth28.jpg'
[16:10:09] Preparing image : '0synth20.jpg'
[16:10:09] Preparing image : '0synth12.jpg'
[16:10:09] Preparing image : '0synth24.jpg'

I would expect your machine to perform as well, but there is something off in this phase. I'm not sure if this is something due to your hardware or if there is something software (another application perhaps?) slowing down the system. Make sure to check in Zephyr where the temporary directory is (by default is in your %tmp% windows installation directory, so your C: drive which hopefully is the fastest).

Try changing the temporary directory from zephyr's option on a different drive, restart zephyr and try again. Hopefully we'll see if we're on the right track

[manocheese]

I've moved the cache, but it was still using the TEMP folder on the C drive. So I moved that to another drive too. Still the same.

I then updated drivers, added exceptions for those folders in to all the antivirus programs, disabled Backblaze and everything else that might slow down access.

Both drives I tested are NVME drives, I'm going to try moving things to a non-nvme drive, in case it's a driver issue with the drives.

[manocheese]

Same thing with a normal HDD, switching over didn't affect the results at all. I also test without CUDA enabled, in case it was having trouble loading textures in to the GPUs. It wasn't that either.

Side note: Running without CUDA made me appreciate how much of a difference it makes quite a lot.

[manocheese]

Forgot to mention. I used the Samsung Drive program to run a TRIM optimisation on all the drives.

I'm running it again and watching Resource Monitor, file access is usually under 5ms response time, but does seem to spike up to 20-30 sometimes. That could be it.

I might do a clean install of Windows, in case that helps.

[Andrea Alessi]

If you can do that at least we can start excluding external causes, that would help! Keep us posted!

[manocheese]

Windows reset, still doing the same thing.

[Andrea Alessi]

Hmm then something else is off. Random stuff to check, sorry if this is a bit shooting in the dark:

- temperature ? nvme ssds might throttle the speed due to temperature. Check the airflow ?

- bios settings? Maybe there is something off in the bios settings ?

- have you tried switching drivers? I assume you're using samsung drivers, if not, try them. If yes, just as a test, what happens if you switch to microsoft's driver?

[manocheese]

The other drives do exactly the same thing. So I don't think it's the NVME issue, I've attached a benchmark that shows the IOPS hitting some fairly ridiculous speeds.

I'm running the benchmark on a couple of other PCs to compare the results to. Was the test PC you looked at Windows 10 and, if so, did it have the creator's update?

[Andrea Alessi]

We have two of the same workstation, one with windows8 and the other one with windows10 (with update). I don't remember which one we ran the test with to be honest, but they both perform well so i don't think that's the issue. Very odd, i'm thinking if we're missing something

[manocheese]

Tested it on the office PC. It doesn't even have an NVME drive. It appears to be doing the same thing. I've attached the log and uploaded the results.

[Andrea Alessi]

I'll have a look at them with Roberto as soon as we can, i'm starting to running out of ideas

[manocheese]

Thanks. Don't worry about it if you don't find anything, I'm thinking it's more of an OS problem and that's something I can investigate myself. And it's not like it doesn't process things very quickly.

In case it matters, I only have 3.701 on Steam Lite. I do use Pro, but I'm stuck on 3.3.