Page 4 of 5

Re: GPU speed database

Posted: Mon Jan 14, 2019 9:13 pm
by djmarksf
Glad you found my results useful. I saw several interesting observations in that data:

1) use of an external GPU (in my case, the Vega 64) with a 2017 MacBook Pro with Thunderbolt 3 connectivity gave quite disappointing results. Best result went from 4 frames a second to 6.21, which hardly seems worth it given the ~$1,000 cost of the enclosure plus the GPU. I guess the one good thing in that scenario is the GPU by itself handling 5.71 FPS, presumably freeing up the CPU for other things while processing video. If my results are anything to go by, rational use of eGPU's on Macs would be a very "niche" scenario.

2) the difference in performance of the exact same GPU card when moved from the Thunderbolt enclosure to a Windows PC is pretty enormous, going from 5.71 to 19.2 FPS (GPU only).

3) The improvement from upgrading from the Vega 64 to RTX 2080 Ti is less than one might expect given the raw specs for the two GPU's.

Re: GPU speed database

Posted: Mon Jan 14, 2019 9:49 pm
by NVTeam
djmarksf wrote: Mon Jan 14, 2019 9:13 pm 2) the difference in performance of the exact same GPU card when moved from the Thunderbolt enclosure to a Windows PC is pretty enormous, going from 5.71 to 19.2 FPS (GPU only).
We also noticed that the same AMD GPUs may perform significantly better in Windows than in macOS. For example, please see the figures related to Radeon RX Vega 64 in this blog post. The difference seems to be caused by some inefficiency of the GPU drivers in macOS. That may in principle be improved by the developers of the GPU drivers over time.

Vlad

Re: GPU speed database

Posted: Wed Jan 16, 2019 12:37 am
by djmarksf
NVTeam wrote: Mon Jan 14, 2019 9:49 pm We also noticed that the same AMD GPUs may perform significantly better in Windows than in macOS. For example, please see the figures related to Radeon RX Vega 64 in this blog post. The difference seems to be caused by some inefficiency of the GPU drivers in macOS. That may in principle be improved by the developers of the GPU drivers over time.
Yes, I did read that post. Your figures for the Vega 64 were 11.6 on Mac, 17.5 on Windows. Those must be some really inefficient Mac drivers if the hardware was otherwise similar.

I have to think that my much larger difference (5.71 Mac, 19.2 Windows) was partially due to the Mac setup being a dual-core i5 laptop with the Vega 64 in an eGPU case, while the Windows machine is a much faster 16-core i9 X299 setup with the Vega 64 plugged into a PCI slot. I don't think thermal throttling was happening in the eGPU case because it's very well-ventilated (vents covering most of the side panels, two large fans on one side as well as the three fans on the Vega 64 itself).

eGPU's are being marketed as a "solution" for Macs that lack any internal expansion capability, so I found it discouraging that in real life it provided so little benefit for this use.

Re: GPU speed database

Posted: Thu Jan 24, 2019 2:21 pm
by AndCycle
I really do think lack of data for GPU purchasing reference is something shouldn't happened in this day,
gosh we have internet and we can upload data if anyone push the test button and willing to share the info.

i7-8700 with ASUS DUAL-RTX2060-O6G
Neat Bench (Neat Image 8.4.0, Neat Video 4.8.5) x64
Copyright (c) 1999-2018 Neat Image team, Neat Video team, ABSoft.
All Rights Reserved.

GPU detection log:

Looking for NVIDIA CUDA-capable devices...
CUDA driver version: 10000
NVIDIA CUDA initialized successfully.
Checking CUDA GPU #1:
GPU device name is: GeForce RTX 2060
4997 MB available during initialization (6144 MB total)
Check passed - will attempt to use the device

Looking for AMD OpenCL-capable devices...
Failed to initialize OpenCL.
If you use an AMD card, please install the latest AMD driver with OpenCL support.

Neat Video benchmark:

Frame Size: 1920x1080 progressive
Bitdepth: 8 bits per channel
Mix with Original: Disabled
Temporal Filter: Enabled
Quality Mode: Normal
Radius: 2 frames
Dust and Scratches: Disabled
Slow Shutter: Disabled
Spatial Filter: Enabled
Quality Mode: Normal
Frequencies High, Mid, Low
Artifact Removal: Enabled
Detail Recovery: Disabled
Edge Smoothing: Disabled
Sharpening: Disabled


Detecting the best combination of performance settings:
running the test data set on up to 12 CPU cores and on up to 1 GPU
GeForce RTX 2060: 4997 MB currently available (6144 MB total), using up to 100%

CPU only (1 core): 2.24 frames/sec
CPU only (2 cores): 4.61 frames/sec
CPU only (3 cores): 6.49 frames/sec
CPU only (4 cores): 8.47 frames/sec
CPU only (5 cores): 10.1 frames/sec
CPU only (6 cores): 11.4 frames/sec
CPU only (7 cores): 11.2 frames/sec
CPU only (8 cores): 10.9 frames/sec
CPU only (9 cores): 10.5 frames/sec
CPU only (10 cores): 9.71 frames/sec
CPU only (11 cores): 9.09 frames/sec
CPU only (12 cores): 8.55 frames/sec
GPU only (GeForce RTX 2060): 19.6 frames/sec
CPU (1 core) and GPU (GeForce RTX 2060): 16.1 frames/sec
CPU (2 cores) and GPU (GeForce RTX 2060): 17.5 frames/sec
CPU (3 cores) and GPU (GeForce RTX 2060): 20 frames/sec
CPU (4 cores) and GPU (GeForce RTX 2060): 21.7 frames/sec
CPU (5 cores) and GPU (GeForce RTX 2060): 23.8 frames/sec
CPU (6 cores) and GPU (GeForce RTX 2060): 25 frames/sec
CPU (7 cores) and GPU (GeForce RTX 2060): 24.4 frames/sec
CPU (8 cores) and GPU (GeForce RTX 2060): 23.8 frames/sec
CPU (9 cores) and GPU (GeForce RTX 2060): 23.3 frames/sec
CPU (10 cores) and GPU (GeForce RTX 2060): 22.2 frames/sec
CPU (11 cores) and GPU (GeForce RTX 2060): 21.7 frames/sec
CPU (12 cores) and GPU (GeForce RTX 2060): 20.8 frames/sec

Best combination: CPU (6 cores) and GPU (GeForce RTX 2060)

Re: GPU speed database

Posted: Fri Jan 25, 2019 6:07 am
by Fifonik
NVTeam wrote: Mon Jan 14, 2019 11:01 am Perhaps it just makes sense for testers to run the tests with the 'standard' settings and submit that to the database.
But I guess it would be excessive to not accept other results.
There is only two options: understand and store all data or refuse unexpected data.
For example, results with different "Quality Mode" are not comparable so I cannot just ignore the field. I have to add a new field for the option into DB + modify search so user will need to specify it to get adequate results.

I'll try to fix the tool over weekend.

Re: GPU speed database

Posted: Tue Jan 29, 2019 2:29 am
by Fifonik
Sorry, it took longer, then I expected. It were some changes in recent versions that was quite hard to add with current implementation. So I ended up re-doing it from the scratch. Still working on it...

A few questions:
1. In 4.0.8 (I have one log from the version) it was only one Quality option - outside Temporal & Spatial sections. Was it applied to both sections?

2. Are there any information what is "Detail Recovery" option mentioned in NeatBench log? Is it something that will be implemented in future version so I need to adjust parser?

3. Are there any reasons that NeatBench has "Artifact Removal" option enabled?
I tried to allow the database to accept NeatBench's results as well (it is much easier for end user to run NeatBench and upload results). I compared NeatBench results vs NeatVideo results (both 4.8.5) on my system. All options except "Artifact Removal" were the same. NeatBench was slower by ~10%. So they are not directly comparable :(

Thanks.

Re: GPU speed database

Posted: Tue Jan 29, 2019 8:02 am
by NVTeam
Thank you for your work!

I will comment below.

1. Since version 4.5, there are two separate quality settings now, one in Temporal section and another in Spatial section. Each section can be controlled independently.

2. Detail Recovery is only available in Neat Image, so it is shown as disabled in other cases. If the database only records logs of Neat Video performance, then there is no need to take it into account.

3. Artifact Removal is enabled by default in Spatial filter of Neat Video, which is why it is also enabled in Neat Bench.
The speed of Neat Video inside host application may be slightly different than the speed measured by Neat Bench, for example because the host application may be also doing some work in the background or may be occupying some memory resources, while in case of Neat Bench there is no such host application and its influence.

Thank you,
Vlad

Re: GPU speed database

Posted: Tue Jan 29, 2019 9:40 am
by Fifonik
1. Yes, I see that in last version they are separated. However, I'm trying to keep backward compatibility and need to understand how the Quality option was used in 4.0.x. Was it applied for temporal + spatial or to only one of the section?

2. OK, will ignore it.

3. Oh, I see. I did not know that the option is enabled by default in Neat Video. Looks like I disabled it long time ago and it was stored somewhere. So NeatBench options are 'defaults' for NeatVideo. Will double check logs in database (I'm keeping original logs that were added) and probably update database. And sure will accept NeatBench's logs.
I'm aware about proper benchmarking so I tried my best. I compared NeatVideo vs NeatBench on the same system that was not busy (but who really know what windows 10 doing in background?). I run tests a few times. Thought about averaging values, but results were pretty consistent: 11.1, 11.1, 11.0 (NeatBench) and 11.1, 11.1, 11.1 (NeatVideo) so I just used 11.1 (with the same settings where Artifact Removal disabled). With the Removal Artifacts enabled numbers were 10.5.
Anyway, I only did these testing to check if the option resource expensive or not. I did it for different options and found that I can ignore some of them that only change values by about 1-2%.

Re: GPU speed database

Posted: Tue Jan 29, 2019 11:04 am
by NVTeam
1. Yes, it was synchronized in versions before 4.5.

3. The host application usually doesn't do much background work when Neat Video is open but it may occupy some GPU memory, which may in some cases affect the speed a bit. Anyway, what you observe is quite normal: usually the influence of the host application is not significant.

Thank you,
Vlad

Re: GPU speed database

Posted: Sun Feb 03, 2019 1:07 pm
by Fifonik
Done (+ added info about my RX 470).
It is possible just select log info from the forum and paste it there.

Most important changes:
[+] Neat Bench log supported
[+] System extracts GPU and Neat Video version info from the log so in most cases you do not need to specify it manually
[+] New Neat Video versions and new GPU will be created automatically if not exist in database yet
[+] Tool generates link to search results that you can publish somewhere (see example with RX 470 above)
[*] UI updated
[-] Unfortunately, multiple logs cannot be inserted in the textarea any longer.

Re: GPU speed database

Posted: Sun Feb 03, 2019 4:19 pm
by NVTeam
Thank you very much!

Dear forum users, please try to submit your new measurements to the database.

Thank you,
Vlad

Re: GPU speed database

Posted: Mon May 13, 2019 9:28 pm
by Fifonik
As defaults in v5 is changed ('Very Low' is enabled now) I will need to make modifications in the database and separate records with 'High, Mid, Low' from 'High, Mid, Low, Very Low'.

Re: GPU speed database

Posted: Mon May 13, 2019 9:33 pm
by NVTeam
Thanks a lot for your help!

Vlad

Re: GPU speed database

Posted: Tue May 14, 2019 1:10 am
by karuchie
If you want to compare directly with the old version.

Command of NeatBench4 like NeatBench5.
NeatBench4 vlf+ 32

Command of NeatBench5 like NeatBench4.
NeatBench5 vlf- 8

Re: GPU speed database

Posted: Tue May 14, 2019 3:06 am
by Fifonik
Sure, I read about this already. I was talking about updating the GPU speed database tool.