[Solved] AE 5.5 & multi-CPU-redering vs. settings in Nea

[INDAVO]

Hello,

first of all: THANK YOU FOR THE CUDA SUPPORT. ITS AWSOME.

The whole story:

With my old Core2Quad and Neatvideo 2.6 I only rendered 720x576 PAL DV stuff @ 5fps (quite slow).

Now I finally got new fresh hardware:

Core i7 2600K @ 4.4GHz OC (stable)
32GB 1600MHz DDR3 RAM (the full load)
Point Of View Geforce 560 Ti with 2GB VRAM

So I puchased the new Neatvideo 3.0 and installed Adobe After Effects CS5.5 and did a test: Wow, 30fps for my 720x576 PAL DV Video with CPU and GPU redering settings activated in neatvideo.

Now the problem:

I noticed in the task manager that neatvideo does not use the whole CPU with acitvated CPU (8 cores) and GPU (all memory) redering. Only about 50 to 60%. With only GPU setting the CPU runs in idle. With only CPU (8 cores) setting it also does not use the whole CPU but about 70-80%.

So I tried some settings in AE itself. I activated the multiprocessor feature in AE with all cores (3GB RAM for each of the 8 cores) and activated only CPU rendering in neatvideo.
Now the render-process used 100% of the CPU with all cores and was quite a little faster. I also think that the memory usage was more efficient.

After that I thought I could activate the GPU afterburner in neat and set it to CPU and GPU with all cores and memory.
The problem was that it rendered but some dosboxes with yellow font came up saying that the memory of the gpu wasn't allocatable (or quite something like that).

Is that a bug? What are the optimum settings to get neatvideo render with 100% CPU and 100% GPU?

If you need any screenshots of my settings in AE and neatvideo just send me a message.

Thank you,

INDAVO

[NVTeam]

I noticed in the task manager that neatvideo does not use the whole CPU with acitvated CPU (8 cores) and GPU (all memory) redering. Only about 50 to 60%.

The Task Manager shows how both AE and its components use CPU. Neat Video is just one of those components so it is not possible to see there how much CPU Neat Video alone uses.

You can make the overall AE render process use more CPU though, using multiprocessing in AE.

I activated the multiprocessor feature in AE with all cores (3GB RAM for each of the 8 cores) and activated only CPU rendering in neatvideo.
Now the render-process used 100% of the CPU with all cores and was quite a little faster. I also think that the memory usage was more efficient.

After that I thought I could activate the GPU afterburner in neat and set it to CPU and GPU with all cores and memory.
The problem was that it rendered but some dosboxes with yellow font came up saying that the memory of the gpu wasn't allocatable (or quite something like that).

Your video card has 2GB of memory. That amount must be divided into equal pieces between copies of AE render that are doing multiprocessing. If you have 8 copies, 2GB should be divided into 8 parts. The amount of GPU memory allowed to be used by Neat Video should be about 10% (you should set that in NV Preferences, otherwise it will run out of GPU memory right away because you run more than one copy of the filter) . Practically 10% is just about 150MB, which is likely to be insufficient for processing. In these circumstances, I would try to reduce the number of AE render instances. Set it to 2 instead. Set the amount of GPU memory allowed to NV to 50%. Perhaps this will work better.

Also, I recommend to update to v3.1.

Vlad

[INDAVO]

Hi Vlad,

thank you for the quick reply. I just noticed the new version and installed it.
Wow. Now it's 40fps with 4 CPU cores and GPU (optimum benchmark settings)

So I now set the multiprocessor support in AE to 4 cores and reduced the RAM to 1.5GB for each. And in neat I divided the GPU memory by 4 and set it a little lower.

Now it works! Allthough you said that the taskmanager does not show the exact usage of neat it now was @ 100% CPU usage and it rendered quite faster than ever.

So thank you for the help with these settings (perhaps you should take this into the manual)
And of course a big thank you for the new 3.1 version.

INDAVO

[NVTeam]

I am glad it works better.

Also, I checked how AE multiprocessing works with NV today and found that running 2 multiprocessors in AE works faster than 4, and also faster than 1. At least on my machine. So I recommend to test that directly.

Vlad

[dkitsov]

Do I then divide the memory for the total memory of all of the GPUs or still for each one individually?

[NVTeam]

I think you can adjust the memory settings only individually, per GPU, there.
I recommend to try it on defaults first, it may not require adjusting at all if you use a small amount of multiprocessors in AE (like 2, which was the optimal amount in my direct tests).

Vlad

[dkitsov]

So I am running a 12 core Xeon with 24Gb my GPUs are GTX580 x2, with 3GB each. I also have a Quadro 4000 with 2GB of ram. The answer depends on if NV is using GPUs separately or together. Lets say if it is using them together and I want to run 4 cores plus two GPUs then I would have to allocate 50% of GPU ram on each of the GPUs, then each of the AE instances would have 1.5GB of video RAM. If NV uses GPUs separately then I have to allocate 25% of memory on each GPU? Each of 4 AE instances would use both of the GPUs per instance with .75 GB of video RAM each?
How's my logic? Is my understanding faulty?
Thank you.

[NVTeam]

The number of NV instances running at the same time is equal to the number of AE instances you allocate in AE preferences. Each instance of NV will use all GPUs enabled in NV preferences and the amount of RAM of the corresponding GPUs specified in NV preferences. So, if you have 2 AE (NV) instances then set the memories of 580's to 50%, 4 instances - then 25%.
I guess 4000 will not be very useful in that setup.

I wondering what will be the optimum number of AE instances in this case. Could you also post your speed figures from the Optimize box for frame 1920x1080, radius=1 please?

Vlad

[dkitsov]

So, for some reason those numbers are inconsistent (as is the amount of the available memory for each GPU). But here their are: I either get 15.8 fps with 2 x GTX580, or 16.4 with 2 xGPX 580 plus 9 cores. This is for 8-bit color.

I wondering what will be the optimum number of AE instances in this case. Could you also post your speed figures from the Optimize box for frame 1920x1080, radius=1 please?

Vlad

[NVTeam]

Thank you. Some fluctuations are normal. The CPU contribution may also depend on other system processes. Generally, 16 fps is a very good result.

Have you checked which number of AE multiprocessing copies achive the highest speed (shortest time) of actual rendering? Could be 2, could be more, as you have many cores.

Vlad

[dkitsov]

Vlad, here is the data:
Method: Source 1min 34sec 07frames cineform AVI, 1920x1080 23.976p, Trillions of Colors, 444 up-sample on decode.
Render to either Millions of Colors plus or to Trillions of Colors plus depending on the comp settings, Cineform HighHD Avi.

Each of the processor's cores has a 2GB allowance.
Full purge and/or restart of AE if warranted by a certain condition described below. GUI runs on Quadro 4000. NV selected 5 cores of the cpu plus 2x 580 GTX GPUs.

1. No MP enabled in AE. Both of GTX 580 GPUs allowed 100%
5:21

2. 2 cores MP, 8-bit. GPUs @ 100%
3:21

3. 2 cores; 16-bit. GPUs @ 100%
5:10

4. 3 cores; 8-bit. @100%
3:19

5. 4 cores; 8-bit. @100%
3:57

6. 4 cores; 8-bit. @25%
3:25

7. 5 cores; 8-bit. @20%
3:33

During the last two tests error screen appears with an error 999 (not sure will have to replicate). During the last 4 tests when going back to NV interface reported GPU memory is significantly lower (800mb vs original 2970mb each). Memory reported is back to original upon AE restart. Makes me think that perhaps what runs well for one clip/composition in a render queue might not work for the following clips and create out of memory errors.

Dima

Thank you. Some fluctuations are normal.
Have you checked which number of AE multiprocessing copies achive the highest speed (shortest time) of actual rendering? Could be 2, could be more, as you have many cores.

Vlad

[NVTeam]

Thank you for the details!

an error 999

That is quite likely related to the shortage of GPU memory too.

During the last 4 tests when going back to NV interface reported GPU memory is significantly lower (800mb vs original 2970mb each). Memory reported is back to original upon AE restart. Makes me think that perhaps what runs well for one clip/composition in a render queue might not work for the following clips and create out of memory errors.

Yes, once GPUs are used from multiple instances of AE and NV, the GPU memory may become a limiting factor. Which is why it is probably best to use only 2 instances of AE MP on that machine. That is stable in terms of memory and that also ensures practically the best rendering speed. 3 instances are a bit faster but that may be just a measurement fluctuation and even if not, the difference is really small.

So, 2 instances seem to be the best choice on your machine too.

Thank you once again,
Vlad

[dkitsov]

So I run into something peculiar. After I disable GPU processing and try to run the render NV says not enough GPU memory.

[NVTeam]

It looks like there is a small bug there.. Which makes it necessary to restart AE when those NV settings are changed, so that the new setting were used in all instances of AE/NV. We will try to fix this in the next update. In the meantime please simply restart AE.

Thank you,
Vlad