Highlight Preservation when doing sharpening

suggest a way to improve Neat Video
Post Reply
apefos
Posts: 14
Joined: Thu Jun 28, 2012 11:45 am

Highlight Preservation when doing sharpening

Post by apefos »

I perceived that when increasing the sharpening in neatvideo the highlights clips. I measured a part of the image which was at 247 value and after sharpening is become 255. So after sharpening there is the loss of some highlight textures and unwanted clipping in the signal in the bright areas.

I perceived that this issue is more pronounced when increasing more the sharpening amount, which is useful for Canon DSLR.

This thing happens in other sharpen filters also, not an issue only in neatvideo.

My suggestion is to implement some kind of HighLight Preservation, a checkbox near the sharpening settings, this feature would be used to avoid the increase in luminance in bright areas when doing sharpening.

the sharpening in neatvideo is awesome, does not show pixelation, if this highlight preservation can be implemented, it would make it even better.
NVTeam
Posts: 2745
Joined: Thu Sep 01, 2005 4:12 pm
Contact:

Post by NVTeam »

Thank you for the suggestion. Could you send us a small sample project with clip to help us reproduce that directly?

Thank you,
Vlad
apefos
Posts: 14
Joined: Thu Jun 28, 2012 11:45 am

Post by apefos »

OK, Vlad, I prepared a project, and also more tests, please read below:

I perceived that the issue happens to the shadows dark areas also, in the opposite way. when doing sharpening the dark areas gets more dark, so there are two issues, highlights gets more bright and shadows gets more dark. The test I did below uses aggressive settings to increase the perceiving of the issues, but the issues happens with low amount of sharpening also as explained in the end.

The project:

shows both things, shadows and highlights issues:

there are 3 original mov videos from Canon DSLR,

includes jpg files showing before/after results,

there are the neatvideo noise profile and noise preset files included,

also two text files with careful explanation of the tests, telling how to perceive the issues, and the suggestions to solve

to reproduce the results as can be seen in the jpg files, load each mov video to neatvideo and load the noise profile and noise preset to see the differences before/after sharpening in the dark areas and bright areas. zoom in the neatvideo window to 200% or 400% to perceive it better.

after seeing the results with aggressive sharpening settings, lower the sharpening amounts to Y + conservative + slider1=80, slider2=10, slider3=0 this works good for canon 720p, but also shows some amount of highlight clipping and shadow crush. less sharpening but the issues still exist, smaller but it is there...

please download the project at link below (zip file with main project and another txt file with the second test, more refined settings, and idea of dynamic range sliders):

https://www.dropbox.com/sh/g6rrm4vwvb6z ... zXXva?dl=0

Best Regards

Thanks

Apefos
NVTeam
Posts: 2745
Joined: Thu Sep 01, 2005 4:12 pm
Contact:

Post by NVTeam »

Thank you very much. We will download and run it on our machines. I will let you know when we have any comments after that.

Thank you,
Vlad
apefos
Posts: 14
Joined: Thu Jun 28, 2012 11:45 am

Post by apefos »

thanks, Vlad,

your comments will be much appreciated.

I uploaded the noise presets with the sharpening settings which I use for Canon DSLR, two presets.

In these presets the highlight and shadows shows less clipping/crushing because the amount of sharpening is small, but with a careful look you can perceive that the shadows are darker and highlights are brighter after sharpening compared to the original image.

you can apply these presets to the three mov files to test, download these presets here:

https://www.dropbox.com/sh/muc1nledij8d ... lQu1a?dl=0

Best Regards

Apefos
NVTeam
Posts: 2745
Joined: Thu Sep 01, 2005 4:12 pm
Contact:

Post by NVTeam »

Thank you. We took a look at the test case and could in principle see the problem but it didn't seem very big. It only shows when extreme settings are used, so in most cases that would not be visible.

The best solution perhaps would be to render such data in 32-bit precision, so that highlights and darks were fully preserved even if they go out of normal range. It may be useful to bring them back to the normal range by compressing the range, that should be possible to do using Premiere own tools then.

We will check if it makes any practical sense to implement any non-linear treatment to apply less sharpening specifically in highlights/darks. That may require some extra computations and may potentially slow down rendering.

Anyway, thank you very much for the suggestion and testing materials.

Kind regards,
Vlad
apefos
Posts: 14
Joined: Thu Jun 28, 2012 11:45 am

Post by apefos »

Interesting idea, Vlad,

the non-linear sharpening would be awesome! an option to change linear/non-linear with a checkbox or a slider (a slider would change the non-linear curve), or one checkbox/slider for shadows one checkbox/slider for highlights, separately... you decide.

so user can chose for a faster render in linear sharpening or a slower render in a non-linear sharpening (with fixed ccheckbox curve or custom curve with slider).

again, awesome idea!

I will do tests to perceive the suggestion to use the 32bit workflow and recover shadows and highlights using another effect/filter, but it seems that the render can increase also due to be using another filter, so having it alreay in neatvideo would be faster human working time.

You know, we indie videomakers/filmmakers are always trying to get the best film look...

Thanks!
apefos
Posts: 14
Joined: Thu Jun 28, 2012 11:45 am

Post by apefos »

hello, Vlad,

based on your idea, I developed three curves for sharpening, please take a look:

luminance value x sharpening amount (percentage), three tables:
(...) = from 73 to 183 in luminance

000 = 000% - 000% - 000% -
008 = 000% - 000% - 000% -
016 = 025% - 016% - 012% -
024 = 050% - 033% - 025% -
032 = 075% - 050% - 037% -
040 = 100% - 066% - 050% -
048 = 100% - 083% - 062% -
056 = 100% - 100% - 075% -
064 = 100% - 100% - 087% -
072 = 100% - 100% - 100% -
(...) = 100% - 100% - 100% -
184 = 100% - 100% - 100% -
192 = 100% - 100% - 087% -
200 = 100% - 100% - 075% -
208 = 100% - 083% - 062% -
216 = 100% - 066% - 050% -
224 = 075% - 050% - 037% -
232 = 050% - 033% - 025% -
240 = 025% - 016% - 012% -
248 = 000% - 000% - 000% -
255 = 000% - 000% - 000% -

probably it will need to be a formula to avoid steps on sharpening amount, so the percentage of sharpening will change for each integer change in luminance inside the specific luminance toe/shoe curves range (008-072 to shadows, 184-248 for highlights).

maybe 8 checkbox can solve, 4 for highlights, 4 for shadows:

4 checkbox for highlights: one for linear, then curve 1, curve 2, curve 3

4 checkbox for shadows: one for linear, then curve 1, curve 2, curve 3

what do you think?

thanks.
apefos
Posts: 14
Joined: Thu Jun 28, 2012 11:45 am

Post by apefos »

another interesting solution would be to make the CONSERVATIVE checkbox to be a curve instead of linear. the conservative option already decreases the amount of sharpening, so if it can be logaritmic instead of linear it will solve.
apefos
Posts: 14
Joined: Thu Jun 28, 2012 11:45 am

Post by apefos »

I was thinking about the curve, the non-linear sharpening, then something came up in my mind:

maybe the non-linear sharpening would help to avoid halos also, because the sharpening amount would be loewr in the dark and bright areas with high contrast where the halos are more pronounced after sharpening...
NVTeam
Posts: 2745
Joined: Thu Sep 01, 2005 4:12 pm
Contact:

Post by NVTeam »

Thank you for all the suggestions and ideas you have described so far. We plan to work on the sharpening filter in the near future and will consider ways to improve it. I am sure we will keep in mind your ideas as well.

Thank you,
Vlad
Post Reply