Can NI auto-match best profile when exposure 'pushed'?

questions about practical use of Neat Image
Post Reply
l_d_allan
Posts: 35
Joined: Thu Sep 24, 2009 3:00 pm

Can NI auto-match best profile when exposure 'pushed'?

Post by l_d_allan »

Canon 50D
DPP 3.6.1
CS4
NI 6.1 64-bit plug-in

I've got a bunch of pictures (500+ after throwing away rejects) that were taken of a high school musical with mediocre lighting. Much of the lighting was from spotlights that changed a lot.

The pictures were taken at iso's from 800 to 3200 iso, and many were significantly underexposed. Also, I did a lot of bracketing the exposure in bursts of three, such as 1.3 under-exposed, zero compensation, and 1.3 over-exposed.

Also, many of the pictures are relatively "busy" without good places for NI to find a good sized square that has no detail. Otherwise, I could rely on auto-detection and let NI measure noise rather than using auto-match.

For many, I did the equivalent of "pushing" the ISO. Back with darkrooms, you might be using Tri-400, and develop so that you got the equivalent of ISO 800. My understanding is that this is roughly equivalent to increasing the exposure slider a stop. With both firm and digital images, the noise increases significantly.

My understanding is that NI auto-matches a profile based on:
1. Which camera
2. ISO
3. How much compression?
4. Resolution
5. Sharpness
6. Exposure

Is there a way that NI can detect how much the exposure was increased in post-processing before the NI step? It seems like the noise can be very different for an image that had exposure "pushed" in post-processing by 0 stops, 1 stop, or 2 stops.

With Canon's DPP (Digital Photo Pro) 3.6.1 , the information may be partly available. With "Image Information On", there are fields for:
* Exposure Compensation
* Autoexposure Bracketing.

The point is that it would be helpful if NI could auto-match and use a profile that took into account "pushed iso" rather than what the basic ISO was. This would be the equivalent of NI determining something like "the picture was taken at 1600 iso, but the best profile-match would be "1600-Plus-Two-Stop-Exposure-Slider-Increase").

However, this doesn't show if the exposure slider was increased from the original image during p.p. That makes me doubt whether NI could auto-match that the exposure slider was "pushed".

I suppose I could keep track of which images were "pushed", and process them in a batch. I would have profile folders for:
* no push
* half-stop pushed
* full-stop pushed
* 1.5 stop pushed
* two stops pushed

I did take pictures of the calibration target that were intentionally under-exposed by one and two stops. Then I corrected the exposure by "pushing" with the exposure slider and built NI profiles from those.

The intention is to have them available to refine the noise-profile-matching. My expectation was that this would work ok if a whole series of images were underexposed by a relatively set amount. I sometimes do this when the lighting is fairly constant, but to get an acceptable shutter speed and f-stop for DOF, I intentionally under-exposre. (but perhaps in the future I should increase the ISO and avoid "pushing?)

However, it becomes less productive when the lighting varies a lot. In a series of images, each can vary a lot from the previous without a useful pattern.

I suppose I could use the CS4 Bridge labels to color-code the relatively degree of "pushing" involved. Or use keywords to identify the amount of positive or negative pushing. I want to use ACR 5.5 to correct the exposure, and then have NI handle them in batches.

Another factor is that I expect to be using ACR 5.5's adjustment brush to "paint-in" exposure compensations for high contrast. In a single image, two actors might be in the stoplight and over-exposed, and two actors might be out of the spotlight and under-exposed. My expectation is that noise would be higher in those areas where exposure was increased. I suppose this would really throw off auto-matching and expecially auto-detection.

A related factor, if ETTR was being used (expose-to-the-right), the actual noise could be significanly less than the ISO would predict. (The image is intentionally overexposed in the camera, then the image is 'reverse-pushed' during p.p. by moving the exposure slider to the left). I suppose this could be termed "negative pushing".
NITeam
Posts: 3173
Joined: Sat Feb 01, 2003 4:43 pm
Contact:

Post by NITeam »

I understand the problem of matching vs exposure compensation. It is not that simple to solve automatically as the number of possible scenarios is quite large, which is clear from your message too. Anyway, I will describe how it is now to help you deal with this problem. Neat Image takes into account many EXIF parameters to do matching (including those listed above and some others), but not exposure compensation value. We haven't included it because it is not always properly reflecting the post-processing adjustments applied to the image (not all software tools correctly update that value). Therefore, it is not currently possible to make an automated matching decision based on the exposure compensation values even if they are correctly set by the camera and/or post-processing software. It is still possible to manually separate all profiles into groups with different compensations as you have described above and then switch Neat Image from using one group to another depending on the images you currently work with (like matching against profiles from one sub-folder to process images with EV1.0 stored in an EV1.0 sub-folder, then another profile sub-folder for images with another compensation, etc.).

Another useful thing that may help process such images is Auto Fine-Tune. If you have a single profile set built for EV0.0 images, and match images (with any EV values) against them then, after the matched profile is loaded, additionally Auto Fine-Tune it. This will adjust the profile to the somewhat different noise in the current input image and that may be sufficient to make this noise profile accurate enough to be good for this image.

From our side, I think we will revisit this issue of exposure compensation as we work on new versions of Neat Image to see if it may be now possible to make a more reliable use of it.

Hope this helps,
Vlad
l_d_allan
Posts: 35
Joined: Thu Sep 24, 2009 3:00 pm

Post by l_d_allan »

Thanks for the helpful info.

I took a closer look at the "Action Setup". I hadn't realized I can specify three choices:
* Auto Match
* Auto Profile
* Use current profile (which allows a single profile)
and refine the above with Auto Fine-Tune.

I am wondering in the "Use Current Profile" will be that helpful in the situation I describe, with a variety of ISO's being used. If the ISO's for a batch were all the same, then it might be worthwhile.

But if I'm switching around the default folder to look in, I might as well tweak each image separately, which I'm hoping to avoid. I also don't want to split up the overall batch into a dozen sub-batches.

If your development team (you?) enhances this functionality, perhaps you could allow a custom factor to be specified from CS4 or PSE, such as a keyword or a label. Then the end-user could make this a relatively high priority for doing the auto-match?
NITeam
Posts: 3173
Joined: Sat Feb 01, 2003 4:43 pm
Contact:

Post by NITeam »

Please note that you can use Auto Fine-Tune with both Auto Match and Use current profile.

If we will take into account the exposure compensation (there are some issues) then it will be most likely based on the standard exif value, to ensure compatibility among tools, cameras and workflows. Using a custom keyword or tag is a less desirable candidate solution because it is not standard and therefore not very compatible. Regarding priority, we will check if it will be useful to make it adjustable or there is a value that works best.

Thank you,
Vlad
l_d_allan
Posts: 35
Joined: Thu Sep 24, 2009 3:00 pm

Post by l_d_allan »

Oooops ... I was mistaken that Auto-Tune can be used with Auto-Profile.

I did some further checking. Attached are screen shots. This first is with a blurred image deliberately under-exposed by one stop, and then using ACR 5.5 to increase the exposure by one stop. This could represent "push processing".

Image

The next involves an image deliberately over-exposed by 1.5 stops, then using ACR 5.5 to decrease exposure by 1.5 stops. This could represent ETTR (expose-to-the-right).

Image

Within Bridge, there is a field within the group of "Camera Raw" that indicates that exposure was increased by one stop for the first image, and decreased by 1.5 stop for the second image (also fields for brightness, contrast, clarity, temperature, white balance, etc.)

Also, this information is available to Bridge for images converted to TIFF, JPG, and PSD using the CS4 Image Processor.

I would think that people using ACR would be a large proportion of your customers, including PS, CS, and PSE.

For my Canon 50d, I set up 160+ images ranging from under-exposed by two stops through over-exposed by two stops. This included ISO 100 through ISO 12800 (increments of 1/3rd stops for 100 through 3200, then H1=6400 and H2=12800).
* under 2.0
* under 1.5
* under 1.0
* under 0.5
* baseline
* over 0.5
* over 1.0
* over 1.5
* over 2.0

The idea would be to use this as the setting for Profile directory. Neat-Image would Auto-Profile, taking into account Camera, ISO, and exposure increase/decrease.

Does this sound workable, at least for PS and PSE?
NITeam
Posts: 3173
Joined: Sat Feb 01, 2003 4:43 pm
Contact:

Post by NITeam »

Re: I was mistaken that Auto-Tune can be used with Auto-Profile.

It can but makes little sense because Auto Fine-Tune is already a part of Auto Profile. Using it again would be repeating the same fine-tuning twice, which is useless. It is much more useful when you use Auto Match to load a profile. Auto Fine-Tune can to a large extent resolve the issue of exposure compensations right now.

Regarding the idea about using several values related to exposure compensation taken from various (both standard and non-standard) areas of image metadata, it could work. It would not be very compatible (camera raw settings are proprietory, not standard for all raw converters, and perhaps even not for different versions of ACR itself), but could work.

Again, thank you for the suggestion.

Vlad
l_d_allan
Posts: 35
Joined: Thu Sep 24, 2009 3:00 pm

Post by l_d_allan »

NITeam wrote:Regarding the idea about using several values related to exposure compensation taken from various (both standard and non-standard) areas of image metadata, it could work. It would not be very compatible (camera raw settings are proprietory, not standard for all raw converters, and perhaps even not for different versions of ACR itself), but could work.
Again, it seems like this could be the majority of your potential customer base. I would speculate that the people who purchase a noise reduction profile would be mostly using CS3 or later (but I acknowledge ignorance about this).

If you looked at previous versions of PSE and CS/PS, I wonder how far back fields for "Exposure" and "Brightness" are available?

The intent is to further refine which Auto-Profile is picked, taking into account not only the Camera and ISO, but also under and over exposure adjustments.

HTH ...
Rick
Posts: 3
Joined: Mon Apr 26, 2010 10:04 am

Re: Can NI auto-match best profile when exposure 'pushed'?

Post by Rick »

l_d_allan wrote: For many, I did the equivalent of "pushing" the ISO. Back with darkrooms, you might be using Tri-400, and develop so that you got the equivalent of ISO 800. My understanding is that this is roughly equivalent to increasing the exposure slider a stop. With both firm and digital images, the noise increases significantly.
Hope you don’t mind me jumping in on this interesting thread. I’ve used NI for some time, but am newly playing with RAW processing having just upgraded to Canon 450D.

I have often been tempted to “push” the exposure as described here - especially with my old compact - in order to obtain a fast enough shutter speed in a given situation. But is there really a benefit to be had? Why not just increase ISO in the first place.

Say for example an ISO exposure at ISO800 is pushed by one stop. Then I wonder how that compares to taking the shot at ISO1600 with no push. Presumably the noise patterns will be different: I’m guessing pushed ISO800 means concentrated noise (less pixels affected, but with twice the intensity) whereas ISO1600 would be more scattered (more pixels affected) .

Which of these types of noise would NI deal with best? (assuming both use suitable prepared profiles as already covered in this thread). I am guessing the ISO800 shot would require stronger noise reduction, hence losing more detail ; whereas the ISO1600 shot probably has less detail in the first place.

If the “push” technique really works then I’m surprised that camera manufacturers haven’t jumped in to automate it in-camera?
Post Reply