RGB histogram widget with texture upload for post shader

[claudeha]

Is your feature request related to a problem? Please describe.
Optimizing exposure etc would be easier with a colour histogram.

Describe the solution you'd like

Dockable widget with image RGB Linear/Log histograms a la GIMP, Darktable, etc. Button to capture histogram from buffer (if OpenGL 4 is supported this could be in a compute shader, otherwise read back to CPU). Mode to automatically update after each subframe.

Further, option to automatically upload the histogram (perhaps also rearranged to be able to look up R,G,B values from percentiles) as a texture1d (or two) for the buffer shader to do algorithmic post colouring with (eg, auto levels, equalize, white balance, etc).

Automatic update to be disabled in tiled rendering, unless 2-pass rendering is possible (accumulate all tiles, then buffer shader all tiles with histogram accumulated from all the tiles) - probably simpler to use histogram snapshot from a smaller preview image?

Describe alternatives you've considered
Export as EXR and do post in other software. Breaks flow, and not all post software supports writing post algorithms as GLSL code.

[3Dickulus]

just to get familiar I did a little digging and reading...
khronos OpenGL-Refpages gl2.1 glGetHistogram
University of Pennsylvania OpenGLInsights AsynchronousBufferTransfers search histogram 2nd occurrence
Scatter-based histogram generation developer.amd.com

CERNs histogram processing class :O
... googled "save OpenGL histogram"

would this be part of the buffershader frag(s)? or an auxbuffershader frag?

[claudeha]

I imagine histogram generation happening between accumulation and buffershader. Buffer shader gets the histogram data as an additional input. Might be tricksy to do histogram of HDR buffer. Not sure what current state of art is for histogramming, probably compute shader on GL4 and scatter on GL3.

[3Dickulus]

http://developer.download.nvidia.com/SDK/9.5/Samples/samples.html#imgproc_histogram

uses cg

[3Dickulus]

..hmmm FragM wants core 4.5 profile on linux, or rather that is what is requested but different setups may cause Qt to decide on something else :-/ maybe the buffershader just needs a revisit ?

[3Dickulus]

Let's say the final image is 3840x2160

What would be a good size for a "subsample" histogram? 640x360?
Would the histogram need to be recalculated every frame?
Would that cause inconsistencies requiring dynamic adjustment of exposure to maintain... ambience?

At the outset my thoughts are that it might be a good thing to enhance single images but could get very very complicated over many images if dynamic adjustments are required. A purpose built image processing suite might be better suited to the task.

Just wrote a little doodle for histogram equalization about 75 lines of code, using a smaller image for the initial histogram might make it fast enough to be tolerable on CPU but could easily (I think) be integrated into bufferShaderProgram or a small shader to execute between shaderProgram and bufferShaderProgram?

There's lots of room in bufferShaderProgram for more GLSL.

[3Dickulus]

got 8bit histogram running on CPU

[3Dickulus]

the above runs the image through this code (smoother now) before displaying it from HiresRender->Preview Image

at around line 1830 in MainWindow.cpp

                QImage histoImage;
                histo = new Histogram(&finalImage, &histoImage, this);
                label->setPixmap(QPixmap::fromImage(histoImage));

needs proper weighting, crude, just something to play with, include source...

#ifndef HISTOGRAM_H
#define HISTOGRAM_H

#include <glm/glm.hpp>
#include <glm/ext.hpp>
#include <QObject>
#include <QImage>

class Histogram : public QObject
{
	Q_OBJECT
public:
	Histogram(QImage *const src, QImage *const dst, QObject *parent = 0);
	
public slots:
	void equalization();
private:
	QImage *srcImage;
	QImage *dstImage;
	
};

#endif // HISTOGRAM_H

Histogram::Histogram ( QImage *const src, QImage *const dst, QObject *parent ) : QObject ( parent )
{
    srcImage = src;
    dstImage = dst;
    equalization();
}

void Histogram::equalization()
{
    const int HISTOGRAM_SIZE = 256;
    const float MAX_VALUE = 255.0;
 
    const int w = srcImage->width();
    const int h = srcImage->height();

    *dstImage = QImage ( w, h, QImage::Format_RGB888 );

    uint rhist[HISTOGRAM_SIZE] = {0};
    uint ghist[HISTOGRAM_SIZE] = {0};
    uint bhist[HISTOGRAM_SIZE] = {0};
    // acquire histogram data
    for ( int y = 0; y < h; ++y ) {
        for ( int x = 0; x < w; ++x ) {
            const QRgb pix = srcImage->pixel ( x, y );
            // count distribution
            ++rhist[qRed ( pix )];
            ++ghist[qGreen ( pix )];
            ++bhist[qBlue ( pix )];
        }
    }

    uint rsum = 0;
    uint gsum = 0;
    uint bsum = 0;
    uint rLut[HISTOGRAM_SIZE] = {0};
    uint gLut[HISTOGRAM_SIZE] = {0};
    uint bLut[HISTOGRAM_SIZE] = {0};
    float scale = MAX_VALUE / (w*h);    // scale factor ,so the values in LUT are from 0 to MAX_VALUE
    // build a cumulative histogram as LUT
    for ( int i = 0; i < HISTOGRAM_SIZE; ++i ) {
        rsum += rhist[i];
        rLut[i] = rsum * scale;
        gsum += ghist[i];
        gLut[i] = gsum * scale;
        bsum += bhist[i];
        bLut[i] = bsum * scale;
    }

    // transform image using sum histogram as a Look Up Table
    for ( int y = 0; y < h; ++y ) {
        for ( int x = 0; x < w; ++x ) {
            QRgb pix = srcImage->pixel ( x, y );
            const uchar rlevel = ( uchar ) ( rLut[qRed ( pix )] );
            const uchar glevel = ( uchar ) ( gLut[qGreen ( pix )] );
            const uchar blevel = ( uchar ) ( bLut[qBlue ( pix )] );

            pix = qRgb ( rlevel, glevel, blevel );
            dstImage->setPixel ( x, y, pix );
        }
    }

    return;
}

[3Dickulus]

I found this to be a fascinating read on the subject... https://www.graphics.cornell.edu/pubs/2004/Mol04.pdf

[3Dickulus]

the fastest most efficient? https://www.uni-koblenz.de/~cg/ws0809/Seminar_GPUProgrammierung/Material/09_Algorithms_Histograms/GPUHistogram_Kubias2007.pdf
I understand the principal and what the code does but having some difficulty figuring out how to fit that tiny frag and glCalls into FragM code flow.

[3Dickulus]

Ok, based on GPUHistogram Kubias 2007 the raytracer.frag can be modified to put a lum value in the depth buffer as described in the paper.

image

lum in depth buffer

that's half of it, now just need to render with occlusion query n bin times while increasing z pos from 0 to 1 in 1/n increments... at least that's what I understand from the paper. not an easy task as FragM is already rendering to a quad so the occlusion query plane will have to be moved between Eye pos = 0 and render-plane pos = 1 or the slice-plane will be occluded by the render-plane? not sure how to tackle this... gonna sleep on it ;)

[3Dickulus]

oh! just like the spline shader! post accumulation, the depth buffer will be full of LUM values, so before buffershader executes run the occlusion queries, then hand the histo data to the buffershader? need to create a vbo for the quad, disable depth buffer write and enable depth buffer test.
the paper is describing 256 bins = 8bit, FragM will need more than that... or do it once each for RGB?