Faster Gaussian blur implementation for PhotoModularFX

[tannerhelland]

I notice that PMFX's current Gaussian Blur filter slows down significantly as radius increases. Since (I think?) PhotoModularFX works on floating-point data, it is a good candidate for an IIR implementation of Gaussian Blur, similar to what GIMP uses.

A nice BSD-licensed reference implementation is available here:

http://www.getreuer.info/home/gaussianiir

The original 1994 paper that inspired the approach is available here (free download):

https://www.researchgate.net/publication/236268568_Signal_and_Image_Restoration_Using_Shock_Filters_and_Anisotropic_Diffusion

On a 10-megapixel image (3872x2592), PMFX takes about 135 seconds to process a 3-channel gaussian blur. In PhotoDemon, IIR code very similar to the above project does the same thing in about 6 seconds, and that's including the overhead of moving all data from 8-bit integer RGB to 32-bit floating-point RGB and back again, plus processing a fourth channel (alpha )- so PMFX could be even faster, I think.

[miorsoft]

Thank you, I'll see.
What's the Radius you used for the test ?
I know it's slow, even if it's done using separable (1D) kernels (first gauss blur in X and then in Y).
Alternatively user can use Box Blur ("Box" option, always BLUR node)

[tannerhelland]

200 px radius.

(So sorry that I forgot to mention this in the first post!)

[miorsoft]

I removed some unnecessary IFs
on 3528x2344 (7.8 Mpix) 200px-radius to me it take about 105 secs
low gain....

[miorsoft]

I translated gaussianiir2d from your link http://www.getreuer.info/home/gaussianiir
super Fast !!!!!
( I used numsteps=4)

About license , (If I decide to put this update) do I have to mention that link somewhere ?

PS:
Today I updated "mousewheel issues"

[tannerhelland]

Nice work!!

Fast gaussian blur is really helpful for many things, so I think it is a good one to optimize, and IIR is a great technique for this. (For example, I use gaussian blur at large radius for shadow/highlight correction - if you blend the result with the original image, using different blend modes, you can get many useful effects.)

About license , (If I decide to put this update) do I have to mention that link somewhere ?

At the link, he says the code is licensed by Simplified BSD license. This is one of the easiest open-source licenses (https://opensource.org/licenses/bsd-license.html). You can just put some text in your instruction manual, I think - from the link:

2. Redistributions in binary form must reproduce the above copyright notice, this list of conditions and the following disclaimer in the documentation and/or other materials provided with the distribution.

[miorsoft]

Updated with Gaussian IIR

[Modules]
-Blur; Blur3; Fast Mode
-ToneMap; ToneMap3; LocalMap; DoG; Noise2; PlasticWrap; -Faster (because of Internal faster Blur)
-PlastWrap: Changed 'Details' parameter (Now proportional to image size)
-RGB2HCY and HCY2RGB: reviewed colorspaces conversion algorithm
-EDGES: Reviewed algorithm
[Other]
-Added 'Add FX-Node Selector' Window
-Added Projects-Thumbs Preview. +50 MBytes
-Renaming 'Node' to 'Module'

https://miorsoft.github.io/Site/PhotoModularFX/PhotoModularFX.zip

[miorsoft]

I need a fast Bilateral now.
I thought it can be done in a similar way of IIR Blur.

Then I found Tanner's useful link on my vbForums page:

Fast O(1) bilateral filtering using trigonometric range kernels

Tanner, did you implement it ?
or do you know where I can found some source code to stat from ?
I'm not so good in reading..... (mathematical symbols)

[tannerhelland]

Sorry that I have not had time to try your new Fast Blur implementation! I have been really busy and had not had time for VB6 stuff for many weeks...

Unfortunately, I have not implemented trigonometric range kernel bilateral filtering. :( I don't understand all the math involved, either. I think they do have a source code implementation available here:

http://bigwww.epfl.ch/algorithms/bilateral-filter/

It is implemented as an ImageJ plugin, but maybe there is enough code to get started? I am happy to help if I can find some free time.

One thing I did to improve bilateral performance in PhotoDemon is offer a "Fast" separable option (like gaussian blur). If you run the bilateral filter in the horizontal direction, then run the filter in the vertical direction, it only takes O(w + h) time instead of O(w * h). The results are not exactly the same (because the algorithm is not technically separable) but they are close, and for large radii especially the visual difference is not too noticeable.

On a 10 megapixel photo, bilateral filter radius 25 takes about 11 seconds separable, compared to 240 seconds in regular mode. So for me, this is a good trade-off. Maybe it's an easier thing to try first?

[miorsoft]

Yes, I know about separable Bilateral, but I don't like the results.
For large radii I used a (don't know how to say) "Kernel Sub-Sampling", and the results are quite good. I made it using only some points within Kernel Radius. The points are selected with Poisson disk sampling technique. (a quite good result of quality and speed ...)

Ok, about the link of source code:

I translated to VB6 the code in Bilateral_Filter_Polynomial.java
It easy to do.

It's super fast and the results are good for quite large RANGE sigma

While for Small RANGE Sigmas it becomes strange/odd

It think I translated it correctly

[miorsoft]

Tanner, if you'd like to implement Possion Disc sampling
I found this cool link:
https://bost.ocks.org/mike/algorithms/
Search for "Bridson’s algorithm" for Poisson-disc sampling. it's the algorithm I used.
For big radii, this techinque is very useful, and despite the approximation the results are good.
(in my program I use Poisson Disc in Bilateral and Threshold Blur with a selectable minimun distance between kernel sample points from 6% to 15% of the Kernel Radius.)

[tannerhelland]

Wow, great link!! That is a beautiful visualization, very easy to understand and follow.

Thank you for sharing the screenshots of your results; the large radius looks very good to me. I would need to study the algorithm further to know if the small radii result is expected, or if something else is wrong. (Many approximation algorithms seem to have mixed results at small radii, so it's entirely possible that the small radius problem lies with the algorithm itself.)

I have used Voronoi diagrams before (for "Stained Glass" and "Crystallize" effects) and it is interesting to see a similar approach for sampling. I would like to investigate Bridson's algorithm further for my own code - thank you for showing it to me. I have used some fixed supersampling techniques before (for example, all Distort tools in PhotoDemon require it), but they are much simpler than Bridson's algorithm, and probably not as high-quality.

Re: separable bilateral, I have not noticed problems in my own work with it, but maybe I have not studied it closely? For example, here is my result on your sample image from above, with the displayed settings (the "use estimation" checkbox toggles separable vs standard algorithm):

They look very close to my eye, but it is always possible that I have missed something in my implementation. (Or maybe, a certain combination of settings causes bad results?) I just mention this in case you notice something I didn't, or maybe there are different ways to implement separable bilateral, with varying results!

Either way, if you would like me to test any of these updates for you, please let me know. It may take me a few days but I am always happy to help. Congratulations on your great work on this topic.

[miorsoft]

Thank you very much Tanner.
The example pictures I posted are about Bilateral_Filter_Polynomial.java I translated to VB6 (taken from your link (http://bigwww.epfl.ch/algorithms/bilateral-filter/)
Sorry, but maybe you missed it. The "Wrong" one (the second) is obtained with a small RANGE sigma (not a small kernel radius.) In the other side, It works good for quite big RANGE sigmas , as you see.
To be honest, I did not fully understand the codes of this link. It seems to me that there are more algorithms and I do not know which one is the best.

About Separable, Yes, I was wrong, it is not so bad, I noticed it too in my algorithm.
I do not know why, but I had this idea (that the result was too low-quality), which remained in my mind from the past, without actually carrying out tests.
You are right , Separable is a good approximation.

I'm curious about your 2 parameters : "Color Strenght" and "Color Preservation",
[ also because my filter is using only 1 channel, not all three RGB (or four RGBA). ]

Another algorithm I try to implement without success is this:
Adaptive Manifolds for Real-Time High-Dimensional Filtering
I just try with only 3 manifolds , but I encountered many problems that maybe we'll talk about later.

[tannerhelland]

"Sorry, but maybe you missed it. The "Wrong" one (the second) is obtained with a small RANGE sigma (not a small kernel radius.) In the other side, It works good for quite big RANGE sigmas , as you see."

You're right, I did miss that! Thank you for the clarification. I apologize for not reading your original post carefully.

"I'm curious about your 2 parameters : "Color Strenght" and "Color Preservation",
[ also because my filter is using only 1 channel, not all three RGB (or four RGBA). ]"

Sometimes I try to create names that are easier for users to understand, even if they don't have a technical background. These parameters are just basic adjustments that control the kernel used for intensity differences - e.g., to build a lookup table for RGB colors on the byte range [0, 255]:

    Dim i As Long, k As Long
    
    ReDim m_colorFunc(0 To 255, 0 To 255) As Double
    
    For i = 0 To 255
        For k = 0 To 255
            m_colorFunc(i, k) = Exp(-0.5 * ((Abs(i - k) / colorFactor) ^ colorPower))
        Next k
    Next i

The full source is here.

"Another algorithm I try to implement without success is this:
Adaptive Manifolds for Real-Time High-Dimensional Filtering
I just try with only 3 manifolds , but I encountered many problems that maybe we'll talk about later.

That looks like a very interesting algorithm, especially running in real-time on HD video! I will download the PDF and study it more closely. Thank you for the link.

[miorsoft]

About bilateral I found an interesting code. It is in C
I tried to ask for help on vbforums even though I think the hopes are few.
If you're interested, look here.
http://www.vbforums.com/showthread.php?866895-Translate-Recursive-Bilateral-Filter-from-C-to-VB6
Do you think it's worth trying to translate it?

If you could understand something, let me know.
(do it here on github since you do not log in anymore in vbforums)

[miorsoft]

On my own I found a really simple way to make a very fast edge preserving smoothing filter.
It is done in a similar way to the IIR Blur.
Here are some examples:

Like BLUR IIR it's time invariant.
The speed depends only on the size of the image, not on the smoothing parameter (Sigma or radius)
In the next update it will be available, probably with the names: BilatFast and BilatFast3.
The results are good even if it is not perfect and is a simplification of the algorithm in my previous post. (that maybe one day I will be able to do)