Mypaint already has a wiki covering Linear Light and Color Management:
There is a great GIMP thread here that really shines a light on all of the issues with keeping backwards compatibility and appearances. MyPaint should listen carefully and consider the longer picture and go with a simpler approach:
http://www.gimpusers.com/forums/gimp-developer/14817-why-the-endless-background-conversions-between-linear-and-regular-srgb-trc
I think anyone interested should watch this presentation and try to grasp the concepts. hit present to have animations:
And:
Also:
Scene Linear might be really awesome. You could work on a painting that could be displayed dynamically to allow the viewer to āexploreā your painting. That is, normally information is lost in the shadows and highlights. Your workflow could be to do your painting in the ānormal rangeā. Then, adjust the exposure either up or down and add more details to the shadows and highlight areas.
And:
https://docs.blender.org/manual/en/dev/render/post_process/color_management.html
If we implemented OCIO and Linear reference we could have a very flexible and modern painting tool.
More thoughts Image editors using internal linear gamma color spaces
OpenColorIO doesnāt appear to handle ICC profiles at the moment, but it is on the roadmap for 2.0. However, ICC profiles arenāt anywhere near as good as 3D LUTs (I donāt quite understand why, but look it up)
https://groups.google.com/forum/#!topic/ocio-dev/aSc8rz1W7y8
So, if I understand (not that likely) one would need a 3D LUT of their monitor to get end-to-end color management.
More background:
http://nbviewer.jupyter.org/gist/sagland/3c791e79353673fd24fa
So hereās a (naive) gameplan:
Well hereās my attempt so far to modify the pixbuf of a cairo surface. Havenāt had a lot of success other than being able to fill the pixbuf with any color I want. Hereās a workflow by @troy_s (whoās an invaluable resource!) to help setup OCIO (3d lut, etc)
So here Iām affecting every GUI interface color picker thing, which is pretty nifty. For the actual drawing surface a different technique will be needed, I believe (it might actually be a lot easier). Also, Iām re-creating a new āprocessorā for every single call, I think this is totally wrong. I should be reusing the processor and keeping it around, so Iāll have to figure out how to do that.
https://github.com/briend/mypaint/blob/OCIO/gui/colors/bases.py#L67-L104
Ok, this is getting much better. Almost all the gui picker things are now color managed by OCIO. In the below screenshot you can see all the color thingies on the sides have slightly ādullerā colors than those on the canvas. Go ahead, use the color picker and check the saturation and hue angles. The ones āon canvasā are perfect sRGB colors and the colors on the side panels have been corrected for an āAdobe RGBā monitor.
When I use MyPaint on my Adobe RGB monitor, the colors on the side panels look like the colors you see on canvas here. Likewise, the colors on the canvas are ridiculously saturated and hue-shifted. Next step is to get the actual canvas color-corrected, oh, and the palette in the lower right there is not color corrected yet.
Ok Iām getting a bit farther, Palette is corrected now (still a few more guis to fix). Iāve defined 4 OCIO processors. These should be able to let us choose an arbitrary internal reference space such as linear or flmic_log, while still using the normal cairo widgets that are sRGB. Whenever a pixel is displayed on the screen it will take one of two paths. Either it will be a cairo/gui pixel and take the sRGB to Display path, or it will be a pixel from the drawing surface which is āin referenceā, so that pixel will take the āreference to Displayā path.
Before weāre able to use a different reference weāll have to switch to floating point and remove a lot of stuff that might CLAMP the values, thankfully @maxy has pulled out of the attic some old branches that should really help with this.
But the next step is still to get the actual tiled surface to be processed through the āreference to displayā OCIO processor, and at least MyPaint will look good on any monitor (assuming you have a 3DLUT)
Funny, while dealing with a cairo pixelbuffer is a pain in the neck, getting the pallete to be corrected for display was just these two lines:
hi_rgb = app.sRGB_to_Display.applyRGB(hi_col.get_rgb())
fill_bg_rgb = app.sRGB_to_Display.applyRGB(col.get_rgb())
a note on performance: If you have just one color gui thingy I canāt tell if it is any slower, but if you add 3,4 or ALL, it definitely goes slower. However, the color picker doesnāt need to be super high performance, what Iām really worried about is the actual drawing surfaceā¦ hopefully Iāll know soon how that goes!
Chatting a bit more with @troy_s, I learned that we could support wider gamut like Adobe RGB (or other colorants) pretty easily by switching to Cairoās RGB30 mode that allows 10 bits per pixel (ints). Since we donāt need an alpha channel for UI elements like color pickers, this should be fine. TODO figure out if cairo is using a straight 2.2 gamma or the actual 2-part. If it is straight 2.2 it will be a bit easier.
Ok I have the drawing surface color-corrected. However at the moment the color picker is sampling the display-referred space instead of the internal reference, resulting in this madness:
This video is actually pretty interesting because you can see exactly how the color management is working (Besides the obvious malfunction). For red, each time I pick the color the new color is a different āredā Itās a bit darker but also the hue angle shifts just a bit. Same with Blue. This is because Adobe RGB is āsupposedā to have the exact same colored LEDs for Red and Blue. Obviously my monitor is just a bit off. However, Adobe RGB has a very different green compared to sRGB (rec 709). So, when I color pick my green, each subsequent correct changes the hue angle a LOT. Itās almost turning red by the time I get to the middle. Here is the spread for each RGB:
Red: 0 degrees to 32 degrees
Green: 120 degree to 49 degrees (and I only did a few steps compared to R and B)
Blue: 240 degree to 247 degrees
Performance is really bad when panning, moving layers. The most important thing, drawing, is actually pretty fast and almost not an issue. Right now Iām using Python OCIO and they hint that you should use C++ for high performance applications. MyPaint already uses C++ here and there for pixel operations so Iām going to look at moving it all over to c++ for OCIO stuff.
In the video above, the colors look a dulled because it is meant for an Adobe RGB monitor (MY monitor, specifically). However, when I save the file to disk and open in GIMP, for instance, the color picker returns 255,0,0 for red, 0,255, 0 for green, etc. This is GOOD, it is preserving the internal reference without any display correction.
Hereās a few quick profiling snapshots:
drawing:
moving layer:
color picking:
panning view:
Ok Panning/zoom/rotate/flip are now 100% speed. I was confusing rendering with caching and had the OCIO transform in the wrong spot. Thanks @achadwick!
file:///tmp/mypaint-profile-hxTuAz/20171119-092007-1.png@troy_s has been an enormous help sorting this all out. The end goal is to hopefully have the input colors come from 10 bit Cairo RGB non-linear perceptual space (hijacking the 2.2 gamma hopefully) and then transform to float32 numpy for everything in whatever space you want configuring via OCIO (linear or filmic-log probably). All operations internally will just work with these floats and not do any conversions at all. Finally, just before rendering to screen the numpy array will be transformed for the display device, converted to 8bit int and stuffed back in the mypaint surface/gdkpixbuf and then blitted
God I hope Iāve got that all right.
This might be interesting-- float32 is way faster than float16 numpy: Python Numpy Data Types Performance - Stack Overflow
So for 2 days I didnāt even try flipping my setting in my OCIO config from Reference: SRGB OETF to Reference: Linear. Just flipping that switch is enough for colors to start blending like this:
Instead of this nastiness 
Oh, I didnāt even notice thisā¦ but look at the GUI elements. OCIO is now managing the top bar and the icons. With the sRGB reference it looks washed out, while the linear reference (top) it looks normal (actually maybe normal should be washed-out). I guess this is expected when you literally jam OCIO into every place you see pixel buffers
I started replacing 8bit numpy arrays with 32bit numpy. I resurrected Jon Nordbyās floating point libmypaint branch, too.
Even with OCIO active, performance seems to be actually much better than normal mypaint, which I suppose is partly because numpy32 is faster, but also there is less conversions going on, like all the fix15ā¦ Clearly, color for the tiles is horribly broken but at least it is compiling and running!
Here is the required OCIO libmypaint branch (really nothing to do with OCIO just keeping the name consistent):
https://github.com/briend/libmypaint/tree/OCIO
and the mypaint branch:
https://github.com/briend/mypaint/tree/OCIO
Memory usage for one layer is about 150MB for a 1080P screen. zoomed out to 50% (4X more area) and filling in the entire screen with a big air brush, the memory is 500MB. So, really not that bad at all IMO. I created 10 more layers and did some painting on each and my memory is at 1.5GB. I did basically the same thing in Krita (2000x4000 32bit) and created 10 more layers and it was about the same memory usage, so at least that much makes sense.
My assumption is that the color data is being skewed by some horrible incorrect data mismatch going from 8bit to 32bit, so hopefully the performance is not going to vanish as soon as I fix that.
So I was a little bit too eager to replace every uint8 with a float, when GDK buffer definitely needs to remain 8 bit. So I undid a lot of stuff and finally got something that actually doesnāt look broken until closer examination:
Iām not really doing anything to convert from 8bit int 0-255 and floats, which might explain this. It looks pretty neat but not useful yet. Iām hesitant to clamp the float into values 0-1, since Iām looking to implement scene-referred painting. If I can ever get my head wrapped around it. . . .
Update, Iāve got the background colors handled, I think, and alpha is working as expected. Iām constraining alpha to 0-1.0, which makes a lot of sense. Thereās no reason to allow that to be bigger. However, Iām feeding bad values into libmypaint, I think, which expects RGB to be floats between 0-1. So, right now I have no color data but itās looking nice:
More progress! Fixed the color issue, and got the 32bit float numpy tiles āmanagedā by OCIO, and flipped my OCIO config to Linear Reference to do a blend test. This is with the ānoiseā disabled for now, so there is some banding.
More progress. The internal model is almost completely transitioned to linear light and associated alpha (premultiplied color). Here in this video you can see how you can build up light intensity beyond what the display can show (without adjusting exposure, still need to add that OCIO controller). Then you can add opaque ādarknessā back to bring back the original lines (since they were there all the time just brighter than the white).
Painting with a āzero opacityā or zero alpha brush is not intuitive at first glance, so Iām looking at either adding an addition control (alpha) or possibly changing the default blend mode to Multiply, and make this current ānormalā mode into a new option like āadditive modeā.
From the OpenEXR technical introduction:
Calling the color channels āpremultipliedā does not mean that the color values in an image have actually been multiplied by alpha at some point during the creation of the image, or that pixels with zero alpha and non-zero color channels are illegal. Non-zero color with zero alpha is legal; such a pixel represents an object that emits light even though it is completely transparent, for example, a candle flame or a lens flare.
Hereās a better demo, now I have an āAlphaā slider. This is just a boolean toggle to make the Opacity slider adjust ONLY the alpha channel (thanks @troy_s for that tip!). This lets you paint normally, or flip it to paint with control of the alpha (occlusion). So, if you want to paint a flame or something that isnāt blocking light, you flip that alpha switch and then you can reduce opacity to zero and paint with light.
Also, you can go back and āerase the occlusionā from things on canvas already.
and a continuation:
Iāve changed the setting to āOcclusionā (for now, for lack of a better word) and made a brush that gradually becomes transparent (loses occlusion) towards the end of the stroke. Once it is backlit you can see the difference between Opacity and Occlusion:
How do you convert from 10 bit Cairo RGB to float32 numpy? Maybe you have some fast trick using numpy.right_shift() or something. Hereās my solution, possibly I confused the order of colors, maybe your solution is faster or cleaner:
surface = cairo.ImageSurface(cairo.FORMAT_RGB30, width, height)
...
buf = surface.get_data()
bgr = np.ndarray(shape=(height, width, 1), dtype=np.uint32, buffer=buf)
bgr_bg_b = np.right_shift(bgr, (((0,10,20),),))
bgr_bg0_b00 = np.left_shift(bgr_bg_b, (((0,10,20),),))
r_g_b = bgr_bg0_b00
r_g_b[:,:,0:2] -= bgr_bg0_b00[:,:,1:3]
r_g_b = r_g_b/2**30
r_g_b *=255
r_g_b = r_g_b.astype(np.uint8)
cv2.imwrite("r_g_b.png", r_g_b)
What do you mean by āCairo RGB non-linear perceptual spaceā? I thought that Cairo canāt correctly handle non-linear/perceptual color spaces such as sRGB.
Thank you!
Probably better than subtracting shifted versions is to use bitwise_and.
The question about non-linear/perceptual color spaces remains.
I didnāt understand anything, but itās very interesting
