Seeking feedback: pigment setting options

jpll · October 17, 2019, 7:49am

Some background

Anyone who has tried out any of the alpha builds will know about the new Pigment setting that enables spectral colour mixing (implemented by @briend) which makes colours blend in a more natural way (and generally with much nicer results). The drawback to the spectral mixing is performance - especially noticeable on slower systems or high resolutions (when working with large brushes, that is).

Right now, the pigment setting is on by default. This means that any brush that does not include that setting - almost all of them - will have spectral mixing enabled.

I’m working on a branch that allows users to override the defaults, and would like some feedback on the functionality. The branch adds a submenu called Pigment options to the Color menu. Right now this menu contains two mutually exclusive options (only one can be active at a time): Pigment setting on by default and Pigment setting off by default. The default value is on by default.

What these options do is determine whether or not to enable spectral mixing for brushes that do not define that setting - meaning all older brushes. However, if you explicitly set the value of the Pigment setting for a brush, switching to either of these options will not affect it.

I’m considering whether to expand these options with Pigment setting always on and Pigment setting always off.

Additionally, @briend floated the idea of adding a modifier - similar to the eraser/alpha lock/colorize modifiers - that would enable/disable pigment for the active brush. Such a modifier would provide more flexibility when working with multiple brushes and needing to switch Pigment on/off for the same brush somewhat frequently.

Questions

Should the pigment setting be on by default?

It is a bit unusual for a new setting to be enabled by default, when it changes the behaviour of components that were designed without it in mind. On the other hand, most of the old brushes work really well with spectral mixing and having it enabled by default makes it less likely that users will fail to notice its existence and impact.

Personally I think it should be left on by default - it is a major new feature after all.

Is the Pigment options menu a good solution, and does it need those two extra options?

Other suggestions are appreciated.

Overriding unspecified defaults is enough.
Having options for always-on/always-off is necessary.

0 voters

Should a mode/modifier for switching Pigment on/off also be added

There is no reason for this to be mutually exclusive with the options menu. It would not be part of the same group as the existing modifiers, because it would not be mutually exclusive with them (the Pigment modifier could be enabled at the same time as e.g. the Alpha Locking modifier)

It’s probably not that hard to add, it’s just a question of whether it’s needed (and how to represent it in the gui).

Yes, a modifier is necessary for my workflow.
No, a modifier is not necessary for my workflow.

0 voters

That’s all, any input is appreciated!

toniw · October 17, 2019, 7:27pm

I personally think its enough to have an on/off switch in the brush settings and have old brushes use the default (on) to keep it as simple as possible.

The only reason to have a global off-switch of some kind - for me - would be because of the performance impact, which is quite noticable (even using unsafe-optimizations). Is It likely that the pigment mode’s performance will significantly improve in the future?

Also, is this still the relevant thread for the pigment mode topic: Smudge Tweaks testers needed ?

jpll · October 17, 2019, 7:49pm

When you say brush settings, do you mean the brush settings editor, the options panel (with the small selection of sliders), or the Brush menu in the menu list?

My concern is that the only way for a user to get non-pigment by default at the moment is to explicitly disable the setting and resave all the brushes they don’t want to use pigment with (I would guess that inking, sketching and filler brushes would be the primary cases).

The off-by-default/on-by-default options (that can be bound to key combinations, of course) would essentially be a (sort of) backwards-compability measure.

In the near future I think it’s unlikely. Brien experimented with a floating-point backend representation a couple of months ago that, iirc, sped things up a fair bit. The tradeoff is increased memory consumption and some performance degradation for other kinds of blending.

Not sure about that one.

briend · October 18, 2019, 2:12am

I think we have some tough decisions to make on a lot of things, really. Maybe I should make a new thread on the topic.
Suffice it to say the current alpha’s pigment mode is really pretty horrible and was premature, for sure (oops). I think we can blame @jpll for that since he wasn’t around to stop me from merging it.

That Smudge thread is defunct pretty much, but all that stuff is still in my personal branch. I have just one branch for mypaint and libmypaint now that you can try if you are brave.

jpll · October 18, 2019, 3:52pm

Heh, yes I can’t deny my negligence there Though I don’t think I would have necessarily vetoed the merge even if I was in a position to do so.

Just for the record, personally I basically always have pigment enabled, except when sketching/inking at really high resolutions, but that’s because most of the time performance is not an issue when sitting on a modern 3.8GHz desktop CPU. However it definitely is an issue for those who are using tablet pc’s, older/slower laptops or just older/slower hardware in general. For those people some kind of option like this would be necessary to allow them to make use of the other new features/fixes that have been introduced since the last release, with minimal inconvenience.

It probably does make more sense to place these options under the Brush menu and/or the tool options panel. After all, the color menu only contains options/actions for picking or changing the current color, while the brush menu already contains additional brush-related stuff. One argument against adding a switch in the tool options panel is that all of those sliders relate only to the active brush, whereas the new switch would act on the default value of a setting itself.

Anyway, there’s no need to stress a decision on this - the basic patch is already written (with only the added default-options), and can be applied at any time (or not at all).

toniw · November 10, 2019, 4:38pm

So I tried setting pigment mode to zero for performance (because I’m painting zoomed out 50% with high dabs count) and noticed that the blended colors seem too bright. @briend I tried your fork and that problem does not occur there but I also don’t see any change when using the pigment slider (neither in color nor performance). Is there a way around this / can the old 1.2.1 behaviour still be used in later versions?

jpll · November 10, 2019, 6:03pm

In the latest libmypaint a pigment value of 0 will result in the old smudging being used - anything else will use the newer and heavier (but more physically realistic) version. This behavior was added in the libmypaint commit 1130ac889dda1ab629817180455dd776dd52a340 which I merged earlier today.

However, it still shouldn’t look like that from just setting the pigment to 0 and smudging. That middle one looks more like the effect of using the Normal layer type when the eotf value is set to 2.2 (default).

To partially answer the last question, I have a patch that adds an option to switch to a “Legacy Color Mode” (currently on a per-session basis) that sets the eotf to 1.0 and inverts any color transform that has already been applied to loaded color data. It needs some more work before it can be added to master though. If you’re interested, I can push the branch somewhere, with a clear “use at your own risk” warning.

edit: I can’t replicate your result w. current master of mypaint/libmypaint, but I did notice that I introduced a problem by upping the number of pixels used in the color sampling function used in the smudge routine; the accumulated errors caused by the fastpow power function approximation causes the output to be darker than it should be (that was the case before as well, but fewer samples makes the effect more gradual). So that needs to be fixed by either lowering the number of samples or switching to using powf in that function. If you are using the latest master, this may have had some impact on your tests.

toniw · November 10, 2019, 8:31pm

I am indeed using the latest master, but i also noticed this behavior a week ago or so.

I can easily reproduce it using a clean config. I fill a layer with red and use any smudge brush with low pressure to blend. I do this on a single layer and it happens for both Normal and Pigment Layer type equally.

I found the setting “display.colorspace_EOTF” inside my settings.json and tried setting it from 2.2 to 1.0 as you mentioned. When setting pigment then to 0 it now looks like the “old” color mixing, which is fine for me for now . If you want me to test stuff like this you can send me a a link to the branch

jpll · November 10, 2019, 9:06pm

Hmm, good that the eotf setting solves it for you, but it’s weird if you get that output for both the Normal and the Pigment layer types. Could you upload and link to a minimal example?

briend · November 11, 2019, 6:40am

The EOTF of 1.0 will make the pigment mode really terrible, unfortunately (much too dark!).

I’m trying to think of another way to have these models co-exist. Maybe something like this could make sense:

I’m thinking if we changed the layers window UI so that it is divided into three “Domains”, we could have a pipeline that doesn’t flip flop around. Domains would be a bit like Layer Groups, except they would not be rendered isolated, they would simply enforce that they are adjacent in the stack and not co-mingled with other domains.

After the Log Domain layers are composited down, it would be transformed to linear via exp() and converted from N spectral channels to RGB. Then it would continue to composite the Linear Domain layers. After the last Linear domain layer is composited down, it is transformed to the Display Domain via the OETF (^1/2.2 or ideally piecewise). Finally any legacy layers in the Display Domain are composited and the result goes to screen (after Color Management? hah).

Any particular Domain could be totally empty, except the background layer in the LOG Domain. For traditional painting you’d probably only use the LOG domain layers. Each domain would have its own possibly overlapping list of modes that make sense for it. For instance, “Colorize” wouldn’t make sense for the Spectral LOG domain, since it would need to flip flop to RGB and back again.

LOG Domain layers would be a special subclass with extended data model for the extra channels. Likewise, libmypaint would have to handle # of channels dynamically (which would likely change when you move between domains), and we’d have to prevent trying to use Colorize or Color settings when using LOG domain (to avoid flip flops). Then we could just eliminate the Pigment Setting slider entirely, and libmypaint wouldn’t have to blend the two models internally, which IMO is pretty useless and very inefficient.

Too complicated? Yeah, probably I’m probably overlooking something important, as well. I would much rather abandon all of the legacy stuff and just do one model. We could try to decide if we want MyPaint (3.x?) to be a whole new thing, a totally specialized traditional physically modeled painting app without any baggage. . . not sure if most would agree about that move though.

jpll · November 11, 2019, 6:15pm

Yep, maybe it’s better to have a more general legacy switch that, apart from changing the eotf and reverting transforms, also changes the default layer type to Normal and sets the pigment default to off. I think that would be sufficient for backwards compatibility. If we add a --legacy flag as well, that would make it easier for users to create different shortcuts/aliases for when they want to work on older files (or files that need to be trivially compatible with external editors) without having to use an older version.

MyPaint has been used by thousands* of people for some 13 years (probably quite a few .ora files around as a result) so if possible I’d like to not break backwards compatibility and WYSIWYG interoperability (copy/paste and external editing) without a fallback for this release.

* The number of downloads does not translate to actual users of course, but fwiw the windows installers for 1.2.1 have over 200k downloads between them from the github release alone. I would guess that the number of active users numbee at least a few thousand, but who knows.

On another note, my fix for the artifacts caused by dark colors introduces some new and fresh artifacts(!) when smudging from a bright color to an extremely dark color. At least this source of artifacts is a little more benign than the others, since smudging further does clear up the offending pixels, but I’ll have to try to find out what the actual root cause of these artifacts are instead of blindly adding more conditionals (unless the amended conditional I just thought of happens to work ).

toniw · November 14, 2019, 4:25pm

What kind of example would be most helpful? an .ora file, a video of the process or something else?

jpll · November 14, 2019, 4:59pm

An .ora file would be better, maybe along with a screenshot of that file, to see if there’s some discrepancy.

toniw · November 14, 2019, 9:26pm

Screenshot:

ORA-File:

download

Used Brush:

Dieterle/Fan#1 (but any smudge brush works)

Settings:

"display.colorspace_EOTF": 2.2,
"display.colorspace": "srgb",

jpll · November 14, 2019, 10:50pm

Yep, that is consistent with the behavior here on latest everything. And switching to eotf 1.0 w. pigment slider turned all the way down makes the behavior the same as in 1.2.1. Unfortunately I don’t think it’s feasible to have full legacy mixing co-exist with the pigment-friendlier eotf of 2.2. At least not without adapting the old blending model (which might very well be impossible, @briend would have to chime in on that one).

briend · November 15, 2019, 12:43am

Combining models is totally possible, but it’s a trade off with performance. GIMP, for instance, has a menu to select legacy layer modes or linear modes, but presumably that will do a lot of flip-flopping and you should notice performance drops when mixing things up.

I think the simplest thing to do would be to immediately start encoding a version mypaint-2.x into the ORA format metadata somewhere so that we can treat old files special; maybe it makes sense to open old files with EOTF at 1.0 and also default pigment to zero and mode to normal (basically like @jpll describes with the legacy switch). Legacy switch makes sense too, I think.

I’m still thinking about the 3-domain model. . . maybe someday. It would be great; perfect backwards compatibility and efficient mixing of models. . .

jpll · November 15, 2019, 7:11pm

That is probably the most sensible thing to do. Even though by now there are probably a lot of files making use of pigment and eotf=2.2 (I have quite a few myself) the overall inconvenience of having to explicitly state “I don’t want this file to use legacy” is probably less than the opposite, given a reasonable assumption about the volume of files out there.

Of course, we could have a heuristic (that could be turned off) that when opening an .ora checks if there are any pigment-type layers in the stack, and takes that into account when making the decision.

Maybe instead of storing a mypaint-2.x attribute we should store a mypaint:eotf="2.2" / mypaint:eotf="1.0" attribute instead, and assume that if the file lacks the attribute, and any pigment layers, it is a legacy file. Otherwise we’ll have to be careful about not writing the mypaint-2.x attribute when saving legacy files.

briend · November 16, 2019, 5:32am

Argh, that’s a really good point. Seems like we’ll need to use heuristics, that’s a bummer. There’s no way to be 100% sure in the case where someone didn’t use a pigment layer. Ah well, hindsight is 20/20. We should really do both; put the EOTF and the verbose mypaint:version in the stack.xml. . . or maybe the mypaint-settings.json? Not sure which would be more appropriate or if it matters at all. Guessing Andrew added the json for convenience and to keep the stack.xml from getting too cluttered with mypaint extensions.

So if we load an old legacy file (one without mypaint:eotf, or any pigment layers) we can set the eotf to 1.0. Then it should be safe to write the file with mypaint:version:“2.x.x” as well as mypaint:eotf:“1.0”. Do you think it would be ok to automatically “upgrade” the legacy file to 2.x, as long as we capture the eotf?

jpll · November 16, 2019, 10:16am

Agreed about storing both. As for whether to use a new namespaced attribute or a new doc-local settings key/value, I think that depends on where the info is less likely to be deleted by other applications (GIMP/Krita) and whether other applications will want to access it.

I will test whether:

a) Unknown/unused xml attributes are erased/overwritten indiscriminately (I think we probably do…).

b) The mypaint-settings.json file is retained when re-saving an ora from GIMP/Krita.

Afaict the ora spec does not go into details about how to handle these kind of things.

Edit: It’s all tossed away, and on second thought that probably is the right thing to do - if you (the application) leave in a bunch of data you know nothing about, you also don’t know if you’ve broken any dependencies or assumptions that the data relies on. We’ll just have to be clear about these caveats in the documentation.

I think the doc-local mypaint-settings.json was added primarily to support the storing of layer views - which naturally don’t really fit into the stack itself. I can see it also being useful for a lot of other things though - storing symmetry states, color and brush histories etc.

I can’t think of any problems with it. We might also want to coordinate that part with the Krita team, so that the new 2.x.x ora files can be rendered correctly in Krita without having to manually switch to linear sRGB, but that’s somewhat of a separate issue.

jpll · November 23, 2019, 2:56pm

Compat features branch: GitHub - jplloyd/mypaint at backwards-compatibility

Fairly rough implementation of basic backwards compatibility.

Two compatibility modes: 1.x and 2.x, with the default being configurable (default default is 2.x ). The default mode is the one used when starting MyPaint.
By default, files without pigment layers (or mypaint:eotf=“1.0”) are opened in 1.x compat mode. Files with pigment layers (or mypaint:eotf != “1.0”) are opened in 2.x compat mode. This behavior is configurable with additional options: always open as 1.x and always open as 2.x.
Each compat mode has options for default layer type and whether pigment should be on or off by default. Defaults here are (Normal, Off) and (Pigment, On) for 1.x and 2.x respectively. Changing these options does not require reloading for the change to take effect.
In 1.x compat mode, the otherwise eotf-dependent tile transformations will just delegate to the old versions that are significantly faster.

Screenshot of the preferences tab

Caveats / Missing features
The scratchpad is not handled very well. I have not included the patch for reverting the transforms directly on the tiles, so the scratchpad is simply saved and reloaded when the compat mode changes.
In order for the scratchpad to always function as expected, it basically needs to be cleared after switching modes; the stroke map colors might not match stroke appearance and the layer type is not automatically switched to the default of the active compat mode.

Brush color is not updated correctly on eotf change (just noticed). Not a big issue, as brush color is usually changed frequently anyway (and it is easy to do so), but still an issue.

No on-the-fly compat mode switching.
Transforming back and forth is a (slightly) destructive operation, but making backups of all tiles can eat up a lot of memory, so there are no perfect solutions for making the switch reversible (which is necessary, it’s either that or tossing away the undo history when switching*). Additionally, the stroke maps would need to be updated as well. In short, it’s a bit more work than I thought.

* Or apply the transforms to the undo history too, but let’s not even contemplate that.

Do try out the branch if you have old (and new) .ora files available to try out. The only thing that should be at risk is your scratchpad.

As long as the actions of the current compat implementation are non-destructive I don’t think there is a problem amending these features down the line if requested.