With or without options: additional measurements

[FAMILIAR-project]

There are some options that have the same values in the dataset or the values are unbalanced (lack of diversity). It calls to further explore why (to a reasonable extent). The following applies to 4.13.3

The idea is to measure N configurations with an option o activated and deactivated (we measure 2 x N configurations), and then analyze the difference.

In fact, to reduce the cost of the experiment, we can make the economy of measuring with or without since we have already in the database some measurements.

I propose the following procedure

1. given an option o (eg CC_OPTIMIZE_FOR_SIZE), randomly pick a configuration in the database with 'n' value (says)
2. "mutate" the configuration by changing the value to 'y' (warning: we have to ensure that the configuration is valid and constraints are respected)
3. repeat N times the procedure

Let us take a concrete example. o = "CONFIG_CC_OPTIMIZE_FOR_SIZE" and N=100.
We consider dataset 4.13.3
For doing the first step, we can randomly pick a configuration using the dataframe

import pandas as pd
import tuxml
df = tuxml.load_dataset()
# 0 value is 'n' value, 1 is 'y' value, 2 is 'm' value
df.query("CC_OPTIMIZE_FOR_SIZE == 0")['cid'][:100] # TODO real random ;) 

problem: we need the associated .config files (we might need to restore the database, at least locally).

for the second step, we need to do something like:
vmacher:linux-4.13.3 macher1$ scripts/config -e CONFIG_CC_OPTIMIZE_FOR_SIZE -d CONFIG_CC_OPTIMIZE_FOR_PERFORMANCE
using this nice utility:
https://github.com/torvalds/linux/blob/v4.13/scripts/config

note that we have to manually fix the possible inconsistency and proactively deactivate CONFIG_CC_OPTIMIZE_FOR_PERFORMANCE (the two options are choices and mutually exclusive)
(make olddefconfig is OK)

once done (we have n=100 new .config files), we have to call tuxml/kernel generator with --config

We need some local scripts to prepare/create .config files we want to measure.
And then use tuxml to populate the database.

It is also needed to locate where in the database are configurations with and without option (what are the cids?)

[FAMILIAR-project]

We need the source of Linux 4.13.3:
https://cdn.kernel.org/pub/linux/kernel/v4.x/linux-4.13.3.tar.xz
(for mutating the configuration)

specifics of CONFIG_CC_OPTIMIZE_FOR_SIZE:
https://github.com/torvalds/linux/blob/v4.13/init/Kconfig#L1038-L1051

(it may be harder to other options in general to mutate configs)

[FAMILIAR-project]

interesting report thanks to @psaffray experiments!
we have two cases:

• the mutation of configuration is smooth (we just activate an option and no impact on other parts)...
• the mutation is more tricky: the activation of CONFIG_CC_OPTIMIZE_FOR_SIZE leads to the modification of other values

We way wonder why we have such situations?
Well, let's take an example (see below for the complete log):
With 95587.config : +KMEMCHECK n
in this case, the difference between the original configuration and the "mutated" one is not only about CONFIG_CC_OPTIMIZE_FOR_SIZE and CONFIG_CC_OPTIMIZE_FOR_PERFORMANCE, but also about CONFIG_KMEMCHECK (n value, instead of yes originally)

The reason can be found here:
https://cateee.net/lkddb/web-lkddb/KMEMCHECK.html
( CONFIG_DEBUG_KERNEL ) && (! CONFIG_X86_USE_3DNOW ) && ( CONFIG_SLUB || CONFIG_SLAB ) && (! CONFIG_CC_OPTIMIZE_FOR_SIZE ) && (! CONFIG_FUNCTION_TRACER )
https://github.com/torvalds/linux/blob/v4.13/lib/Kconfig.kmemcheck

so when CONFIG_CC_OPTIMIZE_FOR_SIZE has y, CONFIG_KMEMCHECK has n value.
in the original config CONFIG_KMEMCHECK may have been set to y (and for other cases the value was already n)

With 60969.config :

With 75541.config :
-KMEMCHECK n

With 36050.config :

With 109991.config :

With 95587.config :
-KMEMCHECK n

With 52158.config :
-KMEMCHECK n

With 61410.config :
-KMEMCHECK n

With 84961.config :
-KMEMCHECK n

With 85715.config :

With 108678.config :
-KMEMCHECK n

With 50460.config :

With 84391.config :

With 51643.config :
-KMEMCHECK y
-KMEMCHECK_BITOPS_OK y
-KMEMCHECK_DISABLED_BY_DEFAULT n
-KMEMCHECK_ENABLED_BY_DEFAULT n
-KMEMCHECK_ONESHOT_BY_DEFAULT y
-KMEMCHECK_PARTIAL_OK y
-KMEMCHECK_QUEUE_SIZE 64
-KMEMCHECK_SHADOW_COPY_SHIFT 5
+DEBUG_PAGEALLOC n
+X86_DIRECT_GBPAGES y

With 65027.config :
-KMEMCHECK n

With 100817.config :
-KMEMCHECK n

With 66733.config :

With 115083.config :

With 31167.config :

With 109504.config :

With 56140.config :
-KMEMCHECK y
-KMEMCHECK_BITOPS_OK n
-KMEMCHECK_DISABLED_BY_DEFAULT n
-KMEMCHECK_ENABLED_BY_DEFAULT n
-KMEMCHECK_ONESHOT_BY_DEFAULT y
-KMEMCHECK_PARTIAL_OK n
-KMEMCHECK_QUEUE_SIZE 64
-KMEMCHECK_SHADOW_COPY_SHIFT 5
+DEBUG_PAGEALLOC n
+SLUB_DEBUG_ON n
+X86_DIRECT_GBPAGES y

With 43228.config :

With 114560.config :
-KMEMCHECK n

With 103074.config :

With 116131.config :

With 88408.config :

With 68438.config :
-KMEMCHECK n

With 30199.config :

With 83256.config :

With 101863.config :

With 90527.config :
-KMEMCHECK y
-KMEMCHECK_BITOPS_OK y
-KMEMCHECK_DISABLED_BY_DEFAULT n
-KMEMCHECK_ENABLED_BY_DEFAULT n
-KMEMCHECK_ONESHOT_BY_DEFAULT y
-KMEMCHECK_PARTIAL_OK n
-KMEMCHECK_QUEUE_SIZE 64
-KMEMCHECK_SHADOW_COPY_SHIFT 5
+DEBUG_PAGEALLOC n
+X86_DIRECT_GBPAGES y

With 50322.config :

With 42915.config :

With 50374.config :
-KMEMCHECK y
-KMEMCHECK_BITOPS_OK y
-KMEMCHECK_DISABLED_BY_DEFAULT n
-KMEMCHECK_ENABLED_BY_DEFAULT n
-KMEMCHECK_ONESHOT_BY_DEFAULT y
-KMEMCHECK_PARTIAL_OK y
-KMEMCHECK_QUEUE_SIZE 64
-KMEMCHECK_SHADOW_COPY_SHIFT 5
+DEBUG_PAGEALLOC n
+X86_DIRECT_GBPAGES y

With 43243.config :
-KMEMCHECK n

With 80134.config :
-KMEMCHECK y
-KMEMCHECK_BITOPS_OK y
-KMEMCHECK_DISABLED_BY_DEFAULT n
-KMEMCHECK_ENABLED_BY_DEFAULT n
-KMEMCHECK_ONESHOT_BY_DEFAULT y
-KMEMCHECK_PARTIAL_OK y
-KMEMCHECK_QUEUE_SIZE 64
-KMEMCHECK_SHADOW_COPY_SHIFT 5
+DEBUG_PAGEALLOC n
+X86_DIRECT_GBPAGES y

With 81591.config :

With 42391.config :

With 55333.config :

With 99104.config :

With 43684.config :
-KMEMCHECK y
-KMEMCHECK_BITOPS_OK y
-KMEMCHECK_DISABLED_BY_DEFAULT n
-KMEMCHECK_ENABLED_BY_DEFAULT n
-KMEMCHECK_ONESHOT_BY_DEFAULT y
-KMEMCHECK_PARTIAL_OK n
-KMEMCHECK_QUEUE_SIZE 64
-KMEMCHECK_SHADOW_COPY_SHIFT 5
+DEBUG_PAGEALLOC n
+SLUB_DEBUG_ON n
+X86_DIRECT_GBPAGES y

With 72791.config :
-KMEMCHECK n

With 44115.config :

With 73441.config :

With 84612.config :

With 68862.config :
-KMEMCHECK y
-KMEMCHECK_BITOPS_OK y
-KMEMCHECK_DISABLED_BY_DEFAULT n
-KMEMCHECK_ENABLED_BY_DEFAULT n
-KMEMCHECK_ONESHOT_BY_DEFAULT y
-KMEMCHECK_PARTIAL_OK n
-KMEMCHECK_QUEUE_SIZE 64
-KMEMCHECK_SHADOW_COPY_SHIFT 5
+DEBUG_PAGEALLOC n
+X86_DIRECT_GBPAGES y

With 53035.config :

With 95624.config :
-KMEMCHECK n

With 50900.config :
-KMEMCHECK n

With 67378.config :
-KMEMCHECK y
-KMEMCHECK_BITOPS_OK y
-KMEMCHECK_DISABLED_BY_DEFAULT n
-KMEMCHECK_ENABLED_BY_DEFAULT n
-KMEMCHECK_ONESHOT_BY_DEFAULT y
-KMEMCHECK_PARTIAL_OK n
-KMEMCHECK_QUEUE_SIZE 64
-KMEMCHECK_SHADOW_COPY_SHIFT 5
+DEBUG_PAGEALLOC n
+SLUB_DEBUG_ON n
+X86_DIRECT_GBPAGES y

With 96062.config :
-KMEMCHECK y
-KMEMCHECK_BITOPS_OK y
-KMEMCHECK_DISABLED_BY_DEFAULT n
-KMEMCHECK_ENABLED_BY_DEFAULT n
-KMEMCHECK_ONESHOT_BY_DEFAULT y
-KMEMCHECK_PARTIAL_OK y
-KMEMCHECK_QUEUE_SIZE 64
-KMEMCHECK_SHADOW_COPY_SHIFT 5
+DEBUG_PAGEALLOC n
+SLUB_DEBUG_ON n
+X86_DIRECT_GBPAGES y

With 71224.config :
-KMEMCHECK y
-KMEMCHECK_BITOPS_OK y
-KMEMCHECK_DISABLED_BY_DEFAULT n
-KMEMCHECK_ENABLED_BY_DEFAULT n
-KMEMCHECK_ONESHOT_BY_DEFAULT y
-KMEMCHECK_PARTIAL_OK n
-KMEMCHECK_QUEUE_SIZE 64
-KMEMCHECK_SHADOW_COPY_SHIFT 5
+DEBUG_PAGEALLOC n
+X86_DIRECT_GBPAGES y

With 70567.config :

With 92557.config :

With 62687.config :
-KMEMCHECK y
-KMEMCHECK_BITOPS_OK y
-KMEMCHECK_DISABLED_BY_DEFAULT n
-KMEMCHECK_ENABLED_BY_DEFAULT n
-KMEMCHECK_ONESHOT_BY_DEFAULT y
-KMEMCHECK_PARTIAL_OK n
-KMEMCHECK_QUEUE_SIZE 64
-KMEMCHECK_SHADOW_COPY_SHIFT 5
+DEBUG_PAGEALLOC n
+X86_DIRECT_GBPAGES y

With 83842.config :

With 49677.config :

With 92246.config :
-KMEMCHECK n

With 53285.config :

With 34406.config :

With 73749.config :

With 79660.config :

With 101703.config :

With 54022.config :

With 93850.config :
-KMEMCHECK n

With 92478.config :
-KMEMCHECK n

With 105976.config :
-KMEMCHECK y
-KMEMCHECK_BITOPS_OK y
-KMEMCHECK_DISABLED_BY_DEFAULT n
-KMEMCHECK_ENABLED_BY_DEFAULT n
-KMEMCHECK_ONESHOT_BY_DEFAULT y
-KMEMCHECK_PARTIAL_OK n
-KMEMCHECK_QUEUE_SIZE 64
-KMEMCHECK_SHADOW_COPY_SHIFT 5
+DEBUG_PAGEALLOC n
+X86_DIRECT_GBPAGES y

With 48029.config :
-KMEMCHECK y
-KMEMCHECK_BITOPS_OK n
-KMEMCHECK_DISABLED_BY_DEFAULT n
-KMEMCHECK_ENABLED_BY_DEFAULT n
-KMEMCHECK_ONESHOT_BY_DEFAULT y
-KMEMCHECK_PARTIAL_OK y
-KMEMCHECK_QUEUE_SIZE 64
-KMEMCHECK_SHADOW_COPY_SHIFT 5
+DEBUG_PAGEALLOC n
+SLUB_DEBUG_ON n
+X86_DIRECT_GBPAGES y

With 56481.config :
-KMEMCHECK n

With 97877.config :
-KMEMCHECK n

With 62122.config :
-KMEMCHECK n

With 98599.config :

With 71741.config :

With 45607.config :
-KMEMCHECK y
-KMEMCHECK_BITOPS_OK y
-KMEMCHECK_DISABLED_BY_DEFAULT n
-KMEMCHECK_ENABLED_BY_DEFAULT n
-KMEMCHECK_ONESHOT_BY_DEFAULT y
-KMEMCHECK_PARTIAL_OK n
-KMEMCHECK_QUEUE_SIZE 64
-KMEMCHECK_SHADOW_COPY_SHIFT 5
+DEBUG_PAGEALLOC n
+X86_DIRECT_GBPAGES y

With 57131.config :
-KMEMCHECK n

With 44016.config :
-KMEMCHECK y
-KMEMCHECK_BITOPS_OK y
-KMEMCHECK_DISABLED_BY_DEFAULT n
-KMEMCHECK_ENABLED_BY_DEFAULT n
-KMEMCHECK_ONESHOT_BY_DEFAULT y
-KMEMCHECK_PARTIAL_OK y
-KMEMCHECK_QUEUE_SIZE 64
-KMEMCHECK_SHADOW_COPY_SHIFT 5
+DEBUG_PAGEALLOC n
+SLUB_DEBUG_ON n
+X86_DIRECT_GBPAGES y

With 57521.config :
-KMEMCHECK n

With 30576.config :

With 53306.config :
-KMEMCHECK y
-KMEMCHECK_BITOPS_OK n
-KMEMCHECK_DISABLED_BY_DEFAULT n
-KMEMCHECK_ENABLED_BY_DEFAULT n
-KMEMCHECK_ONESHOT_BY_DEFAULT y
-KMEMCHECK_PARTIAL_OK n
-KMEMCHECK_QUEUE_SIZE 64
-KMEMCHECK_SHADOW_COPY_SHIFT 5
+DEBUG_PAGEALLOC n
+X86_DIRECT_GBPAGES y

With 59549.config :
-KMEMCHECK n

With 112869.config :

With 32131.config :

With 55305.config :
-KMEMCHECK n

With 110840.config :

With 115630.config :
-KMEMCHECK y
-KMEMCHECK_BITOPS_OK y
-KMEMCHECK_DISABLED_BY_DEFAULT n
-KMEMCHECK_ENABLED_BY_DEFAULT n
-KMEMCHECK_ONESHOT_BY_DEFAULT y
-KMEMCHECK_PARTIAL_OK y
-KMEMCHECK_QUEUE_SIZE 64
-KMEMCHECK_SHADOW_COPY_SHIFT 5
+DEBUG_PAGEALLOC n
+SLUB_DEBUG_ON n
+X86_DIRECT_GBPAGES y

With 73234.config :
-KMEMCHECK y
-KMEMCHECK_BITOPS_OK y
-KMEMCHECK_DISABLED_BY_DEFAULT n
-KMEMCHECK_ENABLED_BY_DEFAULT n
-KMEMCHECK_ONESHOT_BY_DEFAULT y
-KMEMCHECK_PARTIAL_OK n
-KMEMCHECK_QUEUE_SIZE 64
-KMEMCHECK_SHADOW_COPY_SHIFT 5
+DEBUG_PAGEALLOC n
+X86_DIRECT_GBPAGES y

With 52542.config :

With 68416.config :

With 32297.config :
-KMEMCHECK y
-KMEMCHECK_BITOPS_OK y
-KMEMCHECK_DISABLED_BY_DEFAULT n
-KMEMCHECK_ENABLED_BY_DEFAULT n
-KMEMCHECK_ONESHOT_BY_DEFAULT y
-KMEMCHECK_PARTIAL_OK y
-KMEMCHECK_QUEUE_SIZE 64
-KMEMCHECK_SHADOW_COPY_SHIFT 5
+DEBUG_PAGEALLOC n
+X86_DIRECT_GBPAGES y

With 104080.config :
-KMEMCHECK y
-KMEMCHECK_BITOPS_OK y
-KMEMCHECK_DISABLED_BY_DEFAULT n
-KMEMCHECK_ENABLED_BY_DEFAULT n
-KMEMCHECK_ONESHOT_BY_DEFAULT y
-KMEMCHECK_PARTIAL_OK y
-KMEMCHECK_QUEUE_SIZE 64
-KMEMCHECK_SHADOW_COPY_SHIFT 5
+DEBUG_PAGEALLOC n
+SLUB_DEBUG_ON n
+X86_DIRECT_GBPAGES y

With 42060.config :
-KMEMCHECK y
-KMEMCHECK_BITOPS_OK y
-KMEMCHECK_DISABLED_BY_DEFAULT n
-KMEMCHECK_ENABLED_BY_DEFAULT n
-KMEMCHECK_ONESHOT_BY_DEFAULT y
-KMEMCHECK_PARTIAL_OK n
-KMEMCHECK_QUEUE_SIZE 64
-KMEMCHECK_SHADOW_COPY_SHIFT 5
+DEBUG_PAGEALLOC n
+X86_DIRECT_GBPAGES y

With 80652.config :
-KMEMCHECK y
-KMEMCHECK_BITOPS_OK n
-KMEMCHECK_DISABLED_BY_DEFAULT n
-KMEMCHECK_ENABLED_BY_DEFAULT n
-KMEMCHECK_ONESHOT_BY_DEFAULT y
-KMEMCHECK_PARTIAL_OK y
-KMEMCHECK_QUEUE_SIZE 64
-KMEMCHECK_SHADOW_COPY_SHIFT 5
+DEBUG_PAGEALLOC n
+X86_DIRECT_GBPAGES y

With 38821.config :
-KMEMCHECK n

With 60417.config :
-KMEMCHECK n

With 88988.config :
-KMEMCHECK n

With 107624.config :

With 66622.config :
-KMEMCHECK n

With 99413.config :

With 75796.config :
-KMEMCHECK n

With 47488.config :

With 114194.config :
-KMEMCHECK n

With 41985.config :
-KMEMCHECK n

With 103430.config :
-KMEMCHECK n

With 78480.config :

With 101222.config :

With 62700.config :

With 100711.config :
-KMEMCHECK y
-KMEMCHECK_BITOPS_OK y
-KMEMCHECK_DISABLED_BY_DEFAULT n
-KMEMCHECK_ENABLED_BY_DEFAULT n
-KMEMCHECK_ONESHOT_BY_DEFAULT y
-KMEMCHECK_PARTIAL_OK n
-KMEMCHECK_QUEUE_SIZE 64
-KMEMCHECK_SHADOW_COPY_SHIFT 5
+DEBUG_PAGEALLOC n
+SLUB_DEBUG_ON n
+X86_DIRECT_GBPAGES y

With 88747.config :

With 79695.config :
-KMEMCHECK n

With 89894.config :

With 82330.config :

With 30196.config :

With 32779.config :
-KMEMCHECK y
-KMEMCHECK_BITOPS_OK n
-KMEMCHECK_DISABLED_BY_DEFAULT n
-KMEMCHECK_ENABLED_BY_DEFAULT n
-KMEMCHECK_ONESHOT_BY_DEFAULT y
-KMEMCHECK_PARTIAL_OK y
-KMEMCHECK_QUEUE_SIZE 64
-KMEMCHECK_SHADOW_COPY_SHIFT 5
+DEBUG_PAGEALLOC n
+SLUB_DEBUG_ON n
+X86_DIRECT_GBPAGES y

With 82072.config :

With 91657.config :

With 39580.config :

With 59533.config :
-KMEMCHECK n

With 74557.config :

With 34827.config :

With 73129.config :

With 44188.config :

With 110107.config :
-KMEMCHECK n

With 77330.config :
-KMEMCHECK n

With 89691.config :

With 116166.config :
-KMEMCHECK y
-KMEMCHECK_BITOPS_OK y
-KMEMCHECK_DISABLED_BY_DEFAULT n
-KMEMCHECK_ENABLED_BY_DEFAULT n
-KMEMCHECK_ONESHOT_BY_DEFAULT y
-KMEMCHECK_PARTIAL_OK y
-KMEMCHECK_QUEUE_SIZE 64
-KMEMCHECK_SHADOW_COPY_SHIFT 5
+DEBUG_PAGEALLOC n
+SLUB_DEBUG_ON n
+X86_DIRECT_GBPAGES y

With 39673.config :
-KMEMCHECK y
-KMEMCHECK_BITOPS_OK n
-KMEMCHECK_DISABLED_BY_DEFAULT n
-KMEMCHECK_ENABLED_BY_DEFAULT n
-KMEMCHECK_ONESHOT_BY_DEFAULT y
-KMEMCHECK_PARTIAL_OK y
-KMEMCHECK_QUEUE_SIZE 64
-KMEMCHECK_SHADOW_COPY_SHIFT 5
+DEBUG_PAGEALLOC n
+SLUB_DEBUG_ON n
+X86_DIRECT_GBPAGES y

With 39512.config :

With 106202.config :

With 53468.config :

With 109554.config :

With 66741.config :

With 102931.config :
-KMEMCHECK y
-KMEMCHECK_BITOPS_OK n
-KMEMCHECK_DISABLED_BY_DEFAULT n
-KMEMCHECK_ENABLED_BY_DEFAULT n
-KMEMCHECK_ONESHOT_BY_DEFAULT y
-KMEMCHECK_PARTIAL_OK y
-KMEMCHECK_QUEUE_SIZE 64
-KMEMCHECK_SHADOW_COPY_SHIFT 5
+DEBUG_PAGEALLOC n
+X86_DIRECT_GBPAGES y

With 75881.config :

With 80267.config :

With 90512.config :
-KMEMCHECK n

With 98530.config :

With 74065.config :
-KMEMCHECK n

With 31576.config :

With 41759.config :

With 85918.config :
-KMEMCHECK y
-KMEMCHECK_BITOPS_OK n
-KMEMCHECK_DISABLED_BY_DEFAULT n
-KMEMCHECK_ENABLED_BY_DEFAULT n
-KMEMCHECK_ONESHOT_BY_DEFAULT y
-KMEMCHECK_PARTIAL_OK y
-KMEMCHECK_QUEUE_SIZE 64
-KMEMCHECK_SHADOW_COPY_SHIFT 5
+DEBUG_PAGEALLOC n
+SLUB_DEBUG_ON n
+X86_DIRECT_GBPAGES y

With 92564.config :
-KMEMCHECK y
-KMEMCHECK_BITOPS_OK n
-KMEMCHECK_DISABLED_BY_DEFAULT n
-KMEMCHECK_ENABLED_BY_DEFAULT n
-KMEMCHECK_ONESHOT_BY_DEFAULT y
-KMEMCHECK_PARTIAL_OK y
-KMEMCHECK_QUEUE_SIZE 64
-KMEMCHECK_SHADOW_COPY_SHIFT 5
+DEBUG_PAGEALLOC n
+X86_DIRECT_GBPAGES y

With 51007.config :
-KMEMCHECK y
-KMEMCHECK_BITOPS_OK y
-KMEMCHECK_DISABLED_BY_DEFAULT n
-KMEMCHECK_ENABLED_BY_DEFAULT n
-KMEMCHECK_ONESHOT_BY_DEFAULT y
-KMEMCHECK_PARTIAL_OK y
-KMEMCHECK_QUEUE_SIZE 64
-KMEMCHECK_SHADOW_COPY_SHIFT 5
+DEBUG_PAGEALLOC n
+SLUB_DEBUG_ON n
+X86_DIRECT_GBPAGES y

With 60249.config :

With 84342.config :
-KMEMCHECK n

With 106994.config :
-KMEMCHECK n

With 56096.config :
-KMEMCHECK n

With 97496.config :
-KMEMCHECK n

With 87079.config :

With 108210.config :
-KMEMCHECK n

With 38919.config :
-KMEMCHECK y
-KMEMCHECK_BITOPS_OK y
-KMEMCHECK_DISABLED_BY_DEFAULT n
-KMEMCHECK_ENABLED_BY_DEFAULT n
-KMEMCHECK_ONESHOT_BY_DEFAULT y
-KMEMCHECK_PARTIAL_OK n
-KMEMCHECK_QUEUE_SIZE 64
-KMEMCHECK_SHADOW_COPY_SHIFT 5
+DEBUG_PAGEALLOC n
+X86_DIRECT_GBPAGES y

With 50711.config :
-KMEMCHECK y
-KMEMCHECK_BITOPS_OK n
-KMEMCHECK_DISABLED_BY_DEFAULT n
-KMEMCHECK_ENABLED_BY_DEFAULT n
-KMEMCHECK_ONESHOT_BY_DEFAULT y
-KMEMCHECK_PARTIAL_OK y
-KMEMCHECK_QUEUE_SIZE 64
-KMEMCHECK_SHADOW_COPY_SHIFT 5
+DEBUG_PAGEALLOC n
+SLUB_DEBUG_ON n
+X86_DIRECT_GBPAGES y

With 99895.config :
-KMEMCHECK y
-KMEMCHECK_BITOPS_OK n
-KMEMCHECK_DISABLED_BY_DEFAULT n
-KMEMCHECK_ENABLED_BY_DEFAULT n
-KMEMCHECK_ONESHOT_BY_DEFAULT y
-KMEMCHECK_PARTIAL_OK n
-KMEMCHECK_QUEUE_SIZE 64
-KMEMCHECK_SHADOW_COPY_SHIFT 5
+DEBUG_PAGEALLOC n
+X86_DIRECT_GBPAGES y

With 42330.config :

With 107830.config :

With 38164.config :
-KMEMCHECK y
-KMEMCHECK_BITOPS_OK y
-KMEMCHECK_DISABLED_BY_DEFAULT n
-KMEMCHECK_ENABLED_BY_DEFAULT n
-KMEMCHECK_ONESHOT_BY_DEFAULT y
-KMEMCHECK_PARTIAL_OK n
-KMEMCHECK_QUEUE_SIZE 64
-KMEMCHECK_SHADOW_COPY_SHIFT 5
+DEBUG_PAGEALLOC n
+SLUB_DEBUG_ON n
+X86_DIRECT_GBPAGES y

With 68002.config :

With 107696.config :

With 72682.config :
-KMEMCHECK n

With 111392.config :
-KMEMCHECK n

With 41781.config :

With 101229.config :
-KMEMCHECK y
-KMEMCHECK_BITOPS_OK y
-KMEMCHECK_DISABLED_BY_DEFAULT n
-KMEMCHECK_ENABLED_BY_DEFAULT n
-KMEMCHECK_ONESHOT_BY_DEFAULT y
-KMEMCHECK_PARTIAL_OK n
-KMEMCHECK_QUEUE_SIZE 64
-KMEMCHECK_SHADOW_COPY_SHIFT 5
+DEBUG_PAGEALLOC n
+X86_DIRECT_GBPAGES y

With 82685.config :

With 106332.config :

With 60738.config :

With 68557.config :

With 90165.config :
-KMEMCHECK y
-KMEMCHECK_BITOPS_OK n
-KMEMCHECK_DISABLED_BY_DEFAULT n
-KMEMCHECK_ENABLED_BY_DEFAULT n
-KMEMCHECK_ONESHOT_BY_DEFAULT y
-KMEMCHECK_PARTIAL_OK n
-KMEMCHECK_QUEUE_SIZE 64
-KMEMCHECK_SHADOW_COPY_SHIFT 5
+DEBUG_PAGEALLOC n
+SLUB_DEBUG_ON n
+X86_DIRECT_GBPAGES y

With 96706.config :
-KMEMCHECK n

With 50835.config :
-KMEMCHECK n

With 66760.config :

With 100394.config :
-KMEMCHECK n

With 61904.config :

With 103562.config :

With 81684.config :

With 50736.config :
-KMEMCHECK y
-KMEMCHECK_BITOPS_OK y
-KMEMCHECK_DISABLED_BY_DEFAULT n
-KMEMCHECK_ENABLED_BY_DEFAULT n
-KMEMCHECK_ONESHOT_BY_DEFAULT y
-KMEMCHECK_PARTIAL_OK n
-KMEMCHECK_QUEUE_SIZE 64
-KMEMCHECK_SHADOW_COPY_SHIFT 5
+DEBUG_PAGEALLOC n
+SLUB_DEBUG_ON n
+X86_DIRECT_GBPAGES y

With 100845.config :

With 86132.config :

With 43676.config :

With 53472.config :

With 96100.config :
-KMEMCHECK y
-KMEMCHECK_BITOPS_OK n
-KMEMCHECK_DISABLED_BY_DEFAULT n
-KMEMCHECK_ENABLED_BY_DEFAULT n
-KMEMCHECK_ONESHOT_BY_DEFAULT y
-KMEMCHECK_PARTIAL_OK y
-KMEMCHECK_QUEUE_SIZE 64
-KMEMCHECK_SHADOW_COPY_SHIFT 5
+DEBUG_PAGEALLOC n
+SLUB_DEBUG_ON n
+X86_DIRECT_GBPAGES y

With 59380.config :
-KMEMCHECK n

With 34875.config :
-KMEMCHECK n

With 42348.config :
-KMEMCHECK n

With 95934.config :
-KMEMCHECK n

With 57592.config :

With 88593.config :
-KMEMCHECK y
-KMEMCHECK_BITOPS_OK y
-KMEMCHECK_DISABLED_BY_DEFAULT n
-KMEMCHECK_ENABLED_BY_DEFAULT n
-KMEMCHECK_ONESHOT_BY_DEFAULT y
-KMEMCHECK_PARTIAL_OK n
-KMEMCHECK_QUEUE_SIZE 64
-KMEMCHECK_SHADOW_COPY_SHIFT 5
+DEBUG_PAGEALLOC n
+SLUB_DEBUG_ON n
+X86_DIRECT_GBPAGES y

With 89478.config :

With 35978.config :
-KMEMCHECK y
-KMEMCHECK_BITOPS_OK n
-KMEMCHECK_DISABLED_BY_DEFAULT n
-KMEMCHECK_ENABLED_BY_DEFAULT n
-KMEMCHECK_ONESHOT_BY_DEFAULT y
-KMEMCHECK_PARTIAL_OK n
-KMEMCHECK_QUEUE_SIZE 64
-KMEMCHECK_SHADOW_COPY_SHIFT 5
+DEBUG_PAGEALLOC n
+SLUB_DEBUG_ON n
+X86_DIRECT_GBPAGES y

With 48339.config :

With 82988.config :
-KMEMCHECK n

With 57761.config :
-KMEMCHECK n

With 105051.config :

With 89591.config :

With 58855.config :

With 33749.config :
-KMEMCHECK n

With 108762.config :
-KMEMCHECK y
-KMEMCHECK_BITOPS_OK n
-KMEMCHECK_DISABLED_BY_DEFAULT n
-KMEMCHECK_ENABLED_BY_DEFAULT n
-KMEMCHECK_ONESHOT_BY_DEFAULT y
-KMEMCHECK_PARTIAL_OK n
-KMEMCHECK_QUEUE_SIZE 64
-KMEMCHECK_SHADOW_COPY_SHIFT 5
+DEBUG_PAGEALLOC n
+X86_DIRECT_GBPAGES y

With 116641.config :

With 70409.config :

With 53199.config :
-KMEMCHECK y
-KMEMCHECK_BITOPS_OK y
-KMEMCHECK_DISABLED_BY_DEFAULT n
-KMEMCHECK_ENABLED_BY_DEFAULT n
-KMEMCHECK_ONESHOT_BY_DEFAULT y
-KMEMCHECK_PARTIAL_OK y
-KMEMCHECK_QUEUE_SIZE 64
-KMEMCHECK_SHADOW_COPY_SHIFT 5
+DEBUG_PAGEALLOC n
+SLUB_DEBUG_ON n
+X86_DIRECT_GBPAGES y

With 43079.config :
-KMEMCHECK y
-KMEMCHECK_BITOPS_OK n
-KMEMCHECK_DISABLED_BY_DEFAULT n
-KMEMCHECK_ENABLED_BY_DEFAULT n
-KMEMCHECK_ONESHOT_BY_DEFAULT y
-KMEMCHECK_PARTIAL_OK n
-KMEMCHECK_QUEUE_SIZE 64
-KMEMCHECK_SHADOW_COPY_SHIFT 5
+DEBUG_PAGEALLOC n
+X86_DIRECT_GBPAGES y

With 77763.config :
-KMEMCHECK y
-KMEMCHECK_BITOPS_OK y
-KMEMCHECK_DISABLED_BY_DEFAULT n
-KMEMCHECK_ENABLED_BY_DEFAULT n
-KMEMCHECK_ONESHOT_BY_DEFAULT y
-KMEMCHECK_PARTIAL_OK n
-KMEMCHECK_QUEUE_SIZE 64
-KMEMCHECK_SHADOW_COPY_SHIFT 5
+DEBUG_PAGEALLOC n
+SLUB_DEBUG_ON n
+X86_DIRECT_GBPAGES y

With 104679.config :
-KMEMCHECK y
-KMEMCHECK_BITOPS_OK n
-KMEMCHECK_DISABLED_BY_DEFAULT n
-KMEMCHECK_ENABLED_BY_DEFAULT n
-KMEMCHECK_ONESHOT_BY_DEFAULT y
-KMEMCHECK_PARTIAL_OK y
-KMEMCHECK_QUEUE_SIZE 64
-KMEMCHECK_SHADOW_COPY_SHIFT 5
+DEBUG_PAGEALLOC n
+SLUB_DEBUG_ON n
+X86_DIRECT_GBPAGES y

edit:
(cascading effects: https://cateee.net/lkddb/web-lkddb/X86_DIRECT_GBPAGES.html
https://cateee.net/lkddb/web-lkddb/DEBUG_PAGEALLOC.html)

[FAMILIAR-project]

I propose to follow a similar methodology for:

• KASAN_INLINE (be careful you should deactivated KASAN_OUTLINE) https://github.com/torvalds/linux/blob/v4.13/lib/Kconfig.kasan#L23-L46
• SLOB
• OPTIMIZE_INLINING
• MATH_EMULATION
• CC_STACKPROTECTOR_STRONG
• CC_STACKPROTECTOR_REGULAR
• LGUEST_GUEST

by similar methodology I mean:

• reuse of procedures to compute existing configurations with this new option
• "diff" about the mutation
• data/CIDs
• notebook reporting on results / summarizing key result

[psaffray]

First I tested option 'OPTIMIZE_INLINING' because it had no dependency.
Here is the diffconfig logs (empty for all, which means 'make olddefconfig' did not change anything after having changed 'CONFIG_OPTIMIZE_INLINING' from 0 to 1, so the sample is valid)
logs : diffs.txt
Results for compilation after switching the value have been successfully produced then.

[psaffray]

Concerning KASAN_INLINE :
when I try to get the .config that have this option set to 0 I get an error

df = tuxml.load_dataset()
# 0 value is 'n' value, 1 is 'y' value, 2 is 'm' value
test = df.query("KASAN_INLINE == 0")['cid'].sample(n=200, random_state=1)

UndefinedVariableError: name 'KASAN_INLINE' is not defined
It appears to me that there are no configuration where KASAN_INLINE is ever set.
So I arranged otherwise to select configurations where neither INLINE nor OUTLINE where active, but when I run 'make olddefconfig' after enabling KASAN_INLINE, it just simply removes it.
-KASAN_INLINE y
-KASAN_OUTLINE n
Finally I tried to select configurations with KASAN_INLINE=0 and KASAN_OUTLINE=1, and that worked well !
By switching these two options, the 'make olddefconfig' does not modify anything so I will be able to launch the compilation part now.

[psaffray]

Concerning SLOB :
I selected config files where (CONFIG_SLOB == 0 and CONFIG_EXPERT == 1)
because SLOB depends on EXPERT.
But when enabling SLOB, 'make olddefconfig' systematically changes other options.
For example at least these three lines seem to appear everytime

-HAVE_ALIGNED_STRUCT_PAGE y
-HAVE_HARDENED_USERCOPY_ALLOCATOR y
 SLUB y -> n

logs : diffs.txt

[psaffray]

Concerning MATH_EMULATION :
Depends on ( CONFIG_MODIFY_LDT_SYSCALL ) && ( CONFIG_X86_32 )
Firstly, I haven't found any configuration with this option present (enabled or not) in the dataset
I am unable to find a way so that the 'make olddefconfig' does not remove my changings.
Whatever configuration I take, when I enable MATH_EMULATION, I also enable the dependancies for the case they are not already set. 'make olddefconfig' automatically removes the MATH_EMULATION option line...
I would guess that it is impossible because we are using X86_64 and it is not compatible?

[psaffray]

Concerning CC_STACKPROTECTOR_STRONG :
By changing the option from 0 to 1, we also need to change others :

From:

# CONFIG_CC_STACKPROTECTOR is not set
CONFIG_CC_STACKPROTECTOR_NONE=y
# CONFIG_CC_STACKPROTECTOR_STRONG is not set

To:

CONFIG_CC_STACKPROTECTOR=y
# CONFIG_CC_STACKPROTECTOR_NONE is not set
CONFIG_CC_STACKPROTECTOR_STRONG=y

logs : diffs.txt

For CC_STACKPROTECTOR_REGULAR, it is the same :

From:

# CONFIG_CC_STACKPROTECTOR is not set
CONFIG_CC_STACKPROTECTOR_NONE=y
# CONFIG_CC_STACKPROTECTOR_REGULAR is not set

To:

CONFIG_CC_STACKPROTECTOR=y
# CONFIG_CC_STACKPROTECTOR_NONE is not set
CONFIG_CC_STACKPROTECTOR_REGULAR=y

I reused the same .config files but enabled REGULAR instead of STRONG

[psaffray]

You can check the results here https://github.com/TuxML/misc/blob/master/analysis/merge_dataframes.ipynb