Bidding: how to test symmetry / implement symmetry for board rotation (so the the model is independent on the cardinal direction)?

[ThePokerDude]

I'm not yet familiar with the code so maybe bidding symmetry is already implemented.

Let's say the training set of boards is unbalanced - so for example N has on average 11 HCP.
In this case I would expect that bidding would change if we rotate the board even if dealer is also rotated.

Let's take board 1 of the bidding data set

K76.965.Q63.9854 AT83.AJ.J9754.73 Q42.84.T2.AKQT62 J95.KQT732.AK8.J
N None P P 1C 1H P 1N P 2N P 3D P 3H P 4H P P P

Let's assume that BEN would bid identically.

What about these 3 other options:

J95.KQT732.AK8.J K76.965.Q63.9854 AT83.AJ.J9754.73 Q42.84.T2.AKQT62
E None P P 1C 1H P 1N P 2N P 3D P 3H P 4H P P P

Q42.84.T2.AKQT62 J95.KQT732.AK8.J K76.965.Q63.9854 AT83.AJ.J9754.73
S None P P 1C 1H P 1N P 2N P 3D P 3H P 4H P P P

AT83.AJ.J9754.73 Q42.84.T2.AKQT62 J95.KQT732.AK8.J K76.965.Q63.9854
W None P P 1C 1H P 1N P 2N P 3D P 3H P 4H P P P

Are we sure that bidding will be identical for all 4 options?
How could we test it for some boards?

If it turns out that NN is assmmetrical - should we just create 3 additional copies of each board (which seems simple but ineffective)
or is there a clever way?

In another github project - KataGo (best open source NN Go engine) - this question also arose.
The answer of the main developer was:

Training already takes into account the symmetry of the board, and there isn't much further improvement to be done. Every time any piece of data is seen, it is randomly mirrored 8 possible ways when training the neural net, which massively improves neural net's training stability.

lightvector/KataGo#333

[ThorvaldAagaard]

In the original BEN, there was no handling of symmetry, and Lorand trusted that enough samples would cover all situations. In the current implementation I have added symmetry, so North and South are following the same rules, and so for East and West. As NS and EW might play different systems I have decided not to have symmetry between the two sides.

So originally
J95.KQT732.AK8.J K76.965.Q63.9854 AT83.AJ.J9754.73 Q42.84.T2.AKQT62
E None P P 1C 1H P 1N P 2N P 3D P 3H P 4H P P P

AT83.AJ.J9754.73 Q42.84.T2.AKQT62 J95.KQT732.AK8.J K76.965.Q63.9854
W None P P 1C 1H P 1N P 2N P 3D P 3H P 4H P P P

This would in the input become 2 different training input as there on the second would be two more PAS_START in the bidding sequence, and the training would see it as different.

If you look at the training script bidding_coverage.py you will see a parameter sameforboth, signaling to the code, that I want same training for North and South.

In the same script I am also reducing overrepresented samples (The exact same sequence can only occur 10 times) but I haven't dah time to find if 10 is the right number - training is taking long time

And in the same script I am also repeating sequences, where the vulnerability is missing, ie I have a sequence where both are vulnerable, but the same sequences does not exist when both are non-vulnerable.

Finally the training has two input dataset, one where the deals are selected based on common situations - typical a convention. That is about 600.000 deals and another with 5 mill totally random. The first is the primary dataset, and the second is then used to fill the gaps. Here I only select max 2 of the same bidding sequence. Again a guess what the best value is for that number.

So yes, I think for now that we have the symmetry needed

[ThePokerDude]

Thanks, I see. That seems very reasonable.
I dowloaded ben-0.2.0-alpha and in that version it was not yet included.
I'm a programming beginner and have little experience with github so I did not know how to download the latest version -
but I'll figure out :).
But I'm experienced with SQL, reading log files and code configurations and I'm playing bridge for about 25 years (also chess and go), so I hope I can contribute.

I always wanted to learn programming and this project is really fascinating and motivating so I want to go for it :).

[ThorvaldAagaard]

ChatGPT can help you get the latest version :-)
Drop me a mail at thorvald@netbridge.dk and I will send you an invite to our discord-server