Thread: 3D dice results are not random
-
November 29th, 2020, 00:46 #1
- Join Date
- Aug 2019
- Posts
- 2,025
3D dice results are not random
After comparing FGU's "physical" D8 dice results with FGU's own "non-physical" /die command I concluded that FGU's physical dice engine seems to be bugged and does not create random numbers. FGU's non-physical /die pseudo-random number generator (PRNG) has its own flaws, but it is mostly functional compared to the current 3D implementation (judging by D8 rolls).
Originally Posted by Moon Wizard
Originally Posted by ENT "Pseudorandom Number Sequence Test Program"Code:Chi square distribution for 375000 samples is 13393.71, and randomly would exceed this value less than 0.01 percent of the times.
Originally Posted by PractrandCode:length= 8 kilobytes (2^13 bytes), time= 0.2 seconds Test Name Raw Processed Evaluation BCFN(2+0,13-9,T) R= +29.3 p = 7.5e-8 very suspicious DC6-9x1Bytes-1 R= +30.1 p = 2.9e-13 FAIL ...and 23 test result(s) without anomalies
BCFN failures where the first parameter listed is low are typically similar in meaning to a failure on a DC6 parameterization.Code:length= 8 kilobytes (2^13 bytes), time= 1.3 seconds Test Name Raw Processed Evaluation BCFN(0+0,13-9,T) R= +65.6 p = 2.7e-16 FAIL ! DC6-9x1Bytes-1 R= +30.1 p = 2.9e-13 FAIL DC6-6x2Bytes-1 R= +27.4 p = 1.9e-12 FAIL DC6-5x4Bytes-1 R= +34.7 p = 3.4e-15 FAIL !
Failures on FPF parameterizations can have similar meaning.Code:length= 32 kilobytes (2^15 bytes), time= 0.3 seconds Test Name Raw Processed Evaluation BCFN(0+0,13-9,T) R=+244.9 p = 3.8e-56 FAIL !!!! BCFN(0+1,13-9,T) R=+271.2 p = 4.6e-62 FAIL !!!! BCFN(0+2,13-9,T) R=+143.0 p = 2.6e-34 FAIL !!! DC6-9x1Bytes-1 R= +75.6 p = 1.5e-37 FAIL !!! DC6-6x2Bytes-1 R= +99.0 p = 5.4e-41 FAIL !!! DC6-5x4Bytes-1 R=+109.0 p = 1.1e-57 FAIL !!!! FPF-14+6/32:(0,14-8) R= +14.1 p = 3.4e-10 very suspicious FPF-14+6/32:(1,14-9) R= +11.2 p = 2.6e-7 unusual FPF-14+6/32:all R= +12.2 p = 1.2e-7 very suspicious FPF-14+6/16:(0,14-8) R= +22.4 p = 4.0e-16 FAIL FPF-14+6/16:(1,14-8) R= +22.3 p = 4.1e-16 FAIL FPF-14+6/16:(2,14-9) R= +14.0 p = 3.9e-9 suspicious FPF-14+6/16:all R= +28.0 p = 2.5e-18 FAIL ! FPF-14+6/16:cross R= +7.3 p = 1.6e-5 mildly suspicious FPF-14+6/4:(0,14-6) R= +47.5 p = 2.2e-36 FAIL !!! FPF-14+6/4:(1,14-7) R= +29.8 p = 1.6e-23 FAIL !! FPF-14+6/4:(2,14-8) R= +38.1 p = 1.9e-27 FAIL !! FPF-14+6/4:(3,14-8) R= +12.1 p = 9.6e-9 suspicious FPF-14+6/4:all R= +63.7 p = 3.9e-50 FAIL !!!! FPF-14+6/4:cross R= +28.6 p = 1.1e-20 FAIL !!
Originally Posted by RaBiGeTe
https://www.fourmilab.ch/random/
https://pracrand.sourceforge.net/
https://cristianopi.altervista.org/RaBiGeTe_MT/
https://en.wikipedia.org/wiki/Randomness_testsLast edited by Weissrolf; November 29th, 2020 at 10:04.
-
November 29th, 2020, 00:56 #2
- Join Date
- Aug 2019
- Posts
- 2,025
Let me clarify this up-front: Please stay on topic of the PRNG engine. Don't be defensive, but open to discussion. I have no interest in seeing FGU fail at such a fundamental mechanic, but want to see it fixed and improved. Thanks in advance!
-
November 29th, 2020, 01:18 #3
Ah, interesting
But I think some explanation of the data is needed, for example, what is shown in the last graphic? The axes are not labelled etc. A bit difficult to follow without explanation what is calculatedMy extensions for 3.5e and Pathfinder
Bug reports please here
-
November 29th, 2020, 01:23 #4
I also don't know how to read that.
From your 200 d8 examples, I would expect something like 25 of each result. Does your sample data show a range of 24-26 results for each value, or is range more like 5-50?
Problems? See; How to Report Issues, Bugs & Problems
On Licensing & Distributing Community Content
Community Contributions: Gemstones, 5E Quick Ref Decal, Adventure Module Creation, Dungeon Trinkets, Balance Disturbed, Dungeon Room Descriptions
Note, I am not a SmiteWorks employee or representative, I'm just a user like you.
-
November 29th, 2020, 01:26 #5
- Join Date
- Aug 2019
- Posts
- 2,025
I quoted the explanations, but here are the links to the corresponding web-sites. For the graph I added the labeling of the axes in the quote now, but all we really need to know here is that the lines are not even broadly close to each other and that FGU's /die tests considerably better. It's just another visualization of how very badly FGU's physical engine fails.
A simpler and better visualization is in the first image that demonstrates how FGU's physical D8 ebb and flow within only 200 rolls. That's why I put it first and the other one last.
https://www.fourmilab.ch/random/
https://pracrand.sourceforge.net/
https://cristianopi.altervista.org/RaBiGeTe_MT/
https://en.wikipedia.org/wiki/Randomness_testsLast edited by Weissrolf; November 29th, 2020 at 01:44.
-
November 29th, 2020, 01:34 #6
- Join Date
- Aug 2019
- Posts
- 2,025
If it says "Fail" in too many tests then it's not random. Especially Practrand usually needs a huuuuge amount of data (like terabytes) to falsify randomness of good RNGs, but FGU's D8 data is so problematic that is manages with just a few kilobytes. FGU's /die data also fails, but later and at a different test (pointing to different problems).
From your 200 d8 examples, I would expect something like 25 of each result. Does your sample data show a range of 24-26 results for each value, or is range more like 5-50?
What I did was to create binary data out of the dice rolls where each byte consist of numbers 0-255. Since FGU's physical dice cannot roll D256(-1) I rolled D8(-1) instead, stripped the five 00000 high bits and formed one long bitstream of 375000 bytes (=1 mio rolls of D8). Each D8's binary representation is between 000 to 111, 2.67 D8 make one byte between 00000000 to 11111111. For comparison I did the very same with random.org rolls of 0-7.
The first image is a bitmap where each binary 0 is black and each binary 1 is white. I found out that there is a period of only 600 pixels/bits where you see the ebb and flow of dark and light areas repeat in the bitmap image. 600 bits = 75 bytes x 2.67 = 200 rolls of D8. The image does not tell us which results were rolled, but it tells us that there are periodic ups and downs of more 1 bits vs. more 0 bits in the stream of consecutive rolls.
Again, not everyone has to understand what this all means, all you need to see is that both PCG and random.org look like random noise (the kind of static noise you may remember from old TVs), but FGU's D8 rolls show a clear periodic pattern.Last edited by Weissrolf; November 29th, 2020 at 01:49.
-
November 29th, 2020, 01:48 #7
I mean it would be nice when just every technical term is shortly introduced with its definition, to avoid that one needs to go through the links to understand that So, basically just an introduction of what you did and what the values represent, and how to read that, not just listing the result like Fail or Success That would certainly improve readability for people who are not experienced with all these technical terms, basically it would be good when the reader can interpret the result on one's own (now one simply needs to trust the Fail message). I understand though when that's annoying to do, because you did it in your free time etc.
My extensions for 3.5e and Pathfinder
Bug reports please here
-
November 29th, 2020, 01:58 #8
- Join Date
- Aug 2019
- Posts
- 2,025
It's rather impossible to do. We are not scientists here who even understand what this all means beyond fail and success. There just is no way to give a "short introduction". These are well established randomness tests and all we can do is trust their results, just like scientists around the world trust them and test them. Everyone wanting to go deeper needs to read through manuals and web-sites of scientists/coders who create and test random number generators (and even their disputes in papers and forums), and possibly study mathematics.
The first bitmap image is the most down-to-earth representation of FGU's PRNG failure I can give you. It is pure visualization of the numbers FGU rolled as black/white dots and should display random noise. Instead it displays visibly repeated patterns. That must not happen!
I did far more work on this than I should have done, but this represents a real number-crunching examination of FGU's physical dice results, free from any "I rolled too many crits today" bias. Not psychology, but mathematics. Does it go over my own head? Sure, but that's why Practrand tries to categorize it in readable fails and suspicions for dummies like us.Last edited by Weissrolf; November 29th, 2020 at 02:03.
-
November 29th, 2020, 02:00 #9
- Join Date
- Aug 2019
- Posts
- 2,025
And again, what really matters is the comparison to better (FGU /die) and proper (PCG, random.org) working RNGs. Even if we don't understand a thing about all the results we can easily see that one fails hard where others succeed.
I specifically did not include specific FGU /die results in the discussion yet, because physical 3D dice fails so much worse that we need to concentrate on them first.
-
November 29th, 2020, 02:25 #10
Last edited by smelton; November 29th, 2020 at 02:29.
Thread Information
Users Browsing this Thread
There are currently 1 users browsing this thread. (0 members and 1 guests)
Bookmarks