I was working with my friend on a Random Number Generator (RNG) research paper. We need to evaluate the quality of our random bitstream result using the NIST SP800-90B entropy test.
Since this is my first time working in the RNG field, I tried to gather all information about evaluating the quality of an RNG. The SP800-90B standard is the starting point. I tried to read it again and again since there is so much jargon, one of them that took me a long time to understand is the Independent and Identically Distributed (IID). I got the concept right but each article seems to explain different complicated cases and examples which made me confuse even more. Eventually, I end up with this explanation:
Independent is when a value is not affected by the other value.
For example, if you roll two dice, the result does not depend on each other.Identically Distributed is when the probability of any specific outcome is the same.
For example, if you flip a coin, you had 50 50 chances of getting head or tail. That value doesn’t change every time you flip a coin. On the other hand, if you have a collection of weighted coins, where each coin had a different probability of head or tail, that would be not identically distributed.In machine learning, IID often implies that all of the data from the training set comes from the same process and that data is not related to each other.
Terms: Independent and Identically Distributed (IID) by IntuitiveML (https://www.youtube.com/watch?v=EGKbPww2_rc)
Now I can sleep well.