On statistical tests for randomness included in the NIST SP800-22 test suite and based on the binomial distribution

In this paper we review some statistical tests included in the NIST SP 800-22 suite, which is a collection of tests for the evaluation of both true-random (physical) and pseudorandom (algorithmic) number generators for cryptographic applications. The output of these tests is the so-called p-value which is a random variable whose distribution converges to the uniform distribution in the interval [0,1] when testing an increasing number of samples from an ideal generator. Here, we compute the exact non-asymptotic distribution of p-values produced by few of the tests in the suite, and propose some computation-friendly approximations. This allows us to explain why intensive testing produces false-positives with a probability much higher than the expected one when considering asymptotic distribution instead of the true one. We also propose a new approximation for the Spectral Test reference distribution, which is more coherent with experimental results.