Volume 2, Issue 2 Empirical Active Information 47 Empirical Active Information Eric Holloway DOI: 10.33014/issn.2640-5652.2.2.holloway.4 Active information is the log ratio of the performance of a search with (!) and without (") information about the tar- get it is searching for, using the notation of self information # ($) = ! log2 $, #+ = log2 ! " (1) = # (") ! # (!). (2) As proven in Bartlett (2020), active information is a canoni- cal specified complexity, and therefore by Montañez (2018), Pr(#+ > %) " 2!! . (3) This is posed in an ideal setting where we know both of these performance values. However, in the empirical setting, although we can per- haps take the baseline as a given value, we must infer the performance of the search with target information by obser- vation. In this case, we will only consider a search across binary strings. All finite domains can be cast as binary strings, so the proof here is general enough for all finite domains, although perhaps not always the most convenient representation. We’ve observed & bits and of those ' are zeros. We can think of this in terms of a prediction task, where ones cor- respond to correct predictions and zeros correspond to in- correct predictions. For sake of demonstration we set ' " &/2, (4) which we can always guarantee by flipping the prediction if more than half are incorrect. From the assumption in (4) we define the probabilities $ = '/& (5) ( = 1 ! '/& (6) such that $ " (. We want to figure out what kind of prediction accuracy we can expect on unseen data, and we want to err on the side of caution and not be over optimistic. Stated more formally, what is the Bernoulli distribution probability ) of ones that generated this sequence, based on a certain confidence probability of *? First, here is an upper bound on the binomial quotient tail derived from the binomial cumulative distribution: "! #=0 " & + # $$%(&% " "! #=0 " & + # $#(%!# " 1 (7) "! #=0 " & + # " $!$%(!&% (8) = 2%' ($) . (9) The left hand side of (7) is derived by keeping the proba- bility constant during the summation, instead of changing as + is incremented. This guarantees the left hand side is a lower bound on the binomial cumulative distribution, since we’ve selected the smallest probability in the summation and kept it constant. The jump from $!$%(!&% in (8) to 2%' ($) in (9) is based on the entropy function: , ($) = !$ log2 $ ! ( log2 ( (10) 2' ($) = $!$(!& . (11) Now, let us return to our goal of estimating the accuracy on unseen data. The estimate can be wrong in two ways. 1. Too low. If so, then great! 2. Too high. If so, we want to know the probability the estimate is too high. So, stated more formally, we want to pick an ) for our estimate such that any Bernoulli distribution with a smaller probability " of generating a one, i.e. 0.5 " " < ) , will have a probability less than * of generating ' or fewer zeros. In other words, * is the probability we’ve over estimated our model accuracy. The probability * is thus: * = "! #=0 " & + # (1 ! ))#)%!# " "! #=0 " & + # )% (12) " 2%' ($))% (13) log2 * " &(, ($) + log2 )). (14) The substitution to get the right hand side in (12) is the same sort of reasoning used to get the left hand side in (7), except reversed. In (7) the goal was to get a lower bound https://dx.doi.org/10.33014/issn.2640-5652.2.2.holloway.4 48 Letters and Notes on the cumulative distribution. In (12) the goal is to get an upper bound. So, we pick the largest probability in the summation, )%, which is larger than (1 ! ))#)%!# because ) # 0.5. The reduction from (12) to (13) is based on the substitu- tion from (9). The use of the logarithm in (14) measures the probability in bits to make the formula a bit easier to manipulate. Now, we want to derive ) from a predefined value for *, for example * = 2!(. This means we need to pick a value for ) such that the right hand side of (14) evaluates to !-. If we substitute in ) = 2!' ($)!(/%, (15) we achieve just that result, log2 * " &(, ($) + log2 (2!' ($)!(/%)) (16) = &(, ($) ! , ($) ! -/&) = !-. (17) As long as &, ($)+- < & then from (15) ) > 0.5, and we are able to predict better than 50/50 odds. Here we can also see why our very first assumption in (4) is necessary. If ' > &/2 then since the entropy function , ($) is symmetrical we can substitute in a value for $ such that (15) gives us a estimate that is higher than the observed accuracy rate, i.e. ( < ) , which is a contradiction since the estimate is a lower bound and so should be lower than the observed accuracy. Now we can finalize our concept of empirical active informa- tion per sample by substituting in ! = ) for (1) and setting - = 0, #)+ = # (") ! , ($). (18) And as at the beginning, the probability bound still applies. Pr(#)+ > %) " 2!! . (19) Bartlett, Jonathan (2020). “Active Information is a Spec- ified Complexity Model”. In: Communications of the Blyth Institute 2.2. Montañez, George D (2018). “A Unified Model of Complex Specified Information”. In: BIO-Complexity 2018. About This Journal The Purpose of the Journal Paper Submission Policies Other Journal Content From the Editors Annie CrawfordAnnie CrawfordMetaphor and Meaning in the Teleological Language of Biology Introduction The History of Teleology in the Biological Sciences Teleological language is Essential to Biology If teleological language is essential to biology, then life must be teleological Conclusion: Life All the Way Down Salvador CordovaSalvador CordovaFisher's Fundamental Theorem of Natural Selection Isn't Fundamental After All Introduction: The Problem of Defining Fitness Absolute ``Darwinian'' Fitness vs. Relative Fitness Mean and Variance of Relative Fitnesses Numerical Examples to Illustrate Fisher's Theorem Discussion Conclusion Eric HollowayEric HollowayTutorial: Bioinformatics Basics Introduction Genetic Code and Sequence Translation Sequencing and Assembly Accessing Data Finding Things With BLAST Summary Letters and Notes Eric HollowayYou Cannot Get Meaning From Randomness Jonathan BartlettActive Information is a Specified Complexity Model Eric HollowayEvolution in the Valley of Illusions Jonathan BartlettIs Active Information Applicable to Biology? Eric HollowayEmpirical Active Information News