One-sided coin — Dr. Işık Barış Fidaner

“It is the disk of Odin,” the old man said in a patient voice, as though he were speaking to a child. “It has but one side. There is not another thing on earth that has but one side. So long as I hold it in my hand I shall be king.” (The Disk by Jorge Luis Borges)

Dr. Işık Barış Fidaner

yield1
Source: Decision Making Coin

For a regular coin, a toss always yields an outcome, which is either Heads or Tails. Let’s say that the coin yields Heads with probability p and Tails with probability 1-p.

The technical term “probability” implies that the two outcomes Heads & Tails are mutually exclusive and their “probabilities” sum up to 1.

Given these two values, the total entropy is -p*log(p)-(1-p)*log(1-p). This entropy takes its limit values at three values of p:

1) The coin always yields Heads: p = 1, entropy = 0.

2) The coin yields Heads with p = 1/2 (and yields Tails with 1/2): entropy = log(2), which is the maximum value.

3) The coin never yields Heads (p = 0) so it always yields Tails: entropy = 0.

For the other values, we get the intermediate entropies:

yield0

Two dashed lines show the first term and the second term in the entropy formula.

The convenience of this formulation is that when the logarithm is computed with base 2, log(2) becomes 1 and it’s interpreted as “1 bit” of information produced by the coin toss.

In accordance with the laws of probability, a regular coin may yield two mutually exclusive outcomes with total probability 1. Now we’ll describe a one-sided coin that may only yield a single outcome.

The physical shape of a coin ensures that one of the surfaces will be visible to determine the outcome. However, an outcome is ultimately determined as a symbol in the tosser’s language. If the symbol on the surface does not exist in the tosser’s language (e.g. “Tails” is unknown), or if there is no symbol on the surface and there is no “zero symbol” (for the absence of a symbol) in the tosser’s language, then the coin toss may not yield an outcome for that particular surface.

yield2
In a one-sided coin, the Tails symbol is erased (either from the coin or from the tosser’s language or from both). This new coin either yields Heads (with potentiality p) or it does not yield an outcome at all. (Since the laws of probability does not apply, it is inappropriate to call it a “probability”, so we call it a “potentiality”.)

When the second surface is not expressible in the tosser’s language as a symbolic outcome, its information is zero by definition. For such a coin, total entropy is -p*log(p), which takes its limit values at three values of p:

1) The coin always yields Heads: p = 1, entropy = 0.

2) The coin yields Heads with p = 1/e (and doesn’t yield an outcome with 1 – 1/e): entropy = 1/e, which is the maximum value.

3) The coin never yields Heads (p = 0) so it never yields an outcome: entropy = 0.

For the general case p = 1/x, entropy is log(x)/x:

yield0b

Let’s compare the two coins:

— In the regular coin, there are two symbols, and two information terms in the entropy formula. In the one-sided coin, there is only one symbol, and only one information term in the entropy formula.

— In the regular coin, maximum entropy is log(2). In the one-sided coin, maximum entropy is 1/e.

— In the regular coin, maximum entropy is attained at p = 1/2 (50%). In the one-sided coin, maximum entropy is attained at p = 1/e (~37%).

In other words, when we’re able to observe only a single symbol, it’s more surprising for us if that symbol occurs less than half (precisely 1/e = ~37%) of the time.

Note that the base of the logarithm does not change these numbers, even with log base 2, the number 1/e persists for the one-sided coin.

A regular coin can be simulated by two one-sided coins (with potentialities p and q) simply by asserting the condition p + q = 1. The outcomes won’t form a regular sequence of symbols as with the regular coin but they’ll produce the same information.

See also:

Entropy as a statistical test of over/under-representation

Lack and Excess in Entropy Agglomeration

The so-called “Third Axiom of Probability” is false

2 comments

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s