Understanding Shannon's Entropy metric for Information · HackerPapers discussions

Understanding Shannon's Entropy metric for Information — mentioned by dswilkerson

Thu, 07 Oct 2021 16:12:16 GMT

I read your arxiv.org article on Shannon Entropy: https://arxiv.org/abs/1405.2061 What I am amazed at is that arxiv accepted this. I submitted exactly the same argument to them on 27 Jan 2009. They rejected it. Here it is on my home page: http://dsw.users.sonic.net/entropy.html Entropy is (1) the expected value of (2) the information of an event. You might want to post a link to your answer here and see which of our explanations is more popular: https://math.stackexchange.com/questions/331103/intuitive-ex...

Paper and discussion history. A mention is not an endorsement.

Understanding Shannon's Entropy metric for Information — HN submission

Thu, 07 Oct 2021 13:24:22 GMT

Understanding Shannon's Entropy metric for Information [pdf]

100 points; 42 discussion replies on HN. Paper and source history. A submission is not an endorsement.

Understanding Shannon's Entropy metric for Information — mentioned by refactor_master

Wed, 27 Jan 2021 22:49:48 GMT

It does indeed sound like it’s written the wrong way around. Random information = high entropy. A common word is therefore low entropy. Here’s a good analogy I found: > Informally, the amount of information in an email is proportional to the amount of “surprise” its reading causes. For example, if an email is simply a repeat of an earlier email, then it is not informative at all. On the other hand, if say the email reveals the outcome of a cliff-hanger election, then it is highly informative. Similarly, the information in a variable is tied to the amount of surprise that value of the variable…

Paper and discussion history. A mention is not an endorsement.