Questions tagged [information-theory]

The tag has no usage guidance.

Filter by
Sorted by
Tagged with
1 vote
0 answers
47 views

What contributions from Glavieux & Thitimajshima were "merged" into Claude Berrou's now-famous 1993 talk "Near Shannon limit error-correcting coding"?

At 01:15:13 in the delightful video The Bit Player | Claude Shannon | Father of Information Theory | IEEE Information Theory Society Andrea Goldsmith recounts ...
user avatar
  • 1,942
3 votes
0 answers
102 views

Pre-20th century sources on information theory?

What are some pre-19th or pre-20th century sources on information theory? Do they exist?
user avatar
  • 5,011
4 votes
1 answer
276 views

Why was the 'differential entropy' from information theory so named?

The entropy of a distribution $p$ on a discrete set $\mathcal{X}$ is defined as $$H(p) = -\sum_{x \in \mathcal{X}} p_x \log p_x.$$ Shannon in his classic paper [1] defines the analogue for continuous ...
user avatar
6 votes
2 answers
317 views

Why doesn't Morse code try to achieve more compression?

I was working on the Morse code Wikipedia article , when I noticed something strange. There might be a reason, but I can't find it. As you can see, the code symbol ...
user avatar