Claude Shannon, a legacy transcending digital technology

Claude Shannon, a major scientist from the second half of the 20th century, marked his era with his communication theory. His work triggered a digital metamorphosis that today affects all levels of our societies. To celebrate what would have been Shannon’s 100th birthday this year, the Institut Henri Poincaré will pay tribute to the scientist with a conference on October 26 to 28. At this event, Olivier Rioul, a researcher at Télécom ParisTech, will provide insight into the identity of this pioneer in the communication field, and will present part of the legacy he left behind.

Tags:

# Digital

# history of sciences

# information theory

# Télécom ParisTech

# theory of communication

A mathematical genius, Claude Shannon was also a lover of games and the author of an article entitled “Programming a computer to play chess”. Credits: LIVA Linzer Veranstaltungsgesellschaft mbH.

Read the post

What are turbo codes?

Claude Elwood Shannon. The name is not well known by the general public. And yet, if the digital revolution the world is experiencing today had a father, it would doubtless be this man, born in 1916 in a small town in Michigan. His life, which ended in 2001, received little media coverage. Unlike Alan Turing, no Hollywood blockbusters have been dedicated to him. Nor has his identity been mythicized by artistic circles, as was the case for Einstein. “Shannon led an ordinary life, and perhaps that is why nobody talks about him,” observes Olivier Rioul, researcher in digital communications at Télécom ParisTech.

Though his life was not particularly extraordinary, Claude Shannon’s work, on the other hand, was thrilling in many ways. In 1948, he wrote an article entitled A mathematical theory of communication. “Its publication came as revolution in the scientific world,” explains Olivier Rioul. In this article, Claude Shannon introduced the concept of bits of information. He also outlined – for the first time – a schematic diagram of a communication channel, which included all the active parts involved in transmitting a signal, from its source to its destination.

Capture-d’écran-2016-10-25-à-09.19.31-1024x433

First schematic diagram of a communication system, published by Claude Shannon in 1948. He explained that a channel could be “a coaxial cable, a band of radio frequencies, a beam of light, etc.”

This element may seem trivial. And yet its impacts have been felt far beyond Shannon’s field of study. The diagram has inspired presentations of paradigms in linguistics, representing the sequence of a speech; in psychology, showcasing the mechanisms of understanding and perceiving a message; and even in biology, where it has been used to describe the transcription of DNA in greater detail. The enthusiasm for this type of schematic depiction was so great that in 1956, Claude Shannon sought to curb this general surge of excitement in an editorial entitled “The Bandwagon”. In the editorial, he reminded readers that, above all, his work represented a branch of mathematics, and his theory could not resolve all the problems in every discipline.

Shannon and his magic formula

Yet in addition to his channel diagram, it was above all a formula published in a paper in 1948 that went on to mark the scientific community: C=W.log(1+SNR). With this mathematical expression, Shannon defined the maximum capacity of a transmission channel, in other words, the quantity of information that can be transmitted in a reliable manner. It shows that this capacity depends solely on the channel’s bandwidth, and the relationship between the strength of the transmitted signal and the noise in the channel. Based on these results, every channel has a throughput limit, below which the message transmitted from the transmitter to the receiver is not altered.

“Shannon’s strength lies in having succeeded in obtaining this result in a theoretical way,” insists Olivier Rioul. “Shannon did not provide the solution required to reach this limit, but showed that it exists, for all channels.” It would not be until 43 years later, with the work of Berrou, Glavieux and Thitimajshima in 1991, that Shannon’s limit would be nearly reached for the first time with the development of turbo codes.

Olivier Rioul believes the story behind this formula is out of the ordinary, and has been the subject of many historical approximations. “And the time was ripe. In 1948 – the year in which Claude Shannon made his results public – seven other scientists published similar formulas,” he explains, based on research carried out with José Carlos Magossi on the history of this formula.

However, the results obtained by Shannon’s peers were sometimes inaccurate and sometimes inspired by Shannon’s prior work, and therefore not very original. And all of them were part of the same environment, were in contact with each other or participated in the same conferences. All except Jacques Laplume, a French engineer who obtained a correct formula similar to Shannon’s at almost the same time. Yet what he lacked and what kept him from leaving his mark on history was the enormous contribution of the rest of Shannon’s theory.

A pioneer in communications, but that’s not all…

While his work represents the beginnings of modern digital communications, Claude Shannon also left behind a much greater legacy. In 1954, behavioral psychologist Paul Fitts published his law, named after him, which is used to model human movements. In his scientific article, he explicitly cited Shannon’s theorem, referring to his channel capacity formula. “Today we use Fitt’s formula to study human-computer interactions,” explains Olivier Rioul, who worked with a PhD student on reconciling this law with Shannon’s theory.

The scope of Shannon’s work therefore far exceeds the realms of information and communication theory. As a lover of games, he developed one of the first machines for playing chess. He was also one of the pioneers of artificial intelligence and machine learning, with his demonstration in 1950 of a robotic mouse that could find its way through a labyrinth and remember the optimal route.

Although Shannon’s life was not necessarily extraordinary in the literal sense, he was undeniably an extraordinary man. As for his lack of fame – which the centenary celebration of his birth seeks to remedy – he himself had said, referring to his information theory, “In the beginning I didn’t think it would have an enormous impact. I enjoyed working on these problems, just like I enjoyed working on lots of other problems, without any ulterior motives for money or fame. And I think that a lot of other scientists have this same approach, they work because of their love of the game.”

Claude Shannon, code wizard

To celebrate the centenary of Claude Shannon’s birth, the Institut Henri Poincaré is organizing a conference dedicated to the information theory, from October 26 to 28. The conference will explore the mathematician’s legacy in current research, and areas that are taking shape in the field he created. Institut Mines-Télécom, a partner of the event, along with CNRS, UPMC and Labex Carmin, will participate through presentations from four of its researchers: Olivier Rioul, Vincent Gripon, Jean-Louis Dessalles and Gérard Battail.

To find out more about Shannon’s life and work, CNRS has created a website that recounts his journey.