Shannon's Channel Capacity Limit and the Architecture of AI-Based Semantic Communication

Shannon's Law

Today's communication has developed on the basis of Shannon's Law (Shannon's Law). This is a theory that determines the limits of network communication, and the efforts have been aimed at raising the maximum channel capacity of the formula described below. For example, there are technologies such as MIMO (Multiple Input Multiple Output). These technologies are part of an effort to achieve higher channel capacity based on Shannon's theory.

Shannon's Law is expressed by the following formula that calculates channel capacity.

These represent Channel Capacity, Bandwidth, and Signal-to-Noise Ratio, respectively. Put simply, the maximum communication speed (C) is obtained by multiplying the total amount of physical frequency resources the system can use (B) by the efficiency (log₂(1 + S/N)), which indicates how many bits of information can be carried per 1 Hz of that resource according to signal quality (S/N).

With the emergence of the law for calculating channel capacity, the communications industry began to devote its efforts to increasing channel capacity. For roughly 70 years thereafter, the innovation in communications has largely been achieved through improvements in channel capacity.

However, in the present era, processing resources have advanced tremendously. Communication has progressed from sending text to now sending spatial vectors. Reliably transmitting all of this data in segments has begun to reach its limits. For example, the data generated by autonomous vehicles can amount to several terabytes per day, and it is nearly impossible to sustain this with current communication networks. Simply laying more cables and deploying more antennas reaches economic and physical limits.

Thus, moving away from the paradigm of accurately transmitting all bits, a paradigm that seeks to transmit only context by introducing intelligence between communicating parties has begun to be studied. (*The concept itself has existed for decades.) This is a change driven by the recent strong development of intelligent models and the growing need to communicate even larger volumes of data.

This is called Semantic Communication, as it is communication that exchanges meaning.

Semantic Communication (Semantic communication)

Whereas semantic communication previously transmitted the entire data, it now aims to transmit only the core meaning contained within it, that is, the context.

This problem was already raised in the communication model of Shannon and Weaver, who divided the maturity of communication into three levels.

1. Technical problem: How accurately can a symbol be transmitted? (This is the core area of my theory.)
2. Semantic problem: How accurately does the transmitted symbol convey the desired 'meaning'?
3. Effectiveness problem: How effectively does the conveyed meaning influence the behavior of the recipient?

Up to now, the development of communication has nearly solved the technical problem, and it now undertakes the task of addressing the semantic and effectiveness problems.

The difference between maturity level 1 and levels 2 and 3 (semantic communication) is typically illustrated by the burning house example.

A house is on fire.

In the current communication paradigm, this scene is digitized piece by piece, and it is transmitted as a photograph.

In semantic communication, instead of sending all the data, as in "black smoke is coming from the window and flames are visible," only the core 'meaning' is conveyed, such as "Fire outbreak, immediate dispatch required." This approach boldly omits unnecessary information and aims to make the recipient take a specific action (dispatch).

If end-to-end communication shares the same knowledge base within the category of firefighting, this can dramatically reduce the amount of transmitted data required to grasp the situation.

The core encoding/decoding logic of semantic communication is a communication paradigm but operates on top of the application layer. On the transmitting side, a semantic encoder converts the given data into semantic data, and on the receiving side, a semantic decoder processes it into a form that downstream sources can use. Both will likely take the form of inference models or the like that possess the same knowledge base, and through this, communication that exchanges semantics becomes possible without transmitting massive data.

Naturally, this is guaranteed on the foundation of the completeness of the existing communication paradigm. First, symbols must be accurately transmittable at the technical level, and this maturity level has already been achieved. The main challenge now is how well the transmitted symbols convey and interpret the semantics of information, and research on this has only just begun.

However, unlike traditional syntactic communication systems, communication systems based on this semantic context depend on AI and similar technologies for reliability, so there is a very high likelihood of problems arising. Even with the same knowledge base, different interpretations may emerge from the black-box domain of the model.

Afterword

It is said that in 6G (sixth-generation) mobile communication, such semantic communication will be applied to become an intelligent internet system, but there is a question mark over why a paradigm operating above the application layer has become a research subject for mobile carriers. In my intuition, mobile carriers play the role of guaranteeing maturity level 1, in which symbols and bits are accurately transmitted at the technical level, and I think the point at which semantic communication operates already belongs to the domain of application programs.

On the other hand, there is also a doubt about whether a new paradigm can exist in communication technology, where reliability must be laid down as the fundamental value. I also hold this doubt, and personally I take a somewhat negative position.

Nevertheless, the reason I am writing this article is that I think the next paradigm of mobile communication is unfolding in a rather interesting form. The introduction of satellite internet for expanding channel capacity is practically a settled fact with the emergence of Project Kuiper, Starlink, and others, and the attempt to break through the limits that had been constrained by Shannon's Law in a new form was quite fascinating.