Contents

Abstract.............................................................................................................................................1

1.OverviewofSemanticCommunication.............................................................................................6

2.KeyModulesofSemanticCommunication.......................................................................................8

2.1SemanticKnowledgeBase.........................................................................................................8

2.1.1ModernSemanticCommunicationEnabledbySemanticKnowledgeBase.................................8

2.1.2StructuralSystemofSemanticKnowledgeBaseforSemanticCommunication........................11

2.1.3ConstructionMethodofSemanticKnowledgeBase...................................................................12

2.1.4ExamplesofSemanticKnowledgeBasesSupportingSemanticCommunication......................14

2.2JointSemantic-ChannelCodingandDecoding..........................................................................16

2.2.1SemanticCodingandDecoding..................................................................................................16

2.2.2JointSemantic-ChannelCodingandDecoding...........................................................................16

2.3SemanticInformationTransmission..........................................................................................19

2.3.1SemanticInformationTransmissionandChallenges..................................................................19

2.3.2AddressingModernCommunicationChallenges:ExamplesofSemanticInformation

TransmissionSystem........................................................................................................................20

2.4CompatibilityofSemanticCommunicationwithExistingSystems..............................................23

2.4.1CompatibilityofSemanticCommunicationwithSourceCoding...............................................24

2.4.2CompatibilityofSemanticCommunicationwithHierarchicalArchitectureofClassical

CommunicationSystems.....................................................................................................................29

3.Single-ModalSemanticCommunication.........................................................................................32

3.1Text-OrientedSemanticCommunication...................................................................................33

3.2Speech-OrientedSemanticCommunication...............................................................................37

3.3Image-OrientedSemanticCommunication................................................................................42

3.4Video-OrientedSemanticCommunication.................................................................................44

3.4.1TheContextofVideoTransmission............................................................................................44

3.4.2SemanticVideoTransmission.....................................................................................................45

3.4.3SemanticVideoConference........................................................................................................45

3.4.4Conclusion...................................................................................................................................48

4.SemanticNoiseSuppression..........................................................................................................50

4.1RobustTextSemanticCommunication......................................................................................50

4.2RobustSpeechSemanticCommunication..................................................................................52

4.3RobustImageSemanticCommunication...................................................................................53

3/155

5.MultimodalSemanticCommunication............................................................................................56

5.1MultimodalSemanticCommunicationArchitecture...................................................................56

5.1.1ModalFusionDesigninMultimodalSemanticCommunication................................................58

5.1.2NoiseImmunityDesigninMultimodalSemanticCommunication............................................59

5.2Multi-TaskSemanticCommunication.......................................................................................60

5.2.1WhatTriggersStudiesonMulti-TaskSemanticCommunicationSystems................................60

5.2.2Multi-TaskSemanticCommunicationSystemModel................................................................62

5.2.3MainTechnologiesandProspectsofMulti-TaskSemanticCommunication.............................63

5.3Multi-UserSemanticCommunication.......................................................................................64

5.3.1MajorTechnologiesUsedinMultiuserSemanticCommunication............................................65

5.3.2MultiuserSemanticCommunicationSystemArchitecture.........................................................66

5.3.3ResourceAllocationinMultiuserSemanticCommunication.....................................................68

6.DigitalSemanticCommunicationandWaveformOptimization........................................................70

6.1DigitalSemanticCommunicationNetwork................................................................................70

6.2OFDM-basedSemanticCommunicationWaveformOptimization...............................................72

6.3Two-WaySemanticCommunication.........................................................................................75

6.4APredictiveChannel-basedSemanticCommunication..............................................................79

7.ResourceManagementinSemanticCommunication........................................................................83

7.1ConstructionofResourceAllocationModelinSemanticCommunication...................................83

7.2OptimizationObjectivesofResourceAllocation........................................................................85

7.3ResourceManagementStrategyinSemanticCommunication.....................................................86

7.4SemanticHybridAutomaticRepeatreQuest..............................................................................87

7.4.1TraditionalHybridAutomaticRepeatreQuestStrategies...........................................................87

7.4.2SemanticHybridAutomaticRepeatSystem...............................................................................88

7.4.3Conclusion..................................................................................................................................91

7.5SemanticErrorDetection.........................................................................................................92

7.5.1TraditionalCheckandErrorDetectionMethods........................................................................92

7.5.2SemanticSimilarityDetectionTechnology.................................................................................93

7.5.3SentenceSemanticSimilarityDetection.....................................................................................95

7.5.4Conclusion..................................................................................................................................97

8.DistributedSemanticCommunication.............................................................................................98

8.1MultimodalSemanticRelayArchitecture..................................................................................98

8.1.1CompositionandDeploymentofMultimodalSemanticRelayArchitecture...........................101

4/155

8.1.2AdvantagesofMultimodalSemanticRelayArchitecture.........................................................102

8.2DistributedCollaborativeDevice-ServerInference..................................................................104

8.2.1CompositionandDeploymentofDistributedCollaborativeDevice-ServerInference.............104

8.2.2ExperimentalPerformanceAnalysisofDistributedCollaborativeDevice-ServerInference...105

9.SecurityChallengesandCountermeasuresinSemanticCommunication.........................................107

9.1ModelSecurityinSemanticCommunication...........................................................................107

9.1.1AdversarialAttacksandDefense..............................................................................................108

9.1.2PoisoningAttacksandDefense.................................................................................................110

9.1.3FutureProspects........................................................................................................................112

9.2CryptographicTechniquesinSemanticCommunication...........................................................112

9.3PrivacyProtectioninSemanticCommunication.......................................................................115

9.4SemanticWatermarkingforSecureCommunication................................................................117

10.SemanticCommunicationforSatelliteSystems...........................................................................122

10.1EfficientSemanticTransmissioninSatelliteSystems.............................................................122

10.1.1Multi-SatelliteSemanticTransmissionSystem.......................................................................123

10.2HighlyReliableSemanticTransmissioninSatelliteSystems..................................................124

10.2.1SemanticTransmissionunderLowLinkBudget....................................................................124

10.2.2BeamformingforSatelliteApplicationsBasedonSemanticTransmission...........................125

10.2.3SemanticTransmissionBasedonChannelPrediction............................................................126

10.3IntelligentSemanticTransmissioninSatelliteSystems...........................................................126

11.ApplicationScenarios..............................................................................................................128

11.1HolographicCommunication................................................................................................128

11.2AutonomousDriving.............................................................................................................131

11.3DigitalSafeVillage............................................................................................................133

11.4IndustrialIntelligentManufacturing......................................................................................134

11.5SmartInstrument............................................................................................................................135

12.KeyChallenges.....................................................................................................................................137

12.1ChallengestoSemanticInformationTheory..........................................................................137

12.2SemanticCommunicationBasedonLargeModels.................................................................138

12.3Standardization...............................................................................................................................141

13.SummaryandOutlook.............................................................................................................143

14.References..............................................................................................................................144

15.ListofAbbreviations...............................................................................................................153

5/155

1.OverviewofSemanticCommunication

ClaudeShannonandWarrenWeaverpointedoutthatcommunicationcanbedividedinto

threelayers:syntactic,semantic,andpragmatic.Traditionalcommunication,developedbasedon

ClaudeShannon'sclassicalinformationtheory,fallsintothesyntacticlayer,whichevaluates

networkperformanceusingmetricssuchasbiterrorratio,symbolerrorrate,andtransmission

ratewithoutconsideringthemeaningofsymbols.Itismainlyusedtosolvetechnicalchallenges

relatedtothecorrecttransmissionofbitsorsymbols.Semanticcommunicationfocuseson

conveyingthemeaningandsignificanceofinformation,ratherthanjustfocusingonthesymbols

themselves.Itismainlyusedtoaddresshowtoaccuratelyconveythemeaningoftransmitted

symbolsandhowthereceivercaninfluencethesystembehaviorinthedesiredway.Semantic

communicationhasbeenproposedinsyncwithsyntacticcommunication.However,becauseof

technicallimitationsandspecificscenariodemands,peoplehavepredominantlyfocusedon

syntacticcommunication.Nevertheless,withrapidadvancementincommunicationtechnology,

thecapacityoftraditionalcommunicationsystemshasgraduallyapproachedtheShannonlimit.

Ontheotherhand,thankstothegrowthofartificialintelligencetechnologyandtheincreasing

demandforintelligentcommunicationin6Gnetworks,semanticcommunicationhasonceagain

becomeapopulartechnology.

Semanticcommunicationisanovelcommunicationparadigmthatcanaddressthe

expressionandtransmissionofinformationmeaningsatthesemanticlayer.Itinvolvesmoving

someorallaspectsoftheunderstandingofinformationmeaningstothetransmitter,thereby

reducingtransmissionvolumeandloweringbandwidthrequirements.Semanticcommunication

differsfromtraditionalcommunicationinthefollowingaspects.First,themodeofinformation

representation.Semanticcommunicationupgradessymbolrepresentationtosemanticfeatures

tailoredforcommunicationscenariosandtasks,movingtheextractionandunderstandingof

semanticfeaturesofthesourcecontenttothetransmitter.Second,theevaluationcriteriafor

qualityofservice.Syntacticcommunicationtypicallymeasuresthequalityofserviceusing

metricssuchassymbolerrorrateandpacketlossrate,whichdonotdirectlyreflectuser

experienceandothersubjectiveaspectsofquality.Dependingonscenariosandtasks,semantic

communicationdefinesthequalityofservicejointlyusingobjectivesemanticaccuracyand

6/155

subjectivelysensedquality.Comparedtotraditionalcommunication,semanticcommunication

systemsaremoreefficientintransmission.Bytransmittingonlycrucialsemanticinformation

ratherthanallinformation,semanticcommunicationrequireslesstransmissionbandwidth,which

canenhancetransmissionreliabilityandcapacity,consequentlyimprovingwirelesstransmission

efficiency.Meanwhile,thereconstructionofrawinformationrequiresasemanticdecoding

model.Accordingly,semanticcommunicationcanenhancedatasecurityundercertainconditions.

Thesystemmodelofsemanticcommunicationisshowninthefigurebelow.Semantic

communicationmainlyfocusesonthesemanticrepresentation,transmission,andreconstruction

ofsourcecontent,aswellassemantic-basedwirelesstransmission.Keyprocessesinclude

knowledgebaseconstruction,semanticcodinganddecoding,andchannelcodinganddecoding.

Intermsoftheory,semanticcommunicationresearchismainlyinspiredbyClaudeShannon's

informationtheory.Itdefinessemanticentropy,semanticrate-distortion,andsemanticchannel

capacitybyreplacingstatisticalprobabilitywithlogicalprobability,andthenestablishesa

theoreticalframeworkforsemanticcommunication.Ontheotherhand,thankstothe

developmentofartificialintelligenceandenhancingdataprocessingcapabilities,algorithms

relatedtotheextraction,coding,andtransmissionofsemanticfeaturesbasedondeeplearning

havebeensuccessfullyappliedtovarioustypesofsources.Semanticcommunicationholdsbroad

applicationprospectsinfieldssuchasmultimediacommunication,augmentedrealityand

immersivecommunication.

Figure1.1SemanticCommunicationSystemModel

7/155

2.KeyModulesofSemanticCommunication

2.1SemanticKnowledgeBase

2.1.1ModernSemanticCommunicationEnabledbySemantic

KnowledgeBase

Figure2.1ModernSemanticCommunicationArchitectureEnabledbySemanticKnowledge

Base

AlongwiththerapiddevelopmentofmobilecommunicationandInternettechnologies,the

demandhassurgedforhigh-speedandlow-latencyradioaccess.Traditionalcommunication

systemshaveapproachedtheShannonlimit,makingiturgenttoachievebreakthroughsinnew

communicationtechnologies.Semanticcommunication,asanendogenousintelligenceandnovel

brain-likeinformationinteractionmechanism,involvesprocessessuchassemanticelement

extraction,recognition,understanding,transmission,andreasoningakintoinformation

transmissionamonghumans.Intraditionalcommunication,sourcesymbolsaremappedtobit

streamsbasedonpresetcodingschemes,withmappingfunctionsdesignedusingpractical

experienceandaccuratemathematicalmodels.Insemanticcommunication,sourcesaremapped

tosemanticstreamsbasedonthesemanticbase(Seb)usingcodingsystemspoweredbyartificial

8/155

intelligencealgorithms,withmappingfunctionsestablishedbyaneuralnetworksystemdriven

bybothdataandmodels[1].

Unliketraditionalcommunication,whichprocessessignalsbasedonsymbol-bitstreams,

semanticcommunicationdoesthatbasedonthesemanticbase.Thesemanticbaseisthe

minimumtransmissionandprocessingunitinsemanticcommunicationbutcurrentlylacksa

precisedefinition.Broadlyspeaking,itcoversallsemantic-relatedfeaturesextractedfromsource

information.However,semanticfeaturesextractedbydifferentmethodsorsemanticfeaturesin

variousformsundoubtedlypossessdistinctcharacteristics,withsomeexhibitingclear

advantagesordisadvantages,whileothersrequirejudgmentbasedonspecificcommunication

tasksandrequirements.Thatisoneofthereasonsbehindtheuncertaintyofthesemanticbase.

Afterall,regardingthesemanticelementsincludedinthesamepieceofexactinformation,their

understandingmaynotbecompletelyconsistentindifferentcommunicationnodes.Thatisthe

reasonwhythesemanticbase'sdefinition,acquisitionmethod,andmultimodaluniversal

semanticextractionneedfurtherexploration.

Thesemanticknowledgebaseisaresearchdirectionwithinthesemanticbaseframework,

representingastructuredandmemory-capableknowledgenetworkmodelthatcanprovide

relevantsemanticknowledgedescriptionsfordatainformation.AsdepictedinFigure2.1,the

semanticknowledgebaseforsemanticcommunicationcanbedividedintothesourceknowledge

base,channelknowledgebase,andtaskknowledgebase,offeringmulti-levelsemantic

knowledgerepresentationsforsourcedata(e.g.,text,images,videos,etc.),channeltransmission

environment(e.g.positionandshapeinformationofobstaclesduringtransmission,position

informationofintelligentreflectingsurface(IRS)),andtaskrequirements(e.g.image

classification,3Dreconstructionandsemanticsegmentation)[1].Inend-to-endsemantic

communication,atthetransmittingend,thetransmittercanefficientlyacquiremulti-level

semanticknowledgedescriptionsofsourcedata,channelsemanticestimationsoftransmission

environment,andspecificsemanticrequirementsfordownstreamtasksbasedonitssource,

channel,andtaskknowledgebase.Subsequently,itperformsjointsemantic-channelcodingfor

informationtobetransmitted.Atthereceivingend,thereceiver,basedonitssemantic

knowledgebase,performsknowledgequeryandunderstandingofreceivedinformationto

completejointsemantic-channeldecoding.

9/155

Thesemanticknowledgebasebooststhedevelopmentofsemanticcommunication.For

instance,currentdeeplearning-basedjointsource-channelcodinganddecodingmethodsrequire

extensivetrainingbasedonlarge-scalerelevantdatatoobtainsuitablecommunicationmodels

tailoredtospecifictasks.Thishighconsumptionofdataandtimeresourcesimpedesthe

widespreadadoptionofdeepjointsource-channelcoding.Asapriorknowledgesource,the

semanticknowledgebaseoffersefficientandstandardizedsemanticsupportforsemantic

communication,effectivelyacceleratingthetrainingofjointsource-channelcodinganddecoding

networks,andreducingthedependencyonmassivedatafornetworktrainingforspecific

communicationtasks.

Next-generationwirelesscommunicationtechnologiesfaceincreasinglycomplexand

diversecommunicationscenariosandapplicationrequirements.Thedemandforinterconnected

scenariosinvolvingintelligentagentsusingdifferentcommunicationprotocolswillsignificantly

increase.Knowledgesharingthroughthesemanticknowledgebasewillhelpestablishaunified

knowledgebackgroundforanefficienthuman-machine-thinginterconnectednetwork,enhancing

interactionefficiencyamongheterogeneousprotocol-basedintelligentagents.Intelligentagents

employingdifferentsemanticcommunicationprotocolsexhibitsignificantdifferencesin

definingandrepresentingthesemanticbase,whilethesemanticknowledgebasecanpartially

addresschallengesrelatedtonon-uniformsemanticbasespecificationsandphysicallayersignal

specificationsduringcross-protocolsemanticcommunication.Asufficientlylargesemantic

knowledgebasecancoverrepresentationspecificationsofsemanticbasesandsignalsforvarious

protocols.Distributingittoheterogeneousintelligentagentsengagingincross-protocolsemantic

communicationcaneffectivelyassisttheseagentsintask-specifictraining.Furthermore,onthe

basisofthesemanticknowledgebase,codinganddecodingcomponentscanbeupdatedoradded

flexiblywithouttheneedtodeleteormodifyexistingsemanticandphysicallayermodulesof

intelligentagents.Typically,thesemanticknowledgebaseisdeployedonedgeservers.The

heterogeneousprotocol-basedintelligentagentsmerelyneedtorequestandloadrelevant

semanticknowledgefromtheknowledgebasebeforeestablishinganewconnection.The

knowledgebasedoesn'thavetobeinvolvedinthecommunicationprocess.

Thesemanticknowledgebaseprovidesglobalknowledgebackgroundandstoragesearch

servicesforextracting,recognizing,transmitting,understanding,andreasoningsemantic

elementsduringsemanticcommunication.Itdefinesanefficientsearchspaceandstandardizes

10/155

searchpaths,significantlyenhancingtheflexibilityofsemanticcommunicationandsupporting

theapplicationofsemanticcommunicationinmorecommunicationscenarios.Itisoneofthe

keyenablingtechnologiesforsemanticcommunication.

2.1.2StructuralSystemofSemanticKnowledgeBaseforSemantic

Communication

Tobeginwith,weintroducethestructureofthesemanticknowledgebase,includingits

interfacesandinternalorganizationalstructure(asshowninFigure2.2).

Figure2.2SchematicDiagramofSemanticKnowledgeBase

①Designofknowledgebaseinterface

Considerasimplescenariowhereasourceandasinksharethesameknowledgebase.At

thesourceend,semanticknowledgeaboutthesource,tasks,andchannelsobtainedfromthe

knowledgebaseisadoptedtoguidecodingandtransmission.Atthesinkend,theknowledge

baseisappliedtodecodereceivedsignals.Acrucialstepinthisprocessishowtoretrievethe

requiredknowledgefromtheknowledgebase,whichwerefertoasknowledgeretrieval.By

representingsemanticknowledgefromdifferentmodalsignalsourcesintheformofsemantic

featurevectors,wecanuseaunifiedinterfacetohandleknowledgeretrievalfordifferentmodal

signalsources,ensuringtheconsistencyandscalabilityofthesemanticlibraryinterface.The

followingdefinestheinputandoutputofthesemanticknowledgebase.

Input:Typesofsemanticknowledge(source,task,orchannel)anddatasourceinformation

(e.g.numericaldata,character,text,image,speech,videoandpointcloud);

11/155

Output:Alistcontainingmultiplesetsofsemanticvectors.Thislistdescribesthemulti-

levelsemanticknowledgeoftheinputsignal,wherethelengthofthelistindicatesthenumberof

levels.

Forexample,insourcesemanticknowledgeretrieval,inputtinganimagereturnsmultiple

setsofsemanticvectors,witheachsetrepresentingalevel.Forobjectsegmentationtasks,the

returnedsemanticvectorscancharacterizeinformationsuchas<objectcategoryinformation>,

<textureinformation>,<objectpositioninformation>,and<head,body,hands,andlegsposition

information>.

②Organizationalstructureofsemanticknowledge

Theknowledgebasewillconsistofthreetypesofsub-semanticknowledgebases:source,

task,andchannel.Theknowledgebaseinternallystoresseveralsetsofsemanticbasespre-

constructedbasedonthedatasourcesandtargettasks,aswellascomputationalmodelsthatcan

beusedtodynamicallyconstructsemanticbasesinreal-time.Whennewdatasourcesareinput

forsemanticknowledgeretrieval,iftheexistingsemanticbasesmeetthecommunicationtask

requirements,theywillbedirectlyusedtorepresenttheinputdatasourcesandreturnsemantic

knowledgevectors;iftheexistingsemanticbasesdonotmeetthecommunicationtask

requirements,computationalmodelswillbeusedtodynamicallyconstructsemanticbasesthat

meettherequirementsforsemanticknowledgerepresentationofinputdatasourcesinreal-time.

2.1.3ConstructionMethodofSemanticKnowledgeBase

Currentmethodsforconstructingsemanticknowledgebasesforend-to-endsemantic

transmissionincludeknowledge