March2024ExpertInsightsPERSPECTIVEONATIMELYPOLICYISSUEBILVACHANDRAAnalyzingHarmsfromAI-GeneratedImagesandSafeguardingOnlineAuthenticityThedemocratizationofimage-generatingartificialintelligence(AI)toolswithoutregulatoryguardrailshasamplifiedpreexistingharmsontheinternet.TheemergenceofAIimagesontheinternetbeganwithgenera-tiveadversarialnetworks(GANs),whichareneuralnetworkscontaining1(1)ageneratoralgorithmthatcreatesanimageand(2)adiscriminatoralgorithmtoassesstheimage’squalityand/oraccuracy.Throughseveralcollaborativeroundsbetweenthegeneratoranddiscriminator,afinalAIimageisgenerated(Alqahtani,Kavakli-Thorne,andKumar,2021).ThisPersonDoesNotE,asitecreatedbyanUberengineerthatgeneratesGANimagesofrealisticpeople,launchedinFebru-ary2019toawestruckaudiences(Paez,2019),withseriousimplicationsforexploita-tioninsuchareasofabuseaswidespreadscamsandsocialengineering.ThiswasjustthebeginningforAI-generatedimagesandtheirexploitationontheinternet.Overtime,AIimagegenerationadvancedawayfromGANsandtowarddiffusionmodels,whichproducehigher-qualityimagesandmoreimagevarietythanGANs.DiffusionmodelsworkbyaddingGaussiannoise2tooriginaltrainingdataimagesthroughaforwarddiffusionprocessandthen,throughareverseprocess,slowlyremovingthenoiseandresynthesiz-ingtheimagetorevealanew,cleangeneratedimage(Ho,Jain,andAbbeel,2020).Diffusionmodelsarepairedwithneuralnetworktechniquestomaptext-to-imagecapabili-ties,knownastext-imageencoders(ContrastiveLanguage-ImagePre-training[CLIP]wasamilestoneinthisspace),toallowmodelstoprocessvisualconcepts(Kim,Kwon,andChulYe,2022).Thus,thecommercializationofdif-fusionmodels(DALL-E,StableDiffusion,Midjourney,Imagen,andothers)putthepowerofsyntheticimagegen-erationinthehandsoftheuseronaglobalscale.Theriseofimagegenerationtoolshasintroducedsyntheticformsofsuchsafetyharmsasmis-anddisinformation,extremism,andnonconsensualintimateimagery(NCII),causingfur-therdisarrayanddamageintheinternetecosystem.ThesocietalharmsfromAIimagegenerationtoolshaveyettobeeffectivelyaddressedfromaregulatorystandpointbecauseofanexusofpolicychallenges,suchascopyrightprotection,dataprivacy,ethics,andcontrac-tualrequirements.RecentfearsrisingfromgenerativeAIhavesomewhatmovedthepolicyneedleonAIregula-tion,sparkinggreatinterestinCongressandtheexecutivebranch(asevidencedbytheSenateAIinsightforumsandPresidentJoeBiden’sexecutiveorderonAI,respectively[WhiteHouse,2023b]).However,withoutlegislationaddressingsafetyandsocietalissuesrelatedtogenerativeAI,executingacoherentregulatorystrategytoaddresstheharmfuleffectsofAI-generatedimagesontheinternetisatallorder.(Asofthiswriting,Section230oftheCom-municationsDecencyActof1996servesasthesolepieceoflegislationforinternetregulation[U.S.Code,Title47,Section230].)Inthispaper,IdelveintosafetyharmsandchallengesfromAI-generatedimagesandhowsuchimagesaffectauthenticityontheinternet.Thefirstsectionoutlinestheroleofimageauthenticityontheinternet.Inthesecondsec-tion,Ireviewthetechnicalsafetychallengesandharmsfortheimagegenerationspace,thenlookatindustrysolutionstoauthenticity,includingthepromiseofprovenancesolu-tionsandissueswithimplementingthem.Thethirdsectionoutlinesseveralpolicyconsiderationstotacklethisnewpar-adigmthatlargelyfocusonprovenance,givenitspromiseasanauthenticitysolutionandrelevanceinpolicyconversa-tions.Inthispaper,contentauthenticityinthecontextofimagesreferstoestablishingtransparentinformationaboutimages(bothhumanandAI-generated),whetherinorigin,context,authorship,orotherareas,inawaythatisaccessi-bletousersontheinternet.Throughout,thispaperfocusesAbbreviationsAIartificialintelligenceC2PACoalitionforContentProvenanceandAuthenticityCAICIDContentAuthenticityInitiativecivilinvestigativedemandCLIPCSAMFTCGANGIFCTNCIIContrastiveLanguage-ImagePre-trainingchildsexualabusematerialFederalTradeCommissiongenerativeadversarialnetworkGlobalInternetForumtoCounterTerrorismnonconsensualintimateimageryNationalInstituteofStandardsandTechnologyNISTOECDOrganisationforEconomicCo-operationandDevelopment2oncontentauthenticity,asitcouldplayakeyroleinshap-ingpublictrustinimagecontentbroadlyandcoversawideswathofissues,fromdisinformationtosyntheticNCII.TheissueisnotonlyadecreaseincontentTheRoleofImageAuthenticityontheInternetauthenticitybutalsoalackofknowledgeandtoolsamongmanyuserstohelpnavigatethisparadigmshiftintheinformationdomain.Imageshaveplayedanimportantroleinthehistoryoftheinternet,informingpeopleaboutcurrentevents,sparkingemotionalresponsestowaratrocitiesandinjustice,gal-vanizingindividualstosupportacause,andmuchmore.Researchshowsthatpeopletendtorespondmoreviscer-allytoimagesthantheydototextonline(MedillSchoolofJournalism,2019).Studiessuggestthatthebrainprocessesvisualstimulimorerapidlythanitdoeswords(AlpuimandEhrenberg,2023).Furthermore,imagescanincreaseaviewer’sperceptionofthetruthfulnessofanaccompanyingstatement(AlpuimandEhrenberg,2023).However,theeraoftreatingimagesas“proof”fromasocialstandpointisrapidlychanging.(somephotorealistic)spreadwidelyontheinternet,con-juringfakecrowdsofIsraelismarchinginsupportoftheIsraeligovernmentorunusualimagesofGazanchildreninthemidstofexplosions;theseimageswererarelylabeledasAI-generated(KleeandRamirez,2023).Imageauthenticityontheinternetisinjeopardy,asAI-generatedimageswithoutproofofprovenance,ortheoriginofagivenimage,areaffectinghowpeopleperceivecurrenteventsandpublicfigures.Theissueisnotonlyadecreaseincontentauthenticitybutalsoalackofknowl-edgeandtoolsamongmanyuserstohelpnavigatethisparadigmshiftintheinformationdomain.OpportunisticactorsaretakingadvantageofaccessibleAItoolstoreducetrustincontentandmedia,particularlyduringtumultu-ousperiods,suchasviolentconflicts.AnexampleofthisistheinflammatoryAI-generatedimagesspawnedafterHamas’sOctober7,2023,attackinIsraelandthesub-sequentconflictinGaza.MultipleAI-generatedimagesSolutionstothisproblemthatfocusoncontentauthen-ticitycouldbethemostvaluablebyprovidingindividualswithmoretransparencyaboutcontentthattheyconsumeand,therefore,moreagencyintermsofhowtointerpretthatcontent.Forexample,solutionsthatprovidemetadatainformationtotheuser,suchasauthorship,geolocationdata,andtoolsusedintheeditingofanimage,canallbeusefulforuserinterpretation.Thoughtheresearchbehindauthenticitymeasuresaffectingusertrustincontentisnotconclusive,andcontentauthenticityisnotasilverbulletforsolvingallAI-drivenharmsontheinternet,contentauthenticitysolutionsareastepintherightdirection.3Image-generationtechnologywillcontinuetoevolve,becomemoreadvanced,andlikelybecomemorephotoreal,increasingtheescalationpotentialforharm.Thereisnofoolproofwaytomakethesemodelsentirelysafeforuse.TostartsolvingissuesofAI-generatedimagesfuelingdis-information,harmfulpropaganda,NCII,andothersafetyharms,theUnitedStatesmustfirstdevelopsolutionsinregardtoauthenticityandimprovethepublic’saccesstoinformationabouttheseimages.ThefirststepinnavigatingthisshiftistocomprehendhowAIimagegenerationtoolsproducesafetyissuesandchallengesandhowcurrentsafeguardsareinsufficienttotackletheproblem.challengesareduetobiasesandharmsfromtrainingdata,theexistenceofopen-sourceimagemodels,andapiece-mealapproachtocontentmoderationattheuserlevel.ThecurrentAIimagegenerationspacemainlyconsistsoftext-to-imagediffusionmodels,suchasMidjourney,DALL-E,andStableDiffusion,thatgenerateimagesbasedonuserprompts.Understandingthefundamentalsofsafetyissuesintext-to-imagediffusionmodelscanshowwhythesemodelscanproduceunsafeimages.Furthermore,divingdeepintotrainingdata,open-sourcemodels,andcontentmoderationrevealsthatthesetechnicalmitigationsaresimplynotenoughtopreventthegenerationofharmfulcontent,andtheUnitedStatesneedsauthenticitysolutionstomanagerisksandharms.Imagegenerationmodelsreflectsocietalandrepresen-tationalbiasesontheinternetbecausetheyaretrainedondatascrapedfromtheinternet.Forexample,therearefarmoreimagessexualizingwomenontheinternetcomparedwithsimilarimagesofmenandfarmoreimagesofmeninprofessionalroles(doctor,lawyer,engineer,etc.)thansimilarimagesofwomen.Imagemodelsconceptualizetheserepresentationalbiasesandhavebecomequiteadeptatgeneratingcontentthatbothoversexualizeswomenandhighlightsmeninprofessionalpositions(Heikkil?,2022).Recentresearchshowsthatthereisstillmuchworktobedonetomakeimagemodelssaferandlessbiased,astherearestillsevereoccupationalbiasesinmodels,whichresultintheexclusionofgroupsofpeoplefromgeneratedresults(NaikandNushi,2023).Furthermore,RestofWorldin2023conductedanexperimentwithMidjourneythatshowedthatthetooltypicallyrepresentednationalitiesusingharmfulstereotypes:Imagesofa“Mexicanperson”mainlyshowedamaninasombrero,andimagesofNewSafetyChallengesinImageGenerationSafetychallengesintheAIimagegenerationlandscapebeginatthetechnicallevel,andthemostsignificantsafetyImagegenerationmodelsreflectsocietalandrepresentationalbiasesbecausetheyaretrainedondatascrapedfromtheinternet.4Delhialmostexclusivelyshowedpollutedstreets(Turk,2023).Theinternetisinherentlybiased,giventhenatureofitshumaninputs:Whenyouscrapehighlybiaseddata,youwillgenerateitaswell.nately,thetrade-offbetweensafetyandwhatconstitutesaqualityproductisoftendifficult.Thediscussionofsafetychallengeswithimagegen-erationwouldbeincompletewithouthighlightingtheopen-sourcespaceforthiscriticaltechnology.Thedebatesaroundopen-sourcegenerativeAIareabundant:Thosewhoareinfavorhighlightthebenefitsofopenaccessinimprovingmodelsandtheirsafetycapabilitieswithwiderresearcheraccess,andthoseagainstarefocusedonthepotentialformalignuseandchallengeswithmonitor-ingandcontrollingmodeluse.Withimagegenerationinparticular,openaccesshasbenefitedmaliciousactorsinremovingsafeguardsandgeneratingharmfulcontentatscalewiththeuseoffine-tunedopen-sourcemodels.Forexample,whenStableDiffusionwasopen-sourcedin2022,UnstableDiffusionwasborn.UnstableDiffusionwasalargeserverthatusedStableDiffusion’sopen-sourcemodelwithreducedsafeguardstocreatenot-suitable-for-workplacecontent(WiggersandSilberling,2022).AnevenmorealarmingexampleisCivitAI,asitecreatedforAIimagegenerationthatallowsuserstobrowsethousandsofmodelsinordertogeneratepornographiccontentandsyntheticchildsexualabuseimagery,streamliningthenonconsensualAIporneconomy(Maiberg,2023a).AmorerecentexampleisaStanfordInternetObservatoryinvesti-gationthatfoundhundredsofimagesofconfirmedchildsexualabusematerial(CSAM)inanopendatasetcalledLAION-5B,whichiscommonlyusedinimagegenerationmodels,suchasStableDiffusion(Thiel,2023).Theuseofopen-sourceimagegenerationposesethicalconcernsrelatedtochildexploitation,consent,andfairuse.Fromaconsentandfairuseperspective,suchusecoulddispro-portionatelyaffectsexworkers,whomayhavetheirimagesSafetyharmsinthesemodelsalsostemfromtrain-ingdata.Tostart,datalabelingislargelyoutsourcedtoprovidersthatspecializeinscaledlabeling,whichiscost-effective.However,thisprocesscanintroducebiasesandinaccuraciesinhumanlabelsfortrainingdata(SmithandRustagi,2020).Whendevelopersscrapedatafromtheinternetforimagegeneration,theirmainmethodtoensurethatmodelsaresafeistofiltertrainingdataandattempttoreducetheprevalenceofharmfulcontentinthetrainingprocess(SmithandRustagi,2020).However,thismethodiscontingentonhoweffectivethesefiltersareinrootingoutharmfulcontentwhileensuringthattheyarenottooconservativeinexcludingtrainingdatasothatthemodelsarestilltrainedonawiderangeofdataandretainqualityandcreativityintheirgenerations.Furthermore,evenwithrobustsafetyfiltering,theabilityformodelstodeducecon-ceptsacrossdifferenttypesofbenignimagescanresultinthegenerationofaharmfulimage.Forexample,animagemodelthatistrainedonimagesofbeachesbutnotonpor-nographywillunderstandhowthehumanbodylookswithswimsuitsorminimalclothing.Thesamemodelcouldalsobetrainedonimagesofchildrengoingtoschool,playingoutside,etc.—allofwhicharebenignimages.Thesemodelcapabilitiescombined(withoutfurthersafetymeasures)willlikelyallowthemodeltocreateimagesofscantilycladorevennudechildrenthroughmalignpromptengineer-ingtechniques.However,mostdeveloperswouldstillwanttrainingdatatohaveimagesofbeachesandchildrentoensurehighqualityandeffectivegenerations.Unfortu-5scrapedfromtheinternetduringthemodeltrainingpro-cess,allowingtheirlikenesstobereproducedsyntheticallywithouttheirconsentandwithoutcompensation.Despitethebenefitsofopen-sourcesoftwareensuringgreateraccesstoimagemodels,ithasacceleratedsafetyharms,especiallythoserelatedtosexualcontentandconsent.Last,itisvaluabletonotethecontentmoderationsifiersthatidentifyviolativepromptsandimageoutputclassifiersthatclassifyimagesthatshouldbeblocked,bothofwhichareusedinDALL-E3topreventharmfulgenerations(David,2023).Althoughsuchmoderationismoreholisticthanjustkeyword-blocking,theseclassifiersarenotfoolproof.Specifically,red-teamersforDALL-E3foundthat(1)themodelrefusalsondisinformationcouldbebypassedbyaskingforstylechanges,(2)themodelcouldproducerealisticimagesoffictitiousevents,and(3)publicfigurescouldbegeneratedbythemodelthroughpromptengineeringandcircumvention(OpenAI,2023).Broadly,contentmoderation,throughbothclassifiersandkeyword-blocking,wasneverbuilttoinfercontext,andanattempttodosoispracticallyanimpossibletask.Forexample,userscanattempttobypassmanykeywordandclassifiersafeguardsbyusingvisualsynonyms,suchas“flesh-coloredeggplant”(foraphallicimage),“redliquid”forblood,and“skin-coloredsheertop”togeneratenudity;thelistbecomesendless.Theseharmsandsafetychallengesareshapingcontentauthenticityontheinternet.Thoughmuchofthespotlightofcontentauthenticityisonmis-anddisinformation,theexistenceofsyntheticNCII,syntheticextremistpropa-ganda,andmorealsodirectlyshapesusers’perceptionsofrealityandcancausegreatharmtoindividualsandsociet-ies.Unfortunately,fromatechnicalperspective,eradicat-ingallthesesafetyharmsseemsunlikely,giventhechal-lengesofcontentmoderation,biasandsafetyissueswithtrainingdata,andtheexistenceoftailor-madeopen-sourcemodels.Instead,theUnitedStatesshouldlooktocreategreatertransparencyaroundtheseimagesthroughsolu-tionsthatpromoteaccesstoinformationabouttheoriginofcontent.Thesesolutionsmaybelessfruitfultomitigatesafeguardsofclassifiersandkeyword-blocking,whichareusedinmanyimagegenerationtools,andtodetailwhytheyareinsufficientforsafety.Safety,asafirstprincipleacrosstheAIdevelopmentlifecycle,isthepathtoensur-ingsafeimagegeneration,startingfromfilteringdatapriortotraining.Whenthemajorityofsafetyworkisdoneinamonitoringcapacitythroughcontentmoderation,harm-fulgenerationsarelikelytofallthroughthecracks.ThecurrentimagegenerationmodelsonthemarketcannotentirelypreventthegenerationofNCIIcontent,syntheticCSAM,extremistpropaganda,andmisinformation.Someofthem,suchasStableDiffusion,relyonaweakformofcontentmoderation:blockingspecifickeywordsfrombeingused.StableDiffusionenactedkeyword-blockingforwordsrelatedtothehumanreproductivesystemtoattempttopreventthegenerationofpornography(Heikkil?,2023).Bing’sAIImageGenerator,poweredbyDALL-E3,blockedthekeywords“twintowers”topreventanyharm-fuldepictionsof9/11(David,2023).Despitekeyword-drivensafeguards,botaccountswereabletospreadanAI-generatedimageofthePentagononfire,whichbrieflywentviral(Bond,2023).Keyword-drivenmoderationisatbestapiecemealsolutionthatonlyhelpstomoderatelow-hangingfruitandcanhardlycoverthemultitudeofharmfulnarrativesatscale.Anotherformofmoderationcanbedonethroughclassifiers,suchaspromptinputclas-6syntheticNCIIcontent,giventhatthecoreofmitigatingNCIIisnotjustdecipheringwhetherthecontentisAI-generatedbutalsoensuringlegalrecourseandaccount-abilityforindividualsandentitiesthatdistributeandcreatesuchcontent.However,authenticitysolutionssuchasprovenanceandwatermarkingcouldhelpdissuadeanddisincentivizemaliciousactorsfromusingtoolsthatadoptthesesafeguardsandhelpdebunkphotorealisticNCII,disinformation,andmuchmore.Framingthisissueasoneofcontentauthenticitycouldempowerusersandputtheonusofresponsibilityintermsofsecuringtransparencyandaccountabilityforsafetyissuesinimagegenerationmodelsontechnologyprovid-ersandgovernmententitiesthatcanenforceregulationstocombattheseharms.Authenticitysolutionssuchasprovenanceandwatermarkingcouldhelpdissuadeanddisincentivizemaliciousactors.determiningwhetheranimageisauthentic,despiteadver-sarialmotives.Auser-centricapproachtothisissueiskey,giventhatthesuccessofcontentauthenticityinitiativescanbeshapedbyuserexperiences,personalbias,and/orbeliefsabouttechnology.IndustrySolutionstoPreserveImageAuthenticityTheissueofpreservingauthenticityontheinternetisnotnew.Fakeaccounts,bots,phishingemails,andmorehavebeenpersistentissuesforyears.Challengeswithdeepfakesstartedtosparkmore-deliberateconversationsaboutphotoandvideoauthenticity;in2017,aReddituserexploitedGoogle’sopen-sourcedeep-learninglibrarytopostpor-nographicface-swapimages(Adee,2020).Now,thefighttopreserveauthenticityhasbecomeevenmorecrucial,givenalackofpolicysafeguardsandsufficientplatform-levelenforcement,aswellasthespeedatwhichAIimagegenerationisimproving.WhenexaminingtheissueofharmscausedbyAI-generatedimagesthroughthelensof“authenticity,”itisimportanttothinkthroughwhatkindsoftoolsandtechnologieswillbestsupportindividualsinWatermarking,Hashing,andDetectionTherehavebeenseveraldebatesabouttherightapproachtoauthenticity.Theconceptofwatermarking

hasdomi-natedtheconversationandisatermthattheWhiteHouseincludedinitsvoluntaryAIcommitmentsforindustry(WhiteHouse,2023a).However,thereisconfusionandalackofconsensusinthefieldaboutthedefinitionofawatermarkandhoweffectivewatermarkscanbe.AhelpfulwaytodefinewatermarkingisprovidedbythePartnershiponAI:aformofdisclosurethatcanbevisibleorinvis-ibletotheuserandincludes“modificationsintoapieceofcontentthatcanhelpsupportinterpretationsofhowthe7oughlyvettedhashesofconfirmedCSAM(NationalCenterforMissingandExploitedChildren,undated)andterroristcontent(GIFCT,undated),respectively.Hashingcanproveusefulintermsofsharingcontentamongsocialmediaplatformsthatveryclearlybelongsinspecificcategoriesofabuse,butitislesspracticalforuseacrossavarietyofcontent,giventhathashingoccursretroactivelyandcannotbedonewellatscale.Itisalsosusceptibletoadversarialattacksandisvulnerabletodatabaseintegrityissuesanddiscrepanciescausedbyhumanreviewinthecontentattri-butionprocess(Ofcom,2022).Last,anapproachthathasbeendiscussedforseveralyearsisdetection.Bothestablishedcompaniesandsmallerstartups—suchasIntel(Clayton,2023),Optic(Kovtun,2023),andRealityDefender(Wiggers,2023)—havepro-duceddeepfakedetectionsolutions.Thoughthetechnologycanbepromising,itcomeswithahostofissues.Tradition-ally,inthecybersecurityspace,detectionandevasionareacat-and-mousegame,withdetectionneedingtoconstantlyimproveasbothadversarialactorsandthetechnologyitselfimprove.RealityDefender’schiefexecutiveofficerclaimsthatprovenanceandwatermarkingsolutionsareweaker,giventhattheyrequirebuy-in,andthatRealityDefender’sproduct,whichisfocusedoninference(determiningtheprobabilityofsomethingbeingfake),isamorerobustsolution(Goode,2023).However,evenwithhighratesofefficacy,theonuswouldstillbeonuserstogaugehowmuchtheyshouldtrustapieceofcontentbasedonaprob-abilitymetricalone.Furthermore,currentimagedetectioncapabilitieshaveaccuracyissues,asreportedinaBellingcatinvestigation.BellingcatassessedatoolbyOptic(called“AIorNot”)anddeterminedthatitwassuccessfulinidentify-ingAI-generatedandrealimagesquiteaccurately,exceptDetectionandevasionareacat-and-mousegame,withdetectionneedingtoconstantlyimproveasbothadversarialactorsandthetechnologyitselfimprove.contentwasgeneratedand/oredited”(PartnershiponAI,2023).Watermarking(bothvisibleandinvisible/metadata-based)canbeausefuldisclosureforthegeneralpublicforcontentinterpretation,butitisfarfromaholisticsolution,giventheneedforwatermarkstoberobustagainstadver-sarialattacks,toovercomechallengestosecurewidespreadadoption,andtobeunderstandablebyaconsumeroruseracrossdifferentsocialplatformsandhardwaredevices.Anotherapproachinthefieldishashing,orfinger-printingimagecontent,whichhappensafteranimageiscreated.Cryptographichashingisusedtodetermineexactmatches,whereasperceptualhashingcanfindsimilarmatchesthatmaynotbeexactlythesameimage(Ofcom,2022).ThemeritsofhashinghavebeenparticularlyevidentintheidentificationofCSAMandterroristcontent.Forexample,theNationalCenterforMissingandExploitedChildrenandtheGlobalInternetForumtoCounterTer-rorism(GIFCT)arenonprofitorganizationsthatusehash-sharingplatformswithtechnologycompaniesofthor-8whenAI-generatedimageswerebothcompressedandpho-torealistic,inwhichcaseaccuracydroppedsignificantly(Kovtun,2023).Imagecompression(arelativelycommonpractice)canassistmalignactorsinevadingdetection,especiallyonsocialmediaplatforms,whichgenerallycom-pressalluploadedimages.Detectiontoolsarenotfoolproofandarefragiletominorperturbations.eration(Zhang,Chapman,andLeFevre,2009).InanAIimagegenerationcontext,provenancecouldmaptheoriginofanimagethroughacryptographichashorsignaturethatisappliedandattachedtothecontent,isstoredsecurelythroughencryption,andis“tamper-evident”orisabletoshowwhethertheimagehasbeenalteredinanyway.Meta-datainformation,availabletousersthroughlabels,couldalsohelpdefinethetrustworthinessofanimage.Inprac-tice,makingprovenanceasuccessgoesfarbeyondacryp-tographicsignatureandrequiresthewidespreadadoptionofoneormanyinteroperableframeworksacrossdifferentmediums:hardware(e.g.,cameras,smartphones),editingsoftware(e.g.,photoeditingprograms,face-swapapps),andpublishingandsharingentities(e.g.,newsmedia,socialmediaplatforms),whichisaverychallengingtaskinpractice.Theindustryecosystemhasralliedaroundtheappli-cationofprovenancetosupportcontentauthenticityeco-systemsforAI-generatedimages.IndustryleadersincludeAdobe,Intel,Microsoft,andTruepic,allofwhicharemembersoftheContentAuthenticityInitiative(CAI)andtheCoalitionforContentProvenanceandAuthenticity(C2PA).Bothgroupsarefocusedoncross-industrypar-ticipationtotackletheissuesofmediatransparencyandcontentprovenance,withtheC2PAframeworkunderlyingmanyoftheseinitiatives,andhavereleasedproductsthatusetheC2PAframework.Forexample,Adobelaunchedits“ContentCredentials”featurein2023,whichusestheC2PAstandardtoallowtheattachmentofsecure,tamper-evidentmetadataonanexportordownload(Quach,2023).TheC2PAframeworkisaninteroperablespecificationthat“enablestheauthorsofprovenancedatatosecurelybindstatementsofprovenancedatatoinstancesofcontentusingProvenanceThoughnoindividualsolutiontocontentauthenticityisholistic,provenance

isemergingasausefultooltopro-activelypreserveoriginmetadataand/oranyeditingorchangestoagivenpieceofcontent.Detectionmethodsriskbeinglessusefulastechnologycontinuestoimproveandevolvewhileplacingtheonusofusingdetectiontoolsonausereverytimetheycomeacrosscontentthattheydeemtobesuspicious.Furthermore,suchmethodscanlackaccu-racy,furtherobfuscatingthedecisionmakingprocessforanindividualtoassesscontentforitsauthenticity.Prove-nanceapproaches,suchasestablishingtheoriginofapieceofcontentthroughsecuredmetadata,aregenerallymorerobust,giventhattheyfocusontheoriginofcontentratherthanprovingwhethersomethingisrealorfake.Further-more,whenimplementedwell,provenancesolutionscanbeincorporatedacrossthecontentsupplychain—inAIimagegenerationtools,socialmediaplatforms,newssites,andmore—sothatmetadatainformationisreadilyavailabletoauserandcancomplementandbeusedintandemwithwatermarkingandfingerprintinginitiatives.Examiningtheoriginalapproachesto