CHECKPOINTRESEARCH

AISECURITYREPORT

CHECKPOINTAIREPORT?1STANNUALEDITION

TABLEOFCONTENTS

02

01

04

AIFOR

ENTERPRISES

05

SECURITYFOR,

BY&WITHAI

03

AITHREATS

INTRODUCTION

AIFORRESEARCH

INTRODUCTION

CHECKPoiNTTM

01INTRODUCTION

02AITHREATS

AIMODELSINTHEDARKWEB

THENEWSOCIALENGINEERING

TARGETINGLLMACCOUNTS

AIFORMALWARE

03AIFORRESEARCH

AIFORAPTHUNTING

AIVULNERABILITYRESEARCH

04AIFORENTERPRISES

05SECURITYFOR,BY,&WITHAI

CP<>

CHECKPOINTRESEARCH

4AISECURITYREPORT2025

INTRODUCTION

TheAcceleratingFutureofAIforCyberOffendersandDefenders

AIisrevolutionizingindustries,andcybercrimeandcybersecurityarenodifferent.AdoptingAIinenterprises—andunfortunatelybythreatactorsaswell—enhancesefficiency,scale,andimpact.Atthispointintime,webelieveit’sessentialtopauseandassessthecurrentstate

andfutureofAIandcybersecurity.

HowareattackersusingAI,andwhatcomesnext?Ascyberdefenders,howcanweleverageAItoenhanceoursecurityeffortsandprotectourorganizationsmoreeffectively?ThesearethequestionsaddressedinthefirsteditionoftheCheckPointResearchAISecurityReport.

Ourfocuszeroesinon:

?Theriseofautonomousandinteractivesocialengineeringacrosstext,audio,andvideo

?ThejailbreakingandweaponizationofLLMs

?Theautomationofmalwaredevelopmentanddatamining

?AIadoptioninenterprisesandtheirassociatedrisks

?Theemergenceofdatapoisoninginthewildandlarge-scaledisinformationamplifiedbyGenAItools

?TheAItoolsthatfightfirewithfire-protectingyourorganizationfromthemostadvancedthreats

AIthreatsarenolongertheoretical—they'rehereandevolvingrapidly.AsaccesstoAItoolsbecomesmorewidespread,threatactorsexploitthisshiftintwokeyways:byleveragingAIto

enhancetheircapabilitiesandtargetingorganizationsandindividualsadoptingAItechnologies.

Thefollowingpagesprovideacomprehensiveunderstandingofthesethreats,allowingreaderstonavigatetheintricate

landscapeofAIsecurity.

Toasecurefutureofinnovationandsuccess,

LotemFinkelstein,

DirectorofCheckPointResearch

AITHREATS

02

6AISECURITYREPORT2025

CHECKPoiNTTM

01INTRODUCTION

02AITHREATS

THEAIMODELSUSEDBYCYBERCRIMINALS

CybercriminalsarecloselymonitoringtrendsinmainstreamAI

AIMODELSINTHEDARKWEB

adoption.Wheneveranewlargelanguagemodel(LLM)isreleasedtothepublic,undergroundactorsquicklytestitspotentialfor

misuse(figure1).Currently,ChatGPTandOpenAI’sAPIarethe

mostpopularmodelsforcybercriminals,whileotherslikeGoogleGemini,MicrosoftCopilot,andAnthropicClaudearequickly

gainingpopularity.Thelandscapeischangingwiththelaunchofopen-sourcemodelslikeDeepSeekandQwenbyAlibaba.Thesemodelsenhanceaccessibility,haveminimalusagerestrictions,andareavailableinfreetiers,makingthemakeyassettocrime.

THENEWSOCIALENGINEERING

TARGETINGLLMACCOUNTS

AIFORMALWARE

03AIFORRESEARCH

AIFORAPTHUNTING

AIVULNERABILITYRESEARCH

04AIFORENTERPRISES

05SECURITYFOR,BY,&WITHAI

CP<>

CHECKPOINTRESEARCH

Figure1–UndergroundforumdiscussiononharnessingDeepSeekformalwaredevelopment

7AISECURITYREPORT2025

CHECKPoiNTTM

01INTRODUCTION

TheDevelopmentofMaliciousAIModels

02AITHREATS

Cybercriminalsareexploiting

mainstreamplatformsandcreating

AIMODELSINTHEDARKWEB

THENEWSOCIALENGINEERING

TARGETINGLLMACCOUNTS

AIFORMALWARE

andsellingspecializedmaliciousLLMmodelsexplicitlytailoredforcyber

crime(figure2).ThesedarkLLM

modelsaredesignedtocircumvent

thesafeguardsestablishedforethicalmodelsandareactivelymarketedashackingtools.

03AIFORRESEARCH

AIFORAPTHUNTING

AIVULNERABILITYRESEARCH

Figure2–OnionGPT,anexampleofadarkAImodelcreatedinTor

04AIFORENTERPRISES

05SECURITYFOR,BY,&WITHAI

CP<>

CHECKPOINTRESEARCH

8AISECURITYREPORT2025

CHECKPoiNTTM

ThenotoriousAImodelWormGPT

01INTRODUCTION

02AITHREATS

wascreatedbyjailbreakingChatGPT(figure3).Marketedasthe“ultimatehackingAI,”itcangeneratephishingemails,writemalware,andcraft

socialengineeringscriptswithout

ethicalconstraints.ATelegram

AIMODELSINTHEDARKWEB

THENEWSOCIALENGINEERING

TARGETINGLLMACCOUNTS

AIFORMALWARE

channelpromotesitsuseforfraud,

botnetcreation,andcyberintrusion,offeringsubscriptionshighlightingthecommercializationofdarkAI.

03AIFORRESEARCH

AIFORAPTHUNTING

AIVULNERABILITYRESEARCH

04AIFORENTERPRISES

05SECURITYFOR,BY,&WITHAI

Figure3–MaliciousAIserviceWormGPTadvertisedonaTelegramchannel

CP<>

CHECKPOINTRESEARCH

9AISECURITYREPORT2025

CHECKPoiNTTM

01INTRODUCTION

02AITHREATS

AIMODELSINTHEDARKWEB

THENEWSOCIALENGINEERING

TARGETINGLLMACCOUNTS

AIFORMALWARE

03AIFORRESEARCH

AIFORAPTHUNTING

AIVULNERABILITYRESEARCH

Figure4-HackerGPTcapabilitiespublishedonacybercrimeforum

04AIFORENTERPRISES

05SECURITYFOR,BY,&WITHAI

whileothersmodifyopen-sourcemodels.AsmainstreamAImodelsevolve,sodotheirdarkcounterparts.

AnewwaveofdarkAImodels,suchasGhostGPT,FraudGPT,

andHackerGPT(figure4),servespecificaspectsofcybercrime.SomemodelswraparoundmainstreamAIwithjailbreaks,

CP<>

CHECKPOINTRESEARCH

CHECKPoiNTTM

TheRiseofFakeAIPlatforms

01INTRODUCTION

ThedemandforAI-basedserviceshas

02AITHREATS

ledtotheemergenceoffakeAIplatformsdesignedtodeceiveusersanddistributemalware,stealsensitivedata,orenablefinancialfraud.Examplesinclude

HackerGPTLite(figure5),whichseemstobeanAItoolbutissuspectedtobe

aphishingwebsiteandfakeDeepSeekdownloadsites,which,inreality,

distributemalware.

AIMODELSINTHEDARKWEB

THENEWSOCIALENGINEERING

TARGETINGLLMACCOUNTS

AIFORMALWARE

03AIFORRESEARCH

Inone

case

,amaliciousdistributed

ChromeextensionimitatingChatGPT

wasdesignedtostealusercredentials.Onceinstalled,thisharmfulextensionhijackedFacebooksessioncookies,

grantingattackerscompleteaccesstovictims'accountsandallowingthemtooperateremotely.

Figure5-AsuspectedphishingAI-servicewebsitetargetingcybercriminals

AIFORAPTHUNTING

AIVULNERABILITYRESEARCH

04AIFORENTERPRISES

05SECURITYFOR,BY,&WITHAI

CP<>

CHECKPOINTRESEARCH

10AISECURITYREPORT2025

11AISECURITYREPORT2025

CHECKPoiNTTM

AI-POWEREDSOCIALENGINEERING:ANEWERA

02AITHREATS

01INTRODUCTION

Socialengineering—manipulatingindividualsintoactions

theywouldn’ttypicallytake—iscentraltomanycyberattacks.

AIMODELSINTHEDARKWEB

THENEWSOCIALENGINEERING

TARGETINGLLMACCOUNTS

AIFORMALWARE

Evenattacksthatexploitpurelytechnicalvulnerabilitiesoften

beginwithasocialengineeringscheme.Attackersleverage

text,audio,andimagerytoconvincinglyimpersonatespecific

individualsorgeneratehumanvoices,fosteringtrusttodeceivetheirtargets.WithrecentadvancementsinAI,attackerscan

03AIFORRESEARCH

createauthentic-lookingmaterialsatscale,conductautomatedchats,andholdreal-timeaudioandvideoconferenceswhile

impersonatingothers.

AIFORAPTHUNTING

AIVULNERABILITYRESEARCH

AstheseAI-driventoolsproliferateoncriminalforumsand

incidentsrise,ourrelianceonaudioandvisualcuesforidentityconfirmationiscriticallycompromised.Fullyautonomous

04AIFORENTERPRISES

audiodeepfaketoolsforlarge-scalephonescamsarealreadyavailable,meaningthatrecognizingafamiliarfaceorvoice

05SECURITYFOR,BY,&WITHAI

isnolongersufficientproofofidentity;instead,interactionsmustbereinforcedbyadditionalauthenticationmeasures.

AI-drivensocialengineeringisalreadyinfluencingreal-worldcybercrime.TheFBIrecently

warned

thepublicthatcybercriminals

increasinglyleverageAI-generatedtext,images,audio,and

CP<>

CHECKPOINTRESEARCH

videotoenhancetheirattacks.Thisdemonstratesthatattackers

nowpossesssophisticatedcapabilitiespreviouslyunavailable,significantlyboostingtheeffectivenessofdeceptionandfraud.

Onlinefraudreliesonbothqualityandquantity.Evenpoorly

phrasedscams,likesextortionemails—canbeprofitablewhen

senttomillionsofpotentialvictims,eveniftheysucceedwith

onlyasmallpercentage.Attackers,therefore,aimtoenhance

boththequalityoftheirimpersonationsandtheautomationleveloftheiroperations,maximizingimpactwhileminimizingcostly

humanresources.AIfacilitatesadvancementsintheseareas,

creatinghighlyconvincingtext,audio,andvideoinmultiple

languagesandenablinginteractivechatbots,resultingin

automatedagentscapableofengagingmaliciouslywithvictims.

ADeepDiveintoDeepfake

TechnologiesandTheirExploitation

Thefollowingsectionswillexaminetheexploitationscenariosinvolvingeachmediatype(text,audio,images,andvideo),detailtherelatedservicesactivelyadvertisedwithincriminalforums,andanalyzereportsfromactualincidentstoillustratethe

practicalimplicationsoftheseevolvingthreats.

12AISECURITYREPORT2025

CHECKPoiNTTM

TheMaturityLevelsofDeepfakeAutomation

01INTRODUCTION

GenerativeAItechnologiesspanaspectrumofsophisticationandmaturity,rangingfrombasicofflinegenerationoftext,images,

andvideostoadvancedonlinemanipulationrequiringrealindividuals’involvement,suchasface-swappingorvoiceimitation.Atthehighestlevel,fullyautonomous,real-timegenerationproducesconvincingcontentinstantly,dynamicallyrespondingtounsuspectingindividualsduringinteractions.

02AITHREATS

AIMODELSINTHEDARKWEB

THENEWSOCIALENGINEERING

MEDIATYPE

OFFLINEGENERATION

REALTIMEGENERATION

FULLYAUTONOMOUS

TARGETINGLLMACCOUNTS

AIFORMALWARE

03AIFORRESEARCH

AIFORAPTHUNTING

AIVULNERABILITYRESEARCH

TEXT

04AIFORENTERPRISES

AUDIO

05SECURITYFOR,BY,&WITHAI

VIDEO

Pre-rendered

scriptsoremails

Pre-recorded

impersonations

Pre-created

deepfakevideos

Real-timegeneratedresponses

Real-timevoicemanipulation

Liveface-swappingorvideoalteration

AI-generated,fullyinteractiveconversations

FullyAI-driven

conversationalaudio

Completelyautomated,AI-generated

interactivevideo

CP<>

CHECKPOINTRESEARCH

(RedVmarksmaturitylevelalreadyavailableinmarketsandexploitedinthewild)

CHECKPoiNTTM

AI-GeneratedTextualSocialEngineering

01INTRODUCTION

TheavailabilityofChatGPTandotherLargeLanguageModel

02AITHREATS

(LLM)chatbotssince2022hasloweredbarrierstogeneratingconvincingtext,enhancingthequalityofphishingemails.

Attackers,whooftencomefromdifferentlinguisticandculturalbackgroundsfromtheirvictims,previouslyfacedsignificant

AIMODELSINTHEDARKWEB

languagebarriers.However,LLMtechnologynowallows

THENEWSOCIALENGINEERING

TARGETINGLLMACCOUNTS

AIFORMALWARE

attackerstocraftmessageseffortlesslywithnative-likefluencyandculturalnuances.

LLMtechnology

nowallows

attackersto

craftmessages

effortlesslywith

native-likefluency

andcultural

nuances.

Inarecentcase,CheckPointHarmonyEmail&Collaborationblockedasextortioncampaignthatuseddiversetextual

03AIFORRESEARCH

phrasingtoavoiddetection.Eachemailinthethousandsof

AIFORAPTHUNTING

AIVULNERABILITYRESEARCH

messagesuniquelyrewordedtheurgencyof"Timeisrunningout,"usingexpressionslike"Thehourglassisnearlyempty

foryou"or"You'reapproachingtheendofyourtime."Sincesextortioncampaignstypicallydonotcontaintraditional

IndicationsofCompromise(IoCs)likemaliciousURLsor

04AIFORENTERPRISES

attachments,apartfromcryptocurrencywalletaddresses,

detectionreliesheavilyontextanalysis,furthercomplicatingdefensemeasures.

05SECURITYFOR,BY,&WITHAI

CP<>

CHECKPOINTRESEARCH

13AISECURITYREPORT2025

14AISECURITYREPORT2025

CHECKPoiNTTM

AreviewofDarkwebforumsreveals

01INTRODUCTION

variousAI-assistedtoolsdesigned

specificallytostreamlinethecreation

02AITHREATS

andmanagementofphishingandspamcampaigns(figure6).Forexample,

GoMailPro(figure8),pricedat$500permonth,integratedChatGPTin

AIMODELSINTHEDARKWEB

THENEWSOCIALENGINEERING

TARGETINGLLMACCOUNTS

AIFORMALWARE

2023toautomatethegenerationof

spamandphishingemails.Recent

2024updates,includingcapabilitiesto

recoverblockedemailaccountsusedforspamdistribution,furtherenhancedthesolution.

Figure6–AdvertisementofanAI-assistedSpamAgent

03AIFORRESEARCH

AIFORAPTHUNTING

AIVULNERABILITYRESEARCH

04AIFORENTERPRISES

05SECURITYFOR,BY,&WITHAI

CP<>

CHECKPOINTRESEARCH

Figure7–AdvertisementofanAI-assistedSpamAgent

15AISECURITYREPORT2025

CHECKPoiNTTM

AnotherexampleofanAItextual

01INTRODUCTION

applicationisthe"BusinessInvoice

Swapper”(figure8)developedbythecybercriminalgroupGXCTeam.Itis

02AITHREATS

designedtofacilitateBusinessEmail

Compromise(BEC)byautomatically

AIMODELSINTHEDARKWEB

THENEWSOCIALENGINEERING

TARGETINGLLMACCOUNTS

AIFORMALWARE

scanningcompromisedemailaccountsforinvoicesorpaymentinstructions.

Italtersbankingdetailstoredirect

fundstoattacker-controlledaccounts.

LeveragingAI,itseamlesslyovercomeslanguagebarriers,manageslargedatavolumesefficiently,andautomates

03AIFORRESEARCH

distribution,enhancingthescalabilityandimpactoffraudulentemailattacks.

AIFORAPTHUNTING

AIVULNERABILITYRESEARCH

Figure8–Advertisementofthe“BusinessInvoiceSwapper”inadarkwebforum

04AIFORENTERPRISES

05SECURITYFOR,BY,&WITHAI

CP<>

CHECKPOINTRESEARCH

CHECKPoiNTTM

AI-GeneratedAudioDeepFakes

CybercriminalsincreasinglyemployAI-generatedaudio,or"audiodeepfakes,"toexecutesophisticatedimpersonationscams.Thistechnologyproduceshighlyrealisticreplicasofindividuals'voices,enhancingscammers'abilitytodeceivevictims.Voicesamplesonsocialmedia—fromcelebritiestoeverydayusers—provideampleresourcesforattackers.

PopularplatformslikeElevenLabsandtoolsliketheopen-

sourceRetrieval-basedVoiceConversion(

RVC

)algorithm

canproduceconvincingaudiousingjusttenminutesofvoicesamples.Thesetechnologieshavebeenusedinextortion

caseswherecriminalsfalselyclaimthatafamilymemberhasbeen

kidnapped

orinan

emergency

andaskforurgentmoneytransfers.

Arecent

case

inItalyreportedlyinvolvedscammersemployingliveAI-assistedaudiodeepfaketechnologytoconvincingly

impersonatethevoiceofdefenseministerGuidoCrosetto.The

attackersaimedtoextortmoneyfromhisaffluentcontactsby

falselyclaimingthefundswereneededforhostagerelease.

Severalhigh-profileindividuals,includingdesignerGiorgio

Armani,weretargeted.Atleastonevictimwhoknewtheministerwasdeceivedandtransferredasignificantsum.

TheX137Telegrammanagementconsole,advertisedon

01INTRODUCTION

Darkwebforums,isanexampleofafullyautonomousAI-basedtextualinteractiveagent.Thistoolautomatestaskswithin

02AITHREATS

text-basedplatforms,simultaneouslyconductingreal-timeconversationswithmultipleusersaccordingtodesignatedtasks.UsingGeminiAI,X137monitors,summarizes,and

AIMODELSINTHEDARKWEB

engagesinTelegramcommunicationswithuncensored,hacking-relatedinsights.

THENEWSOCIALENGINEERING

TARGETINGLLMACCOUNTS

AIFORMALWARE

TheprimarycontributionoftheseAI-driventoolsistheirability

toscalecriminaloperations,overcomingpreviousbottlenecks

associatedwithemployinglinguisticallyandculturallyproficientmanpower.AI-generatedtextenablescybercriminalsto

03AIFORRESEARCH

overcomelanguageandculturalbarriers,significantlyenhancingtheirabilitytoexecutesophisticatedreal-timeandoffline

AIFORAPTHUNTING

AIVULNERABILITYRESEARCH

communicationattacks.UncensoredLLM-basedchatbotscanconvincinglyandeffectivelymanagemultiplecommunicationthreadssimultaneously.

04AIFORENTERPRISES

Inadditiontofinanciallymotivatedcriminals,nation-state

actorsarealsoincreasinglyleveraginggenerativeAIto

enhancesocialengineeringschemes.Google

reports

that

05SECURITYFOR,BY,&WITHAI

Iranian,Russian,andChineseAPTandinformationoperationsactorshaveusedAItoolslikeGeminiforcontentcreation,

localization,andpersonadevelopment.OpenAI’s

report

similarlyfoundtheuseofthesecapabilitiesininfluence

CP<>

CHECKPOINTRESEARCH

operations,streamliningphishing,influencecampaigns,andreconnaissance.

16AISECURITYREPORT2025

17AISECURITYREPORT2025

CHECKPoiNTTM

01INTRODUCTION

02AITHREATS

AIMODELSINTHEDARKWEB

THENEWSOCIALENGINEERING

TARGETINGLLMACCOUNTS

AIFORMALWARE

03AIFORRESEARCH

Figure9-RecruitmentofAIdevelopersfortelephonysystem

AIFORAPTHUNTING

AIVULNERABILITYRESEARCH

Recent

research

showsthatpeoplecannolongerreliablydistinguishbetweengenuinevoicesandAI-generatedaudio.Discussionsoncriminalforumsincreasinglyfocuson

04AIFORENTERPRISES

criminaltelephonysystems(figure9).AdvertisementsondarkwebforumsexplicitlyseekAIdeveloperstoimplementAI-drivencapabilitiesintophone-basedscams.

05SECURITYFOR,BY,&WITHAI

integratingAI-generatedtextandaudiointocomprehensive

CP<>

CHECKPOINTRESEARCH

18AISECURITYREPORT2025

CHECKPoiNTTM

01INTRODUCTION

02AITHREATS

AIMODELSINTHEDARKWEB

THENEWSOCIALENGINEERING

TARGETINGLLMACCOUNTS

AIFORMALWARE

03AIFORRESEARCH

Figure10-AdpresentingthecapabilitiesofanAI-enhancedtelephonysystem

AIFORAPTHUNTING

AIVULNERABILITYRESEARCH

04AIFORENTERPRISES

TheseAI-basedcallsystemsarealreadyavailableforpurchase

andareprimarilyusedas

OTPbots

.Thebotscallpotentialvictimsandfollowpredeterminedscriptstoobtainone-timepassword,

05SECURITYFOR,BY,&WITHAI

CP<>

CHECKPOINTRESEARCH

mostlytofinancialservicesaccounts..Moreadvancedplatformsnowprovideflexibleconversationalstructures,adaptingscenariosinrealtimebyanalyzingvictimresponsesdynamically.

OneservicelaunchedinJanuary2025highlightshowthese

systemscanseamlesslymanagenumerouslanguagesand

handlemultiplesimultaneousinteractions,significantlyenhancingscalabilitycomparedtotraditionalphonescamsthat

rely

extensivelyonskilledhumanlabor(figure10).Inaconversationwithsuchaseller,theyexplained,“Wemakeaskeletonof

aspeech,accordingtowhichtheAIwillguidetheclient,”

emphasizingitscapabilitytogooff-scriptandmanageunexpectedscenariosacross"anytopic,anyfield,anylanguage,anyvoice.”

19AISECURITYREPORT2025

CHECKPoiNTTM

SampleconversationswereprovidedinRussianandSpanish,

01INTRODUCTION

showcasingthesystem'smultilingualproficiency(figure11).TheseadvancedAItelephonyplatformscostabout$20,000,including

02AITHREATS

trainingandsupport,orarebilledatapproximately$500baserateplus$1.50perminute,dramaticallyreducingtheneedformany

qualifiedhumanoperators.

AIMODELSINTHEDARKWEB

THENEWSOCIALENGINEERING

TARGETINGLLMACCOUNTS

AIFORMALWARE

03AIFORRESEARCH

AIFORAPTHUNTING

AIVULNERABILITYRESEARCH

04AIFORENTERPRISES

05SECURITYFOR,BY,&WITHAI

Figure11-Conversationwithtelephonysystemseller

CP<>

CHECKPOINTRESEARCH

20AISECURITYREPORT2025

CHECKPoiNTTM

01INTRODUCTION

AI-GeneratedVisualDeepFakes

02AITHREATS

Criminalforumsshowthegrowing

useofAI-generatedimagestobypassKnowYourCustomer(KYC)identity

AIMODELSINTHEDARKWEB

THENEWSOCIALENGINEERING

TARGETINGLLMACCOUNTS

AIFORMALWARE

verification.BasicAI-drivenservices

offertheabilitytocreateconvincing

identitiestoregisternewaccounts

fraudulently,unlockfrozenaccounts,

orhijacklegitimateaccountsbyforginguseridentities.Pricestypicallystart

03AIFORRESEARCH

around$70forsimpleAI-generated

images.Moresophisticatedcriminal

servicestargetingmajorKYCproviders

AIFORAPTHUNTING

AIVULNERABILITYRESEARCH

-suchasONFIDO,SUMSUB,and

JUMIO-commandhigherfeesorevendemandapercentageofthefundsfromhijackedaccounts(figure12).

04AIFORENTERPRISES

05SECURITYFOR,BY,&WITHAI

CP<>

CHECKPOINTRESEARCH

Figure12-Accountverificationandunlockingserviceadvertisement

21AISECURITYREPORT2025

CHECKPoiNTTM

Criminalserviceproviderstypically

01INTRODUCTION

receivetheverificationlinkanddirectlycompletetheidentityverification

(figure13).Pricingvariesbyregion,

02AITHREATS

withEuropeanandCIScountriespayingaround$350andservicesfortheUS

AIMODELSINTHEDARKWEB

THENEWSOCIALENGINEERING

TARGETINGLLMACCOUNTS

AIFORMALWARE

andCanadareachingupto$500,especiallyforfalsifieddocuments.

Trustinganonymouscriminalswithaccesstofrozenaccountscarries

significantriskforclients;such

transactionsarefeasibleonlydueto

03AIFORRESEARCH

establishedreputationmechanismsandcomprehensivemitigationprocedures

withintheseillicitmarketplaces.

AIFORAPTHUNTING

AIVULNERABILITYRESEARCH

04AIFORENTERPRISES

05SECURITYFOR,BY,&WITHAI

Figure13-CybercrimeservicethatoffersKYCverification.

CP<>

CHECKPOINTRESEARCH

22AISECURITYREPORT2025

CHECKPoiNTTM

Pre-recordedDeepfakeVideos

AI-generatedvideodeepfakesareincreasinglyexploitedfor

fraud,primarilythroughfaceandaudioswappinginpre-

recordedvideos.Thesevideosoftenfalselydepictwell-knownindividualsendorsingscams,includinginvestmentfraud.Thetechnicalbarrierislowerforpre-recordeddeepfakevideos,

makingtheseserviceswidelyaccessibleincriminalforums.

Pricesrangefromafewhundredtoseveralthousanddollars,dependingonvideolengthandquality.Arecent

operation

inTbilisi,Georgia,useddeepfakevideosfeaturingpublicfiguressuchasBenFogleandMartinLewistopromotefraudulent

cryptocurrencyinvestments,deceivingover6,000victimsin

theUKandCanadaandresultingin$35millioninlosses.

Beyondfinancialfraud,AI-fabricatedvideoshavebeenwidely

deployed

inpoliticalinfluencecampaignsandelection-relateddisinformationeffortsworldwide.

Real-timeVideoManipulation

01INTRODUCTION

Whilepre-recordeddeepfakevideosarecommon,real-time

videomanipulationpresentsamoreadvancedchallenge.

02AITHREATS

Thoughhigh-endAIvideogeneratorslikeOpenAI’sSoraremainrestrictiveanddonotpermitreal-timeintegration,lower-

AIMODELSINTHEDARKWEB

THENEWSOCIALENGINEERING

TARGETINGLLMACCOUNTS

AIFORMALWARE

resolutionreal-timeface-swappingtoolsarealreadyaccessibleandinactiveuse.Paidservicesandtoolsofferedoncriminal

forumsandopen-sourcesolutionsthatrequirearelativelylowhardwareinvestmentmakereal-timedeepfakeattacksincreasinglyavailable.

Theimpactoftheseadvancementsisalreadyevidentinreal-

03AIFORRESEARCH

AIFORAPTHUNTING

AIVULNERABILITYRESEARCH

worldfraudcases.Inearly2024,BritishengineeringfirmArupp

suffered

a￡20millionlossaftercybercriminalsuseddeepfakevideotechnologytoimpersonateseniorexecutivesduringa

livevideocall,convincinganemployeetotransferfundstofraudulentaccounts.

04AIFORENTERPRISES

05SECURITYFOR,BY,&WITHAI

CP<>

CHECKPOINTRESEARCH

23AISECURITYREPORT2025

CHECKPoiNTTM

Inone

case

,pornographicmaterialsandAI-basedaudio-videotoolswereused

toimpersonateapornstar.Throughlivechatinteractions,dozensofmenwere

coercedintocommittingvarioussexualcrimes.Recordingswerelaterusedforpornographicdistribution.Theattackerandatleastfourofhiscontactshave

beendetained.

01INTRODUCTION

02AITHREATS

AIMODELSINTHEDARKWEB

THENEWSOCIALENGINEERING

TARGETINGLLMACCOUNTS

AIFORMALWARE

Inanotherrecentcase,AUS-based

engineerreportedanidentitytheft

attemptusingAI-generatedface-

03AIFORRESEARCH

swappingtechnologyduringanonline

technicalinterview(figure14).While

thismayhavebeenanisolatedcase

AIFORAPTHUNTING

AIVULNERABILITYRESEARCH

ofindividualfraud,growing

evidence

suggestsbroadercampaignsare

linkedtostate-sponsoredespionage

orfinanciallymotivatedoperations.As

04AIFORENTERPRISES

real-timedeepfaketechnologybecomesincreasinglyaccessible,suchfraudulent

Figure14–Identitytheftattemptbylivefaceswapduringinterview

attemptsareexpectedtoescalate.

05SECURITYFOR,BY,&WITHAI

CP<>

CHECKPOINTRESEARCH

CHECKPoiNTTM

01INTRODUCTION

02AITHREATS

TARGETINGOFLLMACCOUNTS

AsthepopularityofgenerativeAIplatformscontinuestorise,sodoestheirvalueinthecybercriminalunderground.Accessto

AIMODELSINTHEDARKWEB

THENEWSOCIALENGINEERING

LLMservicesenablesattackerstouseAIformaliciouspurposesandrepresentsatradablecommodity.Asaresult,LLMaccountshavebecomeasignificanttargetforcybercriminals.

Cybercriminalshaveestablishe