1

?2024CommonSenseMedia.Allrightsreserved.CommonSenseandotherassociatednamesandlogosaretrademarksofCommonSenseMedia,a501(c)(3)nonprofitorganization(FEIN:41-2024986).

PAGE

10

GenerativeAI

inK–12Education:

ChallengesandOpportunities

UpdatedAugust2024

TableofContents

Introduction

HowtoreadthispaperBackgroundResponsibleAI

Education-specificconsiderations

ExamplesofprinciplesandguidelinesforresponsibleAICautionarynotes

HowAmericansviewAIGeneralattitudesAwarenessandknowledgeUsage

AIliteracyamongteachersThestateofschoolpolicy

PolicyapproachesintheU.S.,acrosstheAtlantic,andbeyondUnitedStates

EuropeanUnion

UnitedKingdomandAustraliaGovernancemodels

Opportunities

Student-supportingAdaptivelearningCreativity

Project-basedlearningandcollaborationTeacher-supporting

LessonplanningandcontentdifferentiationCoachingandfeedback

GradingandassessmentsSystem-supporting

DatainteroperabilityAdministrationandlogisticsParentengagement

SeckingerHigh:AllInonAIRisks,harms,andchallenges

InaccuraciesandhallucinationsBiasandinequity

LackofrepresentationInformationcensorshipCheatingandplagiarism

OverrelianceontechnologyandlossofcriticalthinkingDatasecurity

ConsentandprivacyHumanreplacement

WhatstudentsthinkRecommendationsConclusionAcknowledgmentsAPPENDIX

Resources,links,andinformation

Educatorsnapshot:howonesocialstudiesteacherisnavigatingAIEducatorsnapshot:howonefourthgradeteacherusesKyronLearningEducatorsnapshot:howonefourthgrademathteacherusesTeachFX

About

Introduction

Lessthantwoyearsago,generativeAImovedfromadevelopingtechnologystillmostlyconfinedtoresearchpurposesandacademicdebatetowidespreadusage.OpenAIpubliclyreleasedtheirtext-to-imagemodelDALL-E2inSeptember2022,1followedbytheirAItextchatbotChatGPTthatNovember.2ByJanuary2023,

ChatGPThadclocked100millionusers—thefastest-growingconsumerapplicationever.3Ayearlater,userscouldchoosefromseveralchatbotcompetitors,includingGoogleGeminiandAnthropic'sClaude,andthousandsofapplicationsbuiltonthesetoolshadfloodedthemarket.

Accordingtocountlessheadlines,theadventofgenerativeAImarkedthebeginningofa"revolution"4acrosseveryimaginablesectorofbusinessandsociety,includingbanking,manufacturing,healthcare,law,entertainment,retail,design,and,ofcourse,education.Butheadlinesareheadlines,andwe'vebeenthroughtechnologicalrevolutionsbefore(thinkautomobiles,television,nuclearpower,computing,andtheinternetinthelastcentury,tosaynothingofindustry,railways,andelectricityinthemoredistantpast).WhatmakesAIdifferent?Isthisrevolutionthemotherofalltechnologicalrevolutions?Howdowekeepupwith,andmakesenseof,thelatestdevelopments?

Inresearchingandreportingthispaper,wespoketomorethan40peopleaboutAI,runningthegamutfromexpertPhDstoeighthgraders.WealsodrewonthecollectivewisdomofCommonSense,whichhasbuiltawellspringofexpertiseandknowledgeongenerativeAIoverthelastfewyearsacrossitseducation,policy,research,advocacy,andgrowthteams.We'vetriedtobringtogetherpeoplewhodon'tnormallyinteractwitheachotherinordertostitchdisparateviewpointsintomoreofaquilt.Acrossthatwidespectrumofvoices,weheardasimilarlywiderangeofreactions—fear,consternation,excitement,anxiety,andoptimism—sometimessimultaneously.Noneofthosereactionsarewrong.AlmosttwoyearsafterChatGPT'srelease,welackclearguidelines,guardrails,orpolicy,evenasweentertaingreathopesforthefuture.

OuraimistoprovideanoverviewofgenerativeAIinK–12education,includingbackgroundonthetechnology;thestateofAIliteracyandschoolguidelines;opportunities;risks;andrecommendationsforthefield.

1OpenAI.(2022,September28).DALL·Enowavailablewithoutwaitlist.

/blog/dall-e-now-available-without-waitlist

2OpenAI.(2022,November30).IntroducingChatGPT.

/blog/chatgpt

3Hu,K.(2023,February2).ChatGPTsetsrecordforfastest-growinguserbase–analystnote.Reuters.

/technology/chatgpt-sets-record-fastest-growing-user-base-analyst-note-2023-02-01/

4Thorbecke,C.(2023,November30).AyearafterChatGPT'srelease,theAIrevolutionisjustbeginning.CNN.

/2023/11/30/tech/chatgpt-openai-revolution-one-year/index.html

;Candelon,F.,Burtsev,M.,Jha,G.,Sack,D.,Zhukov,L.,&ZuluagaMartínez,D.(2023,December1).ChatGPT'sAIrevolution:Howmanagementcanadapt.Fortune.

/2023/12/01/chatgpt-ai-revolution-management-bcg/

;Rosenbaum,E.MITdatashow'industrialrevolution-level'leapforworkersusingAI.(2023,May10).CNBC.

/2023/05/10/mit-data-show-industrial-revolution-level-leap-for-workers-using-ai.html

;Kessler,S.(2023,June10).TheA.I.revolutionwillchangework.Nobodyagreeshow.NewYorkTimes.

/2023/06/10/business/ai-jobs-work.html

;Fernandez,B.(2023,August28).GenerativeAIpromisesaneconomicrevolution.Managingthedisruptionwillbecrucial.WallStreetJournal.

/articles/generative-ai-promises-an-economic-revolution-managing-the-disruption-will-be-crucial-b1c0f054

Howtoreadthispaper

Readerswillnotethatthetitleofthispaperfocuseson"generativeAI,"butwediscussAImorebroadlythroughout;it'snearlyimpossibletodisentangleonefromtheotherbecausethey'vebecomesynonymousinmostpeople'sminds.Whenweusetheterm"AI,"wearereferencingAImorebroadly.Weuse"generative"andotherqualifierswhenthatistheareaoffocus.Finally,giventherapidpaceofchange,weacknowledgethatthispaperrepresentsasnapshotintime,butwe'vetriedtofocusasmuchaspossibleonconsiderationsthatwon'tgeteasilyresolvedinthenearterm.

WebeginourpaperwithbackgroundinformationaboutAIingeneral,giventheimportanceofAIliteracy.WeincludeasummaryofthetenetsofresponsibleAI,aswellasexamplesofprinciplesthatfollowthosetenets,tohelpsetafoundationforhowwemightapproachAIineducation.Wealsowantedtobustsomecommonlyheldmyths,soweincludedafewcautionarynotestocomplementthisintroductorymaterial.

Next,werunthroughresultsfrommultiplesurveystoprovideahigh-levelviewofsentiment,usage,andunderstandingofgenerativeAIamongAmericansingeneral,andyoungpeople,parents,andteachersinparticular.Andwhilesurveyresultsareinformative,wealsowantedtotalkdirectlywithpeopletogetaqualitativesenseofhowtheyfeelaboutthistechnology.You'llnotequotesfromteachersandparentsthroughoutthispaper,aswellasasectiondedicatedtotakeawaysfromconversationswithyoungpeople.WeincludedfullQ&AswiththreeteachersintheAppendix.

Intermsofpolicy,weofferanoverviewbothofselectcurrentgovernmentpolicyandofschool/districtpolicy.(Spoileralert:There'smuchmoretobedoneonbothfronts.)

Allofthat—backgroundanddefinitions,exampleprinciples,surveyresults,andpolicyoverviews—servesasafoundationforunderstandingoursectionsonopportunitiesandrisks.Andthroughout,certainthemescomeuprepeatedly:

Hypevs.reality

Datasecurityandprivacy

Transparency,trust,andaccountability

Biasandequity

Humanoversight

Thesesamethemesalsorunthroughoutourrecommendations.

ThisisstillearlydaysforgenerativeAI,andwedon'tclaimtohavealltheanswers.Butwehopethispapercanfunctionasaguidetothisevolvinglandscapeandhelpeducationalstakeholdersmakeinformeddecisionsaboutwhen,how,andwhethertouseAItoolsineducationalsettings.

Background

Ask10peoplethedefinitionofartificialintelligenceandyou'llget20answers.Merriam-WebsterdefinesAIas"thecapabilityofcomputersystemsoralgorithmstoimitateintelligenthumanbehavior."5Thatsumsitupsuccinctly,butother,moretechnicallyaccuratedefinitionsrefrainfromanyreferencetohumans,giventherisksofanthropomorphizingthetechnology.TheEuropeanCommission'sHigh-LevelExpertGrouponArtificialIntelligence,forexample,definesAIas"systemsthatdisplayintelligentbehaviorbyanalyzingtheirenvironmentandtakingactions—withsomedegreeofautonomy—toachievespecificgoals."6AndtheGoogleDevelopersglossaryusesthisdefinition:"anon-humanprogramormodelthatcansolvesophisticatedtasks."7AIhasbeenaroundfordecades,fromrecommendationenginesonNetflixandvoiceassistantssuchasSiritocustomerservicechatbots,searchengines,andmonitoringsystemsthatpredictwhenfactorymachineswillrequiremaintenance.ThisearlierformofAIperformsdiscretetasksandrespondstoaparticularsetofinputs.8Thesesystemscanlearnfromdataandmakepredictionsbasedonthatdata,butthey'renotproducingoriginalcontent.

GenerativeAI,ontheotherhand,cangeneratecontent,suchastext,images,audio,andvideo,thatiscomplex,coherent,andoriginal,inresponsetoasubmittedpromptorquery,bylearningfromlargereferencesetsofexistingdata.9Alexasettingatimerforyourturkeywhileyourhandsarecoveredinrawpoultrygoopisconvenient,butChatGPTgeneratinganoriginalsonnetaboutyoursecondcousininthestyleofPetrarchisprettymind-blowing.

ThespectrumofreactionstogenerativeAIweencounteredasweresearchedthisreportaresimilartothespectrumintheAIindustryitself,thoughnotaspolarized.Ononeend,"doomers"advocateforaslowerapproachtoAIdevelopmentduetoconcernsaboutthespeculativerisksthatsomebelieveitposestohumankind'sfuturesurvival.10Ontheotherend,"boomers"believethepositivebenefitsfaroutweighanyrisks,andthatdevelopmentcan'tandshouldn'tbecurbed.Butwhiletheextremesgetmostoftheattention,neitheraddressesthecomplicatedrealityandmyriadconcernsofcurrentusage.Inthemiddle,there'sagrowingefforttodevelopresponsible,trustworthyAIthatworksforallpeopleinthehereandnow.Thevastmajorityofpeopleingeneralalsoliesomewhereinthemiddleinhowthey'rethinkingaboutAIintheirownlives:skeptical,butcurious.Andthey'relookingforguidance.

There'sonebigproblemwiththeemphasisonpreparingforapossiblefutureworldwhereAIrulessupreme:Thatvisionobscurespresent-daychallengesthatneedtobeaddressed.Longtermism—thefocusonensuringthathumankindwillsurvivemillionsofyearsfromnow11—isterribleforAI,saysTracyPizzoFrey,aformerGoogleAI

5Merriam-Webster.(n.d.).Artificialintelligence.InMerriam-Wdictionary.

/dictionary/artificial%20intelligence#dictionary-entry-1

6High-LevelExpertGrouponArtificialIntelligence.(2018,December18).AdefinitionofAI:Maincapabilitiesandscientificdisciplines.

EuropeanCommission.

https://ec.europa.eu/futurium/en/system/files/ged/ai_hleg_definition_of_ai_18_december_1.pdf

7GoogleforDevelopers.(n.d.)Machinelearningglossary.

/machine-learning/glossary

8Marr,B.(2023,July24).ThedifferencebetweengenerativeAIandtraditionalAI:Aneasyexplanationforanyone.Forbes.

/sites/bernardmarr/2023/07/24/the-difference-between-generative-ai-and-traditional-ai-an-easy-explanatio

n-for-anyone/

9CommonSenseMedia.AIrating:ChatGPT.

/ai-ratings/chatgpt

;McKinsey&Company.(n.d.).WhatisgenerativeAI?

/featured-insights/mckinsey-explainers/what-is-generative-ai

10TheSamAltmandramapointstoadeepersplitinthetechworld.(2023,November19).TheEconomist.

/business/2023/11/19/the-sam-altman-drama-points-to-a-deeper-split-in-the-tech-world

;EdinburghFuturesInstitute.(2023,November23).OpenAIchiefexecutive'sfiring,rehiringcastsnewlightonthewarbetweenAIdoomers,boomers.

https://efi.ed.ac.uk/openai-chief-executives-firing-rehiring-casts-new-light-on-the-war-between-ai-doomers-boomers/

11Samuel,S.(2023,September7).SiliconValley'svisionforAI?It'sreligion,repackaged.FuturePerfect.Vox.

/the-highlight/23779413/silicon-valleys-ai-religion-transhumanism-longtermism-ea

executivewholedthedevelopmentofCommonSenseMedia'sAIprinciples."Somepeoplearetalkingaboutcolonizingotherplanetsandnotthinkingaboutclimatechangeandsystemicracism.It'sasleightofhandtotalkaboutexistentialrisk—itcoversupwhatishappeningnow."

Ineducation,PizzoFreyseessimilaritiestotheriseofedtechoverthepast20years,anditsaccelerationduringandsincethepandemic."Whatyouhaveatahighlevelisaprettybifurcatedexperience:peoplewhoarereally,reallyexcitedaboutAIandnewtechnologyandwanttoimplementasfastaspossible,andpeoplewhoarereallyskeptical,especiallyfrontlineeducatorswhowerealreadyskepticaloftechnologyineducationandarealsoskepticalofAI."AIlikelywon'tbeanovernightrevolution,butmoreofanevolution,aseriesofsmallerstepsandchangesthatcanadduptobigtransformationsovertime."Wecouldletthathappenbydefaultorwecouldbeintentionalaboutit.Weneedtoquestionwhomakesdecisions.Rightnowit'shappeningmorebydefault,"saysPizzoFrey.

ResponsibleAI

TheresponsibleAImovementrespondstothatcallforintentionality.ResponsibleAIreferstoartificialintelligencethatisdesigned,developed,deployed,andusedinawaythatalignswithhumanvaluesandminimizesrisksandharms.12Thinkofitasafocusonhowtomakethesetoolsandtechnologiesworkbetter—thatis,moreaccurately,fairly,andethically—today,notin10years.Whilethere'snofixeddefinitionofexactlywhatresponsibleAIlookslikeinpractice,consensusisemergingaroundcertainthemes.

"Hundredsofprinciplesandchartersexistacrosstheglobe,andthey'reremarkablysimilarinsomepredictableways,"saysPizzoFrey."Theytendtoalltouchonunfairbias,transparencyandexplainability,andprivacyandsafety."Here,explainabilityreferstoimprovinghumanunderstandingofhowAIworks,whichalsobuildstrustinthesystem'sresults.13

Mostpeoplewon'tbeabletounderstandthedeepertechnicalaspectsofhowChatGPTworks,buteveryoneshouldbeabletounderstand,inplainlanguage,thebasicsofhowthetechnologyfunctions.

Mostpeoplewon'tbeabletounderstandthedeepertechnicalaspectsofhowChatGPTworks,buteveryoneshouldbeabletounderstand,inplainlanguage,thebasicsofhowthetechnologyfunctions.Thatalsomeansthattechcompaniescan'thidebehind"blackbox"terminologyasanexcuseforalackofexplanation.

"Humanintheloop"isaphrasethatalsocropsupinmanysetsofAIprinciples:theneedforahumantobepresentatcertainpointsinthesystem,ratherthantheuseofAIasawholesalealternativetohuman

decision-making.Ineducation,thisideaofhumaninvolvementisespeciallyimportant.TheU.S.Departmentof

12Nguyen,A.,Ngo,H.N.,Hong,Y.,Dang,B.,&Nguyen,B.T.(2023).Ethicalprinciplesforartificialintelligenceineducation.EducationandInformationTechnologies.28(4):4221–4241.

/10.1007%2Fs10639-022-11316-w

;IEEEStandardsAssociation.(n.d.).Generalprinciples.

/wp-content/uploads/import/documents/other/ead_general_principles.pdf

13Turri,V.(2022,January17).WhatisexplainableAI?SoftwareEngineeringInstituteblog,CarnegieMellonUniversity.

/blog/what-is-explainable-ai/

Education'sMay2023report("ArtificialIntelligenceandtheFutureofTeachingandLearning")issuesthedirectiveto"alwayscentereducatorsininstructionalloops,"furtherdetailingthoseloopsas:

Theloopinwhichteachersmakemoment-to-momentdecisionsastheydotheimmediateworkofteaching.

Theloopinwhichteacherspreparefor,plan,andreflectonteaching,whichincludesprofessionaldevelopment.

TheloopinwhichteachersparticipateindecisionsaboutthedesignofAI-enabledtechnologies,participateinselectingthetechnologies,andshapetheevaluationoftechnologies—thussettingacontextfornotonlytheirownclassroombutthoseoffellowteachersaswell.14

The2024NationalEducationTechnologyPlanalsomakesapointtoemphasizehumansintheloop:"Regardlessoftechnologists'claims,noemergingtechnologywillinandofitselfsolvethelong-standingchallengesfacedbyeducationalsystems.Asnewtechnologiesemerge,educatorsmustbeinvolvedindesigninganddevelopingdigitaltoolsforclassroomuse.Evaluatingthesetechnologiesforpotentialclassroomuseatthestate,schoolsystem,orschoollevelshouldbeiterativeandincludeeducatorandstudentinput."15Similarly,emergingstate-levelguidancealsoplaceshumansatthecenter.AccordingtoWashingtonstate'sguidance,forexample:"InK–12education,usesofAIshouldalwaysstartwithhumaninquiryandalwaysendwithhumanreflection,humaninsight,andhumanempowerment."16

PizzoFreygoesastepfurtherthanthe"humanintheloop"principle.What'sneeded,shesays,especiallyincaseswhereAIcanhaveadirectandmeaningfulimpactonpeople'slivesandopportunities(suchaseducation),ismeaningfulhumancontrol."Youcannot,andfromaprincipledperspectiveshouldnot,separatehumansfromAI,"shesays.Infact,AIissociotechnical;thetechnologycannotbedivorcedfromthehumanswhoinformandshapeitsuse.17

Education-specificconsiderations

AsstakeholdersdebateandclarifytheappropriateplaceofgenerativeAIineducation,it'simportanttokeepinmindsuchthemesasbias,transparency,andsafety.It'salsoworthhighlightingsomeadditionalconsiderationsaroundAIinaneducationalcontext.

First,we'retalkingaboutchildren.Thismaybestatingtheobvious,butthefactthatchildrendeservegreateranddifferentprotectionsthanadultsdoesn'tgetforegroundedenoughinconversationsaroundAIineducation.Ina2021papersummarizingAIresearchers'responsestoquestionsabouttheethicsofAIineducation,thefocusonyoungpeoplewasassociatedwithissuesofconsent,learningagency,dataprivacy,andalgorithmicbias,aswellaslonger-term"impactsofengagingwithyoungpeopleduringaformativeperiod."Saidonerespondent:"Weare

14U.S.DepartmentofEducation,OfficeofEducationalTechnology.(2023,May).Artificialintelligenceandthefutureofteachingandlearning:Insightsandrecommendations.

/documents/ai-report/ai-report.pdf

15U.S.DepartmentofEducation,OfficeofEducationalTechnology.(2024.)Nationaleducationaltechnologyplan.

16WashingtonOfficeofSuperintendentofPublicInstruction.(2024,January18).Human-centeredAIguidanceforK–12publicschools.

https://ospi.k12.wa.us/sites/default/files/2024-01/human-centered-ai-guidance-k-12-public-schools.pdf

17CommonSenseMedia.AIinitiative.

/ai

;Weidinger,L.,Rauh,M.,Marchal,N.,Manzini,A.,Hendricks,L.A.,Mateos-Garcia,J.,Bergman,S.,Kay,J.,Griffin,C.,Bariach,B.,Gabriel,I.,Rieser,V.,&Isaac,W.(2023,October31).SociotechnicalsafetyevaluationofgenerativeAIsystems.GoogleDeepMind.

/10.48550/arXiv.2310.11986

influencingimpressionableyoungpeople,andthatcomeswithamoralobligationtobeascorrectandproperaspossible."18

Twootherconsiderationsareimportanttorecognizeasschoolsevaluate,review,andpossiblyadoptnew

AI-poweredtools.OneisthattheimpactsofAIineducationalsettings,whetherviamoretraditionalalgorithmsormorerecentgenerativeAItechnology,aren'tnecessarilyreadilyapparent.AmalfunctioningAIsystemina

self-drivingcarcouldresultinimmediate,significantbodilyharmordeath,whereastheeffectsofAIonchildrenmaybeharder,andtakelonger,toidentify.Educatorsandschoolleadersnonethelessneedtoconsiderthosepotentialdownstreameffectsnow,andresearchwillberequiredtostudyoutcomes.

ThesecondisthatAIeducationaltoolsmayembodyassumptionsaboutwhichinstructionalstrategiesarebestforlearners,andnotenoughattentionisalwayspaidtowhichpedagogyisdrivingaspecificAItool.19Intherushtoreleaseandevaluatenewproducts,oftenthereisnotenoughfocusonthepedagogicalapproachthateachproducthasadopted—ortheefficacyofthetoolatall."Pedagogy-drivenAIineducationisoftenconsideredasgoodregardlessof[whether]thepedagogychosentodriveAIisanappropriateoneornotinthefirstplace,"anotherresearchercommentedinthatsamepaperontheethicsofAIineducation.20

ExamplesofprinciplesandguidelinesforresponsibleAI

Frameworksneedroadmapsthatspellouthowtheyworkinpractice,andanumberofentitieshavebegunreleasingactionableAIprinciplesandguidelinestothatend.In2023,CommonSenseMediareleaseditsfirstratingsofAIproducts,whichencompassmanyofthethemescommontoresponsibleAIframeworks.Gearedtowardaneducator,family,andpolicyaudience,theratingsaregroundedinasetofeightcoreprinciplesforhowtothinkaboutAIanditsimpactsonadultsandchildren:

PutPeopleFirst

PromoteLearning

PrioritizeFairness

HelpPeopleConnect

BeTrustworthy

UseDataResponsibly

KeepKidsandTeensSafe

BeTransparentandAccountable21

Similarly,DigitalPromise,aglobalnonprofitworkingtoexpandeducationalopportunitiesthroughresearchandtechnology,listsfourcomponentsinitsAIevaluationframework:transparency,safety,ethics,andimpact.22AndtheNationalInstituteofStandardsandTechnologyincludesthesecharacteristicsoftrustworthyAI:

18Holmes,W.,Porayska-Pomsta,K.,Holstein.K.,Sutherland,E.,Baker,T.,BuckinghamShum,S.,Santos,O.C.,Rodrigo,M.T.,Cukurova,M.,Bittencourt,I.I.,&Koedinger,K.R.(2022).EthicsofAIineducation:Towardsacommunity-wideframework.InternationalJournalofArtificialIntelligenceinEducation,32:504–526.

/10.1007/s40593-021-00239-1

19Ibid.

20Ibid.Jurenka,I.,Kunesch,M.,McKee,K.R.,Gillick,D.,Zhu,S.,Wiltberger,S.,Phal,S.M.,Hermann,K.,Kasenberg,D.,Bhoopchand,A.,Anand,A.,P?slar,M.,Chan,S.,Wang,L.,She,J.,Mahmoudieh,P.,Rysbek,A.,Ko,W.-J.,Huber,A.,...&Ibrahim,L.(2024,May14).TowardsresponsibledevelopmentofgenerativeAIforeducation:Anevaluation-drivenapproach.GoogleDeepMind.

/abs/2407.12687

21CommonSenseMedia.(n.d.).HowwereviewandrateAIproducts.

/ai-ratings/how-we-rate-and-review

22Mills,K.,Ruiz,P.,&

Lee

,K.(2024,February21).RevealinganAIliteracyframeworkforlearnersandeducators.DigitalPromise.

/2024/02/21/revealing-an-ai-literacy-framework-for-learners-and-educators/

Safe

Secureandresilient

Explainableandinterpretable

Privacy-enhanced

Fair—withharmfulbiasmanaged

Validandreliable

Accountableandtransparent23

Transparencyandaccountabilityaretwoofthetoughestnutstocrack.AIalgorithmsareoftendifficulttounderstand,evenforexpertsinthefield,andlargelanguagemodels(LLMs)likeChatGPTareoftenreferredtoas"blackboxes,"orsystemswhoseinnerworkingsareinvisibletotheuser.24

Amara,25anAIethicsresearcher,notesthatalotofthepeoplewhocreatethesesystemsdon'tunderstandhowthesystemsareworkingthemselves.

Asforaccountability,Amaraacknowledgesthevalueofpolicy,butaddsthatAIsystemsaredifficulttoregulateevenwhenruleshavebeencreated.ShepointstoaNewYorkCitylawineffectsince2023thatmandatesanindependentauditforanyAIhiringtool.CompaniesmustalsodisclosetheuseofanyAItoolsinthehiringprocesstoprospectiveemployees.26Amaraquestionsthecity'sabilitytomonitorforviolationsandenforceanyconsequences."Wehaven'tfiguredouthowtoregulatethesesystems,"shesays.

Frameworksandprinciplesareimportantnotjustforunderstanding,evaluating,andregulatingproductsalreadyonthemarket—theyalsoneedtobeimplementedatthedesignanddevelopmentstages,beforeusersareexposedtopotentialrisksandharms.Andit'scrucialtobeexplicitaboutthevaluesthatinformthedesignofAIsystems,platforms,andtools,becausethosesamevaluesinformthetools'outcomes."Ifwebuildthesetoolswithequityinmind,itwillhelpusbuildequityintherestofoursystems.Ifwedesignthemtotrackkids,they'lltrackkids,"saysErinMote,executivedirectorofInnovateEDU."Technologyisn'tneutral,andit'salsobuiltbyhumans."

23NationalInstituteofStandardsandTechnology.(2023,January).Artificialintelligenceriskmanagementframework.

/nistpubs/ai/NIST.AI.100-1.pdf

24Bagchi,S.,&TheConversationU.S.(2023,May26).WhyweneedtoseeinsideAI'sblackbox.ScientificAmerican.

/article/why-we-need-to-see-inside-ais-black-box/

25Wehaveusedapseudonymattheresearcher'srequest.

26NewYorkCityDepartmentofConsumerAffairs.(n.d.).Automatedemploymentdecisiontools.

/site/dca/about/automated-employment-decision-tools.page

ExamplesofAIPrinciples

CommonSenseMedia

DigitalPromise

NIST(NationalInstitute

ofStandardsandTechnology)

PutPeopleFirst

Transparency

Safe

PromoteLearning

Safety

Secureandresilient

PrioritizeFairness

Ethics

Explainableandinterpretable

HelpPeopleConnect

Impact

Privacy-enhanced

BeTrustworthy

Fair-withharmfulbiasmanaged

ProtectOurPrivacy

Validandreliable

KeepKidsandTeensSafe

Accountableandtransparent

BeTransparentandAccountable

Cautionarynotes

Beforeweturntothemeatofthiswhitepaper,thereareafewcautionarynotestokeepinmind:

Don'tbelieve(all)thehype

Yes,generativeAIrepresentsatechnologicaladvancement,onethatwecouldn'thaveimaginedjustagenerationago.Butthere'sareasonthattheSubstackblogcalled"AISnakeOil,"writtenbyPrincetonprofessorArvindNarayananandgraduatestudentSayashKapoor,ispopular.27Alotofwhat'sbeingsoldasAIisn'tnearlyasgoodasit'smadeouttobe.Marketingspeakhasbeenaroundforever,andsalesteamsareouttomakeaprofit.Caveatemptor.

Don'tforgethistory

Thetechindustryisfuture-facingbydesign.Thatcanbegoodforinnovation,butbadforhistoricalcontext.AlthoughgenerativeAIisnew,algorithmshavebeeninoperationfordecades,withmanyofthesamebiasproblemsthatarepoppinguptoday.28Andineducation,technologyhasbeenseepingintotheclassroomfor100years.AudreyWatters'bookTeachingMachinestracesthehistoryofpersonalizedlearningbacktoSidneyPressey'sautomatedgradingmachinesofthe1920s,followedbythemore

well-knowninventionsfromB.F.Skinner."Teachingmachinesareoftende