9781802207675 Autonomous Weapons Systems and the Protection of the Human Person An International Law Analysis

《9781802207675 Autonomous Weapons Systems and the Protection of the Human Person An International Law Analysis》由会员上传分享，免费在线阅读，更多相关内容在行业资料-天天文库。

©DiegoMauri2022Coverimage:©KathrynBrimblecombe-Fox,Manhunting,2017,gouacheonpaper,56×75cm.Allrightsreserved.Nopartofthispublicationmaybereproduced,storedinaretrievalsystemortransmittedinanyformorbyanymeans,electronic,mechanicalorphotocopying,recording,orotherwisewithoutthepriorpermissionofthepublisher.PublishedbyEdwardElgarPublishingLimitedTheLypiatts15LansdownRoadCheltenhamGlosGL502JAUKEdwardElgarPublishing,Inc.WilliamPrattHouse9DeweyCourtNorthamptonMassachusetts01060USAAcataloguerecordforthisbookisavailablefromtheBritishLibraryLibraryofCongressControlNumber:2022932894ThisbookisavailableelectronicallyintheLawsubjectcollectionhttp://dx.doi.org/10.4337/9781802207675ISBN9781802207668(cased)ISBN9781802207675(eBook)DiegoMauri-9781802207675DownloadedfromPubFactoryat05/28/202203:48:15AMviaDukeUniversityLawLibrary

1ContentsForewordbyMaryEllenO’ConnellviAcknowledgmentsviiListofabbreviationsix1Introduction:AutonomousWeaponsSystemsandtheProtectionoftheHumanPerson12Sketchingthedebate143Autonomousweaponssystemsunderinternationalhumanrightslaw674Autonomousweaponssystemsunderinternationalhumanitarianlaw1185Autonomousweaponssystemsandthe‘accountabilitygap’conundrum1766Buildingthebridgetothefuture235Bibliography260Index284vDiegoMauri-9781802207675DownloadedfromPubFactoryat05/28/202203:48:25AMviaDukeUniversityLawLibrary

2ForewordWeaponsrepresentoneofthegreatparadoxesofhumanity.Humanbeingsconstantlyresearchanddevelopnewweapons—newwaysofkillingwithincreasedlethality,easeofuse,greaterprecision,andcapabilitiesofallkinds.Atthesametime,weconstantlycontemplatethemoralityofusingweapons.Moralassessmentleadstoeffortsatlegalregulation,includingbanningspe-cificweapons.Theparadoxhasbeenonfulldisplayforadecadeandahalfregardingfullyautonomousroboticweaponsandtheefforttobanorstrictlylimittheiruse.Thisbookistherarestudyofallthreeaspectsoftheweaponsparadoxasregardsautonomousweapons.Itisafullyintegratedaccountofwhatweneedtoknowinordertounderstandthecurrentdebateaboutusingautonomousweaponswithsomeregulationorbanningthemaltogether.Theauthorexpertlyexplainsthecomplexnewtechnologicalresearchinvolvedinautonomousweapons.Theseareweaponsthatremovetheneedforahumanresponsibilityforthedecisiontotakethelifeofanotherperson.Hehasappropriatelychosenhumanrightsasthestandardagainstwhichtoassessthemoralityofthisrecentdevelopmentinweaponstechnology.Heisthenabletoconsiderissuessuchaswhethertobanorregulate,andhowtodoso.Thediscussionisengaging,allowingreaderstocometotheirown,informedconclusions.Ihavehadtheprivilegeofdiscussingthelawandmoralityofhigh-techweaponswithDiegoMaurisincehisresearchvisittotheUniversityofNotreDamein2018.Ifoundhimtobeoneoftherarehumanbeingswiththeabilitytowritecomprehensivelyononeofthegreatestchallengestothelawandmoralityofarmscontrol.Hecamewithawealthofknowledge,whichhegenerouslyshared,aswellasexperienceintheHolySee’sdelegationtotheprincipalnegotiationsconcernedwithregulatingautonomousweapons.Hecamealsowithadeterminedpurposetodeepenhisknowledgeandunderstand-ing.Thisbookisatestamenttohissuccessandanimportantcontributiontolawandethics.MaryEllenO’ConnellRobertandMarionShortProfessorofLawUniversityofNotreDame,USAviDiegoMauri-9781802207675DownloadedfromPubFactoryat05/28/202203:48:27AMviaDukeUniversityLawLibrary

3AcknowledgmentsThisbookistheresultofalong-runningjourneythatwouldnothavebeenpossiblewithoutthetravellingcompanionsIwasundeservedlyluckyenoughtohave.Ideally,thisjourneystartedduringmyinternshipatthePermanentMissionoftheHolySeeattheUnitedNationsinGenevainsummer2015:itwasthenthat,accidentally,Icameacrosssomepaperworkonasubjectthatwastotallyunknowntome,‘LethalAutonomousWeaponsSystems’(LAWS).ThatcuriousassonancewiththeEnglishtermfortherulesoflaw—paradoxicalforaweaponssystemwhich,onfirstimpression,threatenedtoclashhead-onwitheverymoralandlegalrule—wastooprecisetobeleftaside.Ilaterhadtheuniqueprivilegetojoin,first,theHolySee’sDelegationtotheMeetingofExpertsand,later,theGroupofGovernmentalExperts,whichgavemeaccessnotonlytoawealthoftechnicaldocumentsandinsightsintothedip-lomaticarena,butalso—andaboveall—toanetworkoffacesandexpertisethatIwouldrarelymeetoutside.ThisiswhyIamgratefulprimarilytotheMission’spersonnel,fromMsgrTomasiandMsgrJurkovič(whoservedasPermanentRepresentativesduringmystays)toallcolleaguesandfriendsIhavehadthechancetomeetduringtheseyears.Mysincerestandaffectionatethanksgotomymentor,PasqualeDeSena,whofatherlysupportedmenotonlyinstructuringthisbook,butalsointakingmyfirststepsintheacademia.IreallylackthewordstoproperlythankMicaelaFrulli,withwhomIhavethehonourtoworkinFlorence:herencouragingattentionsandenthusiasticspurs,especiallyatthebeginningofthisadventure,havebeenanunexpectedgiftforme.Iwasthenluckyenoughtodock,forafewmonths,attheUniversityofNotreDame(US),whereImetMaryEllenO’Connell—herstronginclinationtowardstheneedtocombinelawandmoralitywasarealsourceofinspirationformyreflections—andattheEuropeanCentreforConstitutionalandHumanRights(ECCHR)inBerlin,whereIworkedcloselywithChantalMelonionrelatedprojects.Myresearchalsobenefitedfromseveralpresentationsanddiscussions,forwhichIamgratefultoAndreaSaccucci,LuisaVierucci,DeborahRusso,MariaChiaraVitucci,GabrieleDellaMorte,GiulioBartolini,AlessandraGianelliandChiaraVenturini.Lastly,IhavetodedicateaspecialmentiontothepersonneloftheIstitutodiScienzeMilitariAeronauticheoftheItalianAirForceand,inparticular,toColonelAndreaRossi:theyhavealwaysbeenavailabletomeetviiDiegoMauri-9781802207675DownloadedfromPubFactoryat05/28/202203:48:28AMviaDukeUniversityLawLibrary

4viiiAutonomousweaponssystemsandtheprotectionofthehumanpersonmeduringthedraftingprocessandsharetheirviewsonthemilitaryimplica-tionsofautonomousweaponssystems(AWS).Thosewhoagreedtodedicatetheirtimeandenergytomyworkaremany—andnowthatIthinkaboutit,IfeelIhavetoapologize.Thefirstis,beyonddoubt,DanieleAmoroso,withwhomIhaveconsumedhours(especiallyduringvacationdays)discussingAWS:withouthisbrilliantintuitionsandwittycritiques,mymodestreflectionsonthistopicwouldhavebeentrivial.LauraMagiandOliviaLopesPegnaacceptedtoreadandreviewportionsofthebook,givingmethoughtfulsuggestionsonwhattoinsertandchange.MartinaBuscemi’ssharpeyesightwasindispensableforstrengtheningthestructure;thesamegoesforMarcoRoscini,whohelpedmewiththebookproject.LorenzoAcconciamessaandFrancescaTammone—whomIregardasfriendsevenbeforecolleagues—readpreviousversionsofthebookandsharedtheirstimulatingviewswithme.Finally,EdwardElgar’spublishingteam(CarolineCornish,AmberWattsandBenBoothinparticular)providedmewithsuperbandpromptassistancethroughtheeditorialprocess.Asanantidotetosolitudecharacterizingworkssuchaswritingamono-graph,doctorsrecommendcompanionshipbytrustworthyandfriendlyfaces.Ilistenedtothem.Thus,thenumberofpeoplewhohavecontributedtomakemyheavydayslighterissogreatthatIcannotnamenameshere.Butthisrelievesme,becauseIknowfromtheaffectionthatbindsusthat,whentheyreadtheselines,theywillimmediatelyunderstandIamreferringtothem.Iamthankfulbeyondwordsforwhattheirsupportandfriendshipmeanstome.ToGloria,mylove,andtomyparentsandfamilygoesonelast,heart-feltthankyouforbeingmycompassthroughthisjourney.DiegoMauri-9781802207675DownloadedfromPubFactoryat05/28/202203:48:28AMviaDukeUniversityLawLibrary

5AbbreviationsACHPRAfricanCharteronHumanandPeoples’RightsACHRAmericanConventiononHumanRightsACommHPRAfricanCommissiononHumanandPeoples’RightsAFRLAirForceResearchLaboratoryAIartificialintelligenceAPadditionalprotocolAPIAdditionalProtocolItotheGenevaConventionsAPIIAdditionalProtocolIItotheGenevaConventionsARSIWAArticlesontheResponsibilityofStatesforInternationallyWrongfulActsATSAlienTortStatuteAWSautonomousweaponssystemsB&HRbusinessandhumanrightsBCIbrain–computerinterfaceCCFcontinuouscombatantfunctionCCPRHumanRightsCommitteeCCWConventiononCertainConventionalWeaponsCDEMcollateraldamageestimationmethodologyCSKRCampaigntoStoptheKillerRobotsDARPADefenseAdvancedResearchProjectsAgencyDMZDemilitarizedZoneDoDDepartmentofDefenseDPHdirectlyparticipantsinhostilitiesEBOeffect-basedoperationsECHREuropeanConventiononHumanRightsECommHREuropeanCommissionofHumanRightsECtHREuropeanCourtofHumanRightsixDiegoMauri-9781802207675DownloadedfromPubFactoryat05/28/202203:48:30AMviaDukeUniversityLawLibrary

6xAutonomousweaponssystemsandtheprotectionofthehumanpersonEUEuropeanUnionFTCAFederalTortClaimsActGCGenevaConventionGC36GeneralCommentNo36(2018)onart6ICCPRGDPRGeneralDataProtectionRegulationGGEGroupofGovernmentalExpertsHRWHumanRightsWatchIACinternationalarmedconflictIACommHRInter-AmericanCommissionofHumanRightsIACtHRInter-AmericanCourtofHumanRightsICCInternationalCriminalCourtICCPRInternationalCovenantonCivilandPoliticalRightsICCStRomeStatuteoftheInternationalCriminalCourtICJInternationalCourtofJusticeICLinternationalcriminallawICRACInternationalCommitteeforRobotArmsControlICRCInternationalCommitteeoftheRedCrossICTRInternationalCriminalTribunalforRwandaICTYInternationalCriminalTribunalfortheFormerYugoslaviaIEDimprovisedexplosivesdeviceIEEEInstituteofElectricalandElectronicsEngineersIHLinternationalhumanitarianlawIHRLinternationalhumanrightslawILCInternationalLawCommissionILOInternationalLabourOrganizationIMTInternationalMilitaryTribunalIMTFEInternationalMilitaryTribunalfortheFarEastIOinternationalorganizationiPRAWInternationalPanelontheRegulationofAutonomousWeaponsISRintelligence,surveillanceandreconnaissanceJCEjointcriminalenterpriseLARlethalautonomousrobotsDiegoMauri-9781802207675DownloadedfromPubFactoryat05/28/202203:48:30AMviaDukeUniversityLawLibrary

7AbbreviationsxiLAWSlethalautonomousweaponssystemsLOAClawofarmedconflictMHCmeaningfulhumancontrolMoDMinistryofDefenceMoEmeetingofexpertsNATONorthAtlanticTreatyOrganizationNGOnon-governmentalorganizationNIACnon-internationalarmedconflictNLWnon-lethalweaponsPMSCprivatemilitaryandsecuritycontractorOECDOrganisationforEconomicCo-operationandDevelopmentOEIWGOpen-EndedIntergovernmentalWorkingGroupOHCHROfficeoftheHighCommissionerforHumanRightsOODAobserve,orient,decide,actPCIJPermanentCourtofInternationalJusticePMSCprivatemilitaryandsecuritycontractorSCSLSpecialCourtforSierraLeoneSIPRIStockholmInternationalPeaceResearchInstituteSIrUSsuperfluousinjuryandunnecessarysufferingUDHRUniversalDeclarationonHumanRightsUKUnitedKingdomofGreatBritainandNorthernIrelandUNUnitedNationsUNCUnitedNationsCharterUNGAUnitedNationsGeneralAssemblyUNHRCUnitedNationsHumanRightsCouncilUNIDIRUnitedNationsInstituteforDisarmamentResearchUNSCUnitedNationsSecurityCouncilUNSGUnitedNationsSecretaryGeneralUSUnitedStatesofAmericaDiegoMauri-9781802207675DownloadedfromPubFactoryat05/28/202203:48:30AMviaDukeUniversityLawLibrary

8DiegoMauri-9781802207675DownloadedfromPubFactoryat05/28/202203:48:30AMviaDukeUniversityLawLibrary

91.Introduction:AutonomousWeaponsSystemsandtheProtectionoftheHumanPersonWhymen?Menareforyoumerenumbers,nomore!(FSchiller,DonKarlos,InfantvonSpanien,ActV,SceneX(GeorgJoachimGöschen1787,translatedbyFKimmich,OpenBookPublishers2018))AUSintelligenceanalystoncesaidthat‘thestatewhichcontemporaryarmshavereachedisasirrevocableasoriginalsin’.1Referringtothedevelopmentofnuclearweapons,theauthorendedupconfessingtheunmentionable,namely,thatthetendencytoproduceevermorepowerfulandevermoredestructivearmamentsissomehowinherentinhumannatureand,therefore,asinescapableasoriginalsin.Humanitywouldthereforeneedanewkindofbaptismtoredeemitselffromthisguilt.However,theredoesnotyetseemtobeaMessiahonthehorizon—onthecontrary,high-performanceweaponsareconstantlydevelopedtoapplyforceagainstobjectsandindividuals.Writinginaquitedifferentage(andfromanentirelydifferentperspective),oneofthemostcontroversiallawyersofthetwentiethcentury,CarlSchmitt,concludedhismasterpieceDerNomosderErdequotingafamousaffirma-tionofHegel,accordingtowhich‘duringthetransitionfromfeudalismtoabsolutismhumanityneededgunpowder,andimmediatelyitwasthere’.2AccordingtotheGermanjurist,newformsofpowerrequireappropriateformsofprojectingforceagainstthe‘other’:everyhumanagehasthustheweaponitneeds.Thosewordswereputdownattheendofthe1940s,adecadeinwhichhumanityborewitnesstothedestructivepowerofanunprecedentedweapon,thenuclearbomb.IthadbeentheexhibitionofthatuncontrollableforcethatplayedsuchanimportantroleinterminatingWorldWarII.Atthatmomenttheentireworldwasawaitingnolessthan40yearsofuninterrupted1RO’Connell,OfArmsandMen.AHistoryofWar,WeaponsandAggression(OUP1989)11.2CSchmitt,TheNomosoftheEarthintheInternationalLawoftheJusPublicumEuropeaum(origednGreven1950)(trandannotGLUlmen,TelosPress2006)321(emphasisadded).1DiegoMauri-9781802207675DownloadedfromPubFactoryat05/28/202203:48:32AMviaDukeUniversityLawLibrary

102Autonomousweaponssystemsandtheprotectionofthehumanpersontensionbetweentwopoles,undertheconstantthreatofanuclearwarcapableofendinghumanpresenceonearth.Thisbookisdevotedtoatechnologicaldevelopmentinthefieldofarma-mentsthathasrecentlyelicitedanintensedebateamongdiplomats,militaryexperts,scientists,philosophersand(international)lawyers,namely,auton-omousweaponssystems(AWS).Inbrief,theseareweaponssystemsthatcanselectandengagetargetsautonomously,thatis,withouthumaninterven-tion.Differentperspectivesanddisciplines—ethical,military,politicalandlegal—areinterestedinthoseweapons.Thisbookadoptsaninternationallawperspective;moreprecisely,itisbasedonthenormsprotectingthehumanperson.Beforeadvancingthebook’smaintheses,thischapterplacesAWSintheevolutionarypatternthatweaponshaveexperiencedthroughhumanhistory.Whichstageofthe‘evolution’ofweaponsareweattoday?Whichweapon(s)doesourepochneed?Caninternationallawkeepthisevolutionarypatternundercontrol?Thefirsttwosections(1.1and1.2)provideanswerstothesequestions,inordertoputforwardAWS’smostsalientfeatures.Thefinalsection(1.3)presentsthebook’smainargumentsandstructure.1.1USINGFORCEAGAINSTTHEINDIVIDUAL:ANEVOLUTIONARYPATTERNIfonewantedtosketchthehistoryoftheuseofforceacrosshumanages,anappropriateanglewouldbethatofremoteness:mainlyintuitive,thegoalofastrategicuseofforceagainstindividualsismaximizingthereceiver’slosseswhileminimizingtheirown.Projectinglethalitywithoutprojectingvulnera-bilityisadesirableoutcomeinwar,asitisinanyothercontextwhereforcecouldbeused(forexample,policing).However,fromadiachronicperspectivetheneedforcombatantstobeintegrallyremovedfromlife-threateningrisksseemstohaveflourishedquiterecently.Inmoreancienttimes,suchremotionwasperceivedasascandal.AnexamplefromGreekliteraturemayhelp.InBookXIIIoftheIliad,Idomeneus,respondingtohiscomradeMeriones(accordingtotradition,hisnephew)whohadaskedhimforsomespearstouseinbattle,vigorouslydefendedthemoralsuperiorityofhand-to-handcombat:‘mywayisnottofightmybattlesstandingfarawayfrommyenemies’.3Remotenesswasperceivedasanintolerableaffronttotrueheroism,whichrequiredpersonalconfrontationandthusexposureofone’sownlifetolethalrisk.Actingotherwise—escapingfromtheenemy’srangeofshot—wasamanifestation3TheIliadofHomer(trRLattimore)(UniversityofChicagoPress2011)bookXIII,262–3.DiegoMauri-9781802207675DownloadedfromPubFactoryat05/28/202203:48:32AMviaDukeUniversityLawLibrary

11Introduction3ofcowardicedeservingpublicshame.Chivalryhasalwaysbeenacentripetalforceinwarfare(pushingcombatantstothebattlespace),yetithasalwaysbeenassociatedwithdifferent,morecentrifugalforcesprojectingcombatantsfarawayfromdanger.Thatis,nochivalriccodehaseverpreventedthedevelop-mentanddeploymentofweaponswhichallowedtheiruserstosubjugatetheenemyatadistance,withoutexposingthemselvestotherisksassociatedwithhand-to-handconfrontation.Heroism,whilerhetoricallybewitching,couldnotkeeppacewithefficiencyandriskreduction:thisseemsenoughtojustifyourcontemporaryarsenals.Ifwewantedtosketchaprécisofhowforcehasbeenprojectedinthehistoryofmankind,wecouldidentifythefollowingstagesatleast.Attheverybegin-ning,theforcethatwasusedinthefieldwasmuscle-powered:itwashumanandanimalmusclesthatpoweredallprehistoric,classicalandearlymedievalwarfare.Rocks,sticks—convenientlyshapedandsharpenedforhunting—clubs,daggersandbowscompensatedthescarceoffensivecapabilitiesofthehumanbody’sstructuresincethemostancienttimes.4DuringtheMiddleAge,moresophisticatedweapons,suchasthecrossbowandthecatapult(oneoftheearliestexamplesofmechanicartillery),enteredthefield.5Importantly,thetypeofremotenessensuredbycrossbowsraisedgreatconcernduringthetwelfthcentury,tothepointthattheywerebanned(albeittemporarily)underCanonno.29ofthesecondEcumenicalLateranCouncil(1139).6Thereasonsforthisprohibitionweretwofold.First,theuseofcrossbowsrequiredlittletraining:unlikethearcher,thecrossbowmandidnotneedtobephysicallyvigorous,andhisvolumeoffirewasnotlimitedbyfatigue.Second,theirusemadeitpossibleforsimplepeasantstoshootdownknights,whichlookedintolerableto‘apoliticalandsocialstructureweldedtogetherwithchivalricfightingskillsacquiredatagreatcost’.7Thesecondstagetobeidentifiedischaracterizedbychemical-poweredforce.Chinawasattheforefrontindiscoveringtherecipeforamixtureofsalt-peter,charcoalandsulfur,whichwasinitiallynamed‘blackpowder’andlaterregardedastheveryfirstrevolutioninmilitaryaffairs.8Behindaninnocuous,purelydescriptivelabel,thispowderconcealedadisruptiveforcethathadno4O’Connell(n1)22.5Ibid65.6NPTanner(ed),DecreesofEcumenicalCouncils(GeorgetownUniversityPress1990)203(‘[w]e[theSupremePontiff]prohibitunderanathemathatmurderousartofcrossbowmenandarchers,whichishatefultoGod,tobeemployedagainstChristiansandCatholicsfromnowon’.7O’Connell(n1)95.8GParker,TheMilitaryRevolution.MilitaryInnovationandtheRiseoftheWest,1500–1800(CUP1996).DiegoMauri-9781802207675DownloadedfromPubFactoryat05/28/202203:48:32AMviaDukeUniversityLawLibrary

124Autonomousweaponssystemsandtheprotectionofthehumanpersonprecedentsatthattime.Gunpowder,andthenbullets,broughtaboutanentirelynewspeciesofweapons,suchastheharquebus,theflintlockmusketand,even-tually,thepistol.9Heavyartillerydevelopedatthesamepace.CoupledwiththecapitalisticmentalitythatspreadacrossEuropebythefifteenthcentury,morelethalweaponsbecameavailabletoanincreasingnumberofcombatants.However,theveritableturningpointofchemical-poweredforceoccurredduringthesecondhalfofthenineteenthcentury,thankstothediscoveryofdynamite.Thismaterielwasemployedtoproducepracticalhighexplosivestobetransformedintomunitionsandtobedeliveredinshells,bombsand,eventually,missiles.Machineguns,too,wereinventedinthisperiod.Aswerecrossbowsduringthetwelfthcentury,machinegunsalsowereaccusedofbeingimmoralweaponssincetheybestowedoverlyunbalancedpowerontheirusers.10Asforthecrossbowssomecenturiesbefore,thisargumentprovedinsufficienttojustifytheirprohibition:providedthattheycouldbeusedinaccordancewiththelawsofwar,nooverwhelmingreasonfortheirbancouldbefound.Instead,machinegunshavebeenregularlyemployedinarmedcon-flict,especiallyinaerialwarfare.Curiously,atthebeginningofthetwentiethcenturytheskybecamethebirthplaceofentirelynewhorizonsofremotenessinapplyinglethalforce,evenmoresothanthatwhichoccurredatseaduringthepreviouscenturies.11Sufficeittoconsiderthataerialbombingshavebeenlabelledasuncivilizedmeansofwarfarebecauseoftheirindiscriminateuseagainstcivilianpopulationcenters;however,theyprovedefficientagainstsolelymilitarytargets(particularlytroopconcentrations)andextremelyval-uablefromthetacticalviewpoint,andtheirusewaseventuallyaccepted.Insummary,ifchemicalpowerallowedforincreaseddistancebetweenthecom-batantandthetarget,technologyadvancementshavebeenpushingremotenessevenfurther,supplyingdeliveryplatformsthatassurebothaccuracyandreducedexposuretorisks.Athirdstepintheevolutionarypatternisundoubtedlyrepresentedbynuclear-powereduseofforce.Itisbecauseoftheirdestructivepowerthatnuclearweaponsdeservedsomepopularepithets,suchas‘theabsoluteweapon’,whichwasdevelopedby‘wizards’eagertodragtheentireplanetintoan‘Armageddon’scenarioandconsignmankindto‘oblivion’.Also,itispreciselyowingtotheimpactthatnuclearweaponshadontargetedpopu-lationsandbroadlyonpublicopinionthatthedoctrineofnucleardeterrence9O’Connell(n1)117–18.10Ibid233:‘[a]weaponwhichsprayedbulletslikeagardenhosesprayedwaternotonlycontradicted[the]values[ofindividualheroism],itpromisedtomakeamockeryofthem’.11JAOlsen(ed),AHistoryofAirWarfare(PotomacBooks2010)1–2.DiegoMauri-9781802207675DownloadedfromPubFactoryat05/28/202203:48:32AMviaDukeUniversityLawLibrary

13Introduction5wasborn.12Nuclear-poweredforcewitnessesthatthequestfordestructivenessmayeventuallywipeouthumanity.Itisonthisbasis—notremotenessandpro-gressivedistancingfromthefield—thatattempts(todatewithlimitedsuccess)havebeenmadetoprohibitthisweaponry.13Humanityisnowconsideringanuclear-freefuture,whichcannotberegardedotherthanpositively.Thisthree-stepanalysishasservedamerelydescriptivepurpose,thatis,tobrieflyshowtheprogressthattechnologyhasmadepossiblethroughthecenturies.However,someissueshaveemerged.First,thetendencytodevelopmorepowerfulweaponsgoeshand-in-handwiththatofdistancingtheappliersofforcefromtheirreceivers.Second,thatnewspeciesofweaponspopulatethefielddoesnotrevoketheutilityoflessadvancedweapons:thenuclearweaponhasnotmadefoldingknivesdisappear.Instead,thisdiversityinthechoiceofweaponsallowsforgraduatedusesofforce,dependingonthecircumstancesofparticularcases.Third,weaponreliabilityandefficiencyandstandofffromharmcanbeidentifiedasdrivingfactorsthatguidetechnologicaldevelop-mentsinthisfield.Fourth,andmoreimportantlyforthepurposesofthisbook,theadventofhighlypowerfulweaponsentrustingtheiruserswithmorepowertoapplyforcewithoutexposingtoriskhastraditionallybeenaccompaniedwithethicalrestraints:policymakers,philosophers,militaryexpertsand,even,lawyershaverepeatedlyraisedquestionsastotheadmissibilityoftheiruseand,insomeinstances,workedfortheirregulationorprohibition.1.2TOWARDS‘ARTIFICIALINTELLIGENCE(AI)-POWERED’FORCETheenormousprogressmadeinthefieldofcomputersciencewasboundtoinfluencemanyaspectsofeverydaylife.Itwasjustamatteroftimebeforethisprogressinvestedinthefieldofweaponstoo.Computersciences’impactonweaponeeringisacomplex,multi-facetedissuethatwoulddeserveamon-ographicworkinitself.However,forourpurposesandwiththeusualcaveat,tworecentdevelopmentscanbesingledout.Thefirstdevelopmentisrepresentedbytheconstructionof‘cyberspace’,anentirelynewspacethroughwhichforcecanbedeliveredoverlongdis-tances.14‘Cyberweapons’canbedefinedas‘computercode[s]that[are]used,ordesignedtobeused,withtheaimofthreateningorcausingphysical,12DJordan(ed),UnderstandingModernWarfare(CUP2016),405ff.13TreatyontheProhibitionofNuclearWeapons(adopted7July2017,enteredintoforce22January2021)CN.478.2020.TREATIES-XXVI-9.14MNSchmitt(gened),TallinnManual2.0ontheInternationalLawApplicabletoCyberOperations(CUP2017).DiegoMauri-9781802207675DownloadedfromPubFactoryat05/28/202203:48:32AMviaDukeUniversityLawLibrary

146Autonomousweaponssystemsandtheprotectionofthehumanpersonfunctional,ormentalharmtostructures,systems,orlivingbeings’.15Legalscholarshiponthisnewtypeofweapons,andtheirimplicationsontraditionalconceptssuchasthenotionof‘attack’,hasflourishedtodate.16Itiswidelyacknowledgedthatcyberweapons,ifemployedagainststrategicalinfrastruc-ture,cancausedevastatingharmtothepopulationofanentirecountry(ifnotmorethanone).17Inadditiontoensuringenhancedremoteness(astheusercanbesafelyplacedalongdistancefromthetarget),cyberweaponsensurealsoanelevateddegreeofanonymity,asmadeclearbythelong-debatedissuesofattributionraisedbytheiruse.TherecentattacksontheIraniannuclearplantatNatanzandthedifficultiesassociatedwithattributingthemtoaparticu-largroup(andthustoastate)areillustrativeofthedisruptiveandobscurecapabilitiescharacterizingtheseweapons.18Thesecapabilitieswillonlybemagnifiedbytheadventofincreasinglyautomated(andautonomous)cyberweapons—thatis,toolsthatcandetectthreats,analyzetargetvulnerabilities,developresponsesandeffectuatethemwithouthumanintervention—whichhaveattractedattentioninscholarshiponlyrecently.19Theseconddevelopmentisrepresentedbyunmannedweaponssystems.Commonlyknownas‘drones’—aftertheEnglishtermindicatingthemalebeeandtheengine’shum—thesesystemsconsistofacraftthatdoesnotcarryahumanoperatorandthatcanbeequippedwithalethalornon-lethal15TRidandPMcBurney,‘CyberWeapons’(2012)157TheRUSIJ6,7.16NTsagourias,‘TheLegalStatusofCyberspace’,inNTsagouriasandRBuchan(eds),ResearchHandbookonInternationalLawandCyberspace(EdwardElgar2017);MRoscini,CyberOperationsandtheUseofForceinInternationalLaw(OUP2014).Foratheoreticalappraisal,seeSPietropaoli,‘Cyberspazio.Ultimafrontieradell’inimi-cizia?Guerre,nemiciepiratialtempodellarivoluzionedigitale’(2019)8Rivistadifilosofiadeldiritto379.17WBoothby,‘DeceptionintheModern,CyberBattlespace’,inJDOhlin,KGovernandCFinkelstein(eds),CyberWar:LawandEthicsforVirtualConflicts(OUP2015);MNSchmitt,‘TheLawofCyberWarfare:QuoVadis?’(2018)25StanfordL&PolReview269.18JGol,‘IranBlasts:WhatIsBehindMysteriousFiresatKeySites?’(BBC,6July2020)https://www.bbc.com/news/world-middle-east-53305940accessed31August2021.19NCRowe,‘DistinctiveEthicalChallengesofCyberweapons’,inJDOhlin,KGovernandCFinkelstein(eds),CyberWar:LawandEthicsforVirtualConflicts(OUP2015)317.SeealsoMKaminska,DBroedersandFCristiano,‘LimitingViralSpread:AutomatedCyberOperationsandthePrinciplesofDistinctionandDiscriminationintheGreyZone’,inTJančárková,LLindström,GViskyandPZozt(eds),GoingViral.202113thInternationalConferenceonCyberConflict(NATOCCDCOEPublications2021).Onautonomouscyber-weapons,seeRLiivojaandAVäljataga(eds),AutonomousCyberCapabilitiesunderInternationalLaw(NATOCCDCOEPublications2021).DiegoMauri-9781802207675DownloadedfromPubFactoryat05/28/202203:48:32AMviaDukeUniversityLawLibrary

15Introduction7payload.20Firstemployedforintelligence,surveillance,andreconnaissance(ISR)operations,droneswereeventuallyarmedbymountinglaser-guidedmissiles(thefamous‘Hellfire’),whichmarkedaturningpointinthismilitarytechnology.Currently,thisclassofuninhabitedweaponshasevolvedconsid-erably,andthespeciesthatpopulatestatearsenalsarevariedandperfectlyadaptedtotheenvironmentinwhichtheyoperate,beitair,seaorland(eitheraboveorbelowground).Unmannedvehiclesshowtwomainfeatures.First,theydonotcontemplateahumanonboard:theiroperatorissafelyplacedinagivenenvironmentfromwhichitmaneuversthecraft,whilethelatteroperatesinanotherenvironment.21Second,physicaldistanceisparadoxicallymatchedwithunprecedentedclosenesstothebattlefield.Thisispossibleowingtoaplethoraofinfraredandhigh-resolutionvideocameras,aswellaselectromagnetic,gammaray,biologicalandchemicalsensors,whichprojecttheiroperatorsintotheoperationalscene.Insummary,thehumanoperatorisatthesametimephysicallydistantfromwhatheorsheisobserving(unlikeanaircraftpilot)andmentallyclosetoit,whichresultsinthefosteringofa‘PlayStation’mentality(andcorrelateddisease,suchasformsofpost-traumaticstressdisorder).22Theimpactofunmannedvehiclesonthewayforceisusedincontemporaryscenarios,aswellastheirimplicationsforinternationallaw,havebeenexploredindetailbyscholars.23Cyberweaponsandunmannedweaponssystemsensurestandofffromharm,greatefficiencyand,toadegree,anonymityfortheirusers.ThesedevicesareboundtoprogressevenfurtherowingtodevelopmentsinthefieldofAI-relatedtechniques,suchasmachinelearning.Increasinglymorefunctionswillbeentrustedtothemachine,whichwillbeabletodeliberateparticularcoursesofactionbasedoninformationgatheredandinstructionsgivenbytheoperator.RecallingSchmitt’squoteattheopeningofthischapter,whatdoesourepochprovideuswith?Whichweapondoweneed?Statistically,attackscarriedoutthrougharmeddroneshavebeenfoundtobehighlyimprecise:thisismainlyowingtopoordecision-makingbyhumanagents.24Beingable20PWSinger,WiredforWar.TheRoboticsRevolutionandConflictinthe21stCentury(PenguinBooks2009)60.21GChamayou,Théoriedudrone(LaFabriqueEditions2013)37.22‘ReportoftheUnitedNationsSpecialRapporteuronextrajudicial,summaryorarbitraryexecutions,PhilipAlston’(28May2010)A/HRC/14/24/Add.6para84.23FRuschi,‘Ilvolodeldrone.Versounaguerrapostumana?Unariflessionedifilosofiadeldirittointernazionale’(2016)13JuraGentium12;CHeyns,DAkande,LHill-CawthorneandTChengeta,‘TheInternationalFrameworkRegulatingtheUseofArmedDrones’(2016)65ICLQ791.24JDorseyandNAmaral,‘MilitaryDronesinEurope.EnsuringTransparencyandAccountability’,ChathamHouseResearchPaper(April2021).SeealsoCMeloni,DiegoMauri-9781802207675DownloadedfromPubFactoryat05/28/202203:48:32AMviaDukeUniversityLawLibrary

168Autonomousweaponssystemsandtheprotectionofthehumanpersontooperateclosertothehostileenvironmentissurelyanasset;reducingriskofmisjudgmentswithoutlosingefficiencywouldbeavaluableassettoo.Dronesarestillloadedwithahumanpresence,whichmayexposethemtopoorandnot-so-rapiddecisions.Atoolisneededforoperatingwiththesameriskexposuregrantedbydrones,butmorequicklyandefficiently—morehumanely,ifpossible.Arguably,ourepochneeds‘robots’,andimmediatelytheyarethere.Perfectlysuitablefor‘dirty,dull,anddangerous’jobs,theseAI-poweredsystemswouldnotexposetheiruserstoanydirectharmand,moreimportantly,theywouldnotsufferfromcloudedjudgmentsoremotionaloverreactions—alltoohumandrawbacksresponsibleforthecommissionofegregiouscrimesinhistory.Asoneoftheoften-quotedmantrasinsupportofAWSputsit,‘robotsdon’trape’:whilerightfullyshroudedwithcontroversy,thisaffirmationsuggeststhatwell-programmedautonomoussystemswouldsimplybeincapableofcommittingatrocitiesinthefield.25Addingthusanothersteptotheevolutionarypatternsketchedabove,thecurrentfrontierintheuseofforcecanbelabelledas‘AI-powered’:technologiesfeaturingmoreorlessadvancedAItechniquesarecombinedwithsystemscapableofreleasingforceintherealworld.Theadventofthisweaponry,featuringsoftwareandhardwarecomponents,isoftenregardedasarevolutioninmilitaryaffairs.Itisnotthefirsttimeinhistorythatparticularprogressisregardedasrevolutionary:wealreadyencounteredthisexpressionwhenbrowsingchemical-poweredforce.WhileitisundeniablethatAWSwillresultinanincreasedremoval(ordispersal)ofhumanpresencefromtheactofdeliberatingaparticularuseofforce,theassumptionfromwhichthisbookstartsisthatthefootprintofawalkingrobot-soldiercanbeconsideredtheresultoflong,millennialevolutionarypatterns,itsmainfeaturesbeingclearlydetectableinpreviousarmaments.Havingsaidthis,whatremainsconstantinallthoseevolutionarystepsisthat,inthemosttrivialform,forceismeanttobeemployedagainstobjectsandindividuals.Asregardsthelatter,thesemaybeenemycombatantsduringhostilities—warfareisthedomaininwhichweaponshavebeentraditionallydevelopedanddeployed—or,morerecently,suspectsduringlawenforce-mentorpolicingoperations.Inbothinstances,whatisatstakeisthehumanpersonandtheircorerights,suchaslifeandphysicalintegrity.Asshowninthefollowing,theprotectionofthehumanpersonisoneofthevaluesthat‘StateandIndividualResponsibilityforTargetedKillingsbyDrones’,inEDiNucciandFSantonideSio(eds),DronesandResponsibility:Legal,PhilosophicalandSocio-TechnicalPerspectivesonRemotelyControlledWeapons(Routledge2016).25MHomayounnejadandREOverill,‘The“RobotsDon’tRapeControversy”’,inJGow,EDijxhoorn,RKerrandGVerdirame(eds),RoutledgeHandbookofWar,LawandTechnology(Routledge2019).DiegoMauri-9781802207675DownloadedfromPubFactoryat05/28/202203:48:32AMviaDukeUniversityLawLibrary

17Introduction9contemporaryinternationallawstrivestodefend,bothduringhostilitiesandinlawenforcementoperations.Oneofthelensesthroughwhichtherelation-shipbetweeninternationallawandtechnology—specifically,AWS—canbestudiedisthustherulesandprinciplesoftheinternationallegalorderthataimtoprotectthehumanpersonfromtheuseofforcebystates.Atleastinthepasttwocenturies,numerousnormsofinternationallawhaveflourishedtorestrainstates’powertoapplyforceagainstindividuals:Statesarenotfreetousewhatevermeansortechniquetodoso:limitationsexist,andmostoftheselimitationsareneededtoprotectthehumanperson—ascombatant,civilianor,morerecently,inhimselforherself.1.3SETTINGSAIL:RESEARCHQUESTIONSANDSTRUCTUREOFTHEBOOKTheanalysisconductedthusfarbegsthefollowinginterrogatives:howdocon-straintstotheuseofforceagainstindividualsapplywhenAWSarefielded?Whichrulesandprinciplescouldbeimpinged?Isthereariskofaccountabilitygapsopeningup?IsthecurrentdebateonAWSgoingintherightdirection,iftheobjectiveistoprovideadequateregulationofthisnewweaponry?Isexistinglawsuitableforcopingwithit,orshouldnewlawbeadopted?ThisbookaddressesthechallengesposedbyAWStointernationalnormsstrivingtoprotectandpromotethehumanperson,adistinctivetraitoftheinternationallegalorder—atleastinthepastcentury.Thosenormsarecontainedininter-nationalhumanrightslaw(IHRL)andinternationalhumanitarianlaw(IHL).Thisbookstravelsalongbeatentracks,butwiththeaimofdiscoveringnewtrails.Tothisend,itreliesonmapdetailsthatarerarelyfollowed.Letusnowillustratethemoutofmetaphor.Ontheonehand,beatentracksare:(1)thelong-lastingquandaryastowhethertheemergingnotionofmeaningfulhumancontrol(MHC)—encapsulatingtheneedthatincreasinglyautonomoussystemsbeplacedunderacertaintypeofcontrolbyhumanoperators—ismerelyadiplomaticmovetohelptheinternationaldebateonAWSprogress;(2)theanalysisofprimaryobligationspertainingtoIHRLandIHLandtheimpactthatAWSisexpectedtohaveonthem;and(3)theissueofvacuumsopeningupinthelawofinter-nationalresponsibility(accountabilitygaps)asaconsequenceofAI-poweredweaponry.Legalscholarshiphasbeenengaginginextensivediscussionsofthosetopics.Ontheotherhand,mapdetailsleadingusdiscovernewtrailsarerepresentedbytwinnotionsthatarefrequentlyinvokedinthedebateonAWS,namely,‘humandignity’and‘humanity’.AsthemainstreamapproachtoAWSgoes,theseweaponssystemsshouldbetested—asoccursforanyotherweaponry—againstnormsregulatingtheuseofforceagainsthumantargets.First,whileDiegoMauri-9781802207675DownloadedfromPubFactoryat05/28/202203:48:32AMviaDukeUniversityLawLibrary

1810AutonomousweaponssystemsandtheprotectionofthehumanpersonIHRLandIHLdivergesignificantlyastotherequirementsforemployingpotentiallylethalforce,theybothcontainrulesimposingadutytodistinguishbetweenpermissibleandimpermissibletargets,todosetheforcethatcanbeused,toadoptprecautionsand,even,toconductinvestigationsintoparticularlossesoflife.26Thisbookaddressestheserulesbyhighlightingthechallengesthatfuturehuman–machineinteractionsarelikelytoposetothem.However,ifourinquiryterminatedhere,theanalysiswouldbeincomplete.Providedthatallrulesarerespectedinagivenscenario(thatis,permissibletargetsareselectedandengaged,andtheforceappliedlivesuptothelegalrequirementsauthorizingit),canitstillbesaidthatforcedeliveredthroughAWSisinlinewiththetarget’sinherentworth?Thisiswheretheconceptsof‘humandignity’and‘humanity’kickin.Ourmainargumentisthat,iftheimpactofAWSonthenormsprotectingthehumanpersonistobeassessed,wecannotavoidathoroughreflectionbasedonthosetwinnotions—astheyarefrequentlyinvokedinthedebateonAWS,totheextentthatthereisalmostnostateoractorquestioningtheirimportance.27Yetimbuedwithevidentethicalvalues,theirlegalrelevanceisrarelydealtwith—thisholdstrueingeneralaswellasforAWSspecifically.Thethesesthisbookadvancescanbesummarizedasfollows:(1)humandignityandhumanityencapsulatethesamecorevalue,namely,theneedthat,inbotharmedconflictandlawenforcement,humanbeings’inherentworthbeprotected;28(2)normatively,theyqualifyandoperateas‘generalprinciples’of,respectively,IHRLandIHL;29(3)theycanthusbeemployedtoadvanceaprincipledinterpretationofexistinglaw(delegelataperspective)andtopromotenewlaw(delegeferendaperspective).30However,whetherthosegeneralprinciplesofhumandignityandhumanitycanbeunderstoodasimposingadegreeofhumancontroloncriticalfunctionsperformedbyAWS(essentially,targetselectionandengagement)—totheextentthatAWSoperatingwithoutacceptableformsofhumancontrolwouldbeillegal—isanopenandparamountquestion,whichremainsmostlyneglectedinthedebate.Onthisbasis,ouranalysisfirstframestheissueofAWSbytakingintoaccountthediscussionsthathavebeeninitiatedintheUnitedNations(UN)frameworkalmostadecadeago(Chapter2).Areviewofthemainpositionsheldbyrelevantactors(bothstateandnon-state)willhelpunderstandthecontentandthelimitsofthosedebates,whichseemtobeincapableofleading26SeeCh3,ss3.2–3.4andCh4,ss4.2–4.4.(Thisandallsimilarcitationsrefertochaptersandsectionsinthisvolume.)27SeeCh3,s3.5.2andCh4,s4.5.3.28SeeCh2,s2.3.4.29SeeCh3,s3.5andCh4,s4.5.30SeeCh6,s6.2.DiegoMauri-9781802207675DownloadedfromPubFactoryat05/28/202203:48:32AMviaDukeUniversityLawLibrary

19Introduction11tosubstantiveconclusions.Thischapterfocusesonthemostsalientcom-ponentsofAWSbeforeturningtothelawunderwhichtheanalysiswillbeconducted:asourfocusisontheprotectionofthehumanperson,weconsiderIHRLandIHLquabranchesofinternationallawregulatinghow(potentiallylethal)forcecanbeusedagainstindividuals.ThetheoreticalmodelsoftherelationshipbetweenIHRLandIHLarepresentedattheendofthechapter,whichwillallowforadvancingournotionofhumandignityandhumanityasthetheoreticalcompasstorelyonduringtheanalysisofprimarynorms.InChapters3and4(which,onaparwithhumandignityandhumanity,canberegardedastwinchapters),wedelvedeeperinprimarynormsofinter-nationallawpertainingtoIHRLandIHL,inthatorder;themethodologicalreasonsforthischoiceareexplainedlater,oncetherelationshipbetweenthosetwobranchesofinternationallawisappraised.31Incidentally,thisorderistheoreticallysound:IHRLstrivestoprotectthehumanpersoninallcontexts,whileIHLputsinplaceguaranteesthatarespecificforgivencontexts.TheissueofthecompatibilityofAWSwithIHRLandIHLhasreceivedextensiveattentioninscholarship:themainrulesgoverningtheuseofforceagainstindividuals,especiallyduringarmedconflict,havebeenexploredindetail.However,extantlegalanalysesareunsatisfactorywhenitcomestoassessingAWSagainstthegeneralprinciplesofhumandignityandhumanity.Inlinewiththebook’smainthesis,thesetwinchaptersaimatdrawingcommonconclusionsastotheextenttowhichthegeneralprinciplesofhumandignityandhumanityimposehumancontrolovercriticalfunctionsoftheweaponssystems.Thebookthenturns,inChapter5,totheissueofresponsibility.ThecurrentnarrativeonAWScentersmostlyontheallegedexistenceofaccountabilitygapsinevitablyresultingfromtheuseofthisweaponry,whichwouldbeenoughtosponsortheiroutrightprohibition.Thegoaloftheanalysisistoprovewhetherthisargumentiswellgrounded.First,individualresponsibilityunderinternationalcriminallaw(ICL)isexamined.Second,theroleoflegalentities,suchascompanies,producingAWSisscrutinizedthroughthelensoftheiraccountability:theemergingfieldofbusinessandhumanrights(B&HR)offersapromisinglegalbackgroundagainstwhichAWScanbeassessed.Finally,theanalysisturnstothesubjectofinternationallawthatwillnaturallyresorttoAWSinexercisingitsinherentprerogatives,namely,states.Here,theanalysisfollowsamethodologicallyorthodoxpath,bycoveringboththesub-jectiveandtheobjectiveelementsthatconstitutetheinternationalwrongdoingaspertheDraftArticlesontheResponsibilityofStatesforInternationally31SeeCh2,s2.3.2.DiegoMauri-9781802207675DownloadedfromPubFactoryat05/28/202203:48:32AMviaDukeUniversityLawLibrary

2012AutonomousweaponssystemsandtheprotectionofthehumanpersonWrongfulActs(ARSIWA).32Thischapterisofparamountimportanceforourmainthesis:admittingthataccountabilitygapscanbefilled,theoverarchingquestionremainsofwhetherAWSliveuptothegeneralprinciplesofhumandignityandhumanity.SothedebateonAWSdoesnotterminatewiththeadop-tionoftechniquesforallocatingresponsibilityincasethingsgowrong:evenwhenAWSbehaveastheyareexpectedtoandtargettherighttargets,issuesofhumandignityandhumanitystillarise.Inlightofprimarynormsandthelawofinternationalresponsibilityinitsvariousforms,ourfinalstepisanattempttopavethewayforfurtherreflec-tions,bycombiningbothdelegelataanddelegeferendaperspectivesthathaveemergedthusfar(Chapter6).First,thenotionofMHCisreconstruedtoassessitslegalnature:doesitstandasanindependentruleorprincipleofinter-nationallaw,oristhismerelyarhetoricaldevicethatrisksmuddlingtheissue?Second,andconsequently,arenewedappraisalofthecategoryof‘generalprinciples’thatisofrelevancewhendiscussingAWSisproposed:whilemostlyneglectedduringthedebate,thislegalcategory—ifusedproperly—couldbeusefullyinvokedbystatesandotheractorsasalegalyardsticktorein-forcetheirpositions.Third,theproposedappraisalofhumancontrolthroughthelensofgeneralprinciplesofIHRLandIHLwillallowfortheextensionofourreflectionstoothertechnologiesand,eventually,foraglobalappreciationoftherolethatinternationallawcanplayvis-à-vistroublingchangesinthewayforceis(andwillbe)employedagainstindividuals.Inconclusion,discussingAWSfromaperspectiveembracinghumandignityandhumanityaslegallodestarsistimelyandundelayable,asthisfavorsabetterframingoftheargumentsputforwardabouthowtoregulateAWS.Theinternationalcommunityishavingahardtimeprovidinganeffec-tivenormativeframeworkforAWS:existinglawisdeemedinsufficient,andadoptingnewlawappearsarduous.Statesthataremoreinterestedindevel-opingAWS‘buytime’andopposeanylegallybindinginstrumentonAWS;thoseactorsthatareagainsttheincreasinglyautonomousweaponrypushforanewtreatyorattheveryleastnon-bindingnormstoorientthedebate,andbasetheirargumentsonconceptssuchasthoseofhumandignityandhumanitywithoutadequatelysubstantiatingthemfromalegalstandpoint.Againstthisbackground,asoundandprincipledreflectiononthelawasitis(lexlata)andthelawasitshouldbetocopewiththesechallenges(lexferenda)—twodistinctbutnotentirelysegregateddimensions33—isgreatlyneeded,toclarify32ILC,‘ResponsibilityofStatesforInternationallyWrongfulActswithCommentaries’(9June2001)A/CN.4/SER.A/2001/Add.1.33ACassese,‘Soliloquy’,inACassese,TheHumanDimensionofInternationalLaw:SelectedPapersofAntonioCassese(PGaetaandSZappalàeds,OUP2008).DiegoMauri-9781802207675DownloadedfromPubFactoryat05/28/202203:48:32AMviaDukeUniversityLawLibrary

21Introduction13ambiguitiesandhelpthedebateprogress.Thefinalgoalremainstheeffectiveprotectionofthehumanpersonintheinternationallegalorder.Thisbookisintendedtoofferahumblecontributiontothatreflection.DiegoMauri-9781802207675DownloadedfromPubFactoryat05/28/202203:48:32AMviaDukeUniversityLawLibrary

222.SketchingthedebateIfthereaderengagedinaquickreconnaissanceofkeydocumentsonauton-omousweaponssystems(AWS),heorshemaybesurprisedtocomeacrossadenseforestofacronyms,definitionsandconceptsthatseemtooffernowayout.Expressionssuchas‘autonomous’—asopposedto‘semi-autonomous’,‘automated’or‘fullyautonomous’—weaponssystems,‘humancontrol’(attimesprecededby‘meaningful’orreplacedby‘appropriatelevelsofhumanjudgment’)areoftenjuxtaposedinliststhatvarygreatly,dependingontheirauthors.Thereadermayevenfeeldiscomfortatwhatappearstobeplayingwithwords:quotingHumptyDumptyinthefamousLewisCarroll’sThroughtheLookingGlass,‘whenIuseaword…itmeansjustwhatIchooseittomean’.Togiveanexampleofthisterminologicaljumble,theborderlinebetweenarmeddronesandAWSdoesnotseemsoclear-cut.IntheIntroductiontothisbookitwasarguedthatAWScanbeunderstoodasthelaststepintheevolu-tionarypatternofprojectingforcefromafar:theytakethecoreideaofarmeddronestoahigherlevelinthattheydelegatespecificcriticaldecisionstothemachine.1However,keepingAWSandarmeddronesdistinct,whilerelativelyeasyintheory,mayturnouttobetrickyinpractice:sometimes,evenwhenhumaninterventionisrequiredtodischargeforceagainstthetarget(aswithexistingunmannedsystems),humandecision-makersarepresentedwithavastnumberofcomputer-generateddatathattheyarenotinapositiontomeaning-fullyreviewandthus,inmanyinstances,tocontest.Whilethesystemdoesnottakedecisionsautonomously(thatis,withouthumanintervention),humanpresenceseemsnothingmorethannominal.Inadditiontothisempiricaldif-ficulty,thenumerousdocuments—reports,positionandworkingpapers,statedeclarationsandacademicarticles—dedicatedtothesenewtechnologiesandtheuseofforceagainstindividualsfailtoproperlydistinguishbetweenAWSandothersystems.Havingareliablecompassisvitalinordernottogetlost.Thischapteraimstoexplainthesebasicnotionsandprovideaworkingdefi-nitionofAWSandassociatedconcepts,beforeventuringintothelegalanaly-sisofthetechnology.Forthispurpose,thechaptersummarizesthekeypointsofthedebateonAWSthathaveemergedthusfar(section2.1).Theoverview1SeeCh1,s1.2.14DiegoMauri-9781802207675DownloadedfromPubFactoryat05/28/202203:48:39AMviaDukeUniversityLawLibrary

23Sketchingthedebate15isthenfollowedbythedelimitationofthesubject-matterofthebook:thethreefoundationalconceptsof‘autonomy’,‘criticalfunctions’and‘humancontrol’areexplained(section2.2).Thethirdsection(2.3)isdevotedtoframingthesubject-matterinlegalterms,thatis,fromtheperspectiveofinternationallaw.Concludingremarksareprovidedinsection2.4.2.1GENERALOVERVIEWOFTHEDEBATETheinternationaldebateonAWShaslastedforyears,sothematerialthathasbeenamassedthusfar(encompassingstatements,workingpapers,commonpositions,politicaldeclarations,andsoon)isimmense.Itisthereforeappro-priatetosortthroughthedebate’smainsteps,withaviewtoinsulatingthecoreissuesthatwillbetheobjectsofthefollowingchapters.Forthispurpose,itmustbedisclosedinadvancethattherehasbeennorevolutionaryadvance-mentintheunderstandingofcoreissuesrelatedtoAWS;instead,actorspartic-ipatinginthedebatehavenarroweddowntheirfocustospecificelementsthatraisedmoreattentionthanothers.Twomainphasesofthedebatecanbeoutlined.Thefirst(section2.1.1)takesplacebeforetheissueofAWSwastakenupwithintheframeworkoftheConventiononCertainConventionalWeapons(CCW),whenmeetingsofstatesandnon-stateactorsbegan,in2014,andarestillconvenedonaregularbasis.Atthattimethedebatewassituatedmostlyalongthelinesofscholarlyworks,andaturningpointwasrepresentedbytheReportofthe(then)UNSpecialRapporteuronextrajudicial,summaryorarbitraryexecutionsonAWStotheHumanRightsCouncil.Thatmarkedthebeginningofstates’andotheractors’interestinAWS.Thesecondphase(2.1.2),stillongoing,isrepresentedbythedebatehostedbytheCCW.2.1.1FromEarlyDocumentstotheHeynsReportWhileitisalwayshardtopinpointadateinthetimelineincontextssuchasthis,itcanbearguedthattheinternationalcommunity—understoodhereinalimitedfashion,thatis,asgroupsofstatesgatheredwithintheframeworkofinternationalorganizations(IOs)—establishedtoshowinterestinthetopicofAWSin2013grossomodo.ItwasthatyearwhenthethenUNSpecialRapporteuronextrajudicial,summaryorarbitraryexecutions,ChristofHeyns,submitteda22-pagereporttotheUNHumanRightsCouncil(UNHRC)tack-lingtheissueof‘Lethalautonomousrobotsandtheprotectionoflife’(hereaf-DiegoMauri-9781802207675DownloadedfromPubFactoryat05/28/202203:48:39AMviaDukeUniversityLawLibrary

2416Autonomousweaponssystemsandtheprotectionofthehumanpersonter:HeynsReport).2Thisdocumentwassorelevantthatitisstillregardedasareferencepointincurrentdebates.Beforeturningtotheanalysisofthereportanditskeypassages,however,itisimportanttoshedlightonthebackgrounddocumentsthattoanextentinspiredtheSpecialRapporteur’sanalysis.Withoutalookback,keypassagesofthereportcouldhardlybegrasped.First,scholarshiphadalreadydedicatedsomecontributions—mostlydoctrinalarticles—toAWS,frombothatechni-calandmoral/legalperspective.3Statesaswellhadalreadybeguntopublishdocumentssettingtheirpolicyvis-à-vistheuseofautonomyformilitarysystems:forinstance,theUKfirsttackledtheissueinthe2011JointDoctrineNote2/11.4Thedoctrine,whichisnolongerinforce,distinguishedbetween‘remotelypilotedaircraft(system)’and‘automated’/‘autonomous’systems,andprovidedajointgeneralanalysisofthethreeclassesofweapons.Thatis,theissueofAWSdidnotreceivespecificattentionbytheMinistryofDefence(MoD).Inananalogousfashion,thesameyear(2011),ontheoccasionofthe31stInternationalConferenceoftheRedCrossandtheRedCrescent,areportpreparedbytheInternationalCommitteeoftheRedCross(ICRC)anddedicatedtothechallengesposedbynewtechnologiestointernationalhuman-itarianlaw(IHL)wassubmittedforthediscussion:5forthefirsttime,theissueofAWSwasexpresslyanalyzedinanofficialreportbytheorganization.Whilethismaysuggestthattheconferencewasthefirstforuminwhichstateactorsofficiallydiscussedthetopic,itmustbenotedthatnoofficialstatementonAWSwasinsertedinanyfinalreportsoftheconference.Inadditiontothis,AWSweredealtwithundertheumbrellaof‘newtechnologies’onaparwithotherweaponsandplatforms.Theywerenotmadetheobjectofspecificattention.TwodatesinNovember2012changedeverythingandtookthe(thenembry-onic)debatetothenextlevel.Overthecourseoftwodays,twodiametricallyopposeddocumentswerereleased:theUSDepartmentofDefense(DoD)’sDirective3000.096and,onlyafewhourspreviously,HumanRightsWatch(HRW)’sreport‘LosingHumanity’,draftedintandemwiththeInternational2‘ReportoftheUnitedNationsSpecialRapporteuronExtrajudicial,SummaryorArbitraryExecutions,ChristofHeyns’(2013)A/HRC/23/47.3AKrishnan,KillerRobots:LegalityandEthicalityofAutonomousWeapons(Routledge2009).4UKMoD,‘JointDoctrineNote2/11:TheUKApproachtoUnmannedAircraftSystems’(JDN2/11,2011).5ICRC,‘InternationalHumanitarianLawandtheChallengesofContemporaryArmedConflicts’,Reportofthe31stInternationalConferenceoftheRedCrossandRedCrescent(October2011)38ff.6USDoD,‘DirectiveNumber3000.09’(21November2012,revisited8May2017)(Directive3000.09).DiegoMauri-9781802207675DownloadedfromPubFactoryat05/28/202203:48:39AMviaDukeUniversityLawLibrary

25Sketchingthedebate17HumanRightsClinicatHarvardLawSchool.7Whiletheformertakesaposi-tiveapproachtoincreasedautonomyinmilitarysystems,andhasinviewtheadventofAWS,thelatterstrivestoexposethenegativesideeffectsassociatedwiththatnewtechnology,andmarksthelaunchofthepro-bancampaign.Acoalitionofnon-governmentalorganizations(NGOs)hadestablishedtheCampaigntoStopKillerRobots(CSKR)onlyonemonthbefore.Theseantitheticaldocumentswereimportantforatleasttworeasons.First,theyprovidedearlydefinitionsofAWSthatresonatedinsubsequentdebates:forinstance,accordingtoDirective3000.09,‘autonomousweaponssystems’arethose‘that,onceactivated,canselectandengagetargetswithoutfurtherinterventionbyahumanoperator’.8ThisdefinitionisparticularlyeffectiveasitcapturesthekeyelementsofAWS,asdiscussedbelow.Second,criticalissuesassociatedwithAWSwerealreadyzoomedinon.WhiletheDoD’sDirective3000.09containedmostlytechnicaldirectionsvis-à-visthedeploy-mentof‘autonomous’and‘semi-autonomous’weaponssystems,‘LosingHumanity’conductedalegalandmoralanalysisofAWS,assessingboththeircompatibilitywithIHLandtheaccountabilitygapsthattheirusemayopenup.Importantly,‘LosingHumanity’hastobecreditedfortheoftenuseddis-tinctionamongsystemsenvisaging‘human-in-the-loop’,‘human-on-the-loop’and‘human-out-of-the-loop’modesofcontrol.Theargumentslaiddownbythereportstimulatedthereactionsofseveralscholars,whotookthesideofeitherthepro-banfrontorthepro-AWSone,somestandingonamiddleground.Bythen,themachineryofdebatingAWShadbeensetinmotion.ThisledthentotheHeynsReport.ItissomehowpropheticthatthefirstinternationalforumthatdealtwithAWSwastheUNHumanRightsCouncil.Asarguedmoreextensivelyinthefollowing,aperspectivebasedprimarilyonIHRLhasbeenmissingsincetheissueofAWSwastakenupbystateswithintheCCWframework,whereIHL’sobligationsaretheprimaryconcern.TheHeynsReportdedicatedanentiresectionto‘lethalautonomousrobots’(LAR)—anexpressionlaterabandoned,byHeynshimselftoo,owingtoitsvagueness—andexaminedtheircompatibilitywithbothIHLandIHRL.Theanalysiswasthusexpandedtostateactivitiesinvolvingtheuseofforceoutsidearmedconflict,namely,forlaw-enforcementpurposes.TheHeynsReportleansheavilyondefinitionsandnotionscontainedinpreviousschol-arship,statedocuments,theDoD’sDirective3000.09and,evenmoreso,in‘LosingHumanity’,forinstanceinreaffirmingtheconceptofretaininghumanpresenceintheoperational‘loop’.Afteracomprehensivelegalandethical7HumanRightsWatchandInternationalHumanRightsClinicatHarvardLawSchool,‘LosingHumanity.TheCaseagainstKillerRobots’(19November2012).8Directive3000.09(n6)GlossaryptII.DiegoMauri-9781802207675DownloadedfromPubFactoryat05/28/202203:48:39AMviaDukeUniversityLawLibrary

2618Autonomousweaponssystemsandtheprotectionofthehumanpersonanalysis—touchinguponalsoissuesrelatedtodisarmamentanduseofforceininterstaterelations—thereportrecommendsstatestoplacenationalmoratoriaonLARsasanimmediatestep,callingonIOstosupportthemoratoriaandengageinafruitfuldebateonthistechnology.9Regrettably,therecommenda-tionaddressedtotheUNregardingtheconventionofa‘HighPanelonLARs’bytheUNHighCommissionerforHumanRightshasnotbeenimplementedtodate.10AlmostatthesametimeastheHeynsReportwassubmittedtotheHumanRightsCouncil,twootherdocumentsdedicatedtoAWSwereissued.ThefirstistheUNSecretary-General(UNSG)’sreportontheprotectionofciviliansinarmedconflictof2013,submittedtotheUNSecurityCouncil.11Forthefirsttimeinmorethantenyearsofreportssubmittedonthattopic,theUNSGaddressednotonlytheissueofarmeddrones,butalso‘autonomousweaponssystems,orwhatareknownas“killerrobots”’andtheircompliancewith‘internationalhumanitarianandhumanrightslaw’.12Hence,stateswereurgedtoengagein‘discussions’tobeinitiated‘immediatelyandnotoncethetechnologyhasbeendevelopedandproliferated’,participationthereintobeextendedto‘UnitedNationsactors,ICRCandcivilsociety’.13TheissueofAWSwastakenupalsobysubsequentreports.14Theseconddocumentisanotherreport—thistimecommissionedbytheEuropeanUnion’s(EU’s)Parliament’sSecretary-General—whichfocusedontheimplicationsofarmeddronesand‘unmannedrobots’ontheenjoymentofhumanrights.15Whiletheabove-mentioneddistinctionbetweensystemshavinghumans‘in’,‘on’and‘outof’theoperationalloopwaspresent,16thereportfocusedmoreonremotelypilotedplatforms,endowedwithlowdegreesofautonomy,andcontainedalegalanalysisofthistechnologybothinsideandoutsidearmedconflict.Issuesrelatedtointernationalresponsibilityandtheroleofinternationaladjudicatorybodieswerecoveredaswell.TogetherwiththeHeynsReport,thisseconddocumentdeservescreditinthatitidentified9HeynsReport(n2)paras118and116.10HeynsReport(n2)para114.11‘ReportoftheSecretary-GeneralontheProtectionofCiviliansinArmedConflict’(22November2013)S/2013/689.12Ibidpara29.13Ibid.14Seemorerecently‘ProtectionofCiviliansinArmedConflict.ReportoftheSecretary-General’(6May2020)S/2020/266.15Directorate-GeneralforExternalPoliciesoftheUnion,PolicyDepartment,‘HumanRightsImplicationsoftheUsageofDronesandUnmannedRobotsinWarfare’(2013)EXPO/B/DROI/2012/12.16Ibid6.DiegoMauri-9781802207675DownloadedfromPubFactoryat05/28/202203:48:39AMviaDukeUniversityLawLibrary

27Sketchingthedebate19basiclegalissues,therebysettingthestageforthesoon-to-comeinternationaldebateonAWS.2.1.2TheDebateintheCCWFramework(andBeyond)AfterabriefappearanceattheUNHRC,thetopicofAWSwastakenupbystateswithinadifferentframework—attheveryleast,theydidimplementoneoftheHeynsReport’srecommendations.17The2013MeetingoftheStatespartytotheCCWdecidedtoconveneameetingofexperts(MoE)inthefollowingyear,withthemandateto‘discussthequestionsrelatedtoemergingtechnologiesintheareaoflethalautonomousweaponssystems,inthecontextoftheobjectivesandpurposesoftheConvention’.18DuringthefirstMoE(2014),threefeaturesemergedthathavecharacterizedthedebatetodate.First,theobjectofthedebateare‘lethal’AWS(hencetheacronymLAWS):inusingthisexpression,statesstressakeyfeatureofAWS,namely,thattheiruseislikelytoaffecttargets’bodilyintegritytothepointofcausingdeath.Second,theopportunitythatrepresentativesofNGOs(suchasHRW,AmnestyInternationalandothermembersofthenew-bornCSKR)andcivilsociety(suchasuniversitiesandresearchinstitutes)joinedthedebateonaparwithstates(bothpartiestotheCCW,onlysignatoriesandnon-partiesasobservers)waswelcomed,andtheirroleas‘awareness-raisers’onAWSfullyacknowledged.19Thesenon-stateactorstooktheflooronnumerousoccasionsduringthedebate,and—asinlaterMoEs—contributedtothedebatebysub-mittingnumerousworkingpapers.ThiscircumstanceisnotpeculiartoAWS:theincreasingparticipationofNGOsandcivilsocietyinfieldsregardingweapons,disarmamentandarmscontrolcanbeinscribedinamoregeneralandrecenttrendcharacterizinginternationallaw-making.20Third,asmadeclear17HeynsReport(n2)para121:‘[p]articipateininternationaldebateandtrans-governmentaldialogueontheissueofLARs’.18MeetingoftheHighContractingPartiestotheConventiononProhibitionsorRestrictionsontheUseofCertainConventionalWeaponsWhichMayBeDeemedtoBeExcessivelyInjuriousortoHaveIndiscriminateEffects,‘Finalreport’(16December2013)CCW/MSP/2013/10para32.19MeetingoftheHighContractingPartiestotheConventiononProhibitionsorRestrictionsontheUseofCertainConventionalWeaponsWhichMayBeDeemedtoBeExcessivelyInjuriousortoHaveIndiscriminateEffects,‘Reportofthe2014infor-malMeetingofExpertsonLethalAutonomousWeaponsSystems(LAWS)’(11June2014)CCW/MSP/2014/3para15.20CRyngaert,‘Non-StateActors:CarvingoutaSpaceinaState-CentredInternationalLegalSystem’(2016)63NetherlandsInt’lLR183;Jd’Aspremont(ed),ParticipationintheInternationalLegalSystem.MultiplePerspectivesonNon-stateActorsinInternationalLaw(Routledge2011).DiegoMauri-9781802207675DownloadedfromPubFactoryat05/28/202203:48:39AMviaDukeUniversityLawLibrary

2820Autonomousweaponssystemsandtheprotectionofthehumanpersonbytheagendaofthemeeting,themainlegalfocuswasoncompliancewithIHL(IHRLbeingrelegatedtoasecondaryrole)andpossibleaccountabilitygapsresultingfromAWS;thedebatealsotoucheduponkeyprinciplessuchas‘humandignity’andtheMartensClause.21ThefirstMoEwasthenfollowedbyasecond,in2015,22andathird,in2016.23Statesandnon-stateactorsdiscussedtechnicalaspectsandlegalissuesatlength.Thedebateaddressedalsothewidespreadandproblematicconceptof‘meaningfulhumancontrol’(MHC),asregardswhichdelegationswereincapableofreachinganagreement,asisillustratedmorethoroughlylater.24Whatismoreimportant,the2016reportcontainedsomerecommendationsthatweresubmittedtotheReviewConferenceoftheCCW;inparticular,theestablishmentofan‘open-endedGroupofGovernmentalExperts(GGE)inaccordancewithestablishedpractice[oftheCCW]’wassuggestedasawaytoenhancethedebateonAWS.25ThefirstGGEwasthenconvenedinNovember2017.ItaddressedthetopicofAWSfromamulti-facetedperspective,rangingfromtechnologicalchallengestomilitaryeffects,fromlegaltoethicalquestions,andstrovetocovercross-cuttingissues.26TheGGEwascapableofidentifyingthreeareasofworkforsubsequentsessions,namely:(1)thedefinitionaleffortinsearchingforacommonunderstandingofAWS;(2)theassessmentofhuman–machineinteractionscharacterizingthosesystems;(3)theproposalofpossiblelegaloptionsforregulatingthem.27Afourthissuewasaddedduringthe201821‘Reportofthe2014InformalMeetingofExperts’(n19)paras26–33.22MeetingoftheHighContractingPartiestotheConventiononProhibitionsorRestrictionsontheUseofCertainConventionalWeaponsWhichMayBeDeemedtoBeExcessivelyInjuriousortoHaveIndiscriminateEffects,‘Reportofthe2015InformalMeetingofExpertsonLethalAutonomousWeaponsSystems(LAWS)’(2June2015)CCW/MSP/2015/3.23FifthReviewConferenceoftheHighContractingPartiestotheConventiononProhibitionsorRestrictionsontheUseofCertainConventionalWeaponsWhichMayBeDeemedtoBeExcessivelyInjuriousortoHaveIndiscriminateEffects,‘Reportofthe2016InformalMeetingofExpertsonLethalAutonomousWeaponsSystems(LAWS)’(10June2016)CCW/CONF.V/2.24Ibidpara15.Seeinfras2.2.3.25IbidAnnexpara3.26GroupofGovernmentalExpertsoftheHighContractingPartiestotheConventiononProhibitionsorRestrictionsontheUseofCertainConventionalWeaponsWhichMayBeDeemedtoBeExcessivelyInjuriousortoHaveIndiscriminateEffects,‘Reportofthe2017GroupofGovernmentalExpertsonLethalAutonomousWeaponsSystems(LAWS)’(20November2017)CCW/GGE.1/2017/CRP.1para12.27Ibid.DiegoMauri-9781802207675DownloadedfromPubFactoryat05/28/202203:48:39AMviaDukeUniversityLawLibrary

29Sketchingthedebate21session,thatis,possiblemilitaryimplicationsrelatedtoAWS—atopicthatagaindemonstratesthemulti-facetedapproachanimatingtheCCWforum.28Importantly,theGGEagreedonalistofconclusionsandrecommenda-tionsthatwasinsertedinthe2017reportandsummarizedthekeypointsthatemergedduringthedebate.29Inthe2018session,thislastwasfurtherdis-cussedandeventuallytransformedinto‘PossibleGuidingPrinciples’.30Atfirstsight,theseprinciplesreaffirmsolemnlythecontinuingimportanceforIHLto‘applyfully’toAWS,31aswellastheneedforretaining‘humanresponsibility’fordecisionsregardingtheuseofAWS.32The2019sessionoftheGGEaddedanotherguidingprinciplefocusingspecificallyonhuman–machineinterac-tion.33However,theeffectivescopeoftheseprinciplesmustnotbeoveresti-mated:inadditiontobeingvoidofbindingforce,theyhavebeenpurposefullyformulatedinvagueterms,astheyaretheresultofanegotiatingprocessbasedontheruleofconsensus.Theextenttowhichtheircontent,inspiteoftheirnon-bindingnature,correspondstotreatyandcustomaryinternationallawisexploredthroughoutthisbook.Inaccordancewiththedeliberationsofthe2019MeetingoftheHighContractingPartiestotheCCW,the2020–2021sessionsoftheGGEonAWSwereexpectedto‘exploreandagreeonpossiblerecommendationsonoptionsrelatedtoemergingtechnologiesintheareaoflethalautonomousweaponssystems…takingintoaccountallproposals(past,presentandfuture)’.34WhiletheSARS-CoV2pandemicrisksputtingoffforthcomingdiscussions(astheydidthe2020secondroundofdiscussions),theexpectationisgrowing28GroupofGovernmentalExpertsoftheHighContractingPartiestotheConventiononProhibitionsorRestrictionsontheUseofCertainConventionalWeaponsWhichMayBeDeemedtoBeExcessivelyInjuriousortoHaveIndiscriminateEffects,‘Reportofthe2018sessionoftheGroupofGovernmentalExpertsonEmergingTechnologiesintheAreaofLethalAutonomousWeaponsSystems(LAWS)’(23October2018)CCW/GGE.1/2018/3.29‘Reportofthe2017GroupofGovernmentalExperts’(n26)para16.30‘Reportofthe2018GroupofGovernmentalExperts’(n28)para21.31Ibidprinciple(a).32Ibidprinciple(b).33GroupofGovernmentalExpertsoftheHighContractingPartiestotheConventiononProhibitionsorRestrictionsontheUseofCertainConventionalWeaponsWhichMayBeDeemedtoBeExcessivelyInjuriousortoHaveIndiscriminateEffects,‘Reportofthe2019sessionoftheGroupofGovernmentalExpertsonEmergingTechnologiesintheAreaofLethalAutonomousWeaponsSystems(LAWS)’(25September2019)CCW/GGE.1/2019/3AnnexIV(GuidingPrinciples)principle(c).34MeetingoftheHighContractingPartiestotheConventiononProhibitionsorRestrictionsontheUseofCertainConventionalWeaponsWhichMayBeDeemedtoBeExcessivelyInjuriousortoHaveIndiscriminateEffects,‘FinalReport’(13December2019)CCW/MSP/2019/9para31.DiegoMauri-9781802207675DownloadedfromPubFactoryat05/28/202203:48:39AMviaDukeUniversityLawLibrary

3022AutonomousweaponssystemsandtheprotectionofthehumanpersonthatstateswillbeabletoagreeonalegaldocumentonAWS.Again,thenatureofsuchdocumentsisopenfordebate.MostNGOsandseveralstates—atleast30—callfortheadoptionofalegallybindinginstrumentonAWS,35apro-posalthathasbeenofficiallyformalizedalsoduringtheGGEmeetingsbytheNon-AlignedMovementGroup,36bytheAfricanUnion37andbyajointdocu-mentsignedbyAustria,BrazilandChile.38Othersconsideritmoreappropriatetoagreeonapoliticaldeclaration—diplomaticallyamoreattainablegoal,yetcomingatthecostofpreciselegalregulation.39Otherstates(includingmajormilitarypowerssuchastheUS,theUK,IsraelandRussia)eitherdonotstatetheirpositionoropposeanyproposalstoregulateAWS,callingsuchamove‘premature’giventhepaceoftechnologicaladvancementsinthefield.40WhiletheGGEremains,andislikelytoremaininthenearfuture,themainforumfordebatingAWS,twootherfieldsofdiscussionareworthyofmention.First,attheinternationallevel,theUNGA’sFirstCommittee,whichdealswithdisarmamentandinternationalsecurity,hasbeenconsideringthetopicofAWSforacoupleofyearsnow.41Forinstance,accordingtotheCSKR,nolessthan41stateshavetackledtheissueofAWSattheUNGA,withinterventionsbyhigh-rankinggovernmentmembers(suchasMinistersofForeignAffairs)35Foranupdatedoverview,seeHumanRightsWatch,‘StoppingKillerRobots.CountryPositionsonBanningFullyAutonomousWeaponsandRetainingHumanControl’(10August2020).36Venezuela,‘GeneralPrinciplesonLethalAutonomousWeaponsSystems.WorkingPaperSubmittedonBehalfoftheNonAlignedMovement(NAM)andOtherStatesPartiestotheConventiononConventionalWeaponsGroupofGovernmentalExpertsonLethalAutonomousWeaponsSystems’(28March2018)CCW/GGE.1/2018/WP.1.37Benin,‘StatementonBehalfoftheAfricanGrouptotheConventiononConventionalWeaponsGroupofGovernmentalExpertsonLethalAutonomousWeaponsSystems’(9April2018).38Austria,Brazil,andChile,‘ProposalforaMandatetoNegotiateaLegally-bindingInstrumentthatAddressestheLegal,HumanitarianandEthicalConcernsPosedbyEmergingTechnologiesintheAreaofLethalAutonomousWeaponsSystems(LAWS)’(8August2018)CCW/GGE.2/2018/WP.7.39FranceandGermany,‘ForConsiderationbytheGroupofGovernmentalExpertsonLethalAutonomousWeaponsSystems(LAWS)’(7November2017)CCW/GGE.1/2017/WP.4para12.40UK,StatementforAgendaitem5(e)(25–29March2019)(statingharshlythat‘thecontinueddivergenceofviewsafterseveralyearsofdiscussionsmakestheformu-lationofamutuallyacceptable,practicalandenforceablelegalinstrumentunlikelyinthenearfuture–intheCCWoranyotherforum’).41UNGA,‘ConventiononProhibitionsorRestrictionsontheUseofCertainConventionalWeaponsWhichMayBeDeemedtoBeExcessivelyInjuriousortoHaveIndiscriminateEffects.ReportoftheFirstCommittee’(15November2019)A/74/372.DiegoMauri-9781802207675DownloadedfromPubFactoryat05/28/202203:48:39AMviaDukeUniversityLawLibrary

31Sketchingthedebate23duringthehigh-levelopeningofthesession.42PositionstendtooverlaplargelywiththoseexpressedattheCCW.Second,atthedomesticlevel,manystateorgans(mostlypertainingtothelegislativebranch)havebeguntofocustheirattentiononthetopic.Curiously,somedomesticauthoritiestakeadiametri-callyopposedpositionvis-à-visthepositionsupportedbytheexecutiveintheinternationalforum.Forexample,theParliamentoftheNetherlands—afierceopposertoanylegallybindinginstrumentonAWS43—haspassedaresolu-tioninMay2019callingforabanonAWS.44SimilarresolutionshavebeenadoptedbytheParliamentsofBelgium,45aswellastheEUParliament.46Theseinitiativesmustnotbeunderestimated:whilelittlemaybedoneinfosteringthedebateininternationalfora,domesticauthoritiesquastateauthoritiescontrib-utetodevelopingpracticeonAWS,whichisofrelevanceforthepurposesofouranalysis.Attheendofthisoverview,thetimehascometodrawsomeconclusionsfromalltheabove.TheCCWremains‘anappropriateframeworkfordealingwiththeissue’ofAWS:themainreasonliesinthattheConventionisrecognizedaroleofstriking‘abalancebetweenmilitarynecessityandhumanitarianconsidera-tions’.47AsisarguedmoreextensivelyinChapter3,thesetermscontainaclearreferencetoIHL,whichisconsideredtobethemainbodyofinternationallawofrelevanceinthedebateonAWS.Thisisalsoconfirmedwhen‘appli-cableinternationallaw’isreferredto,asoftentheformula‘inparticularIHL’followsimmediately.48MaintainingthedebateintheCCWforumimpliesthecentralityofIHLinthelegaldiscourseonAWS,anissuewhichistheobjectofcriticalremarksinthefollowing.Inadditiontothis,inspiteofmorethansevenyearsofdiscussions,theCCWforumhasbeenabletoprovideneitherclear-cutdefinitionsnoranormative42CSKR,‘HighLevelConcernsonKillerRobotsatUN’(30October2019)https://www.stopkillerrobots.org/2019/10/unga74/accessed31August2021.43TheNetherlands,Statementatthe2016MoE(referringtoproposalsofmorato-riumas‘inexpedientandunfeasible’).44GovernmentoftheNetherlands,ParliamentaryMotionbyMemberSvenKoopmansetalios,‘Overbeheersingvandeproductie,plaatsing,verspreidingeninzetvannieuwepotentiëlemassavernietigingswapens’(19May2019)KST3369443.45ChambredesRépresentantsdeBelgique,‘Résolutionvisantàinterdirel’uti-lisation,parlaDéfensebelge,derobotstueursetdedronesarmés’(19July2018)3203/3005.46EuropeanParliament,Resolution2018/2752(RSP)of12September2018onautonomousweaponsystems[2018]OJC433/86.47GuidingPrinciples,principle(k);Reportofthe2018GroupofGovernmentalExperts’(n28)para21lit(j).48GuidingPrinciples,principle(c).DiegoMauri-9781802207675DownloadedfromPubFactoryat05/28/202203:48:39AMviaDukeUniversityLawLibrary

3224AutonomousweaponssystemsandtheprotectionofthehumanpersontooltoregulateAWS.Atellingexampleofthisisthecumbersometerminologyemployedtodescribetheobjectofthedebate,namely,‘emergingtechnologiesintheareaoflethalautonomousweaponssystems’:thewidescopeofthisformula,coupledwiththepersistingideawherebyitismoreimportanttoadoptworkingdefinitionsthatdonothamperthedebate,initselfmakesitdifficulttoprogresssubstantiallytowardsanyregulatoryoutcome.Technologicallyadvancedstates(suchastheUS,UK,Israel,RussiaandChina,aswellassomeEuropeanstates)fearthattoo-specificdefinitionswhichcouldincludeexistingplatformsmayjeopardizetheirmilitaryprograms.Insummary:nospecificdefinitionisinsight.Asaconsequenceforthepurposesofthepresentanalysis,itiskeytoinsulatethebasicconceptsthatcharacterizeAWSandtofocusattentiononeachofthem.2.2WHATISINANAME?UNRAVELLINGDEFINITIONALKNOTSInthepreviousparagraphsithasbeenarguedthatstatesandotheractorstendtopreferopen-textured,ifnotpurposelyvague,definitionsofAWSsoasnottoincludeexistingtechnologiesandhampersoon-to-bedevelopmentsintechnology.Lackofspecificdefinitiondoesnotimplythatcertainelements,commontothenumerousdefinitionsproposedsofar,cannotbesingledout.Forinstance,inordertodefine‘autonomy’,referenceisoftenmadetotheconcurringnotionof‘humancontrol’:intuitively,itmaybecontendedthatthemoretheformerincreases,themorethelatterdecreases,inasortofinverseproportionalityratio.However,thescenarioismorecomplex.Plus,towriteintermsof‘autonomy’and‘humancontrol’withoutreferringtotheparticularfunctionsunderscrutinymayleadtomeagerresults.Insummary,insteadofseekingone-size-fits-alldefinitions(which,inprac-tice,haveaslimmerchanceofbeingadoptedinthedebate),itispreferabletoprovidethebasiccoordinatestograsppeculiarcharacteristicsofAWS.Forthatpurpose,thissectionexplorestheconceptsof‘autonomy’(2.2.1),‘criticalfunctions’(2.2.2)and‘humancontrol’(2.2.3).2.2.1TheMeaningof‘Autonomy’inMachines‘Autonomy’isaconceptequallyintuitivetograspandtrickytodefine.ItderivesfromtheGreekautos(thatis,self)andnomos(thatis,law),andsodescribesthehumancharacteristicofbeingabletogiveoneselfyourownlaw.Thenotionofautonomyhasbeendevelopedinnumerousfields:forexample,intheologyitisdescribedas‘therightandfreedomofself-determinationasDiegoMauri-9781802207675DownloadedfromPubFactoryat05/28/202203:48:39AMviaDukeUniversityLawLibrary

33Sketchingthedebate25contrastedwithdeterminationbysomebodyorsomethingelse(heteronomy)’.49Inphilosophy,Kantdefendedtheconceptofmoralautonomyastheabilitytoretainfullauthorityoverone’saction,thusbeingcapableofself-governing(asopposedtobeingobedienttoanexternallyimposedmorality,whichwouldimplytheabove-mentionednotionofheteronomy).50Whenappliedtonon-humanbeings(suchasmachines)thenotionofauton-omyisusuallyreshapedasfollows.First,itsopposite(thatis,heteronomy)isunderstoodasarelationshipofdependencebetweenmachinesandhumans:inbrief,themoreamachineisdependentonahumanbeingwhenperformingitstasks,thelessitcanbedescribedasautonomous.Inter-machineindepend-enceisnotrelevantinrelationtoautonomy.Second,literaturetendstoadoptanarrowdefinitionofautonomy,deliberatelyignoringitsmoralimplications.Forhumanstobeautonomousitisessentialthattheycanactinaccordancewithmoralprinciples:humanbeings‘canmakethingshappenintentionallyandforreasons’.51Conversely,thenotionsof‘will’,‘intention’and‘moralagency’becomeoflittleornoutilitywhendiscussingautonomyinrespectofmachines.Ithasbeenarguedthatthereisnointerestwithinthemilitaryinbuilding‘anethically-infalliblemachine’;rather,andlesspretentiously,thegoalistohave‘amachinethatperformsbetterthanhumansdoonthebattlefield’.52Thisfunctionalistapproachemploysanotionofautonomythatfocusesexclusivelyonperformance(thatis,carryingoutagiventaskoptimally),notontheethicalbackgroundsofanaction.Itfollowsthatamachineisautono-mousnotwhenitmakesmoraljudgmentsonitsown(thatis,withoutrelyingonexternalmoralinfluence),butsimpliciterwhenitperformsanactionintherealworldwithoutexternalintervention.Autonomyinmachinesthereforeismorallyneutral,asdecisionsarenottheresultofamoraljudgment.Onthispremise,thefollowingdefinitionofautonomyhasbeenproposed:‘thecapacitytooperateinthereal-worldenvironmentwithoutanyformofexternalcontrol,oncethemachineisactivatedandatleastinsomeareasofoperation,forextendedperiodsoftime’.53Autonomyisthereforeconceived49‘Autonomy’,inARichardsonandJBowden(eds),TheWestminsterDictionaryofChristianTheology(JohnKnoxPress1983)60.50TEHill,‘KantianAutonomyandContemporaryIdeasofAutonomy’,inOSensen(ed),KantOnMoralAutonomy(CUP2012).51Ibid18.52PLin,GBekeyandKAbney,‘AutonomousMilitaryRobotics:Risk,EthicsandDesign’,ReportfortheUSDepartmentofNavy,OfficeofNavalResearch(20December2008)2(emphasisadded).53Ibid105(emphasisadded).Theauthorsdefine‘robot’as‘apoweredmachinethat(1)senses,(2)thinks(inadeliberative,non-mechanicalsense),and(3)acts’(ibid100).DiegoMauri-9781802207675DownloadedfromPubFactoryat05/28/202203:48:39AMviaDukeUniversityLawLibrary

3426Autonomousweaponssystemsandtheprotectionofthehumanpersonasthemachine’scapabilityof(1)organizingdatapreviouslygatheredviasensorsand(2)performinganactioninaccordancewiththosedata,(3)withoutrelyingonahumanoperator.Similarly,inatellingcomparisonwithbiologicalsystems,ithasbeenemphasizedthatcapabilityofoperatingintherealworldindependentlyofhumancontrolandadaptabilitytochangingenvironmentareimportantfeaturesoflivingorganismsthat,tosomeextentanddependingonthecurrentstatusoftechnology,canbereplicatedinrobotics.54Algorithmsarethemediumthroughwhichmachineautonomyoperates.Castsimply,analgorithmisa‘processorsetofrulestobefollowedincal-culationsorotherproblem-solvingoperations,especiallybyacomputer’.55Inbroaderterms,itis‘anyprocessthatcanbecarriedoutautomatically’.56Systemsemployingalgorithmsforsupportingdecision-makingareprimarilydesignedeithertoanticipateoutcomesthatarenotyetknowableforcertain,ortodetectandcategorizesomethingunknownbutknowableusinginference.57Thisdescriptionofalgorithmsforautonomousdecision-makingprocessesisparticularlyhelpfulforouranalysis.Autonomousweaponssystemswillemployalgorithmicsystemsallowingfortakingreal-lifedecisionsregardingwhethertoselectandengageaparticulartargetornot.Bydoingso,admittedlythesystemwillnotknowthetargetsbutwillbeabletoinscribethemincertaincategoriesiftheymatchthecorrespondingrequirements,andaparticularcourseofactionwillbedeliberatedonthebasisofthatcategorization.Animportantcaveatisduenow.Nomatterhowfarautonomypushesmachinesinthefieldofperformingmorecomplextasks,humanpresenceisalwayspresentataparticularpointoftheprocess.Itisuptohumanpro-grammerstowritethealgorithmsthroughwhichthemachineoperates;again,everyrobotstructurallyembodiesacontrolarchitecture(atsoftwarelevel)thatallowsahumanoperatortosupervise,controlandeventuallydeactivatethemachine.58Thisisakeyelementforthepurposesofthepresentanalysis.Thiscircumstance—theinescapabilityofsomeformofhumanpresence,atleastattheverybeginning(theactofprogrammingamachine)andinperforming54GBekey,AutonomousRobots.FromBiologicalInspirationtoImplementationandControl(MITPress2005)1–2(arguingthatitisexpectablethat‘robots[will]…exhibitincreasinglevelsofautonomyandintelligenceinthenearfeature’,inawaythatisgoingtobeincreasinglycomparabletolivingspecies).55OxfordEnglishDictionary,‘Algorithm’https://www.oed.com/view/Entry/4959?redirectedFrom=algorithm#eidaccessed31August2021.56JChabert(ed),AHistoryofAlgorithms:FromthePebbletotheMicrochip(Springer1999)2.57LEdwardsandMVeale,‘SlavetotheAlgorithm?Whya“RighttoanExplanation”isProbablyNottheRemedyYouareLookingFor’(2017)16DukeL&TechReview18,24.58Chabert(n56)3ff.DiegoMauri-9781802207675DownloadedfromPubFactoryat05/28/202203:48:39AMviaDukeUniversityLawLibrary

35Sketchingthedebate27(some)tasks—leadstoatrivial,yetmeaningful,conclusion:autonomyisbutasophisticatedcamouflageforheteronomy.Itisuptohumanstoconfermachinesautonomy:metaphorically,itisforhumansto‘drawtheline’andthusdecidehowmuchthemachinecanperformwithoutthem.2.2.1.1‘Static’conceptionsofautonomyAccordingtomanyactorsinvolvedinthedebateonAWS,thereisacleardividebetweenwhatcanbedefinedasautonomyandwhatcannot:autonomyisseenasastaticnotion,monolithic,showingdiscreteboundaries.Asanexampleofthistendency,theICRCraisedageneraldefinitionofautonomyinits2011reportdedicatedtoIHLandchallengesincontemporaryarmedconflicts.59Indefining‘autonomousweaponssystems’,theICRCstatesthatthistypeofweaponsystemis‘onethatcanlearnoradaptitsfunctioninginresponsetochangingcircumstancesintheenvironmentinwhichitisdeployed’.60Bycontrast,an‘automatedweaponorweaponssystems’is‘onethatisabletofunctioninaself-containedandindependentmanneralthoughitsemploymentmayinitiallybedeployedordirectedbyahumanoperator’.61TheICRCtakestheexampleofasentrygunthatopensfireuponvoiceverificationofapotentialintruder.Asensorgatherstherelevantdata(thatis,thevoice),thecomputationalprogramelaboratesthem(thatis,matchesitwithregisteredvoices)andtheactuatorsperformtheappropriateaction(thatis,firesorenablesaccessbythesubject).Clear-cutcategoriesof‘autonomy’,‘semi-autonomy’and‘automation’arepresentinearlymilitarydocumentsonAWS,whicharetoberegardedasele-mentsofpracticeinthisemergingfield.The2011UKMoD’sJointDoctrineNotedefinesautonomoussystemsas‘capableofunderstandinghigher-levelintentanddirection…[and]ofdecidingonacourseofactionfromamonganumberofalternatives,withoutdependingonhumanoversightandcontrol,althoughthesemaystillbepresent’,while‘automated’systemsarethose‘that,inresponsetoinputsfromoneormoresensors,[are]programmedtologicallyfollowapredefinedsetofrulesinordertoprovideanoutcome’.62Thesedefini-tionsarerepeatedverbatimintheupdated2017JointDoctrine,63andhavebeencriticizedbytheHouseofLords’SelectCommitteeonArtificialIntelligence59‘InternationalHumanitarianLawandtheChallengesofContemporaryArmedConflicts’(n5).60Ibid39.61Ibid(originalemphasis).62SeeJointDoctrine(n4)2,3.63UKMoD,JointDoctrinePublication0-30.2:UnmannedAircraftSystems(JDP0-30.2,2017)9.DiegoMauri-9781802207675DownloadedfromPubFactoryat05/28/202203:48:39AMviaDukeUniversityLawLibrary

3628Autonomousweaponssystemsandtheprotectionofthehumanpersonforbeingsubstantiallydivergentfromthoseofotherstates,whichwouldriskisolatingtheUKintheinternationaldebate.64Anoften-quoteddefinitionofautonomyinmilitarysystemsiscontainedintheUSDoD’sDirective3000.09.65‘Autonomous’weaponssystemsarethosethatonceactivated,canselectandengagetargetswithoutfurtherinterventionbyahumanoperator.Thisincludeshuman-supervisedautonomousweaponsystemsthataredesignedtoallowhumanoperatorstooverrideoperationoftheweaponsystem,butcanselectandengagetargetswithoutfurtherhumaninputafteractivation.Asstressedbythedirective,humanpresenceisguaranteed:humanoperators,whileretainingspecificprerogatives(forexample,toabortthemission),havealimited—ifnotnegligible—roleintheperformanceofparticularfunctions,suchastargetselectionandengagement.Onthecontrary,human-supervisedweaponssystemsandsemi-autonomousweaponsystemsenvisageamoresubstantialroleforhumanoperators,respectively,to‘interveneandterminateengagements,includingintheeventofaweaponsystemfailure,beforeunac-ceptablelevelsofdamageoccur’andtoselect‘individualtargetsorspecifictargetgroups’(leavingthesubsequentengagementtotheweaponsystem).66Asanexampleofthesesystems,‘fire-and-forget’missilesarecited.Thisallowsustosingleoutasubjectthatisanalyzedindetaillater:thefunctionsassociatedwithincreasingautonomymatter.OtherkeydocumentsonAWSadoptastaticapproachtothenotionofautonomy.TheHeynsReportdescribesLARsas‘roboticweaponsystemsthat,onceactivated,canselectandengagetargetswithoutfurtherinterventionbyahumanoperator.Theimportantelementisthattherobothasanautonomous“choice”regardingselectionofatargetandtheuseoflethalforce’.67Thisdefi-nition,inwhichtheconceptofautomationasopposedtoautonomyremainsimplicit,attachesgreaterimportancetothespecificfunctionperformedbythemachine(namely,targetselectionandengagement).Bythesametoken,HRW’sreport‘LosingHumanity’distinguishesamong(1)human-in-the-loop64HouseofLords,SelectionCommitteeonArtificialIntelligence,‘AIintheUK:Ready,WillingandAble?’(16April2018)HLPaper100.65Directive3000.09(n6)13.66Ibid14.67HeynsReport(n2)38.DiegoMauri-9781802207675DownloadedfromPubFactoryat05/28/202203:48:39AMviaDukeUniversityLawLibrary

37Sketchingthedebate29weapons,68(2)human-on-the-loopweapons69and(3)human-out-of-the-loopweapons.70Thisdistinctionisremarkablefortworeasons.First,theloopthesedefinitionsrefertoisa‘shortloop’,asitislimitedtothesoledecision-makingprocessthatculminatesintheuseofforceagainsttheselectedtarget:itthereforefitsperfectlyintotheobjectofouranalysis.Second,thisthree-layerdefinitioncapturestheintensityofhumanpresenceasregardssomefunctions(asintheHeynsReport)thatareofgreaterinterest.DuringtheCCWdebate,manystateshaveproposedtheirdefinitionsofautonomy,mostlybycirculatingworkingpapersonthesubject.TheNetherlandsunderstandsan‘autonomousweaponsystem’asaweaponthat,withouthumanintervention,selectsandengagestargetsmatchingcertainpredefinedcriteria,followingahumandecisiontodeploytheweaponontheunderstandingthatanattack,oncelaunched,cannotbestoppedbyhumanintervention.71EstoniaandFinlandmakeadifferencebetween(1)automation,(2)auton-omyand(3)independence.72Importantly,thetwodelegationsmakethepointthatthecategoriesmaynotbeofrealhelpinadvancingthediscus-sions:forinstance,aweaponssystemthatmayoperateaccordingtoasetofpre-programmedrules(asforautomatedsystems)doesnotimplyabsenceofunforeseenresults,astheoperationalenvironmentmaybesuchthattheweaponsystemfollowsanunexpectedcourseofaction.73AccordingtoRussia,‘autonomousweaponssystems’aredefinedas‘anunmannedpieceoftechni-calequipmentthatisnotamunitionandisdesignedtoperformmilitaryandsupporttasksunderremotecontrolbyanoperator,autonomouslyorusingthecombinationofthesemethods’,while‘semi-autonomousweaponsystem’68HumanRightsWatchandInternationalHumanRightsClinicatHarvardLawSchool(n7)2:‘robotsthatcanselecttargetsanddeliverforceonlywithahumancommand’.69Ibid:‘robotsthatcanselecttargetsanddeliverforceundertheoversightofahumanoperatorwhocanoverridetherobots’actions’.70Ibid:‘robotsthatarecapableofselectingtargetsanddeliveringforcewithoutanyhumaninputorinteraction’.71TheNetherlands,‘ExaminationofVariousDimensionsofEmergingTechnologiesintheAreaofLethalAutonomousWeaponsSystems,intheContextoftheObjectivesandPurposesoftheConvention’(9October2017)CCW/GGE.1/2017/WP.2para5.72EstoniaandFinland,‘CategorizingLethalAutonomousWeaponsSystems–ATechnicalandLegalPerspectivetoUnderstandingLAWS’(24August2018)CCW/GGE.2/2018/WP.2paras4ff.73Ibidpara6.DiegoMauri-9781802207675DownloadedfromPubFactoryat05/28/202203:48:39AMviaDukeUniversityLawLibrary

3830Autonomousweaponssystemsandtheprotectionofthehumanpersonisasystem‘requiringinvolvementoftheoperator’.74Herethedividinglinebetweenthetwocategoriesdoesnotseemsovisible—perhaps,ratherthanaweakness,thishastoberegardedasastrength.Assomanyactorsdrawtheirdividinglinesbetweenthesecategories,adifferentapproachtoautonomymaybepreferable.Insummary,thereseemstobeapointinavoidingstrict,andultimatelysuffocating,definitions.2.2.1.2‘Dynamic’conceptionsofautonomyAnotherapproachto‘autonomy’istoconsideritnotasa‘monolith’,butasa‘spectrum’describingvariouslevelsofinteractionbetweenthehumanoperatorandthemachine.Inthisrespect,ithasbeenarguedthatautonomyismorethanmereindependencefromhumans:itsurelyincludesit,butitdoesnotcoincidewithit.Developingthislineofthought,amulti-facetedapproachtodefiningautonomyhasbeenadoptedwhichfocuseson‘commonsetsoftraits’whichapplyacrosstypesofmachinesystems.75Thesecommonsetsare(1)frequencyofoperatorinteraction(automation),(2)toleranceforenviron-mentaluncertainty(adaptability)and(3)levelofassertivenessorabilitytochangeanoperativeplantobetteraccomplishtheassignedmissionwithoutaspecificinterventionbytheoperator(discretion).76Changingadefined(flex-ible)planinordertoachieveadefined(rigid)goalcanbedescribedasaformofcreativity.77Assumingthattheabove-mentionedsetsoftraitsmayvaryinscale,somehaveinferredthatdrawingabrightlinebetweenautomationandautonomyisimpossible;instead,thelattershouldbeseenasacontinuumoraspectrum.78Amongmany,engineerSheridanhasproposedwhathasbeenregardedasthe‘bestknownscale’formeasuringautonomy:79histen-levelmodelisaccom-paniedbyprecisecriteriaagainstwhichtheautonomyofagivensystemcan74RussianFederation,‘Russia’sApproachestotheElaborationofaWorkingDefinitionandBasicFunctionsofLethalAutonomousWeaponsSystemsintheContextofthePurposesandObjectivesoftheConvention’(4April2018)CCW/GGE.1/2018/WP.6para2.75WCMarraandSKMcNeil,‘Understanding“TheLoop”:RegulatingtheNextGenerationofWarMachines’(2013)36HarvardJofL&PublicPolicy1139.76Ibid1148ff.77OGClark,RKokandRLacroix,‘MindandAutonomyinEngineeredBiosystems’(1999)12EngineeringApplicationsofArtificialIntelligence389,390(‘[a]lthoughabsolutecreativityisanunattainablegoal,anyentitywithsomedegreeofautonomymustbecreativeenoughtoformulateatleastafewofitsowngoalsandbehavioralguidelines’).78Ibid.79GCoppinandFLegras,‘AutonomySpectrumandPerformancePerceptionIssuesinSwarmSupervisoryControl’(2012)100ProceedingsoftheIEEE590,592.DiegoMauri-9781802207675DownloadedfromPubFactoryat05/28/202203:48:39AMviaDukeUniversityLawLibrary

39Sketchingthedebate31beobjectivelyevaluated.80Level1correspondstoautomation,whilelevel10indicatesfullautonomy;betweenthetwoextremes,thesystemisendowedwithcrescentautonomousfunctionalities.81Inbetween,thesystem‘offersacompletesetofactions’(level2),‘narrowstheselectiondowntoafew’(3),or‘suggestsone’(4)and‘executesthatsuggestionifthehumanapproves’(5),‘allowsthehumanarestrictedtimetovetobeforeautomaticexecution’(6),‘executesautomatically,thennecessarilyinformsthehuman’(7),‘informsthehumanafterexecutiononlyifthehumanasks’(8),or‘informsthehumanafterexecutionifit,thecomputer,decidestodoso’(9).82Farfrombeingatechnicalproposal,theconceptionofautonomyasadynamicscalehasbeenendorsedbysomestatesaswell.Forexample,theUSAirForceResearchLaboratory(AFRL)hasproposedits11-levelautonomyspectrum(morefocusedonhumandependenceatthedecisionandactionstagesoftheobserve,orient,decide,act—OODA—loop):remotelypilotedvehicle(1);executepre-plannedmissionremotely(2);changeablemission(3);robustresponsetoreal-timefaults/events(4)fault/eventadaptivevehicle(5);real-timemulti-vehiclecoor-dination(6);real-timemulti-vehiclecooperation(7);battlespaceknowledge(8);battlespacesinglecognizance(9);battlespaceswarmcognizance(10);fullyautonomous(11).83Spectrumsareusefulforclassifyingbothexistingandfutureweapons:forinstance,theNazilandtorpedoGoliath,aremote-controlledvehiclecarryingexplosivestobedrivenintoenemytanks,doesnotexceedlevel0ofSheridan’sscale,byreasonofitsinherentdependenceonhuman–machineinteraction,lackofenvironmentalawarenessanddiscretion.Analogously,thefamousTomahawkmissilemusthaveitstargetsetpriortolaunch,inadditiontobeingunabletodynamicallyreactandadapttoenvironmentaluncertainty,thusplacingitatnomorethanlevel1ofAFRL’sspectrum.84Recentunmannedaerialvehicles(suchasthePredator,theReaperandtheGlobalHawk),aswellasgroundvehicles(suchasthePackbotandtheSWORDS)andmaritimevehi-80MarraandMcNeil(n75).81RParasumaran,TBSheridanandCDWickens,‘AModelforTypesandLevelsofHumanInteractionwithAutomation’(2000)30ProceedingsoftheIEEE286,287.82Ibid.83ESholes,‘EvolutionofaUAVAutonomyClassificationTaxonomy’,Remarksatthe2007IEEEAerospaceConference(2007)http://ieeexplore.ieee.org/stamp/stamp.jsp?tp=&arnumber=4161585accessed31August2021,1.Athoroughcompar-isonbetweenSheridan’smodelandtheAirForce’smodeliscarriedoutbyMarraandMcNeil(n75)1163–4(arguingthatthelatteraccuratelyshowsthatagivensystemmay‘mixandmatch’autonomyandautomationwithregardtodifferentstages).84Level1indicatesthatamachine‘execute[s]pre-plannedmissionremotely’,withoutbeingabletomodifysuchmissions(whichisthedegreeofautonomyattainedatlevel2):seeSholes(n83)3.DiegoMauri-9781802207675DownloadedfromPubFactoryat05/28/202203:48:39AMviaDukeUniversityLawLibrary

4032Autonomousweaponssystemsandtheprotectionofthehumanpersoncles(suchastheAegisseadefensesystem)althoughtechnicallyapttoauton-omousfunctioning,arecurrentlyoperatedatthelowerendofthespectrum.85Tosumup,conceivingautonomyasaspectrumandhumaninterventionasascaleallowsforabetterunderstandingofhowAWScanbeexpectedtooperate,andmayalsomakeiteasiertocompareandcontrastthedefinitionsproposedabove.Forexample,fromtheperspectiveofthehuman–machinerelationship,onemayeasilypairHRW’scategorieswiththoseoftheDoD’sDirective3000.09:‘semi-autonomousweaponsystems’wouldcor-respondto‘human-in-the-loopweapons’;‘human-supervisedautonomousweapons’to‘human-on-the-loopweapons’;‘fullyautonomousweapons’to‘human-out-of-the-loop’.However,thisoperationmayturnouttobeinconclu-sive,ifotherconcepts—thefunctionstobeperformedbythemachineandthedegreeofhumaninterventiontherein—arenotexploredcontextually.2.2.2The‘Functions’underAnalysisTheprevioussubsectionhasdemonstratedthattheconceptofautonomylendsitselftomultifariousinterpretation,totheextentthattowriteaboutautonomywithoutconsideringthespecificfunctionsofamachineisatbestpointless.Alandmineandatoasterbothexercisesomefunctionswithouthumaninter-vention(respectively,explodinguponphysicalcontactandtoastingasliceofbread),ergoautonomously,buttheyraisedifferentconcerns.ItisthereforecrucialtounderstandwhichfunctionsaretakenintoconsiderationwhendebatingAWS.Asapreliminaryremark,thedecision-makingprocessthattakesplaceinoperationscontemplatingtheuseofforce(firstandforemostmilitaryopera-tions)iscommonlydistilledintothe‘OODAloop’.Accordingtothismodel,originallydevelopedbyJohnBoydinanattempttojustifyAmericansuperi-orityoverKoreansduringtheKoreanwar,ahumansoldierfirst‘observes’thesurroundingenvironment,gatheringasmuchdataaspossiblethroughhumansenses,then‘orients’himself(orhisvehicle)towardstheadversary,‘decides’whichcourseofactionisbestsuitableforreachingthegoaland,eventually,‘acts’(thatis,executesthedecision).HenceAWSmustbecapableofperform-ingsomeorallthesestepswithoutrelyingonhumanintervention.Inthissubsection,weoutlinethenotionof‘criticalfunctions’andconnectitwiththetargetingprocess.Thiswillhelpusunderstandinwhichfieldsincreas-ingautonomyislikelytoraisethemostintriguingquestions.85Foranin-depthdiscussionofthefeaturesofeachvehicle,seeMarraandMcNeil(n75)1178–85.DiegoMauri-9781802207675DownloadedfromPubFactoryat05/28/202203:48:39AMviaDukeUniversityLawLibrary

41Sketchingthedebate332.2.2.1Theconceptof‘criticalfunctions’:from‘lethality’to‘useofforce’Intheidentificationofthefunctionsthataregoingtobeassociatedwithautonomy,aground-breakingconcepthasbeenputforwardbytheICRCina2014report.86Thereportfocusesonautonomyinrespectofthesole‘criticalfunctions’thataweaponsystemmayperform,namely,‘acquiring,tracking,selectingandattackingtargets’.87Thesefunctionsarecontrastedwith‘non-criticalfunctions’,whichencompassnavigation(forexample,take-off,landing,loiteringandrouteplanning)andsensorcontrol.Theconceptof‘crit-icalfunctions’hasbeenconstantlyre-statedbytheICRC.88Criticalfunctionshavebeenboileddowntotargetselection(whichcomprisesthesearch,identi-fication,trackinganddetectionofaparticulartarget)andengagement(whichconstitutestheactofapplyingforceinordertoneutralize,damageordestroyaparticulartarget).89Towriteintermsof‘criticalfunctions’makesexplicitafeaturethathasbeenimplicitlytakenintoaccountwhenshapingadefinitionofAWS.Directive3000.09understandsautonomyinrespectoftargetselectionandengagement;90similarly,HRWrefersto‘selectingtargetsanddeliveringforce’withcompleteautonomy.91ThisisconfirmedalsobytheobjectofthedebateattheCCW,namely,‘emergingtechnologiesintheareaofLethalAutonomousWeaponsSystems’(emphasisadded).Ithasbeenarguedalreadythattheadjective‘Lethal’hasbeenpresentsincethebeginningofthedebate,whichimpliesthattheweaponsystem’sfunctionsunderanalysisarethosethatareamenabletoprovokingthetarget’sdeath.Expressionssuchas‘deadlyweapons’or‘killerrobots’—extremelypopularamongNGOsandcivilsociety—reflectapeculiarchoiceoffunctionstobeconcernedwith.Somestateshaveproposedabroaderapproach.AccordingtoEstoniaandFinland,whilediscussingthenotionof‘lethality’:Thereisnoclearreasontoexcludeless-than-lethalweaponsfromthediscussion–lethalityisnotadefiningfeatureofanyweaponsystem,autonomousorotherwise.Aninstrumentthatisintendedtocauseless-than-lethalinjuriestopersons,orharm86ICRC,‘AutonomousWeaponsSystems.Technical,Military,LegalandHumanitarianAspects’,ExpertMeeting(March2014).87Ibid62.88ICRC,‘AutonomousWeaponsSystems.ImplicationsofIncreasinglyAutonomyintheCriticalFunctionsofWeapons’,ExpertMeeting(March2016).89Ibid71.90Directive3000.09(n6)13.91HumanRightsWatchandInternationalHumanRightsClinicatHarvardLawSchool(n7)2.DiegoMauri-9781802207675DownloadedfromPubFactoryat05/28/202203:48:39AMviaDukeUniversityLawLibrary

4234Autonomousweaponssystemsandtheprotectionofthehumanpersontoobjects,isnonethelessaweapon.Also,aweaponintendedtobeless-than-lethalmaywellprovetobelethalincertaincircumstances.92However,thismorecomprehensiveapproachhasbeenrejectedbyotherstates.Chinamaintainsastricterapproachto‘lethality’(‘whichmeanssufficientpayload(charge)andformeanstobelethal’).93Morestraightforwardly,Japansupportsthepositionwhereby‘itisappropriatetolimitthediscussiononlytoautonomousweaponssystemwithlethality’.94Therefore,tofillthenotionof‘criticalfunctions’withappropriatecontentitisessentialtodeterminewhetheronlylethal,orpotentiallylethal,orless-than-lethalor,again,non-lethalcoursesofactionaretobeincluded.Besides,thisambiguitystillisanopenissuealsointheinternationaldebate.95AwayoutofthisconceptualambiguityisofferedbythedebateattheCCW.Asthe2018reportstates,severaldelegationsexpressedtheviewthatafocusonlethalitywouldfailtoaddressinjuriestopersonsordamagetoobjectsthatareprotectedbyIHL,…whileothersfeltthattheterm‘lethal’asacharacteristicneededtobefurtherexaminedinthelightofthefundamentalnotionofuseofforce,whichtriggerslegalobligationsunderinternationallawirrespectiveoflethality.96Whenappliedagainstanindividual,forcecanresultinlethalornon-lethalout-comes,irrespectiveofwhethertheweaponisdesignedtokilloronlytoinjure:agunshotmaybedeadlyornotdependingonavarietyofcontingencies.Thecommontraitdeadlyandnon-deadlyoutcomesofaparticularengagementshareisthatbotharetheresultofaparticularuseofforce.Itisthereforeappropriatethattheissueof‘criticalfunctions’beanalyzedthroughtheprismoftheuseofforce.92EstoniaandFinland(n72)para1.93China,‘PositionPaper’(11April2018)CCW/GGE.1/2018/WP.7para3.94Japan,‘Possibleoutcomeof2019GroupofGovernmentalExpertsandFutureActionsofInternationalCommunityonLethalAutonomousWeaponsSystems’(22March2019)CCW/GGE.1/2019/WP.3para5.95UK,‘HumanMachineTouchpoints:TheUnitedKingdom’sPerspectiveonHumanControloverWeaponDevelopmentandTargetingCycles’(8August2018)CCW/GGE.2/2018/WP.1paras7,8(‘[t]heGGEdebateon11April2018sawmanyparticipantscallforhumancontrolof“criticalfunctions”,oftenspecificallyreferringto“selectandengage”,butitwasunclearpreciselywhatthesetermsactuallymean…TheGGEshouldlooktoestablishwhichfunctionsare“critical”andmustbesubjecttohumancontrol,andwhichcouldbedelegatedtomachinesoperatingunderothersafeguards’).96‘Reportofthe2018SessionoftheGroupofGovernmentalExperts’(n28)para12.DiegoMauri-9781802207675DownloadedfromPubFactoryat05/28/202203:48:39AMviaDukeUniversityLawLibrary

43Sketchingthedebate35Whilethereisnointernationaltreatyregulatingexpresslythiscategoryofweapons,non-lethalweapons(NLW)canbedefinedas‘weapon[s]whichareexplicitlydesignedanddevelopedtoincapacitateorrepelpersonnel,withalowprobabilityoffatalityorpermanentinjury,ortodisableequipment,withminimalundesireddamageorimpactontheenvironment’.97Accordingtoothers,‘less-lethal’shouldbepreferred‘sincetheuseofanyweaponcanhavefatalconsequences’.98HeretheexpressionNLWispreferable,astheemphasisisontheintendedeffectthataweaponisdesignedtoproduce(toinjurysolelyinlieuofkilling)—afeaturethatwillbeofuseinthefollowingchapters.Non-lethalweaponsmayincludeacousticweapons,electromagneticweapons,mechanicalandkineticdevices(suchaswatercannons),aswellaschemicalagentsdesignedtoincapacitatetheirtargets.99Non-lethalweaponshavebeenincreasinglyadoptedbystatesnotonlyinarmedconflictscenarios,butalso—preciselybyvirtueoftheirreduceddegreeoflethality—inlaw-enforcementorevenpeace-keepingoperations,inwhichtherelevantmandatedictatesstrictrulesforusingforce.100Thequestionof‘lethality’hasadirectbearingonthenatureofthetargetsconsidered.Autonomousweaponssystemsmay(andprobablywill)beusedagainstnon-humantargets(forexample,enemytanks,compoundsandprop-ertyingeneral),therebyraisingtheissueoftheircompatibilitywithrulesandprinciplesofinternationallaw.However,asexplainedinmoredetailbelow,theinternationalprotectionofthehumanpersoniscurrentlyassuredbyamoresubstantivesetofnormsthanwithregardtoobjects:itisthereforeinlightofthenormsontheuseofforceagainsthumantargetsthat‘criticalfunctions’ofAWSwillbeassessed.Forthepurposesofthepresentwork,theexpression‘criticalfunctions’isunderstoodasthoseresultingintheactualdischargeofforceagainstahumantarget,bothinarmedconflictandinlaw-enforcementsituations.Functionsfallingoutsidetheprocessleadingtotheuseofforce(forexample,flight,navigationandloitering)orsituatedatearlystagesoftheprocess(forexample,identificationofpoliticalandstrategicobjectivestobepursuedduringamili-97NATO,‘NATOPolicyonnon-lethalweapons’(1999,lastupdated2008)https://www.nato.int/cps/ic/natohq/official_texts_27417.htm?selectedLocale=enaccessed31August2021.98GenevaAcademyofInternationalHumanitarianLawandHumanRights,‘GenevaGuidelinesonLess-LethalWeaponsandRelatedEquipmentinLawEnforcement’(July2018).99NATO(n97).100Whilethelegalanalysisofobligationsincumbentonstatesinthesescenariosisleftforthefollowing,sufficeittorecallhereMSossai,‘TheDemandsofFutureOperationsandthePromiseofNon-orLess-LethalWeapons’(2018)21YIHL1.DiegoMauri-9781802207675DownloadedfromPubFactoryat05/28/202203:48:39AMviaDukeUniversityLawLibrary

4436Autonomousweaponssystemsandtheprotectionofthehumanpersontarycampaign),whileprogressivelyinvolvingtheuseofalgorithmsandthusraisingunescapablelegalissues,arenotregardedas‘critical’.2.2.2.2‘Criticalfunctions’andthe‘targetingprocess’:someclarificationsInthedebateonAWS,andmorespecificallyon‘criticalfunctions’and‘humancontrol’,referenceisoftenmadetothe‘targetingprocess’,thatis,thecyclethataimsatthedestructionorneutralizationofopponent’sforcesandequipment.101Towriteintermsof‘targetingprocess’hasundoubtedadvan-tages,butalsopresentssomesetbacks.Startingwiththeformer,thenotionof‘targetingprocess’helpsunderstandthecomplexityoffunctionsinwhichhumansandmachinescaninteract.Bothstrategicandtacticalassessmentsareinvolved.TakingtheexampleoftheNorthAtlanticTreatyOrganization(NATO),thetargetingprocessenvisagesasix-phasecycle:(1)analysisofcommander’sgoals;(2)targetdevelopment;(3)analysisofcapabilities;(4)assignmentofcapabilities;(5)planningandexecutionofthemission;(6)assessmentoftheresults.102Thesecondphaseis,inturn,composedofdifferentsub-steps:targetanalysis,vetting,validation,nominationandprioritization.103Theactualdischargeofforceoccursinphase5,wheretheengagementofthetargettakesplace;however,itsselectionhasbeenmadesubsequenttonumeroussteps,inwhichhumanoperatorsmayhaveinteractedwithmachines(forexample,algorithmsre-elaboratingdatacollectedfromtheoperationalfield).Thatis,thespecificselectionismadebytheAWSastheresultofalongchainofstages,whichcouldbetrackedbackeventopoliticaldecision-makers.Whilethisaspectwillbeofrelevanceasregards‘humancontrol’,104sufficeittostressthattargetselectionandengage-menthavetobeunderstoodwithregardtoanactualdischargeofforceagainstaparticularindividual:theseare‘criticalfunctions’forourpurposes.Thisseemstoreceiveconfirmationbyotherapproaches,whichprefertheemploymentoftheterm‘attack’,mostlyinasmuchasnotionispresentinIHL.Article49AdditionalProtocolIdefinesanattackas‘actsofviolenceagainsttheadversary,whetherinoffenceorindefence’.105Asthecommentarytothis101MACEkelhof,‘LiftingtheFogofTargeting:“AutonomousWeapons”andHumanControlthroughtheLensofMilitaryTargeting’(2018)71NavalWarCollegeR1,4.102NATO,‘AlliedJointDoctrineforJointTargeting.EditionAVersion1’(April2016)AJP-3.9.103Ibid.104Seeinfras2.2.3.105ProtocolAdditionaltotheGenevaConventionsof12August1949,andrelatingtotheProtectionofVictimsofInternationalArmedConflicts(adopted8June1977,enteredintoforce7December1978)1125UNTS3(API)art49.DiegoMauri-9781802207675DownloadedfromPubFactoryat05/28/202203:48:39AMviaDukeUniversityLawLibrary

45Sketchingthedebate37rulemakesclearinrespectofmine-laying,an‘attack’isbelievedtooccurwhenforceisdischargedagainstthetarget.106ProjectingthisonAWS,itmeansthataweaponsystemisconsideredto‘attack’whenitreleasesforce—lethalornot—againstaselectedtarget.However,anapproachbasedexclusivelyonthe‘targetingprocess’turnsouttobemisleading.Asthisconcepthasbeenformulatedinthemilitaryfield,itisunsuitableforcoveringallthecircumstancesinwhichforcecanbeusedagainstahumantargetoutsidethebattlefield,namely,duringlaw-enforcementoperations.Inaccordancewithourcomprehensiveapproachtotheuseofforceasexplainedabove,itispreferabletoadoptamoregeneralunderstandingof‘criticalfunctions’,whichincludethetargetingprocessasitisregulatedinmilitarystrategywithoutbeinglimitedtothisarea.Byconceiving‘criticalfunctions’asthosepertainingtotheactualdischargeofforceagainstaparticu-larhumantarget,scenariosdifferentfromarmedconflictarecomprisedtoo.2.2.3‘HumanControl’:A‘Meaningful’Concept?Aftertheillustrationoftheconceptsof‘autonomy’and‘criticalfunctions’ofaweaponsystem,itistimetoturntoanissuethathasremainedinthebackgroundthusfar,namely,theroleofahumanoperatorthroughouttheperformanceofthe‘criticalfunctions’.Whenitcomestoselectingandengag-ingaparticulartarget,thehumanoperatorhastraditionallyplayedacrucialrole:toshootanarrow,tofireacannonortobombasitehavealwaysbeenhumans’call.Thesameappliestomoremoderntechnologies,suchasarmeddrones,albeitthepracticeof‘signaturestrikes’—involvingtheuseofalgo-rithmsforre-elaboratinggathereddatainordertoselecttargetsonthebasisofapattern-of-lifeanalysis—contributestoconfiningthehumanoperatortoamoremarginalrole.ThestakesofAWStechnologyarethatcriticalfunctionsoftheweaponmaybeperformedabsentahumandecision-maker:inthis,theyconstituteanunprecedentedstepintheevolutionaryprocessofremotenessintheuseofforce.107Theneedtoretainhumanpresenceintheuseofforceagainsthumantargetshasbeeninvokedfromtheearlieststagesofthedebate.Perhapssurprisingly,itisoneofthe(few)elementswhichallactorsinvolvedinthedebateonAWSseemtoagreeupon,ascommentatorshavenoted.108This,however,is106YSandoz,CSwinarskiandBZimmermann(eds),CommentaryontheAdditionalProtocolsof8June1977totheGenevaConventionsof12August1949(MartinusNijhoff1987)para1881.107SeeCh1.108RCrootof,‘AMeaningfulFloorfor“MeaningfulHumanControl”’(2016)30TempleIntlandCompLI53(‘[t]otheextentthereisanyconsensusamongStates,banDiegoMauri-9781802207675DownloadedfromPubFactoryat05/28/202203:48:39AMviaDukeUniversityLawLibrary

4638Autonomousweaponssystemsandtheprotectionofthehumanpersononconditionthatnooneasksthemwhattheactualmeaningofthatnotionis.ThisiswhyanadamantquestondefiningMHChasbeenperceivedattimesascounterproductiveintheCCWdebate.109Forourpurposes,though,under-standingMHCiskeyfordelimitatingthesubject-matter.Inthissubsection,weprovideageneraloverviewofthenotionsofMHCthathavebeenadvancedthusfar,andlaythefoundationforansweringthemostpressingquestionforourpurposes,namely,whetherMHCcorrespondstoalegalconcept.2.2.3.1The(emerging)notionof‘meaningfulhumancontrol’:narrowandbroadunderstandingsTheneedtoretainhumanpresenceinthetargetingprocessledtotheelabora-tionoftheconceptofMHCasthekeyrequisitefordevelopinganddeployingAWS.ThelabelhasbecomemainstreamnotonlyinthedebateattheCCW,butalsoinrelevantscholarship,acquiringaplethoraofdifferentunderstandings.ThefirstactortoemploytheconceptofMHCwasaUK-basedNGO,Article36,whichin2013releasedapapercommentingontheUK’spolicyonAWS.110Participatinginthe2014MoE,Article36stressedtheimportanceofpreserving‘deliberativemoralreasoning,byhumanbeings,overindividualattack’;again,theNGO’srepresentativespecifiedthatitlieswithintheveryessenceofIHLtohave‘humancommanders…tomakedeliberativecasebycase[judgments]onthelegalityofindividualattacks’.111TheconceptofMHCisconstruednarrowlyashumanagents(namely,the‘commander’)havetoparticipate:(1)actively(‘deliberative’),whichexcludespurelynominalinter-vention;(2)inactualcontexts(‘casebycase’),meaningtakingintoaccountallthecharacteristicsofspecificsituations;(3)inparticulardecisionsresultinginauseofforceagainstatarget(‘individual’),whichexcludesinterventiononlyattheearlystageofprogramminganAWS.112‘Meaningful’canbeunderstoodadvocates,andbanskepticsregardingtheregulationof[AWS],itisgroundedintheideathatallweaponryshouldbesubjectto[MHC]’).109France,‘Interventiongénérale’(13–17April2015)(‘cettenotionnepermetpasderépondredemanièreadéquateauxexigencesdecaractérisationdesSALAcarelleesttropvagueetpourraitconduireàengloberdanslepérimètredenosdiscussionsdessystèmesquinerépondentpasaucritèred’autonomiequejeviensd’énumérer’).110Article36,‘KillerRobots:UKGovernmentPolicyonFullyAutonomousWeapons’(2013)http://www.article36.org/wp-content/uploads/2013/04/Policy_Paper1.pdfaccessed31August2021.111Article36,‘Statement’(13May2014).112RMoyes,‘Meaningfulhumancontroloverindividualattacks’,inICRC,‘AutonomousWeaponsSystems.ImplicationsofIncreasinglyAutonomyintheCriticalFunctionsofWeapons’ExpertMeeting(March2016)46.DiegoMauri-9781802207675DownloadedfromPubFactoryat05/28/202203:48:39AMviaDukeUniversityLawLibrary

47Sketchingthedebate39astheoppositeof‘absent’,butalso‘nominal’:113itisarequirementstemmingfromtheintuitivepremisethathavinghumanoperators‘simplypressinga“fire”button…isnotsufficient’.114Ithasbeensaidthatnotwithstandingtheinherentlysubjectivenotionof‘meaningful’,theextenttowhichaformof‘humanintervention’isrequiredhasanimplicationonthetemporaldimensionofcontrol:‘sufficienttime’mustbegiventoahumanoperatortointerveneintheprocess.115OtheractorshavetakenasimilarrestrictivepositiononMHC.HumanRightsWatch,forexample,hasclaimedthat‘thereshouldalwaysbeMHCoftargetingandkilldecisionsinanyindividualattackonotherhumans’,116embracinganarrowunderstandingoftheconceptandlimitingitsscopetohumantargets—whichwouldexcludecaseswheretargetsare,say,militaryobjectsinarmedconflict.TheCSKRnotedthe‘importanceofalwaysmain-tainingMHCovertargetingandattackdecisions’.117TheICRCstressedtheneedforMHCwhenitcomesto‘criticalfunctions’oftheweapon,butseemslessdemandinginrelationtocontroloverindividualattacks.118Alongthesameline,computerscientistSharkeyproposesanarrowlycon-struedconceptofMHCwhenhearguesthathumanagents(commandersoroperators)musthavefullcontextualandsituationalawarenessofaspecificattack,beabletoperceiveunexpectedchangeincircumstances,retainthepowertosuspendoraborttheattack,andhavetimefordeliberationonthe113TChengeta,‘DefiningtheEmergingNotionof“MeaningfulHumanControl”inWeaponSystems’(2017)49NewYorkUniversityIntlLandPolitics833(explain-ingthat,albeit‘deliberatively’present,humans‘canalsobevictimstoerrorsofinduc-tivereasoning,suchasautomationbias,assimilationbias,andconfirmationbias).Formoreontheriskofnominalintervention,seeNESharkey,‘TowardsaPrinciplefortheHumanSupervisoryControlofRobotWeapons’(2014)2Politica&Società305(con-tendingthat‘levels’ofsupervisorycontrolwhereprogramsselectstargetandhumanmustapprovebeforeattackorvetoinrestrictedtimearebothsuitabletogenerate‘auto-mationbias’andtheywouldthereforenotbeacceptable).114Moyes(n112)46(concludingthattosaysomekindofhumancontrolisneces-saryisnotenough,asit‘mustbeinsomewaysubstantial’).115UNIDIR,‘TheWeaponizationofIncreasinglyAutonomousTechnologies:ConsideringHowMeaningfulHumanControlmightmovethediscussionforward’(2014)1.116HumanRightsWatch,‘StatementbyHumanRightsWatchtotheConventiononConventionalWeaponsinformalMeetingofExpertsonLethalAutonomousWeaponsSystems’(13May2014).117CampaigntoStopKillerRobots,‘StatementtotheConventiononConventionalWeaponsmeetingofexperts’(13May2014).118NESharkey,‘GuidelinesfortheHumanControlofWeaponsSystems’,ICRACWorkingPaperNo3(April2018).DiegoMauri-9781802207675DownloadedfromPubFactoryat05/28/202203:48:39AMviaDukeUniversityLawLibrary

4840Autonomousweaponssystemsandtheprotectionofthehumanpersonsignificanceofthetarget.119Sharkeyaddsthatthescenariowhereacomputerprogramprovidesalistoftargetsandleavesthehumanoperatorthechoiceofinitiatinganattackisacceptableonlyifthehumandeliberatesaboutthetargetbeforeattacking;andthisdeliberation,heargues,hastoprecedeanyattack.120Raisingthebarevenhigher,ChengetaproposesanunderstandingofMHCthat(1)requireshumanjudgmentinrealtimeoverindividualattacks,(2)requireshumanauthorizationforeachuseofforce,(3)allowsforequippingthemachinewithanabortmechanismand(4)includestheobligationtomonitorAWSwhileexecutingthedecision.121FewstatessupportexpresslyanarrowunderstandingofMHC.TheHolySee—whosepositionamongthosewhofiercelycontrastthedevelopmentanddeploymentofAWSisillustratedmoreindepthinthefollowing—122hasconsistentlyarguedthatindecisions‘regardinglifeanddeathforotherhumanbeings’MHC‘mustalwaysbepresent’.123Brazilarguesthat‘thereshouldalwaysbeahumanintheloop,withtheprerogativeandeffectivesituationalawarenessandunderstandingofthesystemtomakecriticaldecisions,particu-larlythoserelatedtolifeanddeath,andwithfullcapacitytointerruptauton-omousprocesses’.124DuringtheMoEphase,somestates,suchasGermany125andIreland,126adoptedapositionarguablysupportinganarrowunderstandingofMHC.GermanyandFrance’s2017workingpaperseeminglygoesinthesamedirection,where,whileillustratingtheproposalforapoliticaldeclaration119Ibid.120NESharkey,‘StayingintheLoop:HumanSupervisoryControlofWeapons’,inNBhuta,SBeck,RGeiss,HLiuandCKress(eds),AutonomousWeaponsSystems(CUP2016).ThisapproachisfullyconsistentwithunderstandingsofMHCasexclud-ingmerely‘nominal’humancontrol.121Chengeta(n113)888–9(acknowledgingthat‘maintainingMHCoverweaponsystemsmeansretainingthecontrol-dependentrelationshipbetweenhumansandweaponsystemsfortheircriticalfunctions’andthatasaresult‘aproperlyandnor-mativelyconstrueddefinitionofMHCistheequivalenttoabanonfullyautonomousweapons’).122SeeinfraCh3,s3.5andCh4,s4.5.123HolySee,‘StatementbyH.E.ArchbishopSilvanoTomasi’(13May2014).124Brazil,‘Statementatthe2017GroupofGovernmentalExpertsonLethalAutonomousWeaponsSystems’(March2019).125Germany,‘GeneralStatementbyGermany’(13–16May2014)(‘thereshouldbeacommonunderstandingintheinternationalcommunitythatitisindispensabletomaintainhumancontroloverthedecisiontokillanotherhumanbeing’).126Ireland,‘GeneralStatementattheCCWInformalConsultationsonLethalAutonomousWeapons’(13May2014)(stressingtheimportanceofmaintaining‘humancontrolovertheuseofforce’asputforwardbytheICRC).DiegoMauri-9781802207675DownloadedfromPubFactoryat05/28/202203:48:39AMviaDukeUniversityLawLibrary

49Sketchingthedebate41onAWS,itisarguedthat‘humansshouldcontinuetobeabletomakeultimatedecisionswithregardtotheuseoflethalforce’.127AccordingtoabroaderunderstandingofMHC,humandeliberationateachandeveryengagementcouldbedonewithout.Roordaarguesthat,oncondi-tionthatthestepsofthetargetingprocessareproperlyfollowed,AWSmayremainundersufficientcontroleventhoughthecommanderisnotinvolvedinaparticularengagement.128AsimilarconclusionissupportedbyphilosophicalinquiriesinthefieldofMHC.SantoniDeSioandvandenHovenproposeanaccountofMHCbasedonatwofoldcondition:‘tracking’(thatis,thesystemmustbeabletorespondtothemoralvaluesoftheuserandtorelevantfactsintheoperationalenvironment)and‘tracing’(thatis,thesystem’sactionmustbeattributedtoahumanbeingforaccountabilitypurposes).129Inpractice,‘beingincontroldoesnotnecessarilyrequiretheactofdirectcontrollingfromaposi-tionthatiscontiguousinspaceandtimeorisaproximatecause’:130humandeliberationneednottobepresentateachandeveryengagement.Somelegalscholarssupportthispositiontoo.131ThisapproachtoMHChaseventuallyprevailedintheGGEphase.Instate-mentsandworkingpapersdeliveredatthisstage,nationalpoliciesonAWSoftenresonate.Forinstance,USDirective3000.09requires‘appropriatelevelsofhumanjudgmentovertheuseofforce’.132ThereisanactualdifferencebetweenthisformulaandMHC.Whiletheterm‘appropriate’replaces‘mean-ingful’,moreconcerningisthatreferenceto‘individualattacks’hasmadethewayforamoregeneric‘useofforce’.133TheUShasre-stateditsviewduringtheGGE.Inthe2018workingpaper,itisexplainedthat‘appropriate’istobepreferredasa‘flexible’termreflectingthat‘thereisnotafixed,one-size-fits-all127FranceandGermany(n39)para12lit(c)(emphasisadded).128MRoorda,‘NATO’sTargetingProcess:EnsuringHumanControlover(andLawfulUseof)“Autonomous”Weapons”’,inAPWilliamsandPScharre(eds),AutonomousSystems:IssuesforDefencePolicymakers(NATOCommunicationsandInformationAgency2016).129FSantonideSioandJvandenHoven,‘MeaningfulHumanControloverAutonomousSystems:APhilosophicalAccount’(2018)5FrontiersinRoboticsandAI1.130Ibid10.131MNSchmitt,‘AutonomousWeaponSystemsandInternationalHumanitarianLaw:AReplytotheCritics’(2013)1HarvardNatlSecurityJFeatures1,33(acceptingthat‘ahumanmightnotbeincontrolofaparticularengagement’).132Directive3000.09(n6)para4.133DSaxon,‘AHumanTouch:AutonomousWeapons,DoDDirective3000.09andtheInterpretationof“AppropriateLevelsofHumanJudgmentovertheUseofForce”’,inNBhuta,SBeck,RGeiss,HLiuandCKress(eds),AutonomousWeaponsSystems(CUP2016)(explainingthatthereasonwhytheUSrejectsthetermMHCliesinthatitdoesnotfullycapturetherolehumansplayindevelopinganddeployingAWS).DiegoMauri-9781802207675DownloadedfromPubFactoryat05/28/202203:48:39AMviaDukeUniversityLawLibrary

5042Autonomousweaponssystemsandtheprotectionofthehumanpersonlevelofhumanjudgmentthatshouldbeappliedtoeverycontext’.134Moreover,thenotionofMHCitselfisbelievedto‘obscureratherthanclarifythegenuinechallengesinthisarea’:135whatmattersistoensurethataweaponsystemeffectuatesitsuser’sintentwhileperformingasetoffunctions.Thisresultcanbeachievedthroughpropertraining,doctrine,andtacticsforoperatorsandcommanders.TheUKalsoadoptsapositioninwhich‘humancontrol’hastobeappraisedthroughouttheentire‘lifecycle’ofaweaponsystem.136Again,theunderlyingideaisthathumancontrolneednotberetainedataparticularengagement,providedthathumanoperatorscanunderstandhowthemachineisexpectedtooperateinaparticularenvironment.Asaconsequence,thefocushadbetterbemovedtotheinteractionsbetweenhumansandmachinesinper-formingthevarioustasks:insoarguing,theUKadoptsanapproachconsistentwithitsdomesticmilitarystrategy,whichhasalreadymovedtowardsamodelofhuman–machineteaming.137Russiasupportstheneedtoretain‘duelevelofhumancontrol’andagnosticallyarguesthatitshouldbeuptostatestoestablishhowtodoit.138EstoniaandFinlandassertmorestraightforwardlythat‘humancontroldoesnotnecessarilyhavetobeexercisedcontemporaneouslywiththedeliveryofforce’,addingthat‘meaningfulhumancontrolcannotalwaysrequirethetechnicalcapabilitytocancelanattackthathasalreadycommenced’.139Thislatterconclusionmaybetoohasty,asitgoeswellbeyondwhatmostadvancedstatesargue.OtherstatesthatcanbeenumeratedamongthosethatsupportabroadconceptofMHCaretheNetherlands140andIsrael.141LetusnowproceedwithbriefconclusionsonMHCandthestakesassoci-atedwithit,beforeturningtothelegalanalysisofthatconcept.AccordingtoabroadunderstandingofMHC,humandeliberationmaynotbepresentateachandeveryuseofforceagainstanindividual.Humancontrolcanbeexercisedthroughthedesignofasystemandbyensuringitsoperationalreliabilityandpredictability:thiscontrolwouldbe‘meaningful’inasmuchashumanagentsaretaskedwithprogrammingthesystem.142AbroadunderstandingofMHCwouldthereforerequirehumanpresence‘overtheweapon’,withoutfurther134US,‘Human-MachineInteractionintheDevelopment,DeploymentandUseofEmergingTechnologiesintheAreaofLethalAutonomousWeaponsSystems’(28August2018)CCW/GGE.2/2018/WP.4para9.135Ibidpara47.136UK,‘HumanMachineTouchpoints’(n95).137UKMoD,‘JointConceptNote1/18.Human-MachineTeaming’,JCN1/18(2018).138RussianFederation(n74).139EstoniaandFinland(n72)paras19,20.140TheNetherlands(n71).141Israel,‘StatementbyMrAsafSegev’(26March2019).142SeeUNIDIR(n115)3.DiegoMauri-9781802207675DownloadedfromPubFactoryat05/28/202203:48:39AMviaDukeUniversityLawLibrary

51Sketchingthedebate43specificationandreferencetoindividualattacks.Somehaveexplicitlyrejected‘narrow’understandingsofMHConthegroundthatsuchapproachwouldfail‘toacknowledgethathumans,includingsupposedexperts,arepronetoalegionoferrors’.143Thisisaninterestingreflection,whoseworthliesinthatitfocusesonacost-effectiveanalysis:havinghumanoperators‘incontrol’ofamachinemayleadtosuboptimaloutcomes,whichinIHRL/IHLmeansnothinglessthantargetingthewrongperson.Toputitintermsofmoralphilosophy,therewouldbestrongconsequentialistargumentsrunningagainstnarrowconceptsofMHC.1442.2.3.2Anopenquestion:whatlegalstatusforMHC?HavingreviewedcurrentunderstandingoftheconceptofMHC,itisnowtimetoinquirewhetherMHC—ineitherofitsunderstandings—isalegalrequirementforallweaponsingeneral,andAWSinparticular,andwhatarethenormativesourcesestablishingthisrequirement.Isitlegallypermis-sibletoemployAWSthatdonotensurehumanpresenceateachandeveryengagement?Isitsufficienttohavehumanoperatorssomewhereintheloop(atthedevelopingstage,attheactivationoftheweaponsystemorintheactofsupervising)inorderfortherequirementofMHCtobefulfilled?Ineitherofthese,thiswouldimplythatAWSincapableofensuringMHCoveruseofforceareillegal(thatis,irrespectiveoftheadoptionofaninternationallegalinstrumentonthem).Alternatively,itmaybecontendedthatMHCdoesnotexistasalegalrequirement:ifAWSweretobeprohibited,then,theadoptionofanadhocinstrumentwouldbeinevitable.Oneauthorhasarguedthathumancontrolovermachinesis‘somethingthathashistoricallybeentakenforgranted—assumedbutneverstated’.145However,toaffirmthatMHCisalegalrequirementsimplybecausetech-nologiesthathaveexisteduntilnowcannotdowithouthumaninterventionwouldbeanapodicticargument:itpostulatespreciselywhatitisexpectedtodemonstrate.Instead,theissueistheextenttowhichexistingnormsofinterna-143PMargulies,‘MakingAutonomousWeaponsAccountable:CommandResponsibilityforComputer-GuidedLethalForceinArmedConflicts’,inJDOhlin(ed),ResearchHandbookonRemoteWarfare(EdwardElgar2017),435.144GTamburrini,‘OnBanningAutonomousWeaponsSystems:FromDeontologicaltoWideConsequentialistReasons’,inNBhuta,SBeck,RGeiss,HLiuandCKress(eds)AutonomousWeaponsSystems(CUP2016)(arguingthatthereisnorealcontrastbetweenconsequentialistanddeontologicalargumentsinnormativeethics,astheycon-vergeinprovidingstrongsupportforapre-emptivebanonLAWS).Alongthesameline,seeDAmorosoandGTamburrini,‘TheEthicalandLegalCaseagainstAutonomyinWeaponsSystems’(2018)18GlobalJurist1.145PAsaro,‘JusNascendi:RoboticWeaponsandtheMartensClause’,inRCalo,AMFroomkinandIKerr(eds),RobotLaw(EdwardElgar2016),385.DiegoMauri-9781802207675DownloadedfromPubFactoryat05/28/202203:48:39AMviaDukeUniversityLawLibrary

5244Autonomousweaponssystemsandtheprotectionofthehumanpersontionallawasidentifiedinthefollowingchapters—treatyandcustomaryrules,andgeneralprinciples—imposestandardsofhumancontroloveramachine’scriticalfunctionsthatmaybeencapsulatedinthenotionofMHC.Accordingtosomeauthors,theutilityofMHCasalegalnotionissomethingtobedoubtfulabout.146Thisisoneofthekeyquestionsthisbookaimstoanswer.WhenactorsandcommentatorsrefertoMHC,dotheyre-stateexistinglaw,orpushforthedevelopmentofanewlegalrequirement?IsMHCsomethingdifferentfromexistingrequirementsimposedbyinternationallaw,thusendowedwithauton-omouslegalstanding?Inallcases,whichrolecanitcanplayinthedebateonAWS?ThenotionofMHCwillaccompanyouranalysisofprimarynormsofinternationallawandofthelawofinternationalresponsibility,whichwillallowustodrawconclusionsastoitslegalstandinginthefinalchapter.2.3WHATISTHEREFORINTERNATIONALLAW?TheprevioussectionhasexplainedthatthemostsalientfeatureofAWSisthattheyseparateparticulardeliberationof(potentiallylethal)forcefromhumanintervention:whileremainingincontrolonthewiderloop,humanoperatorswouldnotbeaskedtoapproveparticulardecisionsinvolvingtheuseofforce.Againstthisbackground,theboundariesofthepresentresearch—thatis,thesubject-matterthroughwhichthereaderwillbeguided—appearmoreevidentnow.Still,itisamapfulloftrailsandevenpitfalls:metaphorsaside,theissueofAWScouldbetacklednotonlyby(international)lawyers,butalsobyengi-neers,philosophers,diplomats,expertsinmilitaryaffairs,andsoon.AsstatedinChapter1,thecompassthisbookhaschosenisthatprovidedbyinternationallaw.Thetimehascometoprovideamoreprecisedelimitationofourscopeofanalysis,asadmittedlythetopicoftheuseofforceisbroadininternationallaw.Tobeginwith,onemayconsidertheissueoftheuseofforcewithintheframeworkoftherelationsbetweenstates(orbetweenstatesandnon-stateactors).Thisfieldisregulatedbythejusadbellum,whichsetstheconditionsrecurringwhen‘wagingwar’isallowed:whatisatstakehereisasupra-individual,externaldimensionoftheuseofforce(2.3.1).Inaddition,onecanapproachtheissueoftheuseofforcefromaninternal146TMarauhn,‘MeaningfulHumanControl–andthePoliticsofInternationalLaw’,inWHeintschelvonHeinegg,RFrauandTSinger(eds),DehumanizationofWarfare(Springer2018)(arguingthattheconceptmayserveforsolvingdiplomatichurdles,buthaslittletonolegalstanding,anditmayevenriskjeopardizingexistingnormsofinternationallaw);MACEkelhof,‘ComplicationsofaCommonLanguage:WhyitissoHardtoTalkaboutAutonomousWeapons’(2017)22JofConflict&SecL311(arguingforthelackofasharedvocabulary).DiegoMauri-9781802207675DownloadedfromPubFactoryat05/28/202203:48:39AMviaDukeUniversityLawLibrary

53Sketchingthedebate45dimension(thatis,withinaninter-individualperspective):inthisscenario,stateorgans—militarycommandersandsoldiers,lawenforcementofficialsandagents,indeedallindividuals—applyforceagainstotherindividuals.ReferencehereistothetwobranchesofIHL(orjusinbello)andIHRL(2.3.2).Asillustratedbelow,thisbookadoptsthelatterapproachandwillfocusontheinter-individualdimensionoftheuseofforce.Thefollowingsubsectionisdevotedtopresentingthewell-knowndichotomybetween‘rules’and‘prin-ciples’asdifferentcategoryof‘norms’pertainingtoagivenlegalorder(forourpurposes,IHRLandIHL):anoverviewofdoctrineanditspracticalrele-vanceinthedebateonAWSisthusprovided(2.3.3).Finally,thenotionsandfunctionsofhumandignityandhumanity,quatheoreticalunderpinningsofthebookandgeneralprinciplesofIHRLandIHL,respectively,arediscussedbriefly(2.3.4),andwillbeanalyzedindetailinthefollowingchapters.2.3.1TheJusadBellumDimensionoftheUseofForceAsiswidelyknown,internationallaw’scurrentapproachvis-à-vistheuseofforceininternationalrelationsisremarkablyrestrictive.Thecornerstoneofjusadbellumisarticle2(4)oftheUnitedNationsCharter(UNC),whichprohibitsboththethreatandtheuseofforce‘againsttheterritorialintegrityorpoliticalindependenceofanystate,orinanyothermannerinconsistentwiththePurposesoftheUnitedNations’.147Forcemayberesortedtobystatesonlyinself-defense(article51UNC)orbytheSecurityCouncil(UNSC)pursuanttoitsprerogativesunderChapterVIIoftheUNC.Theprohibitiononthethreatanduseofforceenshrinesoneofthecorevaluesoftheinternationallegalorder:itcreatesanergaomnesobligationandranksasajuscogensnorm.Inlightofthis,itisevenarguedthattheexpressionjuscontrabellumwouldbemoreappropriatethanjusadbellum.148Atfirstsight,AWSdonotappeartorunagainstjusadbellumperse.First,nointernationalinstrumentondisarmamentandarmscontrolprohibitsthem:astatethatdecidedtodevelopanddeploythemwouldnotactinbreachofanyinternationalobligationsincumbentuponit.Second,itmustbekeptinmindthatanyweaponsthatarenotspecificallyprohibitedmustbeusedinaccord-ancewiththerulesregulatingjusadbellum.Whilethetypeofweaponchosentoapplyforceisirrelevant(ifnotprohibited),thefinalreleaseofforcemustcomplywithexistingrules,againstwhichAWShavetobeassessedtoo.149147(24October1945)1UNTSXVI.148RKolb,InternationalLawontheMaintenanceofPeace.JusContraBellum(EdwardElgar2017).149MWagner,‘AutonomousWeaponSystem’inRWolfrum(ed),MaxPlanckEncyclopediaofPublicInternationalLaw(OUP2016)marginn11.DiegoMauri-9781802207675DownloadedfromPubFactoryat05/28/202203:48:39AMviaDukeUniversityLawLibrary

5446AutonomousweaponssystemsandtheprotectionofthehumanpersonWhatmattersistheoccurrenceofacausethatmayjustifythatuseofforce—forinstance,therequirementsforself-defense,orpriorapprovalbytheUNSC.Thatspecificdecisionstoreleaseforcecanbetakenwithouthumaninterven-tiondoesnotseemtoberelevant.ThelimitedliteratureonallegedinherentinconsistenciesofAWSwithjusadbellumconfirmsourcontention.150Oncloserinspection,oneaspectthatAWSmayaffectmorestraightfor-wardlyconcernstheconceptof‘armedattack’asthebasicrequirementforself-defense.DoesforceappliedthroughAWSqualifyasan‘armedattack’ifastatesuccessfullydemonstratesthatAWSactedinanunexpectedway?Torespondinthenegativemaybeinlinewiththeinternationallaw’strendtowardslimitingthecircumstancesinwhichforcecanbeused(inthisinstance,bywayofself-defensebythestatethatistheaddresseeoftheforce),butmayenduppushingstatestoexertlesscareabouthowAWShavetobeused.Contrariwise,apositiveanswercouldbringaboutworrisomeescalationsofmilitaryforceininternationalrelations.Alegalwayoutofthequagmireisofferedbytheroleof(state’s)intenttohitaspecifictargetasoneoftherequire-mentsfor‘armedattack’tooccur.Althoughnotexpresslyforeseenbyarticle51UNC,thisrequirementseemstohavebeenendorsedbytheInternationalCourtofJustice(ICJ)inrecentcaselaw,andhasstimulatedmixedreactionsamongscholarsastoitscorrespondencewithcustomarylaw.151Interestingasitmaybe,thistopicistoolargetobetackledhere.However,AWSstillseemproblematicforanotherreason.Asiswidelyacknowledged,internationallawontheuseofforcehasbeengraduallyevolvingtowardsthe‘eliminationofthethreatofwar’,astheUnitedNationsGeneralAssembly(UNGA)DeclarationontheRightsofPeoplestoPeacereads.152InthedebateonAWS,itisoftensubmittedthatafurthergroundfor150DAmoroso,‘JusinBelloandJusadBellumArgumentsagainstAutonomyinWeaponsSystems:ARe-Appraisal’(2017)43QuestofInt’lL5;HMRoff,‘LethalAutonomousWeaponsandJusadBellumProportionality’(2015)47CaseWestReserveJInt’lL37.Forareflectiononproportionalityandtheuseofarmeddrones,seeAChehtman,‘TheadBellumChallengeofDrones:RecalibratingPermissibleUseofForce’(2017)28EJIL173.151CaseConcerningOilPlatforms(IranvUnitedStatesofAmerica)(Merits)[2003]ICJRep161;NOchoa-RuizandESalamanca-Aguado,‘ExploringtheLimitsofInternationalLawrelatingtotheUseofForceinSelf-defence’(2005)16EJIL499.152UNGA,DeclarationontheRightofPeoplestoPeace,Res39/11No3(adopted12November1984).DiegoMauri-9781802207675DownloadedfromPubFactoryat05/28/202203:48:39AMviaDukeUniversityLawLibrary

55Sketchingthedebate47prohibitingtheseweaponsisthattheirusewouldfuelanew-generationarmsrace.153Forinstance,astheHolySeehasputit,Minimizingtheriskstoitsownforcesisunderstandableandlegitimate.However,withnocasualtiesortalesofhorrorfromoneside,thedomesticpoliticalcostofwagingwarbecomeslesssignificant.Thisrepresentsanimportantdeterrenttooverly-hastenedmilitaryaction,andisadeterrentthatshouldnotbelightlydisre-garded.Autonomousweaponsystemstechnologymakeswartooeasyandremovesitsrelianceonsoldierlyvirtues.154Otherstatesandnon-stateactorshaverestatedthiscontention.155DuringtheworksoftheGGE,specificitemshavebeendedicatedtosecurityissues.InthewordsofthemorerecentGuidingPrinciples,‘physicalsecurity,appropriatenon-physicalsafeguards(includingcyber-securityagainsthackingordataspoofing),theriskofacquisitionbyterroristgroupsandtheriskofprolifera-tionshouldbeconsidered’whenaddressingAWS.156TheGuidingPrinciplesindicateanissuethatisdirectlyconnectedwithsecurity,namely,theriskofAWSfalling‘in[to]thewronghands’(forexample,terroristgroups):statesthusfeelthatAWShavetobedevelopedinawaythatensurespropercontroloverthesystemintheeventofitbeinghacked.157Whilealltheabovesoundsentirelyreasonable,itishardtoimplyapro-hibitiononAWSfromexistingrulesonthejusadbellum.Instead,itseemsthatjusadbellumrequirementscontinuetoapplytoAWSquaweaponsthatstates(ornon-stateactors)mayemploytouseforceintheirrelations.Surely,advocatesofatreatybanningAWSaremorewillingtounderscorethepoliticalundesirabilityofAWS,derivingfromthembeingaformidably‘tempting’toolforunrestrainedviolence:fromadelegeferendaperspective,theirpositionisunderstandable.However,thejusadbellumapproachtoAWSwillnotmaketheobjectofthepresentinquiry,whichisfocusedonthehumandimensionimpactedbyAWS,asthefollowingsubsectionexplains.153Foranempiricalstudy,seeJHanerandDGarcia,‘TheArtificialIntelligenceArmsRace:TrendsandWorldLeadersinAutonomousWeaponsDevelopment’(2019)10GlobalPolicy331.154HolySee(n123).155Cuba,‘DeclaracióndeClaudiaPérezAlvarez,ConsejeradelaMisiónPermanentedelaRepúblicadeCubaenGinebra.ReunióndeexpertossobreSistemasdearmasautónomasletales’(16April2015);ICRAC,‘OpeningStatementtothe2016UNCCWExpertMeeting’(12April2016).156GuidingPrinciples,principle(f).157HeynsReport(n2)para98.DiegoMauri-9781802207675DownloadedfromPubFactoryat05/28/202203:48:39AMviaDukeUniversityLawLibrary

5648Autonomousweaponssystemsandtheprotectionofthehumanperson2.3.2TheIHRL/IHLDimensionoftheUseofForceMovingtotheinternaldimensionoftheuseofforcebyAWS,focusisontheactualreceiverofforce,thatis—accordingtotheproposedunderstandingof‘criticalfunctions’—thehumanperson.Fromthisperspective,twobranchesofinternationallawcometothefore.Ontheonehand,theprotectionofthehumanpersonisthegoalofthehumanrightsmovement,which—largelyinspiredbypreviousconstitutionalexperi-encesofmanycountriesaroundtheglobe—begantoinfluencetheinternationallegalorderinthemiddleofthetwentiethcentury.TheUniversalDeclarationonHumanRights(UDHR)isregardedashavingbroughtaboutIHRL,whichtodaycomprisesanegregiousnumberoftreatyandcustomarynorms,someofwhichareofjuscogensnature.Ontheotherhand,itcannotbeignoredthattheprotectionofparticularcategoriesofindividualpredatesIHRL,andcanbetracedbacktoancienttimes.Someindividuals,suchasheadsofstate,diplo-matsandforeigners,havebeentheobjectofinternationallawforagesasfarasthelawofstateimmunityandthelawofdiplomaticprotectionareconcerned.Otherindividualswhotraditionallyhavebeenaddresseesofinternationallawnormsarethosewhoqualifyas‘combatants’and‘civilians’inarmedconflict:theirprotectionisensuredbyjusinbello,thebranchofinternationallawthatregulatestheconductofhostilities.Whilehumanrightsareintendedtoprotectthehumanpersoninalltimesandcircumstances,theprotectionensuredbyjusinbelloislimitedtoarmedconflicts.Inprinciple,thescopeofapplicationofIHListhusnarrowerthanIHRL.TheinterplaybetweenIHRLandIHL—moreprecisely,itstheoreticalsystematization—isatopiconwhichinternationallawscholarshavebeenduelingforyears.158Theissuehasbecomesocomplexthatathoroughanalysisofitwouldrequirefarmorethanthe(inevitablysmall)spaceofasubsection.Arecent,andnowperhapsmainstream,accountofthisrelationshiphasbeenadvancedbytheICJinthelandmarkAdvisoryOpinionrenderedintheNuclearWeaponscase.159Askedtoascertainwhetherresorttonuclearweaponswasconsistentwithinternationallaw,theICJtouchedupontheissueofhowtocombineIHRLnorms—namely,theprovisionontherighttolifeas158SeemoreextensivelyHJHeintze,‘TheoriesontheRelationshipbetweenInternationalHumanitarianLawandHumanRightsLaw’inRKolbandGGaggioli(eds),ResearchHandbookonHumanRightsandHumanitarianLaw(EdwardElgar2013).159LegalityoftheThreatorUseofNuclearWeapons(AdvisoryOpinion)[1996]ICJRep226.DiegoMauri-9781802207675DownloadedfromPubFactoryat05/28/202203:48:39AMviaDukeUniversityLawLibrary

57Sketchingthedebate49pertheInternationalCovenantonCivilandPoliticalRights(ICCPR)—andIHL,andheldthat:Inprinciple,therightnotarbitrarilytobedeprivedofone’slifeappliesalsoinhostilities.Thetestofwhatisanarbitrarydeprivationoflife,however,thenfallstobedeterminedbytheapplicablelexspecialis,namely,thelawapplicableinarmedconflictwhichisdesignedtoregulatetheconductofhostilities.160ThelexspecialisapproachwaslaterconfirmedbytheICJintheWallcaseandintheDRCvUgandacase.161Otherlegalinstrumentsresorttothisapproachtoexplaintheinteractionsofthetwobodiesofinternationallaw.162Thisapproachstimulatedanintensedoctrinaldebate.First,itledtotheabandonmentofatraditionalaccountoftherelationshipbetweenIHRLandIHL,encapsulatedbythe‘separation’theory.Accordingtothistheory,eithertheformerbodyoflaworthelatterisapplicableatagivenmoment,dependingonfactualcircumstance:theyaremutuallyexclusiveowingtotheirradicalincompatibility.163Althoughattimesstillevoked,164the‘separation’theoryisnolongersupportable,astheriseofthehumanrightsmovementduringthe1960sand1970sledtoanincreasinglystrongerconvergenceofIHLandIHRL.165SomearguethatthisapproachwasstilllatentintheNuclearWeaponscase,whereresorttotheconceptoflexspecialisadmittedlyservedto‘dismisstherelevanceofhumanrightsaltogether’inthatspecificcase.166Second,it160Ibidpara25.161LegalConsequencesoftheConstructionofaWallintheOccupiedPalestineTerritory(AdvisoryOpinion)[2004]ICJRep136para106;CaseConcerningArmedActivitiesontheTerritoryoftheCongo(DemocraticRepublicoftheCongovUganda)(Merits)[2005]ICJRep168para216.162UNHRC,‘GeneralCommentNo36(2018)onArticle6oftheInternationalCovenantonCivilandPoliticalRights,ontheRighttoLife’(30November2018)CCPR/C/GC/36(GC36)para64.163GIADDraper,‘HumanitarianLawandHumanRights’,inMAMeyerandHMcCoubrey(eds),ReflectionsonLawandArmedConflicts(KluwerLawInternational1998)149(‘[t]heattempttoconfusethetworegimesoflawsisunsupportableintheoryandinadequateinpractice.Thetworegimesarenotonlydistinctbutarediametricallyopposed’).164Heintze(n158)55.165RKolb,‘HumanRightsLawandHumanitarianLaw’inRWolfrum(ed),MaxPlanckEncyclopediaofPublicInternationalLaw(OUP2016)44–6(pointingoutthat‘progressiveconvergence’wasdue,amongothers,tothe‘humanitarianism’thattheGenevaConventionswereimbuedwith:‘[t]heIHLoftheGenevaperiodisrootedintheidealof“humanitarism”.Hence,progressiveinterrelationshipswith[I]HRLquicklybecameunavoidable’).166AccordingtoVGowlland-DebbasandGGaggioli,‘TheRelationshipbetweenInternationalHumanRightsandHumanitarianLaw:AnOverview’,inRKolbandGDiegoMauri-9781802207675DownloadedfromPubFactoryat05/28/202203:48:39AMviaDukeUniversityLawLibrary

5850Autonomousweaponssystemsandtheprotectionofthehumanpersonmadeclearthat,whileinprincipleremainingdistinctandtoadegreealsodivergentincontent,IHRLandIHL‘cancompleteeachotheronspecificpoints’.167Thistheory,referredtoas‘complementarity’theory,isthusbasedonacase-by-casecomparativeapproachaimedatreinforcing,enhancingorbetterinterpretingtheapplicablerules.168TheICJturnedtothisintheWallcase,whereitenvisaged‘threepossiblesituations:somerightsmaybeexclu-sivelymattersof[IHL];othersmaybeexclusivelymattersof[IHRL];yetothersmaybemattersofboththesebranchesofinternationallaw’.169Morerecently,athirdconception,‘harmonization’theory,hasemerged:thistheoryaimsto‘eliminateortonedownthedifferencesbetween[IHRLandIHL]’,170andisgroundedintheideathattheirsystemicintegrationispreferable.171SometimesIHLandIHRLsooverlapthatitisimpossibletokeeptheirrespec-tiveobligationsentirelyseparated,insteadformingaunicum.172Whilethisdebateisintriguing,itseemsmoreusefultofocusonthepurposeofadoptingacertainunderstandingoftherelationshipbetweenIHRLandIHL.173Thesetwobranchesofinternationallawsharesimilartraitsinthattheybothstrivetoprotectthehumanpersonfromstates’abuse,whichmayoccurbothinandoffthebattlefield.Ithasbeenarguedthatthemovetowardsthe‘harmonization’theorycanbetracedbacktothe‘naturalconvergenceofthehumanitarianprincipleunderlyingthesetwobodiesoflaw’.174Inbrief,Gaggioli(eds),ResearchHandbookonHumanRightsandHumanitarianLaw(EdwardElgar2013)85(arguingthattheconceptoflexspecialiswasintendedtooperatenotonlyasaninterpretativeaid,butmorefundamentallyasanexclusionaryprinciple).167Heintze(n158)57.168Kolb(n165)46:‘[i]nsomeareas,[I]HRLcomplementsIHL;inotherareas,IHLstrengthensorinspires[I]HRL.Onebranchofthelawfrequentlyservestointerprettheother’.ThisseemstobetheapproachendorsedbytheILC:see‘ReportoftheStudyGrouponFragmentationofInternationalLaw:DifficultiesArisingfromtheDiversificationandExpansionofInternationalLaw’(13April2006)A/CN.4/L.682paras102–3.169LegalConsequencesoftheConstructionofaWall(n161)para106.170GGaggioli,‘ALegalApproachtoInvestigationsofArbitraryDeprivationsofLifeinArmedConflicts:TheNeedforaDynamicUnderstandingoftheInterplaybetweenIHLandHRL’(2017)36QuestofInt’lL27,36.171Gowlland-DebbasandGaggioli(n166)87(accordingtowhomthelegalbasisofthistheorycanbetracedbacktotheILCReportandliesontheprovisionofart31(3)oftheViennaConventionontheLawofTreaties).172SeeforinstanceHeintze(n158)61–2(discussingtheexampleofthe1989UNConventionontheRightoftheChild).173MMilanović,‘TheLostOriginofLexSpecialis.RethinkingtheRelationshipbetweenHumanRightsandInternationalHumanitarianLaw’inJDOhlin(ed),TheoreticalBoundariesofArmedConflictandHumanRights(CUP2016).174DStephens,‘HumanRightsandArmedConflict:TheAdvisoryOpinionoftheInternationalCourtofJusticeintheNuclearWeaponsCase’(2001)4YaleHumanRightsandDevelopmentLJ1,2.DiegoMauri-9781802207675DownloadedfromPubFactoryat05/28/202203:48:39AMviaDukeUniversityLawLibrary

59Sketchingthedebate51irrespectiveevenofthetheorythatonewouldadopttoaddresstherelationshipbetweenIHLandIHRL,thesepossessa‘commonorigin’fromwhichtheyspringandarticulate.ThedraftpreambleadoptedbytheStockholmConferenceforthefourGenevaConventions,referredtothe‘[r]espectforthepersonalityandthedignityofhumanbeings’astheminimumcommondenominatorofIHLinstruments.175Whiletheproposalofacommonsubstantivepreamblewaseventuallydiscardedowingtodifficultyinachievingaunanimousagreement,theideaofIHLandIHRLbeingmutuallylinkedstartedtogaintraction.176Commonarticle3tothefourGenevaConventionsoffersthelitmustestforthisassertion.Thatprovision—consideredstillasthebedrockregulatoryframeworkfornon-internationalarmedconflict(NIAC)—establishesasetofdutiesincumbentuponbelligerentpartiesthatapply‘asaminimum’.177Thatthe‘minimum’justifyingtheregulationofamattertraditionallyconsideredasbeingexclusivelywithintheirdomesticaffairisrootedinthe‘commonorigin’sharedbyIHLandIHRLismadeclearinthepreambleofAPII.178Internationaladjudicatorybodieshavestressedthe‘commonorigin’atvarioustimes.OfparticularrelevancearespecificfindingsbytheICJ.TheCorfuChannelcasefamouslyreliedon‘elementaryconsiderationsofhuman-ity’,allegedlybindinginbotharmedconflictandpeace.179Referringmorespecificallytocommonarticle3,theICJfoundthatthatprovisionreflecteda‘minimumyardstick’constitutingoneofthe‘fundamentalgeneralprinciples175ICRC,RemarksandProposalssubmittedbytheInternationalCommitteeoftheRedCross(February1949),8(emphasisadded).176AClapham,‘TheComplexRelationshipBetweentheGenevaConventionsandInternationalHumanRightsLaw’,inAClapham,PGaetaandMSassoli(eds),The1949GenevaConventions.ACommentary(OUP2015).SeealsoICRC,CommentaryontheFirstGenevaConvention.Convention(I)fortheAmeliorationoftheConditionoftheWoundedandSickinArmedForcesintheField(CUP2016)paras127–8.177ICRC,CommentarytotheSecondGenevaConvention.Convention(II)fortheAmeliorationoftheConditionofWounded,SickandShipwreckedMembersofArmedForcesatSea(CUP2017)paras536–9,idem,CommentarytotheThirdGenevaConvention.Convention(III)fortheTreatmentofPrisonersofWar(CUP2021)paras548–51.178ProtocolAdditionaltotheGenevaConventionsof12August1949andRelatingtotheProtectionofVictimsofNon-InternationalArmedConflicts(adopted8June1977,enteredintoforce7December1977)1125UNTS609(APII),preamble(‘thehumanitarianprinciplesenshrinedinArticle3…constitutethefoundationofrespectforthehumanpersonincasesofarmedconflictnotofaninternationalcharacter’).Asaconfirmation,seeProtocolAdditionaltotheGenevaConventionsof12August1949andRelatingtotheProtectionofVictimsofInternationalArmedConflicts(adopted8June1977,enteredintoforce7December1977)1125UNTS3(API)art72(‘provisionsofthisSectionareadditionalto…otherapplicablerulesofinternationallawrelatingtotheprotectionoffundamentalhumanrightsduringinternationalarmedconflict’).179CorfuChannelCase(UKvAlbania)(Merits)[1949]ICJRep4paras21–2.DiegoMauri-9781802207675DownloadedfromPubFactoryat05/28/202203:48:39AMviaDukeUniversityLawLibrary

6052Autonomousweaponssystemsandtheprotectionofthehumanpersonofhumanitarianlaw’.180EvenmoreimportantisthecontributiongivenbytheInternationalCriminalTribunalfortheFormerYugoslavia(ICTY)and,generally,ofICLjurisprudence.Whiledealingwiththequestionofwhethercommonarticle3correspondstocustomarylawapplicabletointernationalarmedconflicts,theICTYaffirmed:[b]othhumanrightsandhumanitarianlawfocusonrespectforhumanvaluesandthedignityofthehumanperson.Bothbodiesoflawtakeastheirstartingpointtheconcernforhumandignity,whichformsthebasisofalistoffundamentalminimumstandardsofhumanity.…TheuniversalandregionalhumanrightsinstrumentsandtheGenevaConventionsshareacommon‘core’offundamentalstandardswhichareapplicableatalltimes,inallcircumstancesandtoallparties,andfromwhichnoderogationispermitted.181Legalscholarshipalsofavorsthe‘commonorigin’approach.Pictetendorsestheapplicabilityofa‘minimumofhumanity’acrossalltypesofarmedcon-flict,182whileMeronacknowledgesthat,inspiteoftheirdivergent‘historicalanddoctrinalroots’,IHLandIHRLsharethe‘principleofhumanity’asa‘commondenominator’.183CançadoTrindadearguesthat‘[t]hetreatmentdis-pensedtohumanbeings,inanycircumstances,oughttoabidebytheprincipleofhumanity,whichpermeatesthewholecorpusjurisoftheinternationalprotectionoftherightsofthehumanperson’.184Inourunderstanding,thismeansthatthenotionof‘humandignity’inIHRLandthatof‘humanity’inIHLstemfromthesamecommondenominator:thetwobranchesofinternationallawtouchattheapex.Thiscircumstanceiskey180CaseConcerningMilitaryandParamilitaryActivitiesinandAgainstNicaragua(NicaraguavUS)(Merits)[1986]ICJRep3para218.SeealsoLegalityoftheThreatandUseofNuclearWeapons(n159)para79.181ProsecutorvMucićetal(20February2001)ICTY-96-21-Apara149.SeealsoProsecutorvFurundžija(10December1998)ICTY-95-17/1-Tpara182(originalemphasis)(‘[t]hegeneralprincipleofrespectforhumandignityisthebasicunderpin-ningandindeedtheveryraisond’êtreofinternationalhumanitarianlawandhumanrightslaw;indeedinmoderntimesithasbecomeofsuchparamountimportanceastopermeatethewholebodyofinternationallaw’);ProsecutorvMucićetal(16November1998)ICTY-96-21para543;ProsecutorvBlaskić(3March2000)ICTY-95-14-Tpara154;ProsecutorvAkayesu(2September1998)ICTR-96-4-Tparas565–6;ProsecutorvKambanda(4September1998)ICTR-97-23-Sparas15–16;ProsecutorvSerushago(5February1999)ICTR-98-39-Spara15.182JPictet,Développementetprincipesdudroitinternationalhumanitaire(InstitutHenryDunantandEditionsPédone1983),73–4.183TMeron,‘TheHumanizationofHumanitarianLaw’(2000)94AJIL239,245.184AACançadoTrindade,‘SomeReflectionsonthePrincipleofHumanityinItsWideDimension’,inRKolbandGGaggioli,ResearchHandbookonHumanRightsandHumanitarianLaw(EdwardElgar2013)189(originalemphasis).DiegoMauri-9781802207675DownloadedfromPubFactoryat05/28/202203:48:39AMviaDukeUniversityLawLibrary

61Sketchingthedebate53forthediscussiononAWS.Asweapons,theuseofAWSissubjecttobothIHRLandIHLnormsregulatingtheuseofforceagainstindividuals.NearlyallscholarlyworksonAWStackletheissueofthoseweaponssystems’compatibilitywiththetwobranchesofinternationallaw.Thesameappliestotheinternationaldebate.Whatisinterestingtoexploreisthemethodologyconventionallyfollowedforthepurposesofthislegalanalysis,asIHLreceivesmoreattentionthanIHRLdoes.WhileearlydocumentsonAWS(suchastheHeynsReport)haveamorebalancedfocusonbothbranchesofinternationallaw,intheframeworkoftheCCWIHLtendstoovershadowIHRL.AstheGuidingPrinciplesstate,thedebatehastobeguidedby‘internationallaw,inparticulartheUnitedNationsCharterand[IHL]’;itisIHLthatisaffirmedto‘applyfully’toAWS;itisagainst‘applicableinternationallaw,inparticularIHL’that‘human-machineinteraction’hastobeassessed.185Atleastthreereasonsforthismonopolizationcanbepinpointed.Thefirstispurelylegal:theCCWisanIHLtreaty,asstatedinitsarticle1(‘scopeofapplication’).Inparticular,accordingtoitsparagraph2,theCCWanditsProtocols‘shallnotapplytosituationsofinternaldistur-bancesandtensions,suchasriots,isolatedandsporadicactsofviolence,andotheractsofasimilarnature,asnotbeingarmedconflict’.However,ashasbeenarguedabove,‘separation’theoriesontherelationshipbetweenIHRLandIHLhavebeenprovenflawed:eveninthebattlefieldIHRL‘continuestoapply’.Itcannotbeignoredaltogether.ThisiswhythestrictapproachadoptedbytheGuidingPrincipleshasreceivedcriticism.186Thesecondreasonisofapoliticalnature:giventheruleofconsensusthatinformstheGGE’swork,somestatesopposedanyexplicitreferencetoIHRL,formallyasfallingoutsidetheCCWscopeasabove,substantiallyinordertoneutralizemorerestrictivenormsontheuseofforce.Notsurprisingly,mostreferencestoIHRLcanbefoundinstatementsorworkingpapersofstates,NGOsorotheractorsbelong-ingtothegroupthattakesaskepticalstancetowardsAWS.Thefinalreasonispractical:mostapplicationsofAWSareexpectedtooccurinthemilitaryfield,wheretheseweaponssystemsareexpectedtobefieldedsoonerthanforlaw-enforcementpurposes.However,thisisonlyhalfthestory,andtheassertedpriorityofIHLoverIHRLcaneasilyberebutted.Inreverseorder,therehasbeentremendousprogressintechnologyrelevanttolawenforcement:algorithmsenablingfacialrecognitionandprofilinghavealreadybeendeveloped(andarecur-185GuidingPrinciples,principles(a),(c),(h).186Article36,‘CriticalCommentaryonthe“GuidingPrinciples”’,policynote(November2019).DiegoMauri-9781802207675DownloadedfromPubFactoryat05/28/202203:48:39AMviaDukeUniversityLawLibrary

6254Autonomousweaponssystemsandtheprotectionofthehumanpersonrentlyemployed)bymanystates’authorities,firstandforemostthepolice.187Applicationsinmilitaryfieldsconcurwiththose,andareexpectedtoincrementsignificantlyalongsidedevelopmentsinAIcapabilities.188Moreimportantly,mostscenariosinwhichnewtechnologies—andautonomousdecision-makingprocessesespeciallyviaalgorithms—havebeenexperimentedwithandareregularlyusedcouldhardlybesaidtoberegulatedbyIHL.Theclearestexampleisprovidedbythe‘precursors’ofAWS,namely,armeddrones:theseareemployedalmostexclusivelyincounter-terrorismoperations,outsideareasofactivehostilities(thatis,farfromthe‘operatingrange’ofIHL)asoftenisexpresslyacknowledged.189AssigningprioritytoIHLwouldbeatbestmisleading,ifitwasassumedthatAWScouldbefieldedinoperationalscenariosakintothoseinwhicharmeddronesareemployed.Inaddition,fromatheoreticalstandpointitmakessensetoconsiderIHRLbeforeIHL:thecontinuingapplicabilityofIHRLduringarmedconflicthasbeensufficientlydemonstratedabove.190Insummary,asasetofrulesandprinciplesthatoperateinallcircumstances(or,aslexgeneralis),IHRLshouldprecedeIHL,whichinturnencapsulatesrulesandprinciplesthatoperateinspecificcircumstances(aslexspecialis).Bydoingso,thisbookadoptsareversemethodologyvis-à-visthemainstreamdebateonAWSwhichwillallowforabetterunderstandingoftheimpactthatAWSarelikelytohaveoninternationallawinthelongrun.Letusnowturntoillustratingtheapproachthatwillbefollowedinrespectofthenormscon-tainedineachbranchofinternationallaw.2.3.3TheDichotomybetween‘Rules’and‘Principles’andItsRelevanceforInternationalLaw:TheoreticalPremisesBothIHRLandIHLcontainnormsprotectingthehumanperson.OneofthethesesthatthisbookstrivestoadvanceisthatAWSneedtobetestedagainstallthosenorms,which—inaccordancewithlegaltheory—includerulesandprinciples.Thedichotomybetween‘rules’and‘principles’astwodiversecategoriesoflegal‘norms’isatopicthathasbeentheobjectofextensive187DMauri,‘“AlgorithmicTargetConstruction”andtheChallengesbyInternationalHumanRightsLaw’(2019)Eurojus8.188SAbaimovandMMartellini,‘ArtificialIntelligenceinAutonomousWeaponSystems’,inMMartelliniandRTrapp(eds),21stCenturyPrometheus(Springer2020).189‘ProcedureforApprovingDirectActionagainstTerroristTargetsLocatedOutsidetheUnitedStatesandAreasofActiveHostilities’(22May2013).190Seesupra2.3.2.1.DiegoMauri-9781802207675DownloadedfromPubFactoryat05/28/202203:48:39AMviaDukeUniversityLawLibrary

63Sketchingthedebate55debateinlegaltheory,191andisparticularlyfascinatingwhenprojectedtotheinternationallegalorder.Whileforspaceconstraintsitisnotfeasibletocoverthoroughlythevariousdoctrinalreflections,sufficeittooutlineatleasttwofeaturesmarkingthedistinctionbetweenthecategoriesabove,beforeturningtotheissueofwhetherthisdistinctionispresentalsoininternationallaw.First,‘rules’and‘principles’differintheirstructure.Theformerarebelievedtobecomposedofparticularconditionsbymeetingwhichalegalconsequencefollows:eithertheseconditionsarepresent,andthustheruleisapplied,ornot.Thisiswhyitisarguedthatrulesapplyinan‘allornothingfashion’.192Conversely,principlesdonotderivefromthefulfilmentoftheseconditions:theyareheldtoprovideprimafacieargumentsforreachingapar-ticularconclusioninspecificcircumstances.193Insummary,adifferencecanbeidentifiedinthecontentofthetwocategoriesofnorms:‘rules’wouldbemoredefinedincontentandexpectedtoapplyornotinagivencase,while‘principles’wouldbemorenuancedandincline‘inonedirectionoranother’,guidingtheirappliertowardsasolutionofacasewhererulesfailtoapply.194Second,byreasonoftheirstructuraldifferences,thefunctioningwhenappliedbythe(legal)interpreterdivergessensibly.Rulesgothroughaprocessofsubsumptionandestablishrights,duties,orinturnotherrulesoncreation,modificationorextinctionofotherrules;ifagivenrulewerefoundtoconflictwithanother,itsvaliditymaybecalledintoquestion.Oppositely,principlesrequireaprocessofbalancing:intheeventofconflictbetweenprinciples,theinterpreteriscalledupontoweighttheirrespectivevaluesandestablishwhichprincipleistoprevailinagivencircumstance,inordertoderivearuleapplicabletothecaseathand.195Internationallawtooisfamiliarwiththisdichotomy.196Preliminarily,itmustbenotedthat,farfrombeingamerelydoctrinalissue,thistopichasbeentackledrecentlybytheInternationalLawCommission(ILC),thatisthesubsidiaryorgantotheUNGAtaskedwithpromotingtheprogressivedevel-opmentofinternationallawanditscodification:atitsseventiethsession,theILCdecidedtoincludethetopic‘Generalprinciplesoflaw’initsprogram191A‘father’tothisdichotomyisbeyonddoubtRDworkin,TakingRightsSeriously(HarvardUniversityPress1977).192Ibid24ff.193RAlexy,‘OntheStructureofLegalPrinciples’(2000)13RatioJuris294.194Dworkin(n191).SeealsoRGuastini,Filosofiadeldirittopositivo(Giappichelli2017),73ff.195RAlexy,‘OnBalancingandSubsumption.AStructuralComparison’(2003)16RatioJuris433.196SeeGGaja,‘GeneralPrinciplesofLaw’,inRWolfrum(ed),MaxPlanckEncyclopediaofPublicInternationalLaw(OUP2013).DiegoMauri-9781802207675DownloadedfromPubFactoryat05/28/202203:48:39AMviaDukeUniversityLawLibrary

6456Autonomousweaponssystemsandtheprotectionofthehumanpersonofwork.197Theeffortundertakenistowardascertaininghowthecategoryof‘generalprinciplesoflaw’concretelyworksintheinternationallegalsystem,whichthereforeceasestobeatopicpreserveofdoctrinaldiatribes.Yet,thesubjectitselfremainshighlytheoretical:totakeoneexample,theSpecialRapporteurappointedforthetopicsuggestedthat,inthedichotomybetween‘rules’and‘principles’,thelatterrefer‘toamore“general”and“fundamental”normthanothernormsofinternationallaw’,therebyfollowingaconsolidatedapproachtothedichotomy.198Letusnowdelveintothenormativecategoryofgeneralprinciplesininternationallaw,whosenature,functionsandpurposehaveelicitedheateddebate.Asisknown,article38(1)(c)oftheStatuteoftheICJ(andtheidenticalarticle38(1)(3)oftheStatuteofthePermanentCourtofInternationalJustice)enumerates‘generalprinciplesoflawrecognizedbycivilizednations’amongthesourcesofinternationallawthattheCourtisentitledtoapply.Tellingly,theICJitselfseemedtoopposetheideaofdistinguishingbetweenthetwonormswhenitstatedthat‘theassociationoftheterms“rules”and“principles”isnomorethanthedualexpressiontoconveyoneandthesameidea’.199Atthesametime,itwasuptotheICJtore-affirmthenotionof‘generalprinciples’andapplyittoparticularcases,forinstance,whenitarguedtheexistenceof‘certaingeneralandwell-recognizedprinciples’suchas‘elementaryconsid-erationsofhumanity’.200Ifsuchaqualifiedinterpreterofinternationallawseemstobestrugglingwiththatcategory,therecanbelittlesurpriseinalsofindingwidespreaddisagreementamongstatesandscholars.Letusfocusonsomemajorhurdlesthattheinterpretercouldhardlysteerclearof,namely,thenature,theoriginandthefunctioningoftheseprinciplesintheinternationallegalorder.Afirsthurdleisrepresentedbythescopeofthenotionof‘generalprin-ciplesoflaw’.Whilethewrittenprovisionofarticle38(1)(c)seemstoreferexclusivelytoprinciplesderivedfromnationallegalsystems(thatis,existingascommontotheprincipallegalsystemsoftheworldandtransposedtotheinternationallegalorder),accordingtosomethecategoryisbroadenoughtoincludealsoprinciplesformedwithintheinternationallegalsystem,thatis,197ILC,ProvisionalSummaryRecordofthe3433rdMeeting’(19July2018)A/CN.4/SR.3433.198ILC,‘ReportoftheInternationalLawCommission’(29April–7June,8July–9August2019)A/74/10para218.SeealsoMVásquez-BermúdezandACrosato,‘GeneralPrinciplesofLaw:TheFirstDebatewithintheInternationalLawCommissionandtheSixthCommittee’(2020)19ChineseJIL157.199DelimitationoftheMaritimeBoundaryintheGulfofMaineArea(CanadavUnitedStatesofAmerica)(Merits)[1984]ICJRep246[79].200CorfuChannelCase(n179)22.DiegoMauri-9781802207675DownloadedfromPubFactoryat05/28/202203:48:39AMviaDukeUniversityLawLibrary

65Sketchingthedebate57identifiedthroughaprocessofdeductionfromexistingrulesofinternationallaw.Thisdichotomyissometimesreferredtoasbetweenprinciplesinforodomestico(principesgénérauxdedroit)and‘internationallawprinciples’(principesdudroitinternational).201Itisarguedthatthecategoryofgeneralprinciplesinitselfwasconceivedoutofthewell-establishedconfrontationbetweentworadicallydivergenttheoriesoflaw—legalpositivismandnaturallaw—thattookplaceatthetimeofthedraftingofthePermanentCourtofInternationalJustice(PCIJ)Statute,andthatstillexists.202Inparticular,generalprinciplesaccordingtothesecondmeaningplayanimportantroleinnaturallaw(sometimesreferredtoalsoas‘neo-naturallaw’or‘neo-constitutional-ism’)theories,inasmuchasstructurallyopentoextra-positiveinfluences(forexample,moralimperativessuchasjusticeorfairness).203Arguably,owingtothe‘excessivediscretion’thatgeneralprinciplesformedwithintheinterna-tionallegalsystembestowuponinterpreters,somestatesandauthorsaremoreskepticalaboutthosethanprinciplesinforodomestico.204PerhapsforthesamereasontheICJ,andthePCIJbefore,haveshownreluctanceinapplyingbothcategoriesofgeneralprinciples.205CurrentworksoftheILCacknowledgetheexistenceofthetwovariantsofthecategoryof‘generalprinciplesoflaw’andstrivetoidentifyapositivistfoundationalsoingeneralprinciplesformedwithintheinternationallegalsystem.Theelementof‘recognition’ofthoseprinciplesby‘civilizednations’201BBonaféandPPalchetti,‘RelyingonGeneralPrinciplesinInternationalLaw’,inCBrölmannandYRadi(eds),ResearchHandbookontheTheoryandPracticeofInternationalLawmaking(EdwardElgar2016).SeealsoOSchachter,‘InternationalLawinTheoryandPractice’(1982)Recueildescours9,75;ILC(n198)para211.202RPisilloMazzeschiandAViviani,‘GeneralPrinciplesofInternationalLaw:FromRulestoValues?’,inRPisilloMazzeschiandPDeSena(eds),GlobalJustice,HumanRightsandtheModernizationofInternationalLaw(Springer,2018),123–5(stressingthatthefinalformulationoftheclausewasa‘compromisetext’).203Ibid119–20(arguingthataccordingtoneo-constitutionalism‘intheconstruc-tionoftheprinciples,theinterpreterandespeciallythejudgeexercisesanormalfunc-tionof“discovery”oftheexistingnon-writtenlaw’).SeealsoNTsagourias,‘TheConstitutionalRoleofGeneralPrinciplesofLawinInternationalandEuropeanPerspectives’,inNTsagourias(ed),TransnationalConstitutionalism.InternationalandEuropeanPerspectives(CUP2007).204ILC,‘SecondReportonGeneralPrinciplesofLawbyMarceloVásquez-Bermúdez,SpecialRapporteur’(9April2020)A/CN.4/71para113.205BonaféandPalchetti(n201)169.‘Generalprinciplesoflaw’understoodasderiv-ingfromnaturallawareattimesemployedbysomejudgesintheiropinionsattachedtojudgments.SeeCaseConcerningPulpMillsontheRiverUruguay(ArgentinavUruguay)(Merits)[2010]ICJRep14,SeparateOpinionofJudgeCançadoTrindadepara52;SouthWestAfricaCases(EthiopiavSouthAfrica;LiberiavSouthAfrica)(Merits)[1966]ICJRep6,DissentingOpinionofJudgeTanakapara276.DiegoMauri-9781802207675DownloadedfromPubFactoryat05/28/202203:48:39AMviaDukeUniversityLawLibrary

6658Autonomousweaponssystemsandtheprotectionofthehumanperson(ormoreappropriatelybytheinternationalcommunity)pursuanttoarticle38(1)(c)ismaintainedtotheextentthatitisascertainedthatthey(1)derivefromtreatiesandotherinternationalinstruments,or(2)underliegeneralrulesofconventionalorcustomaryinternationallaw,or(3)areinherentinthebasicfeaturesandfundamentalrequirementsoftheinternationallegalsystem.206Turningtoscholars,thosewhoacknowledgethecategoryofgeneralprinciplesformedwithintheinternationallegalsystemstendtotakeasimilarstance,andstressthatthoseprinciplesconstitutetheinternationallegalorder’s‘axiomaticpremises’207or,asothersputit,asbeing‘fixésapriori,presquedemanièred’axiomes’.208Thosewhoadheretothistheoryoftencitetheprinciplesofjustice,equity,bonafides,territorialsovereignty,equalityofstates,pactasuntservanda,reciprocity,andproportionality.209Ithasbeensuggestedthatotherprinciplesshouldbeaddedtothelist,astheyappeartobecorrespondingtotheinternationallegalorder’scontemporarystructure:namely,theprinciplesof‘humanlifeanddignity’210or‘elementaryconsiderationsofhumanity’.211Someauthorsinsistalsoonthemoralconsistencyofthiscategoryofprinci-ples:itisarguedthattheyexpressthevaluesofthelegalordertheyoperatewithin,andthattheyallowtheinternationallegalordertoundergoaprocessof‘constitutionalization’.212Emphasisonthisfeature,however,doesnotleavelegalpositivismoutofthepicture,asconsent(expressedviatheconceptof‘recognition’illustratedabove)bystatesandtheinternationalcommunityatlargeisalwaysmaintained,206‘Secondreportongeneralprinciples’(n204)paras121ff.207CTomuschat,‘InternationalLaw:EnsuringtheSurvivalofMankindontheEveofaNewCentury.GeneralCourseonPublicInternationalLaw’(1999)281Recueildecours9,161(arguingthatthemostprominentofsuchaxiomaticpremisesundoubt-edlyis‘sovereignequalityofStates’,regardedas‘theprincipalGrundnormofthepresent-dayinternationallegalorder’).208PMDupuy,‘L’unitédel’ordrejuridiqueinternational,CoursGénéraldedroitinternationalpublic’(2002)297Recueildescours9,182.209PisilloMazzeschiandViviani(n202)128ff.210Tomuschat(n207)355–6(recallingtheGrundnormexpressionandbasingtheassumptiononthefactthatsuchprinciplefindsexpressionintheUNCharteraswellasintheUDHR).211PMDupuy(n208)186–7(tracingthisprinciple’soriginbacktoasortofsocialmoralityandpublicorder).212AACançadoTrindade,InternationalLawforHumankind–TowardsaNewJusGentium(MartinusNijhoff2010),496(statingthatinternationallawofthetwentiethcenturywasinspiredby‘ahumanistphilosophy’inwhichgeneralprinciplesplayedaconsiderablyimportantrole);CEggett,‘TheRoleofPrinciplesandGeneralPrinciplesinthe“ConstitutionalProcesses”ofInternationalLaw’(2019)66NetherlandsInt’lLR197.DiegoMauri-9781802207675DownloadedfromPubFactoryat05/28/202203:48:39AMviaDukeUniversityLawLibrary

67Sketchingthedebate59evenbythosetheoriesthatinclinemoretowardsnatural-lawthinking.213Manyarguethatgeneralprinciplesformedwithintheinternationallegalsystemdoultimatelydependonconsentinthattheinterpreterthatdecidestoapplythemhastodemonstratethattheyare‘discernible’andthus‘discover’them.214Itshouldbenoted,however,thatatleastinsomebranchesofinternationallaw(suchasIHRL)theroleofstatepracticeforthepurposeofjustifyingthe‘discovery’ofageneralprincipleisstructurallydifferentfromthatwhichistraditionallyplayedintheascertainmentofcustomaryrules:whenitcomestogeneralprinciples,statepracticeseemstobeinvokedmoreasaconfirmationoftheexistenceofgeneralprinciple,whichisdeducedfromexistingrulesandjustifiedonthebasisofitsaxiologicallyorientatedcontent.215Insummary,themethodologytobefollowedtodemonstratetheexistenceofageneralprinci-pleisallegedlydifferentfromthatregulatingtheidentificationofcustomarynorms.Havingexplainedthenatureandthenormativeprocessthatcharacterizegeneralprinciples,anotherproblematicaspectcanbetracedinthewaytheyaresupposedtofunction.ThistopicisontheILCSpecialRapporteur’sagenda,andwillbetackledinthefollowingmonths.216Again,scholarsdisagreegreatlyonthispoint.Atthelowerendofthespectrum,virtuallyallagreethatgeneralprinciplesserveasahermeneutictooltoassisttheinterpreteringivingcontenttointernationalnorms.217Thisisrarelycontestedasithaslittleornoimpactoninternationallaw-making,andcauseslessanguishtostates.Disagreementsarisewhengeneralprinciplesareacknowledgedinmoreimpactfulfunctions,suchasfillinggapsininternationallaw.Accordingtothisview,generalprinci-plescanbeappliedwhentreatiesandcustomsareabsent.218Asisevident,there213AVerdross,‘Lesprincipesgénérauxdedroitdanslesystèmedessourcesdudroitinternationalpublic’,inPGuggenheim(ed),Receuild’étudesdedroitinterna-tionalenhommageàP.Guggenheim(IUHEI1968),536(referringto‘reconnaissanceexpresse’bytheinternationalcommunity).214MEO’ConnellandCMDay,‘SourcesandtheLegalityandValidityofInternationalLaw.NaturalLawasSourceofExtra-PositiveNorms’inSBessonandJd’Aspremont(eds),TheOxfordHandbookontheSourcesofInternationalLaw(OUP2017)(adoptinganatural-lawapproach).CfBonaféandPalchetti(n201)176(justify-ingthisfindinginlightoftheoriesongeneralprinciplesas‘spontaneouslaw’,thusfromastricterpositivistview).215SeetheconvincingdemonstrationprovidedbyPDeSena,‘Prassi,consuetudineeprincipinelcampodeidirittidell’uomo.Riflessioniinternazionalistiche’(2014)43RagionPratica511.216‘SecondReportonGeneralPrinciplesOfLaw’(n204)para182.217MCBassiouni,‘AFunctionalApproachto“GeneralPrinciplesofInternationalLaw”’(1990)11MichiganJofInt’lL768;BCheng,Generalprinciplesoflaw,asappliedbyinternationalcourtsandtribunals(CUP1953,reprinted1987).218PisilloMazzeschiandViviani(n202)134ff.SeealsoBassiouni(n217).DiegoMauri-9781802207675DownloadedfromPubFactoryat05/28/202203:48:39AMviaDukeUniversityLawLibrary

6860Autonomousweaponssystemsandtheprotectionofthehumanpersonisthechancethatgeneralprinciplescanbeinvokedbyinterpreters—firstandforemost,judgesinsolvingparticularcases—ina‘law-makingprocess’with-heldfromstates.The‘masters’ofgeneralprinciplesare,firstandforemost,judges,taskedwithjus-dicere(‘tospeakthelaw’):whenpresentedwith‘newsituations’inrespectofwhichinternationalruleshavenotyetbeendeveloped,itisincumbentuponthemtoprobeexistinglawand,ifneedbe,todiscoverapreceptapplicabletothecase.219Admittedly,theborderlinebetweentheapplicationofexistingnormsandtheinventionofnewnormscanbeextremelythininthesecases.220Howtoavoidabusesbyinterpreters?AstheworksoftheILCshow,manystateswarnagainstthisscenario,holdingthatthecriteriafordeterminingtheexistenceofgeneralprinciples‘mustnotbeusedasaneasyshortcuttoiden-tifyingnormsofinternationallaw’.221Althoughacategorymaybeappealingforthosewhoaffirmtheexistenceofanormbutlacknormativeevidencethereof(awrittenprovision;opiniojurisandpractice),interpretersremainboundtoprovidereasonsforthesolutiontheysupport.Returningtoourpreviousremarks,itisessentialtokeepinmindthat,whilegeneralprinciplesdonotrequireopiniojurisandpractice(lesttheirdifferencewithcustomarynormsevaporates),theymusthoweverbe‘recognizedbythecommunityofnations’.222Someauthorsunderlinethatthis‘community’doesnotincludejuststates,butalsootherinternationalactorswhoseconductcancontributetoacknowledging—‘discovering’or‘discerning’—theseprinciples.223The‘epis-temiccommunity’isadmittedlybroaderthanthemeresumofstates.Onthisbasis,thatthoseprinciplesarefound‘withintheinternationallegalsystem’(thatis,bydeductionorinferencefromexistinginternationalnorms)wouldshieldtheinterpreterfromaccusationsofarbitrariness.Aconvincingviewhasbeenadvancedaccordingtowhichgeneralprincipleshavea‘transitory’char-acter:theymayforestallthedevelopmentofinternationalnorms—customsandtreaties—byprovingthebasisforbothanevolutionaryinterpretation219SeealsotheconvincingreflectionsadvancedbyECarpanelli,‘GeneralPrinciplesofInternationalLaw:StrugglingwithaSlipperyConcept’,inLPineschi(ed),GeneralPrinciplesofLaw–TheRoleoftheJudiciary(Springer2015).220ReparationforInjuriesSufferedintheServiceoftheUnitedNations(AdvisoryOpinion)[1949]ICJRep174,IndividualOpinionbyJudgeAlvarez190(arguingthatwhenconfrontedtounprecedentedcases‘itisquiteimpossibletosaywherethedevel-opmentoflawendsandwhereitscreationbegins’).221ILC‘SecondReportonGeneralPrinciplesofLaw’(n204)para15;seealsoVásquez-BermúdezandCrosato(n198)171.222ILC‘SecondReportonGeneralPrinciplesofLaw’(n204)para160.SeealsoBConforti,‘Humanitéetrenouveaudelaproductionnormative’,inHumanitéetdroitinternational.MélangesRenéJeanDupuy(Pedone1991).223Eggett(n212)208ff(employingthenotionof‘internationalcommunity’).DiegoMauri-9781802207675DownloadedfromPubFactoryat05/28/202203:48:39AMviaDukeUniversityLawLibrary

69Sketchingthedebate61ofthe(existing)lawandnewnorms(forinstance,byinspiringsubsequentdivergingpractice).224Letuswrapuptheanalysisconductedsofar.Generalprinciplesstandasasourceofinternationallawthat,ratherthangoingbeyondstateconsent,blendspositivistfeatureswithextra-positiveinspirations.Ashasbeenstated,‘lawmakingbygeneralprinciplesisahybridprocesswhichcombinescon-sensualandnon-consensualelements’.225Yetwrittenfromasolidpositivistperspective,thiscontentionisnotunacceptableforthosesupportingamorenaturallaw-orientatedviewongeneralprinciples:‘non-consensual’elementsreflecttheneedtoprotectcorevaluesthatareinherentinagivenlegalorder,irrespectiveofwhethertheyhavebeentranslatedintopositiverules.226Bothviewsdepartfromthebasicassumptionthattheinternationallegalordercannotremain‘blind’inrespectoffundamentalvalues.Thecategoryofgeneralprinciplesoflaw—morespecificallythoseformedwithintheinternationallegalsystems—turnsouttobeofparticularuseindebatingAWS,andthecircumstancethattheongoingdiscussionstendtooverlookthedistinctionbetween‘rules’and‘principles’andthusfailtoappraisegeneralprinciplesasanautonomouslegalsourcerelevantforAWSisworrisome.ThisbookaimstofillthislacunaandproposeaninnovativeapproachtotestingthecompatibilityofAWSwithexistinginternationalnorms.Generalprinciples’potentialinthisfieldemergesonceoneconsidersthat‘thedeductivemethod’(thatis,themethodthroughwhichgeneralprin-ciplesarediscernedbyinterpreters)standsas‘themaincriteriontoestablishtheexistenceofalegalprinciplethathasageneralscopeandmaybeappliedtoasituationnotinitiallyenvisagedbytherulesfromwhichitwasderived’.227Accordingtoourpreviousfindings,generalprinciplesarestructuredtoapplyto‘unusualcases’—thatis,caseswhichnotreatyorcustomaryruleregulate:theyorientatetheinterpretationoftheserules,inordertofillgapsandprovideasolutionforthecase.228Inconclusion,itisourcontentionthatAWSaresituatedpreciselywithintheperimeterofan‘unusualcase’:neverinthehistoryofhumankindhasforcebeenappliedagainstatargetwithouthumaninterventionincriticalfunctions224BonaféandPalchetti(n201)174–5.225Ibid176.226O’ConnellandDay(n214).227ILC‘SecondReportongeneralprinciplesoflaw’(n204)para168(emphasisadded).228Ongeneralprinciplesasnormativesourceprovidinglawforcasesnotregulatedbycustoms(literally,‘unusual’),seethelandmarkworkofLGradoni,‘Consuetudineecasoinconsueto’(2012)95Rivistadidirittointernazionale704.DiegoMauri-9781802207675DownloadedfromPubFactoryat05/28/202203:48:39AMviaDukeUniversityLawLibrary

7062AutonomousweaponssystemsandtheprotectionofthehumanpersonbeforeAWS.229Wemaybeexperiencingoneofthose‘magicalmoments’inwhichnorulecanbefoundthatcoversthiscaseinitsentirety.230Granted,rulesexistininternationallawthatregulatetheuseofforceinlaw-enforcementandarmed-conflictscenarios.However,thathumanagentsdonotinterveneintheselectionandengagementofaparticulartargetissomethingentirelynewinthehistoryofIHRLandIHL.Generalprinciplesmayprovidesuperbguidancethroughoutthisjourney.Inparticular,onemustturntohumandignityandhumanityasgeneralprinciplesofIHRLandIHLrespectively.2.3.4TheNotionsandFunctionsof‘HumanDignity’and‘Humanity’inIHRLandIHL:InSearchofaTheoreticalCompassIntheprevioussubsectionwedelimitatedthescopeofouranalysistoIHRLandIHLquabodiesofinternationallawdealingwiththeuseofforceagainstindividuals,andweillustratedthenormativecategoriesinwhichnormsper-tainingtothetwobranchescanbeframed(rulesandgeneralprinciples).Inlinewiththemainthesisofthisbook,231whatremainstobeassessediswhethertheconceptsof‘humandignity’(aselaboratedinIHRL)and‘humanity’(aselab-oratedinIHL)standasgeneralprinciplesformedwithintheinternationallegalorder,sothattheinterpreter—beitastate,acourtoralegalscholar—isauthor-izedtoresorttothemwhenpresentedwithanunregulated(or‘unusual’)case.Yetomnipresentincurrentlegaldiscourses,theconceptsofhumandignityandhumanity—‘twin’notionsindeed—donotdisplayaunivocalcontent,norhavetheirfunctionsinagivenlegalorderbeenuniversallyagreedupon.Asforthecontent,toillustrateitthoroughlywouldrequireamonographicworkonitsown(andthismaystillbeinsufficient).Afewkeypoints,however,canbeoutlined.Dignityevokesasetofvaluesembodiedinthehumanpersonbyreasonoftheirinherentworth.232Thehumanpersonhastoberespected,andtheirrightseffectivelyguaranteed,forthesolefactofpertainingtothehumanfamily.Religiousapproachesgoshouldertoshoulderwithmoresecularvisionsondignityandtheneedtorespectitinallcircumstances.Thedebateonhumandignity—whichblossomedimmediatelyafterWorldWarII,byinspiringbedrockinstrumentssuchastheUDHR—waspredatedbythedevelopmentofhumanitarianisminthelawsofwar.233Duringthenineteenthcentury,theidea229SeeCh1,ss1.1and1.2.230Gradoni(n228)715.231SeeCh1,s1.3.232CMcCrudden,‘InPursuitofHumanDignity:AnIntroductiontoCurrentDebates’,inCMcCrudden(ed),UnderstandingHumanDignity(OUP2013).233KMujezinovicLarsen,‘A“PrincipleofHumanity”ora“PrincipleofHuman-rightism”?’,inKMujezinovicLarsen,CGuldahlCooperandGNystuen(eds),DiegoMauri-9781802207675DownloadedfromPubFactoryat05/28/202203:48:39AMviaDukeUniversityLawLibrary

71Sketchingthedebate63wherebyenemiescouldbedestroyedatanycostwasdeterminedlyrejected:limitationsexisted.234ThisideahasbeencapturedbrilliantlybyCoupland,whodistinguishedbetweenthenotionof‘humanity-mankind’(referringtotheentiregroupofhumanbeingspopulatingtheworld)and‘humanity-sentiment’(understoodasaformofbenevolentandpositiveattitudetowardstheother).235Humandignityandhumanityconstitutethesourcefromwhichhumanrightsandnormsprotectingindividualsduringhostilitieshavesprung.JeanPictet,tonameone,didnothesitatetorefertohumanityastheessentialprinciplefromwhichallotherprinciplesoftheICRCproceed.236Oneofthemultifariousunderstandingsofthoseconceptsisthattheyentailabasicimperativenottoreduceindividualsto‘meremeans’or‘dispensablequalities’:intheKantianjargon,nottotreatthemasmeans,butends.237Morerecently,adifferentunderstandinghassurfaced,accordingtowhichdignitystandsalsoasasourceofdutiesthatindividualsholdvis-à-viseachother,namely,totreatthemselvesinawaythatdoesnotendangerthespecies.238Forourpurposesinthisbook,welookmoreatthefirstmeaning,asithasgainedmoretractioninthedebateonAWS,whoseconcretewaysofoperatingareheldtoraisequestionsastohowthereceiverof(potentiallylethal)forceistreated.239Theirdignityandhumanityaredirectlycalledintoquestion.Astotheirfunction,numerous(andattimesantithetical)approacheshavebeenfollowed.Onemayevenrecalltheideaofa‘spectrum’(asweemployeditwithregardtoautonomy)toshowhowpolarizedthedebateonthelegalstandingofhumandignityandhumanityis.Onthepremisethatathoroughaccountwillbeprovidedinthefollowingchapters,ourpositionisthat,owingtotheirstructureandthevaluestheyencapsulate,bothhumandignityinIHRLandhumanityinIHLworkasgeneralprinciplesformedwithintheSearchingfora‘PrincipleofHumanity’inInternationalHumanitarianLaw(CUP2013).234AAlexander,‘AShortHistoryofInternationalHumanitarianLaw’(2015)26EJIL109,115.235RCoupland,‘Humanity:WhatItIsandHowDoesItInfluenceInternationalLaw?’(2001)83IRRC969.236LFast,‘UnpackingthePrincipleofHumanity:TensionsandImplications’(2016)97IRRC111.237DAmoroso,AutonomousWeaponsSystemsandInternationalLaw.AStudyonHuman-MachineInteractionsinEthicallyandLegallySensitiveDomains(EdizioniScientificheItaliane2020),167ff.238Ibid170.SeemoreextensivelyPDeSena,‘Dignitàumanainsensooggettivoedirittointernazionale’(2017)11DirittiUmanieDirittoInternazionale573,574(andreferencestherein).239OUlgen,‘KantianEthicsintheAgeofArtificialIntelligenceandRobotics’(2017)43QuestofInt’lL59.DiegoMauri-9781802207675DownloadedfromPubFactoryat05/28/202203:48:39AMviaDukeUniversityLawLibrary

7264Autonomousweaponssystemsandtheprotectionofthehumanpersoninternationallegalsystem.Inadditiontoplayinga‘foundationalrole’inthosebranchesofinternationallaw,byprovidingan‘ethicallymotivatedanswer’totheneedtoensurerespectforhumanbeingswithinandoutsidehostilities,theyperformfunctionsthatare‘typicallyascribedtogeneralprinciplesoflaw’.240Bothnotionshaveemergedinthetwobranchesofinternationallawtogivesubstancetocorevaluesthathaveemergedthroughthedecades(theworthofthehumanperson),andbothfitintothecategoryofgeneralprinciples.However,itmustbeacknowledgedthatdifferentandnuancedunderstand-ingsofthoseconceptshavebeenadvanced,inparticularasregardshumanityunderIHL.Schmitt,forinstance,notesthe‘delicatebalance’betweentheprincipleofhumanityandthecompetingprincipleofmilitarynecessity,whichareregardedas‘foundationalprinciples’ofIHL.241AccordingtoBeer,IHL’staskisto‘balancethedemandsofmilitarynecessityagainsthumanitariancon-cerns’,thatis,tofindanequilibriumthat,althoughshiftingovertime(depend-ingonthenatureofconflict,thecompetinginterestsofactorsinvolved,theweaponsused,andsoon),rendersabidancewithitsnormsrealisticandthuseffective.242InReevesandThurnher’sview,thecurrentoveremphasisonhumanitarianconcernsattheexpensesofmilitarynecessityrisksdiminishingthelikelihoodofcompliancewithIHL,withevidentlynefariouseffectsonindividualsaffectedbyhostilities.243Morerecently,Wintersuggestedthat,owingtothecircumstancethatneithermilitarynecessitynorhumanityarecapableofsupersedingexistingrules,theyshouldberegardedmoreas‘pillars’(thusechoingtheconceptof‘foundationalprinciples’)ratherthanprinciplesstrictosensu.244Thesecontentionsdeservecreditinthattheyaimtokeeplexlataandlexferendadistinct:theinvocationofhumanitarianneedsforpromot-ingnewlaw,especiallybyNGOs,mustnotobliterateexistingrules.Whiledefendable,inourviewthesepositionsarenotconvincing,especiallyfromatheoreticalstandpoint.Tolabelhumanityasa‘foundationalprinciple’orasa‘pillar’makessensefromadescriptiveperspective(inthatitcapturestheirimportanceinthewholenormativeedifice),butitmaysuggestthattheseconceptsareinthemselvesvoidoflegalsignificance.Onthecontrary,tocon-240Amoroso(n237)171ff.241MNSchmitt,‘MilitaryNecessityandHumanityinInternationalHumanitarianLaw:PreservingtheDelicateBalance’(2010)50VirginiaJIL795.242YBeer,‘HumanityConsiderationsCannotReduceWar’sHazardsAlone:RevitalizingtheConceptofMilitaryNecessity’(2016)26EJIL801.243SRReevesandJSThurnher,‘AreWeReachingaTippingPoint?HowContemporaryChallengesAreAffectingtheMilitaryNecessity-HumanityBalance’(2013)4HarvardNat’lSecJFeatures1.244EWinter,‘PillarsnotPrinciples:TheStatusofHumanityandMilitaryNecessityintheLawofArmedConflict’(2020)25JofConflict&SecL1.DiegoMauri-9781802207675DownloadedfromPubFactoryat05/28/202203:48:39AMviaDukeUniversityLawLibrary

73Sketchingthedebate65ceivethemasgeneralprinciples,toberesortedtowheninterpretingexistinglaw,offersatheoreticalframeworkthathelpsanyinterpreterreachasolutionforanunregulatedcase.Whilethelaw-makingprocessbystatesneedstostrikethecorrectbalancebetweencompetinginterestsandvalues(asexemplifiedbythedichotomy‘humanity’/’militarynecessity’underIHL),onemustnotlosesightoftheinterpreter,anditsfundamentalroleinidentifyingrulesandprinciplestoapplytoacase.Itshouldbestressedthatsuchahermeneuticoper-ationrequiresastringentdemonstrationbasedonastrictmethodology,lestthefinalresultbearbitraryand,therefore,unacceptableforthosewhoapplythelaw(firstandforemost,asfarasIHLisconcerned,statesonthebattlefield).Admittedly,itmaybethatsomeinterpreters—especiallyIHRLmonitoringbodies—makeleapsforwardandundulygobeyondexistinglaw,thustendingtoblurthelinebetweendelegelataanddelegeferendaarguments.Thispossibility,however,doesnotrevoketheimportanceofresortingtogeneralprinciplestoaddressthechallengesposedbyunusualcases:thisistherealmofprinciples.Insummary,ourhypothesisremainsvalid:wheneverexistingrulesofIHRLandIHLareinsufficienttocoveraparticularcaseunderallitssalientaspects,interpretersareallowedtoresorttogeneralprinciples.Theirrecognitionbytheinternationalcommunityhastobedemonstratedthroughaprecisemeth-odologyinordertoavoidarbitraryconclusions.Humandignityandhumanityqualifyasgeneralprinciplesof,respectively,IHRLandIHL.Theyareaddressedseparatelysolelytomaintainsymmetryofanalysis;however,aswillbeillustrated,theircontentiscontiguous.Whatremainstobedemonstratediswhethertheycanbeunderstoodasimposingadegreeofhumaninterventionintheperformanceofcriticalfunctions.2.4CONCLUDINGREMARKSLetusnowbrieflyrecapitulatethemainpointsthathaveemergedinthischapter.WedemonstratedthattheissueofAWShasbeentheobjectofexten-sivereflection,bothinscholarshipandamongthewiderpublic:theseweaponssystemsraiseimportantchallengesfrommultifariousperspectives(military,ethical,politicaland,especially,legal),alltakenintoaccountinthedebatewithintheCCWperspectives.Astotheirdefinition,theapproachweadoptedisbasedonaworkingdefi-nitioncapableofunderscoringthemostsalientfeaturesofAWS:thedegreeofhumancontroloverthesystemvariesconsiderablyonaspectrum.Thus,itisarguedthatitmakeslittlesensetoconceiveof‘autonomy’asamonolith:instead,ithastobeappraisedasadynamicconceptthatdescribesthedegreeofhuman–machineinteractionsinagivensystem.Accordingly,theeffectsoftheprogressive‘automation’ofweaponssystemsneedtobemeasuredagainstDiegoMauri-9781802207675DownloadedfromPubFactoryat05/28/202203:48:39AMviaDukeUniversityLawLibrary

7466Autonomousweaponssystemsandtheprotectionofthehumanpersonthefunctionstobeperformed:targetselectionandengagementraisedifferentchallengesthandonavigationandorienteering.Thus,ourapproachconsiderscriticalfunctionsofAWS,asitisthesethatwillimpingemoreoninternationalnormsprotectingthehumanperson.Turningtothelegalanalysis,wehavelimitedourinvestigationtoIHRLandIHL,excludingthejusadbellumdimensionofAWS.Inourview,IHRLandIHLstandasdifferentbodiesofinternationallawthat,whiledevelopedtoapplytodistinctscenarios,mayendupoverlappinginpractice.SinceIHRLisalsoheldtoapplyduringhostilities,thismaycreatenormativeconflictswithIHL.Theapproachweadoptedwasbasedonthefollowingconsiderations:(1)oursegregatedanalysisofIHRLandIHLhastheadvantageofprovidinganassessmentoftheimplicationsofAWSwithouthavingtoresolveonceandforalltheconundrumoftheinterplaybetweenthebodiesoflaw(tothepointthatourpositioncanberegardedasultimately‘agnostic’inthisregard);(2)inordertoprovideforward-lookingreflectionsonAWS,whichremainvalidalsointhelongrun(whenautonomoustechnologywillbecomesuitablealsofornon-militaryoperations),andonthebasisthatIHRLgenerallyremainslexgeneralisvis-à-vistheuseofforceagainstindividuals,IHLwillbeconsideredafterIHRL.WeadheretothethesisaccordingtowhichIHRLandIHLshareacommon‘origin’,namely,respectforandthepromotionofthehumanpersonandtheirowndignity.WhileIHLestablishesasetofrulesthat,bytakingintoaccountcompetinginterestssuchasmilitarynecessity,allowforalessrestrainedresorttoforcethanIHRL,thetwobodiesoflawtouchattheapex:thebook’smainthesisisthusthateveryapplicationof(potentiallylethal)forcethroughAWSneedstobeassessednotonlyagainstIHRL/IHLrules,butalsoagainstthegeneralprinciplesofhumandignity/humanity.Thisjustifiesourreverseorderofanalysisvis-à-visscholarship,andwillprovideforabetterappraisalofthedichotomyrules/principlesineachbranchofinternationallaw.Withthisinmind,wenexttackleprimarynormsofIHRLandIHLthatwillbeimpactedbytheadventofAWS.DiegoMauri-9781802207675DownloadedfromPubFactoryat05/28/202203:48:39AMviaDukeUniversityLawLibrary

753.Autonomousweaponssystemsunderinternationalhumanrightslaw3.1AWSFORLAW-ENFORCEMENTOPERATIONS:PRACTICALSCENARIOSWhilethedebateattheCCWismonopolizedbythefocusonmilitaryappli-cationsofAWS,resultinginapreferenceforlegalanalysisconductedthroughthelensofIHL(asguidingprinciple(a)goes),1theparamountimportanceofIHRLcouldhardlybeoverestimated,foratleastthreereasons.First,itscon-tinuingrelevanceinarmed-conflictscenariosonaparwithIHLhasalreadybeenillustrated;2second,whenitcomestolaw-enforcementscenarios,IHRLstandsasthesolelyapplicablelegalregime;third,althoughAWSaredis-cussedmainlyasamilitary-driventechnology,theirapplicationsinpolicingoperationscannotbeignored.Beforeillustratingthischapter’scontent,itisappropriatetopresentpracticalscenariosinwhichAWSarelikelytobeemployedinthefuture.Remotelycontrolleddevices—unarmedorarmed—areoperatedinbordermonitoring(herebybringingabout‘smartborders’,forinstanceintheEUorattheUS–Mexicoborders),3crowdandriotcontrol,4andeven(extraterritorial1GroupofGovernmentalExpertsoftheHighContractingPartiestotheConventiononProhibitionsorRestrictionsontheUseofCertainConventionalWeaponsWhichMayBeDeemedtoBeExcessivelyInjuriousortoHaveIndiscriminateEffects,‘Reportofthe2019sessionoftheGroupofGovernmentalExpertsonEmergingTechnologiesintheAreaofLethalAutonomousWeaponsSystems(LAWS)’(25September2019)CCW/GGE.1/2019/3AnnexIV.2SeeCh2,s2.3.2.3PHankeandDVitiello,‘High-TechMigrationControlintheEUandBeyond:TheLegalChallengesof“EnhancedOperability”’,inECarpanelliandNLazzerini(eds),UseandMisuseofNewTechnologies(Springer2019);USCustomsandBorderProtection,‘CBP’sAutonomousSurveillanceTowersDeclaredaProgramofRecordalongtheSouthwestBorder’(2July2020).4Forathoroughoverviewofexistingtechnology,seeCHeyns,‘HumanRightsandtheUseofAutonomousWeaponsSystems(AWS)DuringDomesticLawEnforcement’(2016)38HumanRightsQuarterly350,358–61;ASpagnolo,‘Human67DiegoMauri-9781802207675DownloadedfromPubFactoryat05/28/202203:48:40AMviaDukeUniversityLawLibrary

7668Autonomousweaponssystemsandtheprotectionofthehumanpersonand,attimes,domestic)targetedkillingsalmostonadailybasis.5Roboticsisawell-knownpromisingfieldofresearchforimprovinglawenforcement:forexample,interactionbetweenhumansandmachinesallowsforreducingexposuretoriskindangerousscenarios.6Advancementsinautonomyarepartofagreatertrendtowardsthe‘digitalization’oflaw-enforcementoperations,whichcontemplatestheincreasinguseofmachinelearningandpredictivealgorithms:policeauthoritiesofmanyStatesacrosstheglobearetesting—ifnotusingalready—thesetools.7Technologyenablingthebulkinterceptionofdataandtheirre-elaborationthroughadvancedAItechniquesisalreadyinuse;technologyenablingthereleaseofforceagainstaparticulartarget(forexample,arioter,anescapeeoraterrorist)isimminent.8Consideranti-personnelsentryweapons,stationaryormobilesystemsemployedtopatrolspecificsites,perimetersorborders.HerethemostfamousexampleisSouthKorea’sborderprotectionsystemstationedattheDemilitarizedZone(DMZ).9Arotatingturretemploysoptical,thermalandinfraredsensorstoselecthumantargets,butrequiresremoteauthorizationfromanoperatortoengagethem;however,thesystemsarealreadyendowedwithfunctionsallowingthemtotargetindividualsautonomously.10Thatis,thechoicetomaintainthesysteminanautomatedmodeismerelyoneofpolicy—thetechnologyforactualautonomyalreadyexists.RightsImplicationsofAutonomousWeaponSystemsinDomesticLawEnforcement:Sci-FiReflectionsonaLo-FiReality’(2017)43QuestofInt’lL33,39–42.5KBenson,‘“Kill’EmandSortitOutLater”:SignatureDroneStrikesandInternationalHumanitarianLaw’(2014)27PacificMcGeorgeGlobalBusiness&DevelopmentLJ17(focusingonarmedconflictbutdescribingthepracticeof‘signa-turestrike’generally).6CLundbergandHIChristensen,‘AssessmentofMan-portableRobotsforLawEnforcementAgencies’(2007)PerMIS’07:Proceedingofthe2007WorkshoponPerformanceMetricsforIntelligentSystems76(claimingthattheinteractionbetweenhumanagentandunmannedsystemsprovedbeneficialalsoinnegotiatingscenarios).7AZavršnik,‘CriminalJustice,ArtificialIntelligenceSystems,andHumanRights’(2020)20ERAForum567.8ADeeks,NLubellandDMurray,‘MachineLearning,ArtificialIntelligence,andtheUseofForcebyStates’(2019)10JofNat’lSecL&Policy1(discussingAIimplicationsonthewaystateswilluseforceinternationallyanddomestically).9LBlain,‘SouthKorea’sAutonomousRobotGunTurrets:DeadlyfromKilometersAway’(NewAtlas,7December2010)http://newatlas.com/korea-dodamm-super-aegis-autonomos-robot-gun-turret/17198/accessed31August2021(underlyingthatthissystemcan‘findandlockontoahuman-sizedtargetinpitchdarknessatadis-tanceofupto1.36miles’).10SParkin,‘KillerRobots:TheSoldiersthatNeverSleep’(BBC,16July2015)http://www.bbc.com/future/story/20150715-killer-robots-the-soldiers-that-never-sleepaccessed31August2021.DiegoMauri-9781802207675DownloadedfromPubFactoryat05/28/202203:48:40AMviaDukeUniversityLawLibrary

77Autonomousweaponssystemsunderinternationalhumanrightslaw69Thisbeingthecase,athoroughassessmentofAWSthroughthelensofIHRLcannotbedelayed.Inlinewiththeresearchquestionsdrivingthisbook,ourobjectiveistoestablishthedegreetowhich‘humancontrol’mustberetainedinorderforAWStobeoperatedincompliancewithIHRL.OnthepremisethatAWSraisetheissueofusing(potentiallylethal)forceagainsthumantargets,thischapterfirstconsidersthehumanrightsparticularlyatstakeinthesecases,namely,therighttolife(section3.2),andtherightnottobesubjecttotorture,inhumanordegradingtreatment(3.3).Foreachright,attentionisgiventocustomaryandtreatyrules,aswellastothecurrentinter-pretationsthereofbyjudiciaryandquasi-judiciarybodies.OtherrightsareconsideredthatcouldbeimpactedbyAWS(suchastherighttoprivacyortherightnottobediscriminatedagainst)inthesamewayasareothertechnologies(suchasbigdatacollectionandelaboration),herebyraisingissuesthatarenotspecifictoAWS(3.4).TheanalysiseventuallyturnstoasecondtierofsourcesofIHRL,namely,‘generalprinciples’,andwillquestionfirstwhether‘humandignity’—initselfacontestednotion—canbeacknowledgedasageneralprincipleofIHRL,andsecondwhetherthisnotioncanbeinvokedasagroundforrejectingAWSperse(3.5).HavingconcludedoursubstantiveanalysisofIHRLnorms,section3.6isdevotedtoatopicalissueofIHRL,namely,itsapplicationinarmed-conflictscenariosanditsextraterritorialreach:thoseareareasthat,althoughraisingcriticalpointswhoseresolutiondoesnotultimatelydependonthemostsalientfeaturesofAWS,deservementioningforthesakeofcompleteness.Finally,thecrucialpointsemergedthusfarareenumeratedanddiscussedinaconcludingparagraph(3.7).3.2RULESONTHEUSEOFFORCEAGAINSTTHEHUMANPERSON:THERIGHTTOLIFEOftenreferredtoasthe‘supremeright’,11therighttolifeiswhereanyuseoflethalforcebegins.12TherighttolifefindsitsplaceintheUDHR(onapar11UNHRC,‘GeneralCommentNo.6:Article6(RighttoLife)’(30April1982)HRI/GEN/1/Rev.1para1;UNHRC,‘GeneralCommentNo.36(2018)onArticle6oftheInternationalCovenantonCivilandPoliticalRights,ontheRighttoLife’(30November2018)CCPR/C/GC/36(GC36).SeealsoCCPR,BaboeramandothersvSuriname,CommNo146,148-154/83(4April1985)para14.3;ECtHR,Streletz,KesslerandKrenzvGermany,AppsNos34044/96,35532/97and44801/98(22March2001)paras72,87and94;IACtHR,BaldéonGarciavPeru,SeriesCNo147(6April2006).12MEO’Connell,‘TheLawonLethalForceBeginswiththeRighttoLife’(2016)3JontheUseofForce&Int’lL205.DiegoMauri-9781802207675DownloadedfromPubFactoryat05/28/202203:48:40AMviaDukeUniversityLawLibrary

7870Autonomousweaponssystemsandtheprotectionofthehumanpersonwiththerighttolibertyandsecurityofperson)13aswellasinanymajorhumanrightstreaties,suchastheEuropeanConventiononHumanRights(ECHR),14theInternationalCovenantonCivilandPoliticalRights(ICCPR),15theAmericanConventiononHumanRights(ACHR)16andtheAfricanCharteronHumanandPeoples’Rights(ACHPR).17Ithasgainedcustomarystatus,18andcurrentlyitisevenconsideredaspartofjuscogens.19Beforeaddressingtherighttolifefromtheviewpointoftheuseofforce,itseemsappropriatetomakefewremarksonthelegalregimeensuringitsprotection.Thiswillhelpeluci-dateboththerationaleandthescopeoftherulesprotectingtherighttolife.First,thisrighthasbeenproclaimedinsolemn,high-soundingterms.Itisconsideredtobean‘essential’right,fromwhichallothersspring:‘ifapersonisdeprivedofhisrighttolife,allotherhumanrightswillbemeaningless’.20AsshownbytheUDHR’sresorttotheterm‘inherent’,therewasadiscerniblesensethattherighttolifehadasolidbackgroundincustomarylaw,eveninextra-positivelaw.21Therighttolifeholdsacentralplaceinthefieldofhumanrights,itsprotectionhavinginspiredanimmensescholarshiptodate.22Second,therighttolife’sparamountimportancejustifiesitscharacterizationasarightnotsubjecttoderogationandasabsoluteinnature.Regardingtheformer,noderogationtotherighttolifeispermittedinexceptionalsituations,13UniversalDeclarationofHumanRights(adopted10December1948)UNGARes217A(III)(UDHR)art3.14ConventionfortheProtectionofHumanRightsandFundamentalFreedoms(EuropeanConventiononHumanRights,asamended)art2.15InternationalCovenantonCivilandPoliticalRights(adopted16December1966,enteredintoforce23March1976)999UNTS171art6.16AmericanConventiononHumanRights(adopted22November1969,enteredintoforce18July1978)OASTreatySeries36art4.17AfricanCharteronHumanandPeoples’Rights(adopted27June1981,enteredintoforce21October1986)(1982)21ILM58art.4.18ReportoftheUnitedNationsSpecialRapporteuronExtrajudicial,SummaryorArbitraryExecutions,ChristofHeyns’(2013)A/HRC/23/47(HeynsReport)para42.19O’Connell(n12)206;PGormley,‘TheRighttoLifeandtheRuleofNon-Derogability:PeremptoryNormsofJusCogens’,inBGRamcharan(ed),TheRighttoLifeinInternationalLaw(MartinusNijhoff1985).20HKabaalioğlu,‘TheObligationsto“Respect”and“Ensure”theRighttoLife’,inBGRamcharan(ed),TheRighttoLifeinInternationalLaw(MartinusNijhoff1985)160.21Ibid161(citingLordDukeston’sinterventionintheSecondSessionoftheCommissiononHumanRightsaccordingtowhichtherighttolifewas‘partofthelawofnaturewhichwasthefoundationofalllawandinternationallaw’).22YDinstein,‘TheRighttoLife,PhysicalIntegrity,andLiberty’,inLHenkin(ed),TheInternationalBillofRights,TheCovenantonCivilandPoliticalRights(ColumbiaUniversityPress1981).DiegoMauri-9781802207675DownloadedfromPubFactoryat05/28/202203:48:40AMviaDukeUniversityLawLibrary

79Autonomousweaponssystemsunderinternationalhumanrightslaw71suchasarmedconflictorpublicemergencies.23Theprohibitionofderogationfromtreatyprovisionsprotectingtherighttolifeissetoutexplicitlybyarticle15(2)ECHR,article4(2)ICCPR,andarticle27(2)ACHR,andonlyimplicitlybytheAfCHPR.Therationaleofthisprohibitionliesinthecircumstancethattherighttolifeisthe‘prerequisitefortheenjoymentofallotherhumanrights’andtherefore‘hascrucialimportancebothforindividualsandforsocietyasawhole’.24Toderogatefromitwouldmeantodenythisimportanceinpractice.Asforthe‘absolute’characteroftherighttolife,themeaningofthisconceptisdebated.Forinstance,GC36claimsthat‘[a]lthoughitinheresineveryhumanbeing,therighttolifeisnotabsolute’.25Therelativecharacterstemsfromthecircumstancethatarticle6ICCPRallowsfordeprivationsoflifeonconditionoftheirbeing‘non-arbitrary’.Onthecontrary,severalauthorsconsidertherighttolifeasabsoluteinthat‘humanlifemaylawfullybetakenincertainrestrictivesituationsconsistentlywiththerighttolife’.26Thedilemma,however,ismoreapparentthanreal,asthewayoutofitcanbetoconsiderthattherighttolifeisabsoluteasawhole,thatis,includingtheexceptionsthereto.Conditionsforits(lawful)deprivationarethuspartoftheabsolutecharacteroftheright.Asshownbelow,thishasarestrictiveimpactontheinterpretationoftheseconditions,inkeepingwiththemaximexceptionessuntstrictaeinterpretationis.Third,inanattempttodefinewhattheconceptof‘arbitrary’deprivationsoflifemeans,itisheldthattherighttolifeencompassesnotonlynegativeduties(thatis,toabstainfromdeprivingsomeone’slifeabsentspecificrequirements)butalsopositiveduties(thatis,toensuretheenjoymentofittoallconcernedindividuals).Thedistinctionbetweennegativeandpositiveobligationsiswellsettledinscholarship.27Encapsulatedinthemoregeneralobligationto23GC36para1.SeealsoCCPR,SuarezdeGuerrerovColombia,CommNoR.11/45(31March1982)para13.1.24GC36para1.SeealsoECtHR,MakaratzisvGreece,AppNo50385/99(20December2004)para56:‘Article2(1)…,whichsafeguardstherighttolifeandsetsoutthecircumstanceswhendeprivationoflifemaybejustified,ranksasoneofthemostfundamentalprovisionsintheConvention,fromwhichnoderogationispermitted....TogetherwithArticle3,italsoenshrinesoneofthebasicvaluesofthedemocraticsoci-etiesmakinguptheCouncilofEurope’.25SeeGC36para16.26O’Connell(n12)206.27DSheltonandAGould,‘PositiveandNegativeObligations’,inDShelton(ed),TheOxfordHandbookofInternationalHumanRightsLaw(OUP2013);AMowbray,HumanRightsLawinPerspective:TheDevelopmentofPositiveObligationsundertheEuropeanConventiononHumanRightsbytheEuropeanCourtofHumanRights(Hart2004);LLavrysen,‘PositiveObligationsintheJurisprudenceoftheInter-AmericanCourtofHumanRights’(2014)7Inter-AmericanandEuropeanHumanRightsJournalDiegoMauri-9781802207675DownloadedfromPubFactoryat05/28/202203:48:40AMviaDukeUniversityLawLibrary

8072Autonomousweaponssystemsandtheprotectionofthehumanperson‘ensure’therightscontainedinatreaty,28positiveobligationscompleteandfulfillthebasicdutytorefrainfromtakingone’slife:theyaretheproductofabroadinterpretationofthe‘inherent’righttolife.29Suchaffirmativeactionconsistsofbothadutytoprotectlifebyminimizingorpreventingtheuseoflethalforceandadutytoconductinvestigationsoncealossoflifeoccurredincontraventionoftherulehasoccurred.Havingexplainedbasicelementsoftherighttolife,theanalysisnarrowsdowntotheobligationsincumbentonstateswhenresortingtolethalforceagainstindividuals.Law-enforcementscenarios,eitherdomesticallyorextra-territorially,arethusunderscrutiny.Thisbodyofnorms—attimesreferredtoas‘lawoflawenforcement’,30inaquestionablyfragmentaryapproach—iscomposedoftreatylaw,asinterpretedbyrelevantjudiciaryandquasi-judiciarymonitoringbodies,aswellascustomarylawandsoft-lawinstruments,suchasthe1979CodeofConductforLawEnforcementOfficials(hereafter,CodeofConduct)31andthe1990BasicPrinciplesontheUseofForceandFirearmsbyLawEnforcementOfficials(hereafter,UNBasicPrinciples).32Theparamountrequirementforassessingwhetheraparticulardepriva-tionoflifeoccursinkeepingwiththenormsregulatingtherighttolifeis‘non-arbitrariness’:thisisthestartingpointofouranalysis.However,inordertograspitspracticalapplicationandthusitscontent,thisconceptwillbeunpackedandanalyzedundertheprismofnegative(section3.2.1)andpositive(3.2.2)obligations.Eventually,thegeneralnotionofarbitrarinesswillbere-appraisedsoastotestwhetheriteffectivelycontributestothediscussiononAWS(3.2.3).3.2.1NegativeObligationsAlltreatyformulationsoftherighttolifeprovidepermissiblegroundsfordeprivationoflife,eitherthroughanenumeration(asinarticle2ECHR)orbywayofgeneralclauses,suchastherequirementofnon-arbitrarinessasperarticle6ICCPR.Iftheconditionsforapermissibledeprivationoflifearenot94.SeealsoRPisilloMazzeschi,‘Responsabilitédel’Etatpourviolationsdesobli-gationspositivesrelativesauxdroitsdel’homme’(2008)333RecueildesCours175.28ICCPR,art2;ECHR,art1;ACHR,art1;AfCHPR,art1.SeealsoKabaalioğlu(n20).29GC36para3;CCPR,SuarezdeGuerrerovColombia(n23)para93.30SCasey-Maslen(ed),WeaponsunderInternationalHumanRightsLaw(CUP2014)xvi–xvii.31UNGA,Resolution34/169(17December1979)A/RES/34/169.32EighthUNCongressonthePreventionofCrimeandtheTreatmentofOffenders(Havana,27August–7September1990).DiegoMauri-9781802207675DownloadedfromPubFactoryat05/28/202203:48:40AMviaDukeUniversityLawLibrary

81Autonomousweaponssystemsunderinternationalhumanrightslaw73met,theactionresultinginthelossoflifeisinbreachoftherelevantprovision,morespecificallyofthenegativeobligationcontainedtherein.Unpackingthejurisprudenceofjudiciaryandquasi-judiciarymonitoringbodiesontherighttolife,itispossibletosingleoutthreebasicrequirements:legality,necessityandproportionality.Weaddresstheminturn.3.2.1.1LegalityThefirsttenetofthenegativeobligationstemmingfromtherighttolifeisthustheprohibitionoftakingone’slifeabsentalegalbasis:wecanrefertothisastheprincipleoflegality.Briefly,anydeprivationoflifethatlackslegalbasisisarbitraryperse.Thedeprivationoflifemustresultfromtheexerciseofapowerthatisprovidedeitherindomesticlaworininternationallaw,orboth.Article6ICCPR,article2ECHRandarticle4(1)ACHRallexpresslyimposethatlifebeprotectedbylaw,whereastheACHPRdoessoimplicitly.33Theexistenceofalegalandregulatoryframeworkfortheuseoflethalforceisthusaprerequisiteforalawfulresorttoit.TheUNBasicPrinciplesthor-oughlyindicatethatthisframeworkmustbesufficientlydetailedanddescribewhicharmsandammunitionarepermittedinagivencircumstance.34Regionalanduniversalmonitoringbodieshaveconsistentlyrecalledtheimportanceoflaw-enforcementauthoritiesbeingprovidedwithanappropriateoperativeframework.35Inthisrespect,theuseofAWScanbeconsistentwiththeprincipleoflegal-ityaslongasanappropriatenormative(thatis,legalandregulatory)frame-workisputinplacebystateauthoritiesthatregulatetheconditionstowhichlifecanbetakenbylaw-enforcementauthorities.Thishaslittletodowith33AfricanCommissiononHumanandPeoples’Rights(ACommHPR),‘GeneralCommentNo.3ontheAfricanCharteronHumanandPeoples’Rights:TheRighttoLife(Article4)’(GeneralCommentNo3)para7(‘StateshavearesponsibilityundertheChartertodevelopandimplementalegalandpracticalframeworktorespect,protect,promoteandfulfilltherighttolife’).34UNBasicPrinciples,principle1.35LimitingtheanalysistotheECHRsystem,seeECtHR,TagayevaandOthersvRussia,AppNo26562/07andothers(13April2017)paras592–9(concludingthattheabsenceofasufficientlystrongregulatoryframework‘bearsarelevance’ontheissueofproportionality);GiulianiandGaggiovItaly,AppNo23458/02(24March2011)para209(affirmingthattheprotectionoftherighttolifenecessitates‘anappropriatelegalandadministrativeframeworkdefiningthelimitedcircumstancesinwhichlawenforcementofficialsmayuseforceandfirearms,inthelightoftherelevantinterna-tionalstandards’);MakaratzisvGreece(n24)paras56–9,ECHR2004-XI;McCannandOthersvTheUnitedKingdom,AppNo18984/91andothers(27September1995)para150.DiegoMauri-9781802207675DownloadedfromPubFactoryat05/28/202203:48:40AMviaDukeUniversityLawLibrary

8274Autonomousweaponssystemsandtheprotectionofthehumanpersonmattersof‘autonomy’andthedegreesof‘humancontrol’:thatis,itdoesnotseemthatfieldingAWSwillinitselfimpactnegativelyontheruleoflegality.3.2.1.2NecessityTheprominentprinciplegoverningtheuseofforceistheprincipleofneces-sity,whichrequiresthatlethalforceberesortedto‘only…whenstrictlyunavoidableinordertoprotectlife’.36Asinferablefromtheletterofarticle2ECHR,necessityhastobeinterpretedinthelightofastrictandcompellingtest,namely,as‘absolutenecessity’.Thisexpressionmarksthedifferencebetweenthenecessitytestundertherighttolifeandotherhumanrightsprovi-sions,suchastherighttoprivacyorthefreedomofexpression.37Theprincipleofnecessityisthusbuiltupontwofoundations:(1)theabsenceorunavailabilityofless-lethalmeans,makingtheresorttolethalforceanextremaratiocourseofaction;and(2)theexistenceofalegitimatepurposeforresortingtosuchforce.38Asforthefirst,law-enforcementofficialsarerequiredtoexhaustnon-violentmeans(suchaspersuasion,negotiationandmediation)beforeusinglethalforce.39Intheeventthatthesenon-violentmeansproveinadequateorineffective,precedencemustbegiventoless-lethalmeanscapableofneutralizingtheindividualwithouttakinghisorherlife.40Thisleadsinevitablytotheconclusionthatevenpotentiallyviolentsuspectsshouldbearrestedratherthankilled,whensuchaless-lethaloptionisviableandreasonableaccordingtothecircumstancesofthecase.41Temporalconsid-erationsmayplayadecisiveroleinassessingwhethertheuseoflethalforcesatisfiestherequirementof‘absolutenecessity’underthistenet.‘Tickingbomb’scenariosareillustrative.Whensplit-seconddecisionsmustbetaken36UNBasicPrinciples,principle9(emphasisadded).37GiulianiandGaggiovItaly(n35)para176(affirmingthat‘astricterandmorecompellingtestofnecessitymustbeemployedthanthatnormallyapplicablewhendeterminingStateactionis“necessaryinademocraticsociety”underparagraph2ofArticles8and11oftheConvention’).38UNBasicPrinciples,principle4.39Ibidpara7.Forapracticalapplicationofthisrule,seeECtHR,ShchiborshchandKuzminavRussia,AppNo5269/08(16January2014)para238(suggestingthatalawenforcementofficialshouldwaitforappropriateresourcestoarriveinlocobeforedecidingtoresorttolethalforce).40GC36para13.FocusingontheECtHR’scaselaw,seeFinogenovandOthersvRussia,AppNo18299/03andothers(20December2011)paras231–6(dealingwithgas);KhamzayevandOthersvRussia,AppNo1503/02(3May2011)para185(foracaseinvolvingfragmentationbombs);StewartvTheUnitedKingdom[dec],AppNo10044/82(10July1984)paras28–30(foracaseinvolvingrubberbullets).41SuarezdeGuerrerovColombia(n23)para137;IACtHR,NadegeDorzemaetalvDominicanRepublic,SeriesCNo251(24October2012)para85(iii);CodeofConductart3.DiegoMauri-9781802207675DownloadedfromPubFactoryat05/28/202203:48:40AMviaDukeUniversityLawLibrary

83Autonomousweaponssystemsunderinternationalhumanrightslaw75toneutralizeanindividualposingagraveandimminentrisktobystanders,less-lethalmeansmaybeunavailableandresortingtolethalforceturnsouttobetheonlyeffectivesolution.42Asforthelegitimatepurposerequirement,importancemustbeattachedtotheaimsthattheuseoflethalforceisintendedtoachieve.Theseaimsareattimeslistedintheapplicabletreatyprovision,asintheECHR;attimestheyarenot—aswiththeICCPR,wherereferenceissolelytotherequirementofnon-arbitrariness—resultinginanecessaryinterventionbytherelevantmonitoringbody.43TheUNBasicPrinciples,theCodeofConductandothersoft-lawinstrumentsallrequirealegitimateaimtobepresentateachandeveryuseoflethalforceagainstanindividual,whichincludesthepreventionofseriouscrimes,thelawfularrestofasuspectposinganimminentthreatorthepreventionoftheirescape.44InafuturescenarioinvolvingtheuseofAWS,letusimaginethatasuspectisbelievedtobeabouttodetonateabombinacrowdedarea,leavingnotimeforimmobilizingthem:useofpotentiallylethalforceturnsouttobeinevitable.Doesitmakeadifferencewhetherthedeliberationismadebythemachine,withouttheinterventionofthehumanoperator?ItmaybearguedthatifAWShavenocapacitytoreadtheintentionofsus-pects,theywouldbeunabletocomplywiththerequirementathand.Owingtotheirlackofunderstandinghumanintentioninreal-worldsituations,AWScouldnotgraduatetheamountofforcetobeappliedagainstatarget,whichwouldresultinusesofforcethatarenotmade‘strictlyunavoidableinordertoprotectlife’.Whilethiscontentionmakesperfectsense,onemustbecautiousnottojumptoconclusionstoohastily.Tobeginwith,itiseasytorebutthatifAWSaredevelopedthatcan‘read’suchintentionsatleasttothesamedegreeofreliabilityasahumanagentcan,thentheruleonnecessitywillberespected.Theabove-mentionedobjectiontoAWSisthusmerely‘technological’,thatis,destinedtoberebuttedonceappropriateadvancementsintechnologyaremade.Inaddition,requiringthatAWSbeableto‘readtheintentionofsuspects’doesnotautomaticallyimplythatAWSbeequippedwithhigherlevelsofcognition,42McCannvTheUnitedKingdom(n35)para74.43Thelistisconsideredasnumerusclausus:seeWSchabas,TheEuropeanConventiononHumanRights.ACommentary(OUP2015)147(arguingthatbecauseofsuchstrictapproach‘therewillinevitablybedisputesaboutwhetherthiswasthegenuineintentoftheauthorities’).Asimilardraftingwasproposed(albeitunsuc-cessfully)attheCommissiononHumanRightsduringthetravauxpréparatoiresoftheICCPR:seeCKBoyle,‘TheConceptofArbitraryDeprivationofLife’,inBGRamcharan(ed),TheRighttoLifeinInternationalLaw(MartinusNijhoff1985).44UNBasicPrinciples,principle9;CodeofConductart3;CommitteeofMinistersoftheCouncilofEurope,‘EuropeanCodeofPoliceEthics’(19September2001)para37(stipulatingthatlaw-enforcementofficialsmayresorttolethalforce‘onlytotheextentrequiredtoobtainalegitimateobjective’).DiegoMauri-9781802207675DownloadedfromPubFactoryat05/28/202203:48:40AMviaDukeUniversityLawLibrary

8476Autonomousweaponssystemsandtheprotectionofthehumanpersonor‘intelligence’.Forinstance,itmaybesufficientthatAWSbecapableofrequesting(andaccepting)surrenderbysuspectsand,iftheycontinuetoposeathreat,enoughtojustifyresortto(evenlethal)force,ofengagingthetarget.Inrespectofthelatterissue,itispossibletodrawananalogyfromtheparallelissueofsurrenderduringarmedconflict.Arguably,thefeasibilityofAWSrequestingsurrenderforthepurposesofIHLishotlydisputed.Forinstance,Sparrowclaimsthatrecognitionofatarget’sintentiontosurrenderisa‘hardproblem’fortworeasonsmainly:first,itrequiresperception,a‘notoriouslyhardtask’forcomputers;second,itrequiresunderstandingthesignificanceofcontext,whichinvolvesdistinguishinggenuinesurrenderfrom‘fake’surrender.45LetusimagineascenariowhereAWSarefieldedtoneutralizeasuspectbar-ricadedinacompound,armedandshootingatbystandersenteringhisorherlineofsight.Inorderfortheprincipleofnecessitytoberespected,AWSmustbecapableofapproachingthesuspect,orderingsurrenderand,ifheorshedoesnot,preventinghimorherfromposingfurtherthreats.Onecouldnothelpbutnoticethat,contrarytohumanagents,AWSdonotriskanypersonalinjuryandmightevenbeprogrammedtoallowthesuspecttoshootatitbeforeopeningfire,therebyraisingthebarforreleasingforcewellaboveself-defense.Whatismore,itmightnotbenecessarytohaverecoursetolethalforce,asless-lethalweaponsmaybesufficientforthepurposeofneutralizingthesuspect.Finally,anotherbeneficialeffectoffieldingAWSinhigh-risklaw-enforcementoper-ationsmaybetofacilitatenegotiations.Asthegoalsofalaw-enforcementoperationaremaintainingpublicorderandsecurity,andpreventinganddetect-ingcrime,ifresorttolethalforceisanextremaratiochoice,negotiationplaysakeyroleasitallowsforpreventionofthecommissionofacrimewithoutdeprivingthesuspectofhisorherlife.TheUNBasicPrinciplesstresstheimportanceofhavinglaw-enforcementofficialsproperlytrainedandeducatedinrelationtothis.46ProvidedthatfieldingAWSreducesexposureofhumanagentstorisk,thecostofengaginginnegotiationsislowered.WhatmattersforAWStobeusedinconformitywiththeruleofnecessityisthatinagivensituationtheyperform‘atleastaswellashumanwarfighters’.4745RSparrow,‘TwentySecondstoComply:AutonomousWeaponSystemsandtheRecognitionofSurrender’(2015)91Int’lLStud699,705–9(underscoringthenaturalsuperiorityofthehumanbrainininterpretingtheactionsofotherhumanbeingsandidentifyingtheirpossibleintentions,andclaimingthatforamachinetocomeclosetoreplicatingsuchbehavioris‘extremelychallenging’).46UNBasicPrinciples,principle20(usingtheterms‘peacefulsettlementofcon-flicts’and‘methodsofpersuasion,negotiationandmediation’).47Sparrow(n45)709(emphasisadded).ThisapproachisfollowedanddefendedbyRCArkin,‘TheCaseforEthicalAutonomyinUnmannedSystems’(2010)9JofMilitaryEthics332.DiegoMauri-9781802207675DownloadedfromPubFactoryat05/28/202203:48:40AMviaDukeUniversityLawLibrary

85Autonomousweaponssystemsunderinternationalhumanrightslaw77Ifaspecificstandardofconductislawfulwhenadoptedbyahumanagent,andifAWSaredeployedthatensurethesamelevelofperformanceofahumanagent,thenthedeprivationofliferesultingfromtheuseofAWSmaybecon-sideredinconformitywiththeruleofnecessity.3.2.1.3ProportionalityAsthecommentaryonarticle3oftheUNCodeofConductexplains,‘innocase’forcecanbeauthorized‘whichisdisproportionatetothelegitimateobjectivetobeachieved’.48Theprincipleofproportionalitycomesintoplayonlywhentheprincipleofnecessityhasbeenmet.ThecloseconnectionbetweennecessityandproportionalityhasbeendulyoutlinedbytheEuropeanCourtofHumanRights(ECtHR)elsewhere,wheretheprincipleiscastinthewrittenprovision.49TheUNBasicPrinciplesandinternationalcaselawestablishthatanunavoidableresorttoforceisprerequisiteforaproportionalityassessment:thatlawenforcementofficialsmaybegrantedrecoursetolethalforcedoesnotentail‘thatanydegreeofforcemaybeused’.50Providedthatresorttoforcecomplieswiththeruleofnecessity,theguidingstarinassessingwhethertheamountofforceemployedwasdisproportionateisthe‘legitimatepurpose’tobeachieved.Again,thisrequirestheforcetobecalibratedonacase-by-casebasis.Forinstance,onseveraloccasionshumanrightsbodieshavemaintainedthateventhoughlaw-enforcementagentsmayuseforcetopreventasuspectfromescaping,thisuseofforceisnotunconstrained,espe-ciallywhenthesuspectisnotposingathreattoanother’slife.51Movingfromthisnotionofproportionality,itisunsurprisingthatinternationalpracticehasevolvedtowardsafinalgroundofjustifyingdeliberatedeprivationsoflife48CodeofConductart3,commentary(b)(emphasisadded).49ECtHR,HandysidevTheUnitedKingdom,AppNo5493/72(7December1976)para58(‘[t]heUnitedKingdomwassaidtohaveviolatedtheprincipleofproportional-ity,inherentintheadjective“necessary”’).50UNBasicPrinciples,principle5;seealsoBoyle(n43)240.ForafocusontheICCPRsystem,seePMTaylor,ACommentaryontheInternationalCovenantonCivilandPoliticalRights.TheUNHumanRightsCommittee’sMonitoringofICCPRRights(CUP2020)138ff.51ECtHR,NachovaandOthersvBulgaria,AppsNo43577/98and43579/98(6July2005)para95(findingthatanescapingsuspectwhodoesnotposeathreattolifemaynotbeshot‘evenifafailuretouselethalforcemayresultintheopportunitytoarrestthefugitivebeinglost’);ECommHR[dec],FarrellvTheUnitedKingdom,AppNo9013/80(11December1982)(stressingthatthepurposeofarrestingcouldnotjustifyadeliberatedecisiontokillabsentanyimmediatethreattolife).DiegoMauri-9781802207675DownloadedfromPubFactoryat05/28/202203:48:40AMviaDukeUniversityLawLibrary

8678Autonomousweaponssystemsandtheprotectionofthehumanpersoncorrespondingtothe‘protectlife’principle.52ThisprinciplehasfinallyfoundrecognitionintheUNBasicPrinciples.53Asinarmedconflict,theprincipleofproportionalityrequiresabalancingcalculation:inIHRLterms,andwhenitcomestotherighttolife,thebalancetobestruckisbetweentheamountofforce(lethal,less-lethalornoforce),ontheonehand,and‘theseriousnessoftheoffenseandthelegitimateobjectivetobeachieved’,ontheother.54ThecaselawoftheECtHRcanoffertellingexamples:somewillbepickedupthatrelatetothethreegroundsforpermitteduseoflethalforcesetoutatarticle2(2)ECHR.Again,thisanalysisaimsatdemonstratingwhethertheprincipleofproportionalitybyitselfdemandshumandeliberationattheactofstrikingsuchbalance.Accordingto‘defencefromunlawfulviolence’(article2(2)(a)ECHR)itisimportanttoconsiderwhatfollows.Inacaseinvolvingalaw-enforcementoperationaimedatneutralizinganarmedmanwhowasabouttoshootatsomeagents,theCourtaffirmedthat‘theuseoflethalforceinthecircumstancesofthiscase,albeithighlyregrettable,wasnotdisproportionateanddidnotexceedwhatwasabsolutelynecessarytoavertwhatwashonestlyperceived…tobearealandimmediaterisktohislifeandthelivesofhiscolleagues’.55Mutatismutandis,itisclearthatAWSwouldnotactinself-defenseasitdoesnotriskits‘life’.However,itseemsunlikelythat,giventhecostsofthemachines,stateswillnotconsidertheopportunitytoallowAWStofireback,atleastwhentheforceusedagainstitislikelytodestroyit.EveniftheAWSispreventedfromfiringback,itwillacttodefendothers(forexample,ahostageorapotentialtarget)fromunlawfulviolence,sotheCourt’sassessmentremainsvalid.TheparametertheCourtresortedtoinordertoassessthelawfulnessofthelaw-enforcementofficial’sactionwas‘honestperception’or‘honestbelief’.Itisawell-establishedstandardinthesecases;56andismeanttoapplytohumanagents.AselaboratedintheArmanidaSilvacase(probablythemostimportant52HeynsReport(n18)para70(describingthisas‘theguidingstarofthepro-tectionoftherighttolife’).SeealsoO’Connell(n12)206(‘tosavelifeimmedi-ately’);Boyle(n43)241–2(commentingonearlydraftsofart6ICCPRand,namely,aFrench-Lebaneseproposalbasedonthestandardof‘dangertohumanlife’,andexplaininghowsuchpositionwasnotsupportedbymostdomesticsystemsbackthen).53UNBasicPrinciples,principle9.54CCPR,BaumgartenvGermany,CommNo960/2000(31July2003)para9.4.55ECtHR,BubbinsvTheUnitedKingdom,AppNo50196/99(17March2005)paras140,152(emphasisadded).56ThetestwasfirstsetinMcCannandOthers(n35)para200(‘[t]heuseofforcebyagentsoftheState…maybejustifiedunderthisprovisionwhereitisbasedonanhonestbeliefwhichisperceived,forgoodreasons,tobevalidatthetimebutwhichsub-sequentlyturnsouttobemistaken.Toholdotherwisewouldbetoimposeanunrealis-ticburdenontheStateanditslaw-enforcementpersonnelintheexecutionoftheirduty,DiegoMauri-9781802207675DownloadedfromPubFactoryat05/28/202203:48:40AMviaDukeUniversityLawLibrary

87Autonomousweaponssystemsunderinternationalhumanrightslaw79judgmentinthisfieldtodate),thisstandardissubjective,meaningthat‘thereasonablenessofabeliefinthenecessityoflethalforce’mustbedeterminedhavingregardto‘thepositionofthepersonwhousedlethalforce’.57ApplyingthesecategoriestoAWS,itmaybesufficienttodemonstratethat,giventherealsituationAWSareoperatedwithin,theirsystemworkedinawaythatwouldappearreasonabletohumanjudgmentexpost.Thisimpliesthattheremustbesomeformof‘reporting’,‘feedback-based’mechanismsthatcanbecheckedinordertoassesswhyandhowAWSactedastheydid.Thatis,theconductmustbecomprehensibleandunderstandabletorationalagents.Soifahumanagentcould:(1)accessthe‘blackbox’;(2)checkrecordeddata;(3)assessthesystem’sperformance;andthen(4)comparetheresultswithahypotheticalscenarioruledbythehumanagent,andifthisagentfoundthattheAWShadactedasthathumanagentwould,therequirementofproportion-alitywouldberespected.Thesameappliestothelegitimateaimsofeffectingalawfularrestorpreventingescape(article2(2)(b))aswellasquellingriotorinsurrection(article2(2)(c)).‘Reasonableness’againisthestandardwhichtheruleofproportionalityismeasuredagainst.58Theseremarksleadtoanimportantconclusion.Inlightofexistingcaselawonproportionality,whatisdecisiveforafindingofabreachiswhyandhowforceisusedinagivencontext—notwho(orwhat)dischargesit.Thatis,whetheraparticularuseofforceisinviolationoftherighttolifedependsonasituationaljudgmentaboutwhatthe‘applier’offorceperceived,whichseemscompatiblewiththat‘applier’beinganon-humanagent.Whileitisundeniablethatcurrentstandardsof‘reasonableness’,‘honestbelief’or‘honestperception’havebeenelaboratedwiththehumanagencyoftheapplieroflethalforceinmind,thereseemstobenorealobstacleinapplyingthesamestandardstoAWS,providedthattheyensureatleastacomparablelevelofperformance.Yetitismorethanunderstandablethatthisrequirementisverydemandingformachines,asitrequiresanadvancedsituation-awarenessthatmaytakeyears,ifnotdecades,tobedevelopedsuccessfully.ThisiswhymostcommentatorsagreethatAWSwillnotbeabletoperformashumansdo,atleastinthenearfuture.59perhapstothedetrimentoftheirlivesandthoseofothers’).SeealsoECtHR,YükselErdoğanandOthersvTurkey,AppNo57049/00(15February2007)para97.57ECtHR,ArmaniDaSilvavTheUnitedKingdom,AppNo5878/08(30March2016)paras244–8(emphasisadded)(caseconcerningthefatalshootingofaBraziliannationalmistakenlyidentifiedbytheLondonpoliceasasuicidebomber).58ECtHR,PerkandOthersvTurkey,AppNo50739/99(28March2006)para68.59Spagnolo(n4)48;Heyns,‘HumanRightsandtheUseofAutonomousWeaponsSystems’(n4)364–5.DiegoMauri-9781802207675DownloadedfromPubFactoryat05/28/202203:48:40AMviaDukeUniversityLawLibrary

8880AutonomousweaponssystemsandtheprotectionofthehumanpersonLegally,however,humandeliberationattheparticularactofengagingatargetisnotarequirementforcompliancewiththeruleofproportionality—neitherfortheruleofnecessitynorfortheruleoflegality.Itfollowsthatamodelof‘humancontrol’thatisabletounderstandthereasonsforparticularbehaviorsofAWSmaybesufficientforaffirmingthattheycanbedevelopedanddeployedinconformitywiththeaboverules.Thisissueisexploredfurtherbelow.603.2.2PositiveObligationsWhilehelpfulfromatheoreticalstandpoint,thedistinctionbetweennegativeandpositiveobligationisnotalwaysclear-cutinpractice.Mosttimesnegativeandpositiveobligationscannotbetreatedasseparated,non-communicatingsetsofdutiesincumbentonstates;insomecircumstancespositiveobligationsare‘inherent’inthenegativeobligations.61Instead,theycompleteeachotherinawaythatrenderstheprotectionoftherighttolife‘practicalandeffec-tive’.62Thisiswhyithasbeensaidthat‘[w]henlethalforceisusedwithina“policingoperation”bytheauthoritiesitisdifficulttoseparatetheState’sneg-ativeobligations…fromitspositiveobligations’.63Theobligationto‘respect’and‘ensure’therighttolifeimpliesacomplexsetofdutieswhichcanbedistinguishedintwomacro-areas:dutiesofprecautionregardingthechoiceofweaponsanddutytoinvestigateandprosecute.3.2.2.1ThedutyofprecautionregardingthechoiceofweaponsTheprecautionaryrulerequiresthepublicauthoritiestoplanlaw-enforcementoperationssoastominimizetheriskoflossoflife.Thisruleoperatesexante,thatis,beforetheactualemploymentoflethalforceandwithaviewtomin-imizingtheresortthereto.ThedutyofprecautioniswellsettledinIHRL.InthefamousMcCanncasetheECtHRelaboratedonthisruleforthefirsttime.64InassessingtheshootingofthreemembersoftheIrishResistanceArmy(suspectedofhavingonthemaremotecontroldevicetobeusedtodetonateabombnearby)bySpecialAirServicesoldiersinGibraltar,theCourtfound60Sees3.2.3.61ECtHR,VereinGegenTierfabrikenSchweiz(Vgt)vSwitzerland(No2),AppNo32772/02(30June2009)para79(‘inadditiontotheprimarilynegativeundertakingofaStatetoabstainfrominterferenceinConventionguarantees,“theremaybepositiveobligationsinherent”insuchguarantees’).62ECtHR,ÖneryildizvTurkey,AppNo48939/99(30November2004)para69.63FinogenovandOthersvRussia(n40)para208;IACtHR,VelásquezRodríguezvHonduras,SeriesCNo4(29July1988)paras166ff;GC36para21.64McCannvTheUnitedKingdom(n35).DiegoMauri-9781802207675DownloadedfromPubFactoryat05/28/202203:48:40AMviaDukeUniversityLawLibrary

89Autonomousweaponssystemsunderinternationalhumanrightslaw81aviolationoftherighttolifeinthattheUKauthoritieshadfailedtotake‘appropriatecareinthecontrolandorganization’oftheoperation.65Otherhumanrightsbodieshaverepeatedlyrequiredtherespectofprecautionaryruleswhenplanningtheuseofforceagainstindividuals.66TheUNBasicPrinciplesasawholerevolvearoundtheserules.67Asfortheirnature,thoserulescanbeconceivedofasobligationsofduediligence,totheextentthat,farfromimposinganobligationtosucceed,theyrequiretheiraddresseestodoeverythingfeasible(andreasonable)toattainaparticularresult.Thedutyofprecautionisstrictlyconnectedwiththedutytoeducateandtrainlaw-enforcementofficials,especiallythosetaskedwithtaking(poten-tially)lethalaction.UNBasicPrinciplestackletheissueof‘qualifications,trainingandcounselling’extensively,envisagingtheprevious‘completionofspecialtraining’intheuseoffirearms.68Duringthistraining,specialattentionmustbeprovidedto‘issuesofpoliceethicsandhumanrights’and‘alternativestotheuseofforceandfirearms’.69Humanrightsbodieshavedealtwithsitua-tionsinwhichlaw-enforcementofficialsactedinawaythatsuggestedalackofappropriatetraining.Forinstance,theECtHRaffirmedthatmilitarypolicemenmustbetrainedtoassesswhetherthereisanabsolutenecessitytousefirearms,‘notonlyonthebasisoftheletteroftherelevantregulations,butalsowithdueregardtothepre-eminenceofrespectforhumanlifeasafundamentalvalue’.70ThisdutycanbesaidtoapplytoAWSafortiori,astheirusewillrequireappropriateunderstandingofthesystemsandtheirfunctions:law-enforcementofficialsresponsiblefordecidingwhethertofieldAWSandformonitoringtheiroperationsneedtoreceivespecifictraining.ThedutyofprecautionalsocoversanaspectofparamountimportanceinthedebatearoundAWS,namely,thechoiceofweaponsandammunitionsbylaw-enforcementofficials.First,itisheldthatregulationofarmsandcautionintheemploymentthereofarethesignofa‘democraticsociety’.71Usingagivenweaponinsteadofanotherdoeshaveacross-cuttingimpactonpositiveandnegativeobligationsregardingtherighttolife.Intuitively,theuseofaweapon65Ibidpara212.66IACtHR,NadegeDorzemaetalvDominicanRepublic(n41)para87(linkingtheprincipleofprecautionwiththeprincipleofproportionality,andstressinghowtheformer‘isalsorelatedtotheplanningofpreventivemeasures,sinceitinvolvesanassessmentofthereasonablenessoftheuseofforce’).67UNBasicPrinciples,principle3.68Ibidprinciple19.69Ibidprinciple20.70NachovaandOthersvBulgaria(n51)para97(emphasisadded);seealsoMcCannvTheUnitedKingdom(n35)paras211–12(complainingaboutthe‘reflexaction’oflaw-enforcementagents,consideredastheresultofpoortraining).71McCannvTheUnitedKingdom(n35)para212.DiegoMauri-9781802207675DownloadedfromPubFactoryat05/28/202203:48:40AMviaDukeUniversityLawLibrary

9082Autonomousweaponssystemsandtheprotectionofthehumanpersonthatisnotappropriateinthecontextraisesissuesofnecessity,proportionalityandprecaution.TheUNBasicPrinciplesstipulatethatstateofficialsmustbeprovidedwithadequateequipment,inparticularalternativeweaponstofirearms,inordertoensureadifferentiateduseofforce.72TheCommissionofInquiryinSyria(establishedwithintheframeworkoftheHumanRightsCouncil)hasissuedreportscondemningtheuseofsnipers,improvisedexplo-sivedevices(IEDs),shellingandfragmentationmortarbombs,chemicalandthermobaricweapons,barrelbombs,missilesandclustermunitions.73TheInter-AmericanCommissionofHumanRights(IACommHR)andtheHumanRightsCommittee(CCPR)alsohavedealtwithparallelcasesinvolvingtheuseofextremelylethalweaponsbystateagents.74Inlightofpractice,thedutyofprecautionasregardsthechoiceofweaponscanbeconstructedasfollows.First,resorttoindiscriminateweaponsisgener-allyconsideredtobeincontrastwiththedutytotake‘allfeasibleprecautionswithaviewtoavoidingand,inanyevent,minimizing,incidentallossofcivil-ianlife’.75IntheIsayevacase,theECtHRheldthattheuseof‘heavycombatweapons’—namely,FAB-250andFAB-500explosivebombslaunchedfromRussianjets—onthevillageofKatyr-Yurt(Chechnya)wasinconsistentwiththedutytoplanandexecuteanoperation‘withtherequisitecareforthelivesofthecivilianpopulation’.76InthemorerecentTagayevacase,itconsideredgrenadelaunchers,tankcannonsandflame-throwers,concludingthateventhoughthedecisiontoresorttotheuseoflethalforcecouldbejustifiedinsomecircumstances(forexample,duringthestormingofaschoolwhereterroristsareholdinghundredsofhostages),theemploymentofindiscriminateweaponsperseleadstoaviolationoftherighttolife.77Second,theuseofnon-orless-lethalweaponsinsteadoflethalweaponsispreferableinsomescenarios.Forinstance,intheFinogenovcase,theresorttoanincapacitatinggastoter-72UNBasicPrinciples,principle2.73UNHumanRightsCouncil,‘ReportoftheIndependentInternationalCommissionofInquiryontheSyrianArabRepublic’(5February2015)A/HRC/28/69para13.74IACommHR,SantoDomingovColombia,Petition289/2002,ReportNo25/03(6March2003)(foracaseinvolvingtheuseofclustermunitions);CCPR,ConcludingobservationsoftheHumanRightsCommittee:Israel,CCPR/C/79/Add.93(18August1998)para17(urgingIsraeltoenforcestricterlimitationsontheuseofrubberbullets).75TagayevaandOthersvRussia(n35)para573.76IsayevavRussia,AppNo57950/00(24February2004)paras179–201(albeitrecognizingthattheoperationhadbeenplannedandexecutedinthepursuitofalegit-imateaim).TheuseofaerialbombinginpopulatedareashasalsobeencondemnedinBenzerandOthersvTurkey,AppNo23502/06(12November2013)paras89,184(claimingthatthebombingasoccurredwasnot‘acceptableinademocraticsociety’andinconsistentwithanycustomaryandtreatyruleapplicabletoarmedconflict).77TagayevaandOthersvRussia(n35)paras584–611.DiegoMauri-9781802207675DownloadedfromPubFactoryat05/28/202203:48:40AMviaDukeUniversityLawLibrary

91Autonomousweaponssystemsunderinternationalhumanrightslaw83minateahostage-takingsituationintheDubrovkatheaterinMoscowwasnotdeemedincontrastwiththeprincipleofproportionalityas,‘whiledangerous’,it‘wasnotsupposedtokill,incontrast,forexample,tobombsorairmissiles’and‘leftthehostagesahighchanceofsurvival’.78Casessuchasthisconfirmthegeneral‘gradual’approachtotheuseofweaponsbylaw-enforcementagents.79ApplyingtheforegoingtoAWS,thepositiveobligationenshrinedintheruleofprecautionimposesadutytofieldAWSthatarecapableof(1)engagingonlypermissibletargets(inkeepingwiththeprohibitiononindiscriminateweapons),and(2)applyingthecorrectamountofforceasrequiredbytheparticularsituation.Previousremarksonthenegativeobligationsstemmingfromtheprotectionoftherighttolifecomplementthisexanteduty.Onthispremise,thereseemstobelittleornothingfortheconclusionthatAWSwouldbeillegal:again,itisamatterofdevelopingAWSthatcanbeoperatedaccord-ingtothoserequirements.Forinstance,whileAWSwillpresumablycarryfirearms,theycouldbeequippedwithdifferentandno-orless-lethalammu-nitions.Itcannotbedeniedthatthekeyfeaturethatrendersthemsounprec-edentedisthattheweaponsystemitselfmakesadeterminationaroundwhentoresorttoforceagainstindividuals.However,claimingthatAWScannotliveuptostates’positivedutyofrefusingindiscriminateweaponsbecausetheywouldnotbeabletodirectappropriateforceagainstpermissibletargetsfallswithinatechnologicalobjectionthatcanbeeasilyrebutted:ifAWSaredevelopedthatcanmakethistypeofdeterminationasahumanwoulddo,thenthepositiveobligationwillberespected.Again,itismoreamatterofhowtheforceisappliedratherthanwhoorwhatappliesit.3.2.2.2ThedutytoinvestigateandprosecuteThedutytoinvestigatedeprivationsoflifeoccurredincontraventionoftherightathandandtoprosecute,whereappropriate,thoseresponsiblefortheseunlawfuldeprivationsconstitutesapositiveobligation—commonlyreferredtoasa‘procedural’obligation—thathasbeenmadetheobjectofextensivecaselaw.80Itinvolvestheobligationtoinvestigateandprosecutethesedeprivations78FinogenovandOthersvRussia(n40)para232.NotethatonthatoccasiontheCourtdecidednottoconsiderthegasasa‘lethal’or‘non-lethal’weapon,stressingbycontrastthatwhatmatteredwasthatthegas‘was,atbest,potentiallydangerousforanordinaryperson,andpotentiallyfatalforaweakenedperson’.79UNBasicPrinciples,principle2.80GC36para31;CCPR,SathasivamandSaraswathivSriLanka,CommNo1436/2005(8July2008)para6.4;AmirovvRussianFederation,CommNo1447/2006(2April2009)para11.2;OlmedovParaguay,CommNo1828/2008(22March2012)para7.3;ECtHR,ŠilihvSlovenia,AppNo71463/01(9April2009)para153;ErgivDiegoMauri-9781802207675DownloadedfromPubFactoryat05/28/202203:48:40AMviaDukeUniversityLawLibrary

9284Autonomousweaponssystemsandtheprotectionofthehumanpersonwhennecessary:itis‘notanobligationofresultbutofmeansonly’.81Thisdutyservesseveralaims,amongwhichare(1)ensuringthatthoseresponsiblearebroughttojustice,(2)promotingaccountabilityandpreventingimpunity,(3)avoidingdenialofjustice,and(4)eventuallydrawingnecessarylessonsforrevisingpracticesandpolicieswithaviewtoavoidingrepeatedviolations.82Humanrightsbodieshavesofardevelopedaconsiderablecaselawonthedutytoinvestigateandprosecute,contributingtoshapingitsboundariesandcontent.Itisheldthatinvestigationsmustbethorough,independent,accessibletothevictim’sfamily,prompt,capableofleadingtoadeterminationastowhethertheuseofforcewaspermissibleand,finally,allowingappropriatepublicscrutiny.Thatis,theymustbeeffectiveinbeingcapableofestablishingfactsandleadingtotheidentificationofthoseresponsible.83Inaddition,itshouldbehighlightedthatcriminalactionagainstthoseresponsibleisnotrequiredperse,exceptincasesofintentionaldeprivationoflife,inrespectofwhichothermeansofaccountability(suchascivilactionforredress)arepresumedinadequate.84Turkey,AppNo23818/94(28July1998)para82;McCannvTheUnitedKingdom(n35)paras157–64;IACtHR,CantoralHuamaníandGarciaSantaCruzvPeru,SeriesC167(10July2007)para131;PuebloBelloMassacrevColombia,SeriesC140(31January2006)para143;ACtHPR,ZimbabweNGOHumanRightsForumvZimbabwe,No245/02(25May2006)para153.SeealsoOHCHR,TheMinnesotaProtocolontheInvestigationofPotentiallyUnlawfulDeaths(2016):TheRevisedUnitedNationsManualontheEffectivePreventionandInvestigationofExtra-legal,ArbitraryandSummaryExecutions(Geneva2017).SeePLeach,RMurrayandCSandoval,‘TheDutytoInvestigateRighttoLifeViolationsacrossThreeRegionalSystems:HarmonisationorFragmentationofInternationalHumanRightsLaw?’,inCMBuckley,ADonaldandPLeach(eds),TowardsConvergenceinInternationalHumanRightsLawApproachesofRegionalandInternationalSystems(Brill2016).81SeeegŠilihvSlovenia(n80)para193.82GC36para31(referencesomitted).SeealsoECtHR,JasinskisvLatvia,AppNo45744/08(21December2010)para72(arguingthatthe‘essential’purposeofthedutytoinvestigateis‘tosecuretheeffectiveimplementationofthedomesticlawswhichprotecttherighttolifeand,inthosecasesinvolvingStateagentsorbodies,toensuretheiraccountabilityfordeathsoccurringundertheirresponsibility’).83FinogenovandOthersvRussia(n40)para269.TheECtHRhasattimesadopteddifferent-in-namebutequal-in-substancestandards:compareÖneryıldızvTurkey,AppNo48939/99(30November2004)para94(‘exemplarydiligence’)andOpuzvTurkey,AppNo33401/02(9June2009)paras131,137–49(‘duediligence’).ThestandardofduediligenceistheoneadoptedbytheIACtHR:seeLeachetal.(n80)33.84ŠilihvSlovenia(n80)para194;SuarezdeGuerrerovColombia(n23)para13.3(foracaseconcerninganarbitrarykillingperformedbyStateagents);CCPR,BautistadeArellanavColombia,CommNo563/1993(27October1995)para8.2(arguingthat‘purelydisciplinaryandadministrativeremediescannotbedeemedtoconstituteade-quateandeffectiveremedies…intheeventofparticularlyseriousviolationsofhumanDiegoMauri-9781802207675DownloadedfromPubFactoryat05/28/202203:48:40AMviaDukeUniversityLawLibrary

93Autonomousweaponssystemsunderinternationalhumanrightslaw85TheimpactthatAWSarelikelytohaveontheseproceduralobligationsisgreatlydebated.Militaryexpertsarguethattheywillbeequippedwithasystemrecordingandtrackingthevariousactionsundertakenwhenfielded,sothathumanoperatorswillnotonlyhaveaccesstoallgathereddata,butalsobeinapositiontoaudittheperformanceofAWSandunderstandthereasonswhytheytookparticularcoursesofaction.85ThiswouldmeanthatintheeventofAWSusingforceagainstanindividual,investigatorswouldbeabletodeterminewhetherthatforcewaspermissibleornotand,ifthelatter,whatwentwrong(forexample,amalfunctioningofthesystem).Whilethepositivedutyofprecautionappliesexantefacto(thatis,beforetheactualengagement),thedutytoinvestigateandprosecuteislocatedafterthereleaseofforce.RisksatthisstagecompriselackofunderstandabilityofhowtheAWSworked:inorderforthedutyathandtoberespected,itisnecessarythatthealgorithmsallowingpossiblelethalcoursesofactionaredesignedinawaytoensureproperunderstanding.Thismightnotbeaneasytask,especiallywhenself-anddeep-learningalgorithmsareinvolved.Inadditiontothebarrierposedbytheprotectionofproprietaryinterestsoftheownersofthealgorithms,thedecision-makingprocessitselfmaybedifficulttounderstandforhumans.Letusexpandthisfurther.Someauthorsarguethatto‘opentheblack-box’sothatoperatorscanchallengeparticulardecisionstakenbythemachineanddistributeresponsibilityacrosstheactorsinvolvedwouldbeanunrealisticundertaking,orevenahoax.86Others,onthecontrary,believethatappropriatesafeguardstoreduceopacityandsupporthumanoperatorsin‘interrogating’AWSexpostfactocanbedeveloped.Tothisend,itisworthrecallingthatsomeproposalshavebeenadvancedrecentlywithregardtotheneedfor‘verification’mechanismscapableofensuringadequateunderstandingofthefunctioningofAWS.Forinstance,a‘glassbox’approach—asopposedtothe‘black-box’realityofautonomoussystems—hasbeensponsoredbyvirtueofwhichautonomoussystemsoughttobedesignedsothattheyautomaticallyrecordandsecurelystorerelevantinformationregardingspecificattacks,andlaterprovidethisinformationinanunderstandableformat.87Forourreasoning,onconditionthatappropriatetechnologyisinplacethatallowsforproperrights,notablyintheeventofanallegedviolationoftherighttolife’).SeealsoSchabas(n43)137.85USDoDDefenseScienceBoard,‘SummerStudyonAutonomy’(June2016),13,34.86LEdwardsandMVeale,‘SlavetotheAlgorithm?Whya“RighttoanExplanation”isProbablyNottheRemedyYouareLookingFor’(2017)16DukeL&TechnologyR18.87MGubrudandJAltmann,‘ComplianceMeasuresforanAutonomousWeaponsConvention’,ICRACWorkingPaper2(May2013).DiegoMauri-9781802207675DownloadedfromPubFactoryat05/28/202203:48:40AMviaDukeUniversityLawLibrary

9486Autonomousweaponssystemsandtheprotectionofthehumanpersonunderstandingofhowamachinehasoperatedinagivenscenario,respectforthisdutydoesnotdependultimatelyonhavinghumandeliberationataparticu-laruseofforce.ItmaybearguedthatAWSremainunder‘humancontrol’inthattheiractionscanbeadequatelyexplainedexpostfacto.However,itcannotbedeniedthatAWSmaycompromisethedutytoholdaccountablethosewhoareresponsibleforunlawfuluseofforce.Asiswidelyknown,investigatingandprosecutingpotentiallyunlawfuldeprivationsoflifeserveatwofoldpurpose:redressingthevictim(s)andensuringgeneralaccountabilitybypreventingimmunity.88Tobeabletoconductappropriateinvestigationsmayturnouttobeamockery,iftheseinvestigationscouldnotendupwithreasonablechancesofconvictionforthoseallegedlyresponsibleforthedeprivationoflife.Also,theremaybedifficultiesinassigningcriminalresponsibilityforwrongdoingscommittedthroughAWS,especiallywhentheyperforminawaythatisneitherunderstandablenorexplicablebytheiropera-tors.IfthenatureofAWSweresuchthatitrendersimpossibletheimputationofatargetingdecisiontoanindividual(bethistheproducer,thestateagentresponsiblefortheacquisitionoftheweaponsystem,orthosewhodecidedtodeploythemandoversawtheiremployment),thiswouldresultinadefactoimpunity,relaxingthe(inevitablystrict)requirementsforpermissibleuseofforceandmakingtheprotectionoftherighttolifeultimatelyineffective.WhoistobeblamedforthemisdoingsofAWS?Toanswerthisquestion,itisnecessarytodelveintothecontentofthedutytoprosecute.Thedutytoprosecuteand,ifappropriate,topunishthosewhoarefoundtoberesponsiblefordeathsthatoccurredincontraventionoftherighttoliferequiresthatstatesenactanappropriatelegalframework—whichalsoincludes‘effectivecriminalprohibitions’89—or,astheECtHRputsit,‘aneffectiveindependentjudicialsystem…holdingaccountablethoseatfault’.90Importantly,adistinctionhastobedrawnbetweenintentionallossesoflife(irrespectiveofwhethertheallegedauthorisaprivateindividualorapublicagent)andnegligentlossesoflife.While,intheformer,criminal-lawsanctionshavetobeprovidedforinabstractoandimposedinconcreto,inthelatteritisacknowledgedthatdifferentformsofresponsibilitymaysufficeforredress-ingnegligentlossesoflife,providedthatensuingsanctions(forexample,ofciviloradministrativenature)proveeffectiveandproportionatetothegravityofthefacts.91Internationalhumanrightslawmonitoringbodieshave88GC36para27.89GC36para20.90FergecvCroatia,AppNo68516/14(9May2017)para32.91CGrabenwarter(ed),EuropeanConventiononHumanRights.Commentary(Verlag-Hart-Nomos2014)21;ECtHR,VovFrance,AppNo53924/00(8July2004)para90;GeneralCommentNo3(n33)para5;GC36paras20,26.DiegoMauri-9781802207675DownloadedfromPubFactoryat05/28/202203:48:40AMviaDukeUniversityLawLibrary

95Autonomousweaponssystemsunderinternationalhumanrightslaw87developedaconsiderablecaselawregardingnegligentlossesoflife,suchasinthemedicalfieldorhuman-madeandnaturalaccidents,includingmilitaryorpolicetraining.92Notethat,atleastinthosecaseswherethedeathoccursasaresultofnegligentusesofforcesbystateagents,compensatoryremediesofacivilnature(toredressvictims’nextofkin)areheldtobenotenough.Forinstance,inacaseoftheinvoluntaryexplosionofamilitary-laidbombinTurkeythatcausedthedeathoftheapplicant’sson,theECtHRheldthatcriminal-lawsanctionsarethesoleadequatemeansforrepressingnegligentconductsinhazardousdomains,whichcallforadequatemeasuresformini-mizingrisksforindividuals.93Thisassertionissupportedbyothermonitoringbodies’caselaw,andseemsabsolutelyconvincing:itsucceedsinensuringaneffectiveprotectionofhumanlifefromparticularlyhazardousthreatswhoseneutralizationcallsfortheadoptionofeffectivemeasuresbystates.WhileacomprehensiveanalysisofresponsibilitystemmingfromtheuseofAWSisconductedlater,94itisappropriatetoanticipatethosescenariosthataremorelikelytoraisechallengestothedutytoprosecuteviolationsoftherighttolife.IntentionalandnegligenttargetingthroughAWScallfortheopeningofofficialcriminalinvestigationsand,whereappropriate,thepunishmentofthoseresponsible.Tothisend,statesmustequipthemselveswithaneffectivelegalframeworkensuringtheinflictionofcriminal-lawsanctionsinthosecases.Itfollowsthatdomesticlegalsystemshavetobeadaptedinordertocriminalizethisconductwiththeappropriateformsofcriminalresponsibility(forexample,negligence)—asalreadyoccursforultra-hazardousactivities—andintandemwithdifferentformsofresponsibilityforotherpersonsinvolved(forexample,companies):whatmattersisthatstatesputinplacemechanismsallowingfora‘cleardistributionoflinesofresponsibilities’.95Itremainstobeinquiredwhethermisdoingsthatcannotbeattributedtoanymaliciousintentionorfaultonthepartofanyinvolvedindividualsshouldresultincriminalconvictions.Existingcaselawdoesnotrequirepunishmentatallcosts:theobligationtosetupeffectivemechanismsforaccountabilitycanbesatisfiedalsoifthelegalsystemaffordsvictimsaremedybeforecivilcourts,foreitherfaultorno-faultresponsibility.Systematically,thismakessense:imposingcriminalsanctionsforno-faultconductwouldamounttoobjectiveresponsibilityincriminallaw,whichstandsatoddswiththeprohibitiononstrictliabilityforcriminaloffences(aneedthathasbeguntogaintraction,92ECtHR,OrukvTurkey,AppNo33647/04(4February2014);LovyginyvUkraine,AppNo22323/08(23June2016).93OrukvTurkey(n92)para50.94SeeCh5,s.5.1.95TagayevaandOthersvRussia(n35)para570.DiegoMauri-9781802207675DownloadedfromPubFactoryat05/28/202203:48:40AMviaDukeUniversityLawLibrary

9688Autonomousweaponssystemsandtheprotectionofthehumanpersonbutinamorelimitedfashionthanindomesticlegalorders,alsoinIHRL).96Againstthisbackground,theexistenceandtheavailabilityofnon-criminalremedies(suchasthoseofacivilnature)wouldensureaccountabilityalsointhosecases.Summingup,theremarkssketchedsofarleadtotheconclusionthatabsenceofhumaninterventioninthestagesoftargetselectionandengagementdoesnotimplyaviolationofthedutytopunish,providedthat(1)itispossibletotraceexpostfactowhyandhowAWShaveperformedinacertainway,and(2)appropriateformsofresponsibilityareenvisagedfordeprivationsoflifecommittedthroughAWS.3.2.3Re-AppraisingtheNotionof‘Arbitrariness’:AWayForward?Mostformulationsoftherighttolife(andthesamegoesforotherhumanrights,forexample,therighttoliberty)arestructuredonthenotionofnon-arbitrari-ness:onlydeprivationsoflifethatare‘arbitrary’engagetheresponsibilityofstates.Forthisreasonourpreviousanalysistooknon-arbitrarinessasanumbrellarequirementfromwhichallnegativeandpositiveobligationsstem.Itisnowtimetotesttherequirementofnon-arbitrarinessasapossibletoolforregulatingAWSunderIHRLasawhole:ouranalysis,whichweconductvis-à-vistherighttolife,canbeexpandedtootherrightsprotectedbyIHRLandbasedonthenotionof‘non-arbitrariness’.Movingfromthisassumption,someauthorshavearguedthatwithouthumancontrolanyuseofforcewouldbearbitraryassuchandthatthereforetheuseofAWSrunsagainsttherighttolife:‘non-humandecision-makingregardingtheuseoflethalforceis…inherentlyarbitrary,andallresultingdeathsarearbitrarydeprivationsoflife’.97ThisargumentdeservescreditforprovidingafurthergroundforrejectingAWS:methodologically,itisforthisreasonthattheanalysisofsucharequirementhasbeenleftinlastplace.ThissubsectioninquireswhetherthislineofargumentissufficientlycorroboratedbyexistingIHRL:tothisend,itwilltakeintoaccountcurrentinterpretationsofIHRLprovisionsandassessAWSagainstthem.96ECtHR,GIEMsrlvItaly,AppsNos1826/06andothers(28June2018)paras242–6.SeealsoGC36para20(stressingthatcriminalsanctionsattachedtocrimesinvolvingintentionalornegligenthomicidemustremain‘compatiblewithallthepro-visionsoftheCovenant’).97HeynsReport(n18)para90(emphasisadded);PAsaro,‘OnBanningAutonomousWeaponSystems:HumanRights,Automation,andtheDehumanizationofLethalDecision-Making’(2012)94IRRC687,700(‘[t]hedecisiontokillahumancanonlybelegitimateifitisnon-arbitrary,andthereisnowaytoguaranteethattheuseofforceisnotarbitrarywithouthumancontrol,supervision,andresponsibility’).DiegoMauri-9781802207675DownloadedfromPubFactoryat05/28/202203:48:40AMviaDukeUniversityLawLibrary

97Autonomousweaponssystemsunderinternationalhumanrightslaw893.2.3.1‘Arbitrariness’incurrentinterpretationsoftherighttolifeInordertoascertainthedegreetowhich‘humancontrol’mustberetainedforcriticalfunctionsofaweapon,themeaningof‘arbitrariness’forthepur-posesoftheprotectionoftherighttolifehastobeillustrated.Theconceptofnon-arbitrarinessisgenerallyassociatedwiththerequirementoflegality:anyuseofforcethatisnotallowedby(domestic)lawfallsautomaticallyintothecategoryof‘arbitrariness’.98However,theoppositeisnottrue:evenwhenthereisalegalbasisforusinglethalforceagainstanindividual,thatusemaystillbe‘arbitrary’.Thenotionof‘arbitrary’coversnotonlydeprivationsoflifethatoccuragainstthelaw,butalsounjustly,inappropriatelyorunpredictably.Themeaningoftheword‘arbitrary’or‘arbitrarily’wascontestedduringthetravauxpréparatoiresofarticle6ICCPR.99Itwasadisputethathadbegunyearsbefore,atthetimeofthedraftingoftheUDHR,100wheretwoopposinginterpretationswerepromoted:ontheonehand,‘arbitrary’wasconsideredasasynonymfor‘notinconformitywithorprovidedforbylaw’;ontheotherhand,itwasconsideredmoreakinto‘unjust’notwithstandingconformitywithdomesticlaw.101TheprovisionontherighttolifecontainedintheICCPR—discussedatthefifth,sixthandeighthsessionsoftheUNCommissiononHumanRights—102finallyincludedthereferencetothenotionofarbitrarinesstosolvedisagreementsamongdelegatesabouthowbesttodraftthetext,espe-ciallyasfarascapitalpunishmentwasinvolved.10398SeeCHeyns,‘AutonomousWeaponsinArmedConflictandtheRighttoaDignifiedLife:AnAfricanPerspective’(2017)33SouthAfricanJOnHumanRights46,52(affirmingthat‘an“arbitrary”deprivationoflifecanbeseenastakingoflifeinawaythatisinviolationofinternationallaw’).99Boyle(n43)224–6.100Whereitappearedinfourclauses,saveforinart3protectingtherighttolife.Seeibid224.101Ibid225.SeemoreextensivelyPHassan,‘TheWord“Arbitrary”asUsedintheUniversalDeclarationofHumanRights:“Illegal”or“Unjust”?’(1969)10HarvInt’lLJ225(foracomprehensivestudyoftheterminseveralprovisionsoftheDeclarationandtheICCPR).102SeeE/CN.4,SR.90,91,93,94,97,98,101,135,139,140,144,149,150,152,153,199,309,310,311.103Boyle(n43)227–33.Theauthorexplainshowsomedelegatesweremorefavorabletoa‘generallimitationslanguage’avoidingtofixgroundsofexceptions(forinstance,theUSandUSSR),whileotherspushedfora‘detailedlisting’approach.Eventuallythe‘controversial’sentencereferringtothenotionofarbitrarinesswasadopted‘verymuchasalastresort’.SeealsoDNsereko,‘ArbitraryDeprivationofLife:ControlsonPermissibleDeprivations’,inBGRamcharan(ed),TheRighttoLifeinInternationalLaw(MartinusNijhoff1985),247–8(underscoringthatthelanguageeventuallyadoptedbyart6(1)ICCPRhas‘theadvantageofbeinggeneralandcapableofcoveringthedifferentlegalsystemsoftheworld’).DiegoMauri-9781802207675DownloadedfromPubFactoryat05/28/202203:48:40AMviaDukeUniversityLawLibrary

9890AutonomousweaponssystemsandtheprotectionofthehumanpersonSubsequentcaselawshowsthatthenotionof‘arbitrariness’hasbeenemployedtocomplementthatof‘legality’.Deprivationsoflifehavebeendeclaredarbitrarynotonlywhencontrarytodomesticlaw,butalsowheninconsistentwiththerelevanthumanrightsprovision.104Apparently,‘arbi-trariness’wasintendedasacomprehensivenotionaimedtoinclude(withoutspecifying)diversegroundsofexceptiontotheprohibitionofdeprivationoflife—suchasthoselistedintheprovisionofarticle2(2)ECHR.105Ithasbeennoted,forexample,thattheIACtHR’scaselawreferstoanotionof‘arbitrar-iness’goingbeyondtheuseofforcethat‘doesnotconformtotheformallaw,butalsothatwhichisunjust’,where‘unjust’isframedintheprinciplesofnecessityandproportionalityasexplainedabove.106GeneralCommentNo36(2018)onarticle6ICCPR(hereafterGC36)deservesmentioninthatitoffersathoroughrecognitionofwhat‘arbitrari-ness’currentlymeans.Allegedlyitencompasses:(1)inappropriateness;(2)injustice;(3)lackofpredictability;(4)dueprocessoflaw;(5)necessity;(6)proportionality;and(7)reasonableness.107Recently,ithasbeendeclinedinrelationto‘discrimination’andeventhroughthelensofa‘gender-sensitiveapproach’.108Thisisinlinewithscholarship,109whichhasinterpretedthenotionof‘arbitrariness’ascoveringpracticesuchasthedeathpenaltyinflictedonadiscriminatorybasis,110summaryandextrajudicialexecutions,111enforced104CCPR,CamargoGuerrerovColombia,CommNo45/1979(31March1982).SeeBoyle(n43)236(concludingthat‘itwasinsufficientforconformitywiththeCovenanttoestablishthatthedeprivationwasjustifiedintheState’slaw’,thusshowinghowthenotionof‘arbitrariness’islargerinscopethanthatof‘legality’).SeealsomorerecentlyGeneralCommentNo3para12(‘[a]deprivationoflifeisarbitraryifitisimpermissibleunderinternationallaw,orundermoreprotectivedomesticlawprovisions’).105MPedrazzi,‘TheProtectionoftheRighttoLifeinLaw-EnforcementOperations’,inBCortese(ed),StudiinonorediLauraPicchioForlati(Giappichelli2014),109ff.106IACtHR,BarriosFamilyvVenezuela,SeriesC237(24November2011)para49(statingtheuseofforcebyStateagentsmustbeexceptionaland‘limitedbytheprinci-plesofproportionality,necessityandhumanity’).107GC36para18.108‘ReportoftheSpecialRapporteuronextrajudicial,summaryorarbitraryexe-cutionsonagender-sensitiveapproachtoarbitrarykillings,AgnesCallamard’(6June2017)A/HRC/35/23.109Nsereko(n103)257ff.110GC36para16(‘countrieswhichhavenotabolishedthedeathpenaltyandthathavenotratifiedtheSecondOptionalProtocolcanonlyapplythedeathpenaltyinanon-arbitrarymanner,withregardtothemostseriouscrimesandsubjecttoanumberofstrictconditions’).111Alsowhencommittedbyprivateindividualsandpublicauthoritieshavenotade-quatelyprevented,investigatedorprosecutedtheresponsible(thusviolatingtheirpos-itiveobligations).SeeforinstanceIACtHR,GonzálezetalvMexico,SeriesC205(16DiegoMauri-9781802207675DownloadedfromPubFactoryat05/28/202203:48:40AMviaDukeUniversityLawLibrary

99Autonomousweaponssystemsunderinternationalhumanrightslaw91orinvoluntarydisappearances,112lawenforcementabusesofpower,113andmilitaryexcesses,114oftenwithrespectnotonlytotherighttolifebutalsototherighttoliberty.AccordingtoACommHPR’sGeneralCommentNo3totheACHPR,thenotionof‘arbitrariness’appliesto‘[a]nydeprivationofliferesultingfromaviolationoftheproceduralorsubstantivesafeguards’oftherelevanthumanrightstreaty:115nihilpluris.Insummary,whatemergesfromourbriefoverviewoftherequirementofnon-arbitrarinessisthatitiscommonlyemployedtocondensedifferenthypothesesofcontrarinesswithIHRLintoonegenerallabel.Anautonomousmeaningofthenotionseemsuneasytograspinlexlata.3.2.3.2AredeprivationsoflifeinflictedthroughAWS‘arbitrary’perse?Inlightoftheforegoing,thecontentionaccordingtowhichdeathinflictedthroughAWSwouldbe‘arbitrary’perseandthereforeincontraventionoftherighttolifeisdelegeferendaatbest.Instead,whatcanbeinferredfromexist-inglawisthat,forinstance,anunreasonabledeprivationoflifeisprohibitedinsofarasarbitrary.Whenthe‘applier’oflethalforceisamachine—thatis,anon-humanagent—itisarguablethataslongasitispossibletoverifyandexplainhowandwhyAWSactedastheydid,thedeprivationoflifecouldnotbedeemedarbitrary.Countinguponthenexusbetween‘non-arbitrariness’and‘reasonableness’,theissueoftheimpactofAWSontherighttolifemayberephrasedasfollows:ifnoscrutinyispossibleonhowandwhyAWSwereconductedinacertainway(forexample,shootingasuspectallegedlyabouttodetonateabombinapublicplaceàlaArmanidaSilva),thenahumanagent(forexample,theoperator,thepublicprosecutor,thejudgeorthecitizen)willbegivennoexplanationbythesystem.Thismightbethecase,forexample,whenself-learningalgorithmsareinvolved,asthedecision-makingprocessmaybeunintelligiblealsotodatacontrollers.ThissituationwouldsurelyhaveanNovember2009);NadegeDorzemaetalvDominicanRepublic(n41)para97;GeneralCommentNo3(n33)paras38–9.112ECtHR,VarnavaandOthersvTurkey,AppNo16064/90andothers(18September2009)para148.113CCPR,VanAlphenvTheNetherlands,CommNo305/1988(23July1990)para5.8;Gorji-DinkavCameroon,CommNo1134/2002(17March2005)para5.1;ECtHR,XvTheUnitedKingdom,AppNo8416/78(5November1981)paras42–3(foracaseconcerningan‘arbitrary’deprivationoflibertyunderart5ECHR).114IACommHR,HugoBustíosSaavedravPeru,Case10.548ReportNo38/97(16October1997)paras58–61.SeemutatismutandisECtHR,HassanvTheUnitedKingdom,AppNo29750/09(16September2014)para110.115GeneralCommentNo3(n33)para12.DiegoMauri-9781802207675DownloadedfromPubFactoryat05/28/202203:48:40AMviaDukeUniversityLawLibrary

10092Autonomousweaponssystemsandtheprotectionofthehumanpersonimpact,forinstance,onthepositiveobligationtoconductinvestigationsintoan(allegedly)unlawfuldeprivationoflifeaswell.Itisthereforepossibletointerprettherequirementof‘non-arbitrariness’asthateachdecisiontouse(potentiallylethal)forceagainstanindividual(thatis,thealgorithmicprocessleadingtosuchoutcome)mustbeexplainableandunderstandable.Theneedforanunderstandableandexplainablealgorithmicdecision-makingprocesshasinspiredsomelegalscholarshipinthefieldofdataprotection.116Theindividual’srighttounderstandthefunctioningandtheimpactofalgorithmsconcerninghimorherhasbeenrecognizedinseveralprovisionsregardingdataprotectionadoptedintheframeworksoftheCouncilofEurope(ModernizedConventionNo108)andtheEuropeanUnion(GeneralDataProtectionRegulation,GDPR),inbothcasesinbindingterms.117Inparticular,therighttoknowthereasonsthatunderliean‘automated’decisionderivesfromasetofseveralrightssuchastherighttoreceiveexanteinformationfromdatacontrollersandtherighttoaccesstoinformationexpost,thatis,afterthedecision-makingprocesshasbeenundertakenorconcluded.118Anotherkeyprovisionconcerningdataprotectionistherightnottobesubjecttoadecisionsignificantlyaffectinganindividualbasedsolelyonanautomatedprocessingofhisorherdata.119Somescholarshaveproposedasystemicinterpretationofthesetworightssothattheyestablisha‘righttolegibility’:thatis,itisarguedthattransparencyandcomprehensibilityrequirethattherightnottobesubjectto‘automated’decision-makingproscribealsonominalhumaninterventionandthattherightofaccessincludesabroaderrighttoknowtherationalesbehindspecificdecisionsconcerningthedatasubject.120Themainrationaleoftheenvisaged‘righttolegibility’istoensureamoreeffectiveprotectionofthe116ADeeks,‘TheJudicialDemandforExplainableArtificialIntelligence’(2019)119ColumbiaLR1829(referringtotheneedfor‘explainableAI(xAI)’incourtrooms).117SeeModernizedConventionfortheProtectionofIndividualswithRegardtotheProcessingofPersonalData(adopted18May2018),CM/Inf(2018)15-final(CETSNo108)(ModernizedConventionNo108)art9;ExplanatoryReporttotheProtocolamendingtheConventionfortheProtectionofIndividualswithregardtoAutomaticProcessingofPersonalData,CM(2018)2-addfinal(CETSNo223)(ExplanatoryReport),paras71–83;Regulation(EU)2016/679oftheEuropeanParliamentandoftheCouncilof27April2016ontheprotectionofnaturalpersonswithregardtothepro-cessingofpersonaldataandonthefreemovementofsuchdata,andrepealingDirective95/46/EC(GeneralDataProtectionRegulation)OJ2016L119/1(GDPR)art2(2).118ModernizedConventionNo108art9(1);GDPRart15(1).119ModernizedConventionNo108art9(1)(a);GDPRart22(1).120GMalgieriandGComandè,‘WhyaRighttoLegibilityofAutomatedDecision-MakingExistsintheGeneralDataProtectionRegulation’(2017)7Int’lDataPrivacyL243.ContraseeSWachter,BMittelstadtandLFloridi,‘WhyaRighttoExplanationofAutomatedDecision-MakingDoesNotExistintheGeneralDataProtectionRegulation’(2017)7Int’lDataPrivacyL76,92(contendingforinstanceDiegoMauri-9781802207675DownloadedfromPubFactoryat05/28/202203:48:40AMviaDukeUniversityLawLibrary

101Autonomousweaponssystemsunderinternationalhumanrightslaw93rightsofindividualsaffectedby‘automated’decision-makinginfieldswheretheirvulnerabilitycouldbeexploitedmore(forinstance,marketingmanipula-tionsorunfaircommercialpractices).Althoughacomparisonwithdataprotectioninstrumentsmaybeappealingandeffectiveincapturingcommonalities,onemustbecautiousandavoiddrawingfar-fetchedconclusions.Inlargepart,negativeandpositiveobliga-tionsstemmingfromtherighttolifealreadyencompasstherequirementsaspertherighttoprivacy.First,exanteknowledgeofhowalgorithmsinvolvedintheprocessofusingforceagainstindividualsappearstosatisfytherequire-mentof‘non-arbitrariness’quaestablishmentby(domesticandinternational)law.Ithasbeenarguedthattheprotectionoftherighttolifenecessitates‘anappropriatelegalandadministrativeframeworkdefiningthelimitedcircum-stancesinwhichlawenforcementofficialsmayuseforceandfirearms,inthelightoftherelevantinternationalstandards’.121Ifthisregulatoryframeworkisobscure,orfailstoprovideindividualswithunderstandableindicationsabouttheconditionsinwhichAWSresorttolethalforceinlaw-enforcementsituations(theabove-mentioned‘limitedcircumstances’),therequirementoflegalitywillhardlybemet.Second,expostexplanationabouttheprocessthathasledtoaspecificautomateddecisioncomestotheforefromtheperspectiveofthedutytoconductinvestigationsinto(allegedly)unlawfuldeprivationsoflife.Thoseinvestigationshavetobecapableofleadingtoadeterminationofwhethertheforceusedwasjustifiedintheparticularcircumstancesofthecase.122Itisthereforeofparamountimportancethatpublicauthoritiesprovideanintelligibleaccountofhowanautomated(orautonomous)processhasworked:afailureto‘explainhow’—reddererationem—mayleadtoabreachoftherighttolifeunderitsprocedurallimb.123Itistemptingtoinferastrongerargumentfromthesetofrightsanddutiesenshrinedindataprotectioninstruments.Iftheindividualenjoysarightnottobesubjecttoasolelyautomateddecision,thenAWSwouldbeprohibited,atleastwhenemployedinsplit-secondscenariosthatdonotallowforhumanintervention.Oncloserinspection,however,noteventhiscircumstanceseemsthat‘thephrase“solely”suggestsevensomenominalhumaninvolvementmaybesufficient’).121GiulianiandGaggiovItaly(n35)para209.122IsayevavRussia(n76)paras221–3(foracasewheretheineffectivenessoftheinvestigationwaspredicatedinthatithadmade‘fewattemptstofindanexplanationfor…seriousandcredibleallegations’,thusplacingtheneedforanexplanationabouttheuseofforceatthecenteroftherighttolifeunderitsproceduraltenet).123ECtHR,KhodorkovskiyandLebedevvRussia,AppsNos11082/06and13772/05(25July2013)para848(discussingthealgorithmicmethodusedfordistributingcon-victedindividualsamongprisons).DiegoMauri-9781802207675DownloadedfromPubFactoryat05/28/202203:48:40AMviaDukeUniversityLawLibrary

10294Autonomousweaponssystemsandtheprotectionofthehumanpersondecisive.First,thefactthatahumanoperatordoesnotmaterially‘pullthetrigger’doesnotimplythataparticularcourseof(lethal)actionhasoccurredbeyondhumancontrol.Itcanbeargued,assupportersofAWSdo,thatthisactionremainsunderhumancontrolprovidedthathumanoperatorscanunder-standandexplainit,frombothanexanteandexpostperspective.Inkeepingwiththemethodologicalpremisesofthiswork,thedegreeofhumancontrolthathastobeensuredinorderfortherequirementofnon-arbitrarinesstobeabidedbyissuchastoenablethis‘legibility’oftheweaponsystem.Second,onemustnotlosesightoftheinherentdifferencesbetweenlaw-enforcementoperationsandotheractivitiestowhichtherightnottobesubjecttoasolelyautomateddecisionapplies.Therightinitselfissubjecttoexceptions,namely,whenthedecisionisauthorizedbylawandadequatesafeguardsoftheindivid-ual’srightsarelaiddown.124AssumingthattheemploymentofAWSwillberegulatedbylaw(afortioriasproscribedbythepreviouslymentionedruleoflegality)andwillensurethatthespecificconditionsofthe(potential)targetaretakenintodueconsideration,theexceptioncanoperate.Insummary,therequirementof‘non-arbitrariness’isanapttestbenchforAWS.Accordingtoexistinginterpretationsofthenotion,aparticularuseofforceisconsideredtobearbitrarywhennojustificationisprovidedforit,thatis,‘arbitrariness’includes—butitisnotlimitedto—caseswhereforceisusedincontraventionofdomesticlaw,butalsocaseswherehumanrightsstandardsarenotsatisfied.Ifhumaninterventionincriticalfunctionsofaweaponisabsent,thisdoesnotimplythatthesubsequentuseofforceisarbitrary,onconditionthathumanoperatorscanunderstandandexplainthebehaviorofAWS—thatis,thismaysufficeas‘humancontrol’.ThebarfordevelopingAWSissetrelativelyhigh.However,itfollowsthatuseofforcedeliberatedabsenthumaninterventionwouldnotbearbitraryperse.Thatis,eventhroughthelensofnon-arbitrariness,whatmattersishowforceisapplied,notwhoorwhatappliesit.Arightnottobesubjecttosolelyautomateddecisions,parallelingthatwhichhasbeenelaboratedinthefieldofdataprotection,couldhardlybesaidtoexistinIHRL.3.3RULESONTHEUSEOFFORCEAGAINSTTHEHUMANPERSON:THERIGHTNOTTOBESUBJECTTOTORTURE,INHUMANORDEGRADINGTREATMENTAutonomousweaponssystemsarelikely,first,toimpactontherighttolife.However,otherrightswillbeimpingedbythewayAWSwilloperate:first124ModernizedConventionNo108art9(2).DiegoMauri-9781802207675DownloadedfromPubFactoryat05/28/202203:48:40AMviaDukeUniversityLawLibrary

103Autonomousweaponssystemsunderinternationalhumanrightslaw95andforemost,incasesofnon-lethalforce,istherightnottobesubjecttotorture,inhumanordegradingtreatment.Therecanbelittledoubtaboutthefundamentalcharacterofthisrule,ifnotbyvirtueofitscontinuedaffirmationinmostIHRLinstrumentsandrelevantscholarship.125Policeoperationsareasensitivedomainforthisprohibition:episodesofpolicebrutalityduringmanifestationsorinthecontextoflaw-enforcementoperations,aswellastheintentionalrecoursetointerrogationtechniquesinvolvingill-treatmentandpsycho-physicalharmarepartofourdailynews.ThescenariounderscrutinyinvolvestheuseofAWSinpolicingoperationsassubstitutesforhumanagents,typicallyforneutralizingsuspects(forexample,whileescaping,duringdisturbancesorriots,orinhostage-takingsituations).AscenarioinwhichAWSareusedasinstrumentsofpurposefullyinflictingtortureonsuspects—inadditiontobeingmorallyabhorrent—issciencefiction,andisnotconsideredhere.Similartotherighttolife,therightnottobesubjecttotorture,inhumanordegradingtreatmentisnon-derogableandabsoluteincharacter.126Itiswidelyrecognizedascustomaryinnature,andhasattainedthestatusofjuscogensnorm.Itfollowsthatonceaparticularconductqualifiesas‘torture’,‘inhumantreatment’or‘degradingtreatment’—inthemselvesthreedistinctnotions,referringtothegravityandintensityoftheill-treatment—therecanbeno‘exceptionsorjustifyingfactors[or]abalanceofinterests,regardlessofthebehaviourofthepersonconcerned’.127Whenitcomestopolicingoperations,non-lethalusesofforcebylaw-enforcementagentsmayconstitute‘cruel,inhumanordegradingtreatment’.128Thatis,inorderforusesofforcetobeconsistentwiththeprohibition,theymustplacethemselvesbelowaspecific125UDHR,art5;ICCPR,art7;ECHR,art3;ACHR,art5;ACHPR,art5.Theprohi-bitionagainsttorturehasmadetheobjectofanadhocinstrumentattheuniversallevel:ConventionAgainstTortureandOtherCruel,InhumanorDegradingTreatmentorPunishment(adopted10December1984,enteredintoforce26June1987)1465UNTS85(CAT).126Taylor(n50)175.127ECtHR,GäfgenvGermany,AppNo22978/05(1June2010)para107.Formoreonthedistinctionsbetweenthethreecategories,seeIGGaranina,‘ThePoliceandtheProhibitionofTortureandInhumanorDegradingTreatmentorPunishment’,inRAlleweldtandGFickenscher,ThePoliceandInternationalHumanRightsLaw(Springer2016);DWeissbrodtandCHeilman,‘DefiningTortureandCruel,Inhuman,andDegradingTreatment’(2011)29L&Inequality343(clarifyingthatdifferencesinthelayersofill-treatmentdependontheactualtreatmentthevictimisexposedto,andmakingtheexampleoftheUSpost-9/11practiceofinterrogatories).128CodeofConductart5(‘nolawenforcementofficialmayinflict,instigateortol-erateanyactoftortureorothercruel,inhumanordegradingtreatmentorpunishment’).DiegoMauri-9781802207675DownloadedfromPubFactoryat05/28/202203:48:40AMviaDukeUniversityLawLibrary

10496Autonomousweaponssystemsandtheprotectionofthehumanpersonthreshold,therebywithoutattaininga‘minimumlevelofseverity’.129Toevaluatewhetherusesofforcereachthatthreshold,atestof‘justification’hasbeenelaboratedbymonitoringbodiesoftherelevantIHRLinstruments.Thistesthastobehandledcarefully,asitmaygeneratetheimpressionthatalogicofbalancingissubtlymakingitswayintotheprotectionofsuchfundamentalright.130First,aparticularuseofforcemaybejustifiedinasmuchasmadenecessaryingivencircumstances.Thisapplies,forinstance,totechniquesthataimtoimmobilizehighlydangerousprisonersduringtransportation.InacasebeforetheCCPR,theexistenceofarealriskthatsuspectsescaped,andthatstateauthoritieshadtakenallthenecessarymeasurestominimizeembarrassmentduringthephysicalsearch,wasdeemedsufficienttoconcludethatthetreat-mentinquestionwasjustifiedandthereforeitdidnotreachtheminimumthresholdofgravitytoqualifyasdegradingtreatment.131Secondly,justifica-tionoftendependsonanassessmentintermsof‘proportionality’.Forcethatisdisproportionatevis-à-vistheparticularcircumstancesofacasemayqualifyasdegradingtreatment.132Ifthisforceresultedinthelossoflife,therelevantprovisionwouldbetherighttolifeasillustratedabove:itiseasytoseethatthetworegimesarestrictlyconnectedinpractice.Expandingonthispoint,caselawhasthrivedonthenotionofthe‘excessive-ness’offorcetobeusedinpolicingoperations.AccordingtotheECtHR,‘anyuseofphysicalforcewhichhasnotbeenmadestrictlynecessaryby[one’s]ownconductdiminisheshumandignityandisinprincipleaninfringementof[article3ECHR]’.133Leavingthereferencetohumandignityuntillater,thefindingisimportantinthatitstressesthatonlyexcessiveusesofforcecontrastwiththeprohibitionunderscrutiny.Tothisend,domesticlawmustregulatetheconditionstowhichforcecanbeusedthrough‘dangerous…equipment’(suchastear-gasgrenades,morelikelytogeneratetheriskofcausingseriousinjury),inordertoestablisha‘systemofadequateandeffectivesafeguardsagainstarbitraryaction,abuseofforceandavoidableaccidents’.134Highstand-ardsapply,forinstance,incasesofpeacefulprotestsandnon-violentbehavior:129ECtHR,IrelandvTheUnitedKingdom,AppNo5310/71(18January1978)para162.130Taylor(n50)175.131CCPR,CabalandBertranvAustralia,CommNo1020/2001(7August2003)para8.2.132CCPR,GamarravParaguay,CommNo1829/2008(22March2012)para7.4;seealsoIACtHR,LandaetaMejiasBrothersetalvVenezuela,SeriesC281(27August2014).133ECtHR,RibitschvAustria,AppNo18896/91(4December1995)para38.134ECtHR,AbdullahYaşaandOthersvTurkey,AppNo44827/08(16July2013)para43.DiegoMauri-9781802207675DownloadedfromPubFactoryat05/28/202203:48:40AMviaDukeUniversityLawLibrary

105Autonomousweaponssystemsunderinternationalhumanrightslaw97toreactthroughtheuseofforcemayhardlybejustified.135Inlinewiththerighttolife,humanrightsmonitoringbodieshavedevelopedanimmensecaselawonpositiveobligationsstemmingfromtherightathand.Ontheonehand,statesarerequiredtoplanlawenforcementoperationsinordertominimizeanyrecoursetoforce,accordingtothecriteriaoffeasibilityandreasonableness.Ontheother,conductamountingtotheviolationoftheprohibitionontorture,inhumanordegradingtreatmenthavetobeproperlyinvestigatedanddulypunished.136Insummary,itissafetoconcludethatassessmentsintermsofjustificationofaparticularuseofforcearestructurallycontext-related:whatmattersisthatforceisresortedtowhen(strictly)necessary,foralegitimatepurpose,andinkeepingwithdomesticlaw.Usesofforcethatcanbejustifiedonthosebasesaresituatedbelowthe‘minimumlevelofseverity’threshold.LetusdiscussascenarioinwhichAWSareemployedtoneutralizeindivid-uals(forexample,rioters,protestors,orsuspectsengagedinviolentconductsuchasrobberiesorthetakingofhostages):alltherequirementsabovehavetobecompliedwithbylaw-enforcementauthorities.Again,itisvitalthateverydeliberationofforcebeexplainedandunderstoodfrombothanexanteandanexpostperspective.Ourfindingsaboveinrelationto‘arbitrariness’findtheircounterpartintheanalysisconductedthroughthelensof‘justification’offorceinpolicingoperations.Itfollowsthathumancontrolhastobepresenttoadegreetowhichitispossibletoexplainwhyandhowanautonomoussystemappliedforceagainstanindividual(oragroupofindividuals).Humaninter-ventionintheparticulardecisiontoselectandengageatargetisnotnecessary,providedthatthesystemremainsunderhumancontrolinthemeaningabove.GrantedthatAWSequippedwithless-lethalornon-lethalweaponsaresubjecttothesamerulesthatareapplicabletoalreadyexistingweapons,adifferentargumentcanbeputforwardaccordingtowhichasAWSmakereal-timedecisionswithouthumanintervention,theforcetheyusewouldbeinherently‘inhuman’inasmuchasappliedbyanon-humanagent.137Althoughappealing,weregardthisargumentasfar-fetchedatbest,inawaythatdoesnotdifferfromourlineofreasoningwithregardtotherighttolife:ourpreviousfindingsapplyfully.Theconceptof‘inhuman’hasalwaysbeeninterpretedasrelatingtotheactualtreatmentofanindividualratherthantothenatureofthe135ECtHR,CestarovItaly,AppNo6884/11(7April2015).136Taylor(n50)178ff.137Heyns,‘HumanRightsandtheUseofAutonomousWeaponsSystems’(n4),363(‘[s]incemachinesarenothumans,itcanbearguedthattheapplicationofforcebyamachinetoahumanbeingwithoutdirecthumaninvolvementandappropriatelevelsofcontrolisinherently,orbydefinition,“inhuman”treatment’).DiegoMauri-9781802207675DownloadedfromPubFactoryat05/28/202203:48:40AMviaDukeUniversityLawLibrary

10698Autonomousweaponssystemsandtheprotectionofthehumanpersonperpetratorinflictingsuchtreatment.138Thatis,thisargument,whilecapturingamoralexigencethatisfarfromnegligible,mayplayadelegeferendaroleandinspirefuturecaselaw;itcouldhardlybeemployedtoassertthatforceappliedthroughAWSamountsto‘inhumantreatment’initself.Asafinalremark,notethatdifferentconclusionsmightbereachediftheconnectionbetweenthenotionsof‘inhumantreatment’and‘humandignity’isexplored:referencestohumandignityarenumerousinthecaselawofIHRLmonitoringbodiesandscholarship.139Attimes,humanrightsmonitoringbodiessucceedinjustifyingtheevolutionaryinterpretationoftheprohibitionunderscrutinyonthesamebasisprovidedbytheconceptofhumandignity.ThisiswhythenotionofhumandignitymayplayakeyroleinappraisingAWS.Wediscussthistopicmoreextensivelylateron;fornow,sufficeittosaythattheruleprohibitingtorture,inhumanordegradingtreatmentdoesnotruleoutthefutureemploymentofAWS.3.4OTHERHUMANRIGHTSAFFECTEDBYAWS:ABRIEFOVERVIEWTheanalysisconductedsofarhasshownthat,accordingtocurrentinterpre-tationsofIHRLrulesregulatingthoserightsthatareputmoreatstakebyAWS,aslongashumancontrolisensuredonAWS—thatis,itispossibletounderstandandexplaintheirparticularcoursesofaction—theycanbedevelopedanddeployedinaccordancewithIHRL.Thathumaninterventionmaybeabsentdoesnotconstituteperseaviolationofthoserights.Asouranalysisconsidersonly‘criticalfunctions’ofaweaponsysteminthemeaningillustratedpreviously,thesefindingsareenough.However,tobroadenthevisionandtakeintoaccountotherrightsthatmaybeimpingeduponbyAWSwillprovideuswithadditionalinsights,whichbothconfirmourfindingsandinspirefurtherreflection.Thefirsthumanrightcomingtotheforeistherighttoprivacy.140Thetech-nologyAWSwillbeendowedwithmayeasilyturnouttobeatemptingtool138SGreer,‘IstheProhibitionagainstTorture,Cruel,InhumanandDegradingTreatmentReally“Absolute”inInternationalHumanRightsLaw?’(2015)15HumanRightsLR101(testingcurrentinterpretationoftheprohibition).139EWebster,‘InterpretationoftheProhibitionofTorture:MakingSenseof‘Dignity’Talk’(2016)17HumanRightsR371(focusingonthecontributionthatthediscoursearound‘humandignity’givestotheinterpretationandapplicationoftherightnottobesubjecttocruel,inhumanordegradingtreatment).140UDHR,art12UDHR;ICCPR,art17;ECHR,art8;ACHR,art11.Foranhis-toricaloverviewoftheinternationalprovisionsprotectingtherighttoprivacy,seeGDellaMorte,BigDataeprotezioneinternazionaledeidirittiumani.Regoleeconflitti(EditorialeScientifica2018),75ff.DiegoMauri-9781802207675DownloadedfromPubFactoryat05/28/202203:48:40AMviaDukeUniversityLawLibrary

107Autonomousweaponssystemsunderinternationalhumanrightslaw99forstatestoestablishcontinuingsurveillancemechanismsuponindividuals,especiallyintheageofrepeatedterroristthreats.141IHRLinstrumentsprotecttherightnottobesubjectedtoarbitraryorunlawfulinterferencewithanindi-vidual’sprivacy,family,homeorcorrespondenceasakeycomponentoftherighttofreedomofexpressionandthought.However,whileacknowledgingthisrightas‘oneofthefoundationsofademocraticsociety’,thoseinstrumentsallowforlimitationsandrestrictions.142Again,standardsofnecessity,propor-tionalityandnon-arbitrarinessregulatetheextenttowhichtherightcanbelimited,inalessrigidfashionthanwithregardtoabsoluteandnon-derogablerights.Interestingly,thosestandardshavebeeninterpretedbyhumanrightsmoni-toringbodiesinadistinctivelyrestrictivefashion,especiallyasaconsequenceoftechnologicaldevelopments.Thisisthecase,forexample,oftechnologyallowingmassivesurveillanceandthesystematiccollectionandstoringofper-sonaldata.Uponthepremisethat‘anunlimiteddiscretiontosubjectpersonswithin[states’]jurisdictiontosecretsurveillance’wouldrisk‘underminingorevendestroyingdemocracyonthegroundofdefendingit’,theECtHRhasdevelopedaninterestingcaselawthroughthelastdecades.143IntheSzabóandVissycaseitisheldthatwhilethat‘governmentsresorttocutting-edgetechnologies’is‘anaturalconsequenceoftheformstakenbypresent-dayter-rorism’,itwould‘defythepurposeofgovernmenteffortstokeepterrorismatbay…iftheterroristthreatwereparadoxicallysubstitutedforbyaperceivedthreatofunfetteredexecutivepowerintrudingintocitizens’privatespheresbyvirtueofuncontrolledyetfar-reachingsurveillancetechniquesandpreroga-tives’.144Itfollowsthatlegalsafeguardshavetobeputinplacebystateswhenemployingthis‘strategic,large-scaleinterception’ofpersonaldata.145Whatshouldbenotedisthatinassessingtherespondentstate’ssafeguardstheCourtinterpretedtherequirementof‘necessaryinademocraticsociety’containedinarticle8(2)ECHRasrequiringthemorestringent‘strictnecessity’byreasonofthepervasivenessofthecutting-edgesurveillancetechnologiesbasedon141‘ReportoftheSpecialRapporteuronthepromotionandprotectionofhumanrightsandfundamentalfreedomswhilecounteringterrorism,BenEmmerson’(14April2014)A/HRC/25/59paras35ff.142UNGA,‘Therighttoprivacyinthedigitalage’(18December2013)A/RES/68/167preamble.143KlassetalvGermany,AppNo5029/71(6September1978)para49.SeealsoLibertyetalvTheUnitedKingdom,AppNo58243/00(1July2008);KennedyvTheUnitedKingdom,AppNo26839/05(18May2010);RomanZakharovvRussia,AppNo47143/06(4December2015);BigBrothersWatchandOthersvTheUnitedKingdom,AppsNos58170/13andOthers(25May2021).144SzabóandVissyvHungary,AppNo37138/14(12January2016)paras67–8.145Ibidpara67.DiegoMauri-9781802207675DownloadedfromPubFactoryat05/28/202203:48:40AMviaDukeUniversityLawLibrary

108100Autonomousweaponssystemsandtheprotectionofthehumanperson‘automatedandsystemicdatacollection’.146Thatis,therationaletheCourtleansonseemstobethefollowing:themoreimpactfulonhumanrightsasur-veillancemeasureis,themorestringenttheconditionsfortheresorttheretoneedtobe.147InrelationtoAWS,thepurposeofdatacollectionwouldbeinteraliatoidentifypotentialtargets.Inparticular,gathereddataarelikelytobestoredandre-elaboratedforthepurposesofthe‘patternoflife’analysis,anoperationamountingto‘profiling’.148DatainterceptionisnotpeculiartoAWS:theprac-ticeof‘signaturestrikes’conductedviaarmeddronesalreadyraisessimilarissues.149Currently,thereislittleornoIHRLcaselawonsignaturestrikes,sincetheyareconductedontheterritoryofstatesthatarenotpartytoregionalhumanrightstreaties(forexample,Pakistan,YemenorAfghanistan)andonthepartofstatesthathavenotrecognizedindividualapplicationstouniversalmonitoringbodies(forexample,theUSvis-à-vistheICCPR).Litigatingthosestrikesprovesparticularlyhard,notonlybeforehumanrightsmonitoringbodiesbutalsobeforedomesticjudges,wherevictimsofstrikesfaceanumberofobstacles(suchasjurisdictionalbars,stateimmunityandotherjudicialtechniquesofavoidance).150However,giventhattechnologicalprogressrisksdeprivingtherighttoprivacyofitscontent,itisabsolutelyurgentthattheseproceduralobstacleswillbeovercomeinthefuture.Inaddition,thegroundsforlimitingtherighttoprivacymustbereinforced:ablanket,massivecol-lectionofpersonaldataforlaw-enforcementpurposeswouldnotonlyexposevirtuallyeveryindividualinagivengeographicalareatoaviolationoftheirrighttoprivacy,butalsomakethempotentialtargetsthatcanbeselectedandengagedduringpolicingoperations.AnotherhumanrightthatcanbeimpactedbyAWSistherightnottobedis-criminatedagainst,aswidelyrecognizedbothincustomaryandtreatylaw.151Recentscholarshiphasgivenagreatdealofattentiontoissuesofdiscrimi-nationconnectedwiththeemploymentofemergingtechnologies,especially146Ibidparas73,67.147Formoreonthispoint,seeDMauri,‘“AlgorithmicTargetConstruction”andtheChallengesbyInternationalHumanRightsLaw’(2019)Eurojus8.148Spagnolo(n4)44ff.149KHKindervater,‘TheEmergenceofLethalSurveillance:WatchingandKillingintheHistoryofDroneTechnology’(2016)47SecurityDialogue223;Benson(n5);TWallandTMonahan,‘SurveillanceandViolencefromAfar:ThePoliticsofDronesandLiminalSecurity-Scapes’(2011)15TheoreticalCriminology239.150Theissueissocomplexthatitcannotbecoveredhere.SeeDMauri,‘ThePoliticalQuestionDoctrinevis-à-visDrones’‘OutsizedPower’:AntitheticalApproachesinRecentCase-Law’(2020)68QuestofInt’lL3.151UDHR,art7(standingas‘autonomousright’aswellaspartofthegeneralclausecontainedinart1);ICCPR,art26;ECHR,art14andart1Protocol12;ACHR,art24.DiegoMauri-9781802207675DownloadedfromPubFactoryat05/28/202203:48:40AMviaDukeUniversityLawLibrary

109Autonomousweaponssystemsunderinternationalhumanrightslaw101algorithms.152Particularlyalarmingarethebiasesthatmayaffectthem:theriskofflawedalgorithmicprocesses,resultinginundesired(andultimatelyunlawful)outcomesmaybringaboutdiscriminatorytreatmentofindividu-alsinseveralfields,suchascreditscoring.153Ithasbeendemonstratedthatsomealgorithmicprocessesemployedinpolicingandotherjustice-sensitivedomainstendtoberaciallybiased.154Aneffectivelegalframeworkforminimizingtheriskofbiasesmustbeenforcedinpractice.155EnsuringthatalgorithmicprocessesarenotaffectedbyanydiscriminatorybiasesisallthemoreimportantasfarasAWSareconcerned:whatisatstakeareindividuals’fundamentalrights—lifeandpersonalintegrityinthefirstplace.156LimitingouranalysisagaintotheECtHR’scaselaw,therightathandprohibitsnotonlydirectdiscrimination,butalsoindirectdiscrimination,thatis,thosethat‘taketheformofdisproportionatelyprejudicialeffectsofageneralpolicyormeasurewhich,thoughcouchedinneutralterms,discriminatesagainstagroup’andthatdonotrequire‘discriminatoryintent’.157InorderforAWStobeusedinlinewiththisprohibition,itiskeythattheiremploymentdoesnotresultinmeasuresaffectingaspecificgroup,inawaythatisunreasonablydisproportionate.IshumaninterventionrequiredinorderforAWStoliveuptorequirementsderivingfromtherightnottobediscriminatedagainst?Again,takingstockofcurrentcaselaw,theargumentcanbemadethataslongascoursesofactiontakenbyAWScanbeunderstoodandexplainedinordertodemonstratetheirnon-arbitrariness,therightnottobediscriminatedagainstwillberespected.Inconclusion,ourfindingswithregardtotherighttolifeapplyalsotootherrightspotentiallyaffectedbyAWS.Thereisnoneedtore-stateherethepointsmadepreviously:augmentedriskscallforincreasedprotectionbylaw,which152DellaMorte(n140);LMcGregor,DMurrayandVNg,‘InternationalHumanRightsLawasaFrameworkforAlgorithmicAccountability’(2019)68ICLQ309.153FPasquale,TheBlackBoxSociety:TheSecretAlgorithmsThatControlMoneyandInformation(HarvardUniversityPress2015).154RSimmons,‘BigDataandProceduralJustice:LegitimizingAlgorithmsintheCriminalJusticeSystem’(2018)15OhioStateJofCrimL573.SeemoregenerallyAGaraponandFLassègue,Justicedigitale.Révolutiongraphiqueetruptureantro-pologique(PUF2018).155FJZuiderveenBorgesius,‘StrengtheningLegalProtectionagainstDiscriminationbyAlgorithmsandArtificialIntelligence’(2020)24Int’lJHumanRights1(favoringasector-specificapproachforregulatingtheuseofalgorithmsinsensitivedomains).156Spagnolo(n4)45(drawinganalogiesfromtheUSpracticeofemployingalgo-rithmsallegedlycapableofpredictingthedefendant’s‘levelofrisk’incriminalpro-ceedingandconcludingthat‘[s]houldsimilarsoftwarebeintroducedinAWSwhichareusedforlawenforcementactivities,theriskofdiscriminationwouldbehigh’).157BiaovDenmark,AppNo38590/10(24May2016)para103.DiegoMauri-9781802207675DownloadedfromPubFactoryat05/28/202203:48:40AMviaDukeUniversityLawLibrary

110102Autonomousweaponssystemsandtheprotectionofthehumanpersoncanbeattainedbycarefullyadaptingexistingcategoriestonewtechnologies.IHRLinstruments,asinterpretedandappliedvis-à-vistechnologicaladvance-ments,alreadyprovideregulationforAWSandcanbeusedtofixthedegreetowhichhumancontrolhastobeensuredintheperformanceofcriticalfunctions.Again,therequesteddegreeofhumancontrolisthatwhichultimatelyallowsforexplainingandunderstandingparticularcoursesofaction.3.5HUMANDIGNITYASAGENERALPRINCIPLERELEVANTFORTHEUSEOFFORCEHumandignityhasbeendescribedas‘thebasicunderpinningand…theraisond’êtreofinternationalhumanitarianlawandhumanrightslaw’,158‘oneofthemostpervasiveandfundamentalideasintheentirecorpus’ofIHRL.159Scholarshiponhumandignity,aswellasdomesticandinternationalcaselaw,haveblossomedinthepastcentury,whichcurrentlyresultsinthedifficulty—ifnottheimpossibility—ofenucleatingauniversallyacceptednotionof‘humandignity’.‘Humandignity’appearsasapolymorphousconcept,taskedwithdif-ferentanddiscretefunctionsdependingonthecontextswhereitisemployedandthepurposesofitsemployment.Tociteanexampleofthispolymorphism,humandignitymaybeunderstoodassourceofstatedutiestowardsindividuals,butalsoassourceofdutiesthatindividualshavetowardsthemselves.160Thisis158ProsecutorvFurundžijacase(10December1998)ICTY-95-17/1-Tpara183.159PCarozza,‘HumanDignity’,inDShelton(ed),TheOxfordHandbookofInternationalHumanRightsLaw(OUP2013),354.‘Humandignity’isawell-affirmedprinciplealsoindomestic(constitutional)law:seeABarak,HumanDignity.TheConstitutionalValueandtheConstitutionalRight(CUP2015).SeealsoMEO’ConnellandCMDay,‘SourcesandtheLegalityandValidityofInternationalLaw.NaturalLawasSourceofExtra-PositiveNorms’,inSBessonandJd’Aspremont(eds),TheOxfordHandbookontheSourcesofInternationalLaw(OUP2017)575(showingthatthenotionofhumandignityisimbuedwithnatural-lawthinking,andexplainingthe‘renaissance’ofsuchnotioninthesecondhalfofthetwentiethcentury).160PDeSena,‘Dignitàumanainsensooggettivoedirittointernazionale’(2017)11DirittiUmanieDirittoInternazionale573;SHennette-Vauchez,‘AHumanDignitas?RemnantsoftheAncientLegalConceptinContemporaryDignityJurisprudence’(2011)9Int’lJofConst’lL32.DiegoMauri-9781802207675DownloadedfromPubFactoryat05/28/202203:48:40AMviaDukeUniversityLawLibrary

111Autonomousweaponssystemsunderinternationalhumanrightslaw103whyseveralcommentatorsconsiderthisconceptasvacuous,161suspicious,162evendangerous,163andlackinganystablemeaningandcontent.Yetimbuedwithmoralconsiderations,theconceptofhumandignityhasalsoalegaldimensionthatmustbeexplored.OurmainthesisaccordingtowhichhumandignityisageneralprinciplerelevantforthedebateonAWShasbeenoutlinedinChapter1.Timehascometoinvestigatethecontentofsuchgeneralprinciple.Towhatextentisabsenceofhumaninterventionincriticalfunctionsconsistentwithhumandignity?Is‘humancontrol’quahumanoper-ators’abilitytoexplainandunderstandhowtheweaponssystemperformedinaparticularcircumstanceenough?Thissectionisstructuredasfollows:first,thenotionofhumandignityandtheextenttowhichitmaybeconsideredasageneralprincipleofIHRLisinvestigated(3.5.1);andsecond,therelevanceofthisnotioninthedebateonAWSisanalyzedinordertoidentifyweakandstrongspotsinargumentscenteredonhumandignity(3.5.2).3.5.1OverviewofaContestedPrincipleInternationalhumanrightslawinstrumentscontainnumerousreferencestothenotionof‘humandignity’.TheUDHRrefersto‘humandignity’inthepreamble(where‘inherentdignity…ofallmembersofthehumanfamily’isinvoked),inarticle1(whichproclaimsthat‘allhumanbeingsarebornfreeandequalindignityandrights’),164andinasetofprovisionsdedicatedtosocialrights.165Dignityisregardedasauniversalvaluethatpertainstohumanbeings.Nofurtherdefinitionisprovided:stateshadnointerestinstrivingtodefinehumandignity,avagueconceptbeingsufficientforthepurposesofthe161MBagaric-Allan,‘TheVacuousConceptofDignity’(2006)5JofHumanRights257,268(contendingthatthediscourseondignityissoabundantinopen-endedremarksthatitisimpossibletounderstandthatnotion‘inanysensebeyondthetautological’).162PMartens,‘Encoreladignitéhumaine:réflexionsd’unjugesurlapromotionparlesjugesd’unenormesuspecte’,inMVerdussenetal(eds),Lesdroitsdel’hommeauseuildutroisièmemillénaire:mélangesenhommageàPierreLambert(Bruylant2000),562(‘[l]adignitéhumaineestenpassededevenirlanotionlaplusagaçantedelalittératurejudiciaire,tantelleseprêteàdesutilisationsvariées’[‘humandignityistobecomethemostirritatingnotionofjudicialliterature,duetoitsvariedutilizations’]).163ECtHR,VereinigungBildenderKünstlervAustria,AppNo68354/01(25January2007)JointDissentingOpinionofJudgesSpielmanandJebenspara9(‘[p]ournous,ilnes’agitpasduconceptabstraitouindéfinidedignitéhumaine,notionquipeutêtreensoidangereuselorsqu’elleestdenatureàjustifierhâtivementdeslimitationsinacceptablesauxdroitsfondamentaux’).164UDHR,Preambleandart1(emphasesadded).OurdefinitionofhumandignityhasbeenexposedinCh1,s1.3andCh2,s2.3.4.165UDHRarts22and23(3).DiegoMauri-9781802207675DownloadedfromPubFactoryat05/28/202203:48:40AMviaDukeUniversityLawLibrary

112104AutonomousweaponssystemsandtheprotectionofthehumanpersonDeclaration,namely,tohavestatesagreeonalistofrights.Theundefinedandprogrammaticcharacterofhumandignitywasconsistentwiththenon-bindingnatureoftheUDHR,andno‘deepphilosophicaljustification’wasneededthen.166Turningnowtobindinginstruments,bothuniversalandregionalIHRLtreatiesmentionhumandignity,typicallyintheirpreambles.167Dignityisalsopresentinprovisionsrelatingtotherightnottobesubjecttotorture,inhumanordegradingtreatment,168tobefreefromforcedlabor,169toliberty,170toberecognizedalegalstatus,171toprivacy,172toeducation,173andotherrightsspe-cificallyenvisagedforchildren.174Atfirstglance,humandignityisreferredtoinapluralityofcontextsandveryfrequentlywithoutspecificmeaning:somearguethat,again,thepurposewastoformulateaconceptcapableofadaptingtothemostheterogeneouscontextswithoutlosingfunctionality.175Thatis,itdoesnotseemthatanautonomous‘righttodignity’existsinIHRL;instead,thenotionofhumandignityprovidesauniversallyacceptedfoundationofdiversehumanrights.176Tocontendthathumandignityisusedonlyforrhetoricalpurposes,however,isincorrect.Thecircumstancethatthisconceptreferstothefoundationofallhumanrights,thusencapsulatingcorevaluesthatdeserveprotectioninthelegalsystem,leadsustowonderwhetheritcorrespondstoageneralprinciple166RBrownsword,‘HumanDignityfromaLegalPerspective’,inMDüwell,J.Braarvig,R.BrownswordandD.Mieth(eds),TheCambridgeHandbookofHumanDignity(CUP2014),2–3.167InternationalConventionontheEliminationofAllFormsofRacialDiscrimination(adopted21December1965,enteredintoforce4January1969)660UNTS195(ICESCR)preamble;ConventionontheEliminationofAllFormsofDiscriminationagainstWomen(adopted1March1980,enteredintoforce3September1981)1249UNTS13(CEDAW)preamble;CAT,preamble(affirmingthattherightsrecognizedtherein‘derivefromtheinherentdignityofthehumanperson’).168ACHRart5.169ACHRart6(2).170ICCPRart10(1);ConventionontheRightsoftheChild(adopted20November1989,enteredintoforce2September1990)1577UNTS3(CRC)art37(c).171AfCHPRart5.172ACHRart11.173ICESCRart13;CRCart28.174CRCarts39and40.175Carozza(n159)349.SeealsoCMcCrudden,‘HumanDignityandJudicialInterpretationofHumanRights’(2008)19EJIL655.176Carozza(n159)353;Hennette-Vauchez(n160)49ff.SeealsoCO’Mahony,‘ThereIsNoSuchThingasaRighttoDignity’(2012)10Int’lJofConst’lL551(arguingthatduetoinconsistentformulationandapplicationindomesticlegalordersthealleged‘righttodignity’isintermingledwiththatto‘personalautonomy’).DiegoMauri-9781802207675DownloadedfromPubFactoryat05/28/202203:48:40AMviaDukeUniversityLawLibrary

113Autonomousweaponssystemsunderinternationalhumanrightslaw105ofinternationallawor,morespecifically,ofIHRL.177Tothisend,attentionhastobegiventothepracticeofhumanrightsmonitoringbodies,whichisabundantinreferencestohumandignitywheninterpretingtreatyprovisions.Thosebodiesemploythenotionofhumandignityasahermeneutictool.TheexampleofferedbytheECtHRcaselawistelling:whiletheECHRdoesnotcontainanywrittenprovisionmentioningthenotionofhumandignity,theCourthasmadenumerousreferencestothatconceptwhileinterpretingtherightsenshrinedthereinandwithaviewtoadvancingaparticularunderstand-ingthereof.InTyrervtheUnitedKingdom,theECtHRheldthatthepracticeofcorporalpunishmentofchildren,whilenotcausing‘anysevereorlong-lastingeffects’perse,constitutedanassaultonphysicalintegrityanddignityinthattheaddresseeswere‘treatedasanobjectinthepowerofauthorities’.178ThisfindingwasguidedbytheneedtointerprettheECHRinawaythatrenderstherightsenshrinedthereinpracticalandeffective:conductsthatdidnotamounttoaninfringementofrightsinthepastmaycontrastwithchangedsocietalneedsandperception,andmaybefoundunlawfulonthebasisofanevolutionaryinterpretationoftreatyprovisions.Morerecently,ithasbeenaffirmedthatanyexcessiverecoursetoforcebylaw-enforcementofficialsisinsharpcontrastwithhumandignityandthuswitharticle3ECHR.179Inthesamevein,lifeimprisonmentwithoutprospectofreleaseisdeemedtobeincontrastwiththeprohibitionofdegradingorinhumantreatmenttotheextentthatitimpingesonprisoners’dignity,deprivingthemofhopetoregainfreedomsomeday.180Caselawonphysicalintegrityorequalityandnon-discriminationmakesabundantreferencestohumandignity.181Insomecases,humandignityservesevento177SeeCh2,s2.3.3.178AppNo5856/72(25April1978)para33.179Seesupras3.3;seealsoECtHR,BouyidvBelgium,AppNo23380/09(28September2015)para88(recallingtheconceptofhumandignityinacaseofanindi-vidualslappedinthefacebypolicemenwhileincustody).180VinterandOthersvTheUnitedKingdom,AppsNo66069/09andothers(9July2013)para113;ViolavItaly(No2),AppNo77633/16(13June2019)paras113and136.181ECtHR,KuznetovvUkraine,AppNo39042/97(29April2003)(foracaserelat-ingtoconditionsofdetentionthatwerebelievedto‘havecausedhimconsiderablementalsuffering,diminishinghishumandignity’);NachovaandOthersvBulgaria(n51)(shootingoftwoRomafugitivesbymilitarypoliceduringattemptedarrest)IACtHR,JuvenileReeducationInstitutevParaguay,SeriesC112(2September2004)para151(foracaseconcerningconditionsofdetention:‘allpersonsdetainedhavetherighttoliveinprisonconditionsthatareinkeepingwiththeirdignityashumanbeings’);VelásquezRodríguez(n63)para156(prolongeddenialofcommunicationinprison);LoayzaTamayovPeru,SeriesC33(17September1997)para57(excessiveforceincontrollinginmatebehavior);MiguelCastro-CastroPrisonvPeru,SeriesC160(25November2006)paras305–6(prisoninmatesforcedtonakednessforextendedDiegoMauri-9781802207675DownloadedfromPubFactoryat05/28/202203:48:40AMviaDukeUniversityLawLibrary

114106Autonomousweaponssystemsandtheprotectionofthehumanpersonbroadenthescopeofspecificrightsinordertoincludeotherrightsthatbythemselvesarenotprotectedbythetreaty.Thisappliestotherighttohousing,adequatefood,highestattainablestandardofhealth,work,socialsecurityandcultureattheuniversallevel.182Thatis,humandignityhasbeenemployedtoattributenewcontenttoexistingrights(therightnottobesubjectedtotorture,inhumanordegradingtreatment;therighttoequality;therighttolifeunder-stoodas‘adignifiedlife’):whilethisoperationispresentedasinterpretive,theborderlinewithjudiciallaw-makingisadmittedlythin.183Interestinglyforourpurposes,humandignityisoftenusedwiththefollowingmeaning:itisprohibitedtoreduceindividualstomereobjectsormeans,or‘dispensablequantities’—failingtoconsidertheindividualworthofeachhumanperson.184ThisunderstandingisparticularlyusefulwithregardtoAWS.Wrappingupourfindings,humandignitycanberegardedasageneralprincipleformedwithintheinternationallegalsystem,andmorepreciselyinIHRL,atleasttotheextenttowhichinterpreters(firstandforemosthumanrightsmonitoringbodies)resorttoitasahermeneutictooltoadvanceapar-ticularinterpretationoftreatyprovisions,mainlyinordertoadjustthemtoever-changingcircumstances(thedynamicorevolutionaryinterpretation).185Humandignity,althoughindeterminateandvagueincontent,effectivelyper-formsfunctionsthataretypicalofgeneralprinciples,namely,asinterpretiveaidandgap-filler.186Scholarsinclinedtowardsextra-positivetheoriesoflawarekeenonconsideringhumandignityasageneralprinciple,thatis,asaperiodsoftime);AtalaRiffoandDaughtersvChile,SeriesC239(24February2012)para79(discriminationbasedonsexualorientation).SeealsoIACtHR,JuridicalConditionandRightsoftheUndocumentedMigrants,AdvisoryOpinionOC-18/03(17September2003)para119.182IACtHR,YakyeAxaIndigenousCommunityvParaguay,SeriesC125(17June2005)(communityofindigenouspeopledeprivedofadequatefood,water,andhealth-care).SeealsoCESCR,‘GeneralCommentNo4:TheRighttoAdequateHousing(Art.11(1)oftheCovenant)’E/1992/23(13December1991)para7(‘therighttohousingshouldnotbeinterpretedinanarroworrestrictivesensewhichequatesitwith,forexample,theshelterprovidedbymerelyhavingaroofoverone’sheadorviewsshelterexclusivelyasacommodity.Ratheritshouldbeseenastherighttolivesomewhereinsecurity,peaceanddignity’).SeealsoCarozza(n159)355–6.183GeneralCommentNo3(n33)paras3,6,11,36,43.184SeeDAmoroso,AutonomousWeaponsSystemsandInternationalLaw.AStudyonHuman-MachineInteractionsinEthicallyandLegallySensitiveDomains(EdizioniScientificheItaliane2020)168.185GLetsas,ATheoryofInterpretationoftheEuropeanConventiononHumanRights(CUP2009),58ff.186SeePDeSena,Dignitéhumaineetdroitinternational(forthcoming)RecueildesCours.DiegoMauri-9781802207675DownloadedfromPubFactoryat05/28/202203:48:40AMviaDukeUniversityLawLibrary

115Autonomousweaponssystemsunderinternationalhumanrightslaw107‘normativereferencepoint’pointingto‘somesupra-positivevalue’.187Fromthisstandpoint,thenexusbetweenlawandmoralityisemphasized:ashasbeeneminentlyaffirmed,humandignitywouldforma‘conceptualhinge’,a‘portal’throughwhichmoralimperativesareimportedintolaw.188Thisapproachtohumandignityissupported,interalia,bytheworksoftheILCongeneralprinciples:‘respectforhumandignity’isenumeratedamongtheprinciples‘underlyinggeneralrulesofconventionalorcustomaryinternationallaw’.1893.5.2HumanDignityintheDebateonAWSTheconceptof‘humandignity’isoneofthemainweaponsinthearsenalofthosewhoopposethedevelopmentofAWS.AscapturedineffectivetermsbyHeyns,theargumentrootedinhumandignitytakeslogicalprecedenceoverothers,asitreplacesthe‘cantheydoit?’questionwiththe(farmoreimpellent)‘shouldtheydoit?’question.190ItisthusappropriatetosiftthroughthevariouspositionstakenbyactorsparticipatinginthedebateonAWS,inordertoverifywhentheprincipleofhumandignityisemployedandforwhichpurposes.Asforstates,referencestohumandignityarescatteredandcursory.IntheCCWframeworkGermanywasthefirsttoexplicitlyrefertohumandignity:the‘righttodignity’wasusedtoarguefortheneedfora‘principleofhumancontrol’overautonomousweaponry.191Subsequently,referencesto‘humandignity’havebeenmadebyChile,192Ecuador,193SriLanka,194CostaRica,195187Carozza(n159)358.Indifferentterms,seealsoCLeBris,L’humanitésaisieparledroitinternationalpublic(Pedone2012),75ff(arguingthathumandignityandhumanrightsstandasdiscretelegalentitiesandthatthelatter‘joueainsiunrôlede“catalyseur”danslesensoùelleprovoque,parsaseuleprésence,unemutationdeceux-cietce,sansquesanaturen’ensoitelle-mêmemodifiée’[‘playsalsoaroleasa“catalyst”asitbringsabout,byitself,atransmutationofthem,withoutevermodifyingitsownnature’]).188JHabermas,‘TheConceptofHumanDignityandtheRealisticUtopiaofHumanRights’(2010)4Metaphilosophy464,469.189ILC,‘SecondreportongeneralprinciplesoflawbyMarceloVásquez-Bermúdez,SpecialRapporteur’(9April2020)A/CN.4/71paras143–4.190Heyns,‘AutonomousWeaponsSystems:LivingaDignifiedLifeandDyingaDignifiedDeath’(n98)10.191Germany,GeneralStatementatthe2014MoE(May2014).192Chile,‘Intervención’,2015MoE(April2015).193Ecuador,‘Declaración’,2015MoE(April2015).194SriLanka,‘StatementbyH.E.Mr.RavinathaP.Aryasinha’,2016MoE(11April2016).195CostaRica,‘Intervención’,2016MoE(11April2016).DiegoMauri-9781802207675DownloadedfromPubFactoryat05/28/202203:48:40AMviaDukeUniversityLawLibrary

116108AutonomousweaponssystemsandtheprotectionofthehumanpersonBrazil196andGreece.197‘Humandignity’hasbeenemployedmostlyasa‘prin-ciple’,yetwithoutfurtherexploringboththecontentandthescopethereof.Otherstateshaveemployed‘humandignity’eitherinrelationtoethicsormoregenerallyinawaythatdiscardsanypossiblelegalsignificance.198Tothewriter’sbestknowledge,neverhastheconceptofhumandignitybeeninvokedinasubstantivefashionbymilitarypowerssuchastheUS,theUK,Russia,ChinaorIsrael.Itissafetoconcludethatstateshavenotyetdevelopedaclearnormativeunderstandingofwhat‘humandignity’is(andwhatitmayrequireorentail)whenitcomestoAWS.Asfarasnon-stateactorsareconcerned,‘humandignity’hasbeeninvokedoften—forinstancebytheUnitedNationsInstituteforDisarmamentResearch(UNIDIR),HumanRightsWatch,theInternationalCommitteeforRobotArmsControl(ICRAC),PAX,theWomen’sInternationalLeagueforPeaceandFreedom,theUniversidadPablodeOlavidedeSevillaandbytheICRC.199Again,noclearindicationsareinferablefromthesestatements,astheyconflateethicalandlegalarguments.Turningtolegalscholarship,sounderreflectionshavebeenproposedonthenormativecontentofhumandignity.Heyns,formerUNSpecialRapporteur,haswrittenextensivelyonthetopic.200Inhisview,whileacknowledgingthathumandignitymayoperate‘asa“conversationstopper”,becauseofthetendencyofpeopletoreadtheirownpreferencesintoit’,itmakessensetospeakaboutarighttodignityonitsown.201AnormativebasisforsuchunderstandingofhumandignitywouldbeprovidedbytheACHPR;however,inourview,theconceptualdifficultiesthatweoutlinedpreviouslyremain.Insteadofarule,webelieveitismoreconvenienttoregardhumandignityasaprinciple.Inallcases,Heyns’sreflectionsareparticularlyimportantastheydealwiththeaxiologicaldimensionofhumandignity:hecleverlyaffirmsthatsincecriticalfunctionsmaybeperformedwithouthumanintervention,thisendsupreducinganindividual‘tonumbers:thezerosandtheonesofbits’.202Theunderlyingideaofhumandignityfitsperfectlywiththat196Brazil,‘Intervenção’,2017GGE(November2017).197Greece,Interventionatthe2018GGE(April2018).198France,Intervention,2015MoE(April2015)(listingpossiblegroundsfordis-cussingLAWSandincluding‘laquestiondeladignitéhumaine’intheethicalstand-point);Denmark,Intervention,2015MoE(labeling‘dignity’asoneofthe‘almostpoeticalexpressions’putforwardinthedebatesofar).199ICRC,Interventionatthe2018GGE(April2018).FormorestatementsbyNGOsandcivilsocietyrepresentatives,seeAmoroso(n184)214.200HeynsReport(n18)paras92ff.201Heyns,‘HumanRightsandtheuseofAutonomousWeaponsSystems’(n4)367–8.202Ibid370.SeealsoHeynsReport(n18)para95(arguingthatentrustingmachineswiththepowertotakedecisionsresultinginthereceiver’sdeathpaintsanimageofAWS‘assomekindofmechanizedpesticide’).DiegoMauri-9781802207675DownloadedfromPubFactoryat05/28/202203:48:40AMviaDukeUniversityLawLibrary

117Autonomousweaponssystemsunderinternationalhumanrightslaw109sponsoredbytheGermanConstitutionalCourtinthe2006judgmentontheconstitutionalityofthedomesticactauthorizingarmedforcestoshootdownhijackedairplanes:individualshappeningtofindthemselvesonboardoneofthoseaircraftwouldhavetheirdignitydiminished,beingtreatedmorelikeobjectsthanhumanpersons.203Theethicalunderpinningsofhumandignityareself-evident.Humandignitydemandslethaldecisionstobetakenbyhumans(thatis,agentswhocanfullyunderstandthemeaningoftheiractions),insteadofbeinglefttoan‘algorithmic’calculationofmachines(thatis,agentsthatcannotunderstandthemeaningoftheiractions),otherwisetheybecome‘meaning-less’and‘arbitrary’.ThisistheargumentputforwardalsobyAsaro.204FromapurelyKantianperspective,Ulgenassertsthat‘[h]umanmoralreasoninginvolvesacombinationofcomprehension,judgment,experience,andemotions’,205andthatthereplacementofhumandecision-makingwhentargetinganotherhumanisatvariancewiththenotionof‘objectiveend’(asopposedto‘relativeend’).Humandignityisa‘pricelessandirreplaceableobjectiveendpossessedbyallrationalbeings’:removingorsignificantlyreducingface-to-facekillingresultsinthediminishingofahumantarget’sdignity,treatedasa‘relativeend’thanasanendinhimself(orherself).206Whileamachinedeliberatesacourseofactionproducingeffectsintherealworld,thatdecisionisnottheresultofanautonomouswillasthatofahumanagent.Theparadoxof‘machinewill’iswellillustratedbyLambert,whoexplainsthattheveryattributeof‘autono-mous’tothosesystemsisapure‘illusion’thatseekstomaskreality.207AnotherargumentthathasbeenputforwardisthatwhenAWSareemployedpotentialtargetsaredeprivedofhope:themomentwhenAWSdeliberatetouseforceagainstaparticularindividual,furtherself-restraintisnotpossible—itcouldhardlybeotherwise.Inthealgorithmicprocesseverystepiscausallylinkedtothepreviousstep:ifexternalcircumstancesdonotchange,neitherthefinaloutcomedoes.Inbrief:pietascannotbetranslated203Bundesverfassungsgericht,1BvR357/05(15February2006).204PAsaro,‘JusNascendi:RoboticWeaponsandtheMartensClause’,inRCalo,AMFroomkinandIKerr(eds),RobotLaw(EdwardElgar2016),385(‘[f]orthekillingofahumantobemeaningful,itmustbeintentional.Thatis,itmustbedoneforreasonandforpurpose.…Whileautonomoussystemsmaybeprogrammedtoactinacertainway,givenacertainsetofconditions,theycannotunderstandthesignificanceoftheiracts’).205OUlgen,‘KantianEthicsintheAgeofArtificialIntelligenceandRobotics’(2017)43QuestofInt’lL59,79.206Ibid82.207DLambert,‘Uneéthiquenepeutêtrequ’humaine!Réflexionsurleslimitesdesmoralmachines’,inRDoeré,DDanetandGDeBoisboissel(eds),Droneset‘KillerRobots’.Faut-illesinterdire?(PressesUniversitairesdeRenne2015),234–5.DiegoMauri-9781802207675DownloadedfromPubFactoryat05/28/202203:48:40AMviaDukeUniversityLawLibrary

118110Autonomousweaponssystemsandtheprotectionofthehumanpersonintoalgorithms.Incidentally,itisworthnotingthatthisperspectivehasbeenexploredalsobysomeparticipantsinthedebate,suchastheHolySee.208Tojustifyhisposition,Heynsdrawsaninterestingparallelwiththedeathpenaltyandlifeimprisonmentwithoutparole:justasthesepenaltiestendto‘wipeoff’theindividualastheydonotleaveopenanypossibilityof‘hope’,AWScannotstructurallyspeartheirtargets.209Althoughappealing,theparallelholdsuptoamorelimitedextent.Thescenariosinwhich‘hope’isallegedlydeniedtoindividualssentencedtocapitalpunishmentandlifeimprisonmentareverydifferent:inthoseambits,individualsareconfrontedwiththecertaintythat,nomatterhowtheybehave,theirpunishmentwillremainthesame.Withregardtolifeimprisonmentwithoutprospectofrelease,the‘inhumanity’ofthepun-ishmentliesinthat,ultimately,itdefactonullifiesrehabilitation.Thesefindings,althoughconvincinginourview,areoflimiteduseinlaw-enforcementsituations.Inthosecases,(potentiallylethal)forcecanberesortedtoforthepurposeofneutralizingactualthreatsandinaccordancewiththerulesillustratedpreviously.Thisisallthemoresoinsplit-secondscenar-ios,thatis,whereAWSareexpectedtooutperformhumanagents—wheretheyarelikelytobefieldedinthefirstplace.Oncetheconditionsforusingforcearemet,thereisnolegalobligationto‘stepback’onthepartoflaw-enforcementagents;insomescenariosiftheydidsoandrefusedtotakeaction,theymayactinbreachoftheobligationtoprotectlifeandthusengagestateresponsibilityunderIHRL.Thatis,existingrulesontheuseofforceunderIHRLareatmostneutralvis-à-vistheindividual’shopenottobetargetedinthesecircumstanc-es.210Normatively,itthereforeseemsthatHeyns’sinterpretationofhumandignityasrequiringhopeonthepartofthepotentialaddresseeofadecisioninvolvingtheuseofforce,yetprincipled,doesnotcorrespondtothelawasitiscurrently.211However,thisargument—centeredonthe‘humanagencyremoval’—deservescreditinthatitprovidesapressingmoralreasontorejectAWS.212Albeitwithsomedifferences,thispositionisreflectedinotherscholarship.Amongafewauthors,onehascontendedthatsincehumandignity,atfirst208Forfurtheranalysis,seeDMauri,‘TheHolySee’sPositiononLethalAutonomousWeaponsSystems.AnAppraisalthroughtheLensoftheMartensClause’(2020)11JIHLS116.209Heyns,‘HumanRightsandtheuseofAutonomousWeaponsSystems’(n4)371.210Amoroso(n184)189.211Heyns,‘Autonomousweaponsinarmedconflictandtherighttoadignifiedlife’(n98)63(inwhichhejustifiesappealtohopebasingonthefactthathopeisan‘impor-tantmeasureofoptimism’asemphasizedby‘psychologists’,‘religiousandotherworldviews,aswellasphilosophicaltraditions’).212Amoroso(n184)189ff.DiegoMauri-9781802207675DownloadedfromPubFactoryat05/28/202203:48:40AMviaDukeUniversityLawLibrary

119Autonomousweaponssystemsunderinternationalhumanrightslaw111understoodmerelyasa‘socialvalue’,hasbecomea‘constitutionalvalue’and‘right’aswell,targetselectionandengagementwherehumandeliberationisabsentareimpermissible.213AnotherscholarhaswrittenextensivelyontheKantianfoundationofhumandignity,andcontendsthattoentrustanon-humanagentwithtargetinghumanswouldamounttotreatingthepotentialvictimas‘mererelativeendsthatcanbesubjectedtoefficientdisposal’.214Whenjusti-fyingthenormativestatusofhumandignity,theauthorresortstotheconceptof‘elementaryconsiderationsofhumanity’,enshrinedinthecaselawoftheICJ,215arguingthathumandignitywouldbepartofsuchconsiderations.Thisoverviewconfirmsourinitialintuition,namely,thatthecategoryofgeneralprinciplesformedwithintheinternationallegalsystem,andimbuedwithcorevaluesofthatlegalorder,isofparticularusewhenaddressingAWS.Thecoreissuemustnowbetackledofwhethertheprincipleofhumandignitycanbesaidtorequirehumaninterventionintheperformanceofcriticalfunctionsbyaweaponssystem.Iftheanswerwereaffirmative,theprincipleofhumandignitywouldprohibitAWS.Inordertoreachthisconclusion,andinkeepingwiththenatureofgeneralprinciplesassourceofinternationallaw,itmustbeascertainedwhethersuchaprincipleis‘recognized’bytheinterna-tionalcommunity.Themethodologyforthisrecognitionrequiresdeductionfromspecificrulesofinternationallaw,tothepointwhereaprinciplecanbeidentified‘evenintheabsence’oftreatyorcustomarynorms.Theanalysisconductedpreviouslyhasshownthatthisrecognitionisstillcontroversial,foratleasttworeasons.First,inmostinterventionsonAWSreferencestohumandignityareeitherscarceorcursory:thenotionofhumandignityisinvokedonaparwithmoralconsiderations,whichendsupdeprivingtheargumentofitslegalsignificance.Second,frequentlytheprincipleofhumandignityandotherrulesprotectingspecifichumanrightsareintermingledtotheextentthatitisnoteasytokeepthemdistinct.WhenstatesorotheractorsrefertotheneedthatAWSareemployedinamannerthatrespectshumandignity,theymeanthatrulesprotectinglife,physicalintegrity,andotherhumanrightsmustbecompliedwith.Letussumupourmainfindings.ToarguethatAWSareprohibitedonthesolebasisthathumaninterventionincriticalfunctionsisabsentseemstoofar-fetched:acorrespondingprinciplecannotbesaidtohavebeen‘recog-213TChengeta,‘Dignity,Ubuntu,HumanityandAutonomousWeaponSystems(AWS)Debate:AnAfricanPerspective’(2016)13BrazilianJofInt’lL461.214OUlgen,‘HumanDignityinanAgeofAutonomousWeapons:AreWeinDangerofLosingan“ElementaryConsiderationofHumanity”?’(2016)15ESILConferencePaper1,19.215CorfuChannelCase(UKvAlbania)(Merits)[1949]ICJRep4,22;LegalityoftheThreatorUseofNuclearWeapons(AdvisoryOpinion)[1996]ICJRep226para79.DiegoMauri-9781802207675DownloadedfromPubFactoryat05/28/202203:48:40AMviaDukeUniversityLawLibrary

120112Autonomousweaponssystemsandtheprotectionofthehumanpersonnized’bytheinternationalcommunity,atleastforthetimebeing.Toomanydivergentviewshavebeenputforwardthusfar,asthedebatewithintheCCWframeworkdemonstrates.However,thisdoesnotimplythattheprincipleofhumandignityisirrelevantaltogether.Asamplydemonstrated,generalprinci-plesprovideanapttoolforadjustingrulesforever-changingcircumstancesbywayofinterpretation,aswellasforfillinggapsinexistingrules:theyaremeanttoapplyto‘unusualcases’.216ThedevelopmentanddeploymentofAWSwillcertainlyprovideinterpreterswithsuchan‘unusualcase’,namely,thathumaninterventionisexpectedtobeabsentfromtargetselectionandengagement:neverinthehistoryofweaponshassomethingsimilaroccurred.217Ithastobekeptinmind,however,thatwhilegeneralprinciplesnaturallygobeyondaconsent-basedlogicofinternationallaw-makingandfeatureextra-positivethreads,218itisalsoundeniablethat‘recognition’bytheinternationalcommu-nitycannotberenouncedaltogether.219Withthisinmind,itisimperativethatstates,internationalorganizationsandotheractorsinvolvedinthedebateonAWSthatsharethesameskepticismtowardsAWSengageinasubstantiveeffortto‘recognize’humandignityquarequirementofhumaninterventionincriticalfunctions.First,argumentshavetobeformulatedinunequivocallegalterms,byidentifyingmoralimperativesandputtingtheminanormativeshape.Second,iftheyconsiderthathumandignityisaboutwhoappliesforceagainstahumantarget(insteadofwhyandhow),itisvitalthatargumentsareputforwardaccordingly.Manyscholarshavepavedthewayforthesedevelopments,220asdidsomeparticipantsatthedebate,suchastheICRCandimportantlytheHolySee.221Theprospectofsuccessofthoseargumentsarediscussedmorethoroughlyinthefinalchapterofthisbook.222216SeeCh2,s2.3.3.217SeeCh1.218O’ConnellandDay(n159).219BBonaféandPPalchetti,‘Relyingongeneralprinciplesininternationallaw’,inCBrölmannandYRadi(eds),ResearchHandbookontheTheoryandPracticeofInternationalLawmaking(EdwardElgar2016).220Amoroso(n184)189.221Heyns,‘HumanRightsandtheuseofAutonomousWeaponsSystems’(n4);Ulgen,‘HumanDignityinanAgeofAutonomousWeapons;seealsoMauri,‘TheHolySee’sPosition’(n208).222SeeCh6,s6.2.DiegoMauri-9781802207675DownloadedfromPubFactoryat05/28/202203:48:40AMviaDukeUniversityLawLibrary

121Autonomousweaponssystemsunderinternationalhumanrightslaw1133.6IHRLINTHEBATTLEFIELD:INTERPLAYWITHIHLANDISSUESOFEXTRATERRITORIALITYBeforeturningtothischapter’sconclusions,itseemsappropriatetodipintoanissuearoundwhichpracticeandscholarshiphaveevolvedextensively,namely,theextenttowhichIHRLrulesandprinciplesapplyalsoinarmedconflict.Internationalhumanrightslawprotectshumanbeingsinallcir-cumstances,anditsapplicabilityduringarmedconflicthasbeenreaffirmedtimeandtimeagainbynumerousauthorities,fromtheICJtohumanrightsmonitoringbodies.However,asIHListhebranchofinternationallawdealingspecificallywitharmedconflict,overlapsand,thus,conflictsinevitablyoccur.WepreviouslyarguedthattheconcurrentapplicationofIHRLandIHLhasbroughtaboutaheateddebateamonglawyers,whohaveelaboratedaplethoraoftheoreticalmodelstoframetherelationsbetweenthetwobranches(‘sep-aration’,‘complementarity’or‘harmonization’).223Farfrombeingamerelydoctrinalquandary,theinterplaybetweenIHRLandIHLhaspracticaleffectsontheregulationofhostilities.224OneofthebedrockdistinctionsinIHListhedifferencebetweeninternationalarmedconflict(IAC)andNIAC,whichultimatelydependsonthecharacteristicsoftheconflictathand.225Thisbifurcationisreflectedalsointheapplicablelaw:whileIACsareregulatedextensivelybytreatylaw(rangingfromthe1899/1907HagueConventionstothe1949GCandthenAdditionalProtocolI,API),conventionalrulesapplicabletoNIACarelimited.Inadditiontotheminimumlegalyardstickcontainedincommonarticle3,provisionsregulatingNIACcanbefoundinAdditionalProtocolII(APII)and,toacertainextent,intheRomeStatuteoftheInternationalCriminalCourt(ICCSt).Admittedly,NIACsareafieldofconcurrentapplicationofbothIHRLandIHL:ontheonehand,somerulestraditionallyregulatingIAC(alsoofcustomarynature)havebeenextendedtoNIAC;226ontheother,giventhepaucityoftreatyrulesandthecontroversies223SeeCh2,s2.3.2.224CDroege,‘TheInterplaybetweenInternationalHumanitarianLawandInternationalHumanRightsLawinSituationsofArmedConflict’(2007)40IsraeliLR310.SeealsoMSassoli,‘InternationalHumanitarianLawandInternationalHumanRightsLaw’,inBSaulandDAkande(eds),TheOxfordGuidetoInternationalHumanitarianLaw(OUP2020).225DAkande,‘ClassificationofArmedConflicts’,inBSaulandDAkande(eds),TheOxfordGuidetoInternationalHumanitarianLaw(OUP2020).226ProsecutorvTadić(2October1995)ICTY-94-1-AR72para127.SeealsoSecondProtocoltotheHagueConventionof1954fortheProtectionofCulturalPropertyintheEventofArmedConflict(adopted26March1999,enteredintoforce9March2004)2253UNTS212.DiegoMauri-9781802207675DownloadedfromPubFactoryat05/28/202203:48:40AMviaDukeUniversityLawLibrary

122114AutonomousweaponssystemsandtheprotectionofthehumanpersonaroundtheexistenceofcustomaryonesapplicabletoNIAC,manycommenta-torsresorttoIHRLnormstofindawayoutofthequagmire.227Therealityofcurrentarmedconflicts—oftenofanon-internationalnature,attimes‘spillover’conflictsinwhichstateactorsconfrontnon-stateactorsontheterritoryofathirdstatewithoutitsconsent—wouldrequireaclearlegalframeworktobeapplied.228TheUShasbeenconductinga‘GlobalWaronTerror’foralmosttwodecadesnow,targetingindividualsconnectedtoAl-QaedainYemen,SomaliaandPakistan;IsraelusesforceagainstHezbollahinLebanon;UgandaandRwandahavebeeninvolvedinaspill-overconflictintheDemocraticRepublicofCongo;TurkeyregularlyengagesKurdistanWorkers’Party(alsoknownasPKK)membersinNorthernIraqandSyria;theWesterncoalitionagainsttheIslamicStateofIraqandtheLevant(alsoknownasISIL)hasbeenactivelyengagedinSyria.Insummary,practiceisabundant,andtheintersectionbetweenIHL(bothforIACandNIAC)andIHRLrunsalongablurredline.Whileathoroughdiscussionofthisintersection(letalonetheresolutionofallissuesconnectedtherewith)fallsoutsidethescopeofthebook,itisappropriatetofocusonspecificscenariosofactivehostilitiesinwhichIHRLmayapply,especiallyasfarasextraterritorialmilitaryoperationsareconcerned.ThereisbroadpracticeofstatespartytoIHRLinstrumentspartic-ipatinginmilitarycampaignsabroad(forinstanceinAfghanistan,withintheInternationalSecurityAssistanceForce,orISAF,operation,orinIraq)withoutavailingthemselvesofthe‘derogationclauses’containedinthosetreaties.229Atleasttwoissuesarisefromthesescenarios.First,itmaybequestionedwhetherimpugnedconductsbystateforceslocatedextraterritoriallyfallwithinthe‘jurisdiction’ofthetroops-sendingstate.BothregionalanduniversalIHRLmechanismsobligestatestoguaranteehumanrightsestablishedthereintoeveryonewithintheirjurisdiction.230Differentmodelsofjurisdictionhavebeencraftedandtestedtoactive-hostilitiesscenarios:thejurisprudenceofmonitoringbodiessuggeststhatnotonlycontroloveraspecificterritory(suchasduringoccupation,accordingtoa‘territorial’modelofjurisdiction),butalsocontroloverpersons(suchaswhendetainingorinterningindividuals,accordingtoa‘personal’modelofjurisdiction)sufficetoconstitutejurisdic-227MSassoli,‘TheConvergenceofInternationalHumanitarianLawofNon-internationalandofInternationalArmedConflicts:DarkSideofaGoodIdea’,inGBiaggini,ODiggelmann,andCKaufmann(eds),PolisundKosmopolis:FestschriftfürDanielThürer(Nomos2015).228NLubell,ExtraterritorialUseofForceagainstNon-StateActors(OUP2010).229ECHR,art15;ICCPR,art4;ACHR,art27.230ECHR,art1;ICCPR,art2(1);ACHR,art1.DiegoMauri-9781802207675DownloadedfromPubFactoryat05/28/202203:48:40AMviaDukeUniversityLawLibrary

123Autonomousweaponssystemsunderinternationalhumanrightslaw115tion.231Morerecently,atrendhasgainedmomentum,accordingtowhich,jurisdictionmaybeextended,ingivencircumstances,toallindividualswhoseenjoymentofrightscanbeaffectedbystates’conduct(amodelwhichcanbereferredtoas‘impact’jurisdiction).232Second,itmaybequestionedwhether,intheabsenceofaderogationfromIHRLinstruments,themonitoringbodyshouldinterprettheIHRLprovisionsinlightoftherulesandprinciplesenshrinedinIHL.Again,IHRLjurisprudenceissomehowincoherent.WhilesomeregionalcourtssuchastheInter-AmericanCourtofHumanRights(IACtHR)hasshownagenerousattitudetowardssystemicinterpretationoftheACHR,othercourtssuchastheECtHRremainanchoredtothefull-fledgedapplicationoftheECHR,withouttakingintoaccount(atleastsubstantially)IHLprovi-sions.233Thedebateispolarized,withsomeauthorscautioningagainstignor-ingIHLaltogetherinfavoroftheapplicationofsoleIHRL,asthismayendupimposingtooheavyaburdenonstatesandthusmakingtheoverallrespectofhumanrightsinthosecontextsunrealistic.234ApracticalandinterestingscenarioisoneinwhichastateoperatesAWSextraterritorially,forinstance,tohuntdownagroupofarmedrebelslocatedontheterritoryofathirdstate.ArecentreportbytheUNPanelofExpertsonLibyasuggestedthataTurkish-madeautonomoussystem(theSTMKargu-2)mayhave‘remotelyengaged’armedsoldiersofGeneralHaftar’stroops.235231ECtHR,BankovićandorsvBelgiumandors,AppNo52207/99(19December2001);AlSkeinivtheUnitedKingdom,AppNo55721/07(7July2011)paras130–42;GeorgiavRussia(II),AppNo38263/08(21January2021).SeealsoGC36para63(andreferencestherein).232GC36para63.Thismodelhasbeentestedbyscholars,yettakingdivergentviewsontheoreticalpremisesandpracticaleffects.SeeLRaible,HumanRightsUnbound.ATheoryofExtraterritoriality(OUP2020);CMallory,HumanRightsImperialists:TheExtraterritorialApplicationoftheEuropeanConventiononHumanRights(Hart2020);MMilanović,ExtraterritorialApplicationofHumanRightsTreaties.Law,Principles,andPolicy(OUP2011);PDeSena,Lanozionedigiuris-dizionestataleneitrattatisuidirittidell’uomo(Giappichelli2001).233IACommHR,‘ThirdReportontheHumanRightsSituationinColombia’(26February1999);butcontrastwithPantojaOrdoñezandorsvColombia(LasPalmerasvColombia),CommNo11.237(4February2000).AsfortheECHR,seeECtHR,GeorgiavRussia(II)(n232)paras200ff,andcontrastwithHassanvtheUnitedKingdom,AppNo29750/09(16September2014)paras100ff.234Sassoli(n227)397ff(suggestinga‘nuancedandcase-by-case’approach).235‘FinalReportofthePanelofExpertsonLibyaestablishedpursuanttoSecurityCouncilresolution1973(2011)’(8March2021)S/2021/229paras63–4.SeeZKallenborn,‘WasaFlyingKillerRobotUsedinLibya?QuitePossibly’(TheBulletin,20May2021)https://thebulletin.org/2021/05/was-a-flying-killer-robot-used-in-libya-quite-possibly/.DiegoMauri-9781802207675DownloadedfromPubFactoryat05/28/202203:48:40AMviaDukeUniversityLawLibrary

124116AutonomousweaponssystemsandtheprotectionofthehumanpersonIfconfirmed,thiswouldrepresentthefirstuseofanAWSinactualcombatscenarios.GiventhatTurkeyispartytotheECHR,victimsofthestrikecouldevenfileanapplicationtotheECtHRinvokingarticle2ECHR.Insuchcases,thetwoissuesdepictedaboveareboundtoarise.Thehumanrightsmonitoringbody(inourexample,theECtHR)willneedtoestablish,first,whethertheimpugnedconductfallswithinthe‘jurisdiction’ofthestate.Withregardtothis,ouranalysisherecanbebutbriefandfuture-orientated(giventhescarcityofpractice).However,itisworthnotingthat,whiletheECtHRhasrecentlyaffirmedthatactscommittedduringthe‘activephaseofhostilities’(thatis,withina‘contextofchaos’)wouldnottriggerthejurisdictionallinkasrequiredbyarticle1ECHR(suchasarmedconfrontationonthefieldbetweentwoarmies),‘isolatedandspecificactsinvolvinganelementofproximity’coulddo.236Thisdistinctionservedtoestablishjurisdictionvis-à-visatargetedkillingcommittedbystateagentsactingabroad,outsidethecontextofmilitaryoperations.237Providedthatjurisdictionarises,thenthemonitoringbodywillhavetoapplyrelevantprovisions(firstandforemost,therighttolife).TheinterplaybetweenIHRLandIHLwillthencomeintoplay:theclassificationofaconflictasanIACoraNIACmayresultinadifferentgraduationofappli-cablenorms,alsodependingonthebodyathand.Whatemergesfromouranalysisthusfaristhatboththeclassificationofarmedconflicts(withthesubsequentissueoftheinterplaybetweenIHRLandIHL)andtheissueofjurisdictionincasesofextraterritorialusesofforceagainstindividualsraiseimportantchallengesfortheeffectivenessofIHRL—yetirrespectiveofthetypeofweaponused.Thatis,thecircumstancethatAWSmayselectandengageindividualswithouthumaninterventionisnotdecisiveforthispurpose.Instead,itisamatterofwhichbodyoflawappliestoaparticularcase,aquestionthathastobeansweredbyassessingwhetherrequirementsimposedbyIHLfortheexistenceofIACorNIACaresatisfied,andwhetherandtowhatdegreeIHRLinstrumentsapply.Thatis,wecanremain‘agnostic’astowhichbranchofinternationallawregulatesagivenscenario;whatmattersinsteadistodemonstratetowhatextentAWScanbedevelopedinkeepingwithIHRLandIHL.3.7CONCLUDINGREMARKSAutonomousweaponssystemsareexpectedtobringaboutarevolutioninmil-itaryaffairs;however,theirimpactonlawenforcementoperationsandothercontextsinwhichIHRListhemainapplicablelegalframeworkshouldnotbe236GeorgiavRussia(n231)paras132ff.237CartervRussia,AppNo20914/07(21September2021)para130.DiegoMauri-9781802207675DownloadedfromPubFactoryat05/28/202203:48:40AMviaDukeUniversityLawLibrary

125Autonomousweaponssystemsunderinternationalhumanrightslaw117underestimated.Anumberofbasichumanrights,rangingfromtherighttolifeandtherightnottobesubjecttotorture,inhuman,ordegradingtreatment,totherighttoprivacyandnottobediscriminatedagainst,willbeatstakeonceAWSarefielded.Theseweaponssystemsarelikelytobeagamechangeralsooutsidehostilities.TheanalysisconductedinthischapterhasshownthatexistingIHRLrulescan,andthusmust,beappliedtoAWS.Deprivationsoflifearetoberegardedasunlawfulwhen‘arbitrary’:initscurrentunderstanding,therequirementofnon-arbitrarinessimposesaspecificsetof(bothnegativeandpositive)dutiesonlaw-enforcementauthorities.Initsmostessentialform,adeprivationoflifeisarbitrarywhennohumanagentcanunderstandandexplainhowandwhyaparticularcourseofactionhasoccurred.ThesamerationaleholdsforAWS.Thus,whileitcannotbearguedthatexistingrulesrequirehumaninterventionintheselectionandengagementofaparticularhumantarget,theysurelycanbeinterpretedsoastoimposeastandardof‘humancontrol’thatensureexanteandexpost‘legibility’oftheweaponssystems.Admittedly,thisrequirementsetsthebarhighforusingAWSinlaw-enforcementscenarios,whereresortto(lethal)forceissubjecttoaplethoraofconstraints.Ultimately,however,wedemonstratedthatAWSarenotinconsistentwithIHRLrulesperse.OuranalysishasthenturnedfromrulestoprinciplesofIHRL,namely,tohumandignity.Itscharacteras‘foundationalprinciple’ofthebodyofinter-nationallawunderscrutiny(and,accordingtosome,oftheinternationallegalsystemconsideredasawhole)makesitasuperbreferenceforthedebateonAWS.Ithasbeendemonstratedthattheprincipleofhumandignitystandsasa‘generalprinciple’ofIHRL,morepreciselyasaprincipleformedwithintheinternationallegalsystemthatunderliesgeneralrulesoftreatyandcustomarylaw.However,inspiteofitsnormativepotential,theprincipleismostlyleftasidebyactorsparticipatinginthedebateonAWS.Thisdoesnotcomeasasurprisewhenpro-AWSstatesandotheractorsareconsidered;yet,thosewhosupportamoreskepticalpositiontowardsAWSdostrugglewiththeprincipleofhumandignity,whichisofteninvokedonaparwithethicalimperatives(thusnotasalegalstandard)orwithoutsufficientautonomyfromparticularrulescomingtothefore(forexample,therighttolife).Currentreferencestovaguenotionsofhumandignity,althoughprincipled,standmoreasdelegeferendaauspicesthanasastatementofexistinglaw:theyhaveanoptative(thatis,expressingasubjectivedesire)meaning,notanindicative(thatis,describinganactualcircumstance)meaning.HavingconcludedtheanalysisofAWSunderIHRL,thatisthelexgeneralisestablishingasetofrightspertainingtotheindividualassuch,andhavingtoucheduponpracticalscenariosofparallelapplicabilityofIHRLandIHL,ourattentionmustnowturntothelatter,quaspecificbranchofinternationallawregulatingtheuseofforceinhostilities.DiegoMauri-9781802207675DownloadedfromPubFactoryat05/28/202203:48:40AMviaDukeUniversityLawLibrary

1264.Autonomousweaponssystemsunderinternationalhumanitarianlaw4.1AWSFORARMED-CONFLICTSCENARIOS:INSIGHTSFROMCURRENTANDFUTUREBATTLESPACESMostscholarlyworksonAWSaredevotedtotheirimpactonIHLnorms.Itiseasytoseewhy:empirically,theprimeapplicationsofAWS-relatedtech-nologywilloccurinmilitaryoperations.Autonomoustechnologyhasbeenblossominginthemilitaryfield,andthereseemstobelittleornochanceofitslowingdown:asoneUSArmyColonelputit,‘wearebuildingthebridgetothefuturewhilestandingonit’.1AccordingtoaformerUSAirForceChiefScientist,‘by2030machinecapabilitieswillhaveincreasedtothepointthathumanswillhavebecometheweakestcomponentinawidearrayofsystemsandprocesses’.2Increasedautonomyinthebattlefieldhasprovenakeytoolforensuringappropriateoperabilityandimprovedperformanceinawiderangeofactivities.ThischaptertestsAWSagainstthebranchofinternationallawstriv-ingtoprotectthehumanpersoninthespecificfieldofhostilities,namely,IHL.Inparallelwiththepreviouschapter’sstructure,thisopeningsectionprovidesageneraloverviewof(currentandfuture)militaryapplicationsofautonomoustechnology,withaviewtoshowingthereasonswhyIHLdominatesthedebateonAWS.Theadvantagesofautonomoustechnologyinwarfarehavebeenhigh-lightedbyanimpressivenumberofpolicypapers(orsimilar).Forinstance,a2016reportelaboratedbytheUSDefenseScienceBoardanddevotedtothecurrentstatusoftheart,existingchallengesandfuturedevelopmentsoftheoperationaluseofautonomyacrossallwarfightingdomainscontendsthatautonomy‘hasreacheda“tippingpoint”invalue’forthemilitary.3Pulling1QuotedinPSinger,WiredforWar.TheRoboticsRevolutionandConflictinthe21stCentury(PenguinBooks2009)19.2WDahm,‘ReportonTechnologyHorizons:AVisionforAirForceScience&TechnologyDuring2010–2030’(September2010)106.3USDoDDefenseScienceBoard,‘SummerStudyonAutonomy’(June2016)98.118DiegoMauri-9781802207675DownloadedfromPubFactoryat05/28/202203:48:43AMviaDukeUniversityLawLibrary

127Autonomousweaponssystemsunderinternationalhumanitarianlaw119factorsare:(1)reductionofundesiredcasualtiesandadoptionofriskiertacticsceterisparibus;(2)rapidcollectionandelaborationofdatafromthefieldandenhancedqualityofdatalinksevenincontestedenvironments;and(3)increaseofoperationaltempoindecision-making,whichiskeyinscenariossuchascyberoperationsormissiledefense.4Expandingthelatter,speedandagilityarebelievedtooperateasdrivingforcesfornext-generationconflicts:weaponssystemsthatcanengageandrespondinsplit-secondscenarioscouldevenreversethetraditionaltempoofbattlespace.‘Linearspeed’(speedacrosstheground)and,moreimportantly,‘angularspeed’(theabilitytoout-thinkandanticipatetheenemy)arecoreadvantagesensuredbytheadventofAWS.5The2016USreport,however,isjustatrifle.ItwouldbemisleadingtoassumethattheUS—firstcountryinrankingformilitaryexpenditures—isaloneintheautonomousarmsrace:basedonpubliclyavailableinformation,RussiaandChinaaremakingtremendousprogressinthisfield.6OthermilitarypowerssuchastheUK,FranceandIsraelfollow.Turningnowtoexistingweaponsthatalreadyfeatureautonomouscapabili-ties,whichareexpectedtobeamplifiedbyadvancesinAI-relatedtechnologies,onemayconsidersensor-fusedandloiteringmunitions.Thosemunitionsarecommonlyreferredtoas‘fire-and-forget’ashumandecision-makingstartsatthemomentofdischargingthem,whilesubsequentactivityisentirelydepend-entonthesystemitself.OneexampleofthistechnologyistheIsraeliHarpy,ananti-radarweaponthatselectsandengagesradartargetsinaccordancewithpre-programmedtarget‘signatures’.7TheHarpy’sadvancedversionistheHarop:thelattercanselectandengageitstargetsatalongerdistance,eitherbyremotecontrolorautonomously.8Morecomplexandstructuredsystemsaremissile-androcket-defenseweapons,usedforairdefenseofshipsandground4Ibid11–12.5TKAdams,‘FutureWarfareandtheDeclineofHumanDecisionmaking’(2001)31TheUSArmyWarCollegeQuarterlyParameters57(analyzingthe‘ArmyAfterNext(AAN)forces’).6VKozyulin,‘Russia’sautomatedandautonomousweaponsandtheirconsidera-tionfromapolicystandpoint’,inICRC,‘AutonomousWeaponsSystems.ImplicationsofIncreasinglyAutonomyintheCriticalFunctionsofWeapons’,ExpertMeeting(March2016);EBKania,‘“AIWeapons”inChineseMilitaryInnovation’,GlobalChinaReport(April2020).7RCrootof,‘TheKillerRobotsAreHere:LegalandPolicyImplications’(2015)36CardozoLawReview1837.SeealsothelandmarkbookbyPScharre,ArmyofNone(WWNorton2018)andP.Scharre,‘Autonomy,“KillerRobots”andHumanControlintheUseofForce–PartI’(JustSecurity,9July2014).8PLBergenandJRowland,‘WorldofDrones:TheGlobalProliferationofDroneTechnology’,inPLBergenandDRothenberg(eds),DroneWars:TransformingConflict,LawandPolicy(CUP2015),301(‘[t]heHaropcancircleoveratargetforhoursbeforeitisactivatedandsenttothegroundasingle-usemissile’).DiegoMauri-9781802207675DownloadedfromPubFactoryat05/28/202203:48:43AMviaDukeUniversityLawLibrary

128120Autonomousweaponssystemsandtheprotectionofthehumanpersoninstallationsagainstincomingthreats(suchasartillery,mortars,andmannedorunmannedaircrafts).FamousexamplesofthistypeofsystemaretheIsraeliIronDome,amobileland-basedcounter-rocketsystemthatinterceptsincom-ingmissilesatimpressivespeed,theUSPhalanx(aship-basedgunsystem),andC-RAM(aland-basedversionofthePhalanx).Inparticular,bothPhalanxandC-RAMcanbeemployedundertheAegiscombatsystem,whichdis-posesoffouroperationalmodes,rangingfrom‘semiautomatic’to‘casualty’:whereasdecisionsonlethalforcearealwayscontrolledbyahumanoperatorinthefirstmode,thesecondpermitsthesystemtoexertlethalforceindependent-ly.9Importantlyforouranalysis,targetsalwaysremainunmannedmunitions:theseweaponswouldnotqualifyasAWSonthebasisofourdefinition.10ResearchinAI-relatedtechnologieswillbringthosecapabilitiestoahigherlevel.Artificialintelligenceisalreadyexpectedtohave‘amajorimpactonthefutureofautonomousvehicles’andtoprove‘disruptivein[its]militaryappli-cation’.11SystemsendowedwithadvancedAIcapabilitieswillallegedlyhave‘asimilarorgreatercapacitytothinklikeahuman’.12Advancesinmachinelearning,neuralnetworks,evolutionarycomputation,and(obviously)roboticsanticipateweaponssystemsthatarecapableofadaptingtotheoperationalenvironment:ifagentsmayacquire(morecorrectly,‘self-learn’)capabilitiessuchasexploration,obstacleavoidance,areacleaningandlandmarkidentifi-cation,ataparticularpointtheymayalsobecomecapableoftargetselectionandengagement.13Anotherfieldofapplicationthroughwhichtheprogressiveeclipseofhumaninterventionindecision-makingcanbeobservedisthatofswarmingtechniques.Swarmsconsistofdisparateelementsthatcoordinateandadapttheirmovementsanddecisionsthroughsimplesharedrulesinordertogiverisetoanemergentandcoherentwhole.14Currently,swarming9GEMarchant,BAllenby,RArkin,ETBarrettandJBorenstein,‘InternationalGovernanceofAutonomousMilitaryRobots’(2011)12ColumbiaScience&TechnologyLawReview272,287(describingthe‘casualty’setting,whichallowsthesystemtodo‘whatitthinksisnecessarytosavetheship’).10SeeCh2,s2.2.11UKMoD,‘JointDoctrineNote2/11:TheUKApproachtoUnmannedAircraftSystems’(JDN2/11,2011)para623.12JOMcGinnis,‘AcceleratingAI’(2010)104NorthwesternUniversityLR1253(arguingthatadvancedAIcapabilitieswillsoonoutpacehumanjudgmentsandevenprove‘acivilizingforceinwar’).13Marchantetal(n9).14PScharre,‘RoboticsontheBattlefieldPartII.TheComingSwarm’(CNAS,October2014)(underscoringtheimportanceofstudiesinnaturalswarms—suchasants,beesandtermites—whichexhibitextraordinarilycomplexbehaviorwhenaggre-gatedinswarms).Foramoretechnicalanalysis,seeILachow,‘TheUpsideandDownsideofSwarmingDrones’(2017)73BulletinoftheAtomicScientists96.DiegoMauri-9781802207675DownloadedfromPubFactoryat05/28/202203:48:43AMviaDukeUniversityLawLibrary

129Autonomousweaponssystemsunderinternationalhumanitarianlaw121techniquesareresearchedinthemaritimedomain(forexample,theDefenseAdvancedResearchProjectsAgency’s,DARPA’s,Hydraprogram),15aswellasintheaerialone.16Oneimportantdriveforswarmingtechnologyistheimportanceofmassintoday’swarfare,bothstrategicallyandeconomically.17Toconcludeouroverview,developmentsinAI-relatedtechnologiesandswarmingtechniqueswillsurelypavethewayforincreasinglyautonomousweaponry.18TheuseofAWSisexpectedtoimpactonnumerousIHLprovi-sions,asmayhappen,tonameonlyone,withregardtotherulesgoverningneutrality:canneutralstatesacquireAWSinordertorepealintrusionfrombelligerentpartiesduringhostilities?19Thesameappliestotheprotectionoftheenvironmentduringarmedconflict:IHLprovisionsestablishdutiesthatstatesmustabidebyirrespectiveoftheweaponemployed.ThosesetsofrulesapplyfullytoAWS.Inlinewiththequestionsdrivingthisbook,regardhastobegiventorulesandprinciplesprotectingthehumanpersonduringtheabhorrent—seeminglyunavoidable—experiencethatwarisforhumankind.Thereisnodoubtthatthisbranchofinternationallaw—historicallyreferredtoasjusinbelloorlawofarmedconflict(LOAC)—hasbeencharacterizedbyaprogressive‘humanization’ofitsrules,evenbeforetheadventofthehumanrightsmovement.20Thisprocessisreflectedinthewayinwhichitsrulesandprinciplesareinterpretedandappliedinconcreto:thischapterlooksatthesein15Hydraconsistsofadistributedunderseanetworkofunmannedpayloadsandplat-formtosupportmannedvesselsandexpandtheircapabilitiesabove,onandbelowtheocean’ssurface.Seehttps://www.darpa.mil/program/hydra.16AfamousexamplebeingtheUSNavy’sLow-CostUAVSwarmingTechnology(LOCUST),tobedeployedasbothdefensiveandoffensiveweapons.SeeDHambling,‘TheU.S.NavyPlanstoFoilMassive‘SuperSwarm’DroneAttacksbyUsingtheSwarm’sIntelligenceagainstItself’(Forbes,26August2020)https://www.forbes.com/sites/davidhambling/2020/08/26/how-us-navy-plans-to-foil-massive-super-swarm-drone-attacks/accessed31August2021.17Scharre,‘RoboticsontheBattlefieldPartII’(n14)18.18CJenks,‘TheGatheringSwarm:ThePathtoIncreasinglyAutonomousWeaponsSystems’(2017)57Jurimetrics341.SeealsoGCoppinandFLegras,‘AutonomySpectrumandPerformancePerceptionIssuesinSwarmSupervisoryControl’(2012)100ProceedingsoftheIEEE590,602(‘[e]xperimentswithhumanoperatorshaveshownthatalthoughthehumanhasapositiveroletoplayinthecontrolandsupervisionoftheautonomousagents,therepresentationgapbetweenthehumanandswarmintel-ligencecallsformoreadvancedHCI[Human-ComputerInterface]tools’).19PSeger,‘TheLawofNeutrality’,inAClaphamandPGaeta(eds),TheOxfordHandbookofInternationalLawinArmedConflict(OUP2014).20SeealsoTMeron,TheHumanizationofInternationalLaw(MartinusNijhoff2006).DiegoMauri-9781802207675DownloadedfromPubFactoryat05/28/202203:48:43AMviaDukeUniversityLawLibrary

130122AutonomousweaponssystemsandtheprotectionofthehumanpersonordertotestAWS.TheGuidingPrinciplesstressthecontinuingimportanceofIHLasthemainlegalframeworkthroughwhichAWSareregulated.21TheextenttowhichIHLrulesandprinciplesrequirehumancontroltobepresentwhenAWSperformcriticalfunctionsisassessedviaatwo-stepanalysis,inlinewiththemethodologicalapproachweadoptedinthepreviouschapter.First,normspertainingtothechoiceofweaponsinarmedconflict—attimesreferredtoasweaponslaw—areillustrated(section4.2),beforeturningtothenormsregardingthewaytheseweaponscouldbeemployedinhostilities—‘targetinglaw’(4.3).RulesregardingthedutytoinvestigateandprosecuteIHLviolationsareanalyzed,alsowithreferencetoparalleldutiesunderIHRL(4.4).ThedichotomybetweenthesetwobodiesofIHLiswell-knowninrelevantscholarship,andprovidesacomprehensiveoverviewonstates’obligationswhendevelopinganddeployingnewweapons.Second,andfollowingasimilarpatterntothatofthepreviouschapter,attentionispaidtoacornerstoneprincipleofIHL,namely,humanity,asenshrinedintheMartensClause(4.5).TouchpointswithcorrespondingobligationsunderIHRLarehighlightedwhereappropriate.Concludingobservationsensue(4.6).4.2RULESONTHEUSEOFFORCEAGAINSTTHEHUMANPERSON:WEAPONSLAWThereareplentyofIHLnormsdevotedtoregulatingtheuseofweaponsintheconductofhostilities.22However,andperhapssurprisingly,thereisnoagreeddefinitionoftheterm‘weapon’ininternationallaw:whileintuitivelyitcanbedescribedasaninstrumentdesignedorusedforinflictingharmordamage,whetheroffensivelyordefensively,IHLprovidesnouniversaldefinition.ItisforthisreasonthattheICRChasclaimedthatanyguidanceistobesoughtwithindomesticlegalorders,andnot‘acrosstheinternationalcommunity’.23Thevariousdefinitionsprovidedbymoststatestendtocoincideinthattheynotetwofeatures:(1)thecapabilityofdirectlycausingharmaskeycompo-21GroupofGovernmentalExpertsoftheHighContractingPartiestotheConventiononProhibitionsorRestrictionsontheUseofCertainConventionalWeaponsWhichMayBeDeemedtoBeExcessivelyInjuriousortoHaveIndiscriminateEffects,‘Reportofthe2019sessionoftheGroupofGovernmentalExpertsonEmergingTechnologiesintheAreaofLethalAutonomousWeaponsSystems(LAWS)’(25September2019)CCW/GGE.1/2019/3,AnnexIV(GuidingPrinciples)principle(a).22SHaines,‘TheDevelopingLawofWeapons.Humanity,DistinctionandPrecautionsinAttack’,inAClaphamandPGaeta(eds),TheOxfordHandbookofInternationalLawinArmedConflict(OUP2014).23SeeICRC,‘AGuidetotheLegalReviewofNewWeapons,MeansandMethodsofWarfare:MeasurestoImplementArticle36ofAdditionalProtocolIof1977’(Geneva2006),47.DiegoMauri-9781802207675DownloadedfromPubFactoryat05/28/202203:48:43AMviaDukeUniversityLawLibrary

131Autonomousweaponssystemsunderinternationalhumanitarianlaw123nent;and(2)theconceptionofweaponsasinstruments(‘objects’)inthehandsoftheirmasters(‘subjects’).Article36ofAdditionalProtocolItotheGenevaConventions(hereafterAPI)—oneofIHL’skeyprovisionsestablishingthedutytocarryoutlegalreviewofnewarmamentsandtechniquestoassesstheircompliancewithinternationallaw—distinguishesamong‘weapons’,‘means’and‘methodsofwarfare’.24Thefirsttwocategoriesdescribeatoolinitsownstructure,whilethethirdrelatestothemannerinwhichaweaponorameansofwarfareisusedinconflict.25‘Weapons’and‘meansofwarfare’relatetothepreliminaryquestion‘isthetoolitselflawful?’,while‘methodsofwarfare’relatetothesubsequentquestion‘isthewayinwhichthattoolisemployedlawful?’.26Thelatterdistinctionishelpfulinthatitshedslightonakeyfeatureofweaponslaw.Thesenormsaimtoprotectpermissibletargets,thatis,militaryobjectives—combatants,inthefirstplace.Theircharacteristrulyhumani-tarian:evenifaparticularindividualcanbetargetedbyvirtueoftheirstatuswithintheconflict,theycannotbetargetedbywhatevermeans.27Limitationsexist.ThisprinciplereflectstheprofoundideathatinspiredHenryDunant,thefounderoftheRedCrossmovement:eveninthemidstofwar,atypicalhumansentiment—charity—shouldbepreservedandpromoted.28Thesameideaisdiscernibleintargetinglawand,moregenerally,inthemodernedificeofIHL:inthisinstance,adegreeofprotectionisensuredalsotothosewhoarenotsparedbyviolenceinbattlefield.29Beforeturningtotheanalysisofspecificweaponslaw’srules,itmustbeascertainedthatthissetofnormsappliestoAWS.Theyappearslightlydifferentfromstrictosensu‘weapons’:theyare‘weaponssystems’,thatisplatformscombiningoneormoreweapon(s)andtheitemsassociatedwithits24Seeinfras4.2.4.25Haines(n22)276.26Ibid277(emphasesadded)(usingtheexampleofwhitephosphoroustoshowthatwhilelawfulasameansofwarfaredesignedforilluminationorapplyingheattosomething,itmayturnunlawfulifdirectlyemployedagainstcombatants,soasamethodofwarfare).27Meron,TheHumanizationofInternationalLaw(n20)61ff.28TheICRC’smottowas,andstillis.‘InterArmaCaritas’.29GOberleitner,‘HumanitarianLawasaSourceofHumanRightsLaw’,inDShelton(ed),TheOxfordHandbookofInternationalHumanRightsLaw(OUP2013),286(arguingthat‘Dunant’smotivationtoassistwarvictimswasbothdeeplyhuman-itarianandpracticalatthesametime.…Christianhumanismandapracticalsenseforsocialchangesufficedtocreate[the‘Genevalaw’]anditspracticalarrangements,bringingthefateofindividualsintotreatylaw’).DiegoMauri-9781802207675DownloadedfromPubFactoryat05/28/202203:48:43AMviaDukeUniversityLawLibrary

132124Autonomousweaponssystemsandtheprotectionofthehumanpersonemployment.30Thereisthusanoteworthydistinctiontobemade:atoneendistheweapon(forinstance,amachinegun),attheotheristhedeliverysystem(theplatform),bothbeinginter-independent.Forthesakeofclarity,considerthePredator,a27-footunmannedaerialvehiclethatmeritoriouslygainedthetitleof‘queenofAmerica’saerialdroneprogram’.31Originallyemployedforintelligence,surveillanceandreconnaissance(ISR)purposes,itwasthenequippedwithlaser-guidedHellfiremissilesforlethalaction.ThePredatorfeaturesaweapon(themissile)andadeliveryplatform(thevehicle),whichisremotelypilotedbyahumanoperator:the‘weapon’or‘meansofwarfare’istheformer,whereasthelatterinitselfisonlyinstrumentalforthedeliveryoflethalforce.Thus,ifoneacceptsthenotionof‘weapon’asa‘tool’intheuser’shands,itisimpliedthatthe‘tool’includesalsotheautonomousdeliverysystemasawhole:inscenarios,‘weapon’isanotionthatcoversalso‘weaponssystems’.ItfollowsthatweaponslawisapplicabletoAWS.Weaponslawdealswiththelimitationintheuseofweaponsinconflict.Thisidea,althoughancientinhistoryandinspiringthewholebodyofIHL,32gainedmomentumin1868,whentheSaintPetersburgDeclarationonexplo-siveprojectileswasadopted.33TheDeclarationwassignedandratified(oraccededto)by20states,andrepresentedanhistoricalattempttolimittheuseofweaponsbetweenbelligerentstates.Smallexplosiverifleprojectileshadproventheirworthagainstobjects,butwhenusedagainstcombatantstheycausedheavierinjuriesthanothertypesofbullet,equallyeffectiveindisablingtheenemy.34Inparticular,twofoundationalpointsmustbehighlighted.First,inthetextoftheDeclarationitisemphaticallystatedthat‘theprogressofcivi-lizationshouldhavetheeffectofalleviatingasmuchaspossiblethecalamitiesofwar’,thusequatingWesternprogresswithhumanizationofwarfare(tobereachedthroughtheprohibitionofde-humanizingtechnology).Second,statesacknowledgedthatacertaindegreeofopennesstotechnologicaladvance-30See,forinstance,‘Weaponsystem’,inUSDoD,DoDDictionaryofMilitaryandAssociatedTerms(January2021)230.31WCMarraandSKMcNeil,‘Understanding“theLoop”:RegulatingtheNextGenerationofWarMachines’(2013)36HarvardJofL&PublicPolicy1139.32HMcCoubrey,InternationalHumanitarianLaw.ModernDevelopmentintheLimitationofWarfare(2nded,Routledge2019);AAlexander,‘AShortHistoryofInternationalHumanitarianLaw’(2015)26EJIL109;MEO’Connell,‘HistoricalDevelopmentandLegalBasis’,inDFleck(ed),TheHandbookofInternationalHumanitarianLaw(2nded,OUP2013).33DeclarationRenouncingtheUse,inTimeofWar,ofExplosiveProjectilesUnder400GrammesWeight(signed29November/11December1868)138CTS297(StPetersburgDeclaration).34SeeFKalshovenandLZegveld,ConstraintsontheWagingofWar.AnIntroductiontoInternationalHumanitarianLaw(CUP2011),9–10.DiegoMauri-9781802207675DownloadedfromPubFactoryat05/28/202203:48:43AMviaDukeUniversityLawLibrary

133Autonomousweaponssystemsunderinternationalhumanitarianlaw125mentshadtobeensured:‘futureimprovementswhichsciencemayeffectinthearmamentoftroops’willrequirenewdeclarations,upgrades,modifications,inordertorenderthathumanizationeffective.35Subsequentnormativedevelopmentsconfirmedthisidea.IntheframeworkofthetwoHaguePeaceConferencesof1899and1907,numerousinterna-tionalinstrumentswereadoptedtoestablishfurtherlimitationsinthechoiceofweapons.Atthe1899Conference,forinstance,aDeclarationonasphyxiatinggases36andaDeclarationonexpanding(orDum-Dum,afterthenameoftheIndianarsenalwherethisweaponryhadbeenfirstproduced)bullets37wereadopted.Ofparamountimportancewasthe1907HagueConventionIVonwaronlandwithitsannexedregulations:38here,forthefirsttime,therulewasestablishedaccordingtowhich‘therightofbelligerentstoadoptmeansofinjuringtheenemyisnotunlimited’(article22,emphasisadded).Thisrule—whichisrepeatedverbatiminarticle35ofAPI—canrightfullybetakenasthebedrockoftheentireweaponslaw,tothepointthat,accordingtosome,itcanberegardedasageneralprincipleofIHL.39Bothgeneralandspecificprohibitionsonweaponsarecontainedinweaponslaw.Leavingasidespecificprohibitions(whichareonlyconsideredwhenrelevantforthepurposesofderivingprinciplesapplicabletoAWS),itistimetoturntogeneralprohibitionsinordertoidentifyrulesthatmayorientstatesinthedevelopmentandeventuallydeploymentofAWS.InalandmarkAdvisoryOpinionrenderedintheNuclearWeaponscase,theInternationalCourtofJustice(ICJ)fixedthetwo‘cardinalprinciples…constitutingthefabricof[IHL]’.40Thefirst,stemmingfromtheprincipleofdiscrimination,includestheprohibitiontodirectlytargetciviliansandcivilianobjects,41andtheprohibition35(Emphasisadded.)TheSaintPetersburgDeclarationitselfprovidesatellingexample:thecriterionof400gramsofweight,initiallyintendedtodistinguishinfantry-bulletsfromartillery-ammunitionandtooutlawonlytheformer,waslaterabandoned,oncetechnicalchangerenderedthedistinctionimmaterial.36Declaration(IV,2)concerningAsphyxiatingGases(signed29July1899)187CTS435.37Declaration(IV,3)concerningExpandingBullets(signed29July1899)187CTS459.38Convention(IV)respectingtheLawsandCustomsofWaronLandanditsAnnex:RegulationsconcerningtheLawsandCustomsofWaronLand(openedforsig-nature18October1907,enteredintoforce26January2010)187CTS227.39SCasey-Maslen,‘Weapons’,inBSaulandDAkande(eds),TheOxfordGuidetoInternationalHumanitarianLaw(OUP2020)263.40LegalityoftheThreatofUseofNuclearWeapons(AdvisoryOpinion)[1996]ICJRep66para78.41ProtocolAdditionaltotheGenevaConventionsof12August1949,andrelat-ingtotheProtectionofVictimsofInternationalArmedConflicts(adopted8June1977,enteredintoforce7December1978)1125UNTS3(API)art48.SeealsoJMDiegoMauri-9781802207675DownloadedfromPubFactoryat05/28/202203:48:43AMviaDukeUniversityLawLibrary

134126Autonomousweaponssystemsandtheprotectionofthehumanpersononusingindiscriminateweapons,namely,those‘thatareincapableofdistin-guishingbetweencivilianandmilitarytargets’.42Thesecondistheprinciplebyvirtueofwhichitisprohibitedtouseweaponsthatcauselegitimatetargetsunnecessarysufferingorthatuselesslyaggravatetheirsuffering(alsoknownasthesuperfluousinjuryandunnecessarysufferingrule—‘SIrUSrule’—intheICRC’sterminology),43orthatcause‘aharmgreaterthanthatunavoidabletoachievelegitimatemilitaryobjectives’.44Asisevident,alltheserulesofweaponslawenshrinetheideaoflimita-tionasillustratedabove,andhavetobereadinconjunctionwithotherkeyprinciplesofIHLapplicabletotargeting,namely,distinction,proportionalityandprecautionsinattack.Theirimportanceisfurtherstressedastheviolationofthesetworules,whencommittedinIAC,givesrisetoindividualcriminalresponsibilitypursuanttoICL.45Rulesprotectingspecificallythenaturalenvironment—whereindividuals,bothciviliansandcombatants,areboundtolive—canbeincludedtoo,albeitwithlittlerelevanceforthepurposeofouranalysis.Finally,ageneralruleregardingthedevelopmentanddeploymentofweaponsimposesonstatesadutytocarryoutalegalreviewoftheweaponathand:thisruleiscontainedinarticle36ofAPI.Letusnowtacklethosegeneralrulesinturn.4.2.1TheProhibitiononWeaponsCausingUnnecessarySufferingandSuperfluousInjuryArticles23(e)ofHagueConventionIVand35(2)APIprohibittheuseofweaponsthatcauseSIrUS(mauxsuperflus).Itisarulethat,inadditiontobeingenshrinedinmostweaponslawtreaties,hasgainedcustomarystatus.46ThebroadacceptanceofthisruleiscounterbalancedbydivergingviewsonitsHenckaertsandLDoswald-Beck,CustomaryInternationalHumanitarianLaw(2vols)(CUP2005)(CIHL)rule1.42APIart35(2);CIHLrule71.43Convention(IV)art23(2);API,art35(2).AsfortheICRC,seeRCouplandandPHerby,‘ReviewoftheLegalityofWeapons:ANewApproach.TheSIrUSProject’(1999)81IRRC583.44LegalityoftheThreatofUseofNuclearWeapons(n40)para78.45RomeStatuteoftheInternationalCriminalCourt(adopted17July1998,enteredintoforce1July2002)2187UNTS3(ICCSt)art8(2)(b)(xx).46ThecustomarynatureoftheprohibitionwasalsoaffirmedbytheInternationalMilitaryTribunal:seeNurembergjudgment,FranceandOthersvGöring(Hermann)andOthers,JudgmentandSentence,[1946]22IMT203,1October1946.SeeYSandoz,CSwinarskiandBZimmermann(eds),CommentaryontheAdditionalProtocolsof8June1977totheGenevaConventionsof12August1949(MartinusNijhoff1987)(APICommentary)para1419.DiegoMauri-9781802207675DownloadedfromPubFactoryat05/28/202203:48:43AMviaDukeUniversityLawLibrary

135Autonomousweaponssystemsunderinternationalhumanitarianlaw127actualscope:ademinimiscontentisfoundinthat‘aharmgreaterthanthatunavoidabletoachievelegitimatemilitaryobjectives’issurelyproscribed.47InordertotestthelimitsofthisruleandthusthecompatibilityofAWSwithit,fouraspectsareofparticularimportance.First,theSIrUSruleisultimatelybuiltupontheconceptofdisproportion.Intuitively,ifsuperfluousandunnecessaryharmisbanned,itisimpliedthattherecouldbesuchthingas‘necessaryharm’,whichwouldbeallowedconsequently.Thatis,themerefactofcausingharmtotheenemydoesnotpersefallwithintheprohibition—itdoeswhenitexceedsathreshold.Thisthresholdcanbetracedbacktomilitarynecessity:ifSIrUSdonotserveanymilitarypurpose,weaponscausingthemareunlawful.However,militarynecessityisanotiontobehandledwithextremecare,lestanyemploymentofviolenceonbattlefieldisjustifiedforitsmilitary-politicalpurposes,andthewholeideaofIHLaslimitationsimplyfadesaway.48Thefactorthatcancounterbalance,andrestrain,militarynecessityisthehumanitarianpurposeofIHLrules,whichrespondtotheprincipleofhumanity(towhichwereturnlater).49Thecombinationofthesetwoopposingforcesresultsinathresholdofharmthatweaponsareallowedtolawfullycauseinarmedconflicts,ofbothaninternationalandanon-internationalnature.50Somedomesticmilitary47LegalityoftheThreatofUseofNuclearWeapons(n40)para78;CIHL,rule70.SeealsoACassese,‘WeaponsCausingUnnecessarySuffering:AreTheyProhibited?’(1975)58RivistaItalianadiDirittoInternazionale12(commentingonart23(e)ofHagueConventionIV).48PJCameron,‘TheLimitationsonMethodsandMeansofWarfare’(1985)9AustralianYearbookofInternationalLaw241,256(claimingthateventhoughvio-lenceispartandparcelofwarfare,excessiveviolencetransformsitinsavagery,whichiswhatmodernIHLaimstoavoid);GSchwarzenberger,‘TheLawofArmedConflict:ACivilizedInterlude?’(1974)28YbofWorldAffairs293(showingskepticismontherealpossibilityforhumanitarianpurposestorestrain‘thesupremacyofforceintimeofwar’).SeealsoHostageCase,UnitedStatesvList(Wilhelm)andOthers,TrialJudgment,CaseNo7[1948]15ILR632(19February1948)(‘militarynecessityorexpediencydonotjustifyaviolationofpositiverules’).49NSitaropoulos,‘WeaponsandSuperfluousInjuryorUnnecessarySufferinginInternationalHumanitarianLaw:HumanPaininTimeofWarandtheLimitsofLaw’(2001)54RevueHelleniquedeDroitInternational71,76.50ProtocolAdditionaltotheGenevaConventionsof12August1949,andRelatingtotheProtectionofVictimsofNon-InternationalArmedConflicts(adopted8June1977,enteredintoforce7December1978)1125UNTS609(APII)art3(3)(c)(‘con-creteanddirectmilitaryadvantageanticipated’).DiegoMauri-9781802207675DownloadedfromPubFactoryat05/28/202203:48:43AMviaDukeUniversityLawLibrary

136128Autonomousweaponssystemsandtheprotectionofthehumanpersonmanualsendorsethisapproach,51asdoestheICJ.52Conversely,accordingtoothers,thetestfordeterminingwhetheraweaponcausesSIrUSshouldconsistinacomparisonwithweaponsalreadytheobjectofaspecificban:iftheireffectsareanalogous,itfollowsthattheformershouldbeprohibitedperrelationem.53This‘osmotic’prohibitionwasdefended,interalia,byJapaninaprotestlodgedin1945againsttheUSforthenuclearbombingsofHiroshimaandNagasaki.54Yet,thelaterjudgmentdeliveredbytheDistrictCourtofTokyointheShimodacaseseemstodepartfromthisapproach,attachingmoreimportancetotheproportionalitycriterionasconstruedabove.55Second,theSIrUSrulerelatestotheinevitabilityofdeathorseriousperma-nentdisabilityasaconsequenceofaparticularuseofforce.Ifaweaponisofanaturethatrendersdeathorlastingincapacitytheinescapableconsequenceofitsuse,thenitisunlawful.56Deathorincapacitymustbeanaccidentaleffectoftheweapon,andnotitsessentialeffect.57Throughthelensofthissecondaspect,onecanseebetterthereasonwhypoisonandexpandingbullets,aswellasblindinglasersandanti-personnellandmines,havebeenalloutlawed.5851USDoD,LawofWarManual(June2015,updatedDecember2016)para6.6.3(arguingthataweapon‘isonlyprohibitedbythesuperfluousinjuryruleifthesuffer-ingitinflictsisclearlydisproportionatetoitsmilitaryutility’andnotingthat,becauseofthedifficultyofcomparingthosetwovalues,prohibitionisestablishedonlyincasesofmanifestdisproportion).52LegalityoftheThreatofUseofNuclearWeapons(n40)para78.53CIHLrule70,242–3(citingtheMilitaryManualsofArgentinaandtheUS).SeealsoUS,‘MemorandumofLawonSniperuseofOpen-TipAmmunitions’(12October1990)(‘[i]nconductingthebalancingtestnecessarytodetermineaweapon’slegality,theeffectsofaweaponcannotbeviewedinisolation.Theymustbeexaminedagainstcomparableweaponsinuseonthemodernbattlefield’).54Cassese,‘WeaponsCausingUnnecessarySuffering’(n47)30(recallingthestatementofJapan,accordingtowhich‘thebomb[…]farexceeds,initsindiscrimi-nateperformanceanditsatrociouscharacter,poisonousgasesandotherweaponswhichhithertohavebeenbannedbecausetheypossesstheseperformances’).55RyuichiShimodaetalvtheState,DistrictCourtofTokyo,7December1963,(1964)32ILR626;seealsoSitaropoulos(n49)81.56SaintPetersburgDeclarationpreamble.SeealsoTheProceedingsoftheHagueConference,TranslationoftheOfficialTexts,TheConferenceof1899(OUP1920),283(fortheremarksoftheRussianDelegatetothe1899HagueConference,whoarguedthat‘[a]sphyxiatinggases[…]wouldexterminatethewholecrew[ofavessel].Thisprocedurewouldthereforebecontrarytothehumaneideawhichoughttoguideus,namely,thatoffindingmeansofputtingenemiesoutofactionwithoutputtingthemoutoftheworld’).57Cassese,‘WeaponsCausingUnnecessarySuffering’(n30)18(‘aweaponislegitimateif,bystrikingtheadversary,itcaneitherkillorwoundhim,dependingonthecircumstances.Bycontrast,itisnotinkeepingwithinternationallawifitalwaysresultsinkillingallpersonswhoinsomewayhappentobestruckbyit’).58CIHLrule70.DiegoMauri-9781802207675DownloadedfromPubFactoryat05/28/202203:48:43AMviaDukeUniversityLawLibrary

137Autonomousweaponssystemsunderinternationalhumanitarianlaw129Athirdaspectistheavailabilityofalternativemeansofwarfare,militaryadvantagebeingequal:onthisconditionstatesshouldresorttoless-injuringweapons.Whetherresorttoleastharmfulweaponscorrespondstoalegalobli-gation,however,isopenfordiscussion.Onthisthereislittletonostateprac-tice;inaddition,someauthorshavearguedthata‘leastharmfulmeans’ruleinthatcommandersareobligedtoadoptthesemeansinlieuofotherswhichtheystillperceivemoreefficientdoesnotexistassuchinIHL.59Fourth,SIrUShastobeassessedagainstthenatureoftheweapon:60itfollowsthatwhatmattersisthedesignofaweapon,itsnormaluse,andnotanypossibleusethatcanbemadeofsuchweapon.Forinstance,evenarifle,thatis,aweaponwhoselawfulnessisundisputed,canbeusedtohitanenemycombatantlegbyleg,lettinghimdieslowlyorsoastopermanentlydisablehim.Thisisnot,however,theresultoftherifle’sdesign(normaluse),butofaparticularlycruelemploymentoftheweapon.Tosumup,theuseofweaponsthatcauseSIrUSisprohibitedunderweaponslawaslongas:(1)SIrUSaredisproportionatewithrespecttomilitarynecessity;(2)deathandpermanentdisabilityaretheinescapableconsequenceoftheiruse;and(3)arguably,lessharmfulalternativesareviable.Inordertomakethisthree-layerassessment,referenceismadetothenormaluseoftheweapon,accordingtoitsdesign.Owingtothegreatdifficultythatisencoun-teredincalculatingalltheabove-mentionedfactors,andtheinherentsubjec-tivityandcontext-relatednessofthisassessment,someauthorshavecometoconsidertheSIrUSruleasbeingpointless.61Inordertoachievetheresultofterminatingoravoidingtheuseofaparticularweapon,thebeststrategywouldbetoputaspecificbanonit,ratherthanrelyingonthegeneralprohibitionagainstweaponscausingSIrUS.62Whilethisseemsentirelyreasonable,todateAWSarenotregulatedbyanyspecificIHLinstrument;instead,theGuidingPrinciplesre-statetheapplicabilityofIHLandthereforeitsgeneralrulesonweaponslaw.Toarguethatowingtothelackofspecificprohibitionsitis59GSCorn,LRBlank,CJenksandETalbotJensen,‘BelligerentTargetingandtheInvalidityofaLeastHarmfulMeansRule’(2013)89Int’lLStud536.60CIHLrule70(‘[t]heuseofmeansandmethodsofwarfarewhichareofanaturetocausesuperfluousinjuryorunnecessarysufferingisprohibited’).61Cassese,‘WeaponsCausingUnnecessarySuffering’(n47)(makinganin-depthanalysisofrelevantpracticeandconcludingthat,inaccordancewiththegeneralinter-pretiveprincipleexpressedinthemaximutresmagisvaleatquampereat,theprinci-pleofSIrUScanmaintainavalue,albeitlimited,inexceptionalcases,whereabhorrentweaponsareatissue);Sitaropoulos(n49)81(commentingagainonstatepracticeandnotingthatnoweaponhaseverbeenbannedonlybyvirtueoftheprincipleofSIrUS).62ThisisalsotheviewofMeron,TheHumanizationofInternationalLaw(n20)72.DiegoMauri-9781802207675DownloadedfromPubFactoryat05/28/202203:48:43AMviaDukeUniversityLawLibrary

138130AutonomousweaponssystemsandtheprotectionofthehumanpersonimpossibletoassessAWSagainsttheSIrUSrulewouldbeequaltocontendingthatthisruledoesnotexistunderIHL.ThequestionthathastobeanswerediswhetherAWSareinherentlyinconsistentwiththeSIrUSrule.Theirsupportersarguethatthisisnotso,sincetheirsalientfeature—beingabletodeliverpotentiallylethalforceautonomously—‘hasnodirectbearing’ontheprohibition,addingthat‘[i]ndi-vidualsystemscouldbedevelopedthatwouldviolatethesenorms,butautono-mousweaponsystemsarenotprohibitedonthisbasisasacategory’.63Letusnowillustratethiskeypointbywayofanexample.IfAWSwereequippedwithgunsshootingexpandingbullets,thesewouldbeunlawful,notthedeliverysystem.Bycontrast,ifAWSwereequippedwithpermissibleweapons(forexample,bullets),thentheSIrUSrulewouldberespected.BearinginmindthatitisAWSasawhole,namelyascombining‘lethality’and‘autonomy’,thatreallymatters,onemayquestionwhetheritisthatamachinemayapplyforceindependentlyofahumanoperatorwhichrenderstheweaponunlawful,especiallyunderthefirstprongoftheSIrUSrulethatweaponsbeevaluatedonthebasisoftheirnature.Autonomousdecision-makingprocessesrunbyAIsystems,wherehumanoperatorsplaylittleornoroleintheperfor-manceofaspecificcriticalfunction,mayberegardedasalteringthebalancebetweentheexpectedmilitaryadvantageandtheharmcausedtothetarget.Isthe‘harm’provokedbyAWSqualitativelysuperiortoanymilitaryadvan-tage,byvirtueofthe(sole)factthatahumanoperatormaynotbeinvolvedintheparticulardecisiontoapplyforce?Tosomeauthors,toansweraffirma-tivelytothisquestionwouldrequireastretchoftheimagination;tothewriter,thequestioncouldbebetterformulatedoncetheroleof‘humanity’asageneralprincipleofIHLisexplained:weleavethisuntillater.Plus,whileintriguing,itisworthnotingthatthisunderstandingoftheruleisnotsupportedbystateandnon-stateactorsinvolvedinthedebateonAWS.TurningtotheotherrequirementsstemmingfromtheSIrUSrule,admittingthatitsthirdprong(thatis,thatavailablealternativesposinglesserrisksshouldbepreferred)imposesalegaldutyoncommanders(whichisdebatable),ifthereisnoalternative,equallyfeasibleandadvantageousmeans,AWScanbelawfullyemployed.Asforthesecondprong(notrenderingdeathorpermanentdisabilityinevitable),theprohibitionwouldcoverAWSprogrammedtoselectandattacktargetsbutpreventedfromcarryingoutacontext-relatedassessmentofthetarget.Inconclusion,forthepurposesoftheSirUSrule—asithasdevelopedthroughthedecadesandinlightofitscurrentunderstanding—theamount63MNSchmittandJSThurnher,‘“OutoftheLoop”:AutonomousWeaponSystemsandtheLawofArmedConflict’(2013)4HarvNat’lSecJ231,279(emphasisadded).DiegoMauri-9781802207675DownloadedfromPubFactoryat05/28/202203:48:43AMviaDukeUniversityLawLibrary

139Autonomousweaponssystemsunderinternationalhumanitarianlaw131ofhumancontroltobeensuredoncriticalfunctionshastobesufficientforensuringthattheharmcausedbytheemploymentofAWSdoesnotexceedtheexpectedmilitaryadvantage.Providedthatthisevaluationdependsontheammunitionmountedbytheweaponssystemathand,thattheactualdischargeofpowerisdeliberatedthroughanAI-drivenprocessdoesnotappeardecisive.4.2.2TheProhibitiononIndiscriminateWeaponsIfthefirstcoreprovisionofweaponslawdealswiththedegreeofharmthatcanbecausedtoahumantarget,theseconddealswiththewayweaponscanbedirectedtotheirtarget.Weaponslawprohibitsweaponsthatcannotdistinguishbetweenpermissibleandimpermissibletargets.Thisrule,whosefirstformulationdatesbacktothe1899HagueConference,64inspiredimportantIHLinstruments,suchasthe1907HagueConventiononnavalmines,65the1907Declarationonprojectilesandexplosivesdischargedfromballoons,66andthe1925GenevaProtocolagainstasphyxiatinggases.67Eventually,ithasbeencodifiedinmorerecentinstruments,amongwhichtheCCW,68theOttawaConvention69and,moreimportantly,article51(4)(b)and(c)API,anditcurrentlyenjoyscustomarystatus.70Theseprovisionsrespec-tivelyestablishthatmethodsormeansofwarfare‘whichcannotbedirectedataspecificmilitaryobjective’and‘theeffectsofwhichcannotbelimited’constitute‘indiscriminateattacks’andareconsequentlyprohibited.Although,systematically,theprovisionsasperarticle51(4)(b)and(c)areinsertedintoasectiondedicatedtotheprotectionofthe‘civilianpopulation’(andmaythus64ACassese,‘TheProhibitionofIndiscriminateMeansofWarfare’,inBVARöling,RAkkerman,PJVanKriekenandCOPannenborg(eds),DeclarationsonPrinciples:AQuestforUniversalPeace(MartinusNijhoff1977).65Convention(VIII)RelativetotheLayingofAutomaticSubmarineContactMines(adopted18October1907,enteredintoforce26January2010)205CTS331.66Declaration(XIV)ProhibitingtheDischargeofProjectilesandExplosivesfromBalloons(signed18October1907)205CTS403.67ProtocolfortheProhibitionoftheUseofAsphyxiating,PoisonousorOtherGases,andofBacteriologicalMethodsofWarfare(adopted17June1925,enteredintoforce9May1926)94LNTS65(GenevaGasProtocol).68Protocol(II)onProhibitionsorRestrictionsontheUseofMines,Booby-TrapsandOtherDevices(adopted10October1980,enteredintoforce2December1983)1342UNTS168(CCWProtocolII)art1(2).69ConventionontheProhibitionoftheUse,Stockpiling,ProductionandTransferofAnti-PersonnelMinesandontheirDestruction(adopted18September1997,enteredintoforce1March1999)2056UNTS211(OttawaConvention),preamble.70ForthecustomarystatusofthisruleatthetimeoftheadoptionofAPI,contrastWBoothby,WeaponsandtheLawofArmedConflict(OUP2009),75,andCassese,‘TheProhibition’(n64).DiegoMauri-9781802207675DownloadedfromPubFactoryat05/28/202203:48:43AMviaDukeUniversityLawLibrary

140132Autonomousweaponssystemsandtheprotectionofthehumanpersonbeconsideredaspartoftargetinglaw),itisimportanttorememberthattheaboveprohibitionsregardtheweaponitself,andnotanyparticularuseofit.71However,theprohibitiononindiscriminateweaponsandthatonindiscrimi-nateattacksaresocontiguousthattheycomplementeachother:accordingly,ourremarksinthecurrentsubsectionshouldbereadinconjunctionwiththatwhichisarguedlater.72Accordingtothefirstrule—prohibitiononweaponsthatcannotbedirectedtowardspecifictargets—afamousexampleofindiscriminateweaponsare‘blind’weapons.Forinstance,onecanrecalltheGermanV.1andV.2usedduringWorldWarII,whoseinaccuracyequatedtheiruse(especiallyagainsttheUK)tothedeliberatetargetingofcivilians.73Landminesandclustermuni-tionsaretellingexamplestoo,andforthisreasontherehasbeenapowerfulmovementinfavoroftheirabolition.74Turningnowtotherequirementofthelimitationofaweapon’seffects,thisrulehasmaterializedinspecificprohi-bitions.Thisisthecaseofbiologicalandchemicalweapons:inadditiontotheharmtheirusemaycause,factorsthateventuallyledtotheirprohibitionunderIHL(aswellasinotherbranchesofinternationallaw)weretheuncon-trollabilityandunpredictabilityoftheireffects.75Asfarasnuclearweaponsareconcerned,whileitisfarfromhardtomakethecasethatthesameappliestosuchdisruptiveweapons,(in)famouslytheICJwasunabletoaffirmtheirincompatibilitywithinternationallawintheNuclearWeaponscase.76AnalogouslytotheSIrUSrule,inordertoassesstheindiscriminatenessofaweapon,referencehastobemadetoitsnature,thatis,itsnormaluse,andnotanypossible(ab)usesthereof:forthisreason,itisarguedthatunguided71MNSchmitt,‘InternationalHumanitarianLawandtheConductofHostilities’,inBSaulandDAkande(eds),TheOxfordGuidetoInternationalHumanitarianLaw(OUP2020).ConfrontCIHL,Rule71,andCIHL,Rules11,12.SeealsoBoothby,WeaponsandtheLawofArmedConflict(n70)67(outliningthedistinctionbetweenprinciplesofTargetingLawandtheruleofindiscriminatenessforWeaponsLawpurposes).72Seeinfras4.3.1.73SeeCassese,‘TheProhibition’(n64)173.74SeeOttawaConvention,preamble;ConventiononClusterMunitions(adopted3December2008,enteredintoforce1August2010)2688UNTS39(CCM),preamble.75Casey-Maslen(n39)265–6.Foranoverviewonchemical,biological,radio-logicalandnuclearweaponsunderIHLseeDMauri,‘TheUseofCBRNWeaponsinArmedConflict’,inAdeGuttry,MFrulli,FCasolariandLPoli(eds),InternationalObligationsApplicabletoPrevention,Preparedness,ResponsetoandRecoveryfromChemical,BiologicalandRadio-Nuclear(CBRN)Events.TowardsanAll-HazardApproach(Brill,forthcoming).76LegalityoftheThreatofUseofNuclearWeapons(n40),SeparateOpinionofJudgeFleischhauerpara394,DeclarationofJudgeHerczeghpara395,andDeclarationofPresidentBedjaouipara396.DiegoMauri-9781802207675DownloadedfromPubFactoryat05/28/202203:48:43AMviaDukeUniversityLawLibrary

141Autonomousweaponssystemsunderinternationalhumanitarianlaw133projectilesdonotcontrastwiththeprohibitiononindiscriminateweapons,77whilethecontrarycasecouldbemadeforthefirst-generation‘Scud’missile,whichwouldmisstargetsbymorethan1kilometer.78Anothercommontraitisthat,whilegeneralandcomprehensiveinscope,thisrulehasneverledtoaparticularweaponbeingconsideredasprohibited;instead,statepracticeshowsthat,oncetheconvictionthataweaponisintolerablyindiscriminateissufficientlyspread,specifictreatyinstrumentsimposingabanonthatweaponareadopted.79Thatis,withoutdoubtingthatgeneralprohibitionsarelegallybinding,specificbansaremoreeffectiveinconcreto.ThislastremarkbringsustodiscussingAWSundertheruleprohibitingindiscriminateweapons.Itcanbearguedthatifaparticulardecisionresultinginthedeathorinjuryofthetargetedindividual(aswellasinthedestructionofanobject)istakenwithouthumanintervention,this‘doesnotprecludethemfrombeingdirectedatcombatantsandmilitaryobjectives,andneednotresultintheirhavingeffectsthatanattackercannotcontrol’.80Again,thepointisinsteadhowdiscriminatetheammunitionscarriedbythedeliveryplatformsare:ifAWSwereequippedwithbiologicalagentstobereleasedwhenthesystemdeemsitappropriate,thecasecouldbeeasilymadethatthoseweaponswouldbeindiscriminateandthusprohibited—granted,butonthebasisoftheammunitionthattheyhavecarriedanddispersed,notonthebasisthattheattackhasbeendeliberatedwithouthumanintervention.Human–machineinteractionmustbeadequateforensuringthattheweaponsystemoperatesinkeepingwiththeprohibitiononindiscriminateweapons;thatis,thatAWScanbedirectedatpermissibletargetsandthattheireffectscanbecontrolled.Focusingontherequirementofcontrolovertheeffectsofaweapon,ithasbeennotedthatlackofsuchcontrolwasthemaindriverofthebanonchemicalandbiologicalweaponsimmediatelyaftertheWorldWarI,withtheadoptionofthe1925GenevaGasProtocol.Thedegreeofhumancontrolthatthisrulerequireshastoensurethathumanoperatorsmayproperlyoverseetheweapon77ProsecutorvMartić,ICTY-95-11(12June2007)paras462–3(discussingtheusebySerbianforcesofanunguidedprojectilecontainingaclusterwarheadof288bomblets,eachbombletcontaining420pelletsof3mmindiameter,andconcludingthat,byvirtuebothofitsbeingunguidedanditsfiringatthemaximumrange,thatpar-ticularweaponhadtobeconsideredindiscriminateandthusprohibited).78Casey-Maslen(n39)265.79CIHLrule17;Cassese,‘TheProhibition’(n64)(arguingthat,ifitisundeniablethatweaponscausing‘indiscriminatesuffering’arebelievedtobecontrarytointerna-tionallawbymanystates,thereisnostatethathascometodiscontinuetheuseofsuchweaponsasaresultofallegationsbyotherstatesabouttheirindiscriminateness).80SchmittandThurnher(n63)279.DiegoMauri-9781802207675DownloadedfromPubFactoryat05/28/202203:48:43AMviaDukeUniversityLawLibrary

142134Autonomousweaponssystemsandtheprotectionofthehumanpersonsystemduringitsdeployment—initself,thisruledoesnotimposehumaninterventionintheperformanceofparticularcriticalfunctions.81SupportersofAWSclaimthattheirabilitytodiscriminatedepends,interalia,ontheefficiencyoftheiralgorithmsandtheenvironmenttheyarefieldedin.82Itfollowsthatiftechnologyisdevelopedthatensuresadequatecapabil-itiesofdiscriminatingandcontrollingtheeffectsofaweapon,thennothingcould(andshould)halttheadoptionofthatweapon.So,thedevelopmentofAWSendowedwithAI-relatedtechnologiesallowingtheirusertodirecttheweaponagainstspecifictargetsandtocontrolitseffectsmustbeconsideredconsistentwiththeruleunderconsideration.AsmostsupportersofAWSclaim,whatshouldbeaskedofthemachineisthatitperformatleastaswellashumanwarfighters.83Technologicaladvancesmayevenenhancecurrentcapabilities.Inclosing,itispossibletore-stateourconclusionsinlinewiththeSIrUSrule.Initself,thataparticularcourseofactionmaybedeliberatedabsenthumaninterventionhasnodirectbearingontheindiscriminatenatureofAWS.Theprotectionthatthisrule—aswellasothernormsofweaponslaw—ensurestoindividualsquapermissibletargetsofAWScannotbestretchedtoinferabanonthisweaponry.4.2.3TheProhibitiononWeaponsIntendedorExpectedtoCauseWidespread,Long-TermandSevereDamagetotheNaturalEnvironmentAthirdgeneralprohibitionhastobeadded,namely—asenshrinedinarticle35(3)APIandofcustomarynature—proscribingbothmethodsandmeansofwarfareintendedorexpectedtocause‘widespread,long-termandseveredamagetothenaturalenvironment’.84Article55APIextendsamoregeneraldutyofcaretowardstheenvironmentandproscribesreprisalsagainstit.Theseprovisionshavetobeaddedtothosecontainedinotherinternationallawinstrumentsdealing,directlyorindirectly,withtheprotectionoftheenviron-81MHomayounnejad,‘EnsuringFullyAutonomousWeaponsSystemsComplywiththeRuleofDistinctioninAttack’,inSCasey-Maslen,MHomayounnejad,HStaufferandNWeizmann(eds),DronesandOtherUnmannedWeaponsSystemsunderInternationalLaw(Brill2018),147.82MNSchmittandEWWidmar,‘“OnTarget”:PrecisionandBalanceintheContemporaryLawofTargeting’(2014)75JofNat’lSecL&Pol379,398–9(arguingthat‘theproperquestionisnotwhether[AWS]areunlawfulperse,butratherwhethertheiruseinaparticularenvironmentandcombatcontextwillmeetIHLrequirements’).83RCArkin,‘TheCaseforEthicalAutonomyinUnmannedSystems’(2010)9JofMilitaryEthics332.84APIart35(3);CIHLrule45.DiegoMauri-9781802207675DownloadedfromPubFactoryat05/28/202203:48:43AMviaDukeUniversityLawLibrary

143Autonomousweaponssystemsunderinternationalhumanitarianlaw135mentandwhich,eventhoughconceivedforpeacetime,donotceasetoapplyduringhostilities.85SeveralIHLinstrumentsarenowconcernedwiththeprotectionofthenaturalenvironmentduringhostilities(forinstancetheENMODConvention),86whichgivesanaccountoftheimportanceofenvironmentalprotectionasacommongoodininternationallawgenerally,andinarmedconflictspecifically.87AlsotheILChasdevoteditsmostrecentworkstothistopic.88Environmentalpro-tectionbyIHLblossomedfromtheneedtoprotectcivilians,morespecificallythemilieuinwhichthey,andhumankindingeneral,areexpectedtoliveandwhichmustthereforebesparedfromthescourgesofwar:theultimategoalisanalogous.89Thus,rulesareinplacethatprotectenemypropertyfromwantondestruction,prohibitpillage(relevantfortheexploitationofnaturalresources),protectcivilianobject(includingthenaturalenvironment,unlessitistobeconsideredasamilitaryobjective)andthoseindispensabletothesurvivalofthecivilianpopulation.Agreatdealmorecanstillbedone,however,inordertogranteffectiveprotectiontothenaturalenvironment,suchascompensationfor(andassistanceto)victimsofIHLviolationsofnormsprotectingtheenvi-ronmentortheestablishmentofinternationalmonitoring.90ItcansurelybeanticipatedthattheuseofAWSwillimpingeonthenaturalenvironment,asmayoccur,forinstance,ifautonomousswarmsdestroyallorpartofforestswheretheenemyhides.Intheseinstances,rulesprotectingenemypropertyorgoverningtheuseofforceagainstcivilianobjectsormilitaryobjectivescometothefore.However,itdoesnotseemthatcharac-teristicsofAWSaredecisiveinthoseinstances.First,ouranalysisisdevotedtoAWScapableoftaking(potentiallylethal)actionagainstindividuals:rulesgoverningtheuseofforceagainstobjects,yetnotentirelydifferentincontentandespeciallyintheirrationales,isnotdiscussedhere.Secondly,inallcases85ILC,‘ThirdReportontheProtectionoftheEnvironmentinRelationtoArmedConflicts’(3June2016)A/CN.4/700para10.86ConventionontheProhibitionofMilitaryorAnyOtherHostileUseofEnvironmentalModificationTechniques(adopted10December1976,enteredintoforce5October1978)1108UNTS151.87CRPayne,‘ProtectionoftheNaturalEnvironment’,inBSaulandDAkande(eds),TheOxfordGuidetoInternationalHumanitarianLaw(OUP2020);JMHenckaertsandDConstantin,‘ProtectionoftheNaturalEnvironment’,inAClaphamandPGaeta(eds),TheOxfordHandbookofInternationalLawinArmedConflict(OUP2014).88ILC,‘ProtectionoftheEnvironmentinRelationtoArmedConflicts.TextandTitlesoftheDraftPrinciplesProvisionallyAdoptedbytheDraftingCommitteeonFirstReading’(6June2019)A/CN.4/L.937.89Payne(n87)205.90HenckaertsandConstantin(n87)488ff.DiegoMauri-9781802207675DownloadedfromPubFactoryat05/28/202203:48:43AMviaDukeUniversityLawLibrary

144136AutonomousweaponssystemsandtheprotectionofthehumanpersontheuseoftheseAWShastobemeasuredagainstexistingIHLrules,asforallweapons,meansandmethodsofwarfare.Forinstance,chemical,biological,radiologicalandnuclearweaponsraiseimportantchallengesfortheprotectionofthenaturalenvironment,astheyreleaseagentsthatmayremainlatentformonths(ifnotdecades)andthussignificantlyaffecttheenvironment.91ItseemsthatsuchissuesliemorewiththeweaponsthatAWSwillbeequippedwith,ratherthanthecircumstancethattheycanselectandengagetheirowntargetswithouthumanintervention—notdifferentlyfromwhathasbeenarguedabove.4.2.4TheDutytoConductLegalReviewofWeaponsAkeyobligationofweaponslaw,andmoregenerallyofIHL,iscontainedinarticle36API,whichreads:Inthestudy,development,acquisitionoradoptionofanewweapon,meansormethodofwarfare,aHighContractingPartyisunderanobligationtodeterminewhetheritsemploymentwould,insomeorallcircumstances,beprohibitedbythisProtocolorbyanyotherruleofinternationallawapplicabletotheHighContractingParty.92WhiletheissueofwhetherthisprovisioncorrespondstocustomaryIHLstillremainsopen,93currentlyAPIcounts174signatoryparties,renderingtheobli-gationtoconductlegalreviewofnewweaponsspreadalmostworldwide.AscorrectlyfoundbytheCommentary,article36wasintroducedasadeviceforaddressingthechallengesposedbynewtechnologies.ItmaysoundsurprisingthattheCommentary’sauthorsanticipatedthat‘[t]heuseoflongdistance,remotecontrolweapons,orweaponsconnectedtosensorspositionedinthefield,leadstotheautomationofthebattlefieldinwhichthesoldierplaysan91SSaluzzo,‘CBRNWeaponsandtheProtectionoftheEnvironmentduringArmedConflicts’,inAdeGuttry,MFrulli,FCasolariandLPoli(eds),InternationalObligationsApplicabletoPrevention,Preparedness,ResponsetoandRecoveryfromChemical,BiologicalandRadio-Nuclear(CBRN)Events.TowardsanAll-HazardApproach(Brillforthcoming).92APIart36.93NJevglevskaja,‘WeaponsReviewObligationunderCustomaryInternationalLaw’(2018)94Int’lLStud186;DBlakeandJSImburgia,‘“BloodlessWeapons”?TheNeedtoConductLegalReviewsofCertainCapabilitiesandtheImplicationsofDefiningThemas“Weapons”’(2010)66TheAirForceLRev157.Foraninsightofstatepractice,seeIDaoust,RCouplandandRIshoey,‘NewWars,NewWeapons?TheObligationofStatestoAssesstheLegalityofMeansandMethodsofWarfare’(2002)846IRRC345.DiegoMauri-9781802207675DownloadedfromPubFactoryat05/28/202203:48:43AMviaDukeUniversityLawLibrary

145Autonomousweaponssystemsunderinternationalhumanitarianlaw137increasinglylessimportantrole’.94Therationaleoftheruleenshrinedinarticle36emergesclearlynow:initsmostessentialform,itaimsatprovidingstateswithacompassfornavigatingthewatersofnewtechnologies.Everytimenewweapons,meansormethodsofwarfareareabouttoenterthebattlefield,statesareunderadutytoconductalegalreviewinordertoassessthelegalitythereof.Fourkeyaspectsofarticle36havetobehighlighted.First,thedutytoconductlegalreviewsembracesdifferentphasesofweaponsmanufactureandprocurement(rangingfromthe‘study’tothe‘adoption’thereof).Second,theassessmentofnewweaponshastobemadewithregardtotheconcretecircumstanceinwhichthosewillbeused.Whileitseemsreasonabletoholdthatthedutycannotbeinterpretedinordertorequirestatestoforeseeallpossiblemisusesofweapons,itsurelyrequiresstatestotakeintoaccountnormalorexpectableusesofthoseweapons.95Third,article36establishesadutytoadoptnationalimplementingmeasures,whosecontentandscopearethuslefttoeachstate.96Thatis,legalreviewspursuanttoarticle36APIaredomesticprocesseswhichhavetobeconductedinordertoensurerespectforIHLprimarily,andthusthegeneralprohibitionsofweaponscausingSIrUS,ofindiscriminateweapons,ofthoseparticularlyaffectingtheenvironmentandofspecificprohibitions.Statesareobligedtoreviewnotonly‘future’weapons,butalsoexistingweaponsifthelatteraremodifiedafteraninitialrevieworupgradedintheirfunctions.97Someauthorshavesuggestedtheadoptionofamulti-prongapproachtoweaponsreview,whichshouldincludetheanalysisofthedesign,technicalandperformanceaswellasthedeterminationofthe‘normalorexpecteduse’oftheweapon,beforeturningtoassessingcompli-ancewithlaw(includingdomesticlaw).98Finally,therulesagainstwhichnewweaponshavetobeassessedarenotlimitedtothosecontainedinAPI,butexpresslyinclude‘anyotherruleofinternationallaw’bindinguponthestate.Thismeansthatotherbranchesofinternationallaw,suchasIHRL,andrele-vantnormscontainedthereinarecalledintoquestion:bysodoing,article36worksasaformidabletoolforsystemicintegrationininternationallaw.Theselastremarkswillbeofhelplater.99ReviewingAWSmusttakeintoaccounttheinherentcharacteristicsoftheweaponssystem.Traditionalreviewprocessesmaybeinadequateforcoping94APICommentarypara1476(devotedto‘futurearms’).95Ibidpara1469.96Daoust,CouplandandIshoey(n93)347.97Ibid352.98ATattersallandDCopeland,‘ReviewingAutonomousCyberCapabilities’,inRLiivojaandAVäljataga(eds),AutonomousCyberCapabilitiesunderInternationalLaw(NATOCCDCOEPublications2021).99SeeCh6,s6.3.DiegoMauri-9781802207675DownloadedfromPubFactoryat05/28/202203:48:43AMviaDukeUniversityLawLibrary

146138AutonomousweaponssystemsandtheprotectionofthehumanpersonwithAWS,owingtothecomplexityandtheopaquenessofalgorithmsdrivingautonomy.Tothisend,bespokemeasures,suchassoftwaretools,tocheckthesystem’scompliancewithrulesaswellasongoingassessmentagainsttestingandtrainingperformancehavebeensuggestedaswaysforensuringproperreviewofAWS.100Thatis,weagreethatarticle36reviewsapplytoAWS,buttheyneedtobeadaptedtothecharacteristicsofthosesystems.Notethatarticle36isofteninvokedduringthedebateonAWS—oneofthemostengagedNGOsintheforum,Article36,isnamedafterthatproviso.Tobeginwith,article36’scontentisechoedinGuidingPrinciple(e),whichaffirmsthat‘[i]naccordancewithStates’obligationsunderinternationallaw,inthestudy,development,acquisition,oradoptionofanewweapon,meansormethodofwarfare,determinationmustbemadewhetheritsemploy-mentwould,insomeorallcircumstances,beprohibitedbyinternationallaw’.101Importantly,some—includingtheUS,Russiaandotherpro-AWSstates—endorsethedutytocarryoutlegalreviewofAWSinasmuchasnewweaponssoastoargueagainsttheirprematureprohibition.TheUSprovidesanexcellentexample:DoDDirective3000.09envisagesaprocedureforlegallyreviewingautonomoustechnology.102Alsootherstates—suchastheNetherlands—emphasizetheimportanceofensuringlegalreviewofAWS,addingthatsharinginformationamongstates(exceptforinformationhavingakeyvaluefornationalsecurity)wouldbeawelcomestepinoperationalizingarticle36.103RepresentativesofcivilsocietyandNGOsfollowthesamelineofreasoning.TheStockholmInternationalPeaceResearchInstitute(SIPRI),forinstance,hasrecommendedthatstatesbuildonexistingelementsofbestpracticeforconductinglegalreviews,developtransparencyandcooperation,andmutuallysupporteachotherinrelevantfields(includingcyberwarfareandhumanenhancement).104Oncloserinspection,article36doesnotdealwithhowforceisappliedagainsttheindividual;instead,itimposesanobligationtosetoutnationalmechanismsallowingfornewweaponstobetestedinlightofIHLrulesandprinciples,aswellasotherinternationalnormsasapplicable.Itfollows100TattersallandCopeland(n98)241ff.101GuidingPrinciples,principle(e).102DAmoroso,AutonomousWeaponsSystemsandInternationalLaw.AStudyonHuman-MachineInteractionsinEthicallyandLegallySensitiveDomains(Edizionisci-entificheitaliane2020),252ff.103TheNetherlands,‘NationalcommentarybytheKingdomoftheNetherlandsregardingthenationalinterpretationandimplementationoftheGuidingPrinciplesaffirmedbytheGroupofGovernmentalExpertsonEmergingTechnologiesintheAreaofLethalAutonomousWeaponsSystem’(2020).104VBoulaninandMVerbruggen,‘Article36Reviews.DealingwiththeChallengesPosedbyEmergingTechnologies’,SIPRIReport(December2017).DiegoMauri-9781802207675DownloadedfromPubFactoryat05/28/202203:48:43AMviaDukeUniversityLawLibrary

147Autonomousweaponssystemsunderinternationalhumanitarianlaw139that,indevelopingAWS,statesmustensurethatthisnewweaponryisusedincompliancewiththerulesofweaponslawillustratedpreviously,thosepertainingtotargetinglaw,andevenIHRLrulesandprinciplesprotectingthehumanperson.105Importantly,article36requiresthatnewweaponsbeassessedagainstprinciples,suchasthatofhumanity(whichisexploredbelow).Indiscussingprohibitionsandrestrictionbasedoncustomaryandconventionalweaponslaw,theICRC’sGuideconcludeswithashortparagraphdedicatedto‘Prohibitionsorrestrictionsbasedontheprinciplesofhumanityandthedictatesofpublicconscience(the“MartensClause”)’.106Accordingtothehumanitarianinstitution,theprincipleofhumanityasenshrinedintheMartensClausecanbeemployedasabasisforconsideringaweaponunlawful.107ItfollowsthatifAWSwerefoundtobecontrarytothose‘principlesofhumanityanddictatesofpublicconscience’,thentheywouldnotpassthelegalreviewaccordingtoarticle36API.Insummary,thereisnoreasontodenythat:(1)AWShavetobesubjectedtoweaponsreviewaccordingtoarticle36API;(2)limitsandshortcomingsderiv-ingfromthereviewprocessdependmoreonhowthosereviewsareconductedthanontheparticulartechnologyunderreview.Inanyevent,AWSneedtobeassessedagainstIHLatlarge,includingrulesofweaponslaw,oftargetinglaw,andalsogeneralprinciples.4.3RULESONTHEUSEOFFORCEAGAINSTTHEHUMANPERSON:TARGETINGLAWLetusnowturntothesecondprongofrulescontainedinIHLandregulatinghowforcecanbeusedagainsthumans,namely,targetinglaw.Thissetofnormsaddressestheissueofhow(lawful)weaponsmaybelawfullyusedinattacks:itfocusesnotonthetoolemployedtoapplyforce,butonitsuser.Targetinglawisinspiredbythesameideaoflimitationencounteredinweaponslaw:therightofthepartiestoaconflicttoattackeachotherisnotunlimited,soitissafetoarguethatbothbranchesofIHLshareahumanitariantraitascommonfeature.Tothisend,referencehastobemadefirsttoarticle48API,encapsulatingtheprincipleofdistinction:partiestotheconflictareunderanobligationtodistinguishbetweenthecivilianpopulationandcombatants,andbetweencivilianobjectsandmilitaryobjectives,andtodirecttheiroperationsonlyagainstthelatter.ThisobligationappliesequallytoIACsandNIACs,anditis105SeeCh3.106ICRC(n23)17.107Seeinfras.4.5.DiegoMauri-9781802207675DownloadedfromPubFactoryat05/28/202203:48:43AMviaDukeUniversityLawLibrary

148140Autonomousweaponssystemsandtheprotectionofthehumanpersonuniversallyheldascorrespondingtocustomarylaw.108Thedifferencebetweenweaponslawandtargetinglawliesintheaddresseesoftheprotection:whiletheformerisconcernedwithcombatants,thelatteraimstoensurethatforcebedirectedtowardspermissibletargets,therebysparingciviliansfromharm.Normsoftargetinglawimposerightsanddutiesonpartiestoaconflictwhenplanningandcarryingoutan‘attack’.Theaccepteddefinitionof‘attack’isprovidedbyarticle49(1)API,accordingtowhichthetermmeans‘actsofviolenceagainsttheadversary,whetherinoffenceorindefence’.109Therefore,itisnotdisputedthatattacksinvolvingtheuseofAWSwillbesubjecttothosenorms.ApreliminaryissuethathastobedealtwithistheidentificationofthepointatwhichanattackusingAWScommencesandthedurationthereof.Accordingtooneview,whichcanbedescribedasnarrow,an‘attack’com-menceswhenAWSselectatarget.Conversely,abroadviewconsidersthatanattackstartswhenAWSareactivated,asfromthatmomentonanyobjectorindividualthatfitswithinthesystem’stargetprofileisdirectlyendangered.Astheway‘attack’isdefinedhasabearingontheapplicabilityofIHLrules,thesecondviewhastobepreferred.110BeforeturningtotheanalysisofspecificrulescontainedinthisbranchofIHL,itseemsappropriatetotackleanargumentthathasbeenformulatedvis-à-visthefuturedeploymentofAWSinthebattlefield.Accordingtosome,theadventofincreasedautonomouscapabilities,madepossiblebyadvancesinAI,deep-learningandneuralnetworks,willrenderAWSmoreakinto‘com-batants’than‘weapons’:theywillbeabletomaketheirowndecisions(whichiswhat‘tools’donotdobydefinition)tothepointthattheywilltransformintorobo-combatants.111Aweapondoesnotdecidewhenandagainstwhom(orwhat)toshoot—theseareitsmaster’scalls.Thus,weaponssystemscapableofselectingandengagingtargetswithouthumaninterventionwillreplicate(andreplace)humandecision-makingprocess:itwillbeuptothemtoperformthewholechainintheOODAloop,theoperatorretaininglittletonoauthorityovercriticalfunctions.However,theanalogywithcombatantshasreceived108CIHLrule1.109APICommentary,paras1878–82.110VBoulanin,NDavinson,NGoussacandMPCarlsson,‘LimitsonAutonomyinWeaponSystems.IdentifyingPracticalElementsofHumanControl’,SipriandICRCReport(June2020)6.111TChengeta,‘AreAutonomousWeaponSystemstheSubjectofArticle36ofAdditionalProtocolItotheGenevaConventions?’(2016)23JofInt’lL&Pol77.DiegoMauri-9781802207675DownloadedfromPubFactoryat05/28/202203:48:43AMviaDukeUniversityLawLibrary

149Autonomousweaponssystemsunderinternationalhumanitarianlaw141somecritiques,owingtoitsinaccuracy,112andhasbeendiscardedforbeingultimatelypointless.113Thisviewfailstoconsideradequatelythathumansandmachinesarelikelytointeractatvariousstagesoftheoperationalloop.Forinstance,theUKdescribesthisinteractionas‘human–machineteaming’,underscoringthecircumstancethat,whileperformingdifferenttasks,humansandAIsystemswillworktoward‘acommongoal’.114ThesameconceptsareadoptedbyBrazil.115Moregenerally,thecurrenttrendsattheCCWconfirmtheviewaccordingtowhichhumansandautonomoussystemswillformateaminoperatingonthebattlefield.GuidingPrinciple(c)recognizestheparamountimportanceofhuman–machineinteraction,whichmaytakevariousformsandbeimplementedatvariousstagesofthelifecycle.Insteadofremovingthe‘human’elemententirely,technologicaladvancesseemmoreinclinedtoretainitandfindunprecedentedwaystocombineitwiththe‘artificial’element,asrecentstudiesin‘brain–computerinterfaces’havehighlighted.116Whileitisnotpossibletoexpandonthishere,letitsufficethatthosetrendsdemonstratethattoconceiveofAWSas‘combatants’entirelyreplacinghumanswouldbemisleading,whichreiteratestherelevanceoftargetinglawinassessingwhetherthoseautonomoussystemscanbeusedincompliancewithexistingIHLnorms.ForthepurposesofIHLthen,onlyhumanagentswillremain‘combatants’:rulesandprinciplesregardinghowandwhenindividualscanbemadetheobjectof(potentiallylethal)forceduringarmedconflictisaddressedexclu-sivelytothem.Itistimenowtoturntothemainnormsontargetinglaw,namely,theprohibitiononindiscriminateattacks(section4.3.1),thedutytotakeprecautionsinattack(4.3.2)and,finally,thedutytoinvestigateandpros-ecuteviolationsofthesenorms(4.3.3).112RCrootof,‘AutonomousWeaponSystemsandtheLimitsofAnalogy’(2018)9HarvardNatl’lSecJ51.113KAnderson,DReisnerandMWaxman,‘AdaptingtheLawofArmedConflicttoAutonomousWeaponSystems’(2014)90Int’lLStudies386,410(arguingthat‘[w]hethertheactoronthebattlefieldisa“who”ora“what”isnottrulytheissue,butratherhowwellthatactorperformsaccordingtothelawofarmedconflict’).114‘UnitedKingdomExpertPaper:TheHumanRoleinAutonomousWarfare’(18November2020)CCW/GGE.1/2020/WP.62,3.115Brazil,‘LAWSandHumanControl:BrazilianProposalsforWorkingDefinitions’(19August2020)CCW/GGE.1/2020/WP.4,3.116ABinnendijk,TMarlerandEMBartels,‘Brain-ComputerInterfaces.U.S.MilitaryApplicationsandImplications.AnInitialAssessment’,RANDReport(2020).DiegoMauri-9781802207675DownloadedfromPubFactoryat05/28/202203:48:43AMviaDukeUniversityLawLibrary

150142Autonomousweaponssystemsandtheprotectionofthehumanperson4.3.1TheProhibitiononIndiscriminateAttacksStemmingdirectlyfromtheprincipleofdistinctionreferredtopreviously,rulesprohibitingindiscriminateattacksposittheexistenceofthedichotomy‘permissible’or‘impermissible’targets.Whetheranindividualoranobjectcanbemadetheobjectofattackduringhostilitiesrequiresalegalassessment,tobeconductedunderIHL.Ingeneral,thecivilianpopulationandindividualcivilianscannotbemadetheobjectofattackduringhostilities:thisprohibition—enshrinedinarticles51(2)APIand13(2)APII—iscomplementedbyspecificrulesrestrainingthepartiestoaconflict.117Thehumanitariantraitoftheserulessurfacednotlongago:stillatthetimeoftheLieberCode,theideawherebyeverycitizenornativeoftheopposingpartyhadtobeconsideredequaltothe‘enemy’wastheprevailingidea.118TheCodeitselfwasamongthefirstattemptsatleavingbehindthisapproach.119Attheinternationallevel,theruleimposingadutytokeepmilitaryforcesandciviliandistinctforthepurposesoftargetingwasfirstenouncedintheSaintPetersburgDeclaration.120Therulethenmadeitswayinthe1899and1907HagueRegulations,andeventuallyintheGenevaConventions(GC)anditsprotocols:tobemoreprecise,thecategoryof‘civil-ians’asanindependentandprotectedonewasrecognizedinGCIV.121TurningtoindividualsspecificallyprotectedbyIHLnorms,afewexamplesare,wounded,sick,shipwrecked,medicalandreligiouspersonnel,children,thedisabled,andcombatantsthatbecomehorsdecombat.Thosecategoriesofindividualsenjoyagraduatedprotectiondependingonthenatureofthearmedconflictunderconsideration(IACorNIAC);122differencesnotwithstanding,itissafetoarguethattheprohibitiononindiscriminateattacksisparamountforthewholeedificeoftargetinglaw.Severalrulesarerelevantinthisarea,whichmayacquiredifferentimportancewhenAWSarefielded.Thesearesubdividedintwogroupsonthebasisoftheissuesraisedbytheemployment117CIHLrule1.118InstructionsfortheGovernmentofArmiesoftheUnitedStatesintheField,GeneralOrdersNo100byPresidentLincoln(24April1863)(LieberCode)art20.119Ibidart22(‘ascivilizationhasadvancedduringthelastcenturies,sohaslike-wisesteadilyadvanced…thedistinctionbetweentheprivateindividualbelongingtoahostilecountryandthehostilecountryitself,withitsmeninarms’).120SaintPetersburgDeclarationpreamble(establishingthatasaresultofthe‘pro-gressofcivilisation’States’legitimateobjectivesinwarfareshouldbe‘toweakenthemilitaryforcesoftheenemy’).121LHill-Cawthorne,‘PersonsCoveredbyInternationalHumanitarianLaw:MainCategories’,ininBSaulandDAkande(eds),TheOxfordGuidetoInternationalHumanitarianLaw(OUP2020).122Ibid.DiegoMauri-9781802207675DownloadedfromPubFactoryat05/28/202203:48:43AMviaDukeUniversityLawLibrary

151Autonomousweaponssystemsunderinternationalhumanitarianlaw143ofAWS:rulespertainingtothefittingofindividualsintoIHLcategories(derivingfromtheprincipleofdistinction),andrulespertainingtostrikingbalancebetweenmilitaryadvantageandcollateraldamage(stemmingfromtheprincipleofproportionality).4.3.1.1‘Categorizing’targets:rulesstemmingfromtheprincipleofdistinctionTraditionally,humansoldiers—physicallypresentontheground—aresup-posedtoevaluate(sometimeswithinaverysmallwindowoftime)theindividualinfrontofhimorher,anddecidewhethertoattackornot:thisisanoperationinwhichaparticularindividualis‘categorized’aspermissibleorimpermissibletarget.Granted,phrasedinthiswaythiscontentionmaysoundasanoversimplificationofthereality:currently,thetargetingcycleisacomplexprocessfeaturingdifferentandhighlysophisticatedsteps.However,itcapturesthemainissueatstake:itispermissibletoselectandengageaparticulartargetonlyifthistargetfulfillsspecificrequirements,firstandforemostthoseimposedbyIHL,andifitcanbeinscribedinacertain‘category’.Indiscriminateattacksarethosethatfailtoinscribetargetsinthecorrectcategory.Tobeginwith,theruleprohibitingindiscriminateattacksrequiresthepartytoaconflicttorefrainfromadoptingindiscriminatetactics,thatis,methodsofwarfaredirectedatprotectedpersonsorobjectsandthusblatantlyviolatingtheabove-mentionedcategories.Thisruleisencapsulatedinarticle51(4)(a)API,whichproscribesindiscriminateattacksasthose‘whicharenotdirectedataspecificmilitaryobjective’—aprohibitionthatiswidelyacknowledgedascustomary.123Theruleiscomplementarytothatprohibitingindiscriminateweapons,asillustratedpreviously:together,theyformamulti-facetedsetofrulesembodyingthegeneralprincipleofdistinctionasperarticle48API.Accordingtothisbedrockrule,partiestoaconflictmustalwaysdistinguish‘betweenthecivilianpopulationandcombatantsandbetweencivilianobjectsandmilitaryobjectives’,andtheiroperationsshallbedirectedonlytothelatter.Articles51(2)and52(1)specifythisrule.ThefirstdifficultyencounteredbyahumancommanderwhenheorshedecidestofieldAWSistomakesurethatattackscarriedoutthroughthoseweaponssystemsisdirectedonlyatpermissibletargets.Oneofthemainissuesgenerallyassociatedwithremotewarfareisthat,asaresultofdistancinghumansfromthebattlespaceandentrustingautomatedorautonomoussystems123CIHLrule13.DiegoMauri-9781802207675DownloadedfromPubFactoryat05/28/202203:48:43AMviaDukeUniversityLawLibrary

152144Autonomousweaponssystemsandtheprotectionofthehumanpersonwithcriticalfunctions,risksofmistargetingincrease.124Toreduceandremovesuchrisks,AWSmustbeabletoassessthestatusoftheparticulartargetbeforeengagingit,anassessmentthat—dependingonthescenarioundercon-sideration—anautonomoussystemmayhaveahardtimetoperformcorrectly.Letusbeginwithmilitaryobjectivesandillustratethosedifficultiesbywayofanexample.Thecategoryof‘militaryobjectives’includes‘thoseobjectswhichbytheirnature,location,purposeorusemakeaneffectivecontributiontomilitaryactionandwhosetotalorpartialdestruction,captureorneutraliza-tion,inthecircumstancesrulingatthetime,offersadefinitemilitaryadvan-tage’(article52(2)API).125Whileamilitarytargetby‘nature’seemseasytoidentify(forexample,atank:AWSwillonlyneedsensorscapableofdistin-guishingatankfromaciviliancar),126parameterssuchas‘purpose’or‘use’aremoredifficulttoassessandinevitablyrequireacontext-relatedassess-ment.Accordingtooneauthor,themilitarypurposeofacivilianobjectmustbeascertainedonthebasisofa‘reasonable’belief,a‘worstcasescenario’beinginsufficient.127Thoseobjectsarecommonlyreferredtoasdual-use,andexamplesincludepowerplantsorstations,128or‘war-sustaining’objects,suchasoilrevenues,usedforfinancingoneparty’smilitaryeffort.129TheICRC’sCommentaryreferstothecriterionof‘purpose’asindicating‘theintendeduseofanobject’,whilethecriterionof‘use’dealswithits‘presentfunction’.130124ECrawford,‘ThePrincipleofDistinctionandRemoteWarfare’,inJDOhlin(ed),ResearchHandbookofRemoteWarfare(EdwardElgar2017)(explainingtheoperationaldifferencesbetweentargetingobjectsandtargetinghumans);Crootof,‘AutonomousWeaponSystemsandtheLimitsofAnalogy’(n112)56;WBoothby,‘AutonomousAttack–OpportunityorSpectre?’(2013)16YearbookofInternationalHumanitarianLaw71,78–9.125Emphasisadded.SeeHBRobertson,‘ThePrincipleoftheMilitaryObjectiveintheLawofArmedConflict’(1997)8UnitedStatesAirForceAcademyJournalofLegalStudies35,56–7(consideringtheevolutionoftheprincipleandcontrasting‘theold-stylenegativelitofprohibitedtargets’and‘thenew-stylepermissiveprincipleofdefiningthemilitaryobjective’.126MWagner,‘AutonomyintheBattlespace:IndependentlyOperatingWeaponSystemsandtheLawofArmedConflict’,inDSaxon(ed),InternationalHumanitarianLawandtheChallengingTechnologyofWarfare(Brill2013),113(arguingforthefea-sibilityofprogrammingtherudimentarycharacteristicsofobjectsintomachines).127YDinstein,TheConductoftheHostilitiesundertheLawofInternationalArmedConflict(CUP2016),183–4.128WesternFrontcase(2005)26RIAA291paras119–20(foracaseconcerningtheaerialbombardmentoftheHirgigoPowerStation,consideredas‘militaryobjec-tive’perAPIas,eventhoughelectricityproductionhadnotstartedatthetimeofthebombing,itwasintendedthattheEritreanarmywouldbenefitfromit).129Schmitt,‘InternationalHumanitarianLawandtheConductofHostilities’(n71),163(illustratingrecentpracticeofthemilitarycampaignagainstISIS).130APICommentarypara2022.DiegoMauri-9781802207675DownloadedfromPubFactoryat05/28/202203:48:43AMviaDukeUniversityLawLibrary

153Autonomousweaponssystemsunderinternationalhumanitarianlaw145Notethatthenotionof‘definitemilitaryadvantage’hasbeensubjecttoremarkablyliberalinterpretations,eitherbywayofassessingitnotagainstthesingleattack,buttheentiremilitarycampaign,orbyadoptingmilitarydoctrines(suchastheUS‘effect-basedoperations’,orEBO)thatgobeyondthegoalofsimplyweakeningtheenemy,includingalsotheachievementofother‘desiredoutcomes’.131Inourview,tooexpansiveinterpretationsofthosenotionsrisknullifyingtheprotectionaccordedtocivilianobjects,sotheymustbeadoptedwithcaution.Inallcases,inordertobeusedincompliancewiththoserulesAWSmustbeendowedwithalgorithmsallowingthemtocollectandre-elaborateamassivequantityofreal-timedata.Addinganotherlayerofdifficulty,anotherkeyaspecttodeterminethemilitaryvalueofatargetisthe‘definite’(‘précis’,intheFrenchversion)militaryadvantageresultingfromtheattack,tobeassessedhicetnunc:again,achallengeforalgorithms.132Thisiswhy,accordingtosome,AWSshouldbeemployedonlyincluttered,simpli-fiedenvironmentswherereal-timeassessmentscanbemadebyalgorithmsinasufficientlyreliableway.133Thelattershouldapplyallthemoresowhenitcomestoselectingandengaginghumantargets.LimitingouranalysisnowtoIAC,notoriouslythecategoryof‘combatants’includes,interalios,membersoftheregulararmedforces,membersofamilitiaandothervolunteergroupsmeetingtheconditionsenvisagedbyIHLnorms(beingunderresponsiblecommand,wearingattireoremblemsallowingthemtobedistinguishedfromthecivilianpopulation,carryingtheirweaponsopenlyandconductingoperationsinaccordancewithIHL),participantsinalevéeenmasse.134Acivilianiswhoeverdoesnotbelongtoanyofthesecategories.135OtherprotectedpersonsunderIHLinclude131WesternFrontcase(n128)para113;MNSchmitt,‘TargetingandHumanitarianLaw:CurrentIssues’(2003)33IsraelYbonHumanRights59.132Wagner(n126)113(discussingexistingtechnologyandcommentingonthedis-criminationcapabilitiesrequiredforfuturemachines).133MBrenneke,‘LethalAutonomousWeaponSystemsandTheirCompatibilitywithInternationalHumanitarianLaw:APrimerontheDebate’(2020)21YearbookoftheInternationalHumanitarianLaw59;MSassoli,‘AutonomousWeaponsandInternationalHumanitarianLaw:Advantages,OpenTechnicalQuestionsandLegalIssuestobeClarified’(2014)90Int’lLStud308.SeealsoICRC,‘AutonomousWeaponsSystems.Technical,Military,LegalandHumanitarianAspects’,ExpertMeeting(March2014)78(‘currentfixedautonomousweaponsystemsusedinnarrowrolesandoperatinginrelativelystatic,lowclutterenvironmentscanbeprogrammedtodistinguishsimpleobjects’);UKMoD,‘JointDoctrineNote2/11’(n11)para508(con-tendingthatinnot-complexenvironments‘adegreeofautonomousoperationisprob-ablyachievablenowanddatafromprogrammessuchasBrimstoneandALARM,forexample,wouldhavedirectread-across’).134SeeGCIIIarts4A(1),(2),(3),(6);APIart43.135APIart50(1).DiegoMauri-9781802207675DownloadedfromPubFactoryat05/28/202203:48:43AMviaDukeUniversityLawLibrary

154146Autonomousweaponssystemsandtheprotectionofthehumanpersonmedicalandreligiouspersonnel.136Yetthedividebetweenpermissibleandimpermissiblehumantargetsmaynotbesoclear-cutinpractice.Forinstance,therearenumerouscaseswhereitcouldbehardtospotacombatantonthebattlefield,asIHLdoesnotimposeafixed‘dresscode’onregularforcesandcontentsitselfwithrequiringthatirregularforcesjustcarrytheirarmsopenly.137Thewidespreadappearanceoncontemporarybattlefieldsofgroupsthatarenotformallyaffiliatedwiththearmedforcesofapartytotheconflicthasfueledextensivecontroversiesonhowtocategorizeboththesegroupsanditscomponent,asmadeevidentbytheICRC’s‘InterpretiveGuidance’onthetopic.138Forinstance,accordingtotheICRConlythosemembersexertinga‘continuouscombatantfunction’(CCF)aretargetable—arequirementthatdoesnotapplyto‘traditional’combatants,whocanbetargetedevenwhenasleep.Thatis,theydonotenjoythecivilianstatus,butallegedlytherearelim-itationstotheirtargeting.Anotherdifficultscenarioisrepresentedbycivilianswho‘directlyparticipateinhostilities’(DPH)pursuanttoarticles53(3)APIand13(1)APII(henceapplicablealsotoNIAC):thiscategorycoversindivid-ualsthat,withoutbeingformallymembersofanorganizedarmedgroup,con-tributetothemilitaryeffortofonepartytotheconflict(forexample,acivilianplantingIEDsonbehalfofagroup).Asaresultofthisconduct,theylosetheprotectionensuredtocivilians‘forsuchtime’astheyparticipateandcanthusbetargeted.ItmustberememberedthatthecategoriesofCCFandDPHasregulatedbytheICRC’sInterpretiveGuidancebroughtaboutaheateddebateinscholarship,withsomeauthorswelcomingitasapositivestepandotherssubjectingittoharshcriticism,mainlyowingtoitslackofclarityaroundthe‘revolvingdoor’issue(asprotectionfromattackswouldbe‘intermittent’forciviliansengaginginrepeatedhostileactswithoutqualifyingasCCF)andthetreatmentofvoluntaryhumanshields.139Leavingasidethedebatethathasgrownaroundthoseissues—althoughintriguing,farbeyondthereachofthissection—thecruxofthematteristhedegreetowhichhumancontrolofAWSisrequiredbytheruleprohibitingindiscriminateattacks,whenthosemachinesperformcriticalfunctionsonthebattlefield.Anargumentthatisoftenmadevis-à-viscompatibilitywiththoserulesoftargetinglawisthat,as‘categorizing’individualsisanoperation136CIHLrules25,27–9.137EWinter,‘TheCompatibilityofAutonomousWeaponswiththePrincipleofDistinctionintheLawofArmedConflict’(2020)69ICLQ845,861.138ICRC,InterpretiveGuidanceontheNotionofDirectParticipationinHostilitiesunderInternationalHumanitarianLaw(May2009).Asitisnotpossibletoexpandonthedebatehere,seeforreferencesSchmitt‘InternationalHumanitarianLawandtheConductofHostilities’(n71)157.139Amoroso(n102)49ff.(andreferencestherein).DiegoMauri-9781802207675DownloadedfromPubFactoryat05/28/202203:48:43AMviaDukeUniversityLawLibrary

155Autonomousweaponssystemsunderinternationalhumanitarianlaw147requiringasubjective,context-relatedassessmentofdifferentcircumstances(forexample,howindividualsbehaveinthebattlespace)andalgorithmicsystemsareinherentlyatoddswithsuchoperation,itwouldbehardforpartiestoaconflicttoemployAWSinanIHL-compliantway,tothepointthatthiscircumstancewouldsufficeforregardingAWSasprohibitedunderthisrule.140Thisargument,however,provestoomuch.Ultimately,itcontainsatechnolog-icalobjectiontoAWS:ifonelooksatthecapabilitiesofcurrenttechnologies,thentheconclusionmaybewarranted.However,itcannotbeexcludedthattechnologicaladvancesmaytomorrowrenderfeasiblewhattodayappearsimpossible:developmentsinthefieldofdeep-learningalgorithms,tonameone,maymakeitpossibleforAWStobeabletodiscriminatebetweenper-missibleandnon-permissibletargets.141Followingthislineofreasoning,ifalgorithmicsystemsaredevelopedwhichensurethatAWScanbeemployedinawaythatallowsforproperlycategorizingindividuals,initselfthissufficesforconcludingthattheruleprohibitingindiscriminateattacksmaybecom-pliedwith.Thisisnottosaythattodisposeofsuchcapabilitiesinreal-lifeoperationalenvironmentsisfarfromchallenging—itis.Forinstance,oneofthechallengesAWSwillfaceinhostilitieswillberecognitionofacombatant’ssurrender(thatis,willfulplacementhorsdecombatresultingintheiracquisitionofpro-tectedpersonstatus).142Inthisinstance,thatindividualisnolongertargetable:AWSmustbedevelopedinawaythatrendersthemcapableofrecognizingandacceptingsurrender.Appropriatesensorsandalgorithmsfordetectinganddecipheringhumanbehaviorarevitalinthesecases.Someauthorshavecorrectlyhighlightedthetechnologicalchallengesunderlyingthosescenarios,however,thisisnotenoughtosponsoratotalbanonAWS.143140DAkerson,‘TheIllegalityofOffensiveLethalAutonomy’,inDSaxon(ed),InternationalHumanitarianLawandtheChangingTechnologyofWar(Brill2013),79(arguingthatbeingthenotionofmilitaryobjectiveexpressedincasuisticterms,‘[t]heparadigmisthusunsuitableforacomputeralgorithmfortworeasons:itcannotbeexpressedwithprecisionanditsvaluecanonlybedeterminedinthecontextofapplication’).141JGHughes,‘TheLawofArmedConflictIssuesCreatedbyProgrammingAutomaticTargetRecognitionSystemsUsingDeepLearningMethods’(2020)21YbIHL99.142APIart41(2)(b).143RSparrow,‘TwentySecondstoComply:AutonomousWeaponSystemsandtheRecognitionofSurrender’(2016)91Int’lLStud699,705(acknowledgingthatrecog-nizingsurrenderis‘likelytobedifficultforrobots’becauseofthehardnessofpercep-tionasataskforrobotsandthecontext-relatednatureofthemeanscommonlyusedtosurrender,butnotexcludingthechanceoftechnologicaladvancementsaltogether);Boothby,‘AutonomousAttack–OpportunityorSpectre?’(n124)79(‘[d]evelopingsoftwaretoidentifywhenapotentialhumantargetismanifestinganintenttosurren-DiegoMauri-9781802207675DownloadedfromPubFactoryat05/28/202203:48:43AMviaDukeUniversityLawLibrary

156148AutonomousweaponssystemsandtheprotectionofthehumanpersonIfitissafetoarguethatAWSdonotruncountertotheruleprohibitingindiscriminateattacks(thatis,inasmuchashumanoperatorsmaynotinterveneinaparticulardecisionregardingtargetselectionandengagement),whatismoreinterestingtoelucidateistheextenttowhich‘humancontrol’hastobeensuredovercriticalfunctions.Initssimplestform,itcanbearguedthatinordertobeoperatedautonomously,AWSneedtobeableto‘categorize’targetsinordertoensurethattheyare‘discriminated’accordingly.While‘observation’and‘recognition’ofpotentialtargetsareprocessesthatauton-omoussystemsmayperformfarbetterthanhumans,‘judging’whetheritisappropriatetotakeacourseofactionresultingintheapplicationof(lethal)forceismoredubious.144Sothisiswhereahumanpresencemayberequestedtointervene.Thatis,humanoperatorsmayretaincontrolforaslongasitisnecessarytoensurehumaninterventionwhenitisneeded,forinstance,wherethereisdoubtabouthowtocategorizeaparticularindividual.Incaseofdoubtastostatus,targetinglawrequirespartiestoaconflicttopresumethattheindividualisacivilian.145Thisrulecorrespondstocustomarylaw:‘[o]necannotautomaticallyattackanyonewhomightappeardubious’.146AppliedtoAWS,thisruleproscribesautonomoussystemsthatarenotpro-grammedtostepbackfromattackinginthoseinstanceswherethereissomemarginofdoubtastowhetheranindividualfitsintoapermissiblecategory.Inthosecases,humanoperatorsmuststepinandguidethesystem.However,itwouldbefar-fetchedtoconcludethat,everytimeadoubtarisesonwhetheratargetispermissible,humansmustberequestedtointervene.Thismaybeappropriateatearlystagesoftechnologicaldevelopments,butcouldbedis-cardedifautonomoussystemsaretrainedadequatelytodealwithdoubt.Forinstance,ithasbeenarguedthatforAWStooperateautonomously,amarginofdoubtwillneedtobeconvertedinto‘anacceptablelevelofmathematicalprobability,belowwhichthepersonorobjectwouldbeclassifiedascivil-ian’.147Then,ifAWSperforminawaythatcorrespondstothelevelofarea-sonablehumaninthesamebattlespace,thedoubtrulewillbecompliedwith.148Theconclusionwereachedcanbephrasedas:initself,themerefactthathumanoperatorsmaynotinterveneintargetselectionandengagementdoesnotrunagainsttherulesondistinction,onconditionthatAI-relatedtechniquesinfusedintoAWSallowforthepropercategorizationofhumantargets.der,isincapacitatedbywoundsorsicknessorisinthepowerofanadversaryislikelytoprovechallenging’).144Winter,‘TheCompatibility’(n136).145APIart50(1).Thesameholdsforobjects:seeAPIart52(3).146CIHLrule6.147Hughes(n141)112.148SchmittandThurnher(n63)262–5.DiegoMauri-9781802207675DownloadedfromPubFactoryat05/28/202203:48:43AMviaDukeUniversityLawLibrary

157Autonomousweaponssystemsunderinternationalhumanitarianlaw1494.3.1.2Strikingbalances:rulesstemmingfromtheprincipleofproportionalityAnothersetofrulesrequiringadegreeofhumancontroloverautonomousmachinesstemsfromtheprincipleofproportionality.Whilethisprinciplehasinspiredextensivescholarshipininternationallaw,149thefocushereisonitsstanceintargetinglaw.Theprohibitionondirectattacksonciviliansasexplainedpreviouslydoesnotimplythatthesecannotbetargetedatall.Theprincipleofproportionalityrequiresthat,inplanningandexecutinganattack,thereasonablyanticipatedmilitaryadvantageofanoperationbeweighedagainstthereasonablyanticipatedcivilianharm,whichtakestheshapeof‘collateraldamage’.Thatis,civilianharmisnotimpermissibleatall;itisifdisproportionate.150Thisaddsasecondlayerofconceptualdifficulty:notonlywillpartiestoaconflictberequiredtoemploymeansandtacticsallowingthemtoproperlycategorizetargets,buttheywillalsoneedtoassessthemil-itaryadvantagethatmayderivefromcarryingoutaparticularattack,andtocompareitwiththeexpectedresultingcollateraldamage.Collateraldamageisunavoidableinwarfare:onecannotconceivethelatterwithouttheformer.151Surprisingly,inspiteofthis,rulesgivingcontenttoaprincipleofproportionalityinarmedconflicthavebeenformulatedonlyrecently.TheSaintPetersburgDeclarationrecallsproportionalityonlyinitspreamble(andwithregardtounnecessarysuffering),152whiletheGeneva149Onproportionality,seePdeSena,‘ProportionalityandHumanRightsinInternationalLaw:Some…“Utilitarian”Reflections’(2016)99Rivistadidirittointer-nazionale1009;VCannizzaro,Ilprincipiodiproporzionalitànell’ordinamentointer-nazionale(Giuffré2000).Morespecifically,seeVCannizzaro,‘ProportionalityintheLawofArmedConflict’,inAClaphamandPGaeta(eds),TheOxfordHandbookofInternationalLawinArmedConflict(OUP2014);JGGardam,‘ProportionalityandForceinInternationalLaw’(1993)87AJIL391.150Schmitt,‘InternationalHumanitarianLawandtheConductofHostilities’(n71)153ff;seealsoGNoll,‘AnalogyatWar:Proportionality,EqualityandtheLawofTargeting’(2013)43NetherlandsYbIL205(exploringthestructureofanalogicalthinking).151YDinstein,‘Discussion:ReasonablyMilitaryCommandersandReasonableCivilians’(2002)78Int’lLStud219(‘[a]nyoneevenmildlyinterestedininterna-tionalhumanitarianlawmuststrivetobringaboutabetterworldinwhichcivilized[sic]lossesinwarareminimized.Nevertheless,therealisticgoalistominimizecivil-iancasualties,nottoeliminatethemaltogether’).Foracriticalperspective,seeGNoll,‘SacrificialViolenceandTargetinginInternationalHumanitarianLaw’,inPEngdahlandOWrange(eds),LawatWar:TheLawasItWasandtheLawasItShouldBe.LiberAmicorumOveBring(Leiden2008).152(‘[T]hisobjective[toweakenthemilitaryforcesoftheenemy]wouldbeexceededbytheemploymentofarmswhichuselesslyaggravatethesufferingofdisa-bledmen,orrendertheirdeathinevitable’).DiegoMauri-9781802207675DownloadedfromPubFactoryat05/28/202203:48:43AMviaDukeUniversityLawLibrary

158150AutonomousweaponssystemsandtheprotectionofthehumanpersonConventionsremaintotallysilentonthepoint.Eventually,tworulescrystal-lizedinAPI,attheendoflongdiscussions.153Article51(5)(b)affirmsthatanattackis‘indiscriminate’wheneverthe‘incidentallossofcivilianlife,injurytocivilians,damagetocivilianobjects,oracombinationthereof’isexpectedtobe‘excessiveinrelationtotheconcreteanddirectmilitaryadvantageantici-pated’.Accordingtoarticle57(2)(b),thisbeingthecase,thereisanobligationtosuspendorcanceltheattack.SimilarrulesarenotpresentinAPII;theirapplicabilitytoNIACis,however,widelyacknowledged,eitheronthebasisoftheprincipleofhumanity(onwhichseebelow)orbyvirtueofcustomarylaw.154Attheoperationallevel,strikingbalancesinaccordancewiththeprincipleofproportionalityimposesathree-stepexanteanalysis:(1)toestimatecollat-eraldamage;(2)toassessmilitaryadvantage;and(3)todetermineexcessive-ness.155Accordingtothefirstofthese,proportionalityrequirestheattributionofa‘weight’totheexpectedcollateraldamage,whichdependsonthenatureofthetarget(forexample,vulnerablepersonsandminors)andofthedamage(injuryweighinglessthanlossoflife).156Anotherkeyaspectisrepresentedby‘reverberatingeffects’,thatis,delayedconsequencesofanattack.Tothisend,itisimportantnottoover-expandthetemporalwindowoftheanalysis,asthiswouldrenderbalancesmoreuncertainandultimatelydifficulttogoverninpractice.157Aspertheassessmentofmilitaryadvantage,oneofthemostcompellingissuesrevolvesaroundthethornyquestionofself-preservation,whichisofparticularrelevancetoAWS.Anattackinwhichnopersonnelarelost(aswouldbethecasewithAWS)seemsmilitarilyadvantageousbydefinition.However,iftheensuingcontentionwerethattheattackerdisposingofsuperiorarmsisallowedtodestroytheenemycompletelyandregardanyciviliancasualtyascollateral,thiswouldbeanabhorrentscenario,toavoidwhichitisconvincinglysuggestedthatthesecurityoftheattackingforcesbekeptoutofthemilitaryadvantagecalculations.158Thethirdstep—evaluatingexcessiveness—isthemostengaging.Aspro-portionality-basedrulesrequirethatabalancebestruckbetweenantagonisticvalues—namely,militaryadvantageandcollateraldamage(and,forthe153APICommentarypara1976.154CIHLrule14;seealsoProsecutorvKupreškićandOthers,ICTY-95-16-T(14January2000)para524.155Amoroso(n102)78ff.;Dinstein,‘TheConductofHostilities’(n127)157.156Dinstein(n127)160.157Amoroso(n102)81.SeealsoISHendersonandKReece,‘ProportionalityunderInternationalHumanitarianLaw:The“ReasonableMilitaryCommander”andReverberatingEffects’(2018)51VanderbiltJofTransnationalL835.158Amoroso(n102)83.DiegoMauri-9781802207675DownloadedfromPubFactoryat05/28/202203:48:43AMviaDukeUniversityLawLibrary

159Autonomousweaponssystemsunderinternationalhumanitarianlaw151purposesofourreasoning,humanlife)—ithasbeenquestionedwhetherthesecapabilitiescouldbetranslatedintoalgorithmsand,moregenerally,whetherproportionalityassessmentcouldbeentrustedentirelytoAWS.159Comparingthoseinterestsisadifficulttaskforhumansinthefirstplace.160Thebalancingprocessfeaturesinherentlysubjectiveandcontext-relatedassessmentsbasedonparadigmssuchas‘reasonableness’and‘excessiveness’,whichrefertoa‘reasonablywell-informedperson’standard(sometimesreferredtoalsoas‘reasonablecommander’standard).161This,however,doesnotmeanthatsuchstandardsarearbitrary,inthattheyareultimatelydependentontheparticularagent’sbeliefsandknowledge.Onthecontrary,scholarshavedevelopedan‘objective’and‘qualified’modelofreasonableness,whichtakesintoaccounttheexpertisereasonablyrequiredbycommandersinthesamepositionoftheoneunderanalysisandevaluatesitagainstexternal,widelyacceptedconstraints.162Thisnotionofproportionalityescapes,intheconvincingwordsofCannizzaro,‘predeterminedbehaviouralschemes’,totheextentthatitisabletoencapsulate‘apotentiallyinfiniterangeofpossiblecombinations’ofunderlyingvaluesandweights.163Insummary,inordertobecompliedwith,rulesstemmingfromtheprincipleofproportionalityneedtoensurearesultthatsatisfiesthecontext-related,dynamicrequirementsillustratedthusfar.Owingtothenatureofthisbalance,itisbelievedthatalgorithmicsystemsarestructurallyinadequatetoliveuptothatstandard.164Asaconsequence,159JvandenBoogaard,‘ProportionalityandAutonomousWeaponsSystems’(2015)6JIHLS247(testingprosandconsofincreasingautonomyinbattlefield).160Noll,‘AnalogyatWar’(n150)213–15;ICTY,‘FinalReporttotheProsecutorbytheCommitteeestablishedtoreviewtheNATObombingcampaignagainsttheFederalRepublicofYugoslavia’(8June2000)39ILM1257para48(‘[o]necannoteasilyassessthevalueofinnocenthumanlivesasopposedtocapturingaparticularmil-itaryobject’).SeealsoHarvardProgramonHumanitarianPolicyandConflictResearch(HPCR),ManualontheInternationalLawApplicabletoAirandMissileWarfare(2009)92(‘theterm“excessive”isoftenmisinterpreted.Itisnotamatterofcountingciviliancasualtiesandcomparingthemtothenumberofenemycombatantsthathavebeenputoutofaction’).161ProsecutorvGalić,ICTY-98-29-T(5December2003)para58.SeealsoCannizzaro,‘ProportionalityintheLawofArmedConflict’(n149)339.162HendersonandReece(n157)840–41.163Cannizzaro,‘ProportionalityintheLawofArmedConflict’(n149)332.164MNSchmitt,‘AutonomousWeaponSystemsandInternationalHumanitarianLaw:AReplytotheCritics’(2013)4HarvardNat’lSecJFeatures1,21(emphasiz-ingtheuncountable‘imaginablescenariosandvariablesthatmightoccurduringhos-tilities’);SchmittandThurnher(n63)256.SeealsoAPICommentary,para2208(accordingtowhichtheprincipleofproportionalitymustbeinterpretedas‘aquestionofcommonsenseandgoodfaithformilitarycommanders’,who‘mustcarefullyweighupthehumanitarianandmilitaryinterestsatstake’ineveryattack).DiegoMauri-9781802207675DownloadedfromPubFactoryat05/28/202203:48:43AMviaDukeUniversityLawLibrary

160152AutonomousweaponssystemsandtheprotectionofthehumanpersonsomeinvokeabanonAWS,whileothersinsistthat,withappropriateadjust-mentsinhumancontrol,autonomoussystemscouldbeemployedinkeepingwiththerulesonproportionality.165Majormilitaryforcesaroundtheworldhavealreadyinvestedincomputersystemscapableofmakingtheseestimates,onthebasisofobjectivestandardsthatcanbetranslatedintoalgorithmiclanguage(forexample,theUScollateraldamageestimationmethodology,orCDEM).166Thesetechniquesresorttosoftwareprogramstoidentifytheleastharmfultargetingoptions:thatis,assigningvaluestodifferenttypesoftargets,dependingontheircharacteristics,isalreadypossibleandiscurrentlyroutinelyactioned.However,itisobjectedthatthetypeofjudgmentsinvolvedintheproportionalitytest(reasonableness,commonsense,andsoon)areimpossibletotranslateintoalgorithmiccodes,astheyareinherentlymorality-basedjudg-mentsthatrequiretoattributea‘value’toobjectsand,importantly,persons.167Theserulesaddressallthoseinvolvedinanattack,fromthosewhoplanittothosewhoauthorizeitand,eventually,towhoconductit:giventhatrespectfortheprincipleofproportionalitymayrequireinitinere,real-timere-assessments(forinstance,owingtotheunexpectedpresenceofciviliansalteringthebalance),theproblemarisesastowhetherAWSwillbecapableofreadjustingtheirparametersaccordingly.Ourpreviousremarksvis-à-visdistinctionareapplicablehereaswell:whilecurrenttechnologyseemsunsuitableforthattask,andthushumanssteppingincannotbeavoided,advancesinAI-relatedfieldsmaymakeitfeasibleforautonomoussystemsinthefuture.168Letusimagineascenarioinwhich,inadditiontoexanteproportionalityassessmentsconductedbyhumanagents,AWSmayreadjustthebalancedirectlyinthebattlespace.Thatis,itistakenforgrantedthathumanswillnotberemovedaltogetherfromthefield—operationalreadjustmentcouldbeactionedbytheautonomoussystemathand,onconditionthathumanoperatorsremain‘intheloop’.169Theoperationwillbeevaluatedinlightofthesamecriteriaapplicable165Andersonetal(n113)403(‘[a]lthoughthereisageneralsensethatsuchexcesscanbedeterminedintrulygrosscases,thereisnoacceptedformulathatgivesdetermi-nateoutcomesinspecificcases’);SchmittandThurnher(n63)256–7(describingthenotionofexcessivenessas‘theproductofacase-by-caseassessmentthatisevaluatedintermsofitsreasonablenessgiventheattendantcircumstances’,andbelievingittobepossibletoobjectivizethenotionthroughappropriatemethodologiessuchasthe‘col-lateraldamageestimatemethodology’).SeealsoSassoli(n133)331.166JSThurnher,‘FeasiblePrecautionsinAttackandAutonomousWeapons’,inWHeintschelvonHeinegg,RFrauandTSinger(eds),DehumanizationofWarfare.LegalImplicationsofNewWeaponTechnologies(Springer2018).167Amoroso(n102)89.168Sassoli(n133)332;Wagner(n126)121.169VandenBoogaard,‘ProportionalityandAutonomousWeaponsSystems’(n159).DiegoMauri-9781802207675DownloadedfromPubFactoryat05/28/202203:48:43AMviaDukeUniversityLawLibrary

161Autonomousweaponssystemsunderinternationalhumanitarianlaw153todecisionstakenbyhumansoldiers:whethersufficientandreliableinforma-tionhasbeentakenfromtheground,whetherithasbeenelaboratedproperlyandusedreasonablyfordecidingaparticularcourseofaction.Theoutcome,forinstance,thekillingofciviliansintheclosesurroundingsofanenemyhideout,willbeassessedinitself.Again,recallhereourpreviousfindingsontheprincipleofproportionalityinIHRL:whatmattersisthathumanagentsareinapositiontounderstandhowandwhyAWSactedthewaytheydid.Iftheresultisregardedasreasonableincomparisonwiththatofahypotheticalhumanagent,thentherulesatissueherewouldbefullycompliedwith.Ashasbeennoted,‘perfection’isnotastandardinIHL;reasonableness,ensuredbythepossibilitytounderstandandexplainAWS’coursesofaction,is.170Itmustbeacknowledgedthatproportionalityrulesassignmilitarydecision-makershardtasks,namely,tostrikebalancesbetweenmilitaryneedsandhumanvalues:unlikedistinctionrules,theyaremorenuancedandultimatelyopen-natured.Thisiswhatrendersthemhighlyunsuitablefortranslationintoalgorithms.Whilethissoundsentirelyreasonable,thepointweillustratedaboveisthatwecannotruleoutthefuturedevelopmentanddeploy-mentofAWSoperatinginaccordancewithproportionalityrules.Whatmattersisthatagivenresultbecompliantwithwhatcanbereasonablyexpectedfromanattack:adequatelytrainedAWS,endowedwithmachinelearningsoftware,mayensurereliableandsatisfyinglevelsofperformance.4.3.2TheDutytoTakePrecautionsinAttackSimilartothatwhichhasbeenarguedforIHRL,thereareIHLruleswhichimposepositiveconductonpartiestoaconflictwhenplanningandexecutinganattack.Thesedutiescomplementtheprohibitionsillustratedpreviously:ifpartiestoaconflictarerequestednottocarryoutindiscriminateattacks(byadoptingindiscriminatetactics,orbycausingdisproportionatecollateraldamage,orbyemployingindiscriminatemeansandmethodsofwarfare),itseemsreasonabletorequirethattheytakeallfeasibleprecautionstomakesurethatthedeliberatedattackabidesbytherules.171Theseareobligationsofduediligence,inthatpartiestoaconflictareexpectedtocomplywiththestandardofcareenshrinedinthenorm:theyarenotunderanobligationtosucceed,but170SchmittandThurnher(n63)257(‘[n]eitherthehumannorthemachineisheldtoastandardofperfection;inthelawofarmedconflictthestandardisalwaysoneofreasonableness’).171APICommentarypara2191.DiegoMauri-9781802207675DownloadedfromPubFactoryat05/28/202203:48:43AMviaDukeUniversityLawLibrary

162154Autonomousweaponssystemsandtheprotectionofthehumanpersontoemployallmeansreasonablyavailabletotheminordertoreachthedesiredresult.172Oncloserinspection,rulesimposingthesedutiespredatepositiveobli-gationsunderIHRL.Afirstformulationoftheprecautionaryprinciplewascontainedinthe1907HagueConventiononbombardmentbynavalforces,andwaslatertakenupbyotherIHLinstruments.173Currently,mostrulesimposingprecautionsinattackareregardedascustomary.174Precautionaryrulesareintendedforthosewhoplanordecideuponanattack:accordingtosome,referenceistocommandersonly,inasmuchasplaced‘atalevelaboveanindividual’.175Assumingthis,precautionsinattackwouldbeconfinedonlytotheplanningphaseoftheoperation,leavingasidesoldiers’choiceincombat.AsexplainedintheCommentary,thisinterpretationcontrastswiththe‘verylargemajorityofthedelegationsattheDiplomaticConference’,whoseintentwastocoveralsosituations‘whichmayariseduringclosecombat’wheresplit-secondandcriticaldecisionsmustbetakenbyofficers‘ofsub-ordinaterank’.176Thispointiskeyforouranalysis:whileitisoftenheldthatAWSwillbefirstemployedinclutteredandremoteareasgenerallydevoidofcivilians(forexample,underwaterorindesertenvironments),whererespectfortargetinglawruleswouldbeensuredmoreeasily,technologicaladvanceswhichcurrentlyaremerelyworkinprogressmaymakeitpossibletoemployAWSinclosecombatsituations;prototypessuchasDARPA’sAtlastestifytothis.Thisbeingso,itseemsappropriatetounderstandprecautionaryrulesinawaythatalsoincludestheplanningandexecutionofa‘particularsortie’.177Theprovisionenshriningthedutytotakeprecautionsinattack,article57API,openswithageneralprincipleaccordingtowhich‘constantcare’shallbetakentoprotectthe‘civilianpopulation,civiliansandcivilianobject’:theconceptunderlyingtheadjective‘constant’,aswellasthebroadinterpretation172ApplicationoftheConventiononPreventionandPunishmentoftheCrimeofGenocide(BosniaandHerzegovinavSerbiaandMontenegro)[2007]ICJRep43para430.SeeCh5,s5.2.2.173HagueConvention(IX)RespectingBombardmentbyNavalForcesinTimeofWar(signed18October1907,enteredintoforce26January1910)USTS542,art2(3)(establishingadutyforthecommanderofanavalforceto‘takeallduemeasures’soasthatanundefendedtownthatisabouttobeattackedmightsuffer‘aslittleharmaspossible’).174CIHLrule15.175Andersonetal(n113)404.176APICommentarypara2197.177Boothby,‘AutonomousAttack:OpportunityorSpectre?’(n124)80–81(howevermaintainingthatonlyhumans—commanders,officers,programmers—areaddressedbytherule).DiegoMauri-9781802207675DownloadedfromPubFactoryat05/28/202203:48:43AMviaDukeUniversityLawLibrary

163Autonomousweaponssystemsunderinternationalhumanitarianlaw155oftheterm‘militaryoperation’,confirmsthatthisdutybindspartiestoacon-flictthroughoutthecombatphase.178ThefollowingprecautionaryrulescanbeidentifiedintreatyandcustomaryIHL.First,itisrequiredthatpartiestoaconflict‘doeverythingfeasible’toverifythatobjectivestobeattackedfitinthecategoryofpermissibletargets.179Thenotionoffeasibilityplaysapivotalrolehere:althoughleftundefined,itisinterpretedasreferringto‘thoseprecautionswhicharepracticableorpracticallypossibletakingintoaccountallcircumstancesrulingatthetime’inlightofcommonsenseandgoodfaith.180Asforrulesonindiscriminateattacks,precautionaryrulestoorelyonsituational,context-relatedassessmentsandarecalibratedonastandardofreasonableness(whichconceptuallyrhymeswithfeasibility).Second,therearerulesthatimposethechoiceofmeansandmethodsofwarfarewithaviewtominimizingoravoidingcollateraldamageasdescribedabove,aswellasthechoicetoengagetheobjectivewhoseselectionisexpectedtocausetheleastdangerofcollateraldamage.181Third,precautionaryrulesrequirepartiestoaconflicttobothrefrainfromattackingandsuspendorcancelattackswhenitbecomesapparentthatthesecannot(anymore)beconductedinkeepingwithrulesondistinctionandproportion-ality.182Finally,whenattacksareexpectedtoaffectthecivilianpopulation,andunlesscircumstancesdonotmakeitunfeasible,partiesareobligedtogive‘effectiveadvancewarning’inordertolimitcivilians’exposuretoavoidableharm.183Lookingattheserulesasawhole,itissafetoconcludethattheirultimateaimistooperationalizemostweaponslawandtargetinglawrulesanalyzedabove.TheuseofAWSinarmedconflictisexpectedtoraisecriticalchallengesforpartiestoaconflictthatdecidetoresorttothem.Therehavebeenvariousproposalsregardinghowtoimplementprecautionaryruleswhenspecificcriticalfunctionsareentrustedtoautonomoussystemsdoingwithouthumanintervention.Regulationcanbedrawnfromexistingrulesapplicabletootherweapons,suchaslandmines,forinstance,aslongasself-destructionandself-deactivationmechanismsarerequestedwhenfieldinglandminesinagiven178APICommentarypara2191.179APIart51(2)(a)(i).180ProtocolonProhibitionsorRestrictionsontheUseofMines,Booby-TrapsandOtherDevicesasamendedon3May1996(ProtocolII)annexedtotheCCW(adopted3May1996,enteredintoforce3December1998)2048UNTS93(AmendedProtocolII)art3(10)(emphasisadded);APICommentarypara2198.181APIarts57(2)(a)(ii)and57(3).182APIarts57(2)(a)(iii)and57(2)(b).183APIart57(2)(c).DiegoMauri-9781802207675DownloadedfromPubFactoryat05/28/202203:48:43AMviaDukeUniversityLawLibrary

164156Autonomousweaponssystemsandtheprotectionofthehumanpersonarea.184Asthoserulesaimtobindthe‘independent“operation”oflandminesinspatio-temporalterms’andlimitthetypesoftargetsthatcanbeengaged,similarrulesmayeasilyapplytoAWStotheextentthattheyoperationalizeprecautionaryduties,suchasverifyingtargetsandminimizingoravoidingcollateraldamages.185Tothisend,humanoperatorsmustalwaysbeinthepositionofdetectingandshuttingdownAWSthatmayhave‘gonerogue’.186EndowingAWSwitha‘killswitch’—actingasafail-safemechanismintheeventofpatentlyimpermissiblebehaviorbyanautonomoussystem—presentsdeveloperswithtechnologicalchallenges(forinstance,limitingrisksofspoof-ingorhacking),butitisvitaltomaintainhumanoversightandthus‘constantcare’asperthebedrockprecautionaryprinciple.187Withrespecttothis,aviewhasbeenadvancedaccordingtowhichitwouldbepossibletodrawananalogybetweenAWSand‘fire-and-forget’missiles:asthelattercannotbe‘calledback’afterfiring(achangeintheoperationalenvironmentbeingirrelevantfromthatmomenton)andtheircompatibilitywithIHLisnotputintoquestion,similarlyAWSwouldnotbeincontrastwithIHLperseifhumanoperatorsarenotinapositiontocallthembackafterfielding.Again,thisassessmentdependslargelyontheoperationalenvironmentinwhichAWSwillbefieldedandtheexpectedtargetsthesystemisgoingtoselectandengage:whiletheanalogywithfire-and-forgetmissilesmayworkformilitary-populatedenvi-ronments,unclutteredspaces—whereciviliansmayappearunexpectedly—mayrequesthumaninterventionatleastthroughanoverridedeviceordevicesentailingperiodical‘remotecheck-in’bythehumanoperator.188Takingintoaccountallthosefactors,itcanbeconcludedthatrulespertain-ingtoprecautionsinattackrequireadegreeofhumancontrolonautonomoussystemsthatiscapableofensuringinformeddecisionsaboutwhen,whereandhowtoemploythoseweapons.189HumansoperatingAWS(typically,com-manders)havetobeinapositiontounderstandtheweaponssystems’capabil-184AmendedProtocolIIarts5(2)and5(6).SeemoreextensivelyHomayounnejad(n81)andThurnher(n169).185Homayounnejad(n81)146ff.186KEgeland,‘LethalAutonomousWeaponSystemsunderInternationalHumanitarianLaw’(2016)85NordicJIL89,102.187Homayounnejad(n81).SeealsoISHenderson,PKeaneandJLiddy,‘RemoteandAutonomousWarfareSystems:PrecautionsinAttackandIndividualAccountability’,inJDOhlin(ed),ResearchHandbookonRemoteWarfare(EdwardElgar2017),369.188Homayounnejad(n81)156.189CfAndersonetal(n113)406(‘autonomoussystemsmightbebetterabletosatisfythelawinsomeusesandenvironmentsthanothers,butthatisnotamatterofprinciple;itisamatterofwhetherandhowfartechnologicalcapabilityadvancesrela-tivetothelegalstandards’).DiegoMauri-9781802207675DownloadedfromPubFactoryat05/28/202203:48:43AMviaDukeUniversityLawLibrary

165Autonomousweaponssystemsunderinternationalhumanitarianlaw157itiesinagivenscenario,andthustotakeallfeasibleprecautions.190Systemsallowingforenhancedhuman–machineteamingmayprovekeytothisend.Thisiswhytheargumentthataparticulardecisionresultingintheselectionandengagementofatargetcouldbetakenwithouthumaninterventionisnotadecisiveone:precautionarymeasurescouldbeinplacethroughwhichstatesareinapositiontofulfillrequirementsimposedbyIHL,irrespectiveofwhetherhumanagentsinterveneintheperformanceofcriticaldecisions.4.4RULESREGARDINGTHEDUTYTOINVESTIGATEANDPROSECUTEHavingterminatedouranalysisofcorerulesofweaponslawandtargetinglaw,itistimetoturntorulesregardingthedutytoinvestigateandprosecuteparticularviolationscommittedinarmedconflictunderIHL.Therationaleunderlyingthisobligationissimilartothatofthedutytoinvestigateandpros-ecuteIHRLviolationsillustratedabove:tomakesurethatviolationsofIHLcoreprovisions,suchastherulesderivingfromtheprinciplesofdistinction,proportionalityandprecautionsinattack,areproperlyrepressedbypublicauthorities.However,itmustbenotedattheoutsetthattheextensionofIHRLguaranteestoIHLviolations,mainlybywayofinterpretationbyIHRLmoni-toringbodiesandthroughsoft-lawinstruments,hasbeenharshlycriticizedbysomescholars,since,inadditiontogoingbeyondlexlata,thisoperationmayenduphaving‘chillingeffects’onpartiesengaginginhostilities,asitwouldimposeunrealisticburdensuponthem.Weareinclinedtosupportthisposition,yetwithsomeimportantadjustments.IncomparisonwithIHRLnormsregulatingthoseduties,IHLprovisionsaresignificantlyfewerinnumber,sothatithasbeenarguedthatinvestigationsinto(potentially)arbitrarydeprivationsoflifeoccurringinthebattlespacewouldbe‘concealed’indifferentnorms.191Thesecouldbeidentifiedin:theobligationtoinvestigate‘gravebreaches’inIACandmoregenerallywarcrimes(bothtreatyandcustomary);192thedutyto‘suppress’IHLviolations,whichisheldto190SeeSassoli(n133)336(notingthatinspiteofthecontentionthatonlyhumanbeingscanplananddecidetoconductanattack,itispossiblethattheydoso‘tempo-rallyandgeographicallyremovedfromtheattack,aslongastheydefinetheparametersaccordingtowhichtherobotattacks,makesurethatitcomplieswiththemandhasthenecessaryinformationtoapplysuchparameters’).191GGaggioli,‘ALegalApproachtoInvestigationsofArbitraryDeprivationsofLifeinArmedConflicts:TheNeedforaDynamicUnderstandingoftheInterplaybetweenIHLandHRL’(2017)36QuestofInt’lL27,29.192GCIarts49–50;GCIIarts50–51;GCIIIarts129–30;GCIVarts146–7;APIarts11,85,86;CIHLrule158.DiegoMauri-9781802207675DownloadedfromPubFactoryat05/28/202203:48:43AMviaDukeUniversityLawLibrary

166158AutonomousweaponssystemsandtheprotectionofthehumanpersoncoveralsoviolationsofIHLnotclassedaswarcrimes;193andonanevenmoregeneralscale,thedutyto‘respectandensurerespectforIHL’.194Othernormscouldbefoundthatlogicallyentailadutytoinvestigate.195ThecircumstancethatIHLobligationsinthisfieldareunderdevelopedasregardsanalogousobligationsunderIHRLcanbeexplainedbyapluralityofreasons.Empirically,conductinginvestigationsintopotentiallyunlawfuldeaththatoccursinthe‘fogofwar’isanarduoustask.Legally,IHLtreatiesdonotestablishmonitoringbodiesthatarecomparable—instructureorinpowers—withIHRLmonitoringbodies:theenforcementofobligationscontainedinthatbodyoflawisentrustedprimarilytodomesticcourts,andtoalimitedandpartialextenttointernationalorgans(eitherjudiciary,suchasinternationalcriminaltribunals,ornon-judiciary,suchasconciliationorenquiryproceduresortheInternationalHumanitarianFact-FindingCommission).Insummary,whatislackingisanextensivecaselawthatdealsspecificallywithIHL,withtheconsequencethatnormsregulatinginvestigationsinthebattlefieldareobscure.Forinstance,anissuethathasbeendiscussedatlengthandthatcurrentlyisnotcompletelysettlediswhethertheobligationtoinvestigatearisesalsowhenthefactsdonotamountprimafacieto‘gravebreaches’orwarcrimes.196Anotheraspectthatremainsunclearisthecontentofthedutytoinvestigate,namely,whetherIHRLrequirementsthathavebeenconsolidatedvis-à-vistherighttolifeandphysicalintegritycouldbeappliedalsotoviolationsoccurredinarmedconflicts,whereIHListherelevantframework.Tothisend,itishelpfultorecallthetheoriesthathavebeenputforwardinrespectoftherelationshipbetweenIHLandIHRL.197Accordingtosome,thelexspecialisprinciplemakesitinappropriatetorefertoIHRLstandardstogivecontenttoinvestigationsintowarcrimesoccurringduringhostilities:IHLstandards,whicharemarkedlyexiguousandmoreundevelopedthanIHRL’s,mustapply.Thus,investigationsarerequiredonlyifthereisreasonablesuspicionoracred-ibleallegationofwarcrimes,andinanyeventtheydonotimposeexhaustionofallpossibleinvestigatoryoptions;victiminvolvementorperformanceofautopsiesarenotstrictlyrequiredbyIHL.198Accordingtoothers,evenifIHListhelexspecialisdesignedtoapplyintheconductofhostilities,IHRLisnotdiscardedaltogether,butremainsrelevantwhenitcomestofillingthedutyto193GCIart49(3);GCIIart50(3);GCIIIart129(3);GCIVart146(3);APIart86.194GCCommonart1;APIart1(1).195Gaggioli(n191)30–31.196Gaggioli(n191).197SeeCh2,s2.3.3.198MNSchmitt,‘InvestigatingViolationsofInternationalLawinArmedConflicts’(2011)2HarvardNat’lSecJ39.DiegoMauri-9781802207675DownloadedfromPubFactoryat05/28/202203:48:43AMviaDukeUniversityLawLibrary

167Autonomousweaponssystemsunderinternationalhumanitarianlaw159investigatewithcontent.199GiventhepaucityofIHLrules,itwouldbeIHRLtostandaslexspecialisinthisparticularfield.Onthisbasis,oneapproachtounderstandingthecontentofthissetofdutiesunderIHListolookathumanrightsmonitoringbodies’jurisprudence.Evencourts(suchastheECtHR)thathavetraditionallydisplayedarefractoryapproachvis-à-vistheapplicationofIHL,haveinterpretedrelevantIHRLprovisioninlightoftheparticularcircumstancesofthecaseandthecontextinwhichtheallegedviolationhasoccurred.200ThejudgmentrecentlyrenderedintheHanancaseisexemplary:theECtHRwasabletoassesstheinvestigationscarriedoutbyGermanyintoanairstrikeconductedinAfghanistanbyadaptingtherequirementsunderarticle2ECHRtotheparticularcontext(ofactivehostilitiesduringNIAC).201NonormativeconflictwasheldtobeinplacebetweenIHLandIHRL:simply,IHRLrequirementsappliedfullyandrealis-tically(thatis,takingintoaccountthatcontext).202Incidentally,thesefindingsshouldreassurethosewhomaintainthatmonitoringbodies’jurisprudencealwaysimposesunrealisticburdensonstates.Thisjurisprudence,coupledwithsoft-lawdocumentsandotherlegalinstruments,hasthuscontributedtoshapingthecontentofadutytoinvestigate,andinsomecasestoprosecute,armedconflict.203Accordingtoarecentandin-depthstudyonthetopic,basic199Gaggioli(n191)50.200GenerallyontheECtHR,seeLMoir,‘TheEuropeanCourtofHumanRightsandInternationalHumanitarianlaw’,inRKolbandGGaggioli(eds),ResearchHandbookonHumanRightsandHumanitarianLaw(EdwardElgar2013).Foratheoreticalappraisaloftheapproach,seeGaggioli(n91)andLGervasoni,‘AContextual-FunctionalApproachtoInvestigationsintoRighttoLifeViolationsinArmedConflict’(2017)36QuestofInt’lL5.201HananvGermany,AppNo4871/16(16February2021).202Ibidpara199.SeealsoECtHR,Al-SkeinivTheUnitedKingdom,AppNo55721/07(7July2011)para164(‘[t]heCourthasheldthatproceduralobligationunderArticle2continuestoapplyindifficultsecurityconditions,includinginacontextofarmedconflict’);IsayevavRussia,AppNo57950/00(24February2005)paras180,210;ErgivTurkey,AppNo23818/94(28July1998);GüleçvTurkey,AppNo21593/93(27July1998).203AsforthejurisprudenceofIHRLbodies,whileitisimpossibletoprovideathor-oughlisthere,sufficeittorecallIACtHR,CaseofMapiripanMassacrevColombia,SeriesC134(15September2005)para137;CCPR,AmirovvRussia,CommNo1447/2006(22April2009)para11.4.SeealsoGC36para64;OHCHR,TheMinnesotaProtocolontheInvestigationofPotentiallyUnlawfulDeaths(2016):TheRevisedUnitedNationsManualontheEffectivePreventionandInvestigationofExtra-legal,ArbitraryandSummaryExecutions(Geneva2017)para20;UNFact-FindingMissionontheGazaConflict‘ReportonHumanRightsinPalestineandOtherOccupiedTerritories’(25September2009)A/HRC/12/48para1804;UNECOSOC‘PrinciplesontheEffectivePreventionandInvestigationofExtra-Legal,ArbitraryorSummaryExecutions’Resolution1989/65(24May1989).DiegoMauri-9781802207675DownloadedfromPubFactoryat05/28/202203:48:43AMviaDukeUniversityLawLibrary

168160Autonomousweaponssystemsandtheprotectionofthehumanpersonrequirements,suchasindependenceandimpartialityoftheinvestigations,thoroughness,promptness,transparency,involvementofnextofkinandthusfairtrialguaranteesinthewholeproceedings,whichhavebeenelaboratedinIHRL,applyalsotodeathsoccurringinarmedconflict.204Thatis,thestandardagainstwhichinvestigationsregardingactscommittedduringhostilitieshavetobeassessedisoneofeffectiveness,thatis,thecapabilityofestablishingfactsandidentifyingthoseresponsible.205Inourview,thecautiondisplayedsofarbyIHRLmonitoringbodiesindicatesthattheriskofpositingimpractical(ifnotdangerous)standardstoinvestigatingauthorities—oneoftheargumentsadvancedtoretainthelexspecialisapproachinfavorofIHL—maybeoveres-timated.However,itmustbeacknowledgedthatthematterisnotsettled,andthatthejurisprudenceofthosebodiesmustbeassessedagainstsomeresistancebystates.Thatis,doubtsremainastothelexlatanatureofthosefindings.Againstthisbackground,andirrespectiveoftheduty’sbreadth,thedevel-opmentanddeploymentofAWSmustensurerespectforsomebasicrequire-ments.Ashasbeenoutlined,theobligationtoinvestigateinto(potential)violationsofIHLamountingtowarcrimespresupposeslogicallysomesteps,suchastheadequaterecordingattheearliestpossibletimeofmilitaryopera-tions.206Recordingisacriticalfactorforassessingtheeffectsofoperationalactivity,enablinglesson-learninganddetectingpossiblesystemicshortcom-ingsofthemilitaryplanning.TheuseofAWSmayprovebeneficialfortheoverallimprovementofmilitarysorties:thosesystemscouldbeequippedwithsoftwareallowingforthecollection,storage,re-elaborationandretrievalofvastamountsofdata.DatacollectedbyAWSfromtheoperationalfieldandtransmittedtothecommandermaybehelpfulfortakingactionsatthesceneofanincident,reportingpromptlytothecompetentauthorities,andsupportingthelatterinconductingalltheassessmentsofthecase,inordertodecidewhethertolaunchacriminaloradministrativeinvestigation.207Subsequenttothesepreliminarysteps,investigationsmustbeconductedinaneffectiveway,whichrequiresthathumanoperatorsaccessingdatagatheredbyAWStobeinapositiontounderstandandexplainhowandwhytheauton-omoussystembehavedinaparticularcontext.ThisholdstrueirrespectiveofthecompetingviewsontheextentofthedutytoinvestigateinIHL(namely,whetheritislimitedtowarcrimesoralsocoversotherviolations).Thisispar-204SeeGenevaAcademyofInternationalHumanitarianLawandHumanRightsandICRC,GuidelinesonInvestigatingViolationsofInternationalHumanitarianLaw:Law,Policy,andGoodPractice(September2019)(GuidelinesonInvestigating).205ECtHR,FinogenovandOthersvRussia,AppsNos18299/03and27311/03(20December2011)para269.206GuidelinesonInvestigating,guideline1.207Ibidguidelines2–6.DiegoMauri-9781802207675DownloadedfromPubFactoryat05/28/202203:48:43AMviaDukeUniversityLawLibrary

169Autonomousweaponssystemsunderinternationalhumanitarianlaw161ticularlytruefortherequirementoftransparency,whichimposestheprovisionofinformationbothtovictimsandtheirnextofkin,andtothegeneralpublicforoverallscrutiny.208Ifthealgorithmicprocessthathasledtoaparticulartargetselectionandengagementinthebattlefieldremainsobscure,inthathumanoperatorscannotadequatelyelaboraterelevantinformationconcerningthatcourseofaction,thedutytoinvestigateandprosecutemaybehinderedinpractice.Autonomoussystemsemployingmachine-anddeep-learningalgorithmswillraisecriticalissuestothisend.Ourpreviousremarksvis-à-vistheobligationtoinvestigateandprosecuteunderIHRLapplyhere:thedutytoexplainhowanautonomoussystemhasbehavedinaparticularcontexthastobeappraisedthroughthelensofthesetypesofalgorithms,thustakingintoaccounttheirstructureandtheirlimits.209‘Openingtheblackbox’mightbehard:henceappropriatesafeguardsneedtobeinplacethatmakesurethathumanoperatorsunderstandandexplainmachinebehavior.Thedifferencebetweenlaw-enforcementscenariosandarmed-conflictscenariosliesinthecircumstancesrulingatthetimeinwhichanallegedviolationiscommitted:itiswidelyacknowledgedthat,owingtothepeculiarcontext,constraintsandchallengesmayposeobstaclestopublicauthorities’investigations.Inconclusion,inorderforAWStobeoperatedincompliancewiththedutytoinvestigateIHLviolations(inbothitsminimalisticandmaximalistunder-standing,aselucidatedabove)andprosecutethoseresponsibleaccordingly,whatmattersisthatanadequatelevelof‘humancontrol’—inthatparticularcoursesofactionundertakenbytheautonomoussystemcanbeunderstoodandexplained—isensured.Inlinewithourpreviousremarks,humaninterventioninthespecificcriticalfunctionwouldbeunnecessary,providedthatadequatesafeguardsareinplaceensuringtheeffectivenessoftheinvestigationandthesubsequentprosecution.4.5HUMANITYASAGENERALPRINCIPLERELEVANTFORTHEUSEOFFORCETherulesthathavebeendiscussedthusfardonotprohibitAWS.Instead,theyrequireadegreeof‘humancontrol’overcriticalfunctionsthatissufficientforensuringadequateunderstandingofhowAWSperforminthebattlefield.TohavehumanoperatorsinapositiontounderstandcoursesofactionofAWSandtoexplaintheirparticularbehavior—exantefactoandexpostfacto—iskeytoensurerespectfortherulesprohibitingindiscriminateweaponsandattacks,imposingadutyofprecautionandadutytoinvestigateandprosecute208Ibidguideline10.SeealsoHananvGermany(n188)paras230ff.209SeeCh3,s3.2.2.2.DiegoMauri-9781802207675DownloadedfromPubFactoryat05/28/202203:48:43AMviaDukeUniversityLawLibrary

170162AutonomousweaponssystemsandtheprotectionofthehumanpersonparticularIHLviolations.Takingonestepfurther,thoserulesrevolvearoundstandards,suchas‘reasonableness’,‘feasibility’and‘goodfaith’,whichhavetobeappliedinconcreto:whatmattersistheresultsobtainedinthebat-tlespace,ratherthanthatahumandecision-makeractually‘pullsthetrigger’.Thatis,respectforthoseIHLrulesdependsonhowforceisemployedinarmedconflict,ratherthanwho—orwhat—materiallyemploysit.InawaythatmirrorsourpreviousremarksinChapter3,however,itiseasytoseethatthoserulesaresomehowunder-inclusivevis-à-visthemostsalientandunprecedentedfeatureofAWS,thatis,thataparticularoutcome(thatis,thedecisiontoselectandengageaspecifictarget)isdeliberatedwithouthumanintervention.210‘Humancontrol’,ensuredinthewiderloop,willnotmaterializeinahumanoperator’sinterventioninthecriticaldecision.Legally,scenarioslikethisaretherealmofprinciples,notrules.211ItfollowsthatinordertoevaluatethesetechnologicaladvancesthroughthelensofIHL,aninquiryintoprinciplesofthatbranchofinternationallawcannotbedonewithout.Whenitcomestoarmedconflict,arguablyaprincipleof‘humanity’canbesaidtoexist;itevenpredatesthatofhumandignityinIHRL.Thissectionshedslightonthisprinciple,first,byillustratingitshistoricalrootsinIHL(4.5.1),second,bytestingitsnormativecharacteronthebasisofinter-nationaljurisprudenceandliterature(4.5.2),andfinally,byarguingforitsrelevanceinthefieldofAWS(4.5.3).4.5.1HistoricalOverview:TheMartensClause‘[H]ailedasasignificantturningpointinthehistoryof[IHL]’,212theMartensClause—namedaftertheRussiandelegatetotheHaguePeaceConferencewhoproposedit—encapsulatestheprincipleofhumanityinawrittenprovi-sion.213Accordingtoitsfirstformulationinthepreambleofthe1899HagueConvention(II),theclauseestablishesthat,[u]ntilamorecompletecodeofthelawsofwarisissued,…incasesnotincludedintheRegulations…populationsandbelligerentsremainundertheprotectionandthe210SeeCh3,s3.5.211LGradoni,‘Consuetudineecasoinconsueto’(2012)95Rivistadidirittointerna-zionale704.212ACassese,‘TheMartensClause:HalfaLoaforSimplyPieintheSky?’(2000)11EJIL187,188.213ForbasicliteratureontheClauseanditsproponents,seeMSalter,‘ReinterpretingCompetingInterpretationsoftheScopeandPotentialoftheMartensClause’(2012)17JofConflict&SecL403;TMeron,‘TheMartensClause,PrinciplesofHumanityandDictatesofPublicConscience’(2000)94AJIL78;VPustogarov,‘F.F.Martens(1845–1909),aHumanistofModernTimes’(1996)312IRRC300;SMiyazaki,‘TheDiegoMauri-9781802207675DownloadedfromPubFactoryat05/28/202203:48:43AMviaDukeUniversityLawLibrary

171Autonomousweaponssystemsunderinternationalhumanitarianlaw163empireoftheprinciplesofinternationallaw,astheyresultfromtheusagesestab-lishedbetweencivilizednations,fromthelawsofhumanity,andtherequirementsofthepublicconscience.214TheoriginalpurposeoftheclausedoesnotappearasnobleasonemayinferfromitscontinuingimportanceinIHL.TheHaguePeaceConferencewasdiscussingthetreatmentofcivilianstakinguparmsagainstanoccupyingforcewhenadiplomaticimpassearosebetweenthedelegates.Ontheonehand,somestates(especiallymilitarypowers)suggestedtheyshouldberegardedasfranc-tireursandtreatedaccordingly(namely,subjectingthemtocapitalpunishment);ontheotherhand,smallerstates,fearingthattheirterritoriescouldbecomethetheaterofmilitaryoccupation,believedittobemoreappropriatetoextendthestatusofcombatantstothem.215Theclause,whichwasthenproposedasamerecompromisebetweenthesepositionsandpurposelydraftedinvagueterms,wasintendedtoestablishthattheconductofhostilitiesisgovernedbyinternationallawevenabsentspecificIHLrules.216Itactsasapermanentreminderthatwhatprovestobeinhumaneandabhorrentinwarcouldneverbeconsideredlegal,evenifthishasnotbeentranslatedintoatreatyorcustomarynorms.217TheMartensClauseisrepeatedalmostverbatiminnumerousinternationallegalinstruments,suchasthepreambleofthe1907HagueConventionIV,218inthe1949GenevaConventions(intheprovisionsregulatingdenunciationsofthosetreaties),219the1977AdditionalProtocols220andtheCCW.221Inspiteofitsultra-pragmaticorigins,theMartensClauseplayedakeyroleinthedevelopmentofIHL,forinstance,byboostingbroaderandmoreMartensClauseandInternationalHumanitarianLaw’,inCSwinarski(ed),StudiesandEssaysonInternationalHumanitarianLawandRedCrossPrinciplesinHonourofJeanPictet(MartinusNijhoff1984).214Convention(II)withRespecttotheLawsandCustomsofWaronLandanditsAnnex:RegulationsconcerningtheLawsandCustomsofWaronLand(adopted29July1899,enteredintoforce4September1900)187CTS429(HagueConventionII),preamble.215KalshovenandZegveld(n34),11–2.216Cassese,‘TheMartensClause’(n212)46,54.217Ibid69(concludingthatinspiteofitsbroadlydiplomaticratherthanstrictlyhumanitarianrationale,theClausehasbeennonethelessemployedtopromoteabetterprotectionofhumandignity).218Convention(IV)respectingtheLawsandCustomsofWaronLandanditsAnnex:RegulationsconcerningtheLawsandCustomsofWaronLand(adopted18October1907,enteredintoforce26January1910)205CTS277.219GCIart63;GCIIart62;GCIIIart142;GCIVart158.220APIart1.221CCW,preamblepara5.DiegoMauri-9781802207675DownloadedfromPubFactoryat05/28/202203:48:43AMviaDukeUniversityLawLibrary

172164Autonomousweaponssystemsandtheprotectionofthehumanpersonhuman-orientatedinterpretationsofexistinglawandthusenhancingthepro-tectionofthehumanpersonduringhostilities.222The‘principlesofhumanity’and‘dictatesofpublicconscience’aretheveryheartofthewholeedificeofIHL.Consideringitscontent,theclausecanbeunderstoodasrejectingthe‘Lotusprinciple’,accordingtowhich‘whateverisnotexplicitlyprohibitedbyinternationallawispermitted’,inIHL:statesovereigntyinchoosinghowtoconductwarfareisnotunlimited,evenabsentatreatyorcustomarylaw.223Theideaof‘limitation’tosovereigntyhasbeenconsistentlyreaffirmedbytheICJthroughoutthepastcenturywhenitconsideredthatcertainpractices(suchasthelackofnotificationoftheexistenceofaminefieldinterritorialwaters)runagainst‘elementaryconsiderationsofhumanity’,224consideredas‘generalandwell-recognizedprinciples’goingbeyondwrittenlaw.225Thelimitstostatesovereigntyareparticularlyvisibleasfarasweapons,meansandmethodsofwarfareareconcerned.Asahistoricalexample,onemaythinkofcrossbows,whichwerebanned(temporarily)bytheSecondEcumenicalLateralCouncilin1139becauseoftheinequalitiestheygeneratedintherigidchivalricsocietyofthatepoch.226Comingtomorerecentexamples,theStPetersburgDeclarationrepresentsthefirstformalagreementprohibitingtheuseofcertainweaponsinwar,namely,smallexplosiverifleprojectiles.Althougheffectiveagainstobjects,whenusedagainstcombatantstheycausedheavierinjuriesthanothertypesofbullet,equallyeffectiveindisablingtheenemy.Twopointsdeserveattention.First,inthetextoftheDeclarationitisemphaticallystatedthat‘theprogressofcivilizationshouldhavetheeffectofalleviatingasmuchaspossiblethecalamitiesofwar’,thusequatingWesternprogresswithhumanizationofwarfare(tobereachedthroughprohi-bitionofde-humanizingtechnology).Aspreviouslyargued,theStPetersburgDeclarationgavefleshtothegeneralprohibitiononweaponscausingunnec-222Salter(n213)404.223Foranoverviewofthisprinciple,seeAHertogen,‘LettingLotusBloom’(2015)26EJIL901.224SeeCorfuChannel;MilitaryandParamilitaryActivitiesinandagainstNicaragua(NicaraguavUSA)(Merits)[1986]ICJRep226;LegalityoftheThreatorUseofNuclearWeapons(n40).225PMDupuy,‘Les“considerationsélémentairesd’humanité”danslajurisprudencedelaCourinternationaledeJustice’,inRJDupuy(ed),MélangesoffertsàN.Valticos.Droitetjustice(Pedone1999);PMDupuy,‘L’unitédel’ordrejuridiqueinternational,CoursGénéraldedroitinternationalpublic’(2002)297RecueildesCours9,186–7(tracingtheiroriginbacktoasortofmoralityandpublicorder).Foracomparisonbetween‘elementaryconsiderationsofhumanity’andtheMartensClause,seeCassese,‘TheMartensClause’(n212)67.226SeeCh1.DiegoMauri-9781802207675DownloadedfromPubFactoryat05/28/202203:48:43AMviaDukeUniversityLawLibrary

173Autonomousweaponssystemsunderinternationalhumanitarianlaw165essarysufferingandsuperfluousinjury.227Second,statesacknowledgedthatadegreeofopennesstotechnologicaladvancementshadtobeensured:‘futureimprovementswhichsciencemayeffectinthearmamentoftroops’228willrequirenewdeclarations,upgradesandmodifications,inordertorendertheabove-mentionedprocessofhumanizationeffective.The1899/1907HagueConferencesaddednumerousprohibitionsonweapons,meansandmethodsofwarfare.Ofparamountimportanceisthe1907HagueConventionIVonwaronlandwithitsannexedregulations,wheretheruleaccordingtowhich‘therightofbelligerentstoadoptmeansofinjuringtheenemyisnotunlimited’wasfirstelaborated.229Thesameruleisre-statedinAPI.230Onthisbasis,APICommentaryinsiststhatthechoiceofweapons,meansandmethodsofwarfareissubject,interalia,to‘theMartensClause’asenshrinedinarticle1.231Insummary,theclause’shistorydemonstratesthattheunderlyingprinciplesofhumanityanddictatesofpublicconsciencehaveplayedaremarkableroleinprogressingIHL:bornoutoftheneedtobreakadiplomaticdeadlock,itwaslatertranslatedintopreambularclausesandeventreatyprovisions.Whatisdebatedisthelegalsignificanceofthoseinvocationsofhumanity,towhichouranalysisnowturns.4.5.2TheLegalRelevanceofHumanityinIHLThelegalcharacter—thatis,thebindingforce—oftheprincipleofhumanityisoneofthemostdiscussedtopicsinIHL,notonlyforitsintriguinglythe-oreticalcharacter,butalsofortheactualconsequencesofacknowledgingit.Normatively,ithasbeenarguedthattheMartensClauseplaysinIHLarolethatisanalogoustothatplayedbyarticle38oftheICJStatuteingeneralinternationallaw:enumeratingthesourcesofthelegalframeworkathand.232Whilethiscouldhardlybedisputed,thepointremainsastothelegalforceoftheprincipleofhumanityasreferencedintheMartensClause.Thedebateonthisispolarized,andapluralityofdoctrinalandjurisprudentialpositionshasemergedthusfar.Whileitisimpossibletoprovideacomprehensiveoverview227Seesupra4.2.228StPetersburgDeclaration(n33)297.229HagueConvention(IV)art22.230APIart35.231APICommentarypara1406.232JCvandenBoogaard,‘FightingbythePrinciples:PrinciplesasaSourceofInternationalHumanitarianLaw’,inMMatthee,BToebesandMBrus(eds),ArmedConflictandInternationalLaw:InSearchoftheHumanFace(TMCAsserPress2013),20.DiegoMauri-9781802207675DownloadedfromPubFactoryat05/28/202203:48:43AMviaDukeUniversityLawLibrary

174166Autonomousweaponssystemsandtheprotectionofthehumanpersonofallofthem,itseemsconvenienttosketchatleastapictureofthatdebate.233Thiswillprovidesupporttotheargumentadvancedhere,namely,thattheprin-cipleofhumanitycanberegarded—andthusappliedinthefieldofAWS—asa‘generalprinciple’ofIHL,analogouslytotheprincipleofhumandignityinIHRL.Atoneendofthespectrum,therearethosewhodenytheprincipleofhumanityanyautonomouslegallybindingeffects.234Intheirview,theprinci-pleisincapableofprohibitingaspecificweapon,meansandmethodofwarfareperse,asconfirmedbythecircumstancethatthusfarnoneofthemhaseverbeenconsideredillegalonthesolebasisoftheMartensClause.235Instead,bothdomesticandinternationalcaselawseemtoresorttotheclauseeithertoconfirmasolutionalreadyreachablethroughtheapplicationofexistinglaw,236ortoadvanceanewinterpretationofthelaw,237orultimatelytorejectacon-trarioarguments.238Thispositionfindssupportamongsomestates.WrittenandoralsubmissionstotheICJintheLegalityoftheThreatorUseofNuclearWeaponscasegiveimportantindicationsonstates’practice,numerousdelega-tionscontrastinganyclause’sbindingeffectexpressly.239However,onemay233Forfurtherreflections,seealsoECarpanelliE,‘GeneralPrinciplesofInternationalLaw:StrugglingwithaSlipperyConcept’,inLPineschi(ed),GeneralPrinciplesofLaw–TheRoleoftheJudiciary(Springer2015).234SeeGAbi-Saab,‘TheSpecificitiesofHumanitarianLaw’,inCSwinarski(ed),StudiesandEssaysonInternationalHumanitarianLawandRedCrossPrinciplesinHonourofJeanPictet(MartinusNijhoff1984),274–5.SeeYDinstein,‘PrincipleofProportionality’,inKMujezinovicLarsen,CGuldahlCooperandGNystuen(eds),Searchingfora‘PrincipleofHumanity’inInternationalHumanitarianLaw(CUP2013),73(contendingthattheprincipleofhumanity‘isnotanobligation(orasetofobligations)incorporatedperseinpositiveIHL’);KMujezinovicLarsen,‘A“PrincipleofHumanity”ora“PrincipleofHuman-rightism”?’,inKMujezinovicLarsen,CGuldahlCooperandGNystuen(eds)Searchingfora‘PrincipleofHumanity’inInternationalHumanitarianLaw(CUP2013),143(consideringtheprincipleashaving‘little(ifany)legaleffect’andso‘vagueanduncertain’thatitcould‘atbest,beper-ceivedasamoralobligation’).235Cassese,‘TheMartensClause’(n212).236SeeKlinge(SupremeCourtofNorway)(1946)13AnnualDigestPublicIntL263;Rauter(SpecialCourtofCassationofHolland)(1949)16AnnualDigestPublicIntL526.Forinternationalcaselaw,seeProsecutorvMartić,ICTY-95-11-R61(8March1996)para13.237K.W.(ConseildeGuerredeBruxelles)(1950)17ILR388.238SeecasesquotedinCassese,‘TheMartensClause’(n212)60.Alongthisline,seealsoGSchwarzenberger,‘TheLegalityofNuclearWeapons’(1958)11CurrentLegalProblems258(highlightingthattheclausewasadoptedwithaviewtoprevent-ing‘anunintendedandcynicalargumentacontrario’).239LegalityoftheThreatorUseofNuclearWeapons(n40);Cassese(n212)andreferencestherein.DiegoMauri-9781802207675DownloadedfromPubFactoryat05/28/202203:48:43AMviaDukeUniversityLawLibrary

175Autonomousweaponssystemsunderinternationalhumanitarianlaw167questionwhethersuchanapproachtotheclausedoesnotendupneutralizingthescopeoftheprovision,whichwouldstandasamerere-statementofexist-ingnorms.Itistruethattheclausemakesanexplicitreferenceto‘establishedcustom’,thatiscustomarylaw:ifaweapon,meansormethodofwarfareisnotproscribedbytreatylaw,noonewoulddoubtitmaystillbeunlawfulifitisnotconsistentwithacustomarynorm.Iftheclauseterminatedhere,itwouldbeatbestredundant.Yet,theadditionalreferenceto‘principlesofhumanityanddictatesofpublicconscience’impliesthatothersourcesoflawareconsidered:onemaythereforewonderwhetheranotherinterpretationoftheclausehastobepreferred.Relyingonthisargument,asecondgroupofinterpreterssuggeststhattheclausehasanimpactonthesourcesofinternationallaw.240Tobeginwith,authorsthataremoreinclinedtonaturallawstrivetodefendamoreincisiveroleforitininternationallaw-making,andthusconsidertheclauseasanextra-positivesourceoflaw.241O’Connellproposesarevitalizedunderstand-ingofnaturallawasthebasisfortheprohibitionontheuseofforce;inherview,therearerulesandprinciplesininternationallawthatemanatefromextra-positivelaw,thatislaw‘beyondconsent’,theMartensClausefallingwithinthiscategory.242CançadoTrindadearguesthat‘humanconscience’isa‘source’ofinternationallaw,243andfindsthatthe‘publicconscience’aspertheMartensClausemayworkasanatural-lawsourceendowedwithbindingforce.244Salteremphasizesthe‘natural-lawdimensions’oftheclause,andarguesthat‘ithascometooperateasatranslatorofmoralimperativesintoconcretelegaloutcomes’,therebysupportingnotonlyitsroleasjudicialaide,butalsoas‘norm-creatingprinciple’.245Moodrick-EvenKhenconsidersIHLasalegalsystemimbuedwithmoralityandthusconsiderstheclauseasafeatureparticularlyrevealingthetightconnectionwithextra-positivevalues.246SomejudgesattheICJattimesexplicitlyorimplicitlyreferredtoextra-positivelaw240Cassese(n212)42–3(comparingandcontrastingscholarshiponthepoint);RTicehurst,‘TheMartensClauseandtheLawsofArmedConflict’(1997)317IRRC125.241MEO’Connell,TheArtofLawintheInternationalCommunity(CUP2019)19ff.242Ibid80ff.243AACançadoTrindade,InternationalLawforHumankind:TowardsaNewJusGentium(MartinusNijhoff2010)139.244AACançadoTrindade,‘SomeReflectionsonthePrincipleofHumanityinitsWideDimension’,inRKolbandGGaggioli(eds),ResearchHandbookonHumanRightsandHumanitarianLaw(EdwardElgar2013).245Salter(n213)437.246HMoodrick-EvenKhen,‘AidosandDikeinInternationalHumanitarianLaw:IsIHLaLegaloraMoralSystem?(2016)99TheMonist26,34.DiegoMauri-9781802207675DownloadedfromPubFactoryat05/28/202203:48:43AMviaDukeUniversityLawLibrary

176168Autonomousweaponssystemsandtheprotectionofthehumanpersontosupporttheirposition.Forinstance,JudgeTanakaarguedthatthecategoryofgeneralprinciplesincludesalso‘naturallawelement’going‘beyondthelimitoflegalpositivism’.247Alongthesameline,inhisdissentingopinioninNuclearWeapons,JudgeKoromalabelledthequestforaspecificlegalprohi-bitiononweapons,meansandmethodsofwarfare(namely,nuclearweapons)as‘anextremeformofpositivism’.248JudgeShahabuddeenarguesthatboth‘principlesofhumanity’and‘dictatesofpublicconscience’operateasnor-mativesourcesofinternationallaw,albeitacknowledgingthattheirprecisecontenthastobe‘ascertainedinthelightofthechangingconditions’.249Finally,MerontooseemstoadmitthattheMartensClausecanplayanorma-tiverole.Afterassertingthatit‘doesnotallowonetobuildcastlesofsand’,heconcedesthat‘[e]xceptinextremecases,itsreferencestoprinciplesofhuman-ityanddictatesofpublicconsciencecannot,alone,delegitimizeweaponsandmethodsofwar,especiallyincontestedcases’.250Itislegitimatethoughtowonderonwhatbasesalegalsourcecansometimes(veryoften)bevoidofbindingeffect,andsometimes(quiterarely)havethatveryeffect.Eithertheprincipleofhumanityisabindingsourceoritisnot:tertiumnondatur.Cassesehimself,whileremainingloyaltoapositivistapproachtotheclause,acknowledgesthat‘Martensdeservescreditforcraftingsuchaningen-iousblendofnaturallawandpositivism’,having,probablyunknowingly,approachedmoralprinciplesfroman‘apparentlypositivist’perspective.251CasseseapproachestheMartensClauseaslexspecialisincustomarylaw:thetraditionalcomponentsofcustoms—namely,ususandopinion—aremain-tainedbutappraisedinamethodologicallydifferentfashion.Hearguesthattheclause‘loosenstherequirementsprescribedforusus,whileatthesametimeelevatingopinio(jurisornecessitatis)toarankhigherthanthatnormallyadmitted’.252Importantly,thisviewhasbeentakenupalsobytheICTYintheKupreškićcase,whereitheldthatIHLprinciples‘mayemergethroughacus-tomaryprocessunderthepressureofthedemandsofhumanityorthedictatesofpublicconscience,evenwhereStatepracticeisscantorinconsistent’.253It247SouthWestAfrica(EthiopiavSouthAfrica;LiberiavSouthAfrica)(Judgment)[1966]ICJRep285,DissentingOpinionofJudgeTanakapara315.248LegalityoftheThreatorUseofNuclearWeapons(n40),DissentingOpinionofJudgeKoromapara575.249IbidDissentingOpinionofJudgeShahabuddeenpara406(‘theprincipleswouldremainconstant,buttheirpracticaleffectwouldvaryfromtimetotime:theycouldjustifyamethodofwarfareinoneageandprohibititinanother’).250Meron,TheHumanizationofInternationalLaw(n20)88(emphasisadded).251Cassese,‘TheMartensClause’(n212)40.252Ibid67(comparingtheroleplayedbytheMartensClauseinIHLtothatof‘ele-mentaryconsiderationsofhumanity’ingeneralinternationallaw).253ProsecutorvKupreškićetal,ICTY-95-16(14January2000)para527.DiegoMauri-9781802207675DownloadedfromPubFactoryat05/28/202203:48:43AMviaDukeUniversityLawLibrary

177Autonomousweaponssystemsunderinternationalhumanitarianlaw169hasbeenconvincinglynotedthatinthefieldofICLatrendof‘moralization’isundergoing,tothepointthateventheprocessofascertainingcustomarylawisaffectedbythoseprinciples(opinioacquiringmoreweightthanpractice).254Usingthesamelineofreasoning,BenvenutiarguesthattheMartensClause,incodifying‘principlesofhumanity’and‘dictatesofpublicconscience’,referredto‘substantialvalues,inherentinindividualandgrouprelations[that]…byvirtueoftheirfoundations…imposethemselves,bytheirownforce,inthelegalorder’.255Those‘principles’and‘dictates’constitutethe‘universal,andatthesametimehistoricallydeterminedfoundation’ofalltreatyandcustom-aryIHL.256Theselastviewsontheprincipleofhumanityareparticularlyconvincing.Inmostcases,existingIHLrulesproscribeweapons,meansandmethodsthatareatvariancewiththeprincipleofhumanity.Therecouldbecases,however,wherethoserulesdonotcoveraspecificweaponwhosefeaturesnonethelessappearinconsistentwiththeunderlyingideaofhumanity.Thesecasesthusqualifyasunusual:somesalientfeaturesofaweaponarenotcoveredbyexistingrules.Itcanthusbearguedthattheprincipleofhumanityasencap-sulatedintheMartensClauseoperatesasageneralprinciple‘formedwithintheinternationallegalsystem’,andmorepreciselyofIHL.257Inthis,theideathathumanitywouldstandonlyasa‘pillar’(ratherthana‘principle’)ofIHLdoesnotsoundpersuasive.258TheongoingworksoftheILC,259inadditiontocertainstatepractice,260maybeinvokedtothisend.InitsrecentcommentarytoGCIII,theICRChasendorsedsuchaconstructionoftheprincipleof254MFrulli,‘TheContributionofInternationalCriminalTribunalstotheDevelopmentofInternationalLaw:TheProminenceofopiniojurisandtheMoralizationofCustomaryLaw’(2015)14TheLandPracticeofInt’lCourtsandTribunals80.255PBenvenuti,‘LaClausolaMartenselatradizionestoricadeldirittonaturalenellacodificazionedeldirittodeiconflittiarmati’,inScrittidegliallieviinmemoriadiGiuseppeBarile(CEDAM1995)(alltranslationsmine).256Ibid179.257ILC,‘SecondReportonGeneralPrinciplesofLawbyMarceloVásquez-Bermúdez,SpecialRapporteur’(9April2020)A/CN.4/741(SecondReport).258CfEWinter,‘PillarsnotPrinciples:TheStatusofHumanityandMilitaryNecessityintheLawofArmedConflict’(2020)25JofConflictandSecL1.259SecondReport(n257)paras131–4.260LegalityoftheThreatorUseofNuclearWeapons(n40),Letterdated15June1996fromcounselappointedbyNauru,13(yetemployingthecategoryofgeneralprinciplesoflawrecognizedbycivilizednations);StatementbyNorway(onbehalfofNordiccountries)A/C.6/74/SR.31para77(claimingthattheMartensClausehadtobeseenas‘anexampleofaprinciplethathadbeenformedwithintheinternationallegalsystem’).DiegoMauri-9781802207675DownloadedfromPubFactoryat05/28/202203:48:43AMviaDukeUniversityLawLibrary

178170Autonomousweaponssystemsandtheprotectionofthehumanpersonhumanity.261Moreover,thejudgmentrenderedintheKruppcaseconfirmsthecontinuedlegalrelevanceoftheprincipleofhumanityincasesnotenvisagedbyexistingrules—herebyconceivingitasageneralprinciple:byholdingthattheprincipleofhumanityasreferredtointhepreambleofHagueConvention(II)was‘muchmorethanapiousdeclaration’,itaffirmedthatthisprinciplehadbeenincorporated‘intothelegalyardsticktobeappliedifandwhenthespecificprovisions…donotcoverspecificcases’.262TheICJ,intheNuclearWeaponscase,alsoreferredtohumanity,yetwithoutenumeratingitinthelistofthe‘cardinalprinciples’ofIHL.263Moregenerally,ithasbeendemonstratedthatgeneralprinciplesoperateasagap-fillerofexistingrulesinIHL,espe-ciallywithregardtotheapplicabilityofIHLrulesregulatingIACtoNIACortopeacekeepingoperations.264Insummary,generalprinciplescanbeidentifiedinIHLinawayanalogoustoIHRL.Imbuedasitiswithcorevalues,humanitycanbetreatedasageneralprincipleofIHL.Nothing,thus,preventstheapplicationoftheprincipleofhumanitytocaseswhereexistingIHLrules(containedinbothweaponslawandtargetinglaw)failstoadequatelycoverallsalientfeaturesofnewweapons—asoccurswithregardtoAWS.4.5.3HumanityintheDebateonAWSInthe(laconic)wordsoftheICJintheNuclearWeaponscase,humanity‘hasprovedtobeaneffectivemeansofaddressingtherapidevolutionofmilitarytechnology’.265ThissufficestodemonstratetherelevanceoftheprincipleofhumanityforthepurposesofestablishingwhetherandtowhichconditionstheuseofAWScomplieswithIHL.TheprincipleofhumanityhasbeenabundantlyinvokedinthedebateonAWS,byboththesupportersandtheopponentsofthistechnology.Theissuethatremainstobesettlediswhethertheserepeatedinvocationsareconvincingfromalegalstandpoint.261ICRC,CommentarytotheThirdGenevaConvention.Convention(III)fortheTreatmentofPrisonersofWar(CUP2021)paras5523ff.262Kruppetal,Judgmentof31July1948,USMilitaryTribunal,reprintedinAnnualDigestsandReportsofPublicInternationalCases(Lauterpachted,1948)620,622.263LegalityoftheThreatorUseofNuclearWeapons(n40)paras78–9.264VandenBoogaard,‘FightingbythePrinciples’(n232).265LegalityoftheThreatorUseofNuclearWeapons(n40)para78.DiegoMauri-9781802207675DownloadedfromPubFactoryat05/28/202203:48:43AMviaDukeUniversityLawLibrary

179Autonomousweaponssystemsunderinternationalhumanitarianlaw171Statesreferringtotheprincipleofhumanityatlargehavebeennumerousthusfar:266tonameafew,Brazilatthe2014MoE267andatthe2017and2019GGE;268SriLanka269andEcuador270atthe2015MoE;Australia271andSierraLeone272atthe2016MoE;andSriLankaagainatthe2017GGE.273AnotherstatethathasrepeatedlyinvokedtheprincipleofhumandignityasenshrinedintheMartensClauseistheHolySee.274InappealingtotheMartensClause,thesestatestendnottodistinguishbetweenlegalandmoralarguments:inmostcasestheprincipleofhumanityisseenonlythroughthelensofmorality.Nordostatesrefertoaparticularmeaningoftheprinciple:theycontentthemselveswithaffirmingthat‘principlesofhumanity’and‘dictatesofpublicconscience’couldrunagainstAWS,withoutspecifyingwhy.Moreimportantly,theClauseisoftenresortedtoasadelegeferendainstrument.275Owingtoitsevidentlymoralappeal,theprincipleofhumanityrepresentsakeyargumentforNGOsopposingAWS.HumanRightsWatchhasrepeat-edlyreferredtotheclauseinitsreports,enlistingitasoneofthegroundsagainstwhichthelegalityofAWShastobechecked.Itisacknowledgedthat‘thereisnoconsensus’among‘expertsandlaypeople’onwhetherautonomouskillingispermissible;however,‘thereiscertainlyalargenumberforwhom266ICRC,‘EthicsandAutonomousWeaponSystems:AnEthicalBasisforHumanControl?’,Report(3April2018)para15(listingthestatesthathavegenerallyreferredtotheclauseoritscontent:Algeria,Argentina,Austria,Belarus,Brazil,Cambodia,CostaRica,Cuba,Ecuador,Egypt,France,Germany,Ghana,HolySee,India,Kazakhstan,Mexico,Morocco,Nicaragua,Norway,Pakistan,Panama,Peru,RepublicofKorea,SierraLeone,SouthAfrica,SriLanka,Sweden,Switzerland,Turkey,Venezuela,ZambiaandZimbabwe).267Brazil,Statementatthe2014MoE(13May2014).268Brazil,Statementatthe2017GGE(April2017)andBrazil,Statementatthe2019GGE(March2019).269SriLanka,Statementatthe2015MoE(April2015).270Ecuador,Statementatthe2015MoE(April2015).271Australia,Statementatthe2016MoE(April2016).272SierraLeone,Statementatthe2016MoE(April2016).273SriLanka,Statementatthe2017GGE(April2017)(mixinglegalandmoralcon-siderations:‘[e]venifanyoftheexistingIHLprinciplesarefoundtobeinapplicable,thetestofpublicconscienceandlawsofhumanityasreferredtointheMartensClauseprovidescompellingreasonsforestablishingbasicguidingprinciplesonthelegalityoftheuseofLAWS’).274SeemoreextensivelyDMauri,‘TheHolySee’sPositiononLethalAutonomousWeaponsSystems.AnAppraisalthroughtheLensoftheMartensClause’(2020)11JIHLS116.275SeeBrazil,Statementatthe2019GGE(n268):‘[t]heMartensClauseis,there-fore,ofextremerelevancewhenwediscusstheethicalandhumanitarianimpactofincorporatingautonomyfeaturesinweaponssystems,andisamostvaluableguideforelaboratingnewlawfornewphenomena’.DiegoMauri-9781802207675DownloadedfromPubFactoryat05/28/202203:48:43AMviaDukeUniversityLawLibrary

180172Autonomousweaponssystemsandtheprotectionofthehumanpersontheideaisshockingandunacceptable’,and‘Statesshouldtaketheirperspec-tiveintoaccountwhendeterminingthedictatesofpublicconscience’.276TheCSKRtakesthesameline.277ThatAWSarefoundtocontradicttheprincipleofhumanitypushesthoseNGOstoinvokeapre-emptivebantreaty;whethertheiropinioisthatAWSwouldbealreadyprohibitedisneitherdeniednoraffirmed.Amongthosenon-stateactors,aprominentroleinthisisplayedbytheICRC.Initsstatements,theICRCreferstothe‘principlesofhumanity’and‘dictatesofpublicconscience’asstructurallylinkedwithmorality,278distinctfrompositivelaw,279butcapableofactingasa‘portal’connectingmoralityandlaw.Whenitcomestoassessingthenormativestatusoftheprinciple,however,clearargumentsarelacking.Atmost,asillustratedbytheICRC,AWSraiseethicalconcernsthatneedtobetranslatedintolegalterms:shouldthistrans-lationnotoccur,asamatterofprincipletheirdevelopmentanddeploymentwouldbepermissibleunderIHL.Asforscholars,therearethosewhoargueagainstanyuseoftheprincipleofhumanityasarelevantlegalnorm.280OneoftheharshestcriticsisEvans,whoarguesthatappealingto‘principlesofhumanity’and‘dictatesofpublicconscience’hasnootherpurposethanof‘incentiviz[ing]thedisseminationofsensationalist,fear-mongeringrhetoricaimedatpersuadingthepublic,impres-sionableStates[sic]orNGOsthatthechallengedweaponsareabhorrent’.281However,thereareauthorswhoseemmoresupportiveofanormativerolefortheprinciple.O’Connellcontendsthat‘[i]tissimplyinsufficienttosaythatahumanbeingissomewhereinthepicturewhendiscussinglegalandmoralstandardsforkilling’;thisisbecausethereare‘essentialhumanqualities’suchas‘conscience,commonsense,intuition’thataredefinitively‘unprogram-276HumanRightsWatchandInternationalHumanRightsClinicatHarvardLawSchool,‘LosingHumanity.TheCaseagainstKillerRobots’(19November2012)35.277ICRAC,Statementatthe2018GGE(9April2018)(arguingthat‘[d]ictatesofpublicconsciencemustalwaystakeprecedenceoveranyshort-termadvantagethatmightbegainedfromautonomoustechnologies…ICRACreiteratesthespiritoftheMartensClause—thatmoralitycanprovideastrongbasisfornewlaw’).278ICRC,Statementatthe2015MoE(13April2015).279ICRC,Statementatthe2016MoE(April2016)(‘[t]hefundamentalquestionattheheartofconcerns,andirrespectiveofwhethertheycanbeusedincompliancewithIHL,iswhethertheprinciplesofhumanityandthedictatesofpublicconsciencewouldallowmachinestomakelife-and-deathdecisionsinarmedconflictwithouthumaninvolvement’).280SchmittandThurnher(n63)275(contendingthatratherthanbeing‘anoverar-chingprinciple’,itmayatmost‘addresslacunaeinthelaw’,whichhoweverisnotthecaseforAWS).281TDEvans,‘AtWarwithRobots:AutonomousWeaponSystemsandtheMartensClause’(2013)41HofstraLRev697,727.DiegoMauri-9781802207675DownloadedfromPubFactoryat05/28/202203:48:43AMviaDukeUniversityLawLibrary

181Autonomousweaponssystemsunderinternationalhumanitarianlaw173mable’.282ToSparrow,allowingamachinetotakedecisionsabouthumanlifeispatentlycontrarytoanymoralimperative:itwouldmean,forabel-ligerentparty,toliterally‘treat[its]enemieslikevermin’,283thatis,refusingtoacknowledgeasharedbelongingtothehumanrace.AccordingtoHeyns,AWSwouldbeperceivedasatypeof‘mechanizedpesticide’employedbyanenemythatdespiseshisopponent’sdignityandconsiderhislifeundeservingofahumandecisiontobetaken.284‘Whensomeonecomesintothesightsofacomputer,thatpersonisliterallyreducedtonumbers:thezerosandtheonesofbits’.285Ourpreviousremarksvis-à-vistheprincipleofhumandignityanditsrelevanceforthedebateonAWSmaybeexpandedhereaswell.286Summingup,theprincipleofhumanityprovidesasoundlegalbasisforarguingthattoentrustcriticalfunctionstoamachine,whilehumanoperatorswouldretainnominaltonopowertointerveneinthedecisionalloop,isintol-erable.Ouroverviewshows,however,thatstateandnon-stateactorstendtoinvoketheMartensClausemoreasadelegeferendatool—namely,toinflu-encefuturelaw-making.Ontheonehand,actorsinvokingitregarditmerelyasamoralimperative.Ontheotherhand,mosttimesbehindtheinvocationofhumanityliestheneedtodevelopanddeployAWSinawaythatensuresrespectforIHLrules(asoccursforIHRL).Inthiscase,though,providedthat‘humancontrol’isensured,andthusthathumanoperatorsremaininapositiontounderstandandexplaincoursesofactionofAWSinanadequatemanner(thatis,satisfyingrequirementsimposedbyIHLrules),theuseofAWSisnotheldtocontrastwithIHL.Inbothcases,theroleoftheprincipleofhumandignityissubstantivelydownplayed.Thescenariocouldbedifferentifstateandnon-stateactorsengagedwithaseriousconfrontationwiththeprincipleofhumanityanditsactualcontentvis-à-visAWS.InlinewithCassese’sconception,‘recognition’(opinio)bytheinternationalcommunity(formedbystateandnon-stateactors)isalwaysnecessaryforascertainingwhetherageneralprincipleexistsinthelegalorderathand,soastosavethe‘consensual’natureoftheinternationallegal282MEO’Connell,‘BanningAutonomousKilling:TheLegalandEthicalRequirementthatHumansMakeNear-TimeLethalDecisions’,inMEvangelistaandHShue(eds),TheAmericanWayofBombing:EthicalandLegalNorms,FromFlyingFortressestoDrones(CornellUniversityPress2014)232.283RSparrow,‘RoboticWeaponsandtheFutureofWar’,inJWolfendaleandPTripodi(eds),NewWarsandNewSoldiers:MilitaryEthicsintheContemporaryWorld(Ashgate2011).284‘ReportoftheUnitedNationsSpecialRapporteuronExtrajudicial,SummaryorArbitraryExecutions,ChristofHeyns’(2013)A/HRC/23/47(HeynsReport)para95.285CHeyns,‘HumanRightsandtheuseofAutonomousWeaponsSystems(AWS)DuringDomesticLawEnforcement’(2016)38HumanRightsQuarterly350,370.286SeeCh3,s3.5.DiegoMauri-9781802207675DownloadedfromPubFactoryat05/28/202203:48:43AMviaDukeUniversityLawLibrary

182174Autonomousweaponssystemsandtheprotectionofthehumanpersonsystem.287Scholarsmoreinclinedtoextra-positivethinkingandthosemoreanchoredtolegalpositivismwouldagree.Asfurtherconfirmation,sufficetorecallthatthisisthemethodologyadoptedbytheILCinitsstudyongeneralprinciples.288Thatis,whatiscurrentlylackinginthedebateonAWSisthisactof‘recognition’:bycontentingthemselveswithinvokingtheprincipleofhumanityfromadelegeferendaperspective,thoseactorsarguingforabanonAWSendupmissingtheopportunitytoclearlystatetheiropiniothatremovinghumaninterventionfromcriticaldecisionsrunscontrarytotheprincipleofhumanity.4.6CONCLUDINGREMARKSArmedconflictwillprobablybethefirstoperationalscenarioforAWS.Themilitaryhasthegreatinterestinthistechnology,andthemorepermissiverulesapplicabletotheuseofforceinthebattlefield(ifcomparedwiththoseapplica-bletolaw-enforcementscenarios)willrendermilitaryapplicationsofAWSofparticularconvenienceinyearstocome.OurjourneythroughIHLhasledustothefollowingconclusions.Rulespertainingtoweaponslawandtargetinglaw—allemanatingfromprinciplessuchasdistinction,proportionality,andprecautionsinattack—provideaclearlegalyardstickthroughwhichthedevelopmentanddeploymentofAWScanbemeasured.FieldingAWSwillbechallenging:IHLrulesraiseahostofissuesforautonomousdecision-making,fromtargetrecognitioninthe‘fogofwar’(andgiventhecomplexitiesofreal-lifebattlefield,especiallyincurrenttimesof‘asymmetrical’warfare)tostrikingdelicatebalanceswhentakingaparticularcourseofaction.Moreover,thedutytotakeallfeasiblepre-cautionsexantefactoandtoconductappropriateinvestigationsexpostfactowillmaintainhumanoperatorssomewhere‘intheloop’—eveniftheycouldbeabsentfromaparticulardecision.Analogouslytoourpreviousfindingsvis-à-visIHRL,IHLnormsaremodelledinamoldthatattachescriticalimpor-tancetotheoutcomeofaparticularcourseofaction,insteadofthataparticulardecisionhasbeenreachedwithoutmeaningfulinvolvementofahumanoper-ator.ThishasledonecommentatortoconcludethathumanjudgmentisnotessentialforthepurposeofrespectingIHL:tothecontrary,giventheratiooferrorscharacterizinghumandecision-making,itwouldsoundunreasonableto287Cassese(n212).SeealsoCEggett,‘TheRoleofPrinciplesandGeneralPrinciplesinthe“ConstitutionalProcesses”ofInternationalLaw’(2019)66NetherlandsInt’lLR197.288SecondReport(n257)para121.DiegoMauri-9781802207675DownloadedfromPubFactoryat05/28/202203:48:43AMviaDukeUniversityLawLibrary

183Autonomousweaponssystemsunderinternationalhumanitarianlaw175imposeitintheconductofhostilities.289Itfollowsthatanydegreesof‘humancontrol’thatareadequatetoensurerespectforthoseprimarynormsareaccept-ableunderexistingIHL.Objectionsfocusingontheinadequatenessofcurrenttechnologyhavetobetakenseriously;however,anypro-banargumentsbasedontheseobjectionsisself-defeating,asitmaybeexpectedthatataparticulartimetechnologicaladvanceswillallowautonomoussystemstooperatelegally.ThesituationmayappeardifferentifAWSareassessedagainsttheback-groundoftheprincipleofhumanity.ThatourremarksontheprincipleofhumanityseemtomirrorgrossomodothoseaspertheprincipleofhumandignityinIHRLsupportsourinitialintuition,accordingtowhich,humandignityandhumanityoperateas‘twin’concepts,bothqualifyingasgeneralprinciplesrelevantfordiscussingAWS,andourinitialmethodologicalchoiceofinvestigatingIHRLbeforeIHL.290BothprincipleshavethepotentialtoshedlightonthemostsalientfeatureofAWS,thatis,theremovalofhumanpres-encefromtheactofselectingandengagingaparticulartarget.However,inbothcaseswhatislackingisaneffort—bystateandnon-stateactorsinvolvedinthedebate—togivecontenttotheprinciple.Tothisend,proposalsforfillingthoseprincipleswithAWS-specificcontentareadvancedinthefinalchapterofthisbook.291Inconclusion,ouranalysisoftheprimarynormsthatwillbeaffectedbyAWSdemonstratesthatthemostsalientfeatureofAWSisgraspedbyexistingrulesandprinciplesonlytoalimitedextent.Moreoften,theargumentthatultimatedecisionsincriticalfunctionswillbetakenwithouthumaninterven-tionisrelevantunderadifferentperspective,affectingthelawofinternationalresponsibilityatlarge.OneofthereasonsmilitatinginfavorofthecontrarinesstohumandignityandhumanityofAWSwouldbethattheirusewillinevitablyresultinaccountabilitygaps.InsteadoflookingatthestakesconnectedwithAWSoperatingasexpected,mostcommentatorsareconcernedwithAWSoperatingnotasexpected.Thefollowingchapter,whichmarksourdeparturefromprimarynormsofIHRLandIHLandleadsusinthefieldofthelawofinternationalresponsibilityatlarge,isdevotedtothisargument.289ETalbotJensen,‘The(Erroneous)RequirementforHumanJudgment(andError)intheLawofArmedConflict’(2020)96Int’lLStud26.290SeeCh1,s1.3andCh2,s2.3.3.291SeeCh6.DiegoMauri-9781802207675DownloadedfromPubFactoryat05/28/202203:48:43AMviaDukeUniversityLawLibrary

1845.Autonomousweaponssystemsandthe‘accountabilitygap’conundrum5.1DEPICTINGTHREESCENARIOSOFAWS’MISDOINGSInhislandmarkreportonAWS,ChristofHeynsarguedthat‘[i]fanatureofaweaponrendersresponsibilityforitsconsequencesimpossible,itsuseshouldbeconsideredasunethicalandunlawfulasanabhorrentweapon’.1Thisisanargumentthatresonateswithmoststates,NGOs,scholarsandotheractorsinvolvedinthedebateonAWS,andcanbereferredtoasthe‘accountabilitygap’(or‘responsibilitygap’)conundrum.2Initsmostessentialform,AWSareheldtorunagainsttheeffectiveprotectionofthehumanpersontotheextentthattheyrenderresponsibilityformistargetingpracticallyunfeasible.Ifanautonomousmachinedeliberatedaparticularcourseofactionwithouthumanintervention,andifhumancontroloverthatdecision-makingprocesswerenotenoughtoengageanyone’sresponsibility,who—orwhatentity—wouldrespondonbehalfof‘misdoings’committedbythatsystem?Twolinguisticcaveatsaredue.First,theexpression‘accountabilitygaps’willbepreferredasitisoftenemployedinthedebate,anditisadmittedlywiderinscope(byhintingatprimaryobligationstoo);second,theterm‘misdoing’isappropriateasitisvagueenoughtoembraceabroadspectrumofcasesinwhichAWSmisfire.3WhetheraccountabilitygapsinthelawofinternationalresponsibilityopenupwithregardtoAWSand—intheaffirmative—howtofillthemispossiblythemainconcernoftheentiredebatetakingplacewithintheCCWframework.GuidingPrinciple(b)affirmsthat‘[h]umanresponsibilityfordecisionsonthe1‘ReportoftheUnitedNationsSpecialRapporteuronextrajudicial,summaryorarbitraryexecutions,ChristofHeyns’(2013)A/HRC/23/47(HeynsReport)para75.2HumanRightsWatchandtheInternationalHumanRightsClinic,‘MindtheGap.TheLackofAccountabilityforKillerRobots’(9April2015).3DAmorosoandBGiordano,‘WhoIstoBlameforAutonomousWeaponsSystems’Misdoings?’,inECarpanelliandNLazzerini(eds),UseandMisuseofNewTechnologies.ContemporaryChallengesinInternationalandEuropeanLaw(Springer2019).176DiegoMauri-9781802207675DownloadedfromPubFactoryat05/28/202203:48:45AMviaDukeUniversityLawLibrary

185Autonomousweaponssystemsandthe‘accountabilitygap’conundrum177useofweaponssystemsmustberetainedsinceaccountabilitycannotbetrans-ferredtomachines’,animperativetobetakenintoaccount‘acrosstheentirelifecycleoftheweaponssystems’.4Onthisbasis,GuidingPrinciple(d)holdsthat‘[a]ccountabilityfordeveloping,deployingandusingany[AWS]…mustbeensuredinaccordancewithapplicableinternationallaw,includingthroughtheoperationofsuchsystemswithinaresponsiblechainofhumancommandandcontrol’.5Reaffirmingtheimportanceofproperlyestablishingandattrib-utingresponsibilityformisdoingscommittedthroughAWSdoesnotimplythataccountabilitygapsmaynotarise,though:itdoesnotremovetheproblemastowhetherexistinglawofinternationalresponsibilityisaptforAWS.Soitisuptotheinterpretertounpackresponsibilityissues,dosomereconnaissanceofrelevantnormsandtestthemagainstthechallengesposedbyAWS.HavingengagedinprimaryobligationsunderIHRLandIHLinthepreviouschapters,wenowturntoissuesofinternationalresponsibility.Beforeaddressingthenormsregulatinginternationalresponsibilityinitsvariousforms,someprelim-inaryremarksarenecessary.Noonefailstoseethattheriskofaccountabilitygapsisparticularlyacuteinremotewarfare,namely,whereweaponsallowingfor‘asomewhatgreaterdistancingofhumanoperators’fromtheoperationalfieldareemployed.6Intuitively,havingahumanoperatorphysicallyremovedfromtheterrainmayreducepsychologicalinvolvementinthedynamicsofthatenvironmentandthusmoralandlegalawarenessofwhatisgoingonthere.Inaddition,theincreasingintegrationofautonomyindecision-makingprocessesandthetrendtowardsadvancedformsofhuman–machineinteractions(andteaminggenerally)magnifyvacuumsofresponsibility:canitbethataparticularcourseofactionisimputablesolelytothehumanoperator?Towhatdegreedoesautonomyinfluenceoutcomes?Inallinstances,havingsomeonetoholdaccountableforwhathappensinarmedconflictorpoliceoperationsiskeyforatwofoldgoal:deterringfutureviolationsandprovidingvictimswithasenseofretribution.Thebyproductofemployingtechnologiesfurtherseparatingthe‘attacker’fromthe‘attacked’isde-humanizationonbothsidesofforce.74GroupofGovernmentalExpertsoftheHighContractingPartiestotheConventiononProhibitionsorRestrictionsontheUseofCertainConventionalWeaponsWhichMayBeDeemedtoBeExcessivelyInjuriousortoHaveIndiscriminateEffects,‘Reportofthe2019sessionoftheGroupofGovernmentalExpertsonEmergingTechnologiesintheAreaofLethalAutonomousWeaponsSystems(LAWS)’(25September2019)CCW/GGE.1/2019/3AnnexIV(GuidingPrinciples)principle(b).5GuidingPrinciples,principle(d).6MNSchmittandJSThurnher,‘“OutoftheLoop”:AutonomousWeaponSystemsandtheLawofArmedConflict’(2013)4HarvardNat’lSecJ231,276.7PWSinger,WiredforWar(PenguinBooks2009)311–12(quotingapopularPakistanisongthatwentviralin2008duringtheUSdronecampaignintheregion,DiegoMauri-9781802207675DownloadedfromPubFactoryat05/28/202203:48:45AMviaDukeUniversityLawLibrary

186178AutonomousweaponssystemsandtheprotectionofthehumanpersonToavoidallthis,theconceptofmeaningfulhumancontrol(MHC)hasbeenresortedto.8Human–machineinteractions,wherehumansmayfindthem-selvesplayingamarginalroleinamachine’sparticularcoursesofaction,mustensurethatresponsibleindividualsmaybeidentified.Followingtheapproachoutlinedattheoutsetofthisbook,thepurposeofthischapteristo‘scan’thelawofinternationalresponsibilitytoestablishthedegreeofhumancontrolthatAWShavetoensureinordertobedevelopedandemployedlegally.ThereadermayalreadyhaveusedtheirimaginationandthoughtaboutatleastthreescenariosofAWScommittingmisdoingsinthefield.Inthefirst,AWShavebeendevelopedand/orusedpurposelyandpreciselyforcom-mittingviolationsofinternationallaw(IHRLandIHL):thehumanoperator(acommanderorapoliceofficer)decidestousethoseAWSinanoperationwhereitisexpected—ifnotwished—thatthesesystemswillbeincapableof,say,distinguishingproperlybetweenpermissibleandimpermissibletargets.Whatweseekisa‘badintention’,eitheronthepartofthemanufacturer,theprogrammer,thefinaluserorallofthem.Inthesecondscenario,dulyprogrammed,manufacturedandemployedAWSsuddenlybegintobehaveimproperlyinthefield:euphemistically,they‘gorogue’andstarttotargetimpermissibletargets(say,bystandersduringapolicespecialoperationthatcouldnotbeevacuatedbeforehand).Whatweseekhereisnota‘badintention’onthepartofanyhumansinvolvedinthechain:theysurelydidnotwantorreasonablyanticipatethosecoursesofaction.Whatweseekinsteadisnegli-gence:somethingwentwrongatapreviousstage(programmingthesoftware,manufacturingthehardwareordeployingthesysteminaninappropriateenvironment)whichcouldhavebeenavoided.Thethirdandfinalscenariofeaturesthe‘falsepositives’:9AWSselectandengageimpermissibletargetsnotasaresultofmaliciousornegligentprogramming,manufacturingoruse,butasaresultoftheunpredictableinteractionofacomplexweaponssystemswithequallycomplexenvironments.10Thatparticularcourseofactionwasmeanttogothatway:statistically,notallpossiblescenarioscouldbeantici-patedortestedbyproducers,andamarginoferrormustalwaysbetakenintoaccount.Thatis,falsepositives(themachine’smistakes)wouldnotqualifyaswhoselyricsare‘America’sheartlessterrorism/Killingpeoplelikeinsects/Buthonordoesnotfearpower’).8SeeCh2,s2.2.3.9ICRC,‘Autonomy,IntelligenceandRobotics:TechnicalAspectsofHumanControl’(August2019),20(referringtocaseswhere‘thealgorithmincorrectlyidenti-fiesanobject’).10RCrootof,‘WarTorts:AccountabilityforAutonomousWeapons’(2016)164UniversityofPennsylvaniaLR1347,1373(referringtothe‘normalaccident’theory).DiegoMauri-9781802207675DownloadedfromPubFactoryat05/28/202203:48:45AMviaDukeUniversityLawLibrary

187Autonomousweaponssystemsandthe‘accountabilitygap’conundrum179malfunctioning(themachine’sdefect,tobeblamedonproducersorusers).11Thedifficultquestionishowtoallocateresponsibilityinthesecases,giventhepluralityofactorsinvolvedand,moreimportantly,theabsenceofanymentalelementsuchasintentionorfaultonthepartofthoseactors.Inlinewiththepurposeofthisbook,theresponsibilitygapsarisingfromtheuseofAWSinlawenforcementandarmedconflictisdiscussedfromthestandpointofinternationallaw—solutionsthatmustbefoundatthedomesticlevelarenotdiscussedhere.First,whethertheautonomoussysteminitselfcouldbeheldresponsibleforitsmisdoingsisdiscussed(section5.2).Althoughadmittedlythisappearsasafuturistic(nottosaysciencefiction)scenario,itisinterestingtonotethatsomeproposalshavebeenadvancedwithregardtodomesticliabilityregimes.Turningtomoretraditionalscenarios,thefirstsubjecttobeheldresponsibleforthemisdoingsofAWSistheindividualinvolvedintheprogramming,manufacturing,transferandeventuallyuseofthosesystems.Thebranchofinternationallawdealingwithindividualrespon-sibilityforthecommissionofaparticularcategoryofconduct,namely,inter-nationalcrimes,isinternationalcriminallaw(ICL);theanalysisthusconsidersthosecategoriesthatmaybechallengedbytheadventofAWS(andthusthatmayneedtoberevisitedaccordingly)(5.3).AnothersubjectplayingapivotalroleindevelopingAWSarecorporations:whiletheirstatusasactorsintheinternationallegalorderisdebated,thenumberofinternationalnormsaddress-ingthemisarguablyincreasing.Whethermechanismsexistthatensureproperresponsibilityontheirpartattheinternationallevelisthusdiscussed(5.4).Finally,theproverbial‘elephantintheroom’roleisplayedbythesubjectthatwillemployAWSintheoperationalfield,namely,states(5.5).Finally,adisclaimer:theresponsibilityofothernon-stateactors(suchasarmedgroups)andinternationalorganizations(IOs)fortheuseofAWSisnotengagedhere.AsregardsIOs,thecircumstancethattheymaycommitinterna-tionalwrongfulactsimpingingontheprotectionofthehumanperson,iswellknown;stateshavestriventosetupalegalframeworkfordealingwithIOs’internationalresponsibilitywhoseeffectivenessisstillhotlydebatedbyschol-ars.12Asourkeyfindingsinrespectofstateresponsibilitycanbeexpanded,11SBeck,‘TheProblemofAscribingLegalResponsibilityintheCaseofRobotics’(2016)31AI&Society473,475(‘[i]farobotthencausesdamagetoathirdpartybecauseofthesereactions,itishardlyplausiblethattherobotwasdefect;itdidwhatitwassupposedtodo:itreactedtonewinputsandadapteditsbehaviour—thusthemachineisnotdefectiveassuch’).12SeetherecentandcomprehensiveworkofMBuscemi,IllecitidelleNazioniUniteetuteladell’individuo(EditorialeScientifica2020).SeealsoILC,‘DraftArticlesontheResponsibilityofInternationalOrganizations’(2011)A/66/10.DiegoMauri-9781802207675DownloadedfromPubFactoryat05/28/202203:48:45AMviaDukeUniversityLawLibrary

188180Autonomousweaponssystemsandtheprotectionofthehumanpersonmutatismutandis,toIOs,itisbelievedthataseparateanalysiswouldonlyhaveburdenedthechapter,soweleaveitforadedicatedcontributioninthefuture.5.2HOLDINGAWSACCOUNTABLE?CURRENTPROPOSALSANDMAJOROBSTACLESTheideaofholdingautonomoussystems,andmoregenerally‘robots’liableformisdoingsoccurringasaconsequenceoftheirusemaysoundfuturisticandultimatelyunrealistic.Yet,someproposalshavebeenadvancedtothisend,andinterestinglynotonlyonthepartoftoo-visionaryauthors.Forinstance,asfarasthefieldofcivilliabilityisconcerned,in2017theEUParliamentinvitedtheCommissiontoconsider,asoneofthepossiblelegalsolutionstotheregulatorychallengeposedbyAIsystems,creatingaspecificlegalstatusforrobotsinthelongrun,sothatatleastthemostsophisticatedautonomousrobotscouldbeestablishedashavingthestatusofelectronicpersonsresponsibleformakinggoodanydamagetheymaycause,andpossiblyapplyingelectronicpersonalitytocaseswhererobotsmakeautonomousdecisionsorotherwiseinteractwiththirdpartiesindependently.13Theconceptof‘electronicpersonhood’stimulatedanintriguingdebate,foritwasunderstoodasadmittingthatrobotsandmachinescouldbeaddresseesoflegalrightsandduties—anideathat,unsurprisingly,ishighlydivisive.14Accordingtosome,iftheultimategoalistoensureproperaccountabilityformachines’misdoings,conceivingelectronicpersonhoodexnihilowouldraisetroublesomelegalproblems,nottomentionethicalconcerns:existingrulesallocatingresponsibilityacrossnaturalandlegalpersonswouldsuffice.15Bycontrast,tootherstheconceptofelectronicpersonhoodisacceptable:farfromimplyingtheacknowledgmentofindividualrightstoautonomoussystems,itwouldservemerelyfunctionalpurposes(suchascappingliability,redistribut-ingprofitsorensuringgreatertransparency).16Startingfromthecircumstance13EuropeanParliamentResolutionof16February2017withRecommendationstotheCommissiononCivilLawRulesonRobotics(2015/2103(INL))[2018]OJC252,239para59(f).14RMichalczak,‘Animals’RaceAgainsttheMachines’,inVKurkiandTPetrzykowski(eds),LegalPersonhood:Animals,ArtificialIntelligenceandtheUnborn(Springer2017).15JJBryson,MEDiamantisandTDGrant,‘Of,For,andBythePeople:TheLegalLacunaofSyntheticPersons’(2017)25ArtificialIntelligenceL273.16EuropeanParliament,PolicyDepartmentforCitizens’RightsandConstitutionalAffairs,Directorate-GeneralforInternalPolicies,ArtificialIntelligenceandCivilLiability(June2020)PE621.926.DiegoMauri-9781802207675DownloadedfromPubFactoryat05/28/202203:48:45AMviaDukeUniversityLawLibrary

189Autonomousweaponssystemsandthe‘accountabilitygap’conundrum181thatcurrentcivilliabilityrulesareinsufficientfordealingproperlywithAIsystems,othershaveventuredtoidentifyalternativeformsofcivilliabilityforthepurposeoffindingsolutionsforthosenewphenomenologyofdamages,consideringnotonlytheattributionof(limited)personhoodtothesystems,butalsotohuman–machineinteractions.17Turningtothefieldofcriminallaw,theoptionofholdingautonomoussystemsresponsibleformisdoingsamountingtocrimes(accordingtodomesticlaw)hasbeenexploredbysomeauthors.Astraditionallycriminallawservesthepurposeofrepressingmostseriousoffencestocorevaluesofacommu-nitybypunishingtheirauthors,acriminal-lawresponsetothesemisdoingsaddressedtothematerialauthoroftheoffence—possiblyresultinginphysicalharm,ifnotdeath,thereof—isbelievedtobethemostappropriate.Tosome,whilesuchascenarioshouldnotbedisregardedaltogether,profoundchangesinexistingcategoriesofcriminallawwouldberequired.18Alongthesameline,somehavetestedtheideaofholdingrobotsandautonomoussystemscriminallyaccountableinspecificfields(suchasrobotsoperatinginthestockmarket)andonthepremisethattheyareendowedwithelectronicpersonhoodforthepurposeofcivilliabilitytoo.19However,thoseoptionsaredeeplyquestionable:howcouldamodernlegalsystemestablishthecriminalliabilityofthings?Howcouldtheensuing‘punishment’besatisfactoryforvictims?Asubstantivedegreeofskepticismiswarranted.IfholdingAWSaccountableindomesticsystemsappearsfuturisticatbest,thiswouldbeevenmoreremoteintheinternationallegalorder.Onagenerallevel,conceivingsubjectsotherthanstates,ashasbeenthecaseforinterna-tionalorganizations,non-stateactors,individualsand,morerecently,businesscompanies(atleasttoalimiteddegree)isanarduousundertaking,euphemis-tically.Treatiesattributingelectronicpersonhoodorsimilarlegalstatusestomachinesarenotinsight.Eveniftheywere,theywouldlikelycontainrulesobligingstatestoadapttheirdomesticlawstonewformsofcivilliability;theywouldhardlyestablishtheirpersonhoodintheinternationallegalorder.OnthespecificlevelthecurrenttrendintheCCWframeworkseemstogoinadifferentdirectionbyestablishingthatlegalresponsibilityshouldliesolely17MWMonterossi,‘LiabilityfortheFactofAutonomousArtificialIntelligenceAgents.Things,AgenciesandLegalActors’(2020)20GlobalJurist1.18GHallevy,LiabilityforCrimesInvolvingArtificialIntelligenceSystems(Springer2015).19FLagioiaandGSartor,‘AISystemsunderCriminalLaw:ALegalAnalysisandaRegulatoryPerspective’(2020)33Philosophy&Technology433.SeealsoUPagallo,‘AIandBadRobots:TheCriminologyofAutomation’,inMRMcGuireandTJHolt(eds),TheRoutledgeHandbookofTechnology,CrimeandJustice(Routledge2017).DiegoMauri-9781802207675DownloadedfromPubFactoryat05/28/202203:48:45AMviaDukeUniversityLawLibrary

190182Autonomousweaponssystemsandtheprotectionofthehumanpersonwithhumanoperators.20Moreover,theGuidingPrinciplesestablishthatAWSshouldnotbe‘anthropomorphized’inordertokeepthemdistinctfromhumanagents.21Insummary,takingintoaccounttheinternationallegalorder’straditionalfeaturesandmorerecenttrends,itishardtoexpectthatmachineswillberec-ognizedbyanyformsofpersonhoodthere.Eveninthe‘falsepositive’scenariodepictedabove,responsibilitywillrestwithothersubjectsofinternationallaw.Letusnowinquirewhotheseare.5.3INTERNATIONALCRIMINALLAW:FOCUSINGONINDIVIDUALSINVOLVEDINTHEDEVELOPMENTANDDEPLOYMENTOFAWSWhenAWSperformincorrectlyandaccusationsaremade,itseemsnaturaltolookathumanagentsinthefirstplace.Thematerialact(oractusreus)performedthroughAWSwouldeasilyamounttocrimessuchasmurder,manslaughterandsimilar:ifthatwerethecase,itwouldbeuptoeachstatetoinvestigateandpunishthosecrimesinaccordancewithitsdomesticcrim-inallaw.Asfarastheinternationallegalorderisconcerned,asetofnormsregulatingindividualresponsibilityforaspecificcategoryofcrimesexists,thatisinternationalcriminallaw(ICL).Whenaconductamountingtoan‘internationalcrime’iscommitted,individualresponsibilityarisesinadditiontothetraditionalresponsibilityofthestate.Aparticularconductqualifiesas‘internationalcrime’whendirectedagainstfundamentalvaluesoftheinterna-tionallegalorder,thebreachofwhichissogravethatitisnecessarytopunishalsotheindividualresponsibleforit.22InaccordancewithageneralprincipleoflawacrossvariousdomesticlegalordersandofICL,foranaccusedtobefoundguiltyofaninternationalcrimetwokeyelementsmustbeproven:actusreus(thatis,amaterialconductcorrespondingtoatreatyorcustomaryICLprovision)andmensrea(thatis,aculpablestateofmind).Asinferablefromthepreviouschapters,thecategoryofinternationalcrimesthatwouldbemoreeasilycommittedthroughAWS20GuidingPrinciples,principle(b).Inscholarship,seeTChengeta,‘AccountabilityGap:AutonomousWeaponSystemsandModesofResponsibilityinInternationalLaw’(2016)45DenverJofInt’lLandPol1.21Ibidprinciple(i).22EGreppi,‘TheEvolutionofIndividualCriminalResponsibilityunderInternationalLaw’(1999)81IRRC531.SeealsoMFrulli,‘ThePromiseofInternationalCriminalJustice:AchievementsandFailuresinProtectingHumanDignity’,inAGattini,RGarciandiaandPWebb(eds),HumanDignityandInternationalLaw(Brill2021).DiegoMauri-9781802207675DownloadedfromPubFactoryat05/28/202203:48:45AMviaDukeUniversityLawLibrary

191Autonomousweaponssystemsandthe‘accountabilitygap’conundrum183iswarcrimes.23Tonameonlyone,thewarcrimeofintentionallyattackingciviliansasenshrinedinarticles8(2)(b)(i)and8(2)(e)(i)ICCStwillbethefirsttocometothefore.24However,thisiswithoutprejudicetothecircumstancethatAWSisemployedforthecommissionofotherinternationalcrimes,suchasthoseamountingtocrimesagainsthumanity,aggressionorgenocide.Thefirstwouldqualify,forinstance,wheneverAWSwereusedtocommitmurderaspartofawidespreadorsystematicattackdirectedagainstanycivilianpopulation,asarticle7ICCStandrelevantcustomaryICLgo.Thesecondwouldapplytoindividualsresponsibleforanactof‘aggression’asdescribedinarticle8bisICCStand,again,incustomarylaw.Asimilardiscourseappliestogenocide.Fortheavoidanceofdoubt,however,itismoreappropriatetoassumeaperspectivebasedonthecommissionofwarcrimes,whichissuf-ficientforconductingageneraloverviewoftheissuesatstake.Referencetothoseothercrimeswillbemadeonlywhenstrictlyrelevant.WhileAWSareunlikelytoraiseunsurmountableproblemsaspertheactusreuselement,morecomplicatingfactorscouldbeidentifiedwithregardtomensreaandmodesofliability,whichbringaboutasetofcomplementaryissues.Asfortheformer,itmustbeestablishedwhethertherequiredmensreainthecrimesunderconsiderationisaptforAWS,wherehuman–machineinteractionsmaybesuchthathumanagentsplayamarginalroleinthedecision-makingprocess(sees5.3.1below).Astothelatter,thevariousmodesofperpetrationenvisagedbyICLmustbetakenintoaccount,inordertoverifywhethertheyeffectivelyaddressallindividualspotentiallyinvolvedinthedevelopmentanddeploymentofAWS.Softwaredevelopers,engineers,programmersandsimilarcategoriesallplayaroleinassemblingthesystemsthatmaybehaveimproperlyinthefield,inadditiontopolicydecision-makers,militarycommandersandsoldierswithinthechainsofcommand.Thisisaproblemthathasbeeneffectivelydescribedasoneof‘manyhands’.25Notethatmorechallengeswillberaisedbythe(probable)useofAWSbyprivatemilitaryandsecuritycontractors(PMSCs),thatis,companiesspecificallyhiredbystatestoperformstatefunctions(suchasuseofforce)incomplexscenarios.26Theirmanagersoremployersthusplayarolethatisanalogoustothatofstateagents(lawenforcementagentsandofficials,aswellassol-23RCryer,‘WarCrimes’,inBSaulandDAkande(eds),TheOxfordGuidetoInternationalHumanitarianLaw(OUP2020).24RomeStatuteoftheInternationalCriminalCourt(adopted17July1998,enteredintoforce1July2002)2187UNTS3(ICCSt).25AmorosoandGiordano(n3)218ff.26Onthistopictheliteraturehasbecomeextensiveinrecentyears.SeeamongmanyFFrancioniandNRonzitti(eds),WarbyContract:HumanRights,HumanitarianLawandPrivateContractors(OUP2011).DiegoMauri-9781802207675DownloadedfromPubFactoryat05/28/202203:48:45AMviaDukeUniversityLawLibrary

192184Autonomousweaponssystemsandtheprotectionofthehumanpersondiersandcommanders),andoneofICL’scurrentchallengesistoeffectivelypunishinternationalcrimescommittedbythem,whichinevitablyrequiresarevisitationofexistinginstitutions.27Modesofliabilityarethusconsideredinadedicatedsubsection(5.3.2).AconcludingdiscussionofICLissuesraisedbyAWSfollows(5.3.3).5.3.1ConstruingtheMensReaRequirementforCrimesCommittedthroughAWSOneofthebedrockprinciplesofICL,whichhavebeenderivedfromdomesticlegalsystemsandwhichnowcanberegardedasageneralprincipleofthatbranchofinternationallaw,istheprincipleofculpability:noonecanbepun-ishedabsentaculpablestateofmind(nullumcrimen,nullapoenasineculpa).Individualcriminalresponsibilityrestsonthepresenceof‘guilt’onthepartoftheindividual,whichhistoricallymarkedacleardeparturefromstrictliabilityregimes:inatrendalsopresentcurrently,manydomesticsystemsandtheinternationallegalsystemeschew(oratleastsignificantlylimit)formsofstrictliabilityinthecriminalfield.28ThissubsectiontacklestheissueofthementalelementforcrimesthatcouldbecommittedthroughAWSbyfocusing,first,oncrimeswherehighermensrearequirements(dolus)applyand,second,onaformofliabilitythatismoresuitableforincludingculpa,thatis,commandorsuperiorresponsibility.5.3.1.1Dolusdirectus,indirectusandeventualisinICLWhileithasbeennotedthatthereisnoICLcustomaryrulesettingoutageneralidentificationofthevariouscategoriesofmensrea,animportantexceptioncanbefoundintheInternationalCriminalCourt(ICC)Statute(ICCSt).29Takingintoaccountarticle30ICCSt,thebasicmensrearequirementcorrespondstodolus:anindividualistobepunishedforamaterialconductamountingtoacrimewithinthejurisdictionoftheICCwhencommitted‘withintentandknowledge’.30Byestablishingthis,twoformsofdolus,namely,dolusdirectus(wheretheoffenderhasknowledgethathisorherconductwillresultinthematerialelementsofthecrimeandpurposefullyactstoattaintheprohibitedresult)27MFrulli,‘ExploringtheApplicabilityofCommandResponsibilitytoPrivateMilitaryContractors’(2010)15JofConflict&SecL435.28EvanSliedregt,IndividualCriminalResponsibilityinInternationalLaw(OUP2012),17ff.29ACassese,Cassese’sInternationalCriminalLaw(3rded,OUP2013),39.30ICCStart30.SeeWSchabas,TheInternationalCriminalCourt.ACommentaryontheRomeStatute(OUP2010),472ff.DiegoMauri-9781802207675DownloadedfromPubFactoryat05/28/202203:48:45AMviaDukeUniversityLawLibrary

193Autonomousweaponssystemsandthe‘accountabilitygap’conundrum185anddolusindirectus(wheretheoffenderhasknowledgethattheprohibitedresultwilloccurintheordinarycourseofeventsandwiththisinmindheorsheacts),arecovered.31Morecontroversialisathirdformofdolus,namely,doluseventualis,inwhichtheknowledgerequirementasdescribedaboveislower:theoffenderisonlyawarethatbyundertakingaparticularconductheorshewillruntheriskofproducingtheprohibitedresult,anddecidestoactnotwithstandingthisawareness.Sometimesthisthirdformofdolusisreferredtoas‘obliqueintent’inordertomarkitsdifferencefromdirectintent(whichunderliesthetwopreviousformsofdolus).Mostoften,itisusedasasynonymfor‘recklessness’(acommonlawterm):yetcontiguous,thetwotermsarenotstrictlyidentical,evenif—forthepurposesofouranalysis—theycouldbeusedinterchangeably.32Onthebasisofaplainreadingofarticle30,intandemwithrelevantjurisprudence,itcanbearguedthatdoluseventualishasbeenruledoutbythelistofmentalstatesofcrimescontainedintheICCSt.Thisconclusioniswarrantedbecausearticle30(b)(2)defines‘knowledge’as‘awarenessthatacircumstanceexistsoraconsequencewilloccurintheordinarycourseofevents’,thereforerequiringdolusindirectusasaminimum.Thiswasbased,first,onatextualandstrictinterpretationofarticle30:relyingontheliteraltextoftheprovision,theICChasadoptedastandardof‘virtual’or‘practical’certaintyvis-à-visknowledge,findingthatdoluseventualiswouldthereforenotbecapturedbythatprovision.33Second,theargumentwasmadethatthetravauxpréparatoiresofarticle30confirmedthatinterpretation:anyreferencetodoluseventualisorrecklessnesswasexplicitlybanishedbyconsensusofstatesparticipatingattheRomeConference.34Asaconsequence,basedonthegeneralclauseonmensrea,thecurrentunderstandingofdolusintheICCStmaybesaidtoexcludethementalelementformostcrimesthatwillbecommittedthroughAWS.InthescenariosofmaliciousemploymentofAWSforthepurposesofcommittinginternationalcrimes(notonlywarcrimes,butalsogenocide,crimesagainsthumanityand,even,aggression),theconstructionofthemensrearequirementwilltakeplaceonthosebases.Thus,individualsprogrammingAWSinordertousethemforthosepurposes,aswellasindividualsdecidingtofieldthemoroperatingthemwiththoseobjectivesinmind,couldbepunishedalsoinlightofthehighthresholdestablishedbytheICCSt.31BembaICC-01/05-01/08(15June2009)para353;LubangaICC-01/04-01/06)(29January2007)para351;KatangaetalICC-01/04-01/07(20September2008)para530.32VanSliedregt(n28)44.33Bemba(n31)para362.34Schabas(n30)476.DiegoMauri-9781802207675DownloadedfromPubFactoryat05/28/202203:48:45AMviaDukeUniversityLawLibrary

194186AutonomousweaponssystemsandtheprotectionofthehumanpersonWhilethisisinabstractoentirelyreasonableandconvincing,inpracticethisleavesusdissatisfiedinconcreto.Autonomousweaponssystemsthatgorogueinthebattlefield,andthathavebeendevelopedanddeployedbyhumanagentsforeseeingpossiblemisdoings,anddecidingtoactinspiteofsuchawareness(thatis,displayingamentalstateamountingtodoluseventualis),wouldnotbepunishableaccordingtoarticle30ICCSt.Yet,thisisnotthewholestory.First,thetextofarticle30confirmsthattheseprovisionsactasa‘defaultrule’,inthatitapplies‘unlessotherwiseprovided’inthetreaty.35ThereforeitmustbeassessedwhetherthespecificcrimecontainedintheICCStestablishesadiffer-entmensrearequirement—notably,alowerrequirementthandolusdirectusorindirectus.Second,onemustnotforgetthattheICCStisatreatyinstrument,bindingupononlythestatespartytoit.Asfarasthirdstatesareconcerned,customaryICL(whichcontainslowerrequirementsforthementalelement)continuestoapply:36aninquiryacrossmensrearequirementsforthosecrimesisthereforenecessarytoverifyifgapseffectivelyexist.ConsideringthatmoststatesthatarelikelytofieldAWSfirstarenotpartytotheICC(forexample,theUS,Russia,IsraelandChina),customarylawclearlyshouldnotbeneglected.MovingtothespecificcrimesthattheuseofAWSislikelytoresultin,theforemostmightbethatofusingprohibitedweapons,whichamountstoa‘seriousviolationofIHL’committedduringIAC(andapplicablealsotoNIAC).37Thisviolationtriggersbothstateresponsibilityandindividualcrim-inalresponsibilityattheinternationallevel.Inadditiontobeingcustomaryinnature,thisprovisionhasmadeitswayintheICCSt,namely,inarticle8(2)(b)(xx).However,sincethisprovisorequiresweapons,projectiles,materialandmethodstobeincludedinanannextothestatute,itsactualeffectsaremainlysymbolic:forthepurposesofpunishment,itisrequiredthatstatesadoptthisannexinaccordancewiththeproscribedproceduralrules.Otherelementsmakeitdifficulttopunishsuchwarcrimesinconcreto.Tonameone,asarguablefromourpreviouschapters,itwouldbefarfromeasytoarguethatAWSwereindiscriminateweaponsorweaponscausingunnecessarysuffering,andthusthattheirdevelopmentanddeploymentinthemselveswouldamounttothewarcrimeathand.Noparticularissuesconcerningmensreaarisehere;instead,thiswouldbemoreamatterofactusreus.WarcrimeslikelytobecommittedthroughAWSarethoseresultingfromtheviolationoftheIHLrulesondistinctionandproportionality,namely,35Ibid474–5.36RCryer,HFriman,DRobinsonandEWilmshurst,AnIntroductiontoInternationalCriminalLawandProcedure(3rded,CUP2014),382.37HenckaertsJMandDoswald-BeckL,CustomaryInternationalHumanitarianLaw(2vols)(CUP2005)(CIHLstudy)rule156.SeealsoSchabas(n30)243ff.DiegoMauri-9781802207675DownloadedfromPubFactoryat05/28/202203:48:45AMviaDukeUniversityLawLibrary

195Autonomousweaponssystemsandthe‘accountabilitygap’conundrum187thoseof‘makingthecivilianpopulationorindividualcivilians,nottakingadirectpartinhostilities,theobjectofattack’and‘launchinganattackintheknowledgethatsuchattackwillcauseincidentallossofcivilianlife,injurytociviliansordamagetocivilianobjectswhichwouldbeclearlyexcessiveinrelationtotheconcreteanddirectmilitaryadvantageanticipated’.38WhiletheformerofthesecrimesqualifieswhencommittedeitherinIACorinNIAC,asconfirmedbybothcustomaryICLandarticles8(2)(b)(i)and8(2)(b)(e)(i)ICCSt,itisworthnotingthattheRomeStatutecontainsaprovisionregardingsolelyIAC(thatis,article8(2)(b)(iv)):regulationisadmittedlypatchy.Moreinterestinglyforourpurposes,thosespecificcrimesintheICCStariseonlywhencommitted‘intentionally’,whichechoestheAPI’sformulationof‘wilfully’.39Inmensreaterms,thisformulationenvisagesdolusdirectus,consistingoftheintention,asregardsthecrimeofattackingcivilians,tomake‘thecivilianpopulationassuchorindividualciviliansnottakingdirectpartinhostilitiestobetheobjectoftheattack’;40thejurisprudenceoftheICCconfirmsthisholding.41Thecircumstanceinwhichindividualsinvolvedinthetargetingcycleassumetheriskofmistargeting(thatis,areawarethattheAWSmayundertakeunexpectedcoursesofaction,owingtotheirinteractionwiththeoperationalenvironment)wouldqualifyasdoluseventualis,whichraisesthequestionastowhetherthismentalstatecorrespondstothemensrearequirement.Ifinterpretednarrowly,the‘intentionally’requirementaspertheICCStwouldinevitablyexcludethatconduct,aswellasagreatarrayofrisk-basedtargetingchoices.42Conversely,andrelyingondifferentsourcesofinternationallawotherthantheRomeStatuteonthebasisofasystemicinterpretation,notethatAPI’scorrespondingprovision(namely,article85)andcustomaryIHLregardthe‘wilfully’requirementasincludingdoluseven-tualisandrecklessness.43Ithasbeenarguedthatfromtheperspectiveofthe38CIHLrule156.39ProtocolAdditionaltotheGenevaConventionsof12August1949,andrelatingtotheProtectionofVictimsofInternationalArmedConflicts(adopted8June1977,enteredintoforce7December1978)1125UNTS3(API)art85.SeealsoYSandoz,CSwinarski,andBZimmermann(eds),CommentaryontheAdditionalProtocolsof8June1977totheGenevaConventionsof12August1949(MartinusNijhoff1987)(APICommentary)para3474.40ICCStElementsofCrimes,element3.41Katangaetal(n31)para271;AbuGardaICC-02/05-02/09(8February2010)para93(forNIAC).42MBo,‘AutonomousWeaponsandtheResponsibilityGapinLightoftheMensReaoftheWarCrimeofAttackingCiviliansintheICCStatute’(2021)19JICJ1(forthcoming).43APICommentarypara3474(arguingthatsuchformulationincludes‘conceptsof“wrongfulintent”or“recklessness”,namely,theattitudeofanagentwho,withoutDiegoMauri-9781802207675DownloadedfromPubFactoryat05/28/202203:48:45AMviaDukeUniversityLawLibrary

196188AutonomousweaponssystemsandtheprotectionofthehumanpersonICCabroadinterpretationofmensrearequirementsisviableandperfectlylegitimate,andthat,inallcases,customaryICLwouldcontentitselfwithdoluseventualis.44ThefirstobstaclethatinterpreterswillneedtofaceistheadaptationofmensrearequirementstotherealityofAWS.However,inourviewinterpretiveeffortscanbemadetothisend.LexlataseemsadequateforensuringtheproperpunishmentofatleastsomecrimescommittedthroughAWS.5.3.1.2ThelimitedspaceforculpaincurrentICL:thecaseofcommandorsuperiorresponsibilityOurfindingsaboveleaveonlylimitedsatisfaction.MostcasesofmisdoingscommittedthroughAWSwouldbetheresultofnegligence:AWScouldreactinawaythat,whileforeseeable,couldhavebeenavoidedthroughbetterchoicesinthedevelopmentordeploymentphases.Internationalcriminallawdoesnotnormallyenvisionculpaasarelevantmensreaforwarcrimes(andinternationalcrimesingeneral),45thatis,thosesituationsinwhichtheperpetrator—whetherawareoftherisksinvolvedintheconductornot—decidestoactontheconvictionthattheprohibitedconsequencewillnotoccur.However,thereisatleastoneformofliabilitythathasbeenconstruedonthebasisofnegligence,namely,commandorsuperiorresponsibility.46Accordingtothisformofliability(specifictoICL),47militaryorotherhierarchicalsuperiorscanbeheldresponsibleforwarcrimescommittedbytheirsubordinatesiftheyknew,oratleasthadreasontoknowgiventhecir-cumstancesrulingatthattime,thatthosesubordinateswereabouttocommitorwerecommittingawarcrimeandthesuperiorsfailedtotakeallnecessaryandreasonablemeasurestoprevent,repressandreportthecommissionthereof.48Fromthedoctrinalviewpoint,thenatureofcommandorsuperiorresponsibil-beingcertainofaparticularresult,acceptsthepossibilityofithappening;ontheotherhand,ordinarynegligenceorlackofforesightisnotcovered’).SeealsoProsecutorvPrlićetalICTY-04-74-T(29May2013)para192.44Cassese,Cassese’sInternationalCriminalLaw(n29)75–6;Bo(n42)(andref-erencestherein);AmorosoandGiordano(n3)221–2.45ButseeProsecutorvBlaskićICTY-95-14-T(13March2000)para152;ProsecutorvKayishemaandRuzindanaICTR-95-I-T(21May1999)para146(admit-tingthat,inrelationtogravebreaches,‘themensrea…includesbothguiltyintentandrecklessnesswhichmaybelikenedtoseriouscriminalnegligence’).46CMeloni,CommandResponsibilityinInternationalCriminalLaw(AsserPress2010),200ff.ContraseeBBonafé,‘CommandResponsibility’,inACassese(ed),TheOxfordCompaniontoInternationalCriminalJustice(OUP2009).47Cryeretal(n36)384;Meloni(n46).48ICCStart28;ICTYStatuteart7(3);ICTRStatuteart6(3);SCSLStatuteart6(3);APIarts86–7;CIHLrules152–3.DiegoMauri-9781802207675DownloadedfromPubFactoryat05/28/202203:48:45AMviaDukeUniversityLawLibrary

197Autonomousweaponssystemsandthe‘accountabilitygap’conundrum189ityisdebated.Accordingtosomescholars,ithastobeconsideredasawayofimputingresponsibilityforsubordinates’crimestotheirsuperiors(thusstandingasaformofderivativeliability),whileothersregarditasaformofresponsibilitybyomission(thusoperatingasaformofdirectliability).49YetthisdoctrinaldebateanditsreflectionsonICLcaselawcanbeleftasideforthepurposesofouranalysis.50Commandorsuperiorresponsibilityfeaturesthreebasicrequirements:asuperiororsubordinaterelationship,afailuretoactasrequiredandacor-respondingmensrea.51Astothefirst,ithasbeeninterpretedinlightofatestof‘effectivecontrol’,understoodastheabilitytomateriallypreventorpunishsubordinates’criminalconduct.52Whilethenatureofthistesthasmadetheobjectofconsiderabledebate,53itsextensiontotherelationsbetweenacom-manderandAWSisoftenquestioned.Accordingtosome,thedoctrineasitstandstodaywouldapplyonlytohuman-to-humanrelationships,anditsexten-siontoAWSwouldresultinananalogicaloperationthat:(1)normatively,ICLproscribesasageneralprinciple;and(2)theoretically,wouldmakenosense,asthisformofaccountability’spreconditionliesintheconductoftwoentitiesendowedwithmoralagency.54Conversely,othershavearguedthatcommandorsuperiorresponsibilityisaconceptthatmaybere-adaptedtoapplytotherelationshipbetweenAWSandhumans.55Providedthatthisconundrumissolved,theremainingfeaturesofcommandorsuperiorresponsibility—indeedstrictlyintertwined—arenolessproblem-atic.Again,thisisamatterofbothactusreusandmensrea.Letusaddresstheminturn.49Cassese,Cassese’sInternationalCriminalLaw(n29)242ff.50SeeFrulli,‘ExploringtheApplicability(n27)10ff(accordingtowhomthelatterappraisalofcommandorsuperiorresponsibilityhasprevailedinICTY’sandICTR’scaselawaswellasintheICCSt).51ProsecutorvMucićetalICTY-96-21-T(16November1998)para344.52Ibidpara256.53Cryeretal(n36)386–8.54AmorosoandGiordano(n3)223;Crootof(n10)1378–80;Chengeta(n20);RGeissandHLahmann,‘AutonomousWeaponsSystems:AParadigmShiftfortheLawofArmedConflict?’,inJDOhlin(ed),ResearchHandbookonRemoteWarfare(EdwardElgar2017),393.55PMargulies,‘MakingAutonomousWeaponsAccountable:CommandResponsibilityforComputer-GuidedLethalForceinArmedConflicts’,inJDOhlin(ed),ResearchHandbookonRemoteWarfare(EdwardElgar2017),441(suggestingthata‘modestrevisionofthedoctrine’wouldbeenoughforhavingAWS’coursesofactioncoveredadequately).Onthesameline,seeAmorosoandGiordano(n3)223–4(questioninghoweverwhethereffectivecontrolcouldbeensuredonAWSduetotheirincreasedoperationaltempo).DiegoMauri-9781802207675DownloadedfromPubFactoryat05/28/202203:48:45AMviaDukeUniversityLawLibrary

198190AutonomousweaponssystemsandtheprotectionofthehumanpersonThematerialelementofthisformofliabilityliesinafailuretoact(thusanomission),thatis,totakenecessaryandreasonablemeasurestopreventorpunishthesubordinates’criminalbehavior.Asillustratedabove,theconceptof‘reasonableness’iskeyforestablishingthecontentofprimaryobligationsofIHRLandIHL.56Bythesametoken,forthepurposesofICLthisstandardhastobeevaluatedcasebycaseandwithregardtoallfactualcircumstancesrulingatthetime,suchasthedegreeofcontrolexercisedbythesuperiorandthelikelihoodthatsubordinatescommitaspecificcategoryofcrimes;asignif-icantboundaryliesinthatasuperiorcannotbeobligedto‘dotheimpossible’.57Thisfindingistotallyinlinewithageneralprincipleoflaw,encapsulatedinthemaximadimpossibilianemotenetur,whichinturnisstrictlyconnectedwiththemensrearequirementunderlyingcommandorsuperiorresponsibility.Movingnowtothementalelement,notethatarticle28ICCprovidesfortwodifferentsetsofrequirements,dependingonwhetherthesuperiorisamilitarypersonoracivilian.Inthefirstofthese,thesuperiorkneworshouldhaveknownaboutthecrimes(thusembracingdoluseventualisaswellas,accord-ingtosome,negligence).58Onthecontrary,civiliansuperiors’responsibilityarisesiftheyknewaboutthecrimeor‘consciouslydisregardedinformation’regardingthecrime:fromaplainreadingofthetextonlyformsofrecklessnessqualify,merenegligencebeinginsufficient.Thisdoublestandardhasbeenmotivatedbystates’reluctancetobroadenthisareaofresponsibility,butaccordingtosomeitmayultimatelybejustifiedinlightofthedifferentrolesplayedbymilitaryandciviliansuperiors.59Inallcases,commandorsuperiorresponsibilityisnotaformofstrictliability,whichwouldbeatoddswiththeprincipleofculpability:either‘actualknowledge’or‘potentialknowledge’ofthecommissionofacrimeisrequired,thelattertobeunderstoodasthatcom-mandersmustpossess‘alarminginformation’aboutthe(possible)criminalactionofthesubordinate.60Whilethismaybetakenasawayofacknowledg-ingnegligenceasmentalstate(astheICCapparentlydid),theissueremainsunsettled.6156SeeCh3,s3.2.2.3andCh4,s4.3.3.2.57ProsecutorvOrićIT-03-68-T(30June2006)para331.58Meloni(n46)184ff.SeealsoBemba(n31)para429.59Meloni(n46)186.60SeemoreindetailProsecutorv.KrnojelacICTY-97-25-A(17September2003)para155(explicitlyrejectingtheexistenceofageneral‘dutytoknow’incumbentuponthecommander).Thismeansthat‘theassessmentofthecommander’smensreadependsonthespecificcircumstancesofeachcase,andcannotbeinferredmerelyfromthegeneralcontextsurroundingthecommissionofinternationalcrimes’:seeBonafé(n46)271.61Bemba(n31)para429(identifyinga‘differentstandard’inart28oftheICCStatutevis-à-visthecorrespondingprovisionsinothertribunals’statutes).DiegoMauri-9781802207675DownloadedfromPubFactoryat05/28/202203:48:45AMviaDukeUniversityLawLibrary

199Autonomousweaponssystemsandthe‘accountabilitygap’conundrum191ApplyingthosefindingstoAWS,anumberofsubstantiveobstaclesarise.First,theextenttowhichthesuperiorcouldbesaidtohavefailedtoprevent—letalonerepress—crimesallegedlycommittedbyAWSisquestionable,namely,inthosescenarioswherethesystemreactstoenvironmentalstimuliinanunforeseeablemanner.Ashasbeendemonstrated,toobjectthatthesamewouldgoforhumansubordinates(actingtruly‘autonomously’,thatis,exert-ingtheirfreedomofaction)ispointless,asinthelattercasethedutytopunishremainsapplicable.62Second,andagain,thementalstateofthecommanderfieldinganAWScouldatbestqualifyasnegligence,forinstance,whenitignoresdirectivesissuedbythesystemmanufactureraccordingtowhichthatAWScouldonlybeusedreliablyinunclutteredenvironments.Currentinterpretationsofthemensrearequirementestablishingahighthreshold(forinstance,inaccordancewiththemodelof‘personaldereliction’)couldbesupersededbylessstringentjurisprudencebyinternationalordomestictribunalsadjudicatingwarcrimes:thisinterpretationcouldbesponsoredbytheICC,whichalreadyregardsthe‘shouldhaveknown’formulacontainedinarticle28(a)(i)ICCStasculpablenegligence.63Letusnowdrawconcludingremarks.Evenremaininganchoredtoadelegelataperspective,itwouldnotcomeasasurpriseif,bywayofinterpretation,judicialbodiescouldprovideaninterpretationofICLprovisionsthatistai-loredtoAWSandthecomplexrelationshipwithhumanagents.64Asolutionsuchasthiscouldbegroundedinexistingrequirements,suchasthatof‘suf-ficientlyalarminginformation’,tobeappraisedinlightofthespecificitiesofAWS-relatedtechnologies.Asregardsscholarship,someconvincingargu-mentshavebeenputforwardsuggestingthatrulesoncommandresponsibilitymaybeappliedtoautonomouscyberweapons.65Yet,asetofscenarios—thoseinwhichnegligenceonthepartofthesuperiorcouldnotbeestablished,forinstance,intheeventoftotallyunforeseeableinteractionsbetweentheAWSandtheenvironment—wouldnotbecomprised,lestculpabecomeamaskedformofstrictliability,whichispatentlyinconsistentwithICL.Insummary,62AmorosoandGiordano(n3)223.63Cryeretal(n36)390.64Forasimilarconclusion,seeNReitinger,‘AlgorithmicChoiceandSuperiorResponsibility:ClosingtheGapBetweenLiabilityandLethalAutonomybyDefiningtheLineBetweenActorsandTools’(2015)51GonzagaLRev79(holdingthatthecommandorsuperiorresponsibilityadaptstosuperior–subordinaterelationshipwherethelatterarenon-humanagents).65RBuchanandNTsagourias,‘AutonomousCyberWeaponsandCommandResponsibility’(2020)96Int’lLStud645(arguinginfavorofasuperior–subordinaterelationshipbetweenahumanagentandanautonomoussystem).DiegoMauri-9781802207675DownloadedfromPubFactoryat05/28/202203:48:45AMviaDukeUniversityLawLibrary

200192AutonomousweaponssystemsandtheprotectionofthehumanpersonevenifcommandorsuperiorresponsibilityisadaptedtoapplytoAWS,thatsolutionwillworkproperlyonlyinrespectofalimitedsetofcases.5.3.2ApplyingModesofPerpetrationtoIndividualsInvolvedintheDevelopmentandDeploymentofAWSOurfocusthusfarhasbeenonparticularindividualscommittinginternationalcrimesthroughAWS.However,itcanbeexpectedthatapluralityofindividu-als(nottomentionlegalpersons)willplayaroleinthecycleofproductionandemploymentofAWS:programmers,engineers,manufacturers,procurementofficials,stateagents(suchasmilitarycommandersandtheirsubordinates)andpoliticalleaders.66Distributingresponsibilityacrossalltheseplayersmaybehard,giventhedifficultyofreconstructingthecausalchainthatledtoaparticularmisdoingand,consequently,ofestablishingthecontributionbyeachandeveryindividual.Thisissuehasbeenreferredtoasthe‘manyhandsproblem’,alabelwhichisverysuitableforAWS-relatedscenarios.67Asoccurswithdomesticcriminallaw,ICLfeaturesdifferentmodesofperpetrationthataimtoproperlyappraisethecontributionofeachindividualtoaparticularcrimeandthustofairlysentencethem.Thosemodesofper-petrationarenotwatertightcompartments;theymaysubstantiallyoverlapinpractice,anditisuptothecriminalcourttoexertdiscretionandchoose‘themostappropriateheadofresponsibilityunderwhichtoattachcriminalresponsibilitytotheaccused’.68Onthepremisethatthosecategoriesarethusflexible,itistobeascertainedwhethertheyareadequateforensuringaproperdistributionofcriminalresponsibilityacrossallindividualspossiblyinvolvedinAWS-relatedcrimes.Tothisend,wenowaddressexistingformsofco-perpetrationandaidingorabettinginICL.5.3.2.1Formsof(co-)perpetration:thedoctrineofjointcriminalenterpriseThefirstmodeofresponsibilitycomingtotheforeisjointcriminalenterprise(JCE),whichaddressesthecaseofparticipantsinacommoncriminalplan.69Althoughnotspecificallyregulatedinthestatutesofinternationaltribunalandcourts,thisdoctrinehasbeenelaboratedfirstinNurembergandTokyo,andthenexpandedmainlythankstotheICTY’scaselaw.Thismodeofrespon-sibilityencompasses:(1)liabilityforacommonpurpose(JCEI);(2)liability66HeynsReport(n1)para77.67AmorosoandGiordano(n3).68Cryeretal(n36)354.69ACassese,‘TheProperLimitsofIndividualResponsibilityundertheDoctrineofJointCriminalEnterprise’(2007)5JICJ109.DiegoMauri-9781802207675DownloadedfromPubFactoryat05/28/202203:48:45AMviaDukeUniversityLawLibrary

201Autonomousweaponssystemsandthe‘accountabilitygap’conundrum193forparticipationinacommoncriminalplanwithinaninstitutionalframework(JCEII);and(3)liabilitybasedonforesightandvoluntaryassumptionofrisk(JCEIII).70Allindividualsinvolvedinthefurtheranceofthecommonplanaretobetreatedasprincipalperpetratorsforthepurposesofsentencing.ThescholarlydebateonJCE(anditsvariousshapes)issovastthatitwouldbeimpossible,andultimatelyoutsidetheaimsoftheparagraph,totackleitinitsentirety.Sufficeittorecallthat,aspertheactusreus,meremembershipinagroupisnotenough:‘significantcontribution’tothecrimeisrequired.71Asregardsmensrea,whileJCEIandJCEIIfeaturean‘intent’requirement(amountingtodolusdirectusandindirectus),JCEIIIcontentsitselfwithlowermentalelement:accordingtothisform,members—showingtheintentiontoparticipateinandfurtherthecriminalactivityofthegroup—canbeheldresponsibleforcrimescommittedoutsidethecommonplan,butnonethelessoccurringas‘anaturalandforeseeableconsequence’oftheplan.72Thatis,thementalelementofJCEIIIamountstodoluseventualis,whichmakesitthemoreextended(andcontroversial)formofJCE.73Again,culpaisnotenvi-sionedasarelevantmentalelement.74Incontrastwiththetraditionalapproachtothismodeofperpetration,theICCSt‘unpacks’JCEandregulatesthroughthelensofco-perpetrationbasedonjointcontrol(article25(3)(a))andamoreattenuatedformofperpetrationthroughcommonpurpose(article25(3)(d)).Inaddition,itimposesahighermentalstate,inthatitrequiresintentionalpartici-pation,withtheaimoffurtheringthecriminalactivityandintheknowledgeofthegroup’sintentiontocommitthecrime.DoluseventualisandrecklessnessasperJCEIIIwouldthusbeexcluded,eventhoughsomearguethatabroadandinclusiveinterpretationwouldnotbeunwarranted.75AsregardscrimescommittedthroughAWS,theJCEdoctrineanditsvariousapplicationsbyinternationalcourtsmayprovideaneffectivetoolforprose-cutingandpunishingthe‘manyhands’involvedinthechain.Programmers,engineers,manufacturersandstateagentswouldbetreatedasperpetrators(orco-perpetrators),providedthatitisprovenbeyondanyreasonabledoubtthat:70ProsecutorvTadićICTY-94-1-A(15July1995)para220.71ProsecutorvBrđaninandTalićICTY-99-36(3April2007)para430;ProsecutorvMilutinovićetalICTY-99-37-AR72(21May2003)para163.72ProsecutorvNtakirutimanaetalICTR-97-17-A(13December2004)para465.73ProsecutorvTadić(n70)para228(‘responsibilityforacrimeotherthantheoneagreedwiththecommonplanarisesonlyif,underthecircumstancesofthecase:(i)itwasforeseeablethatsuchcrimemightbeperpetratedbyoneorothermembersofthegroupand(ii)theaccusedwillinglyacceptsthatrisk’).74Cryeretal(n36)359.75Ibid362;contraseeCassese,Cassese’sInternationalCriminalLaw(n29)132(arguingthatforesightandvoluntaryassumptionoftherisksufficeforthementalelementofart25(3)(d)).DiegoMauri-9781802207675DownloadedfromPubFactoryat05/28/202203:48:45AMviaDukeUniversityLawLibrary

202194Autonomousweaponssystemsandtheprotectionofthehumanperson(1)acommonplan,designorpurposeisinplacethatinvolvesthecommissionofaninternationalcrime;and(2)theaccusedparticipatedthereinbygivingasignificantcontribution(forinstance,bydevelopingthealgorithmsorpro-grammingandinstallingthesoftware)andwiththerequiredmentalelement(tothepointthatalsodoluseventualismaysuffice).Asregardsthis,thesignif-icantcontributionmaybegivenwellbeforetheoutbreakofthearmedconflictwherethespecificcrimewillbecommitted.76ThemainadvantageofapplyingJCEinthosescenariosisthatallindividualswouldbetreatedandsentencedalike,asprincipalperpetrators,acircumstancereflectingtheequalityoftheircontributiontothecommissionofthecrime.77ThemaindisadvantageisthatthisdoctrinewillcoveronlyalimitedportionofthewiderarrayofmisdoingsthatAWSmaycommitintheoperationalfield:faultycoursesofactionandfalsepositivescouldnotbeincludedforbothactusreus-relatedandmensrea-relatedreasons.Itseemsthatthisdoctrine’smaincriticalpoints—suchasitspropensitytodiminishindividualcriminalliability,totheextentthatsomehavere-phrasedtheacronymas‘justconvicteveryone’—doesnotstrictlydependonAWS,butinsteadtheyarelinkedtothewayinternationaltribunalsandscholarshiphavebeeninterpretingitinpastdecades.785.3.2.2Lessthanperpetration:aidingandabettingAsadistinctmodeofliability,aidingandabettingiswellknownbothinICLandinmostdomesticcriminallawsystems.79Allthosewhosupportedthe‘planning’,‘preparation’and‘execution’ofaninternationalcrimearethussubjecttocriminalsanction.Aidingandabettingimpliespracticalassistance,encouragementormoralsupporttotheprincipalperpetrator.Conceptually,thismayappearasasignificantdifferencefromJCEandotherformsofco-perpetration,whichwouldinferlowersentences.AsthecaselawoftheICTYconfirms,though,thetwomodesofliabilityarenotsegregated,andoverlapsrecur.80Theconstitutiveelementsofaidingandabettingcontemplateactusreusandmensrea.Asregardstheformer,theaiderorabettor’sconductmusthave76TMcFarlandandTMcCormack,‘MindtheGap:CanDevelopersofAutonomousWeaponsSystemsbeLiableforWarCrimes?’(2014)90Int’lLStud361,375.77AmorosoandGiordano(n3)220.78JMBeard,‘AutonomousWeaponsandHumanResponsibilities’(2014)45GeorgetownJofInt’lL617,661.79Cassese,Cassese’sInternationalCriminalLaw(n29)193.80ProsecutorvVasilievićICTY-98-32(25February2004)para182(‘aidingandabettingisaformofresponsibilitywhichgenerallywarrantslowersentencesthanresponsibilityasaco-perpetrator’).DiegoMauri-9781802207675DownloadedfromPubFactoryat05/28/202203:48:45AMviaDukeUniversityLawLibrary

203Autonomousweaponssystemsandthe‘accountabilitygap’conundrum195a‘specificallydirected’and‘substantial’effectonthecommissionofthecrime.81Inspiteoftheapparentlystringentformulation,thestandardhasnotbeensetparticularlyhighinpractice,confirmedbycaseswheretheassistancewasmorethan‘marginalparticipation’,beingheldtosatisfytheactusreusrequirementbyinternationalcourts.82Also,the‘specificdirection’oftheconducttowardsthecommissionofthecrimehasbeendownplayed,asfoundnottocorrespondtocustomaryICL.83Movingontothemensrearequirement,adhoctribunalstendtoaccepta‘knowledge’standard,meaningthattheaiderorabettorisawarethathisorherconductwillsupportthecommissionofthecrime(evenifheorshedoesnotknowtheprecisecrimethatwillbecommit-ted).84Conversely,theICCStsetsthebarhigher:article25(3)(c)criminalizesaidersandabettorsacting‘forthepurposeoffacilitatingthecommission’oftheprincipalperpetrator’scrime.Whileleftundefinedinthestatute,theterm‘purpose’hasbeenregardedbytheICCasleaningmoreontheconceptof‘intent’,whichexcludesthemereawarenessofthesubstantiallikelihoodofacrimeoccurring.85Hence,ithasbeencriticizedforexcessivelylimitingthescopeofaidingandabetting,insharpcontrasttocustomaryICL.86Againstthisbackground,criminalresponsibilityforprogrammers,engi-neersand,moregenerally,developersofAWSmayariseonthefollowingconditions.Asapreliminaryremark,theresponsibilityofarmsproducersanddealersasaccessoriestotheprincipalperpetratorsisaclassicaltopicinICL.TheoftenquotedexampleofaidingorabettingawarcrimeistheZyklonBcase,decidedbytheBritishMilitaryCourtaftertheSecondWorldWar.87TopofficialsoftheGermanfirmwereconvictedasaccessoriestowarcrimescom-mittedbytheNaziregimeonthebasisthattheyhadmanufacturedandsold81ProsecutorvTadić(n70)para229.82ProsecutorvFurundžijaICTY-95-17/1-T(10December1998)para231;ProsecutorvNyiramasuhukoICTR-98-42-A(14December2015)para2083;ProsecutorvTaylorSCLS-03-01-A-1389(26September2013)para371;cfMbarushimanaICC-01/04-01/10(16December2011)para279(‘asubstantialcontributiontothecrimemaybeadded’);AlMahdiICC-01/12-01/15(24March2016)para26(holdingthatassistanceneednottobe‘“substantial”oranyhowqualified’).SeealsoOAHathaway,AFrancis,AHaviland,SReddyKethireddyandATYamamoto,‘AidingandAbettinginInternationalCriminalLaw’(2019)104CornellLRev1593,1611.83Cryeretal(n36)372–3.SeealsoProsecutorvTaylor(n82)para474.84Hathawayetal(n82)1609(providinganexplanatorytable).85Bemba(n31)para97(‘[i]tisnotsufficientthattheaccessorymerelyknowsthathisorherconductwillassisttheprincipalperpetratorinthecommissionoftheoffence’).86Cassese,Cassese’sInternationalCriminalLaw(n29)195.87TeschandOthers(1947)LawReportsofTrialsofWarCriminalsvolI93.Foradifferentscenario,adjudicatedbytheUS,seeUnitedStatesofAmericavCarlKrauchandOthers(1948)8TWC1169.DiegoMauri-9781802207675DownloadedfromPubFactoryat05/28/202203:48:45AMviaDukeUniversityLawLibrary

204196AutonomousweaponssystemsandtheprotectionofthehumanpersonZyklonB(apoisonousgas)foruseinconcentration-campkillings.Inadditiontowarcrimestrialswhereindustrialistswereconvictedasprincipalperpetra-tors,theZyklonBcaseinauguratedatrendaccordingtowhichmanufacturesofweaponstobeemployedinthefieldcouldbeheldcriminallyliableforprovidingthenecessarysupporttooperatives(militarycommanders,politicalleadersand,eventually,soldiers).88Thereisnoreasontoholdthatsimilarfindingswouldnotapplytothedevel-opmentofAWS.89Again,scenarioswheredefensecontractorswererequestedtoproduceautonomoussystemsincapableofproperlydirectingforceagainstpermissibletargetswouldbeeasiertodealwith:if‘knowledge’(or,pursuanttotheapparentlymorerestrictiveformulationprovidedbytheICCSt,‘purpose’)regardingthefuturecommissionofcrimesisdemonstrated,rulesonaidingandabettingwillapplytoallthoseindividualsinvolvedinthedevelopmentofAWS.90Bycontrast,ascenarioinwhichstateorganscommissioncompaniestoproduceAWSforuseinunclutteredspaces(suchasmaritimewarfare),andsubsequentlymilitarycommandersdecidetoemploythosesystemsinopera-tionalfieldspopulatedbyprotectedpersons(typicallyurbanwarfare),wouldnotengagethemodeofliabilityathand:thatis,itwouldbeverydifficulttodemonstratetherequestedmensreaonthepartofdevelopers.Iftheweaponssystemathandisnotillegal(as,atleastprimafacieandcurrently,seemsthecaseforAWS),andgiventhatmostofthemisdoingsofAWSwillbetheresultofunforeseenorunforeseeableinteractionswiththeenvironmentandthesubsequentalgorithmicdecision-makingprocesses,todemonstratethatthemensrearequirementissatisfiedmayturnouttobea‘herculeanchalleng[e]’inthosecases.91Afinalissuedeservingattentionconcernsaspecificelementofwarcrimes,namely,the‘warnexus’betweentheimpugnedconductandthearmedconflict.ThisisarequirementthatissettledincustomaryICL;intheICCElementsofCrimes’words,thecriminalconductmusttakeplace‘inthecontextofand[be]associatedwith’anarmedconflict.92Whenitcomestoestablishingtheresponsibilityofthosewhodevelopweaponsthataretobeusedinhostilities,theissuearisesastowhethertheimpugnedconductislinkedtoanyarmedconflict,particularlyinthosecaseswheretherequestingstateisnotinvolved88Foracomprehensiveoverview,seeMLippman,‘WarCrimesTrialsofGermanIndustrialists:The“OtherSchindlers”’(1995)9TempleInt’l&CompLJ173.89AmorosoandGiordano(n3)218.90Yetitmaybearguedthatart25(3)(c)ICCSt—ifstrictlyinterpreted—wouldrenderprosecutionofthosescenariosextremelyhard.SeeWSchabas,‘EnforcingInternationalHumanitarianLaw:CatchingtheAccomplices’(2001)83IRRC439.91JMBeard(n78)650.92ElementsofCrimes,WarCrimes,art8(2)(a)(i)para4.DiegoMauri-9781802207675DownloadedfromPubFactoryat05/28/202203:48:45AMviaDukeUniversityLawLibrary

205Autonomousweaponssystemsandthe‘accountabilitygap’conundrum197inanyconfrontationagainstotherstatesorarmedgroups.Theabsenceof(aconstitutiveelementoftherequired)actusreusforwarcrimesmakesitselfevident.Forthosereasons,someauthorshavearguedthataidingandabettingstateorganswhennoarmedconflictisimpendingcouldnotbesanctionedbyexistingICL,whichopensupagapinthecaseofAWS.93Twofinalremarksaredue.First,thisismoreaproblemderivingfromthenexusrequirement—asitexistsintreatyandcustomarylaw,andasitisinter-pretedbycourts.Gapsinthisdonotdependonthetechnologyemployedanditsstructuralfeatures,suchastheunpredictabilityofcoursesofactionofAWS(which,bycontrast,isfarmorerelevantunderthemensrearequirement).Second,thatdevelopersofweaponssystemsactingbeforearmedconflictcouldnotbeheldresponsibleforaidingorabettingprincipalperpetratorscommittingwarcrimesthroughtheprovidedweapons,isbynomeanstheobligedconsequenceofjurisprudence.Forinstance,theAppealsChamberoftheICTYleftadoorwideopentothesescenarios,whenitheldthat‘[t]heactusreusofaidingorabettingacrimemayoccurbefore,duringoraftertheprincipalcrimehasbeenperpetrated’.94Whileconcededlygeneralinscope,suchaffirmations—preciselythankstothebreadthoftheirformulation—aresuitableforcoveringscenariosinvolvingthedevelopmentofAWS.95Ourreflectionsonmodesofperpetrationleadtooutcomesthatareanalo-goustothosederivingfromtheanalysisofmensrearequirementsofinterna-tionalcrimes:lexlatacanbeinterpretedandadaptedtocopewithchallengesraisedbyAWS.Inthis,ICLprovidesanadequateframeworkofindividualresponsibility,atleastinrespectofanumberofscenarios.5.3.3OvercomingChallenges:TheContinuingNeedtoRetainthePrincipleofCulpabilityinICLLetusnowdrawsomeconcludingremarksfromouranalysisofICL.AccordingtotheInternationalMilitaryTribunal(IMT),‘crimesagainstinternationallawarecommittedbymen,notabstractentities,andonlybypunishingindividualswhocommitsuchcrimescantheprovisionsofinternationallawbeenforced’.9693McFarlandandMcCormack(n76)376ff.94ProsecutorvBlaškićICTY-95-14-A(29July2004)para48.AsimilarlineofargumentationhasbeenadoptedbytheSCSL:seeProsecutorvTaylor(n82)para484(‘[t]helendingofpracticalassistance,encouragementormoralsupportmayoccurbefore,duringorafterthecrimeorunderlyingoffenceoccurs’).95ContraseeMcFarlandandMcCormack(n76)378(arguingthatcontextualexist-enceofanarmedconflictasrequiredbytheElementsofCrimescouldnotbesubjecttoanalogyasperart22(2)ICCSt).96JudgmentoftheInternationalMilitaryTribunal1946(1947)41AJIL172,221.DiegoMauri-9781802207675DownloadedfromPubFactoryat05/28/202203:48:45AMviaDukeUniversityLawLibrary

206198AutonomousweaponssystemsandtheprotectionofthehumanpersonOneofthebasictenetsofcriminallawingeneral,andICLinparticular,isthatcriminalresponsibilitypertainstotheindividual:onlyculpableindividualsaretobearthepenalconsequencesoftheirownconduct.97Thisisthegistoftheprincipleofculpability.98Asaconsequence,human–machineinteractionsmustbesuchthattheallocationofindividualresponsibilityincasesofinter-nationalcrimescommittedbyAWSremainsfeasible.ThismaywellbetheprioritizedgoalofthenotionofMHC.99ExistingICLissurelyadequateforregulatingacertainsetofinternationalcrimescommittedthroughAWS.Commanders,soldiers,stateofficialsandpoliticalleaderswhodecidetofieldlargelyindiscriminateAWSinhighlypop-ulatedtheaters,suchasurbanwarfare,couldhardlyescapecriminalresponsi-bilityforwarcrimes:nosubstantiveobstaclestoascertainingboththeactusreusandthemensrearequirementswouldmaterialize.Slightlymodifiedsce-nariosfeaturing‘warlords’producingAWSormaliciousprogrammerswouldbeincludedtoo.100Incontrast,othercaseswouldrequireadjustmentstosomeICLdoctrines:doluseventualis,forinstance,maybeenoughtopunishrecklessindividualsforeseeingtheactualriskofthemisdoingsofAWSanddecidingtofieldthemnotwithstandingtheexpectedcoursesofaction.EvenwithregardtotheICCStthisinterpretationwouldbesupportedbythetextofrelevantprovi-sions,withoutraisingissuesofextensiveoranalogicalinterpretation.Thisnotwithstanding,thereisafeelingthataccountabilitygapsaredeemedtoopenup.Autonomousweaponssystems’misdoingscausedbyunpredicta-bleinteractionswiththesurroundingenvironment,whicharemagnifiedbytheinevitableemploymentofadvancedtechniques(firstandforemost,machinelearning),couldhardlyleadtothepunishmentofalltheindividualsinvolved,moreorlessremotely,intheimpugnedcourseofaction.Figuratively,itisasifwewereonabridgeandtherewasabeammissingtoreachtheend:hencethegap.Thequestionishowtoproceed.Giventhatonecannotstandstill,whattodo?Takealeapandjumptotheotherside?Ifyoufailtogatherenoughstrengthforthejump,youriskfalling.Toexpandmensrea,modesofliabilityandevenactusreusmayturnouttobetheproverbialleapinthevoid:thosedoctrines,astheyhavebeenconceivedandapplieduntilnow,aretooheavy97CfEvanSliedregt,‘TheCuriousCaseofInternationalCriminalLiability’(2012)10JICJ1171(arguingthattheprincipleinICLdifferstoacertainextentfromthatindomesticlegalsystems,asithasbeenoriginatedfromaprocessofcriminalizationoflarge-scale,collectiveacts).98ProsecutorvTadić(n70)para186(‘nobodymaybeheldresponsibleforactsortransactionsinwhichhehasnotpersonallyengagedinorinsomeotherwaypartici-pated(nullapoenasineculpa)’).99AmorosoandGiordano(n3)230.100AmorosoandGiordano(n3)218.DiegoMauri-9781802207675DownloadedfromPubFactoryat05/28/202203:48:45AMviaDukeUniversityLawLibrary

207Autonomousweaponssystemsandthe‘accountabilitygap’conundrum199aburdentoentrusttheagilitythisjumprequires.OverstretchingICLrulesandprinciplesmayturnouttobeintolerable,atleastfromatwofoldperspective.Ontheonehand,legally,thismaycauseshortcircuits,asitwouldbeequaltopavingthewayto(not-so-)maskedformsofstrictliability,whichareatoddswithICLandmoderncriminallawingeneral.101Ontheotherhand,politically,itwouldbecounterproductive:inanerasadlycharacterizedbyskepticismtowardsinternationallawanditsinstitutions(amongwhichtheICC,notably),thismayaugmentstates’skepticismtowardsinternationalcriminaljustice,tothefurtherdetrimentofaccountability.ThisisnottosaythatthescopeofICLhastoberestrainedinthenameofmerelydiplomaticorpoliticalreasons—onthecontrary.Itmustbequestionedwhether,atleastinscenariosinvolvingculpaorevenabsenceofmentalpar-ticipationbytheallegedsuspect(s),ICListhecorrectanswertoapressing,ifnotimperativequestion(whotoblamethemisdoingsofAWSon?),orwhetheralternativepathsshouldbefound.Turningtoourpreviousmetaphor,onesolutionmaybetoholdontotheropesandslowlycrossthegap:itmightbedangerous,butitmayleadustotheend.5.4CORPORATERESPONSIBILITYUNDERINTERNATIONALLAW:FOCUSINGONTHELEGALENTITIESDEVELOPINGAWSLetusleavenowtheterrainofindividualcriminalresponsibilityandconsiderotherentitiesinvolved,namely,businesses,whoseroleindevelopingtechnol-ogiestoinfuseAWScouldhardlybeunderestimatedcurrently.102TheUSDoDisinvestingheavilyinbrain–computerinterfaces,103andYouTubevideoenthu-siastswillbefamiliarwithsomeoftheDefenseAdvancedResearchProjectsAgency(DARPA’s)andBostonDynamics’brand-newproducts.104GeneralAtomics,NorthropGrumman,LockheedMartin,BAE,RollsRoyce,Dassault,ThalesandElbitSystemsareafewexamplesofbusinessesthatmaybecitedinthisrespect.105Reportedly,Kalashnikovisactiveinthefieldofdeep-learning101GeissandLahmann(n54)394.102Theterm‘business’isusedinterchangeablywith‘corporation’.103ABinnendijk,TMarlerandEMBartels,‘Brain-ComputerInterfaces.U.S.MilitaryApplicationsandImplications.AnInitialAssessment’,RANDReport(2020).104See,amongmany,https://www.youtube.com/watch?v=uhND7Mvp3f4accessed31August2021.Moregenerally,seeCWMahoney,‘UnitedStatesDefenceContractorsandtheFutureofMilitaryOperations’(2020)36Defense&SecAnalysis180.105HStauffer,‘CorporateLiability:AnAlternativePathtoAccountability?’,inSCasey-Maslen,MHomayounnejad,HStaufferandNWeizmann(eds),DronesandOtherUnmannedWeaponsSystemsunderInternationalLaw(Brill2018),204ff.DiegoMauri-9781802207675DownloadedfromPubFactoryat05/28/202203:48:45AMviaDukeUniversityLawLibrary

208200Autonomousweaponssystemsandtheprotectionofthehumanpersonalgorithmsforautonomousdecision-makinginthebattlespace.106ThenumberofbusinessesoperatinginAWS-relatedmarkets—oftenasgovernments’contractors—issimplyuncountable.107ThisrolewillonlybemagnifiedbytheacquisitionofAWSbyPMSCs,whichwillusetheminperformingtheirfunctions:alongsideindividualresponsibilityformanagersandemployers,theresponsibilityofthecompanyitselfisachallengingissue.108Imaginethefollowingscenario.Amilitaryvesselequippedwithanauton-omoussystemcapableofinterceptingenemymilitaryaircraftentersahostiletheater;suddenly,thesystemdetectsanapproachingaircraftand,withouthumanintervention,decidestouseforceagainstit.Theaircraftishitanddestroyed.Shortlyafterwards,reconnaissanceunitsreachthesitewherethedownedaircraftcrashed,anddiscoverthatitwasnotamilitaryaircraft,butacivilianplanethathaddepartedfromtheenemymilitarybasebyaccident.Well,thestoryisallbutfictional.ThosefactsreallyhappenedintheIranianterritorialwaters,inJuly1988;themilitaryvesselatstakewastheUSSVincennes(equippedwiththecomputerizedAegisairdefensesystem)andthecivilianplanewasanIranianAirbuscarrying290personsaboardwhoalldiedasaresultoftheunintendedengagement.TheheirsofsomeofthedeceasedpassengersandcrewlatersoughtcompensationfromtheUSandbusinessesinvolvedintheconstructionoftheAegisAirDefenceSystem.109ThestorygoesthatfutureaccidentsinvolvingtheuseofAWS—farmoreadvancedthanthethen-operatingAegis—arethuslikelynotonlytohappen,butalsotomultiplycompensationclaimsbeforedomesticcourts.Aninternationalregimefixingcommonrulesandensuringthatvictimsseekandobtainremediesmaythusbeanauspiciousstepforward.Onthispremise,theissueofholdingthosebusinessesaccountableincasesofmisdoingsofAWShassurfacedfromtheveryearlymomentsoftheinterna-tionaldebateonsuchtechnology.110Bycontrast,theGuidingPrinciplesreflectthatneedonlyindirectly,totheextenttowhichtheyestablishthataccountabil-itymustbeensured‘inaccordancewithapplicableinternationallaw’.111Thatis,itistobeascertainedwhetheraccountabilitymechanismsforbusinesses106JGHughes,‘TheLawofArmedConflictIssuesCreatedbyProgrammingAutomaticTargetRecognitionSystemsUsingDeepLearningMethods’(2018)21YbIHL99,101.107FWolfe,‘CompaniesDevelopingLethalAutonomousWeapons,AsGroupsSeekBan,ReportSays’(AviationToday,2December2019)https://www.aviationtoday.com/2019/12/02/companies-developing-lethal-autonomous-weapons-as-groups-seek-ban-report-says/accessed31August2021.108Seesupras5.3.109976F.2d1328(1992)(hereafterKoohicase).110HeynsReport(n1)para79.111GuidingPrinciples,principled.DiegoMauri-9781802207675DownloadedfromPubFactoryat05/28/202203:48:45AMviaDukeUniversityLawLibrary

209Autonomousweaponssystemsandthe‘accountabilitygap’conundrum201operatinginthesectorofAWS-relatedtechnologiesactuallyexistininterna-tionallawand,inthepositive,throughwhichvenues.Thissectionfocuses,first,onexistingmodelsofcorporateresponsibilityunderinternationallaw,inparticularonarecentandpromisingfieldof(or‘movement’in)112internationallaw,namely,businessandhumanrights(B&HRs)(s5.4.1).Secondly,giventhepaucityofthebindingrulesthatcurrentlyexistandtheunlikelinessofsub-stantivechangesinthenearfuture(atleastwithregardtothefieldofAWS),attentionisdevotedtoexistingobstaclestobusinesses’accountabilitybeforedomesticcourts(5.4.2).5.4.1ExistingFormsofCorporateResponsibilityinInternationalLawTraditionally,corporateresponsibilityhasbeenamatterofstates’domesticjurisdiction,notsubjectbythemselvestointernationalnorms.113Thisisbecausecorporationsareestablishedintheterritoryofonestate,conducttheirbusinessthereor,inmanycases,intheterritoryofmorethanonestate.Ifthatremainstrueinprinciple,itcannotbedeniedthat,especiallyinthetwentiethcentury,therehasbeenasignificantmovementtowardstheattri-butionofrightsanddutiestobusinessesdirectlyattheinternationallevel.ImmediatelyaftertheendofWorldWarII,theestablishmentofcriminalcorporateresponsibilityforatrocitiescommittedduring(andalongside)theconflictwasconsideredbuteventuallydiscarded.Asobservedabove,indi-vidualcriminalresponsibilityofbusinessmenthathadplayedaroleinsup-portingthecommissionofNazis’crimeswasestablishedbothatNurembergandinsubsequentprosecutionoflower-rankexecutivesandemployeesofcorporationssuchasI.G.Farben,Flick,KruppandRoechling.114However,thisisarguablydifferentfromholdingacorporationcriminallyresponsible.CriminalcorporateresponsibilitywasnotincludedinthestatutesoftheIMT112ABatesmith,‘CorporateCriminalResponsibilityforWarCrimesandOtherViolationsofInternationalHumanitarianLaw:TheImpactoftheBusinessandHumanRightsMovement’,inCHarvey,JSummersandNDWhite(eds),ContemporaryChallengestotheLawsofWar.EssaysinHonourofProfessorPeterRowe(CUP2014).113RGSteinhardt,‘WeaponsandtheHumanRightsResponsibilitiesofMultinationalCorporations’,inSCasey-Maslen(ed),WeaponsUnderInternationalHumanRightsLaw(CUP2014).114Batesmith(n112)288–9;Lippman(n88).SeealsoUnitedStatesvKrauch(1952)VIIITrialsofWarCriminalsBeforetheNurembergMilitaryTribunalsunderControlCouncilLawNo10,1131ff(‘[w]hereaprivateindividualorajuristicpersonbecomesapartytounlawfulconfiscationofpublicorprivateproperty…acquisition…constitutesconductinviolationof[art47]oftheHagueRegulations’).DiegoMauri-9781802207675DownloadedfromPubFactoryat05/28/202203:48:45AMviaDukeUniversityLawLibrary

210202AutonomousweaponssystemsandtheprotectionofthehumanpersonandtheInternationalMilitaryTribunalfortheFarEast(IMTFE),eventhoughtheconceptofcriminalresponsibilityforlegalpersonswasnotunknowntotheirdrafters:forinstance,theIMThadthepowertodeclarethecriminalnatureof‘groups’or‘organizations’withaviewtosubsequentprosecutionofindividualsinvolvedtherein.115Despitethewidelyacknowledgedimportanceofthefightagainstimpunityforinternationalcrimesandofthekeyroleplayedbypowerfulcorporateactorsinthecommissionofthosecrimes,corporatecriminalresponsibilitywasdiscussedduringthepreparatoryworksofthestatutesoftheICTY,InternationalCriminalTribunalforRwanda(ICTR)andICC,butdiscardedagain.116Seniormanagersandotherindividualsemployedbycorporationscouldthusbeprosecuted,butnotthecompaniesthemselves.117AsonecommentatornotedwithparticularregardtotheICC,thefailureinadoptingamodelofcorporatecriminalresponsibilityisregretful,notonlyinitself,butalsobecauseitsadoptionwouldmarkawelcomedsteptowardstheproperimplementationofthecomplementarityprinciple.118Awarenessoftheneedtoestablishcorporations’responsibilityfortheviolationof(atleastsome)internationalnormsbegantospreadinthesecondhalfofthetwentiethcentury.ThedevelopmentofIHRLconstitutedaboostingfactorinthissense.Manyinternationalorganizations,suchastheOrganisationforEconomicCo-operationandDevelopment(OECD)andtheInternationalLabourOrganization(ILO),havearticulatedcorporations’humanrightsresponsibilities.WithintheUNframework,severalattemptshavebeenmadetocreateaframeworkofhumanrightsobligationsandthusresponsibilitiesforbusinesses,suchastheUNDraftCodeofConductonTransnationalCorporations,elaboratedinthe1970sand1980sbutneverturnedintoalegallybindingdocument.119Thesubsequent2003DraftNorms,theappointmentofJohnRuggieasSpecialRepresentativeoftheSecretary-Generalonthistopicin2005,the‘Framework’submittedbyhimin2008andthentheunprece-dentedadoptionofthe‘GuidingPrinciples’in2011bytheHumanRights115IMTStatute,art10.116Batesmith(n112)290–91;Schabas(n30)224–5(commentingontheexclusionofcorporatecriminalresponsibilityfromthedraftart25(1)ICCStforreasonsoftime).117See,amongmany,ProsecutorvMusemaICTR-96-13-T(27January2000)(foracaseinvolvingateafactoryinthecommissionofinternationalcrimes).118KHaigh,‘ExtendingtheInternationalCriminalCourt’sJurisdictiontoCorporations:OvercomingComplementaryConcerns’(2008)14AustralianJofHumanRights199.119UNDraftInternationalCodeofConductforTransnationalCorporations(1984)23ILM626.SeemoreextensivelyKPSauvant,‘TheNegotiationsoftheUnitedNationsCodeofConductonTransnationalCorporations.ExperienceandLessonsLearned’(2015)16JofWorldInvestment&Trade11.DiegoMauri-9781802207675DownloadedfromPubFactoryat05/28/202203:48:45AMviaDukeUniversityLawLibrary

211Autonomousweaponssystemsandthe‘accountabilitygap’conundrum203CouncilallmarkedimportantstepsinthenewlycreatedfieldofB&HRs.120Thoseprinciplesrevolvearoundthree‘pillars’,namely,states’dutytoprotecthumanrights,corporations’responsibilitytorespecthumanrightswhilecon-ductingtheirbusiness,andaccesstoremedyforvictims.In2014,theHumanRightsCouncilcreatedtheOpen-EndedIntergovernmentalWorkingGroup(OEIWG)mandatedto‘elaborateaninternationallylegallybindinginstrumenttoregulate…theactivitiesoftransnationalcorporationsandotherbusinessenterprises’.121Severalstates—homeofmanymultinationals,suchastheUSandmembersoftheEU,aswellastheEUitself—deniedtheirsupporttotheOEIWG,whichwasabletoelaborateafirsttreatydraftin2018(the‘ZeroDraft’),122a‘RevisedDraft’in2019123anda‘SecondRevisedDraft’in2020.124Whileathoroughanalysisoftheproposedtreatyfallsinevitablyoutsidethescopeofthissubsection,someofitskeyfeaturesarerelevantforourpur-poses.Forexample,businessesinvolvedinthedevelopmentofAWSwillbeincluded,pursuanttothedefinitionsprovidedbyarticle1.125Althoughthosebusinesseswillbe‘defensecontractors’,thiswillnotshieldthemfromtheobli-gationsenshrinedintheinstrument.Inrespectofthis,thedrafttreatyimposesadutyonstatesexclusively,whichincluderegulatingbusinessesoperatingontheirterritoryorwithintheirjurisdictionforcompliancewithhumanrightsaswellasprovidingappropriatecivilandcriminalliabilityfornaturalandlegalpersons:thatis,businesseswouldnotbeaddresseddirectlybytreatyobliga-tions,norcouldtheirresponsibilitybeinvokedbyvictimsbeforeinternationalbodiesestablishedforthatpurpose.126Thus,again,itisstatesthatarerequiredtoconductduediligenceinthefieldofhumanrightsinordertopreventvio-lationscommittedbycorporations.127Moreover,itisincumbentuponstatestoprovidevictimswithadequateremedies,toensurethattheirdomesticlegal120OHCHR,‘GuidingPrinciplesonBusinessandHumanRights:ImplementingtheUnitedNations‘Protect,RespectandRemedy’Framework’(2011)HR/PUB/11/04(GuidingPrinciplesonB&HRs);Batesmith(n112)300ff.121HumanRightsCouncil,res26/9(14July2014)A/HRC/RES/26/9.122UNOEIWGChairmanship,‘LegallyBindingInstrumenttoRegulate,inInternationalHumanRightsLaw,theActivitiesofTransnationalCorporationsandOtherBusinessEnterprises’(16July2018).123UNOEIWGChairmanship,‘RevisedDraft,LegallyBindingInstrumenttoRegulate,inInternationalHumanRightsLaw,theActivitiesofTransnationalCorporationsandOtherBusinessEnterprises’(16July2019).124UNOEIWGChairmanship,‘SecondRevisedDraft,LegallyBindingInstrumenttoRegulate,inInternationalHumanRightsLaw,theActivitiesofTransnationalCorporationsandOtherBusinessEnterprises’(6August2020)(SecondRevisedDraft).125Ibidart1(3)–(4).126Ibidarts3–14.127Ibidart6.DiegoMauri-9781802207675DownloadedfromPubFactoryat05/28/202203:48:45AMviaDukeUniversityLawLibrary

212204Autonomousweaponssystemsandtheprotectionofthehumanpersonsystemsareequippedtoestablishthelegalliabilityofbusinessesandnaturalpersons:tothisend,thedrafttreatyincludesalsonormsofprivateinternationallawregulatingjurisdiction,applicablelawandjudicialcooperation.Althoughasummary,thistouracrossthemainprovisionsofthedrafttreatyconfirmsthecontinuingdomesticdimensionofcorporateresponsibility.MovingfromthegenerallevelofB&HRstothespecificfieldofweapons,nothingchangessubstantially.Whendesigning,manufacturing,selling,trans-ferringand,eventually,evenusing(asisthecaseforPMSCs)128weapons,theactivityofthosecorporationsissignificantlyimpactedbyhumanrightsstand-ards.Manytreaties,forinstance,containobligationstoexpresslyestablishtheliabilityoflegalpersonsforactsamountingtoconductsprohibitedbythosetreaties.129Similarly,soft-lawinstrumentshaveaffirmedthatbusinessesareboundnotonlybyIHRL,butalsoIHLwhentheiractivitiesarecloselylinkedtoarmedconflict:thisisthecaseofa2006documentpublishedbytheICRCanddedicatedtothetopic,130towhichthe2010InternationalCodeofConductforPrivateSecurityServiceProvidershastobeadded.131WithspecificregardtotheAIsector,therehavebeendevelopmentsofnote.Forinstance,the2018TorontoDeclaration—promotedbyAccessNowandAmnestyInternationalasaninstrumentofself-regulationbyprivateactors—establishesthatcorpora-tionsinvolvedinthedevelopmentofAI-relatedtechnologiesshouldrespecthumanrightsatalltimes,byidentifyingpotentialdiscriminatoryoutcomes,takingactiontopreventandmitigateviolations,andensuringtransparency.132OtherdocumentsthathavebeensettingthestageontheimportanceofthehumanrightsdimensionindevelopingAIsystemshavebeendraftedbytheInstituteofElectricalandElectronicsEngineers(IEEE).133128ICRCandSwissFederalDepartmentofForeignAffairs,TheMontreuxDocumentonPertinentInternationalLegalObligationsandGoodPracticesforStatesRelatedtoOperationsofPrivateMilitaryandSecurityCompaniesduringArmedConflict(August2009).129UnitedNationsConventionAgainstTransnationalOrganizedCrime(adopted15November2000,enteredintoforce29September2003)2225UNTS209,art10.SeealsoSteinhardt(n113).130ICRC,BusinessandInternationalHumanitarianLaw(30November2006).131InternationalCodeofConductforPrivateSecurityServiceProviders(9November2010)https://icoca.ch/the-code/accessed31August2021.Foradetailedanalysis,seeMBuzatu,TowardsanInternationalCodeofConductforPrivateSecurityServiceProviders:AViewfromInsideaMultistakeholderProcess(UbiquityPress2018).132AmnestyInternationalandAccessNow,‘TheTorontoDeclaration:ProtectingtheRighttoEqualityandNon-DiscriminationinMachineLearningSystems’(16May2018).133TheIEEEGlobalInitiativeonEthicsofAutonomousandIntelligentSystems,‘EthicallyAlignedDesign(EAD):AVisionforPrioritizingHumanWell-BeingwithAutonomousandIntelligentSystems’(2019).DiegoMauri-9781802207675DownloadedfromPubFactoryat05/28/202203:48:45AMviaDukeUniversityLawLibrary

213Autonomousweaponssystemsandthe‘accountabilitygap’conundrum205Inconclusion,theresponsibilityofcorporationsinvolvedintheweaponrysectorandintheAIsectorformisdoingscommittedthroughtheirproductisprogressivelymakingitswayintheinternationallegalsystem.134Whilecurrentlynoaccountabilitymechanismsexistattheinternationallevel,whichhastobeviewedasaconsequenceofthecurrentstate-of-the-artofthatlegalsystem,afixedpointemergingfromallthehard-andsoft-lawinstrumentsmentionedthusfaristhatthefirstforumwherevictimsmayseekredressremaindomesticlegalsystems.5.4.2TheMainObstaclestoCorporateResponsibilityinDomesticLegalSystems:APrécisThe(current)absenceofinternationalmechanismsofcorporateresponsibilityinevitablyimpliesfragmentation:allegedvictimsneedtoseekredressbeforedomesticcourts,andtheyusuallyencounternumerousobstaclesonthatpath.Asathoroughanalysisofeachandeveryobstacleinalllegalsystemsoftheworldisimpossible,someexamplesareprovidedthatarehelpfultounderstandwhereinternationallawmaycontributetoadjustingdomesticlegalsystems’mainshortcomingandthusensuringproperredresstovictims.Thoseexamplesare:(1)jurisdictionalbars;(2)formsofjudicialrestraintsvis-à-visactivitiescarriedoutintheexerciseofpublicfunctions;and(3)theprevailingmodelofproductliabilityformisdoingscarriedoutbyrobots.Thefirsthurdlethatvictimsseekingredresswouldencounterintakingcor-porationstocourtisrelatedtowheretosuethem.Theabsenceofmechanismsofuniversalciviljurisdictionconstitutesapreliminaryobstacletoaccesscourts,whichhasbeencorrectlypointedoutwhenhumanrightsviolationsareatstakeandtheneedtoprovidevictimswithadequateremediesisparticularlypressing.Itfollowsthatvictimswhointendtolitigatebusinesses’conductwillhavetofaceamultiplicityofdomesticrules(andasperities)onadjudicatoryjurisdiction.135TheUSlegalorderprovidesatellingexampletothisend.Thewell-knownAlienTortStatute(ATS)providesUScourtswithjurisdictiontoreview‘anycivilactionbyanalienforatortonly,committedinviolationof134FormoreontheAIsector,seeECarpanelli,‘TheRoleofCorporationsasStandardSetters:TheCaseofBusinessActorsInvolvedintheDevelopmentandDeploymentofArtificialIntelligenceTools’,inMBuscemi,NLazzerini,LMagiandDRusso(eds),LegalSourcesinBusinessandHumanRights.EvolvingDynamicsinInternationalandEuropeanLaw(Brill2020).135LRoordaandCRyngaert,‘PublicInternationalLawConstraintsontheExerciseofAdjudicatoryJurisdictioninCivilMatters’,inSForlatiandPFranzina(eds),UniversalCivilJurisdiction.WhichWayForward?(Brill2020).DiegoMauri-9781802207675DownloadedfromPubFactoryat05/28/202203:48:45AMviaDukeUniversityLawLibrary

214206AutonomousweaponssystemsandtheprotectionofthehumanpersonthelawofnationsoratreatyoftheUnitedStates’.136Laidquiescentforalmosttwocenturies,theATSwasrediscoveredaroundthe1970sandappliedtoholdindividualsaccountableforhumanrightsviolationscommittedabroad.137However,inmorerecentdecadesthescopeoftheATShasbeenrepeatedlynarroweddownbydomesticjudgments(forinstance,aslongasitsextraterrito-rialapplicabilityisconcerned),tothepointthatitisdoubtedwhetherdomesticcorporationscouldbeeffectivelysuedintheUSfortortscommittedbyitscontactsabroad.138Thus,ifUS-basedcorporationsproducedandsoldAWSabroad,andviasubsidiaries,andviolations‘ofthelawofnations’occurredasaresultoftheemploymentofAWS,USjurisdictionwouldnotbetriggeredsostraightforwardly.AgreatdealwoulddependonwhethertheUScorporation’sconductathandhasits‘focus’inUSterritory.139LookingnowattheothersideoftheAtlanticOcean,EUmemberstateshaveadopteduniformcriteriaforregulatingjurisdictionincivilmatters(includingtorts)inRegulation1215/2012(theRegulation‘BrusselsI-bis’):whilenormsdealingspecificallywithhumanrightsviolationsaremissing,generalandspecificprovisionsonjurisdictionareinplacetoensurethatpersonsdomiciledinamemberstate(inourexample,corporationsinvolvedinthedevelopmentofAWS)canbesuedinthecourtsofthatorothermemberstates.140JurisdictionofEUmemberstates’domesticcourtsincasesofAWS-relatedtortsisthussubjecttoalreadyexistingrules(andlimitsthereof).141Secondly,admittingthatthejurisdictionalbarriersarecrossed,holdingcor-porationsaccountableformisdoingscausedbyAWSrisksbeingalosingbattleinthoselegalsystemswherespecificdefensesapplytoprivatecontractorsthatmanufacturegoodsandequipmentforthestate.TheUSSupremeCourtwasabletocraftadefense—commonlyreferredtoasthe‘government-contractordefense’—wellbeforethecurrenttrendof‘outsourcing’militaryfunctionsto13628USCpara1350.137FilártigavPeña-Irala,630F.2d876(2ndCir.1980).138SosavAlvarez-Machain,542US692(2004);KiobelvRoyalDutchPetroleumCo,596US108(2013);JesnervArabBankPLC,138SCt1386(2018);NestleUSA,IncvDoe,No19—416(SCt,17June2021).139RJRNabisco,IncvEuropeanCmty,136SCt2090,2101(2016).SeeDCSong,‘U.S.CorporateLiabilityUndertheAlienTortStatuteAfterJesnerv.ArabBank,PLC’(2018)21OregonRevofInt’lL1(differentiatingATSlawsuitsagainstUScorpo-rationsfromthoseagainstforeigncorporations,whicharearguedtoencountermoreobstacles).140Regulation(EU)No1215/2012oftheEuropeanParliamentandoftheCouncilof12December2012onjurisdictionandtherecognitionandenforcementofjudgmentsincivilandcommercialmatters[2012]OJL351/1,arts4,7par2.141FormoreontheEuropeandimensionofB&HRs,seeABonfanti(ed),BusinessandHumanRightsinEurope.InternationalLawChallenges(Routledge2019).DiegoMauri-9781802207675DownloadedfromPubFactoryat05/28/202203:48:45AMviaDukeUniversityLawLibrary

215Autonomousweaponssystemsandthe‘accountabilitygap’conundrum207privateentities.142InBoylevUnitedTechnologiesCorp,theestateofamarinewhohaddiedinahelicoptercrashsuedthemanufacturerarguingthattheaccidenthadbeentheresultofproductdefects:theSupremeCourtexpandedthediscretionary-functionexceptiontotheFederalTortClaimsAct(FTCA)—allowingsuitsagainstthegovernmentforitsemployees’torts—tobusinessesthatoperateascontractorswiththegovernment.143Morespecifically,theCourtadoptedathree-prongtesttoapplythedefensetobusinesses:(1)theexistenceof‘precisespecifications’approvedbythegovernmentandsubmittedtotheprivatecontractor;(2)thecircumstancethatthefinalproductconformedtothosespecifications;and(3)thefactthatthecontractor‘warned’thegovern-mentaboutthedangersassociatedwiththeuseofthatproduct.144AnotherexceptionthatwouldshieldbusinessesfromresponsibilitybeforeUScourtsisthe‘combatant-activities’exceptionasregulatedbytheFTCA.145Relevantcaselawsprungfromtargetingaccidentsonthebattlefield,withvictimssuingmanufacturersforthemalfunctioningofweaponsorweaponssystems,asintheKoohicasementionedpreviously.146Inthosecases,UScourtsaffirmedthat‘nodutyofreasonablecareisowedtothoseagainstwhomforceisdirectedasaresultofauthorizedmilitaryaction’.147AsforEUmemberstates,gov-ernmentalexceptionssuchasthoseillustratedthusfarareregulatedmainlybydomesticlaw:theEURegulationonthelawapplicabletonon-contractualobligations(theRegulation‘Rome-II’)doesnotapplytostates’liabilityforactsoromissionsintheexerciseoftheirpublicauthority(actaiureimperii).148Finally,someremarksaredueconcerningproductliability.Admittedly,ourprevioussetofcasescouldbetackledfromthisperspective.Liabilityfordamagecausedbydefectiveproductsrepresentsoneofthemostcommonmodelsofstrictliability,whichdevelopedimmediatelyaftertheindus-trializationprocessforthepurposeofensuringtheproperdistributionofcostsconnectedwithinherentlydangerousactivities.149Productliabilityiscurrentlyregulatedmainlybydomesticorsupranationallaw,theUSandtheEUtraditionallystandingasthetwomajorjuncturesfromacomparative142AFinkelman,‘SuingtheHiredGuns:AnAnalysisofTwoFederalDefensestoTortLawsuitsagainstMilitaryContractors’(2009)34BrooklynJIL395.143487U.S.500,502–3(1988).144Ibid512.14528USCpara2680(j).146Koohi(n109).SeealsoBentzlinvHughesAircraftCo,833FSupp1486(CDCal1993).147Koohi(n109)1337.Formoreonthesedefenses,seeFinkelman(n142)412ff.148Regulation(EC)864/2007oftheEuropeanParliamentandoftheCouncilof11July2007onthelawapplicabletonon-contractualobligations[2007]OJL199/40,art1.149Monterossi(n17).DiegoMauri-9781802207675DownloadedfromPubFactoryat05/28/202203:48:45AMviaDukeUniversityLawLibrary

216208Autonomousweaponssystemsandtheprotectionofthehumanpersonperspective.150Inparticular,EUDirective85/374/EEC(hereaftertheProductLiabilityDirective)establishesthatthemanufacturerofa‘defective’productisliablefordamagescausedbythatproduct,unlessheorsheisabletoprovetheabsenceofanycausalconnectionbetweenthedamageandthedefect.151Inparticular,themanufacturercanescaperesponsibilityifproofisgiventhat‘thestateofscientificandtechnicalknowledgeatthetime…wasnotsuchastoenabletheexistenceofthedefecttobediscovered’.152Inparallel,theUSsystemisfamiliarwiththe‘developmentrisk’defense,whichstraddlesthelinebetweenastrict-liabilitymodelandamodelforextremelyculpableconduct.153Thegististhat,ifexistingmodelsofproductresponsibilityareappliedtelsquelstoAWSmisdoings(or,moregenerally,toAIsystems),producersmaylimitthemselvestorelyingonthestate-of-the-artexistingatthemomentwhentheyputAWSintocirculationorinvokingthe‘developmentrisk’defense.Atthispoint,onemaybetemptedtoclaimvictory:sincemachinelearningtechniquesrendertheinteractionsbetweenanautonomoussystemandtheenvironmentunpredictableexprofesso,itwouldbehardforbusinessestoplaythe‘developmentrisks’card.However,itiseasytoreplythat,oncloserinspection,sinceanAWStakesanunpredictablecourseofactionandinteractswithitssurroundingsinanunanticipatedway,thiscorrespondsexactlytowhatthosesystemsareexpectedtodo.154Thatis,thosecasesamounttoour‘falsepositive’scenarios,tothepointthatthemisdoingsofAWScouldhardlyberegardedas‘defects’forthepurposesofproductliability.Again,apossibleaccountabilitygapopensup,tofillwhich,extensivescholarshiphasbeenproducedsofar.Suggestionshavebeenadvancedastohowtoadaptexistingproductliabilitytothosescenarios,rangingfromrecog-nizinglimitedsubjectivitytoAIsystems(ortohuman–machineinteractions)tobroadeningthescopeofnon-faultbasedliability,155tothequestioningofthosesystems’statusof‘product’andtheirunexpectedbehavior’sstatusof‘defects’.156Tothisend,theEUoffersatellingexampleofhowlawisprogressinginthisfield.ProposalshavebeenmadeforrevisitingtheProductLiabilityDirectiveinordertobringituptodatewithtechnologicaladvancesin150MSShapo,‘ComparingProductsLiability:ConceptsinEuropeanandAmericanLaw’(1993)26CornellInt’lLJ279.151CouncilDirective85/374/EECof25July1985ontheapproximationofthelaws,regulationsandadministrativeprovisionsoftheMemberStatesconcerningliabilityfordefectiveproducts[1985]OJL210/29.152Ibidart7lit(e).153Shapo(n150)303–5.154Beck(n11);seealsoAmorosoandGiordano(n3)227–8.155Monterossi(n17).156Beck(n11).DiegoMauri-9781802207675DownloadedfromPubFactoryat05/28/202203:48:45AMviaDukeUniversityLawLibrary

217Autonomousweaponssystemsandthe‘accountabilitygap’conundrum209thefieldofAI,inparticularbyadoptingstrictliabilitymodelsandenhancingthesafetyofproducts.157In2020,theEUParliamentadoptedaresolutionontheimportanceofensuringaworkingcivilliabilityregimeforAIsystems,andmaderecommendationstotheEUCommissionfordrawinguparegu-lationonthematter—firstandforemost,forthe‘operators’ofAIsystems.158Importantly,theresolutionsupportedalsotheintroductionofadjustmentstotheProductLiabilityDirective(towhichtheproposedActwouldremaincomplementary)andtheestablishmentofastrictliabilityregimethatpre-ventedoperatorsof‘high-riskAI-systems’frominvoking‘duediligence’oran‘autonomousactivity’bythemachineasagroundforexcludingliability.159Themostrecentproposalofregulation—ambitiouslynamed‘RegulationonaEuropeanApproachforArtificialIntelligence’—marksseveralstepsforwardbutfallsoutsidethescopeofourresearch,asitsarticle2(4)excludes‘AIsystemsexclusivelyusedfortheoperationofweaponsorothermilitarypurposes’fromthescopeoftheAct.160Liningupthepointstoucheduponsofar,toholdbusinessesaccountableforviolationsofinternationallaw—andhumanrightsinparticular—beforedomesticcourtsisfraughtwithmanyobstacles.Thissituationmayberegardedastheconsequenceoflimitationsinherentinthosesystemsandlackofcoor-dinationbetweenthem(especiallywhenthebusinessthatplaintiffsintendtosueislocatedfarfromtheterritorywheredamagehasoccurred,orwhenitisaprivatecontractorofstateagencies).TheissueismagnifiedbytheadventofAI-relatedtechnologies,bothgenerallyand,morespecifically,inthefieldofAWS.Theconcernsthatthosetechnologiesraisemaypushlawmakerstoseekforandelaboratesolutions,asistakingplaceintheEUsystem:revisit-ingexistingmodelsofresponsibility—forinstance,byreplacingfault-basedresponsibilitywithno-faultresponsibility—maybeasolution.Also,theEUParliamenthasrecommendedtheadoptionofabindinginstrument(Directive)dedicatedtocorporateaccountabilityandduediligenceobligationsinthefield157EuropeanCommission,ReportfromtheCommissiontotheEuropeanParliament,theCouncilandtheEuropeanEconomicandSocialCommittee.ReportonthesafetyandliabilityimplicationsofArtificialIntelligence,theInternetofThingsandRobotics(19February2020)COM(2020)64final.158EuropeanParliamentresolutionof20October2020withrecommendationstotheCommissiononacivilliabilityregimeforartificialintelligence(2020/2014(INL))P9_TA(2020)0276.159IbidAnnexB,art4(3).160EuropeanCommission,RegulationonaEuropeanApproachtoArtificialIntelligence(21April2021)COM/2021/206final.DiegoMauri-9781802207675DownloadedfromPubFactoryat05/28/202203:48:45AMviaDukeUniversityLawLibrary

218210Autonomousweaponssystemsandtheprotectionofthehumanpersonofhumanrights—includingscenariosrelevantforourpurposes—andtheenvironmentandgoodgovernance.161Insummary,domestic(orsupranational)legislationcouldbepassedtoadjustexistinglimitationsandofferamoreefficientnormativeframeworkfordealingwithAIsystems.However,theriskoffragmentedsolutionsismaterial:thisiswhyaninternationallegalinstrumentregulatingthismatter—byprovidingcommoncriteriaforacomprehensiveandadequatesystemofcorporateresponsibility,adjudicativejurisdiction,applicablelawandmutualassistance,tonameonlysome—wouldsurelybeawelcomedstepinthedirec-tionofaccountability.5.4.3OvercomingChallenges:EnhancingB&HRsInstrumentsandRevisitingExistingLawThestartingpointofourinquiryinthischapterwastotesttheextenttowhichhumancontrolhastobeensuredoverAWSinordertoavoidaccount-abilitygaps.ThataparticularmisdoingcouldnotbetracedbacktooneormoreindividualsraisedthesuspicionofwhetherthefutureuseofAWSinarmed-conflictandlaw-enforcementscenarioswillresultinalackofaccount-abilityonthepartofakeyactorinvolvedinthedevelopmentofthosesystems,namely,businesses.Ouranalysishasledtothefollowingconclusions.First,corporaterespon-sibilityininternationallawisapressingissue,asdemonstratedbythemomentousdevelopmentoftheB&HRsmovementinrecentyears.Second,theabsenceofuniversalmechanismsforadjudicatingcorporateresponsibil-ityoratleastcoordinatingdomesticlegalsystems(inprivateinternationallaw-relatedmatters)representsasignificantdrawback,especiallyasfarasvictims’accesstoeffectiveremediesisconcerned:solutionshavetobefoundindomesticlegalorders,andtheseareinevitablyfragmented.162Limitationsoftheselegalsystemshavebeenexposedextensivelybyscholarship:thepaththatleadstoasatisfyingsetofremediesforhumanrightsabusesisthusfraughtwithobstaclesthatneedtobeproperlyfixed.163161EuropeanParliamentresolutionof10March2021withrecommendationstotheCommissiononcorporateduediligenceandcorporateaccountability(2020/2129(INL))P9_TA(2021)0073.162Seegenerally,foracomparativeperspective,CKessedjianandHCantùRivera(eds),PrivateInternationalLawAspectsofCorporateSocialResponsibility(Springer2020).163FVos,‘EuropeanTrendsinTortLawRemediestoAddressCorporateHumanRightsAbuses’,inABonfanti(ed),BusinessandHumanRightsinEurope.InternationalLawChallenges(Routledge2019).DiegoMauri-9781802207675DownloadedfromPubFactoryat05/28/202203:48:45AMviaDukeUniversityLawLibrary

219Autonomousweaponssystemsandthe‘accountabilitygap’conundrum211Ifthusanaccountabilitygapseemstoopenup,oncloserinspectionthisisowingmoretothecurrentstate-of-the-artofinternationallaw(anddomesticlegalsystems)ratherthaninherentfeaturesofAWS.AstheKoohiaccidentdemonstrates,structurallimitsexistinbothlegalorders.Thesehavetobecorrectedirrespectiveofourdiscourseonautonomy:incidentally,thiscouldbeanaptmomenttodoso.However,ultimately,thedegreeofhumancontrolovercriticalfunctionshaslittleornobearingonallthoseissues.5.5STATERESPONSIBILITY:FOCUSINGONTHELEGALENTITIESDEPLOYINGAWSTheinadequacyofindividualcriminalresponsibilityformostofthemisdo-ingsofAWSthatwilloccurinpractice,coupledwithdifficultiesassociatedwithcorporateresponsibilityundercurrentinternationallaw,haveledsomecommentatorstozoominonstateresponsibility,thatistheresponsibilityoftheentitythatwilldeployandoperateAWS.164Tothewriter’sbestknowl-edge,despiteitstraditionalcentralityininternationallaw,thisistheleastexplored—yetpossiblythemostpromising—prongofinternationalresponsi-bilityvis-à-visAWS.Relyingonstateresponsibilityisdesirableforatleastthreereasons.First,stateresponsibilityandotherformsofinternationalresponsibilityarenotmutu-allyexclusive:forinstance,incaseswhereindividualcriminalresponsibilityarises,stateresponsibilitycouldeasilyfollow.Bysodoing,stateresponsibilitycouldbehelpfulinthatitmayfillpossiblegapsresultingfromtheapplicationofotherformsofresponsibility.Second,itmayoffervictimsgreaterchancesofobtainingadequateredress,consideringthatstateshave‘deeperpockets’incomparisonwithbusinesses—letaloneindividuals.165Third,thismodelofresponsibilitywillpushstates(thatis,themainusersofAWS,atleastinthenearfuture)towardsensuringthattheweaponstheydevelopandthenusecomplywithIHRLandIHL.166Thatis,statesknowingthatlackofduedili-genceontheirpartwillresultintheirresponsibilityattheinternationallevelmaybeinclinedtoinvestmoreinprevention.Stateresponsibilitycouldthen164HeynsReport(n1)para81(‘[i]ngeneral,astrongeremphasisonStateasopposedtoindividualresponsibilitymaybecalledfor’);McFarlandandMcCormack(n76)385(lamentinglackoffocusedscholarship).165Crootof(n10)1390.166DNHammond,‘AutonomousWeaponsandtheProblemofStateAccountability’(2015)15ChicagoJIL652,669(‘[s]houldstateschoosetouseAWSsinspiteoftheserisks,liabilitywouldgivethemareasonatthepurchaseanddeploymentstagestoensurethattheirAWSswillcomplywithinternationallaw’).DiegoMauri-9781802207675DownloadedfromPubFactoryat05/28/202203:48:45AMviaDukeUniversityLawLibrary

220212Autonomousweaponssystemsandtheprotectionofthehumanpersonbeadriverforensuringbettercompliancewithprimaryobligationsconcerningtheprotectionofthehumanperson.Inlinewiththegeneralquestionsdrivingthischapter,whatmattershereiswhetherexistinglawofstateresponsibilityiswellequippedfordealingwiththemisdoingsofAWS.Inparticular,itisinterestingtotestthedegreetowhichhumancontroloverAWSisrequestedbythelawofstateresponsibilityinordertoavoidanygaps.ThisbranchofinternationallawhasbeenlargelycodifiedbytheDraftArticlesonResponsibilityofStatesforInternationallyWrongfulActs,elaboratedbytheILCandadoptedin2001.167Stateresponsi-bilitystemsfromthecommissionofan‘internationallywrongfulact’that(1)isattributabletoastate(s5.5.1)and(2)constitutesabreachofaninternationalobligationofthestate(5.5.2).168TheARSIWAdonottakefault(3)into(direct)consideration,apointwhichiscriticalintheassessmentoftheimpli-cationsofAWSonthelawofinternationalresponsibility:tothisend,thelimitsofexistingfault-basedmodelsofstateresponsibilityfortheviolationofIHRLandIHLnormsareexposedandpossiblewaysforwardtoensuringappropriateaccountabilityareillustrated(5.5.3).5.5.1AttributionofaWrongdoingCommittedthroughAWSAttributingaviolationofIHRLorIHLnormsresultingfromtheuseofAWSdoesnotseemprimafacieproblematic.Statesactprimarilythroughtheirorgans(‘agentandrepresentatives’),169anditwillbeuptostateorgans(policeofficersandmilitarycommanders)todecideif,whenandhowtodeployAWSintheoperationalfield.Accordingtoarticle4ARSIWA,thisconducthastobeconsideredas‘anactofthatState’.170Theissueofwhethercorporateconduct(developingAWStobesoldtogovernments)canbeattributedtothestateviatheARSIWA,fallsoutsidethescopeofthepresentresearch,asitispositedthat,insuchacase,whatistobeimputedtothestateisnottheparticularmisdoingathand,buttheconductofanentityotherthanthestate(thecorpora-tion).Insummary,corefeaturesofAWSwouldnotbedecisiveforthepurposeoftheattributiontest.Inthistraditionalframeworkofattribution,theadventofAI-relatedtech-nologiesmaybringanelementofnoveltytothesituation.Advancesinthefieldofhuman–machineinteractionswillresultinhumanoperators’reduced167ILC,‘ResponsibilityofStatesforInternationallyWrongfulActswithCommentaries’(9June2001)A/CN.4/SER.A/2001/Add.1(ARSIWA).168ARSIWAart2.169GermanSettlersinPoland(1923)PCIJSeriesBNo622.170ArmedActivitiesontheTerritoryofCongo(DRCvUganda)[2005]ICJRep6para242.DiegoMauri-9781802207675DownloadedfromPubFactoryat05/28/202203:48:45AMviaDukeUniversityLawLibrary

221Autonomousweaponssystemsandthe‘accountabilitygap’conundrum213leewayininterveninginthesystem’sdecision-makingprocess.Ifaparticularcourseofactionisdeliberatedabsentanyhumanintervention,astheoperatorremainssomewhereintheloop,couldtheargumentbemadethat,technically,theensuingviolationisnotimputabletotheoperatorandthus,pursuanttothecriteriaoutlinedbytheARSIWA,attributabletothestate?Arguably,adistinctionshouldbemadebetweentheconductconsistingindeployingAWSandtheconductconsistinginselectingandengaginganimpermissibletarget:whiletheformeriscertainlyattributabletothestate,thelattermayraisesomedoubts.Thislineofargument,however,couldbeeasilyobjectedto.ThefirstobjectionliesinthatAWS—yetfeaturingadvancedsystemsforhuman–machineinteractionsandAIcapabilities—havetoberegardedas‘tools’inthehandsoftheirusers.Theyarenotentities‘other’thantheiroperatorsforthepurposesofthelawofstateresponsibilities.Secondly,evenadmittingthattheycouldbeconceivedasautonomousentities,171fromasystematicperspectiveitcanbeobservedthatexistingcriteriaforattributingresponsibilityarecapableoflinkingtheimpugnedconductwiththestatethathasdecidedtodeployAWS.Autonomousweaponssystemswouldberegardedas‘organs’ofthatstate,exercisingexecutivepowersparexcellence—namely,enforcinglawandconductinghostilities.Onagenerallevel,article4ARSIWAremainsapplicable.AsfortheattributionofthemisdoingsofAWSwhichthehumanoperatordidnothaveachancetohaltorprevent,thesewouldqualifyasultraviresactspursuanttoarticle7ARSIWA,asthose‘organs’(AWS)acted,inagivenscenario,‘contrarytoinstructions’receivedbythehumanoperatorbeforedeployment.TherewouldbenoobstacleinfindingthatAWSpurportedtoactonbehalfofthestate,yetexceedingtheprerogativestheyhadbeeninfusedinto,whichwouldmaketheimpugnedconductattributabletothedeployingstate.172ThisconclusionisalsowarrantedbythespecialcriteriaoperatingforattributingresponsibilityforviolationsofIHL,accord-ingtowhichthoseviolationsperpetratedinthecontextofarmedconflictbymembersofthearmedforcesofastate—towhichAWSwouldbeequatedhypothetically—inallcircumstancesengagetheresponsibilityofthatstate.173Subsequently,inourviewtherewouldbenoneedtoresorttootherrulesofattributionsuchasthoseondefactoorgans,entitiesexercisinggovernmental171OnpossibleanalogiesforappraisingAWS,seeRCrootof,‘AutonomousWeaponsSystemsandtheLimitsofAnalogy’(2018)9HarvardNat’lSecJ52.172JCrawford,StateResponsibility.TheGeneralPart(CUP2013),136ff.173ConventionIVRespectingtheLawsandCustomsofWaronLand(adopted18October1907)539UNTS639,art3;API,art91;ARSIWA,Commentary,art7para4;LCondorelliandCKress,‘TheRulesofAttribution:GeneralConsiderations’,inJCrawford,APelletandSOlleson(eds),TheLawofInternationalResponsibility(OUP,2010)227–8.DiegoMauri-9781802207675DownloadedfromPubFactoryat05/28/202203:48:45AMviaDukeUniversityLawLibrary

222214Autonomousweaponssystemsandtheprotectionofthehumanpersonauthority,andthoseplacedatthedisposalofanotherstate.174EvendrawingananalogybetweenAWSandprivatepersonsactingundertheinstruction,direc-tionorcontrolbythestatepursuanttoarticle8ARSIWAseemsanimaginativevirtuositythatwouldaddlittletoouranalysis.Instead,itmaybequestionedwhethertheconductofnon-stategroupsthatsucceedinhijackingAWSandcontrolthemmaybeattributedtothestate.Thisbegsthequestionofwhichmodelof‘control’shouldbeadoptedwhenattributingconductsofprivateindividualstothestate(the‘effectivecontrol’testorthe‘overallcontrol’test),whicharisesirrespectiveoftheweapon’scapabilities.Inourview,also,AWSendowedwithultra-developedAItechniquesmakingthemresemblehumanintelligencewouldstillremain‘tools’inthehandsoftheir‘masters’.Theywouldnotbeindependentactorsintheinternationallegalsystem:humansprogramandusethem.Theissue,thus,iswhethertheactperformedbytheautonomoussystemscanbeimputedtotheindividual(s)usingit:ifthis(orthese)arestateorgans,orentitieswhoseconductcanbeattributedtothestateaccordingtotherulesasweknowthem,themisdoingsofAWSwillbeattrib-utabletothatstate.Forthesakeofcompleteness,however,someremarkscouldbemadewithregardtoadifferent,yettopical,issuethathasemergedduringthedebateattheCCW,namely,thevulnerabilityofAWStointerference,spoofingorhackingbyotherstatesornon-stateactors.175GuidingPrinciple(f)referstotheneedtoadoptappropriatephysicalandnon-physicalsafeguardsagainsttheserisks,aswellasagainstthemoregeneralriskofacquisitionandemploymentby‘terroristgroups’.176Thisrisk,whichintersectswithquestionsrelatedtojusadbellum,hasbeenexploredindetailbyscholars:itisheldthatthecircumstancethatthosesystemsarevulnerabletomaliciousactionsshouldpushstatestoadoptallreasonableprecautionarymeasurestopreventthatscenario,totheextentthat,ifappropriatemeasurescannotbetakentoneutralizetherisk,statesshouldrefrainfromfieldingAWS.177Thisargumentsoundsentirelyconvincing.Yet,thisreferencetothedutytopreventmaliciousexploitationsofthevulnerabilitiesofAWSshiftsthefocusfromthesphereofresponsibilitytothatofprimaryobligations,namely,obligationsofduediligence.178Given174ARSIWA,arts4(2),5,6.175‘Reportofthe2019sessionoftheGGE’para25(a).176GuidingPrinciples2019,principle(f).177SchmittandThurnher(n6)242;ALeveringhausandGGiacca,‘Robo-Wars.TheRegulationofRoboticWeapons’(OxfordMartinPolicyPaper2014),19.178Crawford(n172)158ff;MSassoli,‘StateResponsibilityforViolationsofInternationalHumanitarianLaw’(2002)84IRRC401,411–12(arguingonthebasisofcommonart1totheGenevaConventions).Moregenerally,seeRPisilloMazzeschi,“DueDiligence”eresponsabilitàinternazionaledegliStati(Giuffrè1989).DiegoMauri-9781802207675DownloadedfromPubFactoryat05/28/202203:48:45AMviaDukeUniversityLawLibrary

223Autonomousweaponssystemsandthe‘accountabilitygap’conundrum215thecharacterof‘obligationofconduct’thatthissetofobligationstraditionallyfeatures,itcouldthusbearguedthat,ifastatefailstotakeallmeasurestopreventtheacquisitionofAWSbyotherstatesornon-stategroupsthatwillusethosesystemstocommitviolationsofinternationallaw,thenresponsibilitytotheformerarisesvis-à-visthatfailure.179Theconducttobeattributedwouldconsistinthebreachofduediligenceobligations,ratherthanintheactualwrongdoingcommittedthroughahijackedAWS.Recappingourargumentsthusfar,theapplicationofexistingcriteriaofattributionofawrongdoingtoastatetoascenariowheretherelevantconductconsistsinthemisdoingsofAWS,doesnotappeararduous.Tothisend,humaninterventionintheactoftakingacriticaldecisionisnotrequired;formsofhumancontrolovertheautonomoussystem‘inthewiderloop’areenoughtoattractthewrongdoinginthesphereofstateresponsibility.OntheunderstandingthatAWSwillkeeponfeaturinghuman–machineinteractionstovariousdegrees,evenconceiving‘ultra-intelligent’systemsasmoreakinto‘agents’than‘tools’—assomehavebeguntodo—180wouldnotraiseunsur-mountableobstacles.5.5.2TheBreachofanInternationalObligationbyAWSMovingontothematerialelementofinternationalwrongdoing,staterespon-sibilityarisesiftheconductattributabletothestateconstitutesabreachofaninternationalobligationincumbentuponit.181Inordertoassessthematerialelement,referencemustbemadetotheprimaryobligationsatstake—forthepurposesofouranalysis,thoseobligationspertaintoIHRLandIHL.Alongthelinesoftheanalysisconductedinthepreviouschapters,primaryobligationsincumbentuponstatescanbecharacterizedaseither‘negative’(requiringstatesnottoengageinagivenconduct:forinstance,toabstainfromarbitrarydeprivationsoflife,torefrainfromtargetingdirectlyciviliansorcivilianobjects)or‘positive’(imposingtokeepacertainbehaviorwithaviewtoafinalresult:forinstance,takeallreasonableprecautionarymeasureswhenplanningapolicingoperationinvolvingtheuseoflethalforceoramilitaryattack).Applyinganotherwell-knowndoctrinal‘filter’tothisdichotomy,theformercouldberegardedasobligations‘ofresult’,inthattheyimposeonstatesadutytoactinordertoprovide‘guaranteesofoutcome’:inourexam-ples,thenegativedutytorefrainfromtakinglifearbitrarilyisrespectedas179ApplicationoftheConventiononthePreventionandPunishmentoftheCrimeofGenocide(BosniaandHerzegovinavYugoslavia)(Merits)[2007]ICJRep43para221.180Hallevy(n18).181ARISWAarts2and12.DiegoMauri-9781802207675DownloadedfromPubFactoryat05/28/202203:48:45AMviaDukeUniversityLawLibrary

224216Autonomousweaponssystemsandtheprotectionofthehumanpersonlongasstatesdonotactcontrarytothatprohibition.Oppositely,mostpositivedutieswouldbeconceivedasobligations‘ofconduct’:theyareinthenatureof‘bestefforts’obligations,totheextentthattheyrequirestatestoadopta‘diligent’behavior,thatis,todotheirutmost,toobtainagivenresult.182Asregardsthissetofobligations,internationallegalscholarshiphascoinedthewell-knownlabelof‘duediligence’obligations.183Uponthispremise,itmaynowbewonderedonwhichconditionsthemisdo-ingsofAWSamounttobreachesofIHRLandIHLnorms.Canitbesaidthatastateviolatesaprimaryobligationincumbentuponit,whenAWSselectandengageanimpermissibletarget?Whatifthemisdoingisfoundtobecausedbyunforeseen(andunforeseeable)interactionsbetweenthesystemandtheenvironment?Atwhichconditioncanstatesshieldbehindtheargumentthattheyhavedoneanythingintheirpowertopreventaresult,whichnonethelesshappened?Asisevident,posedinthisway,thoseissuescallintoquestionconceptssuchas‘fault’and‘duediligence’onthepartofstates.Inordertoaddressthemsystematically,ouranalysisturns,first,topossiblecircumstancesprecludingwrongfulnessthatstatesmaybetemptedtoinvokeasjustificationtotheirconduct,andthentothe‘duediligence’standardmostprimaryrulesencapsulate.5.5.2.1Willapleaofforcemajeurebeeffective?IntheeventofAWS‘goingrogue’inthefield,toarguefortheexistenceofabreachofrelevantIHRLandIHLnormsdoesnotseemarduous:factsspeaksforthemselves.Yet,thisistrueonlyatfirstsight.Inthescenariosunderanal-ysis,statesthatdecidedtoemployAWSmayarguethatawrongfulevent(thatis,theparticulartargetingdecision)occurredasaresultofanunforeseeablemalfunctioningofthesystem.Theresorttoincreasingautonomouscapabilitiesintargetingwillmakeittemptingforstates‘topleadforcemajeureinordertoevadeinternationalresponsibilityforunforeseenandunforeseeable“deci-sions”takenbytheautonomoussystem’.184Thepossibilitythatstatesinvoke182Crawford(n172)221–2.183PisilloMazzeschi,“DueDiligence”eresponsabilitàinternazionaledegliStati(n178);PisilloMazzeschi,‘TheDueDiligenceRuleandtheNatureoftheInternationalResponsibilityofStates’(1992)35GermanYbIL9.Morerecently,seeSCassella(ed),Lestandarddeduediligenceetlaresponsabilitéinternationale(Pedone2018)andAOllino,DueDiligenceObligationsinInternationalLaw:ATheoreticalStudy(CUPforthcoming).184Directorate-GeneralforExternalPoliciesoftheUnion,PolicyDepartment,‘HumanRightsImplicationsoftheUsageofDronesandUnmannedRobotsinWarfare’(2013)EXPO/B/DROI/2012/12,39.SeealsoGenevaAcademyofInternationalHumanitarianLawandHumanRights,‘AutonomousWeaponSystemsunderInternationalLaw’(November2014)AcademyBriefingNo8,24.DiegoMauri-9781802207675DownloadedfromPubFactoryat05/28/202203:48:45AMviaDukeUniversityLawLibrary

225Autonomousweaponssystemsandthe‘accountabilitygap’conundrum217thiscircumstanceprecludingwrongfulness—yetremainedmarginalinthedebate—deservesattention,asitmayhaveaninfluenceonhowtounderstandthescopeofhumancontrolthatisrequiredovercriticalfunctions.Anappraisalofforcemajeureasregulatedbyexistingnormsofstateresponsibility,aswellasitsprospectiveapplicationtothemisdoingsofAWS,isthusvital.Circumstancesprecludingwrongfulnesshavetheeffectofpreventingtheengagementofresponsibilityonthepartofstates:thetopicofdefensesininternationallawhasfueledanintensedebateduringthecodificationofthematterintheframeworkoftheILC,andstillcontinuestoraise‘recurringdoctrinalchallenges’forscholars.185Ascodifiedbyarticle23ARSIWA,forcemajeureconsistsin‘anirresistibleforceor…anunforeseenevent,beyondthecontrolofthestate,makingitmateriallyimpossibleinthecircumstancestoperformtheobligation’.Thedefenseofforcemajeurereliesontheprinciplepursuanttowhichnolegalsubjectcanberequestedtodowhattheycannotreasonablydo(adimpossibilianemotenetur,amaximwealreadyencounteredabove):derivingfromRomanLaw,thisprincipleistodayrecognizedbybasicallyalllegalsystemsoftheworld,totheextentthatitcanberegardedasageneralprincipleofinternationallaw.186Forcemajeurehasundergonealongprocessofreconceptualizationduringthepasttwocenturies,culminatinginthecurrentformulationunderARSIWA.187Ittraditionallycoversbothnaturalandhuman-madeevents,andthushypotheticallyalsoamistargetingbyanautonomoussystem,reactingtotheoperationalenvironmentinanunexpectedmanner.188185ILC,‘EighthReportonStateresponsibilitybyMrRobertoAgo,SpecialRapporteur–theinternationallywrongfulactoftheState,sourceofinternationalresponsibility’(1979)A/CN.4/318;SSzurek,‘TheNotionofCircumstancesPrecludingWrongfulness’,inJCrawford,APelletandSOlleson(ed),TheLawofInternationalResponsibility(OUP2010).SeealsoTChristakis,‘Les“circonstancesexcluantl’il-licéité”:uneillusionoptique?’,inOCorten,NAngelet,EDavidandPKlein(eds),Droitdupouvoir,pouvoirdudroit:MélangesoffertsàJeanSalmon(Bruylant2007)and,morerecently,FPaddeu,JustificationandExcuseinInternationalLaw.ConceptandTheoryofGeneralDefences(CUP2018).186FPaddeu,‘AGenealogyofForceMajeureinInternationalLaw’(2011)82BritishYbIL381;ILC,‘“Forcemajeure”and“FortuitousEvent”ascircumstancesprecludingwrongfulness:SurveyofStatepractice,internationaljudicialdecisionsanddoctrine–studypreparedbytheSecretariat’(1978/II(1))ILCYb;SSzurek,‘CircumstancesPrecludingWrongfulnessintheILCArticlesonStateResponsibility:ForceMajeure’,inJCrawford,APelletandSOlleson(ed),TheLawofInternationalResponsibility(OUP2010).187Paddeu,‘AGenealogyofForceMajeure’(n186)437ff(emphasizingitscorre-spondencetocustomarylaw).188ARSIWAart23,Commentarypara3;Paddeu,‘AGenealogyofForceMajeure’(n186)394–5(underscoringthat‘itwasmade-manevents…whichplayedanimpor-DiegoMauri-9781802207675DownloadedfromPubFactoryat05/28/202203:48:45AMviaDukeUniversityLawLibrary

226218AutonomousweaponssystemsandtheprotectionofthehumanpersonUnpackingtherequirementsputforwardbyarticle23ARSIWAforforcemajeuretoqualify,threeelementsdeserveattention:(1)thestateinvokingthedefensecouldhaveno‘control’oftheevent(irresistibility),resultinginthematerialimpossibilitytocomplywiththeprimaryobligation;(2)the‘event’isnotforeseeable(unpredictability);and(3)the‘externality’ofthecircum-stance.LetusapplythoseinahypotheticalscenarioinvolvingAWS.Astotheirresistibilityrequirement,humanoperatorsnotinterveningintheperformanceofcriticaldecisions,whileremainingintheloop,istheessentialfeatureofAWS:thiswouldexcludeanypossibilityforstateorgansto‘stepin’andpreventthesystemfromdeliberatingaparticularcourseofaction.Humancontrolwillbepreservedandoperatorswillalwaysbeabletodeacti-vateAWS—yet,thismayoccurtoolate.Inthatinstance,itisthus‘materiallyimpossible’forthestatetoperformtheobligationsathand.Whilethisisundeniable,morecontroversialistherequirementoftheunpredictabilityoftheevent.IthasbeenalreadyarguedthatdeployingAWSimpliesariskofunanticipatedbehavioroftheweapon,especiallywhenemployingself-learningtechniques.Ithasbeenalsoobservedthat,lookingatthepracticeofdefensecontractors,manufacturerandproducerswillbegivenspecificdirectivesbygovernments,buttheywillalsoprobablydeclinerespon-sibilityforanymalfunctioningthattheynotifytotheircontractors.189Forthestate,itwouldthusbehardtoclaimthatthewrongfuleventwascompletelyunexpected:thatAWSarepronetoerrorsisknownexante—morethanthis,itispartofthegame.Onemayobjectthatwhathastobeunforeseeableisapar-ticularcourseofaction:thatis,thatinsuchsituation(forexample,abattlefieldpopulatedmostlybypermissibletargets)theAWSwouldattackthatparticulartarget.However,itwouldrunagainstthespiritofthisjustification—which,inbeingan‘exception’totherulesofinternationalresponsibility,hastobeinter-pretednarrowly—ifforcemajeurecouldbeinvokedascarteblancheforstatesdecidingtoengageinriskyactivities,suchasemployingAWSinthefield.Tosolvetheconundrum,attentionistobegiventoanothercoreelementofforcemajeure,namely,the‘externality’ofthecircumstance.190Thiselementisreflectedinarticle23(2),whichestablishesthatforcemajeurecannotbeinvokedif:(1)thesituationis‘due,eitheraloneorincombinationwithotherfactors,totheconductofthestate’;or(2)thestatehas‘assumedtheriskofthatsituationoccurring’.Whileonemayinterpretthedecisiontofieldunpredictable-by-definitionAWSasacausalfactortothefinalmisdoing,thetantpartinthedevelopmentofthedefenceofforcemajeure’);ILC,‘“Forcemajeure”and“FortuitousEvent”ascircumstancesprecludingwrongfulness’(n186)70–71.189Seesupra5.4.190Szurek(n186)477.DiegoMauri-9781802207675DownloadedfromPubFactoryat05/28/202203:48:45AMviaDukeUniversityLawLibrary

227Autonomousweaponssystemsandthe‘accountabilitygap’conundrum219latterrequirementseemscrucialforouranalysis.StatesdecidingtoemployAWSplacethemselvesinaquintessentialriskyarea.TheCommentary,though,suggestsanarrowunderstandingofthisnegativerequirement,asitclaimsthat‘theassumptionofriskmustbeunequivocalanddirectedtowardsthosetowhomtheobligationisowed’.191Traditionally,itisheldthatstatesassumetheserisksbymeansofmultilateralorbilateralagreements,orunilat-eraldeclarations.192ItwouldbehardtosaythatthesamegoesforAWSabsentanybindingformulationstothatend.Thisissuewillbeexploredfurtherwhendiscussingpossibledelegeferendasolutions.Inlightofallthis,forcemajeureis,atleastatfirstsight,atemptingoptionforstatestryingtoescapeinternationalresponsibilityforthemisdoingsofAWS;however,existingrulesmaybeinterpretedinordertolimitrecoursetothisdefense.Oncloserinspection,twoadditionalelementsmustbekeptinmindthatconfirmourassertion.First,article26establishesthatnocircum-stanceprecludingwrongfulnesscouldbeeffectiveinrespectofjuscogensnorms:asmostIHRLandIHLrulesandprinciplescomingtotheforecom-monlyacknowledgethisstatus,itfollowsthatstateswillbepreventedfrominvokingforcemajeureif,say,AWSareemployedincontraventionofrulesimposedtodistinguishcombatantsfromcivilians.Appealingtothenatureoftherulesathandwillsuffice.Second,currentlythiscircumstanceprecludingwrongfulnessremainsuntestedvis-à-vistechnologicalrisksasentailedbytheuseofunpredictablealgorithmsinreal-lifescenarios.Lookingatthelawasitcurrentlystands,itmustbenotedthat,ingeneral,forcemajeureisrarelyinvoked—andevenmorerarely,invokedsuccessfully.193ThesamecanbeexpectedincasesinvolvingAWS.Whilethisconclusionisconvincingwithregardtomaliciousornegligentmistargeting,itmayleaveusunsatisfiedinscenariosfeaturingtheselectionandengagementoffalsepositives.Intheseinstances,nofaultonthepartofstates—eitherinthedevelopmentphaseorinthedeploymentphase—couldbefound,totheextentthat,asamatterofprinciple,theinvocationofforcemajeuremayseemwarranted.Beforeconsideringwhichadjustmentscanbemadetothelawofstateresponsibilitytofixthisinconvenience,thenatureoftheprimaryobligationsatstakemustbeinvestigated.191ARSIWAart23,Commentarypara10(emphasisadded).192ILC‘“Forcemajeure”and“FortuitousEvent”ascircumstancesprecludingwrongfulness’(n186)31.193AmocoInternationalFinanceCorporationvTheGovernmentoftheIslamicRepublicofIran(14July1987)IUSCTCaseNo56;AutopistaConcesionadavRepublicofVenezuela(23September2003)ICSIDCaseNoARB/00/5para118;RSMProductionCorpvCentralAfricanRepublic(7December2010)ICSIDCaseNoARB/07/02paras180–85.DiegoMauri-9781802207675DownloadedfromPubFactoryat05/28/202203:48:45AMviaDukeUniversityLawLibrary

228220Autonomousweaponssystemsandtheprotectionofthehumanperson5.5.2.2Willa‘duediligence’argumentbeeffective?Inadditiontoprimaryobligationsestablishingadutytoabstainfromadoptingaparticularbehavior,thereareprimaryobligationsimposingadutytotakeallreasonablestepsforavertingagivenrisk(‘duediligence’obligations):thisisthecase,forinstance,ofIHRLandIHLrulesestablishingadutyofprecautiononthepartofstateswhenconductingoperationsinvolvingtheuseof(poten-tiallylethal)forceagainstindividuals.Thus,thesecondargumentthatstatesmaybetemptedtoadvanceforescapinginternationalresponsibilityiscon-nectedwiththenatureoftheprimaryobligationsathand.Asthoseobligationshavebeendiscussedinthepreviouschapters,welimitouranalysistonotingoutkeyaspectsthathaveadirectbearingoninternationalresponsibility.194MostIHRLandIHLobligationsthatarerelevantforouranalysisencapsu-lateaduediligencestandard.195Duediligencedutiesarepresentinallphasesoftheweapons’lifecycle,fromtheirdevelopmenttotheiremployment.Duringthedevelopmentphase,statesneedtospecifytherequirementsforfutureuse(includingaccuracyandreliabilityoftargetingmechanisms,areastobecovered,andtheweapon’sforeseeableeffectsonobjectsandindividuals)andexercisecontroloverdesignfeaturesandtechnicalorperformancecharac-teristics.196Duringthefieldingstage,statesareunderadutytochooseweaponsappropriateforaparticularoperationalscenario.AccordingtoIHRL,thisisthecaseofthedutytoplanpolicingoperationsinawaytominimizetheriskoflossesoflife:statesarerequestedtoadoptall‘feasible’measurestothatend.197Tonameoneexample,GC36repeatedlyassertsthatstatesarerequestedtotakeappropriatemeasuresagainst‘reasonablyforeseeablethreats’.198Asillustratedinthepreviouschapters,requirementsforresortingto(potentiallylethal)forceunderIHRLaremorestringentthanunderIHL.AsfarasIHLisconcerned,theprincipleofprecautionandtheensuingrulesrequirebelligerentstoadopt‘reasonable’or‘feasible’precautionswhileplanningandconductingamilitaryattack(limitingtothemainones:toverifythattheobjectivestobeattackedarepermissible;toselectmeansandmethodsthatminimizecollateraldamage;toprovidewarningsoftheattack).199Inallcases,therelevantstandardisthat194SeeCh3,s3.2.2.1andCh4,s4.3.2.195Onduediligence,seealsoInternationalLawAssociationStudyGrouponDueDiligenceinInternationalLaw,‘FirstReport’(7March2014).196ICRC,‘AGuidetotheLegalReviewofNewWeapons,MeansandMethodsofWarfare:MeasurestoImplementArticle36ofAdditionalProtocolIof1977’(Geneva2006),18.197TagayevaandOthersvRussia,AppNo26562/07andothers(13April2017)para573.SeemoreextensivelyCh3,s3.2.198GC36paras7,18,21.SeemoreextensivelyCh3,s3.2.2.1.199APIart57(2),(4);APICommentarypara2230.SeealsoMLongobardo,‘TheRelevanceoftheConceptofDueDiligenceforInternationalHumanitarianLaw’DiegoMauri-9781802207675DownloadedfromPubFactoryat05/28/202203:48:45AMviaDukeUniversityLawLibrary

229Autonomousweaponssystemsandthe‘accountabilitygap’conundrum221ofreasonableness(synonymouswithotherexpressionssuchas‘feasibility’or‘adequateness’),whichcanbedescribedasthe‘overarchingparameter’thatduediligenceobligationshavetobemeasuredagainst.200Whatisfeasible,andthusreasonable,dependsonthecircumstancesprevailingatthetime,which,incasesofarmedconflict(inlightoftheconcurrentapplicationofIHRLandIHLasarguedbefore),maysetthebarlowerthaninlaw-enforcementoperations.Whileexcludingpracticallyimpossibleprecautions(asifstatescouldhaveperfectknowledgeofallfactorsinvolved),theobligationtodoeverythingfeasiblemustremainhigh,inlightofbothhumanitarianandmili-taryconsiderations.201Itmaybewonderedwhetherallthosereferencestonotionsof‘foreseeabil-ity’,‘reasonableness’,‘reasonablemanstandard’,‘feasibility’,andsimilar,donotdescribeaparticularstateofmindonthepartofthestate,onethatcanbetermeda‘fault’.Onthis,scholarshipisdividedbetweensupporttoan‘objec-tive’standardofduediligenceanda‘subjective’standardofduediligence.202Aparticularlyconvincingviewhasbeenadvancedaccordingtowhich,whiletobeconstruedobjectively,duediligencerulesontargetingunderbothIHRLandIHLinherentlyencapsulateastandardof‘fault’,totheextentthatifatleastthismentalelementcannotbedemonstratedtoexist,thoserules‘aresimplynotviolated,withtheconsequencethattheobjectiveelementforthepurposesofStateresponsibility…isnotfulfilled’.203InthecontextofAWS,thismeansthatstatesdevelopinganddeployingthisweaponryareexpectedtoadoptallfeasibleandreasonablemeasuresinordertoavoidunwantedscenarios,thatis,theselectionandengagementofanimper-missibletarget.Asillustratedabove,theassessmentofwhetherduediligenceobligationshavebeencompliedwithdependsontheparticularcircumstancesofthecase.204Factorsthatmaycometotheforewhileassessingduediligencevis-à-visthemisdoingsofAWSare,forinstance:whetherAWShavebeenusedincompliancewiththemanufacturer’sdirectives;whetherAWSwereaptfortheoperationalscenarioathand;andwhetherfail-safemechanismswere(2020)37WisconsinJIL44,67.Formoreonrulesregardingprecautionsinattack,seeCh4,s4.3.2.200Ollino(n183)111.201Longobardo(n199)68(andpracticetherein).202MSassoli,‘AutonomousWeaponsandInternationalHumanitarianLaw:Advantages,OpenTechnicalQuestionsandLegalIssuestobeClarified’(2014)90Int’lLStud308,332–5(supportinganobjectivestandardintargeting).203DAmoroso,AutonomousWeaponsSystemsandInternationalLaw.AStudyonHuman-MachineInteractionsinEthicallyandLegallySensitiveDomains(EdizioniScientificheItaliane2020),148.204NMcDonald,‘TheRoleofDueDiligenceinInternationalLaw’(2019)68ICLQ1041.DiegoMauri-9781802207675DownloadedfromPubFactoryat05/28/202203:48:45AMviaDukeUniversityLawLibrary

230222Autonomousweaponssystemsandtheprotectionofthehumanpersoninplacethatcouldbeusedbyhumanoperatorstomanuallyoverridesystemsactingcontrarytotheirinstructions.InscenariosamountingtoanintentionaluseofAWStocommitviolationsofIHRLandIHL,orthosecharacterizedbystate’sfailuretoadoptreasonablemeasurestopreventforeseeablemistarget-ing,itwouldbehardforthatstatetoarguethattheprimaryobligationathanddoesnotmaterialize.Whilekeptdistinctforthesakeofconceptualclarity,theissuesofforcemajeure(thatis,acircumstanceprecludingwrongfulness)andofduediligence(whichpertainsmoretotheprimarynormunderscrutiny)tendtooverlapinpractice.205Mostoften,ashasbeencorrectlynotedinthesub-sectorsofinvestmentlawandenvironmentallaw,theinvocationofforcemajeuretendstoblurintotheissueofthecontentandscopeoftheprimaryobligationsofduediligence.206Therequirementofthe‘unpredictability’ofthefortuitouseventclearlyreferstotheabove-mentionedstandardof‘fault’inthestate’sbehavior.Thatis,ifastatewereinthepositionofanticipatingaparticularunlawfulcourseofactionandfailedtotakeallappropriatemeasurestoneutralizeorminimizeit—thusactingwithnegligenceandatfault—thatstatewouldnotbeabletosuccessfullyinvokeforcemajeure.207Thisconceptualaffinityisconfirmedbypractice:inancientcasesinvolvingnegligentbombing,injuredstatesconsistentlyheldthatasthebreachresultedfromanevident‘lackofproperprecaution’,forcemajeurecouldnotbeinvoked.208ThesamewouldthusapplyforfutureAWS:iftheharmfuleventmaterializesasaconsequenceofastate’sfailuretocomplywithbest-effortrequirementsimposeduponitbyIHRLandIHL,theprimarynormwillbeapplicableandnoforcemajeurecouldexemptthatstatefrominternationalresponsibility.ProvidedthatcasesinvolvingtheuseofAWSdonotexistyet,itispossi-bletoanticipate,onthebasisofthemostrecentcaselaw,whatcouldbetheapproachofmonitoringbodiesvis-à-visthemisdoingsofAWS.LetustaketheECtHR’sjurisprudenceasanexample,althoughthesamemaypertainto205ILC,‘EightReport’paras121ff.206JZrilic,‘ArmedConflictasForceMajeureinInternationalInvestmentLaw’(2019)16ManchesterJofInt’lEconomicL28;MDellinger,‘RethinkingForceMajeureinPublicInternationalLaw’(2017)37PaceLR455.SeealsoPdeSena,‘La“duediligence”etlelienentrelesujetetlerisquequ’ilfautprévenir:quelquesobser-vations’,inS.Cassella(ed)Lestandarddeduediligenceetlaresponsabilitéinterna-tionale(Pedone2018).207ARSIWAart23,Commentarypara3.208ILC,‘“Forcemajeure”and“FortuitousEvent”ascircumstancesprecludingwrongfulness’(n186)para130(regardingtheshootingofanAmericanofficerontheUSSChattanoogabyFrenchofficersintheChineseharbourofChefoo)andparas255–6(regardingthebombingofLaChaux-de-FondsbyGermanairmenduringWorldWarI).DiegoMauri-9781802207675DownloadedfromPubFactoryat05/28/202203:48:45AMviaDukeUniversityLawLibrary

231Autonomousweaponssystemsandthe‘accountabilitygap’conundrum223otherjudiciaryorquasi-judiciarybodiesthatmayapplyIHRLandIHL.TheECtHRhasdevelopedanextensivejurisprudenceregardingtheuseof(lethal)forcebystateagents.Fromaproceduralstandpoint,astheimpugnedeventslielargelywithintheexclusiveknowledgeofstateauthorities,theburdenofproofwillrestonthemtoprovideaconvincingexplanationastowhetherAWSdeliberatedaparticularcourseofactionthathumanoperatorswerenotabletohalt.209Inadditiontothis,theECtHRemphasizesthatinbothplanningandcarryingoutpoliceoperationsregardhastobegiventowhetherauthoritiestook‘appropriatecare’toensureminimizationofrisks,withaviewtoavoidingtheimpositionof‘unrealisticburden’onthestate.210Onamoregenerallevel,itshouldbenotedthattheECtHRhasbeenconstruingtheduediligenceobliga-tionsstemmingfromtherighttolifeandnottobesubjecttotorture,inhumanordegradingtreatmentinincreasinglystricterterms.Onthepremisethatonly‘feasible’measuresarerequestedtobetakenandthattheultimatestandardisoneof‘reasonableness’,211theECtHRwasabletoaffirmtheexistenceofa‘dutyofdiligence’incumbentonstateauthoritiesinrespectoftheuseoffirearmsbyprivateindividuals,interpretingthetraditionalrequirementoftheexistenceofa‘realandimmediaterisktolife’forduediligenceobligationstobetriggeredinaverybroadfashion.212Asadissentingjudgehasnoted,inthelongrunthisapproachmayendupoverstretchingthescopeoftheduediligenceobligationsderivingfromtherighttolife.213Thisgeneraltrend—coveringbothnegativeandpositiveobligations—isofinterestforourpurposes:itseemstoindicatethat,whenconfrontedwithECHRviolationscommittedthroughAWS,theECtHRwillprobablymain-tain,ifnotstrengthen,astrictanddemandingapproach.Providedthatstateshaveactualorputativeknowledgeaboutrealandimmediateriskofharm(suchasthatstemmingfromtheuseofAWSinoperationalscenarios),therearenoseriousobstaclestoestablishingstateresponsibilityforthemisdoingsofAWSincurrenthumanrightsjurisprudence.214Thisresultcouldbejustifiedonthe209TagayevaandOthersvRussia(n197)para586.210McCannandOthersvUnitedKingdom,AppNo18984/91andothers(27September1995)para200.211KukhalashviliandOthersvGeorgia,AppsNos8938/07and41891/07(2April2020)para146(foracaseregardingindiscriminateandexcessiveuseoflethalforceduringanti-riotoperationinprison).212KotilainenandOthersvFinland,AppNo62439/12(17September2020)(foracaseregardingthefailuretoconfiscategunfromadangerousstudentthatlatercom-mittedschoolkillings).213IbidPartlyDissentingOpinionofJudgeEicke.214Formoreonthispoint,seeVStoyanova,‘Fault,KnowledgeandRiskwithintheFrameworkofPositiveObligationsundertheEuropeanConventiononHumanRights’(2020)33LeidenJIL601.DiegoMauri-9781802207675DownloadedfromPubFactoryat05/28/202203:48:45AMviaDukeUniversityLawLibrary

232224Autonomousweaponssystemsandtheprotectionofthehumanpersonbasisofthelong-standingtheoryaccordingtowhichtheprotectionofhumanrightshastobe‘practicalandeffective’,thatis,calibratedagainstthechangingneedsofsocieties.Asillustratedinthepreviouschapters,thereseemstoberoomforevolutiveinterpretationinthisfieldgenerally.Thisnotwithstanding,itisequallyclearthatitwouldbetoodemandingforastatetoanticipateeverypossiblecourseofbehaviorbyAWS,especiallywhenusedincompliancewithproducer’sdirectivesandwithinpermittedenvironments.Therequirementofputativeknowledgemustnotbeinterpretedinawaythatrendersituseless,aswouldoccurifweweresatisfiedwithaffirm-ingthateachandeverytimeAWSmisfire,thisinitselfdemonstratesthatthedeployingstateoughttohavebeenawareofit.Thatis,adoptingabroadnotionofputativeknowledgeand,thus,ahighanddemandingstandardofduediligence,scenariosamountingto‘falsepositive’wouldescaperegulation.Inourinitialhypothesis,thosemisdoingsarebydefinitionoutsidetheperimeterofstates’faultyconduct,inthattheydidadoptallreasonableprecautionarymeasures,buttheunexpectedoutcomehasnonethelessoccurred.Noactualorpotentialknowledgeaboutaparticularmisdoingcanbefound.Letusnowdrawsomecommonconclusionsfromourremarksonforcemajeureandduediligence.Withspecificreferencetothescenariosdescribedabove,itcannotbeexcludedthateitherforcemajeurewillbeinvokedsuccess-fully,orthattheduediligenceobligationsapplyatall.Whilethisistrue,ourcontentionisthatthissituationdoesnotnecessarilyresultinan‘accountabilitygap’thatonlynarrowconceptionsofhumancontrol(implyinghumaninter-ventionateachandeveryperformanceofcriticalfunctions)areequippedtofill.Differentsolutionscouldbefound,andthechallengesdescribedthusfarcouldbedealtwithadequately.5.5.3OvercomingChallenges:MakingtheCaseforan‘AbsoluteLiability’RegimefortheMisdoingsofAWSFromadelegelataperspective,theregimeofstateresponsibilityforbreachesofIHRLandIHLisfault-basedtraditionally.WhatneedstobediscussediswhetherthereareoptionsforadjustingexistinglawinamannerthatitcanfaceAWS-relatedchallengesmoreappropriately.Inthissubsection,anappraisalofthecharacteristicofatraditionalregimeisprovided,beforeturningtodiscus-sionoftheopportunityofano-faultregimeforthemisdoingsofAWSandthemeanstoestablishit.DiegoMauri-9781802207675DownloadedfromPubFactoryat05/28/202203:48:45AMviaDukeUniversityLawLibrary

233Autonomousweaponssystemsandthe‘accountabilitygap’conundrum225Letusbeginwiththelawasithasbeen,andasitis.Theconceptoffaultininternationallawhasbeenaddressedbyextensivescholarship.215Thespecifi-cationofageneralrequirementoffaultasaprerequisitetostateresponsibilitywasdiscussedatlengthbytheILC,butwaseventuallydroppedowingtooppo-sitionfromalargenumberofstatesandtheoreticaldifficulties.216Morepre-cisely,itwasbelievedthattheconceptoffaultpertainedmoretotheprimaryobligationsatstake,ratherthanactingasastructuralelementofresponsibilityatagenerallevel.217Thisholdsnotonlyforduediligenceobligationsasexploredabove,butalsofornegativeobligations.Admittedly,thosedutiesdonotprovidefordolus,culpaorothermensrea,sothatthecorrespondingmodelofresponsibilityisthatof‘objective’(orno-fault)responsibility—thestandardmodelininternationallaw,whennototherwiseprovided.218However,inpracticethisistrueonlyprimafacie.Thatfaultisnotenvisagedasastruc-turalcomponentoftheprimaryobligationsunderscrutinydoesnotruleouttheimportanceof‘psychological’standardsforthepurpose,forinstance,ofthecircumstancesprecludingwrongfulness:‘certainmanifestationsofthe“intention”or“design”behindstateactionmayberelevanttothejustificationsandexcuses’.219Thatis,ifastatedemonstratesthatithasdoneeverythinginitspowertoavoidtheimpugnedmisdoing,itmaysucceedininvokingforcemajeure,atleastinanadmittedlysmallpercentageofcasesqualifyingas‘falsepositives’.Thisisapossibilitythatshouldnotberuledoutapriori,asarguedpreviously.TheriskthatfutureAWSwilltakeunexpected(andunforeseeable)coursesofactionintheoperationalfieldisamaterialone.Individuals,businessesandotherentities(firstandforemost,states)havenointerestinmakingorusingAWSpronetomalfunctioning;onthecontrary,itistotheirownbenefitto215Foranoverviewondoctrinalcontributions,seeILC,‘“Forcemajeure”and“FortuitousEvent”ascircumstancesprecludingwrongfulness’(n186)paras488ff,andIBrownlie,SystemofLawofNations:StateResponsibilityPartI(OUP1983)38ff.SeealsoPdeSena,‘Condottadisingoliorganiecondottadell’apparatostataleintemadicolpanell’illecitointernazionale’(1988)71Rivistadidirittointernazi-onale525;AGattini,‘LanotiondefauteàlalumièreduprojetdeconventiondelaCommissionduDroitInternationalsurlaresponsabilitéinternationale’(1992)3EJIL253;GPalmisano,‘Fault’,inRWolfrum(ed),MaxPlanckEncyclopediaofPublicInternationalLaw(OUP2007).216Crawford(n172)60.217Denmark(onbehalfoftheNordiccountries)(1998/II(1))ILCYb101;ARSIWAart2,Commentarypara10.218Crawford(n172)61(‘oncethebreachofanobligationowedunderaprimaryruleofinternationallawisestablished,thisisprimafaciesufficienttoengagethesec-ondaryconsequencesofresponsibility’);BConfortiandMIovane,Dirittointernazion-ale(12thed,EditorialeScientifica2021),433ff.219Crawford(n172)61.DiegoMauri-9781802207675DownloadedfromPubFactoryat05/28/202203:48:45AMviaDukeUniversityLawLibrary

234226Autonomousweaponssystemsandtheprotectionofthehumanpersonproducefail-safe,efficientandreliableautonomoussystems.Equally,thiswillnotbealwaysthecase;misdoingswilloccur.Whenitisdemonstratedthatastatehasdoneanythinginitspowertopreventthosemisdoings,thatharmfulbehaviordoesnotamounttoaninternationallywrongfulact.HowtobestprotectthevictimsofAWSinthosecases?Astheprevioussubsectionsmadeclear,solutionsmaybefound.However,inourview,falsepositiveswouldhardlyfallwithinthescopeofanevolution-aryinterpretationofexistingprovisions.Ultimately,tostretchexistingrulesmayappearunsatisfyingas,indoingso,thegistoftheproblemraisedbyAWSfailstobegraspedandaddressedproperly.Whatseemsmoreappropriateisanadhocregimecapableofdoingwithoutfaultonthepartofstatesandattributingliabilitywheneverharmoccursasaresultofactivitiesthatarenotprohibited(onthecontrary,thetraditionalregimeofstateresponsibilitywouldbeapplicable).Theinternationallegalorderhasbeendealingwithpermittedbuthazardousactivitiesforseveraldecadesnow.Technologicaladvances,whichenableconsiderableeconomicprogressbutbringaboutactivitiescapableofcausinggreatharmtosociety,hadalreadyledmostdomesticlegalsystemstomovefromafault-basedmodelofcivilliabilitytowardsano-fault,‘objective’model.220Asimilartransitionwasunderwayininternationallawduringthe1960sandthe1970s.First,internationallegalscholarsbegantoaddressthetopicof‘ultra-hazardousactivities’whichindustrialandtechnologicaldevel-opmentsmadepossibleatthattime,mostlyregardingtheenvironment.221TheILC’sworksontheinternationalresponsibilityofstateswereongoingatthattime:SpecialRapporteurAgodecidedtoinsulatethetopicofinternationalliabilityarisingoutoftheperformanceofparticularlawfulactivitiesfromthemaindiscussion,devotedtowrongfulacts,andin1978adedicatedsubjectwasplacedontheILC’sprogramofwork.222Theunderlyinginterestinestab-lishingamechanismofliabilitywas,firstandforemost,toensurethat,astheimpugnedactivitiesarelawfulperse,victimssufferingfromensuingharm220Monterossi(n17)5(andreferencestherein).221CWJenks,‘LiabilityforUltra-HazardousActivitiesinInternationalLaw’(1966)117Recueildescours176(arguingfortheneedofanobjectiveregimeofliabilityatagenerallevel);PMDupuy,LaresponsabilitéinternationaledesEtatspourlesdom-magesd’originetechnologiqueetindustrielle(Pedone1976)(arguingforspecific,treaty-basedregimesofobjectiveliability).222ILC,‘SecondReportonStateResponsibilitybyMrRobertoAgo’(1970/II)ILCYb178;ILC,‘InternationalLiabilityforInjuriousConsequencesArisingoutofActivitiesnotProhibitedbyInternationalLaw’,Reportofthe30thSession(1978/II(2))ILCYb150.DiegoMauri-9781802207675DownloadedfromPubFactoryat05/28/202203:48:45AMviaDukeUniversityLawLibrary

235Autonomousweaponssystemsandthe‘accountabilitygap’conundrum227disposeofadequateremedies.223TheILC’sworkculminatedinthe2001‘Draftarticlesonthepreventionoftransboundaryharmfromhazardousactivities’andthe2006‘Draftprinciplesontheallocationoflossinthecaseoftrans-boundaryharmarisingoutofhazardousactivities’.224Whilethisnormativeframeworkhasbeenconceivedanddevelopedwiththeprotectionoftheenvironmentfromcertainactivities(namely,thosethatarecarriedoutbyprivateentitiessuchasbusinessesintheterritoryofastateandthatproduceharmfuleffectsoutsidethatterritory)inmind,225theconceptof‘absoluteliability’goeswellbeyondit.Spaceactivitiesandthoseinvolvingnuclearenergywereconsideredasfallingwithinthescopeofthestudy.226Atreatyworthmentioningisthe1972ConventiononInternationalLiabilityforDamageCausedbySpaceObjects.227Inparticular,itsarticleIIestablishesaregimeofabsoluteliabilityfordamagecausedbythelaunchingstate(s)onthesurfaceoftheearthandtoaircraftinflight,whilearticleIIIresortstothetraditionalfault-basedmodelofliabilityasregardsdamagecaused‘elsewhere’(typically,tootherspaceobjects).Therationalebehindthistwo-prongedmodelofresponsibilityliesinthat,whileinthelatterscenariostatesareperformingsymmetricalactivities(thatis,navigatinginouterspace),intheformerthehazardousactivitycreatesnon-reciprocalrisks,forwhichaninjuredpartywouldfindithardtoprovefault.228Eventhoughtreaty-basedregimesofabsoluteliabilityexist,itisdisputedthatgeneralruleshaveconsolidatedincustomarylaw,totheextentthattosomecommentatorsthenormsidentifiedbytheILCremainatthelevelofrecommendations.229InordertofillresponsibilitygapsarisingfromtheuseofAWS,somehavearguedinfavorofaregimeofabsoluteliabilityforthesemisdoings.Hammond,forinstance,sponsorsa‘strictliabilityregime’inthatthecorre-223MMontoje,‘TheConceptofLiabilityintheAbsenceofanInternationallyWrongfulAct’,inJCrawford,APelletandSOlleson(eds),TheLawofInternationalResponsibility(OUP2010),504–5.224Respectively,ILC,Reportofthe53rdSession(2001/II(2))ILCYb148,andILC,Reportofthe56thSession(2006/II(2))ILCYb59.225JBarboza,‘InternationalLiabilityfortheInjuriousConsequencesofActsnotProhibitedbyInternationalLawandProtectionoftheEnvironment’(1994)247Recueildescours301.226AEBoyle,‘StateResponsibilityandInternationalLiabilityforInjuriousConsequencesofActsnotProhibitedbyInternationalLaw:ANecessaryDistinction?’(1990)39ICLQ1,4.227(Adopted29November1971,enteredintoforce1September1972)961UNTS187.228EFHennessey,‘LiabilityforDamageCausedbytheAccidentalOperationofaStrategicDefenseInitiativeSystem’(1988)21CornellInt’lLJ317,328.229Montoje(n223)512.DiegoMauri-9781802207675DownloadedfromPubFactoryat05/28/202203:48:45AMviaDukeUniversityLawLibrary

236228Autonomousweaponssystemsandtheprotectionofthehumanpersonspondingactivity—employingautonomoussystemsthatmayinteractunfore-seeablywiththeenvironment—createsnonreciprocalrisksandbenefits.230Inthesamevein,Crootofbelievesthataregimeofstrictliability,tobemodelledonexistingdomesticregimes,isthepreferablesolutiontotheproblemsposedbywhatshecalls‘wartorts’:thismodelfitswiththeinherentunpredictabilityofthesystemsandtheirdangerousnessforhumanindividuals.231AccordingtoAmorosoandGiordano,ithastobeacknowledgedthatthisoptionmaybewelcomedunderadelegeferendaperspective,butdoesnotcorrespondtothelawofstateresponsibilityasitcurrentlystands.232Asamatteroffact,iftheoverarchingproblemanimatingouranalysisistofind‘who’(or‘what’)toblameforAWS’misdoings,expandingthemodelofabsoluteliabilitytoAWSviaanadhoctreatyinstrument—giventheabsenceofageneralregimeincustomaryinternationallaw—wouldbenotonlyanaus-piciousdevelopment,butalsooneperfectlyinlinewithexistinginternationallaw.ThedangersposedbyAWSfitperfectlyinthoseconceptualcategoriesthatmodelsofabsoluteliabilitynormallyemploy.Owingtotheirunpredict-abilityinthefield,AWScanbedescribedas‘risky’,andthustheactivitiescarriedoutthroughtheminlaw-enforcementandmilitaryscenarioscanberegardedas‘ultra-hazardous’.233Thatis,itistobeexpectedthatstatistically,byreasonofthenatureandtheoccurrenceofthoseactivities,harmwilloccurinspiteofanyreasonableprecautionarymeasuresadoptedbytheemployingstate.234Tofunctionproperly,suchanextendedmodelofabsoluteliabilityshouldcoverthefollowingissues.First,ithastobeelucidatedthatharmcausedbythemisdoingsofAWSincludeslossoflifeandpersonalinjury,inlinewiththeexistingprinciple1oftheILC’s2006DraftPrinciples.235Therewouldbenoneedforindulginginthelong-standingdebateonwhethertheharmmustbe‘significant’,thatis,reachingacertainthreshold,whichhasgainedtractionintheenvironmentaldomain.236Theinfringementofindividualintegritywouldsuffice.Second,theregimeofabsoluteliabilityforthemisdoingsofAWSthatcannotbeimputedtomalfunctioningormalicioususeandthatwhichtradition-allyexistsforIHRLandIHLviolationswouldnotbemutuallyexclusive:on230Hammond(n166)670–71.231Crootof,‘WarTorts’(n10)1394–5.232AmorosoandGiordano(n3)225.233STownley,‘TheRiseofRiskinInternationalLaw’(2018)ChicagoJIL594,632.234GHandl,‘LiabilityasanObligationEstablishedbyaPrimaryRuleofInternationalLaw’(1985)16NetherlandsYbInt’lL49,64.235ILC,Reportofthe56thSession(n224).236Montoje(n214)508;Boyle(n226)7.DiegoMauri-9781802207675DownloadedfromPubFactoryat05/28/202203:48:45AMviaDukeUniversityLawLibrary

237Autonomousweaponssystemsandthe‘accountabilitygap’conundrum229thecontrary,theformerwouldcomplementthelatterandfillgapsarisingfromitsapplication.Bydoingthis,statesremainunderadutytopreventharmfuleventsresultingfromtheuseofAWSasencapsulatedbothinIHRLandIHL,theabsoluteliabilitymodelbeingactivatedbymisdoingsthatcannotbetracedtoanybreachoftheduediligencerule.237Lastbutnotleast,ithasbeennotedthatregimesofabsoluteliabilityininternationallawarestronglyorientatedinfavorofthevictims,inthattheyensurecompensationtothosewhosufferfromtheharmfuleffectsofactivitiesthatarenotprohibited.238The2006DraftPrinciplesaimtoensure‘promptandadequatecompensation’tovictimsoftransboundarydamage,aprinciplethatissettledininternationallawregardingenvironmentalmatters.239Giventhedifficultiesillustratedabove,afuturetreatyregulatingthemisdoingsofAWSshouldalsoestablishunambiguousrulesonaspectsofprivateinternationallaw(suchasjurisdiction,choiceoflawandexecutionofjudgments)and,importantly,theunavailabilityofcircum-stancesprecludingwrongfulness,suchasforcemajeure.240Thiswouldalsogiveapplicationtoarticle23(2)(b)ARSIWA(inthatthestatewouldexplicitly‘assumetherisk’ofamisdoingoccurring)andarticle27(b),accordingtowhichtheinvocationofforcemajeureis‘withoutpreju-dice’tocompensationformateriallossescausedbytheactinquestion.Whileitisdisputedwhetherthestateinvokingforcemajeurecanberequestedtopaycompensation,anaffirmativesolutioncouldbeexpresslyestablishedsoastomitigatetheeffectsoftheharmonthevictimsofthemisdoingsofAWS.241Inconclusion,atreaty-basedregimeofabsoluteliabilityininternationallaw,tobeimplementeddomesticallybystates,wouldbeanauspiciousstepforwardtowardsensuringaccountabilityforthemisdoingsofAWS,whichmayalsocorrectshortcomingsassociatedwiththecurrentstate-of-the-art,astheprevioussectionsinthischapterhavehighlighted.Giventhelackof237CfMontoje(n223)507.238LCondorelli,‘Laréparationdesdommagescatastrophiquescausésparlesactiv-itésspatiales’,inLaréparationdesdommagescatastrophiques.TravauxdesXIIIeJournéesd’étudesjuridiquesJeanDabin(Bruylant1990).Onthesamevein,seealsoSMazaroff,‘ExonerationfromLiabilityforDamageCausedbySpaceActivities’(1968)54CornellLRev70(stressingthevictim-centerednatureofabsoluteliability).239ILC,Reportofthe56thSession(n224)principle3.SeealsoTrailSmelter(1941)3RIAA907;UNGA,ReportoftheUnitedNationsConferenceontheHumanEnvironment(15December1972)A/RES/2994(StockholmDeclaration)principle21;UNGA,ReportoftheUnitedNationsConferenceonEnvironmentandDevelopment(14June1992)A/CONF.151/26(RioDeclaration)principle13.240Whichwouldbeinlinewithcurrenttrendsintreaty-basedregimesofabsoluteliability:seeMontoje(n223)510.241ARSIWA,art27(b).Forthedoctrinaldebate,seeCrawford(n172)318ff.DiegoMauri-9781802207675DownloadedfromPubFactoryat05/28/202203:48:45AMviaDukeUniversityLawLibrary

238230Autonomousweaponssystemsandtheprotectionofthehumanpersongeneralnormsofinternationallawonabsoluteliability,242andthepaucityandlimitsoftreatyinstrumentsestablishingit,243suchatreatywouldsurelycontributetothe‘progressivedevelopmentofinternationallaw’inthefieldofultra-hazardousactivities.5.6HOWTOFILLGAPS:ESTABLISHINGANEFFECTIVEFRAMEWORKFORREMEDYINGTHEMISDOINGSOFAWSTheaimofthischapterhasbeentoinvestigatetheextenttowhichhumancontrolhastoberetainedovercriticalfunctionsforthepurposesofinter-nationalresponsibility:thepossibilityofaccountabilitygapsresultingfromAWSisoneofthemainargumentsbrandishedbytheiropponents.Themainconcernistheinhumanenatureofaweaponrenderingdistributionofrespon-sibilityimpossible.Wedemonstratedthat,ifthatisthepurposeofemployingconceptssuchashumandignityandhumanity,bothdelegelataanddelegeferendaAWScanbedevelopedanddeployedinawaythatensuresadequateresponsibilityandthusthatiscompatiblewiththosevalues.WeacknowledgedthatAWSwillprobablyposechallengestothelawofinternationalresponsibilityinabroadsense.Individualcriminalresponsibilityforactsamountingtointernationalcrimeshastobemaintained,nomatterwhichweaponsareemployed.Yet,itwouldrunagainstthespiritofICLtoimposeliabilitybeyondtheprincipleofculpability.Corporateresponsibilityattheinternationallevelisstillatveryembryonicstages:noeffectiveinterna-tionallegalinstrumentisimminent,andtodatedomesticlegalsystemsprovideprospectivevictimswithfragmentedandunsatisfactorychancestomakerighttheirwrongandtoseekandobtainredressaccordingly.Thisshouldbeanalarmcallforstatestoadoptworkingsolutions;yet,itseemsthattheproblemliesmorewiththismodeofresponsibilityinitselfratherthanwiththetechnol-ogyunderscrutiny.Finally,stateresponsibilitycouldbeeasilyadjustedtothemisdoingsofAWS,eitherviaaprincipledinterpretationofexistinglawor—evenmoreappropriately—throughtheadoptionofadhoclegalinstrumentsestablishingaregimeofabsoluteliability.242Montoje(n223)507.243Inadditiontotreatiesmentionedpreviously,seeforinstance,BaselProtocolonLiabilityandCompensationforDamageResultingfromTransboundaryMovementsofHazardousWastesandtheirDisposal(adopted10December1999,notenteredintoforce)UNEP/CHW.1/WG/1/9/2;andNagoya-KualaLumpurSupplementaryProtocolonLiabilityandRedresstotheCartagenaProtocolonBiosafety(adopted15October2010,enteredintoforce5March2018).DiegoMauri-9781802207675DownloadedfromPubFactoryat05/28/202203:48:45AMviaDukeUniversityLawLibrary

239Autonomousweaponssystemsandthe‘accountabilitygap’conundrum231Across-cuttingissuethathasemergedfromthevariousmodesofinterna-tionalresponsibilityanalyzedthusfaristheimportanceofprovidingvictimsofthemisdoingsofAWSwithappropriateremedies:itisthoseindividualswhobearthecostofthisultra-hazardoustechnology,ifthingsgowrong.Numerousinternationalnormsestablishanobligationtomakereparationsincumbentuponthewrongdoerstate—firstandforemostatthegenerallevel—astypically‘secondary’obligations.244InIHRL,arighttoremedyinfavorofvictimsisenvisagedbyrelevanttreatiesandsoft-lawinstruments.245WhiletraditionallyacorrespondingindividualrightdoesnotexistinIHL,sometreatyprovi-sions246and,importantly,the2005UNBasicPrinciplesandGuidelines247epitomizearecenttrendtowardstheacknowledgmentoftheimportanceofensuringappropriateremediesforthosewhosufferfromseriousviolationsofIHL.248RemediesareregulatedalsoinICL,249andtheyconstitutethe‘thirdpillar’oftheB&HRsedifice.250Also,violationsoccurringasaconsequenceoftheuseofweapons,251andthereforeofAWS,requirearighttoremedy.252Inlinewiththeapproachweadopted,itshouldbeinquiredwhetherthesalientfeaturesofAWSrenderremedyingIHRLandIHLviolationsimpossible.Oncloserinspection,itdoes244ARSIWA,art31.Seegenerally,MIovane,Lariparazionenellateoriaenellaprassidell’illecitointernazionale(Giuffré1990).245UDHR,art8;ICCPR,art2;ECHR,art13;ACHR,art25;AfCHPR,art7.SeemoreindetailGBartolini,Riparazioneperviolazionedeidirittiumanieordinamentointernazionale(Jovene2009).246APIart91(shapedasanobligation‘tomakecompensation’).247UNGA,‘BasicPrinciplesandGuidelinesontheRighttoaRemedyandReparationforVictimsofGrossViolationsofInternationalHumanRightsLawandSeriousViolationsofInternationalHumanitarianLaw’(16December2005)A/RES/60/147(UNBasicPrinciplesandGuidelines).248RPisilloMazzeschi,‘ReparationClaimsbyIndividualsforStateBreachesofHumanitarianLawandHumanRights’(2003)1JICJ339(whoconsidersthevic-tim’srighttoreparationinthefieldsofIHRLandthenIHLandarguesthatitisdiffi-culttofinda‘unitary’customaryruleestablishingtheindividual’srighttoreparation).SeealsoCEvans,TheRighttoReparationinInternationalLawforVictimsofArmedConflict(CUP2012);ELCamins,‘NeedsorRights?ExploringtheLimitationsofIndividualReparationsforViolationsofInternationalHumanitarianLaw’(2016)10Int’lJofTransitionalJustice126;CTomuschat,‘StateResponsibilityandtheIndividualRighttoCompensationbeforeNationalCourts’,inAClaphamandPGaeta(eds),TheOxfordHandbookofInternationalLawinArmedConflict(OUP2014).249ICCSt,art75.250UNGuidingPrinciplesonB&HRs,principles25ff;UNOEIWGChairmanship,‘SecondRevisedDraft’(n124)art7.251MBurkeandLPersi-Vicentic,‘RemediesandReparations’,inS.Casey-Maslen(ed),WeaponsunderInternationalHumanRightsLaw(CUP2014).252Chengeta(n20);GenevaAcademyBriefingNo8(n184)23–4.DiegoMauri-9781802207675DownloadedfromPubFactoryat05/28/202203:48:45AMviaDukeUniversityLawLibrary

240232AutonomousweaponssystemsandtheprotectionofthehumanpersonnotseemthatAWSraiseunprecedentedorinsurmountableproblems;instead,theymaymagnifyexistinglimitsinvictims’accesstoadequateremedies,bothdomesticallyandintheinternationallegalorder.253Onelastexamplemayillustratethepoint.FollowingtheUNBasicPrinciplesandGuidelines,forinstance,victimsareentitledtoaccesstojustice,reparationandinforma-tion.254Thefirstoftheserequiresthateffectivejudicialmechanismsareinplaceforvictims,whomustnotbehinderedintheirlegalactionsbeforecourtsandotherbodies—whethertheyarepartofthecriminal,civilordisciplinarysystem.Asforthesecond,reparationcoverstherestorationofpecuniaryandmoraldamagesresultingfromtheunlawfulact.Itshouldbekeptinmindthat,inlinewiththeARSIWA,obligationsstemmingfromthecommissionofaninternationallywrongfulactincludecessation,offerofappropriateassurancesandguaranteesofnon-repetition,andreparation(intheformsofrestitution,compensation,satisfactionandpaymentofinterest).255Focusingonthelatter,itisheldthatformsofsatisfactioninclude‘verificationofthefactsandfullandpublicdisclosureofthetruth’,aswellas‘accesstorelevantinformationconcerningviolationsandreparationmechanisms’.256ItmaybequestionedwhetherAWSendowedwithreducedhumancontrolovercriticalfunctionsareadequateforensuringthisinformation.RecallingourpreviousfindingsonthedutytoinvestigateintoIHRLandIHLviolations,257ifAWSensurethathumanoperatorsmayunderstand(andreceiveexplanationabout)whythesystemshassobehaved,thenthedutyathandwouldbecompliedwith.AllthisholdsincasesofmaliciousornegligentdischargesofforcebyAWS,towhichthelawofstateresponsibilityappliesfully.However,scenar-iosinvolvingfalsepositives,inrespectofwhichnoviolationofIHRLandIHLprimaryobligationsarises,wouldfalloutsidethatregime.Theabsoluteliabil-ityregimeweadvancedabovewouldthenstart.Inthisregard,itmustbenotedthat,intheabsenceofaninternationallywrongfulact,whatistobeacknowl-edgedtovictimsisa‘promptandadequatecompensation’,aimedmoretoavoidvictimsbearingallcostsassociatedwithalawfulactivityratherthantorepairdamages—as,strictly,nounlawfulconducthasbeentaken.ExtendingthisregimetothemisdoingsofAWS,afuturetreatymustthusimposeuponstatesspecificobligationstosetupadomesticlegalframeworkallowingfor253Hammond(n166)677ff(notingtwoimportanthindrancesforvictims,namely,theirinabilitytosuestatesbeforetheICJandthedifficultyinactivatingcivilremediesprovidedbydomesticlegalsystems).TraditionallimitshavebeenexposedinteraliosbyPisilloMazzeschi,‘ReparationClaims’(n248)344ff.254UNBasicPrinciplesandGuidelinespara11.255ARSIWAarts28ff.256UNBasicPrinciplesandGuidelinesparas22,24.257SeeCh3,s3.2.2.2andCh4,s4.4.DiegoMauri-9781802207675DownloadedfromPubFactoryat05/28/202203:48:45AMviaDukeUniversityLawLibrary

241Autonomousweaponssystemsandthe‘accountabilitygap’conundrum233‘promptandadequatecompensation’.Drawingfromthe2006DraftPrinciples,thefuturetreatymayalsoincludeadutytoestablishdedicatedfundsatthedomesticlevel.258Thedutytograntaccesstodomestic(andinternational)rem-edies,aswellastoguaranteeappropriateaccesstoinformationregardingthatparticularbehavior,caneasilybederivedfromtheDraftPrinciplestoo.259Bydoingthis,theentirespectrumofscenariosfeaturingthemisdoingsofAWSwouldberegulated.Summingup,iftheoverarchingissueistoensurethataccountabilitygapsdonotopenupasaconsequenceoftheuseofincreasinglyautonomoussystemsinthefield,ICL,corporateresponsibilityandthelawofstateresponsibilitycanbealladjustedaccordingly—bothdelegelataanddelegeferenda.Thatis,toconstrueanotionofhumancontrolonthebasisoftheneedtoregulateinternationalresponsibilityforthemisdoingsofAWScomesdowntothefol-lowingissue:human–machineinteractionsmustbesufficientfordistributingresponsibilityacrossvariousactors(individuals,businessesandstates)inacoherentandacceptablefashion.Formsofhumancontrolwherebyhumanoperatorsmaynotinterveneineachandeverycriticalfunction,butstillremaininthewiderloop,mayworkiftheultimategoalistopreventaccountabilitygaps:therewouldbenoneedtoprohibitAWSaltogether.IfthemainreasoninfavorofbanningAWSconsistsofthepossibilityofopeningupaccounta-bilitygaps,thentheargumentisnotdecisive:adequateremediescanbebuiltupthroughadhocregulation.ThisconclusionmaysoundunacceptableforskepticsofAWS.Yet,atleastitstandsasaspurforbasingargumentsonmoresolidgrounds,asshowninthenextchapter.5.7CONCLUDINGREMARKSAutonomousweaponssystemsraisekeychallengesnotonlyfromthestand-pointofprimaryobligations(cantheybeusedincompliancewithIHRLorIHLrulesandprinciplesgoverningtheuseofforceagainstindividuals?),butalsowhenitcomestodistributingresponsibilityacrossalltheactorsinvolvedintheirdevelopment,deploymentanduse(whoistoblameforthemisdoingsofAWS?).OuranalysisinthischapteraddressedtheargumentwherebyAWSwouldbe‘abhorrent’machinestotheextentthattheyrenderallocationofresponsibilityimpossible,asaconsequenceofremovinghumaninterventionfromcriticaldecisions.Wedemonstratedthatthisargumentmayberhetori-callyappealing,butaddslittletoourlegalanalysis.258ILC,Reportofthe56thSession(n224)principle4(4).259Ibidprinciple6.DiegoMauri-9781802207675DownloadedfromPubFactoryat05/28/202203:48:45AMviaDukeUniversityLawLibrary

242234AutonomousweaponssystemsandtheprotectionofthehumanpersonThatis,itseemsthat,whilegivingvoicetoanunderstandableconcernoftheinternationalcommunity,anobjectiontoAWSbasedsolelyonaccountabilitygapsisultimatelymisplaced.Internationallawgoverningtheresponsibilityofindividuals,corporationsand,moreimportantly,states(aswellasIOsandothernon-stateactors,whichwehavenotcovered)canbeadaptedandadjustedtocopewithAWS-relatedchallenges.Instead,theargumentpertain-ingtothe‘inhumane’or‘abhorrent’characterofAWSshouldbeconnectedmoretotheissueofwhetherdispensinghumaninterventionfromcriticaldecisionsisinlinewithhumandignityandhumanity.260Coupledwiththe‘technological’objectiontoAWSthatwediscussedinthepreviouschapters(thatis,AWScannotbedevelopedanddeployedinaccordancewithrulesontheuseofforce),the‘accountabilitygap’objectiondoesnotappeardecisive.ThisbringsustothecorevaluesthatshouldanimatethediscussiononAWS,andtothestepsthatshouldbetakentoprogressinourunderstandingoftheimpactofAWSontheprotectionofthehumanperson.Wediscussthosekeypointsinthefollowing(andfinal)chapter.260SeesupraCh3,s3.5andCh4,s4.5.DiegoMauri-9781802207675DownloadedfromPubFactoryat05/28/202203:48:45AMviaDukeUniversityLawLibrary

2436.BuildingthebridgetothefutureTheinternationaldebateonAWSmaysoongettoaturningpoint:the2020–21GGEhasbeenmandatedwith‘clarifying,consideringanddevelopingthenormativeandoperationalframework’onAWS.1Theoutcomesofthisitemweresubmittedtothe2021SixthReviewConferenceoftheCCW,whichmaymarkanauspicious—muchanticipated,asin2021thedebateattheCCWturnedeight—stepforwardtotheregulationofAWS.Onecanwonderwherethepathwilllead:towardsabindinginstrumentofinternationallaw,possiblyintheformofanadditionalprotocoltotheCCW?Whichelementsshouldthistreatycontain?Willanon-bindingdeclaration(containingguidelinesorguidingprinciples)beenoughaswell?Wherewillitstandasregardsexistingnormsofinternationallaw?Itisdifficult,atthetimeofwriting,toanticipatefuturemoves.This,however,shouldnotdistractfromtheimportanceofhavingaprincipleddiscussiononhowtobestregulateAWSinthenearandlong-termfuture.Thisfinalchapterdrawsconcludingreflectionsfromtheanalysisconductedthusfar,withaviewtoexploringwaysforwardandtestingthewatersofpossibleimplicationsofAI-relatedtechnologiesbeyondAWS.Althoughthesewaterswillnotbenavigated,ascrutinizingglance,tosampleatleastpartoftheirnature,maybeusefulinordertoanticipatewhatwillawaitusinthecomingyears.First,theintensedebateonthenotionofmeaningfulhumancontrol(MHC)iswrappedup(section6.1).Second,attentionwillbegiventogeneralprinciplesofinternationallaw(moreprecisely,ofIHRLandIHL),andtheirroleinthedebateonAWS,inordertoexploretheirpotentialandlimitsforthepurposeofprovidinganeffectiveregulationofthisnewtechnologies(6.2).Third,itwillbeassessedwhethercurrentreflectionson‘humancontrol’andontheroleofgeneralprinciplesasasourceofinternationallawcouldinfluenceexistingandsoon-to-betechnologiesthatfeatureaprogressiveremoval(orredefinition)ofhumanpresenceincriticalfunctions(6.3).Finally,closingremarksfollow(6.4).1GroupofGovernmentalExpertsonEmergingTechnologiesintheAreaofLethalAutonomousWeaponsSystems,‘Chairperson’sSummary’(19April2021)CCW/GGE.1/2020/WP.711.235DiegoMauri-9781802207675DownloadedfromPubFactoryat05/28/202203:48:48AMviaDukeUniversityLawLibrary

244236Autonomousweaponssystemsandtheprotectionofthehumanperson6.1RE-FRAMINGTHEDEBATEONMHC:CUIPRODEST?InChapter2thenotionofMHCwasillustratedchieflyasalabelforconvey-ingdifferentunderstandingsoftheextenttowhichhumancontrolhastobegrantedoverAWS.2States,non-stateactorsandscholarshaveproposedabulkofpossibledefinitionsofMHC:findingcommongrounddoesnotseemeasy.However,atleasttwobasicpointscanbefixed.First,humancontrolhastobeensuredacrossthewholelifecycleofAWS,fromtheirdesignanddevelop-menttotheiracquisitionanddeployment.Thismeansthatarangeofdiverseactivities,involvingseveralhumanagents(thesoftwaredesigners;theengi-neers;themanufacturers;thegovernmentalagentsresponsibleforacquisition;militaryandcivilianofficersresponsiblefortheoperations;andtheoperators),havetobecarriedoutbearinginmindtheneedtoretainMHC.GuidingPrinciple(b)encapsulatesthisidea,butwithreferenceto‘humanresponsi-bility’.3Second,andconsequently,thediversityoftheseactivitiesleadstotheimpossibility(orinadequacy)ofadoptinga‘one-size-fits-all’notionofMHC:thedegreeofhumancontrolthathastobeexercisedoverAWSvariesdependingontheactivityathandaswellastheoperationalcontextinwhichthesystemisoperated.4ThesefindingsonMHChavebeenmadepossiblebythelegalanalysiscon-ductedthusfar,astheyarebasedonIHRLandIHLnorms,aswellasthelawofinternationalresponsibility.TimehasthuscometoappraisetheconceptofMHCunderalegalperspective,inordertounderstandwhetheritcorrespondswithanexistingnormofinternationallawand,intheaffirmative,ofwhichnature(6.1.1).ThisleadsustoquestionthelegalstatusofMHCandaskifthisconceptlendsitselftobeunderstoodmoreasapolitical–diplomatictoolforadvancingthedebateintheCCWframework(andbeyond)(6.1.2).2SeeCH2,s2.2.3.3GroupofGovernmentalExpertsoftheHighContractingPartiestotheConventiononProhibitionsorRestrictionsontheUseofCertainConventionalWeaponsWhichMayBeDeemedtoBeExcessivelyInjuriousortoHaveIndiscriminateEffects,‘Reportofthe2019sessionoftheGroupofGovernmentalExpertsonEmergingTechnologiesintheAreaofLethalAutonomousWeaponsSystems(LAWS)’(25September2019)CCW/GGE.1/2019/3AnnexIV(GuidingPrinciples)principle(b).4InternationalPanelontheRegulationofAutonomousWeapons(iPRAW),‘FocusonHumanControl’(August2019)15.DiegoMauri-9781802207675DownloadedfromPubFactoryat05/28/202203:48:48AMviaDukeUniversityLawLibrary

245Buildingthebridgetothefuture2376.1.1TheLegalStatusofMHCInthedebateonAWS,referencestoMHCorcontiguousnotions(suchas‘appropriatelevelsofhumanjudgment’)areabundant.Whathastobeassessednowiswhetherstateandnon-stateactorsreferringtothisconceptasalegalconceptareactuallyre-statingexistingnorms,orpushingfortheformationofnewnorms.Accordingtotheanalysisthathasbeenconductedthusfar,notallIHRL-andIHL-relevanttreatiescontemplateMHC.ItisaconceptthatwasbornoutofthediscussionsaroundAWSandthathadneverbeenexpressedbefore.5Theconceptof‘control’ispresentinmanybranchesofinternationallaw:inadditiontothoseexploredthusfar,thinkofthelawofoccupation,armscontrolaswellasmanyothersub-sectorsoftheinternationallegalorder.6Afortiori,thesamegoesforcustomarylaw:neitheropiniojurisnorgeneralpracticedoactuallyattach,asmadeevidentinthat—inadditiontohavingnostatepracticeinthematter—aunitarynotionofMHCcannotbedrawnfromstatementsandinterventionbyactors.Thatis,whilevirtuallyeveryactoragreesthatsomeformofhumancontrolmustberetainedoverthecriticalfunctionsofAWS,diametricallyopposedpositionsareadvancedvis-à-vistheacceptabilityofAWS.Movingtosoft-lawinstruments,thereisatleastonerecentdevelopmentthathastobeputunderthespotlight,ifonlytoshowthatMHCisstartingtomakeitswayininternationallawbeyondthetraditionalfora.InNovember2015theAfricanCommissiononHumanandPeoples’Rights(ACommHPR)releasedGeneralCommentNo3dedicatedtotherighttolifeunderarticle4.7Inthesectionregulatinguseofforceduringarmedconflictitisestablishedthat:‘[a]nymachineautonomyintheselectionofhumantargetsortheuseofforceshouldbesubjecttomeaningfulhumancontrol’.8ToourknowledgethisisthefirsttimethatMHCisexpresslyreferredtoinaGeneralCommenttoanIHRLtreaty.Althoughvoidofbindingeffect,instrumentssuchasthisareofparamountimportanceastheyguideinterpretersinapplyingthecorresponding5TMarauhn,‘MeaningfulHumanControl–andthePoliticsofInternationalLaw’,inWHeintschelvonHeinegg,RFrauandTSinger(eds),DehumanizationofWarfare(Springer2018).6Forarecentandunitaryappraisalofthenotionof‘control’,seeRBartels,JCvandenBoogaard,PALDucheine,EPouwandJVoetelink(eds),MilitaryOperationsandtheNotionofControlUnderInternationalLaw.LiberAmicorumTerryD.Gill(Springer2021).7AfCommHPR,‘GeneralCommentNo.3ontheAfricanCharteronHumanandPeoples’Rights:TheRighttoLife(Article4)’(adoptedinNovember2015)(GeneralCommentNo3).8Ibidpara35.DiegoMauri-9781802207675DownloadedfromPubFactoryat05/28/202203:48:48AMviaDukeUniversityLawLibrary

246238Autonomousweaponssystemsandtheprotectionofthehumanpersontreatytonew,unprecedentedcasesandmaybeusedtoassesstheevolutionofcustomarylaw.ThattherequirementofMHCisexpressedinanofficialtext,however,availsourinitialintuition—MHChasnotgainedanautonomouslocusstandiininternationallawyet—insteadofcontradictingit.First,asinglereferenceinanon-bindingdocumentcouldinspireaninterpretativepracticebytheACHPR’sjudicialbodiesoritcouldnot:inanycase,currentlytherequirementofMHCisnotlegallybinding.Second,therelevantnotionisnotexplained:itishardforanyinterpretertoderiveaprecisemeaning(broadornarrow)therefrom.Nofootnotecontainingrelevantcitationsisprovided,whichcanbeseenastroublingbyopposersofAWS.9Third,andasfurtherproof,therelevantparagraphconcludes:‘[t]heuseofsuchnewtechnologiesshouldfollowtheestablishedrulesofinternationallaw’.10Again,a(quiteinnovative)referencetoMHCistuneddownbyaconcludingremarkthatrecallsexistingrulesofIHLorIHRL.Lookingatthesupra-nationallevel,notethattheEUhassometimesreferredtoMHCinthefieldofAWS.Thisisthecaseofthe2018EUParliament’sResolutiononAWSaimedaturgingthedevelopmentofacommonpositiononthematterbeforethe2018meetingofstatespartytotheCCW.11Inparticu-lar,theEUParliamentcalledontheHighRepresentativeforForeignAffairsandSecurityPolicy,theEuropeanCouncilandmemberstatestodevelopandadoptacommonpositiononAWS‘thatensuresmeaningfulhumancontroloverthecriticalfunctionsofweaponsystems’.12Giventhenatureoftheactanditsnon-bindingcharacter,theimpactonmemberstates’conductwillpre-sumablybenegligible.However,itshouldnotbeunderestimatedthattheEUinstitutionendowedwiththehighestdemocraticlegitimizationhastakensucharemarkablystrongpositioninfavorofMHC.Thisisevenmoreimportantifoneconsidersthatnationalparliamentsalsohavetakenasimilarstance,forexample,thoseofBelgiumandItaly.13Theseremarksleadustowonderwhether,providedthatopiniojurisandpracticecouldhardlybesaidtoexistcurrently,MHCcouldbeconstruedas9SeeCHeyns,‘Autonomousweaponsinarmedconflictandtherighttoadigni-fiedlife:AnAfricanperspective’(2017)33SouthAfricanJonHumanRights46,66–7(quotingGeneralCommentNo3andaddingthat‘[h]owever,thereisstillgreatuncer-taintyaboutwhattheconceptentails’).10GeneralCommentNo3para35(emphasisadded).11EuropeanParliament,‘Resolutionof12September2018onautonomousweaponsystems(2018/2752(RSP))’[2018]OJC433/86.12Ibidpoint2.13ChambredesRépresentantsdeBelgique,‘Résolutionvisantàinterdirel’uti-lisation,parlaDéfensebelge,derobotstueursetdedronesarmés’(19July2018)3203/3005;CameradeiDeputati,MotionNo1-01776SessionNo898(6December2017).DiegoMauri-9781802207675DownloadedfromPubFactoryat05/28/202203:48:48AMviaDukeUniversityLawLibrary

247Buildingthebridgetothefuture239ageneralprincipleofinternationallaw.ToputitinlinewiththedivisionadoptedbytheILC,itcouldcorrespondasageneralprincipleeitherderivingfromtreatiesorunderlyinggeneraltreatyandcustomaryrules.14AsarohasarguedthatMHCstandsasa‘principle…thathashistoricallybeentakenforgranted—assumedbutneverstated’.15Recallingourpreviousfindingsonthiscategoryofinternationalnorms,itcanbearguedthat,effectively,MHCcorrespondstoan‘axiomaticpremise’oftheinternationallegalorderinthatitpurportstopreservethedignityofthosewhoaretargetedbyAWS.16Ontheonehand,thewidespreadacceptanceofMHCasa‘commondenominator’17inthedebateonAWSisindicativeofits‘recognition’asageneralprinciple.18Ontheother,ithastobeemphasizedthatthereisnoagreeddefinitionofwhatMHCis—onthecontrary,itisunderstoodbyrelevantactorsindiametricallydivergentfashions.19Inclosing,todateMHChasnoindependentlegalstandingininternationallaw.NeithertreatylawnorcustomarylawhavedevelopedtothepointthatMHCcanbeconsideredasabindingrequirementwhenlethalforceisusedagainstahumantarget.Oncloserinspection,thecategoryofgeneralprinci-plesofinternationallawalsoisoflittleornohelp:itcouldhardlybesaidthatasufficientlyhomogeneousandprecisenotionofMHChasbeen‘recognized’bythevariousactors.Somescholarshavecontendedthatthisconcept‘revealsittobedeceptivelyalluringbutsubstantivelyvagueandunhelpful’.20Inourview,thisstatement,whileperemptory,isinprincipleacceptable.Legally,thefindingofarequirementofMHCtobeappliedtoAWSshouldbeappreci-atedfromadelegeferendaperspectiveor,assomescholarshaveputit,asa‘programmaticprinciple’ofinternationallawseekingtopromotetheadoptionofnewlaw.21Thisseemstobeconfirmedinthatstatementsanddeclarations14ILC,‘SecondReportonGeneralPrinciplesofLawbyMarceloVásquez-Bermúdez,SpecialRapporteur’(9April2020)A/CN.4/71para121(SecondReport).15PAsaro,‘JusNascendi,RoboticWeaponsandtheMartensClause’,inRCalo,AMFroomkinandIKerr(eds),RobotLaw(EdwardElgar2016),383(emphasisadded).16CTomuschat,‘InternationalLaw:EnsuringtheSurvivalofMankindontheEveofaNewCentury.GeneralCourseonPublicInternationalLaw’(1999)281Recueildecours9,161.17UNIDIR,‘TheWeaponizationofIncreasinglyAutonomousTechnologies:ConsideringHowMeaningfulHumanControlMightMoveTheDiscussionForward’(2015)3.18SeeCh2,s2.3.3.19Seeibid.20ADeeks,NLubellandDMurray,‘MachineLearning,ArtificialIntelligence,andtheUseofForcebyStates’(2019)10JofNat’lSecL&Policy1,16.21RPisilloMazzeschiandAViviani,‘GeneralPrinciplesofInternationalLaw:FromRulestoValues?’,inRPisilloMazzeschiandPDeSena(eds),GlobalJustice,HumanRightsandtheModernizationofInternationalLaw(Springer2018).DiegoMauri-9781802207675DownloadedfromPubFactoryat05/28/202203:48:48AMviaDukeUniversityLawLibrary

248240AutonomousweaponssystemsandtheprotectionofthehumanpersononMHCcomefrombothsides,thatis,bysupportersofAWSandbytheiropponents.Often,MHCisreferredtoaseithera‘guidingprinciple’,orasmakingthecontentofa‘newlegalnorm’thatstillawaitsadoption,oragainan‘emergingnotion’.22NosurprisethenthatcritiquesofAWSinvokealegallybindinginstrumentthatprohibitsAWS:avaguenotionofMHCasthatwhichisinplacecurrentlydoesnotsufficeinitself,butisinsteadperceivedasaprin-ciplethatcouldinspiredevelopmentsinpositivelaw.236.1.2ThePolitical–DiplomaticValueofMHCWhileoneshouldnotexpectMHCtoworkasalegallyrelevanttool(exceptinadelegeferendaperspective),toholdthatitdoesnotperformotherimportantfunctionswouldbeunjust.Forinstance,ithasbeenheldthatthesuccessofMHCinsideandoutsidetheCCWcanbegroundedinitsbeing‘intuitivelyappealing’.24ForAWSsupporters,thevaguenesscharacterizingithasmadeitpossibleforthemtoendorsethisconceptinthedebatewithoutfallingintothetrapofAWSopposers.25Tothese,inturn,itservedasabasisforadvocatingfortheirexplicitregulation—ifnottheirban.Globally,MHCisemployedasaneatlypoliticizedconcept.Thisconsiderationhasbroughtmostcommen-tatorstotheconclusionthatMHCcouldatmost‘augment’existingtreatyobligations(suchasIHLrulesontargeting),thusstandingasa‘regulatoryconcept’ratherthanasanindependenttreatynorm.26Itfollowsthat,atleastfromapolitical–diplomaticstandpoint,theconceptofMHCisofsomeuse.Giventhesubstantiallydivergingviewsonitscontent,itcurrentlyseemsimplausiblethat,basingonMHC,theactorsinvolvedinthedebateonAWSwillconvergeonacommonground:theissueistoopolarized,22Heyns(n9)66,70.23HumanRightsWatchandInternationalHumanRightsClinic,‘HeedtheCall.AMoralandLegalImperativetoBanKillerRobots’(August2018)44–5(concludinginsupportofa‘preemptivebantreaty’andarguingthat‘dangerstohumanitymorethanjustifythecreationofnewlawthatmaintainshumancontrolovertheuseofforceandprevents[AWS]fromcomingintoexistence’).24JvandenBoogaardandMPRoorda,‘“Autonomous”WeaponsandHumanControl’,inRBartels,JCvandenBoogaard,PALDucheine,EPouwandJVoetelink(eds),MilitaryOperationsandtheNotionofControlUnderInternationalLaw.LiberAmicorumTerryD.Gill(Springer2021),429.25RCrootof,‘AMeaningfulFloorforMeaningfulHumanControl’(2016)30TempleInt’l&CompLJ53,55.26Ibid55.SeealsoMarauhn(n5)212(‘[m]yconclusion…isthattheconceptof[MHC]shouldnotbeframedandusedasalegalconcept.Rather,useshouldbemadeofexistingand(largely)undisputedrulesof[IHL]’).DiegoMauri-9781802207675DownloadedfromPubFactoryat05/28/202203:48:48AMviaDukeUniversityLawLibrary

249Buildingthebridgetothefuture241andavaguelycraftedconceptcannotbetaskedwithimpossiblemissions.27However,byvirtueofitsactingasacatalystforsodiversepositions,MHCmayfueloperationaldiscussionsonhowbesttoensureadequatehumancontrolovercriticalfunctions:thatis,itisanotionthatmayservethepurposesoftheGGE’smandatefor2020–21,namely,todevelopthe‘operationalframe-work’ofAWS.SeveralactorshaveproposedoperationalmodelsofMHC.TheInternationalPanelontheRegulationofAutonomousWeapons(iPRAW)supportsanotionof‘humancontrol’thatembracesbothcontrol‘bydesign’and‘inuse’,andthatallowsforsituationalunderstandingandinterventionbyhumanoperatorswhenappropriate.28Therequirementsforthishumancontrolrangefromaminimumtoamaximumdegree:forinstance,includingformsof‘boxedautonomy’(thatis,ensuringnoimmediatesituationalawarenessandnoimmediateoptionforimmediateintervention),wheretheuseofforceoccursatmachinespeed,maybeacceptable.TheInternationalCommitteeforRobotsArmsControl(ICRAC),foritspart,haspublishedathoroughandconvincingreportexploringthestakesandthecharacteristicsofMHC.29Inparticular,itisarguedthatMHCshouldbeoperationalizedthroughathree-prongtest,encom-passing,inturn,(1)thewhatproperties(forexample,thetypeofmissiontobeaccomplished,whetherdefensiveoroffensive),(2)thewhereproperties(forexample,theoperationalenvironment),and(3)thehowproperties(forexample,theprocessingofinformationandsensory-motorcapabilitiestoreachthegoal).Theminimumdegreeofhumancontrol—correspondingtothehighestlevelofautonomy,wherenofurtherhumaninvolvementispossible—doesnotliveuptotherequirementofMHC:AWSsooperatingwouldthusbelegally(andethically)prohibited.ThisapproachtoMHCisstricterthanthatofiPRAW,asthelatteradmitsformsof‘boxedautonomy’.Irrespectiveofthesedivergences—demonstratingagainthedifficultyofconstruingauni-versallysharednotionofMHC—whatmattersisthatICRACprovidesele-mentsforarticulatingMHCfromanoperationalperspective,embracing‘training’,‘controlbydesign’and‘controlinuse’.Insummary,operationally,thesetwoperspectivedonotseemsodifferent.Finally,amodelofhuman27MACEkelhof,‘ComplicationsofaCommonLanguage:WhyItIssoHardtoTalkaboutAutonomousWeapons’(2017)22JofConflict&SecL311,331(demon-stratingthelimitsofa‘commonvernacular’fromalegalstandpoint,butemphasizingitsutilityinnegotiationsasa‘brilliantstrategy’).28InternationalPanelontheRegulationofAutonomousWeapons,‘FocusontheHuman–MachineRelationinAWS’(March2018).SeealsoInternationalPanelontheRegulationofAutonomousWeapons,‘FocusonHumanControl’(August2019).29DAmorosoandGTamburrini,‘WhatMakesHumanControloverWeaponsSystems“Meaningful”?’,ICRACReport(August2019).DiegoMauri-9781802207675DownloadedfromPubFactoryat05/28/202203:48:48AMviaDukeUniversityLawLibrary

250242AutonomousweaponssystemsandtheprotectionofthehumanpersoncontrolhasbeenproposedalsobytheICRCintandemwiththeStockholmInternationalPeaceResearchInstitute(SIPRI).30Theproposedmodel—aimedatensuringlegalcompliance,ethicalacceptabilityandmilitaryefficiency—envisagescontrol‘overweaponparameters’,‘overenvironment’and‘throughhuman-machineinteraction’.31Thismodelfeaturesacombinationofcontrolmeasuresrelatedtothecharacteristicsoftheenvironment,ofthesystemandoftheuser.AnalogieswithICRAC’smodelofMHCareevident.Againstthisbackground,thefollowingconcludingpointscanbedrawn.Meaningfulhumancontrolandcontiguousnotions—suchas‘appropriatelevelsofhumanjudgment’32—servetofindacommongroundfordiscussion:allagreeontheimportanceofretainingit,butsubstantiallydisagreeonwhattheseconceptsentail.‘Narrow’understandings(requestinghumaninterven-tionincriticalfunctions)and‘broad’understandings(consideringitsufficienttomaintainformsofhumancontrolinthewiderdecisionalloop)ofMHCfindalternatesupportinthedebate.33Asouranalysisinthepreviouschaptershasdemonstrated,behindthenotionofMHClietwosetsofissuesthatactorsinvolvedinthedebateconceiveasofparamountimportance.First,thedevelopmentanddeploymentofAWSmustconformwithIHRLandIHLrulesandprinciplesregardingtheuseof(poten-tiallylethal)forceagainstindividuals.34Tothisend,MHCisresortedtoasatoolforconveyingthisneed.However,ithasbeendemonstratedthatIHRLandIHLalreadydisplaylegalnormsensuringthathumancontrolisexercisedoverweaponsinarmedconflictandlawenforcement:thereisnoneedforanautonomousruleorprinciple.Second,MHCisconsideredasameansforensuringaccountabilityformisdoingsofAWS:intheviewofmostsupportersofthisnotion,humancontrolonautonomousweaponryhastobe‘meaningful’inthatitallowsfortheallocationofresponsibility(orliability)incaseswhereAWS‘gorogue’.35Inthesamevein,ithasbeenobservedthatinthelawofinternationalresponsibilityasitcurrentlystands‘accountabilitygaps’mayopenup;however,thosecanbefilledbothbyadjustingexistinglawand,ideally,byintroducingnewlaw,withoutprohibitingAWS.30VBoulanin,NDavinson,NGoussacandMPCarlsson,‘LimitsonAutonomyinWeaponSystems.IdentifyingPracticalElementsofHumanControl’,SIPRIandICRCReport(June2020).31Ibidix.32USDoD,‘DirectiveNumber3000.09’(21November2012,revisited8May2017)para4.33SeeCh2,s2.2.3.1.34AsdemonstratedinChs3and4.35SeeCh5,s5.6.DiegoMauri-9781802207675DownloadedfromPubFactoryat05/28/202203:48:48AMviaDukeUniversityLawLibrary

251Buildingthebridgetothefuture243Letusnowdrawsomeconclusionsfromtheargumentswehaveputforwardthusfar.Legally,thenotionofMHCfailstofullydetachitselffromexistingnormsofinternationallaw;atbest,itre-statesexistingrulesandprinciples.Underthisperspective,itisredundant:thedebatecouldprogresssimplybyfocusingonlexlataandappraisethechallengesofAWStoit.Thisiswhatwedidinthepreviouschapters.Diplomatically,however,thecontinuingrecurrenceofMHCinthedebatemaybehelpfulinthatitstressestheneedforoperationalizingtheconcept:thismayadvancetheunderstandingofcharacteristicsandimplicationsofAWSfromatechnicalstandpoint,andthenfosterthedebatewithintheCCWframework.Inthelongrun,MHCmayevenorientatedelegeferendainitiativesinthefieldofAWS.However,ifthisistheobjective,oneoughttorelymoreonnormsthat,eitherbywayofinterpretationofexistinglaworbyoutliningtheneedfornewlaw,alreadyperformthisfunc-tion.Thesenormsarethegeneralprinciplesofhumandignityandhumanity,towhichwenowturnourfocus.6.2RE-STATINGINTERNATIONALLAWPROTECTINGTHEHUMANPERSON:WHICHROLEFORGENERALPRINCIPLES?ThosewhoarguethatMHCalreadycorrespondstoa‘generalprinciple’ofinternationallawcommitalogicalfallacy,astheypositexactlywhattheyaresupposedtodemonstrate:theycannotescapethiscirculusinprobando.This,however,doesnotmeanthatgeneralprinciplesofinternationallawplaynoroleinthedebateonAWS—onthecontrary.Letusnowrecapwhathasemergedthusfarinlightofthemainthesisthatweadvancedatthebeginningofouranalysis.Oneofthebook’sstartingassumptionswasthattheinternationallegalorderhasbeenwitnessingarecentandimportanttrendof‘humanization’:theexpan-sionofthehumanrightsmovementhasinfluencedalmostallbranchesofinter-nationallaw,includingIHL.36Inparticular,theideahasbeentestedaccordingtowhichtherespectofhumandignityistheveryraisond’êtreofbothIHRLandIHL:ouranalysisofprimarynormspertainingtothesetwosub-sectorsofinternationallawhasconfirmedthattheideaiswellgrounded,andthatthelegalsourcewhichthat‘basicunderpinning’reflectsisthecategoryof‘generalprinciples’ofinternationallaw.37InlinewiththeILC’sperspectiveonthematter,itisthegeneralprinciples‘formedwithintheinternationallegal36TMeron,TheHumanizationofInternationalLaw(MartinusNijhoff2006).37ProsecutorvFurundžijaICTY-95-17/1-T(10December1998)para182.DiegoMauri-9781802207675DownloadedfromPubFactoryat05/28/202203:48:48AMviaDukeUniversityLawLibrary

252244Autonomousweaponssystemsandtheprotectionofthehumanpersonsystem’—derivedfromtreatiesandotherinstrumentsorunderlyinggeneraltreatyandcustomaryrules—categorywereferto.Drawingfromourremarksinpreviouschapters,wenowwonderaboutwhichrolethesegeneralprinciplescouldplayinthedebateonAWS.Towhatextentcouldgeneralprinciplesbesuccessfullyinvoked?IsthereanychancethatinternationaljudgesquaauthoritativeinterpreterswillresorttothosegeneralprinciplestoadjudicateAWS-relatedcases?Whichoutcomescouldbeexpected?Thesequestionsareaddressedinthefirstsubsection(6.2.1).Ouranalysisthenmovestotheinternationallaw-makingplan,whereitisarguedthattheymayplayaroleascatalystforstateandnon-stateactorsinvolvedinthedebate(6.2.2).6.2.1AHintofPessimism:AWSbeforeInternationalAdjudicatoryBodiesThefirstscenariothatcanbeenvisagedisthatinwhichnonewruleofinterna-tionallawhasbeenadoptedvis-à-visAWS,sothattheirdevelopmentremainssubjecttoexistingrulesandprinciples.Fromthisperspective,animportantroleinguaranteeingtheprotectionofindividualsaffectedbyAWSwillbeplayedbyinternationaladjudicatorybodies.ConsideringtheexistenceofregionalanduniversalIHRLbodiesand,conversely,theabsenceofparallelmechanismsunderIHL,itwillbefortheformertodevelopcaselawonAWS.WhilesomestatesattheforefrontofAI-relatedtechnologyarenotpartytoanybindinginstrumentsallowingforindividualorinter-statecommunications(suchastheUSorChina),atleastintheEuropeancontinentsome(suchastheUKandRussia)arepartytotheICCPRandtheECHR.Itfollowsthatcaselawontherighttolifeandtherightnottobesubjecttotorture,inhumanordegradingtreatmentwilleventuallybeextendedtoattacksconductedthroughAWS.InthepreviouschapteritwasshownthatIHRLandIHLrulesontargetingrestraintheuseofforceagainstindividuals.38Limitationsexistasregardshowandagainstwhomforcecanbeapplied.Sofar,sogood.Anunexploredissuelieswithwho,orwhat,doesmateriallyapplyforce.ThegistofAWSisthathumancontrolmayberestrictedinthattheparticularcourseofaction(thatis,selectingandengagingaparticulartarget)canbedeliberatedthroughanalgorithmicprocessinwhichahumanoperatormaynotintervene.Whileremainingsomewhereinthewiderloop,andthusformally‘incontrol’,heorshemaynotstepinperformingaspecificcriticalfunction.38SeeCh3,ss3.2,3.3,3.4andCh4,ss4.2,4.3,4.4.DiegoMauri-9781802207675DownloadedfromPubFactoryat05/28/202203:48:48AMviaDukeUniversityLawLibrary

253Buildingthebridgetothefuture245Asevidencedbytheanalysisconductedabove,itcanbeanticipatedthatIHRLmonitoringbodieswillresorttowell-knownconcepts,suchasthoseof‘arbitrariness’,‘reasonableness’and‘feasibility’,andadjustthemtotheunprecedentedscenariosunderscrutiny.Principlessuchas‘humandignity’and‘humanity’maythusplayaroleinsupportinganevolutionaryinterpre-tationofexistingrulesinordertoadaptthemtotherealityof‘autonomouskilling’,asoccurswhenthebodiesexpandtheircaselawtonewscenarios.39Thewell-knowndoctrineaccordingtowhichhumanrightscontainedinIHRLinstrumentshavetobeinterpretedandappliedsothattheyare‘practicalandeffective,andnottheoreticalandillusory’willworkaptlyforthepurpose.40Inthefaceofatechnologyraisingsoimportantethicalstakes,thosemonitoringbodiesmayfeelpressuredtoapplyexistingnormsverystrictlyandsetthebarhigh.Thiswouldundoubtedlyrepresentanauspiciousdevelopmenttowardstheeffectiveprotectionofthehumanperson.Letusnowconsideradifferentscenario,inwhichAWSselectandengagepermissibletargets(theenemycombatantorthesuspect)inaccordancewithIHRLandIHLrules.Primafacie,thefindingofabreachofinternationallawonthissolebasisseemsunlikely,ifnotungrounded:iftheconductdoesnotqualifyasarbitraryorunreasonable,evenanevolutionaryinterpretationofexistingrulescannotensurethatresult.Thewiseinterpreter,however,mayarguethat,whileitisundisputedthatrulesontargetinghavebeencompliedwith,thesamemaynotapplytogeneralprin-ciples:byhavinganautonomoussystemdeliberatingaparticularuseofforceabsenthumanintervention,therespondentstate’sconductwouldamounttoaninfringementofthegeneralprinciplesofhumandignityandhumanity.ThisisanargumentthatmayresonatewithsomeparticipantsintheCCWdebate.Granted,thespecificuseofforceconformswithexistingrules.However,astheserulesdonotcoverthecaseentirely(theelementsofnoveltylyinginwhoorwhatultimatelyappliesforce),resorttogeneralprincipleswouldbewarranted.Judicialtechniques(suchasthatofbalancing),availabletojudgesingeneralandtointernationaljudgesinparticular,canbeemployedtocon-vincinglyreachthatconclusion.41Arethosemonitoringbodiesinapositiontoadoptthislineofreasoningand,giventhelackofaruleapplicabletothecaseathand,adjudicateonthebasisofthosegeneralprinciples?Inourview,anaffirmativeanswertothequestion39SeeCh3,s3.5andCh4,s4.5.40ECtHR,AireyvIreland,AppNo6289/73(9October1979)para24.41PDeSenaandLAcconciamessa,‘BalancingTest’,inHRuizFabri(ed)MaxPlanckEncyclopediaofInternationalProceduralLaw(OUPforthcoming).DiegoMauri-9781802207675DownloadedfromPubFactoryat05/28/202203:48:48AMviaDukeUniversityLawLibrary

254246Autonomousweaponssystemsandtheprotectionofthehumanpersonispossibleand,yetwithsomeadjustments,entirelyjustifiable.This,however,doesnotmeanthatitisalsopracticallyfeasible.Ontheonehand,methodologically,theuseofAWScreatesan‘unusualcase’theresolutionofwhichwarrantsresorttogeneralprinciples.42Itisuptotheinterpreter—theinternationaljudge,inourcase—to‘recognize’theexistenceofageneralprinciplebydistillingitfromexistingtreatyorcustom-aryrulesofinternationallaw.Thishermeneuticoperationof‘recognition’hastobecorroboratedbyaconvincingdemonstration,lesttheinterpreterisnotaccusedofarbitrariness.Relianceonunambiguousdeclarationsbystatesandotheractorsinvolvedinthedebate,onaparwithaprincipledappraisalofexistingrules,isvitaltoreachsoundandpersuasiveconclusions.Ontheotherhand,giventhecurrentunderstandingofthegeneralprinciplesofhumandignityandhumanitywithintheCCWframeworkandbeyond,ourfictionalinterpretermayhavedifficultygettingthatfar.Uptonow,whenstateandnon-stateactorsarguethatAWSruncountertohumandignityandhuman-ity,theytendtorelyon(1)lackofcompliancewithIHRLandIHLrules,(2)accountabilitygaps,and(3)ethicalandmoralreasons,whoseconnectionwithlawisnotexplained.Veryrarelyhavethoseactorsengagedinademonstrationthatthescenariounderscrutinyisalreadyproscribedassuch.Inotherwords,whatismissingispreciselytheelementof‘recognition’ofgeneralprinciplesformedwithintheinternationallegalorder.43Thissufficestoconcludethatitwouldbehardforthosemonitoringbodiestopushthemselvesbeyondatel-eologicalinterpretationofexistingrights.Thus,theywillhardlyregardthetargetingofapermissibleobjectiveillegalsolelyonthebasisthatitoccurredviaAWS.Consideringnowinternationalcourtsasauthoritativeinterpretersandappliersofthelaw,wecananticipatethatthosetoowillprobablyrefrainfromadoptinganeffectivestanceagainstAWS.TheICJprovidesatellingexampletothisend.Evenwhenconfrontedwithfarmoredisruptiveweapons,suchasnuclearweapons,theICJfounditselfunabletotakeafirmstanceinfavoroftheirprohibition:itisthewell-known(andcriticized)caseofnuclearweapons.44TheICJwasmoreinclinedtoresortto(andthusapply)generalprinciplesatthebeginningofitsactivity,ashappenedwiththeCorfuChannelcase,whereitinvoked‘elementaryconsiderationsofhumanity’aspartof‘certaingeneralandwell-recognizedprinciples’viableinadjudicatorycases.4542LGradoni,‘Consuetudineecasoinconsueto’(2012)95Rivistadidirittointerna-zionale704.43SecondReportpara121.44LegalityoftheThreatofUseofNuclearWeapons(AdvisoryOpinion)[1996]ICJRep66.45CorfuChannelcase(UnitedKingdomvAlbania)(Merits)[1949]ICJRep4.DiegoMauri-9781802207675DownloadedfromPubFactoryat05/28/202203:48:48AMviaDukeUniversityLawLibrary

255Buildingthebridgetothefuture247Theaimwastocontributetothedevelopmentofinternationallawonthebasisofthecontingentexigenciesandvaluesoftheinternationalcommunity.46Morerecentlythough,itcarefullyavoidedanyreferencestogeneralprinciplesinthenameofafiercelypositiveapproachtointernationallaw,astheJurisdictionalImmunitiescasemadeclear.47Thus,imaginingadisputeinvolvingtheuseofAWS,ormoreprobablyarequestforadvisoryopiniononthatmatter,itseemsimplausiblethattheICJwillventureintoestablishingtheillegalityofAWS.Instead,itwillmoreprobablyre-stateexistingIHRLandIHLoutofrespectformorepositivisttrends,asmadeclearintheFisheriesJurisdictioncase.48Thisreflectionleadsustothefollowing,whichisdedicatedtothelimitedroleplayedbygeneralprinciplesininternationaladjudication.Generally,thecau-tiousstancetowardstheirapplicationresultsfromstates’mistrustforanyjudi-cialtechniquesgranting‘considerableleeway…overformalizedrulescreatedthroughclassicallaw-makingprocedure’.49Similarly,inanycase,thosebodieswillhardlybeinapositiontoproscribeaparticularweaponorweaponssystemwithbindingeffectsuponstates.Theyrenderadvisoryopinions,settlespecificdisputesbetweenstatesoradjudicatestateresponsibilityvis-à-visotherenti-tiesandsubjects:theirhandsarethereforetied.ItcannotbedeniedthattheaffirmationofthecontrarinessofAWStogeneralprinciplesbyadjudicatorybodies(eitheruniversalorregional),yetremote,wouldhaveanauthoritativeweightintheinternationallegalorderand,inparticularcircumstances,mayinspireinternationallaw-making.Thisphenomenonisfullyinlinewiththetraditionalaimofinvokinggeneralprinci-ples:contributingtothedevelopmentofinternationallawinavalue-orientatedfashion.50Itmaybethatthedevelopmentofnewtechnologieslikelytosubverttherelationshipbetweenhumansandmachinespromptsthoseinternational46ReparationforInjuriesSufferedintheServiceoftheUnitedNations(AdvisoryOpinion)[1949]ICJRep174,IndividualOpinionbyJudgeAlvarez190(acknowledg-ingthattheCourtpossessesthepower‘todevelopinternationallawandtocontributetoitscreationinfaceofnewsituations’).47JurisdictionalImmunitiesoftheState(GermanyvItaly)(Merits)[2012]ICJRep99.48FisheriesJurisdiction(UKvIceland)[1974]ICJRep3,para53(‘theCourt,asacourtoflaw,cannotrenderjudgmentsubspecielegisferendae,oranticipatethelawbeforethelegislatorhaslaiditdown’).49VCannizzaro,‘BalancingTest:InternationalCourtofJustice(ICJ)’,inHRuizFabri(ed),MaxPlanckEncyclopediaofInternationalProceduralLaw(OUP2019)para6.50ECarpanelli,‘GeneralPrinciplesofInternationalLaw:StrugglingwithaSlipperyConcept’,inLPineschi(ed),GeneralPrinciplesofLaw–TheRoleoftheJudiciary(Springer2015),141(referringtotheroleofthejudiciaryascontributingtothe‘“mor-alization”ofinternationallawbydetectingandrecognizingwhatamountstogeneralprinciplesofinternationallaw’).DiegoMauri-9781802207675DownloadedfromPubFactoryat05/28/202203:48:48AMviaDukeUniversityLawLibrary

256248Autonomousweaponssystemsandtheprotectionofthehumanpersonbodiestorediscovertheregulatingroleofgeneralprinciples‘innewsitua-tions’—astheoneinvolvingweaponscapableofperformingcriticalfunctionswithouthumanintervention.51Thatis,ifanumberofstateswilltakeafirmandlegallysoundstanceagainstAWS,itisprobablethatinternationaladjudicatorybodieswillfollow.Bydevelopingcaselawonthisunprecedentedmatter,andbyresortingtolong-standingrulesandespeciallyprinciplesofIHRLandIHL,theymaycontribute—albeitindirectly—tofosterrespectforhumandignityandhumanity,inandoutofthebattlefield.6.2.2AHintofOptimism:MakingtheCaseforaTreatyRegulatingAWSInthepreviouschapterswedemonstratedthatAWSaresubjecttobothrulesandprinciplesofIHRL,andtheconceptsofhumandignityandhumanitystandas‘generalprinciplesformedwithintheinternationallegalorder’.Anunderstandingofthoseprinciplessothattheyimposehumaninterventionateachandeveryactofapplying(potentiallylethal)forceagainsthumantargets,however,doesnotcorrespondtowhatstatesandotheractorsinvolvedinthedebateonAWShavebeensupportingthusfar.Asaconfirmationofthis,onemayconsiderthateventhosewhostronglyopposeAWSdonotpushthem-selvestoarguingthatautonomousweaponryisillegal;rather,theyresorttoethicalormoralreasonsandthusmakeargumentsdelegeferenda.OuranalysisabovehasdemonstratedthatargumentsbasedontheillegalityofAWSneedtoberestyled,atleast,tosoundconvincing.Intheirmostessen-tialform,AWSareoftenregardedasillegalintheinternationallegalorderforthefollowingreasons:(1)theycannotliveuptorequirementsthatIHRLandIHLassertfortargetingindividualsinsideandoutsidethebattlespace;(2)incasesofmisdoings,accountabilitygapsareboundtoopenupasaresultoflackofhumaninterventionincriticaldecisions.Whileworthyofconsideration,bothobjectionstoAWScanberefuted.First,itcannotbeexcludedthatfuturedevelopmentswillrenderAWScapableofensuringacceptablestandardsofperformanceandreliability,incomparisonwithhumanagents(whomtheymayoutpaceeventually).Second,accountabilitygapscanbeavoidedoratleastsignificantlyreducedbothbyadjustingexistinglawandbyadoptingnewlawtofixsomeinconveniences.Thatis,ifthesearethebasesonwhichapro-hibitionofAWSissought,ultimatelytheargumentdoesnotseemconvincing.However,wepreviouslyarguedthatgeneralprinciplesplayarolefromadelegeferendaperspective.Apositionthathasrecentlyreceivedsupport51ReparationforInjuriesSufferedintheServiceoftheUnitedNations(n45)IndividualOpinionbyJudgeAlvarez190.DiegoMauri-9781802207675DownloadedfromPubFactoryat05/28/202203:48:48AMviaDukeUniversityLawLibrary

257Buildingthebridgetothefuture249bysomeactorsdeservesmentioning.TheICRChassubmittedapapercon-tainingproposalsandrecommendationsinrelationtothedevelopmentofthenormativeandoperationalframeworkforAWS,tobeconsideredduringthe2021GGE.52HavingacknowledgedtheethicalandlegalchallengesposedbyAWS,theICRCrecommendstheadoptionofalegallybindinginstrumentaimedtoruleout‘unpredictable’AWSandAWS‘thataredesignedorusedtoapplyforceagainstpersons’(thatis,AWSaccordingtothedefinitionadoptedinthisbook).Conversely,AWSthatcantargetonlynon-humantargetsandthatarenot‘unpredictable’havetobesubjecttobroadregulation.Thisisaground-breakingposition,asitmarksthefirsttimethattheICRChasendorsedabanonAWSinsuchastrongfashion—anoutcomethatcanbejustifiedbytheICRC’srecurringinterestinethicalimperatives,inlinewithbothitsgeneralmandateandpreviousinterventionsonAWSspecifically.53TimewilltellifthisdiplomaticmovewillsucceedinadvancingabindinginstrumentregulatingAWS.ThecircumstancethattheprohibitiononAWSisheldtohavefirmethicalroots,intandemwiththeinvocationofanewlegallybindinginstrument,speakstotherolethatgeneralprinciplesofinternationallawarebelievedtoplayinthismatter.Thisisapositioninlinewithotheractors,suchasNGOs,takingacriticalstancetowardsAWSandevenstates(mutatismutandis,theHolySee).Movinginevitablyfromareligiousperspective,theHolySeesupportstheprohibitiononAWSinasmuchasremovingthehumanpresencefromactslikelytoimpingesubstantiallyonhumanintegrity.54Inadditiontoensuringproperdistributionofresponsibilityincaseofmisdoings,thisunderstandingpreservestheinherentcoreofhumandignitybyarguingthateveryuseofforcethatdoesnotinvolvehumanpresenceinitsdeliberatingprocessrunsagainstthehumanperson.55Legally,thepremiseisthatgeneralprinciplesactas‘moralmodulators’or‘portals’throughwhichethicalimper-52ICRC,‘ContributionbytheInternationalCommitteeoftheRedCrossSubmittedtotheChairoftheConventiononCertainConventionalWeapons(CCW)GroupofGovernmentalExpertsonEmergingTechnologiesintheAreaofLethalAutonomousWeaponsSystemsasaProposalforConsensusRecommendationsinRelationtotheClarification,ConsiderationandDevelopmentofAspectsoftheNormativeandOperationalFramework’(11June2021).53ICRC,‘EthicsandAutonomousWeaponsSystems:AnEthicalBasisforHumanControl?’(3April2018).54DMauri,‘TheHolySee’sPositiononLethalAutonomousWeaponsSystems.AnAppraisalthroughtheLensoftheMartensClause’(2020)11JIHLS116.55DAmorosoandGTamburrini,‘TheEthicalandLegalCaseAgainstAutonomyinWeaponsSystems’(2018)18GlobalJurist1.DiegoMauri-9781802207675DownloadedfromPubFactoryat05/28/202203:48:48AMviaDukeUniversityLawLibrary

258250Autonomousweaponssystemsandtheprotectionofthehumanpersonativesaretranslatedinlegalterms.56Thisviewisparticularlyconvincing,asitdrawsfrombothlegalpositivismandnaturallawtheorieswithoutindulgingtheirmoreradicalaspirations.57Writtenlawshouldthereforefollowgeneralprinciples.Generalprinciplessuchasthoseunderscrutinyare‘recognized’and‘discovered’inexistingnorms(ofinternationallaw)throughanoperationthatistypicallycarriedoutbytheinterpreter.Thus,itmakessensethatthoseactorssupportingthisunderstandingofhumandignityandhumanitypushfora‘new’legallybindinginstrumentonAWS.Treatylawisexpectedtoputthoseprinciplesinwriting,literally.DifferentproposalsforatreatyonAWShavebeenputforwardtothisend.58ThatbyHumanRightsWatchandtheHarvardLawSchool’sInternationalHumanRightsClinic,forinstance,canbepraisedinthatitcontainsacon-sideredreflectiononhumandignityandafocuson‘positiveobligations’thatstatesmustcomplywith.59Otherkeypointsawaitingappropriateregulationaretheallocationofresponsibilityincasesofmisdoingand,inanyevent,theestablishmentofeffectivecompensatorymechanismsforvictims,capableoffixingexistingshortcomings.60ThedutytoadoptnationalmeasuresforlegallyreviewingAWS—stemmingfromart36APIandre-statedinGuidingprinciple(e)—couldbeinsertedinthelegallybindinginstrumenttoo,withaviewtofosteringinternationalcooperationandinformationsharingindevel-opingAI-relatedtechnologies.61Somestates,forinstance,Switzerland,haveadvancedproposalsbuiltupona‘compliance-based’approachtoAWS:statesshouldadoptnationalreviewingmechanismstakingintoaccountthepeculiar-itiesofAWS,andthussetupconfidence-buildingmechanismsandsharebestpractices,totheextentthatnationalsecurityinterestsaredulysatisfied.62TheCampaigntoStopKillerRobotscirculatedapaperdedicatedtothepossiblecontentofalegallybindinginstrument,builtaroundthreecoreexigencies:(1)56MEO’ConnellandCMDay,‘SourcesandtheLegalityandValidityofInternationalLaw.NaturalLawasSourceofExtra-PositiveNorms’,inSBessonandJd’Aspremont(eds),TheOxfordHandbookontheSourcesofInternationalLaw(OxfordUniversityPress:Oxford,2017).57SeeCh2,s2.3.3.58Foracomprehensiveoverview,seeDAmoroso,AutonomousWeaponsSystemsandInternationalLaw.AStudyonHuman-MachineInteractionsinEthicallyandLegallySensitiveDomains(EdizioniScientificheItaliane2020)217ff.59HumanRightsWatchandInternationalHumanRightsClinicatHarvardLawSchool,‘NewWeapons,ProvenPrecedent.ElementsofandModelsforaTreatyonKillerRobots’(October2020).60SeeCh5,s5.6.61SeeCh4,s4.2.4.62Switzerland,‘A“Compliance-Based”ApproachtoAutonomousWeaponsSystems’(10November2017)CCW/GGE.1/2017/WP.9paras20–21.DiegoMauri-9781802207675DownloadedfromPubFactoryat05/28/202203:48:48AMviaDukeUniversityLawLibrary

259Buildingthebridgetothefuture251toretainMHCoverAWS;(2)toprohibitAWSlackingMHC;and(3)toadoptasetofpositiveobligationstoensureMHC.63Importantly,notonlyNGOsorrepresentativesofcivilsocietyhaveadvancedproposalsofalegallybindinginstrument,butalsostateshavebeguntosubmitelementsforafuturetreatyonAWS.64We,too,haveadvancedsomeproposalsforafuturelegallybindinginstrument,forinstance,asregardstheneedforanabsoluteliabilityregimeofstateresponsibilityandthusfortheestablishmentofaneffectivemechanismforremedyingvictimsofmisdoingsthatdonotamounttoaninternationallywrongfulact.65Inconclusion,itnowseemsclearwhydelegeferendathoseactorssupporttheprohibition(oratleastthecomprehensiveregulation)ofAWS,andmoregenerally,whyinsistingonthecontentandthefunctionofgeneralprinciplesofinternationallawisarationalandforward-lookingstrategy.ThattheILCisadvancingitsworksongeneralprinciplesinparallelwiththeGGEisaluckycoincidence:onagenerallevel,thosetwoforaofdiscussionmarkarenewedinterestwithintheinternationalcommunityinextra-positivelegalsources.Whentechnologyadvancestothepointthatitcanseriouslyimpingeontherightsanddignityofthehumanpersonandexistingrulesseemunabletoregulateit,thecategoryofgeneralprinciples—shapedforapplyingtothe‘unusualcases’—turnsouttobeanindispensablecompass.66Whatismore,itisaphenomenonworthnotingthatsomeNGOsandrepresentativesofcivilsociety—fiercelydefendingasecularviewofsociety—donothesitatetoresorttoreligiousconcepts(suchasthatofmalainse)whendescribingautonomoustechnologies.67Whilejustifiedonthebasisoftherhetoricalappealthoseexpressionsmayhaveonthegeneralpublic,theseinvoluntaryreferencesshowtheneedtodiscoversolidprinciplesonwhichtogroundlegalrules.Acon-tinuingandconsideredreflectiononhumandignityandhumanityshouldbepursuednow—especiallynow.Thus,itisdiscouragingtoseethatthoseprinciples(onaparwithIHRLrules)areneglectedintheCCWdebateonAWS:onlyafewstatesinsiston63CampaigntoStopKillerRobots,‘KeyElementsofaTreatyonFullyAutonomousWeapons’(November2019).SeemoreextensivelyAmoroso(n57)256ff.64Brazil,ChileandMexico,‘ElementsforaFutureNormativeFrameworkConducivetoaLegallyBindingInstrumenttoAddresstheEthicalHumanitarianandLegalConcernsPosedbyEmergingTechnologiesintheAreaof(Lethal)AutonomousWeapons(LAWS)’(June2021);‘WorkingPapertobeSubmittedbytheBolivarianRepublicofVenezuelaonBehalfoftheNon-AlignedMovement(NAM)andOtherStatesPartiestotheConventiononCertainConventionalWeapons(CCW)’(June2021).65SeeCh5,s5.6.66Gradoni(n41).67SeereferencesinAmoroso(n57)162ff.DiegoMauri-9781802207675DownloadedfromPubFactoryat05/28/202203:48:48AMviaDukeUniversityLawLibrary

260252Autonomousweaponssystemsandtheprotectionofthehumanpersontheirimportanceandrelevanceforthesubject-matter.Theblossomingofstudiesdedicatedtotheroleofgeneralprinciples,evenfromacontemporarynatural-lawperspective,demonstratesthattheinternationalcommunityisinsearchofsolidgrounduponwhichtobuildtheregulationofnewtechnologies:AWSmaythusprovideanaptbenchtest.68ThisisevenmoreimperativeforthosewhoopposeAWSandwhomaydiscoverthe‘unforeseenpotential’inthislegalsource.69TheICRC’sendorsementofabanon‘anti-personnel’AWS,ifsupportedbysoundlegalarguments(asarethosebuiltongeneralprinciples),maystillnotsucceedinleadingtotheadoptionofatreatybanningAWS—evenifitdid,itcanbeexpectedthatmoststatesthathavethecapabil-ityofdevelopingthoseweaponswillnotsignnorratifyit,therebyrenderingthatinstrumentapplicablemoreonpaperthaninreality.Butthiswouldnotbetheendoftheissue.GeneralprinciplesmayindeedinfluencethedebateandinspirereflectionsgoingbeyondAWS,asthefollowingsectionexplains.6.3RE-APPRAISINGHUMANCONTROLBEYONDAWSMorethaneightyearsofdebatewithintheCCWframework—towhichthepreviousdiscussionattheHumanRightsCouncilandcollateralforashouldbeadded—witnesstothecurrenttopicalityofAWS.70Accordingtoarecentsub-missionbyBrazil,ChileandMexicototheGGE,a‘comprehensiveknowledgebaserepository’hasbeenamassedthroughtheyears.71Asalaststepinouranalysis,ithastobewonderedwhetherlegalreflec-tionsconductedwithregardtoAWScanbeextendedtoothertechnologicaldevelopmentsthatsharecommonalitieswiththeweaponsunderscrutiny.Thepurposeofthisconcludingsectionistwofold:toevaluatecurrenttechnologythat,toanextent,forerunsAWS,namely,armeddrones(6.3.1);andtolegallytestfuturedevelopmentsthatmaymagnifyissuesraisedbyAWS(6.3.2).OurhypothesisisthattheinsistenceontheneedforMHCinweaponssystems,fromtheoperationalstandpoint,coupledwiththereappraisalofgeneralprinci-plesaslegalsourcesrelevantforgoverningtechnologicaldevelopments,fromthelegalstandpoint,provideausefulcompasstonavigatethetroublingwatersofnewtechnologies.68MEO’Connell,TheArtofLawintheInternationalCommunity(CUP2019).69CTomuschat,‘ObligationsArisingforStatesWithoutTheirWill’(1993)241Recueildescours198,315.70SeeCh2,s2.1.71Brazil,ChileandMexico(n63).DiegoMauri-9781802207675DownloadedfromPubFactoryat05/28/202203:48:48AMviaDukeUniversityLawLibrary

261Buildingthebridgetothefuture2536.3.1HumansPressingtheButton:TheRealityofExisting‘Automated’SystemsRecallingourinitialremarks,armeddronescanbeconceivedas‘predecessors’ofAWS.72Theseweaponssystemsfeatureformsofhuman–machineinterac-tionsthathavebeenlabelled‘automated’(incontrastto‘autonomous’):humaninterventionintheperformanceofcriticalfunctionsisrequired.However,towhatextentishumaninterventionincurrentprocessesoftargeting‘meaning-ful’,astheconceptofMHCasdevelopedvis-à-visAWSwouldrequire?Targetinghasbecomeacomplexoperation(a‘process’ratherthanan‘action’),inwhichtechnologyplaysanincreasinglycriticalrole.73TheNorthAtlanticTreatyOrganization’sAlliedJointDoctrineforJointTargeting’providesanillustrativeexample.74Theprocessleadingtotheidentificationofatarget(‘targetdevelopment’)featuresseveralphasesandsub-phases,including:targetanalysis,targetvetting,targetvalidation,targetnominationandtargetprioritization.Inthephasededicatedtomissionplanningandforceexecution,anequallycomplexprocedureoccursthatiscapturedby‘find,fix,track,target,engage,exploit,assess’(F2T2E2A).75The82-pagedocumentreleasedbyNATOisself-explanatory.Insteadofprovidingathoroughanal-ysisoftheprocedure(whichfallsoutsidethescopenotonlyofthischapter,butalsoofthewriter’sexpertise),whatisinterestingtonotehereisthatthisproceduralsophisticationaimsatensuringbettertargeting—agoalwhichtechnologicaladvancementssupportinghumandecision-makingareintendedtopursue.Tothisend,recallthatthepracticeoftoday’sdronestrikesenvisagestwoconceptualmodelsoftargeting,namely,‘personalitystrikes’and‘signaturestrikes’.76Theformerreferstostrikestargetingknownindividuals(suchas‘terrorists’or‘Al-Qaedamembers’):thepersonalidentityoftheeligibletargetisthereforeknownbytheauthoritiestakingaction.Thelatter,onthecontrary,targetindividualsonthesolebasisoftheirobservedpatternofbehavior(thatis,‘signature’):individualsarescannedandcategorizedso72SeeCh1.73MACEkelhof,‘LiftingtheFogofTargeting:“AutonomousWeapons”andHumanControlthroughtheLensofMilitaryTargeting’(2018)71NavalWarCollegeRev61.74NATO,‘AlliedJointDoctrineforJointTargeting.EditionAVersion1’(April2016)AJP-3.9.75Ibid.76KBenson,‘“Kill’EmandSortItOutLater”:SignatureDroneStrikesandInternationalHumanitarianLaw’(2014)27PacificMcGeorgeGlobalBusiness&DevelopmentL17,34.DiegoMauri-9781802207675DownloadedfromPubFactoryat05/28/202203:48:48AMviaDukeUniversityLawLibrary

262254Autonomousweaponssystemsandtheprotectionofthehumanpersonthatthedecision-makersdisposeofallinformationnecessarytodeterminetheireligibilityastargets.However,thereismountingevidenceshowingthatsignaturestrikescauseasignificantnumberofimpermissiblecasualties.77Mistargetingcoulddependonpoordataorinadequatere-elaborationbythesystem.Oncloserinspection,giventhatthefinaldecisionrestswithahumanoperator,the‘bad’decisioniscausedbyacombinationofthesystem’serrorandtheoperator’sinabilitytodetectsucherror.Oneofthemainrisksassoci-atedwithhuman–machineinteractionisthe‘automationbias’(orautomationcomplacencyorover-trust),thatis,thepropensityforhumanstoover-relyoncomputer-generatedsolutions.78Otherrelevantrisksareunder-trust(theoppositeofautomationbias)andout-of-the-loopcontrolproblems,whichincreasethemoreasystemoperatesautonomously.79Therefore,humanagentsarethemselves(asforautonomoussystems)pronetoalegionoferrors.Onthispremise,ascenarioinwhichahumanoperatortakesadecisionbasedoncomputer-generatedsolutionsthatheorsheisnotabletodealwithproperly(the‘armeddrone’scenario)andinwhichthemachineitselfperformsthecriticaldecisionunderhumansupervision(the‘autonomousweapons’scenario)donotseemsodivergent.Inbothcases,humancontrol(intheformofdirectinterventionintheformerscenario,andintheformofsupervisioninthelatter)maybeonlynominalinpractice.80Insummary,there-discoveryofgeneralprinciplesofhumandignityandhumanityandtheinsistenceontheneedformeaningfulhumancontrolcouldbringabouta‘positiveexternality’onthehuman–machineinteractionscur-rentlyexistinginthefieldofarmeddrones.Operatively,thetargetingprocesscouldbeadjusted—throughtheadoptionofbestpracticesorguidelines—inordertoensurethathumanoperatorsunderstand(and,ifneedbe,contest)thecomputer-generatedsolution.Normatively,thisformofhumancontrolmayensurebettercompliancewithprimaryobligationsofIHRLandIHL,andthusreducethenumberofcasualties.6.3.2HumansBeingtheButton:EnhancementandBrain–ComputerInteractionsMovingfromexistingformsofhuman–machineinteractionstofuturecom-binations,the(yetlargelyunexplored)fieldofhumanenhancementwillraise77Ibid;forfurtherdata,seehttps://www.thebureauinvestigates.com/projects/drone-waraccessed31August2021.78MLCummings,‘AutomationBiasinIntelligentTimeCriticalDecisionSupportSystems’(September2004)AIAA1stIntelligentSystemsTechnicalConference1.79Boulaninetal(n30)19.80SeealsoEkelhof,‘LiftingtheFogofTargeting’(n72)80.DiegoMauri-9781802207675DownloadedfromPubFactoryat05/28/202203:48:48AMviaDukeUniversityLawLibrary

263Buildingthebridgetothefuture255challengingissues.Humanenhancement(oraugmentation)canbedefinedastheprocessendowinghumanagents(forthepurposesofouranalysis,soldiersoragents)withempoweredcapabilitiesthrougheitherexternaldevicesorbodilyintegration.81Enhancementmaythustakedifferentforms:biochemical(throughtheuseofdrugsorotherchemicalorbiologicalagents),prosthetic(throughtheimplantationofdevices)andcybernetic(featuringbrain–machineinterfacestoconnecttheagent’sbraindirectlytoamachinewithouttheneedformanualinputs).Cyberneticenhancementisofparticularinterestforourdiscourse.Itreliesoninterfacesusingelectrodestocollectbrainsignalsandsendthemtocomputer-controlleddevices.Farfrombeingascience-fiction,futuristicdevel-opment,brain-controlleddevicesinthemilitaryfieldhavealreadybeentestedintherealworld.Forinstance,in2016thefirstmind-controlleddroneracewasorganizedinFlorida.82Reportedly,brain–computerinterface(BCI)technolo-gieswillsoonbeemployedinmilitarycombat:theiranticipatedoperationalutilityrequiresacarefulexaminationofpolicyconsiderationsandsocialimpli-cations.83Theadvantagesofthistechnologyareevident:collectionoflargeramountsofdata;increasedspeedindecision-making;andmoreefficientover-sightofgreatnumbersofdevices(‘swarms’).Asforthelatter,thedifficultyincoordinatingnumeroussystemswithinanetworkwasoneofthecriticalpointsraisedbyautonomyinthebattlefield:BCImayprovideaworkingsolutiontothis,notbyremovinghumanpresencebutbyre-shapingit.Inadditiontolegalchallenges,enhancementtechnologiesraiseethicalquestions.Developmentsinthisdirectionareseenasanattemptto‘robotize’humanagents,inoppositiontotheprogressiveincreaseinautonomy(whichcanberegardedasanattemptto‘humanize’machines),asonerecentdoc-umentpreparedbytheHolySeeargues.84AsregardsIHRLandIHL,therulesandprinciplesillustratedabovearefullyapplicable:developmentoftheenhancementdevicesissubjecttoweaponslaw,andenhancedsoldiersandagentsmustcomplywithtargetinglaw.Internationalresponsibilitywillfollowinaccordancewithapplicablenorms.Yet,sincehumanagentsreceive81HAHarrisonDinnissandJKKleffner,‘Soldier2.0:MilitaryHumanEnhancementandInternationalLaw’(2016)92Int’lLStud432.82MMurphy,‘Thefirstbrain-controlleddroneracejusttookplaceinFlorida’,Quartz(25April2016)https://qz.com/669720/the-first-brain-controlled-drone-race-just-took-place-in-florida/accessed31August2021.83ABinnendijk,TMarlerandEMBartels,‘Brain-ComputerInterfaces.U.S.MilitaryApplicationsandImplications.AnInitialAssessment’,RANDReport(2020).84AdeLaRochefoucauldandSSaldi(eds),‘TheHumanizationofRobotsandtheRobotizationoftheHumanPerson.EthicalReflectionsonLethalAutonomousWeaponsSystemsandAugmentedSoldiers’(workingpaperbytheCaritasInVeritateFoundation).DiegoMauri-9781802207675DownloadedfromPubFactoryat05/28/202203:48:48AMviaDukeUniversityLawLibrary

264256Autonomousweaponssystemsandtheprotectionofthehumanpersonaparticulartreatment,anothersetoflegalhurdlesweighsontheinterpreter:namely,thehumanrightsoftheenhancedsoldieroragent.85Theirbasicrightstolife,bodilyintegrity,andprivacy,thoughtandexpression,maybeimpingedbytheenhancementtechnology,especiallywhenthiscanalterthebrain’sabilitytomakefreechoices.Theissueoftheenhancedhumanagents’rightsiscloselyconnectedwiththeirowndignityashumanpersons:thatis,towhatextentareenhancementtreatments—inparticularthosethatcansignificantlyimpingeonphysicalandpsychiccapabilities—compatiblewiththenotionof‘humandignity’?Inthis,humandignityshouldbeconceivedmoreasaprinciplelimitinganindividual’spowerovertheirownperson.Whetheran‘objective’conceptofhumandignitycorrespondswithageneralprincipleofinternationallaw,andmorepreciselyofIHRL,hasbeenquestionedbysomeauthors.86Inconclusion,inadditiontothetarget’sdignityandhumanity(asencapsu-latedinthegeneralprinciplesillustratedpreviously),onemustnotlosesightoftheforceappliers’dignityandhumanity.Theanticipatedadventofmoreintertwinedhuman–machineinteractions,andofenhancementtechnologies,shouldmarkthebeginningofaseriousandthoughtfulreflectionofhumandignityandhumanityandtheirpotentialasgeneralprinciplesofinternationallaw.Tothisend,thedebateonAWSmaypavethewayforward.6.4DROPPINGANCHOR:WHATCANINTERNATIONALLAWDO?TheapproachtoAWSthatthisbookhasfollowedhasbeensolelylegal:wetestedthewaytheywillforeseeablyoperatethroughthelensesofIHRLandIHLandthelawofinternationalresponsibilitysoastoassesstheirimpactontheprotectionofthehumanperson.Letusnowreviewthekeyissuesexploredacrossthechapters.First,whatinterestsforourlegalanalysisistherealityof‘human–machineinteractions’,whichiswhatthevariousdefinitionsof‘autonomy’,‘criticalfunctions’or‘humancontrol’subtend.Thestudyofhowandtowhatextenthumanagentsandautonomoussystemscaninteract(orteamwithoneanother)goeswellbeyondthetopicofAWS,asitisrelevantfornumerousfieldsofcurrentandfuturesocieties.Theconceptofhuman–machineinteractionsisparticularlyusefulforouranalysis,asitallowsforabetterunderstandingof85DinnisandKefflner(n80)455ff.86PDeSena,‘Dignitàumanainsensooggettivoedirittointernazionale’(2017)11Dirittiumaniedirittointernazionale573;SHennette-Vauchez,‘AHumanDignitas?RemnantsoftheAncientLegalConceptinInternationalDignityJurisprudence’(2011)9Int’lJofConstLaw32.DiegoMauri-9781802207675DownloadedfromPubFactoryat05/28/202203:48:48AMviaDukeUniversityLawLibrary

265Buildingthebridgetothefuture257existingtechnologies(suchasarmeddrones)andanticipatingfuturetech-nologies(suchashumanenhancementinitsvariousforms).TheconceptofMHC,whilevoidofindependentlegalstatus,standsasatoolforcatalyzingdiplomaticpositionsandoperationalizinglegalrequirements.Second,theincreasinglynumerousandcomplexformsofinteractionsbetweenhumansandmachineshavebeentestedthroughthenormsprotectingthehumanpersonintheinternationallegalorder,thatis,IHRLandIHL.Traditionally,thesebranchesofinternationallawhavebeenconcernedwiththewayinwhichforceisapplied:wasaparticularuseofforcediscriminatoryandproportionate?Haveprecautionarymeasuresbeentaken?Canresponsi-bilitybeallocatedadequately,ifAWSmisfire?Ourjourneythroughrelevantprimaryandsecondaryobligationsofinternationallawhasdemonstratedthat,ifthis,andonlythis,iswhatrendersAWS‘abhorrent’weapons,theirbanwouldbeunjustified:itwouldbesufficienttoadjustandrevisitexistingnorms,throughproperinterpretivetechniques.Totheextentthathumanoper-atorscanunderstandandexplainthecoursesofactionofAWS,compliancewithIHRLandIHLruleswillbeensured.87Totheextentthat,ifthingsgowrongandmisdoingsoccur,responsibilitycanbeallocatedappropriatelyandeffectivemechanismsforreparationsareinplace,andcompliancewiththelawofinternationalresponsibilitywillbeensured.88Third,whilethissoundsentirelyreasonable,ithasbeenarguedthatwhatcharacterizesthesesystemsisthataparticulardecisionoverthereleaseof(potentiallylethal)forceagainstindividualsmaybedeliberatedabsenthumanintervention:ahumanoperator,whileremainingsomewhereintheloopandthus‘incontrol’,maynotstepintotheprocess.Isthisanydifferentfromcurrentandpasttoolsdesignedtoapplyforceagainstindividuals?DoesthishaveanimpactonusesofforceagainstpermissibletargetsaccordingtoIHRLandIHLrules?Inourview,inprincipletheanswerisaffirmative:totheextentthatAWSpresenttheinterpreterwiththisuniquefeaturethatexistingrulesfailtocapture(thatis,thefocusshiftsfromthewayforceisapplied,towhoorwhatappliestheforce),resorttothecategoryofprinciplesislegallysound.Generalprinciplesofinternationallaw(formedwithintheinternationallegalorder),namely,thoseofhumandignityandhumanity,canthusbeinvokedinrespectofAWS.However,canitbesaidthatthoseprinciplesareunderstoodasrequiringhumanpresenceateachandeveryapplicationofforceagainsthumantargets?ThedebateattheGGEdoesnotwarrantthisconclusion:eventhosewhomorefiercelyopposeAWSbasetheirargumentsontheir(alleged)failuretocomplywithinternationallawandthe‘accountabilitygaps’ensuing87SeeChs3and4.88SeeCh5.DiegoMauri-9781802207675DownloadedfromPubFactoryat05/28/202203:48:48AMviaDukeUniversityLawLibrary

266258Autonomousweaponssystemsandtheprotectionofthehumanpersonfromtheiruse.Norcaninternationalcaselawbeinvokedtothisend.Thatis,thecontainer(generalprinciples)fitswiththecaseathand,butnotthecontent(humandignityandhumanityarenotunderstoodasprohibitingAWS).Inthewriter’sview,thisshouldbetakenasademonstrationthat,atleasttoday,thesegeneralprinciplesdonotproscribeAWS;instead,theyrequirethatanyhuman–machineinteractionsinthissensitivefieldensurerespectforthecorevaluestheyencapsulate.Theymaythusdrivetheinternationalregulationofnext-to-comeAI-driventechnologies:insummary,theyarenotuseless,noristheirroleinsignificant.Finally,wediscussedtheopportunityofalegallybindinginstrumentonAWS.AnadhoctreatywouldfacilitatetheregulationofAWSbyfixingsomeinconveniencesinexistinglaw.Ataminimum,thisnewlegalinstrument—intheformofasixthadditionalprotocoltotheCCW89orasanindependentlynegotiatedtreatyàlaConventiononClusterMunitions90—shouldcontainrulesontheallocationofstateandcorporateresponsibilityincasesofmisdoingsofAWS,byadoptingabsoluteliabilityregimesandremovingexistingobstaclestowardsvictims’reparation.Ideally,thetreatyshouldalsocontainmecha-nismsforcooperationbetweenstates,exchangesofinformationandsharingof‘bestpractices’forpeacefulpurposes,inlinewiththatwhichisprovidedbyothertreatiesinthefieldofarmscontrolanddisarmament.91Moreover,atreatywouldprovideelementsforoperationalizingthelong-discussedconceptofMHC,whichwillbekeyincopingwithfuturedevelopments(forinstance,inthefieldofcyberneticenhancement).Insummary,intimeswheretechnologymaysubstantivelyaltertherelationshipbetweenhumanagentsandmachinesandprogressivelyremovetheformerfromcriticaldecisions,todisposeofalegallysoundconceptof‘humancontrol’isvital.ThisbooksuggeststhatashiftfromtheperspectiveofbanningAWStogoverntheminkeepingwithrulesandprinciplesofIHRLandIHLandthelawofinternationalresponsibilitymaybewelcomed.InsteadofaskingwhetherAWSlackingnarrowMHCarelawfulornotunderinternationallaw,ourapproachhasbeentoquestionthedegreetowhichhumancontrolhastobemaintainedovercriticalfunctionstoensuretheproperprotectionofthehumanperson,whichisatthecoreofcontemporaryinternationallaw.Thatis,89Brazil,ChileandMexico(n63).90(Adopted3December2008,enteredintoforce1August2010)2688UNTS39.91ConventionontheProhibitionoftheDevelopment,ProductionandStockpilingofBacteriological(Biological)andToxinWeaponsandontheirDestruction(adopted10April1972,enteredintoforce26March1975)1015UNTS163(BWC)artX;ConventionontheProhibitionoftheDevelopment,Production,StockpilingandUseofChemicalWeaponsandontheirDestruction(adopted13January1993,enteredintoforce29April1997)1974UNTS45(CWC)artXI.DiegoMauri-9781802207675DownloadedfromPubFactoryat05/28/202203:48:48AMviaDukeUniversityLawLibrary

267Buildingthebridgetothefuture259thepointisnotwhetherAWSare‘inevitable’or‘evitable’—asarguedintheintroduction,theyareonestepforwardinthe‘evolutionarypattern’oftoolsforapplyingforceagainstindividuals—butinsteadwhatinternationallawcandotoruleovertheserapidchanges.92Thenumerousproposalsfor(eitherbindingornon-binding)instrumentsregulatingAWSdemonstratethatanincreasingnumberofactorsinvolvedinthedebatefeeltheneedtogetpreparedforunprecedentedchangesinhowforcecanbeusedagainstthehumanperson.Whilepreparingtodrafttheselastconclusions,aquotationpoppedupasatypeofconfirmation.Inthewell-knownSunTzu’sTheArtofWar,itiswrittenthat‘thegeneralwhowinsthebattlemakesmanycalculationsinhistemple’.93Ifoneconsidersthatalgorithmsareprocessestobefollowedincal-culations,therecentturntoAI-poweredwarfareappearstoencapsulateanideathatdatesbackmorethantwomillennia.Beingabletocalculatetheinfinitevariablesaccompanyingmilitaryandpoliceoperationsprovidesaclearadvan-tageinefficiency.Thishasremainedconstantthroughthecenturies,tothepointthatcurrentdevelopmentsdobutmagnifyancienttrends.Internationallawcannotberequestedtodomorethanwhatisalreadyinitspowertodo:whenitcomestonewtechnologies,itshouldbeableatleasttoprovideeffectiveregulationtoturbulentchanges—inthecaseofAWS,tostrivetoprotect,tothebestofitscapabilities,thehumanpersonfromunprecedented,impersonalandtroublingformsofusingforceagainstindividuals.Whilethismayseemfrustratingtosome,itshouldnotdistractfromthefactthatIHRLandIHL,withalltheirlimits,haveasay.The(re-)discoveryofgeneralprinciplesencapsulatingcorevaluesoftheinternationallegalorder,combinedwithaprincipledeffortdelegeferendaonthepartofallactorsinvolvedinthedebate(notjuststates),iswhatinterna-tionallawcanofferinthesecriticaltimes.94Thisidealtensionmaynotsoundlikemuch,butitisatleastacompasstorelyonwhennavigatingunknown,unchartedwaters.92SeeCh1.93SunTzu,TheArtofWar(transGLionel)(IxiaPress2019)32(emphasisadded).94Wefirmlyendorsetheideawherebyinternationallawyersshouldcombineanal-ysisoflexlatawithproposalsoflexferenda—yetcarefullykeepingthoseperspec-tivesdistinct.SeeACassese,‘Soliloquy’,inACassese,TheHumanDimensionofInternationalLaw:SelectedPapersofAntonioCassese(PGaetaandSZappalàeds,OUP2008).DiegoMauri-9781802207675DownloadedfromPubFactoryat05/28/202203:48:48AMviaDukeUniversityLawLibrary

268BibliographyAbaimov,SandMartellini,M,‘ArtificialIntelligenceinAutonomousWeaponSystems’,inMMartelliniandRTrapp(eds),21stCenturyPrometheus(Springer2020).Abi-Saab,G,‘TheSpecificitiesofHumanitarianLaw’,inCSwinarski(ed),StudiesandEssaysonInternationalHumanitarianLawandRedCrossPrinciplesinHonourofJeanPictet(MartinusNijhoff1984).Adams,TK,‘FutureWarfareandtheDeclineofHumanDecisionmaking’(2001)31TheUSArmyWarCollegeQuarterlyParameters57.Akande,D,‘ClassificationofArmedConflicts’,inBSaulandDAkande(eds),TheOxfordGuidetoInternationalHumanitarianLaw(OUP2020).Akerson,D,‘TheIllegalityofOffensiveLethalAutonomy’,inDSaxon(ed),InternationalHumanitarianLawandtheChangingTechnologyofWar(Brill2013).Alexander,A,‘AShortHistoryofInternationalHumanitarianLaw’(2015)26EJIL109.Alexy,R,‘OnBalancingandSubsumption.AStructuralComparison’(2003)16RatioJuris433.Alexy,R,‘OntheStructureofLegalPrinciples’(2000)13RatioJuris294.Amoroso,D,‘JusinbelloandjusadbellumArgumentsAgainstAutonomyinWeaponsSystems:ARe-Appraisal’(2017)43QuestofInt’lL5.Amoroso,D,AutonomousWeaponsSystemsandInternationalLaw.AStudyonHuman-MachineInteractionsinEthicallyandLegallySensitiveDomains(EdizioniScientificheItaliane2020).Amoroso,DandGiordano,B,‘WhoIstoBlameforAutonomousWeaponsSystems’Misdoings?’,inECarpanelliandNLazzerini(eds),UseandMisuseofNewTechnologies.ContemporaryChallengesinInternationalandEuropeanLaw(Springer2019).Amoroso,DandTamburrini,G,‘TheEthicalandLegalCaseAgainstAutonomyinWeaponsSystems’(2018)18GlobalJurist1.Amoroso,DandTamburrini,G,‘WhatMakesHumanControloverWeaponsSystems“Meaningful”?’,ICRACReport(August2019).Anderson,K,Reisner,DandWaxman,M,‘AdaptingtheLawofArmedConflicttoAutonomousWeaponSystems’(2014)90Int’lLStud386.Arkin,RC,‘TheCaseforEthicalAutonomyinUnmannedSystems’(2010)9JofMilitaryEthics332.Asaro,P,‘OnBanningAutonomousWeaponSystems:HumanRights,Automation,andtheDehumanizationofLethalDecision-Making’(2012)94IRRC687.Asaro,P,‘JusNascendi:RoboticWeaponsandtheMartensClause’,inRCalo,AMFroomkinandIKerr(eds),RobotLaw(EdwardElgar2016).Bagaric-Allan,M,‘TheVacuousConceptofDignity’(2006)5JofHumanRights257.Barak,A,HumanDignity.TheConstitutionalValueandtheConstitutionalRight(CUP2015).260DiegoMauri-9781802207675DownloadedfromPubFactoryat05/28/202203:48:50AMviaDukeUniversityLawLibrary

269Bibliography261Barboza,J,‘InternationalLiabilityfortheInjuriousConsequencesofActsnotProhibitedbyInternationalLawandProtectionoftheEnvironment’(1994)247Recueildescours301.Bartels,R,vandenBoogard,J,Ducheine,PAL,Pouw,EandVoetelinkJ(eds),MilitaryOperationsandtheNotionofControlUnderInternationalLaw.LiberAmicorumTerryD.Gill(Springer2021).Bartolini,G,Riparazioneperviolazionedeidirittiumanieordinamentointernazionale(Jovene2009).Bassiouni,MC,‘AFunctionalApproachto“GeneralPrinciplesofInternationalLaw”’(1990)11MichiganJofInt’lL768.Batesmith,A,‘CorporateCriminalResponsibilityforWarCrimesandOtherViolationsofInternationalHumanitarianLaw:TheImpactoftheBusinessandHumanRightsMovement’,inCHarvey,JSummersandNDWhite(eds),ContemporaryChallengestotheLawsofWar.EssaysinHonourofProfessorPeterRowe(CUP2014).Beard,JM,‘AutonomousWeaponsandHumanResponsibilities’(2014)45GeorgetownJofInt’lL617.Beck,S,‘TheProblemofAscribingLegalResponsibilityintheCaseofRobotics’(2016)31AI&Society473.Beer,Y,‘HumanityConsiderationsCannotReduceWar’sHazardsAlone:RevitalizingtheConceptofMilitaryNecessity’(2016)26EJIL801.Bekey,G,AutonomousRobots.FromBiologicalInspirationtoImplementationandControl(MITPress2005).Benson,K,‘“Kill’EmandSortItOutLater”:SignatureDroneStrikesandInternationalHumanitarianLaw’(2014)27PacificMcGeorgeGlobalBusiness&DevelopmentLJ17.Benvenuti,P,‘LaClausolaMartenselatradizionestoricadeldirittonaturalenellacodificazionedeldirittodeiconflittiarmati’,inGBarile(ed),ScrittidegliallieviinmemoriadiGiuseppeBarile(CEDAM1995).Bergen,PLandRowland,J,‘WorldofDrones:TheGlobalProliferationofDroneTechnology’,inPLBergenandDRothenberg(eds),DroneWars:TransformingConflict,LawandPolicy(CUP2015).Binnendijk,A,Marler,TandBartels,EM,‘Brain-ComputerInterfaces.U.S.MilitaryApplicationsandImplications.AnInitialAssessment’,RANDReport(2020).Blake,DandImburgia,JS,“‘BloodlessWeapons”?TheNeedtoConductLegalReviewsofCertainCapabilitiesandtheImplicationsofDefiningThemas“Weapons”’(2010)66TheAirForceLR157Bo,M,‘AutonomousWeaponsandtheResponsibilityGapinLightoftheMensReaoftheWarCrimeofAttackingCiviliansintheICCStatute’(2021)19JICJ1(forthcoming).Bonafé,B,‘CommandResponsibility’,inACassese(ed),TheOxfordCompaniontoInternationalCriminalJustice(OUP2009).Bonafé,BandPalchetti,P,‘RelyingonGeneralPrinciplesinInternationalLaw’,inCBrölmannandYRadi(eds),ResearchHandbookontheTheoryandPracticeofInternationalLawmaking(EdwardElgar2016).Bonfanti,A(ed),BusinessandHumanRightsinEurope.InternationalLawChallenges(Routledge2019).Boothby,W,WeaponsandtheLawofArmedConflict(OUP2009).Boothby,W,‘AutonomousAttack–OpportunityorSpectre?’(2013)16YearbookofInternationalHumanitarianLaw71.DiegoMauri-9781802207675DownloadedfromPubFactoryat05/28/202203:48:50AMviaDukeUniversityLawLibrary

270262AutonomousweaponssystemsandtheprotectionofthehumanpersonBoothby,W,‘DeceptionintheModern,CyberBattlespace’,inJDOhlin,KGovernandCFinkelstein,CyberWar:LawandEthicsforVirtualConflicts(OUP2015).Boulanin,VandVerbruggen,M,‘Article36Reviews.DealingwiththeChallengesPosedbyEmergingTechnologies’,SipriReport(December2017)Boulanin,V,Davinson,N,Goussac,NandCarlsson,MP,‘LimitsonAutonomyinWeaponSystems.IdentifyingPracticalElementsofHumanControl’,SipriandICRCReport(June2020).Boyle,AE,‘StateResponsibilityandInternationalLiabilityforInjuriousConsequencesofActsnotProhibitedbyInternationalLaw:ANecessaryDistinction?’(1990)39ICLQ1.Boyle,CK,‘TheConceptofArbitraryDeprivationofLife’,inBGRamcharan(ed),TheRighttoLifeinInternationalLaw(MartinusNijhoff1985).Brenneke,M,‘LethalAutonomousWeaponSystemsandTheirCompatibilitywithInternationalHumanitarianLaw:APrimerontheDebate’(2020)21YearbookoftheInternationalHumanitarianLaw59.Brownlie,I,SystemofLawofNations:StateResponsibilityPartI(OUP1983).Brownsword,R,‘HumanDignityfromaLegalPerspective’,inMDüwell,JBraarvig,RBrownswordandDMieth(eds),TheCambridgeHandbookofHumanDignity(CUP2014).Bryson,JJ,Diamantis,MEandGrant,TD,‘Of,For,andBythePeople:TheLegalLacunaofSyntheticPersons’(2017)25ArtificialIntelligenceL273.Buchan,RandTsagourias,N,‘AutonomousCyberWeaponsandCommandResponsibility’(2020)96Int’lLStud645.Burke,MandPersi-Vicentic,P,‘RemediesandReparations’,inSCasey-Maslen,WeaponsunderInternationalHumanRightsLaw(CUP2014).Buscemi,M,IllecitidelleNazioniUniteetuteladell’individuo(EditorialeScientifica2020).Buzatu,M,TowardsanInternationalCodeofConductforPrivateSecurityServiceProviders:AViewfromInsideaMultistakeholderProcess(UbiquityPress2018).Cameron,PJ,‘TheLimitationsonMethodsandMeansofWarfare’(1985)9AustralianYearbookofInternationalLaw241.Camins,EL,‘NeedsorRights?ExploringtheLimitationsofIndividualReparationsforViolationsofInternationalHumanitarianLaw’(2016)10Int’lJofTransitionalJustice126.CançadoTrindade,AA,InternationalLawforHumankind–TowardsaNewJusGentium(MartinusNijhoff2010).CançadoTrindade,AA,‘SomeReflectionsonthePrincipleofHumanityinItsWideDimension’,inRKolbandGGaggioli(eds),ResearchHandbookonHumanRightsandHumanitarianLaw(EdwardElgar2013).Cannizzaro,V,‘ProportionalityintheLawofArmedConflict’,inAClaphamandPGaeta(eds),TheOxfordHandbookofInternationalLawinArmedConflict(OUP2014).Cannizzaro,V,‘BalancingTest:InternationalCourtofJustice(ICJ)’,inRWolfrum(ed),MaxPlanckEncyclopediaofInternationalProceduralLaw(OUP2019).Cannizzaro,V,Ilprincipiodiproporzionalitànell’ordinamentointernazionale(Giuffré2000).Carozza,P,‘HumanDignity’,inDShelton(ed),TheOxfordHandbookofInternationalHumanRightsLaw(OUP2013).DiegoMauri-9781802207675DownloadedfromPubFactoryat05/28/202203:48:50AMviaDukeUniversityLawLibrary

271Bibliography263Carpanelli,E,‘GeneralPrinciplesofInternationalLaw:StrugglingwithaSlipperyConcept’,inLPineschi(ed),GeneralPrinciplesofLaw–TheRoleoftheJudiciary(Springer2015).Carpanelli,E,‘TheRoleofCorporationsasStandardSetters:TheCaseofBusinessActorsInvolvedintheDevelopmentandDeploymentofArtificialIntelligenceTools’,inMBuscemi,NLazzerini,LMagiandDRusso(eds),LegalSourcesinBusinessandHumanRights.EvolvingDynamicsinInternationalandEuropeanLaw(Brill2020).Casey-MaslenS,‘Weapons’,inBSaulandDAkande(eds),TheOxfordGuidetoInternationalHumanitarianLaw(OUP2020).Casey-Maslen,S(ed),WeaponsunderInternationalHumanRightsLaw(CUP2014).Cassella,S(ed),Lestandarddeduediligenceetlaresponsabilitéinternationale(Pedone2018).Cassese,A,‘WeaponsCausingUnnecessarySuffering:AreTheyProhibited?’(1975)58RivistaItalianadiDirittoInternazionale12.Cassese,A,‘TheProhibitionofIndiscriminateMeansofWarfare’,inBVARöling,RAkkerman,PJVanKriekenandCOPannenborg(eds),DeclarationsonPrinciples:AQuestforUniversalPeace(MartinusNijhoff1977).Cassese,A,‘TheMartensClause:HalfaLoaforSimplyPieintheSky?’(2000)11EJIL187.Cassese,A,‘TheProperLimitsofIndividualResponsibilityundertheDoctrineofJointCriminalEnterprise’(2007)5JICJ109.Cassese,A,‘Soliloquy’,inACassese,TheHumanDimensionofInternationalLaw:SelectedPapersofAntonioCassese(PGaeta,andSZappalàeds,OUP2008).Cassese,A,Cassese’sInternationalCriminalLaw(3rded,OUP2013).Chabert,J,(ed),AHistoryofAlgorithms:FromthePebbletotheMicrochip(Springer1999).Chamayou,G,Théoriedudrone(LaFabriqueEditions2013).Chehtman,A,‘TheAdBellumChallengeofDrones:RecalibratingPermissibleUseofForce’(2017)28EJIL173.Cheng,B,GeneralPrinciplesofLaw,asAppliedByInternationalCourtsandTribunals(CUP1953,repr1987).Chengeta,T,‘AccountabilityGap:AutonomousWeaponSystemsandModesofResponsibilityinInternationalLaw’(2016)45DenverJInt’lLandPol1.Chengeta,T,‘AreAutonomousWeaponSystemstheSubjectofArticle36ofAdditionalProtocolItotheGenevaConventions?’(2016)23JofInt’lL&Pol77.Chengeta,T,‘Dignity,Ubuntu,HumanityandAutonomousWeaponSystems(AWS)Debate:AnAfricanPerspective’(2016)13BrazilianJofInt’lL461.Chengeta,T,‘DefiningtheEmergingNotionof“MeaningfulHumanControl”inWeaponSystems’(2017)49NewYorkUniversityInt’lLandPol833.Christakis,T,‘Les“circonstancesexcluantl’illicéité”:uneillusionoptique?’,inOCorten,NAngelet,EDavidandPKlein(eds),Droitdupouvoir,pouvoirdudroit:MélangesoffertsàJeanSalmon(Bruylant2007).Clapham,A,‘TheComplexRelationshipBetweentheGenevaConventionsandInternationalHumanRightsLaw’,inAClapham,PGaetaandMSassoli(eds),The1949GenevaConventions.ACommentary(OUP2015).Clark,OG,Kok,RandLacroix,R,‘MindandAutonomyinEngineeredBiosystems’(1999)12EngineeringApplicationsofArtificialIntelligence389.DiegoMauri-9781802207675DownloadedfromPubFactoryat05/28/202203:48:50AMviaDukeUniversityLawLibrary

272264AutonomousweaponssystemsandtheprotectionofthehumanpersonCondorelli,L,‘Laréparationdesdommagescatastrophiquescausésparlesactivitésspatiales’,inLaréparationdesdommagescatastrophiques.TravauxdesXIIIeJournéesd’étudesjuridiquesJeanDabin(Bruylant1990).Condorelli,LandKress,C,‘TheRulesofAttribution:GeneralConsiderations’,inJCrawford,APelletandSOlleson(eds),TheLawofInternationalResponsibility(OxfordOUP,2010).Conforti,B,‘Humanitéetrenouveaudelaproductionnormative’,inHumanitéetdroitinternational.MélangesRenéJeanDupuy(Pedone1991).Conforti,BandIovane,M,Dirittointernazionale(12thed,EditorialeScientifica2021).Coppin,GandLegras,F,‘AutonomySpectrumandPerformancePerceptionIssuesinSwarmSupervisoryControl’(2012)100ProceedingsoftheIEEE590.Corn,GS,Blank,LR,Jenks,CandTalbotJensen,E,‘BelligerentTargetingandtheInvalidityofaLeastHarmfulMeansRule’(2013)89Int’lLStud536.Coupland,R,‘Humanity:WhatItIsandHowDoesItInfluenceInternationalLaw?’(2001)83IRRC969.Coupland,RandHerby,P,‘ReviewoftheLegalityofWeapons:ANewApproach.TheSIrUSProject’(1999)81IRRC583.Crawford,E,‘ThePrincipleofDistinctionandRemoteWarfare’,inJDOhlin(ed),ResearchHandbookofRemoteWarfare(EdwardElgar2017).Crawford,J,StateResponsibility.TheGeneralPart(CUP2013).Crootof,R,‘TheKillerRobotsAreHere:LegalandPolicyImplications’(2015)36CardozoLR1837.Crootof,R,‘AMeaningfulFloorfor“MeaningfulHumanControl”’(2016)30TempleInt’landCompLJ53.Crootof,R,‘WarTorts:AccountabilityforAutonomousWeapons’(2016)164UniversityofPennsylvaniaLR1347.Crootof,R,‘AutonomousWeaponSystemsandtheLimitsofAnalogy’(2018)9HarvardNatl’lSecJ51.Cryer,R,‘WarCrimes’,inBSaulandDAkande(eds),TheOxfordGuidetoInternationalHumanitarianLaw(OUP2020).Cryer,R,Friman,H,Robinson,DandWilmshurst,E,AnIntroductiontoInternationalCriminalLawandProcedure(3rded,CUP2014).Cummings,ML,‘AutomationBiasinIntelligentTimeCriticalDecisionSupportSystems’(September2004)AIAA1stIntelligentSystemsTechnicalConference1.D’Aspremont,J(ed),ParticipationintheInternationalLegalSystem.MultiplePerspectivesonNon-StateActorsinInternationalLaw(Routledge2011).Dahm,W,‘ReportonTechnologyHorizons:AVisionforAirForceScience&TechnologyDuring2010–2030’(September2010).Daoust,I,Coupland,RandIshoey,R,‘NewWars,NewWeapons?TheObligationofStatestoAssesstheLegalityofMeansandMethodsofWarfare’(2002)846IRRC345.DeLaRochefoucauld,AandSaldi,S(eds),‘TheHumanizationofRobotsandtheRobotizationoftheHumanPerson.EthicalReflectionsonLethalAutonomousWeaponsSystemsandAugmentedSoldiers’(workingpaperbytheCaritasInVeritateFoundation).DeSena,P,‘Condottadisingoliorganiecondottadell’apparatostataleintemadicolpanell’illecitointernazionale’(1988)71Rivistadidirittointernazionale525.DeSena,P,Lanozionedigiurisdizionestataleneitrattatisuidirittidell’uomo(Giappichelli2001).DiegoMauri-9781802207675DownloadedfromPubFactoryat05/28/202203:48:50AMviaDukeUniversityLawLibrary

273Bibliography265DeSena,P,‘Prassi,consuetudineeprincipinelcampodeidirittidell’uomo.Riflessioniinternazionalistiche’(2014)43RagionPratica511.DeSena,P,‘ProportionalityandHumanRightsinInternationalLaw:Some…“Utilitarian”Reflections’(2016)99Rivistadidirittointernazionale1009.DeSena,P,‘Dignitàumanainsensooggettivoedirittointernazionale’(2017)11DirittiUmanieDirittoInternazionale573.DeSena,P,‘La“duediligence”etlelienentrelesujetetlerisquequ’ilfautpréve-nir:quelquesobservations’,inSCassella(ed),Lestandarddeduediligenceetlaresponsabilitéinternationale(Pedone2018).DeSena,P,Dignitéhumaineetdroitinternational(forthcoming)RecueildesCours.DeSena,PandAcconciamessa,L,‘BalancingTest’,inHRuizFabri(ed),MaxPlanckEncyclopediaofInternationalProceduralLaw(OUPforthcoming).Deeks,A,‘TheJudicialDemandforExplainableArtificialIntelligence’(2019)119ColumbiaLR1829.Deeks,A,Lubell,NandMurray,D,‘MachineLearning,ArtificialIntelligence,andtheUseofForcebyStates’(2019)10JofNat’lSecL&Pol1.DellaMorte,G,BigDataeprotezioneinternazionaledeidirittiumani.Regoleecon-flitti(EditorialeScientifica2018).Dellinger,M,‘RethinkingForceMajeureinPublicInternationalLaw’(2017)37PaceLR455.Dinstein,Y,‘TheRighttoLife,PhysicalIntegrity,andLiberty’,inLHenkin(ed),TheInternationalBillofRights,TheCovenantonCivilandPoliticalRights(ColumbiaUniversityPress1981).Dinstein,Y,‘Discussion:ReasonablyMilitaryCommandersandReasonableCivilians’(2002)78Int’lLStud219.Dinstein,Y,‘PrincipleofProportionality’,inKMujezinovicLarsen,CGuldahlCooperandGNystuen(eds),Searchingfora‘PrincipleofHumanity’inInternationalHumanitarianLaw(CUP2013).Dinstein,Y,TheConductoftheHostilitiesundertheLawofInternationalArmedConflict(CUP2016).Dorsey,DandAmaral,N,‘MilitaryDronesinEurope.EnsuringTransparencyandAccountability’,ChathamHouseResearchPaper(April2021).Draper,GIAD,‘HumanitarianLawandHumanRights’,inMAMeyerandHMcCoubrey(eds),ReflectionsonLawandArmedConflicts(KluwerLawInternational1998).Droege,C,‘TheInterplaybetweenInternationalHumanitarianLawandInternationalHumanRightsLawinSituationsofArmedConflict’(2007)40IsraeliLR310.Dupuy,PM,LaresponsabilitéinternationaledesEtatspourlesdommagesd’originetechnologiqueetindustrielle(Pedone1976).Dupuy,PM,‘Les“considerationsélémentairesd’humanité”danslajurisprudencedelaCourinternationaledeJustice’,inRJDupuy(ed),MélangesoffertsàN.Valticos.Droitetjustice(Pedone1999).Dupuy,PM,‘L’unitédel’ordrejuridiqueinternational,CoursGénéraldedroitinterna-tionalpublic’(2002)297RecueildesCours9.Dworkin,R,TakingRightsSeriously(HarvardUniversityPress1977).Edwards,LandVeale,M,‘SlavetotheAlgorithm?Whya“RighttoanExplanation”IsProbablyNottheRemedyYouareLookingFor’(2017)16DukeL&TechnologyR18.Egeland,K,‘LethalAutonomousWeaponSystemsunderInternationalHumanitarianLaw’(2016)85NordicJIL89.DiegoMauri-9781802207675DownloadedfromPubFactoryat05/28/202203:48:50AMviaDukeUniversityLawLibrary

274266AutonomousweaponssystemsandtheprotectionofthehumanpersonEggett,C,‘TheRoleofPrinciplesandGeneralPrinciplesinthe“ConstitutionalProcesses”ofInternationalLaw’(2019)66NetherlandsInt’lLR197.Ekelhof,MAC,‘ComplicationsofaCommonLanguage:WhyitissoHardtoTalkaboutAutonomousWeapons’(2017)22JofConflict&SecL311.Ekelhof,MAC‘LiftingtheFogofTargeting:“AutonomousWeapons”andHumanControlthroughtheLensofMilitaryTargeting’(2018)71NavalWarCollegeR61.Evans,C,TheRighttoReparationinInternationalLawforVictimsofArmedConflict(CUP2012).Evans,TD,‘AtWarwithRobots:AutonomousWeaponSystemsandtheMartensClause’(2013)41HofstraLR697.Fast,L,‘UnpackingthePrincipleofHumanity:TensionsandImplications’(2016)97IRRC111.FinkelmanA,‘SuingtheHiredGuns:AnAnalysisofTwoFederalDefensestoTortLawsuitsagainstMilitaryContractors’(2009)34BrooklynJIL395.Francioni,FandRonzitti,N(eds),WarbyContract:HumanRights,HumanitarianLawandPrivateContractors(OUP2011).Frulli,M,‘ExploringtheApplicabilityofCommandResponsibilitytoPrivateMilitaryContractors’(2010)15JofConflict&SecL435.Frulli,M,‘TheContributionofInternationalCriminalTribunalstotheDevelopmentofInternationalLaw:TheProminenceofopiniojurisandtheMoralizationofCustomaryLaw’(2015)14TheLandPracticeofInt’lCourtsandTribunals80.Frulli,M,‘ThePromiseofInternationalCriminalJustice:AchievementsandFailuresinProtectingHumanDignity’,inAGattini,RGarciandiaandPWebb(eds),HumanDignityandInternationalLaw(Brill2021).Gaggioli,G,‘ALegalApproachtoInvestigationsofArbitraryDeprivationsoLifeinArmedConflicts:TheNeedforaDynamicUnderstandingoftheInterplaybetweenIHLandHRL’(2017)36QuestofInt’lL27.Gaja,G,‘GeneralPrinciplesofLaw’,inRWolfrum(ed),MaxPlanckEncyclopediaofPublicInternationalLaw(OUP2013).Garanina,IG,‘ThePoliceandtheProhibitionofTortureandInhumanorDegradingTreatmentorPunishment’,inRAlleweldtandGFickenscher(eds),ThePoliceandInternationalHumanRightsLaw(Springer2016).Garapon,AandLassègue,F,Justicedigitale.Révolutiongraphiqueetruptureantro-pologique(PUF2018).Gardam,JG,‘ProportionalityandForceinInternationalLaw’(1993)87AJIL391.Gattini,A,‘LanotiondefauteàlalumièreduprojetdeconventiondelaCommissionduDroitInternationalsurlaresponsabilitéinternationale’(1992)3EJIL253.Geiss,RandLahmann,H,‘AutonomousWeaponsSystems:AParadigmShiftfortheLawofArmedConflict?’,inJDOhlin(ed),ResearchHandbookonRemoteWarfare(EdwardElgar2017).Gervasoni,L,‘AContextual-FunctionalApproachtoInvestigationsintoRighttoLifeViolationsinArmedConflict’(2017)36QuestofInt’lL5.Gormley,P,‘TheRighttoLifeandtheRuleofNon-Derogability:PeremptoryNormsofJusCogens’,inBGRamcharan(ed),TheRighttoLifeinInternationalLaw(MartinusNijhoff1985).Gowlland-Debbas,VandGaggioli,G,‘TheRelationshipBetweenInternationalHumanRightsandHumanitarianLaw:AnOverview’,inRKolbandGGaggioli(eds),ResearchHandbookonHumanRightsandHumanitarianLaw(EdwardElgar2013).DiegoMauri-9781802207675DownloadedfromPubFactoryat05/28/202203:48:50AMviaDukeUniversityLawLibrary

275Bibliography267Grabenwarter,C(ed),EuropeanConventiononHumanRights.Commentary(Verlag-Hart-Nomos2014).Gradoni,L,‘Consuetudineecasoinconsueto’(2012)95Rivistadidirittointernazion-ale704.Greer,S,‘IstheProhibitionagainstTorture,Cruel,InhumanandDegradingTreatmentReally“Absolute”,InternationalHumanRightsLaw?’(2015)15HumanRightsLR101.Greppi,E,‘TheEvolutionofIndividualCriminalResponsibilityunderInternationalLaw’(1999)81IRRC531.Guastini,R,Filosofiadeldirittopositivo(Giappichelli2017).Gubrud,MandAltmann,J,‘ComplianceMeasuresforanAutonomousWeaponsConvention’,ICRACWorkingPaper2(May2013).Habermas,J,‘TheConceptofHumanDignityandtheRealisticUtopiaofHumanRights’(2010)4Metaphilosophy464.Haigh,K,‘ExtendingtheInternationalCriminalCourt’sJurisdictiontoCorporations:OvercomingComplementaryConcerns’(2008)14AustralianJofHumanRights199.Haines,S,‘TheDevelopingLawofWeapons.Humanity,DistinctionandPrecautionsinAttack’,inAClaphamandPGaeta(eds),TheOxfordHandbookofInternationalLawinArmedConflict(OUP2014).Hallevy,G,LiabilityforCrimesInvolvingArtificialIntelligenceSystems(Springer2015).Hammond,DN,‘AutonomousWeaponsandtheProblemofStateAccountability’(2015)15ChicagoJIL652.Handl,G,‘LiabilityasanObligationEstablishedbyaPrimaryRuleofInternationalLaw’(1985)16NetherlandsYbIL49.Haner,JandGarcia,D,‘TheArtificialIntelligenceArmsRace:TrendsandWorldLeadersinAutonomousWeaponsDevelopment’(2019)10GlobalPol331.Hanke,PandVitiello,D,‘High-TechMigrationControlintheEUandBeyond:TheLegalChallengesof“EnhancedOperability”’,inECarpanelliandNLazzerini(eds),UseandMisuseofNewTechnologies(Springer2019).HarrisonDinniss,HAandKleffner,JK,‘Soldier2.0:MilitaryHumanEnhancementandInternationalLaw’(2016)92Int’lLStud432.Hassan,P,‘TheWord“Arbitrary”asUsedintheUniversalDeclarationofHumanRights:“Illegal”or“Unjust”?’(1969)10HarvardInt’lLJ225.Hathaway,OA,Francis,A,Haviland,A,Reddy,Kethireddy,SandYamamoto,AT,‘AidingandAbettinginInternationalCriminalLaw’(2019)104CornellLR1593.Heintze,HJ,‘TheoriesontheRelationshipbetweenInternationalHumanitarianLawandHumanRightsLaw’,inRKolbandGGaggioli(eds),ResearchHandbookonHumanRightsandHumanitarianLaw(EdwardElgar2013).Henckaerts,JMandConstantin,D,‘ProtectionoftheNaturalEnvironment’,inAClaphamandPGaeta(eds),TheOxfordHandbookofInternationalLawinArmedConflict(OUP2014).Henckaerts,JMandDoswald-Beck,L,CustomaryInternationalHumanitarianLaw(2vols)(CUP2005).HendersonISandReeceK,‘ProportionalityunderInternationalHumanitarianLaw:The“ReasonableMilitaryCommander”andReverberatingEffects’(2018)51VanderbiltJofTransnationalL835.DiegoMauri-9781802207675DownloadedfromPubFactoryat05/28/202203:48:50AMviaDukeUniversityLawLibrary

276268AutonomousweaponssystemsandtheprotectionofthehumanpersonHenderson,IS,Keane,PandLiddy,J,‘RemoteandAutonomousWarfareSystems:PrecautionsinAttackandIndividualAccountability’,inJDOhlin(ed),ResearchHandbookonRemoteWarfare(EdwardElgar2017).Hennessey.EF,‘LiabilityforDamageCausedbytheAccidentalOperationofaStrategicDefenseInitiativeSystem’(1988)21CornellInt’lLJ317.Hennette-Vauchez.S,‘AHumanDignitas?RemnantsoftheAncientLegalConceptinContemporaryDignityJurisprudence’(2011)9Int’lJofConst’lL32.Hertogen,A,‘LettingLotusBloom’(2015)26EJIL901.Heyns,C,‘HumanRightsandtheUseofAutonomousWeaponsSystems(AWS)DuringDomesticLawEnforcement’(2016)38HumanRightsQuarterly350.Heyns,C,‘AutonomousWeaponsinArmedConflictandtheRighttoaDignifiedLife:AnAfricanPerspective’(2017)33SouthAfricanJonHumanRights46.Heyns,C,AkandeD,Hill-CawthorneLandChengetaT,‘TheInternationalFrameworkRegulatingtheUseofArmedDrones’(2016)65ICLQ791.Hill,TE,‘KantianAutonomyandContemporaryIdeasofAutonomy’,inOSensen(ed),KantOnMoralAutonomy(CUP2012).Hill-Cawthorne,L,‘PersonsCoveredbyInternationalHumanitarianLaw:MainCategories’,inBSaulandDAkande(eds),TheOxfordGuidetoInternationalHumanitarianLaw(OUP2020).Homayounnejad,M,‘EnsuringFullyAutonomousWeaponsSystemsComplywiththeRuleofDistinctioninAttack’,inSCasey-Maslen,MHomayounnejad,HStaufferandNWeizmann(eds),DronesandOtherUnmannedWeaponsSystemsunderInternationalLaw(Brill2018).Homayounnejad,MandOverill,RE,‘The“RobotsDon’tRape”Controversy’,inJGow,EDijxhoorn,RKerrandGVerdirame(eds),RoutledgeHandbookofWar,LawandTechnology(Routledge2019).Hughes,JG,‘TheLawofArmedConflictIssuesCreatedbyProgrammingAutomaticTargetRecognitionSystemsUsingDeepLearningMethods’(2020)21YbIHL99.Iovane,M,Lariparazionenellateoriaenellaprassidell’illecitointernazionale(Giuffré1990).Jenks,C,‘TheGatheringSwarm:ThePathtoIncreasinglyAutonomousWeaponsSystems’(2017)57Jurimetrics341.Jenks,CW,‘LiabilityforUltra-HazardousActivitiesinInternationalLaw’(1966)117Recueildescours176.Jevglevskaja,N,‘WeaponsReviewObligationunderCustomaryInternationalLaw’(2018)94Int’lLStud186.Jordan,D(ed),UnderstandingModernWarfare(CUP2016).Kabaalioğlu,H,‘TheObligationsto“Respect”and“Ensure’theRighttoLife”’,inBGRamcharan(ed),TheRighttoLifeinInternationalLaw(MartinusNijhoff1985).Kalshoven,FandZegveld,L,ConstraintsontheWagingofWar.AnIntroductiontoInternationalHumanitarianLaw(CUP2011).Kaminska,M,Broeders,DandCristiano,F,‘LimitingViralSpread:AutomatedCyberOperationsandthePrinciplesofDistinctionandDiscriminationintheGreyZone’,inTJančárková,LLindström,GViskyandPZozt(eds),GoingViral.202113thInternationalConferenceonCyberConflict(NATOCCDCOEPublications2021).Kania,EB,‘“AIWeapons”inChineseMilitaryInnovation’,GlobalChinaReport(April2020).Kessedjian,CandCantùRivera,H(eds),PrivateInternationalLawAspectsofCorporateSocialResponsibility(Springer2020).DiegoMauri-9781802207675DownloadedfromPubFactoryat05/28/202203:48:50AMviaDukeUniversityLawLibrary

277Bibliography269Kindervater,KH,‘TheEmergenceofLethalSurveillance:WatchingandKillingintheHistoryofDroneTechnology’(2016)47SecDialogue223.Kolb,R,‘HumanRightsLawandHumanitarianLaw’,inRWolfrum(ed),MaxPlanckEncyclopediaofPublicInternationalLaw(OUP2016).Kolb,R,InternationalLawontheMaintenanceofPeace.JusContraBellum(EdwardElgar2017).Krishnan,A,KillerRobots:LegalityandEthicalityofAutonomousWeapons(Routledge2009).Lachow,I,‘TheUpsideandDownsideofSwarmingDrones’(2017)73BulletinoftheAtomicScientists96.Lagioia,FandSartor,G,‘AISystemsunderCriminalLaw:ALegalAnalysisandaRegulatoryPerspective’(2020)33Philosophy&Technology433.Lambert,D,‘Uneéthiquenepeutêtrequ’humaine!Réflexionsurleslimitesdesmoralmachines’,inRDoeré,DDanetandGDeBoisboissel(eds),Droneset“KillerRobots”.Faut-illesinterdire?(PressesUniversitairesdeRennes2015).Lavrysen,L,‘PositiveObligationsintheJurisprudenceoftheInter-AmericanCourtofHumanRights’(2014)7Inter-AmericanandEuropeanHumanRightsJournal94.LeBris,C,L’humanitésaisieparledroitinternationalpublic(Pedone2012).Leach,P,Murray,RandSandoval,C,‘TheDutytoInvestigateRighttoLifeViolationsacrossThreeRegionalSystems:HarmonisationorFragmentationofInternationalHumanRightsLaw?’,inCMBuckley,ADonaldandPLeach(eds),TowardsConvergenceinInternationalHumanRightsLawApproachesofRegionalandInternationalSystems(Brill2016).Letsas,G,ATheoryofInterpretationoftheEuropeanConventiononHumanRights(CUP2009).Leveringhaus,AandGiacca,G,‘Robo-Wars.TheRegulationofRoboticWeapons’(OxfordMartinPolicyPaper2014).Liivoja,RandVäljataga,A(eds),AutonomousCyberCapabilitiesunderInternationalLaw(NATOCCDCOEPublications2021).Lin,P,Bekey,GandAbney,K,‘AutonomousMilitaryRobotics:Risk,EthicsandDesign’,ReportfortheUSDepartmentofNavy,OfficeofNavalResearch(20December2008).Lippman,M,‘WarCrimesTrialsofGermanIndustrialists:The“OtherSchindlers”’(1995)9TempleInt’l&CompLJ173.Longobardo,M,‘TheRelevanceoftheConceptofDueDiligenceforInternationalHumanitarianLaw’(2020)37WisconsinJIL44.Lubell,N,ExtraterritorialUseofForceagainstNon-StateActors(OUP2010).Lundberg,CandChristensen,HI,‘AssessmentofMan-portableRobotsforLawEnforcementAgencies’(2007)PerMIS’07:Proceedingofthe2007WorkshoponPerformanceMetricsforIntelligentSystems76.Mahoney,CW,‘UnitedStatesDefenceContractorsandtheFutureofMilitaryOperations’(2020)36Defense&SecAnalysis180.Malgieri,GandComandè,G,‘WhyaRighttoLegibilityofAutomatedDecision-MakingExistsintheGeneralDataProtectionRegulation’(2017)7Int’lDataPrivacyL243.Mallory,C,HumanRightsImperialists:TheExtraterritorialApplicationoftheEuropeanConventiononHumanRights(Hart2020).Marauhn,T,‘MeaningfulHumanControl–andthePoliticsofInternationalLaw’,inWHeintschelvonHeinegg,RFrauandTSinger(eds),DehumanizationofWarfare(Springer2018).DiegoMauri-9781802207675DownloadedfromPubFactoryat05/28/202203:48:50AMviaDukeUniversityLawLibrary

278270AutonomousweaponssystemsandtheprotectionofthehumanpersonMarchant,GE,Allenby,B,Arkin,R,Barrett,ETandBorenstein,J,‘InternationalGovernanceofAutonomousMilitaryRobots’(2011)12ColumbiaScience&TechnologyLR272.Margulies,P,‘MakingAutonomousWeaponsAccountable:CommandResponsibilityforComputer-GuidedLethalForceinArmedConflicts’,inJDOhlin(ed),ResearchHandbookonRemoteWarfare(EdwardElgar2017).Marra,WCandMcNei,lSK,‘Understanding“TheLoop”:RegulatingtheNextGenerationofWarMachines’(2013)36HarvardJofL&PublicPol1139.Martens,P,‘Encoreladignitéhumaine:réflexionsd’unjugesurlapromotionparlesjugesd’unenormesuspecte’,inMVerdussenetal(eds),Lesdroitsdel’hommeauseuildutroisièmemillénaire:mélangesenhommageàPierreLambert(Bruylant2000).Mauri,D,‘“AlgorithmicTargetConstruction”andtheChallengesbyInternationalHumanRightsLaw’(2019)Eurojus8.Mauri,D,‘TheHolySee’sPositiononLethalAutonomousWeaponsSystems.AnAppraisalthroughtheLensoftheMartensClause’(2020)11JIHLS116.Mauri,D,‘ThePoliticalQuestionDoctrinevis-à-visDrones’‘OutsizedPower’:AntitheticalApproachesinRecentCase-Law’(2020)68QuestofInt’lL3.Mauri,D,‘TheUseofCBRNWeaponsinArmedConflict’,inAdeGuttry,MFrulli,FCasolariandLPoli(eds),InternationalObligationsApplicabletoPrevention,Preparedness,ResponsetoandRecoveryfromChemical,BiologicalandRadio-Nuclear(CBRN)Events.TowardsanAll-HazardApproach(Brillforthcoming).Mazaroff,S,‘ExonerationfromLiabilityforDamageCausedbySpaceActivities’(1968)54CornellLR70.McCoubrey,H,InternationalHumanitarianLaw.ModernDevelopmentintheLimitationofWarfare(2nded,Routledge2019).McCrudden,C,‘HumanDignityandJudicialInterpretationofHumanRights’(2008)19EJIL655.McCrudden,C,‘InPursuitofHumanDignity:AnIntroductiontoCurrentDebates’,inCMcCrudden,UnderstandingHumanDignity(OUP2013).McDonald,N,‘TheRoleofDueDiligenceinInternationalLaw’(2019)68ICLQ1041.McFarland,T,AutonomousWeaponSystemsandtheLawofArmedConflict.CompatibilitywithInternationalHumanitarianLaw(CUP2020).McFarland,TandMcCormack,T,‘MindtheGap:CanDevelopersofAutonomousWeaponsSystemsbeLiableforWarCrimes?’(2014)90Int’lLStud361.McGinnis,JO,‘AcceleratingAI’(2010)104NorthwesternUniversityLR1253.McGregor,L,Murray,DandNg,V,‘InternationalHumanRightsLawasaFrameworkforAlgorithmicAccountability’(2019)68ICLQ309.Meloni,C,‘StateandIndividualResponsibilityforTargetedKillingsbyDrones’,inEDiNucciandFSantonideSio(eds),DronesandResponsibility:Legal,PhilosophicalandSocio-TechnicalPerspectivesonRemotelyControlledWeapons(Routledge2016).Meloni,C,CommandResponsibilityinInternationalCriminalLaw(AsserPress2010).Meron,T,‘TheHumanizationofHumanitarianLaw’(2000)94AJIL239.Meron,T,‘TheMartensClause,PrinciplesofHumanityandDictatesofPublicConscience’(2000)94AJIL78.Meron,T,TheHumanizationofInternationalLaw(MartinusNijhoff2006).DiegoMauri-9781802207675DownloadedfromPubFactoryat05/28/202203:48:50AMviaDukeUniversityLawLibrary

279Bibliography271Michalczak,R,‘Animals’RaceAgainsttheMachines’,inVKurkiandTPetrzykowski(eds),LegalPersonhood:Animals,ArtificialIntelligenceandtheUnborn(Springer2017).Milanović,M,‘TheLostOriginofLexSpecialis.RethinkingtheRelationshipbetweenHumanRightsandInternationalHumanitarianLaw’,inJDOhlin(ed),TheoreticalBoundariesofArmedConflictandHumanRights(CUP2016).Milanović,M,ExtraterritorialApplicationofHumanRightsTreaties.Law,Principles,andPolicy(OUP2011).Miyazaki,S,‘TheMartensClauseandInternationalHumanitarianLaw’,inCSwinarski(ed),StudiesandEssaysonInternationalHumanitarianLawandRedCrossPrinciplesinHonourofJeanPictet(MartinusNijhoff1984).Moir,L,‘TheEuropeanCourtofHumanRightsandInternationalHumanitarianLaw’,inRKolbandGGaggioli(eds),ResearchHandbookonHumanRightsandHumanitarianLaw(EdwardElgar2013).Monterossi,MW,‘LiabilityfortheFactofAutonomousArtificialIntelligenceAgents.Things,AgenciesandLegalActors’(2020)20GlobalJurist1.Montoje,M,‘TheConceptofLiabilityintheAbsenceofanInternationallyWrongfulAct’,inJCrawford,APelletandSOlleson(eds),TheLawofInternationalResponsibility(OUP2010).Moodrick-EvenKhen,H,‘AidosandDikeinInternationalHumanitarianLaw:IsIHLaLegaloraMoralSystem?(2016)99TheMonist26.Mowbray,A,HumanRightsLawinPerspective:TheDevelopmentofPositiveObligationsundertheEuropeanConventiononHumanRightsbytheEuropeanCourtofHumanRights(Hart2004).Moyes,R,‘MeaningfulHumanControloverIndividualAttacks’,inICRC,‘AutonomousWeaponsSystems.ImplicationsofIncreasinglyAutonomyintheCriticalFunctionsofWeapons’,ExpertMeeting(March2016).MujezinovicLarsen,K,‘A“PrincipleofHumanity”ora“PrincipleofHuman-Rightism”?’,inKMujezinovicLarsen,CGuldahlCooperandGNystuen(eds),Searchingfora‘PrincipleofHumanity’inInternationalHumanitarianLaw(CUP2013).Noll,G,‘SacrificialViolenceandTargetinginInternationalHumanitarianLaw’,inPEngdahlandOWrange(eds),LawAtWar:TheLawasItWasandtheLawasItShouldBe.LiberAmicorumOveBring(Leiden2008).Noll,G,‘AnalogyatWar:Proportionality,EqualityandtheLawofTargeting’(2013)43NetherlandsYbIL205.NserekoD,‘ArbitraryDeprivationofLife:ControlsonPermissibleDeprivations’,inBGRamcharan(ed),TheRighttoLifeinInternationalLaw(MartinusNijhoff1985).O’Connell,ME,‘HistoricalDevelopmentandLegalBasis’,inDFleck(ed),TheHandbookofInternationalHumanitarianLaw(2nded,OUP2013).O’Connell,ME,‘BanningAutonomousKilling:TheLegalandEthicalRequirementthatHumansMakeNear-TimeLethalDecisions’,inMEvangelistaandHShue(eds),TheAmericanWayofBombing:EthicalandLegalNorms,FromFlyingFortressestoDrones(CornellUniversityPress2014).O’Connell,ME,‘TheLawonLethalForceBeginswiththeRighttoLife’(2016)3JontheUseofForce&Int’lL205.O’Connell,ME,TheArtofLawintheInternationalCommunity(CUP2019).O’Connell,MEandDay,CM,‘SourcesandtheLegalityandValidityofInternationalLaw.NaturalLawasSourceofExtra-PositiveNorms’,inSBessonandJd’Aspre-mont,TheOxfordHandbookontheSourcesofInternationalLaw(OUP2017).DiegoMauri-9781802207675DownloadedfromPubFactoryat05/28/202203:48:50AMviaDukeUniversityLawLibrary

280272AutonomousweaponssystemsandtheprotectionofthehumanpersonO’Connell,R,OfArmsandMen.AHistoryofWar,WeaponsandAggression(OUP1989).O’Mahony,C,‘ThereIsNoSuchThingasaRighttoDignity’(2012)10Int’lJofConst’lL551.Oberleitner,G,‘HumanitarianLawasaSourceofHumanRightsLaw’,inDShelton(ed),TheOxfordHandbookofInternationalHumanRightsLaw(OUP2013).Ochoa-Ruiz,NandSalamanca-Aguado,E,‘ExploringtheLimitsofInternationalLawRelatingtotheUseofForceinSelf-defence’(2005)16EJIL499.Ollino,A,DueDiligenceObligationsinInternationalLaw:ATheoreticalStudy(CUPforthcoming).Olsen,JA(ed),AHistoryofAirWarfare(PotomacBooks2010).Paddeu,F,‘AGenealogyofForceMajeureinInternationalLaw’(2011)82BritishYbIL381.Paddeu,F,JustificationandExcuseinInternationalLaw.ConceptandTheoryofGeneralDefences(CUP2018).Pagallo,U,‘AIandBadRobots:TheCriminologyofAutomation’,inMRMcGuireandTJHolt(eds),TheRoutledgeHandbookofTechnology,CrimeandJustice(Routledge2017).Palmisano,G,‘Fault’,inRWolfrum(ed),MaxPlanckEncyclopediaofPublicInternationalLaw(OUP2007).Parasumaran,R,Sheridan,TBandWickens,CD,‘AModelforTypesandLevelsofHumanInteractionwithAutomation’(2000)30ProceedingsoftheIEEE286.Parker,G,TheMilitaryRevolution.MilitaryInnovationandtheRiseoftheWest,1500–1800(CUP1996).Pasquale,F,TheBlackBoxSociety:TheSecretAlgorithmsThatControlMoneyandInformation(HarvardUniversityPress2015).Payne,CR,‘ProtectionoftheNaturalEnvironment’,inBSaulandDAkande(eds),TheOxfordGuidetoInternationalHumanitarianLaw(OUP2020).Pedrazzi,M,‘TheProtectionoftheRighttoLifeinLaw-EnforcementOperations’,inBCortese(ed),StudiinonorediLauraPicchioForlati(Giappichelli2014).Pictet,J,Développementetprincipesdudroitinternationalhumanitaire(InstitutHenryDunantandEditionsPédone1983).Pietropaoli,S,‘Cyberspazio.Ultimafrontieradell’inimicizia?Guerre,nemiciepiratialtempodellarivoluzionedigitale’(2019)8Rivistadifilosofiadeldiritto379.PisilloMazzeschi,R,“DueDiligence”eresponsabilitàinternazionaledegliStati(Giuffrè1989).PisilloMazzeschi,R,‘TheDueDiligenceRuleandtheNatureoftheInternationalResponsibilityofStates’(1992)35GermanYbILL9.PisilloMazzeschi,R,‘ReparationClaimsbyIndividualsforStateBreachesofHumanitarianLawandHumanRights’(2003)1JICJ339.PisilloMazzeschi,R,‘Responsabilitédel’Etatpourviolationsdesobligationspositivesrelativesauxdroitsdel’homme’(2008)333RecueildesCours175.PisilloMazzeschi,RandViviani,A,‘GeneralPrinciplesofInternationalLaw:FromRulestoValues?’,inRPisilloMazzeschiandPDeSena(eds),GlobalJustice,HumanRightsandtheModernizationofInternationalLaw(Springer,2018).Pustogarov,V,‘F.F.Martens(1845–1909),aHumanistofModernTimes’(1996)312IRRC300.Raible,L,HumanRightsUnbound.ATheoryofExtraterritoriality(OUP2020).DiegoMauri-9781802207675DownloadedfromPubFactoryat05/28/202203:48:50AMviaDukeUniversityLawLibrary

281Bibliography273Reeves,SRandThurnher,JS,‘AreWereachingaTippingPoint?HowContemporaryChallengesAreAffectingtheMilitaryNecessity-HumanityBalance’(2013)4HarvardNat’lSecJFeatures1.Reitinger,N,‘AlgorithmicChoiceandSuperiorResponsibility:ClosingtheGapBetweenLiabilityandLethalAutonomybyDefiningtheLineBetweenActorsandTools’(2015)51GonzagaLR79.Richardson,AandBowden,J(eds),TheWestminsterDictionaryofChristianTheology(JohnKnoxPress1983).Rid,TandMcBurney,P,‘CyberWeapons’(2012)157TheRUSIJ6.Robertson,HB,‘ThePrincipleoftheMilitaryObjectiveintheLawofArmedConflict’(1997)8UnitedStatesAirForceAcademyJofLegalStud35.Roff,HM,‘LethalAutonomousWeaponsandJusadBellumProportionality’(2015)47CaseWestReserveJInt’lL37.Roorda,LandRyngaert,C,‘PublicInternationalLawConstraintsontheExerciseofAdjudicatoryJurisdictioninCivilMatters’,inSForlatiandPFranzina(eds),UniversalCivilJurisdiction.WhichWayForward?(Brill2020).Roorda,M,‘NATO’sTargetingProcess:EnsuringHumanControlover(andLawfulUseof)“Autonomous”Weapons”’,inAPWilliamsandPScharre(eds),AutonomousSystems:IssuesforDefencePolicymakers(NATOCommunicationsandInformationAgency2016).Roscini,M,CyberOperationsandtheUseofForceinInternationalLaw(OUP2014).Rowe,NC,‘DistinctiveEthicalChallengesofCyberweapons’,inJDOhlin,KGovern,CFinkelstein,CyberWar:LawandEthicsforVirtualConflicts(OUP2015).RuschiF,‘Ilvolodeldrone.Versounaguerrapostumana?Unariflessionedifilosofiadeldirittointernazionale’(2016)13JuraGentium12.Ryngaert,C,‘Non-StateActors:CarvingOutaSpaceinaState-CentredInternationalLegalSystem’(2016)63NetherlandsInt’lLR183.Salter,M,‘ReinterpretingCompetingInterpretationsoftheScopeandPotentialoftheMartensClause’(2012)17JofConflict&SecL403.Saluzzo,S,‘CBRNWeaponsandtheProtectionoftheEnvironmentduringArmedConflicts’,inAdeGuttry,MFrulli,FCasolariandLPoli(eds),InternationalObligationsApplicabletoPrevention,Preparedness,ResponsetoandRecoveryfromChemical,BiologicalandRadio-Nuclear(CBRN)Events.TowardsanAll-HazardApproach(Brillforthcoming).Sandoz,Y,Swinarski,CandZimmermann,B(eds),CommentaryontheAdditionalProtocolsof8June1977totheGenevaConventionsof12August1949(MartinusNijhoff1987).SantonideSio,FandvandenHoven,J,‘MeaningfulHumanControloverAutonomousSystems:APhilosophicalAccount’(2018)5FrontiersinRoboticsandAI1.Sassoli,M,‘StateResponsibilityforViolationsofInternationalHumanitarianLaw’(2002)84IRRC401.Sassoli,M,‘AutonomousWeaponsandInternationalHumanitarianLaw:Advantages,OpenTechnicalQuestionsandLegalIssuestobeClarified’(2014)90Int’lLStud308.Sassoli,M,‘TheConvergenceofInternationalHumanitarianLawofNon-internationalandofInternationalArmedConflicts:DarkSideofaGoodIdea’,inGBiaggini,ODiggelmann,andCKaufmann(eds),PolisundKosmopolis:FestschriftfürDanielThürer(Nomos2015).DiegoMauri-9781802207675DownloadedfromPubFactoryat05/28/202203:48:50AMviaDukeUniversityLawLibrary

282274AutonomousweaponssystemsandtheprotectionofthehumanpersonSassoli,M,‘InternationalHumanitarianLawandInternationalHumanRightsLaw’,inBSaulandDAkande(eds),TheOxfordGuidetoInternationalHumanitarianLaw(OUP2020).Sauvant,KP,‘TheNegotiationsoftheUnitedNationsCodeofConductonTransnationalCorporations.ExperienceandLessonsLearned’(2015)16JofWorldInvestment&Trade11.Saxon,D,‘AHumanTouch:AutonomousWeapons,DoDDirective3000.09andtheInterpretationof“AppropriateLevelsofHumanJudgmentovertheUseofForce”’,inNBhuta,SBeck,RGeiss,HLiuandCKress(eds),AutonomousWeaponsSystems(CUP2016).Schabas,W,‘EnforcingInternationalHumanitarianLaw:CatchingtheAccomplices’(2001)83IRRC439.Schabas,W,TheEuropeanConventiononHumanRights.ACommentary(OUP2015).Schabas,W,TheInternationalCriminalCourt.ACommentaryontheRomeStatute(OUP2010).Schachter,O,‘InternationalLawinTheoryandPractice’(1982)Recueildescours9.Scharre,P,‘Autonomy,“KillerRobots”andHumanControlintheUseofForce–PartI’(JustSecurity,9July2014).Scharre,P,‘RoboticsontheBattlefieldPartII.TheComingSwarm’(CNAS,October2014).ScharreP,ArmyofNone(WWNorton2018).Schmitt,C,TheNomosoftheEarthintheInternationalLawoftheJusPublicumEuropeaum(transandannotGLUlmen)(TelosPress2006).Schmitt,MN,‘TargetingandHumanitarianLaw:CurrentIssues’(2003)33IsraelYbonHumanRights59.Schmitt,MN,‘MilitaryNecessityandHumanityinInternationalHumanitarianLaw:PreservingtheDelicateBalance’(2010)50VirginiaJIL795.Schmitt,MN,‘InvestigatingViolationsofInternationalLawinArmedConflicts’(2011)2HarvardNat’lSecJ39.Schmitt,MN,‘AutonomousWeaponSystemsandInternationalHumanitarianLaw:AReplytotheCritics’(2013)4HarvardNat’lSecJFeatures1.SchmittMN(gened),TallinnManual2.0ontheInternationalLawApplicabletoCyberOperations(CUP2017).Schmitt,MN,‘TheLawofCyberWarfare:QuoVadis?’(2018)25StanfordL&PolR269.Schmitt,MN,‘InternationalHumanitarianLawandtheConductofHostilities’,inBSaulandDAkande(eds),TheOxfordGuidetoInternationalHumanitarianLaw(OUP2020).Schmitt,MNandThurnher,JS,‘“OutoftheLoop”:AutonomousWeaponSystemsandtheLawofArmedConflict’(2013)4HarvardNat’lSecJ231.Schmitt,MNandWidmar,EW,‘“OnTarget”:PrecisionandBalanceintheContemporaryLawofTargeting’(2014)75JofNat’lSecL&Pol379.Schwarzenberger,G,‘TheLegalityofNuclearWeapons’(1958)11CurrentLegalProblems258.Schwarzenberger,G,‘TheLawofArmedConflict:ACivilizedInterlude?’(1974)28YbofWorldAffairs293.Seger,P,‘TheLawofNeutrality’,inAClaphamandPGaeta(eds),TheOxfordHandbookofInternationalLawinArmedConflict(OUP2014).Shapo,MS,‘ComparingProductsLiability:ConceptsinEuropeanandAmericanLaw’(1993)26CornellInt’lLJ279.DiegoMauri-9781802207675DownloadedfromPubFactoryat05/28/202203:48:50AMviaDukeUniversityLawLibrary

283Bibliography275Sharkey,NE,‘TowardsaPrinciplefortheHumanSupervisoryControlofRobotWeapons’(2014)2Politica&Società305.Sharkey,NE,‘StayingintheLoop:HumanSupervisoryControlofWeapons’,inNBhuta,SBeck,RGeiss,HLiuandCKress(eds),AutonomousWeaponsSystems(CUP2016).Sharkey,NE,‘GuidelinesfortheHumanControlofWeaponsSystems’,ICRACWorkingPaperNo3(April2018).SheltonDandGouldA,‘PositiveandNegativeObligations’,inDShelton(ed),TheOxfordHandbookofInternationalHumanRightsLaw(OUP2013).Simmons,R,‘BigDataandProceduralJustice:LegitimizingAlgorithmsintheCriminalJusticeSystem’(2018)15OhioStateJofCriminalL573.Singer,PW,WiredforWar.TheRoboticsRevolutionandConflictinthe21stCentury(PenguinBooks2009).Sitaropoulos,N,‘WeaponsandSuperfluousInjuryorUnnecessarySufferinginInternationalHumanitarianLaw:HumanPaininTimeofWarandtheLimitsofLaw’(2001)54RevueHelleniquedeDroitInternational71.Song,DC,‘U.S.CorporateLiabilityundertheAlienTortStatuteafterJesnerv.ArabBank,PLC’(2018)21OregonRofInt’lL1.Sossai,M,‘TheDemandsofFutureOperationsandthePromiseofNon-orLess-LethalWeapons’(2018)21YbIHL1.Spagnolo,A,‘HumanRightsImplicationsofAutonomousWeaponSystemsinDomesticLawEnforcement:Sci-FiReflectionsonaLo-FiReality’(2017)43QuestofInt’lL33.Sparrow,R,‘RoboticWeaponsandtheFutureofWar’,inJWolfendaleandPTripodi(eds),NewWarsandNewSoldiers:MilitaryEthicsintheContemporaryWorld(Ashgate2011).Sparrow,R.‘TwentySecondstoComply:AutonomousWeaponSystemsandtheRecognitionofSurrender’(2015)91Int’lLStud699,705–9.Stauffer,H,‘CorporateLiability:AnAlternativePathtoAccountability?’,inSCasey-Maslen,MHomayounnejad,HStaufferandNWeizmann(eds),DronesandOtherUnmannedWeaponsSystemsunderInternationalLaw(Brill2018).Steinhardt,RG,‘WeaponsandtheHumanRightsResponsibilitiesofMultinationalCorporations’,inSCasey-Maslen(ed),WeaponsUnderInternationalHumanRightsLaw(CUP2014).Stephens,D,‘HumanRightsandArmedConflict:TheAdvisoryOpinionoftheInternationalCourtofJusticeintheNuclearWeaponsCase’(2001)4YaleHumanRightsandDevelopmentLJ1.Stoyanova,V,‘Fault,KnowledgeandRiskwithintheFrameworkofPositiveObligationsundertheEuropeanConventiononHumanRights’(2020)33LeidenJIL601.SunTzu,TheArtofWar(transGLionel)(IxiaPress2019).Szurek,S,‘CircumstancesPrecludingWrongfulnessintheILCArticlesonStateResponsibility:ForceMajeure’,inJCrawford,APelletandSOlleson(eds),TheLawofInternationalResponsibility(OUP2010).Szurek,S,‘TheNotionofCircumstancesPrecludingWrongfulness’,inJCrawford,APelletandSOlleson(eds),TheLawofInternationalResponsibility(OUP2010).TalbotJensen,E,‘The(Erroneous)RequirementforHumanJudgment(andError)intheLawofArmedConflict’(2020)96Int’lLStud26.DiegoMauri-9781802207675DownloadedfromPubFactoryat05/28/202203:48:50AMviaDukeUniversityLawLibrary

284276AutonomousweaponssystemsandtheprotectionofthehumanpersonTamburrini,G,‘OnBanningAutonomousWeaponsSystems:FromDeontologicaltoWideConsequentialistReasons’,inNBhuta,SBeck,RGeiss,HLiuandCKress(eds),AutonomousWeaponsSystems(CUP2016).Tanner,NP(ed),DecreesofEcumenicalCouncils(GeorgetownUniversityPress1990).Tattersall,AandCopeland,D,‘ReviewingAutonomousCyberCapabilities’,inRLiivojaandAVäljataga(eds),AutonomousCyberCapabilitiesunderInternationalLaw(NATOCCDCOEPublications2021).Taylor,PM,ACommentaryontheInternationalCovenantonCivilandPoliticalRights.TheUNHumanRightsCommittee’sMonitoringofICCPRRights(CUP2020).Thurnher,JS,‘FeasiblePrecautionsinAttackandAutonomousWeapons’,inWHeintschelvonHeinegg,RFrauandTSinger(eds),DehumanizationofWarfare.LegalImplicationsofNewWeaponTechnologies(Springer2018).Ticehurst,R,‘TheMartensClauseandtheLawsofArmedConflict’(1997)317IRRC125.Tomuschat,C,‘ObligationsarisingforStateswithouttheirWill’(1993)241Recueildescours198.Tomuschat,C,‘InternationalLaw:EnsuringtheSurvivalofMankindontheEveofaNewCentury.GeneralCourseonPublicInternationalLaw’(1999)281Recueildecours9.TomuschatC,‘StateResponsibilityandtheIndividualRighttoCompensationbeforeNationalCourts’,inAClaphamandPGaeta(eds),TheOxfordHandbookofInternationalLawinArmedConflict(OUP2014).Townley,S,‘TheRiseofRiskinInternationalLaw’(2018)ChicagoJIL594.Tsagourias,N,‘TheConstitutionalRoleofGeneralPrinciplesofLawinInternationalandEuropeanPerspectives’,inNTsagourias(ed),TransnationalConstitutionalism.InternationalandEuropeanPerspectives(CUP2007).Tsagourias,N,‘TheLegalStatusofCyberspace’,inNTsagouriasandRBuchan(eds),ResearchHandbookonInternationalLawandCyberspace(EdwardElgar2017).Ulgen,O,‘HumanDignityinanAgeofAutonomousWeapons:AreWeinDangerofLosingan“ElementaryConsiderationofHumanity”?’(2016)15ESILConferencePaper1.UlgenO,‘KantianEthicsintheAgeofArtificialIntelligenceandRobotics’(2017)43QuestofInt’lL59.VandenBoogaard,J,‘FightingbythePrinciples:PrinciplesasaSourceofInternationalHumanitarianLaw’,inMMatthee,BToebesandMBrus(eds),ArmedConflictandInternationalLaw:InSearchoftheHumanFace(TMCAsserPress2013).VandenBoogaard,J,‘ProportionalityandAutonomousWeaponsSystems’(2015)6JIHLS247.VandenBoogaard,JandRoorda,MP,‘“Autonomous”WeaponsandHumanControl’,inRBartels,JvandenBoogard,PALDucheine,EPouwandJVoetelink(eds),MilitaryOperationsandtheNotionofControlUnderInternationalLaw.LiberAmicorumTerryD.Gill(Springer2021).VanSliedregt,E,IndividualCriminalResponsibilityinInternationalLaw(OUP2012).VanSliedregt,E,‘TheCuriousCaseofInternationalCriminalLiability’(2012)10JICJ1171.Vásquez-Bermúdez,MandCrosato,A,‘GeneralPrinciplesofLaw:TheFirstDebatewithintheInternationalLawCommissionandtheSixthCommittee’(2020)19ChineseJIL157.DiegoMauri-9781802207675DownloadedfromPubFactoryat05/28/202203:48:50AMviaDukeUniversityLawLibrary

285Bibliography277Verdros,A,‘Lesprincipesgénérauxdedroitdanslesystèmedessourcesdudroitinternationalpublic’,inPGuggenheim(ed),Receuild’étudesdedroitinternationalenhommageàP.Guggenheim(IUHEI1968).Vos,F,‘EuropeanTrendsinTortLawRemediestoAddressCorporateHumanRightsAbuses’,inABonfanti(ed),BusinessandHumanRightsinEurope.InternationalLawChallenges(Routledge2019).Wachter,S,Mittelstadt,BandFloridi,L,‘WhyaRighttoExplanationofAutomatedDecision-MakingDoesNotExistintheGeneralDataProtectionRegulation’(2017)7Int’lDataPrivacyL76.Wagner,M,‘AutonomyintheBattlespace:IndependentlyOperatingWeaponSystemsandtheLawofArmedConflict’,inDSaxon(ed),InternationalHumanitarianLawandtheChallengingTechnologyofWarfare(Brill2013).Wagner,M,‘AutonomousWeaponSystem’,inRWolfrum(ed),MaxPlanckEncyclopediaofPublicInternationalLaw(OUP2016).Wall,TandMonahan,T,‘SurveillanceandViolencefromAfar:ThePoliticsofDronesandLiminalSecurity-Scapes’(2011)15TheoreticalCriminology239.Webster,E,‘InterpretationoftheProhibitionofTorture:MakingSenseof“Dignity”Talk’(2016)17HumanRightsR371.Weissbrodt,DandHeilman,C,‘DefiningTortureandCruel,Inhuman,andDegradingTreatment’(2011)29L&Inequality343.Winter,E,‘PillarsnotPrinciples:TheStatusofHumanityandMilitaryNecessityintheLawofArmedConflict’(2020)25JofConflict&SecL1.Winter,E,‘TheCompatibilityofAutonomousWeaponswiththePrincipleofDistinctionintheLawofArmedConflict’(2020)69ICLQ845.Završnik,A,‘CriminalJustice,ArtificialIntelligenceSystems,andHumanRights’(2020)20ERAForum567.Zrilic,J,‘ArmedConflictasForceMajeureinInternationalInvestmentLaw’(2019)16ManchesterJofInt’lEconomicL28.ZuiderveenBorgesius,FJ,‘StrengtheningLegalProtectionagainstDiscriminationbyAlgorithmsandArtificialIntelligence’(2020)24Int’lJHumanRights1.WEBSITESArticle36,‘KillerRobots:UKGovernmentPolicyonFullyAutonomousWeapons’(2013)http://www.article36.org/wp-content/uploads/2013/04/Policy_Paper1.pdf.Blain,L,‘SouthKorea’sAutonomousRobotGunTurrets:DeadlyfromKilometersAway’(NewAtlas,7December2010)http://newatlas.com/korea-dodamm-super-aegis-autonomos-robot-gun-turret/17198/.CSKR,‘HighLevelConcernsonKillerRobotsatUN’(30October2019)https://www.stopkillerrobots.org/2019/10/unga74/.Gol,J,‘IranBlasts:WhatIsBehindMysteriousFiresatKeySites?’(BBC,6July2020)https://www.bbc.com/news/world-middle-east-53305940.Hambling,D,‘TheU.S.NavyPlanstoFoilMassive“SuperSwarm”DroneAttacksbyUsingtheSwarm’sIntelligenceagainstItself’(Forbes,26August2020)https://www.forbes.com/sites/davidhambling/2020/08/26/how-us-navy-plans-to-foil-massive-super-swarm-drone-attacks/.Kallenborn,Z,‘WasaFlyingKillerRobotUsedinLibya?QuitePossibly’(TheBulletin,20May2021)https://thebulletin.org/2021/05/was-a-flying-killer-robot-used-in-libya-quite-possibly/.DiegoMauri-9781802207675DownloadedfromPubFactoryat05/28/202203:48:50AMviaDukeUniversityLawLibrary

286278AutonomousweaponssystemsandtheprotectionofthehumanpersonMurphy,M,‘TheFirstBrain-ControlledDroneRaceJustTookPlaceinFlorida’(Quartz25April2016)https://qz.com/669720/the-first-brain-controlled-drone-race-just-took-place-in-florida/.NATO,‘NATOPolicyonNon-LethalWeapons’(1999,lastupdated2008)https://www.nato.int/cps/ic/natohq/official_texts_27417.htm?selectedLocale=en.Parkin,S,‘KillerRobots:TheSoldiersthatNeverSleep’(BBC,16July2015)http://www.bbc.com/future/story/20150715-killer-robots-the-soldiers-that-never-sleep.Sholes,E,‘EvolutionofaUAVAutonomyClassificationTaxonomy’,Remarksatthe2007IEEEAerospaceConference(2007)http://ieeexplore.ieee.org/stamp/stamp.jsp?tp=&arnumber=4161585.TheBureauofInvestigativeJournalism,https://www.thebureauinvestigates.com/projects/drone-war.Wolfe,F,‘CompaniesDevelopingLethalAutonomousWeapons,asGroupsSeekBan,ReportSays’(AviationToday,2December2019)https://www.aviationtoday.com/2019/12/02/companies-developing-lethal-autonomous-weapons-as-groups-seek-ban-report-says/.INTERNATIONALDOCUMENTSThefollowinglistcontainsonlythemaindocuments(reports,workingpapersandthelike)onAWS.StatementsdeliveredintheMoE/GGEsessionsbystates,IOs,NGOsandothersarenotlistedhereastheyareallavailableat:https://meetings.unoda.org/.ACommHPR,‘GeneralCommentNo.3ontheAfricanCharteronHumanandPeoples’Rights:TheRighttoLife(Article4)’.AmnestyInternationalandAccessNow,‘TheTorontoDeclaration:ProtectingtheRighttoEqualityandNon-DiscriminationinMachineLearningSystems’(16May2018).Article36,‘CriticalCommentaryonthe“GuidingPrinciples”’,policynote(November2019).Austria,BrazilandChile,‘ProposalforaMandatetoNegotiateaLegally-BindingInstrumentthatAddressestheLegal,HumanitarianandEthicalConcernsPosedbyEmergingTechnologiesintheAreaofLethalAutonomousWeaponsSystems(LAWS)’(8August2018)CCW/GGE.2/2018/WP.7.Brazil,‘LAWSandHumanControl:BrazilianProposalsforWorkingDefinitions’(19August2020)CCW/GGE.1/2020/WP.4.Brazil,ChileandMexico,‘Elementsforafuturenormativeframeworkconducivetoalegallybindinginstrumenttoaddresstheethicalhumanitarianandlegalcon-cernsposedbyemergingtechnologiesintheareaof(lethal)autonomousweapons(LAWS)’(June2021).CameradeiDeputati,MotionNo1-01776SessionNo898(6December2017).CampaigntoStopKillerRobots,‘KeyElementsofaTreatyOnFullyAutonomousWeapons’(November2019).ChambredesRépresentantsdeBelgique,‘Résolutionvisantàinterdirel’utilisation,parlaDéfensebelge,derobotstueursetdedronesarmés’(19July2018)3203/3005.China,‘PositionPaper’(11April2018)CCW/GGE.1/2018/WP.7.Directorate-GeneralforExternalPoliciesoftheUnion,PolicyDepartment,‘HumanRightsImplicationsoftheUsageofDronesandUnmannedRobotsinWarfare’(2013)EXPO/B/DROI/2012/12.DiegoMauri-9781802207675DownloadedfromPubFactoryat05/28/202203:48:50AMviaDukeUniversityLawLibrary

287Bibliography279EstoniaandFinland,‘CategorizingLethalAutonomousWeaponsSystems–ATechnicalandLegalPerspectivetoUnderstandingLAWS’(24August2018)CCW/GGE.2/2018/WP.2.EuropeanParliament,PolicyDepartmentforCitizens’RightsandConstitutionalAffairs,Directorate-GeneralforInternalPolicies,ArtificialIntelligenceandCivilLiability(June2020)PE621.926.EuropeanParliament,Resolution2018/2752(RSP)of12September2018onautono-mousweaponsystems[2018]OJC433/86.FifthReviewConferenceoftheHighContractingPartiestotheConventiononProhibitionsorRestrictionsontheUseofCertainConventionalWeaponsWhichMayBeDeemedtoBeExcessivelyInjuriousortoHaveIndiscriminateEffects,‘Reportofthe2016InformalMeetingofExpertsonLethalAutonomousWeaponsSystems(LAWS)’(10June2016)CCW/CONF.V/2.FranceandGermany,‘ForConsiderationbytheGroupofGovernmentalExpertsonLethalAutonomousWeaponsSystems(LAWS)’(7November2017)CCW/GGE.1/2017/WP.4.GenevaAcademyofInternationalHumanitarianLawandHumanRights,‘GenevaGuidelinesonLess-LethalWeaponsandRelatedEquipmentinLawEnforcement’(July2018).GenevaAcademyofInternationalHumanitarianLawandHumanRights,‘AutonomousWeaponSystemsunderInternationalLaw’(November2014)AcademyBriefingNo8.GovernmentoftheNetherlands,ParliamentaryMotionbyMemberSvenKoopmansetalios,‘Overbeheersingvandeproductie,plaatsing,verspreidingeninzetvannieuwepotentiëlemassavernietigingswapens’(19May2019)KST3369443.GroupofGovernmentalExpertsoftheHighContractingPartiestotheConventiononProhibitionsorRestrictionsontheUseofCertainConventionalWeaponsWhichMayBeDeemedtoBeExcessivelyInjuriousortoHaveIndiscriminateEffects,‘Reportofthe2017GroupofGovernmentalExpertsonLethalAutonomousWeaponsSystems(LAWS)’(20November2017)CCW/GGE.1/2017/CRP.1.GroupofGovernmentalExpertsoftheHighContractingPartiestotheConventiononProhibitionsorRestrictionsontheUseofCertainConventionalWeaponsWhichMayBeDeemedtoBeExcessivelyInjuriousortoHaveIndiscriminateEffects,‘Reportofthe2018sessionoftheGroupofGovernmentalExpertsonEmergingTechnologiesintheAreaofLethalAutonomousWeaponsSystems(LAWS)’(23October2018)CCW/GGE.1/2018/3.GroupofGovernmentalExpertsoftheHighContractingPartiestotheConventiononProhibitionsorRestrictionsontheUseofCertainConventionalWeaponsWhichMayBeDeemedtoBeExcessivelyInjuriousortoHaveIndiscriminateEffects,‘Reportofthe2019sessionoftheGroupofGovernmentalExpertsonEmergingTechnologiesintheAreaofLethalAutonomousWeaponsSystems(LAWS)’(25September2019)CCW/GGE.1/2019/3.GroupofGovernmentalExpertsonEmergingTechnologiesintheAreaofLethalAutonomousWeaponsSystems,‘Chairperson’sSummary’(19April2021)CCW/GGE.1/2020/WP.7.HarvardProgramonHumanitarianPolicyandConflictResearch(HPCR),ManualontheInternationalLawApplicabletoAirandMissileWarfare(2009).HouseofLords,SelectionCommitteeonArtificialIntelligence,‘AIintheUK:Ready,WillingandAble?’(16April2018)HLPaper100.DiegoMauri-9781802207675DownloadedfromPubFactoryat05/28/202203:48:50AMviaDukeUniversityLawLibrary

288280AutonomousweaponssystemsandtheprotectionofthehumanpersonHumanRightsWatch,‘StoppingKillerRobots.CountryPositionsonBanningFullyAutonomousWeaponsandRetainingHumanControl’(10August2020).HumanRightsWatchandInternationalHumanRightsClinic,‘MindtheGap.TheLackofAccountabilityforKillerRobots’(9April2015).HumanRightsWatchandInternationalHumanRightsClinic,‘HeedtheCall.AMoralandLegalImperativetoBanKillerRobots’(August2018).HumanRightsWatchandInternationalHumanRightsClinicatHarvardLawSchool,‘LosingHumanity.TheCaseagainstKillerRobots’(19November2012).HumanRightsWatchandInternationalHumanRightsClinicatHarvardLawSchool,‘NewWeapons,ProvenPrecedent.ElementsofandModelsforaTreatyonKillerRobots’(October2020).ICRC,RemarksandProposalsSubmittedbytheInternationalCommitteeoftheRedCross(February1949).ICRC,‘AGuidetotheLegalReviewofNewWeapons,MeansandMethodsofWarfare:MeasurestoImplementArticle36ofAdditionalProtocolIof1977’(Geneva2006).ICRC,InterpretiveGuidanceontheNotionofDirectParticipationinHostilitiesunderInternationalHumanitarianLaw(May2009).ICRC,‘AutonomousWeaponsSystems.Technical,Military,LegalandHumanitarianAspects’,ExpertMeeting(March2014).ICRC,‘AutonomousWeaponsSystems.ImplicationsofIncreasinglyAutonomyintheCriticalFunctionsofWeapons’,ExpertMeeting(March2016).ICRC,‘InternationalHumanitarianLawandtheChallengesofContemporaryArmedConflicts’,Reportofthe31stInternationalConferenceoftheRedCrossandRedCrescent(October2011).ICRC,CommentaryontheFirstGenevaConvention.Convention(I)fortheAmeliorationoftheConditionoftheWoundedandSickinArmedForcesintheField(CUP2016).ICRC,CommentarytotheSecondGenevaConvention.Convention(II)fortheAmeliorationoftheConditionofWounded,SickandShipwreckedMembersofArmedForcesatSea(CUP2017).ICRC,‘EthicsandAutonomousWeaponsSystems:AnEthicalBasisforHumanControl?’(3April2018).ICRC,CommentarytotheThirdGenevaConvention.Convention(III)fortheTreatmentofPrisonersofWar(CUP2021).ICRC,‘ContributionbytheInternationalCommitteeoftheRedCrosssubmittedtotheChairoftheConventiononCertainConventionalWeapons(CCW)GroupofGovernmentalExpertsonEmergingTechnologiesintheAreaofLethalAutonomousWeaponsSystemsasaproposalforconsensusrecommendationsinrelationtotheclarification,considerationanddevelopmentofaspectsofthenormativeandopera-tionalframework’(11June2021).ICRCandSwissFederalDepartmentofForeignAffairs,TheMontreuxDocumentonPertinentInternationalLegalObligationsandGoodPracticesforStatesRelatedtoOperationsofPrivateMilitaryandSecurityCompaniesduringArmedConflict(August2009).ICTY,‘FinalReporttotheProsecutorbytheCommitteeestablishedtoreviewtheNATObombingcampaignagainsttheFederalRepublicofYugoslavia’(8June2000)39ILM1257.ILC,‘SecondReportonStateResponsibilitybyMrRobertoAgo’(1970/II)ILCYb178.DiegoMauri-9781802207675DownloadedfromPubFactoryat05/28/202203:48:50AMviaDukeUniversityLawLibrary

289Bibliography281ILC,‘“Forcemajeure”and“FortuitousEvent”ascircumstancesprecludingwrongful-ness:SurveyofStatepractice,internationaljudicialdecisionsanddoctrine–studypreparedbytheSecretariat’(1978/II(1))ILCYb61.ILC,‘InternationalLiabilityforInjuriousConsequencesArisingoutofActivitiesnotProhibitedbyInternationalLaw’,Reportofthe30thSession(1978/II(2))ILCYb150.ILC,‘EighthReportonStateresponsibilitybyMrRobertoAgo,SpecialRapporteur–theinternationallywrongfulactoftheState,sourceofinternationalresponsibility’(1979)A/CN.4/318.ILC,Reportofthe53rdSession(2001/II(2))ILCYb148.ILC,Reportofthe56thSession(2006/II(2))ILCYb59.ILC,‘ReportoftheStudyGrouponFragmentationofInternationalLaw:DifficultiesArisingfromtheDiversificationandExpansionofInternationalLaw’(13April2006)A/CN.4/L.682.ILC,‘ThirdReportontheProtectionoftheEnvironmentinRelationtoArmedConflicts’(3June2016)A/CN.4/700.ILC,‘Provisionalsummaryrecordofthe3433rdmeeting’(19July2018)A/CN.4/SR.3433.ILC,‘Protectionoftheenvironmentinrelationtoarmedconflicts.TextandtitlesofthedraftprinciplesprovisionallyadoptedbytheDraftingCommitteeonfirstreading’(6June2019)A/CN.4/L.937.ILC,‘ReportoftheInternationalLawCommission’(29April–7June,8July–9August2019)A/74/10.ILC,‘SecondReportonGeneralPrinciplesofLawbyMarceloVásquez-Bermúdez,SpecialRapporteur’(9April2020)A/CN.4/71.InternationalPanelontheRegulationofAutonomousWeapons,‘FocusontheHuman–MachineRelationinAWS’(March2018).InternationalPanelontheRegulationofAutonomousWeapons(iPRAW),‘FocusonHumanControl’(August2019).Japan,‘PossibleOutcomeof2019GroupofGovernmentalExpertsandFutureActionsofInternationalCommunityonLethalAutonomousWeaponsSystems’(22March2019)CCW/GGE.1/2019/WP.3.Kozyulin,V,‘Russia’sAutomatedandAutonomousWeaponsandTheirConsiderationfromaPolicyStandpoint’,inICRC,‘AutonomousWeaponsSystems.ImplicationsofIncreasinglyAutonomyintheCriticalFunctionsofWeapons’,ExpertMeeting(March2016).MeetingoftheHighContractingPartiestotheConventiononProhibitionsorRestrictionsontheUseofCertainConventionalWeaponsWhichMayBeDeemedtoBeExcessivelyInjuriousortoHaveIndiscriminateEffects,‘FinalReport’(16December2013)CCW/MSP/2013/10.MeetingoftheHighContractingPartiestotheConventiononProhibitionsorRestrictionsontheUseofCertainConventionalWeaponsWhichMayBeDeemedtoBeExcessivelyInjuriousortoHaveIndiscriminateEffects,‘Reportofthe2014informalMeetingofExpertsonLethalAutonomousWeaponsSystems(LAWS)’(11June2014)CCW/MSP/2014/3.MeetingoftheHighContractingPartiestotheConventiononProhibitionsorRestrictionsontheUseofCertainConventionalWeaponsWhichMayBeDeemedtoBeExcessivelyInjuriousortoHaveIndiscriminateEffects,‘Reportofthe2015InformalMeetingofExpertsonLethalAutonomousWeaponsSystems(LAWS)’(2June2015)CCW/MSP/2015/3.DiegoMauri-9781802207675DownloadedfromPubFactoryat05/28/202203:48:50AMviaDukeUniversityLawLibrary

290282AutonomousweaponssystemsandtheprotectionofthehumanpersonMeetingoftheHighContractingPartiestotheConventiononProhibitionsorRestrictionsontheUseofCertainConventionalWeaponsWhichMayBeDeemedtoBeExcessivelyInjuriousortoHaveIndiscriminateEffects,‘FinalReport’(13December2019)CCW/MSP/2019/9.NATO,‘AlliedJointDoctrineforJointTargeting.EditionAVersion1’(April2016)AJP-3.9.OHCHR,‘GuidingPrinciplesonBusinessandHumanRights:ImplementingtheUnitedNations‘Protect,RespectandRemedy’Framework’(2011)HR/PUB/11/04.OHCHR,TheMinnesotaProtocolontheInvestigationofPotentiallyUnlawfulDeaths(2016):TheRevisedUnitedNationsManualontheEffectivePreventionandInvestigationofExtra-legal,ArbitraryandSummaryExecutions(Geneva2017).‘ProcedureforApprovingDirectActionagainstTerroristTargetslocatedoutsidetheUnitedStatesandAreasofActiveHostilities’(22May2013).‘ReportoftheSpecialRapporteuronextrajudicial,summaryorarbitraryexecutionsonagender-sensitiveapproachtoarbitrarykillings,AgnesCallamard’(6June2017)A/HRC/35/23.‘ReportoftheSpecialRapporteuronthepromotionandprotectionofhumanrightsandfundamentalfreedomswhilecounteringterrorism,BenEmmerson’(14April2014)A/HRC/25/59.‘ReportoftheUnitedNationsSpecialRapporteuronextrajudicial,summaryorarbi-traryexecutions,PhilipAlston’(28May2010)A/HRC/14/24/Add.6.‘ReportoftheUnitedNationsSpecialRapporteuronextrajudicial,summaryorarbi-traryexecutions,ChristofHeyns’(2013)A/HRC/23/47(HeynsReport).RussianFederation,‘Russia’sApproachestotheElaborationofaWorkingDefinitionandBasicFunctionsofLethalAutonomousWeaponsSystemsintheContextofthePurposesandObjectivesoftheConvention’(4April2018)CCW/GGE.1/2018/WP.6.Switzerland,‘A“Compliance-Based”ApproachtoAutonomousWeaponsSystems’(10November2017)CCW/GGE.1/2017/WP.9.TheIEEEGlobalInitiativeonEthicsofAutonomousandIntelligentSystems,‘EthicallyAlignedDesign(EAD):AVisionforPrioritizingHumanWell-BeingwithAutonomousandIntelligentSystems’(2019).TheNetherlands,‘ExaminationofVariousDimensionsofEmergingTechnologiesintheAreaofLethalAutonomousWeaponsSystems,intheContextoftheObjectivesandPurposesoftheConvention’(9October2017)CCW/GGE.1/2017/WP.2.UK,‘HumanMachineTouchpoints:TheUnitedKingdom’sPerspectiveonHumanControloverWeaponDevelopmentandTargetingCycles’(8August2018)CCW/GGE.2/2018/WP.1.UKMoD,‘JointDoctrineNote2/11:TheUKApproachtoUnmannedAircraftSystems’(JDN2/11,2011).UKMoD,JointDoctrinePublication0-30.2:UnmannedAircraftSystems(JDP0-30.2,2017).UKMoD,‘JointConceptNote1/18.Human–MachineTeaming’,JCN1/18(2018).UNECOSOC,‘PrinciplesontheEffectivePreventionandInvestigationofExtra-Legal,ArbitraryorSummaryExecutions’Resolution1989/65(24May1989).UNFact-FindingMissionontheGazaConflict,‘ReportonHumanRightsinPalestineandOtherOccupiedTerritories’(25September2009)A/HRC/12/48.UNHumanRightsCouncil,‘ReportoftheIndependentInternationalCommissionofInquiryontheSyrianArabRepublic’(5February2015)A/HRC/28/69.DiegoMauri-9781802207675DownloadedfromPubFactoryat05/28/202203:48:50AMviaDukeUniversityLawLibrary

291Bibliography283UNOEIWGChairmanship,‘LegallyBindingInstrumenttoRegulate,inInternationalHumanRightsLaw,theActivitiesofTransnationalCorporationsandOtherBusinessEnterprises’(16July2018).UNOEIWGChairmanship,‘RevisedDraft,LegallyBindingInstrumenttoRegulate,inInternationalHumanRightsLaw,theActivitiesofTransnationalCorporationsandOtherBusinessEnterprises’(16July2019).UNOEIWGChairmanship,‘SecondRevisedDraft,LegallyBindingInstrumenttoRegulate,inInternationalHumanRightsLaw,theActivitiesofTransnationalCorporationsandOtherBusinessEnterprises’(6August2020).UNSecretaryGeneral,‘ReportoftheSecretary-Generalontheprotectionofciviliansinarmedconflict’(22November2013)S/2013/689.UNSecretaryGeneral,‘Protectionofciviliansinarmedconflict.ReportoftheSecretary-General’(6May2020)S/2020/266.UNGA,‘Therighttoprivacyinthedigitalage’(18December2013)A/RES/68/167.UNGA,‘ConventiononProhibitionsorRestrictionsontheUseofCertainConventionalWeaponsWhichMayBeDeemedtoBeExcessivelyInjuriousortoHaveIndiscriminateEffects.ReportoftheFirstCommittee’(15November2019)A/74/372.UNIDIR,‘TheWeaponizationofIncreasinglyAutonomousTechnologies:ConsideringHowMeaningfulHumanControlmightmovethediscussionforward’(2014).US,‘Human–MachineInteractionintheDevelopment,DeploymentandUseofEmergingTechnologiesintheAreaofLethalAutonomousWeaponsSystems’(28August2018)CCW/GGE.2/2018/WP.4.USCustomsandBorderProtection,‘CBP’sAutonomousSurveillanceTowersDeclaredaProgramofRecordalongtheSouthwestBorder’(2July2020).USDoD,‘DirectiveNumber3000.09’(21November2012,revisited8May2017).USDoD,LawofWarManual(June2015,updatedDecember2016).USDoDDefenseScienceBoard,‘SummerStudyonAutonomy’(June2016).Venezuela,‘GeneralPrinciplesonLethalAutonomousWeaponsSystems.WorkingPaperSubmittedonBehalfoftheNonAlignedMovement(NAM)andOtherStatesPartiestotheConventiononConventionalWeaponsGroupofGovernmentalExpertsonLethalAutonomousWeaponsSystems’(28March2018)CCW/GGE.1/2018/WP.1.‘WorkingPapertobeSubmittedbytheBolivarianRepublicofVenezuelaonBehalfoftheNon-AlignedMovement(NAM)andOtherStatesPartiestotheConventiononCertainConventionalWeapons(CCW)’(June2021).DiegoMauri-9781802207675DownloadedfromPubFactoryat05/28/202203:48:50AMviaDukeUniversityLawLibrary

292Indexabsoluteliability224–30ARSIWA(DraftArticlesontheaccountabilitygapsandresponsibilityResponsibilityofStatesfor176–80,233–4,257–8InternationallyWrongfulActs)commandorsuperiorresponsibility212–14,217–19,229,232188–92Article36(NGO)38,138constructingmensrearequirementartificialintelligence(AI)7,8,120,134,forcrimescommitted180–81,204–5,208–9,212–13,184–92214,259corporateresponsibilityunder‘attack’,useofterm36–7,140internationallaw199–211attributionofwrongdoing6,212–15establishinganeffectiveframeworkautomatedweapons/weaponssystems27,230–3328,29,253–4holdingautonomousweaponsautomationandautonomy30–31systems(AWS)accountableautonomousweaponssystems(AWS)180–82applicabilityofweaponslaw123–4internationalcriminallaw(ICL)definitionsof14,17,23–4,27–30,182–9933andmeaningfulhumancontroldevelopmentofinternationaldebate(MHC)242on,overview15–24modesofperpetration192–9technologicaldevelopmentofseealsostateresponsibility118–21actusreus182,183,186,193,194–5,autonomy197,198–9advantagesof118–19adimpossibilianemotenetur190,‘boxedautonomy’241217andcriticalfunctions33Aegiscombatsystem120,200dynamicconceptionsof30–32aerialwarfare4meaningof,inmachines24–7AfricanCommissiononHumanandmeasuringandevaluatingdegreeofPeoples’Rights(ACommHPR)30–3273,91,237–8staticconceptionsof27–30aidingandabetting194–7Akerson,D.147Benvenuti,P.169algorithms26,53–4,85,91–3,101,biologicalandchemicalweapons132,109–10,145,147,151–2,161,133,258259‘boxedautonomy’241AlienTortStatute(ATS)205–6brain-computerinteractions255arbitrariness88–94Brazil40‘armedattack’,asconcept46armeddrones6–8,14,18–19,54,100,CampaigntoStopKillerRobots(CSKR)124,253–417,19,22–3,39,250–51284DiegoMauri-9781802207675DownloadedfromPubFactoryat05/28/202203:48:52AMviaDukeUniversityLawLibrary

293Index285Cassese,A.128,129,133,166–7,168,deprivationoflife,seerighttolife173–4‘directlyparticipateinhostilities’(DPH)Chechnya82146chemicalandbiologicalweapons132,discrimination,rightsagainst100–101133,258distinction,principleof139–40,Chengeta,T.39,40,111143–8China34dolusdirectus,indirectusandeventualischivalry3184–8,193,194,198civilliability180–81DraftArticlesontheResponsibilitycollateraldamage149–50,152,155ofStatesforInternationallycombatantsWrongfulActs(ARSIWA)definitionof140–41,145,163212–14,217–19,229,232identificationof145–6drones6–8,14,18–19,54,100,124,commandorsuperiorresponsibility253–4188–92duediligence81,84,153–4,203,compensation87,200,229,231,232–3209–10,211–12,214–15,216,‘complementarity’theory50220–24‘continuouscombatantfunction’(CCF)Dunant,Henry123146dynamite4ConventiononCertainConventionalWeapons(CCW)15,19–22,23–4,‘electronicpersonhood’180–8234,53,107–8,131,141,176–7,environment,prohibitiononweapons181–2,214,235,251–2;seealsoseverelydamaging134–6GroupofGovernmentalExpertsEstonia29,33–4,42(GGE);MeetingofExperts(MoE)EuropeanConventiononHumanRightsConventiononInternationalLiabilityfor(ECHR)70,71,72,73,74,75,DamageCausedbySpaceObjects105,116227EuropeanCourtofHumanRightscorporateresponsibilityunder(ECtHR)71,73,74,77,78–9,internationallaw199–20180–81,82–4,87,88,93,96–7,existingformsof201–599–100,101,103,105,115,116,mainobstaclestocorporate159,160,222–3responsibilityindomesticEuropeanParliament18–19,23,180,legalsystems205–10209–10,238overcomingchallenges210–11EuropeanUnion(EU),andcorporatecriminalcorporateresponsibility201–2responsibility206,207–10criminallaw181;seealsointernationalexcessiveness150–51criminallaw(ICL)‘externality’ofcircumstance218–19criticalfunctions33–7extraterritoriality113–16crossbows3culpability(culpa)184,188–92,193,falsepositives178–9,208,219,232197–9fault/no-fault224–30cyberweapons5–6,7FederalTortClaimsAct(FTCA)207cyberneticenhancement255,258;seeFinland29,33–4,42alsohumanenhancement‘fire-and-forget’munitions28,119,156dataforcemajeure216–19,222,224,225,collection99–100,160229protection92–4foreseeability218‘definitemilitaryadvantage’145France40–41DiegoMauri-9781802207675DownloadedfromPubFactoryat05/28/202203:48:52AMviaDukeUniversityLawLibrary

294286Autonomousweaponssystemsandtheprotectionofthehumanpersongeneralprinciples,roleinre-statingascontestedprinciple,overviewofinternationallawprotectingthe103–7humanperson243–52,257–8andhumanenhancement256GenevaConventionsandProtocols51,andinhumantreatment9852humanenhancement138,254–6,257;dutytoinvestigateandprosecuteseealsocyberneticenhancement157–8humanjudgementandMartensClause165andhumandignity109targeting36–7,142,143,144,146,andproportionalityofuseofforce150,154–578–80weapons123,125,126–7,131–2,human-machineinteractions,asconcept133–4,136–40256–7wrongfulintent187–8human-machinepartnership140–41Germany40–41humanpresence8,26–7,28–9,249–50,‘government-contractordefense’206–7257–8GroupofGovernmentalExperts(GGE)HumanRightsCouncil(UN)15–16,17,20–22,34,41–2,47,122,171,82,202–3177,235,236HumanRightsWatch(HRW)16–17,gunpowder3–428–9,39,171–2,240,250humanunderstandingoffunctioningofHaguePeaceConferencesandautonomousweaponssystemsConventions125,126,131,154,(AWS)85–6,91–2,160–61162–3,165,170humanity10,52,62–5,161–2,175,246‘harmonization’theory50–51indebateonautonomousweaponsheteronomy25,27systems(AWS)170–74Heyns,C.107,108,110historicaloverviewofprincipleHeynsReport(ReportoftheUnited162–5NationsSpecialRapporteurandhumanenhancement256onExtrajudicial,SummaryorlegalrelevanceinInternationalArbitraryExecutions)15–16,HumanitarianLaw(IHL)17–18,19,28,88,108,176165–70HolySee40,47,110,171,249–50andweaponslaw139hope109–10humancontrol37–44,258–9identificationofcombatants145–6andaccountabilitygaps233Iliad2andarbitrariness88–9,93–4illegalityofautonomousweaponsandautonomy24systems(AWS),argumentsbasedandindiscriminateattacks146–8on248andInternationalHumanitarianLawinforodomestico57(IHL)174–5indiscriminateattacks,prohibitiononandjustificationofforce97142–53andprecautionaryprinciple156–7indiscriminateweapons,prohibitiononandtargetingprocess36,37–41131–4seealsomeaningfulhumancontrolintent178,185,187–8,193(MHC)Inter-AmericanCourtofHumanRightshumandignity10,52,62–4,65,102–3,(IACtHR)90–91,105–6,115117,175,243,246internationaladjudicatorybodies,andautonomousweaponssystemsautonomousweaponssystems(AWS)107–12(AWS)before244–8DiegoMauri-9781802207675DownloadedfromPubFactoryat05/28/202203:48:52AMviaDukeUniversityLawLibrary

295Index287internationalarmedconflict(IAC),andrightnottobesubjecttotorture,internationallaw113–14,116inhuman,ordegradingInternationalCommitteeforRobotstreatment94–8ArmsControl(ICRAC)241anduseofforce48–54,257InternationalCommitteeoftheRedseealsorighttolifeCross(ICRC)16,27,33,39,51,InternationalHumanitarianLaw(IHL)63,123,139,146,169–70,171,17,23,66,174–5172,204,242,249,252andduediligence220–23InternationalCourtofJustice(ICJ)dutytoinvestigateandprosecute48–50,51–2,56–7,125–6,132,157–61164,167–8,170,246–7andhumanenhancement255–6InternationalCovenantonCivilandandhumanity62,63–4,65,161–74PoliticalRights(ICCPR)70,71,andmeaningfulhumancontrol72,75,89,90–91(MHC)242InternationalCriminalCourt(ICC)andmilitaryapplicationsofStatute(ICCSt)183,184–8,190,autonomousweapons191,193,195,196systems(AWS),overviewofinternationalcriminallaw(ICL)182–4,118–22230,231relationshipwithInternationalconstructingmensrearequirementHumanRightsLaw(IHRL)forcrimescommitted184–92anddutytoinvestigateandmodesofperpetration192–9prosecute158–60InternationalCriminalTribunalfortherelationshipwithInternationalFormerYugoslavia(ICTY)52,HumanRightsLaw(IHRL)133,151,168,188,190,192–3,andextraterritoriality113–16194–5,197remedies231–2InternationalHumanRightsClinicatanduseofforce48–54,257HarvardLawSchool17,250seealsotargetinglaw;weaponslawInternationalHumanRightsLaw(IHRL)InternationalLawCommission(ILC)66,116–1755–6,57–8,59,60,107,135,212,andcorporateresponsibility202–4222,225,226–7,229,233developmentofcaselaw244–5InternationalMilitaryTribunal(IMT)anddiscrimination100–101126,197,201–2andduediligence220–23internationalorganizations(IOs)179–80andHeynsReport17InternationalPanelontheRegulationofandhumandignity62,63–4,65,AutonomousWeapons(iPRAW)102–12241andhumanenhancement255–6internationalresponsibility,seeimportanceof67accountabilitygapsandandmeaningfulhumancontrolresponsibility(MHC)237–8,242investigationandprosecutionduty83–8,andprivacy98–100157–61relationshipwithInternationalirresistibility218HumanitarianLaw(IHL)anddutytoinvestigateandJapan34,128prosecute158–60jointcriminalenterprise(JCE),doctrinerelationshipwithInternationalof192–4HumanitarianLaw(IHL)andjurisdictionalbarrierstoestablishingextraterritoriality113–16corporateresponsibility205–6remedies231–2jusadbellum45–7DiegoMauri-9781802207675DownloadedfromPubFactoryat05/28/202203:48:52AMviaDukeUniversityLawLibrary

296288Autonomousweaponssystemsandtheprotectionofthehumanpersonjusinbello48,121constructingrequirementforcrimescommittedthrough‘killswitch’156autonomousweaponsKolb,R.49–50systems(AWS)184–92militaryadvantage150militarynecessity127–8lawenforcement53–4,67–8,72militaryobjectives144–5andarbitrariness93,94‘misdoings’,seeaccountabilitygapsanddutyofprecautionregardingchoiceresponsibilityofweapons80–83moralautonomy25andhumandignity105–6,110morality25,172necessityoflethalforce74–7negotiationsandsurrender76proportionalityoflethalforceNATO(NorthAtlanticTreaty77–80Organization)36,253rightnottobesubjecttotorture,naturalenvironment,prohibitiononinhuman,ordegradingweaponsseverelydamagingtreatment94–8134–6legalpositivism57,58–9,61,167–8naturallaw57,61,167–8legalrulesandprinciples,dichotomynegligence86–8,178,188–92,222between54–62Netherlands29lethalautonomousweaponssystemsnon-arbitrariness88–94(LAWS),useofterm19non-internationalarmedconflict(NIAC),lethality33–5andinternationallaw51,113–14,lexspecialisapproach49–50116Libya115–16non-lethalweapons(NLW)35,82–3LosingHumanityreport16–18,28–9nuclearweapons1–2,4–5,48–9,125–6,128,132,168,246machineguns4maliciousexploitations,protectionofobserve,orient,decide,act(OODA)loopautonomousweaponssystems31,32(AWS)from214–15Open-EndedIntergovernmentalWorking‘manyhands’problem183,192Group(OEIWG)203–4MartensClause139,162–4,165–70,171,173PermanentCourtofInternationalJusticemeaningfulhumancontrol(MHC)(PCIJ)56,5738–43,178,236perpetration,modesof192–9legalstatus43–4,237–40personalitystrikes253MeetingofExperts(MoE)debateprecaution,dutyof80–83,153–7,on20220–21political-diplomaticvalueof240–43Predator124inproposedregulation251,258privacy98–100reappraising,beyondautonomousdataprotection92–4weaponssystems(AWS)privatemilitaryandsecuritycontractors252–6(PMSCs)183–4,200,204seealsohumancontrolproductliability207–9MeetingofExperts(MoE)19–20,38,prohibitedweapons,useof18640–41,107–8,171,172proportionality77–80,149–53mensrea182,183,193,194–5,196,197,prosecutionandinvestigationduty83–8,198–9157–61DiegoMauri-9781802207675DownloadedfromPubFactoryat05/28/202203:48:52AMviaDukeUniversityLawLibrary

297Index289protectionofautonomousweaponsSparrow,R.76,147systems(AWS)frommaliciousstateresponsibility211–12exploitations214–15attributionofawrongdoingcommittedthroughreasonableness151,153,190,221autonomousweaponsrecklessness185systems(AWS)212–15regulatingautonomousweaponssystemsbreachofaninternationalobligation(AWS),makingcasefortreatybyautonomousweapons248–52,258systems(AWS)215–24remedies,establishinganeffectiveduediligence220–24frameworkfor230–33forcemajeure216–19,222,224,remoteness2–3,4,7225,229responsibility,seeaccountabilityovercomingchallenges(caseforgapsandresponsibility;state‘absoluteliability’regimeforresponsibility‘misdoings’)224–30reviewofweapons,dutytoconductstatesovereignty164136–9,250SunTzu259rightnottobesubjecttotorture,superfluousinjuryandunnecessaryinhuman,ordegradingtreatmentsufferingrule(SIrUSrule)126–3194–8superiororcommandresponsibilityrighttolife69–72188–92arbitrariness88–94surrender76,147andduediligence223surveillance99–100dutyofprecautionregardingchoiceswarming120–21,255ofweapons80–83Syria82dutytoinvestigateandprosecute83–8targetselection33,35legality73–4and‘armedattack’46necessity74–7attributionofwrongdoing213negativeobligations72–80anddatacollection100positiveobligations80–88andhumandignity111proportionality77–80targetinglaw139–41Roorda,M.41‘categorizing’targets(principleofrulesandprinciples,dichotomybetweendistinction)143–854–62duediligence221Russia29–30dutytotakeprecautionsinattack153–7SaintPetersburgDeclaration124–5,142,andgeneralprinciples245149,164–5principleofproportionality149–53SantonideSio,F.41prohibitiononindiscriminateattacksSchmitt,Carl1142–53Schmitt,M.N.64,134,151–2targetingprocess36–41,253–4self-defense45,46,78transnationalcorporations202–3semi-autonomousweaponssystems28,treatyregulatingautonomousweapons29–30systems(AWS),makingcasefor‘separation’theory49,53248–52,258Sharkey,N.E.39–40Turkey115–16signaturestrikes37,68,100,253–4SouthKorea68Ulgen,O.109,111DiegoMauri-9781802207675DownloadedfromPubFactoryat05/28/202203:48:52AMviaDukeUniversityLawLibrary

298290AutonomousweaponssystemsandtheprotectionofthehumanpersonUNBasicPrinciples73,74,76,77,78,jusadbellumdimensionof45–781,82,231,232legality73–4UNDraftCodeofConductonnecessity74–7TransnationalCorporations202proportionality77–80UNHumanRightsCouncil15–16,17,rightnottobesubjecttotorture,82,202–3inhuman,ordegradingUNSecretary-General(UNSG),reporttreatment94–8ontheprotectionofciviliansinseealsotargetinglaw;weaponslawarmedconflict(2013)18UnitedKingdom(UK)16,27–8,34,42vandenHoven,J.41UnitedNationsCharter(UNC)45UnitedNationsGeneralAssemblywarcrimes(UNGA)FirstCommittee22–3dutytoinvestigateandprosecuteUnitedStates(US)41–2,205–7,208157–61UniversalDeclarationonHumanRightsseealsointernationalcriminallaw(UDHR)48,69–70,103–4(ICL)unmannedweaponssystems,seedrones‘warnexus’196–7unpredictability218weapons,evolutionandcontinuingUSDepartmentofDefense(DoD)developmentof1,3–916–17,28,41weaponslaw122–6USDepartmentofDefense(DoD)dutytoconductlegalreviewofDefenseScienceBoardreportweapons136–9,250(2016)118–19prohibitiononindiscriminateuseofforce34–7,44–5weapons131–4arbitrariness88–94prohibitiononweaponscausingandduediligence223superfluousinjuryanddutyofprecaution80–83,153–7,unnecessarysuffering(SIrUS220–21rule)126–31dutytoinvestigateandprosecuteprohibitiononweaponsseverely83–8,157–61damagingnaturalevolutionof2–5,8–9environment134–6andhumandignity102–12wilfulness187andhumanity161–74IHRL/IHLdimensionof48–54,257ZyklonBcase195–6DiegoMauri-9781802207675DownloadedfromPubFactoryat05/28/202203:48:52AMviaDukeUniversityLawLibrary