资源描述：
《DESIGN AND IMPLEMENTATION OF A HIGH-PERFORMANCE NETWORK INTRUSION PREVENTION SYSTEM设计与高智能实现》由会员上传分享，免费在线阅读，更多相关内容在学术论文-天天文库。
DESIGNANDIMPLEMENTATIONOFAHIGH-PERFORMANCENETWORKINTRUSIONPREVENTIONSYSTEM121KonstantinosXinidis,KostasG.Anagnostakis,EvangelosP.Markatos1InstituteofComputerScience,FoundationforResearchandTechnologyHellas,P.OBox13852Heraklio,GR-711-10Greece{xinidis,markatos}@ics.forth.gr;DistributedSystemsLaboratory,rdCISDepartment,Univ.ofPennsylvania,200S.33Street,Philadelphia,PA19104anagnost@dsl.cis.upenn.eduAbstract:Networkintrusionpreventionsystemsprovideproactivedefenseagainstsecuritythreatsbydetectingandblockingattack-relatedtraffic.Thistaskcanbehighlycomplex,andtherefore,software-basednetworkintrusionpreventionsystemshavedifficultyinhandlinghighspeedlinks.Thispaperdescribesthedesignandimplementationofahigh-performancenetworkintrusionpreventionsystemthatcombinestheuseofsoftware-basednetworkintrusionpreventionsensorsandanetworkprocessorboard.Thenetworkprocessoractsasacustomizedloadbalancingsplitterthatcooperateswithasetofmodifiedcontent-basednetworkintrusiondetectionsensorsinprocessingnetworktraffic.Weshowthatthecomponentsofsuchasystem,ifco-designed,canachievehighperformance,whileminimizingredundantprocessingandcommunication.Wehaveimplementedthesystemusinglow-cost,off-the-shelftechnology:anIXP1200networkprocessorevaluationboardandcommodityPCs.Ourevaluationshowsthatourenhancementscanreducetheprocessingloadofthesensorsbyatleast45%resultinginasystemthatcanhandleafully-loadedGigabitEthernetlinkusingatmostfourcommodityPCs.Keywords:NetworkIntrusionDetectionSystems,NetworkIntrusionPreventionSystems,networkprocessors,loadbalancing1.INTRODUCTIONTheincreasingimportanceofnetworkinfrastructureandservicesalongwiththehighcostanddifficultyofdesigningandenforcingend-systemsecuritypolicieshasresultedingrowinginterestincomplementary,network-levelsecuritymechanisms,asprovidedbyfirewallsandnetworkintrusiondetectionandpreventionsystems.High-performancefirewallsarerathereasytoscaleuptocurrentedge-networkspeedsbecausetheiroperationinvolvesrelativelysimpleoperationssuchasmatchingasetofAccessControlList-typepolicyrulesagainstfixed-sizepacketheaders.Unlikefirewalls,networkintrusionpreventionsystems(NIPSes)aresignificantlymorecomplexand,asaresult,arelaggingbehindroutersandfirewallsinthetechnologycurve.Thecomplexitystemsmainlyfromtheneedtoanalyzenotjustheadersbutalsopacketcontentandhigher-levelprotocols.Moreover,thefunctionofNIPSesneedstobeupdatedwithnewdetectioncomponentsandheuristics,duetothecontinuouslyevolvingnatureofnetworkattacks.Bothcomplexityandtheneedforflexibilitymakeithardtodesignhigh-performanceNIPSes.Application-SpecificIntegratedCircuits(ASICs)lacktheneededflexibilitywhilesoftware-basedsystemsareinherentlylimitedintermsofperformance.Onedesignthatoffersbothflexibilityandperformanceistheuseofmultiplesoftware-basedsystemsbehindahardware-basedloadbalancer.Althoughsuchadesigncanscaleuptoedge-networkspeeds,it 2K.Xinidis,K.G.Anagnostakis,E.P.Markatosstillrequiressignificantresources,intermsofthenumberofsensors,requiredrack-space,etc.Itisthereforeimportanttoconsiderwaysofimprovingtheperformanceofsuchsystems.Thispaperexplorestherolethathigh-speednetworkprocessors(NPs)canplayinscalingupnetworkintrusionpreventionsystems.WefocusonwaysforexploitingtheperformanceandprogrammabilityofNPsforboostingin-linenetworkintrusiondetection.WedescribethearchitectureofaNIPSusingcommodityPersonalComputers(PCs)asnetworkintrusion8detectionsensors,fedbyanIXP1200networkprocessor.Wepresenttheallocationofoperationstocomponentsandthetrade-offswefacedduringdesigningandprototypingthe20system.Forfurtherdetailspleasereferto.Therestofthispaperisorganizedasfollows.InSection2wedescribethearchitectureandimplementationofoursystem,calledDigenisa.InSection3weexaminetheperformancebenefitsofusingNP-basedloadbalancingandacceleration.Wediscussworkthatisrelatedtohigh-performanceintrusionpreventioninSection4.Finally,wesummarizeandcommentonfutureresearchdirectionsinSection5.2.DESIGNANDIMPLEMENTATIONWefacedanumberofdesignchallengesinconstructingDigeniswithrespecttoperformance,flexibilityandscalability:Performance:TheprimarymetricofinterestinthedesignofaNIPSisthroughput.Thatis,tobeabletooperateatnetworkspeedsofatleast1Gbit/swithoutpacketlosses,soastodetectanyattemptedattack.Therefore,thesystemmustbecapableofanalyzingalltheincomingtrafficunderthemoststringentconditions.Networkintrusiondetectionsystems2,5(NIDSes)basedoncommodityPCsareabletomonitoratspeedsmuchlowerthan1Gbit/s.Thisnecessitatestheuseofadistributeddesignwithseveralintrusiondetectionsensors4,11,19operatinginparallelandsupportedbyaloadbalancingtrafficsplitter.Atthesametime,wewanttominimizecostanduseasfewresourcesaspossible.TheuseofanNPimplementingthesplitterappearsreasonable,sinceitislikelytobecheaperthanacustomASIC,whileloadbalancingoperationsseemtobewellwithintheprocessingcapacityofmodernNPs.Wealsowanttominimizethenumberofsensorsneeded.AkeyfocusofourworkisthereforeonhowtoexploittheprocessingcapacityontheNPtoreducetheloadofthesensors.AsecondimportantperformancegoalisminimizingthelatencyinducedbytheNIPS.ThereisadirectrelationshipbetweenlatencyintroducedbyanetworkingdeviceandthemaximumthroughputofTCPflowsb.IftheNIPSwillbeusedattheboundarybetweenanenterprisenetworkandtheInternet,latenciesintheorderofafewmillisecondsmaybetolerable.IftheNIPSisdeployedinternally,andthenetworkneedstosupporthigh-bandwidthlocalservices(suchasfilesharing,etc.)thelatencyrequirementsareevenmorestringent.Particularly,thereisacriticalvaluefortheroundtriptime(RTT)ofapacketineachnetwork.Ifthelatencyisbelowthiscriticalvalue,TCPthroughputisunaffected--itisthelinespeedoftheunderlyingnetworkwhichbecomesthebottleneck--abovethiscriticalvalue,however,TCPthroughputisnegativelyimpacted.ThecriticalvalueforRTTinanetworksupportingGigabitspeedsis0.5milliseconds.Thus,ifwewantthethroughputofTCPtobeunaffected,wemustensurethatthelatencyimposedbyourNIPSislessthan0.5milliseconds.However,GigabitEthernetlinkswillrarelycarryonlyasingleTCPconnection.Rather,aGigabitEthernetlinksupportshundreds,ifnotthousandsofTCPconnections,andthis9multiplexingmitigatestheimpactoflatencyontheoverallthroughputofthelink.Inotherwords,itispossibleaDigenisAkritas,theidealmedievalGreekhero,isaboldwarrioroftheEuphratesfrontier.HewasaproficientwarriorbytheageofthreeandspenttherestofhislifedefendingtheByzantineEmpirefromfrontierinvaders.bRecallthatTCPThroughput=Window/RTTwhereWindowisthemaximumTCPwindowsize(defaultvalueis64Kbytes)andRTTistheroundtriptimeinthenetwork. DesignandImplementationofaHigh-PerformanceNIPS3SplitterSensorsEntryPointG0LoadF0P0BalancerPluinsP1GigabitF1Ethernetg-InterfacesP2F2PNG1FNExitPointFastEthernetInterfacesFigure1.ArchitectureofDigenis.toimposelatencygreaterthan0.5millisecondswithoutaffectingthethroughputofalinkduetothehighnumberofTCPconnections.FlexibilityandScalability:ANIPSneedstobeflexibleandscalable,bothforscalinguptohigherlinkspeedsandmoreexpensivedetectionfunctions,aswellasforupdatingthedetectionheuristics.Iftheprotectionofafasterlinkoramorefine-graineddetectionisrequired,itwouldbedesirabletoreuseasmuchaspossibleoftheexistinghardware.Clearly,thispropertydoesnotholdforASIC-basedNIPSes.However,itisremarkablethatalmostall8,13,19NIPSprovidersignorethisdimension.Furthermore,aprerequisiteofflexibilityissimplicityasextendingacomplexsystemmaybehardanderror-prone.Itisthereforedesirableforthehard-to-programelementsofoursystemtobeasgenericaspossible.2.1ArchitectureDigenisiscomposedofacustomizedloadbalancingsplitterandanumberofcontent-basednetworkintrusiondetectionsensorsconnectedwiththesplitter(Figure1).Thesplitteristheentryandexitpointofthetrafficthatrunsthroughthesystem.Thebasictaskofthesplitteristoevenlydistributethetrafficacrossthesensorsandtotransmitthenon-attackpacketsbacktotheirdestination.Thesensorsareresponsiblefortheheavytaskofinspectingthetrafficforintrusionattempts.Theymaintaintherequiredinformationforrecognizingallthemalicioustrafficanddecidingwhethertoforwardordropthepacket.Foreveryinputpacket,thesplittercomputeswhichsensorwillberesponsibletoanalyzethispacket.Then,itforwardsthepackettothissensorforinspection.Thesensorsearchesforknownattackpatternscontainedinthepacket.Ifapatternisfound,thenthepacketisblocked,otherwisethepacketisforwardedbacktothesplitter.Thesplitterreceivestheanalyzedpacketandtransmitsittoitsdestination.Additionally,Digenissupportsplug-insthatimplementoperationsnecessarytoimprovetheperformanceofthesystem.Aplug-inhastwoparts,onerunningonthesplitterandonerunningonthesensors.Thesetwopartscooperateinordertoaccomplishtheirtask.Inthecontextofthisworkwehavedesignedaplug-inforDigenisthatattemptstominimizethecostofsendingapacketfromasensortothesplitter.Splitter:Thefunctionalityofthesplittercanbedividedintothebasicoperationsandtheplug-insthatprovideadequateoperationstoboostperformance.Thebasicpartofthesplitterintegratesthefunctionalityofaloadbalancer--itisresponsiblefordistributingtheincomingtrafficacrosstheoutputinterfaces(ports).However,itdiffersfromacommonloadbalancerinthatitmustbeflow-preserving,thatis,allthepacketsbelongingtothesameflowcmustbeforwardedtothesameoutputinterface.cIncaseofTCP/UDPtraffic,wedefineaflowtoconsistofallthetrafficofaTCPorUDPconnection.Otherwise,aflowconsistsofallthetrafficoriginatingfromaparticularIPaddressanddestinedtoaparticularIPaddress. 4K.Xinidis,K.G.Anagnostakis,E.P.MarkatosRegardingloadbalancing,therearetwopossibleapproachesthatwecoulduse:stateful3,10,16loadbalancingthatrequiresfromthesystemtoholdstateandhash-basedloadbalancingthatexperiencesgreaterloadimbalances.Forthepurposesofthispaper,weassumethatloadimbalancesaretolerableandusethesimplerhash-basedmethod.TheinputofthehashfunctioniscomposedofthesourceanddestinationIPaddressesofthepacket.Sensor:AsensorisacommodityPCthatrunsamodifiedpopularNIDSandisconnectedwiththesplitter(throughanEthernetconnection).Asensorreceivestrafficfromthesplitterandanalyzesitforpossibleknownattacks.Incasethatanattackisfound,itnotifiesthesplittertoblocktheoffendingpacket(s),otherwiseitinformsthesplitterthatthepacket(s)shouldbeforwarded.Asensormaintainsstateaboutthetrafficitanalyzesinordertooperatecorrectly.ThemaintainedstateincludestheactiveTCPconnectionsithascapturedinthenearpast,TCPconnectionstaggedasoffending,fragmentedpacketsandstatisticsabouttheconnectionspersecondtoTCP/UDPdestinationports.2.1.1ReducingRedundantPacketTransmissionWehavedesignedaplug-inforDigenisthatisresponsibleforreducingredundantpackettransmissiononthesystem.Theideabehindthisplug-inisthefollowing:Supposethatthesplitterstorestemporarily(forafewmilliseconds)thepacketsthatitforwardstothesensorsforanalysis.Thenthereisnoneedforthesensorstosendbacktothesplittertheanalyzedpacket,butonlyauniqueidentifierofthatpacket(PID).BecausethesplitterhaspreviouslystoredthepacketwiththisPID,itcaninferthereferencedpacketandforwardittotheappropriatedestination.TheonlyextraworkforthesplitteristotageachpacketwithaPID,whichisatrivialtask.Althoughtheadditionalprocessingcosttothesplitterfromthisplug-inisminimal,thereductiontotheloadofthesensorsisremarkable.However,thistechniquerequiresfromthesplittertobeequippedwithadditionalmemoryforthebufferingofthepackets.AswewillpresentinSection3,thememoryrequirementsareeasilysatisfiedbymodernNPs.Subsequently,wediscusshowasensorcommunicatesthepacketinformationbacktothesplitter.CommunicationbetweenSplitter-Sensor:Thesplittercommunicateswiththesensorsinordertodecidetheactionthatshouldbeperformed,thatis,forwardordropthepacket.Thisisdonewithacknowledgments(ACKs)fromthesensorstothesplitter.AnACKisanordinaryEthernetpacket.ItconsistsofanEthernetheader,followedbytwobytesdenotingthenumberofpacketsacknowledged(ACKfactor)andfollowedbyasetoffour-byteintegersrepresentingthePIDs.ThereareotherpossibleformatsrequiringlessbytesandsupportinghigherACKfactorsforthisconfiguration.However,thisapproachismorescalable.Thereareseveraloptionsregardingtheinformationthatthesepacketsshouldcontain.Thesensorsmaysendbacktothesplitterthefollowingresponses:1.PositiveACKs:anACKforeverypacketnotrelatedtoanyintrusionattempt.2.PositivecumulativeACKs:anACKforasetofpacketsnotrelatedtoanyintrusionattempt.3.NegativeACKs:anACKforeverypacketthatbelongstoanoffendingsession.4.NegativecumulativeACKs:anACKforasetofpacketsthatbelongtoanattacksession.5.Thepacketreceived.Eachofthesesolutionshasitsprosandcons.Thepacketreceived(PR)scheme,althoughithastheadvantagethatitdoesnotrequirethesplittertotemporaryholdthepacketinmemory,itsuffersfromlowperformance.InSection3,weevaluatesomeoftheseapproaches,withregardtoperformance.AmongpositiveandnegativecumulativeACKs(CACKs)wehavechosentheformerones.NegativeCACKshavetwomajordrawbacks: DesignandImplementationofaHigh-PerformanceNIPS5First,inordertobeabletodistinguishwhenapacketmustbeforwarded,wehavetouseatimeoutvalue.Recallthat,ourNIPSmustnotdropanypacketoranattackmightbemissed.Asaresult,wewouldbeforcedtochooseatimeoutfortheworstcasescenario.Theside-effectisthatpacketswillexperienceahighlatency.Second,itisimpossibleforthesplittertodifferentiatethecasewheretheanalyzedpacketcontainednointrusionfromthecasewherethepacketwasdroppedduetoanerrorcondition.WehavechosenpositiveCACKs(P-CACKs)becausetheysupersedepositiveACKs.2.2ImplementationWehaveimplementedDigenisusinglow-cost,off-the-shelftechnology:anIntelIXP1200EthernetevaluationboardandcommodityPCs.Splitter:WehaveimplementedthesplitterusinganIXP1200networkprocessor.TheIXP1200chipcontainssixmicro-engineswithfourhardwarethreads(contexts)each.Also,thischiphasageneral-purposeStrongARMprocessorcore,aFIFOBusInterface(FBI)unitandbusesforoff-chipmemories(SRAMandSDRAM).ThemaximumaddressableSRAMandSDRAMmemoryare8Mbytesand256Mbytesrespectively.TheFBIunitinterfacestheIXP1200chipwiththemediaaccesscontrol(MAC)unitsthroughtheIXbus.TheFBIalsocontainsahashunitthatcantake48-bitor64-bitdataandproducea48-or64-bithashindex.Inourevaluationboard,anIXF440MACunit(witheightFastEthernetinterfaces)andanIXF1002MACunit(withtwoGigabitEthernetinterfaces)areconnectedtotheIXbus.Wehavedevelopedtheapplicationusingmicro-engineassemblylanguage.Theassignmentofthreadstotasksisdoneasfollows:weassigneightthreadsforthereceivepartoftheGigabitEthernetinterface,onethreadforthereceivepartofeachoftheeightFastEthernetinterfaces,fourthreadsforthetransmitpartoftheeightFastEthernetinterfaces,andfourthreadsforthetransmitpartoftheGigabitEthernetinterface.Fortheimplementationofhash-basedloadbalancing,weusethehashunitoftheIXP1200.Also,forthetemporarystorageoftheincomingpacketsuntiltheyareacknowledgedweuseacircularbufferwhichresidesinSDRAMmemory.ThiscircularbuffermustbelargeenoughtopreventoverwritingpacketsbeforetheirmatchingACKisreceived.Sensor:Thefunctionalityofthesensorhasbeenimplementedbymodifyingthepopular15NIDSSnortversion2.0.2.Thefunctionalityofthesensorcanbedividedintothreedifferentphases:(1)theprotocoldecodingphase,(2)thedetectionphase,and(3)thepreventionphase.Inthefirstphase,therawpacketstreamisseparatedintoconnectionsrepresentingend-to-endactivityofhosts.Aconnection,incaseofIPtraffic,canbeidentifiedbythesourceanddestinationIPaddresses,transportprotocolandUDP/TCPports.Then,anumberofprotocol-basedoperationsareappliedtotheseconnections.Theprotocolhandlingrangesfromnetworklayertoapplicationlayerprotocols.Someoftheoperationsappliedbytheprotocol-basedhandlingareIPdefragmentation,TCPstreamreconstructionandidentificationoftheURIinHTTPrequests.Thesecondphaseconsistsoftheactualdetection.Here,thepacket(oranequivalenthigher-levelprotocoldataunit)ischeckedagainstadatabaseofdetectionheuristicsrepresentingattackpatterns.Thenfollowsthepreventionphase.Theactionofthisphasedependsontheresultofthepreviousone.Ifnoattackisfound,thesensorinformsthesplittertoforwardthepackets.Ifmaliciousactivityisobserved,thenthepreventionengineblocksthesuspicioustrafficbyinformingthesplittertonotforwardthepacketsbelongingtotheoffendingconnection(s).ExtraImplementations:Inadditiontooursplitter,forcomparisonpurposes,wehaveimplementedthefollowingthreeconfigurationsontheIXP1200: 6K.Xinidis,K.G.Anagnostakis,E.P.Markatos•Aforwarder(FWD)thattransmitsthetrafficarrivingataninputGigabitEthernetinterfacetoanoutputGigabitEthernetinterface.•Aloadbalancer(LB)thatimplementsaflow-preservingloadbalancerwiththesameload-balancingcharacteristicsasoursplitter.TheIXP1200receivestrafficfromaGigabitEthernetinterfaceandtransmitsthetraffictoeightFastEthernetinterfaces.•Thelastconfiguration(LB+FWD)implementsthebasicfunctionalityofoursplitter(withoutoptimizations).3.EVALUATIONInthisSectionweexaminetheperformanceofourarchitecture.Wefocusontheimpactofourenhancementstosensor-splittercommunication.Inparticular,wecomparetheperformanceofP-CACKvs.thePRscheme.Wealsoshowthatsuchtechniquescanbeefficientlysupportedbycurrentnetworkprocessorsandthattheydonotsignificantlyimpairforwardinglatency.3.1ExperimentalEnvironmentSplitter:TheperformanceoftheconfigurationsrunningontheIXP1200ismeasured7usingtheIXP1200DeveloperWorkbench(version2.01a).Specifically,weusethetransactorprovidedbyIntel.Thetransactorisacycle-accuratearchitecturalmodeloftheIXP1200hardware.WesimulatetheconfigurationsastheywouldrunonarealIXP1200chip.Weassumeaclockfrequencyof232MHzanda64-bitIXbuswithaclockfrequencyof104MHz.Sensor:Weusea2.66GHzPentiumIVXeonprocessorwithhyper-threadingdisabled.ThePChas512MbytesofDDRDRAMmemoryat266MHz.ThePCIbusis64-bitwideclockedat66MHz.ThehostoperatingsystemisLinux(kernelversion2.4.20,Red-Hat9.0).6TheGigabitEthernetnetworkinterfaceisanIntelPRO/1000MTDualPortServerAdapter.ThesensorsoftwareisamodifiedSnortversion2.0.2,compiledwithgccversion3.2.2.WeturnoffallpreprocessinginSnort.Unlessnotedotherwise,Snortisconfiguredwiththedefaultrule-set.PacketTraces:FortheevaluationofDigenisweusethreepackettraces.TheFORTH.WEBtracewascapturedatICS-FORTHandonlycontainsHTTPtraffic.TheFORTH.LANtracewasalsocapturedatICS-FORTHandcontainstrafficfromaninternalLocalAreaNetwork(LAN).Bothtracescontaintherealpayloadofthepackets.TheIDEVALtracesaretakenfromMITLincolnLaboratoryandwereusedinthe1999DARPA12IntrusionDetectionEvaluation.3.2Results3.2.1PerformanceoftheSplitterAlltheIXP1200configurationsdescribedinSection2(LB,FWD,oursplitter,andLB+FWD)handleatmosttheIPandUDP/TCPheaderoftheincomingpackets.Thus,wearguethatthemostdemandingtrafficfortheseconfigurationsisthetrafficconsistingofahighpercentageofsmallpackets,namely64-bytepacketsd.WesimulatetheaboveconfigurationsandtheresultsshowthatalltheconfigurationsarecapableofsustaininglinedThisisthesmallestpossiblepacketinanEthernetlinkincludingthe4-byteEthernetCRC. DesignandImplementationofaHigh-PerformanceNIPS7speedevenwithtrafficconsistingofonly64-bytepacketse.Thisisexpectedasthetheoretical7forwardingcapacityoftheIXP1200chipisgreaterthan1600Mbit/s.Whilealltheconfigurationssustainlinespeeds,weuseasametricforcomparisontheutilizationofthemicro-enginesandtheutilizationofSRAMandSDRAMmemoriesf.Thesearesomeoftheresourcesthatmaybecomethebottleneck,consideringthattheIXP1200specificationstatesthatthemaximumIXbusthroughputis6Gbit/s.InFigure2wepresenttheaverageutilizationofthemicro-enginesandtheutilizationoftheSRAMandSDRAMmemoriesforthedescribedconfigurations.WeobservethatourapproachisefficientanddoesnotconsumealltheresourcesoftheIXP1200,leavingheadroomforevenmoreoffloadingofthesensors.Particularly,theresultssuggestthattheextracostofthesplittercomparedtotheloadbalancerisaffordableg.8050FWDFWD35FWD70LB45LBLBSPLITTERSPLITTER3040SPLITTER60LB+FWDLB+FWDLB+FWD3525Microengines(%)503020254015203010UtilizationofSDRAM(%)15UtilizationofSRAM(%)Utilizationof20105645121024151864512102415186451210241518PacketSize(bytes)PacketSize(bytes)PacketSize(bytes)Figure2.UtilizationoftheIXP1200micro-engines,SDRAMandSRAMmemoriesfordifferentpacketsizes.ItisobviousthatthesplitterconfigurationdoesnotconsumealltheresourcesoftheIXP1200.3.2.2PerformanceoftheSensorWefirstmeasuretheprocessingcostofasensorfordifferentcoordinationschemesusingthedefaultrule-set.InthisexperimentSnortsimplyreadstrafficfromapackettraceh,performsallthenecessaryNIPSfunctionality,andthentransmitsthecoordinationmessagestoahypotheticalsplitterthroughaGigabitEthernetinterface.Figure3,showsthetimethatSnortspendstoprocessallthepacketsfortheFORTH.WEBtraceincludinguserandsystemtimebreakdown.TheresultsshowthatthehighertheP-CACKfactor,thelessthetotalrunningtimeforSnort.TherunningtimeispracticallythesamewiththeunmodifiedSnortforP-CACKwithfactorequalto128.Also,Snortfinished45%fasterforP-CACKwithfactorequalto128comparedtothePRscheme.Moreover,weobservethatthesystemtimeislowerthantheusertime.ThisconfirmsthefactthatSnortspendsmostofitsprocessingtimeinheaderandcontentmatchingwhichiscountedinusertime.Wealsoobserve(Figures3and4)thattheimprovementoftheP-CACKschemecomparedtothePRschemedependsverymuchonthetraceused:theP-CACKschemeisfrom45%to3.8timesmoreefficientthanthePRscheme.Thereasonisthattheimprovementdependsonthedetectionloadofthesensor.Thesmallerthedetectionload,thebiggertherelativeimprovement.Thisbecomesclearerifwedeterminewheretheimprovementiscomingfrom.TheimprovementstemsfromthefactthattheP-CACKschemereducestheoverheadrequiredforsendingapackettothenetwork(systemtimeinFigures3and4).Ifthedetectionengineofasensorisoverloaded,thenthisoverheadisasmallfractionofthetotalworkloadofthesensorandreducingitdoesnotleadtomuchimprovement.Incontrast,ifthedetectionengineofasensorislightlyloaded,thisoverheadconsumesasignificantfractionofeThesplitterusesP-CACKschemewithafactorofeight.fMoreaccurately,wemeasuretheutilizationofthebusesofSRAMandSDRAMmemories.gWehavetomentionthattheincreasedutilizationofthemicro-enginesinthecaseofthesplitterconfigurationiscausedbytheinstrumentationcodeweaddtomeasuretheperformanceofthesplitter.Whileintheotherconfigurationswedonotaddcodeforevaluationpurposes,weareobligedtodosointhecaseofthesplitter.hWeconfirmthattheharddiskisnotthebottleneckbymeasuringthethroughputoftheharddiskandthetransmitrateofSnort.Asexpected,thetransmitrateofSnortissmallerthanthethroughputofthedisk. 8K.Xinidis,K.G.Anagnostakis,E.P.Markatosthetotalworkloadofthesensorandreducingitresultsinamorenotableimprovement.Forexample,ifthetrafficisruleset-intensive,thenthedetectionloadofthesensorincreasesandtherelativeimprovementissmall.Ontheotherhand,fortrafficthatrequiresfewerrulestobecheckedforeverypacket,thedetectionloadofthesensorwillbeminimalandtheimprovementwillbegreater.508UserTimeUserTimeSystemTime7SystemTime4063054Seconds20Seconds3210100PR116128256OnlyPR116128256OnlyP-CACKP-CACKP-CACKP-CACKDetectionP-CACKP-CACKP-CACKP-CACKDetectionCoordinationSchemeCoordinationSchemeFigure3.Processingcostofasensor(timetoprocessFigure4.Processingcostofasensor(timetoprocessallpacketsinatrace),withuserandsystemtimeallpacketsinatrace),withuserandsystemtimebreakdown(FORTH.WEBtrace).Weobservethatbreakdown(IDEVALtrace).WeobservethattheP-theP-CACKschemewithfactor256is45%moreCACKschemewithfactor256is3.8timesmoreefficientthanthePRscheme.efficientthanthePRscheme.WealsorepeattheexperimentonaPCwithaslowerPentiumIIIprocessorat1.13GHzandthesamePCIbuscharacteristicsandEthernetnetworkinterfaces.Theresults(Figure5)showthattheimprovementissmallercomparedtothefastermachine.Whenweexaminemorecarefullytheresults,weobservethatwhileusertimedoubles,thesystemtimeincreasesonlyby30%.Thishappensbecauseusertimeismainlythetimespentforcontentsearchandheadermatching,whichareprocessorintensivetasks.Onthecontrary,systemtimeisdominatedbythetimespentforcopyingthepacketfrommainmemory,overthePCIbus,totheoutputnetworkinterface,handlinginterruptsandcontrolregistersoftheEthernetdevice.AsthespeedofprocessorsincreasesfasterthanthespeedofPCIbusesandDRAMmemories,wecanarguethat,astechnologyevolves,theeffectofourenhancementswillbeevenmorepronounced–commonprocessorsarealreadyrunningat3.8GHz,sothepreviouslyreportedimprovementisinfactaconservativeresult. DesignandImplementationofaHigh-PerformanceNIPS980UserTimeUserTimeSystemTimeSystemTime703006050200401Full16128SecondsSecondsNone3010020Full116128None101160Full128None01005001000PR116128256OnlyP-CACKP-CACKP-CACKP-CACKDetectionCoordinationSchemeNumberofRulesFigure5.ProcessingcostofaslowersensorFigure6.Performanceofasensorusingincremental(FORTH.WEBtrace).Wecanseethatthenumberofsyntheticrules.Wenoticethatastheimprovementissmallercomparedtothefastersensor.numberofrulesincreasestheimprovementofP-CACKschemeversusPRschemedecreases.Table1.Syntheticruleexample.alerttcpanyany→anyany(ack:1;flags:S;content:”RPCoverflow”;)Alltheaboveexperimentsareperformedusingthedefaultrule-setofSnort.TofurtherunderstandthecorrelationbetweenthedetectionloadofasensorandtheP-CACKschemeimprovementwealsoexperimentwithvariable,syntheticrule-sets.AnexampleruleisshowninTable1.Similarlytothepreviousexperiment,SnortreadstrafficfromapackettraceandsendspacketsoveraGigabitEthernetinterface.TheresultsareshowninFigure6.WeobservethatasthenumberofrulesincreasestheimprovementofP-CACKschemeversusPRschemedecreases.Inotherwords,asdetectionloadincreases,improvementdecreases.AnotherinterestingpointisthatthemaximumrelativeimprovementofP-CACKoverPRisforsmallpacketsof64bytes.Smallpacketsrequirelesstimeforcontentmatchingusertimeandcommunicationsystemtimeisthedominantcostfactor.Inaddition,inthecaseof64-bytepackets,thebottleneckisnottheprocessor,asinthecaseoflargerpackets,butthePCIbus.ThisisclearlyshownintheexperimentsinvolvingtheIDEVALtraces(Figure4),whichcontainmanysmallpacketsforemulatingcertaintypesofattackssuchasSYNflooding.Forthistrace,theP-CACKschemeis3timesmoreefficientcomparedtothePRscheme.ThisisalsoanicesideeffectoftheP-CACKscheme,inthatitmakestheNIPSmorerobustagainstTCPSYNfloodattacks,giventhatsuchattacksconsistofabigfractionofsmallpackets. 10K.Xinidis,K.G.Anagnostakis,E.P.Markatos2601240PRP-CACK10.9220200P-CACK160.8180P-CACK1280.7160Detection-Only0.61400.5P-CACK11200.4P-CACK16100MLFR(Mbps)800.3P-CACK128600.2P-CACK25640Fractionofpackets0.1200PR002468101214161820FORTH.WEBFORTH.LANIDEVALPacketTracesForwardingLatency(millisecondsFigure7.MaximumLossFreeRate(MLFR)ofaFigure8.CDFforlatencyofasensor.Wenoticethatsensorusingdefaultrule-set.latencyincreaseswiththeP-CACKfactor.3.2.3ForwardingLatencyoftheSensorThehighestportionofthelatencyimposedbyourNIPSisduetocontentmatchingonthesensors.ThishappensduetothefactthatcontentmatchingisthesinglemostexpensiveoperationineveryNIPS.Tomeasureforwardinglatency,weusetwohostsAandBwithtwoGigabitEthernetnetworkinterfaceseach,eth0andeth1.WeconnectthetwointerfacesofhostAwiththetwointerfacesofhostBback-to-back.EverythingthathostAsendstonetworkinterfaceeth0/eth1isreceivedbyhostBonnetworkinterfaceeth0/eth1,andviceversa.Host1AreadsatracefromafileandsendstraffictohostB(usingtcpreplay).HostBrunsSnort,whichreceivespacketsfrominterfaceeth0andsendsrepliestointerfaceeth1.HostAmatchesthepackettransmissiontimewiththearrivaltimeofthereplyandcomputesthelatency.Initially,weestimatethemaximumlossfreerate(MLFR)ofasensorbyreplayingapackettraceandmeasuringtherateatwhichthesensorstarteddroppingpackets(Figure7).Inthisexperimentwesettheinputpacketbuffersizeto16Mbytes.WeuseMLFRtocomputethelatencythatasensorimposestoanalyzedpacketswhenreachingitsprocessingcapacity.Inthisexperiment,hostAreplaysFORTH.WEBtraceatthemaximumlossfreerateofeachcommunicationscheme.Weobservethattherearepacketsthatexperienceveryhighlatency.Tounderstandthisphenomenon,wemeasurethetimethatSnortspendsincontent17andheadermatchingusingtherdtscinstructionofthePentiumIVprocessor.Theresultsshowthatthepeaksintimespentforcontentandheadermatchingcoincidewiththepeaksinlatency.Thismeansthat,whentherequiredperpacketoperationsincrease,sodoesthelatency.Aconsequenceofthispropertyisthatpacketsthatrequireasignificantamountofprocessingslowdownotherpacketsthatdonot.Thisisaformofheadofline(HOL)blocking.Figure8showsthecumulativedistributionfunction(CDF)forallACKschemeswhenasensorreceivestrafficattheMLFRofFORTH.WEBtrace.WenoticethatlatencyincreaseswiththeP-CACKfactor.Aninterestingobservationisthatthegraphisheavytailed,meaningthatwhilemostofthepacketsexperiencelowlatency,5%ofthepacketsexhibitveryhighlatency(above20milliseconds).Thesearepacketsthatarereceivedfromasensorwhilethesensorhasatemporaryexcessload.Thismayhappenbecause,forexample,somepacketsrequiretoomanyrulestobechecked.Iftoomanysuchpacketsarereceivedback-to-back,thesystemreaches(orexceeds)itscapacityandthelatencyincreasesconsiderably.3.2.4ForwardingLatencyoftheSplitterWearguethattheoveralllatencythatapacketexperiencesbyourNIPSisduetotheprocessingofthesensorsandnottheforwardingofthesplitter.Also,thecyclesspentbythesplittertoforwardapacketfromtheinputinterfacetoanoutputinterfacedependonlyonthepacketlength.Thismeansthatpracticallyallpacketsofthesamelengthexperiencealmostthesamelatency. DesignandImplementationofaHigh-PerformanceNIPS113.2.5MemoryrequirementsThereisadirectrelationshipbetweenlatencyimportedbythesensorsandrequiredmemoryonthesplitter.Thesplitterneedsmemorytosaveincomingpacketsuntiltheyareacknowledgedbythesensors.TheamountofmemorythesplitterneedsdependsonthehighestpossiblelatencythatourNIPSwilltolerate.Ifwesetthisvalueinareasonablevalue,forexample,200millisecondsthenaccordingtothefactthatourNIPSanalyzestrafficat800Mbit/s,therequiredmemoryisapproximately20Mbytes.ThismeansthatthecircularbufferoftheIXP1200mustbeatleast20Mbytes.ThisisareasonablerequirementconsideringthatthemaximumaddressableSDRAMmemoryoftheIXP1200is256Mbytes.4.SUMMARYANDCONCLUDINGREMARKSWehavepresentedthedesignofDigenis,ahigh-performanceNetworkIntrusionPreventionSystem(NIPS).Thesystemconsistsofacustomizedload-balancingcomponentbuiltusingtheIXP1200NetworkProcessor,andanumberofsensorsimplementedoncommodityPCs.Incontrasttooff-the-shelfloadbalancersusedinNIPSproducts,ourdesignexploitstheprogrammabilityofNPstomovepartoftheintrusionpreventionfunctionalityfromthesensorstotheNP.Wehavefocusedononemethodforboostingsystemperformancebyoptimizingthecoordinationbetweentheloadbalancerandthesensors.Theresultisa45%improvementinperformance,allowingthesystemtoreachspeedsofatleast1Gbit/s.Thereareseveraldirectionsthatwearecurrentlypursuing.First,wearere-examiningthestructureofthesensorsoftware.Inparticular,weconsiderthepossibilityofusingamore14fine-grainedprotocolprocessingmodelsuchastheonedemonstratedbyBro,andwetrytomovepartoftheprotocolprocessingfunctionalitytotheNP.Second,wearelookingatwaysforbuildinga10Gbit/sNIPSusingthird-generationNPs.ACKNOWLEDGEMENTSThisworkwassupportedinpartbytheISTprojectSCAMPI(IST-2001-32404)fundedbytheEuropeanUnion,theGSRTprojectEAR(GSRTcode:USA-022),andbyESTIA,aPAVET-NEprojectfundedbytheGreekGeneralSecretariatofResearchandTechnology(PAVET-NEcode:04BEN8).KostasAnagnostakisisalsosupportedinpartbyONRunderGrantN00014-01-1-0795.KonstantinosXinidisandE.P.MarkatosarealsowithUniversityofCrete.TheworkofKostasAnagnostakiswasdonewhileatICS-FORTH.REFERENCES1.AaronTurnerandMattBing.tcpreplayTool.http://tcpreplay.sourceforge.net.2.S.Antonatos,K.G.Anagnostakis,andE.P.Markatos.Generatingrealisticworkloadsforintrusiondetectionsystems.InProceedingsofthe4thACMSIGSOFT/SIGMETRICSWorkshoponSoftwareandPerformance(WOSP2004),January2004.3.Z.Cao,Z.Wang,andE.W.Zegura.Performanceofhashingbasedschemesforinternetloadbalancing.InProceedingsofIEEEInfocom,pp.323-341,2000.4.Y.Charitakis,K.G.Anagnostakis,andE.Markatos.Anactivesplitterarchitectureforintrusiondetection(shortpaper).InProceedingsoftheTenthIEEE/ACMSymposiumonModeling,Analysis,andSimulationofComputerandTelecommunicationsSystems(MASCOTS2003),October2003.5.Y.Charitakis,D.Pnevmatikatos,E.P.Markatos,andK.G.Anagnostakis.CodegenerationforpacketheaderintrusionanalysisontheIXP1200networkprocessor.InProceedingsofthe7thInternationalWorkshoponSoftwareandCompilersforEmbeddedSystems(SCOPES2003),September2003.6.IntelCorporation.IntelPRO/1000MTDualPortServerAdapter.http://www.intel.com.7.IntelCorporation.IntelIXP1200NetworkProcessor(whitepaper),2000.http://developer.intel.com.8.InternetSecuritySystemsInc.http://www.iss.net. 12K.Xinidis,K.G.Anagnostakis,E.P.Markatos9.IntrusionPreventionSystemsGroupTest-Edition1,NSSGroupLtd.http://www.nss.co.uk/acatalog/.10.L.KenclandJ.Y.L.Boudec.Adaptiveloadsharingfornetworkprocessors.InProceedingsofIEEEInfocom,June2002.11.C.Kruegel,F.Valeur,G.Vigna,andR.Kemmerer.Statefulintrusiondetectionforhigh-speednetworks.InProceedingsoftheIEEESymposiumonSecurityandPrivacy,pp.285-294,May2002.12.R.Lippmann,J.W.Haines,D.J.Fried,J.Korba,andK.Das.The1999DARPAoff-lineintrusiondetectionevaluation.ComputerNetworks,34(4):579-595,October2000.13.NetworkAssociates,Inc.http://www.networkassociates.com.14.V.Paxson.Bro:Asystemfordetectingnetworkintrudersinreal-time.InProceedingsofthe7thUSENIXSecuritySymposium,January1998.15.M.Roesch.Snort:Lightweightintrusiondetectionfornetworks.InProc.ofthesecondUSENIXSymposiumonInternetTechnologiesandSystems,November1999.(Softwareavailablefromhttp://www.snort.org).16.R.Russo,L.Kencl,B.Metzler,andP.Droz.ScalableandadaptiveloadbalancingonIBMPowerNP.Technicalreport,ResearchReport-IBMZurich,August2002.17.Time-StampCounter.http://www.intel.com/design/Xeon/applnots/24161825.pdf.18.TippingPointTechnologiesInc.http://www.tippingpoint.com.19.TopLayerNetworks.http://www.toplayer.com.20.K.Xinidis,K.G.Anagnostakis,andE.P.Markatos.DesignandImplementationofaHigh-PerformanceNetworkIntrusionPreventionSystem.ICS-FORTHTechnicalReport334,March2004.
