Super simple distributed hyperparameter tuning with Keras and Mongo

《Super simple distributed hyperparameter tuning with Keras and Mongo》由会员上传分享，免费在线阅读，更多相关内容在学术论文-天天文库。

1、MattHarveyFollowProductmanagerandmachinelearningenthusiastbasedinSiliconValley.Apr18·3minreadSupersimpledistributedhyperparametertuningwithKerasandMongoOneofthechallengesofhyperparametertuningadeepneuralnetworkisthetimeittakestotrainandevaluateeachsetofparameters.If

2、you’reanythinglikeme,youoftenhavefourorfivenetworksinmindthatyouwanttotry:differentdepth,differentunitsperlayer,etc.Trainingandevaluatingthesenetworksinseriesisfineifthedatasetorparameterspacearesmall.Butwhatifyou’reevolvinganeuralnetworkwithageneticalgorithm,likewedidi

3、nourpreviouspost?Orwhatifonetrain/evalsteptakesadaytocomplete?Weneedparallelism!Here’sasupersimplewaytoachievedistributedhyperparametertuningusingMongoDBasaquasipub/sub,withacontrollerdefiningjobstoprocess,andNworkerstodothetrainingandevaluating.Itlookslikethis:Allth

4、ecodeisavailableonGitHubhere:SuperSimpleDistributedHyperparameterTuningTheControllerThesystemstartswithacontrollerthatcreatesthejobs.Howyouchoosetocreatejobsisuptoyou.Inoursimpleexample,wejustrandomlycreateatwo-layerMLPwitharandomnumberofunitsperlayer.Itgoessomethin

5、glikethis:defmain():"""Toyexampleofaddingjobstobeprocessed."""db=Database('blog-test')whileTrue:print("Creatingjob.")network=gen_network(784,10)#mnistsettingsadd_job('mnist',network,db)sleep_time=random.randint(60,120)print("Waiting%dseconds."%(sleep_time))time.slee

6、p(sleep_time)TheDatabaseJobsarestoredinaMongoDB.WecreateaDatabaseclasswiththefollowingmethods:•insert_job—AddthejobtotheDB.Thejobdocumentincludesthedatasetwewanttotrainon(inourcase,justmnistorcifar10),theKeras-generatedJSONnetworkdefinition,processingandprocessedflags

7、,andanemptydictionarytoholdthemetrics•find_job—Returnsthefirstavailablejobthatisn’tcurrentlybeingprocessedandsetsaflagthatindicatesthejobisbeingworkedon•score_job—UpdatethejobtoindicateithasbeenprocessedandaddtheresultingmetricsTheDBneedstoberemotelyaccessibletoallthe

8、workersifyou’rerunningthistrulydistributed.Initscurrentstate,thecodeonlyacceptsahostargumentsinceIrunthisonAWSinstancesthatshareasecurityg