ArchSummit北京2015-《Twitter实时搜索引擎发展历程》-王天

《ArchSummit北京2015-《Twitter实时搜索引擎发展历程》-王天》由会员上传分享，免费在线阅读，更多相关内容在行业资料-天天文库。

1、Twitterਫ෸൤ᔱ୚කᬰ۸ܲᑕӻՈՕᕨሴॠ2003ଙ๜ᑀླӱԭႴय़਍ᦇᓒ๢ᔮҔ2005ଙᏗॊླӱԭUniversityofNorthCarolinaChapelHillҔ2005ଙ7์ےفGoogle՗Ԫᑏۖ൤ᔱ̵ෛ᳼൤ᔱ̵൤ᔱᨶᰁᒵૡҔ2011ଙ3์ےفTwitter൤ᔱ᮱ᳪૡᛗՔ̶ଃᶾTwitterጱ൤ᔱᨶᰁࢫᴚදᬰਫ෸൤ᔱԾߝ̶@wangtianwww.twitter.com/wangtianwww.linkedin.com/in/wangtianTwitter൤ᔱ༷٭•2008ଙতԭ

2、තᨻጱSummizeلݪ•໐ஞ๐ۓғىᲫਁ൤ᔱ̵ፘى௔൤ᔱ̵Facets൤ᔱ̵Termᕹᦇ…•ፘى๐ۓғᛔۖୌᦓ̵೪ٟ༄ັ̵ፘىവគ…•6700Պ๵വ෈ق᮱ᕑفᔱ୚•๶ᛔTwitter൤ᔱԾߝ•Web̵Android̵iOS̵TweetDeck̵…•๶ᛔلو̵վ֎ٖ݊᮱API•ۖᑏ̵ᒊᗑ̵ଧᑕጱᘏݎ୏՜ٌଫአ̵ໟᶎଫአ•ࠟӱ൤ᔱ๐ۓᦈᴅᘏҁইGNIP҂2008~2009አಁ•വ෈ᤩፗളٟکහഝପ•ٟჿӞӻᤒ੪୏ސӥӞӻ•൤ᔱፗളࣁහഝପᬰᤈMonorail•ݝඪ೮ӣॠૢݦጱහഝ൤ᔱڹᒒMySQL

3、IndexMySQLTimesliceTimesliceTimesliceTimesliceMySQLFirehoseIngestertabletabeltabletable(slave)MySQL(slave)चԭහഝପጱᔱ୚ᳯ᷌௔ᚆ•਻ᰁํᴴಘ઀ࢯᵙ•൤ᔱଚݎ௔•ਫ෸ᑕଶݑහഝପᔱ୚ๅෛ୽ߥۑᚆڹว•ᵙզႲےෛහഝ•ᵙզඪ೮॔๥ጱັᧃ᧍ဩ•෫ဩඪ೮ፘى௔൤ᔱ•൤ᔱڹᒒਂࣁԭๅय़ᔮᕹӾᵙզ୏ݎचԭLuceneጱᔱ୚᯿ٟ•APIؙਂٟ᯿ӥ٭ఘ਻ّ੶postinglistᕮ຅•ਫ෸ๅෛ̵ग़ᕚᑕ෫ᲁ

4、᧛ᦢᳯ•սضඪ೮ᭋ෸ᳵଧጱ൤ᔱ•ඪ೮൉ڹᕮ๳൤ᔱ•قٖਂᔱ୚•ඪ೮ࣈቘහഝ•ෛ໅ᇫݢಘ઀queryᕮ຅•LucenequeryӤᶎᇿᒈ۱ᤰጱහഝᕮ຅•ᥢ᝜queryزᔰ•՗timepartitionکhashpartition•᭛ଶ൉܋ғ5msӾ؀୊᬴•๐ۓ਻ᰁ൉܋ғ10x•ݍᨯሠᛗLucene୏რᐒ܄ັᧃ᧍᥺ғSearchQueryRawQueryappleiphonefilter:imagesnear:meአಁᬌفጱᛔኧ෈๜໒ୗጱັᧃᥢ᝜۸̵ړ᦯̵ᥴຉ඙ᒧҁࣈቘᗝ̵ᐒԻࢶᬦᄁ̵᧍᥺̵ᔄࣳᒵ҂

5、SerializedQuery(*appleiphone[filterimages][geocode33.4-128.225km])ᥢ᝜queryአᇙਧ᧍ဩ୵ୗ۸ᤒᬡQueryAnnotationandRewritingғ൉׀क़᮱හഝඪ೮ጱ᷐क़ຽဳ޾᯿ٟRewrittenQuery(*(+(*appleiphone:w:2.0)@appleiphone:user:v:0.3)[filterimages][geocode33.4-128.225km])ಘ઀queryLuceneQueryBuild

6、ingғኧᔱ୚෫ىqueryฉ੘ԅᇙਧᔱ୚ጱ൤ᔱquery(MUST:text:appleMUST:from_user:apple^0.4)(MUST:text:iphoneMUST:from_user:iphone^0.4)LuceneQueryMUST:internal:__has_imagesGeoQuery(…)ᶎLuceneᔱ୚ጱັᧃ೜ํᔱ୚ऒᕡᜓ*Ԇᥝᩢ੒᨝ᤒᐏොୗෛӞդᔱ୚๐ۓ࢏ғEarlybirdҁ2010҂•Earlybird෬ฎӞӻᔱ୚࢏(Indexer)ԞฎӞӻᔱ୚๐ۓ࢏(

7、IndexServer)•MySQLሿࣁՐአԭྍ޾වඎහഝכਂଧڜ۸ጱ෈໩ҁവ෈҂አಁ•൤ᔱڹᒒդᎱᵱᥝ޾ಅํEarlybirdpartition᭗מᤒᐏMonorail᭦ᬋ൤ᔱڹᒒ൤ᔱڹᒒᵱᥝ޾ಅํpartitionԻၞᔱ୚ၞ࿜ᕚFirehoseIngesterMySQLMySQLMySQLMySQLEarlybirdEarlybird(master)(slave)Earlybirdਂؙ(slave)(partition)(partition)(slave)(partition)॒ቘവ෈හഝහ

8、ഝପՐአԭᔱ୚޾൤ᔱ᮷໩෈୚ᔱஇୌڠྍ޾වඎහഝݎኞࣁᬯ᯾EarlybirdClientEarlybird୏ত൤ᔱᗝSegmentSegmentSegmentSegmentSegmentMySQL43210(slave)๋ෛਫ෸വ෈ෛ෯Segment:ӞӻᇔቘᇿᒈጱLuceneIndexPostingListӾdocumentsೲᆙ෸ᳵᭋଧഭڜෛӞդ൤ᔱ๐ۓ࢏ғBlenderҁ2011҂•Ӟ୏তԅӞӻӾᳵ੶ጱEarlybird໑ᜓᅩڊሿ•ᴫଃᬰᤈٌਙᴫے඙ғᬦᄁ̵ऴشᕮຎ̵…አಁ•ᦡᦇڡ๗ܨᘍᡤ