1、Mongodb 在互聯(lián)網(wǎng)金融的應(yīng)用AgendaAbout KoalaWhy ? RequirementsHow ? ArchitectureProblemsSolutionsFutureWhat is P2PWhat is a P2P?P2P PlatformBorrowerLenderLenderLenderSingle LoanNoteNoteNoteLender :有錢人/投資人, Borrower:缺錢人/借款人 Note: 投資(憑證) Loan:借款標(biāo)的(需求)Investing in multiple P2P?P2P PlatformBorrowerLenderLenderLen

2、derP2P PlatformBorrowerP2P PlatformBorrowerMultiple Loans Multiple Loans Multiple Loans Lender :有錢人/投資人, Borrower:缺錢人/借款人 Note: 投資(憑證) Loan:借款標(biāo)的(需求)What is a P2P fund?P2P PlatformBorrowerLenderLenderLenderP2P PlatformBorrowerP2P PlatformBorrowerP2PFundMultiple LoansMultiple LoansMultiple LoansLender

3、 :有錢人/投資人, Borrower:缺錢人/借款人 Note: 投資(憑證) Loan:借款標(biāo)的(需求) Fund:基金What is a P2P fund?P2P PlatformBorrowerLenderLenderLenderP2P PlatformBorrowerP2P PlatformBorrower考拉 理財Multiple LoansMultiple LoansMultiple LoansLender :有錢人/投資人, Borrower:缺錢人/借款人 Note: 投資(憑證) Loan:借款標(biāo)的(需求) Fund:基金About KoalaTags: Finance ,

4、 P2P mashup，懶人 -通過極致的分散理財，達(dá)到風(fēng)險&收益的平衡 正式上線：2015/5交易額：51+億￥AUM：8+億￥documents count: 千萬 億用戶量：75萬，同時在線: 1002,000About Koala投資策略 和管理系統(tǒng)風(fēng)險監(jiān)控模型和管理系統(tǒng)資產(chǎn)配置模型（源于高盛）基金財務(wù)系統(tǒng)用戶管理系統(tǒng)System architectureTech StackMongodb(core) + Hadoop MySQLNodejs(Typescript), Python, R, Sql, Java(non core)Mongodb supports most of our

5、core systemsWhy? - Requirements for the marketLoans 稀疏多變Borrower 征信信息P2P platforms 稀疏多變市場需求（哪有不多變的需求）數(shù)據(jù)量多、N多維度統(tǒng)計要響應(yīng)快Why? - Requirements for finance修改歷史需要被保留 (不允許刪除操作)操作要提供反操作，而不僅僅是簡單的update備份，備份，備份安全演練Mongodb deploy architectureV1Mongodb deploy architectureV2Mongodb deploy architectureV3Mongodb depl

6、oy architectureV4Mongodb deploy architectureV5坑s:Ops & DevIssues ever met in opsIndex 不中，掃描過多比較耗內(nèi)存Dev誤操作程序連錯db篡改數(shù)據(jù)誤操作誤刪數(shù)據(jù)Issues ever solvedIndex 不中，掃描過多 - explain it + 慢查詢分析: ensureindex比較耗內(nèi)存 - 加內(nèi)存吧，便宜的解決方案Dev誤操作程序連錯db篡改數(shù)據(jù) - VPC & 區(qū)別開環(huán)境 & 做好權(quán)限控制誤操作誤刪數(shù)據(jù) - 延時節(jié)點(diǎn)！Oplog 恢復(fù)Security Summary角色限制: Administra

7、tor/Monitor/Read Write/Read Only不要用默認(rèn)端口設(shè)置password給不同的password 區(qū)分開prod & dev env開發(fā)環(huán)境不允許連prod envBackup SummaryOplog 演練恢復(fù)到指定時間點(diǎn)Full backup Every 6 hoursDaily Backup within Recently 30 DaysDelay instance (1 hour) - 防止 replica set 實(shí)時污染第二個 DataCenter instanceIssues ever met in dev讀寫沒分離壓力 primary過大離線計算ETL

8、導(dǎo)致壓力過大嵌套文檔的數(shù)組長度過大“Schemaless”Issue ever solved in dev讀寫沒分離壓力 primary過大 - 讀寫分離離線計算ETL導(dǎo)致壓力過大 - 讀寫分離 & 同步多一個DB用于計算嵌套文檔的數(shù)組長度過大 - 設(shè)計注意考慮是否是不斷增長、變化應(yīng)用層or DB層限制Schema坑 s:part 2 Problems about TradingCrash while processingTransfer exampleA account 1000transfer 500￥B account 100Transfer example簡單場景: account &

9、 trade are in the same DB system復(fù)雜場景: account & trade are in different service & DB System實(shí)際場景: Different Banks/SystemNeed Transaction?強(qiáng)一致性弱一致性Mongodb SolutionsEmbed documents優(yōu): Sometimes it works & make query faster缺: Sometimes it just doesnt workTwo phase commitMake code looks complicatedOther Sol

10、utionsMySQLMySQL (trade + account service) + Mongodb (other)PostgreSQLCassandraSequoiaDBHybrid alreadyRedis + MongodbRedis + MySQLHybrid Solution for Mysql & MongodbHybrid Solution ProblemsMaintainability: More ComplexForeign KeyJoin TablesSyncOthers Mongodb Only Solution: Events SourcingEvents Sour

11、cing : record every events and replay them after crashTrade offOur Solutions基于Embed docs 設(shè)計應(yīng)用層(ORM) level:dev造了個輪子：mongoose-transact/mongoose-transtion-plugin/waterline-transactHow Transaction Is ImplementHow Transaction Is Implement in CodeUsage Sample CodeBest Practise for Account Trading(Common)可重入函數(shù)掉單檢測:定時檢查&重試單元測試注：支付異步回調(diào)會發(fā)生多次直到處理成功(限