从过去20年软件架构，我们可以得到什么？

Junzhi He

TABLE OF CONTENTS 1

1. What is the “Learning” for human? vs Machine learning
2. What is the knowledge? What is Architect?
3. History of Architect
4. Pattern and Principle
5. Architect and Organization
6. Case Study

TABLE OF CONTENTS 2

1. Bottleneck and Foundation:
1. Storage
2. Consistency
3. Algorithm
4. Transaction
5. Replication and Partition
6. Balance
7. Distribute Components
8. Security
9. Parallel computing specialization

WHAT IS THE “LEARNING” FOR HUMAN?

• 学习不是记忆
• 例子：为什么有些小朋友小学成绩好，但是到了中学或者大学就不行了。
• 记忆的特点：容量有限，难以遍历所有情况
• 矛盾点：有限的例子 vs 无限的问题
• 学习是什么？通过压缩信息找到知识
• 后转1：聊聊刷题和狗家的面试
• 后转2：聊聊高考

​

VS MACHINE LEARNING1

HOW TO LEARN ARCHITECT？

• 为什么我要先给大家干货，再讲例子呢？也就是先读薄，再读厚？
• 群友的平均水平
• 时间所限
• Proactive

番外篇：英/法语，�编程和刷题1

番外篇：英/法语，�编程和刷题2

HISTORY 1 – NAÏVE DISTRIBUTION

• 1970S TO 1980S
• MOTIVATION: HW IS BOTTLENECK
• ACHIVEMENT: PRINCIPLE CONTRIBUTE TODAY
• HP: Network Computing Architecture(NCA) 🡪 RPC (Thrift and gRPC(protobuf))
• CMU: Andrew File System -> GFS, HDFS (Andrew Carnegie, Andrew Mellon)
• MIT: Kerberos, currently used in Windows and MacOS
• Challenge: Difficult than simple philosophy of UNIX
• Reality: Moore's law

​

HISTORY 2 – MONOLITHIC VS SINGLE POINT

• From 1990s to 2005s
• Single Point is positive, no IPC(Inter-Process Communication), simple, efficient
• Monolithic is not positive, but
• supports layer-based-architecture
• supports also modularization by language, function, etc (Multiple JAR、WAR、DLL)
• First bottleneck is lack of isolation and autonomy due to same process.
• Error propagation to entire system, global impact. (memory leak, thread explosion, dead lock, blocking I/O, forever)
• Ex: deploy is hard

HISTORY 2 – MONOLITHIC VS SINGLE POINT

• Difference between Monolithic and Single Point
• 2nd Bottleneck: Partial Update is not passible. (Same as our UI): specific maintenance plan, blue-green deploy, A/B Test is complicated.
• 3^rd Bottleneck: Technical async is difficult

​

​

HISTORY 3 – SERVICE-ORIENTED ARCHITECTURE

• When: From 2000 to 201x
• Information Silo Architecture: Orthogonality
• Microkernel Architecture
• Event-Driven Architecture
• SOA
• 初心 by Unix DCE：让开发人员不必关心服务是远程还是本地，都能够透明地调用服务或者访问资源

HISTORY 3.1 MICROKERNEL ARCHITECTURE

• Technically, it supports both Desktop App and Web app.
• Most times it is impossible to transform components to plug-in

​

HISTORY 3.2 EVENT-DRIVEN ARCHITECTURE�FOUNDATION OF SOA

HISTORY 3.3 SOA

• From 2006 and supports by IBM, Oracle, SAP
• Supports decouple service, reusable components, service registration, configuration, isolation, resilience, composition, etc. 🡪 Redefine software development. 活字印刷
• Remote access: SOAP protocol.
• Enhanced Message Queue: ESB – Enterprise Service Bus.
• Service Data Object
• Philosophy: If architect follow the SOA guideline 八股文, then every problem will be resolved. Not only technic, but also management and organization.

HISTORY 3.3 SOA

• Why SOA failed?
• Market: Same as J2EE, Enterprise platform, but not internet high-tech.
• SOAP protocol is too strict and make the implementation too complicated.
• Strategy: Same as J2EE and EJB, very hard to use even it is open source. It could be used only by few rich enterprise and their architects. The guideline become a constraint. 计划经济
• 忘记了初心, 脱离了人民群众。

HISTORY 4 - MICROSERVICE

• First appeared in 2005 with cloud computing.
• In 2012, As Well-Behaved Unix Services. Forgot “SOA”
• In 2014,
• https://martinfowler.com/articles/microservices.html
• Organized around Business Capability, Decentralized Governance, Componentization via Services, Products not Projects, Decentralized Data Management, Smart Endpoint and Dumb Pipe(直接报我身份证), Design for Failure, Evolutionary Design, Infrastructure Automation
• RPC: RMI（Sun/Oracle）、Thrift（Facebook）、Dubbo（Ali）、gRPC（Google）、Motan2（Sina）、Finagle（Twitter）、brpc（Baidu）、Arvo（Hadoop）、JSON-RPC、REST
• Service Registry: Eureka（Netflix）、Consul（HashiCorp）、Nacos（Ali）、ZooKeeper（Apache）、Etcd（CoreOS）、CoreDNS（CNCF）

​

HISTORY 5 – �CLOUD NATIVE

• Paradox: Static HW infra vs Flexible requirement of Software
• What is boundary?

​

SERVERLESS

• AWS Lambda
• Kubeless

​

SERVERLESS – MY PREDICTION

• PROS:
• No infra, No deployment, Auto-scaling, No operation. Like assembly language to C?
• Good at: short connection, seamless, even-driven: Machine learning, Web General, public API, mobile/wechat app.
• CONS:
• In short term, complex business logic, low-latency, long-connection: Bank, Insurance, Gaming.
• Security.
• Long term positive.

DON’T BE AFRAID

1 – LAYER AND EVENT-DRIVEN

2 - MICROKERNEL AND MICROSERVICES

3 - SPACE-BASED ARCHITECTURE

FROM ANT TO ELEPHANT 1 – MY 1^ST WEBSITE

​

​

​

​

​

​

​

What is “ACID” and “Transaction”? Isolation, Consistency -> Unix Simple

Servlet vs JSP vs JavaScript vs AJAX

Design Pattern?

1^ST WEBSITE – 隐藏的挑战

• 在冰冻的河面走路是很容易的，正如软件开发
• Transaction需求：
• 转账的例子: 我的数据要存进去啊 – D 🡸 故障无处不在
• 我能放弃吗？ - A 🡸 业务逻辑的复杂性，事务的内容有可能很多
• 永远没冲突？ - YES, 在串行的情况下 C 🡸 看下面
• 共享资源 – I 🡸 可惜系统的user很多
• 下面会用到的数据结构：
• 如何快速找到undolog？类似LRU的链表
• 如何快速找到当前版本？二分查找
• 索引？特殊的二叉树 -> B Tree
• binlog的结构？只能添加不能修改
• 多版本控制MVCC中，关于版本查找的区间问题

​

​

UNDO LOG FOR A�REDO LOG FOR D

• UNDO LOG除了回滚还有什么作用？I
• REDO LOG保证了crash safe吗？
• D一定在硬盘？
• 讨厌的Bin Log
• 为什么不用BinLog保证D，不用undo log做备份

ISOLATION – 级别？多线程？锁？悲观还是乐观？快照度和当前读？SELECT FOR UPDATE?

FROM ANT TO ELEPHANT 2 – FE AND BE

FROM ANT TO ELEPHANT 3 – STORAGE AND I/O

95% of time is blocked by I/O and storage is the bottleneck.

CPP cache vs memory vs SSD ?

Memory: speed is good enough and affordable price.

transmissive vs detour

Cold boot

Architect Principle: Add a layer

​

FROM ANT TO ELEPHANT 4 – CPU

• Performance Test team:
• Java Reflection: Meta
• Regex
• String concat
• Memory Copy
• Reduce data transport (payload)
• More expensive single point: Oracle
• Multi-layer:
• Web Server
• Domain Server
• Cache DB
• Result: 20%
• Architect Pattern: Layer based

​

FROM ANT TO ELEPHANT 5 – DB

• Read and Write Separation
• Message Queue
• 解耦异步削峰填谷
• WHERE IS MY CONSISTENCY?
• Architect Pattern:
• Pre-micro-service

FROM ANT TO ELEPHANT 6 – DOMAIN SERVER

• Partition of Domain Server. 🡪 Load Balance
• Consistency Session
• session sync
• unified session service. <- Enterprise choice (We already have Redis)
• certain hash method. <- My choice (Next Step)
• Kill Session
• Ticket System
• DB: Isolation and Lock
• Cache: Lazy delete
• Architect Pattern:
• Space-Based Architecture

FROM ANT TO ELEPHANT 7 – DB AGAIN

• DB data Shading (MySQL)
• Horizontal
• Vertical (Too complicated, redesign data model)
• Domain Shading

​

FROM ANT TO ELEPHANT 8 – WEB SERVER

• Discussion:
• IP or user_id?
• Different Web Server has different IP
• Where to balance: DNS? Web Server?
• DNS: Location, Network Operator,
• Virtual Server
• Architect Principle: One module, One Task
• 10 millions active user

FROM ANT TO ELEPHANT 9 – REVIEW PARTITION (WHY MICROSERVICE ?)

• HW: Cost of domain server is high due to complete session.
• ORG: Release depends more on Human Resource rather than HW. Normally, we have few experts with a large group of SWE
• Error propagation among modules
• Tech require: Different Tech stack: Python for ML, C++ for Cache Redis
• Reality: Could decouple it?
• Not easy: Payment, Product, Order, Even Auth
• Could we continue to reply on Design Pattern?
• Our solution: Hybrid -> 10 millions active login with 500 millions employee

FROM ANT TO ELEPHANT 10.1 – REVIEW DB FOR HYBRID OPTION

• Distribute DB requirement:
• Metadata management
• Consistency
• Partition
• Replication
• Distribute Transaction
• Foundation Algorithm: Bloom Filter, SkipList, LSM Tree, Merkle Hash Tree, Snappy, LZSS
• Architect Principle: Open to addition, close to modification.

FROM ANT TO ELEPHANT 10.2 – ETL

• Behavior data vs Transaction data
• Send data
• Pull and Push with Message Queue(Kafka)
• Batch and Streaming
• Architect Pattern: Event-Driven Architect

​

​

FROM ANT TO ELEPHANT 10.3 – ETL AND AWS LAMBDA

ML is microkernel of core domain components

Kafka will send the data.

Architect Pattern: Microkernel Architecture

​

h

FROM ANT TO ELEPHANT 11.1 – WHY HYBRID?

• New Analytical and Machine learning module will support Microservice and go to NoSQL database.
• Transactional data will be isolated(not partition) by Tenant and doesn’t go to Microservice.
• DB Shading service is only for huge customer.
• Distribute cache and Local cache will be used complementary.
• Distribute Service will be delegated to GCP.

​

FROM ANT TO ELEPHANT 11.2 – MICROSERVICE TECH EVOLUTION – J2EE

Only Java and JVM.

Business:

Not Open Source

heavy and difficult to use.

Tech:

Supports Distribute and Scalability, but it is more Naïve distribute.

Not designed for low latency, high availability

JSP: view in backend

JMS: Not flexible enough

JDBC: Only for RDBMS.

ORG:

stagnation, NOT Agile

​

FROM ANT TO ELEPHANT 11.2 – MICROSERVICE TECH EVOLUTION – SPRING CLOUD

• Service Registry: Netflix Eureka -> Spring Cloud Consul
• Keeps tracking of whats happening in the cluster
• VM and service
• Configure center: Spring Cloud Config -> Spring Cloud Consul
• API Gateway: Netflix Zuul -> Spring Cloud Gateway
• Service Resilience: Netflix Hystrix (latency and fault tolerance library) -> Resilience4j
• Load Balancing: Netfilix Ribbon -> Spring Cloud Loadbalancer
• Discussion:
• Should we use Spring framework on application level to provide functionalities of infra level for Microservice?
• The only reason is the development of microservice infra is still in progress.
• Why its market share is high?

FROM ANT TO ELEPHANT 11.3.1 – MICROSERVICE TECH EVOLUTION – WHY K8S?

• Discussion:
• Spring framework building time is long due to Config、Eureka、Zuul、Hystrix、Ribbon、Feign
• Domain Server and Infra are coupled together.
• Domain: Account, Warehouse, Payment
• Infra: Service Registration, Config, Gateway, Service Resilience(Circuit Breaker, Rate Limiting), Gateway, Load Balancing, Auth

​

FROM ANT TO ELEPHANT 11.3.2 – MICROSERVICE TECH EVOLUTION – WHY K8S?

• Discussion:
• Declarative vs Imperative
• SQL is a good language?
• Infra as configuration
• ML as configuration
• Low code platform

FROM ANT TO ELEPHANT 11.3.3 – MICROSERVICE TECH EVOLUTION – WHY K8S?

• Service Registry: Kubernetes Service
• Configure center: Kubernetes ConfigMap
• API Gateway: Netflix Zuul vs Ingress Controller
• Service Resilience: Netflix Hystrix, K8S doesn’t support Circuit Breaker, Rate Limit
• Load Balancing: Kubernetes Service only in DNS
• Auth: Spring Security OAuth 2 (K8s RBAC supports authorization but we don’t use, why?)
• Others: Monitor, Log analysis, Visualization Service. (K8S doesn’t support)
• Sidecar?
• Architect Principle: Add layer only if required

​

FROM ANT TO ELEPHANT 11.4.1 – MICROSERVICE TECH EVOLUTION – K8S AND ISTIO

• 我们的声音振聋发聩：
• 一个请求在哪个服务上调用失败啦？是 A 有调用 B 吗？还是 C 调用 D 时出错了？为什么这个请求、页面忽然卡住了？怎么调度到这个 Node 上的服务比其他 Node 慢那么多？这个 Pod 有 Bug，消耗了大量的 TCP 链接数……微服务太复杂了，已经学不过来了，让我们回归单体吧
• Objective:
• Most SWEs only focus their own single service code with SprintBoot, ignore SpringCloud and K8S.
• Experts and toolkit Quick troubleshooting crossing service.
• Solution: Manage, Monitor for entire system, not one service.
• Visualization of service call relation.
• Visualization of ETL and ML pipeline.
• Rule-based and Dynamic configuring param of single point circuit breaker, retry and balanced
• Beautiful monitoring
• Log collection and analysis
• Nice to have: ML helps.

、

• Paradox: Static HW infra vs Flexible requirement of Software
• What is boundary?

​

FROM ANT TO ELEPHANT 11.4.2 – MICROSERVICE TECH EVOLUTION – K8S AND ISTIO

• Service Registry: Kubernetes Service
• Configure center: Kubernetes ConfigMap
• API Gateway: Istio Ingress Gateway
• Service Resilience: Envoy
• Load Balancing: Envoy and KubeDNS
• Auth:
• Istio Security for entire system
• Spring Security OAuth2 only works with JWT for Jar

​

FROM ANT TO ELEPHANT 12 – SERVICE MESH

• TCP/IP -> MICROSERVICE -> SIDECAR -> SERVICE MESH

• SIDECAR:
• AIRBNB SYNAPSE AND NERVE
• NETFLIX PRANA

WHAT IS ANTI – PATTERN OF ARCHITECT

• “勤奋”程序员
• 喜欢读源代码，书中自有黄金屋
• 我们的征途是星辰大海
• 重复劳动太多？系统太成熟了？做的事情没挑战？
• 什么都学，但回头就忘
• “普通”程序员
• 速度之上，不耻下问
• 流行就是最好的
• 谜一样的解决了问题：寻找根源，辩证思维：现有结论是否还有疑点？能复现问题吗？自己搜集的信息足够吗？是否有另外解决问题的角度？log可以改进吗？事后针对性的monitor和分析

经济适用的架构

• 理解力 > 技术
• 舍弃不必要的诉求和需要的资源：
• 搜索结果100页
• APP下载排行TOPK: 200名以后只关心每日变化
• 容忍不精确：需要强一致性吗？Transaction vs Behavior？订单 vs 访问量？
• 冷热数据的处理: Newsfeed
• 热数据：大V和网红 -> 数据少，频次高 -> push
• 冷数据：-> 数据多，频次低 -> pull
• 脱离具体场景都是耍流氓
• 必要时降级：类似Windows安全模式，只保证核心服务的安全和健壮
• 不要只看请求频次QPS，还要看请求的集中度，制定规则

BOTTLENECK AND FOUNDATION 2: CONSISTENCY 1�

• Transaction需求：
• 事务结束的两种可能方式：
• commit：提交最小操作单元中的所有操作。-ATOMICITY
• terminate：操作终止，最小操作单元中所有修改无效。
• 数据库操作的环境：
• 共享-多用户并发访问.
• 不稳定-潜在的硬件/软件故障
• 事务所需环境：
• 不共享 - 一个事务内的操作不受其他事务影响 - ISOLATION
• 稳定 - 即使面对系统故障，当前事务的操作也能保留现场 – DURABILITY
• 一个事务开始的过程中必须确保：在该事务结束之前其他事务看不到它的结果。 - ISOLATION
• 如果事务中止：必须确保当前事务所有可能影响数据一致性的操作都会被清理。- CONSISTENCY
• 如果系统出现故障：必须确保重新启动时所有未提交的事务都会被清理。 - FAULT TOLERENCE AND CONSISTENCY
• 思考: DDL需要事务吗？

​

​

BOTTLENECK AND FOUNDATION 2: CONSISTENCY 2�

• ACID之A和D:

​

BOTTLENECK AND FOUNDATION 2: CONSISTENCY 3�

BOTTLENECK AND FOUNDATION 3: ALGORITHM 1 �

BOTTLENECK AND FOUNDATION 3: ALGORITHM 2 �

BOTTLENECK AND FOUNDATION 3: ALGORITHM 3�