一:分布式缓存redis 单进程单线程(一个redis进程是一个redis实例) https://blog.csdn.net/liupeng_qwert/article/details/77263187 https://www.cnblogs.com/syyong/p/6231326.html https://help.aliyun.com/document_detail/52228.html?spm=a2c4g.11186623.6.546.3S95dr http://www.redis.cn/topics/introduction.html
http://www.redis.cn/topics/partitioning.html 二:分布式缓存tair https://blog.csdn.net/farphone/article/details/53522383
三:分布式消息kafka(文件形式存储硬盘) https://help.aliyun.com/document_detail/68156.html?spm=a2c4g.11186623.6.543.IG04U8 https://blog.csdn.net/qq_21989939/article/details/79452065
生产建议2个集群【1个业务1个日志集群,每个集群3台机器(建议配置4c8g2t)】,测试可一台、kafka默认不支持ons的tag标签,但通过透传已经支持, 巨量的partition不支持 集群上部署kafka代码一模一样,但数据是分布式存储,即每台机器上的partition存储数据不一样 kafka比ons性能高大概3倍,因为从代码上进行了优化,包括但不限于线程和nio模式、跑批处理 kafka的延迟和事务的问题 topic与partition 是一对多,一个partition 在磁盘是一个文件 Data written to Kafka is written to disk and replicated for fault-tolerance. Kafka allows producers to wait on acknowledgement so that a write isn't considered complete until it is fully replicated and guaranteed to persist even if the server written to fails.
Kafka uses ZooKeeper so you need to first start a ZooKeeper server if you don't already have one. zookeeper注册节点和topic信息 【http://kafka.apache.org/documentation/】 Kafka的Message存储采用了分区(partition),分段(LogSegment)和稀疏索引这几个手段来达到了高效性