An introduction to state-based CRDTs:修订间差异

来自WHY42
Riguz留言 | 贡献
Riguz留言 | 贡献
第40行: 第40行:
Imagine that you need to build a planet-scale video streaming service. Whenever a user uploads a video, we are replicating it across different data centers located on different continents to maintain good throughput and latency, and in result a better user experience. Additionally we want to show users a view count for that video.
Imagine that you need to build a planet-scale video streaming service. Whenever a user uploads a video, we are replicating it across different data centers located on different continents to maintain good throughput and latency, and in result a better user experience. Additionally we want to show users a view count for that video.
-->
-->
<q>
<q>
试想你需要建造一个全球范围的视频流服务。
试想你需要建造一个全球范围的视频流服务。
第48行: 第49行:
我们还希望把视频的播放次数展示给用户。
我们还希望把视频的播放次数展示给用户。
</q>
</q>
<!--
<!--
Video uploading is a good example of master-slave replication. However things may complicate for such a small feature as view count. With many concurrent users over the entire planet and write-heavy characteristics, using the same approach for counter increments is not a great idea, as this may end up with congestion for more popular videos. We don't need transactions, as the nature of the problem allows us to loose constrains of strong consistency in favor of higher availability of our application. However as described earlier most of the existing solutions are based on exclusive write access to a replicated resource. This is where CRDTs and multi-master scenarios come to play.
Video uploading is a good example of master-slave replication. However things may complicate for such a small feature as view count. With many concurrent users over the entire planet and write-heavy characteristics, using the same approach for counter increments is not a great idea, as this may end up with congestion for more popular videos. We don't need transactions, as the nature of the problem allows us to loose constrains of strong consistency in favor of higher availability of our application. However as described earlier most of the existing solutions are based on exclusive write access to a replicated resource. This is where CRDTs and multi-master scenarios come to play.

2024年9月17日 (二) 13:31的版本

原文链接: https://www.bartoszsypytkowski.com/the-state-of-a-state-based-crdts/
作者: Bartosz Sypytkowski

(翻译)基于状态的CRDT简介

这是一个在以往的一些分享中已经谈论过的主题,但却从未写过。 这里我想写一点关于CRDT(冲突无关复制数据类型)的内容: 它要解决什么样的问题,并通过一些基本的代码实现(F#)来帮助你理解怎样去使用CRDT。

动机

CRDT是解决一个常见的分布式环境下数据同步问题的答案。 尽管人们熟知分布式领域中的此类问题,在过去已经进行了许多尝试来解决, 通常是去中心化的两阶段提交事物的变种。 但他们无一例外有以下的问题:

  • 我们的一个假设是,所有的参与者总是在事务发生期间是可用的,而经常这个假设恰恰是不成立的(比如在边缘计算和移动应用的场景中)
  • 多阶段提交和使用议会制的方案需要在参与者之间进行多次往返通信。这附带带来延迟和吞吐量的开销。而且,当机器增多或者通信距离增加时,这种方式有降低可扩展性的趋势。多数场景中,一旦跨越单个数据中心机房,使用分布式事务都是行不通的
  • 这些方案通常确定一个单一的主节点来进行排他写操作,或者作为单一的数据源。在某些情况下,比如上面提到的场景中,都是不可接受的。

这并非是说先有的解决方案不好。 我们期望在一个更广阔的可能的解决方案的范围中, 找到解决特定条件下的问题的更佳方式。 这其中我期望呈现的一个问题是:

试想你需要建造一个全球范围的视频流服务。 每当用户上传一个视频文件的时候, 都需要复制到位于其他大陆的数据中心中,以便维持良好的延迟和吞吐量, 从而得到更好的用户体验。 不仅如此, 我们还希望把视频的播放次数展示给用户。