-
Notifications
You must be signed in to change notification settings - Fork 2
/
Copy pathgroupcache.html
399 lines (378 loc) · 30.8 KB
/
groupcache.html
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
297
298
299
300
301
302
303
304
305
306
307
308
309
310
311
312
313
314
315
316
317
318
319
320
321
322
323
324
325
326
327
328
329
330
331
332
333
334
335
336
337
338
339
340
341
342
343
344
345
346
347
348
349
350
351
352
353
354
355
356
357
358
359
360
361
362
363
364
365
366
367
368
369
370
371
372
373
374
375
376
377
378
379
380
381
382
383
384
385
386
387
388
389
390
391
392
393
394
395
396
397
398
399
<?xml version="1.0" encoding="utf-8"?>
<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Strict//EN"
"http://www.w3.org/TR/xhtml1/DTD/xhtml1-strict.dtd">
<html xmlns="http://www.w3.org/1999/xhtml" lang="en" xml:lang="en">
<head>
<!-- 2021-01-23 Sat 18:03 -->
<meta http-equiv="Content-Type" content="text/html;charset=utf-8" />
<meta name="viewport" content="width=device-width, initial-scale=1" />
<title>groupcache源码阅读</title>
<meta name="generator" content="Org mode" />
<link rel="stylesheet" type="text/css" href="./css/org-css.css"/>
<link rel="stylesheet" type="text/css" href="./css/gitalk.css"/> <script src="./js/gitalk.min.js"></script>
</head>
<body>
<div id="org-div-home-and-up">
<a accesskey="h" href="index.html"> UP </a>
|
<a accesskey="H" href="index.html"> HOME </a>
</div><div id="content">
<h1 class="title">groupcache源码阅读</h1>
<div id="table-of-contents">
<h2>Table of Contents</h2>
<div id="text-table-of-contents">
<ul>
<li><a href="#org3b0ee44">1. 前言</a></li>
<li><a href="#org9fe4a5b">2. lru.cache结构</a></li>
<li><a href="#org6301d2b">3. cache结构</a></li>
<li><a href="#org5627e74">4. group结构</a></li>
<li><a href="#orgb06115c">5. 分布式结构</a></li>
<li><a href="#org7c24919">6. 总结</a></li>
</ul>
</div>
</div>
<div id="outline-container-org3b0ee44" class="outline-2">
<h2 id="org3b0ee44"><span class="section-number-2">1</span> 前言</h2>
<div class="outline-text-2" id="text-1">
<p>
groupcache是golang写的一个缓存库，因为是内存缓存，所以效率比较高。目前公司会用到，而且源码比较简单，所以饿着肚子通读了它的源码。
groupcache大概可以分为三层结构（自己分的，大雾），每层提供基本的功能，下层为上层提供服务。
第一层是lru包下的cache结构，这层提供了基本的cache功能，提供最近最少使用淘汰缓存数据。
第二层是groupcache.go文件中的cache结构，它是对lru.cache的分装，并提供安全并发功能，合一些基本的统计数据。
第三层是groupcache.go中的group结构, 它提供了简单的分布式缓存支持，flight缓存调用(对于并发的多次读取，只请求一次get，每个并发返回相同的数据)。
</p>
</div>
</div>
<div id="outline-container-org9fe4a5b" class="outline-2">
<h2 id="org9fe4a5b"><span class="section-number-2">2</span> lru.cache结构</h2>
<div class="outline-text-2" id="text-2">
<p>
lru下的cache结构，数据存放在一个双向链表中，并提供一个map映射到key跟列表的元素，链表主要提供lru算法。map主要提供快速查找key。如下图所示, list是标准库中的container/list
</p>
<div class="org-src-container">
<pre class="src src-go"><span style="color: #676E95;">// </span><span style="color: #676E95;">Cache is an LRU cache. It is not safe for concurrent access.</span>
<span style="color: #89DDFF;">type</span> <span style="color: #c792ea;">Cache</span> <span style="color: #89DDFF;">struct</span> {
<span style="color: #676E95;">// </span><span style="color: #676E95;">MaxEntries is the maximum number of cache entries before</span>
<span style="color: #676E95;">// </span><span style="color: #676E95;">an item is evicted. Zero means no limit.</span>
MaxEntries <span style="color: #c792ea;">int</span>
<span style="color: #676E95;">// </span><span style="color: #676E95;">OnEvicted optionally specificies a callback function to be</span>
<span style="color: #676E95;">// </span><span style="color: #676E95;">executed when an entry is purged from the cache.</span>
OnEvicted <span style="color: #89DDFF;">func</span>(<span style="color: #ffcb6b;">key</span> <span style="color: #c792ea;">Key</span>, <span style="color: #ffcb6b;">value</span> <span style="color: #89DDFF;">interface</span>{})
ll *<span style="color: #c792ea;">list.List</span>
cache <span style="color: #89DDFF;">map</span>[<span style="color: #89DDFF;">interface</span>{}]*<span style="color: #c792ea;">list.Element</span>
}
<span style="color: #676E95;">// </span><span style="color: #676E95;">Add adds a value to the cache.</span>
<span style="color: #89DDFF;">func</span> (<span style="color: #ffcb6b;">c</span> *<span style="color: #c792ea;">Cache</span>) <span style="color: #82aaff;">Add</span>(<span style="color: #ffcb6b;">key</span> <span style="color: #c792ea;">Key</span>, <span style="color: #ffcb6b;">value</span> <span style="color: #89DDFF;">interface</span>{}) {
<span style="color: #89DDFF;">if</span> c.cache == <span style="color: #f78c6c;">nil</span> {
c.cache = <span style="color: #82aaff;">make</span>(<span style="color: #89DDFF;">map</span>[<span style="color: #89DDFF;">interface</span>{}]*<span style="color: #c792ea;">list.Element</span>)
c.ll = list.<span style="color: #82aaff;">New</span>()
}
<span style="color: #89DDFF;">if</span> <span style="color: #ffcb6b;">ee</span>, <span style="color: #ffcb6b;">ok</span> := c.cache[key]; ok {
<span style="color: #676E95;">//</span><span style="color: #676E95;">如果拿到key的数据，是热点数据，放在链表最前面,并更新值</span>
c.ll.<span style="color: #82aaff;">MoveToFront</span>(ee)
ee.Value.(*<span style="color: #c792ea;">entry</span>).value = value
<span style="color: #89DDFF;">return</span>
}
<span style="color: #676E95;">//</span><span style="color: #676E95;">没有则存放在list中，并对map建立索引</span>
<span style="color: #ffcb6b;">ele</span> := c.ll.<span style="color: #82aaff;">PushFront</span>(&<span style="color: #c792ea;">entry</span>{key, value})
c.cache[key] = ele
<span style="color: #89DDFF;">if</span> c.MaxEntries != 0 && c.ll.<span style="color: #82aaff;">Len</span>() > c.MaxEntries {
c.<span style="color: #82aaff;">RemoveOldest</span>()
}
}
<span style="color: #676E95;">// </span><span style="color: #676E95;">Get looks up a key's value from the cache.</span>
<span style="color: #89DDFF;">func</span> (<span style="color: #ffcb6b;">c</span> *<span style="color: #c792ea;">Cache</span>) <span style="color: #82aaff;">Get</span>(<span style="color: #ffcb6b;">key</span> <span style="color: #c792ea;">Key</span>) (<span style="color: #ffcb6b;">value</span> <span style="color: #89DDFF;">interface</span>{}, <span style="color: #ffcb6b;">ok</span> <span style="color: #c792ea;">bool</span>) {
<span style="color: #89DDFF;">if</span> c.cache == <span style="color: #f78c6c;">nil</span> {
<span style="color: #89DDFF;">return</span>
}
<span style="color: #89DDFF;">if</span> <span style="color: #ffcb6b;">ele</span>, <span style="color: #ffcb6b;">hit</span> := c.cache[key]; hit {
<span style="color: #676E95;">//</span><span style="color: #676E95;">如果能拿到，则是热点数据，移动到最前面。</span>
c.ll.<span style="color: #82aaff;">MoveToFront</span>(ele)
<span style="color: #89DDFF;">return</span> ele.Value.(*<span style="color: #c792ea;">entry</span>).value, <span style="color: #f78c6c;">true</span>
}
<span style="color: #89DDFF;">return</span>
}
</pre>
</div>
<p>
其他的一些操作都非常简单，不一一列出.
</p>
</div>
</div>
<div id="outline-container-org6301d2b" class="outline-2">
<h2 id="org6301d2b"><span class="section-number-2">3</span> cache结构</h2>
<div class="outline-text-2" id="text-3">
<p>
groupcache中的cache主要是加了并发安全，并添加一些统计数据, 一些操作都是直接调用lru.cache
</p>
<div class="org-src-container">
<pre class="src src-go"><span style="color: #89DDFF;">type</span> <span style="color: #c792ea;">cache</span> <span style="color: #89DDFF;">struct</span> {
mu <span style="color: #c792ea;">sync.RWMutex</span>
nbytes <span style="color: #c792ea;">int64</span> <span style="color: #676E95;">// </span><span style="color: #676E95;">of all keys and values</span>
lru *<span style="color: #c792ea;">lru.Cache</span>
nhit, nget <span style="color: #c792ea;">int64</span>
nevict <span style="color: #c792ea;">int64</span> <span style="color: #676E95;">// </span><span style="color: #676E95;">number of evictions</span>
}
<span style="color: #89DDFF;">func</span> (<span style="color: #ffcb6b;">c</span> *<span style="color: #c792ea;">cache</span>) <span style="color: #82aaff;">add</span>(<span style="color: #ffcb6b;">key</span> <span style="color: #c792ea;">string</span>, <span style="color: #ffcb6b;">value</span> <span style="color: #c792ea;">ByteView</span>) {
c.mu.<span style="color: #82aaff;">Lock</span>()
<span style="color: #89DDFF;">defer</span> c.mu.<span style="color: #82aaff;">Unlock</span>()
<span style="color: #89DDFF;">if</span> c.lru == <span style="color: #f78c6c;">nil</span> {
c.lru = &<span style="color: #c792ea;">lru.Cache</span>{
<span style="color: #f78c6c;">OnEvicted</span>: <span style="color: #89DDFF;">func</span>(<span style="color: #ffcb6b;">key</span> <span style="color: #c792ea;">lru.Key</span>, <span style="color: #ffcb6b;">value</span> <span style="color: #89DDFF;">interface</span>{}) {
<span style="color: #ffcb6b;">val</span> := value.(<span style="color: #c792ea;">ByteView</span>)
c.nbytes -= <span style="color: #82aaff;">int64</span>(<span style="color: #82aaff;">len</span>(key.(<span style="color: #c792ea;">string</span>))) + <span style="color: #82aaff;">int64</span>(val.<span style="color: #82aaff;">Len</span>())
c.nevict++
},
}
}
c.lru.<span style="color: #82aaff;">Add</span>(key, value)
c.nbytes += <span style="color: #82aaff;">int64</span>(<span style="color: #82aaff;">len</span>(key)) + <span style="color: #82aaff;">int64</span>(value.<span style="color: #82aaff;">Len</span>())
}
<span style="color: #89DDFF;">func</span> (<span style="color: #ffcb6b;">c</span> *<span style="color: #c792ea;">cache</span>) <span style="color: #82aaff;">get</span>(<span style="color: #ffcb6b;">key</span> <span style="color: #c792ea;">string</span>) (<span style="color: #ffcb6b;">value</span> <span style="color: #c792ea;">ByteView</span>, <span style="color: #ffcb6b;">ok</span> <span style="color: #c792ea;">bool</span>) {
c.mu.<span style="color: #82aaff;">Lock</span>()
<span style="color: #89DDFF;">defer</span> c.mu.<span style="color: #82aaff;">Unlock</span>()
c.nget++
<span style="color: #89DDFF;">if</span> c.lru == <span style="color: #f78c6c;">nil</span> {
<span style="color: #89DDFF;">return</span>
}
<span style="color: #ffcb6b;">vi</span>, <span style="color: #ffcb6b;">ok</span> := c.lru.<span style="color: #82aaff;">Get</span>(key)
<span style="color: #89DDFF;">if</span> <span style="color: #89DDFF; font-weight: bold;">!</span>ok {
<span style="color: #89DDFF;">return</span>
}
c.nhit++
<span style="color: #89DDFF;">return</span> vi.(<span style="color: #c792ea;">ByteView</span>), <span style="color: #f78c6c;">true</span>
}
</pre>
</div>
</div>
</div>
<div id="outline-container-org5627e74" class="outline-2">
<h2 id="org5627e74"><span class="section-number-2">4</span> group结构</h2>
<div class="outline-text-2" id="text-4">
<p>
group结构如下:
</p>
<div class="org-src-container">
<pre class="src src-go"><span style="color: #89DDFF;">type</span> <span style="color: #c792ea;">Group</span> <span style="color: #89DDFF;">struct</span> {
<span style="color: #676E95;">//</span><span style="color: #676E95;">group的名字，必须唯一</span>
name <span style="color: #c792ea;">string</span>
<span style="color: #676E95;">//</span><span style="color: #676E95;">getter方法，用于从缓存失效后从数据库或其他地方获取数据.</span>
getter <span style="color: #c792ea;">Getter</span>
<span style="color: #676E95;">//</span><span style="color: #676E95;">分布式支持</span>
peersOnce <span style="color: #c792ea;">sync.Once</span>
peers <span style="color: #c792ea;">PeerPicker</span>
cacheBytes <span style="color: #c792ea;">int64</span> <span style="color: #676E95;">// </span><span style="color: #676E95;">limit for sum of mainCache and hotCache size</span>
<span style="color: #676E95;">//</span><span style="color: #676E95;">两个缓存</span>
<span style="color: #676E95;">// </span><span style="color: #676E95;">mainCache is a cache of the keys for which this process</span>
<span style="color: #676E95;">// </span><span style="color: #676E95;">(amongst its peers) is authoritative. That is, this cache</span>
<span style="color: #676E95;">// </span><span style="color: #676E95;">contains keys which consistent hash on to this process's</span>
<span style="color: #676E95;">// </span><span style="color: #676E95;">peer number.</span>
mainCache <span style="color: #c792ea;">cache</span>
<span style="color: #676E95;">// </span><span style="color: #676E95;">hotCache contains keys/values for which this peer is not</span>
<span style="color: #676E95;">// </span><span style="color: #676E95;">authoritative (otherwise they would be in mainCache), but</span>
<span style="color: #676E95;">// </span><span style="color: #676E95;">are popular enough to warrant mirroring in this process to</span>
<span style="color: #676E95;">// </span><span style="color: #676E95;">avoid going over the network to fetch from a peer. Having</span>
<span style="color: #676E95;">// </span><span style="color: #676E95;">a hotCache avoids network hotspotting, where a peer's</span>
<span style="color: #676E95;">// </span><span style="color: #676E95;">network card could become the bottleneck on a popular key.</span>
<span style="color: #676E95;">// </span><span style="color: #676E95;">This cache is used sparingly to maximize the total number</span>
<span style="color: #676E95;">// </span><span style="color: #676E95;">of key/value pairs that can be stored globally.</span>
hotCache <span style="color: #c792ea;">cache</span>
<span style="color: #676E95;">// </span><span style="color: #676E95;">loadGroup ensures that each key is only fetched once</span>
<span style="color: #676E95;">// </span><span style="color: #676E95;">(either locally or remotely), regardless of the number of</span>
<span style="color: #676E95;">// </span><span style="color: #676E95;">concurrent callers.</span>
<span style="color: #676E95;">// </span><span style="color: #676E95;">在缓存命中失败的时候减少调用</span>
loadGroup <span style="color: #c792ea;">flightGroup</span>
_ <span style="color: #c792ea;">int32</span> <span style="color: #676E95;">// </span><span style="color: #676E95;">force Stats to be 8-byte aligned on 32-bit platforms</span>
<span style="color: #676E95;">// </span><span style="color: #676E95;">Stats are statistics on the group.</span>
Stats <span style="color: #c792ea;">Stats</span>
}
</pre>
</div>
<p>
先说flightGroup结构比较简单，这个结构主要是在缓存失效的时候，减少对底层的访问。比如一个缓存数据失效了，这个时候同时会有很多人调用接口，缓存都没有命中，就会对数据库发起很多次调用，其实这个时候只要调用一次就行了，其他的都是相同的数据。
</p>
<div class="org-src-container">
<pre class="src src-go"><span style="color: #89DDFF;">func</span> (<span style="color: #ffcb6b;">g</span> *<span style="color: #c792ea;">Group</span>) <span style="color: #82aaff;">Do</span>(<span style="color: #ffcb6b;">key</span> <span style="color: #c792ea;">string</span>, <span style="color: #ffcb6b;">fn</span> <span style="color: #89DDFF;">func</span>() (<span style="color: #89DDFF;">interface</span>{}, <span style="color: #c792ea;">error</span>)) (<span style="color: #89DDFF;">interface</span>{}, <span style="color: #c792ea;">error</span>) {
g.mu.<span style="color: #82aaff;">Lock</span>()
<span style="color: #89DDFF;">if</span> g.m == <span style="color: #f78c6c;">nil</span> {
g.m = <span style="color: #82aaff;">make</span>(<span style="color: #89DDFF;">map</span>[<span style="color: #c792ea;">string</span>]*<span style="color: #c792ea;">call</span>)
}
<span style="color: #676E95;">//</span><span style="color: #676E95;">对于同一个key的调用等待其他goroutine调用完成</span>
<span style="color: #89DDFF;">if</span> <span style="color: #ffcb6b;">c</span>, <span style="color: #ffcb6b;">ok</span> := g.m[key]; ok {
g.mu.<span style="color: #82aaff;">Unlock</span>()
c.wg.<span style="color: #82aaff;">Wait</span>()
<span style="color: #676E95;">//</span><span style="color: #676E95;">直接返回数据</span>
<span style="color: #89DDFF;">return</span> c.val, c.err
}
<span style="color: #676E95;">//</span><span style="color: #676E95;">同个key的调用放在map中</span>
<span style="color: #ffcb6b;">c</span> := <span style="color: #82aaff;">new</span>(<span style="color: #c792ea;">call</span>)
c.wg.<span style="color: #82aaff;">Add</span>(1)
g.m[key] = c
g.mu.<span style="color: #82aaff;">Unlock</span>()
<span style="color: #676E95;">//</span><span style="color: #676E95;">获取数据</span>
c.val, c.err = <span style="color: #82aaff;">fn</span>()
c.wg.<span style="color: #82aaff;">Done</span>()
<span style="color: #676E95;">//</span><span style="color: #676E95;">删除限制</span>
g.mu.<span style="color: #82aaff;">Lock</span>()
<span style="color: #82aaff;">delete</span>(g.m, key)
g.mu.<span style="color: #82aaff;">Unlock</span>()
<span style="color: #89DDFF;">return</span> c.val, c.err
}
</pre>
</div>
<p>
来看看group的get方法是怎么使用的
</p>
<div class="org-src-container">
<pre class="src src-go"><span style="color: #89DDFF;">func</span> (<span style="color: #ffcb6b;">g</span> *<span style="color: #c792ea;">Group</span>) <span style="color: #82aaff;">Get</span>(<span style="color: #ffcb6b;">ctx</span> <span style="color: #c792ea;">Context</span>, <span style="color: #ffcb6b;">key</span> <span style="color: #c792ea;">string</span>, <span style="color: #ffcb6b;">dest</span> <span style="color: #c792ea;">Sink</span>) <span style="color: #c792ea;">error</span> {
g.peersOnce.<span style="color: #82aaff;">Do</span>(g.initPeers)
g.Stats.Gets.<span style="color: #82aaff;">Add</span>(1)
<span style="color: #89DDFF;">if</span> dest == <span style="color: #f78c6c;">nil</span> {
<span style="color: #89DDFF;">return</span> errors.<span style="color: #82aaff;">New</span>(<span style="color: #c3e88d;">"groupcache: nil dest Sink"</span>)
}
<span style="color: #676E95;">//</span><span style="color: #676E95;">在缓存中查看key</span>
<span style="color: #ffcb6b;">value</span>, <span style="color: #ffcb6b;">cacheHit</span> := g.<span style="color: #82aaff;">lookupCache</span>(key)
<span style="color: #89DDFF;">if</span> cacheHit {
g.Stats.CacheHits.<span style="color: #82aaff;">Add</span>(1)
<span style="color: #89DDFF;">return</span> <span style="color: #82aaff;">setSinkView</span>(dest, value)
}
<span style="color: #676E95;">// </span><span style="color: #676E95;">Optimization to avoid double unmarshalling or copying: keep</span>
<span style="color: #676E95;">// </span><span style="color: #676E95;">track of whether the dest was already populated. One caller</span>
<span style="color: #676E95;">// </span><span style="color: #676E95;">(if local) will set this; the losers will not. The common</span>
<span style="color: #676E95;">// </span><span style="color: #676E95;">case will likely be one caller.</span>
<span style="color: #ffcb6b;">destPopulated</span> := <span style="color: #f78c6c;">false</span>
<span style="color: #676E95;">//</span><span style="color: #676E95;">如果没有在缓存中找到数据，就从getter方法中load进来</span>
<span style="color: #ffcb6b;">value</span>, <span style="color: #ffcb6b;">destPopulated</span>, <span style="color: #ffcb6b;">err</span> := g.<span style="color: #82aaff;">load</span>(ctx, key, dest)
<span style="color: #89DDFF;">if</span> err != <span style="color: #f78c6c;">nil</span> {
<span style="color: #89DDFF;">return</span> err
}
<span style="color: #89DDFF;">if</span> destPopulated {
<span style="color: #89DDFF;">return</span> <span style="color: #f78c6c;">nil</span>
}
<span style="color: #89DDFF;">return</span> <span style="color: #82aaff;">setSinkView</span>(dest, value)
}
<span style="color: #89DDFF;">func</span> (<span style="color: #ffcb6b;">g</span> *<span style="color: #c792ea;">Group</span>) <span style="color: #82aaff;">lookupCache</span>(<span style="color: #ffcb6b;">key</span> <span style="color: #c792ea;">string</span>) (<span style="color: #ffcb6b;">value</span> <span style="color: #c792ea;">ByteView</span>, <span style="color: #ffcb6b;">ok</span> <span style="color: #c792ea;">bool</span>) {
<span style="color: #676E95;">//</span><span style="color: #676E95;">这个方法比较简单，从是从maincache和hotcache中读取数据</span>
<span style="color: #89DDFF;">if</span> g.cacheBytes <= 0 {
<span style="color: #89DDFF;">return</span>
}
value, ok = g.mainCache.<span style="color: #82aaff;">get</span>(key)
<span style="color: #89DDFF;">if</span> ok {
<span style="color: #89DDFF;">return</span>
}
value, ok = g.hotCache.<span style="color: #82aaff;">get</span>(key)
<span style="color: #89DDFF;">return</span>
}
<span style="color: #89DDFF;">func</span> (<span style="color: #ffcb6b;">g</span> *<span style="color: #c792ea;">Group</span>) <span style="color: #82aaff;">load</span>(<span style="color: #ffcb6b;">ctx</span> <span style="color: #c792ea;">Context</span>, <span style="color: #ffcb6b;">key</span> <span style="color: #c792ea;">string</span>, <span style="color: #ffcb6b;">dest</span> <span style="color: #c792ea;">Sink</span>) (<span style="color: #ffcb6b;">value</span> <span style="color: #c792ea;">ByteView</span>, <span style="color: #ffcb6b;">destPopulated</span> <span style="color: #c792ea;">bool</span>, <span style="color: #ffcb6b;">err</span> <span style="color: #c792ea;">error</span>) {
g.Stats.Loads.<span style="color: #82aaff;">Add</span>(1)
<span style="color: #676E95;">//</span><span style="color: #676E95;">loadGroup减少对底层的调用，上面已经说了</span>
<span style="color: #ffcb6b;">viewi</span>, <span style="color: #ffcb6b;">err</span> := g.loadGroup.<span style="color: #82aaff;">Do</span>(key, <span style="color: #89DDFF;">func</span>() (<span style="color: #89DDFF;">interface</span>{}, <span style="color: #c792ea;">error</span>) {
<span style="color: #89DDFF;">if</span> <span style="color: #ffcb6b;">value</span>, <span style="color: #ffcb6b;">cacheHit</span> := g.<span style="color: #82aaff;">lookupCache</span>(key); cacheHit {
g.Stats.CacheHits.<span style="color: #82aaff;">Add</span>(1)
<span style="color: #89DDFF;">return</span> value, <span style="color: #f78c6c;">nil</span>
}
g.Stats.LoadsDeduped.<span style="color: #82aaff;">Add</span>(1)
<span style="color: #89DDFF;">var</span> <span style="color: #ffcb6b;">value</span> <span style="color: #c792ea;">ByteView</span>
<span style="color: #89DDFF;">var</span> <span style="color: #ffcb6b;">err</span> <span style="color: #c792ea;">error</span>
<span style="color: #676E95;">//</span><span style="color: #676E95;">如果能从远程获取，就从分布式的其他机子获取就从其他机子获取，因为其他机器也是缓存数据比数据库快</span>
<span style="color: #89DDFF;">if</span> <span style="color: #ffcb6b;">peer</span>, <span style="color: #ffcb6b;">ok</span> := g.peers.<span style="color: #82aaff;">PickPeer</span>(key); ok {
value, err = g.<span style="color: #82aaff;">getFromPeer</span>(ctx, peer, key)
<span style="color: #89DDFF;">if</span> err == <span style="color: #f78c6c;">nil</span> {
g.Stats.PeerLoads.<span style="color: #82aaff;">Add</span>(1)
<span style="color: #89DDFF;">return</span> value, <span style="color: #f78c6c;">nil</span>
}
g.Stats.PeerErrors.<span style="color: #82aaff;">Add</span>(1)
<span style="color: #676E95;">// </span><span style="color: #cc9393;">TODO</span><span style="color: #676E95;">(bradfitz): log the peer's error? keep</span>
<span style="color: #676E95;">// </span><span style="color: #676E95;">log of the past few for /groupcachez? It's</span>
<span style="color: #676E95;">// </span><span style="color: #676E95;">probably boring (normal task movement), so not</span>
<span style="color: #676E95;">// </span><span style="color: #676E95;">worth logging I imagine.</span>
}
<span style="color: #676E95;">//</span><span style="color: #676E95;">调用getter方法，获取数据(从数据库，或者其他地方)</span>
value, err = g.<span style="color: #82aaff;">getLocally</span>(ctx, key, dest)
<span style="color: #89DDFF;">if</span> err != <span style="color: #f78c6c;">nil</span> {
g.Stats.LocalLoadErrs.<span style="color: #82aaff;">Add</span>(1)
<span style="color: #89DDFF;">return</span> <span style="color: #f78c6c;">nil</span>, err
}
g.Stats.LocalLoads.<span style="color: #82aaff;">Add</span>(1)
destPopulated = <span style="color: #f78c6c;">true</span> <span style="color: #676E95;">// </span><span style="color: #676E95;">only one caller of load gets this return value</span>
<span style="color: #676E95;">//</span><span style="color: #676E95;">把数据存放在cache中</span>
g.<span style="color: #82aaff;">populateCache</span>(key, value, &g.mainCache)
<span style="color: #89DDFF;">return</span> value, <span style="color: #f78c6c;">nil</span>
})
<span style="color: #89DDFF;">if</span> err == <span style="color: #f78c6c;">nil</span> {
value = viewi.(<span style="color: #c792ea;">ByteView</span>)
}
<span style="color: #89DDFF;">return</span>
}
</pre>
</div>
</div>
</div>
<div id="outline-container-orgb06115c" class="outline-2">
<h2 id="orgb06115c"><span class="section-number-2">5</span> 分布式结构</h2>
<div class="outline-text-2" id="text-5">
<p>
上面提到了从分布式机子获取缓存使用的是getFromPeer方法,下面是源码
</p>
<div class="org-src-container">
<pre class="src src-go"><span style="color: #676E95;">//</span><span style="color: #676E95;">每一个分布式的服务都需要实现一个Get方法，接口描述文件在proto文件中</span>
<span style="color: #89DDFF;">func</span> (<span style="color: #ffcb6b;">g</span> *<span style="color: #c792ea;">Group</span>) <span style="color: #82aaff;">getFromPeer</span>(<span style="color: #ffcb6b;">ctx</span> <span style="color: #c792ea;">Context</span>, <span style="color: #ffcb6b;">peer</span> <span style="color: #c792ea;">ProtoGetter</span>, <span style="color: #ffcb6b;">key</span> <span style="color: #c792ea;">string</span>) (<span style="color: #c792ea;">ByteView</span>, <span style="color: #c792ea;">error</span>) {
<span style="color: #ffcb6b;">req</span> := &<span style="color: #c792ea;">pb.GetRequest</span>{
<span style="color: #f78c6c;">Group</span>: &g.name,
<span style="color: #f78c6c;">Key</span>: &key,
}
<span style="color: #ffcb6b;">res</span> := &<span style="color: #c792ea;">pb.GetResponse</span>{}
<span style="color: #676E95;">//</span><span style="color: #676E95;">从远端得到数据</span>
<span style="color: #ffcb6b;">err</span> := peer.<span style="color: #82aaff;">Get</span>(ctx, req, res)
<span style="color: #89DDFF;">if</span> err != <span style="color: #f78c6c;">nil</span> {
<span style="color: #89DDFF;">return</span> <span style="color: #c792ea;">ByteView</span>{}, err
}
<span style="color: #ffcb6b;">value</span> := <span style="color: #c792ea;">ByteView</span>{<span style="color: #f78c6c;">b</span>: res.Value}
<span style="color: #676E95;">// </span><span style="color: #cc9393;">TODO</span><span style="color: #676E95;">(bradfitz): use res.MinuteQps or something smart to</span>
<span style="color: #676E95;">// </span><span style="color: #676E95;">conditionally populate hotCache. For now just do it some</span>
<span style="color: #676E95;">// </span><span style="color: #676E95;">percentage of the time.</span>
<span style="color: #676E95;">//</span><span style="color: #676E95;">哈哈，这里随机放在hotCache中,有意思</span>
<span style="color: #89DDFF;">if</span> rand.<span style="color: #82aaff;">Intn</span>(10) == 0 {
g.<span style="color: #82aaff;">populateCache</span>(key, value, &g.hotCache)
}
<span style="color: #89DDFF;">return</span> value, <span style="color: #f78c6c;">nil</span>
}
</pre>
</div>
<p>
然后groupcache实现了一个Get接口，源码在http.go中, 有兴趣的可以看看。这里就不加了。
</p>
</div>
</div>
<div id="outline-container-org7c24919" class="outline-2">
<h2 id="org7c24919"><span class="section-number-2">6</span> 总结</h2>
<div class="outline-text-2" id="text-6">
<p>
所以现在可以看到一次groupcache的读取过程如下:
</p>
<ol class="org-ol">
<li>从mainCache中读取</li>
<li>从hotCache中读取</li>
<li>从分布式集群中的其他机器读取</li>
<li>从底层(数据库或其他地方)读取</li>
</ol>
<p>
不过groupcache只支持get方法，你不能自己控制cache过期。这点不大方便，因为我们总希望实时获取数据。
</p>
</div>
</div>
</div>
<div id="postamble" class="status">
<div id="gitalk" /> <script> var gitalk = new Gitalk({
clientID: 'f30e66bb5ab9089aa742',
clientSecret: '5d256d445447bd4db16540c2aab0e0884218ed12',
repo: 'guidao.github.io',
owner: 'guidao',
admin: ['guidao'],
id: location.pathname, // Ensure uniqueness and length less than 50
distractionFreeMode: false // Facebook-like distraction free mode
})
gitalk.render('gitalk') </script>
</div>
</body>
</html>