-
Notifications
You must be signed in to change notification settings - Fork 0
Expand file tree
/
Copy pathsearch.xml
More file actions
216 lines (102 loc) · 95.8 KB
/
search.xml
File metadata and controls
216 lines (102 loc) · 95.8 KB
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
<?xml version="1.0" encoding="utf-8"?>
<search>
<entry>
<title>详解G1垃圾回收器</title>
<link href="2022/09/17/xiang-jie-g1-la-ji-hui-shou-qi/"/>
<url>2022/09/17/xiang-jie-g1-la-ji-hui-shou-qi/</url>
<content type="html"><![CDATA[<p>摘自知乎: <a href="https://zhuanlan.zhihu.com/p/130992385" target="_blank" rel="noopener">https://zhuanlan.zhihu.com/p/130992385</a></p><p>[TOC]</p><h4><span id="jian-jie">简介:</span><a href="#jian-jie" class="header-anchor">#</a></h4><p>Garbage First(G1)是垃圾收集领域的最新成果，同时也是HotSpot在JVM上力推的垃圾收集器，并赋予取代CMS的使命。如果使用Java 8/9，那么有很大可能希望对G1收集器进行评估。本文详细首先对JVM其他的垃圾收集器进行总结，并与G1进行了简单的对比；然后通过G1的内存模型、G1的活动周期，对G1的工作机制进行了介绍；同时还在介绍过程中，描述了可能需要引起注意的优化点。</p><h5><span id="jvm-gc-shou-ji-qi-de-hui-gu-yu-bi-jiao">JVM GC收集器的回顾与比较</span><a href="#jvm-gc-shou-ji-qi-de-hui-gu-yu-bi-jiao" class="header-anchor">#</a></h5><blockquote><p>到JDK1.8为止的垃圾回收器图如下</p><p><img src="https://gitee.com/fa20091001/blogImage/raw/master/picGoimg/image-20201209155318912.png" alt="image-20201209155318912"></p><p>其实他们的一个迭代的过程是随着技术的演化进行的</p><h3><span id="chuan-xing-shou-ji-qi-zu-he-serial-serial-old">串行收集器组合 Serial + Serial Old</span><a href="#chuan-xing-shou-ji-qi-zu-he-serial-serial-old" class="header-anchor">#</a></h3><blockquote><p>开启选项：-XX:+SerialGC</p><p>串行收集器是最基本、发展时间最长、久经考验的垃圾收集器，也是client模式下的默认收集器配置。</p><p>串行收集器采用单线程stop-the-world的方式进行收集。当内存不足时，串行GC设置停顿标识，待所有线程都进入安全点(Safepoint)时，应用线程暂停，串行GC开始工作，采用单线程方式回收空间并整理内存。单线程也意味着复杂度更低、占用内存更少，但同时也意味着不能有效利用多核优势。事实上，串行收集器特别适合堆内存不高、单核甚至双核CPU的场合。</p></blockquote><h3><span id="bing-xing-shou-ji-qi-zu-he-parallel-scavenge-parallel-old">并行收集器组合 Parallel Scavenge + Parallel Old</span><a href="#bing-xing-shou-ji-qi-zu-he-parallel-scavenge-parallel-old" class="header-anchor">#</a></h3><blockquote><p>开启选项：-XX:+UseParallelGC或-XX:+UseParallelOldGC(可互相激活)</p><p>并行收集器是以关注吞吐量为目标的垃圾收集器，也是server模式下的默认收集器配置，对吞吐量的关注主要体现在年轻代Parallel Scavenge收集器上。</p><p>并行收集器与串行收集器工作模式相似，都是stop-the-world方式，只是暂停时并行地进行垃圾收集。年轻代采用复制算法，老年代采用标记-整理，在回收的同时还会对内存进行压缩。关注吞吐量主要指年轻代的Parallel Scavenge收集器，通过两个目标参数-XX:MaxGCPauseMills和-XX:GCTimeRatio，调整新生代空间大小，来降低GC触发的频率。并行收集器适合对吞吐量要求远远高于延迟要求的场景，并且在满足最差延时的情况下，并行收集器将提供最佳的吞吐量。</p></blockquote><h3><span id="bing-fa-biao-ji-qing-chu-shou-ji-qi-zu-he-parnew-cms-serial-old">并发标记清除收集器组合 ParNew + CMS + Serial Old</span><a href="#bing-fa-biao-ji-qing-chu-shou-ji-qi-zu-he-parnew-cms-serial-old" class="header-anchor">#</a></h3><blockquote><p>开启选项：-XX:+UseConcMarkSweepGC</p><p>并发标记清除(CMS)是以关注延迟为目标、十分优秀的垃圾回收算法，开启后，年轻代使用STW式的并行收集，老年代回收采用CMS进行垃圾回收，对延迟的关注也主要体现在老年代CMS上。</p><p>年轻代ParNew与并行收集器类似，而老年代CMS每个收集周期都要经历：初始标记、并发标记、重新标记、并发清除。其中，初始标记以STW的方式标记所有的根对象；并发标记则同应用线程一起并行，标记出根对象的可达路径；在进行垃圾回收前，CMS再以一个STW进行重新标记，标记那些由mutator线程(指引起数据变化的线程，即应用线程)修改而可能错过的可达对象；最后得到的不可达对象将在并发清除阶段进行回收。值得注意的是，初始标记和重新标记都已优化为多线程执行。CMS非常适合堆内存大、CPU核数多的服务器端应用，也是G1出现之前大型应用的首选收集器。</p><p>但是CMS并不完美，它有以下缺点：</p><blockquote><ol><li>由于并发进行，CMS在收集与应用线程会同时会增加对堆内存的占用，也就是说，CMS必须要在老年代堆内存用尽之前完成垃圾回收，否则CMS回收失败时，将触发担保机制，串行老年代收集器将会以STW的方式进行一次GC，从而造成较大停顿时间；</li><li>标记清除算法无法整理空间碎片，老年代空间会随着应用时长被逐步耗尽，最后将不得不通过担保机制对堆内存进行压缩。CMS也提供了参数-XX:CMSFullGCsBeForeCompaction(默认0，即每次都进行内存整理)来指定多少次CMS收集之后，进行一次压缩的Full GC。</li></ol></blockquote></blockquote><h3><span id="garbage-first-g1">Garbage First (G1)</span><a href="#garbage-first-g1" class="header-anchor">#</a></h3><blockquote><p>开启选项：-XX:+UseG1GC</p><p>上面的几组垃圾收集器组合,都有几个共同的特点:</p><ul><li>年轻代,老年代是独立具有连续的内存块</li><li>年轻代收集使用单eden,双survivor进行复制算法;</li><li>老年代收集必须扫描整个老年代区域;</li><li>都是以尽可能少而快地执行GC为设计原则.</li></ul><p>G1垃圾收集器也是以关注延迟为目标、服务器端应用的垃圾收集器，被HotSpot团队寄予取代CMS的使命，也是一个非常具有调优潜力的垃圾收集器。虽然G1也有类似CMS的收集动作：初始标记、并发标记、重新标记、清除、转移回收，并且也以一个串行收集器做担保机制，但单纯地以类似前三种的过程描述显得并不是很妥当。事实上，G1收集与以上三组收集器有很大不同：</p><ul><li>G1的设计原则是”首先收集尽可能多的垃圾(Garbage - First)”。因此，G1并不会等内存耗尽(串行、并行)或者快耗尽(CMS)的时候开始垃圾收集，而是在内部- 采用了启发式算法，在老年代找出具有高收集收益的分区进行收集。同时G1可以根据用户设置的暂停时间目标自动调整年轻代和总堆大小，暂停目标越短年轻代空间越小、总空间就越大；</li><li>G1采用内存分区(Region)的思路，将内存划分为一个个相等大小的内存分区，回收时则以分区为单位进- 行回收，存活的对象复制到另一个空闲分区中。由于都是以相等大小的分区为单位进行操作，因此G1天- 然就是一种压缩方案(局部压缩)；</li><li>G1虽然也是分代收集器，但整个内存分区不存在物理上的年轻代与老年代的区别，也不需要完全独立的- survivor(to space)堆做复制准备。G1只有逻辑上的分代概念，或者说每个分区都可能随G1的运行在不- 同代之间前后切换；</li><li>G1的收集都是STW的，但年轻代和老年代的收集界限比较模糊，采用了混合(mixed)收集的方式。即每次- 收集既可能只收集年轻代分区(年轻代收集)，也可能在收集年轻代的同时，包含部分老年代分区(混合- 收集)，这样即使堆内存很大时，也可以限制收集范围，从而降低停顿</li></ul><h4><span id="g1-de-nei-cun-mo-xing">G1的内存模型</span><a href="#g1-de-nei-cun-mo-xing" class="header-anchor">#</a></h4><blockquote><p>分区概念: </p><p><img src="https://gitee.com/fa20091001/blogImage/raw/master/picGoimg/image-20201209171626803.png" alt="image-20201209171626803"></p><h3><span id="fen-qu">分区</span><a href="#fen-qu" class="header-anchor">#</a></h3><p>分区 Region</p><p>G1采用了分区(Region)的思路，将整个堆空间分成若干个大小相等的内存区域，每次分配对象空间将逐段地使用内存。因此，在堆的使用上，G1并不要求对象的存储一定是物理上连续的，只要逻辑上连续即可；每个分区也不会确定地为某个代服务，可以按需在年轻代和老年代之间切换。启动时可以通过参数-XX:G1HeapRegionSize=n可指定分区大小(1MB~32MB，且必须是2的幂)，默认将整堆划分为2048个分区。</p><h3><span id="qia-pian">卡片</span><a href="#qia-pian" class="header-anchor">#</a></h3><p>卡片 Card</p><p>在每个分区内部又被分成了若干个大小为512 Byte卡片(Card)，标识堆内存最小可用粒度所有分区的卡片将会记录在全局卡片表(Global Card Table)中，分配的对象会占用物理上连续的若干个卡片，当查找对分区内对象的引用时便可通过记录卡片来查找该引用对象(见RSet)。每次对内存的回收，都是对指定分区的卡片进行处理。</p><h3><span id="dui">堆</span><a href="#dui" class="header-anchor">#</a></h3><p>堆 Heap</p><p>G1同样可以通过-Xms/-Xmx来指定堆空间大小。当发生年轻代收集或混合收集时，通过计算GC与应用的耗费时间比，自动调整堆空间大小。如果GC频率太高，则通过增加堆尺寸，来减少GC频率，相应地GC占用的时间也随之降低；目标参数-XX:GCTimeRatio即为GC与应用的耗费时间比，G1默认为9，而CMS默认为99，因为CMS的设计原则是耗费在GC上的时间尽可能的少。另外，当空间不足，如对象空间分配或转移失败时，G1会首先尝试增加堆空间，如果扩容失败，则发起担保的Full GC。Full GC后，堆尺寸计算结果也会调整堆空间。</p><h2><span id="fen-dai-mo-xing">分代模型</span><a href="#fen-dai-mo-xing" class="header-anchor">#</a></h2><p><img src="https://gitee.com/fa20091001/blogImage/raw/master/picGoimg/v2-2e01e8281ddd51fb2820ba8e97c96933_720w.jpg" alt="img"></p><h3><span id="fen-dai">分代</span><a href="#fen-dai" class="header-anchor">#</a></h3><p>分代 Generation</p><p>分代垃圾收集可以将关注点集中在最近被分配的对象上，而无需整堆扫描，避免长命对象的拷贝，同时独立收集有助于降低响应时间。虽然分区使得内存分配不再要求紧凑的内存空间，但G1依然使用了分代的思想。与其他垃圾收集器类似，G1将内存在逻辑上划分为年轻代和老年代，其中年轻代又划分为Eden空间和Survivor空间。但年轻代空间并不是固定不变的，当现有年轻代分区占满时，JVM会分配新的空闲分区加入到年轻代空间。</p><p>整个年轻代内存会在初始空间-XX:G1NewSizePercent(默认整堆5%)与最大空间-XX:G1MaxNewSizePercent(默认60%)之间动态变化，且由参数目标暂停时间-XX:MaxGCPauseMillis(默认200ms)、需要扩缩容的大小以及分区的已记忆集合(RSet)计算得到。当然，G1依然可以设置固定的年轻代大小(参数-XX:NewRatio、-Xmn)，但同时暂停目标将失去意义。</p><p>总的来说，G1可以做到让你自己来设置垃圾回收对系统的影响，他自己通过把内存拆分为大量小Region,以及追踪每个Region中可以回收的对象大小和预估时间，最后在垃圾回收的时候，尽量把垃圾回收对系统造成的影响控制在你指定的范围内，同时在有限的时间内尽量回收可能多的垃圾对象</p><p>其实这个也就是一个简单的的介绍，下面一篇我们来说说线上环境G1垃圾回收器的具体使用和参数设置</p></blockquote></blockquote></blockquote>]]></content>
<tags>
<tag> JVM </tag>
</tags>
</entry>
<entry>
<title>MySQL综合练习</title>
<link href="2020/12/31/mysql-zong-he-lian-xi/"/>
<url>2020/12/31/mysql-zong-he-lian-xi/</url>
<content type="html"><![CDATA[<h4><span id="yi-jian-biao-yu-ju">一. 建表语句:</span><a href="#yi-jian-biao-yu-ju" class="header-anchor">#</a></h4><blockquote><pre class=" language-sql"><code class="language-sql"><span class="token comment" spellcheck="true">-- 创建部门表</span><span class="token keyword">DROP</span> <span class="token keyword">TABLE</span> <span class="token keyword">IF</span> <span class="token keyword">EXISTS</span> dept<span class="token punctuation">;</span> <span class="token keyword">CREATE</span> <span class="token keyword">TABLE</span> dept<span class="token punctuation">(</span> <span class="token comment" spellcheck="true">-- 部门编号</span> deptno <span class="token keyword">int</span> <span class="token keyword">PRIMARY</span> <span class="token keyword">KEY</span><span class="token punctuation">,</span> <span class="token comment" spellcheck="true">-- 部门名称</span> dname <span class="token keyword">VARCHAR</span><span class="token punctuation">(</span><span class="token number">14</span><span class="token punctuation">)</span><span class="token punctuation">,</span> <span class="token comment" spellcheck="true">-- 部门所在地</span> loc <span class="token keyword">VARCHAR</span><span class="token punctuation">(</span><span class="token number">13</span><span class="token punctuation">)</span> <span class="token punctuation">)</span><span class="token punctuation">;</span><span class="token comment" spellcheck="true">-- 向部门表插入数据</span><span class="token keyword">INSERT</span> <span class="token keyword">INTO</span> dept <span class="token keyword">VALUES</span> <span class="token punctuation">(</span><span class="token number">10</span><span class="token punctuation">,</span><span class="token string">'ACCOUNTING'</span><span class="token punctuation">,</span><span class="token string">'NEW YORK'</span><span class="token punctuation">)</span><span class="token punctuation">;</span><span class="token keyword">INSERT</span> <span class="token keyword">INTO</span> dept <span class="token keyword">VALUES</span> <span class="token punctuation">(</span><span class="token number">20</span><span class="token punctuation">,</span><span class="token string">'RESEARCH'</span><span class="token punctuation">,</span><span class="token string">'DALLAS'</span><span class="token punctuation">)</span><span class="token punctuation">;</span><span class="token keyword">INSERT</span> <span class="token keyword">INTO</span> dept <span class="token keyword">VALUES</span> <span class="token punctuation">(</span><span class="token number">30</span><span class="token punctuation">,</span><span class="token string">'SALES'</span><span class="token punctuation">,</span><span class="token string">'CHICAGO'</span><span class="token punctuation">)</span><span class="token punctuation">;</span><span class="token keyword">INSERT</span> <span class="token keyword">INTO</span> dept <span class="token keyword">VALUES</span> <span class="token punctuation">(</span><span class="token number">40</span><span class="token punctuation">,</span><span class="token string">'OPERATIONS'</span><span class="token punctuation">,</span><span class="token string">'BOSTON'</span><span class="token punctuation">)</span><span class="token punctuation">;</span><span class="token comment" spellcheck="true">-- 创建员工表</span><span class="token keyword">DROP</span> <span class="token keyword">TABLE</span> <span class="token keyword">IF</span> <span class="token keyword">EXISTS</span> emp<span class="token punctuation">;</span><span class="token keyword">CREATE</span> <span class="token keyword">TABLE</span> emp<span class="token punctuation">(</span> <span class="token comment" spellcheck="true">-- 员工编号</span> empno <span class="token keyword">int</span> <span class="token keyword">PRIMARY</span> <span class="token keyword">KEY</span><span class="token punctuation">,</span> <span class="token comment" spellcheck="true">-- 员工姓名</span> ename <span class="token keyword">VARCHAR</span><span class="token punctuation">(</span><span class="token number">10</span><span class="token punctuation">)</span><span class="token punctuation">,</span> <span class="token comment" spellcheck="true">-- 工作岗位</span> job <span class="token keyword">VARCHAR</span><span class="token punctuation">(</span><span class="token number">9</span><span class="token punctuation">)</span><span class="token punctuation">,</span> <span class="token comment" spellcheck="true">-- 直属领导</span> mgr <span class="token keyword">int</span><span class="token punctuation">,</span> <span class="token comment" spellcheck="true">-- 入职时间</span> hiredate <span class="token keyword">DATE</span><span class="token punctuation">,</span> <span class="token comment" spellcheck="true">-- 工资</span> sal <span class="token keyword">double</span><span class="token punctuation">,</span> <span class="token comment" spellcheck="true">-- 奖金</span> comm <span class="token keyword">double</span><span class="token punctuation">,</span> <span class="token comment" spellcheck="true">-- 所属部门</span> deptno <span class="token keyword">int</span><span class="token punctuation">)</span><span class="token punctuation">;</span><span class="token comment" spellcheck="true">-- 为员工表表添加外键约束</span><span class="token keyword">ALTER</span> <span class="token keyword">TABLE</span> emp <span class="token keyword">ADD</span> <span class="token keyword">CONSTRAINT</span> <span class="token keyword">FOREIGN</span> <span class="token keyword">KEY</span> EMP<span class="token punctuation">(</span>deptno<span class="token punctuation">)</span> <span class="token keyword">REFERENCES</span> dept <span class="token punctuation">(</span>deptno<span class="token punctuation">)</span><span class="token punctuation">;</span><span class="token comment" spellcheck="true">-- 向员工表插入数据</span><span class="token keyword">INSERT</span> <span class="token keyword">INTO</span> emp <span class="token keyword">VALUES</span><span class="token punctuation">(</span><span class="token number">7369</span><span class="token punctuation">,</span><span class="token string">'SMITH'</span><span class="token punctuation">,</span><span class="token string">'CLERK'</span><span class="token punctuation">,</span><span class="token number">7902</span><span class="token punctuation">,</span><span class="token string">"1980-12-17"</span><span class="token punctuation">,</span><span class="token number">800</span><span class="token punctuation">,</span><span class="token boolean">NULL</span><span class="token punctuation">,</span><span class="token number">20</span><span class="token punctuation">)</span><span class="token punctuation">;</span><span class="token keyword">INSERT</span> <span class="token keyword">INTO</span> emp <span class="token keyword">VALUES</span><span class="token punctuation">(</span><span class="token number">7499</span><span class="token punctuation">,</span><span class="token string">'ALLEN'</span><span class="token punctuation">,</span><span class="token string">'SALESMAN'</span><span class="token punctuation">,</span><span class="token number">7698</span><span class="token punctuation">,</span><span class="token string">'1981-02-20'</span><span class="token punctuation">,</span><span class="token number">1600</span><span class="token punctuation">,</span><span class="token number">300</span><span class="token punctuation">,</span><span class="token number">30</span><span class="token punctuation">)</span><span class="token punctuation">;</span><span class="token keyword">INSERT</span> <span class="token keyword">INTO</span> emp <span class="token keyword">VALUES</span><span class="token punctuation">(</span><span class="token number">7521</span><span class="token punctuation">,</span><span class="token string">'WARD'</span><span class="token punctuation">,</span><span class="token string">'SALESMAN'</span><span class="token punctuation">,</span><span class="token number">7698</span><span class="token punctuation">,</span><span class="token string">'1981-02-22'</span><span class="token punctuation">,</span><span class="token number">1250</span><span class="token punctuation">,</span><span class="token number">500</span><span class="token punctuation">,</span><span class="token number">30</span><span class="token punctuation">)</span><span class="token punctuation">;</span><span class="token keyword">INSERT</span> <span class="token keyword">INTO</span> emp <span class="token keyword">VALUES</span><span class="token punctuation">(</span><span class="token number">7566</span><span class="token punctuation">,</span><span class="token string">'JONES'</span><span class="token punctuation">,</span><span class="token string">'MANAGER'</span><span class="token punctuation">,</span><span class="token number">7839</span><span class="token punctuation">,</span><span class="token string">'1981-04-02'</span><span class="token punctuation">,</span><span class="token number">2975</span><span class="token punctuation">,</span><span class="token boolean">NULL</span><span class="token punctuation">,</span><span class="token number">20</span><span class="token punctuation">)</span><span class="token punctuation">;</span><span class="token keyword">INSERT</span> <span class="token keyword">INTO</span> emp <span class="token keyword">VALUES</span><span class="token punctuation">(</span><span class="token number">7654</span><span class="token punctuation">,</span><span class="token string">'MARTIN'</span><span class="token punctuation">,</span><span class="token string">'SALESMAN'</span><span class="token punctuation">,</span><span class="token number">7698</span><span class="token punctuation">,</span><span class="token string">'1981-09-28'</span><span class="token punctuation">,</span><span class="token number">1250</span><span class="token punctuation">,</span><span class="token number">1400</span><span class="token punctuation">,</span><span class="token number">30</span><span class="token punctuation">)</span><span class="token punctuation">;</span><span class="token keyword">INSERT</span> <span class="token keyword">INTO</span> emp <span class="token keyword">VALUES</span><span class="token punctuation">(</span><span class="token number">7698</span><span class="token punctuation">,</span><span class="token string">'BLAKE'</span><span class="token punctuation">,</span><span class="token string">'MANAGER'</span><span class="token punctuation">,</span><span class="token number">7839</span><span class="token punctuation">,</span><span class="token string">'1981-05-01'</span><span class="token punctuation">,</span><span class="token number">2850</span><span class="token punctuation">,</span><span class="token boolean">NULL</span><span class="token punctuation">,</span><span class="token number">30</span><span class="token punctuation">)</span><span class="token punctuation">;</span><span class="token keyword">INSERT</span> <span class="token keyword">INTO</span> emp <span class="token keyword">VALUES</span><span class="token punctuation">(</span><span class="token number">7782</span><span class="token punctuation">,</span><span class="token string">'CLARK'</span><span class="token punctuation">,</span><span class="token string">'MANAGER'</span><span class="token punctuation">,</span><span class="token number">7839</span><span class="token punctuation">,</span><span class="token string">'1981-06-09'</span><span class="token punctuation">,</span><span class="token number">2450</span><span class="token punctuation">,</span><span class="token boolean">NULL</span><span class="token punctuation">,</span><span class="token number">10</span><span class="token punctuation">)</span><span class="token punctuation">;</span><span class="token keyword">INSERT</span> <span class="token keyword">INTO</span> emp <span class="token keyword">VALUES</span><span class="token punctuation">(</span><span class="token number">7788</span><span class="token punctuation">,</span><span class="token string">'SCOTT'</span><span class="token punctuation">,</span><span class="token string">'ANALYST'</span><span class="token punctuation">,</span><span class="token number">7566</span><span class="token punctuation">,</span><span class="token string">'1987-07-03'</span><span class="token punctuation">,</span><span class="token number">3000</span><span class="token punctuation">,</span><span class="token boolean">NULL</span><span class="token punctuation">,</span><span class="token number">20</span><span class="token punctuation">)</span><span class="token punctuation">;</span><span class="token keyword">INSERT</span> <span class="token keyword">INTO</span> emp <span class="token keyword">VALUES</span><span class="token punctuation">(</span><span class="token number">7839</span><span class="token punctuation">,</span><span class="token string">'KING'</span><span class="token punctuation">,</span><span class="token string">'PRESIDENT'</span><span class="token punctuation">,</span><span class="token boolean">NULL</span><span class="token punctuation">,</span><span class="token string">'1981-11-17'</span><span class="token punctuation">,</span><span class="token number">5000</span><span class="token punctuation">,</span><span class="token boolean">NULL</span><span class="token punctuation">,</span><span class="token number">10</span><span class="token punctuation">)</span><span class="token punctuation">;</span><span class="token keyword">INSERT</span> <span class="token keyword">INTO</span> emp <span class="token keyword">VALUES</span><span class="token punctuation">(</span><span class="token number">7844</span><span class="token punctuation">,</span><span class="token string">'TURNER'</span><span class="token punctuation">,</span><span class="token string">'SALESMAN'</span><span class="token punctuation">,</span><span class="token number">7698</span><span class="token punctuation">,</span><span class="token string">'1981-09-08'</span><span class="token punctuation">,</span><span class="token number">1500</span><span class="token punctuation">,</span><span class="token number">0</span><span class="token punctuation">,</span><span class="token number">30</span><span class="token punctuation">)</span><span class="token punctuation">;</span><span class="token keyword">INSERT</span> <span class="token keyword">INTO</span> emp <span class="token keyword">VALUES</span><span class="token punctuation">(</span><span class="token number">7876</span><span class="token punctuation">,</span><span class="token string">'ADAMS'</span><span class="token punctuation">,</span><span class="token string">'CLERK'</span><span class="token punctuation">,</span><span class="token number">7788</span><span class="token punctuation">,</span><span class="token string">'1987-07-13'</span><span class="token punctuation">,</span><span class="token number">1100</span><span class="token punctuation">,</span><span class="token boolean">NULL</span><span class="token punctuation">,</span><span class="token number">20</span><span class="token punctuation">)</span><span class="token punctuation">;</span><span class="token keyword">INSERT</span> <span class="token keyword">INTO</span> emp <span class="token keyword">VALUES</span><span class="token punctuation">(</span><span class="token number">7900</span><span class="token punctuation">,</span><span class="token string">'JAMES'</span><span class="token punctuation">,</span><span class="token string">'CLERK'</span><span class="token punctuation">,</span><span class="token number">7698</span><span class="token punctuation">,</span><span class="token string">'1981-12-03'</span><span class="token punctuation">,</span><span class="token number">950</span><span class="token punctuation">,</span><span class="token boolean">NULL</span><span class="token punctuation">,</span><span class="token number">30</span><span class="token punctuation">)</span><span class="token punctuation">;</span><span class="token keyword">INSERT</span> <span class="token keyword">INTO</span> emp <span class="token keyword">VALUES</span><span class="token punctuation">(</span><span class="token number">7902</span><span class="token punctuation">,</span><span class="token string">'FORD'</span><span class="token punctuation">,</span><span class="token string">'ANALYST'</span><span class="token punctuation">,</span><span class="token number">7566</span><span class="token punctuation">,</span><span class="token string">'1981-12-03'</span><span class="token punctuation">,</span><span class="token number">3000</span><span class="token punctuation">,</span><span class="token boolean">NULL</span><span class="token punctuation">,</span><span class="token number">20</span><span class="token punctuation">)</span><span class="token punctuation">;</span><span class="token keyword">INSERT</span> <span class="token keyword">INTO</span> emp <span class="token keyword">VALUES</span><span class="token punctuation">(</span><span class="token number">7934</span><span class="token punctuation">,</span><span class="token string">'MILLER'</span><span class="token punctuation">,</span><span class="token string">'CLERK'</span><span class="token punctuation">,</span><span class="token number">7782</span><span class="token punctuation">,</span><span class="token string">'1981-01-23'</span><span class="token punctuation">,</span><span class="token number">1300</span><span class="token punctuation">,</span><span class="token boolean">NULL</span><span class="token punctuation">,</span><span class="token number">10</span><span class="token punctuation">)</span><span class="token punctuation">;</span><span class="token comment" spellcheck="true">-- 创建工资等级表</span><span class="token keyword">DROP</span> <span class="token keyword">TABLE</span> <span class="token keyword">IF</span> <span class="token keyword">EXISTS</span> salgrade<span class="token punctuation">;</span><span class="token keyword">CREATE</span> <span class="token keyword">TABLE</span> salgrade<span class="token punctuation">(</span> <span class="token comment" spellcheck="true">-- 等级</span> grade <span class="token keyword">int</span><span class="token punctuation">,</span> <span class="token comment" spellcheck="true">-- 最低工资</span> losal <span class="token keyword">double</span><span class="token punctuation">,</span> <span class="token comment" spellcheck="true">-- 最高工资</span> hisal <span class="token keyword">double</span><span class="token punctuation">)</span><span class="token punctuation">;</span><span class="token comment" spellcheck="true">-- 向工资等级表插入数据</span><span class="token keyword">INSERT</span> <span class="token keyword">INTO</span> salgrade <span class="token keyword">VALUES</span> <span class="token punctuation">(</span><span class="token number">1</span><span class="token punctuation">,</span><span class="token number">700</span><span class="token punctuation">,</span><span class="token number">1200</span><span class="token punctuation">)</span><span class="token punctuation">;</span><span class="token keyword">INSERT</span> <span class="token keyword">INTO</span> salgrade <span class="token keyword">VALUES</span> <span class="token punctuation">(</span><span class="token number">2</span><span class="token punctuation">,</span><span class="token number">1201</span><span class="token punctuation">,</span><span class="token number">1400</span><span class="token punctuation">)</span><span class="token punctuation">;</span><span class="token keyword">INSERT</span> <span class="token keyword">INTO</span> salgrade <span class="token keyword">VALUES</span> <span class="token punctuation">(</span><span class="token number">3</span><span class="token punctuation">,</span><span class="token number">1401</span><span class="token punctuation">,</span><span class="token number">2000</span><span class="token punctuation">)</span><span class="token punctuation">;</span><span class="token keyword">INSERT</span> <span class="token keyword">INTO</span> salgrade <span class="token keyword">VALUES</span> <span class="token punctuation">(</span><span class="token number">4</span><span class="token punctuation">,</span><span class="token number">2001</span><span class="token punctuation">,</span><span class="token number">3000</span><span class="token punctuation">)</span><span class="token punctuation">;</span><span class="token keyword">INSERT</span> <span class="token keyword">INTO</span> salgrade <span class="token keyword">VALUES</span> <span class="token punctuation">(</span><span class="token number">5</span><span class="token punctuation">,</span><span class="token number">3001</span><span class="token punctuation">,</span><span class="token number">9999</span><span class="token punctuation">)</span><span class="token punctuation">;</span></code></pre></blockquote><h4><span id="er-cha-xun-yu-ju">二. 查询语句:</span><a href="#er-cha-xun-yu-ju" class="header-anchor">#</a></h4><blockquote><ol><li><p>查询部门编号为30的部门的员工详细信息</p><pre class=" language-SQL"><code class="language-SQL">SELECT * FROM emp WHERE deptno = 30;</code></pre></li><li><p>查询从事clerk工作的员工的编号、姓名以及其部门号</p><pre class=" language-SQL"><code class="language-SQL">SELECT empno,ename,deptno FROM emp WHERE job = ‘clerk’;</code></pre></li><li><p>查询奖金多于基本工资的员工的信息</p><pre class=" language-SQL"><code class="language-SQL">SELECT * FROM emp WHERE comm > sal;</code></pre></li><li><p>查询奖金多于基本工资60%的员工的信息</p><pre class=" language-SQL"><code class="language-SQL">SELECT * FROM emp WHERE comm > sal * 0.6;</code></pre></li><li><p>查询部门编号为10的部门经理和部门编号为20的部门中工作为CLERK的职员信息</p><pre class=" language-SQL"><code class="language-SQL">SELECT * FROM emp WHERE deptno = 10 AND job=‘MANAGER’ OR deptno = 20 AND job = ‘CLERK’;</code></pre></li><li><p>查询部门编号为10的部门经理或部门编号为20的部门工作为CLERK的职员信息或者既不是经理也不是CLERK但是工资高于2000的员工信息</p><pre class=" language-SQL"><code class="language-SQL"> SELECT * FROM emp WHERE deptno = 10 AND job=‘MANAGER’ OR deptno = 20 AND job = ‘CLERK’ OR job NOT IN (‘MANAGER’,‘CLERK’) AND sal > 2000 ;</code></pre></li><li><p>查询获得奖金的员工的信息</p><pre class=" language-SQL"><code class="language-SQL">SELECT * FROM emp WHERE comm > 0 and comm is not null;</code></pre></li><li><p>查询奖金少于100或者没有获得奖金的员工的信息</p><pre class=" language-SQL"><code class="language-SQL">SELECT * FROM emp WHERE comm < 100 OR comm IS NULL;</code></pre></li><li><p>查询姓名以A、B、S开头的员工的信息</p><pre class=" language-SQL"><code class="language-SQL">SELECT * FROM emp WHERE ename LIKE ‘A%’ OR ename LIKE ‘B%’ OR ename LIKE ‘S%’;</code></pre></li><li><p>查询找到姓名长度为6个字符的员工的信息</p><pre class=" language-mysql"><code class="language-mysql">SELECT * FROM emp WHERE LENGTH(ename) = 6;</code></pre></li><li><p>查询姓名中不包含R字符的员工信息。</p><pre class=" language-sql"><code class="language-sql"><span class="token keyword">SELECT</span> <span class="token operator">*</span> <span class="token keyword">FROM</span> emp <span class="token keyword">WHERE</span> ename <span class="token operator">NOT</span> <span class="token operator">LIKE</span> ‘<span class="token operator">%</span>R<span class="token operator">%</span>’<span class="token punctuation">;</span></code></pre></li><li><p>查询员工的详细信息并按姓名排序</p><pre class=" language-sql"><code class="language-sql"><span class="token keyword">SELECT</span> <span class="token operator">*</span> <span class="token keyword">FROM</span> emp <span class="token keyword">ORDER</span> <span class="token keyword">BY</span> ename <span class="token keyword">ASC</span><span class="token punctuation">;</span></code></pre></li><li><p>查询员工的信息并按工作降序工资升序排列</p><pre class=" language-sql"><code class="language-sql"><span class="token keyword">SELECT</span> <span class="token operator">*</span> <span class="token keyword">FROM</span> emp <span class="token keyword">ORDER</span> <span class="token keyword">BY</span> job <span class="token keyword">DESC</span> <span class="token punctuation">,</span> sal <span class="token keyword">ASC</span><span class="token punctuation">;</span></code></pre></li><li><p>计算员工的日薪(按30天计)</p><pre class=" language-sql"><code class="language-sql"><span class="token keyword">SELECT</span> ename<span class="token punctuation">,</span>sal<span class="token operator">/</span><span class="token number">30</span> <span class="token punctuation">`</span><span class="token keyword">AS</span> <span class="token string">'日薪'</span><span class="token punctuation">`</span> <span class="token keyword">FROM</span> emp<span class="token punctuation">;</span></code></pre></li><li><p>查询姓名中包含字符A的员工的信息</p><pre class=" language-sql"><code class="language-sql"><span class="token keyword">SELECT</span> <span class="token operator">*</span> <span class="token keyword">FROM</span> emp <span class="token keyword">WHERE</span> ename <span class="token operator">LIKE</span> ‘’<span class="token operator">%</span>A<span class="token operator">%</span>’<span class="token punctuation">;</span></code></pre></li><li><p>查询拥有员工的部门的部门名和部门号</p><pre class=" language-SQL"><code class="language-SQL">SELECT DISTINCT d.dname, d.deptno FROM dept d,emp e WHERE d.deptno = e.deptno;Select DISTINCT d.deptno,d.dname from emp e, dept d where e.JOB IS NOTNULL;</code></pre></li><li><p>查询工资多于smith的员工信息。</p><pre class=" language-SQL"><code class="language-SQL">SELECT *FROM emp WHERE sal > (SELECT sal FROM emp WHERE ename = ‘smith’);</code></pre></li></ol></blockquote>]]></content>
<tags>
<tag> MySQL </tag>
</tags>
</entry>
<entry>
<title>Redis缓存解决办法</title>
<link href="2020/12/29/redis-huan-cun-shi-xiao/"/>
<url>2020/12/29/redis-huan-cun-shi-xiao/</url>
<content type="html"><![CDATA[<h3><span id="yi-huan-cun-chu-li-liu-cheng">一 缓存处理流程</span><a href="#yi-huan-cun-chu-li-liu-cheng" class="header-anchor">#</a></h3><blockquote><p>前台请求,后台先从缓存中取数据,取到直接返回结果,取不到时从数据库中区,数据库取到更新缓存,并返回结果,数据库也没有取到数据,那直接返回空结果.</p></blockquote><p> <img src="https://gitee.com/fa20091001/blogImage/raw/master/picGoimg/20180919143214712" alt="img"></p><h3><span id="er-huan-cun-chuan-tou">二 缓存穿透</span><a href="#er-huan-cun-chuan-tou" class="header-anchor">#</a></h3><h5><span id="miao-shu">描述:</span><a href="#miao-shu" class="header-anchor">#</a></h5><p>缓存穿透是指缓存和数据库中都没有的数据,而用户不断发起请求,如发起为id为”-1”的数据或者id为特别大不存在的数据.这时的用户很可能是攻击者,攻击会导致数据库压力过大.</p><h5><span id="jie-jue-fang-an">解决方案:</span><a href="#jie-jue-fang-an" class="header-anchor">#</a></h5><ol><li>接口层增加校验,如用户鉴权,id做基础校验,id<=0的直接拦截</li><li>从缓存取不到的数据,在数据库中也没有取到,这时也可一件key-value对写为key-null,缓存有效时间可以设置短点,如30秒(设置太长会导致正常情况也没法使用).这样可以防止攻击用户反复用同一个id暴力攻击</li></ol><h3><span id="san-huan-cun-ji-chuan">三 缓存击穿:</span><a href="#san-huan-cun-ji-chuan" class="header-anchor">#</a></h3><h5><span id="miao-shu">描述:</span><a href="#miao-shu" class="header-anchor">#</a></h5><ol><li>缓存击穿是指缓存中没有数据,但是数据库中有的数据(一般指缓存时间到期),这是由于并发用户特别多,同时读缓存没有读到数据,有同时去数据库中去数据,引起数据库压力瞬间增大,造成过大压力</li></ol><h5><span id="jie-jue-fang-an">解决方案:</span><a href="#jie-jue-fang-an" class="header-anchor">#</a></h5><ol><li><p>设置热点数据永不过期</p></li><li><p>加互斥锁,互斥锁如下</p></li></ol><h6><span id="shuo-ming">说明:</span><a href="#shuo-ming" class="header-anchor">#</a></h6><ol><li>缓存中有数据,直接走上述代码13行后返回结果</li><li>缓存中没有数据,第一个进入的线程,获取锁并从数据库去取数据,没释放锁之前,其它并行进入的线程会等待100ms,在重新去缓存取数据,这样防止都去数据库重复取数据,重复向缓存中更新数据情况出现.</li><li>这是简化处理,理论上如果能根据key值加锁就更好,就是从线程A 从数据库取key1的数据并不妨碍线程B取key2的数据,上面明显不足</li></ol>]]></content>
<tags>
<tag> Redis </tag>
</tags>
</entry>
<entry>
<title>ES的多种检索方式</title>
<link href="2020/12/29/es-de-duo-chong-jian-suo-fang-shi/"/>
<url>2020/12/29/es-de-duo-chong-jian-suo-fang-shi/</url>
<content type="html"><![CDATA[<p><strong>## ES的多种检索方式</strong></p><h4><span id="1-cha-xun-quan-bu">1.查询全部：</span><a href="#1-cha-xun-quan-bu" class="header-anchor">#</a></h4><p> GET /ecommerce/product/_search</p><p> took:1 #耗费了多少毫秒</p><p> time_out:false #是否超时</p><pre class=" language-json"><code class="language-json"> <span class="token property">"_shards"</span> <span class="token operator">:</span> <span class="token punctuation">{</span> <span class="token property">"total"</span> <span class="token operator">:</span> <span class="token number">1</span><span class="token punctuation">,</span> <span class="token property">"successful"</span> <span class="token operator">:</span> <span class="token number">1</span><span class="token punctuation">,</span> <span class="token property">"skipped"</span> <span class="token operator">:</span> <span class="token number">0</span><span class="token punctuation">,</span> <span class="token property">"failed"</span> <span class="token operator">:</span> <span class="token number">0</span> <span class="token punctuation">}</span></code></pre><p> “hits.total”:6 查询结果的数量</p><p> “max_score” : 1.0 score的含义，就是document对于一个search的相关度的匹配分数，越相关，就越匹配，分数越高</p><p> “hits.hits”:包含了匹配搜索的document的详细数据</p><pre><code>GET /ecommerce/product/_search?q=name:yagao&sort=price:desc</code></pre><p><font color="#ABCDE">—— 生产中并不常用 —–</font></p><p><strong>### DSL（Domain Specified Language）：特定领域的语言</strong></p><p> 1.查询所有的商品</p><pre><code> GET /ecommerce/product/_search</code></pre><pre><code>{ "query":{ "match_all":{} } }</code></pre><p> 2.降序排序</p><pre><code>GET /ecommerce/product/_search</code></pre><pre class=" language-json"><code class="language-json"><span class="token punctuation">{</span> <span class="token property">"query"</span><span class="token operator">:</span><span class="token punctuation">{</span> <span class="token property">"match"</span><span class="token operator">:</span><span class="token punctuation">{</span> <span class="token property">"name"</span><span class="token operator">:</span><span class="token string">"yagao"</span> <span class="token punctuation">}</span> <span class="token punctuation">}</span><span class="token punctuation">,</span> <span class="token property">"sort"</span><span class="token operator">:</span><span class="token punctuation">[</span> <span class="token punctuation">{</span><span class="token property">"price"</span><span class="token operator">:</span><span class="token string">"desc"</span><span class="token punctuation">}</span> <span class="token punctuation">]</span> <span class="token punctuation">}</span></code></pre><p> 3.分页查询</p><pre><code>GET /ecommerce/product/_search</code></pre><pre class=" language-json"><code class="language-json"><span class="token punctuation">{</span> <span class="token property">"query"</span><span class="token operator">:</span><span class="token punctuation">{</span> <span class="token property">"match_all"</span><span class="token operator">:</span><span class="token punctuation">{</span><span class="token punctuation">}</span> <span class="token punctuation">}</span><span class="token punctuation">,</span> <span class="token property">"from"</span><span class="token operator">:</span><span class="token number">1</span><span class="token punctuation">,</span> <span class="token property">"size"</span><span class="token operator">:</span><span class="token number">2</span> <span class="token punctuation">}</span></code></pre><p> 4.指定查询输出的字段</p><pre><code> GET /ecommerce/product/_search</code></pre><pre class=" language-json"><code class="language-json"> <span class="token punctuation">{</span> <span class="token property">"query"</span><span class="token operator">:</span><span class="token punctuation">{</span> <span class="token property">"match_all"</span><span class="token operator">:</span><span class="token punctuation">{</span><span class="token punctuation">}</span> <span class="token punctuation">}</span><span class="token punctuation">,</span> <span class="token property">"_source"</span><span class="token operator">:</span><span class="token punctuation">[</span><span class="token string">"name"</span><span class="token punctuation">,</span><span class="token string">"price"</span><span class="token punctuation">]</span> <span class="token punctuation">}</span> <span class="token punctuation">}</span></code></pre><p> 5.对数据进行过滤</p><pre><code>GET /ecommerce/product/_search</code></pre><pre class=" language-json"><code class="language-json"> <span class="token punctuation">{</span> <span class="token property">"query"</span><span class="token operator">:</span><span class="token punctuation">{</span> <span class="token property">"bool"</span><span class="token operator">:</span><span class="token punctuation">{</span> <span class="token property">"must"</span><span class="token operator">:</span><span class="token punctuation">{</span> <span class="token property">"match"</span><span class="token operator">:</span><span class="token punctuation">{</span> <span class="token property">"name"</span><span class="token operator">:</span><span class="token string">"yagao"</span> <span class="token punctuation">}</span> <span class="token punctuation">}</span><span class="token punctuation">,</span> <span class="token property">"filter"</span><span class="token operator">:</span><span class="token punctuation">{</span> <span class="token property">"range"</span><span class="token operator">:</span><span class="token punctuation">{</span> <span class="token property">"price"</span><span class="token operator">:</span><span class="token punctuation">{</span> <span class="token property">"gt"</span><span class="token operator">:</span><span class="token number">11</span> <span class="token punctuation">}</span> <span class="token punctuation">}</span> <span class="token punctuation">}</span> <span class="token punctuation">}</span> <span class="token punctuation">}</span> <span class="token punctuation">}</span> </code></pre><p> 6.全文检索</p><pre><code> GET /ecommerce/product/_search</code></pre><pre class=" language-json"><code class="language-json"><span class="token punctuation">{</span> <span class="token property">"query"</span><span class="token operator">:</span><span class="token punctuation">{</span> <span class="token property">"match"</span><span class="token operator">:</span><span class="token punctuation">{</span> <span class="token property">"producer"</span><span class="token operator">:</span><span class="token string">"yagao producer"</span> <span class="token punctuation">}</span> <span class="token punctuation">}</span> <span class="token punctuation">}</span></code></pre><p> 7.短语搜索</p><pre class=" language-json"><code class="language-json">GET /ecommerce/product/_search</code></pre><pre class=" language-json"><code class="language-json"><span class="token punctuation">{</span> <span class="token property">"query"</span><span class="token operator">:</span><span class="token punctuation">{</span> <span class="token property">"match_phrase"</span><span class="token operator">:</span><span class="token punctuation">{</span> <span class="token property">"producer"</span><span class="token operator">:</span><span class="token string">"yaogao producer"</span> <span class="token punctuation">}</span> <span class="token punctuation">}</span> <span class="token punctuation">}</span></code></pre><p> 8.高亮</p><pre class=" language-json"><code class="language-json">GET /ecommerce/product/_search</code></pre><pre class=" language-json"><code class="language-json"> <span class="token punctuation">{</span> <span class="token property">"query"</span><span class="token operator">:</span><span class="token punctuation">{</span> <span class="token property">"match_phrase"</span><span class="token operator">:</span><span class="token punctuation">{</span> <span class="token property">"producer"</span><span class="token operator">:</span><span class="token string">"yaogao producer"</span> <span class="token punctuation">}</span> <span class="token punctuation">}</span><span class="token punctuation">,</span> <span class="token property">"highlight"</span><span class="token operator">:</span><span class="token punctuation">{</span> <span class="token property">"fields"</span><span class="token operator">:</span><span class="token punctuation">{</span> <span class="token property">"producer"</span><span class="token operator">:</span><span class="token punctuation">{</span><span class="token punctuation">}</span> <span class="token punctuation">}</span> <span class="token punctuation">}</span> <span class="token punctuation">}</span></code></pre>]]></content>
<tags>
<tag> Elasticsearch </tag>
</tags>
</entry>
<entry>
<title>Redis笔记(布隆,缓存,持久化)</title>
<link href="2020/12/29/redis-bi-ji-bu-long-huan-cun-chi-jiu-hua/"/>
<url>2020/12/29/redis-bi-ji-bu-long-huan-cun-chi-jiu-hua/</url>
<content type="html"><![CDATA[<h2><span id="redis-bi-ji-bu-long-huan-cun-chi-jiu-hua">Redis笔记(布隆,缓存,持久化)</span><a href="#redis-bi-ji-bu-long-huan-cun-chi-jiu-hua" class="header-anchor">#</a></h2><blockquote><p>redis模块地址: <a href="https://redis.io/modules" target="_blank" rel="noopener">https://redis.io/modules</a></p><p>redis 中文地址: <a href="http://redis.cn/" target="_blank" rel="noopener">http://redis.cn/</a></p><p>Redis可以自定义添加模块</p><p><img src="https://gitee.com/fa20091001/blogImage/raw/master/picGoimg/image-20201229155131550.png" alt="image-20201229155131550"></p></blockquote><h3><span id="bu-long-guo-lu-qi-mo-kuai">布隆过滤器模块:</span><a href="#bu-long-guo-lu-qi-mo-kuai" class="header-anchor">#</a></h3><h4><span id="an-zhuang-bu-long-guo-lu-qi">安装布隆过滤器</span><a href="#an-zhuang-bu-long-guo-lu-qi" class="header-anchor">#</a></h4><blockquote><pre class=" language-shell"><code class="language-shell">//下载wget https://github.com/RedisBloom/RedisBloom/archive/master.zip//解压unzip master.zip//编译make //编译后生成 redisbloom.so 并cp到指定目录cp ./redisbloom.so /usr/local/redis///启动./bin/redis-server --loadmodule ./redisbloom.so</code></pre></blockquote><h5><span id="jie-jue-wen-ti-chang-jing-huan-cun-chuan-tou"><em>解决问题场景:缓存穿透</em></span><a href="#jie-jue-wen-ti-chang-jing-huan-cun-chuan-tou" class="header-anchor">#</a></h5><ol><li>如果穿透了,不存在</li><li>client增加redis中的key,value标记</li><li>数据库增加元素</li><li>完成元素对bloom的添加</li></ol><h5><span id="gai-lu-jie-jue-wen-ti-bu-ke-neng-bai-fen-zhi-bai-zheng-que">概率解决问题 不可能百分之百正确</span><a href="#gai-lu-jie-jue-wen-ti-bu-ke-neng-bai-fen-zhi-bai-zheng-que" class="header-anchor">#</a></h5><ol><li>有的向bitmap中标记</li><li>请求的可能会被误标记</li><li>一定概率会大量减少放行: 穿透</li><li>成本低</li></ol><p>总结: 布隆过滤器判断中 如果出现1 不一定是1 有可能是碰撞导致; 如果出现0 一定是0</p><blockquote><blockquote><p><img src="https://gitee.com/fa20091001/blogImage/raw/master/picGoimg/image-20201229163548134.png" alt="image-20201229163548134"></p></blockquote><p><font color="#sdfg">问题: bloom布隆过滤器; counting bloom; cukcoo 布谷鸟过滤器</font></p></blockquote><h3><span id="reidis-zuo-wei-huan-cun-de-shi-yong">Reidis作为缓存的使用:</span><a href="#reidis-zuo-wei-huan-cun-de-shi-yong" class="header-anchor">#</a></h3><h4><span id="redis-zuo-wei-shu-ju-huan-cun-de-que-bie">redis作为数据/缓存的却别:</span><a href="#redis-zuo-wei-shu-ju-huan-cun-de-que-bie" class="header-anchor">#</a></h4><ol><li>缓存数据并不重要</li><li>缓存不是全量数据</li><li>缓存应该随着访问变化</li><li>热点数据</li></ol><h4><span id="key-de-you-xian-qi">key的有限期:</span><a href="#key-de-you-xian-qi" class="header-anchor">#</a></h4><blockquote><ol><li><p>不会随着访问延长时间</p></li><li><p>发生写,会剔除过期时间</p></li><li><p>倒计时,且redis不能延长</p></li><li><p>定时 EXPIREAT</p></li><li><p>业务逻辑自己补全</p></li></ol><h5><span id="guo-qi-shi-jian-pan-ding-yuan-li">过期时间判定原理:</span><a href="#guo-qi-shi-jian-pan-ding-yuan-li" class="header-anchor">#</a></h5><ol><li><p>被动访问时判定</p></li><li><p>周期轮询判定(增量)</p><p>目的稍微牺牲下内存,但是确保了redis性能为王</p></li></ol></blockquote><h4><span id="nei-cun-da-xiao">内存大小</span><a href="#nei-cun-da-xiao" class="header-anchor">#</a></h4><ol><li><p>maxmemory</p><blockquote><p><code>maxmemory</code>配置指令用于配置Redis存储数据时指定限制的内存大小。通过redis.conf可以设置该指令，或者之后使用CONFIG SET命令来进行运行时配置。</p><p>例如为了配置内存限制为100mb，以下的指令可以放在<code>redis.conf</code>文件中。</p><pre class=" language-conf"><code class="language-conf">maxmemory 100mb</code></pre></blockquote></li><li><p>回收策略: </p><blockquote><p>当maxmemory限制达到的时候Redis会使用的行为由 Redis的maxmemory-policy配置指令来进行配置。</p><p>以下的策略是可用的:</p><ul><li><strong>noeviction</strong>:返回错误当内存限制达到并且客户端尝试执行会让更多内存被使用的命令（大部分的写入指令，但DEL和几个例外）</li><li><strong>allkeys-lru</strong>: 尝试回收最少使用的键（LRU），使得新添加的数据有空间存放。</li><li><strong>volatile-lru</strong>: 尝试回收最少使用的键（LRU），但仅限于在过期集合的键,使得新添加的数据有空间存放。</li><li><strong>allkeys-random</strong>: 回收随机的键使得新添加的数据有空间存放。</li><li><strong>volatile-random</strong>: 回收随机的键使得新添加的数据有空间存放，但仅限于在过期集合的键。</li><li><strong>volatile-ttl</strong>: 回收在过期集合的键，并且优先回收存活时间（TTL）较短的键,使得新添加的数据有空间存放。</li></ul><p>如果没有键满足回收的前提条件的话，策略<strong>volatile-lru</strong>, <strong>volatile-random</strong>以及<strong>volatile-ttl</strong>就和noeviction 差不多了。</p><p>选择正确的回收策略是非常重要的，这取决于你的应用的访问模式，不过你可以在运行时进行相关的策略调整，并且监控缓存命中率和没命中的次数，通过RedisINFO命令输出以便调优。</p><p>一般的经验规则:</p><ul><li>使用<strong>allkeys-lru</strong>策略：当你希望你的请求符合一个幂定律分布，也就是说，你希望部分的子集元素将比其它其它元素被访问的更多。如果你不确定选择什么，这是个很好的选择。.</li><li>使用<strong>allkeys-random</strong>：如果你是循环访问，所有的键被连续的扫描，或者你希望请求分布正常（所有元素被访问的概率都差不多）。</li><li>使用<strong>volatile-ttl</strong>：如果你想要通过创建缓存对象时设置TTL值，来决定哪些对象应该被过期。</li></ul><p><strong>allkeys-lru</strong> 和 <strong>volatile-random</strong>策略对于当你想要单一的实例实现缓存及持久化一些键时很有用。不过一般运行两个实例是解决这个问题的更好方法。</p><p>为了键设置过期时间也是需要消耗内存的，所以使用<strong>allkeys-lru</strong>这种策略更加高效，因为没有必要为键取设置过期时间当内存有压力时。</p></blockquote></li></ol><h3><span id="redis-chi-jiu-hua-ji-zhi">Redis持久化机制:</span><a href="#redis-chi-jiu-hua-ji-zhi" class="header-anchor">#</a></h3><blockquote><p>Redis 提供了不同级别的持久化方式:</p><ol><li>RDB持久化方式能够在指定的时间间隔能对你的数据快照存储</li><li>AOF持久化方式记录每次对服务器写的操作,当服务器重启的时候会重新执行这些命令来恢复原始的数据,AOF命令以Redis协议最佳保存每次写的操作到文件末尾,Redis还能对AOF文件进行后台重写,使得AOF文件的体积不至于过大</li><li>也可以同时开启两种持久化方式,在这种情况下,当redis重启的时候会优先载入AOF文件来恢复原始的数据,因为在通常情况下AOF文件保存的数据集要比RDB文件保存的数据集要完整</li></ol></blockquote><h4><span id="rdb-chi-jiu-hua-fang-shi">RDB持久化方式:</span><a href="#rdb-chi-jiu-hua-fang-shi" class="header-anchor">#</a></h4><h5><span id="rdb-de-you-dian">RDB的优点:</span><a href="#rdb-de-you-dian" class="header-anchor">#</a></h5><ol><li>RDB是一个非常紧凑的文件,它包含了某个时间点的数据集,非常适用于数据集的备份,比如可以在每个小时保存一下过去24小时内的数据,同事每天保存过去30天的数据,这样即使出了问题也可以根据需求恢复到不同的版本的数据集.</li><li>RDB是一个紧凑的单一文件,很方便传输到另一个远端数据中心,非常适用于灾难恢复.</li><li>RDB在保存RDB文件时会父进程唯一需要做的是fork出一个子进程,接下来全部由子进程来做,父进程不需要在做其他IO操作,所以RDB持久化方式可以最大化redis的性能.</li><li>与AOF相比较, 在恢复大的数据集的时候,RDB方式会更快一些 </li></ol><h5><span id="rdb-de-que-dian">RDB的缺点:</span><a href="#rdb-de-que-dian" class="header-anchor">#</a></h5><ol><li>如果Redis服务意外停止工作(断电)的情况下会丢失的数据,虽然可以配置不同的save时间点(如每隔5分钟并且对数据集有100个写的操作),是redis要完整的保存整个数据集是一个比较繁重的工作,通常每隔5分钟或者更久做一次完整的保存,加入在redis意外宕机,可能会丢失几分钟的数据.</li><li>RDB需要经常fork子进程来保存数据集到硬盘上,当数据及比较大的时候,fork的过程就非常耗时,可能会导致Redis在一些毫秒级内不能响应客户端的请求,如果数据及巨大并且CPU性能不是很好的青口下,这种情况会持续几秒,AOF也需要fork,但是你可以调节重写日志文件的频率来提高数据及的耐久度.</li></ol><h5><span id="kuai-zhao">快照:</span><a href="#kuai-zhao" class="header-anchor">#</a></h5><blockquote><p>在默认情况下,redis将数据库快照保存在名字为dump.rdb的二进制文件中.可以对Redis进行设置,让它在”N秒内数据集至少有M个改动”这一条件被满足时,自动报存一次数据及.也可以通过调用SAVE和BGSAVE,手动让Redis进行数据集保存操作</p><pre class=" language-conf"><code class="language-conf">save 60 1000</code></pre><p>这种持久化方式称为快照 snapshotting.</p></blockquote><h5><span id="gong-zuo-fang-shi">工作方式:</span><a href="#gong-zuo-fang-shi" class="header-anchor">#</a></h5><p>当redis需要保存dump.rdb文件时,服务器执行以下操作: </p><ol><li>redis调用forks,同事拥有父进程和子进程</li><li>子进程将数据集写入到一个临时的RDB文件中</li><li>当子进程完成新RDB文件写入时,Redis用新RDB文件替换原来的RDB文件,并删除旧的RDB文件</li></ol><p>这种工作方式使得Redis可以从写时复制(copy-on-write)机制中获益</p>]]></content>
<tags>
<tag> Redis </tag>
</tags>
</entry>
<entry>
<title>Docker实战之入门以及Dockerfile(三)</title>
<link href="2016/03/09/docker-shi-zhan-zhi-ru-men-yi-ji-dockerfile-san/"/>
<url>2016/03/09/docker-shi-zhan-zhi-ru-men-yi-ji-dockerfile-san/</url>
<content type="html"><![CDATA[<p>#Docker实战之入门以及Dockerfile(三)<br><a href="http://git.oschina.net/dockerf/docker-practice/blob/master/Docker%E5%AE%9E%E6%88%98%E4%B9%8B%E5%85%A5%E9%97%A8%E4%BB%A5%E5%8F%8ADockerfile%28%E4%B8%80%29.md?dir=0&filepath=Docker%E5%AE%9E%E6%88%98%E4%B9%8B%E5%85%A5%E9%97%A8%E4%BB%A5%E5%8F%8ADockerfile%28%E4%B8%80%29.md&oid=fd533b0b7665712fdafd3142a61e326ef416dbe1&sha=f11d77675b6ad637a21f26ece6bb11b9e4a66386" target="_blank" rel="noopener">Docker实战之入门以及Dockerfile(一)</a></p><p><a href="http://git.oschina.net/dockerf/docker-practice/blob/master/Docker%E5%AE%9E%E6%88%98%E4%B9%8B%E5%85%A5%E9%97%A8%E4%BB%A5%E5%8F%8ADockerfile%28%E4%BA%8C%29.md?dir=0&filepath=Docker%E5%AE%9E%E6%88%98%E4%B9%8B%E5%85%A5%E9%97%A8%E4%BB%A5%E5%8F%8ADockerfile%28%E4%BA%8C%29.md&oid=76ce88d4a3af4e797873fac07f8f84c4ac72602a&sha=f11d77675b6ad637a21f26ece6bb11b9e4a66386" target="_blank" rel="noopener">Docker实战之入门以及Dockerfile(二)</a></p><p>文章内容，由【<a href="https://csphere.cn/training" target="_blank" rel="noopener">Docker实训课程</a>】</p><p><a href="http://pan.baidu.com/s/1hq2COGc" target="_blank" rel="noopener">第一讲视频</a>翻译整理而成<br><a href="https://github.com/nicescale/docker-training" target="_blank" rel="noopener">培训代码</a> <a href="https://github.com/nicescale/docker-training" target="_blank" rel="noopener">https://github.com/nicescale/docker-training</a><br><a href="http://market.aliyun.com/products/56014007/jxsc000181.html" target="_blank" rel="noopener">虚拟机镜像</a> <a href="http://market.aliyun.com/products/56014007/" target="_blank" rel="noopener">http://market.aliyun.com/products/56014007/</a><br>##应用镜像</p><p>##csphere/wordpress:4.2</p><pre><code># cd docker-training/wordpress/# ls -a. license.txt wp-config-sample.php wp-login.php.. readme.html wp-content wp-mail.phpDockerfile wp-activate.php wp-cron.php wp-settings.php.dockerignore wp-admin wp-includes wp-signup.phpindex.php wp-blog-header.php wp-links-opml.php wp-trackback.phpinit.sh wp-comments-post.php wp-load.php xmlrpc.php/docker-training/wordpress# cat Dockerfile from csphere/php-fpm:5.4add init.sh /init.shentrypoint ["/init.sh", "/usr/bin/supervisord", "-n", "-c", "/etc/supervisord.conf"]</code></pre><p>使用docker后，在项目代码目录下，写Dockerfile文件，非常方便把项目代码直接打包到docker镜像中，如有哪些文件不想被打包进去，可以在<code>.dockerignore</code>文件中定义</p><p>Dockerfile解析：</p><ul><li>wordpress镜像是基于csphere/php-fpm:5.4来进行构建</li><li><code>ONBUILD</code>指令生效，把代码文件拷贝到网站根目录下</li><li><code>init.sh</code>脚本对WordPress连接mysql数据库进行配置，固运行wordpress镜像后，只需要进行配置WordPress即可，数据库已准备就绪！</li></ul><p>生成WordPress镜像</p><p><code>docker build -t csphere/wordpress:4.2 .</code></p><p>查看当前主机本地都有哪些docker镜像</p><p><code>docker images</code></p><p>创建WordPress准备</p><p>查看主机ip地址</p><p><code>ifconfig eth0</code><br>192.168.1.20</p><p>创建WordPress容器：</p><pre><code>docker run -d -p 80:80 --name wordpress -e WORDPRESS_DB_HOST=192.168.1.20 -e WORDPRESS_DB_USER=admin -e WORDPRESS_DB_PASSWORD=csphere2015 csphere/wordpress:4.249d0cddb4e6998a43285fe09165030ba80485065867b9cb8fae9fbdb97cd077f</code></pre><p>参数解析：</p><ul><li>-d 后台运行</li><li>-p 80:80 将宿主机的80端口映射到容器的80端口</li><li>–name wordpress 给容器命名为wordpress</li><li>-e WORDPRESS_DB_HOST=192.168.1.20 数据库主机的ip地址（或者域名）</li><li>-e WORDPRESS_DB_USER=admin 数据库的用户，默认是admin</li><li>-e WORDPRESS_DB_PASSWORD=csphere2015 登陆数据的密码，默认是csphere2015</li><li>csphere/wordpress:4.2使用此镜像创建WordPress容器</li></ul><p>访问<code>http://your_ip</code>，选择语言，并进行设置<code>wordpress</code></p><p><img src="https://discuss.csphere.cn/uploads/default/optimized/2X/5/5e887120f584095151915ef2ddbc6c6f6d8e4885_1_422x500.png" alt></p><h2><span id="entrypoint-he-cmd-de-qu-bie">ENTRYPOINT和CMD的区别</span><a href="#entrypoint-he-cmd-de-qu-bie" class="header-anchor">#</a></h2><p>ENTRYPOINT解析</p><p>定义：</p><p>An ENTRYPOINT allows you to configure a container that will run as an executable</p><p>运行一个Docker容器像运行一个程序一样</p><p>ENTRYPOINT的使用方法：</p><p>1.ENTRYPOINT [“executable”, “param1”, “param2”] (the preferred exec form)</p><blockquote><p>推荐使用1方法，启动起来后，pid为1</p></blockquote><p>2.ENTRYPOINT command param1 param2 (shell form) </p><blockquote><p>启动起来后，pid号为shell命令执行完的pid号</p></blockquote><p>CMD解析</p><p>CMD的使用方法：</p><p>1.CMD [“executable”,”param1”,”param2”] (exec form, this is the preferred form)</p><blockquote><p>运行一个可执行的文件并提供参数</p></blockquote><p>2.CMD [“param1”,”param2”] (as default parameters to ENTRYPOINT) </p><blockquote><p>为ENTRYPOINT指定参数</p></blockquote><p>3.CMD command param1 param2 (shell form) </p><blockquote><p>是以”/bin/sh -c”的方法执行的命令</p></blockquote><p>实战测试<code>CMD</code></p><pre><code>vim DockerfileFROM centos:centos7.1.1503CMD ["/bin/echo", "This is test cmd"]</code></pre><p>生成cmd镜像<br><code>docker build -t csphere/cmd:0.1 .</code><br>生成cmd容器，进行测试<br><code>docker run -it --rm csphere/cmd:0.1</code><br><code>This is test cmd</code><br>测试是否可以替换<code>cmd</code>的命令</p><pre><code>docker run -it csphere/cmd:0.1 /bin/bash[root@c1963a366319 /]#</code></pre><p>测试结果，在Dockerfile中定义的<code>CMD</code>命令,在执行<code>docker run</code>的时候，<code>CMD</code>命令可以被替换。</p><p>实战测试<code>ENTRYPOINT</code></p><pre><code>FROM centos:centos7.1.1503ENTRYPOINT ["/bin/echo", "This is test entrypoint"]</code></pre><p>生成ent(entrypoint)镜像<br><code>docker build -t csphere/ent:0.1 .</code></p><p>生成ent容器，进行测试</p><pre><code>docker run -it csphere/ent:0.1This is test entrypoint</code></pre><p>测试是否可以替换<code>entrypoint</code>的命令</p><pre><code>docker run -it csphere/ent:0.1 /bin/bashThis is test entrypoint /bin/bash</code></pre><p>测试结果，在Dockerfile定义的<code>ENTRYPOINT</code>命令，通过以上方式不能被替换</p><p>实战再次测试<code>ENTRYPOINT</code></p><p><code>docker run -it --entrypoint=/bin/bash csphere/ent:0.1</code><br>测试结果，ENTRYPOINT命令也可以被替换，需要在执行<code>docker run</code>时添加<code>--entrypoint=</code>参数，此方法多用来进行调试</p><p>##更多精彩内容，访问：<a href="https://discuss.csphere.cn" target="_blank" rel="noopener">cSphere-希云社区</a></p><p>说明，文章由<a href="https://csphere.cn" target="_blank" rel="noopener">cSphere-希云</a>所有，转载请整体转载，并保留原文链接，且不得修改原文！</p><p>转载，请联系”cSphere”微信公众号<br><img src="https://discuss.csphere.cn/uploads/default/original/2X/1/1f82a3e93d907fc9fb5acf552f2646b74f5706d5.jpg" alt></p>]]></content>
<categories>
<category> Docker </category>
</categories>
<tags>
<tag> Docker </tag>
</tags>
</entry>
<entry>
<title>Docker实战之入门以及Dockerfile(一)</title>
<link href="2016/03/09/docker-shi-zhan-zhi-ru-men-yi-ji-dockerfile-yi/"/>
<url>2016/03/09/docker-shi-zhan-zhi-ru-men-yi-ji-dockerfile-yi/</url>
<content type="html"><![CDATA[<p>#Docker实战之入门以及Dockerfile(一) </p><p>文章内容，由【<a href="https://csphere.cn/training" target="_blank" rel="noopener">Docker实训课程</a>】</p><p><a href="http://pan.baidu.com/s/1hq2COGc" target="_blank" rel="noopener">第一讲视频</a>翻译整理而成</p><p><a href="https://github.com/nicescale/docker-training" target="_blank" rel="noopener">培训代码</a> <a href="https://github.com/nicescale/docker-training" target="_blank" rel="noopener">https://github.com/nicescale/docker-training</a></p><p><a href="http://market.aliyun.com/products/56014007/jxsc000181.html" target="_blank" rel="noopener">虚拟机镜像</a> <a href="http://market.aliyun.com/products/56014007/" target="_blank" rel="noopener">http://market.aliyun.com/products/56014007/</a></p><p>###首先Docker是软件工业上的集装箱技术</p><p>###回顾，在没有集装箱出现以前，传统运输行业中，会存在这些问题：</p><ul><li>在运输过程中，货物损坏</li><li>装卸、运输货物，效率低下</li><li>运输手续繁多及运输环节多</li><li>劳动强度大，及船舶周转慢</li></ul><p>###在集装箱出现后，完全改变了这种状况，是由于集装箱：</p><ul><li>规则标准化，大大减少了包装费用</li><li>大大提升了货物装卸效率、及运输效率</li><li>不同种运输工具之间转换更容易</li></ul><p>###所以，集装箱出现是传统行业中的一次重大变革</p><p>###传统软件行业中存在的问题</p><ul><li>软件更新发布低效</li><li>业务无法敏捷</li><li>环境一致性，难于保证</li><li>不同环境之间迁移成本太高</li><li>软件开发商，交付实施周期长—成本高</li></ul><p>###有了Docker，以上问题，有望或者说在很大程度上可以得到解决</p><h2><span id="er-docker-de-zu-cheng">二、Docker的组成</span><a href="#er-docker-de-zu-cheng" class="header-anchor">#</a></h2><h4><span id="docker-shi-yi-ge-c-s-jia-gou">Docker是一个C/S架构</span><a href="#docker-shi-yi-ge-c-s-jia-gou" class="header-anchor">#</a></h4><p><img src="https://discuss.csphere.cn/uploads/default/optimized/2X/4/4ae4f9f4583f00766f8126e232d0d5e60042a363_1_666x500.jpg" alt></p><ul><li>Docker Client： Docker的客户端</li><li>Docker Server： Docker daemon的主要组成部分，接收用户通过Docker Client发送的请求，并按照相应的路由规则实现路由分发</li><li>Docker Registry： Registry是Docker镜像的中央存储仓库（pull/push）</li></ul><blockquote><p>通过docker pull命令可以把Registry上的docker镜像，下载到服务器本地</p></blockquote><blockquote><p>通过docker push命令可以把服务器本地的docker镜像，上传到Registry上</p></blockquote><p><strong>Registry在构建自动化平台，起着非常重要的作用！</strong></p><p><img src="https://discuss.csphere.cn/uploads/default/optimized/2X/6/6643a54f646047a9dbf31cce424408b4d3fe5ee3_1_666x500.jpg" alt></p><p><em>提示：Docker镜像运行之后会成为Docker容器—-通过 docker run命令</em></p><p>####Docker容器启动速度非常快，体现在2个方面；</p><p>1.磁盘占用空间小，因为docker镜像采用了分层技术，构建的镜像大小，只有自身的大小，不包含父镜像的大小</p><p>2.内存消耗少，docker容器共享的宿主机的内核，没有操作的进程消耗</p><p>##Docker实战准备</p><ol><li>首先登陆<a href="http://git.oschina.net" target="_blank" rel="noopener">OSChina Git</a></li><li>将<a href="http://git.oschina.net/dockerf/docker-training" target="_blank" rel="noopener">docker-training</a>项目Fork到自己的仓库</li><li>使用自己熟悉的SSH工具连接到服务器</li><li>执行<code>git clone https://git.oschina.net/*YOURNAME*/docker-training.git</code>，将你的远程仓库clone到服务器</li></ol><p><a href="http://git.oschina.net/progit/" target="_blank" rel="noopener">Git 使用指南</a></p><p>后续会构建4个docker镜像，分别为：</p><blockquote><p>centos7 (基础镜像)</p></blockquote><blockquote><p>php-fpm mysql（中间件镜像）</p></blockquote><blockquote><p>worpdress(应用镜像)</p></blockquote><p>####什么是Dockerfile？<br><img src="https://discuss.csphere.cn/uploads/default/original/2X/8/8f12e062a5b2c17e8f502b01fc87948ee77453df.jpg" alt></p><blockquote><p>Dockerfile是自动构建docker镜像的配置文件，Dockerfile中的命令非常类似linux shell下的命令</p></blockquote><blockquote><p>Dockerfile,可以让用户自定义构建docker镜像，支持以 # 开头的注释行</p></blockquote><blockquote><p>一般，Dockerfile分为4部分</p></blockquote><ul><li>基础镜像（父镜像）信息</li><li>维护者信息</li><li>镜像操作命令</li><li>容器启动命令</li></ul><p>####为何把Dockerfile存放到git仓库中，并为每个项目创建git仓库？</p><blockquote><p>方便通过自动化平台，自动构建docker镜像</p></blockquote><p>##三、Dockerfile介绍</p><p>###基础镜像csphere/centos:7.1</p><pre><code>## MAINTAINER Carson,C.J.Zeong <zcy@nicescale.com># DOCKER-VERSION 1.6.2## Dockerizing CentOS7: Dockerfile for building CentOS images#FROM centos:centos7.1.1503MAINTAINER Carson,C.J.Zeong <zcy@nicescale.com>ENV TZ "Asia/Shanghai"ENV TERM xtermADD aliyun-mirror.repo /etc/yum.repos.d/CentOS-Base.repoADD aliyun-epel.repo /etc/yum.repos.d/epel.repoRUN yum install -y curl wget tar bzip2 unzip vim-enhanced passwd sudo yum-utils hostname net-tools rsync man && \ yum install -y gcc gcc-c++ git make automake cmake patch logrotate python-devel libpng-devel libjpeg-devel && \ yum install -y --enablerepo=epel pwgen python-pip && \ yum clean allRUN pip install supervisorADD supervisord.conf /etc/supervisord.confRUN mkdir -p /etc/supervisor.conf.d && \ mkdir -p /var/log/supervisorEXPOSE 22ENTRYPOINT ["/usr/bin/supervisord", "-n", "-c", "/etc/supervisord.conf"]</code></pre><p><code>FROM centos:centos7.1.1503</code> </p><blockquote><p>基于<strong>父镜像</strong>构建其他docker镜像，_父镜像_：可以通过docker pull 命令获得，也可以自己制作</p></blockquote><p><code>MAINTAINER Carson,C.J.Zeong <zcy@nicescale.com></code> </p><blockquote><p>Dockerfile维护者</p></blockquote><p><code>ENV TZ "Asia/Shanghai"</code></p><blockquote><p>ENV（environment）设置环境变量，一个Dockerfile中可以写多个。以上例子是：设置docker容器的时区为Shanghai</p></blockquote><p>######Dockerfile中有2条指令可以拷贝文件</p><p><code>ADD aliyun-mirror.repo /etc/yum.repos.d/CentOS-Base.repo</code> </p><blockquote><p>拷贝本地文件到docker容器里，还可以拷贝URL链接地址下的文件，ADD还具有解压软件包的功能(支持gzip, bzip2 or xz)</p></blockquote><p><code>COPY test /mydir</code> </p><blockquote><p>拷贝本地文件到docker容器</p></blockquote><p><code>RUN yum install -y curl wget....</code> </p><blockquote><p>RUN命令，非常类似linux下的shell命令 <code>(the command is run in a shell - /bin/sh -c - shell form)</code></p></blockquote><blockquote><p>在Dockerfile中每执行一条指令（ENV、ADD、RUN等命令），都会生成一个docker image layer</p></blockquote><p><code>RUN pip install supervisor</code> </p><blockquote><p>supervisor进程管理系统，推荐使用</p></blockquote><p><code>ADD supervisord.conf /etc/supervisord.conf</code></p><blockquote><p>添加supervisor的主配置文件，到docker容器里</p></blockquote><p><code>RUN mkdir -p /etc/supervisor.conf.d</code></p><blockquote><p>创建存放启动其他服务”supervisor.conf”的目录，此目录下的所有以.conf结尾的文件，在启动docker容器的时候会被加载</p></blockquote><p><code>EXPOSE 22</code></p><blockquote><p>端口映射 <code>EXPOSE <host_port>:<container_port></code></p></blockquote><blockquote><p>推荐使用<code>docker run -p <host_port>:<container_port></code> 来固化端口</p></blockquote><p><code>ENTRYPOINT ["/usr/bin/supervisord", "-n", "-c", "/etc/supervisord.conf"]</code></p><blockquote><p>一个Dockerfile中只有最后一条<code>ENTRYPOINT</code>生效，并且每次启动docker容器，都会执行<code>ENTRYPOINT</code></p></blockquote><p>######以上文件就是用来生成第一个docker镜像的Dockerfile，通过<code>docker build</code>指令来生成docker镜像</p><p><code>docker build -t csphere/centos:7.1 .</code></p><blockquote><p>如果Dockerfile在当前目录下，输入点<code>.</code>就可以进行加载当前目录下的<code>Dockerfile</code></p></blockquote><blockquote><p>如果不在当前目录下需要运行<code>docker build -t csphere/centos:7.1 <Dockerfile_dir></code>加载相对路径下的<code>Dockerfile</code></p></blockquote><p>docker镜像的命名规则 <code>registry_url/namespace/image_name:tag</code> 默认<code>tag</code>是<code>latest</code></p><blockquote><p>在构建Docker镜像时，如果有自己内部的yum源，替换成自己内部的yum源地址，可以加快构建速度。</p></blockquote><blockquote><p>如果第一次构建失败，会有部分镜像layer生成，第二次构建会基于第一次构建所生成的layer（use cache），继续构建</p></blockquote><pre><code>Step 10 : EXPOSE 22 ---> Running in 0ed1c5479ebc ---> c57a5bac41c8Removing intermediate container 0ed1c5479ebcStep 11 : ENTRYPOINT /usr/bin/supervisord -n -c /etc/supervisord.conf ---> Running in e16c7ac2fd45 ---> 185ef7b101a8Removing intermediate container e16c7ac2fd45Successfully built 185ef7b101a8</code></pre><p>可以看到每执行一条<code>Dockerfile</code>的指令都会生成一个镜像的layer<code>c57a5bac41c8</code> <code>185ef7b101a8</code> 最后<code>185ef7b101a8</code>这个是docker镜像的ID,<code>185ef7b101a8</code>是由<code>c57a5bac41c8</code> <code>185ef7b101a8</code>…layers叠加而成，体现了docker镜像是分层的</p><pre><code># docker images 查看当前主机本地有哪些docker镜像 REPOSITORY TAG IMAGE ID CREATED VIRTUAL SIZEcsphere/centos 7.1 185ef7b101a8 40 minutes ago 451.9 MB</code></pre><p>通过docker镜像生成一个docker容器</p><p><code>docker help run</code> #查看<code>docker run</code>命令的使用方法</p><p>#####介绍日常工作中经常用到的参数:</p><p><code>docker run -it</code> #启动docker容器在前端<br><code>docker run -d</code> #启动docker容器在后台</p><p><code>docker run -p</code><br><code>docker run -P</code></p><blockquote><p>在Dockerfile中有一条指令是EXPOSE 22，如果使用<code>-P</code>，宿主机会随机选择一个<code>没有被使用的端口</code>和docker<code>容器的22端口</code>做<code>端口映射</code>,如果docker主机或者容器重启后，宿主机又会随机选择一个没有被使用的端口和docker容器的22端口做端口映射，这样端口会发生<code>变化</code></p></blockquote><blockquote><p>如果使用<code>-p</code>,比如<code>2222:22</code>，这样不管是docker主机或者容器重启后，2222:22端口都是这样来映射，<code>不会发生改变</code> </p></blockquote><p>生成docker容器</p><p><code>docker run -d -p 2222:22 --name base csphere/centos:7.1</code></p><p><code>37ac69acf47d501ffc61d8883ae2ba362a132d11e46897212a92b1936e0a0593</code></p><p>参数说明：</p><ul><li>-d 后台运行</li><li>-it 前台交互式运行</li><li>-P 22 将宿主机的一个未使用的随机端口映射到容器的22端口</li><li>-p 2222:22 将宿主机的2222端口映射到容器的22端口</li><li>–name base 给容器命名为base</li><li>csphere/centos:7.1 使用这个镜像镜像创建docker容器</li></ul><p>查看Docker容器</p><p><code>docker ps</code></p><blockquote><p><code>ps</code>默认只会显示容器在“running”的状态的，容器列表</p></blockquote><p><code>docker ps -a</code></p><blockquote><p><code>ps -a</code> 会查看到所有的容器列表</p></blockquote><p>下一篇<a href="http://git.oschina.net/dockerf/docker-practice/blob/master/Docker%E5%AE%9E%E6%88%98%E4%B9%8B%E5%85%A5%E9%97%A8%E4%BB%A5%E5%8F%8ADockerfile%28%E4%BA%8C%29.md?dir=0&filepath=Docker%E5%AE%9E%E6%88%98%E4%B9%8B%E5%85%A5%E9%97%A8%E4%BB%A5%E5%8F%8ADockerfile%28%E4%BA%8C%29.md&oid=76ce88d4a3af4e797873fac07f8f84c4ac72602a&sha=f11d77675b6ad637a21f26ece6bb11b9e4a66386" target="_blank" rel="noopener">Docker实战之入门以及Dockerfile(二)</a></p><hr><p>说明，文章由<a href="https://csphere.cn" target="_blank" rel="noopener">cSphere-希云</a>所有，转载请整体转载，并保留原文链接，且不得修改原文！</p><p>转载，请联系”cSphere”微信公众号<br><img src="https://discuss.csphere.cn/uploads/default/original/2X/1/1f82a3e93d907fc9fb5acf552f2646b74f5706d5.jpg" alt></p>]]></content>
<categories>
<category> Docker </category>
</categories>
<tags>
<tag> Docker </tag>
</tags>
</entry>
<entry>
<title>Docker实战之入门以及Dockerfile(二)</title>
<link href="2016/03/09/docker-shi-zhan-zhi-ru-men-yi-ji-dockerfile-er/"/>
<url>2016/03/09/docker-shi-zhan-zhi-ru-men-yi-ji-dockerfile-er/</url>
<content type="html"><![CDATA[<p>#Docker实战之入门以及Dockerfile(二)<br>上一篇<a href="http://git.oschina.net/dockerf/docker-practice/blob/master/Docker%E5%AE%9E%E6%88%98%E4%B9%8B%E5%85%A5%E9%97%A8%E4%BB%A5%E5%8F%8ADockerfile%28%E4%B8%80%29.md?dir=0&filepath=Docker%E5%AE%9E%E6%88%98%E4%B9%8B%E5%85%A5%E9%97%A8%E4%BB%A5%E5%8F%8ADockerfile%28%E4%B8%80%29.md&oid=fd533b0b7665712fdafd3142a61e326ef416dbe1&sha=f11d77675b6ad637a21f26ece6bb11b9e4a66386" target="_blank" rel="noopener">Docker实战之入门以及Dockerfile(一)</a></p><p>文章内容，由【<a href="https://csphere.cn/training" target="_blank" rel="noopener">Docker实训课程</a>】</p><p><a href="http://pan.baidu.com/s/1hq2COGc" target="_blank" rel="noopener">第一讲视频</a>翻译整理而成</p><p><a href="https://github.com/nicescale/docker-training" target="_blank" rel="noopener">培训代码</a> <a href="https://github.com/nicescale/docker-training" target="_blank" rel="noopener">https://github.com/nicescale/docker-training</a></p><p><a href="http://market.aliyun.com/products/56014007/jxsc000181.html" target="_blank" rel="noopener">虚拟机镜像</a> <a href="http://market.aliyun.com/products/56014007/jxsc000181.html" target="_blank" rel="noopener">http://market.aliyun.com/products/56014007/jxsc000181.html</a><br>##中间件镜像</p><p>培训代码 <a href="https://github.com/nicescale/docker-training" target="_blank" rel="noopener">https://github.com/nicescale/docker-training</a></p><p>虚拟机镜像 <a href="http://market.aliyun.com/products/56014007/jxsc000181.html" target="_blank" rel="noopener">http://market.aliyun.com/products/56014007/jxsc000181.html</a><br>##csphere/php-fpm:5.4</p><pre><code># cd docker-training/php-fpm/# ls Dockerfile nginx_nginx.conf supervisor_nginx.confnginx_default.conf php_www.conf supervisor_php-fpm.conf</code></pre><p>各文件解释：</p><blockquote><p>nginx_nginx.conf 替换默认的nginx.conf文件</p></blockquote><blockquote><p>nginx_default.conf 替换默认的default.conf文件</p></blockquote><blockquote><p>php_<a href="http://www.conf" target="_blank" rel="noopener">www.conf</a> 修改apache用户为nginx</p></blockquote><blockquote><p>supervisor_nginx.conf 添加启动nginx的supervisor文件</p></blockquote><blockquote><p>supervisor_php-fpm.conf 添加启动php-fpm的supervisor文件 </p></blockquote><pre><code># cat Dockerfile ## MAINTAINER Carson,C.J.Zeong <zcy@nicescale.com># DOCKER-VERSION 1.6.2## Dockerizing php-fpm: Dockerfile for building php-fpm images#FROM csphere/centos:7.1MAINTAINER Carson,C.J.Zeong <zcy@nicescale.com># Set environment variableENV APP_DIR /appRUN yum -y install nginx php-cli php-mysql php-pear php-ldap php-mbstring php-soap php-dom php-gd php-xmlrpc php-fpm php-mcrypt && \ yum clean allADD nginx_nginx.conf /etc/nginx/nginx.confADD nginx_default.conf /etc/nginx/conf.d/default.confADD php_www.conf /etc/php-fpm.d/www.confRUN sed -i 's/;cgi.fix_pathinfo=1/cgi.fix_pathinfo=0/' /etc/php.iniRUN mkdir -p /app && echo "<?php phpinfo(); ?>" > ${APP_DIR}/info.phpEXPOSE 80 443ADD supervisor_nginx.conf /etc/supervisor.conf.d/nginx.confADD supervisor_php-fpm.conf /etc/supervisor.conf.d/php-fpm.confONBUILD ADD . /appONBUILD RUN chown -R nginx:nginx /app</code></pre><p>命令解析：</p><p><code>ONBUILD ADD . /app</code></p><p><code>ONBUILD</code> 在生成当前docker镜像的时候不生效，在子镜像生效；<code>ONBUILD</code>在产品发布时起着非常重要的作用！举例</p><blockquote><p>A镜像中有<code>ONBUILD</code>指令，在构建A镜像时<code>ONBUILD</code>指令不执行；B镜像<code>FROM A</code>,在构建B镜像时<code>ONBUILD</code>指令开始执行；</p></blockquote><p>如何给docker镜像命名：</p><ul><li>registry-url: registry服务器的域名或者ip</li><li>namespace: </li><li>image-name: docker镜像的名字</li><li>tag: docker镜像的版本号，推荐使用应用服务的版本号来命名，如<code>php-fpm:5.4</code></li></ul><p>生成php-fpm镜像</p><p><code>docker build -t csphere/php-fpm:5.4 .</code></p><pre><code>Step 12 : ONBUILD add . /app ---> Running in 9e21ede67350 ---> 7541483a5a76Removing intermediate container 9e21ede67350Step 13 : ONBUILD run chown -R nginx:nginx /app ---> Running in ab55fc7a46a1 ---> c61699e8c237Removing intermediate container ab55fc7a46a1Successfully built c61699e8c237</code></pre><p>生成website容器：</p><p><code>docker run -d -p 8080:80 --name website csphere/php-fpm:5.4</code><br><code>da30b15d3518320f4150b20ef329e59432a65610968977277879578b5fd8f4f7</code></p><p>参数解释：</p><ul><li>-d 后台运行</li><li>-p 8080:80 将宿主机的8080端口映射到容器的80端口</li><li>–name website 给容器命名为website</li><li>csphere/php-fpm:5.4 使用这个镜像镜像创建docker容器</li></ul><p>使用浏览器访问：<code>http://your_ip:8080/info.php</code></p><p><img src="https://discuss.csphere.cn/uploads/default/original/2X/c/c002db15d4ed47329927a97066d485bce874ccfb.png" alt></p><p>如何进入一个正在运行的docker容器？</p><p><code>docker exec -it website /bin/bash</code></p><pre><code># supervisorctl 查看当前容器中使用supervisor启动了哪些服务 nginx RUNNING pid 9, uptime 0:23:15php-fpm RUNNING pid 10, uptime 0:23:15</code></pre><p>##csphere/mysql:5.5</p><pre><code>cat Dockerfile ## MAINTAINER Carson,C.J.Zeong <zcy@nicescale.com># DOCKER-VERSION 1.6.2## Dockerizing Mariadb: Dockerfile for building Mariadb images#FROM csphere/centos:7.1MAINTAINER Carson,C.J.Zeong <zcy@nicescale.com>ENV DATA_DIR /var/lib/mysql# Install MariadbRUN yum install -y mariadb mariadb-server && \ yum clean allADD mysqld_charset.cnf /etc/my.cnf.d/COPY scripts /scriptsRUN chmod +x /scripts/startEXPOSE 3306VOLUME ["/var/lib/mysql"]ENTRYPOINT ["/scripts/start"]</code></pre><p>命令解析：</p><p><code>VOLUME ["/var/lib/mysql"]</code></p><blockquote><p><code>VOLUME</code>指令，宿主机文件目录和docker容器文件目录做映射</p></blockquote><p><code>ENTRYPOINT ["/scripts/start"]</code></p><blockquote><p><code>ENTRYPOINT</code>在每次启动docker容器时都会被执行，此例，是运行了一个shell脚本”/scripts/start”</p></blockquote><p>每次启动都会运行<code>/scripts/start</code>脚本，脚本内容如下：</p><pre><code># cat start #!/bin/bashset -e## When Startup Container script#if [[ -e /scripts/firstrun ]]; then # config mariadb /scripts/firstrun_maria rm /scripts/firstrunelse # Cleanup previous mariadb sockets if [[ -e ${DATA_DIR}/mysql.sock ]]; then rm -f ${DATA_DIR}/mysql.sock fifiexec /usr/bin/mysqld_safe</code></pre><p>脚本解析：</p><ul><li><p><code>set -e</code> 脚本中只要有一行有错误，就会中断脚本执行</p></li><li><p>如果firstrun文件存在，执行firstrun_maria脚本，如果不存在，删除mysql.sock文件，并启动Mariadb</p></li></ul><blockquote><p>firstrun_maira脚本是初始化Mariadb，以及设置数据库用户和密码，详情内容请自行阅读<a href="http://git.oschina.net/dockerf/docker-training/blob/master/mysql/scripts/firstrun_maria?dir=0&filepath=mysql%2Fscripts%2Ffirstrun_maria&oid=788bfb61d8cc45a33b60cde5a0e98899ee08f808&sha=3a86a4767292c267af0794628efb76fe31e754e6" target="_blank" rel="noopener">脚本文件</a></p></blockquote><p>构建mysql docker镜像</p><p><code>docker build -t csphere/mysql:5.5 .</code></p><p>###docker volume 保证删除容器后，数据不被删除</p><ul><li>保存容器中的数据</li><li>数据共享</li></ul><p>使用方法：</p><p>1.在Dockerfile中定义VOLUME[“/data”]</p><p>2.通过<code>docker run -d -v <host_dir>:<container_dir></code></p><p>###案例：</p><ol><li>创建mysql容器，不挂载docker volume，删除后，数据是否存在</li><li>创建mysql容器，挂载docker volume，删除后，数据是否存在</li></ol><p>运行不挂载docker volume的mysql容器</p><p><code># docker run -d -p 3306:3306 --name dbserver csphere/mysql:5.5</code><br><code>0a3092395c1e6a84f0ecd5383799f210519c5aefc82cbb7ee2ed1a471fc463f5</code></p><p>删除docker容器，容器里面的数据都会随着容器被删除而删除</p><pre><code># docker rm dbserverError response from daemon: Cannot destroy container dbserver: Conflict, You cannot remove a running container. Stop the container before attempting removal or use -fError: failed to remove containers: [dbserver]</code></pre><p>参数解释：</p><ul><li><code>docker rm</code> 删除状态为“Exited”的docker容器</li><li><code>docker rm -f</code> 强制删除docker容器</li></ul><p>运行挂载docker volume的mysql容器</p><p><code>docker run -d -p 3306:3306 -v /var/lib/docker/vfs/dir/mydata:/var/lib/mysql csphere/mysql:5.5</code><br><code>f49165d5e081b8bd8af9cb9c0bbbeb6545d45f857c1a852646c105</code><br><code>docker exec -it f49 /bin/bash</code></p><p>登陆数据库创建mydb数据库</p><pre><code># mysql# show databases;# create database mydb;Query OK, 1 row affected (0.00 sec)# show databases;+--------------------+| Database |+--------------------+| information_schema || mydb || mysql || performance_schema || test |+--------------------+5 rows in set (0.00 sec)# exitexit</code></pre><p>查看主机文件目录下，是否已生成mydb数据库目录文件</p><pre><code># ls /var/lib/docker/vfs/dir/mydata/aria_log.00000001 ibdata1 ib_logfile1 mysql performance_schemaaria_log_control ib_logfile0 mydb mysql.sock test</code></pre><pre><code>停止docker容器# docker stop f49165d5e081f49165d5e081删除docker容器，查看`mydb`目录是否被删除# docker rm f49165d5e081f49165d5e081# ls /var/lib/docker/vfs/dir/mydata/ 验证，挂载docker volume后，容器被删除掉，数据还在aria_log.00000001 ibdata1 ib_logfile1 mysql performance_schemaaria_log_control ib_logfile0 mydb mysql.sock test</code></pre><p>新创建一个容器，挂载到刚才的数据目录下，是否可以把之前的数据库加载回来</p><p><code>docker run -d -p 3306:3306 --name newdb -v /var/lib/docker/vfs/dir/mydata:/var/lib/mysql csphere/mysql:5.5</code><br><code>29418b93d4d4a00a86169c568b6f952e71d25b155d7f6b8012d953022691b2b8</code></p><p><code>docker exec -it newdb /bin/bash</code></p><pre><code># mysqlMariaDB [(none)]> show databases;+--------------------+| Database |+--------------------+| information_schema || mydb || mysql || performance_schema || test |+--------------------+5 rows in set (0.00 sec)</code></pre><p>验证结果: 只要保证数据在，重新创建一个容器挂载回之前的数据目录，业务即可恢复（容器可随意删除、创建）</p><p>下一篇<a href="http://git.oschina.net/dockerf/docker-practice/blob/master/Docker%E5%AE%9E%E6%88%98%E4%B9%8B%E5%85%A5%E9%97%A8%E4%BB%A5%E5%8F%8ADockerfile%28%E4%B8%89%29.md?dir=0&filepath=Docker%E5%AE%9E%E6%88%98%E4%B9%8B%E5%85%A5%E9%97%A8%E4%BB%A5%E5%8F%8ADockerfile%28%E4%B8%89%29.md&oid=4b0000ad67c06e7f1a3c57edf69e215636b3dd2a&sha=f11d77675b6ad637a21f26ece6bb11b9e4a66386" target="_blank" rel="noopener">Docker实战之入门以及Dockerfile(三)</a></p><hr><p>说明，文章由<a href="https://csphere.cn" target="_blank" rel="noopener">cSphere-希云</a>所有，转载请整体转载，并保留原文链接，且不得修改原文！</p><p>转载，请联系”cSphere”微信公众号</p><p><img src="https://discuss.csphere.cn/uploads/default/original/2X/1/1f82a3e93d907fc9fb5acf552f2646b74f5706d5.jpg" alt></p>]]></content>
<categories>
<category> Docker </category>
</categories>
<tags>
<tag> Docker </tag>
</tags>
</entry>
<entry>
<title>大数据实时数仓</title>
<link href="1970/01/01/da-shu-ju-shi-shi-shu-cang/"/>
<url>1970/01/01/da-shu-ju-shi-shi-shu-cang/</url>
<content type="html"><![CDATA[]]></content>
</entry>
<entry>
<title>Docker实战之Dockerfile</title>
<link href="1970/01/01/docker-shi-zhan-zhi-dockerfile/"/>
<url>1970/01/01/docker-shi-zhan-zhi-dockerfile/</url>
<content type="html"><![CDATA[<div class="toc"><!-- toc --><ul><li><a href="#docker-zu-jian">Docker 组件:</a></li></ul><!-- tocstop --></div>## Docker 组件: Docker Client:Docker的客户端 Docker Server：Docker daemon的主要组成部分，Docker Client发送的请求，并按照实现路由分发。 Docker Images：Docker镜像运行之后变成容器 Docker Registry：Registry是Docker镜像的中央存储仓库]]></content>
</entry>
<entry>
<title>阿里分布式事物框架(seata)</title>
<link href="1970/01/01/a-li-fen-bu-shi-shi-wu-kuang-jia-seata/"/>
<url>1970/01/01/a-li-fen-bu-shi-shi-wu-kuang-jia-seata/</url>
<content type="html"><![CDATA[<h1><span id="a-li-fen-bu-shi-shi-wu-kuang-jia-seata">阿里分布式事物框架（seata）</span><a href="#a-li-fen-bu-shi-shi-wu-kuang-jia-seata" class="header-anchor">#</a></h1><p>Github地址： <a href="(https://github.com/seata/seata)">seata</a></p>]]></content>
</entry>
</search>