Site updated: 2023-11-28 00:01:39

CNAME

@@ -0,0 +1 @@
+hawyior.top

machine-learning-notes/index.html

@@ -82,12 +82,12 @@ <h1 class="p-name article-title" itemprop="headline name">
 
 <div id="article-toc">
     <h2 class="widget-title">目录</h2>
-    <ol class="toc"><li class="toc-item toc-level-1"><a class="toc-link" href="#machine-learning"><span class="toc-number">1.</span> <span class="toc-text">Machine Learning</span></a><ol class="toc-child"><li class="toc-item toc-level-2"><a class="toc-link" href="#%E7%BA%BF%E6%80%A7%E5%9B%9E%E5%BD%92"><span class="toc-number">1.1.</span> <span class="toc-text">线性回归</span></a><ol class="toc-child"><li class="toc-item toc-level-3"><a class="toc-link" href="#%E6%A2%AF%E5%BA%A6%E4%B8%8B%E9%99%8D"><span class="toc-number">1.1.1.</span> <span class="toc-text">梯度下降</span></a></li><li class="toc-item toc-level-3"><a class="toc-link" href="#%E5%A4%9A%E5%85%83%E7%BA%BF%E6%80%A7%E5%9B%9E%E5%BD%92"><span class="toc-number">1.1.2.</span> <span class="toc-text">多元线性回归</span></a></li></ol></li></ol></li></ol>
+    <ol class="toc"><li class="toc-item toc-level-1"><a class="toc-link" href="#course1"><span class="toc-number">1.</span> <span class="toc-text">Course1</span></a><ol class="toc-child"><li class="toc-item toc-level-2"><a class="toc-link" href="#%E7%BA%BF%E6%80%A7%E5%9B%9E%E5%BD%92"><span class="toc-number">1.1.</span> <span class="toc-text">线性回归</span></a><ol class="toc-child"><li class="toc-item toc-level-3"><a class="toc-link" href="#%E6%A2%AF%E5%BA%A6%E4%B8%8B%E9%99%8D"><span class="toc-number">1.1.1.</span> <span class="toc-text">梯度下降</span></a></li><li class="toc-item toc-level-3"><a class="toc-link" href="#%E5%A4%9A%E5%85%83%E7%BA%BF%E6%80%A7%E5%9B%9E%E5%BD%92"><span class="toc-number">1.1.2.</span> <span class="toc-text">多元线性回归</span></a></li><li class="toc-item toc-level-3"><a class="toc-link" href="#%E7%89%B9%E5%BE%81%E7%BC%A9%E6%94%BE"><span class="toc-number">1.1.3.</span> <span class="toc-text">特征缩放</span></a></li><li class="toc-item toc-level-3"><a class="toc-link" href="#%E7%89%B9%E5%BE%81%E5%B7%A5%E7%A8%8B"><span class="toc-number">1.1.4.</span> <span class="toc-text">特征工程</span></a></li></ol></li><li class="toc-item toc-level-2"><a class="toc-link" href="#%E5%88%86%E7%B1%BB-%E9%80%BB%E8%BE%91%E5%9B%9E%E5%BD%92"><span class="toc-number">1.2.</span> <span class="toc-text">分类-逻辑回归</span></a><ol class="toc-child"><li class="toc-item toc-level-3"><a class="toc-link" href="#sigmoid%E5%87%BD%E6%95%B0"><span class="toc-number">1.2.1.</span> <span class="toc-text">sigmoid函数</span></a></li><li class="toc-item toc-level-3"><a class="toc-link" href="#%E5%86%B3%E7%AD%96%E8%BE%B9%E7%95%8C"><span class="toc-number">1.2.2.</span> <span class="toc-text">决策边界</span></a></li><li class="toc-item toc-level-3"><a class="toc-link" href="#%E6%88%90%E6%9C%AC%E5%87%BD%E6%95%B0"><span class="toc-number">1.2.3.</span> <span class="toc-text">成本函数</span></a></li><li class="toc-item toc-level-3"><a class="toc-link" href="#%E6%A2%AF%E5%BA%A6%E4%B8%8B%E9%99%8D-1"><span class="toc-number">1.2.4.</span> <span class="toc-text">梯度下降</span></a></li><li class="toc-item toc-level-3"><a class="toc-link" href="#%E8%BF%87%E6%8B%9F%E5%90%88%E9%97%AE%E9%A2%98"><span class="toc-number">1.2.5.</span> <span class="toc-text">过拟合问题</span></a></li></ol></li></ol></li></ol>
 </div>
 
     <div class="e-content article-entry" itemprop="articleBody">
 
-        <h1 id="machine-learning">Machine Learning</h1>
+        <h1 id="course1">Course1</h1>
 <p>监督学习<span class="bd-box"><h-char class="bd bd-beg"><h-inner>：</h-inner></h-char></span>输入特征x<span class="bd-box"><h-char class="bd bd-beg"><h-inner>，</h-inner></h-char></span>输出目标y<span class="bd-box"><h-char class="bd bd-beg"><h-inner>。</h-inner></h-char></span>对数据集进行预测<span class="bd-box"><h-char class="bd bd-beg"><h-inner>，</h-inner></h-char></span>分为<strong>回归</strong>和<strong>分类</strong></p>
 <p>无监督学习<span class="bd-box"><h-char class="bd bd-beg"><h-inner>：</h-inner></h-char></span>输入特征x<span class="bd-box"><h-char class="bd bd-beg"><h-inner>，</h-inner></h-char></span>没有目标y<span class="bd-box"><h-char class="bd bd-beg"><h-inner>，</h-inner></h-char></span>对数据集进行聚类预测</p>
 <h2 id="线性回归">线性回归</h2>
@@ -120,6 +120,62 @@ <h3 id="多元线性回归">多元线性回归</h3>
 <p><code>·</code>为两个向量的点积(dot)<span class="bd-box"><h-char class="bd bd-beg"><h-inner>。</h-inner></h-char></span><span class="markdown-them-math-inline">$\vec{w} \cdot \vec{x} = w_1*x_1+w_2*x_2+....+w_n*x_n$</span></p>
 <p><strong>矢量化</strong><span class="bd-box"><h-char class="bd bd-beg"><h-inner>：</h-inner></h-char></span>代码简洁<span class="bd-box"><h-char class="bd bd-beg"><h-inner>、</h-inner></h-char></span>运行速度快</p>
 <p>PS: 正规方程<span class="bd-box"><h-char class="bd bd-beg"><h-inner>：</h-inner></h-char></span>某些机器学习库在后端求<span class="markdown-them-math-inline">$w,b$</span>的方法<span class="bd-box"><h-char class="bd bd-beg"><h-inner>，</h-inner></h-char></span>只适用于线性回归<span class="bd-box"><h-char class="bd bd-beg"><h-inner>，</h-inner></h-char></span>而且速度慢<span class="bd-box"><h-char class="bd bd-beg"><h-inner>，</h-inner></h-char></span>不要求掌握</p>
+<h3 id="特征缩放">特征缩放</h3>
+<p>加快梯度下降速度</p>
+<p>避免特征的取值范围差异过大<span class="bd-box"><h-char class="bd bd-beg"><h-inner>，</h-inner></h-char></span>将其进行缩放</p>
+<ul>
+<li>
+<p>除以最大值<span class="bd-box"><h-char class="bd bd-beg"><h-inner>，</h-inner></h-char></span><span class="markdown-them-math-inline">$x_{1,scale} = \frac{x_1}{max}$</span></p>
+</li>
+<li>
+<p>均值归一化</p>
+<ul>
+<li>求均值<span class="markdown-them-math-inline">$\mu$</span></li>
+<li><span class="markdown-them-math-inline">$x_1 = \frac{x_1-\mu}{max-min}$</span></li>
+</ul>
+</li>
+<li>
+<p><code>Z-score</code>归一化</p>
+<ul>
+<li>求标准差<span class="markdown-them-math-inline">$\sigma$</span><span class="bd-box"><h-char class="bd bd-beg"><h-inner>，</h-inner></h-char></span>均值<span class="markdown-them-math-inline">$\mu$</span></li>
+<li><span class="markdown-them-math-inline">$x_1 = \frac{x_1-\mu}{\sigma}$</span></li>
+</ul>
+</li>
+</ul>
+<p><strong>选择合适学习率</strong><span class="bd-box"><h-char class="bd bd-beg"><h-inner>：</h-inner></h-char></span>从0.001开始<span class="bd-box"><h-char class="bd bd-beg"><h-inner>，</h-inner></h-char></span>每次乘以3<span class="bd-box"><h-char class="bd bd-beg"><h-inner>，</h-inner></h-char></span>对比<span class="markdown-them-math-inline">$J(w,b)$</span>与迭代次数的关系<span class="bd-box"><h-char class="bd bd-beg"><h-inner>，</h-inner></h-char></span>选择合适的<span class="markdown-them-math-inline">$\alpha$</span></p>
+<h3 id="特征工程">特征工程</h3>
+<p>利用直觉设计新特征<span class="bd-box"><h-char class="bd bd-beg"><h-inner>，</h-inner></h-char></span>通常通过转化与组合<span class="bd-box"><h-char class="bd bd-beg"><h-inner>，</h-inner></h-char></span>使模型做出更准确的预测</p>
+<p><strong>多项式回归</strong><span class="bd-box"><h-char class="bd bd-beg"><h-inner>：</h-inner></h-char></span>可以添加<span class="markdown-them-math-inline">$x^q$</span>项更好地拟合数据图像<span class="bd-box"><h-char class="bd bd-beg"><h-inner>，</h-inner></h-char></span><span class="markdown-them-math-inline">$f(x)=w_1x^3+w_2x^2+w_1x^1+b$</span></p>
+<h2 id="分类-逻辑回归">分类-逻辑回归</h2>
+<p>解决二分类问题</p>
+<h3 id="sigmoid函数">sigmoid函数</h3>
+<p>输出介于<span class="markdown-them-math-inline">$(0,1)$</span></p>
+<p><span class="markdown-them-math-inline">$g(z)= \frac{1}{1+e^{-z}},z \subseteq R$</span></p>
+<p><span class="markdown-them-math-inline">$f_{\vec{w},b}(\vec{x})=g(\vec{w} · \vec{x}+b) = \frac{1}{1+e^{-(\vec{w} · \vec{x}+b)}}$</span></p>
+<h3 id="决策边界">决策边界</h3>
+<p>以0.5作为阈值<span class="bd-box"><h-char class="bd bd-beg"><h-inner>，</h-inner></h-char></span>当<span class="markdown-them-math-inline">$\vec{w} · \vec{x}+b \ge 0$</span><span class="bd-box"><h-char class="bd bd-beg"><h-inner>，</h-inner></h-char></span>取值1<span class="bd-box"><h-char class="bd bd-beg"><h-inner>；</h-inner></h-char></span>当<span class="markdown-them-math-inline">$\vec{w} · \vec{x}+b &lt;0$</span><span class="bd-box"><h-char class="bd bd-beg"><h-inner>，</h-inner></h-char></span>取值0</p>
+<p><span class="markdown-them-math-inline">$\vec{w} · \vec{x}+b = 0$</span>称为决策边界</p>
+<p>也适用于多项式回归</p>
+<h3 id="成本函数">成本函数</h3>
+<p>如果使用平方误差成本函数<span class="bd-box"><h-char class="bd bd-beg"><h-inner>，</h-inner></h-char></span>有多个局部最小值<span class="bd-box"><h-char class="bd bd-beg"><h-inner>，</h-inner></h-char></span><span class="markdown-them-math-inline">$J(w,b)$</span>是不是凸函数<span class="bd-box"><h-char class="bd bd-beg"><h-inner>，</h-inner></h-char></span>不适用于逻辑回归</p>
+<p>定义<span class="markdown-them-math-inline">$J(w,b)=\frac{1}{m}\sum_{i-1}^{m}L(f_{w,b}(x^{(i)},y^{(i)})$</span></p>
+<p>其中</p>
+<p><span class="markdown-them-math-inline">$L(f_{w,b}(x^{(i)},y^{(i)})=-log(f_{w,b}(x^{(i)})) \quad if \quad y^{(i)}=1$</span></p>
+<p><span class="markdown-them-math-inline">$L(f_{w,b}(x^{(i)},y^{(i)})=-log(1-f_{w,b}(x^{(i)})) \quad if \quad y^{(i)}=0$</span></p>
+<p><strong>简化</strong>成本函数</p>
+<p><span class="markdown-them-math-inline">$L(f_{w,b}(x^{(i)},y^{(i)})=-y^{(i)} log(f_{w,b}(x^{(i)})) - (1-y^{(i)})log(1-f_{w,b}(x^{(i)}))$</span></p>
+<p>得到</p>
+<p><span class="markdown-them-math-inline">$J(w,b) = -\frac{1}{m} (y^{(i)} log(f_{w,b}(x^{(i)})) + (1-y^{(i)})log(1-f_{w,b}(x^{(i)})))$</span></p>
+<p>使得成本函数是凸函数<span class="bd-box"><h-char class="bd bd-beg"><h-inner>，</h-inner></h-char></span>便于实现梯度下降</p>
+<h3 id="梯度下降-1">梯度下降</h3>
+<p>对导数项分别求导</p>
+<p><span class="markdown-them-math-inline">$\frac{\partial{J(w,b)}}{\partial{w}} = \frac{1}{m} \sum_{i=1}^{m} (f(x^i)-y^i)x^i$</span></p>
+<p><span class="markdown-them-math-inline">$\frac{\partial{J(w,b)}}{\partial{b}} = \frac{1}{m} \sum_{i=1}^{m} (f(x^i)-y^i)$</span></p>
+<p>其中<span class="markdown-them-math-inline">$f(x^i) =  \frac{1}{1+e^{-(\vec{w} · \vec{x}+b)}}$</span></p>
+<p>可以使用相似方法进行特征缩放</p>
+<h3 id="过拟合问题">过拟合问题</h3>
+<p>过拟合虽然可能完美通过训练集<span class="bd-box"><h-char class="bd bd-beg"><h-inner>，</h-inner></h-char></span>但是有高方差<span class="bd-box"><h-char class="bd bd-beg"><h-inner>。</h-inner></h-char></span>应该避免欠拟合<span class="bd-box"><h-char class="bd bd-end"><h-inner>（</h-inner></h-char></span>高偏差<span class="bd-box"><h-char class="bd bd-beg"><h-inner>）</h-inner></h-char></span>和过拟合<span class="bd-box"><h-char class="bd bd-end"><h-inner>（</h-inner></h-char></span>高方差<span class="bd-box"><h-char class="bd bd-beg"><h-inner>）</h-inner></h-char><h-char class="bd bd-beg"><h-inner>。</h-inner></h-char></span></p>
+<p><strong>解决过拟合</strong></p>
 
 
     </div>