index.xml

<?xml version="1.0" encoding="utf-8" standalone="yes"?>
<rss version="2.0" xmlns:atom="http://www.w3.org/2005/Atom">
  <channel>
    <title>ps126.5</title>
    <link>https://tjdoc.github.io/</link>
    <description>Recent content on ps126.5</description>
    <generator>Hugo -- gohugo.io</generator>
    <language>en-us</language>
    <lastBuildDate>Tue, 02 Feb 2021 15:22:09 +0900</lastBuildDate><atom:link href="https://tjdoc.github.io/index.xml" rel="self" type="application/rss+xml" />
    <item>
      <title>CNN m4w2</title>
      <link>https://tjdoc.github.io/posts/2021-02-02-coursera_m4w2/</link>
      <pubDate>Tue, 02 Feb 2021 15:22:09 +0900</pubDate>
      
      <guid>https://tjdoc.github.io/posts/2021-02-02-coursera_m4w2/</guid>
      <description>classic networks LeNet-5 AlexNet  many similarities with LeNet5 but much bigger. (10K parameters vs 60M parameters) Uses ReLU instead of traditional sigmoid, tanh Multiple GPUs local response normalization (not used much nowadays)  VGG-16 simplified the nn architechture
ResNets networks in networks and 1x1 convolutions  pulling layers reduce nh, nw 1x1 convolution lets you reduce (or increase) number of channels  inception network </description>
    </item>
    
    <item>
      <title>convolutional neural network</title>
      <link>https://tjdoc.github.io/posts/2021-02-01-coursera_m4w1/</link>
      <pubDate>Mon, 01 Feb 2021 11:37:08 +0900</pubDate>
      
      <guid>https://tjdoc.github.io/posts/2021-02-01-coursera_m4w1/</guid>
      <description>Computer vision convolution operator
summation of the elementwise multiplication with the convolution operator
 python: conv-forward tensorflow: tf.nn.conv2d keras: conv2d  padding with padding, you can
 prevent shrinking output prevent throwing away info from edge padding can be as large as you want  Valid and same convolutions  valid: no padding same: pad so that output size is the same as the input size  you may want odd number filters</description>
    </item>
    
    <item>
      <title>m2w3p tesorflow tutorial</title>
      <link>https://tjdoc.github.io/posts/2021-01-28-coursera_m2w3p/</link>
      <pubDate>Thu, 28 Jan 2021 23:52:40 +0900</pubDate>
      
      <guid>https://tjdoc.github.io/posts/2021-01-28-coursera_m2w3p/</guid>
      <description>tf_utils.py import h5py import numpy as np import tensorflow as tf import math def load_dataset(): train_dataset = h5py.File(&amp;#39;datasets/train_signs.h5&amp;#39;, &amp;#34;r&amp;#34;) train_set_x_orig = np.array(train_dataset[&amp;#34;train_set_x&amp;#34;][:]) # your train set features train_set_y_orig = np.array(train_dataset[&amp;#34;train_set_y&amp;#34;][:]) # your train set labels test_dataset = h5py.File(&amp;#39;datasets/test_signs.h5&amp;#39;, &amp;#34;r&amp;#34;) test_set_x_orig = np.array(test_dataset[&amp;#34;test_set_x&amp;#34;][:]) # your test set features test_set_y_orig = np.array(test_dataset[&amp;#34;test_set_y&amp;#34;][:]) # your test set labels classes = np.array(test_dataset[&amp;#34;list_classes&amp;#34;][:]) # the list of classes train_set_y_orig = train_set_y_orig.reshape((1, train_set_y_orig.shape[0])) test_set_y_orig = test_set_y_orig.reshape((1, test_set_y_orig.</description>
    </item>
    
    <item>
      <title>Improving Deep Neural Networks M2W3</title>
      <link>https://tjdoc.github.io/posts/2021-01-28-coursera_m2w3/</link>
      <pubDate>Thu, 28 Jan 2021 18:19:55 +0900</pubDate>
      
      <guid>https://tjdoc.github.io/posts/2021-01-28-coursera_m2w3/</guid>
      <description>Hyperparameters in the order of importance
 alpha - most important   beta (momentum, default: 0.9) number of hidden units mini-batch size   number of layers learning rate decay beta1, beta2, epsilon (Adam optimization, 0.9, 0.999, 10^-8)  Tuning process  Try random sampling, don&amp;rsquo;t use grid  it is difficult to know in advance which setting will work better   use coarse to fine sampling once you have some idea which range works better - sample that region more densely  picking hyperparameters at random  uniformly random (random version of np.</description>
    </item>
    
    <item>
      <title>coursera m2w2p</title>
      <link>https://tjdoc.github.io/posts/2021-01-27-coursera_m2w2p/</link>
      <pubDate>Wed, 27 Jan 2021 16:16:06 +0900</pubDate>
      
      <guid>https://tjdoc.github.io/posts/2021-01-27-coursera_m2w2p/</guid>
      <description>op utils import numpy as np import matplotlib.pyplot as plt import h5py import scipy.io import sklearn import sklearn.datasets def sigmoid(x): &amp;#34;&amp;#34;&amp;#34; Compute the sigmoid of x Arguments: x -- A scalar or numpy array of any size. Return: s -- sigmoid(x) &amp;#34;&amp;#34;&amp;#34; s = 1/(1+np.exp(-x)) return s def relu(x): &amp;#34;&amp;#34;&amp;#34; Compute the relu of x Arguments: x -- A scalar or numpy array of any size. Return: s -- relu(x) &amp;#34;&amp;#34;&amp;#34; s = np.</description>
    </item>
    
    <item>
      <title>Optimization Algorithms M2W2</title>
      <link>https://tjdoc.github.io/posts/2021-01-26-coursera_m2w2/</link>
      <pubDate>Tue, 26 Jan 2021 10:29:44 +0900</pubDate>
      
      <guid>https://tjdoc.github.io/posts/2021-01-26-coursera_m2w2/</guid>
      <description>mini batch gradient descent 1 epoch: pass through training set. In this case, 5000 gradient decsents.
training with mini batch gradient descent Mini batch Cost function is much more noisier than the batch gradient descent, but nonetheless the overall trend will decrease.
choosing your mini-batch size  mini-batch size: m \(\implies\) Batch gradient descent  Smooth convergence of the cost Takes too long per iteration   mini-batch size: 1 \(\implies\) Stochastic gradient descent  Does not completely converge (noisy cost minimization) Loses speedup from vectorization   mini-batch size: between 1 and m \(\implies\) best performance in practice  Fastest learning Vectorization (~1000 batch size) Make progress without passing the entire training set    Guidelines for choosing your mini-batch size</description>
    </item>
    
    <item>
      <title>coursera m2w1p</title>
      <link>https://tjdoc.github.io/posts/2021-01-25-coursera_m2w1p/</link>
      <pubDate>Mon, 25 Jan 2021 16:53:28 +0900</pubDate>
      
      <guid>https://tjdoc.github.io/posts/2021-01-25-coursera_m2w1p/</guid>
      <description>Initialization init util import numpy as np import matplotlib.pyplot as plt import h5py import sklearn import sklearn.datasets def sigmoid(x): &amp;#34;&amp;#34;&amp;#34; Compute the sigmoid of x Arguments: x -- A scalar or numpy array of any size. Return: s -- sigmoid(x) &amp;#34;&amp;#34;&amp;#34; s = 1/(1+np.exp(-x)) return s def relu(x): &amp;#34;&amp;#34;&amp;#34; Compute the relu of x Arguments: x -- A scalar or numpy array of any size. Return: s -- relu(x) &amp;#34;&amp;#34;&amp;#34; s = np.</description>
    </item>
    
    <item>
      <title>Improving Deep Neural Networks m2w1</title>
      <link>https://tjdoc.github.io/posts/2021-01-24-coursera_m2w1/</link>
      <pubDate>Mon, 25 Jan 2021 10:10:50 +0900</pubDate>
      
      <guid>https://tjdoc.github.io/posts/2021-01-24-coursera_m2w1/</guid>
      <description>Train / dev / test sets Previously:
 Train: Test = 70% : 30% Train : dev : test = 60% : 20% : 20%  However, in the big data era where you have more than 1 million sets
 Train : dev : test = 1,000,000 : 10,000 : 10,000 Train : dev : test = 99.5% : 0.25% : 0.25% Train : dev : test = 99.5% : 0.</description>
    </item>
    
    <item>
      <title>coursera m1w4p2 review</title>
      <link>https://tjdoc.github.io/posts/2021-01-23-m1w4p2_review/</link>
      <pubDate>Sat, 23 Jan 2021 20:02:24 +0900</pubDate>
      
      <guid>https://tjdoc.github.io/posts/2021-01-23-m1w4p2_review/</guid>
      <description>python stuff learned elementwise min max for relu, you use elementwise maximum function which can be done by
np.maximum / np.minimum  np.dot for a given numpy array a and b, for a dot product, both expressions are identical
np.dot(a, b) a.dot(b)  decision boundary plot import matplotlib.pyplot as plt import numpy as np import sklearn import sklearn.datasets import sklearn.linear_model def plot_decision_boundary(model, X, y): # Set min and max values and give it some padding x_min, x_max = X[0, :].</description>
    </item>
    
    <item>
      <title>m1w4p2</title>
      <link>https://tjdoc.github.io/posts/2021-01-22-coursera_m1w4p2/</link>
      <pubDate>Fri, 22 Jan 2021 21:09:54 +0900</pubDate>
      
      <guid>https://tjdoc.github.io/posts/2021-01-22-coursera_m1w4p2/</guid>
      <description>import numpy as np import matplotlib.pyplot as plt import h5py def sigmoid(Z): &amp;#34;&amp;#34;&amp;#34; Implements the sigmoid activation in numpy Arguments: Z -- numpy array of any shape Returns: A -- output of sigmoid(z), same shape as Z cache -- returns Z as well, useful during backpropagation &amp;#34;&amp;#34;&amp;#34; A = 1/(1+np.exp(-Z)) cache = Z return A, cache def relu(Z): &amp;#34;&amp;#34;&amp;#34; Implement the RELU function. Arguments: Z -- Output of the linear layer, of any shape Returns: A -- Post-activation parameter, of the same shape as Z cache -- a python dictionary containing &amp;#34;A&amp;#34; ; stored for computing the backward pass efficiently &amp;#34;&amp;#34;&amp;#34; A = np.</description>
    </item>
    
    <item>
      <title>coursera m1w4 p1</title>
      <link>https://tjdoc.github.io/posts/2021-01-22-coursera_m1w4p1/</link>
      <pubDate>Fri, 22 Jan 2021 18:24:13 +0900</pubDate>
      
      <guid>https://tjdoc.github.io/posts/2021-01-22-coursera_m1w4p1/</guid>
      <description>dnn utility import numpy as np def sigmoid(Z): &amp;#34;&amp;#34;&amp;#34; Implements the sigmoid activation in numpy Arguments: Z -- numpy array of any shape Returns: A -- output of sigmoid(z), same shape as Z cache -- returns Z as well, useful during backpropagation &amp;#34;&amp;#34;&amp;#34; A = 1/(1+np.exp(-Z)) cache = Z return A, cache def relu(Z): &amp;#34;&amp;#34;&amp;#34; Implement the RELU function. Arguments: Z -- Output of the linear layer, of any shape Returns: A -- Post-activation parameter, of the same shape as Z cache -- a python dictionary containing &amp;#34;A&amp;#34; ; stored for computing the backward pass efficiently &amp;#34;&amp;#34;&amp;#34; A = np.</description>
    </item>
    
    <item>
      <title>Neural Networks and Deep Learning W4: Deepl L-layer neural network</title>
      <link>https://tjdoc.github.io/posts/2021-01-22-coursera_m1w4/</link>
      <pubDate>Fri, 22 Jan 2021 16:13:24 +0900</pubDate>
      
      <guid>https://tjdoc.github.io/posts/2021-01-22-coursera_m1w4/</guid>
      <description>Terminology  L number of layers \(n^{[l]}\) number of units in layer l \(a^{[l]}\) activations in layer l, \( a^{[l]} = g^{[l]} (z^{[l]}) \)  Forward propagation in deep network $$ \begin{aligned} Z^{[l]} =&amp;amp; W^{[l]} A^{[l-1]} + b^{[l]} \\
A^{[l]} =&amp;amp; g^{[l]} (Z^{[l]}) \end{aligned} $$
where \( A^{[0]} = X \)
You will need an explict for-loop for computing each layers.
Getting your matrix dimensions right $$ \begin{aligned} W^{[l]} &amp;amp;: (n^{[l]}, n^{[l-1]}) \\</description>
    </item>
    
    <item>
      <title>Take every thought captive</title>
      <link>https://tjdoc.github.io/posts/2021-01-20-2cor10/</link>
      <pubDate>Wed, 20 Jan 2021 10:09:43 +0900</pubDate>
      
      <guid>https://tjdoc.github.io/posts/2021-01-20-2cor10/</guid>
      <description>2cor 10:3. For although we do live in the world, we do not wage war in a worldly way; 2cor 10:4. because the weapons we use to wage war are not worldly. On the contrary, they have God’s power for demolishing strongholds. We demolish arguments 2cor 10:5. and every arrogance that raises itself up against the knowledge of God; we take every thought captive and make it obey the Messiah.</description>
    </item>
    
    <item>
      <title>M1W3 Code</title>
      <link>https://tjdoc.github.io/posts/2021-01-19-m1w3_code/</link>
      <pubDate>Tue, 19 Jan 2021 19:12:37 +0900</pubDate>
      
      <guid>https://tjdoc.github.io/posts/2021-01-19-m1w3_code/</guid>
      <description>Planar utils import matplotlib.pyplot as plt import numpy as np import sklearn import sklearn.datasets import sklearn.linear_model def plot_decision_boundary(model, X, y): # Set min and max values and give it some padding x_min, x_max = X[0, :].min() - 1, X[0, :].max() + 1 y_min, y_max = X[1, :].min() - 1, X[1, :].max() + 1 h = 0.01 # Generate a grid of points with distance h between them xx, yy = np.</description>
    </item>
    
    <item>
      <title>Neural Networks and Deep Learning W3: Shallow Neural Network</title>
      <link>https://tjdoc.github.io/posts/2021-01-19-coursera_m1w3/</link>
      <pubDate>Tue, 19 Jan 2021 10:27:23 +0900</pubDate>
      
      <guid>https://tjdoc.github.io/posts/2021-01-19-coursera_m1w3/</guid>
      <description>2 layer neural network Unlike logistic regression, neural network contains hidden layer(s). The figure above depicts a shallow neural network, 2 layer NN to be precise. The input layer (layer zero) does not count as a layer so the two layers are 1 hidden layer and the output layer. With the hidden layer, layer numbers are identified by the bracket superscript.
vectorizing across multiple examples activation functions If you don&amp;rsquo;t know which activation function works best, test them.</description>
    </item>
    
    <item>
      <title>Psalm 27</title>
      <link>https://tjdoc.github.io/posts/2021-01-19-ps27/</link>
      <pubDate>Tue, 19 Jan 2021 06:55:37 +0900</pubDate>
      
      <guid>https://tjdoc.github.io/posts/2021-01-19-ps27/</guid>
      <description>By David:
Adonai is my light and salvation;
whom do I need to fear?
Adonai is the stronghold of my life;
of whom should I be afraid?
When evildoers assailed me
to devour my flesh,
my adversaries and foes,
they stumbled and fell.
If an army encamps against me,
my heart will not fear;
if war breaks out against me,
even then I will keep trusting.
Just one thing have I asked of Adonai;</description>
    </item>
    
    <item>
      <title>Neural Networksand Deep Learning W2: Lab</title>
      <link>https://tjdoc.github.io/posts/2021-01-18-coursera_m1w2_lab/</link>
      <pubDate>Mon, 18 Jan 2021 18:42:17 +0900</pubDate>
      
      <guid>https://tjdoc.github.io/posts/2021-01-18-coursera_m1w2_lab/</guid>
      <description>packages import numpy as np import matplotlib.pyplot as plt import h5py import scipy from PIL import Image from scipy import ndimage from lr_utils import load_dataset %matplotlib inline load dataset import numpy as np import h5py def load_dataset(): train_dataset = h5py.File(&amp;#39;datasets/train_catvnoncat.h5&amp;#39;, &amp;#34;r&amp;#34;) train_set_x_orig = np.array(train_dataset[&amp;#34;train_set_x&amp;#34;][:]) # your train set features train_set_y_orig = np.array(train_dataset[&amp;#34;train_set_y&amp;#34;][:]) # your train set labels test_dataset = h5py.File(&amp;#39;datasets/test_catvnoncat.h5&amp;#39;, &amp;#34;r&amp;#34;) test_set_x_orig = np.array(test_dataset[&amp;#34;test_set_x&amp;#34;][:]) # your test set features test_set_y_orig = np.</description>
    </item>
    
    <item>
      <title>Coursera W2 Numpy Code</title>
      <link>https://tjdoc.github.io/posts/2021-01-18-coursera_m1w2_code/</link>
      <pubDate>Mon, 18 Jan 2021 14:57:04 +0900</pubDate>
      
      <guid>https://tjdoc.github.io/posts/2021-01-18-coursera_m1w2_code/</guid>
      <description>Sigmoid function import math import numpy as np def basic_sigmoid(x): return 1/(1+math.exp(-x)) def sigmoid(x) return 1/(1+np.exp(-x)) Derivative of sigmoid $$ \sigma &#39; (x) = \sigma(x)(1-\sigma(x)) $$
def sigmoid_derivative(x): s = sigmoid(x) ds = s*(1-s) return ds Reshaping arbitrary dimension numpy array to a column vector def image2vector(image): &amp;#34;&amp;#34;&amp;#34; Argument: image -- a numpy array of shape (length, height, depth) Returns: v -- a vector of shape (length*height*depth, 1) &amp;#34;&amp;#34;&amp;#34; # v = image.</description>
    </item>
    
    <item>
      <title>Galatians 6:9</title>
      <link>https://tjdoc.github.io/posts/2021-01-18-gal69/</link>
      <pubDate>Mon, 18 Jan 2021 00:18:49 +0900</pubDate>
      
      <guid>https://tjdoc.github.io/posts/2021-01-18-gal69/</guid>
      <description>NKRV gal 6:9 우리가 선을 행하되 낙심하지 말지니 포기하지 아니하면 때가 이르매 거두리라
CJB gal 6:9. So let us not grow weary of doing what is good; for if we don’t give up, we will in due time reap the harvest.
 Amen!</description>
    </item>
    
    <item>
      <title>Neural Networks and Deep Learning W2: Logistic Regression as a Neural Network</title>
      <link>https://tjdoc.github.io/posts/2021-01-15-coursera_m1w2/</link>
      <pubDate>Fri, 15 Jan 2021 10:52:30 +0900</pubDate>
      
      <guid>https://tjdoc.github.io/posts/2021-01-15-coursera_m1w2/</guid>
      <description>Logistic Regression Given x, we want \({\hat y} = P(y=1 |_x)\) where
input vector: \( x \in \Re^{n_x} \)
Parameters: \(w \in \Re^{n_x}\), \(b \in \Re \)
Output: \({\hat y} = \sigma (w^T x + b)\)
Here, \( \sigma(z) = \frac{1}{1+e^{-z}} \) is the sigmoid function that enforces \( 0 \le {\hat y} \le 1 \)
Given \( \{{ (x^{(1)} , y^{(1)}) , &amp;hellip;, (x^{(m)} , y^{(m)}) \}} \)
we want \( {\hat y}^{(i)} \approx y^{(i)} \)</description>
    </item>
    
    <item>
      <title>Psalm 20</title>
      <link>https://tjdoc.github.io/posts/2021-01-15-ps20/</link>
      <pubDate>Fri, 15 Jan 2021 08:47:21 +0900</pubDate>
      
      <guid>https://tjdoc.github.io/posts/2021-01-15-ps20/</guid>
      <description>For the leader. A psalm of David: May Adonai answer you in times of distress, may the name of the God of Ya‘akov protect you. May he send you help from the sanctuary and give you support from Tziyon. May he be reminded by all your grain offerings and accept the fat of your burnt offerings. (Selah) May he grant you your heart’s desire and bring all your plans to success.</description>
    </item>
    
    <item>
      <title>sleep</title>
      <link>https://tjdoc.github.io/posts/2021-01-15-sleep/</link>
      <pubDate>Fri, 15 Jan 2021 05:52:58 +0900</pubDate>
      
      <guid>https://tjdoc.github.io/posts/2021-01-15-sleep/</guid>
      <description>NKRV ps 127:2. 너희가 일찍이 일어나고 늦게 누우며 수고의 떡을 먹음이 헛되도다 그러므로 여호와께서 그의 사랑하시는 자에게는 잠을 주시는도다
  KJV ps 127:2. It is vain for you to rise up early, to sit up late, to eat the bread of sorrows: for so he giveth his beloved sleep.
  CJB ps 127:2. In vain do you get up early and put off going to bed, working hard to earn a living; for he provides for his beloved, even when they sleep.</description>
    </item>
    
    <item>
      <title>Neural Networks and Deep Learning W1: What is a neural network?</title>
      <link>https://tjdoc.github.io/posts/2021-01-14-coursera_m1w1/</link>
      <pubDate>Thu, 14 Jan 2021 21:06:56 +0900</pubDate>
      
      <guid>https://tjdoc.github.io/posts/2021-01-14-coursera_m1w1/</guid>
      <description>Terminology  x: input y: output Deep Learning: Same thing as Neural Network but sounds better brandwise  Application Standard Neural Network
 Real estate Online Advertising  Convolutional NN (CNN)
 Image data Photo tagging  Recurrent NN (RNN)
 Data invoving temporal component Speech recognition Machine transaltion  Custom Hybrid
 Automous driving  Data kinds  Supervised Data: Database Unstructured Data  audio image text    Why is deep learning taking off?</description>
    </item>
    
    <item>
      <title>Start</title>
      <link>https://tjdoc.github.io/posts/2021-01-14-start/</link>
      <pubDate>Thu, 14 Jan 2021 02:38:37 +0900</pubDate>
      
      <guid>https://tjdoc.github.io/posts/2021-01-14-start/</guid>
      <description>They that sow in tears shall reap in joy. ps 126:5
 start</description>
    </item>
    
  </channel>
</rss>