- Äɲ䵤줿¹Ô¤Ï¤³¤Î¿§¤Ç¤¹¡£
- ºï½ü¤µ¤ì¤¿¹Ô¤Ï¤³¤Î¿§¤Ç¤¹¡£
#author("2019-11-20T04:35:55+00:00","","")
#mynavi(µ¡³£³Ø½¬¤Î°Ù¤Î¿ô³Ø¤Î´ðÁÃ)
#setlinebreak(on);
#html(<script async src="https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/MathJax.js?config=TeX-MML-AM_CHTML"></script>)
* Ìܼ¡ [#d8db4f2c]
#contents
- ´ØÏ¢
-- [[µ¡³£³Ø½¬¤Î°Ù¤Î¿ô³Ø¤Î´ðÁÃ]]
-- [[Python³Ð¤¨½ñ¤]]
-- [[IPython¤Î»È¤¤Êý¥á¥â]]
-- [[numpyÆþÌç]]
-- [[pandasÆþÌç]]
-- [[matplotlib¤Ç¥°¥é¥ÕºîÀ®]]
-- [[scikit-learn¤ò»ÈÍѤ·¤¿½Å²óµ¢Ê¬ÀÏ]]
-- [[Chainer¤Ç½Å²óµ¢Ê¬ÀϤò¤ä¤Ã¤Æ¤ß¤ë]]
* ³µÍ× [#g644cbdc]
#html(<div style="padding-left: 10px;">)
#TODO
¤³¤³¤Ç¤Ï¡¢ºÇµÞ¹ß²¼Ë¡¤ò»ÈÍѤ·¤Æ½Å²óµ¢Ê¬ÀϤò½ÐÍè¤ë¤À¤±¥é¥¤¥Ö¥é¥êÅù¤ò»ÈÍѤ»¤º¤Ë¥¹¥¯¥é¥Ã¥Á¤Ç¼ÂÁõ¤·¤Æ¤ß¤ë¡£
#html(</div>)
* ¥â¥Ç¥ë¼° [#ff473705]
#html(<div style="padding-left: 10px;">)
* ¼ÂÁõ [#b1b871c1]
#html(){{
ñ²óµ¢Ê¬ÀϤΥâ¥Ç¥ë¼°¤¬¡¡
<math>
<mover><mo>y</mo><mo>^</mo></mover>
<mo>=</mo>
<msub><mo>w</mo><mi>1</mi></msub>
<mo>x</mo>
<mo>+</mo>
<msub><mo>w</mo><mi>0</mi></msub>
</math>
¡¡¤Ç¤¢¤ë¤Î¤ËÂФ·¤Æ¡¢
}}
#html(){{
<div>
ÆÃħÎ̤¬Ê£¿ô¤¢¤ë½Å²óµ¢Ê¬ÀϤξì¹ç¤Î¥â¥Ç¥ë¼°¤Ï¡¡
<math>
<mover><mo>y</mo><mo>^</mo></mover>
<mo>=</mo>
<msub><mo>w</mo><mi>1</mi></msub>
<msub><mo>x</mo><mi>1</mi></msub>
<mo>+</mo>
<msub><mo>w</mo><mi>2</mi></msub>
<msub><mo>x</mo><mi>2</mi></msub>
<mo>+</mo>
<msub><mo>w</mo><mi>0</mi></msub>
<msub><mo>x</mo><mi>0</mi></msub>
</math>
¡¡¤È¤Ê¤ë¡£¡Ê ¹ÔÎó·×»»ÍÑ¤Ë <math><msub><mo>x</mo><mi>0</mi></msub><mo>=</mo><mi>1</mi></math> ¤È¤¹¤ë ¡Ë
</div>
}}
°Ê²¼¤Î¤è¤¦¤Ê¥Ç¡¼¥¿¤¬¤¢¤ë»þ¡¢
| x1 | x2 | y |h
| 70 | 68 | 87 |
| 75 | 82 | 86 |
| 80 | 71 | 93 |
#html(){{
<div style="padding: 10px 0;">
1¹ÔÌÜ¤Ï <math>
<mover><mo>y</mo><mo>^</mo></mover>
<mo>=</mo>
<msub><mo>w</mo><mi>1</mi></msub>
<mo>*</mo>
<mi>70</mi>
<mo>+</mo>
<msub><mo>w</mo><mi>2</mi></msub>
<mo>*</mo>
<mi>68</mi>
<mo>+</mo>
<msub><mo>w</mo><mi>0</mi></msub>
<mo>*</mo>
<mi>1</mi>
</math> ¤È¤Ê¤ê¡¢
</div>
°Ê²¼¤Î¹ÔÎó·×»»¤Ç <math><mover><mo>y</mo><mo>^</mo></mover></math> ¤òµá¤á¤ë»ö¤¬¤Ç¤¤ë¡£
<div style="padding: 10px 0;">
<math>
<mfenced open="[" close="]">
<mtable>
<mtr><mtd><mn>70</mn></mtd><mtd><mn>68</mn></mtd><mn>1</mn></mtd></mtr>
<mtr><mtd><mn>75</mn></mtd><mtd><mn>82</mn></mtd><mn>1</mn></mtd></mtr>
<mtr><mtd><mn>80</mn></mtd><mtd><mn>71</mn></mtd><mn>1</mn></mtd></mtr>
</mtable>
</mfenced>
<mfenced open="[" close="]">
<mtable>
<mtr><mtd><mn><msub><mo>w</mo><mi>1</mi></msub></mn></mtd></mtr>
<mtr><mtd><mn><msub><mo>w</mo><mi>2</mi></msub></mn></mtd></mtr>
<mtr><mtd><mn><msub><mo>w</mo><mi>0</mi></msub></mn></mtd></mtr>
</mtable>
</mfenced>
<mo>=</mo>
<mfenced open="[" close="]">
<mtable>
<mtr><mtd><mi>70</mi><mo>*</mo><msub><mo>w</mo><mi>1</mi></msub><mo>+</mo><mi>68</mi><mo>*</mo><msub><mo>w</mo><mi>2</mi></msub><mo>+</mo><mi>1</mi><mo>*</mo><msub><mo>w</mo><mi>0</mi></msub></mtd></mtr>
<mtr><mtd><mi>75</mi><mo>*</mo><msub><mo>w</mo><mi>1</mi></msub><mo>+</mo><mi>82</mi><mo>*</mo><msub><mo>w</mo><mi>2</mi></msub><mo>+</mo><mi>1</mi><mo>*</mo><msub><mo>w</mo><mi>0</mi></msub></mtd></mtr>
<mtr><mtd><mi>80</mi><mo>*</mo><msub><mo>w</mo><mi>1</mi></msub><mo>+</mo><mi>71</mi><mo>*</mo><msub><mo>w</mo><mi>2</mi></msub><mo>+</mo><mi>1</mi><mo>*</mo><msub><mo>w</mo><mi>0</mi></msub></mtd></mtr>
</mtable>
</mfenced>
</math>
</div>
¤³¤Î°Ù¡¢ÆÃħÎ̤ιÔÎó¤òX¡¢¥Ñ¥é¥á¡¼¥¿¤Î¹ÔÎó¤òW¤È¤·¤¿¾ì¹ç¡¢°Ê²¼¤Î¥â¥Ç¥ë¼°¤Çɽ¤¹»ö¤¬½ÐÍè¤ë¡£
<div style="padding: 10px 0;">
<math>
<mover><mo>y</mo><mo>^</mo></mover>
<mo>=</mo>
<mo>X</mo>
<mo>W</mo>
</math>
</div>
}}
#html(</div>)
// ¥â¥Ç¥ë¼°
* ¥³¥¹¥È´Ø¿ô [#u868c2e4]
#html(<div style="padding-left: 10px;">)
¥³¥¹¥È´Ø¿ô¤Ï°Ê²¼¤ÎÄ̤ꡣ
#html(){{
<div>
<math>
<mrow>
<mo>j</mo><mo>(</mo><mo>W</mo><mo>)</mo>
<mo>=</mo>
</mrow>
<mfrac><mi>1</mi><mo>2m</mo></mfrac>
<munderover><mi>∑</mi><mrow><mi>i</mi><mo>=</mo><mn>1</mn></mrow><mi>m</mi></munderover>
<msup><mrow><mo>(</mo> <mover><mi>y</mi><mo>^</mo></mover> <mo>-</mo> <msub><mi>y</mi><mi>i</mi></msub> <mo>)</mo> </mrow><mi>2</mi></msup>
<mo>=</mo>
<mfrac><mi>1</mi><mo>2m</mo></mfrac>
<msup>
<mrow><mo>(</mo><mo>XW</mo><mo>-</mo><mo>y</mo><mo>)</mo></mrow>
<mo>T</mo>
</msup>
<mrow><mo>(</mo><mo>XW</mo><mo>-</mo><mo>y</mo><mo>)</mo></mrow>
</math>
</div>
}}
#html(</div>)
// ¥³¥¹¥È´Ø¿ô
* ºÇµÞ¹ß²¼Ë¡ [#g528f800]
#html(<div style="padding-left: 10px;">)
ºÇµÞ¹ß²¼Ë¡¡Ê¥Ù¥¯¥È¥ë²½¡Ë¤Î¥â¥Ç¥ë¼°¤Ï°Ê²¼¤ÎÄ̤ꡣ
#html(){{
<div>
<math>
<mo>W</mo>
<mo>:=</mo>
<mo>W</mo>
<mo>-</mo>
<mo>¦Á</mo>
<mfrac><mo>1</mo><mo>m</mo></mfrac>
<msup><mo>X</mo><mo>T</mo></msup>
<mo>(</mo><mo>XW</mo><mo>-</mo><mo>y</mo><mo>)</mo>
</math>
</div>
}}
#html(</div>)
// ºÇµÞ¹ß²¼Ë¡
* Àµµ¬²½ [#n73377f0]
#html(<div style="padding-left: 10px;">)
ÆÃħÎ̤ÎÈϰϤ¬Î󤴤Ȥ˰ۤʤë¾ì¹ç¡¢¥Ç¡¼¥¿¤òÀµµ¬²½¤¹¤ë»ö¤Ë¤è¤Ã¤Æ¡¢Àµ¤·¤¯Ê¬ÀϤò¹Ô¤¦»ö¤¬¤Ç¤¤ë¡£
** z-socre Normalization (ɸ½à²½) [#j3f49df1]
#html(<div style="padding-left: 10px;">)
Ê¿¶Ñ¤¬0¡¢É¸½àÊк¹¤¬1¤È¤Ê¤ë¤è¤¦¤ËÄ´À°¤¹¤ë¡£
#html(){{
<div style="padding-bottom: 10px;">
<math>
<msub><mo>x</mo><mi>1</mi></msub>
<mo>=</mo>
<mfrac>
<mrow><msub><mo>x</mo><mi>1</mi></msub><mo>-</mo><msub><mo>x</mo><mi>mean</mi></msub></mrow>
<msub><mo>x</mo><mi>std</mi></msub>
</mfrac>
</math>
</div>
}}
#html(</div>)
// z-socre Normalization
** min-max normalization [#o4c1e2e1]
#html(<div style="padding-left: 10px;">)
ºÇÂçÃͤ¬1ºÇ¾®Ãͤ¬0¤È¤Ê¤ë¤è¤¦¤ËÄ´À°¤¹¤ë¡£
#html(){{
<div>
<math>
<msub><mo>x</mo><mi>1</mi></msub>
<mo>=</mo>
<mfrac>
<mrow><msub><mo>x</mo><mi>1</mi></msub><mo>-</mo><msub><mo>x</mo><mi>mean</mi></msub></mrow>
<mrow><msub><mo>x</mo><mi>max</mi></msub><mo>-</mo><msub><mo>x</mo><mi>min</mi></msub></mrow>
</mfrac>
</math>
</div>
}}
#html(</div>)
// min-max normalization
#html(</div>)
// Àµµ¬²½
* ¥µ¥ó¥×¥ë¼ÂÁõ [#ddf2ff35]
#html(<div style="padding-left: 10px;">)
°Ê²¼¤Î¥Ç¡¼¥¿¤«¤é¡¢Ç¤°Õ¤ÎÉô²°ÌÌÀѤÈÃÛǯ¿ô¤Î»þ¤Ë²ÈĤ¬¤¤¤¯¤é¤Ë¤Ê¤ë¤«¤ò¡¢½Å²óµ¢Ê¬ÀϤò»ÈÍѤ·¤Æ¿äÏÀ¤¹¤ë½èÍý¤ò¼ÂÁõ¤¹¤ë¡£
|ÌÌÀÑ|ÃÛǯ¿ô|²ÈÄÂ|h
|40.24|7|6.2|
|72.48|4|9.3|
|72.48|4|9.3|
|43.85|0|7.5|
|88.33|0|13.3|
¢¨ &ref(sample_rent1.csv);
sample_rent1.py
#mycode2(python){{
"""½Å²óµ¢Ê¬ÀÏ¥µ¥ó¥×¥ë."""
import numpy as np
import pandas as pd
import matplotlib.pyplot as plt
from mpl_toolkits.mplot3d import Axes3D
# ¥Ç¡¼¥¿Æɤ߹þ¤ß
data = np.loadtxt("sample.csv", delimiter=",", skiprows=1)
def normalize(X, data=None):
"""Àµµ¬²½(Z-score normalization)."""
data = X if data is None else data
m = data.shape[0]
X_norm = np.zeros((data.shape[0], data.shape[1]))
for i in range(data.shape[1]):
X_norm[:, i] = (data[:, i] - float(np.mean(X[:, i]))) / float(np.std(X[:, i]))
x = data[:, :3]
y = data[:, 3]
#y = np.transpose([data[:, 3]])
m = len(y)
# x0¤òÄɲÃ
X_norm = np.column_stack([np.ones([m,1]), X_norm])
# Àµµ¬²½(Z-score normalization)
def norm(X):
X_norm = np.zeros((X.shape[0], X.shape[1]))
X_mean = np.zeros((1, X.shape[1]))
X_std = np.zeros((1, X.shape[1]))
for i in range(X.shape[1]):
X_mean[:, i] = np.mean(X[:, i])
X_std[:, i] = np.std(X[:, i])
X_norm[:, i] = (X[:, i] - float(X_mean[:, i])) / float(X_std[:, i])
return X_norm, X_mean, X_std
return X_norm
weight_int = np.zeros((4, 1))
def cost(x, y, w):
"""¥³¥¹¥È´Ø¿ô."""
xw = x.dot(w)
return np.dot((xw - y).T, (xw - y)) / (2*m)
# ¤â¤·¤¯¤Ï
#diff = np.power((x.dot(w) - y), 2)
#return diff.sum(axis=0) / (2 * len(y))
# ¥³¥¹¥È´Ø¿ô
def cost(x, y, weight):
def gradient_descent(x, y, w, alpha, iter_num):
"""ºÇµÞ¹ß²¼Ë¡."""
m = len(y)
J = 0
y_hut = x.dot(weight)
diff = np.power((y_hut - np.transpose([y])), 2)
#diff = np.power((y_hut - y), 2)
J = (1.0 / (2 * m)) * diff.sum(axis=0)
return J
costs = np.zeros((iter_num, 1))
for i in range(iter_num):
w = w - alpha * (1.0/m) * np.transpose(x).dot(x.dot(w) - y)
costs[i] = cost(x, y, w)
return w, costs
# ¥³¥¹¥È´Ø¿ô(¥Ù¥¯¥È¥ë²½ÈÇ)
def cost2(x, y, weight):
m = len(y)
y_shaped = np.transpose([y])
xw = x.dot(weight)
result = np.dot((xw - y_shaped).T, (xw - y_shaped)) / (2*m)
#J = (1.0 / (2 * m)) * result
return result
if __name__ == "__main__":
def gradient_descent(x, y, weight, alpha, iter_num):
# --------------------------
# ºÇµÞ¹ß²¼Ë¡¤Ë¤è¤ë½Å²óµ¢Ê¬ÀÏ
# --------------------------
# ¥Ç¡¼¥¿Æɤ߹þ¤ß
data = np.loadtxt("data/sample_rent1.csv", delimiter=",", skiprows=1)
x = data[:, 1:3]
y = data[:, 3:4]
# ¥Ç¡¼¥¿¤Î¸Ä¿ô
m = len(y)
j_hist = np.zeros((iter_num, 1))
for i in range(iter_num):
weight = weight - alpha * (1.0/m) * np.transpose(x).dot(x.dot(weight) - np.transpose([y]))
j_hist[i] = cost(x, y, weight)
return weight, j_hist
# Àµµ¬²½¤·¡¢x0¤òÄɲÃ
X_norm = normalize(x)
if __name__ == "__main__":
# ½é´üÃÍ
w_int = np.zeros((3, 1))
# ³Ø½¬Î¨¡¢³Ø½¬²ó¿ô
# ³Ø½¬Î¨
alpha = 0.01
iter_num = 500
# ·±Îý¥Ç¡¼¥¿¤òÀµµ¬²½¤·¤Æ¡¢x0¤ÎÎó¤òÄɲ乤ë
x_padded = np.column_stack([np.ones([m,1]), X_norm])
# ³Ø½¬²ó¿ô
iter_num = 1000
# Àµµ¬²½¤·¤¿¥Ç¡¼¥¿¤ò»ÈÍѤ·¤ÆºÇµÞ¹ß²¼Ë¡¤Ç W ¤òµá¤á¤ë
weight, j_hist = gradient_descent(x_padded, y, weight_int, alpha, iter_num)
# ºÇµÞ¹ß²¼Ë¡¤Ë¤è¤ëʬÀϤμ¹Ô
w, costs = gradient_descent(X_norm, y, w_int, alpha, iter_num)
print("weight: ", weight)
# --------------------------
# ºîÀ®¤·¤¿¥â¥Ç¥ë¤ò»ÈÍѤ·¤ÆÊ̤Υǡ¼¥¿¤òͽ¬¤·¤Æ¤ß¤ë
# --------------------------
z = np.array([[60, 10], [50, 10], [40, 10]])
result = normalize(x, z).dot(w)
for i in range(z.shape[0]):
print("¹¤µ: {}Ö, ÃÛǯ¿ô: {}ǯ ¢â {:0.1f}Ëü±ß".format(z[i,0], z[i,1], result[i,0]))
# ³Ø½¬Ëè¤Î¥³¥¹¥È¤Î²Ä»ë²½
plt.plot(range(j_hist.size), j_hist[:, 0], "r")
plt.xlabel("number of iterations")
plt.ylabel("cost J")
plt.grid(True)
# --------------------------
# ¥°¥é¥Õɽ¼¨
# --------------------------
fig = plt.figure(figsize=(10, 5))
# 3D¥°¥é¥Õ
ax = fig.add_subplot(1, 2, 1, projection='3d')
ax.scatter(data[:, 1], data[:, 2], data[:, 3], color="#ef1234")
ax.set_xlabel("Size")
ax.set_ylabel("Age")
ax.set_zlabel("Rent")
# ³Ø½¬Ëè¤Î¥³¥¹¥È
ax2 = fig.add_subplot(1, 2, 2)
ax2.plot(range(costs.size), costs[:, 0], "r")
ax2.set_xlabel("iterations")
ax2.set_ylabel("cost")
ax2.grid(True)
plt.show()
}}
·ë²Ì
#myterm2(){{
¹¤µ: 60Ö, ÃÛǯ¿ô: 10ǯ ¢â 8.1Ëü±ß
¹¤µ: 50Ö, ÃÛǯ¿ô: 10ǯ ¢â 7.2Ëü±ß
¹¤µ: 40Ö, ÃÛǯ¿ô: 10ǯ ¢â 6.3Ëü±ß
}}
&ref(sample_rent1.png);
#html(</div>)