关于人工智能:2Softmax回归

Softmax回归

LR解决的是二分类问题，Softmax用于多分类场景。假如数据样本为：
$\{ ({x_1},{y_1}),({x_2},{y_2}),…,({x_m},{y_m})\}$，其中${y_i} \in \{ 1,2,…,K\}$，有K个类别，${y_i}$为j的概率是：

$$
p({y_i} = j|{x_i};\theta ) = \frac{{{e^{\theta _j^T{x_i}}}}}{{\sum\limits_{k = 1}^K {{e^{\theta _k^T{x_i}}}} }}
$$

预计每一类的概率为：

$$
{h_\theta }({x_i}) = \left[ {\begin{array}{c}
{p({y_i} = 1|{x_i};\theta )}\\
{p({y_i} = 2|{x_i};\theta )}\\
{…}\\
{p({y_i} = k|{x_i};\theta )}
\end{array}} \right] = \frac{1}{{\sum\limits_{k = 1}^K {{e^{\theta _k^T{x_i}}}} }}\left[ {\begin{array}{c}
{{e^{\theta _1^T{x_i}}}}\\
{{e^{\theta _2^T{x_i}}}}\\
{…}\\
{{e^{\theta _K^T{x_i}}}}
\end{array}} \right]
$$

最终要求的是参数${\theta _1},{\theta _2},…,{\theta _k}$,代价函数：

$$
\begin{array}{l}
L(\theta ) = – \frac{1}{m}[\sum\limits_{i = 1}^m {\sum\limits_{j = 1}^k {I({y_i} = j)\log \frac{{{e^{\theta _j^T{x_i}}}}}{{\sum\limits_{k = 1}^K {{e^{\theta _k^T{x_i}}}} }}} } ],\\
I({y_i} = j) = \left\{ {\begin{array}{c}
{1,{y_i} \in j}\\
{0,{y_i} \notin j}
\end{array}} \right.
\end{array}
$$

同样用梯度降落法：

$$
\frac{{\partial L(\theta )}}{{\partial {\theta _j}}} = – \frac{1}{m}\sum\limits_{i = 1}^m {{x_i}(I({y_i} = j) – p({y_i} = j|{x_i};\theta ))}
$$

Softmax的参数有冗余。