电脑基础 · 2023年4月19日

sigmoid函数求导

sigmoid函数求导

      • 推导过程

sigmoid函数:
f
(
x
)
=
1
1
+
e

x
f(x)= \frac{1}{1+e^{-x}}
f(x)=1+ex1

sigmoid函数的导数:
f

(
x
)
=
f
(
x
)
(
1

f
(
x
)
)
f'(x)=f(x)(1-f(x))
f(x)=f(x)(1f(x))

推导过程

  • 首先,对
    f
    (
    x
    )
    f(x)
    f(x)
    进行变形:

    f
    (
    x
    )
    =
    1
    1
    +
    e

    x
    =
    1
    1
    +
    1
    e
    x
    =
    (
    1
    +
    1
    e
    x
    )

    1
    =
    (
    e
    x
    e
    x
    +
    1
    e
    x
    )

    1
    =
    (
    e
    x
    +
    1
    e
    x
    )

    1
    =
    e
    x
    e
    x
    +
    1
    =
    (
    e
    x
    +
    1
    )

    1
    e
    x
    +
    1
    =
    e
    x
    +
    1
    e
    x
    +
    1

    1
    e
    x
    +
    1
    =
    1

    1
    e
    x
    +
    1
    =
    1

    (
    e
    x
    +
    1
    )

    1
    \begin{aligned} f(x)&= \frac{1}{1+e^{-x}} \\ &= \frac{1}{1+\frac{1}{e^x}} \\ &=(1+\frac{1}{e^x})^{-1} \\ &=(\frac{e^x}{e^x}+\frac{1}{e^x})^{-1} \\ &=(\frac{e^{x}+1}{e^x})^{-1} \\ &=\frac{e^x}{e^{x}+1} \\ &=\frac{(e^{x}+1)-1}{e^{x}+1} \\ &=\frac{e^{x}+1}{e^{x}+1}-\frac{1}{e^{x}+1} \\ &=1-\frac{1}{e^{x}+1} \\ &=1-(e^{x}+1)^{-1} \end{aligned}
    f(x)=1+ex1=1+ex11=(1+ex1)1=(exex+ex1)1=(exex+1)1=ex+1ex=ex+1(ex+1)1=ex+1ex+1ex+11=1ex+11=1(ex+1)1
  • 求导:

注意使用链式法则求导


f

(
x
)
=
(
1

(
e
x
+
1
)

1
)

=
(

1
)
(

1
)
(
e
x
+
1
)

2
e
x
=
(
e
x
+
1
)

2
e
x
=
(
e
x
+
1
)

1
(
e
x
+
1
)

1
e
x
\begin{aligned} f'(x)&=(1-(e^{x}+1)^{-1})' \\ &=(-1)(-1)(e^{x}+1)^{-2} e^{x}\\ &=(e^{x}+1)^{-2} e^{x}\\ &=(e^{x}+1)^{-1}(e^{x}+1)^{-1} e^{x} \end{aligned}
f(x)=(1(ex+1)1)=(1)(1)(ex+1)2ex=(ex+1)2ex=(ex+1)1(ex+1)1ex

由前面提到的
f
(
x
)
f(x)
f(x)
的变形可知:

f
(
x
)
=
1
1
+
e

x
=
(
1
+
e

x
)

1
=
e
x
e
x
+
1
=
e
x
(
e
x
+
1
)

1
\begin{aligned} f(x)&=\frac{1}{1+e^{-x}} =(1+e^{-x})^{-1}=\frac{e^{x}}{e^{x}+1}=e^{x}(e^{x}+1)^{-1} \end{aligned}
f(x)=1+ex1=(1+ex)1=ex+1ex=ex(ex+1)1

所以

f

(
x
)
=
(
e
x
+
1
)

1

(
e
x
+
1
)

1
e
x
=
(
e
x
+
1
)

1

e
x
(
e
x
+
1
)

1
=
(
e
x
+
1
)

1

(
1
+
e

x
)

1
=
1
e
x
+
1

1
1
+
e

x
=
(
e
x
+
1
)

e
x
e
x
+
1

1
1
+
e

x
=
(
e
x
+
1
e
x
+
1

e
x
e
x
+
1
)

1
1
+
e

x
=
(
1

e
x
e
x
+
1
)

1
1
+
e

x
=
(
1

1
1
+
e

x
)

1
1
+
e

x
=
(
1

f
(
x
)
)

f
(
x
)
=
f
(
x
)
(
1

f
(
x
)
)
\begin{aligned} f'(x)&=(e^{x}+1)^{-1} \cdot (e^{x}+1)^{-1} e^{x} \\ &= (e^{x}+1)^{-1} \cdot e^{x}(e^{x}+1)^{-1} \\ &=(e^{x}+1)^{-1} \cdot (1+e^{-x})^{-1} \\ &=\frac{1}{e^{x}+1} \cdot \frac{1}{1+e^{-x}} \\ &=\frac{(e^{x}+1)-e^{x}}{e^{x}+1} \cdot \frac{1}{1+e^{-x}} \\ &=(\frac{e^{x}+1}{e^{x}+1}-\frac{e^{x}}{e^{x}+1}) \cdot \frac{1}{1+e^{-x}} \\ &=(1-\frac{e^{x}}{e^{x}+1}) \cdot \frac{1}{1+e^{-x}} \\ &=(1-\frac{1}{1+e^{-x}}) \cdot \frac{1}{1+e^{-x}} \\ &=(1-f(x)) \cdot f(x) \\ &=f(x)(1-f(x)) \end{aligned}
f(x)=(ex+1)1(ex+1)1ex=(ex+1)1ex(ex+1)1=(ex+1)1(1+ex)1=ex+111+ex1=ex+1(ex+1)ex1+ex1=(ex+1ex+1ex+1ex)1+ex1=(1ex+1ex)1+ex1=(11+ex1)1+ex1=(1f(x))f(x)=f(x)(1f(x))