Visualizing and Understanding2014-中文译文 3.78MB

p731heminyang

资源文件列表:

Visualizing and Understanding2014_中文译文.zip 大约有2个文件

Visualizing and Understanding.pdf 2.25MB
Visualizing and Understanding2014_中文译文.docx 1.81MB

资源介绍:

dfNet深度卷积网络论文

<link href="/image.php?url=https://csdnimg.cn/release/download_crawler_static/css/base.min.css" rel="stylesheet"/><link href="/image.php?url=https://csdnimg.cn/release/download_crawler_static/css/fancy.min.css" rel="stylesheet"/><link href="/image.php?url=https://csdnimg.cn/release/download_crawler_static/89567205/raw.css" rel="stylesheet"/><div id="sidebar" style="display: none"><div id="outline"></div></div><div class="pf w0 h0" data-page-no="1" id="pf1"><div class="pc pc1 w0 h0"><img alt="" class="bi x0 y0 w1 h1" src="/image.php?url=https://csdnimg.cn/release/download_crawler_static/89567205/bg1.jpg"/><div class="t m0 x1 h2 y1 ff1 fs0 fc0 sc0 ls0 ws0">Visualizing<span class="_ _0"> </span>and<span class="_ _0"> </span>Understanding</div><div class="t m0 x2 h2 y2 ff1 fs0 fc0 sc0 ls1 ws0">Con<span class="_ _1"></span>v<span class="_ _1"></span>olutional<span class="_ _0"> </span>Net<span class="_ _1"></span>w<span class="_ _1"></span>orks</div><div class="t m0 x3 h3 y3 ff2 fs1 fc0 sc0 ls2 ws0">Matthew<span class="_ _2"> </span>D.<span class="_ _2"> </span>Zeiler<span class="_ _2"> </span>and<span class="_ _2"> </span>Rob<span class="_ _2"> </span>F<span class="_ _3"></span>ergus</div><div class="t m0 x4 h4 y4 ff3 fs2 fc0 sc0 ls3 ws0">Dept.<span class="_ _4"> </span>of<span class="_ _4"> </span>Com<span class="_ _1"></span>puter<span class="_ _4"> </span>Science,</div><div class="t m0 x5 h4 y5 ff3 fs2 fc0 sc0 ls4 ws0">New<span class="_ _4"> </span>Y<span class="_ _5"></span>ork<span class="_ _4"> </span>Univ<span class="_ _1"></span>ersi<span class="_ _1"></span>ty<span class="_ _3"></span>,<span class="_ _4"> </span>USA</div><div class="t m0 x6 h4 y6 ff4 fs2 fc0 sc0 ls5 ws0">{<span class="ff5 ls6">zeiler,fergus</span>}<span class="ff5 ls6">@cs.nyu.edu</span></div><div class="t m0 x7 h4 y7 ff6 fs2 fc0 sc0 ls7 ws0">Abstrac<span class="_ _1"></span>t.<span class="_ _6"> </span><span class="ff3 ls8">Large<span class="_ _7"> </span>Conv<span class="_ _1"></span>olution<span class="_ _8"></span>al<span class="_ _2"> </span>Network<span class="_ _2"> </span>mod<span class="_ _8"></span>els<span class="_ _2"> </span>have<span class="_ _4"> </span>recently<span class="_ _2"> </span>d<span class="_ _8"></span>emon-</span></div><div class="t m0 x7 h4 y8 ff3 fs2 fc0 sc0 ls9 ws0">strated<span class="_"> </span>impressive<span class="_"> </span>classiﬁcation<span class="_ _6"> </span>performance<span class="_ _9"> </span>on<span class="_"> </span>the<span class="_"> </span>ImageNet<span class="_"> </span>b<span class="_ _a"></span>ench-</div><div class="t m0 x7 h4 y9 ff3 fs2 fc0 sc0 lsa ws0">mark<span class="_ _4"> </span>Krizhe<span class="_ _1"></span>vsky<span class="_ _4"> </span><span class="ff7 lsb">et<span class="_ _2"> </span>al.<span class="_ _7"> </span></span><span class="lsc">[18].<span class="_ _4"> </span>Ho<span class="_ _1"></span>we<span class="_ _1"></span>ver<span class="_ _4"> </span>there<span class="_ _4"> </span>is<span class="_ _4"> </span>no<span class="_ _4"> </span>clear<span class="_ _4"> </span>understandi<span class="_ _1"></span>ng<span class="_ _4"> </span>of</span></div><div class="t m0 x7 h4 ya ff3 fs2 fc0 sc0 lsc ws0">wh<span class="_ _1"></span>y<span class="_ _4"> </span>they<span class="_ _b"> </span>perform<span class="_ _c"> </span>so<span class="_ _c"> </span>w<span class="_ _1"></span>ell,<span class="_ _c"> </span>or<span class="_ _c"> </span>how<span class="_ _b"> </span>they<span class="_ _b"> </span>might<span class="_ _b"> </span>b<span class="_ _8"></span>e<span class="_ _c"> </span>impro<span class="_ _1"></span>v<span class="_ _1"></span>ed.<span class="_ _c"> </span>In<span class="_ _c"> </span>this<span class="_ _b"> </span>pap<span class="_ _8"></span>er</div><div class="t m0 x7 h4 yb ff3 fs2 fc0 sc0 lsd ws0">w<span class="_ _1"></span>e<span class="_ _b"> </span>explore<span class="_ _d"> </span>both<span class="_ _b"> </span>iss<span class="_ _1"></span>ues.<span class="_ _b"> </span>W<span class="_ _5"></span>e<span class="_ _d"> </span>introduce<span class="_ _d"> </span>a<span class="_ _d"> </span>nov<span class="_ _1"></span>el<span class="_ _d"> </span>visualiz<span class="_ _1"></span>ation<span class="_ _b"> </span>tec<span class="_ _1"></span>hnique<span class="_ _d"> </span>that</div><div class="t m0 x7 h4 yc ff3 fs2 fc0 sc0 lse ws0">giv<span class="_ _1"></span>es<span class="_ _d"> </span>insigh<span class="_ _1"></span>t<span class="_ _b"> </span>in<span class="_ _1"></span>to<span class="_ _d"> </span>the<span class="_ _d"> </span>function<span class="_ _d"> </span>of<span class="_ _d"> </span>intermedi<span class="_ _1"></span>ate<span class="_ _d"> </span>feature<span class="_ _d"> </span>la<span class="_ _1"></span>yers<span class="_ _d"> </span>and<span class="_ _d"> </span>the<span class="_ _d"> </span>oper-</div><div class="t m0 x7 h4 yd ff3 fs2 fc0 sc0 lsf ws0">ation<span class="_ _b"> </span>of<span class="_ _c"> </span>the<span class="_ _b"> </span>classiﬁer.<span class="_ _c"> </span>Used<span class="_ _c"> </span>in<span class="_ _b"> </span>a<span class="_ _b"> </span>diagnostic<span class="_ _c"> </span>role,<span class="_ _c"> </span>these<span class="_ _b"> </span>v<span class="_ _8"></span>isualizations<span class="_ _c"> </span>allow</div><div class="t m0 x7 h4 ye ff3 fs2 fc0 sc0 ls10 ws0">us<span class="_ _4"> </span>to<span class="_ _4"> </span>ﬁnd<span class="_ _4"> </span>model<span class="_ _2"> </span>arc<span class="_ _1"></span>hitectur<span class="_ _1"></span>es<span class="_ _4"> </span>that<span class="_ _4"> </span>outp<span class="_ _8"></span>erform<span class="_ _c"> </span>Krizhevsky<span class="_ _c"> </span><span class="ff7 lsb">et<span class="_ _2"> </span>al.<span class="_ _2"> </span></span><span class="ls11">on<span class="_ _2"> </span>the</span></div><div class="t m0 x7 h4 yf ff3 fs2 fc0 sc0 ls12 ws0">ImageNet<span class="_ _7"> </span>classiﬁcat<span class="_ _8"></span>ion<span class="_ _7"> </span>b<span class="_ _a"></span>enchmark.<span class="_ _2"> </span>W<span class="_ _5"></span>e<span class="_ _2"> </span>also<span class="_ _e"> </span>p<span class="_ _8"></span>erform<span class="_ _7"> </span>an<span class="_ _7"> </span>ablation<span class="_ _7"> </span>st<span class="_ _8"></span>udy</div><div class="t m0 x7 h4 y10 ff3 fs2 fc0 sc0 ls13 ws0">to<span class="_ _b"> </span>discov<span class="_ _1"></span>er<span class="_ _b"> </span>th<span class="_ _8"></span>e<span class="_ _b"> </span>p<span class="_ _8"></span>erformance<span class="_ _c"> </span>contribution<span class="_ _b"> </span>from<span class="_ _c"> </span>diﬀerent<span class="_ _b"> </span>mo<span class="_ _8"></span>del<span class="_ _b"> </span>lay<span class="_ _1"></span>ers.<span class="_ _c"> </span>W<span class="_ _5"></span>e</div><div class="t m0 x7 h4 y11 ff3 fs2 fc0 sc0 ls12 ws0">show<span class="_ _2"> </span>our<span class="_ _7"> </span>ImageN<span class="_ _8"></span>et<span class="_ _7"> </span>mo<span class="_ _8"></span>del<span class="_ _e"> </span>generalizes<span class="_ _7"> </span>well<span class="_ _2"> </span>t<span class="_ _8"></span>o<span class="_ _7"> </span>other<span class="_ _7"> </span>dat<span class="_ _8"></span>asets:<span class="_ _7"> </span>wh<span class="_ _8"></span>en<span class="_ _7"> </span>the</div><div class="t m0 x7 h4 y12 ff3 fs2 fc0 sc0 ls14 ws0">softmax<span class="_ _4"> </span>classiﬁer<span class="_ _2"> </span>is<span class="_ _c"> </span>retrain<span class="_ _8"></span>ed,<span class="_ _4"> </span>it<span class="_ _4"> </span>convincingly<span class="_ _4"> </span>b<span class="_ _8"></span>eats<span class="_ _4"> </span>the<span class="_ _c"> </span>cu<span class="_ _8"></span>rrent<span class="_ _c"> </span>state-of-</div><div class="t m0 x7 h4 y13 ff3 fs2 fc0 sc0 ls15 ws0">the-art<span class="_ _2"> </span>results<span class="_ _2"> </span>on<span class="_ _4"> </span>Caltech-101<span class="_ _2"> </span>and<span class="_ _2"> </span>Caltec<span class="_ _1"></span>h-<span class="_ _8"></span>256<span class="_ _2"> </span>datasets.</div><div class="t m0 x8 h5 y14 ff1 fs3 fc0 sc0 ls16 ws0">1<span class="_ _f"> </span>In<span class="_ _1"></span>tro<span class="_ _a"></span>duction</div><div class="t m0 x8 h3 y15 ff2 fs1 fc0 sc0 ls17 ws0">Since<span class="_ _2"> </span>their<span class="_ _7"> </span>introduction<span class="_ _7"> </span>by<span class="_ _2"> </span>LeCun<span class="_ _2"> </span><span class="ff8 ls18">et<span class="_ _7"> </span>al.<span class="_ _7"> </span></span><span class="ls19">[20]<span class="_ _2"> </span>in<span class="_ _2"> </span>the<span class="_ _2"> </span>early<span class="_ _2"> </span>1990’s,<span class="_ _4"> </span>Con<span class="_ _1"></span>volut<span class="_ _1"></span>ional</span></div><div class="t m0 x8 h3 y16 ff2 fs1 fc0 sc0 ls1a ws0">Net<span class="_ _1"></span>works<span class="_ _4"> </span>(con<span class="_ _1"></span>vnets)<span class="_ _2"> </span>ha<span class="_ _1"></span>ve<span class="_ _4"> </span>demonstrated<span class="_ _4"> </span>excellen<span class="_ _1"></span>t<span class="_ _2"> </span>p<span class="_ _8"></span>erformance<span class="_ _4"> </span>at<span class="_ _2"> </span>tasks<span class="_ _2"> </span>suc<span class="_ _1"></span>h<span class="_ _2"> </span>as</div><div class="t m0 x8 h3 y17 ff2 fs1 fc0 sc0 ls1b ws0">hand-written<span class="_ _7"> </span>digit<span class="_ _7"> </span>classiﬁcation<span class="_ _2"> </span>and<span class="_ _7"> </span>face<span class="_ _7"> </span>detection.<span class="_ _7"> </span>In<span class="_ _2"> </span>the<span class="_ _7"> </span>la<span class="_ _8"></span>st<span class="_ _2"> </span>18<span class="_ _2"> </span>mo<span class="_ _8"></span>nths,<span class="_ _2"> </span>sev-</div><div class="t m0 x8 h3 y18 ff2 fs1 fc0 sc0 ls1b ws0">eral<span class="_ _7"> </span>pap<span class="_ _8"></span>ers<span class="_ _7"> </span>have<span class="_ _2"> </span>shown<span class="_ _2"> </span>that<span class="_ _e"> </span>they<span class="_ _7"> </span>ca<span class="_ _8"></span>n<span class="_ _7"> </span>also<span class="_ _7"> </span>deliver<span class="_ _7"> </span>outstanding<span class="_ _7"> </span>p<span class="_ _8"></span>erfor<span class="_ _8"></span>mance<span class="_ _7"> </span>on</div><div class="t m0 x8 h3 y19 ff2 fs1 fc0 sc0 ls1c ws0">more<span class="_ _d"> </span>c<span class="_ _1"></span>hallenging<span class="_ _d"> </span>visual<span class="_ _10"> </span>classiﬁcation<span class="_ _10"> </span>tasks.<span class="_ _10"> </span>Cir<span class="_ _8"></span>esan<span class="_ _10"> </span><span class="ff8 ls18">et<span class="_ _d"> </span>al.<span class="_ _d"> </span></span><span class="ls1d">[4]<span class="_ _10"> </span>demonst<span class="_ _1"></span>rate<span class="_ _10"> </span>state-of-</span></div><div class="t m0 x8 h3 y1a ff2 fs1 fc0 sc0 ls1d ws0">the-art<span class="_ _10"> </span>performance<span class="_ _10"> </span>on<span class="_ _10"> </span>NORB<span class="_ _10"> </span>and<span class="_ _d"> </span>CIF<span class="_ _3"></span>AR<span class="_ _1"></span>-10<span class="_ _d"> </span>datasets.<span class="_ _10"> </span>Most<span class="_ _10"> </span>notabl<span class="_ _1"></span>y<span class="_ _5"></span>,<span class="_ _10"> </span>Krizhevsky</div><div class="t m0 x8 h3 y1b ff8 fs1 fc0 sc0 ls18 ws0">et<span class="_ _2"> </span>al.<span class="_ _2"> </span><span class="ff2 ls1e">[18]<span class="_ _4"> </span>sho<span class="_ _1"></span>w<span class="_ _2"> </span>record<span class="_ _4"> </span>beating<span class="_ _4"> </span>p<span class="_ _8"></span>erformance<span class="_ _4"> </span>on<span class="_ _2"> </span>the<span class="_ _4"> </span>ImageNet<span class="_ _c"> </span>2012<span class="_ _2"> </span>classiﬁ<span class="_ _1"></span>cation</span></div><div class="t m0 x8 h3 y1c ff2 fs1 fc0 sc0 ls1e ws0">benchm<span class="_ _1"></span>ark,<span class="_ _b"> </span>with<span class="_ _d"> </span>their<span class="_ _b"> </span>con<span class="_ _1"></span>vnet<span class="_ _b"> </span>model<span class="_ _b"> </span>achie<span class="_ _1"></span>ving<span class="_ _d"> </span>an<span class="_ _b"> </span>error<span class="_ _d"> </span>rate<span class="_ _b"> </span>of<span class="_ _b"> </span>16.4%,<span class="_ _d"> </span>compared</div><div class="t m0 x8 h3 y1d ff2 fs1 fc0 sc0 ls1f ws0">to<span class="_ _c"> </span>the<span class="_ _b"> </span>2nd<span class="_ _c"> </span>place<span class="_ _b"> </span>result<span class="_ _c"> </span>of<span class="_ _c"> </span>26.1%.<span class="_ _b"> </span>F<span class="_ _5"></span>ollo<span class="_ _1"></span>wing<span class="_ _b"> </span>on<span class="_ _b"> </span>from<span class="_ _c"> </span>this<span class="_ _c"> </span>w<span class="_ _1"></span>ork,<span class="_ _c"> </span>Girshic<span class="_ _1"></span>k<span class="_ _b"> </span><span class="ff8 ls18">et<span class="_ _c"> </span>al.<span class="_ _4"> </span></span><span class="ls20">[10]</span></div><div class="t m0 x8 h3 y1e ff2 fs1 fc0 sc0 ls20 ws0">hav<span class="_ _1"></span>e<span class="_ _2"> </span>shown<span class="_ _2"> </span>leading<span class="_ _7"> </span>detection<span class="_ _2"> </span>p<span class="_ _8"></span>erfo<span class="_ _8"></span>rmance<span class="_ _7"> </span>on<span class="_ _2"> </span>the<span class="_ _7"> </span>P<span class="_ _5"></span>ASCAL<span class="_ _7"> </span>VOC<span class="_ _2"> </span>dataset.<span class="_ _7"> </span>Sev-</div><div class="t m0 x8 h3 y1f ff2 fs1 fc0 sc0 ls21 ws0">eral<span class="_ _d"> </span>factors<span class="_ _d"> </span>are<span class="_ _b"> </span>responsible<span class="_ _d"> </span>for<span class="_ _b"> </span>this<span class="_ _d"> </span>dramatic<span class="_ _d"> </span>impro<span class="_ _1"></span>vemen<span class="_ _1"></span>t<span class="_ _d"> </span>in<span class="_ _d"> </span>p<span class="_ _8"></span>erformance:<span class="_ _d"> </span>(i)<span class="_ _d"> </span>the</div><div class="t m0 x8 h3 y20 ff2 fs1 fc0 sc0 ls1b ws0">av<span class="_ _5"></span>ailability<span class="_ _2"> </span>of<span class="_ _e"> </span>m<span class="_ _1"></span>uch<span class="_ _2"> </span>la<span class="_ _8"></span>rger<span class="_ _7"> </span>training<span class="_ _7"> </span>s<span class="_ _8"></span>ets,<span class="_ _7"> </span>with<span class="_ _e"> </span>millions<span class="_ _7"> </span>of<span class="_ _7"> </span>la<span class="_ _8"></span>be<span class="_ _8"></span>led<span class="_ _7"> </span>exa<span class="_ _8"></span>mples;<span class="_ _2"> </span>(ii)</div><div class="t m0 x8 h3 y21 ff2 fs1 fc0 sc0 ls22 ws0">powerful<span class="_ _d"> </span>GPU<span class="_ _c"> </span>implement<span class="_ _1"></span>atio<span class="_ _8"></span>ns,<span class="_ _b"> </span>mak<span class="_ _8"></span>ing<span class="_ _b"> </span>the<span class="_ _b"> </span>training<span class="_ _c"> </span>of<span class="_ _b"> </span>very<span class="_ _b"> </span>larg<span class="_ _8"></span>e<span class="_ _b"> </span>mo<span class="_ _8"></span>dels<span class="_ _b"> </span>pra<span class="_ _8"></span>cti-</div><div class="t m0 x8 h3 y22 ff2 fs1 fc0 sc0 ls5 ws0">cal<span class="_ _c"> </span>and<span class="_ _c"> </span>(iii)<span class="_ _4"> </span>b<span class="_ _8"></span>etter<span class="_ _c"> </span>mo<span class="_ _8"></span>del<span class="_ _c"> </span>r<span class="_ _8"></span>egulariza<span class="_ _8"></span>tion<span class="_ _b"> </span>stra<span class="_ _8"></span>tegies,<span class="_ _b"> </span>s<span class="_ _8"></span>uch<span class="_ _b"> </span>as<span class="_ _c"> </span>Drop<span class="_ _8"></span>out<span class="_ _c"> </span>[1<span class="_ _8"></span>4].</div><div class="t m0 x9 h3 y23 ff2 fs1 fc0 sc0 ls23 ws0">Despite<span class="_ _2"> </span>this<span class="_ _2"> </span>encourag<span class="_ _8"></span>ing<span class="_ _4"> </span>pr<span class="_ _8"></span>ogress,<span class="_ _4"> </span>ther<span class="_ _8"></span>e<span class="_ _2"> </span>is<span class="_ _2"> </span>still<span class="_ _2"> </span>little<span class="_ _2"> </span>insight<span class="_ _4"> </span>into<span class="_ _2"> </span>the<span class="_ _2"> </span>in<span class="_ _1"></span>ternal</div><div class="t m0 x8 h3 y24 ff2 fs1 fc0 sc0 ls24 ws0">operation<span class="_ _c"> </span>and<span class="_ _4"> </span>behavior<span class="_ _b"> </span>of<span class="_ _4"> </span>these<span class="_ _c"> </span>complex<span class="_ _c"> </span>mo<span class="_ _8"></span>dels,<span class="_ _c"> </span>or<span class="_ _4"> </span>how<span class="_ _b"> </span>they<span class="_ _c"> </span>achiev<span class="_ _1"></span>e<span class="_ _c"> </span>such<span class="_ _b"> </span>go<span class="_ _8"></span>o<span class="_ _8"></span>d</div><div class="t m0 x8 h3 y25 ff2 fs1 fc0 sc0 ls25 ws0">performance.<span class="_ _7"> </span>F<span class="_ _5"></span>rom<span class="_ _e"> </span>a<span class="_ _e"> </span>scientiﬁc<span class="_ _7"> </span>standp<span class="_ _8"></span>oi<span class="ls22">n<span class="_ _1"></span>t,<span class="_ _e"> </span>this<span class="_ _e"> </span>is<span class="_ _e"> </span>deeply<span class="_ _e"> </span>unsa<span class="_ _8"></span>tisfactory<span class="_ _5"></span>.<span class="_ _7"> </span>With-</span></div><div class="t m0 x8 h3 y26 ff2 fs1 fc0 sc0 ls17 ws0">out<span class="_ _7"> </span>clear<span class="_ _7"> </span>unders<span class="_ _8"></span>tanding<span class="_ _7"> </span>of<span class="_ _7"> </span>how<span class="_ _2"> </span>a<span class="_ _8"></span>nd<span class="_ _2"> </span>why<span class="_ _2"> </span>they<span class="_ _e"> </span>work,<span class="_ _2"> </span>the<span class="_ _7"> </span>development<span class="_ _2"> </span>of<span class="_ _7"> </span>b<span class="_ _8"></span>etter</div><div class="t m0 x8 h3 y27 ff2 fs1 fc0 sc0 ls5 ws0">mo<span class="_ _8"></span>dels<span class="_ _7"> </span>is<span class="_ _7"> </span>reduced<span class="_ _7"> </span>to<span class="_ _7"> </span>trial-a<span class="_ _8"></span>nd-error.<span class="_ _2"> </span>In<span class="_ _7"> </span>this<span class="_ _7"> </span>pap<span class="_ _8"></span>er<span class="_ _7"> </span>we<span class="_ _2"> </span>introduce<span class="_ _7"> </span>a<span class="_ _7"> </span>visualiza<span class="_ _8"></span>tion</div><div class="t m0 x8 h6 y28 ff9 fs4 fc0 sc0 ls26 ws0">D.<span class="_ _b"> </span>Fle<span class="_ _8"></span>et<span class="_ _b"> </span>et<span class="_ _c"> </span>al.<span class="_ _c"> </span>(Ed<span class="_ _8"></span>s.):<span class="_ _b"> </span>ECCV<span class="_ _c"> </span>2014,<span class="_ _c"> </span>Part<span class="_ _b"> </span>I,<span class="_ _c"> </span>LNCS<span class="_ _c"> </span>8689,<span class="_ _c"> </span>p<span class="_ _8"></span>p.<span class="_ _c"> </span>818–833,<span class="_ _b"> </span>2014.</div><div class="t m0 xa h6 y29 ff9 fs4 fc0 sc0 ls5 ws0">c</div><div class="t m0 x8 h6 y2a ffa fs4 fc0 sc0 ls5 ws0"><span class="_ _b"> </span><span class="ff9 ls27">Springer<span class="_ _b"> </span>Internat<span class="_ _1"></span>ional<span class="_ _b"> </span>Publishing<span class="_ _d"> </span>Switzerland<span class="_ _b"> </span>2014</span></div><a class="l"><div class="d m1"></div></a><a class="l"><div class="d m1"></div></a><a class="l"><div class="d m1"></div></a><a class="l"><div class="d m1"></div></a><a class="l"><div class="d m1"></div></a><a class="l"><div class="d m1"></div></a><a class="l"><div class="d m1"></div></a></div><div class="pi" data-data='{"ctm":[2.037137,0.000000,0.000000,2.037137,0.000000,0.000000]}'></div></div><div id="pf2" class="pf w0 h0" data-page-no="2"><div class="pc pc2 w0 h0"><img class="bi x0 y0 w1 h1" alt="" src="/image.php?url=https://csdnimg.cn/release/download_crawler_static/89567205/bg2.jpg"><div class="t m0 xb h4 y2b ff3 fs2 fc0 sc0 ls28 ws0">Visual<span class="_ _1"></span>izing<span class="_ _4"> </span>and<span class="_ _4"> </span>Understanding<span class="_ _b"> </span>Con<span class="_ _1"></span>voluti<span class="_ _1"></span>onal<span class="_ _4"> </span>Netw<span class="_ _5"></span>orks<span class="_ _11"> </span>819</div><div class="t m0 x8 h3 y1 ff2 fs1 fc0 sc0 ls17 ws0">technique<span class="_ _2"> </span>that<span class="_ _e"> </span>reveals<span class="_ _2"> </span>the<span class="_ _e"> </span>input<span class="_ _7"> </span>stimuli<span class="_ _7"> </span>that<span class="_ _e"> </span>excite<span class="_ _7"> </span>individual<span class="_ _e"> </span>feature<span class="_ _7"> </span>ma<span class="_ _8"></span>ps<span class="_ _7"> </span>at</div><div class="t m0 x8 h3 y2c ff2 fs1 fc0 sc0 ls1b ws0">any<span class="_ _e"> </span>layer<span class="_ _e"> </span>in<span class="_ _12"> </span>the<span class="_ _12"> </span>mo<span class="_ _8"></span>del.<span class="_ _12"> </span>It<span class="_ _12"> </span>also<span class="_ _13"> </span>allows<span class="_ _e"> </span>us<span class="_ _12"> </span>to<span class="_ _12"> </span>observe<span class="_ _13"> </span>the<span class="_ _13"> </span>evolution<span class="_ _12"> </span>of<span class="_ _13"> </span>features</div><div class="t m0 x8 h3 y2d ff2 fs1 fc0 sc0 ls1d ws0">during<span class="_ _7"> </span>training<span class="_ _2"> </span>and<span class="_ _e"> </span>to<span class="_ _e"> </span>diagnose<span class="_ _2"> </span>p<span class="_ _8"></span>oten<span class="_ _1"></span>tial<span class="_ _7"> </span>problems<span class="_ _7"> </span>with<span class="_ _e"> </span>the<span class="_ _7"> </span>mo<span class="_ _8"></span>del.<span class="_ _7"> </span>The<span class="_ _e"> </span>visu-</div><div class="t m0 x8 h3 y2e ff2 fs1 fc0 sc0 ls29 ws0">alization<span class="_ _13"> </span>technique<span class="_ _12"> </span>we<span class="_ _13"> </span>prop<span class="_ _8"></span>ose<span class="_ _12"> </span>uses<span class="_ _12"> </span>a<span class="_ _12"> </span>mul<span class="_ _1"></span><span class="ls2a">ti-lay<span class="_ _1"></span>ered<span class="_ _12"> </span>Decon<span class="_ _1"></span>volutional<span class="_ _e"> </span>Netw<span class="_ _1"></span>ork</span></div><div class="t m0 x8 h3 y2f ff2 fs1 fc0 sc0 ls2b ws0">(decon<span class="_ _1"></span>vnet),<span class="_ _4"> </span>as<span class="_ _2"> </span>prop<span class="_ _8"></span>osed<span class="_ _2"> </span>b<span class="_ _1"></span>y<span class="_ _2"> </span>Zeiler<span class="_ _2"> </span><span class="ff8 ls18">et<span class="_ _7"> </span>al.<span class="_ _7"> </span></span><span class="ls2c">[29<span class="_ _8"></span>],<span class="_ _2"> </span>to<span class="_ _2"> </span>pro<span class="_ _a"></span>ject<span class="_ _2"> </span>the<span class="_ _7"> </span>feature<span class="_ _2"> </span>a<span class="_ _8"></span>ctiv<span class="_ _5"></span>a<span class="_ _8"></span>tions</span></div><div class="t m0 x8 h3 y30 ff2 fs1 fc0 sc0 ls2d ws0">back<span class="_ _c"> </span>to<span class="_ _2"> </span>the<span class="_ _4"> </span>input<span class="_ _2"> </span>pixel<span class="_ _4"> </span>space.<span class="_ _4"> </span>W<span class="_ _5"></span>e<span class="_ _2"> </span>also<span class="_ _4"> </span>pe<span class="_ _8"></span>rform<span class="_ _4"> </span>a<span class="_ _2"> </span>sensitivit<span class="_ _1"></span>y<span class="_ _2"> </span>analysis<span class="_ _4"> </span>of<span class="_ _4"> </span>the<span class="_ _2"> </span>clas-</div><div class="t m0 x8 h3 y31 ff2 fs1 fc0 sc0 ls17 ws0">siﬁer<span class="_ _2"> </span>output<span class="_ _2"> </span>by<span class="_ _2"> </span>occluding<span class="_ _7"> </span>p<span class="_ _8"></span>ortions<span class="_ _2"> </span>of<span class="_ _2"> </span>the<span class="_ _7"> </span>input<span class="_ _2"> </span>ima<span class="_ _8"></span>ge,<span class="_ _2"> </span>revealing<span class="_ _4"> </span>which<span class="_ _2"> </span>parts<span class="_ _2"> </span>of</div><div class="t m0 x8 h3 y32 ff2 fs1 fc0 sc0 ls1d ws0">the<span class="_ _4"> </span>scene<span class="_ _2"> </span>are<span class="_ _2"> </span>importan<span class="_ _1"></span>t<span class="_ _4"> </span>for<span class="_ _2"> </span>classiﬁcation<span class="_ _1"></span>.</div><div class="t m0 x9 h3 y33 ff2 fs1 fc0 sc0 ls2a ws0">Using<span class="_ _4"> </span>these<span class="_ _4"> </span>to<span class="_ _8"></span>ols,<span class="_ _4"> </span>we<span class="_ _c"> </span>start<span class="_ _4"> </span>with<span class="_ _4"> </span>the<span class="_ _4"> </span>architecture<span class="_ _c"> </span>of<span class="_ _4"> </span>Krizhevsky<span class="_ _4"> </span><span class="ff8 ls18">et<span class="_ _2"> </span>al.<span class="_ _2"> </span></span><span class="ls20">[18]<span class="_ _4"> </span>a<span class="_ _8"></span>nd</span></div><div class="t m0 x8 h3 y34 ff2 fs1 fc0 sc0 ls2e ws0">explore<span class="_ _12"> </span>diﬀeren<span class="_ _1"></span>t<span class="_ _12"> </span>architectures,<span class="_ _12"> </span>discov<span class="_ _1"></span>e<span class="ls2">ring<span class="_ _13"> </span>ones<span class="_"> </span>that<span class="_ _13"> </span>outp<span class="_ _8"></span>erform<span class="_"> </span>their<span class="_ _13"> </span>r<span class="_ _8"></span>esults</span></div><div class="t m0 x8 h3 y35 ff2 fs1 fc0 sc0 ls20 ws0">on<span class="_ _7"> </span>ImageNet.<span class="_ _e"> </span>W<span class="_ _3"></span>e<span class="_ _e"> </span>then<span class="_ _7"> </span>explor<span class="_ _8"></span>e<span class="_ _7"> </span>the<span class="_ _7"> </span>gener<span class="_ _8"></span>alization<span class="_ _7"> </span>ability<span class="_ _2"> </span>of<span class="_ _7"> </span>the<span class="_ _e"> </span>mo<span class="_ _8"></span>del<span class="_ _7"> </span>to<span class="_ _7"> </span>other</div><div class="t m0 x8 h3 y36 ff2 fs1 fc0 sc0 ls23 ws0">datasets,<span class="_ _13"> </span>just<span class="_ _13"> </span>retra<span class="_ _8"></span>ining<span class="_ _e"> </span>the<span class="_ _13"> </span>softmax<span class="_ _13"> </span>c<span class="_ _8"></span>lassiﬁer<span class="_ _e"> </span>on<span class="_ _13"> </span>top.<span class="_ _13"> </span>As<span class="_ _13"> </span>s<span class="_ _8"></span>uch,<span class="_ _e"> </span>this<span class="_ _13"> </span>is<span class="_ _13"> </span>a<span class="_ _13"> </span>form</div><div class="t m0 x8 h3 y37 ff2 fs1 fc0 sc0 ls2d ws0">of<span class="_ _e"> </span>sup<span class="_ _8"></span>ervised<span class="_ _e"> </span>pre-tr<span class="_ _8"></span>aining,<span class="_ _e"> </span>which<span class="_ _e"> </span>con<span class="_ _1"></span>tra<span class="_ _8"></span>sts<span class="_ _e"> </span>with<span class="_ _e"> </span>the<span class="_ _e"> </span>unsup<span class="_ _8"></span>ervised<span class="_ _13"> </span>pre-training</div><div class="t m0 x8 h3 y38 ff2 fs1 fc0 sc0 ls21 ws0">methods<span class="_ _2"> </span>popularized<span class="_ _4"> </span>by<span class="_ _4"> </span>Hin<span class="_ _1"></span>ton<span class="_ _2"> </span><span class="ff8 ls18">et<span class="_ _7"> </span>al.<span class="_ _7"> </span></span><span class="ls19">[13]<span class="_ _2"> </span>and<span class="_ _4"> </span>others<span class="_ _4"> </span>[1,26].</span></div><div class="t m0 x8 h7 y39 ffb fs1 fc0 sc0 ls2f ws0">1.1<span class="_ _14"> </span>Related<span class="_ _2"> </span>W<span class="_ _5"></span>ork</div><div class="t m0 x8 h3 y3a ffb fs1 fc0 sc0 ls30 ws0">Visualiz<span class="_ _1"></span>ation:<span class="_ _b"> </span><span class="ff2">Visualizing<span class="_ _b"> </span>features<span class="_ _4"> </span>to<span class="_ _c"> </span>gain<span class="_ _c"> </span>int<span class="_ _1"></span>uition<span class="_ _c"> </span>ab<span class="_ _8"></span>out<span class="_ _c"> </span>the<span class="_ _4"> </span>net<span class="_ _1"></span>work<span class="_ _b"> </span>is<span class="_ _c"> </span>com-</span></div><div class="t m0 x8 h3 y3b ff2 fs1 fc0 sc0 ls17 ws0">mon<span class="_"> </span>p<span class="_ _1"></span>ra<span class="_ _8"></span>ctice,<span class="_ _12"> </span>but<span class="_"> </span>mostly<span class="_ _12"> </span>limited<span class="_"> </span>to<span class="_ _12"> </span>the<span class="_"> </span>1st<span class="_ _12"> </span>lay<span class="_ _1"></span>er<span class="_ _12"> </span>where<span class="_"> </span>pro<span class="_ _8"></span>jectio<span class="_ _8"></span>ns<span class="_ _12"> </span>to<span class="_"> </span>pixel</div><div class="t m0 x8 h3 y3c ff2 fs1 fc0 sc0 ls31 ws0">space<span class="_ _4"> </span>are<span class="_ _4"> </span>p<span class="_ _8"></span>o<span class="_ _8"></span>ssible.<span class="_ _4"> </span>In<span class="_ _2"> </span>higher<span class="_ _4"> </span>lay<span class="_ _1"></span>ers<span class="_ _4"> </span>alter<span class="_ _8"></span>nate<span class="_ _4"> </span>metho<span class="_ _8"></span>ds<span class="_ _2"> </span>m<span class="_ _1"></span>ust<span class="_ _4"> </span>b<span class="_ _8"></span>e<span class="_ _2"> </span>used.<span class="_ _4"> </span>[8<span class="_ _8"></span>]<span class="_ _4"> </span>ﬁnd<span class="_ _4"> </span>the</div><div class="t m0 x8 h3 y3d ff2 fs1 fc0 sc0 ls32 ws0">optimal<span class="_ _e"> </span>stimulus<span class="_ _e"> </span>for<span class="_ _13"> </span>each<span class="_ _e"> </span>unit<span class="_ _e"> </span>by<span class="_ _e"> </span>per<span class="_ _8"></span>fo<span class="ls33">rming<span class="_ _e"> </span>gradient<span class="_ _7"> </span>descent<span class="_ _e"> </span>in<span class="_ _e"> </span>image<span class="_ _e"> </span>space</span></div><div class="t m0 x8 h3 y3e ff2 fs1 fc0 sc0 ls20 ws0">to<span class="_ _4"> </span>ma<span class="_ _8"></span>ximize<span class="_ _4"> </span>the<span class="_ _4"> </span>unit’s<span class="_ _2"> </span>activ<span class="_ _1"></span>ation.<span class="_ _4"> </span>This<span class="_ _2"> </span>requires<span class="_ _4"> </span>a<span class="_ _2"> </span>careful<span class="_ _4"> </span>initialization<span class="_ _2"> </span>and<span class="_ _4"> </span>do<span class="_ _8"></span>es</div><div class="t m0 x8 h3 y3f ff2 fs1 fc0 sc0 ls2c ws0">not<span class="_ _4"> </span>give<span class="_ _4"> </span>a<span class="_ _8"></span>n<span class="_ _1"></span>y<span class="_ _2"> </span>information<span class="_ _4"> </span>a<span class="_ _8"></span>b<span class="_ _8"></span>out<span class="_ _2"> </span>the<span class="_ _4"> </span>unit’s<span class="_ _2"> </span>inv<span class="_ _5"></span>aria<span class="_ _8"></span>nces.<span class="_ _4"> </span>Motiv<span class="_ _5"></span>a<span class="_ _8"></span>ted<span class="_ _4"> </span>by<span class="_ _4"> </span>the<span class="_ _2"> </span>latter’s</div><div class="t m0 x8 h3 y40 ff2 fs1 fc0 sc0 ls34 ws0">short-coming<span class="_ _8"></span>,<span class="_ _2"> </span>[1<span class="_ _8"></span>9]<span class="_ _e"> </span>(extending<span class="_ _e"> </span>an<span class="_ _7"> </span>idea<span class="_ _e"> </span>by<span class="_ _7"> </span>[2])<span class="_ _e"> </span>show<span class="_ _7"> </span>how<span class="_ _7"> </span>the<span class="_ _e"> </span>Hessian<span class="_ _e"> </span>of<span class="_ _7"> </span>a<span class="_ _e"> </span>given</div><div class="t m0 x8 h3 y41 ff2 fs1 fc0 sc0 ls1b ws0">unit<span class="_ _e"> </span>may<span class="_ _e"> </span>b<span class="_ _8"></span>e<span class="_ _13"> </span>co<span class="_ _8"></span>mputed<span class="_ _e"> </span>numerically<span class="_ _e"> </span>ar<span class="_ _8"></span>ound<span class="_ _e"> </span>the<span class="_ _13"> </span>optimal<span class="_ _13"> </span>resp<span class="_ _8"></span>onse,<span class="_ _13"> </span>giving<span class="_ _e"> </span>s<span class="_ _8"></span>ome</div><div class="t m0 x8 h3 y42 ff2 fs1 fc0 sc0 ls35 ws0">insigh<span class="_ _1"></span>t<span class="_ _b"> </span>in<span class="_ _1"></span>to<span class="_ _b"> </span>in<span class="_ _1"></span>v<span class="_ _5"></span>ariances.<span class="_ _b"> </span>The<span class="_ _b"> </span>problem<span class="_ _d"> </span>is<span class="_ _b"> </span>that<span class="_ _b"> </span>for<span class="_ _b"> </span>higher<span class="_ _d"> </span>lay<span class="_ _1"></span>ers,<span class="_ _d"> </span>the<span class="_ _b"> </span>inv<span class="_ _3"></span>a<span class="_ _8"></span>riances<span class="_ _d"> </span>are</div><div class="t m0 x8 h3 y43 ff2 fs1 fc0 sc0 ls36 ws0">extremely<span class="_ _b"> </span>complex<span class="_ _c"> </span>so<span class="_ _c"> </span>are<span class="_ _c"> </span>p<span class="_ _8"></span>o<span class="_ _8"></span>orly<span class="_ _c"> </span>captured<span class="_ _b"> </span>by<span class="_ _c"> </span>a<span class="_ _c"> </span>simple<span class="_ _c"> </span>quadratic<span class="_ _b"> </span>approximat<span class="_ _1"></span>ion.</div><div class="t m0 x8 h3 y44 ff2 fs1 fc0 sc0 ls34 ws0">Our<span class="_ _4"> </span>appro<span class="_ _8"></span>ach,<span class="_ _c"> </span>by<span class="_ _c"> </span>co<span class="_ _8"></span>nt<span class="_ _1"></span>ra<span class="_ _8"></span>st,<span class="_ _4"> </span>provides<span class="_ _c"> </span>a<span class="_ _2"> </span>non-parametr<span class="_ _8"></span>ic<span class="_ _c"> </span>view<span class="_ _2"> </span>of<span class="_ _4"> </span>inv<span class="_ _5"></span>ariance,<span class="_ _4"> </span>show-</div><div class="t m0 x8 h3 y45 ff2 fs1 fc0 sc0 ls32 ws0">ing<span class="_ _c"> </span>which<span class="_ _c"> </span>patterns<span class="_ _4"> </span>from<span class="_ _4"> </span>the<span class="_ _c"> </span>tr<span class="_ _8"></span>aining<span class="_ _c"> </span>set<span class="_ _c"> </span>a<span class="_ _8"></span>ctiv<span class="_ _5"></span>a<span class="_ _8"></span>te<span class="_ _c"> </span>the<span class="_ _4"> </span>feature<span class="_ _4"> </span>map.<span class="_ _4"> </span>Our<span class="_ _c"> </span>appr<span class="_ _8"></span>oach</div><div class="t m0 x8 h3 y46 ff2 fs1 fc0 sc0 ls37 ws0">is<span class="_ _2"> </span>similar<span class="_ _2"> </span>to<span class="_ _2"> </span>contem<span class="_ _1"></span>p<span class="_ _8"></span>orary<span class="_ _2"> </span>w<span class="_ _1"></span>ork<span class="_ _2"> </span>by<span class="_ _2"> </span>Simon<span class="_ _1"></span>y<span class="_ _1"></span>an<span class="_ _2"> </span><span class="ff8 ls38">et<span class="_ _e"> </span>al.<span class="_ _e"> </span></span><span class="ls39">[2<span class="_ _8"></span>3]<span class="_ _7"> </span>who<span class="_ _2"> </span>demo<span class="_ _8"></span>nstrate<span class="_ _7"> </span>how</span></div><div class="t m0 x8 h3 y47 ff2 fs1 fc0 sc0 ls21 ws0">saliency<span class="_ _b"> </span>maps<span class="_ _4"> </span>can<span class="_ _4"> </span>b<span class="_ _8"></span>e<span class="_ _c"> </span>obtained<span class="_ _4"> </span>from<span class="_ _c"> </span>a<span class="_ _4"> </span>convn<span class="_ _1"></span>et<span class="_ _4"> </span>by<span class="_ _c"> </span>pro<span class="_ _a"></span>jecting<span class="_ _c"> </span>back<span class="_ _b"> </span>from<span class="_ _4"> </span>the<span class="_ _c"> </span>fully</div><div class="t m0 x8 h3 y48 ff2 fs1 fc0 sc0 ls21 ws0">connected<span class="_ _7"> </span>lay<span class="_ _1"></span>ers<span class="_ _7"> </span>of<span class="_ _e"> </span>the<span class="_ _e"> </span>netw<span class="_ _1"></span>ork,<span class="_ _7"> </span>instead<span class="_ _e"> </span>of<span class="_ _7"> </span>the<span class="_ _e"> </span>conv<span class="_ _1"></span>olutional<span class="_ _2"> </span>features<span class="_ _e"> </span>that<span class="_ _7"> </span>we</div><div class="t m0 x8 h3 y49 ff2 fs1 fc0 sc0 ls3a ws0">use.<span class="_ _b"> </span>Girshick<span class="_ _d"> </span><span class="ff8 ls18">et<span class="_ _4"> </span>al.<span class="_ _4"> </span></span><span class="ls22">[10]<span class="_ _c"> </span>show<span class="_ _d"> </span>visualizatio<span class="_ _8"></span>ns<span class="_ _b"> </span>that<span class="_ _b"> </span>identify<span class="_ _b"> </span>patches<span class="_ _b"> </span>within<span class="_ _c"> </span>a<span class="_ _b"> </span>dataset</span></div><div class="t m0 x8 h3 y4a ff2 fs1 fc0 sc0 ls22 ws0">that<span class="_ _4"> </span>ar<span class="_ _8"></span>e<span class="_ _c"> </span>r<span class="_ _8"></span>esp<span class="_ _8"></span>onsible<span class="_ _4"> </span>for<span class="_ _4"> </span>s<span class="_ _8"></span>trong<span class="_ _4"> </span>activ<span class="_ _1"></span>ations<span class="_ _4"> </span>at<span class="_ _4"> </span>hig<span class="_ _8"></span>her<span class="_ _4"> </span>lay<span class="_ _1"></span>ers<span class="_ _4"> </span>in<span class="_ _4"> </span>the<span class="_ _2"> </span>mo<span class="_ _8"></span>del.<span class="_ _4"> </span>Our<span class="_ _4"> </span>vi-</div><div class="t m0 x8 h3 y4b ff2 fs1 fc0 sc0 ls23 ws0">sualizations<span class="_ _13"> </span>diﬀer<span class="_ _12"> </span>in<span class="_ _e"> </span>tha<span class="_ _8"></span>t<span class="_ _13"> </span>they<span class="_ _13"> </span>a<span class="_ _8"></span>re<span class="_ _13"> </span>not<span class="_ _13"> </span>just<span class="_ _13"> </span>crops<span class="_ _12"> </span>of<span class="_ _13"> </span>input<span class="_ _13"> </span>imag<span class="_ _8"></span>es,<span class="_ _13"> </span>but<span class="_ _13"> </span>rather</div><div class="t m0 x8 h3 y4c ff2 fs1 fc0 sc0 ls39 ws0">top-down<span class="_ _2"> </span>pro<span class="_ _a"></span>jectio<span class="_ _8"></span>ns<span class="_ _2"> </span>that<span class="_ _7"> </span>reveal<span class="_ _2"> </span>str<span class="_ _8"></span>uctures<span class="_ _2"> </span>within<span class="_ _7"> </span>ea<span class="_ _8"></span>ch<span class="_ _2"> </span>patch<span class="_ _2"> </span>that<span class="_ _7"> </span>stimulate<span class="_ _2"> </span>a</div><div class="t m0 x8 h3 y4d ff2 fs1 fc0 sc0 ls2c ws0">particular<span class="_ _2"> </span>feature<span class="_ _2"> </span>map.</div><div class="t m0 x8 h3 y4e ffb fs1 fc0 sc0 ls3b ws0">F<span class="_ _3"></span>eatu<span class="_ _8"></span>re<span class="_ _9"> </span>Ge<span class="_ _8"></span>nerali<span class="_ _8"></span>zation:<span class="_ _13"> </span><span class="ff2 ls5">O<span class="_ _8"></span>ur<span class="_ _e"> </span>demo<span class="_ _8"></span>nstration<span class="_ _13"> </span>of<span class="_ _e"> </span>the<span class="_ _13"> </span>g<span class="_ _8"></span>eneralization<span class="_ _13"> </span>ability<span class="_ _e"> </span>of</span></div><div class="t m0 x8 h3 y4f ff2 fs1 fc0 sc0 ls24 ws0">con<span class="_ _1"></span>vnet<span class="_ _2"> </span>features<span class="_ _7"> </span>is<span class="_ _2"> </span>also<span class="_ _7"> </span>explored<span class="_ _2"> </span>in<span class="_ _7"> </span>concurrent<span class="_ _4"> </span>work<span class="_ _2"> </span>by<span class="_ _2"> </span>Donah<span class="_ _1"></span>ue<span class="_ _2"> </span><span class="ff8 ls38">et<span class="_ _e"> </span>al.<span class="_ _13"> </span></span><span class="ls39">[7]<span class="_ _7"> </span>and</span></div><div class="t m0 x8 h3 y50 ff2 fs1 fc0 sc0 ls2c ws0">Girshick<span class="_ _12"> </span><span class="ff8 ls18">et<span class="_ _9"> </span>al.<span class="_ _6"> </span></span><span class="ls21">[10].<span class="_ _12"> </span>They<span class="_"> </span>u<span class="_ _1"></span>se<span class="_"> </span>the<span class="_ _12"> </span>con<span class="_ _1"></span>vnet<span class="_ _9"> </span>featu<span class="_ _1"></span>res<span class="_"> </span>to<span class="_ _12"> </span>obtain<span class="_ _12"> </span>state-of-the<span class="_ _1"></span>-art</span></div><div class="t m0 x8 h3 y51 ff2 fs1 fc0 sc0 ls5 ws0">per<span class="_ _8"></span>formance<span class="_ _4"> </span>o<span class="_ _8"></span>n<span class="_ _2"> </span>Caltec<span class="_ _1"></span>h-1<span class="_ _8"></span>01<span class="_ _4"> </span>and<span class="_ _4"> </span>the<span class="_ _2"> </span>Sun<span class="_ _2"> </span>scenes<span class="_ _4"> </span>dataset<span class="_ _2"> </span>in<span class="_ _4"> </span>the<span class="_ _2"> </span>former<span class="_ _2"> </span>case,<span class="_ _4"> </span>and</div><div class="t m0 x8 h3 y52 ff2 fs1 fc0 sc0 ls29 ws0">for<span class="_ _2"> </span>ob<span class="_ _a"></span>ject<span class="_ _4"> </span>detection<span class="_ _2"> </span>o<span class="_ _8"></span>n<span class="_ _4"> </span>the<span class="_ _2"> </span>P<span class="_ _5"></span>ASCAL<span class="_ _2"> </span>VOC<span class="_ _4"> </span>dataset,<span class="_ _2"> </span>in<span class="_ _2"> </span>the<span class="_ _2"> </span>latter.</div><div class="t m0 x8 h5 y53 ff1 fs3 fc0 sc0 ls3c ws0">2<span class="_ _f"> </span>Approach</div><div class="t m0 x8 h3 y54 ff2 fs1 fc0 sc0 ls5 ws0">W<span class="_ _3"></span>e<span class="_ _7"> </span>use<span class="_ _7"> </span>standa<span class="_ _8"></span>rd<span class="_ _2"> </span>fully<span class="_ _7"> </span>sup<span class="_ _8"></span>ervised<span class="_ _2"> </span>convnet<span class="_ _2"> </span>mo<span class="_ _8"></span>dels<span class="_ _2"> </span>thr<span class="_ _8"></span>oughout<span class="_ _7"> </span>the<span class="_ _2"> </span>pa<span class="_ _8"></span>p<span class="_ _8"></span>er,<span class="_ _2"> </span>as<span class="_ _7"> </span>de-</div><div class="t m0 x8 h3 y55 ff2 fs1 fc0 sc0 ls1b ws0">ﬁned<span class="_ _4"> </span>by<span class="_ _4"> </span>LeCun<span class="_ _2"> </span><span class="ff8 ls18">et<span class="_ _7"> </span>al.<span class="_ _2"> </span></span><span class="ls33">[20]<span class="_ _2"> </span>and<span class="_ _4"> </span>Krizhevsky<span class="_ _2"> </span><span class="ff8 ls18">et<span class="_ _2"> </span>al.<span class="_ _2"> </span></span><span class="ls34">[1<span class="_ _8"></span>8].<span class="_ _4"> </span>These<span class="_ _2"> </span>mo<span class="_ _8"></span>dels<span class="_ _2"> </span>map<span class="_ _2"> </span>a<span class="_ _4"> </span>color</span></span></div><a class="l"><div class="d m1"></div></a><a class="l"><div class="d m1"></div></a><a class="l"><div class="d m1"></div></a><a class="l"><div class="d m1"></div></a><a class="l"><div class="d m1"></div></a><a class="l"><div class="d m1"></div></a><a class="l"><div class="d m1"></div></a><a class="l"><div class="d m1"></div></a><a class="l"><div class="d m1"></div></a><a class="l"><div class="d m1"></div></a><a class="l"><div class="d m1"></div></a><a class="l"><div class="d m1"></div></a><a class="l"><div class="d m1"></div></a><a class="l"><div class="d m1"></div></a></div><div class="pi" data-data='{"ctm":[2.037137,0.000000,0.000000,2.037137,0.000000,0.000000]}'></div></div><div id="pf3" class="pf w0 h0" data-page-no="3"><div class="pc pc3 w0 h0"><img class="bi x0 y0 w1 h1" alt="" src="/image.php?url=https://csdnimg.cn/release/download_crawler_static/89567205/bg3.jpg"><div class="t m0 x8 h4 y2b ff3 fs2 fc0 sc0 ls3d ws0">820<span class="_ _15"> </span>M.D.<span class="_ _4"> </span>Zeile<span class="_ _1"></span>r<span class="_ _2"> </span>and<span class="_ _c"> </span>R.<span class="_ _4"> </span>F<span class="_ _5"></span>ergus</div><div class="t m0 x8 h3 y1 ff2 fs1 fc0 sc0 ls3e ws0">2D<span class="_ _b"> </span>input<span class="_ _4"> </span>image<span class="_ _b"> </span><span class="ffc ls5">x</span></div><div class="t m0 xc h8 y56 ffd fs4 fc0 sc0 ls5 ws0">i</div><div class="t m0 xd h3 y1 ff2 fs1 fc0 sc0 ls3a ws0">,<span class="_ _c"> </span>via<span class="_ _b"> </span>a<span class="_ _c"> </span>ser<span class="_ _8"></span>ies<span class="_ _b"> </span>of<span class="_ _c"> </span>lay<span class="_ _1"></span>ers,<span class="_ _b"> </span>to<span class="_ _c"> </span>a<span class="_ _c"> </span>probability<span class="_ _b"> </span>vector<span class="_ _e"> </span>ˆ<span class="_ _16"></span><span class="ffc ls5">y</span></div><div class="t m0 xe h8 y56 ffd fs4 fc0 sc0 ls5 ws0">i</div><div class="t m0 xf h3 y1 ff2 fs1 fc0 sc0 ls3f ws0">over<span class="_ _4"> </span>th<span class="_ _8"></span>e<span class="_ _c"> </span><span class="ffc ls5">C<span class="_ _7"> </span></span><span class="ls40">dif-</span></div><div class="t m0 x8 h3 y2c ff2 fs1 fc0 sc0 ls2d ws0">ferent<span class="_ _c"> </span>classes.<span class="_ _4"> </span>Ea<span class="_ _8"></span>ch<span class="_ _c"> </span>lay<span class="_ _1"></span>er<span class="_ _4"> </span>consists<span class="_ _4"> </span>of<span class="_ _4"> </span>(i)<span class="_ _2"> </span>conv<span class="_ _1"></span>olution<span class="_ _4"> </span>of<span class="_ _4"> </span>the<span class="_ _2"> </span>previous<span class="_ _4"> </span>lay<span class="_ _1"></span>er<span class="_ _4"> </span>output</div><div class="t m0 x8 h3 y2d ff2 fs1 fc0 sc0 ls17 ws0">(or,<span class="_ _c"> </span>in<span class="_ _4"> </span>the<span class="_ _4"> </span>case<span class="_ _4"> </span>of<span class="_ _c"> </span>the<span class="_ _4"> </span>1st<span class="_ _4"> </span>lay<span class="_ _1"></span>er,<span class="_ _c"> </span>the<span class="_ _4"> </span>input<span class="_ _4"> </span>imag<span class="_ _8"></span>e)<span class="_ _c"> </span>with<span class="_ _4"> </span>a<span class="_ _c"> </span>set<span class="_ _4"> </span>of<span class="_ _4"> </span>learned<span class="_ _c"> </span>ﬁlters<span class="_ _8"></span>;<span class="_ _c"> </span>(ii)</div><div class="t m0 x8 h3 y2e ff2 fs1 fc0 sc0 ls2d ws0">passing<span class="_ _2"> </span>the<span class="_ _7"> </span>resp<span class="_ _8"></span>onses<span class="_ _2"> </span>throug<span class="ls2">h<span class="_ _2"> </span>a<span class="_ _7"> </span>rectiﬁed<span class="_ _2"> </span>linear<span class="_ _2"> </span>function<span class="_ _7"> </span>(<span class="ffc ls41">relu</span><span class="ls5">(<span class="_ _1"></span><span class="ffc">x<span class="ff2 ls42">)=m<span class="_ _17"></span>a<span class="_ _17"></span>x<span class="_ _17"></span>(<span class="_ _17"></span><span class="ffc ls43">x,<span class="_ _10"> </span><span class="ff2 ls44">0));</span></span></span></span></span></span></div><div class="t m0 x8 h3 y2f ff2 fs1 fc0 sc0 ls17 ws0">(iii)<span class="_ _13"> </span>[optiona<span class="_ _8"></span>lly]<span class="_ _13"> </span>max<span class="_ _13"> </span>p<span class="_ _8"></span>o<span class="_ _8"></span>o<span class="_ _8"></span>ling<span class="_ _13"> </span>ov<span class="_ _1"></span>er<span class="_ _13"> </span>lo<span class="_ _8"></span>cal<span class="_ _13"> </span>neig<span class="_ _8"></span>h<span class="_ _1"></span>b<span class="_ _8"></span>or<span class="_ _8"></span>ho<span class="_ _8"></span>o<span class="_ _8"></span>ds<span class="_ _e"> </span>a<span class="_ _8"></span>nd<span class="_ _13"> </span>(iv)<span class="_ _13"> </span>[o<span class="_ _8"></span>ptionally]<span class="_ _13"> </span>a</div><div class="t m0 x8 h3 y30 ff2 fs1 fc0 sc0 ls35 ws0">local<span class="_ _2"> </span>con<span class="_ _1"></span>trast<span class="_ _2"> </span>operation<span class="_ _4"> </span>that<span class="_ _2"> </span>normalizes<span class="_ _4"> </span>the<span class="_ _2"> </span>responses<span class="_ _2"> </span>across<span class="_ _4"> </span>feature<span class="_ _4"> </span>maps.<span class="_ _2"> </span>F<span class="_ _3"></span>or</div><div class="t m0 x8 h3 y31 ff2 fs1 fc0 sc0 ls3e ws0">more<span class="_ _4"> </span>details<span class="_ _4"> </span>o<span class="_ _8"></span>f<span class="_ _c"> </span>thes<span class="_ _8"></span>e<span class="_ _4"> </span>op<span class="_ _8"></span>era<span class="_ _8"></span>tions,<span class="_ _4"> </span>see<span class="_ _4"> </span>[18]<span class="_ _4"> </span>and<span class="_ _4"> </span>[1<span class="_ _8"></span>6].<span class="_ _4"> </span>The<span class="_ _4"> </span>top<span class="_ _4"> </span>few<span class="_ _4"> </span>layers<span class="_ _c"> </span>of<span class="_ _4"> </span>the<span class="_ _2"> </span>net-</div><div class="t m0 x8 h3 y32 ff2 fs1 fc0 sc0 ls21 ws0">w<span class="_ _1"></span>ork<span class="_ _2"> </span>are<span class="_ _2"> </span>con<span class="_ _1"></span>ve<span class="_ _1"></span>ntion<span class="_ _1"></span>al<span class="_ _2"> </span>ful<span class="_ _1"></span>ly-connected<span class="_ _2"> </span>net<span class="_ _5"></span>works<span class="_ _4"> </span>and<span class="_ _2"> </span>the<span class="_ _2"> </span>ﬁnal<span class="_ _4"> </span>lay<span class="_ _1"></span>er<span class="_ _2"> </span>is<span class="_ _2"> </span>a<span class="_ _2"> </span>soft<span class="_ _1"></span>max</div><div class="t m0 x8 h3 y33 ff2 fs1 fc0 sc0 ls45 ws0">classiﬁer.<span class="_ _4"> </span>Fig.<span class="_ _2"> </span>3<span class="_ _2"> </span>shows<span class="_ _4"> </span>the<span class="_ _2"> </span>model<span class="_ _2"> </span>used<span class="_ _2"> </span>in<span class="_ _2"> </span>many<span class="_ _4"> </span>of<span class="_ _2"> </span>our<span class="_ _2"> </span>experiments.</div><div class="t m0 x9 h3 y34 ff2 fs1 fc0 sc0 ls34 ws0">W<span class="_ _3"></span>e<span class="_ _c"> </span>train<span class="_ _b"> </span>these<span class="_ _b"> </span>mo<span class="_ _8"></span>dels<span class="_ _b"> </span>using<span class="_ _b"> </span>a<span class="_ _c"> </span>larg<span class="_ _8"></span>e<span class="_ _d"> </span>set<span class="_ _b"> </span>of<span class="_ _b"> </span><span class="ffc ls5">N<span class="_ _7"> </span><span class="ff2">lab<span class="_ _8"></span>eled<span class="_ _b"> </span>ima<span class="_ _8"></span>ges<span class="_ _b"> </span><span class="ffe">{</span></span><span class="ls43">x,<span class="_ _10"> </span>y<span class="_ _8"></span></span><span class="ffe">}</span></span>,<span class="_ _b"> </span>where<span class="_ _b"> </span>la<span class="_ _8"></span>bel</div><div class="t m0 x8 h9 y35 ffc fs1 fc0 sc0 ls5 ws0">y</div><div class="t m0 x10 h8 y57 ffd fs4 fc0 sc0 ls5 ws0">i</div><div class="t m0 x9 h3 y58 ff2 fs1 fc0 sc0 ls31 ws0">is<span class="_ _2"> </span>a<span class="_ _7"> </span>discrete<span class="_ _7"> </span>v<span class="_ _5"></span>ar<span class="_ _8"></span>iable<span class="_ _2"> </span>indicating<span class="_ _7"> </span>the<span class="_ _2"> </span>tr<span class="_ _8"></span>ue<span class="_ _2"> </span>class.<span class="_ _7"> </span>A<span class="_ _2"> </span>cro<span class="_ _8"></span>ss-entrop<span class="_ _1"></span>y<span class="_ _4"> </span>lo<span class="_ _8"></span>ss<span class="_ _2"> </span>function,</div><div class="t m0 x8 h3 y59 ff2 fs1 fc0 sc0 ls21 ws0">suitabl<span class="_ _1"></span>e<span class="_ _e"> </span>for<span class="_ _2"> </span>image<span class="_ _7"> </span>classiﬁcation,<span class="_ _4"> </span>is<span class="_ _e"> </span>used<span class="_ _2"> </span>to<span class="_ _e"> </span>compare<span class="_"> </span>ˆ<span class="_ _16"></span><span class="ffc ls5">y</span></div><div class="t m0 x11 h8 y5a ffd fs4 fc0 sc0 ls5 ws0">i</div><div class="t m0 x12 h3 y5b ff2 fs1 fc0 sc0 ls46 ws0">and<span class="_ _2"> </span><span class="ffc ls5">y</span></div><div class="t m0 x13 h8 y5a ffd fs4 fc0 sc0 ls5 ws0">i</div><div class="t m0 x14 h3 y5b ff2 fs1 fc0 sc0 ls39 ws0">.<span class="_ _7"> </span>The<span class="_ _e"> </span>para<span class="_ _8"></span>meters</div><div class="t m0 x8 h3 y5c ff2 fs1 fc0 sc0 ls5 ws0">of<span class="_ _7"> </span>the<span class="_ _7"> </span>netw<span class="_ _1"></span>ork<span class="_ _7"> </span>(ﬁlters<span class="_ _7"> </span>in<span class="_ _e"> </span>the<span class="_ _7"> </span>conv<span class="_ _1"></span>olutiona<span class="_ _8"></span>l<span class="_ _2"> </span>lay<span class="_ _1"></span>ers,<span class="_ _2"> </span>weight<span class="_ _2"> </span>matrices<span class="_ _7"> </span>in<span class="_ _7"> </span>the<span class="_ _e"> </span>fully-</div><div class="t m0 x8 h3 y5d ff2 fs1 fc0 sc0 ls35 ws0">connected<span class="_ _2"> </span>lay<span class="_ _1"></span>ers<span class="_ _2"> </span>and<span class="_ _7"> </span>biases)<span class="_ _e"> </span>are<span class="_ _2"> </span>trained<span class="_ _7"> </span>by<span class="_ _7"> </span>bac<span class="_ _1"></span>k-propagating<span class="_ _4"> </span>the<span class="_ _e"> </span>deriv<span class="_ _5"></span>ativ<span class="_ _1"></span>e<span class="_ _7"> </span>of</div><div class="t m0 x8 h3 y5e ff2 fs1 fc0 sc0 ls35 ws0">the<span class="_ _2"> </span>loss<span class="_ _7"> </span>with<span class="_ _2"> </span>resp<span class="_ _8"></span>ect<span class="_ _7"> </span>to<span class="_ _7"> </span>the<span class="_ _2"> </span>parameters<span class="_ _2"> </span>throughout<span class="_ _2"> </span>the<span class="_ _2"> </span>net<span class="_ _1"></span>work,<span class="_ _2"> </span>and<span class="_ _2"> </span>up<span class="_ _8"></span>dating</div><div class="t m0 x8 h3 y5f ff2 fs1 fc0 sc0 ls21 ws0">the<span class="_ _2"> </span>parameters<span class="_ _2"> </span>via<span class="_ _7"> </span>sto<span class="_ _8"></span>c<span class="_ _1"></span>hastic<span class="_ _2"> </span>gradient<span class="_ _2"> </span>descen<span class="_ _1"></span>t.<span class="_ _2"> </span>Details<span class="_ _2"> </span>of<span class="_ _7"> </span>training<span class="_ _2"> </span>are<span class="_ _7"> </span>given<span class="_ _2"> </span>in</div><div class="t m0 x8 h3 y60 ff2 fs1 fc0 sc0 ls47 ws0">Section<span class="_ _4"> </span>3<span class="_ _8"></span>.</div><div class="t m0 x8 h7 y61 ffb fs1 fc0 sc0 ls19 ws0">2.1<span class="_ _14"> </span>Visualiz<span class="_ _1"></span>ation<span class="_ _2"> </span>with<span class="_ _e"> </span>a<span class="_ _7"> </span>Decon<span class="_ _5"></span>vnet</div><div class="t m0 x8 h3 y62 ff2 fs1 fc0 sc0 ls39 ws0">Understanding<span class="_ _c"> </span>the<span class="_ _c"> </span>op<span class="_ _8"></span>eratio<span class="_ _8"></span>n<span class="_ _d"> </span>o<span class="_ _8"></span>f<span class="_ _b"> </span>a<span class="_ _c"> </span>convnet<span class="_ _b"> </span>requires<span class="_ _b"> </span>interpreting<span class="_ _b"> </span>the<span class="_ _c"> </span>feature<span class="_ _c"> </span>ac<span class="_ _8"></span>tiv-</div><div class="t m0 x8 h3 y63 ff2 fs1 fc0 sc0 ls48 ws0">it<span class="_ _1"></span>y<span class="_ _10"> </span>in<span class="_ _10"> </span>intermediate<span class="_ _10"> </span>la<span class="_ _1"></span>yers.<span class="_ _18"> </span>W<span class="_ _3"></span>e<span class="_ _d"> </span>presen<span class="_ _1"></span>t<span class="_ _10"> </span>a<span class="_ _10"> </span>nov<span class="_ _1"></span>el<span class="_ _18"> </span>way<span class="_ _18"> </span>to<span class="_ _10"> </span><span class="ff8 ls2f">map<span class="_ _10"> </span>these<span class="_ _18"> </span>activiti<span class="_ _1"></span>es<span class="_ _d"> </span>b<span class="_ _5"></span>ack<span class="_ _10"> </span>to<span class="_ _10"> </span>the</span></div><div class="t m0 x8 h3 y64 ff8 fs1 fc0 sc0 ls49 ws0">input<span class="_ _b"> </span>pix<span class="_ _8"></span>el<span class="_ _b"> </span>spac<span class="_ _5"></span>e<span class="ff2 ls23">,<span class="_ _b"> </span>showing<span class="_ _d"> </span>wha<span class="_ _8"></span>t<span class="_ _d"> </span>input<span class="_ _c"> </span>pattern<span class="_ _b"> </span>originally<span class="_ _b"> </span>caused<span class="_ _b"> </span>a<span class="_ _b"> </span>given<span class="_ _d"> </span>activ<span class="_ _5"></span>a<span class="_ _8"></span>tion</span></div><div class="t m0 x8 h3 y65 ff2 fs1 fc0 sc0 ls2c ws0">in<span class="_ _2"> </span>the<span class="_ _4"> </span>featur<span class="_ _8"></span>e<span class="_ _4"> </span>maps.<span class="_ _2"> </span>W<span class="_ _3"></span>e<span class="_ _2"> </span>p<span class="_ _8"></span>erfor<span class="_ _8"></span>m<span class="_ _2"> </span>this<span class="_ _4"> </span>mapping<span class="_ _2"> </span>with<span class="_ _2"> </span>a<span class="_ _4"> </span>Deconv<span class="_ _1"></span>ol<span class="_ _8"></span>utional<span class="_ _4"> </span>Netw<span class="_ _1"></span>ork</div><div class="t m0 x8 h3 y66 ff2 fs1 fc0 sc0 ls23 ws0">(deconvnet)<span class="_ _c"> </span>Zeiler<span class="_ _c"> </span><span class="ff8 ls18">et<span class="_ _4"> </span>al.<span class="_ _2"> </span></span><span class="ls33">[29].<span class="_ _b"> </span>A<span class="_ _4"> </span>deconvnet<span class="_ _b"> </span>can<span class="_ _4"> </span>be<span class="_ _4"> </span>thought<span class="_ _c"> </span>of<span class="_ _c"> </span>a<span class="_ _8"></span>s<span class="_ _c"> </span>a<span class="_ _4"> </span>con<span class="_ _1"></span>vnet<span class="_ _4"> </span>mo<span class="_ _8"></span>del</span></div><div class="t m0 x8 h3 y67 ff2 fs1 fc0 sc0 ls34 ws0">that<span class="_ _2"> </span>uses<span class="_ _7"> </span>the<span class="_ _7"> </span>same<span class="_ _7"> </span>comp<span class="_ _8"></span>onents<span class="_ _2"> </span>(ﬁltering,<span class="_ _2"> </span>p<span class="_ _8"></span>o<span class="_ _8"></span>oling<span class="_ _8"></span>)<span class="_ _2"> </span>but<span class="_ _7"> </span>in<span class="_ _7"> </span>reverse,<span class="_ _2"> </span>so<span class="_ _2"> </span>instead<span class="_ _7"> </span>of</div><div class="t m0 x8 h3 y68 ff2 fs1 fc0 sc0 ls23 ws0">mapping<span class="_ _b"> </span>pixels<span class="_ _d"> </span>to<span class="_ _b"> </span>features<span class="_ _b"> </span>do<span class="_ _8"></span>es<span class="_ _b"> </span>the<span class="_ _b"> </span>opp<span class="_ _8"></span>osite.<span class="_ _b"> </span>In<span class="_ _b"> </span>Zeiler<span class="_ _d"> </span><span class="ff8 ls18">et<span class="_ _b"> </span>al.<span class="_ _b"> </span></span><span class="ls25">[29],<span class="_ _b"> </span>decon<span class="_ _1"></span>vnets<span class="_ _b"> </span>were</span></div><div class="t m0 x8 h3 y69 ff2 fs1 fc0 sc0 ls3a ws0">prop<span class="_ _8"></span>osed<span class="_ _4"> </span>as<span class="_ _4"> </span>a<span class="_ _4"> </span>way<span class="_ _c"> </span>of<span class="_ _4"> </span>p<span class="_ _8"></span>erforming<span class="_ _4"> </span>unsup<span class="_ _8"></span>ervised<span class="_ _2"> </span>learning.<span class="_ _4"> </span>Here,<span class="_ _4"> </span>they<span class="_ _4"> </span>a<span class="_ _8"></span>re<span class="_ _c"> </span>no<span class="_ _8"></span>t<span class="_ _4"> </span>used</div><div class="t m0 x8 h3 y6a ff2 fs1 fc0 sc0 ls4a ws0">in<span class="_ _b"> </span>any<span class="_ _b"> </span>learning<span class="_ _b"> </span>capacit<span class="_ _1"></span>y<span class="_ _5"></span>,<span class="_ _b"> </span>just<span class="_ _c"> </span>as<span class="_ _c"> </span>a<span class="_ _c"> </span>prob<span class="_ _8"></span>e<span class="_ _c"> </span>of<span class="_ _b"> </span>an<span class="_ _c"> </span>already<span class="_ _b"> </span>trained<span class="_ _c"> </span>con<span class="_ _1"></span>vnet.</div><div class="t m0 x9 h3 y6b ff2 fs1 fc0 sc0 ls3a ws0">T<span class="_ _5"></span>o<span class="_ _2"> </span>examine<span class="_ _7"> </span>a<span class="_ _2"> </span>convnet,<span class="_ _2"> </span>a<span class="_ _2"> </span>deconvnet<span class="_ _2"> </span>is<span class="_ _2"> </span>a<span class="_ _8"></span>ttached<span class="_ _4"> </span>to<span class="_ _7"> </span>each<span class="_ _2"> </span>of<span class="_ _2"> </span>its<span class="_ _7"> </span>lay<span class="_ _1"></span>ers,<span class="_ _2"> </span>as<span class="_ _7"> </span>illus-</div><div class="t m0 x8 h3 y6c ff2 fs1 fc0 sc0 ls4b ws0">trated<span class="_ _b"> </span>in<span class="_ _b"> </span>Fig.<span class="_ _b"> </span>1(top),<span class="_ _b"> </span>providin<span class="_ _1"></span>g<span class="_ _b"> </span>a<span class="_ _c"> </span>cont<span class="_ _1"></span>inuou<span class="_ _1"></span>s<span class="_ _c"> </span>path<span class="_ _b"> </span>bac<span class="_ _1"></span>k<span class="_ _c"> </span>to<span class="_ _b"> </span>image<span class="_ _b"> </span>pixels.<span class="_ _b"> </span>T<span class="_ _5"></span>o<span class="_ _c"> </span>start,</div><div class="t m0 x8 h3 y6d ff2 fs1 fc0 sc0 ls17 ws0">an<span class="_ _7"> </span>input<span class="_ _e"> </span>image<span class="_ _7"> </span>is<span class="_ _7"> </span>presented<span class="_ _2"> </span>to<span class="_ _e"> </span>the<span class="_ _7"> </span>convnet<span class="_ _2"> </span>and<span class="_ _e"> </span>features<span class="_ _7"> </span>computed<span class="_ _e"> </span>throughout</div><div class="t m0 x8 h3 y6e ff2 fs1 fc0 sc0 ls2f ws0">the<span class="_ _c"> </span>lay<span class="_ _1"></span>ers.<span class="_ _b"> </span>T<span class="_ _5"></span>o<span class="_ _4"> </span>examine<span class="_ _c"> </span>a<span class="_ _4"> </span>given<span class="_ _b"> </span>convnet<span class="_ _b"> </span>activ<span class="_ _5"></span>ation,<span class="_ _c"> </span>we<span class="_ _c"> </span>set<span class="_ _4"> </span>all<span class="_ _c"> </span>other<span class="_ _c"> </span>activ<span class="_ _5"></span>a<span class="_ _8"></span>tions<span class="_ _c"> </span>in</div><div class="t m0 x8 h3 y6f ff2 fs1 fc0 sc0 ls22 ws0">the<span class="_ _2"> </span>lay<span class="_ _1"></span>er<span class="_ _2"> </span>to<span class="_ _2"> </span>zero<span class="_ _2"> </span>and<span class="_ _2"> </span>pass<span class="_ _2"> </span>the<span class="_ _7"> </span>feature<span class="_ _2"> </span>maps<span class="_ _7"> </span>as<span class="_ _2"> </span>input<span class="_ _2"> </span>to<span class="_ _2"> </span>the<span class="_ _7"> </span>attached<span class="_ _4"> </span>deco<span class="_ _8"></span>nvnet</div><div class="t m0 x8 h3 y70 ff2 fs1 fc0 sc0 ls1b ws0">lay<span class="_ _1"></span>er.<span class="_ _2"> </span>Then<span class="_ _7"> </span>we<span class="_ _2"> </span>successively<span class="_ _2"> </span>(i)<span class="_ _2"> </span>unp<span class="_ _8"></span>o<span class="_ _8"></span>ol,<span class="_ _7"> </span>(ii)<span class="_ _e"> </span>rectify<span class="_ _2"> </span>a<span class="_ _8"></span>nd<span class="_ _2"> </span>(iii)<span class="_ _e"> </span>ﬁlter<span class="_ _2"> </span>to<span class="_ _7"> </span>r<span class="_ _8"></span>econstruct</div><div class="t m0 x8 h3 y71 ff2 fs1 fc0 sc0 ls17 ws0">the<span class="_ _2"> </span>activity<span class="_ _4"> </span>in<span class="_ _2"> </span>the<span class="_ _2"> </span>lay<span class="_ _1"></span>er<span class="_ _2"> </span>beneath<span class="_ _2"> </span>that<span class="_ _2"> </span>gave<span class="_ _c"> </span>r<span class="_ _8"></span>ise<span class="_ _4"> </span>to<span class="_ _2"> </span>the<span class="_ _2"> </span>chosen<span class="_ _4"> </span>a<span class="_ _8"></span>ctiv<span class="_ _5"></span>a<span class="_ _8"></span>tion.<span class="_ _2"> </span>This<span class="_ _4"> </span>is</div><div class="t m0 x8 h3 y72 ff2 fs1 fc0 sc0 ls5 ws0">then<span class="_ _2"> </span>rep<span class="_ _8"></span>eated<span class="_ _2"> </span>un<span class="_ _1"></span>til<span class="_ _2"> </span>input<span class="_ _7"> </span>pixel<span class="_ _2"> </span>space<span class="_ _2"> </span>is<span class="_ _2"> </span>reached.</div><div class="t m0 x8 h3 y73 ffb fs1 fc0 sc0 ls1e ws0">Unpo<span class="_ _a"></span>oling:<span class="_ _4"> </span><span class="ff2 ls20">In<span class="_ _7"> </span>the<span class="_ _2"> </span>co<span class="_ _8"></span>nvnet,<span class="_ _2"> </span>the<span class="_ _2"> </span>max<span class="_ _7"> </span>po<span class="_ _a"></span>oling<span class="_ _2"> </span>o<span class="_ _8"></span>p<span class="_ _8"></span>eration<span class="_ _2"> </span>is<span class="_ _7"> </span>non-inv<span class="_ _1"></span>ertible,<span class="_ _2"> </span>how-</span></div><div class="t m0 x8 h3 y74 ff2 fs1 fc0 sc0 ls48 ws0">ever<span class="_ _13"> </span>we<span class="_ _13"> </span>can<span class="_ _12"> </span>obtain<span class="_ _12"> </span>an<span class="_ _12"> </span>approximate<span class="_ _13"> </span>inv<span class="_ _1"></span>er<span class="ls4c">se<span class="_ _13"> </span>by<span class="_ _12"> </span>recording<span class="_ _13"> </span>the<span class="_ _12"> </span>lo<span class="_ _a"></span>cations<span class="_ _12"> </span>of<span class="_ _13"> </span>the</span></div><div class="t m0 x8 h3 y75 ff2 fs1 fc0 sc0 ls34 ws0">maxima<span class="_ _e"> </span>within<span class="_ _13"> </span>each<span class="_ _e"> </span>p<span class="_ _8"></span>o<span class="_ _8"></span>oling<span class="_ _13"> </span>regio<span class="_ _8"></span>n<span class="_ _e"> </span>in<span class="_ _e"> </span>a<span class="_ _13"> </span>set<span class="_ _e"> </span>o<span class="_ _8"></span>f<span class="_ _e"> </span><span class="ff8 ls4d">switch<span class="_ _13"> </span></span><span class="ls4c">v<span class="_ _1"></span>ariables.<span class="_ _e"> </span>In<span class="_ _13"> </span>the<span class="_ _13"> </span>decon-</span></div><div class="t m0 x8 h3 y76 ff2 fs1 fc0 sc0 ls5 ws0">vnet,<span class="_ _e"> </span>the<span class="_ _e"> </span>unpo<span class="_ _8"></span>oling<span class="_ _13"> </span>op<span class="_ _8"></span>eratio<span class="_ _8"></span>n<span class="_ _7"> </span>uses<span class="_ _e"> </span>these<span class="_ _e"> </span>switches<span class="_ _7"> </span>to<span class="_ _e"> </span>place<span class="_ _e"> </span>the<span class="_ _e"> </span>reco<span class="_ _8"></span>nstructions</div><div class="t m0 x8 h3 y77 ff2 fs1 fc0 sc0 ls20 ws0">from<span class="_ _2"> </span>the<span class="_ _7"> </span>lay<span class="_ _1"></span>er<span class="_ _2"> </span>ab<span class="_ _8"></span>ov<span class="_ _1"></span>e<span class="_ _2"> </span>int<span class="_ _1"></span>o<span class="_ _2"> </span>a<span class="_ _8"></span>ppropria<span class="_ _8"></span>te<span class="_ _2"> </span>loca<span class="_ _8"></span>tions,<span class="_ _2"> </span>preserving<span class="_ _2"> </span>the<span class="_ _2"> </span>structur<span class="_ _8"></span>e<span class="_ _2"> </span>of<span class="_ _2"> </span>the</div><div class="t m0 x8 h3 y78 ff2 fs1 fc0 sc0 ls2c ws0">stim<span class="_ _1"></span>ulus.<span class="_ _2"> </span>See<span class="_ _2"> </span>Fig.<span class="_ _2"> </span>1(b<span class="_ _8"></span>ottom)<span class="_ _7"> </span>for<span class="_ _2"> </span>an<span class="_ _2"> </span>illustration<span class="_ _2"> </span>of<span class="_ _2"> </span>the<span class="_ _2"> </span>pro<span class="_ _8"></span>cedur<span class="_ _8"></span>e.</div><div class="t m0 x8 h3 y79 ffb fs1 fc0 sc0 ls4e ws0">Rectiﬁ<span class="_ _8"></span>cation:<span class="_ _0"> </span><span class="ff2 ls45">The<span class="_"> </span>convnet<span class="_"> </span>uses<span class="_ _6"> </span><span class="ff8 ls4f">re<span class="_ _a"></span>l<span class="_ _a"></span>u<span class="_ _19"> </span></span><span class="ls20">non-linearities,<span class="_ _0"> </span>whic<span class="_ _1"></span>h<span class="_ _0"> </span>rectify<span class="_ _6"> </span>the<span class="_ _0"> </span>fea-</span></span></div><div class="t m0 x8 h3 y7a ff2 fs1 fc0 sc0 ls32 ws0">ture<span class="_ _7"> </span>ma<span class="_ _8"></span>ps<span class="_ _7"> </span>thus<span class="_ _7"> </span>ensur<span class="_ _8"></span>ing<span class="_ _7"> </span>the<span class="_ _e"> </span>feature<span class="_ _7"> </span>ma<span class="_ _8"></span>ps<span class="_ _7"> </span>ar<span class="_ _8"></span>e<span class="_ _7"> </span>always<span class="_ _2"> </span>p<span class="_ _8"></span>ositive.<span class="_ _7"> </span>T<span class="_ _5"></span>o<span class="_ _7"> </span>o<span class="_ _8"></span>btain<span class="_ _7"> </span>v<span class="_ _1"></span>alid</div><a class="l"><div class="d m1"></div></a><a class="l"><div class="d m1"></div></a><a class="l"><div class="d m1"></div></a><a class="l"><div class="d m1"></div></a><a class="l"><div class="d m1"></div></a><a class="l"><div class="d m1"></div></a><a class="l"><div class="d m1"></div></a><a class="l"><div class="d m1"></div></a></div><div class="pi" data-data='{"ctm":[2.037137,0.000000,0.000000,2.037137,0.000000,0.000000]}'></div></div><div id="pf4" class="pf w0 h0" data-page-no="4"><div class="pc pc4 w0 h0"><img class="bi x0 y0 w1 h1" alt="" src="/image.php?url=https://csdnimg.cn/release/download_crawler_static/89567205/bg4.jpg"><div class="t m0 xb h4 y2b ff3 fs2 fc0 sc0 ls28 ws0">Visual<span class="_ _1"></span>izing<span class="_ _4"> </span>and<span class="_ _4"> </span>Understanding<span class="_ _b"> </span>Con<span class="_ _1"></span>voluti<span class="_ _1"></span>onal<span class="_ _4"> </span>Netw<span class="_ _5"></span>orks<span class="_ _11"> </span>821</div><div class="t m0 x8 h3 y7b ff2 fs1 fc0 sc0 ls32 ws0">feature<span class="_ _c"> </span>recons<span class="_ _8"></span>tructions<span class="_ _b"> </span>a<span class="_ _8"></span>t<span class="_ _c"> </span>each<span class="_ _b"> </span>layer<span class="_ _d"> </span>(which<span class="_ _c"> </span>also<span class="_ _c"> </span>sho<span class="_ _8"></span>uld<span class="_ _b"> </span>b<span class="_ _8"></span>e<span class="_ _c"> </span>p<span class="_ _8"></span>o<span class="_ _8"></span>sitive),<span class="_ _b"> </span>we<span class="_ _b"> </span>pa<span class="_ _8"></span>ss<span class="_ _c"> </span>the</div><div class="t m0 x8 h3 y7c ff2 fs1 fc0 sc0 ls23 ws0">reconstructed<span class="_ _2"> </span>signal<span class="_ _2"> </span>through<span class="_ _2"> </span>a<span class="_ _2"> </span><span class="ff8 ls4f">re<span class="_ _8"></span>l<span class="_ _a"></span>u<span class="_ _e"> </span></span>non-linear<span class="_ _8"></span>it<span class="_ _1"></span>y</div><div class="t m0 x15 h6 y7d ff9 fs4 fc0 sc0 ls5 ws0">1</div><div class="t m0 x16 h3 y7e ff2 fs1 fc0 sc0 ls5 ws0">.</div><div class="t m0 x8 h3 y7f ffb fs1 fc0 sc0 ls50 ws0">Filtering:<span class="_ _2"> </span><span class="ff2 ls1e">The<span class="_ _7"> </span>convnet<span class="_ _2"> </span>uses<span class="_ _7"> </span>learned<span class="_ _2"> </span>ﬁlters<span class="_ _7"> </span>to<span class="_ _e"> </span>con<span class="_ _1"></span>volv<span class="_ _1"></span>e<span class="_ _2"> </span>the<span class="_ _7"> </span>feature<span class="_ _7"> </span>maps<span class="_ _7"> </span>from</span></div><div class="t m0 x8 h3 y80 ff2 fs1 fc0 sc0 ls51 ws0">th<span class="_ _8"></span>e<span class="_ _2"> </span>p<span class="_ _8"></span>r<span class="_ _8"></span>e<span class="_ _8"></span>vi<span class="_ _8"></span>o<span class="_ _8"></span>u<span class="_ _8"></span>s<span class="_ _4"> </span>l<span class="_ _8"></span>aye<span class="_ _8"></span>r<span class="_ _8"></span>.<span class="_ _2"> </span>T<span class="_ _5"></span>o<span class="_ _2"> </span>a<span class="_ _8"></span>p<span class="_ _8"></span>pr<span class="_ _8"></span>ox<span class="_ _8"></span>i<span class="_ _8"></span>ma<span class="_ _8"></span>t<span class="_ _8"></span>e<span class="_ _8"></span>ly<span class="_ _2"> </span>inv<span class="ls2">ert<span class="_ _4"> </span>this,<span class="_ _2"> </span>the<span class="_ _2"> </span>decon<span class="_ _1"></span>vnet<span class="_ _2"> </span>uses<span class="_ _4"> </span>transp<span class="_ _8"></span>osed</span></div><div class="t m0 x8 h3 y81 ff2 fs1 fc0 sc0 ls23 ws0">versions<span class="_ _2"> </span>of<span class="_ _e"> </span>the<span class="_ _7"> </span>same<span class="_ _e"> </span>ﬁlters<span class="_ _7"> </span>(as<span class="_ _7"> </span>o<span class="_ _8"></span>ther<span class="_ _7"> </span>auto<span class="_ _8"></span>enco<span class="_ _8"></span>der<span class="_ _7"> </span>mo<span class="_ _a"></span>dels,<span class="_ _7"> </span>such<span class="_ _7"> </span>as<span class="_ _7"> </span>RBMs),<span class="_ _7"> </span>but</div><div class="t m0 x8 h3 y82 ff2 fs1 fc0 sc0 ls20 ws0">applied<span class="_ _e"> </span>to<span class="_ _7"> </span>the<span class="_ _e"> </span>rectiﬁed<span class="_ _e"> </span>maps,<span class="_ _e"> </span>not<span class="_ _e"> </span>the<span class="_ _e"> </span>output<span class="_ _e"> </span>of<span class="_ _e"> </span>the<span class="_ _e"> </span>lay<span class="_ _1"></span>er<span class="_ _7"> </span>b<span class="_ _8"></span>eneath.<span class="_ _e"> </span>In<span class="_ _e"> </span>practice</div><div class="t m0 x8 h3 y83 ff2 fs1 fc0 sc0 ls23 ws0">this<span class="_ _2"> </span>means<span class="_ _2"> </span>ﬂipping<span class="_ _2"> </span>each<span class="_ _4"> </span>ﬁlter<span class="_ _2"> </span>vertically<span class="_ _4"> </span>a<span class="_ _8"></span>nd<span class="_ _2"> </span>horizontally<span class="_ _5"></span>.</div><div class="t m0 x9 h3 y84 ff2 fs1 fc0 sc0 ls52 ws0">Note<span class="_ _2"> </span>that<span class="_ _2"> </span>we<span class="_ _4"> </span>do<span class="_ _7"> </span>not<span class="_ _2"> </span>use<span class="_ _2"> </span>an<span class="_ _1"></span>y<span class="_ _7"> </span>con<span class="_ _1"></span>trast<span class="_ _2"> </span>normali<span class="_ _1"></span>zation<span class="_ _2"> </span>operations<span class="_ _4"> </span>when<span class="_ _2"> </span>in<span class="_ _7"> </span>this</div><div class="t m0 x8 h3 y85 ff2 fs1 fc0 sc0 ls5 ws0">reconstruction<span class="_ _c"> </span>path.<span class="_ _4"> </span>Pr<span class="_ _8"></span>o<span class="_ _a"></span>jecting<span class="_ _b"> </span>down<span class="_ _c"> </span>from<span class="_ _4"> </span>higher<span class="_ _c"> </span>lay<span class="_ _1"></span>ers<span class="_ _c"> </span>uses<span class="_ _4"> </span>the<span class="_ _c"> </span>switch<span class="_ _b"> </span>setting<span class="_ _8"></span>s</div><div class="t m0 x8 h3 y86 ff2 fs1 fc0 sc0 ls39 ws0">generated<span class="_ _e"> </span>by<span class="_ _e"> </span>the<span class="_ _e"> </span>ma<span class="_ _8"></span>x<span class="_ _e"> </span>p<span class="_ _8"></span>o<span class="_ _8"></span>oling<span class="_ _e"> </span>in<span class="_ _13"> </span>the<span class="_ _13"> </span>convnet<span class="_ _e"> </span>on<span class="_ _e"> </span>the<span class="_ _13"> </span>wa<span class="_ _1"></span>y<span class="_ _e"> </span>up.<span class="_ _e"> </span>As<span class="_ _13"> </span>these<span class="_ _e"> </span>s<span class="_ _8"></span>witch</div><div class="t m0 x8 h3 y87 ff2 fs1 fc0 sc0 ls22 ws0">settings<span class="_ _4"> </span>a<span class="_ _8"></span>re<span class="_ _4"> </span>p<span class="_ _8"></span>eculiar<span class="_ _2"> </span>to<span class="_ _4"> </span>a<span class="_ _2"> </span>giv<span class="_ _1"></span>en<span class="_ _2"> </span>input<span class="_ _4"> </span>ima<span class="_ _8"></span>ge,<span class="_ _4"> </span>the<span class="_ _2"> </span>reconstructio<span class="_ _8"></span>n<span class="_ _4"> </span>obtained<span class="_ _4"> </span>fro<span class="_ _8"></span>m<span class="_ _4"> </span>a</div><div class="t m0 x8 h3 y88 ff2 fs1 fc0 sc0 ls22 ws0">single<span class="_ _7"> </span>activ<span class="_ _1"></span>ation<span class="_ _7"> </span>thus<span class="_ _2"> </span>re<span class="_ _8"></span>sem<span class="_ _1"></span>bles<span class="_ _7"> </span>a<span class="_ _7"> </span>small<span class="_ _7"> </span>piece<span class="_ _e"> </span>of<span class="_ _7"> </span>the<span class="_ _7"> </span>or<span class="_ _8"></span>iginal<span class="_ _2"> </span>input<span class="_ _e"> </span>image,<span class="_ _7"> </span>with</div><div class="t m0 x8 h3 y89 ff2 fs1 fc0 sc0 ls53 ws0">structures<span class="_ _7"> </span>weigh<span class="_ _1"></span>ted<span class="_ _2"> </span>according<span class="_ _e"> </span>to<span class="_ _7"> </span>their<span class="_ _e"> </span><span class="ls2c">contribution<span class="_ _2"> </span>tow<span class="_ _1"></span>ar<span class="_ _8"></span>d<span class="_ _7"> </span>to<span class="_ _7"> </span>the<span class="_ _e"> </span>feature<span class="_ _e"> </span>acti-</span></div><div class="t m0 x8 h3 y8a ff2 fs1 fc0 sc0 ls5 ws0">v<span class="_ _5"></span>atio<span class="_ _8"></span>n.<span class="_ _7"> </span>Since<span class="_ _e"> </span>the<span class="_ _e"> </span>mo<span class="_ _8"></span>del<span class="_ _7"> </span>is<span class="_ _e"> </span>trained<span class="_ _e"> </span>discriminatively<span class="_ _3"></span>,<span class="_ _7"> </span>they<span class="_ _e"> </span>implicitly<span class="_ _e"> </span>show<span class="_ _2"> </span>which</div><div class="t m0 x8 h3 y8b ff2 fs1 fc0 sc0 ls2c ws0">parts<span class="_ _2"> </span>of<span class="_ _2"> </span>the<span class="_ _2"> </span>input<span class="_ _2"> </span>image<span class="_ _4"> </span>a<span class="_ _8"></span>re<span class="_ _2"> </span>discriminative.<span class="_ _4"> </span>Note<span class="_ _2"> </span>that<span class="_ _2"> </span>these<span class="_ _2"> </span>pro<span class="_ _1a"></span>jections<span class="_ _4"> </span>a<span class="_ _8"></span>re<span class="_ _2"> </span><span class="ff8 ls2f">not</span></div><div class="t m0 x8 h3 y8c ff2 fs1 fc0 sc0 ls1b ws0">samples<span class="_ _b"> </span>fro<span class="_ _8"></span>m<span class="_ _b"> </span>the<span class="_ _c"> </span>mo<span class="_ _8"></span>del,<span class="_ _c"> </span>since<span class="_ _c"> </span>there<span class="_ _c"> </span>is<span class="_ _c"> </span>no<span class="_ _c"> </span>genera<span class="_ _8"></span>tive<span class="_ _d"> </span>pro<span class="_ _8"></span>cess<span class="_ _c"> </span>inv<span class="_ _1"></span>olved.<span class="_ _b"> </span>The<span class="_ _b"> </span>who<span class="_ _8"></span>le</div><div class="t m0 x8 h3 y8d ff2 fs1 fc0 sc0 ls2c ws0">pro<span class="_ _8"></span>cedure<span class="_ _2"> </span>is<span class="_ _4"> </span>similar<span class="_ _2"> </span>to<span class="_ _2"> </span>backpropping<span class="_ _4"> </span>a<span class="_ _2"> </span>single<span class="_ _2"> </span>strong<span class="_ _4"> </span>activ<span class="_ _1"></span>ation<span class="_ _2"> </span>(rather<span class="_ _2"> </span>than<span class="_ _4"> </span>the</div><div class="t m0 x8 h3 y8e ff2 fs1 fc0 sc0 ls17 ws0">usual<span class="_ _4"> </span>g<span class="_ _8"></span>radients),<span class="_ _c"> </span>i.e.<span class="_ _2"> </span>computing</div><div class="t m0 x17 h8 y8f ffd fs4 fc0 sc0 ls54 ws0">∂h</div><div class="t m0 x18 h8 y90 ffd fs4 fc0 sc0 ls54 ws0">∂X</div><div class="t m0 x19 ha y91 fff fs5 fc0 sc0 ls5 ws0">n</div><div class="t m0 x1a h3 y92 ff2 fs1 fc0 sc0 ls55 ws0">,w<span class="_ _17"></span>h<span class="_ _17"></span>e<span class="_ _17"></span>r<span class="_ _17"></span>e<span class="ffc ls5">h<span class="_ _4"> </span></span><span class="ls39">is<span class="_ _2"> </span>the<span class="_ _2"> </span>elemen<span class="_ _1"></span>t<span class="_ _2"> </span>of<span class="_ _4"> </span>the<span class="_ _4"> </span>featur<span class="_ _8"></span>e<span class="_ _4"> </span>ma<span class="_ _8"></span>p</span></div><div class="t m0 x8 h3 y93 ff2 fs1 fc0 sc0 ls56 ws0">with<span class="_ _12"> </span>the<span class="_"> </span>strong<span class="_ _12"> </span>activ<span class="_ _1"></span>ation<span class="_ _12"> </span>a<span class="_ _8"></span>nd<span class="_ _12"> </span><span class="ffc ls5">X</span></div><div class="t m0 x1b h8 y94 ffd fs4 fc0 sc0 ls5 ws0">n</div><div class="t m0 x1c h3 y95 ff2 fs1 fc0 sc0 ls17 ws0">is<span class="_ _12"> </span>the<span class="_"> </span>input<span class="_ _12"> </span>image.<span class="_ _12"> </span>How<span class="_ _1"></span>ever,<span class="_ _13"> </span>it<span class="_"> </span>diﬀers<span class="_ _12"> </span>in</div><div class="t m0 x8 h3 y96 ff2 fs1 fc0 sc0 ls57 ws0">that<span class="_ _e"> </span>(i)<span class="_ _13"> </span>the<span class="_ _13"> </span>the<span class="_ _13"> </span><span class="ff8 ls4f">re<span class="_ _a"></span>l<span class="_ _8"></span>u<span class="_ _9"> </span></span><span class="ls3a">is<span class="_ _e"> </span>imp<span class="_ _8"></span>o<span class="_ _8"></span>sed<span class="_ _e"> </span>indep<span class="_ _8"></span>endently<span class="_ _e"> </span>a<span class="_ _8"></span>nd<span class="_ _e"> </span>(ii)<span class="_ _13"> </span>contrast<span class="_ _7"> </span>nor<span class="_ _8"></span>malization</span></div><div class="t m0 x8 h3 y97 ff2 fs1 fc0 sc0 ls4b ws0">operations<span class="_ _2"> </span>are<span class="_ _2"> </span>not<span class="_ _2"> </span>used.<span class="_ _2"> </span>A<span class="_ _e"> </span>general<span class="_ _4"> </span>shortcoming<span class="_ _2"> </span>of<span class="_ _2"> </span>our<span class="_ _2"> </span>approach<span class="_ _4"> </span>is<span class="_ _7"> </span>that<span class="_ _2"> </span>it<span class="_ _2"> </span>only</div><div class="t m0 x8 h3 y98 ff2 fs1 fc0 sc0 ls2d ws0">visualizes<span class="_ _4"> </span>a<span class="_ _2"> </span>single<span class="_ _2"> </span>activ<span class="_ _5"></span>ation,<span class="_ _2"> </span>not<span class="_ _4"> </span>the<span class="_ _2"> </span>joint<span class="_ _4"> </span>activity<span class="_ _4"> </span>present<span class="_ _c"> </span>in<span class="_ _2"> </span>a<span class="_ _2"> </span>la<span class="_ _1"></span>yer.<span class="_ _4"> </span>Neverthe-</div><div class="t m0 x8 h3 y99 ff2 fs1 fc0 sc0 ls36 ws0">less,<span class="_ _7"> </span>as<span class="_ _13"> </span>we<span class="_ _7"> </span>show<span class="_ _7"> </span>in<span class="_ _13"> </span>Fig.<span class="_ _e"> </span>6,<span class="_ _e"> </span>these<span class="_ _e"> </span>visualizations<span class="_ _2"> </span>are<span class="_ _e"> </span>a<span class="_ _8"></span>ccurate<span class="_ _7"> </span>representat<span class="_ _1"></span>ions<span class="_ _e"> </span>of</div><div class="t m0 x8 h3 y9a ff2 fs1 fc0 sc0 ls2c ws0">the<span class="_ _4"> </span>input<span class="_ _2"> </span>pattern<span class="_ _2"> </span>that<span class="_ _4"> </span>stimulates<span class="_ _4"> </span>the<span class="_ _2"> </span>given<span class="_ _4"> </span>feature<span class="_ _2"> </span>map<span class="_ _4"> </span>in<span class="_ _4"> </span>the<span class="_ _2"> </span>mo<span class="_ _8"></span>del:<span class="_ _2"> </span>when<span class="_ _4"> </span>the</div><div class="t m0 x8 h3 y9b ff2 fs1 fc0 sc0 ls22 ws0">parts<span class="_ _2"> </span>of<span class="_ _2"> </span>the<span class="_ _4"> </span>o<span class="_ _8"></span>riginal<span class="_ _2"> </span>input<span class="_ _2"> </span>image<span class="_ _2"> </span>corresp<span class="_ _8"></span>onding<span class="_ _2"> </span>to<span class="_ _2"> </span>the<span class="_ _2"> </span>pattern<span class="_ _2"> </span>are<span class="_ _2"> </span>o<span class="_ _8"></span>ccluded,<span class="_ _4"> </span>we</div><div class="t m0 x8 h3 y9c ff2 fs1 fc0 sc0 ls2c ws0">see<span class="_ _2"> </span>a<span class="_ _2"> </span>distinct<span class="_ _2"> </span>drop<span class="_ _2"> </span>in<span class="_ _2"> </span>activity<span class="_ _4"> </span>within<span class="_ _2"> </span>the<span class="_ _2"> </span>fea<span class="_ _8"></span>ture<span class="_ _2"> </span>map.</div><div class="t m0 x8 h5 y9d ff1 fs3 fc0 sc0 ls58 ws0">3<span class="_ _f"> </span>T<span class="_ _3"></span>raining<span class="_ _12"> </span>Det<span class="_ _8"></span>ails</div><div class="t m0 x8 h3 y9e ff2 fs1 fc0 sc0 ls31 ws0">W<span class="_ _3"></span>e<span class="_ _e"> </span>now<span class="_ _7"> </span>describ<span class="_ _8"></span>e<span class="_ _e"> </span>the<span class="_ _e"> </span>lar<span class="_ _8"></span>ge<span class="_ _7"> </span>convnet<span class="_ _7"> </span>mo<span class="_ _8"></span>del<span class="_ _e"> </span>that<span class="_ _e"> </span>will<span class="_ _7"> </span>b<span class="_ _8"></span>e<span class="_ _e"> </span>visua<span class="_ _8"></span>lized<span class="_ _7"> </span>in<span class="_ _e"> </span>Section<span class="_ _e"> </span>4.</div><div class="t m0 x8 h3 y9f ff2 fs1 fc0 sc0 ls4c ws0">The<span class="_ _b"> </span>architecture,<span class="_ _d"> </span>shown<span class="_ _d"> </span>in<span class="_ _b"> </span>Fig.<span class="_ _b"> </span>3,<span class="_ _b"> </span>i<span class="ls59">s<span class="_ _c"> </span>similar<span class="_ _b"> </span>to<span class="_ _b"> </span>that<span class="_ _b"> </span>used<span class="_ _c"> </span>by<span class="_ _d"> </span>Krizhevsky<span class="_ _b"> </span><span class="ff8 ls18">et<span class="_ _4"> </span>al.<span class="_ _4"> </span></span><span class="ls17">[18]</span></span></div><div class="t m0 x8 h3 ya0 ff2 fs1 fc0 sc0 ls17 ws0">for<span class="_ _e"> </span>ImageNet<span class="_ _e"> </span>cla<span class="_ _8"></span>ssiﬁcation.<span class="_ _e"> </span>One<span class="_ _e"> </span>diﬀerence<span class="_ _e"> </span>is<span class="_ _13"> </span>that<span class="_ _e"> </span>the<span class="_ _13"> </span>sparse<span class="_ _e"> </span>connections<span class="_ _e"> </span>used</div><div class="t m0 x8 h3 ya1 ff2 fs1 fc0 sc0 ls3a ws0">in<span class="_ _e"> </span>Krizhevsky’s<span class="_ _e"> </span>lay<span class="_ _1"></span>ers<span class="_ _e"> </span>3,4,5<span class="_ _e"> </span>(due<span class="_ _e"> </span>to<span class="_ _13"> </span>the<span class="_ _e"> </span>mo<span class="_ _8"></span>del<span class="_ _e"> </span>b<span class="_ _8"></span>eing<span class="_ _13"> </span>split<span class="_ _e"> </span>acr<span class="_ _8"></span>oss<span class="_ _e"> </span>2<span class="_ _e"> </span>GPUs)<span class="_ _e"> </span>are</div><div class="t m0 x8 h3 ya2 ff2 fs1 fc0 sc0 ls5 ws0">replaced<span class="_ _7"> </span>with<span class="_ _e"> </span>dense<span class="_ _e"> </span>connections<span class="_ _e"> </span>in<span class="_ _e"> </span>our<span class="_ _7"> </span>m<span class="ls5a">o<span class="_ _8"></span>del.<span class="_ _e"> </span>Other<span class="_ _e"> </span>imp<span class="_ _8"></span>ortant<span class="_ _2"> </span>diﬀerences<span class="_ _7"> </span>re-</span></div><div class="t m0 x8 h3 ya3 ff2 fs1 fc0 sc0 ls5 ws0">lating<span class="_ _7"> </span>to<span class="_ _7"> </span>lay<span class="_ _1"></span>ers<span class="_ _2"> </span>1<span class="_ _e"> </span>and<span class="_ _2"> </span>2<span class="_ _e"> </span>w<span class="_ _1"></span>ere<span class="_ _7"> </span>made<span class="_ _7"> </span>following<span class="_ _2"> </span>insp<span class="_ _8"></span>ection<span class="_ _7"> </span>of<span class="_ _7"> </span>the<span class="_ _7"> </span>visualiza<span class="_ _8"></span>tions<span class="_ _2"> </span>in</div><div class="t m0 x8 h3 ya4 ff2 fs1 fc0 sc0 ls1d ws0">Fig.<span class="_ _4"> </span>5,<span class="_ _2"> </span>as<span class="_ _2"> </span>described<span class="_ _4"> </span>in<span class="_ _2"> </span>Section<span class="_ _4"> </span>4.1.</div><div class="t m0 x9 h3 ya5 ff2 fs1 fc0 sc0 ls5b ws0">The<span class="_ _c"> </span>mo<span class="_ _8"></span>del<span class="_ _c"> </span>w<span class="_ _1"></span>as<span class="_ _c"> </span>trained<span class="_ _b"> </span>on<span class="_ _c"> </span>the<span class="_ _c"> </span>ImageNe<span class="ls5c">t<span class="_ _b"> </span>2012<span class="_ _c"> </span>training<span class="_ _b"> </span>set<span class="_ _c"> </span>(1.3<span class="_ _c"> </span>million<span class="_ _c"> </span>images,</span></div><div class="t m0 x8 h3 ya6 ff2 fs1 fc0 sc0 ls1a ws0">spread<span class="_ _10"> </span>ov<span class="_ _1"></span>er<span class="_ _10"> </span>1000<span class="_ _10"> </span>diﬀerent<span class="_ _10"> </span>classes)<span class="_ _10"> </span>[6].<span class="_ _10"> </span>Each<span class="_ _10"> </span>R<span class="_ _1"></span>GB<span class="_ _d"> </span>image<span class="_ _10"> </span>was<span class="_ _10"> </span>preprocessed<span class="_ _10"> </span>by<span class="_ _10"> </span>resiz-</div><div class="t m0 x8 h3 ya7 ff2 fs1 fc0 sc0 ls35 ws0">ing<span class="_ _c"> </span>the<span class="_ _c"> </span>smallest<span class="_ _b"> </span>dimension<span class="_ _c"> </span>to<span class="_ _c"> </span>256,<span class="_ _c"> </span>cropping<span class="_ _b"> </span>the<span class="_ _4"> </span>cen<span class="_ _1"></span>ter<span class="_ _c"> </span>256x256<span class="_ _b"> </span>region,<span class="_ _c"> </span>subtract-</div><div class="t m0 x8 h3 ya8 ff2 fs1 fc0 sc0 ls1c ws0">ing<span class="_ _4"> </span>the<span class="_ _2"> </span>p<span class="_ _8"></span>er-pixel<span class="_ _4"> </span>mea<span class="_ _8"></span>n<span class="_ _4"> </span>(acro<span class="_ _8"></span>ss<span class="_ _4"> </span>all<span class="_ _4"> </span>ima<span class="_ _8"></span>ges)<span class="_ _4"> </span>and<span class="_ _2"> </span>then<span class="_ _4"> </span>using<span class="_ _2"> </span>10<span class="_ _4"> </span>diﬀer<span class="_ _8"></span>ent<span class="_ _c"> </span>sub-cro<span class="_ _8"></span>ps</div><div class="t m0 x8 h3 ya9 ff2 fs1 fc0 sc0 ls1f ws0">of<span class="_ _c"> </span>size<span class="_ _c"> </span>224x224<span class="_ _b"> </span>(corners<span class="_ _b"> </span>+<span class="_ _c"> </span>center<span class="_ _c"> </span>with(<span class="_ _1"></span>out)<span class="_ _c"> </span>horizon<span class="_ _1"></span>tal<span class="_ _c"> </span>ﬂips).<span class="_ _b"> </span>Sto<span class="_ _8"></span>ch<span class="_ _1"></span>astic<span class="_ _c"> </span>gradien<span class="_ _1"></span>t</div><div class="t m0 x8 h3 yaa ff2 fs1 fc0 sc0 ls1d ws0">descen<span class="_ _1"></span>t<span class="_ _d"> </span>with<span class="_ _b"> </span>a<span class="_ _b"> </span>mini-b<span class="_ _1"></span>atch<span class="_ _d"> </span>size<span class="_ _d"> </span>of<span class="_ _d"> </span>128<span class="_ _b"> </span>was<span class="_ _d"> </span>used<span class="_ _d"> </span>to<span class="_ _b"> </span>update<span class="_ _d"> </span>the<span class="_ _b"> </span>parameters,<span class="_ _d"> </span>start<span class="_ _1"></span>ing</div><div class="t m0 x8 h3 yab ff2 fs1 fc0 sc0 ls20 ws0">with<span class="_ _2"> </span>a<span class="_ _2"> </span>lea<span class="_ _8"></span>rning<span class="_ _2"> </span>rate<span class="_ _2"> </span>of<span class="_ _2"> </span>10</div><div class="t m0 x6 h6 yac ffa fs4 fc0 sc0 ls5 ws0">−<span class="ff9">2</span></div><div class="t m0 x1d h3 yad ff2 fs1 fc0 sc0 ls1e ws0">,<span class="_ _2"> </span>in<span class="_ _2"> </span>conjun<span class="_ _1"></span>ction<span class="_ _2"> </span>with<span class="_ _2"> </span>a<span class="_ _2"> </span>momen<span class="_ _1"></span>tum<span class="_ _4"> </span>term<span class="_ _2"> </span>of<span class="_ _2"> </span>0<span class="ffc ls5">.</span><span class="ls46">9.<span class="_ _2"> </span>W<span class="_ _3"></span>e</span></div><div class="t m0 x1e hb yae ff10 fs6 fc0 sc0 ls5 ws0">1</div><div class="t m0 x1f h4 yaf ff3 fs2 fc0 sc0 lsb ws0">W<span class="_ _5"></span>e<span class="_ _e"> </span>also<span class="_"> </span>tried<span class="_"> </span>rectifying<span class="_"> </span>using<span class="_"> </span>the<span class="_ _e"> </span>binary<span class="_"> </span>mask<span class="_"> </span>imposed<span class="_"> </span>by<span class="_ _e"> </span>the<span class="_"> </span>f<span class="_ _1"></span>eed-<span class="_ _8"></span>forw<span class="_ _1"></span>ard<span class="_"> </span><span class="ff7 ls5d">re<span class="_ _a"></span>l<span class="_ _a"></span>u</span></div><div class="t m0 x1f h4 yb0 ff3 fs2 fc0 sc0 ls3c ws0">op<span class="_ _8"></span>eration,<span class="_ _2"> </span>but<span class="_ _2"> </span>the<span class="_ _4"> </span>resultin<span class="_ _8"></span>g<span class="_ _4"> </span>visu<span class="_ _8"></span>alizations<span class="_ _2"> </span>were<span class="_ _4"> </span>signiﬁcantly<span class="_ _2"> </span>less<span class="_ _4"> </span>clear.</div><a class="l"><div class="d m1"></div></a><a class="l"><div class="d m1"></div></a><a class="l"><div class="d m1"></div></a><a class="l"><div class="d m1"></div></a><a class="l"><div class="d m1"></div></a><a class="l"><div class="d m1"></div></a><a class="l"><div class="d m1"></div></a><a class="l"><div class="d m1"></div></a></div><div class="pi" data-data='{"ctm":[2.037137,0.000000,0.000000,2.037137,0.000000,0.000000]}'></div></div><div id="pf5" class="pf w0 h0" data-page-no="5"><div class="pc pc5 w0 h0"><img class="bi x0 y0 w1 h1" alt="" src="/image.php?url=https://csdnimg.cn/release/download_crawler_static/89567205/bg5.jpg"><div class="t m0 x8 h4 y2b ff3 fs2 fc0 sc0 ls3d ws0">822<span class="_ _15"> </span>M.D.<span class="_ _4"> </span>Zeile<span class="_ _1"></span>r<span class="_ _2"> </span>and<span class="_ _c"> </span>R.<span class="_ _4"> </span>F<span class="_ _5"></span>ergus</div><div class="t m2 x20 hc yb1 ff11 fs7 fc0 sc0 ls5e ws1">Layer Below P<span class="_ _1"></span>ooled Maps </div><div class="t m2 x21 hc yb2 ff11 fs7 fc0 sc0 ls5f ws2">Feature Maps </div><div class="t m2 x22 hc yb3 ff11 fs7 fc0 sc0 ls5 ws0">Rectied Feature Maps </div><div class="t m3 x23 hd yb4 ff12 fs8 fc0 sc0 ls5 ws0"></div><div class="t m3 x23 hd yb5 ff12 fs8 fc0 sc0 ls5 ws0">!"</div><div class="t m3 x23 hd yb6 ff12 fs8 fc0 sc0 ls5 ws0"></div><div class="t m3 x23 hd yb7 ff12 fs8 fc0 sc0 ls60 ws0"></div><div class="t m2 x24 hc yb8 ff11 fs7 fc0 sc0 ls5 ws0">Pooled Maps </div><div class="t m3 x23 hd yb9 ff12 fs8 fc0 sc0 ls5 ws0"></div><div class="t m2 x3 hc yba ff11 fs7 fc0 sc0 ls61 ws0">Reconstruction </div><div class="t m2 x25 hc ybb ff11 fs7 fc0 sc0 ls5e ws1">Rectied Unpooled Maps </div><div class="t m2 x3 hc ybc ff11 fs7 fc0 sc0 ls5 ws0">Unpooled Maps </div><div class="t m3 x26 hd ybd ff12 fs8 fc0 sc0 ls5 ws0"></div><div class="t m3 x27 hd ybe ff12 fs8 fc0 sc0 ls5 ws0">!</div><div class="t m3 x28 he ybf ff12 fs9 fc0 sc0 ls5 ws0"></div><div class="t m3 x29 hd yc0 ff12 fs8 fc0 sc0 ls5 ws0">"</div><div class="t m3 x2a hd yc1 ff12 fs8 fc0 sc0 ls5 ws0"></div><div class="t m3 x2b hd yc2 ff12 fs8 fc0 sc0 ls60 ws0"></div><div class="t m2 x2c hc yc3 ff11 fs7 fc0 sc0 ls62 ws3">Layer Above </div><div class="t m2 x3 hc yc4 ff11 fs7 fc0 sc0 ls61 ws0">Reconstruction </div><div class="t m3 x2d hd yc5 ff12 fs8 fc0 sc0 ls63 ws0"></div><div class="t m3 x2e hd yc6 ff12 fs8 fc0 sc0 ls5 ws0"></div><div class="t m4 x2f hf yc7 ff13 fsa fc1 sc0 ls5 ws0">Unpooling </div><div class="t m5 x30 h10 yc8 ff13 fsb fc1 sc0 ls64 ws4">Max Locations </div><div class="t m5 x31 h10 yc9 ff13 fsb fc1 sc0 ls65 ws0">“Switches”<span class="_ _5"></span> </div><div class="t m4 x32 hf yca ff13 fsa fc1 sc0 ls5 ws0">P<span class="_ _1"></span>ooling </div><div class="t m6 x33 h11 ycb ff13 fsc fc1 sc0 ls5 ws0">P<span class="_ _1"></span>ooled Maps </div><div class="t m7 x34 h12 ycc ff13 fsd fc2 sc0 ls5 ws0">Feature Map </div><div class="t m6 x35 h11 ycd ff13 fsc fc1 sc0 ls66 ws5">Layer Above </div><div class="t m6 x36 h11 yce ff13 fsc fc1 sc0 ls67 ws0">Reconstruction </div><div class="t m6 x1b h11 ycf ff13 fsc fc1 sc0 ls5 ws0">Unpooled </div><div class="t m6 x1b h11 yd0 ff13 fsc fc1 sc0 ls5 ws0">Maps </div><div class="t m6 x37 h11 ycf ff13 fsc fc1 sc0 ls5 ws0">Rectiﬁed </div><div class="t m6 x38 h11 yd0 ff13 fsc fc1 sc0 ls5 ws0">Featur<span class="_ _1"></span>e Maps </div><div class="t m0 x8 h4 yd1 ff6 fs2 fc0 sc0 ls68 ws0">Fig.<span class="_ _10"> </span>1.<span class="_ _d"> </span><span class="ff3 ls14">T<span class="_ _5"></span>op<span class="_ _8"></span>:<span class="_ _d"> </span>A<span class="_ _b"> </span>deconvnet<span class="_ _b"> </span>la<span class="_ _1"></span>yer<span class="_ _d"> </span>(left)<span class="_ _b"> </span>attached<span class="_ _d"> </span>to<span class="_ _b"> </span>a<span class="_ _b"> </span>convnet<span class="_ _d"> </span>lay<span class="_ _1"></span>er<span class="_ _b"> </span>(right).<span class="_ _d"> </span>The<span class="_ _b"> </span>deconvnet</span></div><div class="t m0 x8 h4 yd2 ff3 fs2 fc0 sc0 ls69 ws0">will<span class="_ _b"> </span>recon<span class="_ _8"></span>struct<span class="_ _c"> </span>an<span class="_ _b"> </span>ap<span class="_ _8"></span>proximate<span class="_ _b"> </span>version<span class="_ _b"> </span>of<span class="_ _c"> </span>the<span class="_ _b"> </span>convnet<span class="_ _b"> </span>featu<span class="_ _8"></span>res<span class="_ _b"> </span>from<span class="_ _c"> </span>the<span class="_ _c"> </span>lay<span class="_ _1"></span>er<span class="_ _b"> </span>b<span class="_ _8"></span>en<span class="_ _8"></span>eath.</div><div class="t m0 x8 h4 yd3 ff3 fs2 fc0 sc0 ls12 ws0">Bottom:<span class="_ _e"> </span>An<span class="_ _7"> </span>illust<span class="_ _8"></span>ration<span class="_ _e"> </span>of<span class="_ _e"> </span>the<span class="_ _e"> </span>unp<span class="_ _8"></span>o<span class="_ _8"></span>oling<span class="_ _e"> </span>op<span class="_ _8"></span>eration<span class="_ _e"> </span>in<span class="_ _e"> </span>the<span class="_ _7"> </span>d<span class="_ _8"></span>econvnet,<span class="_ _7"> </span>usin<span class="_ _8"></span>g<span class="_ _7"> </span><span class="ff7 ls9">switches</span></div><div class="t m0 x8 h4 yd4 ff3 fs2 fc0 sc0 ls15 ws0">which<span class="_ _d"> </span>record<span class="_ _c"> </span>the<span class="_ _b"> </span>lo<span class="_ _8"></span>cation<span class="_ _c"> </span>of<span class="_ _b"> </span>the<span class="_ _b"> </span>lo<span class="_ _8"></span>cal<span class="_ _b"> </span>max<span class="_ _c"> </span>in<span class="_ _b"> </span>each<span class="_ _d"> </span>p<span class="_ _8"></span>o<span class="_ _8"></span>olin<span class="_ _8"></span>g<span class="_ _b"> </span>region<span class="_ _c"> </span>(colored<span class="_ _b"> </span>zon<span class="_ _8"></span>es)<span class="_ _b"> </span>dur<span class="_ _8"></span>ing</div><div class="t m0 x8 h4 yd5 ff3 fs2 fc0 sc0 ls6a ws0">p<span class="_ _8"></span>o<span class="_ _8"></span>oling<span class="_ _2"> </span>in<span class="_ _4"> </span>th<span class="_ _8"></span>e<span class="_ _4"> </span>convnet<span class="_ _8"></span>.<span class="_ _4"> </span>The<span class="_ _2"> </span>black/white<span class="_ _2"> </span>bars<span class="_ _4"> </span>are<span class="_ _2"> </span>negat<span class="_ _8"></span>ive/positive<span class="_ _2"> </span>activ<span class="_ _1"></span>ations<span class="_ _2"> </span>within</div><div class="t m0 x8 h4 yd6 ff3 fs2 fc0 sc0 ls6b ws0">the<span class="_ _4"> </span>feature<span class="_ _c"> </span>map.</div><div class="t m0 x8 h3 yd7 ff2 fs1 fc0 sc0 ls19 ws0">anneal<span class="_ _b"> </span>the<span class="_ _b"> </span>learning<span class="_ _c"> </span>rate<span class="_ _b"> </span>throughout<span class="_ _b"> </span>trainin<span class="_ _1"></span>g<span class="_ _c"> </span>manual<span class="_ _1"></span>ly<span class="_ _c"> </span>when<span class="_ _c"> </span>the<span class="_ _c"> </span>v<span class="_ _5"></span>alidation<span class="_ _b"> </span>error</div><div class="t m0 x8 h3 yd8 ff2 fs1 fc0 sc0 ls23 ws0">plateaus.<span class="_ _4"> </span>Drop<span class="_ _8"></span>out<span class="_ _c"> </span>[1<span class="_ _8"></span>4]<span class="_ _c"> </span>is<span class="_ _4"> </span>used<span class="_ _4"> </span>in<span class="_ _4"> </span>the<span class="_ _4"> </span>fully<span class="_ _4"> </span>connected<span class="_ _4"> </span>lay<span class="_ _1"></span>ers<span class="_ _c"> </span>(6<span class="_ _4"> </span>and<span class="_ _4"> </span>7)<span class="_ _4"> </span>with<span class="_ _4"> </span>a<span class="_ _4"> </span>ra<span class="_ _8"></span>te</div><div class="t m0 x8 h3 yd9 ff2 fs1 fc0 sc0 ls50 ws0">of<span class="_ _b"> </span>0.5.<span class="_ _c"> </span>All<span class="_ _c"> </span>wei<span class="_ _1"></span>ghts<span class="_ _b"> </span>are<span class="_ _b"> </span>initialized<span class="_ _b"> </span>to<span class="_ _c"> </span>10</div><div class="t m0 x39 h6 yda ffa fs4 fc0 sc0 ls5 ws0">−<span class="ff9">2</span></div><div class="t m0 x3a h3 ydb ff2 fs1 fc0 sc0 ls6c ws0">and<span class="_ _c"> </span>biases<span class="_ _c"> </span>are<span class="_ _b"> </span>set<span class="_ _c"> </span>to<span class="_ _c"> </span>0<span class="_ _8"></span>.</div><div class="t m0 x9 h3 ydc ff2 fs1 fc0 sc0 ls34 ws0">Visualization<span class="_ _12"> </span>o<span class="_ _8"></span>f<span class="_ _12"> </span>the<span class="_ _12"> </span>ﬁr<span class="_ _8"></span>st<span class="_ _12"> </span>lay<span class="_ _1"></span>er<span class="_ _12"> </span>ﬁlters<span class="_"> </span>during<span class="_ _12"> </span>training<span class="_"> </span>rev<span class="_ _1"></span>eals<span class="_ _12"> </span>that<span class="_ _12"> </span>a<span class="_"> </span>few<span class="_ _12"> </span>of</div><div class="t m0 x8 h3 ydd ff2 fs1 fc0 sc0 ls50 ws0">them<span class="_ _4"> </span>dominate.<span class="_ _4"> </span>T<span class="_ _5"></span>o<span class="_ _2"> </span>comb<span class="_ _1"></span>at<span class="_ _2"> </span>this,<span class="_ _4"> </span>we<span class="_ _4"> </span>renormalize<span class="_ _4"> </span>each<span class="_ _4"> </span>ﬁlter<span class="_ _2"> </span>in<span class="_ _4"> </span>the<span class="_ _2"> </span>conv<span class="_ _1"></span>olution<span class="_ _1"></span>al</div><div class="t m0 x8 h3 yde ff2 fs1 fc0 sc0 ls6d ws0">lay<span class="_ _1"></span>ers<span class="_ _c"> </span>whose<span class="_ _4"> </span>RMS<span class="_ _4"> </span>v<span class="_ _5"></span>alue<span class="_ _2"> </span>exceeds<span class="_ _c"> </span>a<span class="_ _4"> </span>ﬁxed<span class="_ _4"> </span>radius<span class="_ _2"> </span>of<span class="_ _c"> </span>10</div><div class="t m0 x3b h6 ydf ffa fs4 fc0 sc0 ls5 ws0">−<span class="ff9">1</span></div><div class="t m0 x23 h3 ye0 ff2 fs1 fc0 sc0 ls17 ws0">to<span class="_ _4"> </span>this<span class="_ _2"> </span>ﬁxed<span class="_ _c"> </span>ra<span class="_ _8"></span>dius.<span class="_ _4"> </span>This</div><div class="t m0 x8 h3 ye1 ff2 fs1 fc0 sc0 ls1b ws0">is<span class="_ _2"> </span>crucia<span class="_ _8"></span>l,<span class="_ _2"> </span>esp<span class="_ _8"></span>ecially<span class="_ _7"> </span>in<span class="_ _7"> </span>the<span class="_ _2"> </span>ﬁrs<span class="_ _8"></span>t<span class="_ _2"> </span>layer<span class="_ _4"> </span>of<span class="_ _7"> </span>the<span class="_ _7"> </span>mo<span class="_ _8"></span>del,<span class="_ _7"> </span>where<span class="_ _2"> </span>the<span class="_ _7"> </span>input<span class="_ _7"> </span>ima<span class="_ _8"></span>ges<span class="_ _2"> </span>a<span class="_ _8"></span>re</div><div class="t m0 x8 h3 ye2 ff2 fs1 fc0 sc0 ls6e ws0">roughly<span class="_ _d"> </span>in<span class="_ _c"> </span>the<span class="_ _c"> </span>[-128,128]<span class="_ _d"> </span>range.<span class="_ _b"> </span>As<span class="_ _b"> </span>in<span class="_ _c"> </span>Krizhevsky<span class="_ _b"> </span><span class="ff8 ls18">et<span class="_ _4"> </span>al.<span class="_ _2"> </span></span><span class="ls22">[18],<span class="_ _b"> </span>we<span class="_ _b"> </span>pr<span class="_ _8"></span>o<span class="_ _8"></span>duce<span class="_ _c"> </span>multiple</span></div><div class="t m0 x8 h3 ye3 ff2 fs1 fc0 sc0 ls1b ws0">diﬀerent<span class="_ _2"> </span>crops<span class="_ _2"> </span>and<span class="_ _2"> </span>ﬂips<span class="_ _2"> </span>of<span class="_ _2"> </span>ea<span class="_ _8"></span>ch<span class="_ _4"> </span>training<span class="_ _7"> </span>example<span class="_ _2"> </span>to<span class="_ _2"> </span>b<span class="_ _8"></span>o<span class="_ _8"></span>ost<span class="_ _7"> </span>training<span class="_ _2"> </span>set<span class="_ _2"> </span>size.<span class="_ _2"> </span>W<span class="_ _5"></span>e</div><div class="t m0 x8 h3 ye4 ff2 fs1 fc0 sc0 ls6e ws0">stopped<span class="_ _b"> </span>training<span class="_ _d"> </span>after<span class="_ _b"> </span>70<span class="_ _b"> </span>ep<span class="_ _8"></span>o<span class="_ _8"></span>ch<span class="_ _1"></span>s,<span class="_ _b"> </span>whic<span class="_ _1"></span>h<span class="_ _b"> </span>to<span class="_ _8"></span>ok<span class="_ _b"> </span>around<span class="_ _b"> </span>12<span class="_ _b"> </span>da<span class="_ _1"></span>ys<span class="_ _b"> </span>on<span class="_ _c"> </span>a<span class="_ _b"> </span>single<span class="_ _d"> </span>GTX580</div><div class="t m0 x8 h3 ye5 ff2 fs1 fc0 sc0 ls20 ws0">GPU,<span class="_ _2"> </span>using<span class="_ _2"> </span>an<span class="_ _2"> </span>implemen<span class="_ _1"></span>tation<span class="_ _2"> </span>ba<span class="_ _8"></span>sed<span class="_ _2"> </span>on<span class="_ _2"> </span>[18].</div><div class="t m0 x8 h5 ye6 ff1 fs3 fc0 sc0 ls4d ws0">4<span class="_ _f"> </span>Con<span class="_ _1"></span>vnet<span class="_ _12"> </span>Visualizat<span class="_ _8"></span>ion</div><div class="t m0 x8 h3 ye7 ff2 fs1 fc0 sc0 ls31 ws0">Using<span class="_ _7"> </span>the<span class="_ _7"> </span>mo<span class="_ _8"></span>del<span class="_ _7"> </span>describ<span class="_ _8"></span>ed<span class="_ _2"> </span>in<span class="_ _7"> </span>Sectio<span class="_ _8"></span>n<span class="_ _2"> </span>3,<span class="_ _7"> </span>we<span class="_ _2"> </span>now<span class="_ _2"> </span>use<span class="_ _7"> </span>the<span class="_ _7"> </span>deconvnet<span class="_ _2"> </span>to<span class="_ _7"> </span>visualize</div><div class="t m0 x8 h3 ye8 ff2 fs1 fc0 sc0 ls1e ws0">the<span class="_ _4"> </span>feature<span class="_ _4"> </span>activ<span class="_ _1"></span>ations<span class="_ _4"> </span>on<span class="_ _4"> </span>the<span class="_ _2"> </span>ImageNet<span class="_ _4"> </span>v<span class="_ _1"></span>alidati<span class="_ _1"></span>on<span class="_ _2"> </span>set.</div><div class="t m0 x8 h3 ye9 ffb fs1 fc0 sc0 ls52 ws0">F<span class="_ _3"></span>eatur<span class="_ _1"></span>e<span class="_ _0"> </span>Visualiz<span class="_ _1"></span>ation:<span class="_ _12"> </span><span class="ff2 ls4b">Fig.<span class="_ _12"> </span>2<span class="_ _12"> </span>shows<span class="_ _13"> </span>feature<span class="_ _13"> </span>visualization<span class="_ _1"></span>s<span class="_ _12"> </span>from<span class="_ _9"> </span>our<span class="_ _12"> </span>model</span></div><div class="t m0 x8 h3 yea ff2 fs1 fc0 sc0 ls1d ws0">once<span class="_ _12"> </span>training<span class="_ _13"> </span>is<span class="_ _12"> </span>complete.<span class="_ _13"> </span>F<span class="_ _5"></span>or<span class="_ _12"> </span>a<span class="_ _9"> </span>giv<span class="_ _1"></span>en<span class="_ _12"> </span>feature<span class="_ _13"> </span>map,<span class="_ _12"> </span>we<span class="_ _13"> </span>show<span class="_ _13"> </span>the<span class="_ _12"> </span>top<span class="_ _12"> </span>9<span class="_ _9"> </span>acti-</div><div class="t m0 x8 h3 yeb ff2 fs1 fc0 sc0 ls34 ws0">v<span class="_ _5"></span>atio<span class="_ _8"></span>ns,<span class="_ _e"> </span>each<span class="_ _e"> </span>pro<span class="_ _a"></span>jected<span class="_ _13"> </span>separately<span class="_ _e"> </span>down<span class="_ _e"> </span>to<span class="_ _13"> </span>pixel<span class="_ _e"> </span>spa<span class="_ _8"></span>ce,<span class="_ _e"> </span>revealing<span class="_ _e"> </span>the<span class="_ _13"> </span>diﬀerent</div><a class="l"><div class="d m1"></div></a><a class="l"><div class="d m1"></div></a><a class="l"><div class="d m1"></div></a><a class="l"><div class="d m1"></div></a><a class="l"><div class="d m1"></div></a></div><div class="pi" data-data='{"ctm":[2.037137,0.000000,0.000000,2.037137,0.000000,0.000000]}'></div></div>

100+评论昵称:

captcha

类型	标题	大小	时间
	中文版VC6.0（32&64bit）rjazz.zip	30.4MB	9月前

	BackToTop 置顶组件（VUE2 后台）	3.47KB	9月前
	LCD12864.zip	2.12MB	9月前
	VMware 全家桶算号器keygen 5-8 (包括 Tanzu、NSX)	65.16KB	9月前
	包含xss攻击的pdf文件	21.38KB	9月前

	LockCop工具（排查死锁问题）	632.11KB	9月前
	iOS MFI认证代码及文档	3.15MB	9月前
	家政保洁上门预约小程序	1.28MB	9月前