[Deep learning] Gradient Descent Optimizer

이미지
https://icim.nims.re.kr/post/easyMath/70 알기 쉬운 산업수학 알기 쉬운 산업수학 Gradient Descent Optimizer 2017년 12월 13일(수) | 김민중 URL  (1) 주어진 목적함수  f f 의 최솟값(minimum)을 찾아가는 알고리즘으로 다음과 같은 방식으로 최솟값을 찾아간다. 초기값  x 0 x 0 을 임의로 설정, 적당한 Learning rate  α α  설정 n ≥ 0 n ≥ 0 인 정수에 대해서  x n + 1 x n + 1 은 다음과 같이 정의한다. x n + 1 : = x n − α ⋅ ∇ f ( x n ) x n + 1 := x n − α ⋅ ∇ f ( x n ) 주의사항 함수 f f 의 모양이 convex가 아닌 경우 global minimum이 아닌 local minimum으로  x n x n 이 수렴할 가능성이 있다. Learning rate  α α  값이 큰 경우 최솟값으로  x n x n 이 수렴하는 것이 아니라 발산할 수 있다. Learning rate  α α  값이 작은 경우 수렴하는 속도가 지나치게 느릴 수 있다. 방정식  2 ⋅ x = 10 2 ⋅ x = 10  의 근을 Gradient Descent를 이용해서 찾아보자. 목적함수  f ( x ) : = ( 10 − 2 x ) 2 f ( x ) := ( 10 − 2 x ) 2 으로 설정하겠다( f f 의 최솟값인 0이 되게 하는  x x 값이 우리가 원하는 방정식의 근이다). 초기값  x 0 = 0 x 0 = 0 , Learning rate  α = 0.05 α = 0.05 으로 설정 f ′ ( x ) = 4 ( 2 x − 10 ) f ′ ( x ) = 4 ( 2 x − 10 )...
댓글 쓰기

[Blogger][Syntax Highlight] How to syntax highlight in blogger

<HowTo>


Edit template

Insert bleow lines

between <head> and </head>


<!-- INCLUDE SyntaxHighlighter Components (css and javascript) -->
<link href='http://alexgorbatchev.com/pub/sh/current/styles/shCore.css' rel='stylesheet' type='text/css'/>
<link href='http://alexgorbatchev.com/pub/sh/current/styles/shThemeDefault.css' rel='stylesheet' type='text/css'/>
<script src='http://alexgorbatchev.com/pub/sh/current/scripts/shCore.js' type='text/javascript'/>
<!-- END INCLUDE SyntaxHighlighter Components -->

<!-- INCLUDE YOUR BRUSHES WHICH YOU WANT TO USE HERE -->
<script src='http://alexgorbatchev.com/pub/sh/current/scripts/shBrushCpp.js' type='text/javascript'/>
<script src='http://alexgorbatchev.com/pub/sh/current/scripts/shBrushCSharp.js' type='text/javascript'/>
<script src='http://alexgorbatchev.com/pub/sh/current/scripts/shBrushXml.js' type='text/javascript'/>
<!-- END INCLUDE BRUSHES -->
<!-- INIT SyntaxHighlighter -->
<script type='text/javascript'>
   SyntaxHighlighter.config.bloggerMode = true;
   SyntaxHighlighter.all();
</script>
<!-- END INIT SyntaxHighlighter -->



< Support Language>


Brush nameBrush aliasesFile name
ActionScript3as3, actionscript3shBrushAS3.js
Bash/shellbash, shellshBrushBash.js
ColdFusioncf, coldfusionshBrushColdFusion.js
C#c-sharp, csharpshBrushCSharp.js
C++cpp, cshBrushCpp.js
CSScssshBrushCss.js
Delphidelphi, pas, pascalshBrushDelphi.js
Diffdiff, patchshBrushDiff.js
Erlangerl, erlangshBrushErlang.js
GroovygroovyshBrushGroovy.js
JavaScriptjs, jscript, javascriptshBrushJScript.js
JavajavashBrushJava.js
JavaFXjfx, javafxshBrushJavaFX.js
Perlperl, plshBrushPerl.js
PHPphpshBrushPhp.js
Plain Textplain, textshBrushPlain.js
PowerShellps, powershellshBrushPowerShell.js
Pythonpy, pythonshBrushPython.js
Rubyrails, ror, rubyshBrushRuby.js
ScalascalashBrushScala.js
SQLsqlshBrushSql.js
Visual Basicvb, vbnetshBrushVb.js
XMLxml, xhtml, xslt, html, xhtmlshBrushXml.js


<Reference>

https://android-expert-blog.blogspot.kr/2015/12/how-to-integrate-syntaxhighlighter-to.html


<Example>

using System;

namespace HelloWorld
{
    class Hello
    {
        public static void Main(String[] args)
         {
             Console.WriteLine("Hello World");
         }
    }
}

<Example Code>


<pre class="brush:csharp" >

using System;

namespace HelloWorld
{
    class Hello
    {
        public static void Main(String[] args)
         {
             Console.WriteLine("Hello World");
         }
    }
}

</pre>

댓글

댓글 쓰기

이 블로그의 인기 게시물

@import' when modules are disabled

이미지
<Error Message> ......./RootViewController.h:13:1: Use of '@import' when modules are disabled [Solution] Xcode -> Build Settings -> Enable Modules -> Yes Xcode -> Build Settings -> Link Frameworks Automatically -> Yes Reference :  http://stackoverflow.com/questions/27479248/ios-use-of-import-when-modules-are-disabled-error
자세한 내용 보기

MagicaVoxel black screen

<Error> When you run magicavoxel that it only gives you a black screen. <Solution> if you get black screen on MacOS, try this: extract the whole folder (go to the "MagicaVoxel" applications folder) move "MagicaVoxel-mac.app" outside the folder move "MagicaVoxel-mac.app" back to the folder again <Reference> raywenderich
자세한 내용 보기

[Anaconda][Opencv][Spyder] How to install opencv to anaconda (spyder).

< Install opencv to anaconda : Method 1> 1. Download opencv from  http://opencv.org/releases.html 2. Extract module to a folder. 3. Copy cv2.pyd from "\opencv\build\python\2.7\x64"                       to    "anaconda install folder\Lib\site-packages" 4. Close anaconda and spyder. 5. Re-run spyder from anaconda. 6. Test it ( from command prompt :  import cv2 ) < Install opencv to anaconda : Method 2> 1. Run spyder 2. Run command window in spyder. 3. activate some environment. ( This is optional ) 4. install opencv with below command. 5. conda install -c menpo opencv36. Test it ( from command prompt : import cv2 ) <Reference> Method 1> https://www.youtube.com/watch?v=9hb0gYCv3YI Method 2> https://github.com/conda/conda/issues/2448 https://www.blogger.com/blogger.g?blogID=8309050393695147092#editor/target=post;postID=335116236004254132;onPublishedMenu=allposts;o...
자세한 내용 보기