[Deep learning] Gradient Descent Optimizer

이미지
https://icim.nims.re.kr/post/easyMath/70 알기 쉬운 산업수학 알기 쉬운 산업수학 Gradient Descent Optimizer 2017년 12월 13일(수) | 김민중 URL  (1) 주어진 목적함수  f f 의 최솟값(minimum)을 찾아가는 알고리즘으로 다음과 같은 방식으로 최솟값을 찾아간다. 초기값  x 0 x 0 을 임의로 설정, 적당한 Learning rate  α α  설정 n ≥ 0 n ≥ 0 인 정수에 대해서  x n + 1 x n + 1 은 다음과 같이 정의한다. x n + 1 : = x n − α ⋅ ∇ f ( x n ) x n + 1 := x n − α ⋅ ∇ f ( x n ) 주의사항 함수 f f 의 모양이 convex가 아닌 경우 global minimum이 아닌 local minimum으로  x n x n 이 수렴할 가능성이 있다. Learning rate  α α  값이 큰 경우 최솟값으로  x n x n 이 수렴하는 것이 아니라 발산할 수 있다. Learning rate  α α  값이 작은 경우 수렴하는 속도가 지나치게 느릴 수 있다. 방정식  2 ⋅ x = 10 2 ⋅ x = 10  의 근을 Gradient Descent를 이용해서 찾아보자. 목적함수  f ( x ) : = ( 10 − 2 x ) 2 f ( x ) := ( 10 − 2 x ) 2 으로 설정하겠다( f f 의 최솟값인 0이 되게 하는  x x 값이 우리가 원하는 방정식의 근이다). 초기값  x 0 = 0 x 0 = 0 , Learning rate  α = 0.05 α = 0.05 으로 설정 f ′ ( x ) = 4 ( 2 x − 10 ) f ′ ( x ) = 4 ( 2 x − 10 )...
댓글 쓰기

[caffe][compile] Allow 32F and 8U data input

0. source : data_transformer.cpp

branch : master <---- has problem data type 32F
branch : opencl <---  it's the answer
https://github.com/BVLC/caffe/blob/opencl/src/caffe/data_transformer.cpp

*** Target source is depend on build tool :  check make or cmake which one is your build tool for your caffe.

1.  allow the data type

// (FTschopp) Fixed for float data
CHECK(cv_img.depth() == CV_8U || cv_img.depth() == CV_32F)
<< "Image data type must be unsigned byte or 4 byte float";

2. convert it , properly

// int_tp top_index = (c * height + h) * width + w;
Dtype pixel;
if (cv_img.depth() == CV_8U) {
pixel = static_cast<Dtype>(ptr[img_index++]);
} else {
pixel = static_cast<Dtype>((reinterpret_cast<const float*>(ptr))
[img_index++]);
}

3. full code
near by 225 line


template<typename Dtype>
void DataTransformer<Dtype>::Transform(const cv::Mat& cv_img,
Blob<Dtype>* transformed_blob) {
const int crop_size = param_.crop_size();
const int img_channels = cv_img.channels();
const int img_height = cv_img.rows;
const int img_width = cv_img.cols;

// Check dimensions.
const int channels = transformed_blob->channels();
const int height = transformed_blob->height();
const int width = transformed_blob->width();
const int num = transformed_blob->num();

CHECK_EQ(channels, img_channels);
CHECK_LE(height, img_height);
CHECK_LE(width, img_width);
CHECK_GE(num, 1);

// CHECK(cv_img.depth() == CV_8U) << "Image data type must be unsigned byte";
// (FTschopp) Fixed for float data
CHECK(cv_img.depth() == CV_8U || cv_img.depth() == CV_32F)
<< "Image data type must be unsigned byte or 4 byte float";

const Dtype scale = param_.scale();
const bool do_mirror = param_.mirror() && Rand(2);
const bool has_mean_file = param_.has_mean_file();
const bool has_mean_values = mean_values_.size() > 0;

CHECK_GT(img_channels, 0);
CHECK_GE(img_height, crop_size);
CHECK_GE(img_width, crop_size);

Dtype* mean = NULL;
if (has_mean_file) {
CHECK_EQ(img_channels, data_mean_.channels());
CHECK_EQ(img_height, data_mean_.height());
CHECK_EQ(img_width, data_mean_.width());
mean = data_mean_.mutable_cpu_data();
}
if (has_mean_values) {
CHECK(mean_values_.size() == 1 || mean_values_.size() == img_channels) <<
"Specify either 1 mean_value or as many as channels: " << img_channels;
if (img_channels > 1 && mean_values_.size() == 1) {
// Replicate the mean_value for simplicity
for (int c = 1; c < img_channels; ++c) {
mean_values_.push_back(mean_values_[0]);
}
}
}

int h_off = 0;
int w_off = 0;
cv::Mat cv_cropped_img = cv_img;
if (crop_size) {
CHECK_EQ(crop_size, height);
CHECK_EQ(crop_size, width);
// We only do random crop when we do training.
if (phase_ == TRAIN) {
h_off = Rand(img_height - crop_size + 1);
w_off = Rand(img_width - crop_size + 1);
} else {
h_off = (img_height - crop_size) / 2;
w_off = (img_width - crop_size) / 2;
}
cv::Rect roi(w_off, h_off, crop_size, crop_size);
cv_cropped_img = cv_img(roi);
} else {
CHECK_EQ(img_height, height);
CHECK_EQ(img_width, width);
}

CHECK(cv_cropped_img.data);

Dtype* transformed_data = transformed_blob->mutable_cpu_data();
int top_index;
for (int h = 0; h < height; ++h) {
const uchar* ptr = cv_cropped_img.ptr<uchar>(h);
int img_index = 0;
for (int w = 0; w < width; ++w) {
for (int c = 0; c < img_channels; ++c) {
if (do_mirror) {
top_index = (c * height + h) * width + (width - 1 - w);
} else {
top_index = (c * height + h) * width + w;
}
// int top_index = (c * height + h) * width + w;
// Dtype pixel = static_cast<Dtype>(ptr[img_index++]);
Dtype pixel;
if (cv_img.depth() == CV_8U) {
pixel = static_cast<Dtype>(ptr[img_index++]);
} else {
pixel = static_cast<Dtype>((reinterpret_cast<const float*>(ptr))
[img_index++]);
}
if (has_mean_file) {
int mean_index = (c * img_height + h_off + h) * img_width + w_off + w;
transformed_data[top_index] =
(pixel - mean[mean_index]) * scale;
} else {
if (has_mean_values) {
transformed_data[top_index] =
(pixel - mean_values_[c]) * scale;
} else {
transformed_data[top_index] = pixel * scale;
}
}
}
}
}
}
#endif // USE_OPENCV

댓글

댓글 쓰기

이 블로그의 인기 게시물

@import' when modules are disabled

이미지
<Error Message> ......./RootViewController.h:13:1: Use of '@import' when modules are disabled [Solution] Xcode -> Build Settings -> Enable Modules -> Yes Xcode -> Build Settings -> Link Frameworks Automatically -> Yes Reference :  http://stackoverflow.com/questions/27479248/ios-use-of-import-when-modules-are-disabled-error
자세한 내용 보기

MagicaVoxel black screen

<Error> When you run magicavoxel that it only gives you a black screen. <Solution> if you get black screen on MacOS, try this: extract the whole folder (go to the "MagicaVoxel" applications folder) move "MagicaVoxel-mac.app" outside the folder move "MagicaVoxel-mac.app" back to the folder again <Reference> raywenderich
자세한 내용 보기

[Anaconda][Opencv][Spyder] How to install opencv to anaconda (spyder).

< Install opencv to anaconda : Method 1> 1. Download opencv from  http://opencv.org/releases.html 2. Extract module to a folder. 3. Copy cv2.pyd from "\opencv\build\python\2.7\x64"                       to    "anaconda install folder\Lib\site-packages" 4. Close anaconda and spyder. 5. Re-run spyder from anaconda. 6. Test it ( from command prompt :  import cv2 ) < Install opencv to anaconda : Method 2> 1. Run spyder 2. Run command window in spyder. 3. activate some environment. ( This is optional ) 4. install opencv with below command. 5. conda install -c menpo opencv36. Test it ( from command prompt : import cv2 ) <Reference> Method 1> https://www.youtube.com/watch?v=9hb0gYCv3YI Method 2> https://github.com/conda/conda/issues/2448 https://www.blogger.com/blogger.g?blogID=8309050393695147092#editor/target=post;postID=335116236004254132;onPublishedMenu=allposts;o...
자세한 내용 보기