Anurag1734/cuda-error-resolution-analysis
07
1[2 {3 "post_stream": {4 "posts": [5 {6 "id": 207046,7 "name": "Andrei Roibu",8 "username": "aroibu1",9 "avatar_template": "/user_avatar/discuss.pytorch.org/aroibu1/{size}/23271_2.png",10 "created_at": "2020-06-26T10:25:16.052Z",11 "cooked": "<p>Firstly, apologise if these are silly questions!</p>\n<p>1 I am wondering what is the default initialisation utilised for Conv layers and is this dependent on the nonlinearity selected for after the layer?<br>\n2. When using a SELU nonlinearity, does the network automatically initialise the weights using the LeCun Normal Initialisation? If not, how could I implement weight initialisation manually to use the LeCun Normal?</p>\n<p>A bit of context:</p>\n<p>Reading through the various blog posts and questions from the past few years, for (1) I managed to find two opposing opinions: either that PyTorch automatically initialises all weights to LeCun Normal, or that PyTorch initialises weights based on the non-linearity used after the Conv Layer (Xavier for Tanh and Kaiming He for ReLU and ReLU derivated). However, when I check the source code (<a href=\"https://github.com/pytorch/pytorch/blob/master/torch/nn/modules/conv.py\" rel=\"nofollow noopener\">https://github.com/pytorch/pytorch/blob/master/torch/nn/modules/conv.py</a>), it appears that the default weight initlisation is Kaiming:</p>\n<pre><code class=\"lang-auto\"> def reset_parameters(self) -> None:\n init.kaiming_uniform_(self.weight, a=math.sqrt(5))\n if self.bias is not None:\n fan_in, _ = init._calculate_fan_in_and_fan_out(self.weight)\n bound = 1 / math.sqrt(fan_in)\n init.uniform_(self.bias, -bound, bound)\n</code></pre>\n<p>In this case, my understanding is that that Kaiming is the default weight initialisation for Conv layers, no matter the following nonlinearity? Thus, most of the posts I have previously read are outdated.</p>\n<p>Then, in order to implement the LeCun Normal initialisation, do I need to rewrite reset_parameters in my own code so that it overwrites the default PyTorch code?</p>",12 "post_number": 1,13 "post_type": 1,14 "posts_count": 8,15 "updated_at": "2020-06-26T10:25:16.052Z",16 "reply_count": 0,17 "reply_to_post_number": null,18 "quote_count": 0,19 "incoming_link_count": 3345,20 "reads": 82,21 "readers_count": 81,22 "score": 16696.4,23 "yours": false,24 "topic_id": 87012,25 "topic_slug": "default-weight-initialisation-for-conv-layers-including-selu",26 "display_username": "Andrei Roibu",27 "primary_group_name": null,28 "flair_name": null,29 "flair_url": null,30 "flair_bg_color": null,31 "flair_color": null,32 "flair_group_id": null,33 "badges_granted": [],34 "version": 1,35 "can_edit": false,36 "can_delete": false,37 "can_recover": false,38 "can_see_hidden_post": false,39 "can_wiki": false,40 "link_counts": [41 {42 "url": "https://github.com/pytorch/pytorch/blob/master/torch/nn/modules/conv.py",43 "internal": false,44 "reflection": false,45 "title": "pytorch/conv.py at master · pytorch/pytorch · GitHub",46 "clicks": 3947 }48 ],49 "read": true,50 "user_title": "",51 "bookmarked": false,52 "actions_summary": [53 {54 "id": 2,55 "count": 156 }57 ],58 "moderator": false,59 "admin": false,60 "staff": false,61 "user_id": 29207,62 "hidden": false,63 "trust_level": 2,64 "deleted_at": null,65 "user_deleted": false,66 "edit_reason": null,67 "can_view_edit_history": true,68 "wiki": false,69 "post_url": "/t/default-weight-initialisation-for-conv-layers-including-selu/87012/1",70 "can_accept_answer": false,71 "can_unaccept_answer": false,72 "accepted_answer": false,73 "topic_accepted_answer": true,74 "can_vote": false75 },76 {77 "id": 207058,78 "name": "Nikan Doosti",79 "username": "Nikronic",80 "avatar_template": "/user_avatar/discuss.pytorch.org/nikronic/{size}/71873_2.png",81 "created_at": "2020-06-26T11:32:58.306Z",82 "cooked": "<p>Hi,</p>\n<p>For the first question, please see these posts:</p>\n<ol>\n<li><a href=\"https://discuss.pytorch.org/t/clarity-on-default-initialization-in-pytorch/84696\" class=\"inline-onebox\">Clarity on default initialization in pytorch</a></li>\n<li><a href=\"https://discuss.pytorch.org/t/cnn-default-initialization-understanding/85353/2\" class=\"inline-onebox\">CNN default initialization understanding</a></li>\n</ol>\n<p>I have explained the magic number <code>math.sqrt(5)</code> so you can also get the idea behind the relation between non-linearity and init method. Acuatlly, default initialization is uniform.</p>\n<p>Also, see this reply in the github thread about it <a href=\"https://github.com/pytorch/pytorch/issues/15314#issuecomment-477448573\" rel=\"nofollow noopener\">https://github.com/pytorch/pytorch/issues/15314#issuecomment-477448573</a></p>\n<p>About the second question, you can reinitialize weights after initializing them using default values. To do so, you can create your init function similar to available cases in <code>torch.nn.init</code> package and use a code similar to following snippet:</p>\n<pre><code class=\"lang-auto\">def init_weights(m):\n \"\"\"\n Initialize weights of layers using Kaiming Normal (He et al.) as argument of \"Apply\" function of\n \"nn.Module\"\n :param m: Layer to initialize\n :return: None\n \"\"\"\n\n if isinstance(m, nn.Conv2d):\n nn.init.kaiming_normal_(m.weight, mode='fan_out')\n nn.init.constant_(m.bias, 0)\n elif isinstance(m, nn.BatchNorm2d):\n nn.init.constant_(m.weight, 1)\n nn.init.constant_(m.bias, 0)\n\nmodel.apply(init_weights)\n</code></pre>\n<p>Bests</p>",83 "post_number": 2,84 "post_type": 1,85 "posts_count": 8,86 "updated_at": "2020-06-27T14:01:10.797Z",87 "reply_count": 1,88 "reply_to_post_number": null,89 "quote_count": 0,90 "incoming_link_count": 58,91 "reads": 75,92 "readers_count": 74,93 "score": 325.0,94 "yours": false,95 "topic_id": 87012,96 "topic_slug": "default-weight-initialisation-for-conv-layers-including-selu",97 "display_username": "Nikan Doosti",98 "primary_group_name": null,99 "flair_name": null,100 "flair_url": null,101 "flair_bg_color": null,102 "flair_color": null,103 "flair_group_id": null,104 "badges_granted": [],105 "version": 1,106 "can_edit": false,107 "can_delete": false,108 "can_recover": false,109 "can_see_hidden_post": false,110 "can_wiki": false,111 "link_counts": [112 {113 "url": "https://discuss.pytorch.org/t/clarity-on-default-initialization-in-pytorch/84696",114 "internal": true,115 "reflection": false,116 "title": "Clarity on default initialization in pytorch",117 "clicks": 251118 },119 {120 "url": "https://discuss.pytorch.org/t/cnn-default-initialization-understanding/85353/2",121 "internal": true,122 "reflection": false,123 "title": "CNN default initialization understanding",124 "clicks": 176125 },126 {127 "url": "https://github.com/pytorch/pytorch/issues/15314#issuecomment-477448573",128 "internal": false,129 "reflection": false,130 "title": "Kaiming init of conv and linear layers, why gain = sqrt(5) · Issue #15314 · pytorch/pytorch · GitHub",131 "clicks": 31132 }133 ],134 "read": true,135 "user_title": "",136 "bookmarked": false,137 "actions_summary": [138 {139 "id": 2,140 "count": 1141 }142 ],143 "moderator": false,144 "admin": false,145 "staff": false,146 "user_id": 12783,147 "hidden": false,148 "trust_level": 2,149 "deleted_at": null,150 "user_deleted": false,151 "edit_reason": null,152 "can_view_edit_history": true,153 "wiki": false,154 "post_url": "/t/default-weight-initialisation-for-conv-layers-including-selu/87012/2",155 "can_accept_answer": false,156 "can_unaccept_answer": false,157 "accepted_answer": true,158 "topic_accepted_answer": true159 },160 {161 "id": 207066,162 "name": "Andrei Roibu",163 "username": "aroibu1",164 "avatar_template": "/user_avatar/discuss.pytorch.org/aroibu1/{size}/23271_2.png",165 "created_at": "2020-06-26T12:43:27.550Z",166 "cooked": "<p>Thanks <a class=\"mention\" href=\"/u/nikronic\">@Nikronic</a>! This makes things slightly more clear to me. However, I still have some questions, if that’s okay?</p>\n<p>I understand the use of <strong>math.sqrt(5)</strong> and how this ties in with the nonlinearity, IF the nonlinearity is ReLU or LeakyReLU. In my case, I am using a PReLU non-linearity for now. This is similar to LeakyReLU, and the Kaiming Loss has been created for it. Given this, would the <strong>math.sqrt(5)</strong> still be a good choice for the a-parameter in Kaiming Loss?</p>\n<p>Also, thank you for the advise on reinitilisation!</p>",167 "post_number": 3,168 "post_type": 1,169 "posts_count": 8,170 "updated_at": "2020-06-26T12:44:32.206Z",171 "reply_count": 1,172 "reply_to_post_number": 2,173 "quote_count": 0,174 "incoming_link_count": 4,175 "reads": 62,176 "readers_count": 61,177 "score": 37.4,178 "yours": false,179 "topic_id": 87012,180 "topic_slug": "default-weight-initialisation-for-conv-layers-including-selu",181 "display_username": "Andrei Roibu",182 "primary_group_name": null,183 "flair_name": null,184 "flair_url": null,185 "flair_bg_color": null,186 "flair_color": null,187 "flair_group_id": null,188 "badges_granted": [],189 "version": 1,190 "can_edit": false,191 "can_delete": false,192 "can_recover": false,193 "can_see_hidden_post": false,194 "can_wiki": false,195 "read": true,196 "user_title": "",197 "reply_to_user": {198 "id": 12783,199 "username": "Nikronic",200 "name": "Nikan Doosti",201 "avatar_template": "/user_avatar/discuss.pytorch.org/nikronic/{size}/71873_2.png"202 },203 "bookmarked": false,204 "actions_summary": [],205 "moderator": false,206 "admin": false,207 "staff": false,208 "user_id": 29207,209 "hidden": false,210 "trust_level": 2,211 "deleted_at": null,212 "user_deleted": false,213 "edit_reason": null,214 "can_view_edit_history": true,215 "wiki": false,216 "post_url": "/t/default-weight-initialisation-for-conv-layers-including-selu/87012/3",217 "can_accept_answer": false,218 "can_unaccept_answer": false,219 "accepted_answer": false,220 "topic_accepted_answer": true221 },222 {223 "id": 207072,224 "name": "Nikan Doosti",225 "username": "Nikronic",226 "avatar_template": "/user_avatar/discuss.pytorch.org/nikronic/{size}/71873_2.png",227 "created_at": "2020-06-26T13:03:10.231Z",228 "cooked": "<p>Great, If you are using anything else rather than LeakyReLU, you need to get proper value based on <code>gain</code> of that particular activation function.</p>",229 "post_number": 4,230 "post_type": 1,231 "posts_count": 8,232 "updated_at": "2020-06-26T13:03:10.231Z",233 "reply_count": 1,234 "reply_to_post_number": 3,235 "quote_count": 0,236 "incoming_link_count": 2,237 "reads": 60,238 "readers_count": 59,239 "score": 42.0,240 "yours": false,241 "topic_id": 87012,242 "topic_slug": "default-weight-initialisation-for-conv-layers-including-selu",243 "display_username": "Nikan Doosti",244 "primary_group_name": null,245 "flair_name": null,246 "flair_url": null,247 "flair_bg_color": null,248 "flair_color": null,249 "flair_group_id": null,250 "badges_granted": [],251 "version": 1,252 "can_edit": false,253 "can_delete": false,254 "can_recover": false,255 "can_see_hidden_post": false,256 "can_wiki": false,257 "read": true,258 "user_title": "",259 "reply_to_user": {260 "id": 29207,261 "username": "aroibu1",262 "name": "Andrei Roibu",263 "avatar_template": "/user_avatar/discuss.pytorch.org/aroibu1/{size}/23271_2.png"264 },265 "bookmarked": false,266 "actions_summary": [267 {268 "id": 2,269 "count": 1270 }271 ],272 "moderator": false,273 "admin": false,274 "staff": false,275 "user_id": 12783,276 "hidden": false,277 "trust_level": 2,278 "deleted_at": null,279 "user_deleted": false,280 "edit_reason": null,281 "can_view_edit_history": true,282 "wiki": false,283 "post_url": "/t/default-weight-initialisation-for-conv-layers-including-selu/87012/4",284 "can_accept_answer": false,285 "can_unaccept_answer": false,286 "accepted_answer": false,287 "topic_accepted_answer": true288 },289 {290 "id": 207082,291 "name": "Andrei Roibu",292 "username": "aroibu1",293 "avatar_template": "/user_avatar/discuss.pytorch.org/aroibu1/{size}/23271_2.png",294 "created_at": "2020-06-26T13:35:54.709Z",295 "cooked": "<p>I understand. Do you have any suggestions on how this can be done, or any sources that I can consult? I had a look at <em>torch.nn.init.calculate_gain</em> but ‘prelu’ is not a supported nonlinearity.</p>",296 "post_number": 5,297 "post_type": 1,298 "posts_count": 8,299 "updated_at": "2020-06-26T13:35:54.709Z",300 "reply_count": 1,301 "reply_to_post_number": 4,302 "quote_count": 0,303 "incoming_link_count": 6,304 "reads": 55,305 "readers_count": 54,306 "score": 46.0,307 "yours": false,308 "topic_id": 87012,309 "topic_slug": "default-weight-initialisation-for-conv-layers-including-selu",310 "display_username": "Andrei Roibu",311 "primary_group_name": null,312 "flair_name": null,313 "flair_url": null,314 "flair_bg_color": null,315 "flair_color": null,316 "flair_group_id": null,317 "badges_granted": [],318 "version": 1,319 "can_edit": false,320 "can_delete": false,321 "can_recover": false,322 "can_see_hidden_post": false,323 "can_wiki": false,324 "read": true,325 "user_title": "",326 "reply_to_user": {327 "id": 12783,328 "username": "Nikronic",329 "name": "Nikan Doosti",330 "avatar_template": "/user_avatar/discuss.pytorch.org/nikronic/{size}/71873_2.png"331 },332 "bookmarked": false,333 "actions_summary": [],334 "moderator": false,335 "admin": false,336 "staff": false,337 "user_id": 29207,338 "hidden": false,339 "trust_level": 2,340 "deleted_at": null,341 "user_deleted": false,342 "edit_reason": null,343 "can_view_edit_history": true,344 "wiki": false,345 "post_url": "/t/default-weight-initialisation-for-conv-layers-including-selu/87012/5",346 "can_accept_answer": false,347 "can_unaccept_answer": false,348 "accepted_answer": false,349 "topic_accepted_answer": true350 },351 {352 "id": 207090,353 "name": "Nikan Doosti",354 "username": "Nikronic",355 "avatar_template": "/user_avatar/discuss.pytorch.org/nikronic/{size}/71873_2.png",356 "created_at": "2020-06-26T13:51:00.574Z",357 "cooked": "<p>Yes, you need to find gain yourself in this case. Actually, I am not familiar with calculating gain so I cannot help with that. Let me know if you found anything.</p>",358 "post_number": 6,359 "post_type": 1,360 "posts_count": 8,361 "updated_at": "2020-06-26T13:51:00.574Z",362 "reply_count": 0,363 "reply_to_post_number": 5,364 "quote_count": 0,365 "incoming_link_count": 3,366 "reads": 49,367 "readers_count": 48,368 "score": 39.8,369 "yours": false,370 "topic_id": 87012,371 "topic_slug": "default-weight-initialisation-for-conv-layers-including-selu",372 "display_username": "Nikan Doosti",373 "primary_group_name": null,374 "flair_name": null,375 "flair_url": null,376 "flair_bg_color": null,377 "flair_color": null,378 "flair_group_id": null,379 "badges_granted": [],380 "version": 1,381 "can_edit": false,382 "can_delete": false,383 "can_recover": false,384 "can_see_hidden_post": false,385 "can_wiki": false,386 "read": true,387 "user_title": "",388 "reply_to_user": {389 "id": 29207,390 "username": "aroibu1",391 "name": "Andrei Roibu",392 "avatar_template": "/user_avatar/discuss.pytorch.org/aroibu1/{size}/23271_2.png"393 },394 "bookmarked": false,395 "actions_summary": [396 {397 "id": 2,398 "count": 1399 }400 ],401 "moderator": false,402 "admin": false,403 "staff": false,404 "user_id": 12783,405 "hidden": false,406 "trust_level": 2,407 "deleted_at": null,408 "user_deleted": false,409 "edit_reason": null,410 "can_view_edit_history": true,411 "wiki": false,412 "post_url": "/t/default-weight-initialisation-for-conv-layers-including-selu/87012/6",413 "can_accept_answer": false,414 "can_unaccept_answer": false,415 "accepted_answer": false,416 "topic_accepted_answer": true417 },418 {419 "id": 207107,420 "name": "Andrei Roibu",421 "username": "aroibu1",422 "avatar_template": "/user_avatar/discuss.pytorch.org/aroibu1/{size}/23271_2.png",423 "created_at": "2020-06-26T15:06:03.979Z",424 "cooked": "<p>Okay - this is a continuation of the message stream I had with <a class=\"mention\" href=\"/u/nikronic\">@Nikronic</a>, and is my solution to calculating the gain in order to properly use PReLU nonlinearity. I have not yet implemented this myself, but it’s what makes sense to me after reading the derivation in the original paper: <a href=\"https://arxiv.org/pdf/1502.01852.pdf\" rel=\"noopener nofollow ugc\">https://arxiv.org/pdf/1502.01852.pdf</a>. I’m writing this both for the community and my own later use <img src=\"https://discuss.pytorch.org/images/emoji/apple/slight_smile.png?v=12\" title=\":slight_smile:\" class=\"emoji\" alt=\":slight_smile:\" loading=\"lazy\" width=\"20\" height=\"20\"></p>\n<p>So, we start from the std definition in the paper:</p>\n<p><img src=\"https://discuss.pytorch.org/uploads/default/original/3X/c/2/c2b62d5f7589b16dddf81e3d4ef9bfffe8b77d85.png\" alt=\"Screenshot 2020-06-26 at 15.46.44\" data-base62-sha1=\"rMuWpOqh7iSKCXXBgT8DNYKx2ux\" width=\"273\" height=\"62\"></p>\n<p>If we rearrange this, we obtain that the standard deviation (=sqrt(Var)) is given by the following two, which are the values obtained from <a href=\"https://pytorch.org/docs/stable/nn.init.html\" class=\"inline-onebox\" rel=\"noopener nofollow ugc\">torch.nn.init — PyTorch 2.1 documentation</a>. The gain is basically identically calculated to the one in LeakyReLU.</p>\n<p><img src=\"https://discuss.pytorch.org/uploads/default/original/3X/0/f/0ffb7e1904ac988fa8af03a66f6a66cb3dae0352.png\" alt=\"Screenshot 2020-06-26 at 15.51.30\" data-base62-sha1=\"2hnYUlKfyD2dl6gsN7YUG6T1KH8\" width=\"182\" height=\"60\"></p>\n<p><div class=\"lightbox-wrapper\"><a class=\"lightbox\" href=\"https://discuss.pytorch.org/uploads/default/original/3X/c/7/c7737fc70c4f6afe649eb8b49c5cb12309179c13.png\" data-download-href=\"https://discuss.pytorch.org/uploads/default/c7737fc70c4f6afe649eb8b49c5cb12309179c13\" title=\"Screenshot 2020-06-26 at 15.51.54\"><img src=\"https://discuss.pytorch.org/uploads/default/optimized/3X/c/7/c7737fc70c4f6afe649eb8b49c5cb12309179c13_2_377x500.png\" alt=\"Screenshot 2020-06-26 at 15.51.54\" data-base62-sha1=\"ssqthM6fLJ8VVbNf4X3q74ptL5V\" width=\"377\" height=\"500\" srcset=\"https://discuss.pytorch.org/uploads/default/optimized/3X/c/7/c7737fc70c4f6afe649eb8b49c5cb12309179c13_2_377x500.png, https://discuss.pytorch.org/uploads/default/original/3X/c/7/c7737fc70c4f6afe649eb8b49c5cb12309179c13.png 1.5x, https://discuss.pytorch.org/uploads/default/original/3X/c/7/c7737fc70c4f6afe649eb8b49c5cb12309179c13.png 2x\" data-dominant-color=\"F7F7F8\"><div class=\"meta\"><svg class=\"fa d-icon d-icon-far-image svg-icon\" aria-hidden=\"true\"><use href=\"#far-image\"></use></svg><span class=\"filename\">Screenshot 2020-06-26 at 15.51.54</span><span class=\"informations\">389×515 12 KB</span><svg class=\"fa d-icon d-icon-discourse-expand svg-icon\" aria-hidden=\"true\"><use href=\"#discourse-expand\"></use></svg></div></a></div></p>\n<p>The equation derived in the original paper is the one used for the normal distribution, not the uniform one <code>torch.nn.init.kaiming_normal_</code> . So, in theory, we could just use this one. However, this is not possible, as the <code>kaiming_normal_</code> function in PyTorch calls <code>torch.nn.init.calculate_gain</code> which does not accept PReLU as a nonlinearity. Thus, we need a workaround this issue:</p>\n<p>The alternative is to just calculate our own standard deviation, which is actually easier than I thought. In the paper, they suggest initiating the <strong>negative_slope</strong> to whatever value we also use to initiate it in our PReLU. For PyTorch, that would be <em><strong>0.25</strong></em> (<a href=\"https://pytorch.org/docs/stable/nn.html#prelu\" class=\"inline-onebox\" rel=\"noopener nofollow ugc\">torch.nn — PyTorch 2.1 documentation</a>). We also need to calculate the <strong>fan_mode</strong>, for which we can look at how this is calculated in the PyTorch source (<a href=\"https://pytorch.org/docs/stable/_modules/torch/nn/init.html#kaiming_normal_\" rel=\"noopener nofollow ugc\">https://pytorch.org/docs/stable/<em>modules/torch/nn/init.html#kaiming_normal</em></a>):</p>\n<pre><code class=\"lang-auto\">def _calculate_fan_in_and_fan_out(tensor):\n dimensions = tensor.dim()\n if dimensions < 2:\n raise ValueError(\"Fan in and fan out can not be computed for tensor with fewer than 2 dimensions\")\n\n num_input_fmaps = tensor.size(1)\n num_output_fmaps = tensor.size(0)\n receptive_field_size = 1\n if tensor.dim() > 2:\n receptive_field_size = tensor[0][0].numel()\n fan_in = num_input_fmaps * receptive_field_size\n fan_out = num_output_fmaps * receptive_field_size\n\n return fan_in, fan_out\n</code></pre>\n<p>Following the calcualtion of the std, and knowing we have a mean of 0, we can rewrite the code provided here (Example 10 - <a href=\"https://www.programcreek.com/python/example/107693/torch.nn.PReLU\" class=\"inline-onebox\" rel=\"noopener nofollow ugc\">Python Examples of torch.nn.PReLU</a>) to produce our required weight initialisation:</p>\n<pre><code class=\"lang-auto\"> for m in self.modules():\n if isinstance(m, nn.Conv2d):\n n = fan_in\n negative_slope = 0.25\n m.weight.data.normal_(0, math.sqrt(2. / (n * (1 + negative_slope ** 2)) ) )\n elif isinstance(m, nn.BatchNorm2d):\n m.weight.data.fill_(1)\n m.bias.data.zero_() \n</code></pre>\n<p>Of course, if one uses to use fan_out, they can just replace it using the code above.</p>\n<p><a class=\"mention\" href=\"/u/nikronic\">@Nikronic</a> - does what I wrote above make sense? Asking as I think you have more experience with these things than I do.</p>",425 "post_number": 7,426 "post_type": 1,427 "posts_count": 8,428 "updated_at": "2020-06-27T14:01:10.748Z",429 "reply_count": 1,430 "reply_to_post_number": null,431 "quote_count": 0,432 "incoming_link_count": 56,433 "reads": 48,434 "readers_count": 47,435 "score": 309.6,436 "yours": false,437 "topic_id": 87012,438 "topic_slug": "default-weight-initialisation-for-conv-layers-including-selu",439 "display_username": "Andrei Roibu",440 "primary_group_name": null,441 "flair_name": null,442 "flair_url": null,443 "flair_bg_color": null,444 "flair_color": null,445 "flair_group_id": null,446 "badges_granted": [],447 "version": 2,448 "can_edit": false,449 "can_delete": false,450 "can_recover": false,451 "can_see_hidden_post": false,452 "can_wiki": false,453 "link_counts": [454 {455 "url": "https://pytorch.org/docs/stable/nn.init.html",456 "internal": false,457 "reflection": false,458 "title": "torch.nn.init — PyTorch master documentation",459 "clicks": 10460 },461 {462 "url": "https://pytorch.org/docs/stable/_modules/torch/nn/init.html#kaiming_normal_",463 "internal": false,464 "reflection": false,465 "title": "torch.nn.init — PyTorch master documentation",466 "clicks": 8467 },468 {469 "url": "https://pytorch.org/docs/stable/nn.html#prelu",470 "internal": false,471 "reflection": false,472 "title": "torch.nn — PyTorch master documentation",473 "clicks": 3474 },475 {476 "url": "https://arxiv.org/pdf/1502.01852.pdf",477 "internal": false,478 "reflection": false,479 "clicks": 3480 },481 {482 "url": "https://www.programcreek.com/python/example/107693/torch.nn.PReLU",483 "internal": false,484 "reflection": false,485 "clicks": 3486 },487 {488 "url": "https://discuss.pytorch.org/uploads/default/original/3X/c/7/c7737fc70c4f6afe649eb8b49c5cb12309179c13.png",489 "internal": true,490 "reflection": false,491 "clicks": 0492 }493 ],494 "read": true,495 "user_title": "",496 "bookmarked": false,497 "actions_summary": [498 {499 "id": 2,500 "count": 1501 }502 ],503 "moderator": false,504 "admin": false,505 "staff": false,506 "user_id": 29207,507 "hidden": false,508 "trust_level": 2,509 "deleted_at": null,510 "user_deleted": false,511 "edit_reason": null,512 "can_view_edit_history": true,513 "wiki": false,514 "post_url": "/t/default-weight-initialisation-for-conv-layers-including-selu/87012/7",515 "can_accept_answer": false,516 "can_unaccept_answer": false,517 "accepted_answer": false,518 "topic_accepted_answer": true519 },520 {521 "id": 207478,522 "name": "Nikan Doosti",523 "username": "Nikronic",524 "avatar_template": "/user_avatar/discuss.pytorch.org/nikronic/{size}/71873_2.png",525 "created_at": "2020-06-28T05:34:22.835Z",526 "cooked": "<p>Hi, Sorry for my late answer, I am struggling with final exams! <img src=\"https://discuss.pytorch.org/images/emoji/apple/smiley.png?v=12\" title=\":smiley:\" class=\"emoji\" alt=\":smiley:\" loading=\"lazy\" width=\"20\" height=\"20\"></p>\n<p>First, thank you for your deep explanation. Secondly, I do not have strong mathematical background, so I do not think I am eligble to validate this but sounds fine to me.</p>\n<p>Another point I would like to mention is that PyTorch uses uniform for initializing weights in convs and linear layers so if gain in PReLU is identical to LeakyReLU, then to achieve the range of <code>[-1/sqrt(fan_mode), 1/sqrt(fan_mode)]</code> for uniform distribution, still we need to consider <code>negative_slope=sqrt(5)</code> where otherwise it will lead to a different scenario.</p>\n<p>I think we need to discuss this as a feature request so the main developers can help us with that. So, I think it would be great idea to create an issue on github.<br>\nHere is another <a href=\"https://github.com/pytorch/pytorch/issues/24991\" rel=\"noopener nofollow ugc\">issue</a> related to this idea that may help. Furthermore, this thread considered another perspective which I have no clue what they are talking about <img src=\"https://discuss.pytorch.org/images/emoji/apple/sweat_smile.png?v=12\" title=\":sweat_smile:\" class=\"emoji\" alt=\":sweat_smile:\" loading=\"lazy\" width=\"20\" height=\"20\">.</p><aside class=\"quote\" data-post=\"1\" data-topic=\"20854\">\n <div class=\"title\">\n <div class=\"quote-controls\"></div>\n <img loading=\"lazy\" alt=\"\" width=\"24\" height=\"24\" src=\"https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/jpeg729/48/3464_2.png\" class=\"avatar\">\n <a href=\"https://discuss.pytorch.org/t/calculate-gain-tanh/20854\">Calculate_gain('tanh')</a> \n </div>\n <blockquote>\n I understand the results returned by calculate_gain for linear, relu, leaky_relu and sigmoid. \nCan anyone tell me why calculate_gain('tanh') returns 5/3 ?\n </blockquote>\n</aside>\n\n<p>If you created the issue on github, could you please also tag me so I can keep track of things? my username is <a href=\"https://github.com/Nikronic\" rel=\"noopener nofollow ugc\">Nikronic</a></p>\n<p>Thank you</p>",527 "post_number": 8,528 "post_type": 1,529 "posts_count": 8,530 "updated_at": "2020-06-28T05:40:33.080Z",531 "reply_count": 0,532 "reply_to_post_number": 7,533 "quote_count": 0,534 "incoming_link_count": 24,535 "reads": 42,536 "readers_count": 41,537 "score": 123.4,538 "yours": false,539 "topic_id": 87012,540 "topic_slug": "default-weight-initialisation-for-conv-layers-including-selu",541 "display_username": "Nikan Doosti",542 "primary_group_name": null,543 "flair_name": null,544 "flair_url": null,545 "flair_bg_color": null,546 "flair_color": null,547 "flair_group_id": null,548 "badges_granted": [],549 "version": 2,550 "can_edit": false,551 "can_delete": false,552 "can_recover": false,553 "can_see_hidden_post": false,554 "can_wiki": false,555 "link_counts": [556 {557 "url": "https://github.com/pytorch/pytorch/issues/24991",558 "internal": false,559 "reflection": false,560 "title": "[Feature request] Add support for selu activation to calculate_gain function. · Issue #24991 · pytorch/pytorch · GitHub",561 "clicks": 3562 },563 {564 "url": "https://discuss.pytorch.org/t/calculate-gain-tanh/20854",565 "internal": true,566 "reflection": false,567 "title": "Calculate_gain('tanh')",568 "clicks": 0569 },570 {571 "url": "https://github.com/Nikronic",572 "internal": false,573 "reflection": false,574 "title": "Nikronic (Mohammad Doosti Lakhani) · GitHub",575 "clicks": 0576 }577 ],578 "read": true,579 "user_title": "",580 "reply_to_user": {581 "id": 29207,582 "username": "aroibu1",583 "name": "Andrei Roibu",584 "avatar_template": "/user_avatar/discuss.pytorch.org/aroibu1/{size}/23271_2.png"585 },586 "bookmarked": false,587 "actions_summary": [],588 "moderator": false,589 "admin": false,590 "staff": false,591 "user_id": 12783,592 "hidden": false,593 "trust_level": 2,594 "deleted_at": null,595 "user_deleted": false,596 "edit_reason": null,597 "can_view_edit_history": true,598 "wiki": false,599 "post_url": "/t/default-weight-initialisation-for-conv-layers-including-selu/87012/8",600 "can_accept_answer": false,601 "can_unaccept_answer": false,602 "accepted_answer": false,603 "topic_accepted_answer": true604 }605 ],606 "stream": [607 207046,608 207058,609 207066,610 207072,611 207082,612 207090,613 207107,614 207478615 ]616 },617 "timeline_lookup": [618 [619 1,620 1948621 ],622 [623 2,624 1947625 ],626 [627 8,628 1946629 ]630 ],631 "suggested_topics": [632 {633 "fancy_title": "How to resolve with the issue of adjusting the gaps in my ‘time_idx’ feature in my stock market dataset?",634 "id": 213245,635 "title": "How to resolve with the issue of adjusting the gaps in my 'time_idx' feature in my stock market dataset?",636 "slug": "how-to-resolve-with-the-issue-of-adjusting-the-gaps-in-my-time-idx-feature-in-my-stock-market-dataset",637 "posts_count": 1,638 "reply_count": 0,639 "highest_post_number": 1,640 "image_url": null,641 "created_at": "2024-11-21T05:01:10.859Z",642 "last_posted_at": "2024-11-21T05:01:10.919Z",643 "bumped": true,644 "bumped_at": "2024-11-21T05:01:10.919Z",645 "archetype": "regular",646 "unseen": false,647 "pinned": false,648 "unpinned": null,649 "visible": true,650 "closed": false,651 "archived": false,652 "bookmarked": null,653 "liked": null,654 "tags_descriptions": {},655 "like_count": 0,656 "views": 26,657 "category_id": 1,658 "featured_link": null,659 "has_accepted_answer": false,660 "posters": [661 {662 "extras": "latest single",663 "description": "Original Poster, Most Recent Poster",664 "user": {665 "id": 28027,666 "username": "Satyam_sharma",667 "name": "Satyam sharma",668 "avatar_template": "/user_avatar/discuss.pytorch.org/satyam_sharma/{size}/20898_2.png",669 "trust_level": 0670 }671 }672 ]673 },674 {675 "fancy_title": "RuntimeError: mat1 and mat2 shapes cannot be multiplied (1x3 and 20x10)",676 "id": 213493,677 "title": "RuntimeError: mat1 and mat2 shapes cannot be multiplied (1x3 and 20x10)",678 "slug": "runtimeerror-mat1-and-mat2-shapes-cannot-be-multiplied-1x3-and-20x10",679 "posts_count": 1,680 "reply_count": 0,681 "highest_post_number": 1,682 "image_url": null,683 "created_at": "2024-11-26T21:12:05.959Z",684 "last_posted_at": "2024-11-26T21:12:06.050Z",685 "bumped": true,686 "bumped_at": "2024-11-26T21:12:06.050Z",687 "archetype": "regular",688 "unseen": false,689 "pinned": false,690 "unpinned": null,691 "visible": true,692 "closed": false,693 "archived": false,694 "bookmarked": null,695 "liked": null,696 "tags_descriptions": {},697 "like_count": 0,698 "views": 23,699 "category_id": 1,700 "featured_link": null,701 "has_accepted_answer": false,702 "posters": [703 {704 "extras": "latest single",705 "description": "Original Poster, Most Recent Poster",706 "user": {707 "id": 81152,708 "username": "dpalate",709 "name": "Devashish Palate",710 "avatar_template": "/user_avatar/discuss.pytorch.org/dpalate/{size}/74220_2.png",711 "trust_level": 0712 }713 }714 ]715 },716 {717 "fancy_title": "Temporal Fusion Transformer Error",718 "id": 213925,719 "title": "Temporal Fusion Transformer Error",720 "slug": "temporal-fusion-transformer-error",721 "posts_count": 3,722 "reply_count": 1,723 "highest_post_number": 3,724 "image_url": null,725 "created_at": "2024-12-06T20:29:08.583Z",726 "last_posted_at": "2024-12-17T19:48:46.480Z",727 "bumped": true,728 "bumped_at": "2024-12-17T19:48:46.480Z",729 "archetype": "regular",730 "unseen": false,731 "pinned": false,732 "unpinned": null,733 "visible": true,734 "closed": false,735 "archived": false,736 "bookmarked": null,737 "liked": null,738 "tags_descriptions": {},739 "like_count": 0,740 "views": 290,741 "category_id": 1,742 "featured_link": null,743 "has_accepted_answer": false,744 "posters": [745 {746 "extras": null,747 "description": "Original Poster",748 "user": {749 "id": 81366,750 "username": "himanshu_birla",751 "name": "himanshu birla",752 "avatar_template": "/user_avatar/discuss.pytorch.org/himanshu_birla/{size}/74403_2.png",753 "trust_level": 0754 }755 },756 {757 "extras": null,758 "description": "Frequent Poster",759 "user": {760 "id": 3534,761 "username": "ptrblck",762 "name": "",763 "avatar_template": "/user_avatar/discuss.pytorch.org/ptrblck/{size}/1823_2.png",764 "admin": true,765 "moderator": true,766 "trust_level": 2767 }768 },769 {770 "extras": "latest",771 "description": "Most Recent Poster",772 "user": {773 "id": 81553,774 "username": "Juan_pablo_Mazo_quin",775 "name": "Juan pablo Mazo quintero",776 "avatar_template": "/user_avatar/discuss.pytorch.org/juan_pablo_mazo_quin/{size}/74572_2.png",777 "trust_level": 1778 }779 }780 ]781 },782 {783 "fancy_title": "How to liberate CUDA Memory succesfully?",784 "id": 214081,785 "title": "How to liberate CUDA Memory succesfully?",786 "slug": "how-to-liberate-cuda-memory-succesfully",787 "posts_count": 2,788 "reply_count": 0,789 "highest_post_number": 2,790 "image_url": "https://discuss.pytorch.org/uploads/default/original/3X/a/f/af1fa59e98fb53eb6f25e7f62b3371828b56f185.png",791 "created_at": "2024-12-11T01:04:45.736Z",792 "last_posted_at": "2024-12-11T14:08:23.450Z",793 "bumped": true,794 "bumped_at": "2024-12-11T14:08:23.450Z",795 "archetype": "regular",796 "unseen": false,797 "pinned": false,798 "unpinned": null,799 "visible": true,800 "closed": false,801 "archived": false,802 "bookmarked": null,803 "liked": null,804 "tags_descriptions": {},805 "like_count": 0,806 "views": 230,807 "category_id": 1,808 "featured_link": null,809 "has_accepted_answer": false,810 "posters": [811 {812 "extras": null,813 "description": "Original Poster",814 "user": {815 "id": 81441,816 "username": "Jose_Leyva",817 "name": "Jose Leyva",818 "avatar_template": "/user_avatar/discuss.pytorch.org/jose_leyva/{size}/74460_2.png",819 "trust_level": 1820 }821 },822 {823 "extras": "latest",824 "description": "Most Recent Poster",825 "user": {826 "id": 3534,827 "username": "ptrblck",828 "name": "",829 "avatar_template": "/user_avatar/discuss.pytorch.org/ptrblck/{size}/1823_2.png",830 "admin": true,831 "moderator": true,832 "trust_level": 2833 }834 }835 ]836 },837 {838 "fancy_title": "Changing the backbone",839 "id": 212284,840 "title": "Changing the backbone",841 "slug": "changing-the-backbone",842 "posts_count": 3,843 "reply_count": 0,844 "highest_post_number": 3,845 "image_url": null,846 "created_at": "2024-10-29T15:27:33.798Z",847 "last_posted_at": "2024-11-08T05:37:31.700Z",848 "bumped": true,849 "bumped_at": "2024-11-08T05:37:31.700Z",850 "archetype": "regular",851 "unseen": false,852 "pinned": false,853 "unpinned": null,854 "visible": true,855 "closed": false,856 "archived": false,857 "bookmarked": null,858 "liked": null,859 "tags_descriptions": {},860 "like_count": 0,861 "views": 97,862 "category_id": 1,863 "featured_link": null,864 "has_accepted_answer": false,865 "posters": [866 {867 "extras": "latest",868 "description": "Original Poster, Most Recent Poster",869 "user": {870 "id": 76804,871 "username": "afnhs-gs",872 "name": "semba",873 "avatar_template": "/letter_avatar_proxy/v4/letter/a/c5a1d2/{size}.png",874 "trust_level": 1875 }876 },877 {878 "extras": null,879 "description": "Frequent Poster",880 "user": {881 "id": 3534,882 "username": "ptrblck",883 "name": "",884 "avatar_template": "/user_avatar/discuss.pytorch.org/ptrblck/{size}/1823_2.png",885 "admin": true,886 "moderator": true,887 "trust_level": 2888 }889 }890 ]891 }892 ],893 "tags_descriptions": {},894 "fancy_title": "Default weight initialisation for Conv layers (including SELU)",895 "id": 87012,896 "title": "Default weight initialisation for Conv layers (including SELU)",897 "posts_count": 8,898 "created_at": "2020-06-26T10:25:15.991Z",899 "views": 5455,900 "reply_count": 5,901 "like_count": 5,902 "last_posted_at": "2020-06-28T05:34:22.835Z",903 "visible": true,904 "closed": false,905 "archived": false,906 "has_summary": false,907 "archetype": "regular",908 "slug": "default-weight-initialisation-for-conv-layers-including-selu",909 "category_id": 1,910 "word_count": 1333,911 "deleted_at": null,912 "user_id": 29207,913 "featured_link": null,914 "pinned_globally": false,915 "pinned_at": null,916 "pinned_until": null,917 "image_url": null,918 "slow_mode_seconds": 0,919 "draft": null,920 "draft_key": "topic_87012",921 "draft_sequence": null,922 "unpinned": null,923 "pinned": false,924 "current_post_number": 1,925 "highest_post_number": 8,926 "deleted_by": null,927 "actions_summary": [928 {929 "id": 4,930 "count": 0,931 "hidden": false,932 "can_act": false933 },934 {935 "id": 8,936 "count": 0,937 "hidden": false,938 "can_act": false939 },940 {941 "id": 10,942 "count": 0,943 "hidden": false,944 "can_act": false945 },946 {947 "id": 7,948 "count": 0,949 "hidden": false,950 "can_act": false951 }952 ],953 "chunk_size": 20,954 "bookmarked": false,955 "topic_timer": null,956 "message_bus_last_id": 0,957 "participant_count": 2,958 "show_read_indicator": false,959 "thumbnails": null,960 "slow_mode_enabled_until": null,961 "accepted_answer": {962 "post_number": 2,963 "username": "Nikronic",964 "name": "Nikan Doosti",965 "excerpt": "Hi, \nFor the first question, please see these posts: \n\n<a href=\"https://discuss.pytorch.org/t/clarity-on-default-initialization-in-pytorch/84696\" class=\"inline-onebox\">Clarity on default initialization in pytorch</a>\n<a href=\"https://discuss.pytorch.org/t/cnn-default-initialization-understanding/85353/2\" class=\"inline-onebox\">CNN default initialization understanding</a>\n\nI have explained the magic number math.sqrt(5) so you can also get the idea behind the relation between non-linearity and init method. Acuatlly, default initi…"966 },967 "can_vote": false,968 "vote_count": 0,969 "user_voted": false,970 "discourse_zendesk_plugin_zendesk_id": null,971 "discourse_zendesk_plugin_zendesk_url": "https://your-url.zendesk.com/agent/tickets/",972 "details": {973 "can_edit": false,974 "notification_level": 1,975 "participants": [976 {977 "id": 12783,978 "username": "Nikronic",979 "name": "Nikan Doosti",980 "avatar_template": "/user_avatar/discuss.pytorch.org/nikronic/{size}/71873_2.png",981 "post_count": 4,982 "primary_group_name": null,983 "flair_name": null,984 "flair_url": null,985 "flair_color": null,986 "flair_bg_color": null,987 "flair_group_id": null,988 "trust_level": 2989 },990 {991 "id": 29207,992 "username": "aroibu1",993 "name": "Andrei Roibu",994 "avatar_template": "/user_avatar/discuss.pytorch.org/aroibu1/{size}/23271_2.png",995 "post_count": 4,996 "primary_group_name": null,997 "flair_name": null,998 "flair_url": null,999 "flair_color": null,1000 "flair_bg_color": null,1001 "flair_group_id": null,1002 "trust_level": 21003 }1004 ],1005 "created_by": {1006 "id": 29207,1007 "username": "aroibu1",1008 "name": "Andrei Roibu",1009 "avatar_template": "/user_avatar/discuss.pytorch.org/aroibu1/{size}/23271_2.png"1010 },1011 "last_poster": {1012 "id": 12783,1013 "username": "Nikronic",1014 "name": "Nikan Doosti",1015 "avatar_template": "/user_avatar/discuss.pytorch.org/nikronic/{size}/71873_2.png"1016 },1017 "links": [1018 {1019 "url": "https://discuss.pytorch.org/t/clarity-on-default-initialization-in-pytorch/84696",1020 "title": "Clarity on default initialization in pytorch",1021 "internal": true,1022 "attachment": false,1023 "reflection": false,1024 "clicks": 251,1025 "user_id": 12783,1026 "domain": "discuss.pytorch.org",1027 "root_domain": "pytorch.org"1028 },1029 {1030 "url": "https://discuss.pytorch.org/t/cnn-default-initialization-understanding/85353/2",1031 "title": "CNN default initialization understanding",1032 "internal": true,1033 "attachment": false,1034 "reflection": false,1035 "clicks": 176,1036 "user_id": 12783,1037 "domain": "discuss.pytorch.org",1038 "root_domain": "pytorch.org"1039 },1040 {1041 "url": "https://github.com/pytorch/pytorch/blob/master/torch/nn/modules/conv.py",1042 "title": "pytorch/conv.py at master · pytorch/pytorch · GitHub",1043 "internal": false,1044 "attachment": false,1045 "reflection": false,1046 "clicks": 39,1047 "user_id": 29207,1048 "domain": "github.com",1049 "root_domain": "github.com"1050 },1051 {1052 "url": "https://github.com/pytorch/pytorch/issues/15314#issuecomment-477448573",1053 "title": "Kaiming init of conv and linear layers, why gain = sqrt(5) · Issue #15314 · pytorch/pytorch · GitHub",1054 "internal": false,1055 "attachment": false,1056 "reflection": false,1057 "clicks": 31,1058 "user_id": 12783,1059 "domain": "github.com",1060 "root_domain": "github.com"1061 },1062 {1063 "url": "https://pytorch.org/docs/stable/nn.init.html",1064 "title": "torch.nn.init — PyTorch master documentation",1065 "internal": false,1066 "attachment": false,1067 "reflection": false,1068 "clicks": 10,1069 "user_id": 29207,1070 "domain": "pytorch.org",1071 "root_domain": "pytorch.org"1072 },1073 {1074 "url": "https://pytorch.org/docs/stable/_modules/torch/nn/init.html#kaiming_normal_",1075 "title": "torch.nn.init — PyTorch master documentation",1076 "internal": false,1077 "attachment": false,1078 "reflection": false,1079 "clicks": 8,1080 "user_id": 29207,1081 "domain": "pytorch.org",1082 "root_domain": "pytorch.org"1083 },1084 {1085 "url": "https://www.programcreek.com/python/example/107693/torch.nn.PReLU",1086 "title": null,1087 "internal": false,1088 "attachment": false,1089 "reflection": false,1090 "clicks": 3,1091 "user_id": 29207,1092 "domain": "www.programcreek.com",1093 "root_domain": "programcreek.com"1094 },1095 {1096 "url": "https://github.com/pytorch/pytorch/issues/24991",1097 "title": "[Feature request] Add support for selu activation to calculate_gain function. · Issue #24991 · pytorch/pytorch · GitHub",1098 "internal": false,1099 "attachment": false,1100 "reflection": false,1101 "clicks": 3,1102 "user_id": 12783,1103 "domain": "github.com",1104 "root_domain": "github.com"1105 },1106 {1107 "url": "https://pytorch.org/docs/stable/nn.html#prelu",1108 "title": "torch.nn — PyTorch master documentation",1109 "internal": false,1110 "attachment": false,1111 "reflection": false,1112 "clicks": 3,1113 "user_id": 29207,1114 "domain": "pytorch.org",1115 "root_domain": "pytorch.org"1116 },1117 {1118 "url": "https://arxiv.org/pdf/1502.01852.pdf",1119 "title": null,1120 "internal": false,1121 "attachment": false,1122 "reflection": false,1123 "clicks": 3,1124 "user_id": 29207,1125 "domain": "arxiv.org",1126 "root_domain": "arxiv.org"1127 }1128 ]1129 },1130 "bookmarks": []1131 },1132 {1133 "post_stream": {1134 "posts": [1135 {1136 "id": 207338,1137 "name": "Mjavan",1138 "username": "887574002",1139 "avatar_template": "/user_avatar/discuss.pytorch.org/887574002/{size}/35835_2.png",1140 "created_at": "2020-06-27T16:09:32.564Z",1141 "cooked": "<p>Hi, I am trying to practice semantic segmentation using Pascal Voc. I wrote a subclass of VOCSegmentation, and I defined transformation. When I try to read one element of train_set, I got the following error,</p>\n<p><strong>call</strong>() takes 1 positional argument but 3 were given</p>\n<p>I can’t understand where the problem is. Here is my code:</p>\n<p>class myVOCSegmentation(VOCSegmentation):</p>\n<pre><code>def __getitem__(self,index):\n \n img = Image.open(self.images[index].convert('RGB'))\n \n target = Image.open(self.masks[index])\n \n if self.transforms is not None:\n\n \n agumented = self.transforms(image= np.array(img),mask = np.array(target))\n \n img = agumented['image']\n \n target = agumented['mask']\n \n # here we have 20 classes, so we set data larger than 20 equal to 0\n target[target>20]=0\n \n img = to_tensor(img)\n \n target = torch.from_numpy(target).type(torch.long)\n \n return(img,target) \n</code></pre>\n<p>h,w = 520,520</p>\n<p>mean = [0.485, 0.456, 0.406]<br>\nstd = [0.229, 0.224, 0.225]</p>\n<p>transform_train = Compose([Resize(h,w),<br>\nHorizontalFlip(p=0.5),<br>\nNormalize(mean=mean,std=std)])</p>\n<p>transform_val = Compose([Resize(h,w),Normalize(mean=mean,std=std)])</p>\n<p>PATH = ‘./data’</p>\n<p>train_set =VOCSegmentation(PATH,year=‘2012’,download = True,image_set =‘train’,transforms=transform_train)</p>\n<p>val_set = VOCSegmentation(PATH,year=‘2012’,download = True,image_set =‘val’,transforms=transform_val)</p>\n<p>for item in train_set:<br>\nprint(item)</p>\n<pre><code>break\n</code></pre>\n<p><strong>call</strong>() takes 1 positional argument but 3 were given</p>",1142 "post_number": 1,1143 "post_type": 1,1144 "posts_count": 13,1145 "updated_at": "2020-06-27T16:09:32.564Z",1146 "reply_count": 3,1147 "reply_to_post_number": null,1148 "quote_count": 0,1149 "incoming_link_count": 163,1150 "reads": 10,1151 "readers_count": 9,1152 "score": 822.0,1153 "yours": false,1154 "topic_id": 87125,1155 "topic_slug": "error-in-datset-of-pascalvoc",1156 "display_username": "Mjavan",1157 "primary_group_name": null,1158 "flair_name": null,1159 "flair_url": null,1160 "flair_bg_color": null,1161 "flair_color": null,1162 "flair_group_id": null,1163 "badges_granted": [],1164 "version": 1,1165 "can_edit": false,1166 "can_delete": false,1167 "can_recover": false,1168 "can_see_hidden_post": false,1169 "can_wiki": false,1170 "read": true,1171 "user_title": null,1172 "bookmarked": false,1173 "actions_summary": [],1174 "moderator": false,1175 "admin": false,1176 "staff": false,1177 "user_id": 33302,1178 "hidden": false,1179 "trust_level": 1,1180 "deleted_at": null,1181 "user_deleted": false,1182 "edit_reason": null,1183 "can_view_edit_history": true,1184 "wiki": false,1185 "post_url": "/t/error-in-datset-of-pascalvoc/87125/1",1186 "can_accept_answer": false,1187 "can_unaccept_answer": false,1188 "accepted_answer": false,1189 "topic_accepted_answer": null,1190 "can_vote": false1191 },1192 {1193 "id": 207342,1194 "name": "",1195 "username": "harsha_g",1196 "avatar_template": "/letter_avatar_proxy/v4/letter/h/4491bb/{size}.png",1197 "created_at": "2020-06-27T16:45:23.480Z",1198 "cooked": "<aside class=\"quote no-group\" data-username=\"887574002\" data-post=\"1\" data-topic=\"87125\">\n<div class=\"title\">\n<div class=\"quote-controls\"></div>\n<img loading=\"lazy\" alt=\"\" width=\"24\" height=\"24\" src=\"https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/887574002/48/35835_2.png\" class=\"avatar\"> 887574002:</div>\n<blockquote>\n<p>VOCSegmentation</p>\n</blockquote>\n</aside>\n<p>Without knowing how your <code>call()</code> functions looks like, it seems that it has been defined as:</p>\n<pre><code class=\"lang-auto\">def call(self):\n ...\n</code></pre>\n<p>However, you may be calling it somewhere as:</p>\n<pre><code class=\"lang-auto\">my_object.call(input_arg1, input_arg2)\n</code></pre>\n<p>which is why you see that error.</p>",1199 "post_number": 2,1200 "post_type": 1,