Anurag1734/cuda-error-resolution-analysis
07
1[2 {3 "post_stream": {4 "posts": [5 {6 "id": 340843,7 "name": "",8 "username": "alexx",9 "avatar_template": "/letter_avatar_proxy/v4/letter/a/ee59a6/{size}.png",10 "created_at": "2022-04-11T07:04:05.440Z",11 "cooked": "<p>I want to create a model that contains a network that learns to estimate rotation angles for individual data points.</p>\n<p>However, with my current implementation, the Gradients of the angle embedding network become None.</p>\n<p>Based on a suggestion here: <a href=\"https://discuss.pytorch.org/t/differentiable-affine-transforms-with-grid-sample/79305\" class=\"inline-onebox\">Differentiable affine transforms with grid_sample</a></p>\n<blockquote>\n<p>or use <code>torch.cat</code> or <code>torch.stack</code> to create <code>theta</code> in the <code>forward</code> method from the parameters.</p>\n</blockquote>\n<p>I tried using .stack() and .cat() on the list of rotation matrices; however my gradients still become None.<br>\nI display the gradients after the backward computation with this command:</p>\n<pre><code class=\"lang-auto\">print([(param.grad,name) for name, param in model.named_parameters()] )\n</code></pre>\n<p>and this is the output</p>\n<pre><code class=\"lang-auto\">... ,(None, 'angle.0.weight'), (None, 'angle.0.bias'), (None, 'angle.2.weight'), (None, 'angle.2.bias')]\n</code></pre>\n<p>This is the code that I’m trying to adapt for my purpose ( The original author of the code is Ghassen HAMROUNI, <a href=\"https://github.com/GHamrouni\" class=\"inline-onebox\" rel=\"noopener nofollow ugc\">GHamrouni (Ghassen Hamrouni) · GitHub</a>). The issue occurs in the Net Class in the method called stn().</p>\n<pre><code class=\"lang-auto\"># License: BSD\n# Author: Ghassen Hamrouni\n\nfrom __future__ import print_function\nimport torch\nimport torch.nn as nn\nimport torch.nn.functional as F\nimport torch.optim as optim\nimport torchvision\nfrom torchvision import datasets, transforms\nimport matplotlib.pyplot as plt\nimport numpy as np\n\nplt.ion() # interactive mode\n</code></pre>\n<p><strong>Loading some data</strong></p>\n<pre><code class=\"lang-auto\">device = torch.device(\"cuda\" if torch.cuda.is_available() else \"cpu\")\n\n# Training dataset\ntrain_loader = torch.utils.data.DataLoader(\n datasets.MNIST(root='.', train=True, download=True,\n transform=transforms.Compose([\n transforms.ToTensor(),\n transforms.Normalize((0.1307,), (0.3081,))\n ])), batch_size=64, shuffle=True, num_workers=4)\n# Test dataset\ntest_loader = torch.utils.data.DataLoader(\n datasets.MNIST(root='.', train=False, transform=transforms.Compose([\n transforms.ToTensor(),\n transforms.Normalize((0.1307,), (0.3081,))\n ])), batch_size=64, shuffle=True, num_workers=4)\n</code></pre>\n<p><strong>stn() is the method where I’m performing the operation that results in the None gradients</strong></p>\n<pre><code class=\"lang-auto\">class Net(nn.Module):\n def __init__(self):\n super(Net, self).__init__()\n self.conv1 = nn.Conv2d(1, 10, kernel_size=5)\n self.conv2 = nn.Conv2d(10, 20, kernel_size=5)\n self.conv2_drop = nn.Dropout2d()\n self.fc1 = nn.Linear(320, 50)\n self.fc2 = nn.Linear(50, 10)\n\n self.angle = nn.Sequential(\n nn.Linear(28*28, 5),\n nn.ReLU(True),\n nn.Linear(5, 1)\n )\n\n # Spatial transformer network forward function\n def stn(self, x):\n\n angles = torch.arctan(self.angle(x.squeeze().reshape( (x.shape[0],28*28) )))*2\n theta = torch.stack([torch.tensor([[[torch.cos(t), -torch.sin(t), 0.0], [torch.sin(t), torch.cos(t), 0.0]] for t in angles], requires_grad = True )]).squeeze()\n grid = F.affine_grid(theta, x.size())\n x = F.grid_sample(x, grid, mode = \"bilinear\")\n\n return x\n\n def forward(self, x):\n # transform the input\n x = self.stn(x)\n\n # Perform the usual forward pass\n x = F.relu(F.max_pool2d(self.conv1(x), 2))\n x = F.relu(F.max_pool2d(self.conv2_drop(self.conv2(x)), 2))\n x = x.view(-1, 320)\n x = F.relu(self.fc1(x))\n x = F.dropout(x, training=self.training)\n x = self.fc2(x)\n return F.log_softmax(x, dim=1)\n\n\nmodel = Net().to(device)\n</code></pre>\n<p><strong>Training code</strong></p>\n<pre><code class=\"lang-auto\">optimizer = optim.SGD(model.parameters(), lr=0.01)\n\n\ndef train(epoch):\n model.train()\n for batch_idx, (data, target) in enumerate(train_loader):\n data, target = data.to(device), target.to(device)\n\n optimizer.zero_grad()\n output = model(data)\n loss = F.nll_loss(output, target)\n loss.backward()\n print([(param.grad,name) for name, param in model.named_parameters()] )\n optimizer.step()\n if batch_idx % 500 == 0:\n print('Train Epoch: {} [{}/{} ({:.0f}%)]\\tLoss: {:.6f}'.format(\n epoch, batch_idx * len(data), len(train_loader.dataset),\n 100. * batch_idx / len(train_loader), loss.item()))\n\nfor epoch in range(1, 20 + 1):\n train(epoch)\n</code></pre>\n<p>I’m happy for any suggestions on how to solve this issue. Thank you in advance!</p>\n<p><strong>SOLUTION:</strong><br>\nI managed to solve the issue, this is how I changed the theta matrix:</p>\n<pre><code class=\"lang-auto\">theta = torch.stack( [ torch.stack([torch.stack([torch.cos(t).unsqueeze(dim=0), -torch.sin(t).unsqueeze(dim=0), torch.zeros(1)]), torch.stack([torch.sin(t).unsqueeze(dim=0), torch.cos(t).unsqueeze(dim=0), torch.zeros(1)])]) for t in angles] ).squeeze() \n</code></pre>",12 "post_number": 1,13 "post_type": 1,14 "posts_count": 1,15 "updated_at": "2022-04-11T08:26:18.462Z",16 "reply_count": 0,17 "reply_to_post_number": null,18 "quote_count": 0,19 "incoming_link_count": 739,20 "reads": 22,21 "readers_count": 21,22 "score": 3729.4,23 "yours": false,24 "topic_id": 148796,25 "topic_slug": "differentiable-and-learnable-rotations-with-grid-sample",26 "display_username": "",27 "primary_group_name": null,28 "flair_name": null,29 "flair_url": null,30 "flair_bg_color": null,31 "flair_color": null,32 "flair_group_id": null,33 "badges_granted": [],34 "version": 2,35 "can_edit": false,36 "can_delete": false,37 "can_recover": false,38 "can_see_hidden_post": false,39 "can_wiki": false,40 "link_counts": [41 {42 "url": "https://discuss.pytorch.org/t/differentiable-affine-transforms-with-grid-sample/79305",43 "internal": true,44 "reflection": false,45 "title": "Differentiable affine transforms with grid_sample",46 "clicks": 5147 },48 {49 "url": "https://github.com/GHamrouni",50 "internal": false,51 "reflection": false,52 "title": "GHamrouni (Ghassen Hamrouni) · GitHub",53 "clicks": 654 }55 ],56 "read": true,57 "user_title": null,58 "bookmarked": false,59 "actions_summary": [60 {61 "id": 2,62 "count": 263 }64 ],65 "moderator": false,66 "admin": false,67 "staff": false,68 "user_id": 54889,69 "hidden": false,70 "trust_level": 1,71 "deleted_at": null,72 "user_deleted": false,73 "edit_reason": null,74 "can_view_edit_history": true,75 "wiki": false,76 "post_url": "/t/differentiable-and-learnable-rotations-with-grid-sample/148796/1",77 "can_accept_answer": false,78 "can_unaccept_answer": false,79 "accepted_answer": false,80 "topic_accepted_answer": null,81 "can_vote": false82 }83 ],84 "stream": [85 34084386 ]87 },88 "timeline_lookup": [89 [90 1,91 129492 ]93 ],94 "suggested_topics": [95 {96 "fancy_title": "Gradient and the tensor dtype inconsistencies",97 "id": 219066,98 "title": "Gradient and the tensor dtype inconsistencies",99 "slug": "gradient-and-the-tensor-dtype-inconsistencies",100 "posts_count": 4,101 "reply_count": 1,102 "highest_post_number": 4,103 "image_url": null,104 "created_at": "2025-04-14T14:07:44.627Z",105 "last_posted_at": "2025-04-15T14:13:39.474Z",106 "bumped": true,107 "bumped_at": "2025-04-15T14:13:39.474Z",108 "archetype": "regular",109 "unseen": false,110 "pinned": false,111 "unpinned": null,112 "visible": true,113 "closed": false,114 "archived": false,115 "bookmarked": null,116 "liked": null,117 "tags_descriptions": {},118 "like_count": 0,119 "views": 126,120 "category_id": 5,121 "featured_link": null,122 "has_accepted_answer": false,123 "posters": [124 {125 "extras": null,126 "description": "Original Poster",127 "user": {128 "id": 29433,129 "username": "cltexe",130 "name": "Omer Faruk Soylemez",131 "avatar_template": "/user_avatar/discuss.pytorch.org/cltexe/{size}/41817_2.png",132 "trust_level": 1133 }134 },135 {136 "extras": null,137 "description": "Frequent Poster",138 "user": {139 "id": 77908,140 "username": "mycul",141 "name": "",142 "avatar_template": "/user_avatar/discuss.pytorch.org/mycul/{size}/72394_2.png",143 "trust_level": 2144 }145 },146 {147 "extras": "latest",148 "description": "Most Recent Poster",149 "user": {150 "id": 3534,151 "username": "ptrblck",152 "name": "",153 "avatar_template": "/user_avatar/discuss.pytorch.org/ptrblck/{size}/1823_2.png",154 "admin": true,155 "moderator": true,156 "trust_level": 2157 }158 }159 ]160 },161 {162 "fancy_title": "Loss is always around 5.6 and doesn’t decrease, Accuracy is at 0.64% - 0.84%",163 "id": 220008,164 "title": "Loss is always around 5.6 and doesn't decrease, Accuracy is at 0.64% - 0.84%",165 "slug": "loss-is-always-around-5-6-and-doesnt-decrease-accuracy-is-at-0-64-0-84",166 "posts_count": 3,167 "reply_count": 0,168 "highest_post_number": 3,169 "image_url": null,170 "created_at": "2025-05-13T23:15:10.333Z",171 "last_posted_at": "2025-05-14T01:03:59.571Z",172 "bumped": true,173 "bumped_at": "2025-05-14T01:03:59.571Z",174 "archetype": "regular",175 "unseen": false,176 "pinned": false,177 "unpinned": null,178 "visible": true,179 "closed": false,180 "archived": false,181 "bookmarked": null,182 "liked": null,183 "tags_descriptions": {},184 "like_count": 0,185 "views": 52,186 "category_id": 5,187 "featured_link": null,188 "has_accepted_answer": false,189 "posters": [190 {191 "extras": null,192 "description": "Original Poster",193 "user": {194 "id": 84269,195 "username": "Ben_Sin",196 "name": "Ben Sin",197 "avatar_template": "/user_avatar/discuss.pytorch.org/ben_sin/{size}/77003_2.png",198 "trust_level": 0199 }200 },201 {202 "extras": null,203 "description": "Frequent Poster",204 "user": {205 "id": 9081,206 "username": "JuanFMontesinos",207 "name": "Juan Montesinos",208 "avatar_template": "/user_avatar/discuss.pytorch.org/juanfmontesinos/{size}/76115_2.png",209 "trust_level": 2210 }211 },212 {213 "extras": "latest",214 "description": "Most Recent Poster",215 "user": {216 "id": 1438,217 "username": "vdw",218 "name": "Chris",219 "avatar_template": "/user_avatar/discuss.pytorch.org/vdw/{size}/10074_2.png",220 "trust_level": 2221 }222 }223 ]224 },225 {226 "fancy_title": "Modifying an instance segmentation model to incorporate images and measurement data",227 "id": 217250,228 "title": "Modifying an instance segmentation model to incorporate images and measurement data",229 "slug": "modifying-an-instance-segmentation-model-to-incorporate-images-and-measurement-data",230 "posts_count": 1,231 "reply_count": 0,232 "highest_post_number": 1,233 "image_url": null,234 "created_at": "2025-02-27T21:52:00.292Z",235 "last_posted_at": "2025-02-27T21:52:00.336Z",236 "bumped": true,237 "bumped_at": "2025-02-28T14:35:17.605Z",238 "archetype": "regular",239 "unseen": false,240 "pinned": false,241 "unpinned": null,242 "visible": true,243 "closed": false,244 "archived": false,245 "bookmarked": null,246 "liked": null,247 "tags_descriptions": {},248 "like_count": 0,249 "views": 18,250 "category_id": 5,251 "featured_link": null,252 "has_accepted_answer": false,253 "posters": [254 {255 "extras": "latest single",256 "description": "Original Poster, Most Recent Poster",257 "user": {258 "id": 77908,259 "username": "mycul",260 "name": "",261 "avatar_template": "/user_avatar/discuss.pytorch.org/mycul/{size}/72394_2.png",262 "trust_level": 2263 }264 }265 ]266 },267 {268 "fancy_title": "Data not balanced",269 "id": 214375,270 "title": "Data not balanced",271 "slug": "data-not-balanced",272 "posts_count": 6,273 "reply_count": 3,274 "highest_post_number": 6,275 "image_url": "https://discuss.pytorch.org/uploads/default/optimized/3X/d/6/d657e80bcc250c583e5f136f133a110e250f7f2c_2_1024x526.png",276 "created_at": "2024-12-18T22:54:17.331Z",277 "last_posted_at": "2024-12-19T09:57:08.773Z",278 "bumped": true,279 "bumped_at": "2024-12-19T09:57:08.773Z",280 "archetype": "regular",281 "unseen": false,282 "pinned": false,283 "unpinned": null,284 "visible": true,285 "closed": false,286 "archived": false,287 "bookmarked": null,288 "liked": null,289 "tags_descriptions": {},290 "like_count": 2,291 "views": 312,292 "category_id": 5,293 "featured_link": null,294 "has_accepted_answer": false,295 "posters": [296 {297 "extras": "latest",298 "description": "Original Poster, Most Recent Poster",299 "user": {300 "id": 46221,301 "username": "mathwseg",302 "name": "mathwseg",303 "avatar_template": "/user_avatar/discuss.pytorch.org/mathwseg/{size}/39221_2.png",304 "trust_level": 1305 }306 },307 {308 "extras": null,309 "description": "Frequent Poster",310 "user": {311 "id": 41396,312 "username": "soulitzer",313 "name": "",314 "avatar_template": "/letter_avatar_proxy/v4/letter/s/839c29/{size}.png",315 "trust_level": 2316 }317 },318 {319 "extras": null,320 "description": "Frequent Poster",321 "user": {322 "id": 41458,323 "username": "J_Johnson",324 "name": "J Johnson",325 "avatar_template": "/user_avatar/discuss.pytorch.org/j_johnson/{size}/55494_2.png",326 "trust_level": 2327 }328 }329 ]330 },331 {332 "fancy_title": "YOLO + ViTPose + LSTM for sequence classification",333 "id": 220695,334 "title": "YOLO + ViTPose + LSTM for sequence classification",335 "slug": "yolo-vitpose-lstm-for-sequence-classification",336 "posts_count": 1,337 "reply_count": 0,338 "highest_post_number": 1,339 "image_url": null,340 "created_at": "2025-06-10T10:37:55.193Z",341 "last_posted_at": "2025-06-10T10:37:55.233Z",342 "bumped": true,343 "bumped_at": "2025-06-10T10:37:55.233Z",344 "archetype": "regular",345 "unseen": false,346 "pinned": false,347 "unpinned": null,348 "visible": true,349 "closed": false,350 "archived": false,351 "bookmarked": null,352 "liked": null,353 "tags_descriptions": {},354 "like_count": 0,355 "views": 68,356 "category_id": 5,357 "featured_link": null,358 "has_accepted_answer": false,359 "posters": [360 {361 "extras": "latest single",362 "description": "Original Poster, Most Recent Poster",363 "user": {364 "id": 84239,365 "username": "Soham_Bhaumik",366 "name": "Soham Bhaumik",367 "avatar_template": "/user_avatar/discuss.pytorch.org/soham_bhaumik/{size}/75355_2.png",368 "trust_level": 1369 }370 }371 ]372 }373 ],374 "tags_descriptions": {},375 "fancy_title": "Differentiable and learnable rotations with grid_sample",376 "id": 148796,377 "title": "Differentiable and learnable rotations with grid_sample",378 "posts_count": 1,379 "created_at": "2022-04-11T07:04:05.356Z",380 "views": 1411,381 "reply_count": 0,382 "like_count": 2,383 "last_posted_at": "2022-04-11T07:04:05.440Z",384 "visible": true,385 "closed": false,386 "archived": false,387 "has_summary": false,388 "archetype": "regular",389 "slug": "differentiable-and-learnable-rotations-with-grid-sample",390 "category_id": 5,391 "word_count": 657,392 "deleted_at": null,393 "user_id": 54889,394 "featured_link": null,395 "pinned_globally": false,396 "pinned_at": null,397 "pinned_until": null,398 "image_url": null,399 "slow_mode_seconds": 0,400 "draft": null,401 "draft_key": "topic_148796",402 "draft_sequence": null,403 "unpinned": null,404 "pinned": false,405 "current_post_number": 1,406 "highest_post_number": 1,407 "deleted_by": null,408 "actions_summary": [409 {410 "id": 4,411 "count": 0,412 "hidden": false,413 "can_act": false414 },415 {416 "id": 8,417 "count": 0,418 "hidden": false,419 "can_act": false420 },421 {422 "id": 10,423 "count": 0,424 "hidden": false,425 "can_act": false426 },427 {428 "id": 7,429 "count": 0,430 "hidden": false,431 "can_act": false432 }433 ],434 "chunk_size": 20,435 "bookmarked": false,436 "topic_timer": null,437 "message_bus_last_id": 0,438 "participant_count": 1,439 "show_read_indicator": false,440 "thumbnails": null,441 "slow_mode_enabled_until": null,442 "can_vote": false,443 "vote_count": 0,444 "user_voted": false,445 "discourse_zendesk_plugin_zendesk_id": null,446 "discourse_zendesk_plugin_zendesk_url": "https://your-url.zendesk.com/agent/tickets/",447 "details": {448 "can_edit": false,449 "notification_level": 1,450 "participants": [451 {452 "id": 54889,453 "username": "alexx",454 "name": "",455 "avatar_template": "/letter_avatar_proxy/v4/letter/a/ee59a6/{size}.png",456 "post_count": 1,457 "primary_group_name": null,458 "flair_name": null,459 "flair_url": null,460 "flair_color": null,461 "flair_bg_color": null,462 "flair_group_id": null,463 "trust_level": 1464 }465 ],466 "created_by": {467 "id": 54889,468 "username": "alexx",469 "name": "",470 "avatar_template": "/letter_avatar_proxy/v4/letter/a/ee59a6/{size}.png"471 },472 "last_poster": {473 "id": 54889,474 "username": "alexx",475 "name": "",476 "avatar_template": "/letter_avatar_proxy/v4/letter/a/ee59a6/{size}.png"477 },478 "links": [479 {480 "url": "https://discuss.pytorch.org/t/differentiable-affine-transforms-with-grid-sample/79305",481 "title": "Differentiable affine transforms with grid_sample",482 "internal": true,483 "attachment": false,484 "reflection": false,485 "clicks": 51,486 "user_id": 54889,487 "domain": "discuss.pytorch.org",488 "root_domain": "pytorch.org"489 },490 {491 "url": "https://github.com/GHamrouni",492 "title": "GHamrouni (Ghassen Hamrouni) · GitHub",493 "internal": false,494 "attachment": false,495 "reflection": false,496 "clicks": 6,497 "user_id": 54889,498 "domain": "github.com",499 "root_domain": "github.com"500 }501 ]502 },503 "bookmarks": []504 },505 {506 "post_stream": {507 "posts": [508 {509 "id": 340853,510 "name": "Louis FOUQUET",511 "username": "Louis_FOUQUET",512 "avatar_template": "/user_avatar/discuss.pytorch.org/louis_fouquet/{size}/48650_2.png",513 "created_at": "2022-04-11T08:04:43.285Z",514 "cooked": "<p>Hello,<br>\nI’m using the fasterrcnn_resnet50_fpn and there are two parameters pretrained (full model pretrained on COCO) and pretrained_backbone (only backbone pretrained on ImageNet).<br>\nSo I was wondering for the COCO model if this model was also trained using the ImageNet backbone or was it completely from scratch ?</p>",515 "post_number": 1,516 "post_type": 1,517 "posts_count": 1,518 "updated_at": "2022-04-11T08:04:43.285Z",519 "reply_count": 0,520 "reply_to_post_number": null,521 "quote_count": 0,522 "incoming_link_count": 62,523 "reads": 6,524 "readers_count": 5,525 "score": 311.2,526 "yours": false,527 "topic_id": 148804,528 "topic_slug": "is-the-faster-rcnn-resnet50-pretrained-on-coco-from-scratch",529 "display_username": "Louis FOUQUET",530 "primary_group_name": null,531 "flair_name": null,532 "flair_url": null,533 "flair_bg_color": null,534 "flair_color": null,535 "flair_group_id": null,536 "badges_granted": [],537 "version": 1,538 "can_edit": false,539 "can_delete": false,540 "can_recover": false,541 "can_see_hidden_post": false,542 "can_wiki": false,543 "read": true,544 "user_title": null,545 "bookmarked": false,546 "actions_summary": [],547 "moderator": false,548 "admin": false,549 "staff": false,550 "user_id": 54894,551 "hidden": false,552 "trust_level": 1,553 "deleted_at": null,554 "user_deleted": false,555 "edit_reason": null,556 "can_view_edit_history": true,557 "wiki": false,558 "post_url": "/t/is-the-faster-rcnn-resnet50-pretrained-on-coco-from-scratch/148804/1",559 "can_accept_answer": false,560 "can_unaccept_answer": false,561 "accepted_answer": false,562 "topic_accepted_answer": null,563 "can_vote": false564 }565 ],566 "stream": [567 340853568 ]569 },570 "timeline_lookup": [571 [572 1,573 1294574 ]575 ],576 "suggested_topics": [577 {578 "fancy_title": "List out of range when using boundings boxes in object detection",579 "id": 212657,580 "title": "List out of range when using boundings boxes in object detection",581 "slug": "list-out-of-range-when-using-boundings-boxes-in-object-detection",582 "posts_count": 1,583 "reply_count": 0,584 "highest_post_number": 1,585 "image_url": null,586 "created_at": "2024-11-07T10:27:30.812Z",587 "last_posted_at": "2024-11-07T10:27:30.862Z",588 "bumped": true,589 "bumped_at": "2024-11-07T10:27:30.862Z",590 "archetype": "regular",591 "unseen": false,592 "pinned": false,593 "unpinned": null,594 "visible": true,595 "closed": false,596 "archived": false,597 "bookmarked": null,598 "liked": null,599 "tags_descriptions": {},600 "like_count": 0,601 "views": 152,602 "category_id": 5,603 "featured_link": null,604 "has_accepted_answer": false,605 "posters": [606 {607 "extras": "latest single",608 "description": "Original Poster, Most Recent Poster",609 "user": {610 "id": 58229,611 "username": "Rexedoziem",612 "name": "Rexedoziem",613 "avatar_template": "/user_avatar/discuss.pytorch.org/rexedoziem/{size}/52029_2.png",614 "trust_level": 1615 }616 }617 ]618 },619 {620 "fancy_title": "Error in Encoder Decoder Based Model",621 "id": 214303,622 "title": "Error in Encoder Decoder Based Model",623 "slug": "error-in-encoder-decoder-based-model",624 "posts_count": 2,625 "reply_count": 0,626 "highest_post_number": 2,627 "image_url": null,628 "created_at": "2024-12-17T10:53:49.759Z",629 "last_posted_at": "2025-10-21T23:07:15.141Z",630 "bumped": true,631 "bumped_at": "2025-10-21T23:07:15.141Z",632 "archetype": "regular",633 "unseen": false,634 "pinned": false,635 "unpinned": null,636 "visible": true,637 "closed": false,638 "archived": false,639 "bookmarked": null,640 "liked": null,641 "tags_descriptions": {},642 "like_count": 0,643 "views": 87,644 "category_id": 5,645 "featured_link": null,646 "has_accepted_answer": false,647 "posters": [648 {649 "extras": null,650 "description": "Original Poster",651 "user": {652 "id": 81422,653 "username": "Idrees11",654 "name": "Idrees Bhat",655 "avatar_template": "/user_avatar/discuss.pytorch.org/idrees11/{size}/74448_2.png",656 "trust_level": 1657 }658 },659 {660 "extras": "latest",661 "description": "Most Recent Poster",662 "user": {663 "id": 83624,664 "username": "sorenmadsen",665 "name": "Soren",666 "avatar_template": "/user_avatar/discuss.pytorch.org/sorenmadsen/{size}/78491_2.png",667 "trust_level": 1668 }669 }670 ]671 },672 {673 "fancy_title": "Faster Vit Hierarchical Attention",674 "id": 214037,675 "title": "Faster Vit Hierarchical Attention",676 "slug": "faster-vit-hierarchical-attention",677 "posts_count": 1,678 "reply_count": 0,679 "highest_post_number": 1,680 "image_url": null,681 "created_at": "2024-12-10T08:36:29.626Z",682 "last_posted_at": "2024-12-10T08:36:29.684Z",683 "bumped": true,684 "bumped_at": "2024-12-10T08:36:29.684Z",685 "archetype": "regular",686 "unseen": false,687 "pinned": false,688 "unpinned": null,689 "visible": true,690 "closed": false,691 "archived": false,692 "bookmarked": null,693 "liked": null,694 "tags_descriptions": {},695 "like_count": 0,696 "views": 127,697 "category_id": 5,698 "featured_link": null,699 "has_accepted_answer": false,700 "posters": [701 {702 "extras": "latest single",703 "description": "Original Poster, Most Recent Poster",704 "user": {705 "id": 81424,706 "username": "hussainmir05",707 "name": "Hussain mir",708 "avatar_template": "/user_avatar/discuss.pytorch.org/hussainmir05/{size}/72721_2.png",709 "trust_level": 0710 }711 }712 ]713 },714 {715 "fancy_title": "CNN with Custom Convolutions, loss NAN",716 "id": 214137,717 "title": "CNN with Custom Convolutions, loss NAN",718 "slug": "cnn-with-custom-convolutions-loss-nan",719 "posts_count": 1,720 "reply_count": 0,721 "highest_post_number": 1,722 "image_url": "https://discuss.pytorch.org/uploads/default/original/3X/a/5/a5183b79ade7a3d44fca8ca047f0d878a1f250b8.png",723 "created_at": "2024-12-12T06:44:53.771Z",724 "last_posted_at": "2024-12-12T06:44:53.827Z",725 "bumped": true,726 "bumped_at": "2024-12-12T06:44:53.827Z",727 "archetype": "regular",728 "unseen": false,729 "pinned": false,730 "unpinned": null,731 "visible": true,732 "closed": false,733 "archived": false,734 "bookmarked": null,735 "liked": null,736 "tags_descriptions": {},737 "like_count": 0,738 "views": 82,739 "category_id": 5,740 "featured_link": null,741 "has_accepted_answer": false,742 "posters": [743 {744 "extras": "latest single",745 "description": "Original Poster, Most Recent Poster",746 "user": {747 "id": 81464,748 "username": "sharom_m",749 "name": "",750 "avatar_template": "/user_avatar/discuss.pytorch.org/sharom_m/{size}/73009_2.png",751 "trust_level": 1752 }753 }754 ]755 },756 {757 "fancy_title": "Getting “Could not initialize NNPACK! Reason: Unsupported hardware.” warning even though NNPACK is enabled",758 "id": 214397,759 "title": "Getting \"Could not initialize NNPACK! Reason: Unsupported hardware.\" warning even though NNPACK is enabled",760 "slug": "getting-could-not-initialize-nnpack-reason-unsupported-hardware-warning-even-though-nnpack-is-enabled",761 "posts_count": 1,762 "reply_count": 0,763 "highest_post_number": 1,764 "image_url": null,765 "created_at": "2024-12-19T09:39:58.123Z",766 "last_posted_at": "2024-12-19T09:39:58.160Z",767 "bumped": true,768 "bumped_at": "2024-12-19T09:39:58.160Z",769 "archetype": "regular",770 "unseen": false,771 "pinned": false,772 "unpinned": null,773 "visible": true,774 "closed": false,775 "archived": false,776 "bookmarked": null,777 "liked": null,778 "tags_descriptions": {},779 "like_count": 0,780 "views": 238,781 "category_id": 5,782 "featured_link": null,783 "has_accepted_answer": false,784 "posters": [785 {786 "extras": "latest single",787 "description": "Original Poster, Most Recent Poster",788 "user": {789 "id": 81586,790 "username": "KirilloCirillo",791 "name": "",792 "avatar_template": "/user_avatar/discuss.pytorch.org/kirillocirillo/{size}/74611_2.png",793 "trust_level": 1794 }795 }796 ]797 }798 ],799 "tags_descriptions": {},800 "fancy_title": "Is the faster-rcnn-resnet50 pretrained on COCO from scratch?",801 "id": 148804,802 "title": "Is the faster-rcnn-resnet50 pretrained on COCO from scratch?",803 "posts_count": 1,804 "created_at": "2022-04-11T08:04:43.228Z",805 "views": 477,806 "reply_count": 0,807 "like_count": 0,808 "last_posted_at": "2022-04-11T08:04:43.285Z",809 "visible": true,810 "closed": false,811 "archived": false,812 "has_summary": false,813 "archetype": "regular",814 "slug": "is-the-faster-rcnn-resnet50-pretrained-on-coco-from-scratch",815 "category_id": 5,816 "word_count": 48,817 "deleted_at": null,818 "user_id": 54894,819 "featured_link": null,820 "pinned_globally": false,821 "pinned_at": null,822 "pinned_until": null,823 "image_url": null,824 "slow_mode_seconds": 0,825 "draft": null,826 "draft_key": "topic_148804",827 "draft_sequence": null,828 "unpinned": null,829 "pinned": false,830 "current_post_number": 1,831 "highest_post_number": 1,832 "deleted_by": null,833 "actions_summary": [834 {835 "id": 4,836 "count": 0,837 "hidden": false,838 "can_act": false839 },840 {841 "id": 8,842 "count": 0,843 "hidden": false,844 "can_act": false845 },846 {847 "id": 10,848 "count": 0,849 "hidden": false,850 "can_act": false851 },852 {853 "id": 7,854 "count": 0,855 "hidden": false,856 "can_act": false857 }858 ],859 "chunk_size": 20,860 "bookmarked": false,861 "topic_timer": null,862 "message_bus_last_id": 0,863 "participant_count": 1,864 "show_read_indicator": false,865 "thumbnails": null,866 "slow_mode_enabled_until": null,867 "can_vote": false,868 "vote_count": 0,869 "user_voted": false,870 "discourse_zendesk_plugin_zendesk_id": null,871 "discourse_zendesk_plugin_zendesk_url": "https://your-url.zendesk.com/agent/tickets/",872 "details": {873 "can_edit": false,874 "notification_level": 1,875 "participants": [876 {877 "id": 54894,878 "username": "Louis_FOUQUET",879 "name": "Louis FOUQUET",880 "avatar_template": "/user_avatar/discuss.pytorch.org/louis_fouquet/{size}/48650_2.png",881 "post_count": 1,882 "primary_group_name": null,883 "flair_name": null,884 "flair_url": null,885 "flair_color": null,886 "flair_bg_color": null,887 "flair_group_id": null,888 "trust_level": 1889 }890 ],891 "created_by": {892 "id": 54894,893 "username": "Louis_FOUQUET",894 "name": "Louis FOUQUET",895 "avatar_template": "/user_avatar/discuss.pytorch.org/louis_fouquet/{size}/48650_2.png"896 },897 "last_poster": {898 "id": 54894,899 "username": "Louis_FOUQUET",900 "name": "Louis FOUQUET",901 "avatar_template": "/user_avatar/discuss.pytorch.org/louis_fouquet/{size}/48650_2.png"902 }903 },904 "bookmarks": []905 },906 {907 "post_stream": {908 "posts": [909 {910 "id": 340832,911 "name": "Siladittya Manna",912 "username": "Siladittya_Manna",913 "avatar_template": "/user_avatar/discuss.pytorch.org/siladittya_manna/{size}/26984_2.png",914 "created_at": "2022-04-11T05:17:27.233Z",915 "cooked": "<p>I am using two version of my loss function</p>\n<p>In version 1, I am using <code>torch.mean(torch.square(Tensor))</code> to calculate the loss</p>\n<p>In version 2, I am using <code>Tensor.pow_(2).sum()/D</code> to clculate the loss</p>\n<p>But the behavior of the loss in the two versions are completely different. The version 2 works fine, but using version 1 I get horrible results.</p>\n<p>Both the experiments have the exactly same hyper-parameter configuration.</p>\n<p>Does not version 1 and 2, represent the same thing?</p>",916 "post_number": 1,917 "post_type": 1,918 "posts_count": 3,919 "updated_at": "2022-04-11T05:17:58.856Z",920 "reply_count": 1,921 "reply_to_post_number": null,922 "quote_count": 0,923 "incoming_link_count": 857,924 "reads": 12,925 "readers_count": 11,926 "score": 4282.4,927 "yours": false,928 "topic_id": 148789,929 "topic_slug": "difference-between-torch-mean-torch-square-tensor-and-tensor-pow-2-sum-d",930 "display_username": "Siladittya Manna",931 "primary_group_name": null,932 "flair_name": null,933 "flair_url": null,934 "flair_bg_color": null,935 "flair_color": null,936 "flair_group_id": null,937 "badges_granted": [],938 "version": 1,939 "can_edit": false,940 "can_delete": false,941 "can_recover": false,942 "can_see_hidden_post": false,943 "can_wiki": false,944 "read": true,945 "user_title": null,946 "bookmarked": false,947 "actions_summary": [],948 "moderator": false,949 "admin": false,950 "staff": false,951 "user_id": 34624,952 "hidden": false,953 "trust_level": 2,954 "deleted_at": null,955 "user_deleted": false,956 "edit_reason": null,957 "can_view_edit_history": true,958 "wiki": false,959 "post_url": "/t/difference-between-torch-mean-torch-square-tensor-and-tensor-pow-2-sum-d/148789/1",960 "can_accept_answer": false,961 "can_unaccept_answer": false,962 "accepted_answer": false,963 "topic_accepted_answer": true,964 "can_vote": false965 },966 {967 "id": 340833,968 "name": "Arul",969 "username": "InnovArul",970 "avatar_template": "/user_avatar/discuss.pytorch.org/innovarul/{size}/5282_2.png",971 "created_at": "2022-04-11T05:25:17.905Z",972 "cooked": "<aside class=\"quote no-group\" data-username=\"Siladittya_Manna\" data-post=\"1\" data-topic=\"148789\">\n<div class=\"title\">\n<div class=\"quote-controls\"></div>\n<img loading=\"lazy\" alt=\"\" width=\"24\" height=\"24\" src=\"https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/siladittya_manna/48/26984_2.png\" class=\"avatar\"> Siladittya_Manna:</div>\n<blockquote>\n<p>torch.mean(torch.square(Tensor))</p>\n</blockquote>\n</aside>\n<p>Note that <code>torch.mean()</code> will divide the sum by the count of all the dimensions. i.e., if the tensor is 4D (B,D,H,W), the division factor is (B*D*H*W). If the tensor is 2D (B,D), the division factor is (B*D).<br>\nI am not sure about the tensor dimension in your case. Check it maybe.</p>",973 "post_number": 2,974 "post_type": 1,975 "posts_count": 3,976 "updated_at": "2022-04-11T05:25:48.758Z",977 "reply_count": 1,978 "reply_to_post_number": null,979 "quote_count": 1,980 "incoming_link_count": 3,981 "reads": 12,982 "readers_count": 11,983 "score": 22.4,984 "yours": false,985 "topic_id": 148789,986 "topic_slug": "difference-between-torch-mean-torch-square-tensor-and-tensor-pow-2-sum-d",987 "display_username": "Arul",988 "primary_group_name": null,989 "flair_name": null,990 "flair_url": null,991 "flair_bg_color": null,992 "flair_color": null,993 "flair_group_id": null,994 "badges_granted": [],995 "version": 1,996 "can_edit": false,997 "can_delete": false,998 "can_recover": false,999 "can_see_hidden_post": false,1000 "can_wiki": false,1001 "read": true,1002 "user_title": "",1003 "bookmarked": false,1004 "actions_summary": [],1005 "moderator": false,1006 "admin": false,1007 "staff": false,1008 "user_id": 998,1009 "hidden": false,1010 "trust_level": 2,1011 "deleted_at": null,1012 "user_deleted": false,1013 "edit_reason": null,1014 "can_view_edit_history": true,1015 "wiki": false,1016 "post_url": "/t/difference-between-torch-mean-torch-square-tensor-and-tensor-pow-2-sum-d/148789/2",1017 "can_accept_answer": false,1018 "can_unaccept_answer": false,1019 "accepted_answer": true,1020 "topic_accepted_answer": true1021 },1022 {1023 "id": 340846,1024 "name": "Siladittya Manna",1025 "username": "Siladittya_Manna",1026 "avatar_template": "/user_avatar/discuss.pytorch.org/siladittya_manna/{size}/26984_2.png",1027 "created_at": "2022-04-11T07:13:47.613Z",1028 "cooked": "<p>Yes. That was the issue. Thanks for pointing it out.</p>",1029 "post_number": 3,1030 "post_type": 1,1031 "posts_count": 3,1032 "updated_at": "2022-04-11T07:13:47.613Z",1033 "reply_count": 0,1034 "reply_to_post_number": 2,1035 "quote_count": 0,1036 "incoming_link_count": 4,1037 "reads": 11,1038 "readers_count": 10,1039 "score": 22.2,1040 "yours": false,1041 "topic_id": 148789,1042 "topic_slug": "difference-between-torch-mean-torch-square-tensor-and-tensor-pow-2-sum-d",1043 "display_username": "Siladittya Manna",1044 "primary_group_name": null,1045 "flair_name": null,1046 "flair_url": null,1047 "flair_bg_color": null,1048 "flair_color": null,1049 "flair_group_id": null,1050 "badges_granted": [],1051 "version": 1,1052 "can_edit": false,1053 "can_delete": false,1054 "can_recover": false,1055 "can_see_hidden_post": false,1056 "can_wiki": false,1057 "read": true,1058 "user_title": null,1059 "reply_to_user": {1060 "id": 998,1061 "username": "InnovArul",1062 "name": "Arul",1063 "avatar_template": "/user_avatar/discuss.pytorch.org/innovarul/{size}/5282_2.png"1064 },1065 "bookmarked": false,1066 "actions_summary": [],1067 "moderator": false,1068 "admin": false,1069 "staff": false,1070 "user_id": 34624,1071 "hidden": false,1072 "trust_level": 2,1073 "deleted_at": null,1074 "user_deleted": false,1075 "edit_reason": null,1076 "can_view_edit_history": true,1077 "wiki": false,1078 "post_url": "/t/difference-between-torch-mean-torch-square-tensor-and-tensor-pow-2-sum-d/148789/3",1079 "can_accept_answer": false,1080 "can_unaccept_answer": false,1081 "accepted_answer": false,1082 "topic_accepted_answer": true1083 }1084 ],1085 "stream": [1086 340832,1087 340833,1088 3408461089 ]1090 },1091 "timeline_lookup": [1092 [1093 1,1094 12941095 ]1096 ],1097 "suggested_topics": [1098 {1099 "fancy_title": "Inconsistencies between PyTorch and NumPy when performing 32-bit floating-point sums",1100 "id": 212931,1101 "title": "Inconsistencies between PyTorch and NumPy when performing 32-bit floating-point sums",1102 "slug": "inconsistencies-between-pytorch-and-numpy-when-performing-32-bit-floating-point-sums",1103 "posts_count": 5,1104 "reply_count": 3,1105 "highest_post_number": 5,1106 "image_url": null,1107 "created_at": "2024-11-13T14:10:23.178Z",1108 "last_posted_at": "2024-11-13T19:58:41.733Z",1109 "bumped": true,1110 "bumped_at": "2024-11-13T19:58:41.733Z",1111 "archetype": "regular",1112 "unseen": false,1113 "pinned": false,1114 "unpinned": null,1115 "visible": true,1116 "closed": false,1117 "archived": false,1118 "bookmarked": null,1119 "liked": null,1120 "tags_descriptions": {},1121 "like_count": 2,1122 "views": 122,1123 "category_id": 1,1124 "featured_link": null,1125 "has_accepted_answer": true,1126 "posters": [1127 {1128 "extras": "latest",1129 "description": "Original Poster, Most Recent Poster",1130 "user": {1131 "id": 80884,1132 "username": "sgolodetz",1133 "name": "Stuart Golodetz",1134 "avatar_template": "/user_avatar/discuss.pytorch.org/sgolodetz/{size}/73975_2.png",1135 "trust_level": 11136 }1137 },1138 {1139 "extras": null,1140 "description": "Frequent Poster, Accepted Answer",1141 "user": {1142 "id": 3534,1143 "username": "ptrblck",1144 "name": "",1145 "avatar_template": "/user_avatar/discuss.pytorch.org/ptrblck/{size}/1823_2.png",1146 "admin": true,1147 "moderator": true,1148 "trust_level": 21149 }1150 }1151 ]1152 },1153 {1154 "fancy_title": "Why kernels different streams can’t in parallel",1155 "id": 215618,1156 "title": "Why kernels different streams can't in parallel",1157 "slug": "why-kernels-different-streams-cant-in-parallel",1158 "posts_count": 2,1159 "reply_count": 0,1160 "highest_post_number": 2,1161 "image_url": "https://discuss.pytorch.org/uploads/default/optimized/3X/f/f/ff20ec5488f7548c57f81427e2ca0e83062f5ee9_2_1024x801.png",1162 "created_at": "2025-01-20T07:24:07.095Z",1163 "last_posted_at": "2025-01-20T13:51:41.366Z",1164 "bumped": true,1165 "bumped_at": "2025-01-20T13:51:41.366Z",1166 "archetype": "regular",1167 "unseen": false,1168 "pinned": false,1169 "unpinned": null,1170 "visible": true,1171 "closed": false,1172 "archived": false,1173 "bookmarked": null,1174 "liked": null,1175 "tags_descriptions": {},1176 "like_count": 0,1177 "views": 206,1178 "category_id": 1,1179 "featured_link": null,1180 "has_accepted_answer": false,1181 "posters": [1182 {1183 "extras": null,1184 "description": "Original Poster",1185 "user": {1186 "id": 72471,1187 "username": "shadowshadow",1188 "name": "",1189 "avatar_template": "/user_avatar/discuss.pytorch.org/shadowshadow/{size}/62985_2.png",1190 "trust_level": 21191 }1192 },1193 {1194 "extras": "latest",1195 "description": "Most Recent Poster",1196 "user": {1197 "id": 3534,1198 "username": "ptrblck",1199 "name": "",1200 "avatar_template": "/user_avatar/discuss.pytorch.org/ptrblck/{size}/1823_2.png",