Anurag1734/cuda-error-resolution-analysis
07
1[2 {3 "post_stream": {4 "posts": [5 {6 "id": 429994,7 "name": "Scolpe",8 "username": "Scolpe",9 "avatar_template": "/user_avatar/discuss.pytorch.org/scolpe/{size}/61541_2.png",10 "created_at": "2024-01-12T17:37:21.123Z",11 "cooked": "<p>I am currently trying to understand whether the situation that I’ve encountered is a normal behaviour or a bug. I run experiments that include training models in a simulated distributed environment. Without going into unnecessary details, in each round, clients train on a local trainset, test it against the local test set and report the values. The values are then stored in a csv file together with models that were tested.</p>\n<p>To run a validation check, I fix the seed and load the local model and the local test set. Subsequently, I perform a test evaluation. What bothers me is the fact that the values reported in a csv file (test values recorded during simulation) are not fully aligned with test values that I obtain when checking the simulation validity afterwards.</p>\n<p>This implies that the same model (with the same weights) tested on the same dataset and with fixed seed obtains two different results. As the model stabilizes (for N rounds, we will have N different models), the difference between the value reported in a csv file and one obtained during replication is close to 0. As an example, I am pasting the log below:</p>\n<pre><code class=\"lang-auto\">0: Iteration, Abs. Loss Diff.: 1.0015488862991333, Abs. Acc. Diff.: 0.25\n1: Iteration, Abs. Loss Diff.: 0.0009263801574705965, Abs. Acc. Diff.: 0.0\n2: Iteration, Abs. Loss Diff.: 0.005786736011505145, Abs. Acc. Diff.: 0.0\n3: Iteration, Abs. Loss Diff.: 0.003574820756912178, Abs. Acc. Diff.: 0.0\n4: Iteration, Abs. Loss Diff.: 0.007152392864227308, Abs. Acc. Diff.: 0.0\n5: Iteration, Abs. Loss Diff.: 0.0015836870670318248, Abs. Acc. Diff.: 0.0\n6: Iteration, Abs. Loss Diff.: 0.00476664781570435, Abs. Acc. Diff.: 0.0\n7: Iteration, Abs. Loss Diff.: 0.003446925878524798, Abs. Acc. Diff.: 0.0\n8: Iteration, Abs. Loss Diff.: 0.0017982900142670122, Abs. Acc. Diff.: 0.0\n9: Iteration, Abs. Loss Diff.: 0.0006368839740753529, Abs. Acc. Diff.: 0.0\n10: Iteration, Abs. Loss Diff.: 0.009332650899887107, Abs. Acc. Diff.: 0.0\n11: Iteration, Abs. Loss Diff.: 0.0002723556756972778, Abs. Acc. Diff.: 0.0\n12: Iteration, Abs. Loss Diff.: 0.010622120499610865, Abs. Acc. Diff.: 0.0\n13: Iteration, Abs. Loss Diff.: 0.004144576042890535, Abs. Acc. Diff.: 0.0\n14: Iteration, Abs. Loss Diff.: 0.00525180220603938, Abs. Acc. Diff.: 0.0\n15: Iteration, Abs. Loss Diff.: 0.013058926761150391, Abs. Acc. Diff.: 0.0\n16: Iteration, Abs. Loss Diff.: 0.008403560966253276, Abs. Acc. Diff.: 0.0\n17: Iteration, Abs. Loss Diff.: 0.012890378683805492, Abs. Acc. Diff.: 0.0\n18: Iteration, Abs. Loss Diff.: 0.015538938939571367, Abs. Acc. Diff.: 0.0\n19: Iteration, Abs. Loss Diff.: 0.03375539824366569, Abs. Acc. Diff.: 0.0\n20: Iteration, Abs. Loss Diff.: 0.0018654009699821117, Abs. Acc. Diff.: 0.0\n21: Iteration, Abs. Loss Diff.: 0.008243808336555913, Abs. Acc. Diff.: 0.0\n22: Iteration, Abs. Loss Diff.: 0.00302100986242293, Abs. Acc. Diff.: 0.0\n23: Iteration, Abs. Loss Diff.: 0.004521983098238702, Abs. Acc. Diff.: 0.0\n24: Iteration, Abs. Loss Diff.: 0.008875386621803094, Abs. Acc. Diff.: 0.0\n...\n46: Iteration, Abs. Loss Diff.: 0.046149560796329814, Abs. Acc. Diff.: 0.0\n47: Iteration, Abs. Loss Diff.: 0.0725968092895346, Abs. Acc. Diff.: 0.0\n48: Iteration, Abs. Loss Diff.: 0.03759608950349502, Abs. Acc. Diff.: 0.0\n49: Iteration, Abs. Loss Diff.: 0.05040962719998787, Abs. Acc. Diff.: 0.0\n</code></pre>\n<p>Even though the value is stabilizing, I find this behaviour strange. Can it be due to an inherent randomness of some of the PyTorch components? The full code is much to complex to demonstrate fully, but I am also including my testing function.</p>\n<pre><code class=\"lang-auto\">def test_loop(net: torch.nn,\n testdata = torch.utils.data.DataLoader):\n net.to(device)\n net.eval()\n criterion = nn.CrossEntropyLoss()\n test_loss = 0\n correct = 0\n total = 0\n y_pred = []\n y_true = []\n losses = []\n \n with torch.no_grad():\n for _, dic in enumerate(testdata):\n inputs = dic['image']\n targets = dic['label']\n inputs, targets = inputs.to(device), targets.to(device)\n outputs = net(inputs)\n \n ######################\n outputs = outputs.cpu()\n targets = targets.cpu()\n #######################\n \n total += targets.size(0)\n test_loss = criterion(outputs, targets)\n losses.append(test_loss)\n pred = outputs.argmax(dim=1, keepdim=True)\n correct += pred.eq(targets.view_as(pred)).sum().item()\n y_pred.append(pred)\n y_true.append(targets)\n \n test_loss = np.mean(losses)\n accuracy = correct / total\n \n \n y_true = [item.item() for sublist in y_true for item in sublist]\n y_pred = [item.item() for sublist in y_pred for item in sublist]\n\n...\n \n return {\n 'test_loss': test_loss,\n 'accuracy': accuracy,\n...\n 'false_positive_rate': false_positive_rate\n }\n \n</code></pre>",12 "post_number": 1,13 "post_type": 1,14 "posts_count": 3,15 "updated_at": "2024-01-12T17:37:21.123Z",16 "reply_count": 0,17 "reply_to_post_number": null,18 "quote_count": 0,19 "incoming_link_count": 35,20 "reads": 6,21 "readers_count": 5,22 "score": 171.2,23 "yours": false,24 "topic_id": 195330,25 "topic_slug": "different-test-results-with-a-fixed-weights-and-fixed-seed",26 "display_username": "Scolpe",27 "primary_group_name": null,28 "flair_name": null,29 "flair_url": null,30 "flair_bg_color": null,31 "flair_color": null,32 "flair_group_id": null,33 "badges_granted": [],34 "version": 1,35 "can_edit": false,36 "can_delete": false,37 "can_recover": false,38 "can_see_hidden_post": false,39 "can_wiki": false,40 "read": true,41 "user_title": null,42 "bookmarked": false,43 "actions_summary": [],44 "moderator": false,45 "admin": false,46 "staff": false,47 "user_id": 67235,48 "hidden": false,49 "trust_level": 1,50 "deleted_at": null,51 "user_deleted": false,52 "edit_reason": null,53 "can_view_edit_history": true,54 "wiki": false,55 "post_url": "/t/different-test-results-with-a-fixed-weights-and-fixed-seed/195330/1",56 "can_accept_answer": false,57 "can_unaccept_answer": false,58 "accepted_answer": false,59 "topic_accepted_answer": null,60 "can_vote": false61 },62 {63 "id": 430000,64 "name": "",65 "username": "ptrblck",66 "avatar_template": "/user_avatar/discuss.pytorch.org/ptrblck/{size}/1823_2.png",67 "created_at": "2024-01-12T18:57:41.983Z",68 "cooked": "<p>I don’t see if and where you’ve enabled deterministic algorithms as described in the <a href=\"https://pytorch.org/docs/stable/notes/randomness.html\">Reproducibility docs</a>. Did you check the docs and followed them?</p>",69 "post_number": 2,70 "post_type": 1,71 "posts_count": 3,72 "updated_at": "2024-01-12T18:57:41.983Z",73 "reply_count": 1,74 "reply_to_post_number": null,75 "quote_count": 0,76 "incoming_link_count": 2,77 "reads": 6,78 "readers_count": 5,79 "score": 31.2,80 "yours": false,81 "topic_id": 195330,82 "topic_slug": "different-test-results-with-a-fixed-weights-and-fixed-seed",83 "display_username": "",84 "primary_group_name": null,85 "flair_name": null,86 "flair_url": null,87 "flair_bg_color": null,88 "flair_color": null,89 "flair_group_id": null,90 "badges_granted": [],91 "version": 1,92 "can_edit": false,93 "can_delete": false,94 "can_recover": false,95 "can_see_hidden_post": false,96 "can_wiki": false,97 "link_counts": [98 {99 "url": "https://pytorch.org/docs/stable/notes/randomness.html",100 "internal": false,101 "reflection": false,102 "title": "Reproducibility — PyTorch 2.1 documentation",103 "clicks": 4104 }105 ],106 "read": true,107 "user_title": "",108 "bookmarked": false,109 "actions_summary": [110 {111 "id": 2,112 "count": 1113 }114 ],115 "moderator": true,116 "admin": true,117 "staff": true,118 "user_id": 3534,119 "hidden": false,120 "trust_level": 2,121 "deleted_at": null,122 "user_deleted": false,123 "edit_reason": null,124 "can_view_edit_history": true,125 "wiki": false,126 "post_url": "/t/different-test-results-with-a-fixed-weights-and-fixed-seed/195330/2",127 "can_accept_answer": false,128 "can_unaccept_answer": false,129 "accepted_answer": false,130 "topic_accepted_answer": null131 },132 {133 "id": 430075,134 "name": "Scolpe",135 "username": "Scolpe",136 "avatar_template": "/user_avatar/discuss.pytorch.org/scolpe/{size}/61541_2.png",137 "created_at": "2024-01-13T12:05:46.232Z",138 "cooked": "<p>Yes, I’ve first followed the documents on reproductibility.</p>\n<p>I am fixing seeds and enabling deterministic algorithms in the main script from which I am running the simulation. The script opens with imports and the following lines:</p>\n<pre><code class=\"lang-auto\">random.seed(42)\nnp.random.seed(42)\ntorch.cuda.manual_seed(42)\ntorch.cuda.manual_seed_all(42)\ntorch.use_deterministic_algorithms(True)\ntorch.backends.cudnn.deterministic = True\ntorch.backends.cudnn.benchmark = False\n</code></pre>\n<p>Then I call <code> CUBLAS_WORKSPACE_CONFIG=:16:8 python script.py args</code>. The script calls other libraries and modules and runs a full simulation cycle.<br>\nWhen I am analyzing the results in the jupyter notebook, I am using the same commands:</p>\n<pre><code class=\"lang-auto\">random.seed(42)\nnp.random.seed(42)\ntorch.cuda.manual_seed(42)\ntorch.cuda.manual_seed_all(42)\ntorch.use_deterministic_algorithms(True)\ntorch.backends.cudnn.deterministic = True\ntorch.backends.cudnn.benchmark = False\n</code></pre>\n<p>The only thing that I am not fixing is the seed for the testloader. However, given that the datasets are already partitioned into training/testing data…my guess is that it should not make much difference (?). But maybe I am wrong on this one.</p>",139 "post_number": 3,140 "post_type": 1,141 "posts_count": 3,142 "updated_at": "2024-01-13T12:05:46.232Z",143 "reply_count": 0,144 "reply_to_post_number": 2,145 "quote_count": 0,146 "incoming_link_count": 4,147 "reads": 5,148 "readers_count": 4,149 "score": 21.0,150 "yours": false,151 "topic_id": 195330,152 "topic_slug": "different-test-results-with-a-fixed-weights-and-fixed-seed",153 "display_username": "Scolpe",154 "primary_group_name": null,155 "flair_name": null,156 "flair_url": null,157 "flair_bg_color": null,158 "flair_color": null,159 "flair_group_id": null,160 "badges_granted": [],161 "version": 1,162 "can_edit": false,163 "can_delete": false,164 "can_recover": false,165 "can_see_hidden_post": false,166 "can_wiki": false,167 "read": true,168 "user_title": null,169 "reply_to_user": {170 "id": 3534,171 "username": "ptrblck",172 "name": "",173 "avatar_template": "/user_avatar/discuss.pytorch.org/ptrblck/{size}/1823_2.png"174 },175 "bookmarked": false,176 "actions_summary": [],177 "moderator": false,178 "admin": false,179 "staff": false,180 "user_id": 67235,181 "hidden": false,182 "trust_level": 1,183 "deleted_at": null,184 "user_deleted": false,185 "edit_reason": null,186 "can_view_edit_history": true,187 "wiki": false,188 "post_url": "/t/different-test-results-with-a-fixed-weights-and-fixed-seed/195330/3",189 "can_accept_answer": false,190 "can_unaccept_answer": false,191 "accepted_answer": false,192 "topic_accepted_answer": null193 }194 ],195 "stream": [196 429994,197 430000,198 430075199 ]200 },201 "timeline_lookup": [202 [203 1,204 652205 ],206 [207 3,208 651209 ]210 ],211 "suggested_topics": [212 {213 "fancy_title": "How to use DataLoader pin memory when data is a mix of cpu and gpu",214 "id": 219625,215 "title": "How to use DataLoader pin memory when data is a mix of cpu and gpu",216 "slug": "how-to-use-dataloader-pin-memory-when-data-is-a-mix-of-cpu-and-gpu",217 "posts_count": 2,218 "reply_count": 0,219 "highest_post_number": 2,220 "image_url": null,221 "created_at": "2025-04-30T10:05:08.269Z",222 "last_posted_at": "2025-04-30T15:01:57.023Z",223 "bumped": true,224 "bumped_at": "2025-04-30T15:01:57.023Z",225 "archetype": "regular",226 "unseen": false,227 "pinned": false,228 "unpinned": null,229 "visible": true,230 "closed": false,231 "archived": false,232 "bookmarked": null,233 "liked": null,234 "tags_descriptions": {},235 "like_count": 0,236 "views": 51,237 "category_id": 37,238 "featured_link": null,239 "has_accepted_answer": false,240 "posters": [241 {242 "extras": null,243 "description": "Original Poster",244 "user": {245 "id": 863,246 "username": "jj0mst",247 "name": "Jj0mst",248 "avatar_template": "/user_avatar/discuss.pytorch.org/jj0mst/{size}/36570_2.png",249 "trust_level": 1250 }251 },252 {253 "extras": "latest",254 "description": "Most Recent Poster",255 "user": {256 "id": 3534,257 "username": "ptrblck",258 "name": "",259 "avatar_template": "/user_avatar/discuss.pytorch.org/ptrblck/{size}/1823_2.png",260 "admin": true,261 "moderator": true,262 "trust_level": 2263 }264 }265 ]266 },267 {268 "fancy_title": "How to load a large text file into datasets for pretraining llm",269 "id": 214357,270 "title": "How to load a large text file into datasets for pretraining llm",271 "slug": "how-to-load-a-large-text-file-into-datasets-for-pretraining-llm",272 "posts_count": 3,273 "reply_count": 1,274 "highest_post_number": 3,275 "image_url": null,276 "created_at": "2024-12-18T09:53:07.657Z",277 "last_posted_at": "2024-12-19T04:27:59.983Z",278 "bumped": true,279 "bumped_at": "2024-12-19T04:27:59.983Z",280 "archetype": "regular",281 "unseen": false,282 "pinned": false,283 "unpinned": null,284 "visible": true,285 "closed": false,286 "archived": false,287 "bookmarked": null,288 "liked": null,289 "tags_descriptions": {},290 "like_count": 1,291 "views": 194,292 "category_id": 37,293 "featured_link": null,294 "has_accepted_answer": false,295 "posters": [296 {297 "extras": "latest",298 "description": "Original Poster, Most Recent Poster",299 "user": {300 "id": 50549,301 "username": "jinooooo",302 "name": "Jino Rohit",303 "avatar_template": "/user_avatar/discuss.pytorch.org/jinooooo/{size}/72578_2.png",304 "trust_level": 1305 }306 },307 {308 "extras": null,309 "description": "Frequent Poster",310 "user": {311 "id": 3534,312 "username": "ptrblck",313 "name": "",314 "avatar_template": "/user_avatar/discuss.pytorch.org/ptrblck/{size}/1823_2.png",315 "admin": true,316 "moderator": true,317 "trust_level": 2318 }319 }320 ]321 },322 {323 "fancy_title": "Tensors cannot ‘to’ other devices right",324 "id": 214528,325 "title": "Tensors cannot 'to' other devices right",326 "slug": "tensors-cannot-to-other-devices-right",327 "posts_count": 2,328 "reply_count": 0,329 "highest_post_number": 2,330 "image_url": "https://discuss.pytorch.org/uploads/default/original/3X/5/c/5ce697bebf0ebdb4c0a38b272cc77f43604167e0.png",331 "created_at": "2024-12-22T09:09:59.804Z",332 "last_posted_at": "2024-12-22T14:51:38.294Z",333 "bumped": true,334 "bumped_at": "2024-12-22T14:51:38.294Z",335 "archetype": "regular",336 "unseen": false,337 "pinned": false,338 "unpinned": null,339 "visible": true,340 "closed": false,341 "archived": false,342 "bookmarked": null,343 "liked": null,344 "tags_descriptions": {},345 "like_count": 1,346 "views": 129,347 "category_id": 37,348 "featured_link": null,349 "has_accepted_answer": false,350 "posters": [351 {352 "extras": null,353 "description": "Original Poster",354 "user": {355 "id": 81656,356 "username": "yuexu0312",357 "name": "Yue Xu",358 "avatar_template": "/user_avatar/discuss.pytorch.org/yuexu0312/{size}/74676_2.png",359 "trust_level": 0360 }361 },362 {363 "extras": "latest",364 "description": "Most Recent Poster",365 "user": {366 "id": 3534,367 "username": "ptrblck",368 "name": "",369 "avatar_template": "/user_avatar/discuss.pytorch.org/ptrblck/{size}/1823_2.png",370 "admin": true,371 "moderator": true,372 "trust_level": 2373 }374 }375 ]376 },377 {378 "fancy_title": "How to make __getitem__ more efficient in loading images?",379 "id": 216591,380 "title": "How to make __getitem__ more efficient in loading images?",381 "slug": "how-to-make-getitem-more-efficient-in-loading-images",382 "posts_count": 4,383 "reply_count": 1,384 "highest_post_number": 4,385 "image_url": null,386 "created_at": "2025-02-12T21:25:42.053Z",387 "last_posted_at": "2025-02-14T08:12:22.246Z",388 "bumped": true,389 "bumped_at": "2025-02-14T08:12:22.246Z",390 "archetype": "regular",391 "unseen": false,392 "pinned": false,393 "unpinned": null,394 "visible": true,395 "closed": false,396 "archived": false,397 "bookmarked": null,398 "liked": null,399 "tags_descriptions": {},400 "like_count": 1,401 "views": 157,402 "category_id": 37,403 "featured_link": null,404 "has_accepted_answer": false,405 "posters": [406 {407 "extras": null,408 "description": "Original Poster",409 "user": {410 "id": 82649,411 "username": "onedeadmatch",412 "name": "dani",413 "avatar_template": "/user_avatar/discuss.pytorch.org/onedeadmatch/{size}/75627_2.png",414 "trust_level": 1415 }416 },417 {418 "extras": "latest",419 "description": "Most Recent Poster",420 "user": {421 "id": 9081,422 "username": "JuanFMontesinos",423 "name": "Juan Montesinos",424 "avatar_template": "/user_avatar/discuss.pytorch.org/juanfmontesinos/{size}/76115_2.png",425 "trust_level": 2426 }427 }428 ]429 },430 {431 "fancy_title": "Unexpected DataLoader hanging when torch.set_num_threads in worker init",432 "id": 219991,433 "title": "Unexpected DataLoader hanging when torch.set_num_threads in worker init",434 "slug": "unexpected-dataloader-hanging-when-torch-set-num-threads-in-worker-init",435 "posts_count": 1,436 "reply_count": 0,437 "highest_post_number": 1,438 "image_url": null,439 "created_at": "2025-05-13T13:58:40.791Z",440 "last_posted_at": "2025-05-13T13:58:40.835Z",441 "bumped": true,442 "bumped_at": "2025-05-13T13:58:40.835Z",443 "archetype": "regular",444 "unseen": false,445 "pinned": false,446 "unpinned": null,447 "visible": true,448 "closed": false,449 "archived": false,450 "bookmarked": null,451 "liked": null,452 "tags_descriptions": {},453 "like_count": 0,454 "views": 43,455 "category_id": 37,456 "featured_link": null,457 "has_accepted_answer": false,458 "posters": [459 {460 "extras": "latest single",461 "description": "Original Poster, Most Recent Poster",462 "user": {463 "id": 49289,464 "username": "Sohrab_Andaz",465 "name": "Sohrab Andaz",466 "avatar_template": "/user_avatar/discuss.pytorch.org/sohrab_andaz/{size}/42486_2.png",467 "trust_level": 1468 }469 }470 ]471 }472 ],473 "tags_descriptions": {},474 "fancy_title": "Different test results with a fixed weights and fixed seed",475 "id": 195330,476 "title": "Different test results with a fixed weights and fixed seed",477 "posts_count": 3,478 "created_at": "2024-01-12T17:37:20.941Z",479 "views": 297,480 "reply_count": 1,481 "like_count": 1,482 "last_posted_at": "2024-01-13T12:05:46.232Z",483 "visible": true,484 "closed": false,485 "archived": false,486 "has_summary": false,487 "archetype": "regular",488 "slug": "different-test-results-with-a-fixed-weights-and-fixed-seed",489 "category_id": 37,490 "word_count": 924,491 "deleted_at": null,492 "user_id": 67235,493 "featured_link": null,494 "pinned_globally": false,495 "pinned_at": null,496 "pinned_until": null,497 "image_url": null,498 "slow_mode_seconds": 0,499 "draft": null,500 "draft_key": "topic_195330",501 "draft_sequence": null,502 "unpinned": null,503 "pinned": false,504 "current_post_number": 1,505 "highest_post_number": 3,506 "deleted_by": null,507 "actions_summary": [508 {509 "id": 4,510 "count": 0,511 "hidden": false,512 "can_act": false513 },514 {515 "id": 8,516 "count": 0,517 "hidden": false,518 "can_act": false519 },520 {521 "id": 10,522 "count": 0,523 "hidden": false,524 "can_act": false525 },526 {527 "id": 7,528 "count": 0,529 "hidden": false,530 "can_act": false531 }532 ],533 "chunk_size": 20,534 "bookmarked": false,535 "topic_timer": null,536 "message_bus_last_id": 0,537 "participant_count": 2,538 "show_read_indicator": false,539 "thumbnails": null,540 "slow_mode_enabled_until": null,541 "can_vote": false,542 "vote_count": 0,543 "user_voted": false,544 "discourse_zendesk_plugin_zendesk_id": null,545 "discourse_zendesk_plugin_zendesk_url": "https://your-url.zendesk.com/agent/tickets/",546 "details": {547 "can_edit": false,548 "notification_level": 1,549 "participants": [550 {551 "id": 67235,552 "username": "Scolpe",553 "name": "Scolpe",554 "avatar_template": "/user_avatar/discuss.pytorch.org/scolpe/{size}/61541_2.png",555 "post_count": 2,556 "primary_group_name": null,557 "flair_name": null,558 "flair_url": null,559 "flair_color": null,560 "flair_bg_color": null,561 "flair_group_id": null,562 "trust_level": 1563 },564 {565 "id": 3534,566 "username": "ptrblck",567 "name": "",568 "avatar_template": "/user_avatar/discuss.pytorch.org/ptrblck/{size}/1823_2.png",569 "post_count": 1,570 "primary_group_name": null,571 "flair_name": null,572 "flair_url": null,573 "flair_color": null,574 "flair_bg_color": null,575 "flair_group_id": null,576 "admin": true,577 "moderator": true,578 "trust_level": 2579 }580 ],581 "created_by": {582 "id": 67235,583 "username": "Scolpe",584 "name": "Scolpe",585 "avatar_template": "/user_avatar/discuss.pytorch.org/scolpe/{size}/61541_2.png"586 },587 "last_poster": {588 "id": 67235,589 "username": "Scolpe",590 "name": "Scolpe",591 "avatar_template": "/user_avatar/discuss.pytorch.org/scolpe/{size}/61541_2.png"592 },593 "links": [594 {595 "url": "https://pytorch.org/docs/stable/notes/randomness.html",596 "title": "Reproducibility — PyTorch 2.1 documentation",597 "internal": false,598 "attachment": false,599 "reflection": false,600 "clicks": 4,601 "user_id": 3534,602 "domain": "pytorch.org",603 "root_domain": "pytorch.org"604 }605 ]606 },607 "bookmarks": []608 },609 {610 "post_stream": {611 "posts": [612 {613 "id": 429976,614 "name": "bashir",615 "username": "bash",616 "avatar_template": "/letter_avatar_proxy/v4/letter/b/439d5e/{size}.png",617 "created_at": "2024-01-12T14:40:20.562Z",618 "cooked": "<p>Hi please i need help i have a discriminator architecture of</p>\n<pre><code class=\"lang-auto\">class Discriminator(nn.Module):\n def __init__(self):\n super(Discriminator, self).__init__()\n\n self.main = nn.Sequential(\n nn.Conv2d(3, 16, kernel_size=3, stride=2, padding=1),\n nn.LeakyReLU(0.2),\n nn.Dropout(0.25),\n nn.Conv2d(16, 32, kernel_size=3, stride=2, padding=1),\n nn.ZeroPad2d((0, 1, 0, 1)),\n nn.BatchNorm2d(32, momentum=0.82),\n nn.LeakyReLU(0.25),\n nn.Dropout(0.25),\n nn.Conv2d(32, 64, kernel_size=3, stride=2, padding=1),\n nn.BatchNorm2d(64, momentum=0.82),\n nn.LeakyReLU(0.2),\n nn.Dropout(0.25),\n nn.Conv2d(64, 128, kernel_size=3, stride=2, padding=1),\n nn.BatchNorm2d(128, momentum=0.82),\n nn.LeakyReLU(0.25),\n nn.Dropout(0.25),\n nn.Conv2d(128, 256, kernel_size=3, stride=1, padding=1),\n nn.BatchNorm2d(256, momentum=0.8),\n nn.LeakyReLU(0.25),\n nn.Dropout(0.25),\n nn.Flatten(),\n nn.Linear(256 * 4 * 4, 1), \n #nn.Linear(32, 73984), \n nn.Sigmoid()\n )\n\n def forward(self, x):\n return self.main(x)\n</code></pre>\n<p>the nn.linear output is</p>\n<pre><code class=\"lang-auto\"> Linear(in_features=4096, out_features=1, bias=True)\n</code></pre>\n<p>i am trying to change the nn.linear to in_features=32, out_features=73984 from my architecture please help</p>",619 "post_number": 1,620 "post_type": 1,621 "posts_count": 5,622 "updated_at": "2024-01-12T14:40:20.562Z",623 "reply_count": 0,624 "reply_to_post_number": null,625 "quote_count": 0,626 "incoming_link_count": 20,627 "reads": 7,628 "readers_count": 6,629 "score": 101.4,630 "yours": false,631 "topic_id": 195323,632 "topic_slug": "runtimeerror-mat1-and-mat2-shapes-cannot-be-multiplied-32x73984-and-4096x1",633 "display_username": "bashir",634 "primary_group_name": null,635 "flair_name": null,636 "flair_url": null,637 "flair_bg_color": null,638 "flair_color": null,639 "flair_group_id": null,640 "badges_granted": [],641 "version": 1,642 "can_edit": false,643 "can_delete": false,644 "can_recover": false,645 "can_see_hidden_post": false,646 "can_wiki": false,647 "read": true,648 "user_title": null,649 "bookmarked": false,650 "actions_summary": [],651 "moderator": false,652 "admin": false,653 "staff": false,654 "user_id": 72395,655 "hidden": false,656 "trust_level": 1,657 "deleted_at": null,658 "user_deleted": false,659 "edit_reason": null,660 "can_view_edit_history": true,661 "wiki": false,662 "post_url": "/t/runtimeerror-mat1-and-mat2-shapes-cannot-be-multiplied-32x73984-and-4096x1/195323/1",663 "can_accept_answer": false,664 "can_unaccept_answer": false,665 "accepted_answer": false,666 "topic_accepted_answer": null,667 "can_vote": false668 },669 {670 "id": 429983,671 "name": "",672 "username": "smth",673 "avatar_template": "/user_avatar/discuss.pytorch.org/smth/{size}/13_2.png",674 "created_at": "2024-01-12T15:12:05.483Z",675 "cooked": "<p>change this line</p>\n<pre><code class=\"lang-auto\">nn.Linear(256 * 4 * 4, 1), \n</code></pre>\n<p>to:</p>\n<pre><code class=\"lang-auto\">nn.Linear(256 * 4 * 4, 32), \n</code></pre>\n<p>And then uncomment the line <code>#nn.Linear(32, 73984), </code></p>\n<p>That’s it, you’ll be all set</p>",676 "post_number": 2,677 "post_type": 1,678 "posts_count": 5,679 "updated_at": "2024-01-12T15:12:05.483Z",680 "reply_count": 1,681 "reply_to_post_number": null,682 "quote_count": 0,683 "incoming_link_count": 1,684 "reads": 7,685 "readers_count": 6,686 "score": 11.4,687 "yours": false,688 "topic_id": 195323,689 "topic_slug": "runtimeerror-mat1-and-mat2-shapes-cannot-be-multiplied-32x73984-and-4096x1",690 "display_username": "",691 "primary_group_name": null,692 "flair_name": null,693 "flair_url": null,694 "flair_bg_color": null,695 "flair_color": null,696 "flair_group_id": null,697 "badges_granted": [],698 "version": 1,699 "can_edit": false,700 "can_delete": false,701 "can_recover": false,702 "can_see_hidden_post": false,703 "can_wiki": false,704 "read": true,705 "user_title": "PyTorch Dev, Facebook AI Research",706 "title_is_group": false,707 "bookmarked": false,708 "actions_summary": [],709 "moderator": true,710 "admin": true,711 "staff": true,712 "user_id": 1,713 "hidden": false,714 "trust_level": 2,715 "deleted_at": null,716 "user_deleted": false,717 "edit_reason": null,718 "can_view_edit_history": true,719 "wiki": false,720 "post_url": "/t/runtimeerror-mat1-and-mat2-shapes-cannot-be-multiplied-32x73984-and-4096x1/195323/2",721 "can_accept_answer": false,722 "can_unaccept_answer": false,723 "accepted_answer": false,724 "topic_accepted_answer": null725 },726 {727 "id": 429997,728 "name": "bashir",729 "username": "bash",730 "avatar_template": "/letter_avatar_proxy/v4/letter/b/439d5e/{size}.png",731 "created_at": "2024-01-12T18:04:42.097Z",732 "cooked": "<p>yes i have done that but this is the error</p>\n<pre><code class=\"lang-auto\">RuntimeError: mat1 and mat2 shapes cannot be multiplied (32x278784 and 32x278784)\n\n</code></pre>",733 "post_number": 3,734 "post_type": 1,735 "posts_count": 5,736 "updated_at": "2024-01-12T18:04:42.097Z",737 "reply_count": 1,738 "reply_to_post_number": 2,739 "quote_count": 0,740 "incoming_link_count": 1,741 "reads": 6,742 "readers_count": 5,743 "score": 11.2,744 "yours": false,745 "topic_id": 195323,746 "topic_slug": "runtimeerror-mat1-and-mat2-shapes-cannot-be-multiplied-32x73984-and-4096x1",747 "display_username": "bashir",748 "primary_group_name": null,749 "flair_name": null,750 "flair_url": null,751 "flair_bg_color": null,752 "flair_color": null,753 "flair_group_id": null,754 "badges_granted": [],755 "version": 1,756 "can_edit": false,757 "can_delete": false,758 "can_recover": false,759 "can_see_hidden_post": false,760 "can_wiki": false,761 "read": true,762 "user_title": null,763 "reply_to_user": {764 "id": 1,765 "username": "smth",766 "name": "",767 "avatar_template": "/user_avatar/discuss.pytorch.org/smth/{size}/13_2.png"768 },769 "bookmarked": false,770 "actions_summary": [],771 "moderator": false,772 "admin": false,773 "staff": false,774 "user_id": 72395,775 "hidden": false,776 "trust_level": 1,777 "deleted_at": null,778 "user_deleted": false,779 "edit_reason": null,780 "can_view_edit_history": true,781 "wiki": false,782 "post_url": "/t/runtimeerror-mat1-and-mat2-shapes-cannot-be-multiplied-32x73984-and-4096x1/195323/3",783 "can_accept_answer": false,784 "can_unaccept_answer": false,785 "accepted_answer": false,786 "topic_accepted_answer": null787 },788 {789 "id": 430001,790 "name": "",791 "username": "smth",792 "avatar_template": "/user_avatar/discuss.pytorch.org/smth/{size}/13_2.png",793 "created_at": "2024-01-12T18:57:45.355Z",794 "cooked": "<p>you must’ve not done what I mentioned. that error message would come if you put the numbers in the wrong order.</p>",795 "post_number": 4,796 "post_type": 1,797 "posts_count": 5,798 "updated_at": "2024-01-12T18:57:45.355Z",799 "reply_count": 0,800 "reply_to_post_number": 3,801 "quote_count": 0,802 "incoming_link_count": 0,803 "reads": 5,804 "readers_count": 4,805 "score": 1.0,806 "yours": false,807 "topic_id": 195323,808 "topic_slug": "runtimeerror-mat1-and-mat2-shapes-cannot-be-multiplied-32x73984-and-4096x1",809 "display_username": "",810 "primary_group_name": null,811 "flair_name": null,812 "flair_url": null,813 "flair_bg_color": null,814 "flair_color": null,815 "flair_group_id": null,816 "badges_granted": [],817 "version": 1,818 "can_edit": false,819 "can_delete": false,820 "can_recover": false,821 "can_see_hidden_post": false,822 "can_wiki": false,823 "read": true,824 "user_title": "PyTorch Dev, Facebook AI Research",825 "title_is_group": false,826 "reply_to_user": {827 "id": 72395,828 "username": "bash",829 "name": "bashir",830 "avatar_template": "/letter_avatar_proxy/v4/letter/b/439d5e/{size}.png"831 },832 "bookmarked": false,833 "actions_summary": [],834 "moderator": true,835 "admin": true,836 "staff": true,837 "user_id": 1,838 "hidden": false,839 "trust_level": 2,840 "deleted_at": null,841 "user_deleted": false,842 "edit_reason": null,843 "can_view_edit_history": true,844 "wiki": false,845 "post_url": "/t/runtimeerror-mat1-and-mat2-shapes-cannot-be-multiplied-32x73984-and-4096x1/195323/4",846 "can_accept_answer": false,847 "can_unaccept_answer": false,848 "accepted_answer": false,849 "topic_accepted_answer": null850 },851 {852 "id": 430072,853 "name": "bashir",854 "username": "bash",855 "avatar_template": "/letter_avatar_proxy/v4/letter/b/439d5e/{size}.png",856 "created_at": "2024-01-13T10:10:28.189Z",857 "cooked": "<p>yes i understand what you are saying but it will give the same error</p>\n<pre><code class=\"lang-auto\">RuntimeError: mat1 and mat2 shapes cannot be multiplied (32x73984 and 4096x3)\n</code></pre>\n<pre><code class=\"lang-auto\">(16): Conv2d(128, 256, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1))\n (17): BatchNorm2d(256, eps=1e-05, momentum=0.8, affine=True, track_running_stats=True)\n (18): LeakyReLU(negative_slope=0.25)\n (19): Dropout(p=0.25, inplace=False)\n (20): Flatten(start_dim=1, end_dim=-1)\n (21): Linear(in_features=4096, out_features=3, bias=True)\n (22): Linear(in_features=32, out_features=73984, bias=True)\n (23): Sigmoid()\n )\n)\n</code></pre>",858 "post_number": 5,859 "post_type": 1,860 "posts_count": 5,861 "updated_at": "2024-01-13T10:10:28.189Z",862 "reply_count": 0,863 "reply_to_post_number": null,864 "quote_count": 0,865 "incoming_link_count": 0,866 "reads": 3,867 "readers_count": 2,868 "score": 0.6,869 "yours": false,870 "topic_id": 195323,871 "topic_slug": "runtimeerror-mat1-and-mat2-shapes-cannot-be-multiplied-32x73984-and-4096x1",872 "display_username": "bashir",873 "primary_group_name": null,874 "flair_name": null,875 "flair_url": null,876 "flair_bg_color": null,877 "flair_color": null,878 "flair_group_id": null,879 "badges_granted": [],880 "version": 1,881 "can_edit": false,882 "can_delete": false,883 "can_recover": false,884 "can_see_hidden_post": false,885 "can_wiki": false,886 "read": true,887 "user_title": null,888 "bookmarked": false,889 "actions_summary": [],890 "moderator": false,891 "admin": false,892 "staff": false,893 "user_id": 72395,894 "hidden": false,895 "trust_level": 1,896 "deleted_at": null,897 "user_deleted": false,898 "edit_reason": null,899 "can_view_edit_history": true,900 "wiki": false,901 "post_url": "/t/runtimeerror-mat1-and-mat2-shapes-cannot-be-multiplied-32x73984-and-4096x1/195323/5",902 "can_accept_answer": false,903 "can_unaccept_answer": false,904 "accepted_answer": false,905 "topic_accepted_answer": null906 }907 ],908 "stream": [909 429976,910 429983,911 429997,912 430001,913 430072914 ]915 },916 "timeline_lookup": [917 [918 1,919 652920 ],921 [922 5,923 651924 ]925 ],926 "suggested_topics": [927 {928 "fancy_title": "Error: ‘PyArray_Descr’ has no member named ‘elsize’",929 "id": 213726,930 "title": "Error: ‘PyArray_Descr’ has no member named ‘elsize’",931 "slug": "error-pyarray-descr-has-no-member-named-elsize",932 "posts_count": 3,933 "reply_count": 1,934 "highest_post_number": 3,935 "image_url": null,936 "created_at": "2024-12-03T04:22:38.936Z",937 "last_posted_at": "2024-12-05T15:39:22.289Z",938 "bumped": true,939 "bumped_at": "2024-12-05T15:39:22.289Z",940 "archetype": "regular",941 "unseen": false,942 "pinned": false,943 "unpinned": null,944 "visible": true,945 "closed": false,946 "archived": false,947 "bookmarked": null,948 "liked": null,949 "tags_descriptions": {},950 "like_count": 0,951 "views": 1250,952 "category_id": 1,953 "featured_link": null,954 "has_accepted_answer": false,955 "posters": [956 {957 "extras": "latest",958 "description": "Original Poster, Most Recent Poster",959 "user": {960 "id": 81262,961 "username": "tuler",962 "name": "",963 "avatar_template": "/user_avatar/discuss.pytorch.org/tuler/{size}/74315_2.png",964 "trust_level": 0965 }966 },967 {968 "extras": null,969 "description": "Frequent Poster",970 "user": {971 "id": 3534,972 "username": "ptrblck",973 "name": "",974 "avatar_template": "/user_avatar/discuss.pytorch.org/ptrblck/{size}/1823_2.png",975 "admin": true,976 "moderator": true,977 "trust_level": 2978 }979 }980 ]981 },982 {983 "fancy_title": "Is my code correct for calculating score during validation?",984 "id": 213685,985 "title": "Is my code correct for calculating score during validation?",986 "slug": "is-my-code-correct-for-calculating-score-during-validation",987 "posts_count": 2,988 "reply_count": 0,989 "highest_post_number": 2,990 "image_url": null,991 "created_at": "2024-12-02T08:46:38.230Z",992 "last_posted_at": "2024-12-02T08:47:20.551Z",993 "bumped": true,994 "bumped_at": "2024-12-02T08:47:20.551Z",995 "archetype": "regular",996 "unseen": false,997 "pinned": false,998 "unpinned": null,999 "visible": true,1000 "closed": false,1001 "archived": false,1002 "bookmarked": null,1003 "liked": null,1004 "tags_descriptions": {},1005 "like_count": 0,1006 "views": 34,1007 "category_id": 1,1008 "featured_link": null,1009 "has_accepted_answer": false,1010 "posters": [1011 {1012 "extras": "latest single",1013 "description": "Original Poster, Most Recent Poster",1014 "user": {1015 "id": 75241,1016 "username": "jawi289p",1017 "name": "",1018 "avatar_template": "/user_avatar/discuss.pytorch.org/jawi289p/{size}/61673_2.png",1019 "trust_level": 11020 }1021 }1022 ]1023 },1024 {1025 "fancy_title": "Need advice on how to modify the GRU to made it work with raw vibration data",1026 "id": 216261,1027 "title": "Need advice on how to modify the GRU to made it work with raw vibration data",1028 "slug": "need-advice-on-how-to-modify-the-gru-to-made-it-work-with-raw-vibration-data",1029 "posts_count": 4,1030 "reply_count": 1,1031 "highest_post_number": 4,1032 "image_url": null,1033 "created_at": "2025-02-05T09:42:18.194Z",1034 "last_posted_at": "2025-02-06T06:48:34.494Z",1035 "bumped": true,1036 "bumped_at": "2025-02-06T06:48:34.494Z",1037 "archetype": "regular",1038 "unseen": false,1039 "pinned": false,1040 "unpinned": null,1041 "visible": true,1042 "closed": false,1043 "archived": false,1044 "bookmarked": null,1045 "liked": null,1046 "tags_descriptions": {},1047 "like_count": 1,1048 "views": 58,1049 "category_id": 1,1050 "featured_link": null,1051 "has_accepted_answer": false,1052 "posters": [1053 {1054 "extras": null,1055 "description": "Original Poster",1056 "user": {1057 "id": 67255,1058 "username": "Mystify2823",1059 "name": "",1060 "avatar_template": "/user_avatar/discuss.pytorch.org/mystify2823/{size}/61564_2.png",1061 "trust_level": 11062 }1063 },1064 {1065 "extras": "latest",1066 "description": "Most Recent Poster",1067 "user": {1068 "id": 19553,1069 "username": "anantguptadbl",1070 "name": "Anant Gupta",1071 "avatar_template": "/user_avatar/discuss.pytorch.org/anantguptadbl/{size}/17784_2.png",1072 "trust_level": 21073 }1074 }1075 ]1076 },1077 {1078 "fancy_title": "Why can the attribute “device” not be resolved within the namespace of torch?",1079 "id": 216720,1080 "title": "Why can the attribute \"device\" not be resolved within the namespace of torch?",1081 "slug": "why-can-the-attribute-device-not-be-resolved-within-the-namespace-of-torch",1082 "posts_count": 5,1083 "reply_count": 2,1084 "highest_post_number": 5,1085 "image_url": null,1086 "created_at": "2025-02-15T16:48:11.651Z",1087 "last_posted_at": "2025-02-16T15:14:44.858Z",1088 "bumped": true,1089 "bumped_at": "2025-02-16T15:14:44.858Z",1090 "archetype": "regular",1091 "unseen": false,1092 "pinned": false,1093 "unpinned": null,1094 "visible": true,1095 "closed": false,1096 "archived": false,1097 "bookmarked": null,1098 "liked": null,1099 "tags_descriptions": {},1100 "like_count": 0,1101 "views": 1682,1102 "category_id": 1,1103 "featured_link": null,1104 "has_accepted_answer": false,1105 "posters": [1106 {1107 "extras": null,1108 "description": "Original Poster",1109 "user": {1110 "id": 82705,1111 "username": "make1234",1112 "name": "",1113 "avatar_template": "/user_avatar/discuss.pytorch.org/make1234/{size}/74826_2.png",1114 "trust_level": 11115 }1116 },1117 {1118 "extras": null,1119 "description": "Frequent Poster",1120 "user": {1121 "id": 82720,1122 "username": "classic-21",1123 "name": "Priyanshu Singh",1124 "avatar_template": "/user_avatar/discuss.pytorch.org/classic-21/{size}/75688_2.png",1125 "trust_level": 01126 }1127 },1128 {1129 "extras": "latest",1130 "description": "Most Recent Poster",1131 "user": {1132 "id": 3534,1133 "username": "ptrblck",1134 "name": "",1135 "avatar_template": "/user_avatar/discuss.pytorch.org/ptrblck/{size}/1823_2.png",1136 "admin": true,1137 "moderator": true,1138 "trust_level": 21139 }1140 }1141 ]1142 },1143 {1144 "fancy_title": "Weight becomes nan after step but grad is normal",1145 "id": 220179,1146 "title": "Weight becomes nan after step but grad is normal",1147 "slug": "weight-becomes-nan-after-step-but-grad-is-normal",1148 "posts_count": 11,1149 "reply_count": 7,1150 "highest_post_number": 11,1151 "image_url": null,1152 "created_at": "2025-05-20T03:46:25.169Z",1153 "last_posted_at": "2025-07-06T18:25:46.514Z",1154 "bumped": true,1155 "bumped_at": "2025-07-06T18:25:46.514Z",1156 "archetype": "regular",1157 "unseen": false,1158 "pinned": false,1159 "unpinned": null,1160 "visible": true,1161 "closed": false,1162 "archived": false,1163 "bookmarked": null,1164 "liked": null,1165 "tags_descriptions": {},1166 "like_count": 7,1167 "views": 202,1168 "category_id": 1,1169 "featured_link": null,1170 "has_accepted_answer": true,1171 "posters": [1172 {1173 "extras": "latest",1174 "description": "Original Poster, Most Recent Poster",1175 "user": {1176 "id": 6296,1177 "username": "pytorcher",1178 "name": "",1179 "avatar_template": "/letter_avatar_proxy/v4/letter/p/7ea924/{size}.png",1180 "trust_level": 11181 }1182 },1183 {1184 "extras": null,1185 "description": "Frequent Poster, Accepted Answer",1186 "user": {1187 "id": 41458,1188 "username": "J_Johnson",1189 "name": "J Johnson",1190 "avatar_template": "/user_avatar/discuss.pytorch.org/j_johnson/{size}/55494_2.png",1191 "trust_level": 21192 }1193 },1194 {1195 "extras": null,1196 "description": "Frequent Poster",1197 "user": {1198 "id": 79061,1199 "username": "Ori_Yarden",1200 "name": "Ori Yarden, PhD",