Anurag1734/cuda-error-resolution-analysis
07
1[2 {3 "post_stream": {4 "posts": [5 {6 "id": 349321,7 "name": "Jacob Lambert",8 "username": "Jacob_Lambert",9 "avatar_template": "/user_avatar/discuss.pytorch.org/jacob_lambert/{size}/48480_2.png",10 "created_at": "2022-06-01T08:33:06.512Z",11 "cooked": "<p>Dear PyTorch community,</p>\n<p>I’m facing a rather weird problem I’m not sure how to debug anymore - my only guess is a hardware related issue.</p>\n<p>I’ve been successfully training a not-so-complex CNN on a 1080Ti (<a href=\"https://github.com/maudzung/Complex-YOLOv4-Pytorch\" rel=\"noopener nofollow ugc\">complex-yolo</a>), trying to improve it.<br>\nOn a new PC with a 2080Ti, I got crashes in training. After a lot of debugging, I noticed my model suddenly blows up (outputing Infs and NaNs) early in training for no apparent reason - there were no issues with the input data, or targets. Reverting code to states I <em>know</em> trained properly, lowering learning rate, simplifying things, nothing helped. I ran with a fixed seed and deterministic mode to debug the issue.</p>\n<p>In desperation I swapped out the 2080Ti for the old 1080Ti, ran the code without any other changes and could successfully train. I’ve had no other issues with the 2080Ti. I can do inference on trained models without issues, I can resume training.</p>\n<p>I’m not even sure what info is helpful for debugging this, or where to start. I used the exact same system with both cards:</p>\n<ul>\n<li>Nvidia Driver Version: 515.43.04</li>\n<li>System CUDA 11.4,</li>\n<li>Python Environment:\n<ul>\n<li>pytorch=1.5.0=py3.6_cuda10.2.89_cudnn7.6.5_0</li>\n<li>cudatoolkit=10.2.89=hfd86e86_1</li>\n</ul>\n</li>\n</ul>\n<p>What can I look into to figure out if there’s something wrong with my hardware - or if there’s some code compatibility issue that leads divergence in training?</p>\n<p>Thank you.</p>",12 "post_number": 1,13 "post_type": 1,14 "posts_count": 2,15 "updated_at": "2022-06-01T08:33:06.512Z",16 "reply_count": 0,17 "reply_to_post_number": null,18 "quote_count": 0,19 "incoming_link_count": 35,20 "reads": 8,21 "readers_count": 7,22 "score": 176.6,23 "yours": false,24 "topic_id": 153074,25 "topic_slug": "model-trains-on-1080ti-but-quickly-diverges-on-2080ti",26 "display_username": "Jacob Lambert",27 "primary_group_name": null,28 "flair_name": null,29 "flair_url": null,30 "flair_bg_color": null,31 "flair_color": null,32 "flair_group_id": null,33 "badges_granted": [],34 "version": 1,35 "can_edit": false,36 "can_delete": false,37 "can_recover": false,38 "can_see_hidden_post": false,39 "can_wiki": false,40 "link_counts": [41 {42 "url": "https://github.com/maudzung/Complex-YOLOv4-Pytorch",43 "internal": false,44 "reflection": false,45 "title": "GitHub - maudzung/Complex-YOLOv4-Pytorch: The PyTorch Implementation based on YOLOv4 of the paper: \"Complex-YOLO: Real-time 3D Object Detection on Point Clouds\"",46 "clicks": 247 }48 ],49 "read": true,50 "user_title": null,51 "bookmarked": false,52 "actions_summary": [],53 "moderator": false,54 "admin": false,55 "staff": false,56 "user_id": 56423,57 "hidden": false,58 "trust_level": 0,59 "deleted_at": null,60 "user_deleted": false,61 "edit_reason": null,62 "can_view_edit_history": true,63 "wiki": false,64 "post_url": "/t/model-trains-on-1080ti-but-quickly-diverges-on-2080ti/153074/1",65 "can_accept_answer": false,66 "can_unaccept_answer": false,67 "accepted_answer": false,68 "topic_accepted_answer": null,69 "can_vote": false70 },71 {72 "id": 349393,73 "name": "",74 "username": "eqy",75 "avatar_template": "/user_avatar/discuss.pytorch.org/eqy/{size}/46233_2.png",76 "created_at": "2022-06-01T16:49:31.232Z",77 "cooked": "<p>This certainly seems unexpected—I would check if this is still visible on a more current version of PyTorch (e.g., >= 1.11) on the off chance that it’s a bug that has been already fixed. If it’s still visible I would see if there’s a specific layer or part of the model where the outputs diverge substantially.</p>",78 "post_number": 2,79 "post_type": 1,80 "posts_count": 2,81 "updated_at": "2022-06-01T16:49:31.232Z",82 "reply_count": 0,83 "reply_to_post_number": null,84 "quote_count": 0,85 "incoming_link_count": 0,86 "reads": 7,87 "readers_count": 6,88 "score": 1.4,89 "yours": false,90 "topic_id": 153074,91 "topic_slug": "model-trains-on-1080ti-but-quickly-diverges-on-2080ti",92 "display_username": "",93 "primary_group_name": null,94 "flair_name": null,95 "flair_url": null,96 "flair_bg_color": null,97 "flair_color": null,98 "flair_group_id": null,99 "badges_granted": [],100 "version": 1,101 "can_edit": false,102 "can_delete": false,103 "can_recover": false,104 "can_see_hidden_post": false,105 "can_wiki": false,106 "read": true,107 "user_title": "",108 "bookmarked": false,109 "actions_summary": [],110 "moderator": false,111 "admin": false,112 "staff": false,113 "user_id": 35275,114 "hidden": false,115 "trust_level": 2,116 "deleted_at": null,117 "user_deleted": false,118 "edit_reason": null,119 "can_view_edit_history": true,120 "wiki": false,121 "post_url": "/t/model-trains-on-1080ti-but-quickly-diverges-on-2080ti/153074/2",122 "can_accept_answer": false,123 "can_unaccept_answer": false,124 "accepted_answer": false,125 "topic_accepted_answer": null126 }127 ],128 "stream": [129 349321,130 349393131 ]132 },133 "timeline_lookup": [134 [135 1,136 1242137 ]138 ],139 "suggested_topics": [140 {141 "fancy_title": "Non blocking copy from CPU to GPU",142 "id": 213522,143 "title": "Non blocking copy from CPU to GPU",144 "slug": "non-blocking-copy-from-cpu-to-gpu",145 "posts_count": 2,146 "reply_count": 0,147 "highest_post_number": 2,148 "image_url": null,149 "created_at": "2024-11-27T13:17:50.447Z",150 "last_posted_at": "2024-11-28T09:42:41.192Z",151 "bumped": true,152 "bumped_at": "2024-11-28T09:42:41.192Z",153 "archetype": "regular",154 "unseen": false,155 "pinned": false,156 "unpinned": null,157 "visible": true,158 "closed": false,159 "archived": false,160 "bookmarked": null,161 "liked": null,162 "tags_descriptions": {},163 "like_count": 0,164 "views": 162,165 "category_id": 1,166 "featured_link": null,167 "has_accepted_answer": false,168 "posters": [169 {170 "extras": "latest single",171 "description": "Original Poster, Most Recent Poster",172 "user": {173 "id": 81162,174 "username": "shira",175 "name": "shira",176 "avatar_template": "/user_avatar/discuss.pytorch.org/shira/{size}/74225_2.png",177 "trust_level": 1178 }179 }180 ]181 },182 {183 "fancy_title": "How do I get gradients of a CNN one time only (without making it sticky)?",184 "id": 218244,185 "title": "How do I get gradients of a CNN one time only (without making it sticky)?",186 "slug": "how-do-i-get-gradients-of-a-cnn-one-time-only-without-making-it-sticky",187 "posts_count": 15,188 "reply_count": 13,189 "highest_post_number": 15,190 "image_url": null,191 "created_at": "2025-03-25T12:29:09.029Z",192 "last_posted_at": "2025-03-29T05:54:49.671Z",193 "bumped": true,194 "bumped_at": "2025-03-29T05:54:49.671Z",195 "archetype": "regular",196 "unseen": false,197 "pinned": false,198 "unpinned": null,199 "visible": true,200 "closed": false,201 "archived": false,202 "bookmarked": null,203 "liked": null,204 "tags_descriptions": {},205 "like_count": 5,206 "views": 107,207 "category_id": 1,208 "featured_link": null,209 "has_accepted_answer": true,210 "posters": [211 {212 "extras": null,213 "description": "Original Poster",214 "user": {215 "id": 31824,216 "username": "Prasad_Raghavendra",217 "name": "Prasad Raghavendra",218 "avatar_template": "/user_avatar/discuss.pytorch.org/prasad_raghavendra/{size}/24424_2.png",219 "trust_level": 2220 }221 },222 {223 "extras": "latest",224 "description": "Most Recent Poster, Accepted Answer",225 "user": {226 "id": 41458,227 "username": "J_Johnson",228 "name": "J Johnson",229 "avatar_template": "/user_avatar/discuss.pytorch.org/j_johnson/{size}/55494_2.png",230 "trust_level": 2231 }232 }233 ]234 },235 {236 "fancy_title": "How to (efficiently) apply a function without a “dim” argument to each row of a 2D tensor?",237 "id": 215365,238 "title": "How to (efficiently) apply a function without a \"dim\" argument to each row of a 2D tensor?",239 "slug": "how-to-efficiently-apply-a-function-without-a-dim-argument-to-each-row-of-a-2d-tensor",240 "posts_count": 1,241 "reply_count": 0,242 "highest_post_number": 1,243 "image_url": null,244 "created_at": "2025-01-14T10:42:36.374Z",245 "last_posted_at": "2025-01-14T10:42:36.448Z",246 "bumped": true,247 "bumped_at": "2025-01-14T10:42:36.448Z",248 "archetype": "regular",249 "unseen": false,250 "pinned": false,251 "unpinned": null,252 "visible": true,253 "closed": false,254 "archived": false,255 "bookmarked": null,256 "liked": null,257 "tags_descriptions": {},258 "like_count": 0,259 "views": 83,260 "category_id": 1,261 "featured_link": null,262 "has_accepted_answer": false,263 "posters": [264 {265 "extras": "latest single",266 "description": "Original Poster, Most Recent Poster",267 "user": {268 "id": 82080,269 "username": "Matt_T1",270 "name": "Matt T.",271 "avatar_template": "/user_avatar/discuss.pytorch.org/matt_t1/{size}/75097_2.png",272 "trust_level": 1273 }274 }275 ]276 },277 {278 "fancy_title": "Torch.linalg.lstsq: (Batch element 0): Argument 6 has illegal value",279 "id": 216565,280 "title": "Torch.linalg.lstsq: (Batch element 0): Argument 6 has illegal value",281 "slug": "torch-linalg-lstsq-batch-element-0-argument-6-has-illegal-value",282 "posts_count": 6,283 "reply_count": 4,284 "highest_post_number": 6,285 "image_url": null,286 "created_at": "2025-02-12T07:27:24.277Z",287 "last_posted_at": "2025-02-19T02:48:46.645Z",288 "bumped": true,289 "bumped_at": "2025-02-19T02:48:46.645Z",290 "archetype": "regular",291 "unseen": false,292 "pinned": false,293 "unpinned": null,294 "visible": true,295 "closed": false,296 "archived": false,297 "bookmarked": null,298 "liked": null,299 "tags_descriptions": {},300 "like_count": 0,301 "views": 118,302 "category_id": 1,303 "featured_link": null,304 "has_accepted_answer": false,305 "posters": [306 {307 "extras": null,308 "description": "Original Poster",309 "user": {310 "id": 82561,311 "username": "CarlosD1119",312 "name": "Deng Yibin",313 "avatar_template": "/user_avatar/discuss.pytorch.org/carlosd1119/{size}/75541_2.png",314 "trust_level": 0315 }316 },317 {318 "extras": "latest",319 "description": "Most Recent Poster",320 "user": {321 "id": 3534,322 "username": "ptrblck",323 "name": "",324 "avatar_template": "/user_avatar/discuss.pytorch.org/ptrblck/{size}/1823_2.png",325 "admin": true,326 "moderator": true,327 "trust_level": 2328 }329 }330 ]331 },332 {333 "fancy_title": "How to Overcome EEG Data Overfitting in Deep Learning Models?",334 "id": 219704,335 "title": "How to Overcome EEG Data Overfitting in Deep Learning Models?",336 "slug": "how-to-overcome-eeg-data-overfitting-in-deep-learning-models",337 "posts_count": 1,338 "reply_count": 0,339 "highest_post_number": 1,340 "image_url": null,341 "created_at": "2025-05-02T20:45:48.057Z",342 "last_posted_at": "2025-05-02T20:45:48.090Z",343 "bumped": true,344 "bumped_at": "2025-05-02T20:45:48.090Z",345 "archetype": "regular",346 "unseen": false,347 "pinned": false,348 "unpinned": null,349 "visible": true,350 "closed": false,351 "archived": false,352 "bookmarked": null,353 "liked": null,354 "tags_descriptions": {},355 "like_count": 0,356 "views": 61,357 "category_id": 1,358 "featured_link": null,359 "has_accepted_answer": false,360 "posters": [361 {362 "extras": "latest single",363 "description": "Original Poster, Most Recent Poster",364 "user": {365 "id": 84126,366 "username": "xenablgn",367 "name": "",368 "avatar_template": "/letter_avatar_proxy/v4/letter/x/e0b2c6/{size}.png",369 "trust_level": 0370 }371 }372 ]373 }374 ],375 "tags_descriptions": {},376 "fancy_title": "Model trains on 1080Ti but quickly diverges on 2080Ti",377 "id": 153074,378 "title": "Model trains on 1080Ti but quickly diverges on 2080Ti",379 "posts_count": 2,380 "created_at": "2022-06-01T08:33:06.445Z",381 "views": 580,382 "reply_count": 0,383 "like_count": 0,384 "last_posted_at": "2022-06-01T16:49:31.232Z",385 "visible": true,386 "closed": false,387 "archived": false,388 "has_summary": false,389 "archetype": "regular",390 "slug": "model-trains-on-1080ti-but-quickly-diverges-on-2080ti",391 "category_id": 1,392 "word_count": 313,393 "deleted_at": null,394 "user_id": 56423,395 "featured_link": null,396 "pinned_globally": false,397 "pinned_at": null,398 "pinned_until": null,399 "image_url": null,400 "slow_mode_seconds": 0,401 "draft": null,402 "draft_key": "topic_153074",403 "draft_sequence": null,404 "unpinned": null,405 "pinned": false,406 "current_post_number": 1,407 "highest_post_number": 2,408 "deleted_by": null,409 "actions_summary": [410 {411 "id": 4,412 "count": 0,413 "hidden": false,414 "can_act": false415 },416 {417 "id": 8,418 "count": 0,419 "hidden": false,420 "can_act": false421 },422 {423 "id": 10,424 "count": 0,425 "hidden": false,426 "can_act": false427 },428 {429 "id": 7,430 "count": 0,431 "hidden": false,432 "can_act": false433 }434 ],435 "chunk_size": 20,436 "bookmarked": false,437 "topic_timer": null,438 "message_bus_last_id": 0,439 "participant_count": 2,440 "show_read_indicator": false,441 "thumbnails": null,442 "slow_mode_enabled_until": null,443 "can_vote": false,444 "vote_count": 0,445 "user_voted": false,446 "discourse_zendesk_plugin_zendesk_id": null,447 "discourse_zendesk_plugin_zendesk_url": "https://your-url.zendesk.com/agent/tickets/",448 "details": {449 "can_edit": false,450 "notification_level": 1,451 "participants": [452 {453 "id": 35275,454 "username": "eqy",455 "name": "",456 "avatar_template": "/user_avatar/discuss.pytorch.org/eqy/{size}/46233_2.png",457 "post_count": 1,458 "primary_group_name": null,459 "flair_name": null,460 "flair_url": null,461 "flair_color": null,462 "flair_bg_color": null,463 "flair_group_id": null,464 "trust_level": 2465 },466 {467 "id": 56423,468 "username": "Jacob_Lambert",469 "name": "Jacob Lambert",470 "avatar_template": "/user_avatar/discuss.pytorch.org/jacob_lambert/{size}/48480_2.png",471 "post_count": 1,472 "primary_group_name": null,473 "flair_name": null,474 "flair_url": null,475 "flair_color": null,476 "flair_bg_color": null,477 "flair_group_id": null,478 "trust_level": 0479 }480 ],481 "created_by": {482 "id": 56423,483 "username": "Jacob_Lambert",484 "name": "Jacob Lambert",485 "avatar_template": "/user_avatar/discuss.pytorch.org/jacob_lambert/{size}/48480_2.png"486 },487 "last_poster": {488 "id": 35275,489 "username": "eqy",490 "name": "",491 "avatar_template": "/user_avatar/discuss.pytorch.org/eqy/{size}/46233_2.png"492 },493 "links": [494 {495 "url": "https://github.com/maudzung/Complex-YOLOv4-Pytorch",496 "title": "GitHub - maudzung/Complex-YOLOv4-Pytorch: The PyTorch Implementation based on YOLOv4 of the paper: \"Complex-YOLO: Real-time 3D Object Detection on Point Clouds\"",497 "internal": false,498 "attachment": false,499 "reflection": false,500 "clicks": 2,501 "user_id": 56423,502 "domain": "github.com",503 "root_domain": "github.com"504 }505 ]506 },507 "bookmarks": []508 },509 {510 "post_stream": {511 "posts": [512 {513 "id": 349248,514 "name": "Megh Bhalerao",515 "username": "Megh_Bhalerao",516 "avatar_template": "/user_avatar/discuss.pytorch.org/megh_bhalerao/{size}/13547_2.png",517 "created_at": "2022-05-31T19:13:07.988Z",518 "cooked": "<p>Hi all,<br>\nI have a <code>conv2d</code> layer parameters of shape say <code>(x,y,z,w)</code>, and I have a binary mask of the same shape <code>(x,y,z,w)</code>, I want to set the <code>requires_grad</code> attribute to every <strong>single</strong> weight parameter in this <code>conv2d</code> layer, or to be more clear I want to set <code>x*y*z*w</code> <code>requires_grad</code> values, i.e. for every weight value inside this conv filter acoording to the binary mask given. At places which have 1 in the binary mask, the <code>requires_grad = True</code> and vice versa. Is there a neat way of doing this other than looping over the entire filter?<br>\nThanks for your help!<br>\n– Megh</p>",519 "post_number": 1,520 "post_type": 1,521 "posts_count": 2,522 "updated_at": "2022-05-31T19:13:07.988Z",523 "reply_count": 1,524 "reply_to_post_number": null,525 "quote_count": 0,526 "incoming_link_count": 154,527 "reads": 13,528 "readers_count": 12,529 "score": 777.6,530 "yours": false,531 "topic_id": 153040,532 "topic_slug": "set-requires-grad-according-to-a-binary-mask",533 "display_username": "Megh Bhalerao",534 "primary_group_name": null,535 "flair_name": null,536 "flair_url": null,537 "flair_bg_color": null,538 "flair_color": null,539 "flair_group_id": null,540 "badges_granted": [],541 "version": 1,542 "can_edit": false,543 "can_delete": false,544 "can_recover": false,545 "can_see_hidden_post": false,546 "can_wiki": false,547 "read": true,548 "user_title": null,549 "bookmarked": false,550 "actions_summary": [],551 "moderator": false,552 "admin": false,553 "staff": false,554 "user_id": 20077,555 "hidden": false,556 "trust_level": 2,557 "deleted_at": null,558 "user_deleted": false,559 "edit_reason": null,560 "can_view_edit_history": true,561 "wiki": false,562 "post_url": "/t/set-requires-grad-according-to-a-binary-mask/153040/1",563 "can_accept_answer": false,564 "can_unaccept_answer": false,565 "accepted_answer": false,566 "topic_accepted_answer": null,567 "can_vote": false568 },569 {570 "id": 349392,571 "name": "K. Frank",572 "username": "KFrank",573 "avatar_template": "/letter_avatar_proxy/v4/letter/k/ecb155/{size}.png",574 "created_at": "2022-06-01T16:39:30.080Z",575 "cooked": "<p>Hi Megh!</p>\n<aside class=\"quote no-group\" data-username=\"Megh_Bhalerao\" data-post=\"1\" data-topic=\"153040\" data-full=\"true\">\n<div class=\"title\">\n<div class=\"quote-controls\"></div>\n<img loading=\"lazy\" alt=\"\" width=\"24\" height=\"24\" src=\"https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/megh_bhalerao/48/13547_2.png\" class=\"avatar\"> Megh_Bhalerao:</div>\n<blockquote>\n<p>to be more clear I want to set <code>x*y*z*w</code> <code>requires_grad</code> values, i.e. for every weight value inside this conv filter acoording to the binary mask given.</p>\n</blockquote>\n</aside>\n<p>There is no way to do what you want – with or without using loops.</p>\n<p>The <code>requires_grad</code> property applies to the entire tensor, not to elements<br>\nof the tensor individually.</p>\n<p>If your use case for this is to “freeze” just certain elements of the <code>Conv2d</code><br>\n<code>weight</code>, that is, to only update some of the elements of <code>weight</code> when<br>\ntraining, the safest way is to store a copy of the elements prior to calling<br>\nthe update, and then restore them after the update. E.g., something like<br>\nthis:</p>\n<pre data-code-wrap=\"python\"><code class=\"lang-python\">with torch.no_grad():\n saved_weights = conv_layer.weight.clone()\n\n# run training loop\n...\n # inside of training loop\n opt.zero_grad()\n loss.backward()\n opt.step()\n with torch.no_grad():\n conv_layer.weight[mask.logical_not()] = saved_weights[mask.logical_not()]\n</code></pre>\n<p>Best.</p>\n<p>K. Frank</p>",576 "post_number": 2,577 "post_type": 1,578 "posts_count": 2,579 "updated_at": "2022-06-01T16:39:30.080Z",580 "reply_count": 0,581 "reply_to_post_number": null,582 "quote_count": 1,583 "incoming_link_count": 15,584 "reads": 12,585 "readers_count": 11,586 "score": 77.4,587 "yours": false,588 "topic_id": 153040,589 "topic_slug": "set-requires-grad-according-to-a-binary-mask",590 "display_username": "K. Frank",591 "primary_group_name": null,592 "flair_name": null,593 "flair_url": null,594 "flair_bg_color": null,595 "flair_color": null,596 "flair_group_id": null,597 "badges_granted": [],598 "version": 1,599 "can_edit": false,600 "can_delete": false,601 "can_recover": false,602 "can_see_hidden_post": false,603 "can_wiki": false,604 "link_counts": [605 {606 "url": "https://discuss.pytorch.org/t/how-to-stop-updating-the-parameters-of-a-part-of-a-layer-in-a-cnn-model-not-the-parameters-of-the-whole-layer/153173/2",607 "internal": true,608 "reflection": true,609 "title": "How to stop updating the parameters of a part of a layer in a CNN model (not the parameters of the whole layer)?",610 "clicks": 2611 }612 ],613 "read": true,614 "user_title": null,615 "bookmarked": false,616 "actions_summary": [],617 "moderator": false,618 "admin": false,619 "staff": false,620 "user_id": 18088,621 "hidden": false,622 "trust_level": 2,623 "deleted_at": null,624 "user_deleted": false,625 "edit_reason": null,626 "can_view_edit_history": true,627 "wiki": false,628 "post_url": "/t/set-requires-grad-according-to-a-binary-mask/153040/2",629 "can_accept_answer": false,630 "can_unaccept_answer": false,631 "accepted_answer": false,632 "topic_accepted_answer": null633 }634 ],635 "stream": [636 349248,637 349392638 ]639 },640 "timeline_lookup": [641 [642 1,643 1243644 ],645 [646 2,647 1242648 ]649 ],650 "suggested_topics": [651 {652 "fancy_title": "Torch.linalg.lstsq: (Batch element 0): Argument 6 has illegal value",653 "id": 216565,654 "title": "Torch.linalg.lstsq: (Batch element 0): Argument 6 has illegal value",655 "slug": "torch-linalg-lstsq-batch-element-0-argument-6-has-illegal-value",656 "posts_count": 6,657 "reply_count": 4,658 "highest_post_number": 6,659 "image_url": null,660 "created_at": "2025-02-12T07:27:24.277Z",661 "last_posted_at": "2025-02-19T02:48:46.645Z",662 "bumped": true,663 "bumped_at": "2025-02-19T02:48:46.645Z",664 "archetype": "regular",665 "unseen": false,666 "pinned": false,667 "unpinned": null,668 "visible": true,669 "closed": false,670 "archived": false,671 "bookmarked": null,672 "liked": null,673 "tags_descriptions": {},674 "like_count": 0,675 "views": 118,676 "category_id": 1,677 "featured_link": null,678 "has_accepted_answer": false,679 "posters": [680 {681 "extras": null,682 "description": "Original Poster",683 "user": {684 "id": 82561,685 "username": "CarlosD1119",686 "name": "Deng Yibin",687 "avatar_template": "/user_avatar/discuss.pytorch.org/carlosd1119/{size}/75541_2.png",688 "trust_level": 0689 }690 },691 {692 "extras": "latest",693 "description": "Most Recent Poster",694 "user": {695 "id": 3534,696 "username": "ptrblck",697 "name": "",698 "avatar_template": "/user_avatar/discuss.pytorch.org/ptrblck/{size}/1823_2.png",699 "admin": true,700 "moderator": true,701 "trust_level": 2702 }703 }704 ]705 },706 {707 "fancy_title": "H100 vs A100 Memory Usage Difference",708 "id": 214869,709 "title": "H100 vs A100 Memory Usage Difference",710 "slug": "h100-vs-a100-memory-usage-difference",711 "posts_count": 8,712 "reply_count": 5,713 "highest_post_number": 8,714 "image_url": null,715 "created_at": "2025-01-02T05:02:00.260Z",716 "last_posted_at": "2025-02-05T21:15:17.407Z",717 "bumped": true,718 "bumped_at": "2025-02-05T21:15:17.407Z",719 "archetype": "regular",720 "unseen": false,721 "pinned": false,722 "unpinned": null,723 "visible": true,724 "closed": false,725 "archived": false,726 "bookmarked": null,727 "liked": null,728 "tags_descriptions": {},729 "like_count": 1,730 "views": 722,731 "category_id": 1,732 "featured_link": null,733 "has_accepted_answer": false,734 "posters": [735 {736 "extras": null,737 "description": "Original Poster",738 "user": {739 "id": 75268,740 "username": "Ty4Reading",741 "name": "Ty",742 "avatar_template": "/user_avatar/discuss.pytorch.org/ty4reading/{size}/69516_2.png",743 "trust_level": 1744 }745 },746 {747 "extras": null,748 "description": "Frequent Poster",749 "user": {750 "id": 3534,751 "username": "ptrblck",752 "name": "",753 "avatar_template": "/user_avatar/discuss.pytorch.org/ptrblck/{size}/1823_2.png",754 "admin": true,755 "moderator": true,756 "trust_level": 2757 }758 },759 {760 "extras": null,761 "description": "Frequent Poster",762 "user": {763 "id": 41997,764 "username": "bdhirsh",765 "name": "Brian Hirsh",766 "avatar_template": "/user_avatar/discuss.pytorch.org/bdhirsh/{size}/74038_2.png",767 "trust_level": 2768 }769 },770 {771 "extras": "latest",772 "description": "Most Recent Poster",773 "user": {774 "id": 82492,775 "username": "ekomp",776 "name": "",777 "avatar_template": "/user_avatar/discuss.pytorch.org/ekomp/{size}/75513_2.png",778 "trust_level": 1779 }780 }781 ]782 },783 {784 "fancy_title": "Pytorch training and testing loop confusion",785 "id": 212847,786 "title": "Pytorch training and testing loop confusion",787 "slug": "pytorch-training-and-testing-loop-confusion",788 "posts_count": 4,789 "reply_count": 2,790 "highest_post_number": 4,791 "image_url": null,792 "created_at": "2024-11-12T07:17:35.145Z",793 "last_posted_at": "2024-12-01T17:35:34.374Z",794 "bumped": true,795 "bumped_at": "2024-12-01T17:35:34.374Z",796 "archetype": "regular",797 "unseen": false,798 "pinned": false,799 "unpinned": null,800 "visible": true,801 "closed": false,802 "archived": false,803 "bookmarked": null,804 "liked": null,805 "tags_descriptions": {},806 "like_count": 0,807 "views": 288,808 "category_id": 1,809 "featured_link": null,810 "has_accepted_answer": false,811 "posters": [812 {813 "extras": null,814 "description": "Original Poster",815 "user": {816 "id": 80845,817 "username": "Usman1",818 "name": null,819 "avatar_template": "/letter_avatar_proxy/v4/letter/u/c67d28/{size}.png",820 "trust_level": 0821 }822 },823 {824 "extras": "latest",825 "description": "Most Recent Poster",826 "user": {827 "id": 3534,828 "username": "ptrblck",829 "name": "",830 "avatar_template": "/user_avatar/discuss.pytorch.org/ptrblck/{size}/1823_2.png",831 "admin": true,832 "moderator": true,833 "trust_level": 2834 }835 }836 ]837 },838 {839 "fancy_title": "multiprocessing.get_context(‘spawn’).Pool creates too many processes",840 "id": 215951,841 "title": "multiprocessing.get_context('spawn').Pool creates too many processes",842 "slug": "multiprocessing-get-context-spawn-pool-creates-too-many-processes",843 "posts_count": 2,844 "reply_count": 0,845 "highest_post_number": 2,846 "image_url": null,847 "created_at": "2025-01-27T18:04:44.483Z",848 "last_posted_at": "2025-01-27T18:49:00.462Z",849 "bumped": true,850 "bumped_at": "2025-01-27T18:49:00.462Z",851 "archetype": "regular",852 "unseen": false,853 "pinned": false,854 "unpinned": null,855 "visible": true,856 "closed": false,857 "archived": false,858 "bookmarked": null,859 "liked": null,860 "tags_descriptions": {},861 "like_count": 0,862 "views": 200,863 "category_id": 1,864 "featured_link": null,865 "has_accepted_answer": true,866 "posters": [867 {868 "extras": "latest single",869 "description": "Original Poster, Most Recent Poster, Accepted Answer",870 "user": {871 "id": 53378,872 "username": "foshea",873 "name": "Finn",874 "avatar_template": "/letter_avatar_proxy/v4/letter/f/a6a055/{size}.png",875 "trust_level": 1876 }877 }878 ]879 },880 {881 "fancy_title": "Pruning by manipulating weight_mask",882 "id": 216953,883 "title": "Pruning by manipulating weight_mask",884 "slug": "pruning-by-manipulating-weight-mask",885 "posts_count": 2,886 "reply_count": 0,887 "highest_post_number": 2,888 "image_url": null,889 "created_at": "2025-02-20T14:21:28.443Z",890 "last_posted_at": "2025-03-15T22:52:11.716Z",891 "bumped": true,892 "bumped_at": "2025-03-15T22:52:11.716Z",893 "archetype": "regular",894 "unseen": false,895 "pinned": false,896 "unpinned": null,897 "visible": true,898 "closed": false,899 "archived": false,900 "bookmarked": null,901 "liked": null,902 "tags_descriptions": {},903 "like_count": 0,904 "views": 75,905 "category_id": 1,906 "featured_link": null,907 "has_accepted_answer": false,908 "posters": [909 {910 "extras": "latest single",911 "description": "Original Poster, Most Recent Poster",912 "user": {913 "id": 520,914 "username": "ndronen",915 "name": "Nicholas Dronen",916 "avatar_template": "/user_avatar/discuss.pytorch.org/ndronen/{size}/1575_2.png",917 "trust_level": 2918 }919 }920 ]921 }922 ],923 "tags_descriptions": {},924 "fancy_title": "Set requires_grad according to a binary mask",925 "id": 153040,926 "title": "Set requires_grad according to a binary mask",927 "posts_count": 2,928 "created_at": "2022-05-31T19:13:07.934Z",929 "views": 837,930 "reply_count": 0,931 "like_count": 0,932 "last_posted_at": "2022-06-01T16:39:30.080Z",933 "visible": true,934 "closed": false,935 "archived": false,936 "has_summary": false,937 "archetype": "regular",938 "slug": "set-requires-grad-according-to-a-binary-mask",939 "category_id": 1,940 "word_count": 272,941 "deleted_at": null,942 "user_id": 20077,943 "featured_link": null,944 "pinned_globally": false,945 "pinned_at": null,946 "pinned_until": null,947 "image_url": null,948 "slow_mode_seconds": 0,949 "draft": null,950 "draft_key": "topic_153040",951 "draft_sequence": null,952 "unpinned": null,953 "pinned": false,954 "current_post_number": 1,955 "highest_post_number": 2,956 "deleted_by": null,957 "actions_summary": [958 {959 "id": 4,960 "count": 0,961 "hidden": false,962 "can_act": false963 },964 {965 "id": 8,966 "count": 0,967 "hidden": false,968 "can_act": false969 },970 {971 "id": 10,972 "count": 0,973 "hidden": false,974 "can_act": false975 },976 {977 "id": 7,978 "count": 0,979 "hidden": false,980 "can_act": false981 }982 ],983 "chunk_size": 20,984 "bookmarked": false,985 "topic_timer": null,986 "message_bus_last_id": 0,987 "participant_count": 2,988 "show_read_indicator": false,989 "thumbnails": null,990 "slow_mode_enabled_until": null,991 "can_vote": false,992 "vote_count": 0,993 "user_voted": false,994 "discourse_zendesk_plugin_zendesk_id": null,995 "discourse_zendesk_plugin_zendesk_url": "https://your-url.zendesk.com/agent/tickets/",996 "details": {997 "can_edit": false,998 "notification_level": 1,999 "participants": [1000 {1001 "id": 18088,1002 "username": "KFrank",1003 "name": "K. Frank",1004 "avatar_template": "/letter_avatar_proxy/v4/letter/k/ecb155/{size}.png",1005 "post_count": 1,1006 "primary_group_name": null,1007 "flair_name": null,1008 "flair_url": null,1009 "flair_color": null,1010 "flair_bg_color": null,1011 "flair_group_id": null,1012 "trust_level": 21013 },1014 {1015 "id": 20077,1016 "username": "Megh_Bhalerao",1017 "name": "Megh Bhalerao",1018 "avatar_template": "/user_avatar/discuss.pytorch.org/megh_bhalerao/{size}/13547_2.png",1019 "post_count": 1,1020 "primary_group_name": null,1021 "flair_name": null,1022 "flair_url": null,1023 "flair_color": null,1024 "flair_bg_color": null,1025 "flair_group_id": null,1026 "trust_level": 21027 }1028 ],1029 "created_by": {1030 "id": 20077,1031 "username": "Megh_Bhalerao",1032 "name": "Megh Bhalerao",1033 "avatar_template": "/user_avatar/discuss.pytorch.org/megh_bhalerao/{size}/13547_2.png"1034 },1035 "last_poster": {1036 "id": 18088,1037 "username": "KFrank",1038 "name": "K. Frank",1039 "avatar_template": "/letter_avatar_proxy/v4/letter/k/ecb155/{size}.png"1040 },1041 "links": [1042 {1043 "url": "https://discuss.pytorch.org/t/how-to-stop-updating-the-parameters-of-a-part-of-a-layer-in-a-cnn-model-not-the-parameters-of-the-whole-layer/153173/2",1044 "title": "How to stop updating the parameters of a part of a layer in a CNN model (not the parameters of the whole layer)?",1045 "internal": true,1046 "attachment": false,1047 "reflection": true,1048 "clicks": 2,1049 "user_id": 18088,1050 "domain": "discuss.pytorch.org",1051 "root_domain": "pytorch.org"1052 }1053 ]1054 },1055 "bookmarks": []1056 },1057 {1058 "post_stream": {1059 "posts": [1060 {1061 "id": 348865,1062 "name": "Alex ",1063 "username": "80HighDef",1064 "avatar_template": "/user_avatar/discuss.pytorch.org/80highdef/{size}/50052_2.png",1065 "created_at": "2022-05-28T18:05:24.930Z",1066 "cooked": "<p>Hello,</p>\n<p>I am trying to write a Binary Classification for pairs of images taken one after the other from a video. My frames are in black and white and thus only have one channel. I’ve been using 2 Conv3d() layers with one channel to moderate success (~85% accuracy), however it occurred to me that I could just be using Conv2d() layers with multiple channels instead.</p>\n<p>What would be the advantages/disadvantages of using either or for this scenario? I’m thinking I should stick with Conv3d() layers due to the ability to run convolution in the third dimension, but as I’m rather new to this I don’t know how useful this is.</p>\n<p>Note, I did come across <a href=\"https://discuss.pytorch.org/t/video-frames-as-conv2d-channels-or-1-channel-conv3d/130810\">this thread</a> before making this post, but I was hoping someone could give a more in-depth answer and I wasn’t sure if I should bump the thread or not, thus I am making a new post.</p>\n<p>All help is appreciated, thanks in advance!</p>",1067 "post_number": 1,1068 "post_type": 1,1069 "posts_count": 4,1070 "updated_at": "2022-05-28T18:05:24.930Z",1071 "reply_count": 1,1072 "reply_to_post_number": null,1073 "quote_count": 0,1074 "incoming_link_count": 1173,1075 "reads": 15,1076 "readers_count": 14,1077 "score": 5843.0,1078 "yours": false,1079 "topic_id": 152821,1080 "topic_slug": "conv2d-with-multiple-channels-or-conv3d-with-1",1081 "display_username": "Alex ",1082 "primary_group_name": null,1083 "flair_name": null,1084 "flair_url": null,1085 "flair_bg_color": null,1086 "flair_color": null,1087 "flair_group_id": null,1088 "badges_granted": [],1089 "version": 1,1090 "can_edit": false,1091 "can_delete": false,1092 "can_recover": false,1093 "can_see_hidden_post": false,1094 "can_wiki": false,1095 "link_counts": [1096 {1097 "url": "https://discuss.pytorch.org/t/video-frames-as-conv2d-channels-or-1-channel-conv3d/130810",1098 "internal": true,1099 "reflection": false,1100 "title": "Video frames as conv2d channels or 1 channel conv3d",1101 "clicks": 121102 }1103 ],1104 "read": true,1105 "user_title": null,1106 "bookmarked": false,1107 "actions_summary": [],1108 "moderator": false,1109 "admin": false,1110 "staff": false,1111 "user_id": 56338,1112 "hidden": false,1113 "trust_level": 1,1114 "deleted_at": null,1115 "user_deleted": false,1116 "edit_reason": null,1117 "can_view_edit_history": true,1118 "wiki": false,1119 "post_url": "/t/conv2d-with-multiple-channels-or-conv3d-with-1/152821/1",1120 "can_accept_answer": false,1121 "can_unaccept_answer": false,1122 "accepted_answer": false,1123 "topic_accepted_answer": null,1124 "can_vote": false1125 },1126 {1127 "id": 348895,1128 "name": "K. Frank",1129 "username": "KFrank",1130 "avatar_template": "/letter_avatar_proxy/v4/letter/k/ecb155/{size}.png",1131 "created_at": "2022-05-29T05:31:39.581Z",1132 "cooked": "<p>Hi Alex!</p>\n<aside class=\"quote no-group\" data-username=\"80HighDef\" data-post=\"1\" data-topic=\"152821\" data-full=\"true\">\n<div class=\"title\">\n<div class=\"quote-controls\"></div>\n<img loading=\"lazy\" alt=\"\" width=\"24\" height=\"24\" src=\"https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/80highdef/48/50052_2.png\" class=\"avatar\"> 80HighDef:</div>\n<blockquote>\n<p>I’m thinking I should stick with Conv3d() layers due to the ability to run convolution in the third dimension</p>\n</blockquote>\n</aside>\n<p>If your use case is restricted to having just pairs of images as your<br>\nnetwork input (as compared to using more than two frames of the<br>\nvideo, say, 8 or 16), then using <code>Conv2d</code> with <code>in_channels = 2</code><br>\nand <code>kernel_size = (k, k)</code> and <code>Conv3d</code> with <code>in_channels = 1</code>,<br>\n<code>kernel_size = (2, k, k)</code>, and a tensor with a depth dimension of 2<br>\npassed in are essentially equivalent (assuming that <code>out_channels = 1</code>).</p>\n<p>With a depth dimension (and <code>kernel_size</code>) of 2 , you don’t really<br>\nhave anything to convolve over, so you get the same result as the<br>\n2-channel computation.</p>\n<p>Since they’re equivalent, in terms of the result, neither one gives you<br>\na better network. To my mind, <code>Conv2d</code> is stylistically better, because<br>\nit better fits the way I think about what is going on.</p>\n<aside class=\"quote no-group\">\n<blockquote>\n<p>I wasn’t sure if I should bump the thread or not, thus I am making a new post.</p>\n</blockquote>\n</aside>\n<p>A profound thanks to you for not resurrecting that zombie thread.<br>\nExhuming old threads from their rightful interment just adds to the<br>\nnoise.</p>\n<p>Best.</p>\n<p>K. Frank</p>",1133 "post_number": 2,1134 "post_type": 1,1135 "posts_count": 4,1136 "updated_at": "2022-05-29T05:31:39.581Z",1137 "reply_count": 1,1138 "reply_to_post_number": null,1139 "quote_count": 1,1140 "incoming_link_count": 11,1141 "reads": 14,1142 "readers_count": 13,1143 "score": 62.8,1144 "yours": false,1145 "topic_id": 152821,1146 "topic_slug": "conv2d-with-multiple-channels-or-conv3d-with-1",1147 "display_username": "K. Frank",1148 "primary_group_name": null,1149 "flair_name": null,1150 "flair_url": null,1151 "flair_bg_color": null,1152 "flair_color": null,1153 "flair_group_id": null,1154 "badges_granted": [],1155 "version": 1,1156 "can_edit": false,1157 "can_delete": false,1158 "can_recover": false,1159 "can_see_hidden_post": false,1160 "can_wiki": false,1161 "read": true,1162 "user_title": null,1163 "bookmarked": false,1164 "actions_summary": [],1165 "moderator": false,1166 "admin": false,1167 "staff": false,1168 "user_id": 18088,1169 "hidden": false,1170 "trust_level": 2,1171 "deleted_at": null,1172 "user_deleted": false,1173 "edit_reason": null,1174 "can_view_edit_history": true,1175 "wiki": false,1176 "post_url": "/t/conv2d-with-multiple-channels-or-conv3d-with-1/152821/2",1177 "can_accept_answer": false,1178 "can_unaccept_answer": false,1179 "accepted_answer": false,1180 "topic_accepted_answer": null1181 },1182 {1183 "id": 349225,1184 "name": "Alex ",1185 "username": "80HighDef",1186 "avatar_template": "/user_avatar/discuss.pytorch.org/80highdef/{size}/50052_2.png",1187 "created_at": "2022-05-31T16:42:35.483Z",1188 "cooked": "<p>Hi K. Frank,</p>\n<p>Thank you for your response!</p>\n<p>If the out channels were greater than 1 for both Conv2d and Conv3d, are they still equivalent?</p>\n<p>-Alex</p>",1189 "post_number": 3,1190 "post_type": 1,1191 "posts_count": 4,1192 "updated_at": "2022-05-31T16:42:35.483Z",1193 "reply_count": 1,1194 "reply_to_post_number": 2,1195 "quote_count": 0,1196 "incoming_link_count": 10,1197 "reads": 12,1198 "readers_count": 11,1199 "score": 57.4,1200 "yours": false,