Anurag1734/cuda-error-resolution-analysis
07
1[2 {3 "post_stream": {4 "posts": [5 {6 "id": 146102,7 "name": "Cagatay Yildiz",8 "username": "cagatayyildiz",9 "avatar_template": "/user_avatar/discuss.pytorch.org/cagatayyildiz/{size}/17682_2.png",10 "created_at": "2019-11-13T20:40:34.298Z",11 "cooked": "<p>I’m having a segmentation fault error when I try to invert a 150x150 matrix, as simple as this:</p>\n<pre><code class=\"lang-auto\">J = torch.randn([20,150])\ntorch.inverse(J.t()@J)\n</code></pre>\n<p>I didn’t try to find out the exact threshold but for example, 120x120 inversion is fine.</p>\n<p>Also, I am having this issue on my macOS Sierra (10.12.6) with Python 3.7.3, Clang 4.0.1 and torch 1.3.0. My Ubuntu machine with Python 3.7.4, GCC 7.3.0 and 1.1.0 has no problem inverting even 3k-3k matrices.</p>\n<p>Any suggestions?<br>\nThanks!</p>",12 "post_number": 1,13 "post_type": 1,14 "posts_count": 4,15 "updated_at": "2019-11-13T20:40:34.298Z",16 "reply_count": 1,17 "reply_to_post_number": null,18 "quote_count": 0,19 "incoming_link_count": 177,20 "reads": 14,21 "readers_count": 13,22 "score": 892.8,23 "yours": false,24 "topic_id": 60899,25 "topic_slug": "segmentation-fault-when-matrix-inverted",26 "display_username": "Cagatay Yildiz",27 "primary_group_name": null,28 "flair_name": null,29 "flair_url": null,30 "flair_bg_color": null,31 "flair_color": null,32 "flair_group_id": null,33 "badges_granted": [],34 "version": 1,35 "can_edit": false,36 "can_delete": false,37 "can_recover": false,38 "can_see_hidden_post": false,39 "can_wiki": false,40 "read": true,41 "user_title": null,42 "bookmarked": false,43 "actions_summary": [],44 "moderator": false,45 "admin": false,46 "staff": false,47 "user_id": 24323,48 "hidden": false,49 "trust_level": 1,50 "deleted_at": null,51 "user_deleted": false,52 "edit_reason": null,53 "can_view_edit_history": true,54 "wiki": false,55 "post_url": "/t/segmentation-fault-when-matrix-inverted/60899/1",56 "can_accept_answer": false,57 "can_unaccept_answer": false,58 "accepted_answer": false,59 "topic_accepted_answer": true,60 "can_vote": false61 },62 {63 "id": 146106,64 "name": "Alban D",65 "username": "albanD",66 "avatar_template": "/user_avatar/discuss.pytorch.org/alband/{size}/215_2.png",67 "created_at": "2019-11-13T20:47:53.402Z",68 "cooked": "<p>Hi,</p>\n<p>I cannot reproduce this locally with python 3.7 on macos.<br>\nDo you know where the segfaults happen exactly? (using gdb or similar tools)<br>\nAlso, do you see the same problem with <code>torch.inverse(J.t().clone()@J)</code>?<br>\nAlso how did you installed pytorch?</p>",69 "post_number": 2,70 "post_type": 1,71 "posts_count": 4,72 "updated_at": "2019-11-13T20:47:53.402Z",73 "reply_count": 1,74 "reply_to_post_number": null,75 "quote_count": 0,76 "incoming_link_count": 5,77 "reads": 13,78 "readers_count": 12,79 "score": 32.6,80 "yours": false,81 "topic_id": 60899,82 "topic_slug": "segmentation-fault-when-matrix-inverted",83 "display_username": "Alban D",84 "primary_group_name": null,85 "flair_name": null,86 "flair_url": null,87 "flair_bg_color": null,88 "flair_color": null,89 "flair_group_id": null,90 "badges_granted": [],91 "version": 1,92 "can_edit": false,93 "can_delete": false,94 "can_recover": false,95 "can_see_hidden_post": false,96 "can_wiki": false,97 "read": true,98 "user_title": "",99 "bookmarked": false,100 "actions_summary": [],101 "moderator": true,102 "admin": true,103 "staff": true,104 "user_id": 211,105 "hidden": false,106 "trust_level": 4,107 "deleted_at": null,108 "user_deleted": false,109 "edit_reason": null,110 "can_view_edit_history": true,111 "wiki": false,112 "post_url": "/t/segmentation-fault-when-matrix-inverted/60899/2",113 "can_accept_answer": false,114 "can_unaccept_answer": false,115 "accepted_answer": false,116 "topic_accepted_answer": true117 },118 {119 "id": 146143,120 "name": "Yaroslav Bulatov",121 "username": "Yaroslav_Bulatov",122 "avatar_template": "/user_avatar/discuss.pytorch.org/yaroslav_bulatov/{size}/7017_2.png",123 "created_at": "2019-11-14T02:13:24.691Z",124 "cooked": "<p>Singular matrices expose bugs in numalg algorithms…</p>\n<p>Wondering if your call is going through MKL under the hood. I’ve seen crashes in that library for singular matrices which <a href=\"https://software.intel.com/en-us/forums/intel-distribution-for-python/topic/628049\" rel=\"nofollow noopener\">get fixed</a> by setting OMP_NUM_THREADS=1</p>\n<p>Dump utility below could be useful to get repro – save the matrix before the crash. Then you can upload the file and wrap it into easy to run repro like <a href=\"https://github.com/pytorch/pytorch/issues/25978#issue-492018796\" rel=\"nofollow noopener\">here</a></p>\n<pre><code class=\"lang-auto\">def dump(result, fname):\n \"\"\"Save result to file. Load as np.genfromtxt(fname). \"\"\"\n result = to_numpy(result)\n if result.shape == (): # savetxt has problems with scalars\n result = np.expand_dims(result, 0)\n location = fname\n # special handling for integer datatypes\n if (\n result.dtype == np.uint8 or result.dtype == np.int8 or\n result.dtype == np.uint16 or result.dtype == np.int16 or\n result.dtype == np.uint32 or result.dtype == np.int32 or\n result.dtype == np.uint64 or result.dtype == np.int64\n ):\n np.savetxt(location, X=result, fmt=\"%d\", delimiter=',')\n else:\n np.savetxt(location, X=result, delimiter=',')\n print(\"Dumping to\", location)\n</code></pre>",125 "post_number": 3,126 "post_type": 1,127 "posts_count": 4,128 "updated_at": "2019-11-14T17:14:35.428Z",129 "reply_count": 0,130 "reply_to_post_number": null,131 "quote_count": 0,132 "incoming_link_count": 6,133 "reads": 9,134 "readers_count": 8,135 "score": 31.8,136 "yours": false,137 "topic_id": 60899,138 "topic_slug": "segmentation-fault-when-matrix-inverted",139 "display_username": "Yaroslav Bulatov",140 "primary_group_name": null,141 "flair_name": null,142 "flair_url": null,143 "flair_bg_color": null,144 "flair_color": null,145 "flair_group_id": null,146 "badges_granted": [],147 "version": 2,148 "can_edit": false,149 "can_delete": false,150 "can_recover": false,151 "can_see_hidden_post": false,152 "can_wiki": false,153 "link_counts": [154 {155 "url": "https://software.intel.com/en-us/forums/intel-distribution-for-python/topic/628049",156 "internal": false,157 "reflection": false,158 "clicks": 5159 },160 {161 "url": "https://github.com/pytorch/pytorch/issues/25978#issue-492018796",162 "internal": false,163 "reflection": false,164 "title": "Provide a way to select SVD algorithm in PyTorch? · Issue #25978 · pytorch/pytorch · GitHub",165 "clicks": 2166 }167 ],168 "read": true,169 "user_title": null,170 "bookmarked": false,171 "actions_summary": [],172 "moderator": false,173 "admin": false,174 "staff": false,175 "user_id": 11965,176 "hidden": false,177 "trust_level": 2,178 "deleted_at": null,179 "user_deleted": false,180 "edit_reason": null,181 "can_view_edit_history": true,182 "wiki": false,183 "post_url": "/t/segmentation-fault-when-matrix-inverted/60899/3",184 "can_accept_answer": false,185 "can_unaccept_answer": false,186 "accepted_answer": false,187 "topic_accepted_answer": true188 },189 {190 "id": 174715,191 "name": "Cagatay Yildiz",192 "username": "cagatayyildiz",193 "avatar_template": "/user_avatar/discuss.pytorch.org/cagatayyildiz/{size}/17682_2.png",194 "created_at": "2020-03-17T16:15:43.623Z",195 "cooked": "<p>I’m using Anaconda3 on my computer. I don’t really remember how I installed torch but I anyways uninstalled it. When I re-installed it via <code>pip</code>, the error persisted. I once again uninstalled and installed this time via <code>conda</code>. Now, everything seems fine.</p>\n<aside class=\"quote no-group\" data-username=\"cagatayyildiz\" data-post=\"1\" data-topic=\"60899\">\n<div class=\"title\">\n<div class=\"quote-controls\"></div>\n<img loading=\"lazy\" alt=\"\" width=\"24\" height=\"24\" src=\"https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/cagatayyildiz/48/17682_2.png\" class=\"avatar\"> cagatayyildiz:</div>\n<blockquote>\n<p>torch.inverse(J.t()<span class=\"mention\">@J</span>)</p>\n</blockquote>\n</aside>",196 "post_number": 4,197 "post_type": 1,198 "posts_count": 4,199 "updated_at": "2020-03-17T16:15:51.239Z",200 "reply_count": 0,201 "reply_to_post_number": 2,202 "quote_count": 1,203 "incoming_link_count": 4,204 "reads": 6,205 "readers_count": 5,206 "score": 21.2,207 "yours": false,208 "topic_id": 60899,209 "topic_slug": "segmentation-fault-when-matrix-inverted",210 "display_username": "Cagatay Yildiz",211 "primary_group_name": null,212 "flair_name": null,213 "flair_url": null,214 "flair_bg_color": null,215 "flair_color": null,216 "flair_group_id": null,217 "badges_granted": [],218 "version": 1,219 "can_edit": false,220 "can_delete": false,221 "can_recover": false,222 "can_see_hidden_post": false,223 "can_wiki": false,224 "read": true,225 "user_title": null,226 "reply_to_user": {227 "id": 211,228 "username": "albanD",229 "name": "Alban D",230 "avatar_template": "/user_avatar/discuss.pytorch.org/alband/{size}/215_2.png"231 },232 "bookmarked": false,233 "actions_summary": [],234 "moderator": false,235 "admin": false,236 "staff": false,237 "user_id": 24323,238 "hidden": false,239 "trust_level": 1,240 "deleted_at": null,241 "user_deleted": false,242 "edit_reason": null,243 "can_view_edit_history": true,244 "wiki": false,245 "post_url": "/t/segmentation-fault-when-matrix-inverted/60899/4",246 "can_accept_answer": false,247 "can_unaccept_answer": false,248 "accepted_answer": true,249 "topic_accepted_answer": true250 }251 ],252 "stream": [253 146102,254 146106,255 146143,256 174715257 ]258 },259 "timeline_lookup": [260 [261 1,262 2173263 ],264 [265 4,266 2048267 ]268 ],269 "suggested_topics": [270 {271 "fancy_title": "Re-initialise weights and biases to Pytorch default in k-fold cross-validation",272 "id": 220480,273 "title": "Re-initialise weights and biases to Pytorch default in k-fold cross-validation",274 "slug": "re-initialise-weights-and-biases-to-pytorch-default-in-k-fold-cross-validation",275 "posts_count": 3,276 "reply_count": 1,277 "highest_post_number": 3,278 "image_url": null,279 "created_at": "2025-05-31T10:40:31.657Z",280 "last_posted_at": "2025-06-01T08:10:53.073Z",281 "bumped": true,282 "bumped_at": "2025-06-01T08:10:53.073Z",283 "archetype": "regular",284 "unseen": false,285 "pinned": false,286 "unpinned": null,287 "visible": true,288 "closed": false,289 "archived": false,290 "bookmarked": null,291 "liked": null,292 "tags_descriptions": {},293 "like_count": 0,294 "views": 53,295 "category_id": 1,296 "featured_link": null,297 "has_accepted_answer": false,298 "posters": [299 {300 "extras": "latest",301 "description": "Original Poster, Most Recent Poster",302 "user": {303 "id": 73973,304 "username": "NZ1",305 "name": null,306 "avatar_template": "/letter_avatar_proxy/v4/letter/n/c57346/{size}.png",307 "trust_level": 0308 }309 },310 {311 "extras": null,312 "description": "Frequent Poster",313 "user": {314 "id": 3534,315 "username": "ptrblck",316 "name": "",317 "avatar_template": "/user_avatar/discuss.pytorch.org/ptrblck/{size}/1823_2.png",318 "admin": true,319 "moderator": true,320 "trust_level": 2321 }322 }323 ]324 },325 {326 "fancy_title": "OutOfMemoryError: CUDA out of memory with a gan ",327 "id": 214826,328 "title": "OutOfMemoryError: CUDA out of memory with a gan ",329 "slug": "outofmemoryerror-cuda-out-of-memory-with-a-gan",330 "posts_count": 2,331 "reply_count": 0,332 "highest_post_number": 2,333 "image_url": null,334 "created_at": "2024-12-31T15:16:59.381Z",335 "last_posted_at": "2024-12-31T16:26:51.608Z",336 "bumped": true,337 "bumped_at": "2024-12-31T16:26:51.608Z",338 "archetype": "regular",339 "unseen": false,340 "pinned": false,341 "unpinned": null,342 "visible": true,343 "closed": false,344 "archived": false,345 "bookmarked": null,346 "liked": null,347 "tags_descriptions": {},348 "like_count": 0,349 "views": 38,350 "category_id": 1,351 "featured_link": null,352 "has_accepted_answer": false,353 "posters": [354 {355 "extras": null,356 "description": "Original Poster",357 "user": {358 "id": 81807,359 "username": "Freeziey",360 "name": "Freeziey",361 "avatar_template": "/user_avatar/discuss.pytorch.org/freeziey/{size}/74834_2.png",362 "trust_level": 0363 }364 },365 {366 "extras": "latest",367 "description": "Most Recent Poster",368 "user": {369 "id": 41396,370 "username": "soulitzer",371 "name": "",372 "avatar_template": "/letter_avatar_proxy/v4/letter/s/839c29/{size}.png",373 "trust_level": 2374 }375 }376 ]377 },378 {379 "fancy_title": "Torch Execution Time Differences: 1.13.0 vs. 2.4.0",380 "id": 212711,381 "title": "Torch Execution Time Differences: 1.13.0 vs. 2.4.0",382 "slug": "torch-execution-time-differences-1-13-0-vs-2-4-0",383 "posts_count": 7,384 "reply_count": 3,385 "highest_post_number": 7,386 "image_url": null,387 "created_at": "2024-11-08T15:00:16.673Z",388 "last_posted_at": "2024-11-20T10:21:55.122Z",389 "bumped": true,390 "bumped_at": "2024-11-20T10:21:55.122Z",391 "archetype": "regular",392 "unseen": false,393 "pinned": false,394 "unpinned": null,395 "visible": true,396 "closed": false,397 "archived": false,398 "bookmarked": null,399 "liked": null,400 "tags_descriptions": {},401 "like_count": 0,402 "views": 390,403 "category_id": 1,404 "featured_link": null,405 "has_accepted_answer": false,406 "posters": [407 {408 "extras": "latest",409 "description": "Original Poster, Most Recent Poster",410 "user": {411 "id": 80773,412 "username": "nico1",413 "name": "Nico",414 "avatar_template": "/user_avatar/discuss.pytorch.org/nico1/{size}/73863_2.png",415 "trust_level": 1416 }417 },418 {419 "extras": null,420 "description": "Frequent Poster",421 "user": {422 "id": 80724,423 "username": "paulge",424 "name": "",425 "avatar_template": "/letter_avatar_proxy/v4/letter/p/82dd89/{size}.png",426 "trust_level": 2427 }428 },429 {430 "extras": null,431 "description": "Frequent Poster",432 "user": {433 "id": 3534,434 "username": "ptrblck",435 "name": "",436 "avatar_template": "/user_avatar/discuss.pytorch.org/ptrblck/{size}/1823_2.png",437 "admin": true,438 "moderator": true,439 "trust_level": 2440 }441 }442 ]443 },444 {445 "fancy_title": "Fuse adds in TorchScript",446 "id": 217999,447 "title": "Fuse adds in TorchScript",448 "slug": "fuse-adds-in-torchscript",449 "posts_count": 1,450 "reply_count": 0,451 "highest_post_number": 1,452 "image_url": null,453 "created_at": "2025-03-18T22:24:10.497Z",454 "last_posted_at": "2025-03-18T22:24:10.538Z",455 "bumped": true,456 "bumped_at": "2025-03-18T22:29:22.920Z",457 "archetype": "regular",458 "unseen": false,459 "pinned": false,460 "unpinned": null,461 "visible": true,462 "closed": false,463 "archived": false,464 "bookmarked": null,465 "liked": null,466 "tags_descriptions": {},467 "like_count": 0,468 "views": 37,469 "category_id": 1,470 "featured_link": null,471 "has_accepted_answer": false,472 "posters": [473 {474 "extras": "latest single",475 "description": "Original Poster, Most Recent Poster",476 "user": {477 "id": 83351,478 "username": "kamei",479 "name": "kamei",480 "avatar_template": "/user_avatar/discuss.pytorch.org/kamei/{size}/76232_2.png",481 "trust_level": 1482 }483 }484 ]485 },486 {487 "fancy_title": "Help with RTX 5090",488 "id": 220597,489 "title": "Help with RTX 5090",490 "slug": "help-with-rtx-5090",491 "posts_count": 12,492 "reply_count": 9,493 "highest_post_number": 12,494 "image_url": null,495 "created_at": "2025-06-05T22:58:21.679Z",496 "last_posted_at": "2025-08-14T07:44:59.319Z",497 "bumped": true,498 "bumped_at": "2025-08-14T07:44:59.319Z",499 "archetype": "regular",500 "unseen": false,501 "pinned": false,502 "unpinned": null,503 "visible": true,504 "closed": false,505 "archived": false,506 "bookmarked": null,507 "liked": null,508 "tags_descriptions": {},509 "like_count": 2,510 "views": 4351,511 "category_id": 1,512 "featured_link": null,513 "has_accepted_answer": false,514 "posters": [515 {516 "extras": null,517 "description": "Original Poster",518 "user": {519 "id": 84595,520 "username": "sladevlone",521 "name": "",522 "avatar_template": "/letter_avatar_proxy/v4/letter/s/a88e57/{size}.png",523 "trust_level": 1524 }525 },526 {527 "extras": null,528 "description": "Frequent Poster",529 "user": {530 "id": 84351,531 "username": "balazon",532 "name": "Balint Varga",533 "avatar_template": "/user_avatar/discuss.pytorch.org/balazon/{size}/77063_2.png",534 "trust_level": 0535 }536 },537 {538 "extras": null,539 "description": "Frequent Poster",540 "user": {541 "id": 3534,542 "username": "ptrblck",543 "name": "",544 "avatar_template": "/user_avatar/discuss.pytorch.org/ptrblck/{size}/1823_2.png",545 "admin": true,546 "moderator": true,547 "trust_level": 2548 }549 },550 {551 "extras": "latest",552 "description": "Most Recent Poster",553 "user": {554 "id": 85427,555 "username": "SW_R",556 "name": "SW R",557 "avatar_template": "/user_avatar/discuss.pytorch.org/sw_r/{size}/77921_2.png",558 "trust_level": 1559 }560 }561 ]562 }563 ],564 "tags_descriptions": {},565 "fancy_title": "Segmentation fault when matrix inverted",566 "id": 60899,567 "title": "Segmentation fault when matrix inverted",568 "posts_count": 4,569 "created_at": "2019-11-13T20:40:34.242Z",570 "views": 901,571 "reply_count": 1,572 "like_count": 0,573 "last_posted_at": "2020-03-17T16:15:43.623Z",574 "visible": true,575 "closed": false,576 "archived": false,577 "has_summary": false,578 "archetype": "regular",579 "slug": "segmentation-fault-when-matrix-inverted",580 "category_id": 1,581 "word_count": 385,582 "deleted_at": null,583 "user_id": 24323,584 "featured_link": null,585 "pinned_globally": false,586 "pinned_at": null,587 "pinned_until": null,588 "image_url": null,589 "slow_mode_seconds": 0,590 "draft": null,591 "draft_key": "topic_60899",592 "draft_sequence": null,593 "unpinned": null,594 "pinned": false,595 "current_post_number": 1,596 "highest_post_number": 4,597 "deleted_by": null,598 "actions_summary": [599 {600 "id": 4,601 "count": 0,602 "hidden": false,603 "can_act": false604 },605 {606 "id": 8,607 "count": 0,608 "hidden": false,609 "can_act": false610 },611 {612 "id": 10,613 "count": 0,614 "hidden": false,615 "can_act": false616 },617 {618 "id": 7,619 "count": 0,620 "hidden": false,621 "can_act": false622 }623 ],624 "chunk_size": 20,625 "bookmarked": false,626 "topic_timer": null,627 "message_bus_last_id": 0,628 "participant_count": 3,629 "show_read_indicator": false,630 "thumbnails": null,631 "slow_mode_enabled_until": null,632 "accepted_answer": {633 "post_number": 4,634 "username": "cagatayyildiz",635 "name": "Cagatay Yildiz",636 "excerpt": "I’m using Anaconda3 on my computer. I don’t really remember how I installed torch but I anyways uninstalled it. When I re-installed it via pip, the error persisted. I once again uninstalled and installed this time via conda. Now, everything seems fine."637 },638 "can_vote": false,639 "vote_count": 0,640 "user_voted": false,641 "discourse_zendesk_plugin_zendesk_id": null,642 "discourse_zendesk_plugin_zendesk_url": "https://your-url.zendesk.com/agent/tickets/",643 "details": {644 "can_edit": false,645 "notification_level": 1,646 "participants": [647 {648 "id": 24323,649 "username": "cagatayyildiz",650 "name": "Cagatay Yildiz",651 "avatar_template": "/user_avatar/discuss.pytorch.org/cagatayyildiz/{size}/17682_2.png",652 "post_count": 2,653 "primary_group_name": null,654 "flair_name": null,655 "flair_url": null,656 "flair_color": null,657 "flair_bg_color": null,658 "flair_group_id": null,659 "trust_level": 1660 },661 {662 "id": 211,663 "username": "albanD",664 "name": "Alban D",665 "avatar_template": "/user_avatar/discuss.pytorch.org/alband/{size}/215_2.png",666 "post_count": 1,667 "primary_group_name": null,668 "flair_name": null,669 "flair_url": null,670 "flair_color": null,671 "flair_bg_color": null,672 "flair_group_id": null,673 "admin": true,674 "moderator": true,675 "trust_level": 4676 },677 {678 "id": 11965,679 "username": "Yaroslav_Bulatov",680 "name": "Yaroslav Bulatov",681 "avatar_template": "/user_avatar/discuss.pytorch.org/yaroslav_bulatov/{size}/7017_2.png",682 "post_count": 1,683 "primary_group_name": null,684 "flair_name": null,685 "flair_url": null,686 "flair_color": null,687 "flair_bg_color": null,688 "flair_group_id": null,689 "trust_level": 2690 }691 ],692 "created_by": {693 "id": 24323,694 "username": "cagatayyildiz",695 "name": "Cagatay Yildiz",696 "avatar_template": "/user_avatar/discuss.pytorch.org/cagatayyildiz/{size}/17682_2.png"697 },698 "last_poster": {699 "id": 24323,700 "username": "cagatayyildiz",701 "name": "Cagatay Yildiz",702 "avatar_template": "/user_avatar/discuss.pytorch.org/cagatayyildiz/{size}/17682_2.png"703 },704 "links": [705 {706 "url": "https://software.intel.com/en-us/forums/intel-distribution-for-python/topic/628049",707 "title": null,708 "internal": false,709 "attachment": false,710 "reflection": false,711 "clicks": 5,712 "user_id": 11965,713 "domain": "software.intel.com",714 "root_domain": "intel.com"715 },716 {717 "url": "https://github.com/pytorch/pytorch/issues/25978#issue-492018796",718 "title": "Provide a way to select SVD algorithm in PyTorch? · Issue #25978 · pytorch/pytorch · GitHub",719 "internal": false,720 "attachment": false,721 "reflection": false,722 "clicks": 2,723 "user_id": 11965,724 "domain": "github.com",725 "root_domain": "github.com"726 }727 ]728 },729 "bookmarks": []730 },731 {732 "post_stream": {733 "posts": [734 {735 "id": 174714,736 "name": "Asma Sattar",737 "username": "AsmaAdil",738 "avatar_template": "/letter_avatar_proxy/v4/letter/a/53a042/{size}.png",739 "created_at": "2020-03-17T16:11:27.195Z",740 "cooked": "<p>what does this give? total number of nodes in batch or total number of graphs in batch?</p>",741 "post_number": 1,742 "post_type": 1,743 "posts_count": 1,744 "updated_at": "2020-03-17T16:11:27.195Z",745 "reply_count": 0,746 "reply_to_post_number": null,747 "quote_count": 0,748 "incoming_link_count": 54,749 "reads": 6,750 "readers_count": 5,751 "score": 271.2,752 "yours": false,753 "topic_id": 73539,754 "topic_slug": "batch-num-graph-in-pytorch-geometric",755 "display_username": "Asma Sattar",756 "primary_group_name": null,757 "flair_name": null,758 "flair_url": null,759 "flair_bg_color": null,760 "flair_color": null,761 "flair_group_id": null,762 "badges_granted": [],763 "version": 1,764 "can_edit": false,765 "can_delete": false,766 "can_recover": false,767 "can_see_hidden_post": false,768 "can_wiki": false,769 "read": true,770 "user_title": null,771 "bookmarked": false,772 "actions_summary": [],773 "moderator": false,774 "admin": false,775 "staff": false,776 "user_id": 28795,777 "hidden": false,778 "trust_level": 1,779 "deleted_at": null,780 "user_deleted": false,781 "edit_reason": null,782 "can_view_edit_history": true,783 "wiki": false,784 "post_url": "/t/batch-num-graph-in-pytorch-geometric/73539/1",785 "can_accept_answer": false,786 "can_unaccept_answer": false,787 "accepted_answer": false,788 "topic_accepted_answer": null,789 "can_vote": false790 }791 ],792 "stream": [793 174714794 ]795 },796 "timeline_lookup": [797 [798 1,799 2048800 ]801 ],802 "suggested_topics": [803 {804 "fancy_title": "Slurm: Torch not compiled with CUDA enabled",805 "id": 212928,806 "title": "Slurm: Torch not compiled with CUDA enabled",807 "slug": "slurm-torch-not-compiled-with-cuda-enabled",808 "posts_count": 5,809 "reply_count": 6,810 "highest_post_number": 8,811 "image_url": null,812 "created_at": "2024-11-13T13:04:57.351Z",813 "last_posted_at": "2024-11-14T02:51:32.282Z",814 "bumped": true,815 "bumped_at": "2024-11-14T02:51:32.282Z",816 "archetype": "regular",817 "unseen": false,818 "pinned": false,819 "unpinned": null,820 "visible": true,821 "closed": false,822 "archived": false,823 "bookmarked": null,824 "liked": null,825 "tags_descriptions": {},826 "like_count": 0,827 "views": 1298,828 "category_id": 1,829 "featured_link": null,830 "has_accepted_answer": false,831 "posters": [832 {833 "extras": "latest",834 "description": "Original Poster, Most Recent Poster",835 "user": {836 "id": 80887,837 "username": "WuJiayang",838 "name": "",839 "avatar_template": "/letter_avatar_proxy/v4/letter/w/d6d6ee/{size}.png",840 "trust_level": 0841 }842 },843 {844 "extras": null,845 "description": "Frequent Poster",846 "user": {847 "id": 3534,848 "username": "ptrblck",849 "name": "",850 "avatar_template": "/user_avatar/discuss.pytorch.org/ptrblck/{size}/1823_2.png",851 "admin": true,852 "moderator": true,853 "trust_level": 2854 }855 }856 ]857 },858 {859 "fancy_title": "Understanding the traceback of a compiler error with cache size limit",860 "id": 215669,861 "title": "Understanding the traceback of a compiler error with cache size limit",862 "slug": "understanding-the-traceback-of-a-compiler-error-with-cache-size-limit",863 "posts_count": 1,864 "reply_count": 0,865 "highest_post_number": 1,866 "image_url": null,867 "created_at": "2025-01-21T10:43:52.773Z",868 "last_posted_at": "2025-01-21T10:43:52.812Z",869 "bumped": true,870 "bumped_at": "2025-01-21T10:43:52.812Z",871 "archetype": "regular",872 "unseen": false,873 "pinned": false,874 "unpinned": null,875 "visible": true,876 "closed": false,877 "archived": false,878 "bookmarked": null,879 "liked": null,880 "tags_descriptions": {},881 "like_count": 0,882 "views": 53,883 "category_id": 1,884 "featured_link": null,885 "has_accepted_answer": false,886 "posters": [887 {888 "extras": "latest single",889 "description": "Original Poster, Most Recent Poster",890 "user": {891 "id": 82217,892 "username": "Johannes_Vogt",893 "name": "Johannes Vogt",894 "avatar_template": "/user_avatar/discuss.pytorch.org/johannes_vogt/{size}/75220_2.png",895 "trust_level": 1896 }897 }898 ]899 },900 {901 "fancy_title": "On pull requests… progress",902 "id": 215848,903 "title": "On pull requests... progress",904 "slug": "on-pull-requests-progress",905 "posts_count": 4,906 "reply_count": 1,907 "highest_post_number": 4,908 "image_url": null,909 "created_at": "2025-01-25T10:28:24.038Z",910 "last_posted_at": "2025-02-01T18:09:38.173Z",911 "bumped": true,912 "bumped_at": "2025-02-01T18:09:38.173Z",913 "archetype": "regular",914 "unseen": false,915 "pinned": false,916 "unpinned": null,917 "visible": true,918 "closed": false,919 "archived": false,920 "bookmarked": null,921 "liked": null,922 "tags_descriptions": {},923 "like_count": 1,924 "views": 166,925 "category_id": 1,926 "featured_link": null,927 "has_accepted_answer": false,928 "posters": [929 {930 "extras": "latest",931 "description": "Original Poster, Most Recent Poster",932 "user": {933 "id": 70673,934 "username": "kuraga",935 "name": "Alexander Kurakin",936 "avatar_template": "/user_avatar/discuss.pytorch.org/kuraga/{size}/68422_2.png",937 "trust_level": 1938 }939 },940 {941 "extras": null,942 "description": "Frequent Poster",943 "user": {944 "id": 3534,945 "username": "ptrblck",946 "name": "",947 "avatar_template": "/user_avatar/discuss.pytorch.org/ptrblck/{size}/1823_2.png",948 "admin": true,949 "moderator": true,950 "trust_level": 2951 }952 }953 ]954 },955 {956 "fancy_title": "5070Ti+Ubuntu 20.04.6+cuda?",957 "id": 217901,958 "title": "5070Ti+Ubuntu 20.04.6+cuda?",959 "slug": "5070ti-ubuntu-20-04-6-cuda",960 "posts_count": 3,961 "reply_count": 1,962 "highest_post_number": 3,963 "image_url": null,964 "created_at": "2025-03-16T03:22:30.520Z",965 "last_posted_at": "2025-03-19T10:15:03.040Z",966 "bumped": true,967 "bumped_at": "2025-03-19T10:15:03.040Z",968 "archetype": "regular",969 "unseen": false,970 "pinned": false,971 "unpinned": null,972 "visible": true,973 "closed": false,974 "archived": false,975 "bookmarked": null,976 "liked": null,977 "tags_descriptions": {},978 "like_count": 0,979 "views": 273,980 "category_id": 1,981 "featured_link": null,982 "has_accepted_answer": false,983 "posters": [984 {985 "extras": "latest",986 "description": "Original Poster, Most Recent Poster",987 "user": {988 "id": 83301,989 "username": "riva_lei",990 "name": "riva lei",991 "avatar_template": "/user_avatar/discuss.pytorch.org/riva_lei/{size}/76187_2.png",992 "trust_level": 1993 }994 },995 {996 "extras": null,997 "description": "Frequent Poster",998 "user": {999 "id": 3534,1000 "username": "ptrblck",1001 "name": "",1002 "avatar_template": "/user_avatar/discuss.pytorch.org/ptrblck/{size}/1823_2.png",1003 "admin": true,1004 "moderator": true,1005 "trust_level": 21006 }1007 }1008 ]1009 },1010 {1011 "fancy_title": "Not able to install PyTorch via pip",1012 "id": 218970,1013 "title": "Not able to install PyTorch via pip",1014 "slug": "not-able-to-install-pytorch-via-pip",1015 "posts_count": 5,1016 "reply_count": 1,1017 "highest_post_number": 5,1018 "image_url": null,1019 "created_at": "2025-04-11T11:54:04.410Z",1020 "last_posted_at": "2025-04-11T16:39:36.369Z",1021 "bumped": true,1022 "bumped_at": "2025-04-11T16:39:36.369Z",1023 "archetype": "regular",1024 "unseen": false,1025 "pinned": false,1026 "unpinned": null,1027 "visible": true,1028 "closed": false,1029 "archived": false,1030 "bookmarked": null,1031 "liked": null,1032 "tags_descriptions": {},1033 "like_count": 0,1034 "views": 195,1035 "category_id": 1,1036 "featured_link": null,1037 "has_accepted_answer": true,1038 "posters": [1039 {1040 "extras": null,1041 "description": "Original Poster",1042 "user": {1043 "id": 19812,1044 "username": "mahmoodn",1045 "name": "Mahmood Naderan",1046 "avatar_template": "/user_avatar/discuss.pytorch.org/mahmoodn/{size}/13302_2.png",1047 "trust_level": 11048 }1049 },1050 {1051 "extras": "latest",1052 "description": "Most Recent Poster, Accepted Answer",1053 "user": {1054 "id": 3534,1055 "username": "ptrblck",1056 "name": "",1057 "avatar_template": "/user_avatar/discuss.pytorch.org/ptrblck/{size}/1823_2.png",1058 "admin": true,1059 "moderator": true,1060 "trust_level": 21061 }1062 }1063 ]1064 }1065 ],1066 "tags_descriptions": {},1067 "fancy_title": "Batch.num_graph in Pytorch geometric",1068 "id": 73539,1069 "title": "Batch.num_graph in Pytorch geometric",1070 "posts_count": 1,1071 "created_at": "2020-03-17T16:11:27.143Z",1072 "views": 334,1073 "reply_count": 0,1074 "like_count": 0,1075 "last_posted_at": "2020-03-17T16:11:27.195Z",1076 "visible": true,1077 "closed": false,1078 "archived": false,1079 "has_summary": false,1080 "archetype": "regular",1081 "slug": "batch-num-graph-in-pytorch-geometric",1082 "category_id": 1,1083 "word_count": 17,1084 "deleted_at": null,1085 "user_id": 28795,1086 "featured_link": null,1087 "pinned_globally": false,1088 "pinned_at": null,1089 "pinned_until": null,1090 "image_url": null,1091 "slow_mode_seconds": 0,1092 "draft": null,1093 "draft_key": "topic_73539",1094 "draft_sequence": null,1095 "unpinned": null,1096 "pinned": false,1097 "current_post_number": 1,1098 "highest_post_number": 1,1099 "deleted_by": null,1100 "actions_summary": [1101 {1102 "id": 4,1103 "count": 0,1104 "hidden": false,1105 "can_act": false1106 },1107 {1108 "id": 8,1109 "count": 0,1110 "hidden": false,1111 "can_act": false1112 },1113 {1114 "id": 10,1115 "count": 0,1116 "hidden": false,1117 "can_act": false1118 },1119 {1120 "id": 7,1121 "count": 0,1122 "hidden": false,1123 "can_act": false1124 }1125 ],1126 "chunk_size": 20,1127 "bookmarked": false,1128 "topic_timer": null,1129 "message_bus_last_id": 0,1130 "participant_count": 1,1131 "show_read_indicator": false,1132 "thumbnails": null,1133 "slow_mode_enabled_until": null,1134 "can_vote": false,1135 "vote_count": 0,1136 "user_voted": false,1137 "discourse_zendesk_plugin_zendesk_id": null,1138 "discourse_zendesk_plugin_zendesk_url": "https://your-url.zendesk.com/agent/tickets/",1139 "details": {1140 "can_edit": false,1141 "notification_level": 1,1142 "participants": [1143 {1144 "id": 28795,1145 "username": "AsmaAdil",1146 "name": "Asma Sattar",1147 "avatar_template": "/letter_avatar_proxy/v4/letter/a/53a042/{size}.png",1148 "post_count": 1,1149 "primary_group_name": null,1150 "flair_name": null,1151 "flair_url": null,1152 "flair_color": null,1153 "flair_bg_color": null,1154 "flair_group_id": null,1155 "trust_level": 11156 }1157 ],1158 "created_by": {1159 "id": 28795,1160 "username": "AsmaAdil",1161 "name": "Asma Sattar",1162 "avatar_template": "/letter_avatar_proxy/v4/letter/a/53a042/{size}.png"1163 },1164 "last_poster": {1165 "id": 28795,1166 "username": "AsmaAdil",1167 "name": "Asma Sattar",1168 "avatar_template": "/letter_avatar_proxy/v4/letter/a/53a042/{size}.png"1169 }1170 },1171 "bookmarks": []1172 },1173 {1174 "post_stream": {1175 "posts": [1176 {1177 "id": 174437,1178 "name": "Doliolarzz",1179 "username": "doliolarzz",1180 "avatar_template": "/letter_avatar_proxy/v4/letter/d/d78d45/{size}.png",1181 "created_at": "2020-03-16T17:24:19.317Z",1182 "cooked": "<p>Hello,<br>\nA model has time-series structure.<br>\nSo, I’ve split each time-step into each gpu, due to insufficiency of gpu memory.</p>\n<p>But I got an error when calling loss.backward():</p>\n<pre><code class=\"lang-auto\">File \"/home/xxx/.local/lib/python3.7/site-packages/torch/autograd/__init__.py\", line 99, in backward\n allow_unreachable=True) # allow_unreachable flag\nRuntimeError: Function CatBackward returned an invalid gradient at index 1 - expected device cuda:2 but got cuda:0\n</code></pre>\n<p>Please find the minimum code below:</p>\n<pre><code class=\"lang-auto\"> self.config['DEVICE_ALL'] = [0, 2, 3]\n self.backbone = UNet(n_channels=config['IN_LEN']*3 - 2)\n self.geo = nn.Parameter(data=torch.randn(1, 4, self.h, self.w), requires_grad=True)\n self.outConv = nn.Conv2d(68, 1, kernel_size=1)\n self.output_time_length = 3\n\n def forward(self, input):\n\n outputs = []\n cur_input = input\n cur_state = self.get_state(...)\n\n for i in range(self.output_time_length):\n \n dev = self.config['DEVICE_ALL'][i]\n cur_input = cur_input.cuda(dev)\n cur_state = cur_state.cuda(dev)\n x = torch.cat([cur_input, cur_state], 1).cuda(dev)\n\n self.backbone = self.backbone.cuda(dev)\n x = self.backbone(x).cuda(dev)\n\n geo = self.geo.expand(x.shape[0], -1, -1, -1).cuda(dev)\n x = torch.cat([x, geo], 1).cuda(dev)\n\n self.outConv = self.outConv.cuda(dev)\n x = self.outConv(x).cuda(dev)\n\n outputs.append(x)\n cur_input = torch.cat([cur_input[:, 1:], x], 1)\n cur_state = self.get_next_state(...)\n\n return torch.cat(outputs, 1).cuda(dev)\n</code></pre>\n<p>Thanks in advance!</p>",1183 "post_number": 1,1184 "post_type": 1,1185 "posts_count": 12,1186 "updated_at": "2020-03-16T19:41:33.398Z",1187 "reply_count": 0,1188 "reply_to_post_number": null,1189 "quote_count": 0,1190 "incoming_link_count": 610,1191 "reads": 35,1192 "readers_count": 34,1193 "score": 3057.0,1194 "yours": false,1195 "topic_id": 73427,1196 "topic_slug": "multi-gpu-backward-error",1197 "display_username": "Doliolarzz",1198 "primary_group_name": null,1199 "flair_name": null,1200 "flair_url": null,