Anurag1734/cuda-error-resolution-analysis
07
1[2 {3 "post_stream": {4 "posts": [5 {6 "id": 83008,7 "name": "Héctor Corrales",8 "username": "Hector_Corrales",9 "avatar_template": "/letter_avatar_proxy/v4/letter/h/ecd19e/{size}.png",10 "created_at": "2018-12-19T12:54:32.034Z",11 "cooked": "<p>As I understand, in model parallelism, you divide a model and train each part separately in different GPUs.<br>\nAn example code I’ve found is this:</p>\n<blockquote>\n<p>class Network(nn.Module):<br>\ndef <strong>init</strong>(self, split_gpus):<br>\nself.module1 = (some layers)<br>\nself.module2 = (some layers)</p>\n<pre><code> self.split_gpus = split_gpus\n if self.split_gpus:\n self.module1.device(\"cuda:0\")\n self.module2.device(\"cuda:1\")\n\ndef forward(self, x):\n x = self.module1(x)\n if self.split_gpus:\n x = x.device(\"cuda:1\") \n return self.module2(x)\n</code></pre>\n</blockquote>\n<p>My question is, is there a way to join 2 GPUs to be seen as a single GPU with double memory and not having to split the model?<br>\nDoes NVIDIA NVLINK do this?<br>\nIf not, what does NVLINK do?</p>\n<p>Thanks!</p>",12 "post_number": 1,13 "post_type": 1,14 "posts_count": 3,15 "updated_at": "2018-12-19T12:54:57.963Z",16 "reply_count": 0,17 "reply_to_post_number": null,18 "quote_count": 0,19 "incoming_link_count": 1788,20 "reads": 67,21 "readers_count": 66,22 "score": 8968.4,23 "yours": false,24 "topic_id": 32569,25 "topic_slug": "model-parallelism-and-nvidia-nvlink",26 "display_username": "Héctor Corrales",27 "primary_group_name": null,28 "flair_name": null,29 "flair_url": null,30 "flair_bg_color": null,31 "flair_color": null,32 "flair_group_id": null,33 "badges_granted": [],34 "version": 1,35 "can_edit": false,36 "can_delete": false,37 "can_recover": false,38 "can_see_hidden_post": false,39 "can_wiki": false,40 "read": true,41 "user_title": null,42 "bookmarked": false,43 "actions_summary": [44 {45 "id": 2,46 "count": 147 }48 ],49 "moderator": false,50 "admin": false,51 "staff": false,52 "user_id": 14031,53 "hidden": false,54 "trust_level": 0,55 "deleted_at": null,56 "user_deleted": false,57 "edit_reason": null,58 "can_view_edit_history": true,59 "wiki": false,60 "post_url": "/t/model-parallelism-and-nvidia-nvlink/32569/1",61 "can_accept_answer": false,62 "can_unaccept_answer": false,63 "accepted_answer": false,64 "topic_accepted_answer": null,65 "can_vote": false66 },67 {68 "id": 96280,69 "name": "Sammy Sidhu",70 "username": "samster25",71 "avatar_template": "/letter_avatar_proxy/v4/letter/s/858c86/{size}.png",72 "created_at": "2019-02-28T02:26:25.637Z",73 "cooked": "<p>It sounds like what you want is Data Parallelism. Where you have the model replicated on each GPU and they each work on different data. This works if you can do a batch size > 1 on a single one of your GPUs.</p>\n<p>NVLINK provides a faster interconnect compared to PCIe. This allows faster GPU<>GPU communication for either data or model parallelism.</p>",74 "post_number": 2,75 "post_type": 1,76 "posts_count": 3,77 "updated_at": "2019-02-28T02:26:25.637Z",78 "reply_count": 1,79 "reply_to_post_number": null,80 "quote_count": 0,81 "incoming_link_count": 14,82 "reads": 60,83 "readers_count": 59,84 "score": 87.0,85 "yours": false,86 "topic_id": 32569,87 "topic_slug": "model-parallelism-and-nvidia-nvlink",88 "display_username": "Sammy Sidhu",89 "primary_group_name": null,90 "flair_name": null,91 "flair_url": null,92 "flair_bg_color": null,93 "flair_color": null,94 "flair_group_id": null,95 "badges_granted": [],96 "version": 1,97 "can_edit": false,98 "can_delete": false,99 "can_recover": false,100 "can_see_hidden_post": false,101 "can_wiki": false,102 "read": true,103 "user_title": null,104 "bookmarked": false,105 "actions_summary": [],106 "moderator": false,107 "admin": false,108 "staff": false,109 "user_id": 16159,110 "hidden": false,111 "trust_level": 1,112 "deleted_at": null,113 "user_deleted": false,114 "edit_reason": null,115 "can_view_edit_history": true,116 "wiki": false,117 "post_url": "/t/model-parallelism-and-nvidia-nvlink/32569/2",118 "can_accept_answer": false,119 "can_unaccept_answer": false,120 "accepted_answer": false,121 "topic_accepted_answer": null122 },123 {124 "id": 179488,125 "name": "Rubeen Mohammad",126 "username": "Rubeen_Mohammad",127 "avatar_template": "/user_avatar/discuss.pytorch.org/rubeen_mohammad/{size}/11836_2.png",128 "created_at": "2020-04-06T06:02:16.559Z",129 "cooked": "<aside class=\"quote no-group\" data-username=\"samster25\" data-post=\"2\" data-topic=\"32569\">\n<div class=\"title\">\n<div class=\"quote-controls\"></div>\n<img loading=\"lazy\" alt=\"\" width=\"24\" height=\"24\" src=\"https://discuss.pytorch.org/letter_avatar_proxy/v4/letter/s/858c86/48.png\" class=\"avatar\"> samster25:</div>\n<blockquote>\n<p>you want is Data Parallelism.</p>\n</blockquote>\n</aside>\n<p>Hi, I’m very new to Nvlink and before starting up, do we need write any code to enable the usage of nvlink for model parallelism. Could you please give me any example code for running a model on nvlink, if you have any.<br>\nThank you</p>",130 "post_number": 3,131 "post_type": 1,132 "posts_count": 3,133 "updated_at": "2020-04-06T06:02:16.559Z",134 "reply_count": 0,135 "reply_to_post_number": 2,136 "quote_count": 1,137 "incoming_link_count": 8,138 "reads": 30,139 "readers_count": 29,140 "score": 78.0,141 "yours": false,142 "topic_id": 32569,143 "topic_slug": "model-parallelism-and-nvidia-nvlink",144 "display_username": "Rubeen Mohammad",145 "primary_group_name": null,146 "flair_name": null,147 "flair_url": null,148 "flair_bg_color": null,149 "flair_color": null,150 "flair_group_id": null,151 "badges_granted": [],152 "version": 1,153 "can_edit": false,154 "can_delete": false,155 "can_recover": false,156 "can_see_hidden_post": false,157 "can_wiki": false,158 "read": true,159 "user_title": null,160 "bookmarked": false,161 "actions_summary": [162 {163 "id": 2,164 "count": 1165 }166 ],167 "moderator": false,168 "admin": false,169 "staff": false,170 "user_id": 27063,171 "hidden": false,172 "trust_level": 1,173 "deleted_at": null,174 "user_deleted": false,175 "edit_reason": null,176 "can_view_edit_history": true,177 "wiki": false,178 "post_url": "/t/model-parallelism-and-nvidia-nvlink/32569/3",179 "can_accept_answer": false,180 "can_unaccept_answer": false,181 "accepted_answer": false,182 "topic_accepted_answer": null183 }184 ],185 "stream": [186 83008,187 96280,188 179488189 ]190 },191 "timeline_lookup": [192 [193 1,194 2502195 ],196 [197 2,198 2432199 ],200 [201 3,202 2029203 ]204 ],205 "suggested_topics": [206 {207 "fancy_title": "Autograd: Add VJP and JVP rules for aten::aminmax #151186",208 "id": 220318,209 "title": "Autograd: Add VJP and JVP rules for aten::aminmax #151186",210 "slug": "autograd-add-vjp-and-jvp-rules-for-aten-aminmax-151186",211 "posts_count": 1,212 "reply_count": 0,213 "highest_post_number": 1,214 "image_url": null,215 "created_at": "2025-05-25T13:19:23.208Z",216 "last_posted_at": "2025-05-25T13:19:23.245Z",217 "bumped": true,218 "bumped_at": "2025-05-25T13:19:23.245Z",219 "archetype": "regular",220 "unseen": false,221 "pinned": false,222 "unpinned": null,223 "visible": true,224 "closed": false,225 "archived": false,226 "bookmarked": null,227 "liked": null,228 "tags_descriptions": {},229 "like_count": 0,230 "views": 36,231 "category_id": 7,232 "featured_link": null,233 "has_accepted_answer": false,234 "posters": [235 {236 "extras": "latest single",237 "description": "Original Poster, Most Recent Poster",238 "user": {239 "id": 84445,240 "username": "vijayabhaskarev",241 "name": "Vijay",242 "avatar_template": "/letter_avatar_proxy/v4/letter/v/f08c70/{size}.png",243 "trust_level": 0244 }245 }246 ]247 },248 {249 "fancy_title": "How to pinpoint NaN grads?",250 "id": 213942,251 "title": "How to pinpoint NaN grads?",252 "slug": "how-to-pinpoint-nan-grads",253 "posts_count": 3,254 "reply_count": 1,255 "highest_post_number": 3,256 "image_url": null,257 "created_at": "2024-12-07T10:27:15.422Z",258 "last_posted_at": "2024-12-12T20:50:29.568Z",259 "bumped": true,260 "bumped_at": "2024-12-12T20:50:29.568Z",261 "archetype": "regular",262 "unseen": false,263 "pinned": false,264 "unpinned": null,265 "visible": true,266 "closed": false,267 "archived": false,268 "bookmarked": null,269 "liked": null,270 "tags_descriptions": {},271 "like_count": 1,272 "views": 248,273 "category_id": 7,274 "featured_link": null,275 "has_accepted_answer": true,276 "posters": [277 {278 "extras": null,279 "description": "Original Poster, Accepted Answer",280 "user": {281 "id": 79050,282 "username": "CesareMontresor",283 "name": "Cesare Montresor",284 "avatar_template": "/user_avatar/discuss.pytorch.org/cesaremontresor/{size}/72884_2.png",285 "trust_level": 1286 }287 },288 {289 "extras": "latest",290 "description": "Most Recent Poster",291 "user": {292 "id": 41396,293 "username": "soulitzer",294 "name": "",295 "avatar_template": "/letter_avatar_proxy/v4/letter/s/839c29/{size}.png",296 "trust_level": 2297 }298 }299 ]300 },301 {302 "fancy_title": "Missing argument create_graph in torch.func api",303 "id": 215670,304 "title": "Missing argument create_graph in torch.func api",305 "slug": "missing-argument-create-graph-in-torch-func-api",306 "posts_count": 7,307 "reply_count": 4,308 "highest_post_number": 7,309 "image_url": null,310 "created_at": "2025-01-21T11:02:25.625Z",311 "last_posted_at": "2025-01-24T01:44:06.847Z",312 "bumped": true,313 "bumped_at": "2025-01-24T01:44:06.847Z",314 "archetype": "regular",315 "unseen": false,316 "pinned": false,317 "unpinned": null,318 "visible": true,319 "closed": false,320 "archived": false,321 "bookmarked": null,322 "liked": null,323 "tags_descriptions": {},324 "like_count": 0,325 "views": 242,326 "category_id": 7,327 "featured_link": null,328 "has_accepted_answer": false,329 "posters": [330 {331 "extras": "latest",332 "description": "Original Poster, Most Recent Poster",333 "user": {334 "id": 43194,335 "username": "LuoXin-s",336 "name": "Sienna",337 "avatar_template": "/user_avatar/discuss.pytorch.org/luoxin-s/{size}/35927_2.png",338 "trust_level": 1339 }340 },341 {342 "extras": null,343 "description": "Frequent Poster",344 "user": {345 "id": 34294,346 "username": "AlphaBetaGamma96",347 "name": "",348 "avatar_template": "/letter_avatar_proxy/v4/letter/a/3da27b/{size}.png",349 "trust_level": 2350 }351 }352 ]353 },354 {355 "fancy_title": "Can I split my input in multiple embeddings? How would pytorch compute gradients?",356 "id": 212676,357 "title": "Can I split my input in multiple embeddings? How would pytorch compute gradients?",358 "slug": "can-i-split-my-input-in-multiple-embeddings-how-would-pytorch-compute-gradients",359 "posts_count": 2,360 "reply_count": 0,361 "highest_post_number": 2,362 "image_url": null,363 "created_at": "2024-11-08T02:24:12.658Z",364 "last_posted_at": "2024-11-08T04:07:54.481Z",365 "bumped": true,366 "bumped_at": "2024-11-08T04:07:54.481Z",367 "archetype": "regular",368 "unseen": false,369 "pinned": false,370 "unpinned": null,371 "visible": true,372 "closed": false,373 "archived": false,374 "bookmarked": null,375 "liked": null,376 "tags_descriptions": {},377 "like_count": 1,378 "views": 33,379 "category_id": 7,380 "featured_link": null,381 "has_accepted_answer": true,382 "posters": [383 {384 "extras": null,385 "description": "Original Poster",386 "user": {387 "id": 80757,388 "username": "Thomas_J",389 "name": "Thomas J",390 "avatar_template": "/user_avatar/discuss.pytorch.org/thomas_j/{size}/73841_2.png",391 "trust_level": 1392 }393 },394 {395 "extras": "latest",396 "description": "Most Recent Poster, Accepted Answer",397 "user": {398 "id": 3534,399 "username": "ptrblck",400 "name": "",401 "avatar_template": "/user_avatar/discuss.pytorch.org/ptrblck/{size}/1823_2.png",402 "admin": true,403 "moderator": true,404 "trust_level": 2405 }406 }407 ]408 },409 {410 "fancy_title": "Efficient Computation of Second-Order Gradients",411 "id": 221328,412 "title": "Efficient Computation of Second-Order Gradients",413 "slug": "efficient-computation-of-second-order-gradients",414 "posts_count": 2,415 "reply_count": 0,416 "highest_post_number": 2,417 "image_url": null,418 "created_at": "2025-07-07T11:11:16.706Z",419 "last_posted_at": "2025-07-07T17:32:22.523Z",420 "bumped": true,421 "bumped_at": "2025-07-07T17:32:22.523Z",422 "archetype": "regular",423 "unseen": false,424 "pinned": false,425 "unpinned": null,426 "visible": true,427 "closed": false,428 "archived": false,429 "bookmarked": null,430 "liked": null,431 "tags_descriptions": {},432 "like_count": 0,433 "views": 44,434 "category_id": 7,435 "featured_link": null,436 "has_accepted_answer": false,437 "posters": [438 {439 "extras": null,440 "description": "Original Poster",441 "user": {442 "id": 84575,443 "username": "katonapng",444 "name": "Anna",445 "avatar_template": "/user_avatar/discuss.pytorch.org/katonapng/{size}/77256_2.png",446 "trust_level": 0447 }448 },449 {450 "extras": "latest",451 "description": "Most Recent Poster",452 "user": {453 "id": 84935,454 "username": "paulk",455 "name": "",456 "avatar_template": "/letter_avatar_proxy/v4/letter/p/67e7ee/{size}.png",457 "trust_level": 2458 }459 }460 ]461 }462 ],463 "tags_descriptions": {},464 "fancy_title": "Model Parallelism and NVIDIA NVLINK",465 "id": 32569,466 "title": "Model Parallelism and NVIDIA NVLINK",467 "posts_count": 3,468 "created_at": "2018-12-19T12:54:31.980Z",469 "views": 3107,470 "reply_count": 1,471 "like_count": 2,472 "last_posted_at": "2020-04-06T06:02:16.559Z",473 "visible": true,474 "closed": false,475 "archived": false,476 "has_summary": false,477 "archetype": "regular",478 "slug": "model-parallelism-and-nvidia-nvlink",479 "category_id": 7,480 "word_count": 238,481 "deleted_at": null,482 "user_id": 14031,483 "featured_link": null,484 "pinned_globally": false,485 "pinned_at": null,486 "pinned_until": null,487 "image_url": null,488 "slow_mode_seconds": 0,489 "draft": null,490 "draft_key": "topic_32569",491 "draft_sequence": null,492 "unpinned": null,493 "pinned": false,494 "current_post_number": 1,495 "highest_post_number": 3,496 "deleted_by": null,497 "actions_summary": [498 {499 "id": 4,500 "count": 0,501 "hidden": false,502 "can_act": false503 },504 {505 "id": 8,506 "count": 0,507 "hidden": false,508 "can_act": false509 },510 {511 "id": 10,512 "count": 0,513 "hidden": false,514 "can_act": false515 },516 {517 "id": 7,518 "count": 0,519 "hidden": false,520 "can_act": false521 }522 ],523 "chunk_size": 20,524 "bookmarked": false,525 "topic_timer": null,526 "message_bus_last_id": 0,527 "participant_count": 3,528 "show_read_indicator": false,529 "thumbnails": null,530 "slow_mode_enabled_until": null,531 "can_vote": false,532 "vote_count": 0,533 "user_voted": false,534 "discourse_zendesk_plugin_zendesk_id": null,535 "discourse_zendesk_plugin_zendesk_url": "https://your-url.zendesk.com/agent/tickets/",536 "details": {537 "can_edit": false,538 "notification_level": 1,539 "participants": [540 {541 "id": 14031,542 "username": "Hector_Corrales",543 "name": "Héctor Corrales",544 "avatar_template": "/letter_avatar_proxy/v4/letter/h/ecd19e/{size}.png",545 "post_count": 1,546 "primary_group_name": null,547 "flair_name": null,548 "flair_url": null,549 "flair_color": null,550 "flair_bg_color": null,551 "flair_group_id": null,552 "trust_level": 0553 },554 {555 "id": 16159,556 "username": "samster25",557 "name": "Sammy Sidhu",558 "avatar_template": "/letter_avatar_proxy/v4/letter/s/858c86/{size}.png",559 "post_count": 1,560 "primary_group_name": null,561 "flair_name": null,562 "flair_url": null,563 "flair_color": null,564 "flair_bg_color": null,565 "flair_group_id": null,566 "trust_level": 1567 },568 {569 "id": 27063,570 "username": "Rubeen_Mohammad",571 "name": "Rubeen Mohammad",572 "avatar_template": "/user_avatar/discuss.pytorch.org/rubeen_mohammad/{size}/11836_2.png",573 "post_count": 1,574 "primary_group_name": null,575 "flair_name": null,576 "flair_url": null,577 "flair_color": null,578 "flair_bg_color": null,579 "flair_group_id": null,580 "trust_level": 1581 }582 ],583 "created_by": {584 "id": 14031,585 "username": "Hector_Corrales",586 "name": "Héctor Corrales",587 "avatar_template": "/letter_avatar_proxy/v4/letter/h/ecd19e/{size}.png"588 },589 "last_poster": {590 "id": 27063,591 "username": "Rubeen_Mohammad",592 "name": "Rubeen Mohammad",593 "avatar_template": "/user_avatar/discuss.pytorch.org/rubeen_mohammad/{size}/11836_2.png"594 }595 },596 "bookmarks": []597 },598 {599 "post_stream": {600 "posts": [601 {602 "id": 179355,603 "name": "",604 "username": "Absurd",605 "avatar_template": "/user_avatar/discuss.pytorch.org/absurd/{size}/25137_2.png",606 "created_at": "2020-04-05T12:34:36.619Z",607 "cooked": "<p>Supporting that I have a N × 4 tensor which is N rectangular ROIs of a mask(C × H × W), and I want to sum up all value of each channel inside a ROI to get a tensor of size N × C, the only way I can figure out is using a for loop. But this method is very slow, so I want to ask if there are faster way to do this?<br>\nThe for loop method is as below:</p>\n<pre><code class=\"lang-auto\">proposals # N * 4 tensor (ROIs)\nmask # C * H * W tensor\ntotal_numbers = []\nfor i in range(proposals.shape[1]):\n proposal = proposals[i]\n mask_region = mask[:, proposal[0]: proposal[2], proposal[1]: proposal[3]]\n total_number = mask_region.sum(dim=(1, 2))\n total_numbers.append(total_number)\n</code></pre>",608 "post_number": 1,609 "post_type": 1,610 "posts_count": 3,611 "updated_at": "2020-04-05T12:34:36.619Z",612 "reply_count": 1,613 "reply_to_post_number": null,614 "quote_count": 0,615 "incoming_link_count": 371,616 "reads": 29,617 "readers_count": 28,618 "score": 1865.8,619 "yours": false,620 "topic_id": 75432,621 "topic_slug": "how-to-sum-up-all-values-according-to-rectangular-rois-of-a-feature-map-quickly",622 "display_username": "",623 "primary_group_name": null,624 "flair_name": null,625 "flair_url": null,626 "flair_bg_color": null,627 "flair_color": null,628 "flair_group_id": null,629 "badges_granted": [],630 "version": 1,631 "can_edit": false,632 "can_delete": false,633 "can_recover": false,634 "can_see_hidden_post": false,635 "can_wiki": false,636 "read": true,637 "user_title": null,638 "bookmarked": false,639 "actions_summary": [],640 "moderator": false,641 "admin": false,642 "staff": false,643 "user_id": 29552,644 "hidden": false,645 "trust_level": 1,646 "deleted_at": null,647 "user_deleted": false,648 "edit_reason": null,649 "can_view_edit_history": true,650 "wiki": false,651 "post_url": "/t/how-to-sum-up-all-values-according-to-rectangular-rois-of-a-feature-map-quickly/75432/1",652 "can_accept_answer": false,653 "can_unaccept_answer": false,654 "accepted_answer": false,655 "topic_accepted_answer": true,656 "can_vote": false657 },658 {659 "id": 179441,660 "name": "K. Frank",661 "username": "KFrank",662 "avatar_template": "/letter_avatar_proxy/v4/letter/k/ecb155/{size}.png",663 "created_at": "2020-04-05T23:58:49.430Z",664 "cooked": "<p>Hi Absurd!</p>\n<aside class=\"quote no-group\" data-username=\"Absurd\" data-post=\"1\" data-topic=\"75432\" data-full=\"true\">\n<div class=\"title\">\n<div class=\"quote-controls\"></div>\n<img loading=\"lazy\" alt=\"\" width=\"24\" height=\"24\" src=\"https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/absurd/48/25137_2.png\" class=\"avatar\"> Absurd:</div>\n<blockquote>\n<p>this method is very slow, so I want to ask if there are faster way to do this?</p>\n</blockquote>\n</aside>\n<p>There are two issues here:</p>\n<p>First, using for-loops generally slows pytorch down because they<br>\nprevent pytorch from performing operations on entire tensors using<br>\nits built in (and hopefully optimized) tensor operations.</p>\n<p>Second, in your method you sum over each rectangular ROI separately,<br>\npotentially performing some of the same sums repeatedly. There is<br>\na standard approach for avoiding this, where you first perform (in your<br>\ncase) a two-dimensional cumulative sum over your <code>mask</code> tensor, and<br>\nthen reconstruct the ROI sums from the cumulative sum using the<br>\nso-called inclusion-exclusion technique.</p>\n<p>(If you only have a few, or mostly non-overlapping ROIs, performing the<br>\nROI sums directly will be cheaper. But if you have a lot of overlapping<br>\nROIs, inclusion-exclusion can be dramatically more efficient. Also, gpu<br>\nvectorization will affect the trade-off above and beyond a simple count<br>\nof floating-point operations, so if this piece is performance-critical, run<br>\nsome timing tests on realistic data.)</p>\n<p>Here is a pytorch (version 0.3.0) script that reimplements your loop<br>\nsolution with using inclusion-exclusion with pure pytorch tensor<br>\noperations:</p>\n<pre data-code-wrap=\"python\"><code class=\"lang-python\">import torch\ntorch.__version__\n\ntorch.manual_seed (2020)\n\nC = 2\nH = 3\nW = 5\n# N = 5\n\nproposals = torch.LongTensor ([[0, 0, 0, 0], [2, 3, 2, 5], [0, 0, 1, 1], [1, 1, 3, 4], [2, 4, 3, 5]])\nmask = (10.0 * torch.rand (C, H, W)).long() # some random integers\n\nproposals\nmask\n\n# loop version\ntotal_numbers = []\nfor i in range(proposals.shape[0]): # .shape[1] was incorrect\n proposal = proposals[i]\n total_number = torch.zeros (mask.shape[0]).long() # empty ROIs have sum of 0\n if proposal[2] > proposal[0] and proposal[3] > proposal[1]:\n mask_region = mask[:, proposal[0]: proposal[2], proposal[1]: proposal[3]]\n total_number = mask_region.sum (2).sum (1)\n total_numbers.append(total_number)\n\ntotal_numbers = torch.stack (total_numbers, dim = 1) # make a 2d tensor instead of a list of tensors\n\ntotal_numbers\n\n# cumulative sums, inclusion-exclusion version\ncmsk = mask.cumsum (1).cumsum (2) # pre-compute cumulative sums\npmsk = torch.nn.ZeroPad2d ((1, 0, 1, 0))(cmsk) # pad to support inclusion-exclusion indexing\n# inclusion-exclusion ROI sum\nmsum = pmsk[:, proposals[:, 2], proposals[:, 3]] - pmsk[:, proposals[:, 0], proposals[:, 3]] - pmsk[:, proposals[:, 2] , proposals[:, 1]] + pmsk[:, proposals[:, 0], proposals[:, 1]]\n\nmsum\n</code></pre>\n<p>And here is the output:</p>\n<pre><code class=\"lang-plaintext\">>>> import torch\n>>> torch.__version__\n'0.3.0b0+591e73e'\n>>>\n>>> torch.manual_seed (2020)\n<torch._C.Generator object at 0x0000019B5E066630>\n>>>\n>>> C = 2\n>>> H = 3\n>>> W = 5\n>>> # N = 5\n...\n>>> proposals = torch.LongTensor ([[0, 0, 0, 0], [2, 3, 2, 5], [0, 0, 1, 1], [1, 1, 3, 4], [2, 4, 3, 5]])\n>>> mask = (10.0 * torch.rand (C, H, W)).long() # some random integers\n>>>\n>>> proposals\n\n 0 0 0 0\n 2 3 2 5\n 0 0 1 1\n 1 1 3 4\n 2 4 3 5\n[torch.LongTensor of size 5x4]\n\n>>> mask\n\n(0 ,.,.) =\n 4 1 5 1 4\n 2 5 8 2 6\n 5 8 7 4 8\n\n(1 ,.,.) =\n 5 7 0 1 8\n 0 5 8 3 4\n 5 0 7 3 4\n[torch.LongTensor of size 2x3x5]\n\n>>>\n>>> # loop version\n... total_numbers = []\n>>> for i in range(proposals.shape[0]): # .shape[1] was incorrect\n... proposal = proposals[i]\n... total_number = torch.zeros (mask.shape[0]).long() # empty ROIs have sum of 0\n... if proposal[2] > proposal[0] and proposal[3] > proposal[1]:\n... mask_region = mask[:, proposal[0]: proposal[2], proposal[1]: proposal[3]]\n... total_number = mask_region.sum (2).sum (1)\n... total_numbers.append(total_number)\n...\n>>> total_numbers = torch.stack (total_numbers, dim = 1) # make a 2d tensor instead of a list of tensors\n>>>\n>>> total_numbers\n\n 0 0 4 34 8\n 0 0 5 26 4\n[torch.LongTensor of size 2x5]\n\n>>>\n>>> # cumulative sums, inclusion-exclusion version\n... cmsk = mask.cumsum (1).cumsum (2) # pre-compute cumulative sums\n>>> pmsk = torch.nn.ZeroPad2d ((1, 0, 1, 0))(cmsk) # pad to support inclusion-exclusion indexing\n>>> # inclusion-exclusion ROI sum\n... msum = pmsk[:, proposals[:, 2], proposals[:, 3]] - pmsk[:, proposals[:, 0], proposals[:, 3]] - pmsk[:, proposals[:, 2] , proposals[:, 1]] + pmsk[:, proposals[:, 0], proposals[:, 1]]\n>>>\n>>> msum\nVariable containing:\n 0 0 4 34 8\n 0 0 5 26 4\n[torch.LongTensor of size 2x5]\n</code></pre>\n<p>Good luck.</p>\n<p>K. Frank</p>",665 "post_number": 2,666 "post_type": 1,667 "posts_count": 3,668 "updated_at": "2020-04-06T05:41:07.229Z",669 "reply_count": 1,670 "reply_to_post_number": null,671 "quote_count": 1,672 "incoming_link_count": 29,673 "reads": 28,674 "readers_count": 27,675 "score": 217.6,676 "yours": false,677 "topic_id": 75432,678 "topic_slug": "how-to-sum-up-all-values-according-to-rectangular-rois-of-a-feature-map-quickly",679 "display_username": "K. Frank",680 "primary_group_name": null,681 "flair_name": null,682 "flair_url": null,683 "flair_bg_color": null,684 "flair_color": null,685 "flair_group_id": null,686 "badges_granted": [],687 "version": 1,688 "can_edit": false,689 "can_delete": false,690 "can_recover": false,691 "can_see_hidden_post": false,692 "can_wiki": false,693 "link_counts": [694 {695 "url": "https://discuss.pytorch.org/t/an-efficient-way-to-slice-tensors-with-array-indices/77078/2",696 "internal": true,697 "reflection": true,698 "title": "An efficient way to slice tensors with array indices?",699 "clicks": 3700 },701 {702 "url": "https://discuss.pytorch.org/t/how-to-extract-smaller-image-patches-3d/16837/41",703 "internal": true,704 "reflection": true,705 "title": "How to extract smaller image patches (3D)?",706 "clicks": 1707 }708 ],709 "read": true,710 "user_title": null,711 "bookmarked": false,712 "actions_summary": [713 {714 "id": 2,715 "count": 2716 }717 ],718 "moderator": false,719 "admin": false,720 "staff": false,721 "user_id": 18088,722 "hidden": false,723 "trust_level": 2,724 "deleted_at": null,725 "user_deleted": false,726 "edit_reason": null,727 "can_view_edit_history": true,728 "wiki": false,729 "post_url": "/t/how-to-sum-up-all-values-according-to-rectangular-rois-of-a-feature-map-quickly/75432/2",730 "can_accept_answer": false,731 "can_unaccept_answer": false,732 "accepted_answer": true,733 "topic_accepted_answer": true734 },735 {736 "id": 179486,737 "name": "",738 "username": "Absurd",739 "avatar_template": "/user_avatar/discuss.pytorch.org/absurd/{size}/25137_2.png",740 "created_at": "2020-04-06T05:46:58.470Z",741 "cooked": "<p>Thanks for your reply. It helps me a lot! In my case N is large, so the inclusion-exclusion version is much faster than the loop version.</p>",742 "post_number": 4,743 "post_type": 1,744 "posts_count": 3,745 "updated_at": "2020-04-06T05:46:58.470Z",746 "reply_count": 0,747 "reply_to_post_number": 2,748 "quote_count": 0,749 "incoming_link_count": 8,750 "reads": 21,751 "readers_count": 20,752 "score": 44.2,753 "yours": false,754 "topic_id": 75432,755 "topic_slug": "how-to-sum-up-all-values-according-to-rectangular-rois-of-a-feature-map-quickly",756 "display_username": "",757 "primary_group_name": null,758 "flair_name": null,759 "flair_url": null,760 "flair_bg_color": null,761 "flair_color": null,762 "flair_group_id": null,763 "badges_granted": [],764 "version": 1,765 "can_edit": false,766 "can_delete": false,767 "can_recover": false,768 "can_see_hidden_post": false,769 "can_wiki": false,770 "read": true,771 "user_title": null,772 "reply_to_user": {773 "id": 18088,774 "username": "KFrank",775 "name": "K. Frank",776 "avatar_template": "/letter_avatar_proxy/v4/letter/k/ecb155/{size}.png"777 },778 "bookmarked": false,779 "actions_summary": [],780 "moderator": false,781 "admin": false,782 "staff": false,783 "user_id": 29552,784 "hidden": false,785 "trust_level": 1,786 "deleted_at": null,787 "user_deleted": false,788 "edit_reason": null,789 "can_view_edit_history": true,790 "wiki": false,791 "post_url": "/t/how-to-sum-up-all-values-according-to-rectangular-rois-of-a-feature-map-quickly/75432/4",792 "can_accept_answer": false,793 "can_unaccept_answer": false,794 "accepted_answer": false,795 "topic_accepted_answer": true796 }797 ],798 "stream": [799 179355,800 179441,801 179486802 ]803 },804 "timeline_lookup": [805 [806 1,807 2029808 ]809 ],810 "suggested_topics": [811 {812 "fancy_title": "Restrict FCN output to valid convolutions",813 "id": 218022,814 "title": "Restrict FCN output to valid convolutions",815 "slug": "restrict-fcn-output-to-valid-convolutions",816 "posts_count": 1,817 "reply_count": 0,818 "highest_post_number": 1,819 "image_url": null,820 "created_at": "2025-03-19T10:24:29.118Z",821 "last_posted_at": "2025-03-19T10:24:29.158Z",822 "bumped": true,823 "bumped_at": "2025-03-19T10:24:29.158Z",824 "archetype": "regular",825 "unseen": false,826 "pinned": false,827 "unpinned": null,828 "visible": true,829 "closed": false,830 "archived": false,831 "bookmarked": null,832 "liked": null,833 "tags_descriptions": {},834 "like_count": 0,835 "views": 58,836 "category_id": 5,837 "featured_link": null,838 "has_accepted_answer": false,839 "posters": [840 {841 "extras": "latest single",842 "description": "Original Poster, Most Recent Poster",843 "user": {844 "id": 83359,845 "username": "cpegel",846 "name": "",847 "avatar_template": "/user_avatar/discuss.pytorch.org/cpegel/{size}/76245_2.png",848 "trust_level": 1849 }850 }851 ]852 },853 {854 "fancy_title": "Help :frowning: Auto encoder predicts shapes ‘ok’ but lacks colours",855 "id": 214163,856 "title": "Help :( Auto encoder predicts shapes 'ok' but lacks colours",857 "slug": "help-auto-encoder-predicts-shapes-ok-but-lacks-colours",858 "posts_count": 3,859 "reply_count": 2,860 "highest_post_number": 5,861 "image_url": "https://discuss.pytorch.org/uploads/default/original/3X/5/0/501da039c27c2a1804a96da5f766dbf4ec662e9a.png",862 "created_at": "2024-12-12T18:31:06.027Z",863 "last_posted_at": "2024-12-19T14:51:00.132Z",864 "bumped": true,865 "bumped_at": "2024-12-19T14:51:00.132Z",866 "archetype": "regular",867 "unseen": false,868 "pinned": false,869 "unpinned": null,870 "visible": true,871 "closed": false,872 "archived": false,873 "bookmarked": null,874 "liked": null,875 "tags_descriptions": {},876 "like_count": 1,877 "views": 82,878 "category_id": 5,879 "featured_link": null,880 "has_accepted_answer": false,881 "posters": [882 {883 "extras": "latest",884 "description": "Original Poster, Most Recent Poster",885 "user": {886 "id": 81089,887 "username": "Aknw_Fen",888 "name": "Aknw Fen",889 "avatar_template": "/user_avatar/discuss.pytorch.org/aknw_fen/{size}/74156_2.png",890 "trust_level": 2891 }892 },893 {894 "extras": null,895 "description": "Frequent Poster",896 "user": {897 "id": 77908,898 "username": "mycul",899 "name": "",900 "avatar_template": "/user_avatar/discuss.pytorch.org/mycul/{size}/72394_2.png",901 "trust_level": 2902 }903 }904 ]905 },906 {907 "fancy_title": "Fold an overlapping 3D tensor?",908 "id": 213879,909 "title": "Fold an overlapping 3D tensor?",910 "slug": "fold-an-overlapping-3d-tensor",911 "posts_count": 2,912 "reply_count": 0,913 "highest_post_number": 2,914 "image_url": null,915 "created_at": "2024-12-05T22:30:45.221Z",916 "last_posted_at": "2024-12-09T20:00:12.156Z",917 "bumped": true,918 "bumped_at": "2024-12-09T20:00:12.156Z",919 "archetype": "regular",920 "unseen": false,921 "pinned": false,922 "unpinned": null,923 "visible": true,924 "closed": false,925 "archived": false,926 "bookmarked": null,927 "liked": null,928 "tags_descriptions": {},929 "like_count": 0,930 "views": 178,931 "category_id": 5,932 "featured_link": null,933 "has_accepted_answer": true,934 "posters": [935 {936 "extras": "latest single",937 "description": "Original Poster, Most Recent Poster, Accepted Answer",938 "user": {939 "id": 66240,940 "username": "bartley",941 "name": "Brendan",942 "avatar_template": "/user_avatar/discuss.pytorch.org/bartley/{size}/60569_2.png",943 "trust_level": 1944 }945 }946 ]947 },948 {949 "fancy_title": "Faster RCNN maP not improving. Suggestions?",950 "id": 212523,951 "title": "Faster RCNN maP not improving. Suggestions?",952 "slug": "faster-rcnn-map-not-improving-suggestions",953 "posts_count": 1,954 "reply_count": 0,955 "highest_post_number": 1,956 "image_url": "https://discuss.pytorch.org/uploads/default/optimized/3X/c/d/cd182af74b27ee1f734e000202f7c724fa56a41f_2_1024x422.png",957 "created_at": "2024-11-04T22:21:03.523Z",958 "last_posted_at": "2024-11-04T22:21:03.586Z",959 "bumped": true,960 "bumped_at": "2024-11-04T22:24:08.335Z",961 "archetype": "regular",962 "unseen": false,963 "pinned": false,964 "unpinned": null,965 "visible": true,966 "closed": false,967 "archived": false,968 "bookmarked": null,969 "liked": null,970 "tags_descriptions": {},971 "like_count": 0,972 "views": 111,973 "category_id": 5,974 "featured_link": null,975 "has_accepted_answer": false,976 "posters": [977 {978 "extras": "latest single",979 "description": "Original Poster, Most Recent Poster",980 "user": {981 "id": 80684,982 "username": "shanalikhan",983 "name": "Shan Khan",984 "avatar_template": "/user_avatar/discuss.pytorch.org/shanalikhan/{size}/73764_2.png",985 "trust_level": 1986 }987 }988 ]989 },990 {991 "fancy_title": "Adding an LSTM to a trained CNN ResNet during fine-tuning",992 "id": 222256,993 "title": "Adding an LSTM to a trained CNN ResNet during fine-tuning",994 "slug": "adding-an-lstm-to-a-trained-cnn-resnet-during-fine-tuning",995 "posts_count": 1,996 "reply_count": 0,997 "highest_post_number": 1,998 "image_url": null,999 "created_at": "2025-08-11T15:29:18.603Z",1000 "last_posted_at": "2025-08-11T15:29:18.662Z",1001 "bumped": true,1002 "bumped_at": "2025-08-11T15:29:18.662Z",1003 "archetype": "regular",1004 "unseen": false,1005 "pinned": false,1006 "unpinned": null,1007 "visible": true,1008 "closed": false,1009 "archived": false,1010 "bookmarked": null,1011 "liked": null,1012 "tags_descriptions": {},1013 "like_count": 0,1014 "views": 29,1015 "category_id": 5,1016 "featured_link": null,1017 "has_accepted_answer": false,1018 "posters": [1019 {1020 "extras": "latest single",1021 "description": "Original Poster, Most Recent Poster",1022 "user": {1023 "id": 85452,1024 "username": "catb",1025 "name": "catb",1026 "avatar_template": "/letter_avatar_proxy/v4/letter/c/f14d63/{size}.png",1027 "trust_level": 01028 }1029 }1030 ]1031 }1032 ],1033 "tags_descriptions": {},1034 "fancy_title": "How to sum up all values according to rectangular ROIs of a feature map quickly",1035 "id": 75432,1036 "title": "How to sum up all values according to rectangular ROIs of a feature map quickly",1037 "posts_count": 3,1038 "created_at": "2020-04-05T12:34:36.561Z",1039 "views": 1912,1040 "reply_count": 2,1041 "like_count": 2,1042 "last_posted_at": "2020-04-06T05:46:58.470Z",1043 "visible": true,1044 "closed": false,1045 "archived": false,1046 "has_summary": false,1047 "archetype": "regular",1048 "slug": "how-to-sum-up-all-values-according-to-rectangular-rois-of-a-feature-map-quickly",1049 "category_id": 5,1050 "word_count": 853,1051 "deleted_at": null,1052 "user_id": 29552,1053 "featured_link": null,1054 "pinned_globally": false,1055 "pinned_at": null,1056 "pinned_until": null,1057 "image_url": null,1058 "slow_mode_seconds": 0,1059 "draft": null,1060 "draft_key": "topic_75432",1061 "draft_sequence": null,1062 "unpinned": null,1063 "pinned": false,1064 "current_post_number": 1,1065 "highest_post_number": 4,1066 "deleted_by": null,1067 "actions_summary": [1068 {1069 "id": 4,1070 "count": 0,1071 "hidden": false,1072 "can_act": false1073 },1074 {1075 "id": 8,1076 "count": 0,1077 "hidden": false,1078 "can_act": false1079 },1080 {1081 "id": 10,1082 "count": 0,1083 "hidden": false,1084 "can_act": false1085 },1086 {1087 "id": 7,1088 "count": 0,1089 "hidden": false,1090 "can_act": false1091 }1092 ],1093 "chunk_size": 20,1094 "bookmarked": false,1095 "topic_timer": null,1096 "message_bus_last_id": 0,1097 "participant_count": 2,1098 "show_read_indicator": false,1099 "thumbnails": null,1100 "slow_mode_enabled_until": null,1101 "accepted_answer": {1102 "post_number": 2,1103 "username": "KFrank",1104 "name": "K. Frank",1105 "excerpt": "Hi Absurd! \n\nThere are two issues here: \nFirst, using for-loops generally slows pytorch down because they \nprevent pytorch from performing operations on entire tensors using \nits built in (and hopefully optimized) tensor operations. \nSecond, in your method you sum over each rectangular ROI separatel…"1106 },1107 "can_vote": false,1108 "vote_count": 0,1109 "user_voted": false,1110 "discourse_zendesk_plugin_zendesk_id": null,1111 "discourse_zendesk_plugin_zendesk_url": "https://your-url.zendesk.com/agent/tickets/",1112 "details": {1113 "can_edit": false,1114 "notification_level": 1,1115 "participants": [1116 {1117 "id": 29552,1118 "username": "Absurd",1119 "name": "",1120 "avatar_template": "/user_avatar/discuss.pytorch.org/absurd/{size}/25137_2.png",1121 "post_count": 2,1122 "primary_group_name": null,1123 "flair_name": null,1124 "flair_url": null,1125 "flair_color": null,1126 "flair_bg_color": null,1127 "flair_group_id": null,1128 "trust_level": 11129 },1130 {1131 "id": 18088,1132 "username": "KFrank",1133 "name": "K. Frank",1134 "avatar_template": "/letter_avatar_proxy/v4/letter/k/ecb155/{size}.png",1135 "post_count": 1,1136 "primary_group_name": null,1137 "flair_name": null,1138 "flair_url": null,1139 "flair_color": null,1140 "flair_bg_color": null,1141 "flair_group_id": null,1142 "trust_level": 21143 }1144 ],1145 "created_by": {1146 "id": 29552,1147 "username": "Absurd",1148 "name": "",1149 "avatar_template": "/user_avatar/discuss.pytorch.org/absurd/{size}/25137_2.png"1150 },1151 "last_poster": {1152 "id": 29552,1153 "username": "Absurd",1154 "name": "",1155 "avatar_template": "/user_avatar/discuss.pytorch.org/absurd/{size}/25137_2.png"1156 },1157 "links": [1158 {1159 "url": "https://discuss.pytorch.org/t/an-efficient-way-to-slice-tensors-with-array-indices/77078/2",1160 "title": "An efficient way to slice tensors with array indices?",1161 "internal": true,1162 "attachment": false,1163 "reflection": true,1164 "clicks": 3,1165 "user_id": 3534,1166 "domain": "discuss.pytorch.org",1167 "root_domain": "pytorch.org"1168 },1169 {1170 "url": "https://discuss.pytorch.org/t/how-to-extract-smaller-image-patches-3d/16837/41",1171 "title": "How to extract smaller image patches (3D)?",1172 "internal": true,1173 "attachment": false,1174 "reflection": true,1175 "clicks": 1,1176 "user_id": 3534,1177 "domain": "discuss.pytorch.org",1178 "root_domain": "pytorch.org"1179 }1180 ]1181 },1182 "bookmarks": []1183 },1184 {1185 "post_stream": {1186 "posts": [1187 {1188 "id": 178908,1189 "name": "Dmitry Temnov",1190 "username": "Dmitry_Temnov",1191 "avatar_template": "/user_avatar/discuss.pytorch.org/dmitry_temnov/{size}/22301_2.png",1192 "created_at": "2020-04-03T12:49:09.387Z",1193 "cooked": "<p>Hi!<br>\nI’ experimenting with converting Pytorch models to ONNX using this repository(<a href=\"https://github.com/amdegroot/ssd.pytorch\" rel=\"nofollow noopener\">repo</a>) - It is basically implementation of SSD.</p>\n<p>Tracer argues at this script that computes coordinates for prior boxes.(<a href=\"https://github.com/amdegroot/ssd.pytorch/blob/master/layers/functions/prior_box.py\" rel=\"nofollow noopener\">script</a>)<br>\nSpecifically this code - where tensor is converted into python range:</p>\n<pre><code class=\"lang-auto\"> for i, j in product(range(f), repeat=2):\n f_k = self.image_size / self.steps[k]\n # unit center x,y\n cx = (j + 0.5) / f_k\n cy = (i + 0.5) / f_k\n</code></pre>\n<p>After that the whole process hangs. Any idea what might cause this or suggestions how to rewrite this part?</p>\n<p>Thanks a lot for your help!</p>",1194 "post_number": 1,1195 "post_type": 1,1196 "posts_count": 3,1197 "updated_at": "2020-04-03T12:49:34.760Z",1198 "reply_count": 0,1199 "reply_to_post_number": null,1200 "quote_count": 0,