Anurag1734/cuda-error-resolution-analysis
07
1[2 {3 "post_stream": {4 "posts": [5 {6 "id": 243254,7 "name": "Amit Kayal",8 "username": "amitkayal",9 "avatar_template": "/letter_avatar_proxy/v4/letter/a/dfb087/{size}.png",10 "created_at": "2020-11-11T15:27:28.195Z",11 "cooked": "<p>Hello All,</p>\n<p>I am trying to use LSTMCell to form multiple LSTM layer and not using LSTM. Getting following error and looks like it is due due to nn.utils.rnn.pack_padded_sequence but not able to debug further. Could you please help me on this?</p>\n<p><strong>My network definition is:</strong></p>\n<blockquote>\n<p>class RNN(nn.Module):</p>\n<pre><code>def __init__(self, vocab_size, embedding_dim, hidden_dim, output_dim, n_layers, \n\n bidirectional, dropout, pad_idx):\n\n \n\n super().__init__()\n\n \n\n self.n_layers = n_layers\n\n self.embedding_dim = embedding_dim\n\n self.hidden_dim = hidden_dim\n\n # Number of time steps\n\n self.sequence_len = 3\n\n self.embedding = nn.Embedding(vocab_size, embedding_dim, padding_idx = pad_idx)\n\n # Initialize LSTM Cell for the first layer\n\n self.lstm_cell_layer_1 = nn.LSTMCell(self.embedding_dim, self.hidden_dim)\n\n \n\n # Initialize LSTM Cell for the second layer\n\n self.lstm_cell_layer_2 = nn.LSTMCell(self.hidden_dim, self.hidden_dim)\n\n # Initialize LSTM Cell for the second layer\n\n self.lstm_cell_layer_3 = nn.LSTMCell(self.hidden_dim, self.hidden_dim)\n\n ## we would need to initialize the hidden and cell state for each LSTM layer.\n\n \n\n self.fc = nn.Linear(hidden_dim*2, output_dim)\n\n \n\n self.dropout = nn.Dropout(dropout)\n\n \n\ndef forward(self, text, text_lengths):\n\n \n\n #text = [sent len, batch size]\n\n \n\n embedded = self.dropout(self.embedding(text))\n\n \n\n #embedded = [sent len, batch size, emb dim]\n\n ## Initialisation\n\n \n\n #pack sequence\n\n packed_embedded = nn.utils.rnn.pack_padded_sequence(embedded, text_lengths)\n\n # packed_embedded = embedded\n\n print(f'text_lengths{text_lengths}')\n\n print(f'text.size(0){text.size(0)}')\n\n # out = packed_embedded.view(self.sequence_len, text.size(0), -1)\n\n # Creation of cell state and hidden state for layer 1\n\n hidden_state = torch.zeros(self.embedding_dim, self.hidden_dim)\n\n cell_state = torch.zeros(self.embedding_dim, self.hidden_dim)\n\n # Creation of cell state and hidden state for layer 2\n\n hidden_state_2 = torch.zeros(self.embedding_dim, self.hidden_dim)\n\n cell_state_2 = torch.zeros(self.embedding_dim, self.hidden_dim)\n\n # Creation of cell state and hidden state for layer 3\n\n hidden_state_3 = torch.zeros(self.embedding_dim, self.hidden_dim)\n\n cell_state_3 = torch.zeros(self.embedding_dim, self.hidden_dim)\n\n # Weights initialization\n\n torch.nn.init.xavier_normal_(hidden_state)\n\n torch.nn.init.xavier_normal_(cell_state)\n\n torch.nn.init.xavier_normal_(hidden_state_2)\n\n torch.nn.init.xavier_normal_(cell_state_2)\n\n \n\n torch.nn.init.xavier_normal_(hidden_state_3)\n\n torch.nn.init.xavier_normal_(cell_state_3)\n\n ## End of Initialisation\n\n # Unfolding LSTM\n\n for input_t in range(self.sequence_len):\n\n # print(f'packed_embedded.size(0){len(packed_embedded)}')\n\n hidden_state_1, cell_state_1 = self.lstm_cell_layer_1(packed_embedded,(hidden_state, cell_state))\n\n hidden_state_2, cell_state_2 = self.lstm_cell_2(hidden_state_1, (hidden_state_2, cell_state_2))\n\n hidden_state_3, cell_state_3 = self.lstm_cell_3(hidden_state_2, (hidden_state_3, cell_state_3))\n\n \n\n #unpack sequence\n\n output, output_lengths = nn.utils.rnn.pad_packed_sequence(hidden_state_3)\n\n \n\n hidden = self.dropout(torch.cat((hidden_state_3[-2,:,:], hidden_state_3[-1,:,:]), dim = 1))\n\n \n\n return self.fc(hidden)\n</code></pre>\n</blockquote>\n<p><strong>Error</strong>:</p>\n<blockquote>\n<p>AttributeError Traceback (most recent call last)<br>\n in ()<br>\n7 start_time = time.time()<br>\n8<br>\n----> 9 train_loss, train_acc = train(model, train_iterator, optimizer, criterion)<br>\n10 valid_loss, valid_acc = evaluate(model, valid_iterator, criterion)<br>\n11</p>\n<p>5 frames<br>\n in train(model, iterator, optimizer, criterion)<br>\n13<br>\n14 text_lengths = text_lengths.cpu()<br>\n—> 15 predictions = model(text, text_lengths).squeeze(1)<br>\n16<br>\n17 loss = criterion(predictions, batch.label)</p>\n<p>/usr/local/lib/python3.6/dist-packages/torch/nn/modules/module.py in _call_impl(self, *input, **kwargs)<br>\n725 result = self._slow_forward(*input, **kwargs)<br>\n726 else:<br>\n → 727 result = self.forward(*input, **kwargs)<br>\n728 for hook in itertools.chain(<br>\n729 _global_forward_hooks.values(),</p>\n<p> in forward(self, text, text_lengths)<br>\n76 for input_t in range(self.sequence_len):<br>\n77 # print(f’packed_embedded.size(0){len(packed_embedded)}')<br>\n—> 78 hidden_state_1, cell_state_1 = self.lstm_cell_layer_1(packed_embedded,(hidden_state, cell_state))<br>\n79 hidden_state_2, cell_state_2 = self.lstm_cell_2(hidden_state_1, (hidden_state_2, cell_state_2))<br>\n80 hidden_state_3, cell_state_3 = self.lstm_cell_3(hidden_state_2, (hidden_state_3, cell_state_3))</p>\n<p>/usr/local/lib/python3.6/dist-packages/torch/nn/modules/module.py in _call_impl(self, *input, **kwargs)<br>\n725 result = self._slow_forward(*input, **kwargs)<br>\n726 else:<br>\n → 727 result = self.forward(*input, **kwargs)<br>\n728 for hook in itertools.chain(<br>\n729 _global_forward_hooks.values(),</p>\n<p>/usr/local/lib/python3.6/dist-packages/torch/nn/modules/rnn.py in forward(self, input, hx)<br>\n963<br>\n964 def forward(self, input: Tensor, hx: Optional[Tuple[Tensor, Tensor]] = None) → Tuple[Tensor, Tensor]:<br>\n → 965 self.check_forward_input(input)<br>\n966 if hx is None:<br>\n967 zeros = torch.zeros(input.size(0), self.hidden_size, dtype=input.dtype, device=input.device)</p>\n<p>/usr/local/lib/python3.6/dist-packages/torch/nn/modules/rnn.py in check_forward_input(self, input)<br>\n788<br>\n789 def check_forward_input(self, input: Tensor) → None:<br>\n → 790 if input.size(1) != self.input_size:<br>\n791 raise RuntimeError(<br>\n792 “input has inconsistent input_size: got {}, expected {}”.format(</p>\n<p>AttributeError: ‘PackedSequence’ object has no attribute ‘size’</p>\n</blockquote>",12 "post_number": 1,13 "post_type": 1,14 "posts_count": 1,15 "updated_at": "2020-11-11T15:27:28.195Z",16 "reply_count": 0,17 "reply_to_post_number": null,18 "quote_count": 0,19 "incoming_link_count": 1560,20 "reads": 39,21 "readers_count": 38,22 "score": 7797.8,23 "yours": false,24 "topic_id": 102428,25 "topic_slug": "packedsequence-object-has-no-attribute-size-error-with-lstmcell",26 "display_username": "Amit Kayal",27 "primary_group_name": null,28 "flair_name": null,29 "flair_url": null,30 "flair_bg_color": null,31 "flair_color": null,32 "flair_group_id": null,33 "badges_granted": [],34 "version": 1,35 "can_edit": false,36 "can_delete": false,37 "can_recover": false,38 "can_see_hidden_post": false,39 "can_wiki": false,40 "read": true,41 "user_title": null,42 "bookmarked": false,43 "actions_summary": [],44 "moderator": false,45 "admin": false,46 "staff": false,47 "user_id": 31764,48 "hidden": false,49 "trust_level": 1,50 "deleted_at": null,51 "user_deleted": false,52 "edit_reason": null,53 "can_view_edit_history": true,54 "wiki": false,55 "post_url": "/t/packedsequence-object-has-no-attribute-size-error-with-lstmcell/102428/1",56 "can_accept_answer": false,57 "can_unaccept_answer": false,58 "accepted_answer": false,59 "topic_accepted_answer": null,60 "can_vote": false61 }62 ],63 "stream": [64 24325465 ]66 },67 "timeline_lookup": [68 [69 1,70 180971 ]72 ],73 "suggested_topics": [74 {75 "fancy_title": "When I install torch==2.6.0 with whl/cu126, none of cuda dependencies get installed. I have cu126 in the environment already. Is this expected?",76 "id": 218014,77 "title": "When I install torch==2.6.0 with whl/cu126, none of cuda dependencies get installed. I have cu126 in the environment already. Is this expected?",78 "slug": "when-i-install-torch-2-6-0-with-whl-cu126-none-of-cuda-dependencies-get-installed-i-have-cu126-in-the-environment-already-is-this-expected",79 "posts_count": 6,80 "reply_count": 3,81 "highest_post_number": 6,82 "image_url": "https://discuss.pytorch.org/uploads/default/optimized/3X/0/e/0ed4e9db0ae58fbdcaac67a5d5a4183acf24f67f_2_1024x416.png",83 "created_at": "2025-03-19T07:50:20.096Z",84 "last_posted_at": "2025-03-19T15:44:20.367Z",85 "bumped": true,86 "bumped_at": "2025-03-19T15:44:20.367Z",87 "archetype": "regular",88 "unseen": false,89 "pinned": false,90 "unpinned": null,91 "visible": true,92 "closed": false,93 "archived": false,94 "bookmarked": null,95 "liked": null,96 "tags_descriptions": {},97 "like_count": 2,98 "views": 3932,99 "category_id": 1,100 "featured_link": null,101 "has_accepted_answer": false,102 "posters": [103 {104 "extras": null,105 "description": "Original Poster",106 "user": {107 "id": 83355,108 "username": "kofuji",109 "name": null,110 "avatar_template": "/letter_avatar_proxy/v4/letter/k/91b2a8/{size}.png",111 "trust_level": 0112 }113 },114 {115 "extras": null,116 "description": "Frequent Poster",117 "user": {118 "id": 83362,119 "username": "twoudsma",120 "name": "",121 "avatar_template": "/user_avatar/discuss.pytorch.org/twoudsma/{size}/77573_2.png",122 "trust_level": 2123 }124 },125 {126 "extras": "latest",127 "description": "Most Recent Poster",128 "user": {129 "id": 3534,130 "username": "ptrblck",131 "name": "",132 "avatar_template": "/user_avatar/discuss.pytorch.org/ptrblck/{size}/1823_2.png",133 "admin": true,134 "moderator": true,135 "trust_level": 2136 }137 }138 ]139 },140 {141 "fancy_title": "RuntimeError: CUDA error: invalid argument",142 "id": 214062,143 "title": "RuntimeError: CUDA error: invalid argument",144 "slug": "runtimeerror-cuda-error-invalid-argument",145 "posts_count": 9,146 "reply_count": 2,147 "highest_post_number": 9,148 "image_url": null,149 "created_at": "2024-12-10T15:39:59.445Z",150 "last_posted_at": "2025-08-02T10:44:13.170Z",151 "bumped": true,152 "bumped_at": "2025-08-02T10:44:13.170Z",153 "archetype": "regular",154 "unseen": false,155 "pinned": false,156 "unpinned": null,157 "visible": true,158 "closed": false,159 "archived": false,160 "bookmarked": null,161 "liked": null,162 "tags_descriptions": {},163 "like_count": 2,164 "views": 1207,165 "category_id": 1,166 "featured_link": null,167 "has_accepted_answer": false,168 "posters": [169 {170 "extras": null,171 "description": "Original Poster",172 "user": {173 "id": 81431,174 "username": "Andymulb",175 "name": "Andreas",176 "avatar_template": "/user_avatar/discuss.pytorch.org/andymulb/{size}/74455_2.png",177 "trust_level": 1178 }179 },180 {181 "extras": null,182 "description": "Frequent Poster",183 "user": {184 "id": 81449,185 "username": "NailikLN",186 "name": "kilian le nezet",187 "avatar_template": "/user_avatar/discuss.pytorch.org/nailikln/{size}/74471_2.png",188 "trust_level": 0189 }190 },191 {192 "extras": null,193 "description": "Frequent Poster",194 "user": {195 "id": 81492,196 "username": "Doruk_Sonmez",197 "name": "Doruk Sönmez",198 "avatar_template": "/user_avatar/discuss.pytorch.org/doruk_sonmez/{size}/74506_2.png",199 "trust_level": 1200 }201 },202 {203 "extras": null,204 "description": "Frequent Poster",205 "user": {206 "id": 80375,207 "username": "vixp24",208 "name": "",209 "avatar_template": "/user_avatar/discuss.pytorch.org/vixp24/{size}/73472_2.png",210 "trust_level": 1211 }212 },213 {214 "extras": "latest",215 "description": "Most Recent Poster",216 "user": {217 "id": 82530,218 "username": "J4Q8",219 "name": "Jakub Łucki",220 "avatar_template": "/user_avatar/discuss.pytorch.org/j4q8/{size}/75512_2.png",221 "trust_level": 1222 }223 }224 ]225 },226 {227 "fancy_title": "Correct SDPA’s attn_mask for Self-attention",228 "id": 213077,229 "title": "Correct SDPA's attn_mask for Self-attention",230 "slug": "correct-sdpas-attn-mask-for-self-attention",231 "posts_count": 1,232 "reply_count": 0,233 "highest_post_number": 1,234 "image_url": null,235 "created_at": "2024-11-17T14:32:43.904Z",236 "last_posted_at": "2024-11-17T14:32:43.958Z",237 "bumped": true,238 "bumped_at": "2024-11-17T14:32:43.958Z",239 "archetype": "regular",240 "unseen": false,241 "pinned": false,242 "unpinned": null,243 "visible": true,244 "closed": false,245 "archived": false,246 "bookmarked": null,247 "liked": null,248 "tags_descriptions": {},249 "like_count": 0,250 "views": 204,251 "category_id": 1,252 "featured_link": null,253 "has_accepted_answer": false,254 "posters": [255 {256 "extras": "latest single",257 "description": "Original Poster, Most Recent Poster",258 "user": {259 "id": 80966,260 "username": "SotoJnthn",261 "name": "Jonathan",262 "avatar_template": "/user_avatar/discuss.pytorch.org/sotojnthn/{size}/74047_2.png",263 "trust_level": 1264 }265 }266 ]267 },268 {269 "fancy_title": "Seeking advice on computing backprop step by chunks",270 "id": 212924,271 "title": "Seeking advice on computing backprop step by chunks",272 "slug": "seeking-advice-on-computing-backprop-step-by-chunks",273 "posts_count": 2,274 "reply_count": 0,275 "highest_post_number": 2,276 "image_url": null,277 "created_at": "2024-11-13T12:28:34.725Z",278 "last_posted_at": "2024-11-14T14:09:04.532Z",279 "bumped": true,280 "bumped_at": "2024-11-14T14:09:04.532Z",281 "archetype": "regular",282 "unseen": false,283 "pinned": false,284 "unpinned": null,285 "visible": true,286 "closed": false,287 "archived": false,288 "bookmarked": null,289 "liked": null,290 "tags_descriptions": {},291 "like_count": 0,292 "views": 36,293 "category_id": 1,294 "featured_link": null,295 "has_accepted_answer": true,296 "posters": [297 {298 "extras": "latest single",299 "description": "Original Poster, Most Recent Poster, Accepted Answer",300 "user": {301 "id": 80885,302 "username": "meditans",303 "name": "",304 "avatar_template": "/user_avatar/discuss.pytorch.org/meditans/{size}/73976_2.png",305 "trust_level": 1306 }307 }308 ]309 },310 {311 "fancy_title": "There is not libgomp-a34b3233.so.1 in torch/lib when I build the torch based on the V2.4.0 with USE_CUDA=0",312 "id": 218511,313 "title": "There is not libgomp-a34b3233.so.1 in torch/lib when I build the torch based on the V2.4.0 with USE_CUDA=0",314 "slug": "there-is-not-libgomp-a34b3233-so-1-in-torch-lib-when-i-build-the-torch-based-on-the-v2-4-0-with-use-cuda-0",315 "posts_count": 1,316 "reply_count": 0,317 "highest_post_number": 1,318 "image_url": null,319 "created_at": "2025-04-02T03:00:40.547Z",320 "last_posted_at": "2025-04-02T03:00:40.588Z",321 "bumped": true,322 "bumped_at": "2025-04-02T03:00:40.588Z",323 "archetype": "regular",324 "unseen": false,325 "pinned": false,326 "unpinned": null,327 "visible": true,328 "closed": false,329 "archived": false,330 "bookmarked": null,331 "liked": null,332 "tags_descriptions": {},333 "like_count": 0,334 "views": 35,335 "category_id": 1,336 "featured_link": null,337 "has_accepted_answer": false,338 "posters": [339 {340 "extras": "latest single",341 "description": "Original Poster, Most Recent Poster",342 "user": {343 "id": 83584,344 "username": "edwinli",345 "name": "edwinli",346 "avatar_template": "/letter_avatar_proxy/v4/letter/e/ecae2f/{size}.png",347 "trust_level": 0348 }349 }350 ]351 }352 ],353 "tags_descriptions": {},354 "fancy_title": "‘PackedSequence’ object has no attribute ‘size’ error with LSTMCell",355 "id": 102428,356 "title": "'PackedSequence' object has no attribute 'size' error with LSTMCell",357 "posts_count": 1,358 "created_at": "2020-11-11T15:27:28.114Z",359 "views": 2365,360 "reply_count": 0,361 "like_count": 0,362 "last_posted_at": "2020-11-11T15:27:28.195Z",363 "visible": true,364 "closed": false,365 "archived": false,366 "has_summary": false,367 "archetype": "regular",368 "slug": "packedsequence-object-has-no-attribute-size-error-with-lstmcell",369 "category_id": 1,370 "word_count": 683,371 "deleted_at": null,372 "user_id": 31764,373 "featured_link": null,374 "pinned_globally": false,375 "pinned_at": null,376 "pinned_until": null,377 "image_url": null,378 "slow_mode_seconds": 0,379 "draft": null,380 "draft_key": "topic_102428",381 "draft_sequence": null,382 "unpinned": null,383 "pinned": false,384 "current_post_number": 1,385 "highest_post_number": 1,386 "deleted_by": null,387 "actions_summary": [388 {389 "id": 4,390 "count": 0,391 "hidden": false,392 "can_act": false393 },394 {395 "id": 8,396 "count": 0,397 "hidden": false,398 "can_act": false399 },400 {401 "id": 10,402 "count": 0,403 "hidden": false,404 "can_act": false405 },406 {407 "id": 7,408 "count": 0,409 "hidden": false,410 "can_act": false411 }412 ],413 "chunk_size": 20,414 "bookmarked": false,415 "topic_timer": null,416 "message_bus_last_id": 0,417 "participant_count": 1,418 "show_read_indicator": false,419 "thumbnails": null,420 "slow_mode_enabled_until": null,421 "can_vote": false,422 "vote_count": 0,423 "user_voted": false,424 "discourse_zendesk_plugin_zendesk_id": null,425 "discourse_zendesk_plugin_zendesk_url": "https://your-url.zendesk.com/agent/tickets/",426 "details": {427 "can_edit": false,428 "notification_level": 1,429 "participants": [430 {431 "id": 31764,432 "username": "amitkayal",433 "name": "Amit Kayal",434 "avatar_template": "/letter_avatar_proxy/v4/letter/a/dfb087/{size}.png",435 "post_count": 1,436 "primary_group_name": null,437 "flair_name": null,438 "flair_url": null,439 "flair_color": null,440 "flair_bg_color": null,441 "flair_group_id": null,442 "trust_level": 1443 }444 ],445 "created_by": {446 "id": 31764,447 "username": "amitkayal",448 "name": "Amit Kayal",449 "avatar_template": "/letter_avatar_proxy/v4/letter/a/dfb087/{size}.png"450 },451 "last_poster": {452 "id": 31764,453 "username": "amitkayal",454 "name": "Amit Kayal",455 "avatar_template": "/letter_avatar_proxy/v4/letter/a/dfb087/{size}.png"456 }457 },458 "bookmarks": []459 },460 {461 "post_stream": {462 "posts": [463 {464 "id": 79502,465 "name": "Sukanya Kudi",466 "username": "sukanya_kudi",467 "avatar_template": "/letter_avatar_proxy/v4/letter/s/f05b48/{size}.png",468 "created_at": "2018-12-03T07:28:15.928Z",469 "cooked": "<p>Hi,<br>\nIs there a reduce_max equivalent of TF(can take multiple dims as input)?<br>\nthe torch.max operation allows only one dim as input.<br>\nI require this for implementation of RMAC. If there are any other alternatives please suggest.</p>",470 "post_number": 1,471 "post_type": 1,472 "posts_count": 2,473 "updated_at": "2018-12-03T07:34:38.387Z",474 "reply_count": 0,475 "reply_to_post_number": null,476 "quote_count": 0,477 "incoming_link_count": 1702,478 "reads": 63,479 "readers_count": 62,480 "score": 8554.6,481 "yours": false,482 "topic_id": 31130,483 "topic_slug": "pytorch-equivalent-of-tf-reduce-max",484 "display_username": "Sukanya Kudi",485 "primary_group_name": null,486 "flair_name": null,487 "flair_url": null,488 "flair_bg_color": null,489 "flair_color": null,490 "flair_group_id": null,491 "badges_granted": [],492 "version": 2,493 "can_edit": false,494 "can_delete": false,495 "can_recover": false,496 "can_see_hidden_post": false,497 "can_wiki": false,498 "read": true,499 "user_title": null,500 "bookmarked": false,501 "actions_summary": [502 {503 "id": 2,504 "count": 1505 }506 ],507 "moderator": false,508 "admin": false,509 "staff": false,510 "user_id": 13502,511 "hidden": false,512 "trust_level": 0,513 "deleted_at": null,514 "user_deleted": false,515 "edit_reason": null,516 "can_view_edit_history": true,517 "wiki": false,518 "post_url": "/t/pytorch-equivalent-of-tf-reduce-max/31130/1",519 "can_accept_answer": false,520 "can_unaccept_answer": false,521 "accepted_answer": false,522 "topic_accepted_answer": null,523 "can_vote": false524 },525 {526 "id": 79522,527 "name": "Krishna Vishal V",528 "username": "krishnavishalv",529 "avatar_template": "/user_avatar/discuss.pytorch.org/krishnavishalv/{size}/1756_2.png",530 "created_at": "2018-12-03T09:42:50.079Z",531 "cooked": "<p>Is this what you are looking for ?</p>\n<aside class=\"onebox githubblob\">\n <header class=\"source\">\n <a href=\"https://github.com/filipradenovic/cnnimageretrieval-pytorch/blob/master/cirtorch/layers/functional.py#L22\" target=\"_blank\" rel=\"nofollow noopener\">github.com</a>\n </header>\n <article class=\"onebox-body\">\n <h4><a href=\"https://github.com/filipradenovic/cnnimageretrieval-pytorch/blob/master/cirtorch/layers/functional.py#L22\" target=\"_blank\" rel=\"nofollow noopener\">filipradenovic/cnnimageretrieval-pytorch/blob/master/cirtorch/layers/functional.py#L22</a></h4>\n<pre class=\"onebox\"><code class=\"lang-py\"><ol class=\"start lines\" start=\"12\" style=\"counter-reset: li-counter 11 ;\">\n<li># return F.adaptive_max_pool2d(x, (1,1)) # alternative</li>\n<li>\n</li>\n<li>def spoc(x):</li>\n<li>return F.avg_pool2d(x, (x.size(-2), x.size(-1)))</li>\n<li># return F.adaptive_avg_pool2d(x, (1,1)) # alternative</li>\n<li>\n</li>\n<li>def gem(x, p=3, eps=1e-6):</li>\n<li>return F.avg_pool2d(x.clamp(min=eps).pow(p), (x.size(-2), x.size(-1))).pow(1./p)</li>\n<li># return F.lp_pool2d(F.threshold(x, eps, eps), p, (x.size(-2), x.size(-1))) # alternative</li>\n<li>\n</li>\n<li class=\"selected\">def rmac(x, L=3, eps=1e-6):</li>\n<li>ovr = 0.4 # desired overlap of neighboring regions</li>\n<li>steps = torch.Tensor([2, 3, 4, 5, 6, 7]) # possible regions for the long dimension</li>\n<li>\n</li>\n<li>W = x.size(3)</li>\n<li>H = x.size(2)</li>\n<li>\n</li>\n<li>w = min(W, H)</li>\n<li>w2 = math.floor(w/2.0 - 1)</li>\n<li>\n</li>\n<li>b = (max(H, W)-w)/(steps-1)</li>\n</ol></code></pre>\n\n\n </article>\n <div class=\"onebox-metadata\">\n \n \n </div>\n <div style=\"clear: both\"></div>\n</aside>\n",532 "post_number": 2,533 "post_type": 1,534 "posts_count": 2,535 "updated_at": "2018-12-03T09:42:50.079Z",536 "reply_count": 0,537 "reply_to_post_number": null,538 "quote_count": 0,539 "incoming_link_count": 15,540 "reads": 63,541 "readers_count": 62,542 "score": 87.6,543 "yours": false,544 "topic_id": 31130,545 "topic_slug": "pytorch-equivalent-of-tf-reduce-max",546 "display_username": "Krishna Vishal V",547 "primary_group_name": null,548 "flair_name": null,549 "flair_url": null,550 "flair_bg_color": null,551 "flair_color": null,552 "flair_group_id": null,553 "badges_granted": [],554 "version": 1,555 "can_edit": false,556 "can_delete": false,557 "can_recover": false,558 "can_see_hidden_post": false,559 "can_wiki": false,560 "link_counts": [561 {562 "url": "https://github.com/filipradenovic/cnnimageretrieval-pytorch/blob/master/cirtorch/layers/functional.py#L22",563 "internal": false,564 "reflection": false,565 "title": "cnnimageretrieval-pytorch/functional.py at master · filipradenovic/cnnimageretrieval-pytorch · GitHub",566 "clicks": 34567 }568 ],569 "read": true,570 "user_title": null,571 "bookmarked": false,572 "actions_summary": [],573 "moderator": false,574 "admin": false,575 "staff": false,576 "user_id": 3568,577 "hidden": false,578 "trust_level": 2,579 "deleted_at": null,580 "user_deleted": false,581 "edit_reason": null,582 "can_view_edit_history": true,583 "wiki": false,584 "post_url": "/t/pytorch-equivalent-of-tf-reduce-max/31130/2",585 "can_accept_answer": false,586 "can_unaccept_answer": false,587 "accepted_answer": false,588 "topic_accepted_answer": null589 }590 ],591 "stream": [592 79502,593 79522594 ]595 },596 "timeline_lookup": [597 [598 1,599 2519600 ]601 ],602 "suggested_topics": [603 {604 "fancy_title": "Torchvision sm_120 failed installed",605 "id": 217725,606 "title": "Torchvision sm_120 failed installed",607 "slug": "torchvision-sm-120-failed-installed",608 "posts_count": 2,609 "reply_count": 0,610 "highest_post_number": 2,611 "image_url": null,612 "created_at": "2025-03-12T04:29:40.562Z",613 "last_posted_at": "2025-03-12T13:14:27.243Z",614 "bumped": true,615 "bumped_at": "2025-03-12T13:14:27.243Z",616 "archetype": "regular",617 "unseen": false,618 "pinned": false,619 "unpinned": null,620 "visible": true,621 "closed": false,622 "archived": false,623 "bookmarked": null,624 "liked": null,625 "tags_descriptions": {},626 "like_count": 0,627 "views": 305,628 "category_id": 5,629 "featured_link": null,630 "has_accepted_answer": false,631 "posters": [632 {633 "extras": null,634 "description": "Original Poster",635 "user": {636 "id": 83206,637 "username": "Desire6666",638 "name": null,639 "avatar_template": "/letter_avatar_proxy/v4/letter/d/e36b37/{size}.png",640 "trust_level": 0641 }642 },643 {644 "extras": "latest",645 "description": "Most Recent Poster",646 "user": {647 "id": 3534,648 "username": "ptrblck",649 "name": "",650 "avatar_template": "/user_avatar/discuss.pytorch.org/ptrblck/{size}/1823_2.png",651 "admin": true,652 "moderator": true,653 "trust_level": 2654 }655 }656 ]657 },658 {659 "fancy_title": "When using conv3d, a large amount of video memory is occupied",660 "id": 213294,661 "title": "When using conv3d, a large amount of video memory is occupied",662 "slug": "when-using-conv3d-a-large-amount-of-video-memory-is-occupied",663 "posts_count": 10,664 "reply_count": 8,665 "highest_post_number": 10,666 "image_url": null,667 "created_at": "2024-11-22T06:19:47.771Z",668 "last_posted_at": "2024-11-26T16:53:45.639Z",669 "bumped": true,670 "bumped_at": "2024-11-26T16:53:45.639Z",671 "archetype": "regular",672 "unseen": false,673 "pinned": false,674 "unpinned": null,675 "visible": true,676 "closed": false,677 "archived": false,678 "bookmarked": null,679 "liked": null,680 "tags_descriptions": {},681 "like_count": 0,682 "views": 186,683 "category_id": 5,684 "featured_link": null,685 "has_accepted_answer": false,686 "posters": [687 {688 "extras": null,689 "description": "Original Poster",690 "user": {691 "id": 71147,692 "username": "yinjun1131",693 "name": "yinjun",694 "avatar_template": "/letter_avatar_proxy/v4/letter/y/439d5e/{size}.png",695 "trust_level": 1696 }697 },698 {699 "extras": "latest",700 "description": "Most Recent Poster",701 "user": {702 "id": 3534,703 "username": "ptrblck",704 "name": "",705 "avatar_template": "/user_avatar/discuss.pytorch.org/ptrblck/{size}/1823_2.png",706 "admin": true,707 "moderator": true,708 "trust_level": 2709 }710 }711 ]712 },713 {714 "fancy_title": "Encoder-Bottleneck-Decoder Unet Based Semantic Segmentation",715 "id": 214096,716 "title": "Encoder-Bottleneck-Decoder Unet Based Semantic Segmentation",717 "slug": "encoder-bottleneck-decoder-unet-based-semantic-segmentation",718 "posts_count": 3,719 "reply_count": 1,720 "highest_post_number": 3,721 "image_url": null,722 "created_at": "2024-12-11T10:13:08.305Z",723 "last_posted_at": "2024-12-18T06:52:53.050Z",724 "bumped": true,725 "bumped_at": "2024-12-18T06:52:53.050Z",726 "archetype": "regular",727 "unseen": false,728 "pinned": false,729 "unpinned": null,730 "visible": true,731 "closed": false,732 "archived": false,733 "bookmarked": null,734 "liked": null,735 "tags_descriptions": {},736 "like_count": 1,737 "views": 314,738 "category_id": 5,739 "featured_link": null,740 "has_accepted_answer": false,741 "posters": [742 {743 "extras": "latest",744 "description": "Original Poster, Most Recent Poster",745 "user": {746 "id": 81422,747 "username": "Idrees11",748 "name": "Idrees Bhat",749 "avatar_template": "/user_avatar/discuss.pytorch.org/idrees11/{size}/74448_2.png",750 "trust_level": 1751 }752 },753 {754 "extras": null,755 "description": "Frequent Poster",756 "user": {757 "id": 18088,758 "username": "KFrank",759 "name": "K. Frank",760 "avatar_template": "/letter_avatar_proxy/v4/letter/k/ecb155/{size}.png",761 "trust_level": 2762 }763 }764 ]765 },766 {767 "fancy_title": "Import torchvision fails on NVIDIA jetson orin nano",768 "id": 217199,769 "title": "Import torchvision fails on NVIDIA jetson orin nano",770 "slug": "import-torchvision-fails-on-nvidia-jetson-orin-nano",771 "posts_count": 3,772 "reply_count": 1,773 "highest_post_number": 4,774 "image_url": null,775 "created_at": "2025-02-26T18:34:09.423Z",776 "last_posted_at": "2025-03-18T12:34:20.272Z",777 "bumped": true,778 "bumped_at": "2025-03-18T12:34:20.272Z",779 "archetype": "regular",780 "unseen": false,781 "pinned": false,782 "unpinned": null,783 "visible": true,784 "closed": false,785 "archived": false,786 "bookmarked": null,787 "liked": null,788 "tags_descriptions": {},789 "like_count": 0,790 "views": 222,791 "category_id": 5,792 "featured_link": null,793 "has_accepted_answer": true,794 "posters": [795 {796 "extras": "latest",797 "description": "Original Poster, Most Recent Poster",798 "user": {799 "id": 35490,800 "username": "cadip92",801 "name": "Adwait Chandorkar",802 "avatar_template": "/user_avatar/discuss.pytorch.org/cadip92/{size}/27771_2.png",803 "trust_level": 1804 }805 },806 {807 "extras": null,808 "description": "Frequent Poster, Accepted Answer",809 "user": {810 "id": 77908,811 "username": "mycul",812 "name": "",813 "avatar_template": "/user_avatar/discuss.pytorch.org/mycul/{size}/72394_2.png",814 "trust_level": 2815 }816 }817 ]818 },819 {820 "fancy_title": "What is the use of y_range feature",821 "id": 218346,822 "title": "What is the use of y_range feature",823 "slug": "what-is-the-use-of-y-range-feature",824 "posts_count": 2,825 "reply_count": 0,826 "highest_post_number": 2,827 "image_url": null,828 "created_at": "2025-03-27T21:07:12.454Z",829 "last_posted_at": "2025-03-27T23:05:50.254Z",830 "bumped": true,831 "bumped_at": "2025-03-27T23:05:50.254Z",832 "archetype": "regular",833 "unseen": false,834 "pinned": false,835 "unpinned": null,836 "visible": true,837 "closed": false,838 "archived": false,839 "bookmarked": null,840 "liked": null,841 "tags_descriptions": {},842 "like_count": 0,843 "views": 29,844 "category_id": 5,845 "featured_link": null,846 "has_accepted_answer": false,847 "posters": [848 {849 "extras": null,850 "description": "Original Poster",851 "user": {852 "id": 83503,853 "username": "p-karmelita",854 "name": "Piotr Karmelita",855 "avatar_template": "/user_avatar/discuss.pytorch.org/p-karmelita/{size}/76371_2.png",856 "trust_level": 0857 }858 },859 {860 "extras": "latest",861 "description": "Most Recent Poster",862 "user": {863 "id": 3534,864 "username": "ptrblck",865 "name": "",866 "avatar_template": "/user_avatar/discuss.pytorch.org/ptrblck/{size}/1823_2.png",867 "admin": true,868 "moderator": true,869 "trust_level": 2870 }871 }872 ]873 }874 ],875 "tags_descriptions": {},876 "fancy_title": "Pytorch equivalent of TF reduce_max",877 "id": 31130,878 "title": "Pytorch equivalent of TF reduce_max",879 "posts_count": 2,880 "created_at": "2018-12-03T07:28:15.874Z",881 "views": 3141,882 "reply_count": 0,883 "like_count": 1,884 "last_posted_at": "2018-12-03T09:42:50.079Z",885 "visible": true,886 "closed": false,887 "archived": false,888 "has_summary": false,889 "archetype": "regular",890 "slug": "pytorch-equivalent-of-tf-reduce-max",891 "category_id": 5,892 "word_count": 72,893 "deleted_at": null,894 "user_id": 13502,895 "featured_link": null,896 "pinned_globally": false,897 "pinned_at": null,898 "pinned_until": null,899 "image_url": null,900 "slow_mode_seconds": 0,901 "draft": null,902 "draft_key": "topic_31130",903 "draft_sequence": null,904 "unpinned": null,905 "pinned": false,906 "current_post_number": 1,907 "highest_post_number": 2,908 "deleted_by": null,909 "actions_summary": [910 {911 "id": 4,912 "count": 0,913 "hidden": false,914 "can_act": false915 },916 {917 "id": 8,918 "count": 0,919 "hidden": false,920 "can_act": false921 },922 {923 "id": 10,924 "count": 0,925 "hidden": false,926 "can_act": false927 },928 {929 "id": 7,930 "count": 0,931 "hidden": false,932 "can_act": false933 }934 ],935 "chunk_size": 20,936 "bookmarked": false,937 "topic_timer": null,938 "message_bus_last_id": 0,939 "participant_count": 2,940 "show_read_indicator": false,941 "thumbnails": null,942 "slow_mode_enabled_until": null,943 "can_vote": false,944 "vote_count": 0,945 "user_voted": false,946 "discourse_zendesk_plugin_zendesk_id": null,947 "discourse_zendesk_plugin_zendesk_url": "https://your-url.zendesk.com/agent/tickets/",948 "details": {949 "can_edit": false,950 "notification_level": 1,951 "participants": [952 {953 "id": 3568,954 "username": "krishnavishalv",955 "name": "Krishna Vishal V",956 "avatar_template": "/user_avatar/discuss.pytorch.org/krishnavishalv/{size}/1756_2.png",957 "post_count": 1,958 "primary_group_name": null,959 "flair_name": null,960 "flair_url": null,961 "flair_color": null,962 "flair_bg_color": null,963 "flair_group_id": null,964 "trust_level": 2965 },966 {967 "id": 13502,968 "username": "sukanya_kudi",969 "name": "Sukanya Kudi",970 "avatar_template": "/letter_avatar_proxy/v4/letter/s/f05b48/{size}.png",971 "post_count": 1,972 "primary_group_name": null,973 "flair_name": null,974 "flair_url": null,975 "flair_color": null,976 "flair_bg_color": null,977 "flair_group_id": null,978 "trust_level": 0979 }980 ],981 "created_by": {982 "id": 13502,983 "username": "sukanya_kudi",984 "name": "Sukanya Kudi",985 "avatar_template": "/letter_avatar_proxy/v4/letter/s/f05b48/{size}.png"986 },987 "last_poster": {988 "id": 3568,989 "username": "krishnavishalv",990 "name": "Krishna Vishal V",991 "avatar_template": "/user_avatar/discuss.pytorch.org/krishnavishalv/{size}/1756_2.png"992 },993 "links": [994 {995 "url": "https://github.com/filipradenovic/cnnimageretrieval-pytorch/blob/master/cirtorch/layers/functional.py#L22",996 "title": "cnnimageretrieval-pytorch/functional.py at master · filipradenovic/cnnimageretrieval-pytorch · GitHub",997 "internal": false,998 "attachment": false,999 "reflection": false,1000 "clicks": 34,1001 "user_id": 3568,1002 "domain": "github.com",1003 "root_domain": "github.com"1004 }1005 ]1006 },1007 "bookmarks": []1008 },1009 {1010 "post_stream": {1011 "posts": [1012 {1013 "id": 243237,1014 "name": "Shamoon Siddiqui",1015 "username": "shamoons",1016 "avatar_template": "/user_avatar/discuss.pytorch.org/shamoons/{size}/21191_2.png",1017 "created_at": "2020-11-11T14:17:10.854Z",1018 "cooked": "<p>Specifically, I see <a href=\"https://pytorch.org/docs/stable/generated/torch.nn.Transformer.html#torch.nn.Transformer\" rel=\"noopener nofollow ugc\">here in the docs</a>:</p>\n<pre><code class=\"lang-auto\">>>> transformer_model = nn.Transformer(nhead=16, num_encoder_layers=12)\n>>> src = torch.rand((10, 32, 512))\n>>> tgt = torch.rand((20, 32, 512))\n>>> out = transformer_model(src, tgt)\n</code></pre>\n<p>I’m unsure what the <code>tgt</code> is. The docs say <code>tgt – the sequence to the decoder (required).</code>. But I’m not passing a sequence to the decoder. I want the decoder to give me an output, don’t I?</p>",1019 "post_number": 1,1020 "post_type": 1,1021 "posts_count": 1,1022 "updated_at": "2020-11-11T14:37:40.698Z",1023 "reply_count": 0,1024 "reply_to_post_number": null,1025 "quote_count": 0,1026 "incoming_link_count": 75,1027 "reads": 4,1028 "readers_count": 3,1029 "score": 375.8,1030 "yours": false,1031 "topic_id": 102423,1032 "topic_slug": "what-is-the-tgt-in-the-torch-nn-transformer",1033 "display_username": "Shamoon Siddiqui",1034 "primary_group_name": null,1035 "flair_name": null,1036 "flair_url": null,1037 "flair_bg_color": null,1038 "flair_color": null,1039 "flair_group_id": null,1040 "badges_granted": [],1041 "version": 2,1042 "can_edit": false,1043 "can_delete": false,1044 "can_recover": false,1045 "can_see_hidden_post": false,1046 "can_wiki": false,1047 "link_counts": [1048 {1049 "url": "https://pytorch.org/docs/stable/generated/torch.nn.Transformer.html#torch.nn.Transformer",1050 "internal": false,1051 "reflection": false,1052 "title": "Transformer — PyTorch 1.7.0 documentation",1053 "clicks": 31054 }1055 ],1056 "read": true,1057 "user_title": null,1058 "bookmarked": false,1059 "actions_summary": [],1060 "moderator": false,1061 "admin": false,1062 "staff": false,1063 "user_id": 28325,1064 "hidden": false,1065 "trust_level": 2,1066 "deleted_at": null,1067 "user_deleted": false,1068 "edit_reason": null,1069 "can_view_edit_history": true,1070 "wiki": false,1071 "post_url": "/t/what-is-the-tgt-in-the-torch-nn-transformer/102423/1",1072 "can_accept_answer": false,1073 "can_unaccept_answer": false,1074 "accepted_answer": false,1075 "topic_accepted_answer": null,1076 "can_vote": false1077 }1078 ],1079 "stream": [1080 2432371081 ]1082 },1083 "timeline_lookup": [1084 [1085 1,1086 18091087 ]1088 ],1089 "suggested_topics": [1090 {1091 "fancy_title": "What is the difference between the CUDA API and CUDA HW lines in the Nsight Systems GUI?",1092 "id": 216417,1093 "title": "What is the difference between the CUDA API and CUDA HW lines in the Nsight Systems GUI?",1094 "slug": "what-is-the-difference-between-the-cuda-api-and-cuda-hw-lines-in-the-nsight-systems-gui",1095 "posts_count": 1,1096 "reply_count": 0,1097 "highest_post_number": 1,1098 "image_url": "https://discuss.pytorch.org/uploads/default/optimized/3X/2/7/27cbe5e88c4ed57d27e04ca70978791ed7307f48_2_1024x368.png",1099 "created_at": "2025-02-09T04:09:00.331Z",1100 "last_posted_at": "2025-02-09T04:09:00.373Z",1101 "bumped": true,1102 "bumped_at": "2025-02-09T04:09:00.373Z",1103 "archetype": "regular",1104 "unseen": false,1105 "pinned": false,1106 "unpinned": null,1107 "visible": true,1108 "closed": false,1109 "archived": false,1110 "bookmarked": null,1111 "liked": null,1112 "tags_descriptions": {},1113 "like_count": 0,1114 "views": 39,1115 "category_id": 1,1116 "featured_link": null,1117 "has_accepted_answer": false,1118 "posters": [1119 {1120 "extras": "latest single",1121 "description": "Original Poster, Most Recent Poster",1122 "user": {1123 "id": 79055,1124 "username": "hello_e",1125 "name": "hhhh",1126 "avatar_template": "/letter_avatar_proxy/v4/letter/h/f9ae1b/{size}.png",1127 "trust_level": 11128 }1129 }1130 ]1131 },1132 {1133 "fancy_title": "`torch.linalg.svd` uses `cudaMemcpyAsync` that syncs between host and device",1134 "id": 213297,1135 "title": "`torch.linalg.svd` uses `cudaMemcpyAsync` that syncs between host and device",1136 "slug": "torch-linalg-svd-uses-cudamemcpyasync-that-syncs-between-host-and-device",1137 "posts_count": 1,1138 "reply_count": 0,1139 "highest_post_number": 1,1140 "image_url": null,1141 "created_at": "2024-11-22T07:35:51.287Z",1142 "last_posted_at": "2024-11-22T07:35:51.352Z",1143 "bumped": true,1144 "bumped_at": "2024-11-22T07:35:51.352Z",1145 "archetype": "regular",1146 "unseen": false,1147 "pinned": false,1148 "unpinned": null,1149 "visible": true,1150 "closed": false,1151 "archived": false,1152 "bookmarked": null,1153 "liked": null,1154 "tags_descriptions": {},1155 "like_count": 0,1156 "views": 64,1157 "category_id": 1,1158 "featured_link": null,1159 "has_accepted_answer": false,1160 "posters": [1161 {1162 "extras": "latest single",1163 "description": "Original Poster, Most Recent Poster",1164 "user": {1165 "id": 14714,1166 "username": "Rui_Wang",1167 "name": "Rui Wang",1168 "avatar_template": "/user_avatar/discuss.pytorch.org/rui_wang/{size}/37635_2.png",1169 "trust_level": 11170 }1171 }1172 ]1173 },1174 {1175 "fancy_title": "PyTorch scatter max for sparse tensors?",1176 "id": 214472,1177 "title": "PyTorch scatter max for sparse tensors?",1178 "slug": "pytorch-scatter-max-for-sparse-tensors",1179 "posts_count": 1,1180 "reply_count": 0,1181 "highest_post_number": 1,1182 "image_url": null,1183 "created_at": "2024-12-20T21:37:04.327Z",1184 "last_posted_at": "2024-12-20T21:37:04.366Z",1185 "bumped": true,1186 "bumped_at": "2024-12-20T21:37:04.366Z",1187 "archetype": "regular",1188 "unseen": false,1189 "pinned": false,1190 "unpinned": null,1191 "visible": true,1192 "closed": false,1193 "archived": false,1194 "bookmarked": null,1195 "liked": null,1196 "tags_descriptions": {},1197 "like_count": 0,1198 "views": 150,1199 "category_id": 1,1200 "featured_link": null,