Anurag1734/cuda-error-resolution-analysis
07
1[2 {3 "post_stream": {4 "posts": [5 {6 "id": 394547,7 "name": "",8 "username": "jambato",9 "avatar_template": "/user_avatar/discuss.pytorch.org/jambato/{size}/58885_2.png",10 "created_at": "2023-03-28T05:49:30.344Z",11 "cooked": "<p>I have a directory with 5000 numpy files, each containing an array of shape [n, m] where <em>n</em> is the number of sequences of length <em>m</em>. The number of sequences <em>n</em> is different for each file and each sequence constitutes a training sample. So far my solution has been to iterate over every file and create a list containing all the sequences, the list is then fed to<code> __getitem__</code>. This solutions consumes too much memory and slows down training. Is there a more efficient way to build the Dataset?</p>\n<pre><code class=\"lang-auto\">class SequenceDataset(Data.Dataset):\n def __init__(self, signal_dir):\n file_list = os.listdir(signal_dir)\n self.file_count = len(file_list)\n print('file number:', self.file_count)\n self.file_list = file_list\n self.signal_path = signal_dir\n\n self.idx = 0\n self.signal_len = self.get_signal_length()\n\n self.signals_pool, self.len_signals =self.getpool() \n\n\n def __len__(self):\n return self.len_signals\n\n def get_signal_length(self):\n signal = np.load(self.signal_path+'/'+self.file_list[0])\n signal_length = signal.shape[1]\n return signal_length\n \n def getpool(self):\n signals = []\n counter = 0\n for f in self.file_list:\n signal = np.load(self.signal_path+'/'+f)\n signals.append(signal)\n\n signal_array = torch.from_numpy(np.concatenate(signals))\n \n len_signals = signal_array.shape[0]\n\n return signal_array.unsqueeze(2), len_signals\n\n def __getitem__(self, item):\n signal = self.signals_pool[item]\n \n return signal\n</code></pre>",12 "post_number": 1,13 "post_type": 1,14 "posts_count": 3,15 "updated_at": "2023-03-28T05:55:10.221Z",16 "reply_count": 0,17 "reply_to_post_number": null,18 "quote_count": 0,19 "incoming_link_count": 20,20 "reads": 8,21 "readers_count": 7,22 "score": 101.6,23 "yours": false,24 "topic_id": 176057,25 "topic_slug": "dataset-from-variable-shaped-arrays",26 "display_username": "",27 "primary_group_name": null,28 "flair_name": null,29 "flair_url": null,30 "flair_bg_color": null,31 "flair_color": null,32 "flair_group_id": null,33 "badges_granted": [],34 "version": 2,35 "can_edit": false,36 "can_delete": false,37 "can_recover": false,38 "can_see_hidden_post": false,39 "can_wiki": false,40 "read": true,41 "user_title": null,42 "bookmarked": false,43 "actions_summary": [],44 "moderator": false,45 "admin": false,46 "staff": false,47 "user_id": 64698,48 "hidden": false,49 "trust_level": 1,50 "deleted_at": null,51 "user_deleted": false,52 "edit_reason": null,53 "can_view_edit_history": true,54 "wiki": false,55 "post_url": "/t/dataset-from-variable-shaped-arrays/176057/1",56 "can_accept_answer": false,57 "can_unaccept_answer": false,58 "accepted_answer": false,59 "topic_accepted_answer": null,60 "can_vote": false61 },62 {63 "id": 394649,64 "name": "",65 "username": "ptrblck",66 "avatar_template": "/user_avatar/discuss.pytorch.org/ptrblck/{size}/1823_2.png",67 "created_at": "2023-03-28T16:38:21.421Z",68 "cooked": "<p>Instead of preloading the data in the <code>__init__</code> method you could store the paths only and load and process each file lazily in the <code>__getitem__</code> method which would reduce the memory usage.</p>",69 "post_number": 2,70 "post_type": 1,71 "posts_count": 3,72 "updated_at": "2023-03-28T16:38:21.421Z",73 "reply_count": 1,74 "reply_to_post_number": null,75 "quote_count": 0,76 "incoming_link_count": 0,77 "reads": 6,78 "readers_count": 5,79 "score": 21.2,80 "yours": false,81 "topic_id": 176057,82 "topic_slug": "dataset-from-variable-shaped-arrays",83 "display_username": "",84 "primary_group_name": null,85 "flair_name": null,86 "flair_url": null,87 "flair_bg_color": null,88 "flair_color": null,89 "flair_group_id": null,90 "badges_granted": [],91 "version": 1,92 "can_edit": false,93 "can_delete": false,94 "can_recover": false,95 "can_see_hidden_post": false,96 "can_wiki": false,97 "read": true,98 "user_title": "",99 "bookmarked": false,100 "actions_summary": [101 {102 "id": 2,103 "count": 1104 }105 ],106 "moderator": true,107 "admin": true,108 "staff": true,109 "user_id": 3534,110 "hidden": false,111 "trust_level": 2,112 "deleted_at": null,113 "user_deleted": false,114 "edit_reason": null,115 "can_view_edit_history": true,116 "wiki": false,117 "post_url": "/t/dataset-from-variable-shaped-arrays/176057/2",118 "can_accept_answer": false,119 "can_unaccept_answer": false,120 "accepted_answer": false,121 "topic_accepted_answer": null122 },123 {124 "id": 394715,125 "name": "",126 "username": "jambato",127 "avatar_template": "/user_avatar/discuss.pytorch.org/jambato/{size}/58885_2.png",128 "created_at": "2023-03-29T04:26:42.299Z",129 "cooked": "<p>Thank you! I will try that!</p>",130 "post_number": 3,131 "post_type": 1,132 "posts_count": 3,133 "updated_at": "2023-03-29T04:26:42.299Z",134 "reply_count": 0,135 "reply_to_post_number": 2,136 "quote_count": 0,137 "incoming_link_count": 0,138 "reads": 6,139 "readers_count": 5,140 "score": 1.2,141 "yours": false,142 "topic_id": 176057,143 "topic_slug": "dataset-from-variable-shaped-arrays",144 "display_username": "",145 "primary_group_name": null,146 "flair_name": null,147 "flair_url": null,148 "flair_bg_color": null,149 "flair_color": null,150 "flair_group_id": null,151 "badges_granted": [],152 "version": 1,153 "can_edit": false,154 "can_delete": false,155 "can_recover": false,156 "can_see_hidden_post": false,157 "can_wiki": false,158 "read": true,159 "user_title": null,160 "reply_to_user": {161 "id": 3534,162 "username": "ptrblck",163 "name": "",164 "avatar_template": "/user_avatar/discuss.pytorch.org/ptrblck/{size}/1823_2.png"165 },166 "bookmarked": false,167 "actions_summary": [],168 "moderator": false,169 "admin": false,170 "staff": false,171 "user_id": 64698,172 "hidden": false,173 "trust_level": 1,174 "deleted_at": null,175 "user_deleted": false,176 "edit_reason": null,177 "can_view_edit_history": true,178 "wiki": false,179 "post_url": "/t/dataset-from-variable-shaped-arrays/176057/3",180 "can_accept_answer": false,181 "can_unaccept_answer": false,182 "accepted_answer": false,183 "topic_accepted_answer": null184 }185 ],186 "stream": [187 394547,188 394649,189 394715190 ]191 },192 "timeline_lookup": [193 [194 1,195 943196 ],197 [198 2,199 942200 ]201 ],202 "suggested_topics": [203 {204 "fancy_title": "Why does Batch.from_data_list add edge_index, edge_attr, and pos to my Torus objects even when not set?",205 "id": 218544,206 "title": "Why does Batch.from_data_list add edge_index, edge_attr, and pos to my Torus objects even when not set?",207 "slug": "why-does-batch-from-data-list-add-edge-index-edge-attr-and-pos-to-my-torus-objects-even-when-not-set",208 "posts_count": 1,209 "reply_count": 0,210 "highest_post_number": 1,211 "image_url": null,212 "created_at": "2025-04-02T14:51:36.978Z",213 "last_posted_at": "2025-04-02T14:51:37.023Z",214 "bumped": true,215 "bumped_at": "2025-04-02T14:51:37.023Z",216 "archetype": "regular",217 "unseen": false,218 "pinned": false,219 "unpinned": null,220 "visible": true,221 "closed": false,222 "archived": false,223 "bookmarked": null,224 "liked": null,225 "tags_descriptions": {},226 "like_count": 0,227 "views": 62,228 "category_id": 37,229 "featured_link": null,230 "has_accepted_answer": false,231 "posters": [232 {233 "extras": "latest single",234 "description": "Original Poster, Most Recent Poster",235 "user": {236 "id": 83602,237 "username": "Tommaso_Guarniera",238 "name": "Tommaso Guarniera",239 "avatar_template": "/user_avatar/discuss.pytorch.org/tommaso_guarniera/{size}/76467_2.png",240 "trust_level": 0241 }242 }243 ]244 },245 {246 "fancy_title": "How do I create mini-batching to meet my training requirements?",247 "id": 214131,248 "title": "How do I create mini-batching to meet my training requirements?",249 "slug": "how-do-i-create-mini-batching-to-meet-my-training-requirements",250 "posts_count": 1,251 "reply_count": 0,252 "highest_post_number": 1,253 "image_url": null,254 "created_at": "2024-12-12T00:53:12.044Z",255 "last_posted_at": "2024-12-12T00:53:12.091Z",256 "bumped": true,257 "bumped_at": "2024-12-12T00:53:12.091Z",258 "archetype": "regular",259 "unseen": false,260 "pinned": false,261 "unpinned": null,262 "visible": true,263 "closed": false,264 "archived": false,265 "bookmarked": null,266 "liked": null,267 "tags_descriptions": {},268 "like_count": 0,269 "views": 44,270 "category_id": 37,271 "featured_link": null,272 "has_accepted_answer": false,273 "posters": [274 {275 "extras": "latest single",276 "description": "Original Poster, Most Recent Poster",277 "user": {278 "id": 81459,279 "username": "Rajesh_Singh",280 "name": "Rajesh Singh",281 "avatar_template": "/user_avatar/discuss.pytorch.org/rajesh_singh/{size}/73225_2.png",282 "trust_level": 1283 }284 }285 ]286 },287 {288 "fancy_title": "Why doesn’t torch have a memmap-like function?",289 "id": 215060,290 "title": "Why doesn't torch have a memmap-like function?",291 "slug": "why-doesnt-torch-have-a-memmap-like-function",292 "posts_count": 2,293 "reply_count": 0,294 "highest_post_number": 2,295 "image_url": null,296 "created_at": "2025-01-07T09:54:34.142Z",297 "last_posted_at": "2025-01-07T11:11:32.025Z",298 "bumped": true,299 "bumped_at": "2025-01-07T11:11:32.025Z",300 "archetype": "regular",301 "unseen": false,302 "pinned": false,303 "unpinned": null,304 "visible": true,305 "closed": false,306 "archived": false,307 "bookmarked": null,308 "liked": null,309 "tags_descriptions": {},310 "like_count": 0,311 "views": 66,312 "category_id": 37,313 "featured_link": null,314 "has_accepted_answer": false,315 "posters": [316 {317 "extras": null,318 "description": "Original Poster",319 "user": {320 "id": 81918,321 "username": "Rain_river",322 "name": "Rain river",323 "avatar_template": "/user_avatar/discuss.pytorch.org/rain_river/{size}/74943_2.png",324 "trust_level": 0325 }326 },327 {328 "extras": "latest",329 "description": "Most Recent Poster",330 "user": {331 "id": 9081,332 "username": "JuanFMontesinos",333 "name": "Juan Montesinos",334 "avatar_template": "/user_avatar/discuss.pytorch.org/juanfmontesinos/{size}/76115_2.png",335 "trust_level": 2336 }337 }338 ]339 },340 {341 "fancy_title": "ShufflerIterDataPipe but different random seed per distributed rank",342 "id": 212612,343 "title": "ShufflerIterDataPipe but different random seed per distributed rank",344 "slug": "shuffleriterdatapipe-but-different-random-seed-per-distributed-rank",345 "posts_count": 1,346 "reply_count": 0,347 "highest_post_number": 1,348 "image_url": null,349 "created_at": "2024-11-06T12:53:13.376Z",350 "last_posted_at": "2024-11-06T12:53:13.422Z",351 "bumped": true,352 "bumped_at": "2024-11-06T12:53:13.422Z",353 "archetype": "regular",354 "unseen": false,355 "pinned": false,356 "unpinned": null,357 "visible": true,358 "closed": false,359 "archived": false,360 "bookmarked": null,361 "liked": null,362 "tags_descriptions": {},363 "like_count": 0,364 "views": 109,365 "category_id": 37,366 "featured_link": null,367 "has_accepted_answer": false,368 "posters": [369 {370 "extras": "latest single",371 "description": "Original Poster, Most Recent Poster",372 "user": {373 "id": 51970,374 "username": "AlbertZeyer",375 "name": "Albert Zeyer",376 "avatar_template": "/user_avatar/discuss.pytorch.org/albertzeyer/{size}/45362_2.png",377 "trust_level": 2378 }379 }380 ]381 },382 {383 "fancy_title": "Pytorch and Cross Entropy",384 "id": 219822,385 "title": "Pytorch and Cross Entropy",386 "slug": "pytorch-and-cross-entropy",387 "posts_count": 7,388 "reply_count": 5,389 "highest_post_number": 7,390 "image_url": null,391 "created_at": "2025-05-06T21:04:58.525Z",392 "last_posted_at": "2025-05-07T12:47:47.953Z",393 "bumped": true,394 "bumped_at": "2025-05-07T12:47:47.953Z",395 "archetype": "regular",396 "unseen": false,397 "pinned": false,398 "unpinned": null,399 "visible": true,400 "closed": false,401 "archived": false,402 "bookmarked": null,403 "liked": null,404 "tags_descriptions": {},405 "like_count": 1,406 "views": 140,407 "category_id": 37,408 "featured_link": null,409 "has_accepted_answer": false,410 "posters": [411 {412 "extras": null,413 "description": "Original Poster",414 "user": {415 "id": 84178,416 "username": "Teodora-bo",417 "name": "Teodora Bo",418 "avatar_template": "/user_avatar/discuss.pytorch.org/teodora-bo/{size}/76933_2.png",419 "trust_level": 0420 }421 },422 {423 "extras": "latest",424 "description": "Most Recent Poster",425 "user": {426 "id": 3534,427 "username": "ptrblck",428 "name": "",429 "avatar_template": "/user_avatar/discuss.pytorch.org/ptrblck/{size}/1823_2.png",430 "admin": true,431 "moderator": true,432 "trust_level": 2433 }434 }435 ]436 }437 ],438 "tags_descriptions": {},439 "fancy_title": "Dataset from variable shaped arrays",440 "id": 176057,441 "title": "Dataset from variable shaped arrays",442 "posts_count": 3,443 "created_at": "2023-03-28T05:49:30.260Z",444 "views": 394,445 "reply_count": 1,446 "like_count": 1,447 "last_posted_at": "2023-03-29T04:26:42.299Z",448 "visible": true,449 "closed": false,450 "archived": false,451 "has_summary": false,452 "archetype": "regular",453 "slug": "dataset-from-variable-shaped-arrays",454 "category_id": 37,455 "word_count": 235,456 "deleted_at": null,457 "user_id": 64698,458 "featured_link": null,459 "pinned_globally": false,460 "pinned_at": null,461 "pinned_until": null,462 "image_url": null,463 "slow_mode_seconds": 0,464 "draft": null,465 "draft_key": "topic_176057",466 "draft_sequence": null,467 "unpinned": null,468 "pinned": false,469 "current_post_number": 1,470 "highest_post_number": 3,471 "deleted_by": null,472 "actions_summary": [473 {474 "id": 4,475 "count": 0,476 "hidden": false,477 "can_act": false478 },479 {480 "id": 8,481 "count": 0,482 "hidden": false,483 "can_act": false484 },485 {486 "id": 10,487 "count": 0,488 "hidden": false,489 "can_act": false490 },491 {492 "id": 7,493 "count": 0,494 "hidden": false,495 "can_act": false496 }497 ],498 "chunk_size": 20,499 "bookmarked": false,500 "topic_timer": null,501 "message_bus_last_id": 0,502 "participant_count": 2,503 "show_read_indicator": false,504 "thumbnails": null,505 "slow_mode_enabled_until": null,506 "can_vote": false,507 "vote_count": 0,508 "user_voted": false,509 "discourse_zendesk_plugin_zendesk_id": null,510 "discourse_zendesk_plugin_zendesk_url": "https://your-url.zendesk.com/agent/tickets/",511 "details": {512 "can_edit": false,513 "notification_level": 1,514 "participants": [515 {516 "id": 64698,517 "username": "jambato",518 "name": "",519 "avatar_template": "/user_avatar/discuss.pytorch.org/jambato/{size}/58885_2.png",520 "post_count": 2,521 "primary_group_name": null,522 "flair_name": null,523 "flair_url": null,524 "flair_color": null,525 "flair_bg_color": null,526 "flair_group_id": null,527 "trust_level": 1528 },529 {530 "id": 3534,531 "username": "ptrblck",532 "name": "",533 "avatar_template": "/user_avatar/discuss.pytorch.org/ptrblck/{size}/1823_2.png",534 "post_count": 1,535 "primary_group_name": null,536 "flair_name": null,537 "flair_url": null,538 "flair_color": null,539 "flair_bg_color": null,540 "flair_group_id": null,541 "admin": true,542 "moderator": true,543 "trust_level": 2544 }545 ],546 "created_by": {547 "id": 64698,548 "username": "jambato",549 "name": "",550 "avatar_template": "/user_avatar/discuss.pytorch.org/jambato/{size}/58885_2.png"551 },552 "last_poster": {553 "id": 64698,554 "username": "jambato",555 "name": "",556 "avatar_template": "/user_avatar/discuss.pytorch.org/jambato/{size}/58885_2.png"557 }558 },559 "bookmarks": []560 },561 {562 "post_stream": {563 "posts": [564 {565 "id": 394629,566 "name": "Sayak Paul",567 "username": "Sayak_Paul",568 "avatar_template": "/user_avatar/discuss.pytorch.org/sayak_paul/{size}/12743_2.png",569 "created_at": "2023-03-28T15:21:19.185Z",570 "cooked": "<p>I am trying to monkey-patch the <code>forward()</code> method of an <code>nn.Module</code>. Here’s my <code>nn.Module</code>:</p>\n<pre><code class=\"lang-py\">import torch.nn as nn \n\nclass GPT5(nn.Module):\n embed_dim = 768\n num_heads = 12\n q_proj = nn.Linear(embed_dim, embed_dim)\n head_dim = embed_dim // num_heads\n scale = head_dim**-0.5\n\n def forward(self, hidden_states):\n return self.q_proj(hidden_states) * self.scale\n</code></pre>\n<p>The following works as usual:</p>\n<pre><code class=\"lang-py\">import torch \n\ngpt5 = GPT5()\ngpt5(torch.randn(1, 10, 768)).size()\n</code></pre>\n<p>Do monkey-patching:</p>\n<pre><code class=\"lang-py\">gpt5 = GPT5()\nnew_forward = lambda x: l.forward(x) + 1\ngpt5.forward = new_forward\n</code></pre>\n<p>The following then raises an error:</p>\n<pre><code class=\"lang-py\">gpt5(torch.randn(1, 10, 768)).size()\n</code></pre>\n<pre><code class=\"lang-bash\">─────────────────────────────── Traceback (most recent call last) ────────────────────────────────╮\n│ in <module>:1 │\n│ │\n│ /usr/local/lib/python3.9/dist-packages/torch/nn/modules/module.py:1194 in _call_impl │\n│ │\n│ 1191 │ │ # this function, and just call forward. │\n│ 1192 │ │ if not (self._backward_hooks or self._forward_hooks or self._forward_pre_hooks o │\n│ 1193 │ │ │ │ or _global_forward_hooks or _global_forward_pre_hooks): │\n│ ❱ 1194 │ │ │ return forward_call(*input, **kwargs) │\n│ 1195 │ │ # Do not call functions when jit is used │\n│ 1196 │ │ full_backward_hooks, non_full_backward_hooks = [], [] │\n│ 1197 │ │ if self._backward_hooks or _global_backward_hooks: │\n│ in <lambda>:2 │\n│ in <lambda>:2 │\n</code></pre>",571 "post_number": 1,572 "post_type": 1,573 "posts_count": 3,574 "updated_at": "2023-03-28T15:21:19.185Z",575 "reply_count": 0,576 "reply_to_post_number": null,577 "quote_count": 0,578 "incoming_link_count": 744,579 "reads": 12,580 "readers_count": 11,581 "score": 3712.4,582 "yours": false,583 "topic_id": 176095,584 "topic_slug": "monkey-patching-the-forward-pass-of-an-nn-module",585 "display_username": "Sayak Paul",586 "primary_group_name": null,587 "flair_name": null,588 "flair_url": null,589 "flair_bg_color": null,590 "flair_color": null,591 "flair_group_id": null,592 "badges_granted": [],593 "version": 1,594 "can_edit": false,595 "can_delete": false,596 "can_recover": false,597 "can_see_hidden_post": false,598 "can_wiki": false,599 "read": true,600 "user_title": null,601 "bookmarked": false,602 "actions_summary": [],603 "moderator": false,604 "admin": false,605 "staff": false,606 "user_id": 19267,607 "hidden": false,608 "trust_level": 2,609 "deleted_at": null,610 "user_deleted": false,611 "edit_reason": null,612 "can_view_edit_history": true,613 "wiki": false,614 "post_url": "/t/monkey-patching-the-forward-pass-of-an-nn-module/176095/1",615 "can_accept_answer": false,616 "can_unaccept_answer": false,617 "accepted_answer": false,618 "topic_accepted_answer": true,619 "can_vote": false620 },621 {622 "id": 394644,623 "name": "",624 "username": "soulitzer",625 "avatar_template": "/letter_avatar_proxy/v4/letter/s/839c29/{size}.png",626 "created_at": "2023-03-28T16:26:09.245Z",627 "cooked": "<p>Does the following work for you?</p>\n<pre><code class=\"lang-python\">class GPT5(nn.Module):\n embed_dim = 768\n num_heads = 12\n q_proj = nn.Linear(embed_dim, embed_dim)\n head_dim = embed_dim // num_heads\n scale = head_dim**-0.5\n\n def forward(self, hidden_states):\n return self.q_proj(hidden_states) * self.scale\n\ngpt5 = GPT5()\nold_forward = gpt5.forward\n\nnew_forward = lambda x: old_forward(x) + 1\ngpt5.forward = new_forward\n\ngpt5(torch.randn(1, 10, 768))\n\n</code></pre>",628 "post_number": 2,629 "post_type": 1,630 "posts_count": 3,631 "updated_at": "2023-03-28T16:26:09.245Z",632 "reply_count": 1,633 "reply_to_post_number": null,634 "quote_count": 0,635 "incoming_link_count": 17,636 "reads": 11,637 "readers_count": 10,638 "score": 107.2,639 "yours": false,640 "topic_id": 176095,641 "topic_slug": "monkey-patching-the-forward-pass-of-an-nn-module",642 "display_username": "",643 "primary_group_name": null,644 "flair_name": null,645 "flair_url": null,646 "flair_bg_color": null,647 "flair_color": null,648 "flair_group_id": null,649 "badges_granted": [],650 "version": 1,651 "can_edit": false,652 "can_delete": false,653 "can_recover": false,654 "can_see_hidden_post": false,655 "can_wiki": false,656 "read": true,657 "user_title": null,658 "bookmarked": false,659 "actions_summary": [660 {661 "id": 2,662 "count": 1663 }664 ],665 "moderator": false,666 "admin": false,667 "staff": false,668 "user_id": 41396,669 "hidden": false,670 "trust_level": 2,671 "deleted_at": null,672 "user_deleted": false,673 "edit_reason": null,674 "can_view_edit_history": true,675 "wiki": false,676 "post_url": "/t/monkey-patching-the-forward-pass-of-an-nn-module/176095/2",677 "can_accept_answer": false,678 "can_unaccept_answer": false,679 "accepted_answer": true,680 "topic_accepted_answer": true681 },682 {683 "id": 394713,684 "name": "Sayak Paul",685 "username": "Sayak_Paul",686 "avatar_template": "/user_avatar/discuss.pytorch.org/sayak_paul/{size}/12743_2.png",687 "created_at": "2023-03-29T03:51:47.089Z",688 "cooked": "<p>Thanks! It works. Couldn’t realize the fix would be about separately storing the forward in a variable.</p>",689 "post_number": 3,690 "post_type": 1,691 "posts_count": 3,692 "updated_at": "2023-03-29T03:51:47.089Z",693 "reply_count": 0,694 "reply_to_post_number": 2,695 "quote_count": 0,696 "incoming_link_count": 4,697 "reads": 11,698 "readers_count": 10,699 "score": 22.2,700 "yours": false,701 "topic_id": 176095,702 "topic_slug": "monkey-patching-the-forward-pass-of-an-nn-module",703 "display_username": "Sayak Paul",704 "primary_group_name": null,705 "flair_name": null,706 "flair_url": null,707 "flair_bg_color": null,708 "flair_color": null,709 "flair_group_id": null,710 "badges_granted": [],711 "version": 1,712 "can_edit": false,713 "can_delete": false,714 "can_recover": false,715 "can_see_hidden_post": false,716 "can_wiki": false,717 "read": true,718 "user_title": null,719 "reply_to_user": {720 "id": 41396,721 "username": "soulitzer",722 "name": "",723 "avatar_template": "/letter_avatar_proxy/v4/letter/s/839c29/{size}.png"724 },725 "bookmarked": false,726 "actions_summary": [],727 "moderator": false,728 "admin": false,729 "staff": false,730 "user_id": 19267,731 "hidden": false,732 "trust_level": 2,733 "deleted_at": null,734 "user_deleted": false,735 "edit_reason": null,736 "can_view_edit_history": true,737 "wiki": false,738 "post_url": "/t/monkey-patching-the-forward-pass-of-an-nn-module/176095/3",739 "can_accept_answer": false,740 "can_unaccept_answer": false,741 "accepted_answer": false,742 "topic_accepted_answer": true743 }744 ],745 "stream": [746 394629,747 394644,748 394713749 ]750 },751 "timeline_lookup": [752 [753 1,754 942755 ]756 ],757 "suggested_topics": [758 {759 "fancy_title": "Loading model from checkpoint results differ to loading model directly",760 "id": 212514,761 "title": "Loading model from checkpoint results differ to loading model directly",762 "slug": "loading-model-from-checkpoint-results-differ-to-loading-model-directly",763 "posts_count": 3,764 "reply_count": 1,765 "highest_post_number": 3,766 "image_url": "https://discuss.pytorch.org/uploads/default/optimized/3X/5/f/5f8b63b57c5fb949f5ab5ca1980be73ad1741806_2_1024x880.png",767 "created_at": "2024-11-04T17:40:35.061Z",768 "last_posted_at": "2024-11-06T11:09:50.739Z",769 "bumped": true,770 "bumped_at": "2024-11-06T11:09:50.739Z",771 "archetype": "regular",772 "unseen": false,773 "pinned": false,774 "unpinned": null,775 "visible": true,776 "closed": false,777 "archived": false,778 "bookmarked": null,779 "liked": null,780 "tags_descriptions": {},781 "like_count": 0,782 "views": 291,783 "category_id": 1,784 "featured_link": null,785 "has_accepted_answer": true,786 "posters": [787 {788 "extras": "latest",789 "description": "Original Poster, Most Recent Poster, Accepted Answer",790 "user": {791 "id": 71636,792 "username": "sophiamaedler",793 "name": "Sophia Mädler",794 "avatar_template": "/user_avatar/discuss.pytorch.org/sophiamaedler/{size}/66166_2.png",795 "trust_level": 1796 }797 },798 {799 "extras": null,800 "description": "Frequent Poster",801 "user": {802 "id": 3534,803 "username": "ptrblck",804 "name": "",805 "avatar_template": "/user_avatar/discuss.pytorch.org/ptrblck/{size}/1823_2.png",806 "admin": true,807 "moderator": true,808 "trust_level": 2809 }810 }811 ]812 },813 {814 "fancy_title": "Non blocking copy from CPU to GPU",815 "id": 213522,816 "title": "Non blocking copy from CPU to GPU",817 "slug": "non-blocking-copy-from-cpu-to-gpu",818 "posts_count": 2,819 "reply_count": 0,820 "highest_post_number": 2,821 "image_url": null,822 "created_at": "2024-11-27T13:17:50.447Z",823 "last_posted_at": "2024-11-28T09:42:41.192Z",824 "bumped": true,825 "bumped_at": "2024-11-28T09:42:41.192Z",826 "archetype": "regular",827 "unseen": false,828 "pinned": false,829 "unpinned": null,830 "visible": true,831 "closed": false,832 "archived": false,833 "bookmarked": null,834 "liked": null,835 "tags_descriptions": {},836 "like_count": 0,837 "views": 162,838 "category_id": 1,839 "featured_link": null,840 "has_accepted_answer": false,841 "posters": [842 {843 "extras": "latest single",844 "description": "Original Poster, Most Recent Poster",845 "user": {846 "id": 81162,847 "username": "shira",848 "name": "shira",849 "avatar_template": "/user_avatar/discuss.pytorch.org/shira/{size}/74225_2.png",850 "trust_level": 1851 }852 }853 ]854 },855 {856 "fancy_title": "Error in loading tensor/model/input to CUDA",857 "id": 214301,858 "title": "Error in loading tensor/model/input to CUDA",859 "slug": "error-in-loading-tensor-model-input-to-cuda",860 "posts_count": 8,861 "reply_count": 4,862 "highest_post_number": 8,863 "image_url": null,864 "created_at": "2024-12-17T10:14:23.276Z",865 "last_posted_at": "2025-01-14T16:48:43.042Z",866 "bumped": true,867 "bumped_at": "2025-01-14T16:48:43.042Z",868 "archetype": "regular",869 "unseen": false,870 "pinned": false,871 "unpinned": null,872 "visible": true,873 "closed": false,874 "archived": false,875 "bookmarked": null,876 "liked": null,877 "tags_descriptions": {},878 "like_count": 2,879 "views": 390,880 "category_id": 1,881 "featured_link": null,882 "has_accepted_answer": false,883 "posters": [884 {885 "extras": "latest",886 "description": "Original Poster, Most Recent Poster",887 "user": {888 "id": 81541,889 "username": "v_lab",890 "name": "v lab",891 "avatar_template": "/user_avatar/discuss.pytorch.org/v_lab/{size}/74548_2.png",892 "trust_level": 1893 }894 },895 {896 "extras": null,897 "description": "Frequent Poster",898 "user": {899 "id": 81089,900 "username": "Aknw_Fen",901 "name": "Aknw Fen",902 "avatar_template": "/user_avatar/discuss.pytorch.org/aknw_fen/{size}/74156_2.png",903 "trust_level": 2904 }905 },906 {907 "extras": null,908 "description": "Frequent Poster",909 "user": {910 "id": 3534,911 "username": "ptrblck",912 "name": "",913 "avatar_template": "/user_avatar/discuss.pytorch.org/ptrblck/{size}/1823_2.png",914 "admin": true,915 "moderator": true,916 "trust_level": 2917 }918 }919 ]920 },921 {922 "fancy_title": "RuntimeError: Given groups=1, weight of size [128, 3, 3, 3], expected input[1, 1, 512, 512] to have 3 channels, but got 1 channels instead",923 "id": 216781,924 "title": "RuntimeError: Given groups=1, weight of size [128, 3, 3, 3], expected input[1, 1, 512, 512] to have 3 channels, but got 1 channels instead",925 "slug": "runtimeerror-given-groups-1-weight-of-size-128-3-3-3-expected-input-1-1-512-512-to-have-3-channels-but-got-1-channels-instead",926 "posts_count": 9,927 "reply_count": 6,928 "highest_post_number": 9,929 "image_url": null,930 "created_at": "2025-02-17T15:36:50.990Z",931 "last_posted_at": "2025-02-21T15:06:24.012Z",932 "bumped": true,933 "bumped_at": "2025-02-21T15:06:24.012Z",934 "archetype": "regular",935 "unseen": false,936 "pinned": false,937 "unpinned": null,938 "visible": true,939 "closed": false,940 "archived": false,941 "bookmarked": null,942 "liked": null,943 "tags_descriptions": {},944 "like_count": 1,945 "views": 210,946 "category_id": 1,947 "featured_link": null,948 "has_accepted_answer": true,949 "posters": [950 {951 "extras": "latest",952 "description": "Original Poster, Most Recent Poster",953 "user": {954 "id": 82738,955 "username": "Dmitr",956 "name": "",957 "avatar_template": "/letter_avatar_proxy/v4/letter/d/13edae/{size}.png",958 "trust_level": 1959 }960 },961 {962 "extras": null,963 "description": "Frequent Poster, Accepted Answer",964 "user": {965 "id": 41458,966 "username": "J_Johnson",967 "name": "J Johnson",968 "avatar_template": "/user_avatar/discuss.pytorch.org/j_johnson/{size}/55494_2.png",969 "trust_level": 2970 }971 },972 {973 "extras": null,974 "description": "Frequent Poster",975 "user": {976 "id": 18088,977 "username": "KFrank",978 "name": "K. Frank",979 "avatar_template": "/letter_avatar_proxy/v4/letter/k/ecb155/{size}.png",980 "trust_level": 2981 }982 }983 ]984 },985 {986 "fancy_title": "Do pre-trained model weights e.g. ResNet50 get updated?",987 "id": 217485,988 "title": "Do pre-trained model weights e.g. ResNet50 get updated?",989 "slug": "do-pre-trained-model-weights-e-g-resnet50-get-updated",990 "posts_count": 2,991 "reply_count": 0,992 "highest_post_number": 2,993 "image_url": null,994 "created_at": "2025-03-05T17:55:02.790Z",995 "last_posted_at": "2025-03-05T19:21:48.780Z",996 "bumped": true,997 "bumped_at": "2025-03-05T19:21:48.780Z",998 "archetype": "regular",999 "unseen": false,1000 "pinned": false,1001 "unpinned": null,1002 "visible": true,1003 "closed": false,1004 "archived": false,1005 "bookmarked": null,1006 "liked": null,1007 "tags_descriptions": {},1008 "like_count": 0,1009 "views": 33,1010 "category_id": 1,1011 "featured_link": null,1012 "has_accepted_answer": false,1013 "posters": [1014 {1015 "extras": null,1016 "description": "Original Poster",1017 "user": {1018 "id": 83089,1019 "username": "td00",1020 "name": "",1021 "avatar_template": "/letter_avatar_proxy/v4/letter/t/eb8c5e/{size}.png",1022 "trust_level": 01023 }1024 },1025 {1026 "extras": "latest",1027 "description": "Most Recent Poster",1028 "user": {1029 "id": 72430,1030 "username": "Eduardo_Lawson",1031 "name": "Eduardo Lawson da Silva",1032 "avatar_template": "/user_avatar/discuss.pytorch.org/eduardo_lawson/{size}/66899_2.png",1033 "trust_level": 21034 }1035 }1036 ]1037 }1038 ],1039 "tags_descriptions": {},1040 "fancy_title": "Monkey-patching the `forward()` pass of an `nn.Module`",1041 "id": 176095,1042 "title": "Monkey-patching the `forward()` pass of an `nn.Module`",1043 "posts_count": 3,1044 "created_at": "2023-03-28T15:21:19.101Z",1045 "views": 1068,1046 "reply_count": 1,1047 "like_count": 1,1048 "last_posted_at": "2023-03-29T03:51:47.089Z",1049 "visible": true,1050 "closed": false,1051 "archived": false,1052 "has_summary": false,1053 "archetype": "regular",1054 "slug": "monkey-patching-the-forward-pass-of-an-nn-module",1055 "category_id": 1,1056 "word_count": 253,1057 "deleted_at": null,1058 "user_id": 19267,1059 "featured_link": null,1060 "pinned_globally": false,1061 "pinned_at": null,1062 "pinned_until": null,1063 "image_url": null,1064 "slow_mode_seconds": 0,1065 "draft": null,1066 "draft_key": "topic_176095",1067 "draft_sequence": null,1068 "unpinned": null,1069 "pinned": false,1070 "current_post_number": 1,1071 "highest_post_number": 3,1072 "deleted_by": null,1073 "actions_summary": [1074 {1075 "id": 4,1076 "count": 0,1077 "hidden": false,1078 "can_act": false1079 },1080 {1081 "id": 8,1082 "count": 0,1083 "hidden": false,1084 "can_act": false1085 },1086 {1087 "id": 10,1088 "count": 0,1089 "hidden": false,1090 "can_act": false1091 },1092 {1093 "id": 7,1094 "count": 0,1095 "hidden": false,1096 "can_act": false1097 }1098 ],1099 "chunk_size": 20,1100 "bookmarked": false,1101 "topic_timer": null,1102 "message_bus_last_id": 0,1103 "participant_count": 2,1104 "show_read_indicator": false,1105 "thumbnails": null,1106 "slow_mode_enabled_until": null,1107 "accepted_answer": {1108 "post_number": 2,1109 "username": "soulitzer",1110 "name": "",1111 "excerpt": "Does the following work for you? \nclass GPT5(nn.Module):\n embed_dim = 768\n num_heads = 12\n q_proj = nn.Linear(embed_dim, embed_dim)\n head_dim = embed_dim // num_heads\n scale = head_dim**-0.5\n\n def forward(self, hidden_states):\n return self.q_proj(hidden_states) * self.scale\n…"1112 },1113 "can_vote": false,1114 "vote_count": 0,1115 "user_voted": false,1116 "discourse_zendesk_plugin_zendesk_id": null,1117 "discourse_zendesk_plugin_zendesk_url": "https://your-url.zendesk.com/agent/tickets/",1118 "details": {1119 "can_edit": false,1120 "notification_level": 1,1121 "participants": [1122 {1123 "id": 19267,1124 "username": "Sayak_Paul",1125 "name": "Sayak Paul",1126 "avatar_template": "/user_avatar/discuss.pytorch.org/sayak_paul/{size}/12743_2.png",1127 "post_count": 2,1128 "primary_group_name": null,1129 "flair_name": null,1130 "flair_url": null,1131 "flair_color": null,1132 "flair_bg_color": null,1133 "flair_group_id": null,1134 "trust_level": 21135 },1136 {1137 "id": 41396,1138 "username": "soulitzer",1139 "name": "",1140 "avatar_template": "/letter_avatar_proxy/v4/letter/s/839c29/{size}.png",1141 "post_count": 1,1142 "primary_group_name": null,1143 "flair_name": null,1144 "flair_url": null,1145 "flair_color": null,1146 "flair_bg_color": null,1147 "flair_group_id": null,1148 "trust_level": 21149 }1150 ],1151 "created_by": {1152 "id": 19267,1153 "username": "Sayak_Paul",1154 "name": "Sayak Paul",1155 "avatar_template": "/user_avatar/discuss.pytorch.org/sayak_paul/{size}/12743_2.png"1156 },1157 "last_poster": {1158 "id": 19267,1159 "username": "Sayak_Paul",1160 "name": "Sayak Paul",1161 "avatar_template": "/user_avatar/discuss.pytorch.org/sayak_paul/{size}/12743_2.png"1162 }1163 },1164 "bookmarks": []1165 },1166 {1167 "post_stream": {1168 "posts": [1169 {1170 "id": 394645,1171 "name": "Selvam Dharma",1172 "username": "Selvam_Dharma",1173 "avatar_template": "/user_avatar/discuss.pytorch.org/selvam_dharma/{size}/58785_2.png",1174 "created_at": "2023-03-28T16:30:17.544Z",1175 "cooked": "<pre><code class=\"lang-python\">#adding the extra layers \nclass added_layers(nn.Module):\n def __init__(self):\n super(added_layers,self).__init__()\n self.add = nn.Sequential(\n \n nn.AdaptiveMaxPool2d((1, 1)),\n nn.Linear(2048,256),nn.ReLU(),\n nn.Linear(256,3),nn.Softmax(dim=1)\n )\n\n def forward(self,x):\n add = self.add(x)\n\n return add\n\nadded_layer = added_layers().to('cuda')\n\nmodel = nn.Sequential(model,added_layer) \nmodel.to('cuda')\n</code></pre>\n<p>and</p>\n<pre><code class=\"lang-python\">#removing the fully connected layer\nclass nochange(nn.Module):\n def __init__(self):\n super(nochange, self).__init__()\n \n def forward(self, x):\n return x\nmodel.fc = nochange()\n\n#freeze the trained layers\nfor parameters in model.parameters():\n parameters.requires_grad = False\n</code></pre>\n<p>these are my code for combining pretrained model exception with additional layers. I am getting the following error:</p>\n<pre><code class=\"lang-python\">---------------------------------------------------------------------------\nValueError Traceback (most recent call last)\n<ipython-input-13-3b76556c6596> in <module>\n 1 #model training for num_ epoch\n----> 2 training_loss,val_loss = training(model,optimizer,train_loader,val_loader,test_loader,num_epoch,device = 'cuda',save_dir = '/content/drive/MyDrive/Applied_Artificial_intelligence/saved_model')\n 3 plt.plot(training_loss)\n\n12 frames\n/usr/local/lib/python3.9/dist-packages/torch/nn/modules/utils.py in _list_with_default(out_size, defaults)\n 34 return out_size\n 35 if len(defaults) <= len(out_size):\n---> 36 raise ValueError(\n 37 \"Input dimension should be at least {}\".format(len(out_size) + 1)\n 38 )\n\nValueError: Input dimension should be at least 3\n</code></pre>\n<p>what would be the issue here,</p>\n<pre><code class=\"lang-python\">(conv2): DwsConvBlock(\n (activ): ReLU()\n (conv): DwsConv(\n (dw_conv): Conv2d(1536, 1536, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), groups=1536, bias=False)\n (pw_conv): Conv2d(1536, 2048, kernel_size=(1, 1), stride=(1, 1), bias=False)\n )\n (bn): BatchNorm2d(2048, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)\n )\n (activ): ReLU(inplace=True)\n (pool): AvgPool2d(kernel_size=10, stride=1, padding=0)\n )\n )\n (output): Linear(in_features=2048, out_features=1000, bias=True)\n (fc): nochange()\n )\n (1): added_layers(\n (add): Sequential(\n (0): AdaptiveMaxPool2d(output_size=(1, 1))\n (1): Linear(in_features=2048, out_features=256, bias=True)\n (2): ReLU()\n (3): Linear(in_features=256, out_features=3, bias=True)\n (4): Softmax(dim=1)\n )\n )\n)\n</code></pre>\n<p>my last few layers are looking like this</p>",1176 "post_number": 1,1177 "post_type": 1,1178 "posts_count": 5,1179 "updated_at": "2023-03-28T16:32:16.028Z",1180 "reply_count": 0,1181 "reply_to_post_number": null,1182 "quote_count": 0,1183 "incoming_link_count": 1020,1184 "reads": 9,1185 "readers_count": 8,1186 "score": 5096.8,1187 "yours": false,1188 "topic_id": 176103,1189 "topic_slug": "valueerror-input-dimension-should-be-at-least-3",1190 "display_username": "Selvam Dharma",1191 "primary_group_name": null,1192 "flair_name": null,1193 "flair_url": null,1194 "flair_bg_color": null,1195 "flair_color": null,1196 "flair_group_id": null,1197 "badges_granted": [],1198 "version": 1,1199 "can_edit": false,1200 "can_delete": false,