Anurag1734/cuda-error-resolution-analysis
07
1[2 {3 "post_stream": {4 "posts": [5 {6 "id": 448885,7 "name": "Ravi Teja",8 "username": "Ravi_Teja2",9 "avatar_template": "/user_avatar/discuss.pytorch.org/ravi_teja2/{size}/66153_2.png",10 "created_at": "2024-07-13T08:19:27.471Z",11 "cooked": "<p>I’m bit new to using Iterable Datasets. I have been using torchdata to build my dataloaders but they seem to deprecate and even delete this functionality now <img src=\"https://discuss.pytorch.org/images/emoji/apple/smiling_face_with_tear.png?v=12\" title=\":smiling_face_with_tear:\" class=\"emoji\" alt=\":smiling_face_with_tear:\" loading=\"lazy\" width=\"20\" height=\"20\">. So, I want to build Iterable Dataset but it seems DistributedSampler cannot be used on iterable dataset. Any suggestions on how to use DDP on iterable Datasets?</p>\n<p>I’m aware of this large issue: <a href=\"https://github.com/pytorch/pytorch/pull/26547\" class=\"inline-onebox\" rel=\"noopener nofollow ugc\">ChunkDataset API proposal by thiagocrepaldi · Pull Request #26547 · pytorch/pytorch · GitHub</a><br>\nBut I don’t think this functionality is added yet.<br>\nAnd also this post: <a href=\"https://discuss.pytorch.org/t/using-iterabledataset-with-distributeddataparallel/92589\" class=\"inline-onebox\">Using IterableDataset with DistributedDataParallel</a><br>\nHere I don’t find any workable process or examples</p>",12 "post_number": 1,13 "post_type": 1,14 "posts_count": 2,15 "updated_at": "2024-07-13T08:26:03.264Z",16 "reply_count": 0,17 "reply_to_post_number": null,18 "quote_count": 0,19 "incoming_link_count": 647,20 "reads": 17,21 "readers_count": 16,22 "score": 3213.4,23 "yours": false,24 "topic_id": 206244,25 "topic_slug": "ddp-on-iterable-dataset",26 "display_username": "Ravi Teja",27 "primary_group_name": null,28 "flair_name": null,29 "flair_url": null,30 "flair_bg_color": null,31 "flair_color": null,32 "flair_group_id": null,33 "badges_granted": [],34 "version": 3,35 "can_edit": false,36 "can_delete": false,37 "can_recover": false,38 "can_see_hidden_post": false,39 "can_wiki": false,40 "link_counts": [41 {42 "url": "https://discuss.pytorch.org/t/using-iterabledataset-with-distributeddataparallel/92589",43 "internal": true,44 "reflection": false,45 "title": "Using IterableDataset with DistributedDataParallel",46 "clicks": 9047 },48 {49 "url": "https://github.com/pytorch/pytorch/pull/26547",50 "internal": false,51 "reflection": false,52 "title": "ChunkDataset API proposal by thiagocrepaldi · Pull Request #26547 · pytorch/pytorch · GitHub",53 "clicks": 2054 }55 ],56 "read": true,57 "user_title": null,58 "bookmarked": false,59 "actions_summary": [],60 "moderator": false,61 "admin": false,62 "staff": false,63 "user_id": 71628,64 "hidden": false,65 "trust_level": 1,66 "deleted_at": null,67 "user_deleted": false,68 "edit_reason": null,69 "can_view_edit_history": true,70 "wiki": false,71 "post_url": "/t/ddp-on-iterable-dataset/206244/1",72 "can_accept_answer": false,73 "can_unaccept_answer": false,74 "accepted_answer": false,75 "topic_accepted_answer": null,76 "can_vote": false77 },78 {79 "id": 462819,80 "name": "MENG",81 "username": "602389789",82 "avatar_template": "/letter_avatar_proxy/v4/letter/6/54ee81/{size}.png",83 "created_at": "2025-01-08T02:10:00.629Z",84 "cooked": "<p>Maybe you can use something like the IterableDatasetShard to wrap your IterableDataset</p><aside class=\"onebox githubblob\" data-onebox-src=\"https://github.com/huggingface/transformers/blob/main/src/transformers/trainer_pt_utils.py#L798\">\n <header class=\"source\">\n\n <a href=\"https://github.com/huggingface/transformers/blob/main/src/transformers/trainer_pt_utils.py#L798\" target=\"_blank\" rel=\"noopener nofollow ugc\">github.com</a>\n </header>\n\n <article class=\"onebox-body\">\n <h4><a href=\"https://github.com/huggingface/transformers/blob/main/src/transformers/trainer_pt_utils.py#L798\" target=\"_blank\" rel=\"noopener nofollow ugc\">huggingface/transformers/blob/main/src/transformers/trainer_pt_utils.py#L798</a></h4>\n\n\n\n <pre class=\"onebox\"><code class=\"lang-py\">\n <ol class=\"start lines\" start=\"788\" style=\"counter-reset: li-counter 787 ;\">\n <li> for batch_start in range(self.batch_size * self.process_index, self.total_num_samples, self.total_batch_size):</li>\n <li> result += indices[batch_start : batch_start + self.batch_size]</li>\n <li></li>\n <li> return iter(result)</li>\n <li></li>\n <li> def __len__(self):</li>\n <li> # Each shard only sees a fraction of total_num_samples.</li>\n <li> return self.total_num_samples // self.num_processes</li>\n <li></li>\n <li></li>\n <li class=\"selected\">class IterableDatasetShard(IterableDataset):</li>\n <li> \"\"\"</li>\n <li> Wraps a PyTorch `IterableDataset` to generate samples for one of the processes only. Instances of this class will</li>\n <li> always yield a number of samples that is a round multiple of the actual batch size (which is `batch_size x</li>\n <li> num_processes`). Depending on the value of the `drop_last` attribute, it will either stop the iteration at the</li>\n <li> first batch that would be too small or loop with indices from the beginning.</li>\n <li></li>\n <li> On two processes with an iterable dataset yielding of `[0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11]` with a batch size of</li>\n <li> 2:</li>\n <li></li>\n <li> - the shard on process 0 will yield `[0, 1, 4, 5, 8, 9]` so will see batches `[0, 1]`, `[4, 5]`, `[8, 9]`</li>\n </ol>\n </code></pre>\n\n\n\n </article>\n\n <div class=\"onebox-metadata\">\n \n \n </div>\n\n <div style=\"clear: both\"></div>\n</aside>\n",85 "post_number": 2,86 "post_type": 1,87 "posts_count": 2,88 "updated_at": "2025-01-08T02:11:15.879Z",89 "reply_count": 0,90 "reply_to_post_number": null,91 "quote_count": 0,92 "incoming_link_count": 14,93 "reads": 11,94 "readers_count": 10,95 "score": 72.2,96 "yours": false,97 "topic_id": 206244,98 "topic_slug": "ddp-on-iterable-dataset",99 "display_username": "MENG",100 "primary_group_name": null,101 "flair_name": null,102 "flair_url": null,103 "flair_bg_color": null,104 "flair_color": null,105 "flair_group_id": null,106 "badges_granted": [],107 "version": 4,108 "can_edit": false,109 "can_delete": false,110 "can_recover": false,111 "can_see_hidden_post": false,112 "can_wiki": false,113 "link_counts": [114 {115 "url": "https://github.com/huggingface/transformers/blob/main/src/transformers/trainer_pt_utils.py#L798",116 "internal": false,117 "reflection": false,118 "title": "transformers/src/transformers/trainer_pt_utils.py at main · huggingface/transformers · GitHub",119 "clicks": 69120 }121 ],122 "read": true,123 "user_title": null,124 "bookmarked": false,125 "actions_summary": [],126 "moderator": false,127 "admin": false,128 "staff": false,129 "user_id": 53190,130 "hidden": false,131 "trust_level": 1,132 "deleted_at": null,133 "user_deleted": false,134 "edit_reason": null,135 "can_view_edit_history": true,136 "wiki": false,137 "post_url": "/t/ddp-on-iterable-dataset/206244/2",138 "can_accept_answer": false,139 "can_unaccept_answer": false,140 "accepted_answer": false,141 "topic_accepted_answer": null142 }143 ],144 "stream": [145 448885,146 462819147 ]148 },149 "timeline_lookup": [150 [151 1,152 469153 ],154 [155 2,156 291157 ]158 ],159 "suggested_topics": [160 {161 "fancy_title": "Two models in distributed data parallel",162 "id": 216965,163 "title": "Two models in distributed data parallel",164 "slug": "two-models-in-distributed-data-parallel",165 "posts_count": 1,166 "reply_count": 0,167 "highest_post_number": 1,168 "image_url": null,169 "created_at": "2025-02-20T18:33:34.651Z",170 "last_posted_at": "2025-02-20T18:33:34.707Z",171 "bumped": true,172 "bumped_at": "2025-02-20T18:33:34.707Z",173 "archetype": "regular",174 "unseen": false,175 "pinned": false,176 "unpinned": null,177 "visible": true,178 "closed": false,179 "archived": false,180 "bookmarked": null,181 "liked": null,182 "tags_descriptions": {},183 "like_count": 0,184 "views": 57,185 "category_id": 12,186 "featured_link": null,187 "has_accepted_answer": false,188 "posters": [189 {190 "extras": "latest single",191 "description": "Original Poster, Most Recent Poster",192 "user": {193 "id": 82826,194 "username": "mf2055",195 "name": "mf2055",196 "avatar_template": "/user_avatar/discuss.pytorch.org/mf2055/{size}/75785_2.png",197 "trust_level": 0198 }199 }200 ]201 },202 {203 "fancy_title": "Distributed training raises ncclUnhandledCudaError",204 "id": 212254,205 "title": "Distributed training raises ncclUnhandledCudaError",206 "slug": "distributed-training-raises-ncclunhandledcudaerror",207 "posts_count": 2,208 "reply_count": 0,209 "highest_post_number": 2,210 "image_url": null,211 "created_at": "2024-10-29T09:38:07.642Z",212 "last_posted_at": "2024-10-29T09:39:27.244Z",213 "bumped": true,214 "bumped_at": "2024-10-29T09:39:27.244Z",215 "archetype": "regular",216 "unseen": false,217 "pinned": false,218 "unpinned": null,219 "visible": true,220 "closed": false,221 "archived": false,222 "bookmarked": null,223 "liked": null,224 "tags_descriptions": {},225 "like_count": 0,226 "views": 232,227 "category_id": 12,228 "featured_link": null,229 "has_accepted_answer": false,230 "posters": [231 {232 "extras": "latest single",233 "description": "Original Poster, Most Recent Poster",234 "user": {235 "id": 80561,236 "username": "HQJo",237 "name": "",238 "avatar_template": "/user_avatar/discuss.pytorch.org/hqjo/{size}/73651_2.png",239 "trust_level": 1240 }241 }242 ]243 },244 {245 "fancy_title": "Torch.distributed.barrier occupies additional CUDA memory",246 "id": 212502,247 "title": "Torch.distributed.barrier occupies additional CUDA memory",248 "slug": "torch-distributed-barrier-occupies-additional-cuda-memory",249 "posts_count": 1,250 "reply_count": 0,251 "highest_post_number": 1,252 "image_url": null,253 "created_at": "2024-11-04T12:49:06.441Z",254 "last_posted_at": "2024-11-04T12:49:06.619Z",255 "bumped": true,256 "bumped_at": "2024-11-04T12:49:06.619Z",257 "archetype": "regular",258 "unseen": false,259 "pinned": false,260 "unpinned": null,261 "visible": true,262 "closed": false,263 "archived": false,264 "bookmarked": null,265 "liked": null,266 "tags_descriptions": {},267 "like_count": 0,268 "views": 99,269 "category_id": 12,270 "featured_link": null,271 "has_accepted_answer": false,272 "posters": [273 {274 "extras": "latest single",275 "description": "Original Poster, Most Recent Poster",276 "user": {277 "id": 48526,278 "username": "yuan_wentao",279 "name": "yuan wentao",280 "avatar_template": "/user_avatar/discuss.pytorch.org/yuan_wentao/{size}/41715_2.png",281 "trust_level": 1282 }283 }284 ]285 },286 {287 "fancy_title": "FSDP issue with invertible networks",288 "id": 215228,289 "title": "FSDP issue with invertible networks",290 "slug": "fsdp-issue-with-invertible-networks",291 "posts_count": 2,292 "reply_count": 0,293 "highest_post_number": 4,294 "image_url": null,295 "created_at": "2025-01-10T18:29:10.087Z",296 "last_posted_at": "2025-01-17T21:23:20.825Z",297 "bumped": true,298 "bumped_at": "2025-01-17T21:23:20.825Z",299 "archetype": "regular",300 "unseen": false,301 "pinned": false,302 "unpinned": null,303 "visible": true,304 "closed": false,305 "archived": false,306 "bookmarked": null,307 "liked": null,308 "tags_descriptions": {},309 "like_count": 1,310 "views": 237,311 "category_id": 12,312 "featured_link": null,313 "has_accepted_answer": true,314 "posters": [315 {316 "extras": "latest single",317 "description": "Original Poster, Most Recent Poster, Accepted Answer",318 "user": {319 "id": 81997,320 "username": "gseastream",321 "name": "Grant Seastream",322 "avatar_template": "/user_avatar/discuss.pytorch.org/gseastream/{size}/75026_2.png",323 "trust_level": 1324 }325 }326 ]327 },328 {329 "fancy_title": "Reshaping tensors while using model parallelism",330 "id": 218591,331 "title": "Reshaping tensors while using model parallelism",332 "slug": "reshaping-tensors-while-using-model-parallelism",333 "posts_count": 1,334 "reply_count": 0,335 "highest_post_number": 1,336 "image_url": null,337 "created_at": "2025-04-03T18:00:02.837Z",338 "last_posted_at": "2025-04-03T18:00:02.891Z",339 "bumped": true,340 "bumped_at": "2025-04-03T18:00:02.891Z",341 "archetype": "regular",342 "unseen": false,343 "pinned": false,344 "unpinned": null,345 "visible": true,346 "closed": false,347 "archived": false,348 "bookmarked": null,349 "liked": null,350 "tags_descriptions": {},351 "like_count": 0,352 "views": 61,353 "category_id": 12,354 "featured_link": null,355 "has_accepted_answer": false,356 "posters": [357 {358 "extras": "latest single",359 "description": "Original Poster, Most Recent Poster",360 "user": {361 "id": 77721,362 "username": "Ryan_Zotti",363 "name": "Ryan Zotti",364 "avatar_template": "/user_avatar/discuss.pytorch.org/ryan_zotti/{size}/61625_2.png",365 "trust_level": 1366 }367 }368 ]369 }370 ],371 "tags_descriptions": {},372 "fancy_title": "DDP on iterable Dataset?",373 "id": 206244,374 "title": "DDP on iterable Dataset?",375 "posts_count": 2,376 "created_at": "2024-07-13T08:19:27.366Z",377 "views": 761,378 "reply_count": 0,379 "like_count": 0,380 "last_posted_at": "2025-01-08T02:10:00.629Z",381 "visible": true,382 "closed": false,383 "archived": false,384 "has_summary": false,385 "archetype": "regular",386 "slug": "ddp-on-iterable-dataset",387 "category_id": 12,388 "word_count": 128,389 "deleted_at": null,390 "user_id": 71628,391 "featured_link": null,392 "pinned_globally": false,393 "pinned_at": null,394 "pinned_until": null,395 "image_url": null,396 "slow_mode_seconds": 0,397 "draft": null,398 "draft_key": "topic_206244",399 "draft_sequence": null,400 "unpinned": null,401 "pinned": false,402 "current_post_number": 1,403 "highest_post_number": 2,404 "deleted_by": null,405 "actions_summary": [406 {407 "id": 4,408 "count": 0,409 "hidden": false,410 "can_act": false411 },412 {413 "id": 8,414 "count": 0,415 "hidden": false,416 "can_act": false417 },418 {419 "id": 10,420 "count": 0,421 "hidden": false,422 "can_act": false423 },424 {425 "id": 7,426 "count": 0,427 "hidden": false,428 "can_act": false429 }430 ],431 "chunk_size": 20,432 "bookmarked": false,433 "topic_timer": null,434 "message_bus_last_id": 0,435 "participant_count": 2,436 "show_read_indicator": false,437 "thumbnails": null,438 "slow_mode_enabled_until": null,439 "can_vote": false,440 "vote_count": 0,441 "user_voted": false,442 "discourse_zendesk_plugin_zendesk_id": null,443 "discourse_zendesk_plugin_zendesk_url": "https://your-url.zendesk.com/agent/tickets/",444 "details": {445 "can_edit": false,446 "notification_level": 1,447 "participants": [448 {449 "id": 53190,450 "username": "602389789",451 "name": "MENG",452 "avatar_template": "/letter_avatar_proxy/v4/letter/6/54ee81/{size}.png",453 "post_count": 1,454 "primary_group_name": null,455 "flair_name": null,456 "flair_url": null,457 "flair_color": null,458 "flair_bg_color": null,459 "flair_group_id": null,460 "trust_level": 1461 },462 {463 "id": 71628,464 "username": "Ravi_Teja2",465 "name": "Ravi Teja",466 "avatar_template": "/user_avatar/discuss.pytorch.org/ravi_teja2/{size}/66153_2.png",467 "post_count": 1,468 "primary_group_name": null,469 "flair_name": null,470 "flair_url": null,471 "flair_color": null,472 "flair_bg_color": null,473 "flair_group_id": null,474 "trust_level": 1475 }476 ],477 "created_by": {478 "id": 71628,479 "username": "Ravi_Teja2",480 "name": "Ravi Teja",481 "avatar_template": "/user_avatar/discuss.pytorch.org/ravi_teja2/{size}/66153_2.png"482 },483 "last_poster": {484 "id": 53190,485 "username": "602389789",486 "name": "MENG",487 "avatar_template": "/letter_avatar_proxy/v4/letter/6/54ee81/{size}.png"488 },489 "links": [490 {491 "url": "https://discuss.pytorch.org/t/using-iterabledataset-with-distributeddataparallel/92589",492 "title": "Using IterableDataset with DistributedDataParallel",493 "internal": true,494 "attachment": false,495 "reflection": false,496 "clicks": 90,497 "user_id": 71628,498 "domain": "discuss.pytorch.org",499 "root_domain": "pytorch.org"500 },501 {502 "url": "https://github.com/huggingface/transformers/blob/main/src/transformers/trainer_pt_utils.py#L798",503 "title": "transformers/src/transformers/trainer_pt_utils.py at main · huggingface/transformers · GitHub",504 "internal": false,505 "attachment": false,506 "reflection": false,507 "clicks": 69,508 "user_id": 53190,509 "domain": "github.com",510 "root_domain": "github.com"511 },512 {513 "url": "https://github.com/pytorch/pytorch/pull/26547",514 "title": "ChunkDataset API proposal by thiagocrepaldi · Pull Request #26547 · pytorch/pytorch · GitHub",515 "internal": false,516 "attachment": false,517 "reflection": false,518 "clicks": 20,519 "user_id": 71628,520 "domain": "github.com",521 "root_domain": "github.com"522 }523 ]524 },525 "bookmarks": []526 },527 {528 "post_stream": {529 "posts": [530 {531 "id": 462815,532 "name": "Jichen Li",533 "username": "Roy-Kid",534 "avatar_template": "/user_avatar/discuss.pytorch.org/roy-kid/{size}/67112_2.png",535 "created_at": "2025-01-07T23:48:27.078Z",536 "cooked": "<p>Hi everyone,</p>\n<p>I am trying to move my code to tensordict(td) and vmap along batch according to <a href=\"https://discuss.pytorch.org/t/how-to-apply-vmap-on-a-heterogeneous-tensor/214109/5\" class=\"inline-onebox\">How to apply vmap on a heterogeneous tensor - #5 by soulitzer</a>. Since vmap not support batched tensordict, now I just use batchsize = 1.</p>\n<p>Nevertheless, everything worked super well until I needed to rewrite derivative module. I have a td with distance between atoms and predict energy by model, and the I need to calculate force by derivating energy w.r.t distance.</p>\n<p>Before I use <code>torch.grad</code>, very straightforward to use. But within vmap, it raises: <code>element 0 of tensors does not require grad and does not have a grad_fn</code>. I find <code>Batchedtensor</code> of energy has <code>grad_fn</code>, but “vmaped” tensor, the real tensor required_grad is false.</p>\n<p>Apparently(?), we can not combine vmap with torch.grad, but <code>torch.func.grad</code>, according to <a href=\"https://discuss.pytorch.org/t/use-vmap-and-grad-to-calculate-gradients-for-one-layer-independently-for-each-input-in-batch/187556/2\" class=\"inline-onebox\">Use vmap and grad to calculate gradients for one layer independently for each input in batch - #2 by AlphaBetaGamma96</a>, and <a href=\"https://discuss.pytorch.org/t/simple-use-case-compete-per-sample-gradient-with-autograd/207317/4\" class=\"inline-onebox\">Simple use case: Compete per sample gradient with autograd - #4 by AlphaBetaGamma96</a></p>\n<p>Since <code>torch.func.vmap</code> can only specify the position of arguments, how can I derivate <code>energy</code> w.r.t <code>distance</code>. Here is my pesudo code:</p>\n<pre><code class=\"lang-auto\">inputs = td({'distance': tensor})\ninputs = model(inputs) # predict energy\ndedx = torch.grad(inputs['energy'], inputs['distance'], torch.zeros(len(energy), 3))\n# how to replace torch.grad with torch.vmap?\n</code></pre>\n<p>So thx for your help!</p>\n<p>Also post in discussion of tensordict <a href=\"https://github.com/pytorch/tensordict/discussions/1167\" class=\"inline-onebox\" rel=\"noopener nofollow ugc\">Combine vmap, func.grad with tensordict · pytorch/tensordict · Discussion #1167 · GitHub</a></p>",537 "post_number": 1,538 "post_type": 1,539 "posts_count": 1,540 "updated_at": "2025-01-07T23:52:21.145Z",541 "reply_count": 0,542 "reply_to_post_number": null,543 "quote_count": 0,544 "incoming_link_count": 50,545 "reads": 12,546 "readers_count": 11,547 "score": 252.4,548 "yours": false,549 "topic_id": 215086,550 "topic_slug": "combine-vmap-func-grad-with-tensordict",551 "display_username": "Jichen Li",552 "primary_group_name": null,553 "flair_name": null,554 "flair_url": null,555 "flair_bg_color": null,556 "flair_color": null,557 "flair_group_id": null,558 "badges_granted": [],559 "version": 1,560 "can_edit": false,561 "can_delete": false,562 "can_recover": false,563 "can_see_hidden_post": false,564 "can_wiki": false,565 "link_counts": [566 {567 "url": "https://github.com/pytorch/tensordict/discussions/1167",568 "internal": false,569 "reflection": false,570 "title": "Combine vmap, func.grad with tensordict · pytorch/tensordict · Discussion #1167 · GitHub",571 "clicks": 5572 },573 {574 "url": "https://discuss.pytorch.org/t/how-to-apply-vmap-on-a-heterogeneous-tensor/214109/5",575 "internal": true,576 "reflection": false,577 "title": "How to apply vmap on a heterogeneous tensor",578 "clicks": 4579 },580 {581 "url": "https://discuss.pytorch.org/t/use-vmap-and-grad-to-calculate-gradients-for-one-layer-independently-for-each-input-in-batch/187556/2",582 "internal": true,583 "reflection": false,584 "title": "Use vmap and grad to calculate gradients for one layer independently for each input in batch",585 "clicks": 2586 },587 {588 "url": "https://discuss.pytorch.org/t/simple-use-case-compete-per-sample-gradient-with-autograd/207317/4",589 "internal": true,590 "reflection": false,591 "title": "Simple use case: Compete per sample gradient with autograd",592 "clicks": 0593 }594 ],595 "read": true,596 "user_title": null,597 "bookmarked": false,598 "actions_summary": [],599 "moderator": false,600 "admin": false,601 "staff": false,602 "user_id": 72671,603 "hidden": false,604 "trust_level": 1,605 "deleted_at": null,606 "user_deleted": false,607 "edit_reason": null,608 "can_view_edit_history": true,609 "wiki": false,610 "post_url": "/t/combine-vmap-func-grad-with-tensordict/215086/1",611 "can_accept_answer": false,612 "can_unaccept_answer": false,613 "accepted_answer": false,614 "topic_accepted_answer": null,615 "can_vote": false616 }617 ],618 "stream": [619 462815620 ]621 },622 "timeline_lookup": [623 [624 1,625 291626 ]627 ],628 "suggested_topics": [629 {630 "fancy_title": "[torch.export] How to disable sym_size_int?",631 "id": 216199,632 "title": "[torch.export] How to disable sym_size_int?",633 "slug": "torch-export-how-to-disable-sym-size-int",634 "posts_count": 3,635 "reply_count": 1,636 "highest_post_number": 3,637 "image_url": null,638 "created_at": "2025-02-04T03:12:05.401Z",639 "last_posted_at": "2025-02-07T01:12:12.848Z",640 "bumped": true,641 "bumped_at": "2025-02-07T01:12:12.848Z",642 "archetype": "regular",643 "unseen": false,644 "pinned": false,645 "unpinned": null,646 "visible": true,647 "closed": false,648 "archived": false,649 "bookmarked": null,650 "liked": null,651 "tags_descriptions": {},652 "like_count": 0,653 "views": 200,654 "category_id": 31,655 "featured_link": null,656 "has_accepted_answer": false,657 "posters": [658 {659 "extras": "latest",660 "description": "Original Poster, Most Recent Poster",661 "user": {662 "id": 82468,663 "username": "Sung_Hyuck_Hong",664 "name": "Sung Hyuck Hong",665 "avatar_template": "/user_avatar/discuss.pytorch.org/sung_hyuck_hong/{size}/75455_2.png",666 "trust_level": 1667 }668 },669 {670 "extras": null,671 "description": "Frequent Poster",672 "user": {673 "id": 3718,674 "username": "richard",675 "name": "Richard Zou",676 "avatar_template": "/user_avatar/discuss.pytorch.org/richard/{size}/1861_2.png",677 "moderator": true,678 "trust_level": 2679 }680 }681 ]682 },683 {684 "fancy_title": "How to understand the files generated by TORCH_COMPILE_DEBUG",685 "id": 222854,686 "title": "How to understand the files generated by TORCH_COMPILE_DEBUG",687 "slug": "how-to-understand-the-files-generated-by-torch-compile-debug",688 "posts_count": 1,689 "reply_count": 0,690 "highest_post_number": 1,691 "image_url": "https://discuss.pytorch.org/uploads/default/original/3X/b/9/b961d178bff6bf56487eb75ce3c69d16f5b1f32a.png",692 "created_at": "2025-09-04T03:30:32.245Z",693 "last_posted_at": "2025-09-04T03:30:32.310Z",694 "bumped": true,695 "bumped_at": "2025-09-04T03:30:32.310Z",696 "archetype": "regular",697 "unseen": false,698 "pinned": false,699 "unpinned": null,700 "visible": true,701 "closed": false,702 "archived": false,703 "bookmarked": null,704 "liked": null,705 "tags_descriptions": {},706 "like_count": 1,707 "views": 19,708 "category_id": 31,709 "featured_link": null,710 "has_accepted_answer": false,711 "posters": [712 {713 "extras": "latest single",714 "description": "Original Poster, Most Recent Poster",715 "user": {716 "id": 85741,717 "username": "Guitenbay",718 "name": "hummel mao",719 "avatar_template": "/user_avatar/discuss.pytorch.org/guitenbay/{size}/78134_2.png",720 "trust_level": 0721 }722 }723 ]724 },725 {726 "fancy_title": "A node type in export IR graph",727 "id": 213565,728 "title": "A node type in export IR graph",729 "slug": "a-node-type-in-export-ir-graph",730 "posts_count": 2,731 "reply_count": 0,732 "highest_post_number": 2,733 "image_url": null,734 "created_at": "2024-11-28T04:40:32.947Z",735 "last_posted_at": "2024-11-28T05:20:57.217Z",736 "bumped": true,737 "bumped_at": "2024-11-28T05:20:57.217Z",738 "archetype": "regular",739 "unseen": false,740 "pinned": false,741 "unpinned": null,742 "visible": true,743 "closed": false,744 "archived": false,745 "bookmarked": null,746 "liked": null,747 "tags_descriptions": {},748 "like_count": 0,749 "views": 177,750 "category_id": 31,751 "featured_link": null,752 "has_accepted_answer": true,753 "posters": [754 {755 "extras": "latest single",756 "description": "Original Poster, Most Recent Poster, Accepted Answer",757 "user": {758 "id": 75504,759 "username": "mhs4670go",760 "name": "seongwoo chae",761 "avatar_template": "/user_avatar/discuss.pytorch.org/mhs4670go/{size}/69729_2.png",762 "trust_level": 1763 }764 }765 ]766 },767 {768 "fancy_title": "Aot_export_module captures more forward variables/parameters",769 "id": 215320,770 "title": "Aot_export_module captures more forward variables/parameters",771 "slug": "aot-export-module-captures-more-forward-variables-parameters",772 "posts_count": 3,773 "reply_count": 1,774 "highest_post_number": 3,775 "image_url": null,776 "created_at": "2025-01-13T09:52:57.803Z",777 "last_posted_at": "2025-01-23T09:10:36.382Z",778 "bumped": true,779 "bumped_at": "2025-01-23T09:10:36.382Z",780 "archetype": "regular",781 "unseen": false,782 "pinned": false,783 "unpinned": null,784 "visible": true,785 "closed": false,786 "archived": false,787 "bookmarked": null,788 "liked": null,789 "tags_descriptions": {},790 "like_count": 0,791 "views": 65,792 "category_id": 31,793 "featured_link": null,794 "has_accepted_answer": false,795 "posters": [796 {797 "extras": "latest",798 "description": "Original Poster, Most Recent Poster",799 "user": {800 "id": 75805,801 "username": "sycz00",802 "name": "Fabi",803 "avatar_template": "/user_avatar/discuss.pytorch.org/sycz00/{size}/69997_2.png",804 "trust_level": 1805 }806 },807 {808 "extras": null,809 "description": "Frequent Poster",810 "user": {811 "id": 41997,812 "username": "bdhirsh",813 "name": "Brian Hirsh",814 "avatar_template": "/user_avatar/discuss.pytorch.org/bdhirsh/{size}/74038_2.png",815 "trust_level": 2816 }817 }818 ]819 },820 {821 "fancy_title": "TS2EPConverter and `dynamic_shapes`",822 "id": 223472,823 "title": "TS2EPConverter and `dynamic_shapes`",824 "slug": "ts2epconverter-and-dynamic-shapes",825 "posts_count": 1,826 "reply_count": 0,827 "highest_post_number": 1,828 "image_url": null,829 "created_at": "2025-10-05T14:27:54.076Z",830 "last_posted_at": "2025-10-05T14:27:54.128Z",831 "bumped": true,832 "bumped_at": "2025-10-05T14:27:54.128Z",833 "archetype": "regular",834 "unseen": false,835 "pinned": false,836 "unpinned": null,837 "visible": true,838 "closed": false,839 "archived": false,840 "bookmarked": null,841 "liked": null,842 "tags_descriptions": {},843 "like_count": 0,844 "views": 12,845 "category_id": 31,846 "featured_link": null,847 "has_accepted_answer": false,848 "posters": [849 {850 "extras": "latest single",851 "description": "Original Poster, Most Recent Poster",852 "user": {853 "id": 86070,854 "username": "olokobayusuf",855 "name": "Yusuf",856 "avatar_template": "/user_avatar/discuss.pytorch.org/olokobayusuf/{size}/78371_2.png",857 "trust_level": 0858 }859 }860 ]861 }862 ],863 "tags_descriptions": {},864 "fancy_title": "Combine vmap, func.grad with tensordict",865 "id": 215086,866 "title": "Combine vmap, func.grad with tensordict",867 "posts_count": 1,868 "created_at": "2025-01-07T23:48:27.022Z",869 "views": 130,870 "reply_count": 0,871 "like_count": 0,872 "last_posted_at": "2025-01-07T23:48:27.078Z",873 "visible": true,874 "closed": false,875 "archived": false,876 "has_summary": false,877 "archetype": "regular",878 "slug": "combine-vmap-func-grad-with-tensordict",879 "category_id": 31,880 "word_count": 261,881 "deleted_at": null,882 "user_id": 72671,883 "featured_link": null,884 "pinned_globally": false,885 "pinned_at": null,886 "pinned_until": null,887 "image_url": null,888 "slow_mode_seconds": 0,889 "draft": null,890 "draft_key": "topic_215086",891 "draft_sequence": null,892 "unpinned": null,893 "pinned": false,894 "current_post_number": 1,895 "highest_post_number": 1,896 "deleted_by": null,897 "actions_summary": [898 {899 "id": 4,900 "count": 0,901 "hidden": false,902 "can_act": false903 },904 {905 "id": 8,906 "count": 0,907 "hidden": false,908 "can_act": false909 },910 {911 "id": 10,912 "count": 0,913 "hidden": false,914 "can_act": false915 },916 {917 "id": 7,918 "count": 0,919 "hidden": false,920 "can_act": false921 }922 ],923 "chunk_size": 20,924 "bookmarked": false,925 "topic_timer": null,926 "message_bus_last_id": 0,927 "participant_count": 1,928 "show_read_indicator": false,929 "thumbnails": null,930 "slow_mode_enabled_until": null,931 "can_vote": false,932 "vote_count": 0,933 "user_voted": false,934 "discourse_zendesk_plugin_zendesk_id": null,935 "discourse_zendesk_plugin_zendesk_url": "https://your-url.zendesk.com/agent/tickets/",936 "details": {937 "can_edit": false,938 "notification_level": 1,939 "participants": [940 {941 "id": 72671,942 "username": "Roy-Kid",943 "name": "Jichen Li",944 "avatar_template": "/user_avatar/discuss.pytorch.org/roy-kid/{size}/67112_2.png",945 "post_count": 1,946 "primary_group_name": null,947 "flair_name": null,948 "flair_url": null,949 "flair_color": null,950 "flair_bg_color": null,951 "flair_group_id": null,952 "trust_level": 1953 }954 ],955 "created_by": {956 "id": 72671,957 "username": "Roy-Kid",958 "name": "Jichen Li",959 "avatar_template": "/user_avatar/discuss.pytorch.org/roy-kid/{size}/67112_2.png"960 },961 "last_poster": {962 "id": 72671,963 "username": "Roy-Kid",964 "name": "Jichen Li",965 "avatar_template": "/user_avatar/discuss.pytorch.org/roy-kid/{size}/67112_2.png"966 },967 "links": [968 {969 "url": "https://github.com/pytorch/tensordict/discussions/1167",970 "title": "Combine vmap, func.grad with tensordict · pytorch/tensordict · Discussion #1167 · GitHub",971 "internal": false,972 "attachment": false,973 "reflection": false,974 "clicks": 5,975 "user_id": 72671,976 "domain": "github.com",977 "root_domain": "github.com"978 },979 {980 "url": "https://discuss.pytorch.org/t/how-to-apply-vmap-on-a-heterogeneous-tensor/214109/5",981 "title": "How to apply vmap on a heterogeneous tensor",982 "internal": true,983 "attachment": false,984 "reflection": false,985 "clicks": 4,986 "user_id": 72671,987 "domain": "discuss.pytorch.org",988 "root_domain": "pytorch.org"989 },990 {991 "url": "https://discuss.pytorch.org/t/use-vmap-and-grad-to-calculate-gradients-for-one-layer-independently-for-each-input-in-batch/187556/2",992 "title": "Use vmap and grad to calculate gradients for one layer independently for each input in batch",993 "internal": true,994 "attachment": false,995 "reflection": false,996 "clicks": 2,997 "user_id": 72671,998 "domain": "discuss.pytorch.org",999 "root_domain": "pytorch.org"1000 }1001 ]1002 },1003 "bookmarks": []1004 },1005 {1006 "post_stream": {1007 "posts": [1008 {1009 "id": 462814,1010 "name": "",1011 "username": "sr5434",1012 "avatar_template": "/letter_avatar_proxy/v4/letter/s/58956e/{size}.png",1013 "created_at": "2025-01-07T22:53:23.939Z",1014 "cooked": "<p>I am using AlphaZero to program a connect four game. Whenever I input the board, the policy and value outputs stay the same. Even when I input random data, the output never changes. I added print statements after the first convolutional and after the ResNet blocks and they output random values. When I input random data into the Policy and Value heads, they output random data. But when I run the model as it is meant to be run, the output stays the same. Here is my model code:</p>\n<pre data-code-wrap=\"python\"><code class=\"lang-python\">import torch\nfrom torch import nn\nclass Block(nn.Module):\n def __init__(self, num_channels):\n super(Block, self).__init__()\n self.conv = nn.Conv2d(num_channels, num_channels, kernel_size=3, padding=1)\n self.relu = nn.ReLU()\n self.conv2 = nn.Conv2d(num_channels, num_channels, kernel_size=3, padding=1)\n self.batch_norm = nn.BatchNorm2d(num_channels)\n def forward(self, x):\n i = x\n # print(\"Shape of x before the block\", x.shape)\n #x = self.batch_norm(x)\n x = self.relu(x)\n x = self.conv(x)\n # print(\"Shape of x after the first conv\", x.shape)\n #x = self.batch_norm(x)\n x = self.relu(x)\n x = self.conv2(x)\n # print(\"Shape of x after the second conv\", x.shape)\n return x + i\n\nclass PolicyHead(nn.Module):\n def __init__(self, num_channels, num_actions):\n super(PolicyHead, self).__init__()\n self.num_channels = num_channels\n self.conv = nn.Conv2d(num_channels, 2, kernel_size=1)\n self.relu = nn.ReLU()\n self.batch_norm = nn.BatchNorm2d(2)\n self.fc = nn.Linear(in_features=84, out_features=num_actions)\n def forward(self, x):\n x = x.reshape(-1, self.num_channels, 6, 7)\n x = self.conv(x)\n #x = self.batch_norm(x)\n x = self.relu(x)\n x = torch.flatten(x, 1)\n return self.fc(x)\n\nclass ValueHead(nn.Module):\n def __init__(self, num_channels):\n super(ValueHead, self).__init__()\n self.num_channels = num_channels\n self.conv = nn.Conv2d(num_channels, 2, kernel_size=1)\n self.relu = nn.ReLU()\n self.batch_norm = nn.BatchNorm2d(2)\n self.fc = nn.Linear(84, num_channels)\n self.fc2 = nn.Linear(num_channels, 1)\n def forward(self, x):\n x = x.reshape(-1, self.num_channels, 6, 7)\n x = self.conv(x)\n #x = self.batch_norm(x)\n x = self.relu(x)\n x = torch.flatten(x, 1)\n x = self.fc(x)\n x = self.relu(x)\n x = self.fc2(x)\n x = torch.tanh(x)\n return x\n\nclass AlphaZero(nn.Module):\n def __init__(self, num_channels=128, num_actions=7, num_blocks=20):\n super(AlphaZero, self).__init__()\n self.num_channels = num_channels\n self.conv = nn.Conv2d(1, num_channels, kernel_size=3, padding=1)\n self.blocks = nn.Sequential(*[Block(num_channels) for _ in range(num_blocks)])\n self.policy_head = PolicyHead(num_channels, num_actions)\n self.value_head = ValueHead(num_channels)\n def forward(self, x):\n x = x.view(-1, 1, 6, 7)\n x = self.conv(x)\n x = self.blocks(x)\n # print(\"Shape of x before the view\", x.shape)\n x = x.view(-1, self.num_channels)\n # print(\"Shape of x after the view\", x.shape)\n return self.policy_head(x), self.value_head(x)\n</code></pre>\n<p>Here is some code I used to test it:</p>\n<pre data-code-wrap=\"python\"><code class=\"lang-python\">weights = torch.load(\"./model-880\", weights_only=True)\nmodel = AlphaZero(num_channels=32, num_actions=7, num_blocks=8).to(\"cuda\")\nmodel.load_state_dict(weights['model_state_dict'])\nx = torch.randint(10, (1, 6, 7), dtype=torch.float).to(\"cuda\")\nprint(x)\nprint(model(x))\n</code></pre>\n<p>During training, the loss fluctuated.</p>",1015 "post_number": 1,1016 "post_type": 1,1017 "posts_count": 1,1018 "updated_at": "2025-01-07T22:53:23.939Z",1019 "reply_count": 0,1020 "reply_to_post_number": null,1021 "quote_count": 0,1022 "incoming_link_count": 16,1023 "reads": 4,1024 "readers_count": 3,1025 "score": 80.8,1026 "yours": false,1027 "topic_id": 215085,1028 "topic_slug": "model-returning-same-output-despite-random-inputs",1029 "display_username": "",1030 "primary_group_name": null,1031 "flair_name": null,1032 "flair_url": null,1033 "flair_bg_color": null,1034 "flair_color": null,1035 "flair_group_id": null,1036 "badges_granted": [],1037 "version": 1,1038 "can_edit": false,1039 "can_delete": false,1040 "can_recover": false,1041 "can_see_hidden_post": false,1042 "can_wiki": false,1043 "read": true,1044 "user_title": null,1045 "bookmarked": false,1046 "actions_summary": [],1047 "moderator": false,1048 "admin": false,1049 "staff": false,1050 "user_id": 81928,1051 "hidden": false,1052 "trust_level": 1,1053 "deleted_at": null,1054 "user_deleted": false,1055 "edit_reason": null,1056 "can_view_edit_history": true,1057 "wiki": false,1058 "post_url": "/t/model-returning-same-output-despite-random-inputs/215085/1",1059 "can_accept_answer": false,1060 "can_unaccept_answer": false,1061 "accepted_answer": false,1062 "topic_accepted_answer": null,1063 "can_vote": false1064 }1065 ],1066 "stream": [1067 4628141068 ]1069 },1070 "timeline_lookup": [1071 [1072 1,1073 2911074 ]1075 ],1076 "suggested_topics": [1077 {1078 "fancy_title": "Pytorch GPU via conda not working",1079 "id": 212586,1080 "title": "Pytorch GPU via conda not working",1081 "slug": "pytorch-gpu-via-conda-not-working",1082 "posts_count": 5,1083 "reply_count": 3,1084 "highest_post_number": 5,1085 "image_url": null,1086 "created_at": "2024-11-06T04:00:56.989Z",1087 "last_posted_at": "2024-11-06T15:45:21.971Z",1088 "bumped": true,1089 "bumped_at": "2024-11-06T15:45:21.971Z",1090 "archetype": "regular",1091 "unseen": false,1092 "pinned": false,1093 "unpinned": null,1094 "visible": true,1095 "closed": false,1096 "archived": false,1097 "bookmarked": null,1098 "liked": null,1099 "tags_descriptions": {},1100 "like_count": 0,1101 "views": 912,1102 "category_id": 1,1103 "featured_link": null,1104 "has_accepted_answer": false,1105 "posters": [1106 {1107 "extras": "latest",1108 "description": "Original Poster, Most Recent Poster",1109 "user": {1110 "id": 80717,1111 "username": "janko",1112 "name": "",1113 "avatar_template": "/user_avatar/discuss.pytorch.org/janko/{size}/73795_2.png",1114 "trust_level": 11115 }1116 },1117 {1118 "extras": null,1119 "description": "Frequent Poster",1120 "user": {1121 "id": 3534,1122 "username": "ptrblck",1123 "name": "",1124 "avatar_template": "/user_avatar/discuss.pytorch.org/ptrblck/{size}/1823_2.png",1125 "admin": true,1126 "moderator": true,1127 "trust_level": 21128 }1129 }1130 ]1131 },1132 {1133 "fancy_title": "‘c10::Error’ error with an unknown source",1134 "id": 217263,1135 "title": "'c10::Error' error with an unknown source",1136 "slug": "c10-error-error-with-an-unknown-source",1137 "posts_count": 1,1138 "reply_count": 0,1139 "highest_post_number": 1,1140 "image_url": null,1141 "created_at": "2025-02-28T03:45:52.756Z",1142 "last_posted_at": "2025-02-28T03:45:52.793Z",1143 "bumped": true,1144 "bumped_at": "2025-02-28T03:45:52.793Z",1145 "archetype": "regular",1146 "unseen": false,1147 "pinned": false,1148 "unpinned": null,1149 "visible": true,1150 "closed": false,1151 "archived": false,1152 "bookmarked": null,1153 "liked": null,1154 "tags_descriptions": {},1155 "like_count": 0,1156 "views": 201,1157 "category_id": 1,1158 "featured_link": null,1159 "has_accepted_answer": false,1160 "posters": [1161 {1162 "extras": "latest single",1163 "description": "Original Poster, Most Recent Poster",1164 "user": {1165 "id": 82985,1166 "username": "Sina_Moeini",1167 "name": "Sina Moeini",1168 "avatar_template": "/user_avatar/discuss.pytorch.org/sina_moeini/{size}/75916_2.png",1169 "trust_level": 11170 }1171 }1172 ]1173 },1174 {1175 "fancy_title": "Acceleration for a loop",1176 "id": 220215,1177 "title": "Acceleration for a loop",1178 "slug": "acceleration-for-a-loop",1179 "posts_count": 2,1180 "reply_count": 0,1181 "highest_post_number": 2,1182 "image_url": null,1183 "created_at": "2025-05-21T06:28:42.971Z",1184 "last_posted_at": "2025-05-21T11:55:10.472Z",1185 "bumped": true,1186 "bumped_at": "2025-05-21T11:55:10.472Z",1187 "archetype": "regular",1188 "unseen": false,1189 "pinned": false,1190 "unpinned": null,1191 "visible": true,1192 "closed": false,1193 "archived": false,1194 "bookmarked": null,1195 "liked": null,1196 "tags_descriptions": {},1197 "like_count": 1,1198 "views": 45,1199 "category_id": 1,1200 "featured_link": null,