Anurag1734/cuda-error-resolution-analysis
07
1[2 {3 "post_stream": {4 "posts": [5 {6 "id": 469338,7 "name": "af c",8 "username": "af_c",9 "avatar_template": "/user_avatar/discuss.pytorch.org/af_c/{size}/76690_2.png",10 "created_at": "2025-04-17T11:15:26.318Z",11 "cooked": "<p>Suppose i have two function z = f(x), g(z), and i want to<br>\nz1 = f(x1)<br>\nz2 = f(x2)<br>\noptimize MSE(\\gradient(g) (z1), \\gradient(g) (z2)).<br>\nBut when i backward propogate the gradient of parameter of f is all zero.</p>\n<p>Here is the code</p>\n<pre><code class=\"lang-auto\">import torch\nfrom torch import nn\ndef batch_jacobian(func, x, create_graph=True, strict=True):\n def _func_sum(x):\n return func(x).sum(dim=0)\n return torch.autograd.functional.jacobian(_func_sum, x, create_graph=create_graph, strict=strict).permute(1,0,2)\n\ng = nn.Sequential(nn.Linear(2, 4), nn.ReLU(), nn.Linear(4, 8), nn.ReLU(), nn.Linear(8, 3))\nx1 = torch.rand((2, 3))\nx2 = torch.rand((2, 3))\nmseloss = nn.MSELoss()\n\ntheta = torch.arange(6).reshape(3, 2) / 10\ntheta.requires_grad = True\nprint(theta)\nsig = torch.nn.Sigmoid()\ndef f(x):\n return sig(x @ theta)\n\nopt = torch.optim.Adam([theta] + list(g.parameters()), lr=1e-3)\nopt.zero_grad()\nloss = mseloss(batch_jacobian(g, f(x1)), batch_jacobian(g, f(x2)))\nloss.backward()\nopt.step()\n</code></pre>\n<p>And the output of grad of theta is all zero</p>",12 "post_number": 1,13 "post_type": 1,14 "posts_count": 2,15 "updated_at": "2025-04-17T11:21:11.011Z",16 "reply_count": 1,17 "reply_to_post_number": null,18 "quote_count": 0,19 "incoming_link_count": 6,20 "reads": 10,21 "readers_count": 9,22 "score": 37.0,23 "yours": false,24 "topic_id": 219202,25 "topic_slug": "optimize-objective-involving-jacobian",26 "display_username": "af c",27 "primary_group_name": null,28 "flair_name": null,29 "flair_url": null,30 "flair_bg_color": null,31 "flair_color": null,32 "flair_group_id": null,33 "badges_granted": [],34 "version": 3,35 "can_edit": false,36 "can_delete": false,37 "can_recover": false,38 "can_see_hidden_post": false,39 "can_wiki": false,40 "read": true,41 "user_title": null,42 "bookmarked": false,43 "actions_summary": [],44 "moderator": false,45 "admin": false,46 "staff": false,47 "user_id": 83874,48 "hidden": false,49 "trust_level": 0,50 "deleted_at": null,51 "user_deleted": false,52 "edit_reason": null,53 "can_view_edit_history": true,54 "wiki": false,55 "post_url": "/t/optimize-objective-involving-jacobian/219202/1",56 "can_accept_answer": false,57 "can_unaccept_answer": false,58 "accepted_answer": false,59 "topic_accepted_answer": null,60 "can_vote": false61 },62 {63 "id": 469467,64 "name": "K. Frank",65 "username": "KFrank",66 "avatar_template": "/letter_avatar_proxy/v4/letter/k/ecb155/{size}.png",67 "created_at": "2025-04-21T02:09:19.449Z",68 "cooked": "<p>Hi af!</p>\n<aside class=\"quote no-group quote-modified\" data-username=\"af_c\" data-post=\"1\" data-topic=\"219202\" data-full=\"true\">\n<div class=\"title\">\n<div class=\"quote-controls\"></div>\n<img alt=\"\" width=\"24\" height=\"24\" src=\"https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/af_c/48/76690_2.png\" class=\"avatar\"> af_c:</div>\n<blockquote>\n<p>But when i backward propogate the gradient of parameter of f is all zero.<br>\n…</p>\n<pre><code class=\"lang-auto\">def batch_jacobian(func, x, create_graph=True, strict=True):\n def _func_sum(x):\n return func(x).sum(dim=0)\n return torch.autograd.functional.jacobian(_func_sum, x, create_graph=create_graph, strict=strict).permute(1,0,2)\n...\ntheta = torch.arange(6).reshape(3, 2) / 10\ntheta.requires_grad = True\n...\ndef f(x):\n return sig(x @ theta)\n...\nloss = mseloss(batch_jacobian(g, f(x1)), batch_jacobian(g, f(x2)))\n</code></pre>\n<p>And the output of grad of theta is all zero</p>\n</blockquote>\n</aside>\n<p>The problem is that <code>torch.autograd.functional.jacobian()</code> only backpropagates back to<br>\nits <code>inputs</code> argument (your <code>f (x1)</code> and <code>f (x2)</code>). It neither knows nor cares that, say, <code>f (x1)</code><br>\ndepends on <code>theta</code>, you don’t backpropagate through <code>f (x1)</code>, and therefore you never reach<br>\nthe dependence on <code>theta</code>, so you get no <code>.grad</code> for <code>theta</code>.</p>\n<p>I’ve tweaked your script so that the call to <code>f (x)</code> occurs inside of <code>batch_jacobian()</code> and<br>\ntherefore inside of the call to <code>torch.autograd.functional.jacobian()</code>. Doing so does then<br>\nproduce <code>.grad</code> for <code>theta</code>.</p>\n<p>Here is the tweaked script:</p>\n<pre data-code-wrap=\"python\"><code class=\"lang-python\">import torch\nprint (torch.__version__)\n\ntorch.manual_seed (2025)\n\nfrom torch import nn\n\ntheta = torch.arange(6).reshape(3, 2) / 10\ntheta.requires_grad = True\nprint ('theta = ...')\nprint (theta)\nsig = torch.nn.Sigmoid()\ndef f(x):\n return sig(x @ theta)\n\ndef batch_jacobian(func, x, create_graph=True, strict=True):\n def _func_sum(x):\n # return func(x).sum(dim=0)\n return func (f (x)).sum (dim=0)\n return torch.autograd.functional.jacobian(_func_sum, x, create_graph=create_graph, strict=strict).permute(1,0,2)\n\ng = nn.Sequential(nn.Linear(2, 4), nn.ReLU(), nn.Linear(4, 8), nn.ReLU(), nn.Linear(8, 3))\nx1 = torch.rand((2, 3))\nx2 = torch.rand((2, 3))\nmseloss = nn.MSELoss()\n\nopt = torch.optim.Adam([theta] + list(g.parameters()), lr=1e-3)\nopt.zero_grad()\n# loss = mseloss(batch_jacobian(g, f(x1)), batch_jacobian(g, f(x2)))\nloss = mseloss(batch_jacobian(g, x1), batch_jacobian(g, x2))\nloss.backward()\nopt.step()\n\nprint ('loss:', loss)\nprint ('theta = ...')\nprint (theta)\nprint ('theta.grad = ...')\nprint (theta.grad)\n</code></pre>\n<p>And here is its output:</p>\n<pre><code class=\"lang-plaintext\">2.6.0+cu126\ntheta = ...\ntensor([[0.0000, 0.1000],\n [0.2000, 0.3000],\n [0.4000, 0.5000]], requires_grad=True)\nloss: tensor(3.6073e-08, grad_fn=<MseLossBackward0>)\ntheta = ...\ntensor([[0.0010, 0.0990],\n [0.2009, 0.2990],\n [0.4009, 0.4990]], requires_grad=True)\ntheta.grad = ...\ntensor([[-2.0444e-07, 4.2059e-07],\n [-1.0772e-07, 2.4320e-07],\n [-1.5494e-07, 3.4528e-07]])\n</code></pre>\n<p>Best.</p>\n<p>K. Frank</p>",69 "post_number": 2,70 "post_type": 1,71 "posts_count": 2,72 "updated_at": "2025-04-21T02:09:19.449Z",73 "reply_count": 0,74 "reply_to_post_number": null,75 "quote_count": 1,76 "incoming_link_count": 1,77 "reads": 8,78 "readers_count": 7,79 "score": 6.6,80 "yours": false,81 "topic_id": 219202,82 "topic_slug": "optimize-objective-involving-jacobian",83 "display_username": "K. Frank",84 "primary_group_name": null,85 "flair_name": null,86 "flair_url": null,87 "flair_bg_color": null,88 "flair_color": null,89 "flair_group_id": null,90 "badges_granted": [],91 "version": 1,92 "can_edit": false,93 "can_delete": false,94 "can_recover": false,95 "can_see_hidden_post": false,96 "can_wiki": false,97 "read": true,98 "user_title": null,99 "bookmarked": false,100 "actions_summary": [],101 "moderator": false,102 "admin": false,103 "staff": false,104 "user_id": 18088,105 "hidden": false,106 "trust_level": 2,107 "deleted_at": null,108 "user_deleted": false,109 "edit_reason": null,110 "can_view_edit_history": true,111 "wiki": false,112 "post_url": "/t/optimize-objective-involving-jacobian/219202/2",113 "can_accept_answer": false,114 "can_unaccept_answer": false,115 "accepted_answer": false,116 "topic_accepted_answer": null117 }118 ],119 "stream": [120 469338,121 469467122 ]123 },124 "timeline_lookup": [125 [126 1,127 191128 ],129 [130 2,131 188132 ]133 ],134 "suggested_topics": [135 {136 "fancy_title": "How to check gradients for ensemble-like architecture?",137 "id": 212395,138 "title": "How to check gradients for ensemble-like architecture?",139 "slug": "how-to-check-gradients-for-ensemble-like-architecture",140 "posts_count": 4,141 "reply_count": 2,142 "highest_post_number": 4,143 "image_url": null,144 "created_at": "2024-11-01T05:22:20.132Z",145 "last_posted_at": "2024-11-07T09:31:06.861Z",146 "bumped": true,147 "bumped_at": "2024-11-07T09:31:06.861Z",148 "archetype": "regular",149 "unseen": false,150 "pinned": false,151 "unpinned": null,152 "visible": true,153 "closed": false,154 "archived": false,155 "bookmarked": null,156 "liked": null,157 "tags_descriptions": {},158 "like_count": 2,159 "views": 44,160 "category_id": 7,161 "featured_link": null,162 "has_accepted_answer": false,163 "posters": [164 {165 "extras": null,166 "description": "Original Poster",167 "user": {168 "id": 80628,169 "username": "fabrizio_chavez",170 "name": "fabrizio chavez",171 "avatar_template": "/user_avatar/discuss.pytorch.org/fabrizio_chavez/{size}/72477_2.png",172 "trust_level": 1173 }174 },175 {176 "extras": "latest",177 "description": "Most Recent Poster",178 "user": {179 "id": 80629,180 "username": "ruhaan10",181 "name": "Ruhaan",182 "avatar_template": "/letter_avatar_proxy/v4/letter/r/ecae2f/{size}.png",183 "trust_level": 1184 }185 }186 ]187 },188 {189 "fancy_title": "Autograd graph traversal",190 "id": 213658,191 "title": "Autograd graph traversal",192 "slug": "autograd-graph-traversal",193 "posts_count": 3,194 "reply_count": 1,195 "highest_post_number": 3,196 "image_url": null,197 "created_at": "2024-12-01T13:15:22.254Z",198 "last_posted_at": "2024-12-18T23:35:26.182Z",199 "bumped": true,200 "bumped_at": "2024-12-18T23:35:26.182Z",201 "archetype": "regular",202 "unseen": false,203 "pinned": false,204 "unpinned": null,205 "visible": true,206 "closed": false,207 "archived": false,208 "bookmarked": null,209 "liked": null,210 "tags_descriptions": {},211 "like_count": 1,212 "views": 259,213 "category_id": 7,214 "featured_link": null,215 "has_accepted_answer": true,216 "posters": [217 {218 "extras": "latest",219 "description": "Original Poster, Most Recent Poster",220 "user": {221 "id": 81227,222 "username": "valerian.rey",223 "name": "Valérian Rey",224 "avatar_template": "/user_avatar/discuss.pytorch.org/valerian.rey/{size}/74283_2.png",225 "trust_level": 2226 }227 },228 {229 "extras": null,230 "description": "Frequent Poster, Accepted Answer",231 "user": {232 "id": 41396,233 "username": "soulitzer",234 "name": "",235 "avatar_template": "/letter_avatar_proxy/v4/letter/s/839c29/{size}.png",236 "trust_level": 2237 }238 }239 ]240 },241 {242 "fancy_title": "Softmax returning only 0 and 1",243 "id": 216001,244 "title": "Softmax returning only 0 and 1",245 "slug": "softmax-returning-only-0-and-1",246 "posts_count": 2,247 "reply_count": 0,248 "highest_post_number": 2,249 "image_url": null,250 "created_at": "2025-01-28T20:09:14.058Z",251 "last_posted_at": "2025-01-28T21:30:35.605Z",252 "bumped": true,253 "bumped_at": "2025-01-28T21:30:35.605Z",254 "archetype": "regular",255 "unseen": false,256 "pinned": false,257 "unpinned": null,258 "visible": true,259 "closed": false,260 "archived": false,261 "bookmarked": null,262 "liked": null,263 "tags_descriptions": {},264 "like_count": 0,265 "views": 58,266 "category_id": 7,267 "featured_link": null,268 "has_accepted_answer": false,269 "posters": [270 {271 "extras": null,272 "description": "Original Poster",273 "user": {274 "id": 82375,275 "username": "jdeawick",276 "name": "jwick",277 "avatar_template": "/letter_avatar_proxy/v4/letter/j/a87d85/{size}.png",278 "trust_level": 0279 }280 },281 {282 "extras": "latest",283 "description": "Most Recent Poster",284 "user": {285 "id": 3534,286 "username": "ptrblck",287 "name": "",288 "avatar_template": "/user_avatar/discuss.pytorch.org/ptrblck/{size}/1823_2.png",289 "admin": true,290 "moderator": true,291 "trust_level": 2292 }293 }294 ]295 },296 {297 "fancy_title": "[Solved][Pytorch1.12] RuntimeError: one of the variables needed for gradient computation has been modified by an inplace operation",298 "id": 216635,299 "title": "[Solved][Pytorch1.12] RuntimeError: one of the variables needed for gradient computation has been modified by an inplace operation",300 "slug": "solved-pytorch1-12-runtimeerror-one-of-the-variables-needed-for-gradient-computation-has-been-modified-by-an-inplace-operation",301 "posts_count": 1,302 "reply_count": 0,303 "highest_post_number": 1,304 "image_url": null,305 "created_at": "2025-02-13T13:21:16.850Z",306 "last_posted_at": "2025-02-13T13:21:16.887Z",307 "bumped": true,308 "bumped_at": "2025-02-13T13:21:16.887Z",309 "archetype": "regular",310 "unseen": false,311 "pinned": false,312 "unpinned": null,313 "visible": true,314 "closed": false,315 "archived": false,316 "bookmarked": null,317 "liked": null,318 "tags_descriptions": {},319 "like_count": 0,320 "views": 28,321 "category_id": 7,322 "featured_link": null,323 "has_accepted_answer": false,324 "posters": [325 {326 "extras": "latest single",327 "description": "Original Poster, Most Recent Poster",328 "user": {329 "id": 82600,330 "username": "briwa",331 "name": "北千wa",332 "avatar_template": "/user_avatar/discuss.pytorch.org/briwa/{size}/75576_2.png",333 "trust_level": 1334 }335 }336 ]337 },338 {339 "fancy_title": "PINN for 2D Heat Conduction Always Converges to a Constant Solution",340 "id": 217547,341 "title": "PINN for 2D Heat Conduction Always Converges to a Constant Solution",342 "slug": "pinn-for-2d-heat-conduction-always-converges-to-a-constant-solution",343 "posts_count": 3,344 "reply_count": 1,345 "highest_post_number": 3,346 "image_url": null,347 "created_at": "2025-03-07T05:56:37.237Z",348 "last_posted_at": "2025-03-10T14:06:25.055Z",349 "bumped": true,350 "bumped_at": "2025-03-10T14:06:25.055Z",351 "archetype": "regular",352 "unseen": false,353 "pinned": false,354 "unpinned": null,355 "visible": true,356 "closed": false,357 "archived": false,358 "bookmarked": null,359 "liked": null,360 "tags_descriptions": {},361 "like_count": 1,362 "views": 130,363 "category_id": 7,364 "featured_link": null,365 "has_accepted_answer": false,366 "posters": [367 {368 "extras": "latest",369 "description": "Original Poster, Most Recent Poster",370 "user": {371 "id": 83120,372 "username": "YH120323",373 "name": "",374 "avatar_template": "/letter_avatar_proxy/v4/letter/y/b2d939/{size}.png",375 "trust_level": 0376 }377 },378 {379 "extras": null,380 "description": "Frequent Poster",381 "user": {382 "id": 56985,383 "username": "Grant_Norman",384 "name": "Grant Norman",385 "avatar_template": "/user_avatar/discuss.pytorch.org/grant_norman/{size}/55533_2.png",386 "trust_level": 2387 }388 }389 ]390 }391 ],392 "tags_descriptions": {},393 "fancy_title": "Optimize objective involving jacobian",394 "id": 219202,395 "title": "Optimize objective involving jacobian",396 "posts_count": 2,397 "created_at": "2025-04-17T11:15:26.265Z",398 "views": 93,399 "reply_count": 0,400 "like_count": 0,401 "last_posted_at": "2025-04-21T02:09:19.449Z",402 "visible": true,403 "closed": false,404 "archived": false,405 "has_summary": false,406 "archetype": "regular",407 "slug": "optimize-objective-involving-jacobian",408 "category_id": 7,409 "word_count": 599,410 "deleted_at": null,411 "user_id": 83874,412 "featured_link": null,413 "pinned_globally": false,414 "pinned_at": null,415 "pinned_until": null,416 "image_url": null,417 "slow_mode_seconds": 0,418 "draft": null,419 "draft_key": "topic_219202",420 "draft_sequence": null,421 "unpinned": null,422 "pinned": false,423 "current_post_number": 1,424 "highest_post_number": 2,425 "deleted_by": null,426 "actions_summary": [427 {428 "id": 4,429 "count": 0,430 "hidden": false,431 "can_act": false432 },433 {434 "id": 8,435 "count": 0,436 "hidden": false,437 "can_act": false438 },439 {440 "id": 10,441 "count": 0,442 "hidden": false,443 "can_act": false444 },445 {446 "id": 7,447 "count": 0,448 "hidden": false,449 "can_act": false450 }451 ],452 "chunk_size": 20,453 "bookmarked": false,454 "topic_timer": null,455 "message_bus_last_id": 0,456 "participant_count": 2,457 "show_read_indicator": false,458 "thumbnails": null,459 "slow_mode_enabled_until": null,460 "can_vote": false,461 "vote_count": 0,462 "user_voted": false,463 "discourse_zendesk_plugin_zendesk_id": null,464 "discourse_zendesk_plugin_zendesk_url": "https://your-url.zendesk.com/agent/tickets/",465 "details": {466 "can_edit": false,467 "notification_level": 1,468 "participants": [469 {470 "id": 18088,471 "username": "KFrank",472 "name": "K. Frank",473 "avatar_template": "/letter_avatar_proxy/v4/letter/k/ecb155/{size}.png",474 "post_count": 1,475 "primary_group_name": null,476 "flair_name": null,477 "flair_url": null,478 "flair_color": null,479 "flair_bg_color": null,480 "flair_group_id": null,481 "trust_level": 2482 },483 {484 "id": 83874,485 "username": "af_c",486 "name": "af c",487 "avatar_template": "/user_avatar/discuss.pytorch.org/af_c/{size}/76690_2.png",488 "post_count": 1,489 "primary_group_name": null,490 "flair_name": null,491 "flair_url": null,492 "flair_color": null,493 "flair_bg_color": null,494 "flair_group_id": null,495 "trust_level": 0496 }497 ],498 "created_by": {499 "id": 83874,500 "username": "af_c",501 "name": "af c",502 "avatar_template": "/user_avatar/discuss.pytorch.org/af_c/{size}/76690_2.png"503 },504 "last_poster": {505 "id": 18088,506 "username": "KFrank",507 "name": "K. Frank",508 "avatar_template": "/letter_avatar_proxy/v4/letter/k/ecb155/{size}.png"509 }510 },511 "bookmarks": []512 },513 {514 "post_stream": {515 "posts": [516 {517 "id": 453762,518 "name": "Brian Kim",519 "username": "yrkim98",520 "avatar_template": "/user_avatar/discuss.pytorch.org/yrkim98/{size}/72559_2.png",521 "created_at": "2024-09-05T16:20:02.784Z",522 "cooked": "<p>Hi all,<br>\nWe’re working on a ML app that uses pytorch, but we’re having trouble specifying the gpu version of pytorch as a dependency for the build. Our project uses <code>pyproject.toml</code> to specify all dependancies and <code>setuptools</code> for the build. Our goal is to allow both CPU and GPU (if available) runs of pytorch after a user <code>pip install</code>’s our app without any further configuration needed.</p>\n<p>we want to specify <code>torch==2.0.1+cu118</code> for windows and ubuntu users- so that if they have a GPU we will be able to use gpu pytorch. (<code>+cu118</code> will just default to cpu if there is no gpu available). We also want to specify the cpu-only <code>torch==2.0.1</code> for osx users since there is not a CUDA build for OSX.</p>\n<p>I dont believe there is a good way to do this solely using <code>pyproject.toml</code> and <code>setuptools</code>, since we have to specify the <code>index-url</code> for the <code>+cu118</code> version of pytorch which is not supported by PEP and recent versions of python.</p>\n<p>Ideally we dont have to bring in other dependency managers like <code>poetry</code> or <code>pdm</code>- but if there is a good solution with those I will consider it. The perfect solution would work with <code>pyproject.toml</code> and <code>setuptools</code>.</p>\n<p>Thanks for the help</p>",523 "post_number": 1,524 "post_type": 1,525 "posts_count": 3,526 "updated_at": "2024-09-05T23:50:28.778Z",527 "reply_count": 0,528 "reply_to_post_number": null,529 "quote_count": 0,530 "incoming_link_count": 1804,531 "reads": 17,532 "readers_count": 16,533 "score": 8848.4,534 "yours": false,535 "topic_id": 209157,536 "topic_slug": "specifying-gpu-version-of-pytorch-for-python-package-in-pyproject-toml",537 "display_username": "Brian Kim",538 "primary_group_name": null,539 "flair_name": null,540 "flair_url": null,541 "flair_bg_color": null,542 "flair_color": null,543 "flair_group_id": null,544 "badges_granted": [],545 "version": 7,546 "can_edit": false,547 "can_delete": false,548 "can_recover": false,549 "can_see_hidden_post": false,550 "can_wiki": false,551 "read": true,552 "user_title": null,553 "bookmarked": false,554 "actions_summary": [555 {556 "id": 2,557 "count": 1558 }559 ],560 "moderator": false,561 "admin": false,562 "staff": false,563 "user_id": 78709,564 "hidden": false,565 "trust_level": 0,566 "deleted_at": null,567 "user_deleted": false,568 "edit_reason": null,569 "can_view_edit_history": true,570 "wiki": false,571 "post_url": "/t/specifying-gpu-version-of-pytorch-for-python-package-in-pyproject-toml/209157/1",572 "can_accept_answer": false,573 "can_unaccept_answer": false,574 "accepted_answer": false,575 "topic_accepted_answer": null,576 "can_vote": false577 },578 {579 "id": 459695,580 "name": null,581 "username": "julfried",582 "avatar_template": "/letter_avatar_proxy/v4/letter/j/a88e4f/{size}.png",583 "created_at": "2024-11-19T17:05:26.303Z",584 "cooked": "<p>I am facing the same problem. As far as I know, poetry also does not work for this. I am curious to know if you found a solution to this problem or if anyone else has an idea, how this can be achieved.</p>\n<p>Thanks for any help!</p>",585 "post_number": 2,586 "post_type": 1,587 "posts_count": 3,588 "updated_at": "2024-11-19T17:05:26.303Z",589 "reply_count": 0,590 "reply_to_post_number": null,591 "quote_count": 0,592 "incoming_link_count": 9,593 "reads": 13,594 "readers_count": 12,595 "score": 62.6,596 "yours": false,597 "topic_id": 209157,598 "topic_slug": "specifying-gpu-version-of-pytorch-for-python-package-in-pyproject-toml",599 "display_username": null,600 "primary_group_name": null,601 "flair_name": null,602 "flair_url": null,603 "flair_bg_color": null,604 "flair_color": null,605 "flair_group_id": null,606 "badges_granted": [],607 "version": 1,608 "can_edit": false,609 "can_delete": false,610 "can_recover": false,611 "can_see_hidden_post": false,612 "can_wiki": false,613 "read": true,614 "user_title": null,615 "bookmarked": false,616 "actions_summary": [617 {618 "id": 2,619 "count": 1620 }621 ],622 "moderator": false,623 "admin": false,624 "staff": false,625 "user_id": 81005,626 "hidden": false,627 "trust_level": 0,628 "deleted_at": null,629 "user_deleted": false,630 "edit_reason": null,631 "can_view_edit_history": true,632 "wiki": false,633 "post_url": "/t/specifying-gpu-version-of-pytorch-for-python-package-in-pyproject-toml/209157/2",634 "can_accept_answer": false,635 "can_unaccept_answer": false,636 "accepted_answer": false,637 "topic_accepted_answer": null638 },639 {640 "id": 469464,641 "name": "AAA",642 "username": "AbdullahHendy",643 "avatar_template": "/user_avatar/discuss.pytorch.org/abdullahhendy/{size}/76722_2.png",644 "created_at": "2025-04-21T01:29:31.475Z",645 "cooked": "<p>I am trying to do the exact same thing. Wondering if there is a clean solution to this.</p>",646 "post_number": 3,647 "post_type": 1,648 "posts_count": 3,649 "updated_at": "2025-04-21T01:29:31.475Z",650 "reply_count": 0,651 "reply_to_post_number": null,652 "quote_count": 0,653 "incoming_link_count": 2,654 "reads": 9,655 "readers_count": 8,656 "score": 11.8,657 "yours": false,658 "topic_id": 209157,659 "topic_slug": "specifying-gpu-version-of-pytorch-for-python-package-in-pyproject-toml",660 "display_username": "AAA",661 "primary_group_name": null,662 "flair_name": null,663 "flair_url": null,664 "flair_bg_color": null,665 "flair_color": null,666 "flair_group_id": null,667 "badges_granted": [],668 "version": 1,669 "can_edit": false,670 "can_delete": false,671 "can_recover": false,672 "can_see_hidden_post": false,673 "can_wiki": false,674 "read": true,675 "user_title": null,676 "bookmarked": false,677 "actions_summary": [],678 "moderator": false,679 "admin": false,680 "staff": false,681 "user_id": 83920,682 "hidden": false,683 "trust_level": 0,684 "deleted_at": null,685 "user_deleted": false,686 "edit_reason": null,687 "can_view_edit_history": true,688 "wiki": false,689 "post_url": "/t/specifying-gpu-version-of-pytorch-for-python-package-in-pyproject-toml/209157/3",690 "can_accept_answer": false,691 "can_unaccept_answer": false,692 "accepted_answer": false,693 "topic_accepted_answer": null694 }695 ],696 "stream": [697 453762,698 459695,699 469464700 ]701 },702 "timeline_lookup": [703 [704 1,705 415706 ],707 [708 2,709 340710 ],711 [712 3,713 188714 ]715 ],716 "suggested_topics": [717 {718 "fancy_title": "Converting ‘deform_conv2d’ to coreml mlmodel format",719 "id": 213118,720 "title": "Converting ‘deform_conv2d’ to coreml mlmodel format",721 "slug": "converting-deform-conv2d-to-coreml-mlmodel-format",722 "posts_count": 3,723 "reply_count": 0,724 "highest_post_number": 3,725 "image_url": null,726 "created_at": "2024-11-18T14:20:21.602Z",727 "last_posted_at": "2024-11-28T06:39:07.159Z",728 "bumped": true,729 "bumped_at": "2024-11-28T06:39:07.159Z",730 "archetype": "regular",731 "unseen": false,732 "pinned": false,733 "unpinned": null,734 "visible": true,735 "closed": false,736 "archived": false,737 "bookmarked": null,738 "liked": null,739 "tags_descriptions": {},740 "like_count": 0,741 "views": 189,742 "category_id": 14,743 "featured_link": null,744 "has_accepted_answer": false,745 "posters": [746 {747 "extras": "latest",748 "description": "Original Poster, Most Recent Poster",749 "user": {750 "id": 77118,751 "username": "mht_1421",752 "name": "Hedi Turki",753 "avatar_template": "/user_avatar/discuss.pytorch.org/mht_1421/{size}/71166_2.png",754 "trust_level": 1755 }756 },757 {758 "extras": null,759 "description": "Frequent Poster",760 "user": {761 "id": 81183,762 "username": "Naman_Jaswani1",763 "name": "Naman Jaswani",764 "avatar_template": "/user_avatar/discuss.pytorch.org/naman_jaswani1/{size}/74246_2.png",765 "trust_level": 0766 }767 }768 ]769 },770 {771 "fancy_title": "ECC error for working GPU",772 "id": 212417,773 "title": "ECC error for working GPU",774 "slug": "ecc-error-for-working-gpu",775 "posts_count": 3,776 "reply_count": 1,777 "highest_post_number": 3,778 "image_url": null,779 "created_at": "2024-11-01T15:18:54.499Z",780 "last_posted_at": "2024-11-02T06:01:26.347Z",781 "bumped": true,782 "bumped_at": "2024-11-02T06:01:26.347Z",783 "archetype": "regular",784 "unseen": false,785 "pinned": false,786 "unpinned": null,787 "visible": true,788 "closed": false,789 "archived": false,790 "bookmarked": null,791 "liked": null,792 "tags_descriptions": {},793 "like_count": 0,794 "views": 712,795 "category_id": 14,796 "featured_link": null,797 "has_accepted_answer": false,798 "posters": [799 {800 "extras": "latest",801 "description": "Original Poster, Most Recent Poster",802 "user": {803 "id": 80637,804 "username": "Maxim_Lubov",805 "name": "Maxim Lubov",806 "avatar_template": "/user_avatar/discuss.pytorch.org/maxim_lubov/{size}/72922_2.png",807 "trust_level": 0808 }809 },810 {811 "extras": null,812 "description": "Frequent Poster",813 "user": {814 "id": 3534,815 "username": "ptrblck",816 "name": "",817 "avatar_template": "/user_avatar/discuss.pytorch.org/ptrblck/{size}/1823_2.png",818 "admin": true,819 "moderator": true,820 "trust_level": 2821 }822 }823 ]824 },825 {826 "fancy_title": "How to train RNN without a teacher?",827 "id": 213704,828 "title": "How to train RNN without a teacher?",829 "slug": "how-to-train-rnn-without-a-teacher",830 "posts_count": 1,831 "reply_count": 0,832 "highest_post_number": 1,833 "image_url": null,834 "created_at": "2024-12-02T13:35:35.680Z",835 "last_posted_at": "2024-12-02T13:35:35.727Z",836 "bumped": true,837 "bumped_at": "2024-12-02T13:35:35.727Z",838 "archetype": "regular",839 "unseen": false,840 "pinned": false,841 "unpinned": null,842 "visible": true,843 "closed": false,844 "archived": false,845 "bookmarked": null,846 "liked": null,847 "tags_descriptions": {},848 "like_count": 0,849 "views": 46,850 "category_id": 14,851 "featured_link": null,852 "has_accepted_answer": false,853 "posters": [854 {855 "extras": "latest single",856 "description": "Original Poster, Most Recent Poster",857 "user": {858 "id": 81245,859 "username": "lanskoyk",860 "name": "Kirill Lanskoy",861 "avatar_template": "/user_avatar/discuss.pytorch.org/lanskoyk/{size}/74297_2.png",862 "trust_level": 0863 }864 }865 ]866 },867 {868 "fancy_title": "I need help in building Pytorch (CPU) from Source in a Dockerfile",869 "id": 215831,870 "title": "I need help in building Pytorch (CPU) from Source in a Dockerfile",871 "slug": "i-need-help-in-building-pytorch-cpu-from-source-in-a-dockerfile",872 "posts_count": 1,873 "reply_count": 0,874 "highest_post_number": 1,875 "image_url": null,876 "created_at": "2025-01-24T17:17:45.294Z",877 "last_posted_at": "2025-01-24T17:17:45.337Z",878 "bumped": true,879 "bumped_at": "2025-01-24T17:27:07.713Z",880 "archetype": "regular",881 "unseen": false,882 "pinned": false,883 "unpinned": null,884 "visible": true,885 "closed": false,886 "archived": false,887 "bookmarked": null,888 "liked": null,889 "tags_descriptions": {},890 "like_count": 0,891 "views": 169,892 "category_id": 14,893 "featured_link": null,894 "has_accepted_answer": false,895 "posters": [896 {897 "extras": "latest single",898 "description": "Original Poster, Most Recent Poster",899 "user": {900 "id": 82296,901 "username": "CarlosERM",902 "name": "Carlos Eduardo Rocha Miranda",903 "avatar_template": "/user_avatar/discuss.pytorch.org/carloserm/{size}/75281_2.png",904 "trust_level": 1905 }906 }907 ]908 },909 {910 "fancy_title": "Workflow Forge: Open Source Launch - GPU-Native Flow Control for AI Workflows",911 "id": 220746,912 "title": "Workflow Forge: Open Source Launch - GPU-Native Flow Control for AI Workflows",913 "slug": "workflow-forge-open-source-launch-gpu-native-flow-control-for-ai-workflows",914 "posts_count": 1,915 "reply_count": 0,916 "highest_post_number": 1,917 "image_url": null,918 "created_at": "2025-06-12T05:48:58.305Z",919 "last_posted_at": "2025-06-12T05:48:58.345Z",920 "bumped": true,921 "bumped_at": "2025-06-12T05:48:58.345Z",922 "archetype": "regular",923 "unseen": false,924 "pinned": false,925 "unpinned": null,926 "visible": true,927 "closed": false,928 "archived": false,929 "bookmarked": null,930 "liked": null,931 "tags_descriptions": {},932 "like_count": 0,933 "views": 46,934 "category_id": 14,935 "featured_link": null,936 "has_accepted_answer": false,937 "posters": [938 {939 "extras": "latest single",940 "description": "Original Poster, Most Recent Poster",941 "user": {942 "id": 57908,943 "username": "smithblack",944 "name": "Christopher M O'Quinn",945 "avatar_template": "/letter_avatar_proxy/v4/letter/s/edb3f5/{size}.png",946 "trust_level": 1947 }948 }949 ]950 }951 ],952 "tags_descriptions": {},953 "fancy_title": "Specifying GPU version of pytorch for python package in pyproject.toml",954 "id": 209157,955 "title": "Specifying GPU version of pytorch for python package in pyproject.toml",956 "posts_count": 3,957 "created_at": "2024-09-05T16:20:02.727Z",958 "views": 1410,959 "reply_count": 0,960 "like_count": 2,961 "last_posted_at": "2025-04-21T01:29:31.475Z",962 "visible": true,963 "closed": false,964 "archived": false,965 "has_summary": false,966 "archetype": "regular",967 "slug": "specifying-gpu-version-of-pytorch-for-python-package-in-pyproject-toml",968 "category_id": 14,969 "word_count": 283,970 "deleted_at": null,971 "user_id": 78709,972 "featured_link": null,973 "pinned_globally": false,974 "pinned_at": null,975 "pinned_until": null,976 "image_url": null,977 "slow_mode_seconds": 0,978 "draft": null,979 "draft_key": "topic_209157",980 "draft_sequence": null,981 "unpinned": null,982 "pinned": false,983 "current_post_number": 1,984 "highest_post_number": 3,985 "deleted_by": null,986 "actions_summary": [987 {988 "id": 4,989 "count": 0,990 "hidden": false,991 "can_act": false992 },993 {994 "id": 8,995 "count": 0,996 "hidden": false,997 "can_act": false998 },999 {1000 "id": 10,1001 "count": 0,1002 "hidden": false,1003 "can_act": false1004 },1005 {1006 "id": 7,1007 "count": 0,1008 "hidden": false,1009 "can_act": false1010 }1011 ],1012 "chunk_size": 20,1013 "bookmarked": false,1014 "topic_timer": null,1015 "message_bus_last_id": 0,1016 "participant_count": 3,1017 "show_read_indicator": false,1018 "thumbnails": null,1019 "slow_mode_enabled_until": null,1020 "can_vote": false,1021 "vote_count": 0,1022 "user_voted": false,1023 "discourse_zendesk_plugin_zendesk_id": null,1024 "discourse_zendesk_plugin_zendesk_url": "https://your-url.zendesk.com/agent/tickets/",1025 "details": {1026 "can_edit": false,1027 "notification_level": 1,1028 "participants": [1029 {1030 "id": 78709,1031 "username": "yrkim98",1032 "name": "Brian Kim",1033 "avatar_template": "/user_avatar/discuss.pytorch.org/yrkim98/{size}/72559_2.png",1034 "post_count": 1,1035 "primary_group_name": null,1036 "flair_name": null,1037 "flair_url": null,1038 "flair_color": null,1039 "flair_bg_color": null,1040 "flair_group_id": null,1041 "trust_level": 01042 },1043 {1044 "id": 81005,1045 "username": "julfried",1046 "name": null,1047 "avatar_template": "/letter_avatar_proxy/v4/letter/j/a88e4f/{size}.png",1048 "post_count": 1,1049 "primary_group_name": null,1050 "flair_name": null,1051 "flair_url": null,1052 "flair_color": null,1053 "flair_bg_color": null,1054 "flair_group_id": null,1055 "trust_level": 01056 },1057 {1058 "id": 83920,1059 "username": "AbdullahHendy",1060 "name": "AAA",1061 "avatar_template": "/user_avatar/discuss.pytorch.org/abdullahhendy/{size}/76722_2.png",1062 "post_count": 1,1063 "primary_group_name": null,1064 "flair_name": null,1065 "flair_url": null,1066 "flair_color": null,1067 "flair_bg_color": null,1068 "flair_group_id": null,1069 "trust_level": 01070 }1071 ],1072 "created_by": {1073 "id": 78709,1074 "username": "yrkim98",1075 "name": "Brian Kim",1076 "avatar_template": "/user_avatar/discuss.pytorch.org/yrkim98/{size}/72559_2.png"1077 },1078 "last_poster": {1079 "id": 83920,1080 "username": "AbdullahHendy",1081 "name": "AAA",1082 "avatar_template": "/user_avatar/discuss.pytorch.org/abdullahhendy/{size}/76722_2.png"1083 }1084 },1085 "bookmarks": []1086 },1087 {1088 "post_stream": {1089 "posts": [1090 {1091 "id": 469454,1092 "name": "Nikitaved",1093 "username": "nikitaved",1094 "avatar_template": "/user_avatar/discuss.pytorch.org/nikitaved/{size}/76721_2.png",1095 "created_at": "2025-04-20T17:22:38.406Z",1096 "cooked": "<p>Hi,</p>\n<p>suppose I have a layer like<br>\n<code>layer(x) = MyLinearLayer(x) + nn.Linear1(x) + ... + nn.Lineark(x)</code> with no biases.<br>\nSuppose all <code>nn.Linear{i}.weight</code>s have the same shape, and <code>MyLinearLayer</code> represents a proxy of a weight of the very same shape. Because of the linear structure, all the weights in these modules should receive the very same gradient. This gradient I compute in a custom <code>MyLinearLayer.backward</code>. What would be the most efficient way to re-use this gradient for all the modules <code>nn.Linear{i}</code>? Of course, I could write a custom <code>layer.backward</code> to do that, but are there other, simpler ways?</p>\n<p>Thank you!</p>",1097 "post_number": 1,1098 "post_type": 1,1099 "posts_count": 3,1100 "updated_at": "2025-04-20T18:53:57.926Z",1101 "reply_count": 0,1102 "reply_to_post_number": null,1103 "quote_count": 0,1104 "incoming_link_count": 11,1105 "reads": 3,1106 "readers_count": 2,1107 "score": 55.6,1108 "yours": false,1109 "topic_id": 219274,1110 "topic_slug": "most-efficient-way-to-re-use-grad-computations-in-a-layer-which-is-a-linear-combination-of-linear-layers",1111 "display_username": "Nikitaved",1112 "primary_group_name": null,1113 "flair_name": null,1114 "flair_url": null,1115 "flair_bg_color": null,1116 "flair_color": null,1117 "flair_group_id": null,1118 "badges_granted": [],1119 "version": 4,1120 "can_edit": false,1121 "can_delete": false,1122 "can_recover": false,1123 "can_see_hidden_post": false,1124 "can_wiki": false,1125 "read": true,1126 "user_title": null,1127 "bookmarked": false,1128 "actions_summary": [],1129 "moderator": false,1130 "admin": false,1131 "staff": false,1132 "user_id": 83919,1133 "hidden": false,1134 "trust_level": 1,1135 "deleted_at": null,1136 "user_deleted": false,1137 "edit_reason": null,1138 "can_view_edit_history": true,1139 "wiki": false,1140 "post_url": "/t/most-efficient-way-to-re-use-grad-computations-in-a-layer-which-is-a-linear-combination-of-linear-layers/219274/1",1141 "can_accept_answer": false,1142 "can_unaccept_answer": false,1143 "accepted_answer": false,1144 "topic_accepted_answer": null,1145 "can_vote": false1146 },1147 {1148 "id": 469455,1149 "name": "",1150 "username": "ptrblck",1151 "avatar_template": "/user_avatar/discuss.pytorch.org/ptrblck/{size}/1823_2.png",1152 "created_at": "2025-04-20T17:27:09.668Z",1153 "cooked": "<p>I don’t understand the question completely. The incoming <code>dgrad</code> will be the same as you already explained and will be passed to the <code>backward</code> function of each module. The <code>wgrad</code> computation is specific to each module and you won’t be able to reuse anything since the computation depends on the forward activation. The outgoing <code>dgrad</code> computation uses the parameters so also unsure what you want to reuse.<br>\nCould you clarify your use case a bit more?</p>",1154 "post_number": 2,1155 "post_type": 1,1156 "posts_count": 3,1157 "updated_at": "2025-04-20T17:27:09.668Z",1158 "reply_count": 0,1159 "reply_to_post_number": null,1160 "quote_count": 0,1161 "incoming_link_count": 1,1162 "reads": 3,1163 "readers_count": 2,1164 "score": 5.6,1165 "yours": false,1166 "topic_id": 219274,1167 "topic_slug": "most-efficient-way-to-re-use-grad-computations-in-a-layer-which-is-a-linear-combination-of-linear-layers",1168 "display_username": "",1169 "primary_group_name": null,1170 "flair_name": null,1171 "flair_url": null,1172 "flair_bg_color": null,1173 "flair_color": null,1174 "flair_group_id": null,1175 "badges_granted": [],1176 "version": 1,1177 "can_edit": false,1178 "can_delete": false,1179 "can_recover": false,1180 "can_see_hidden_post": false,1181 "can_wiki": false,1182 "read": true,1183 "user_title": "",1184 "bookmarked": false,1185 "actions_summary": [],1186 "moderator": true,1187 "admin": true,1188 "staff": true,1189 "user_id": 3534,1190 "hidden": false,1191 "trust_level": 2,1192 "deleted_at": null,1193 "user_deleted": false,1194 "edit_reason": null,1195 "can_view_edit_history": true,1196 "wiki": false,1197 "post_url": "/t/most-efficient-way-to-re-use-grad-computations-in-a-layer-which-is-a-linear-combination-of-linear-layers/219274/2",1198 "can_accept_answer": false,1199 "can_unaccept_answer": false,1200 "accepted_answer": false,