Anurag1734/cuda-error-resolution-analysis
07
1[2 {3 "post_stream": {4 "posts": [5 {6 "id": 446448,7 "name": "Vijay",8 "username": "rvijayc",9 "avatar_template": "/letter_avatar_proxy/v4/letter/r/47e85d/{size}.png",10 "created_at": "2024-06-19T20:36:38.451Z",11 "cooked": "<p>I am trying to find the <em>positional</em> order of “placeholder” and “output” nodes of a torch FX graph generated by torch.export. Here is an example:</p>\n<pre data-code-wrap=\"python\"><code class=\"lang-python\">import torch\nimport torch.nn as nn\nimport torch.export\n\n# author a model.\nclass MLPNet(nn.Module):\n def __init__(self):\n super(MLPNet, self).__init__()\n self.relu1 = nn.ReLU()\n self.fc1 = nn.Linear(32, 64)\n self.relu1 = nn.ReLU()\n self.fc2 = nn.Linear(64, 32)\n self.relu2 = nn.ReLU()\n \n def forward(self, y, x):\n layer11 = self.relu1(self.fc2(x))\n layer10 = self.relu1(self.fc1(y))\n return layer10, layer11\n \n def name(self):\n return \"MLP\"\n\nmodel = MLPNet().eval()\n\n# generate a uniform distribution of data.\nn_batches = 100\n# generate some example input.\nx_in = torch.distributions.uniform.Uniform(-1, 1).sample([n_batches, 64, 32])\ny_in = torch.distributions.uniform.Uniform(-1, 1).sample([n_batches, 64, 64])\n\n# export the module \nm_export = torch.export.export(model, (x_in[0,:], y_in[0,:]))\nprint('---------------')\nprint('torch.export():')\nprint('---------------')\nm_export.module().graph.print_tabular()\nprint()\n</code></pre>\n<p>This prints the following:</p>\n<pre><code class=\"lang-auto\">opcode name target args kwargs\n------------- ---------- ------------------ ------------------ --------\nget_attr fc2_weight fc2.weight () {}\nget_attr fc2_bias fc2.bias () {}\nget_attr fc1_weight fc1.weight () {}\nget_attr fc1_bias fc1.bias () {}\nplaceholder y y () {}\nplaceholder x x () {}\ncall_function t aten.t.default (fc2_weight,) {}\ncall_function addmm aten.addmm.default (fc2_bias, x, t) {}\ncall_function relu aten.relu.default (addmm,) {}\ncall_function t_1 aten.t.default (fc1_weight,) {}\ncall_function addmm_1 aten.addmm.default (fc1_bias, y, t_1) {}\ncall_function relu_1 aten.relu.default (addmm_1,) {}\noutput output_1 output ((relu_1, relu),) {}\n</code></pre>\n<p>The input nodes are (<code>y</code>, <code>x</code>) - in that order, and the output nodes are (<code>relu_1</code>, <code>relu_0</code>) in that order.</p>\n<p>I am following this approach (using <code>torch.export.graph_signature</code>) to make an association with the input argument order (as in the original module signature), and the corresponding “placeholder” and “output” nodes.</p>\n<pre data-code-wrap=\"python\"><code class=\"lang-python\"># get the input argments of the exported module in positional order.\nfrom torch.export.graph_signature import InputKind, OutputKind\nin_args = [ spec.arg.name \n for spec in m_export.graph_signature.input_specs\n if spec.kind == InputKind.USER_INPUT\n ]\n# for each input argument find the corresponding node in the exported graph.\ngraph = m_export.module().graph\nfor idx, arg in enumerate(in_args):\n print(f'Node for Input Argument #{idx}: ')\n node = graph.find_nodes(op='placeholder', target=arg)\n print(node[0])\n\n# similarly print output nodes in positional order.\nout_args = [ spec.arg.name\n for spec in m_export.graph_signature.output_specs\n if spec.kind == OutputKind.USER_OUTPUT\n ]\nfor idx, arg in enumerate(out_args):\n print(f'Node for Output Argument #{idx}: ')\n node = [ n for n in graph.nodes \n if n.op == 'call_function' and n.name == arg ]\n print(node[0])\n</code></pre>\n<p>This prints the following which seems correct on the first glace -</p>\n<pre><code class=\"lang-auto\">Node for Input Argument #0:\ny\nNode for Input Argument #1:\nx\nNode for Output Argument #0:\nrelu_1\nNode for Output Argument #1:\nrelu\n</code></pre>\n<p>My question is whether or not this approach is reliable for all types of graphs, and/or, if there is a simpler approach available to accomplish the same thing?</p>\n<p>Thank You!</p>",12 "post_number": 1,13 "post_type": 1,14 "posts_count": 2,15 "updated_at": "2024-06-19T20:36:38.451Z",16 "reply_count": 0,17 "reply_to_post_number": null,18 "quote_count": 0,19 "incoming_link_count": 36,20 "reads": 12,21 "readers_count": 11,22 "score": 177.4,23 "yours": false,24 "topic_id": 204947,25 "topic_slug": "how-to-get-positional-order-of-inputs-and-outputs-for-a-graph-that-was-exported-via-torch-export",26 "display_username": "Vijay",27 "primary_group_name": null,28 "flair_name": null,29 "flair_url": null,30 "flair_bg_color": null,31 "flair_color": null,32 "flair_group_id": null,33 "badges_granted": [],34 "version": 1,35 "can_edit": false,36 "can_delete": false,37 "can_recover": false,38 "can_see_hidden_post": false,39 "can_wiki": false,40 "link_counts": [41 {42 "url": "https://discuss.pytorch.org/t/how-to-obtain-input-variable-of-each-operand-of-a-fx-ir/220688/2",43 "internal": true,44 "reflection": true,45 "title": "How to obtain input variable of each operand of a fx ir",46 "clicks": 147 }48 ],49 "read": true,50 "user_title": null,51 "bookmarked": false,52 "actions_summary": [],53 "moderator": false,54 "admin": false,55 "staff": false,56 "user_id": 70828,57 "hidden": false,58 "trust_level": 1,59 "deleted_at": null,60 "user_deleted": false,61 "edit_reason": null,62 "can_view_edit_history": true,63 "wiki": false,64 "post_url": "/t/how-to-get-positional-order-of-inputs-and-outputs-for-a-graph-that-was-exported-via-torch-export/204947/1",65 "can_accept_answer": false,66 "can_unaccept_answer": false,67 "accepted_answer": false,68 "topic_accepted_answer": null,69 "can_vote": false70 },71 {72 "id": 471041,73 "name": "Zhengxu Chen",74 "username": "zhxchen17",75 "avatar_template": "/user_avatar/discuss.pytorch.org/zhxchen17/{size}/75908_2.png",76 "created_at": "2025-05-26T14:28:35.263Z",77 "cooked": "<p>Hi <a class=\"mention\" href=\"/u/rvijayc\">@rvijayc</a> I think your approach is reliable. We always require spec.arg.name equals to the node name in fx graph.</p>",78 "post_number": 2,79 "post_type": 1,80 "posts_count": 2,81 "updated_at": "2025-05-26T14:28:35.263Z",82 "reply_count": 0,83 "reply_to_post_number": null,84 "quote_count": 0,85 "incoming_link_count": 1,86 "reads": 6,87 "readers_count": 5,88 "score": 21.2,89 "yours": false,90 "topic_id": 204947,91 "topic_slug": "how-to-get-positional-order-of-inputs-and-outputs-for-a-graph-that-was-exported-via-torch-export",92 "display_username": "Zhengxu Chen",93 "primary_group_name": null,94 "flair_name": null,95 "flair_url": null,96 "flair_bg_color": null,97 "flair_color": null,98 "flair_group_id": null,99 "badges_granted": [],100 "version": 1,101 "can_edit": false,102 "can_delete": false,103 "can_recover": false,104 "can_see_hidden_post": false,105 "can_wiki": false,106 "read": true,107 "user_title": null,108 "bookmarked": false,109 "actions_summary": [110 {111 "id": 2,112 "count": 1113 }114 ],115 "moderator": false,116 "admin": false,117 "staff": false,118 "user_id": 82975,119 "hidden": false,120 "trust_level": 1,121 "deleted_at": null,122 "user_deleted": false,123 "edit_reason": null,124 "can_view_edit_history": true,125 "wiki": false,126 "post_url": "/t/how-to-get-positional-order-of-inputs-and-outputs-for-a-graph-that-was-exported-via-torch-export/204947/2",127 "can_accept_answer": false,128 "can_unaccept_answer": false,129 "accepted_answer": false,130 "topic_accepted_answer": null131 }132 ],133 "stream": [134 446448,135 471041136 ]137 },138 "timeline_lookup": [139 [140 1,141 493142 ],143 [144 2,145 152146 ]147 ],148 "suggested_topics": [149 {150 "fancy_title": "SYCL: feature test compile failed!",151 "id": 218168,152 "title": "SYCL: feature test compile failed!",153 "slug": "sycl-feature-test-compile-failed",154 "posts_count": 1,155 "reply_count": 0,156 "highest_post_number": 1,157 "image_url": null,158 "created_at": "2025-03-23T15:01:39.213Z",159 "last_posted_at": "2025-03-23T15:01:39.250Z",160 "bumped": true,161 "bumped_at": "2025-03-23T15:01:39.250Z",162 "archetype": "regular",163 "unseen": false,164 "pinned": false,165 "unpinned": null,166 "visible": true,167 "closed": false,168 "archived": false,169 "bookmarked": null,170 "liked": null,171 "tags_descriptions": {},172 "like_count": 0,173 "views": 93,174 "category_id": 41,175 "featured_link": null,176 "has_accepted_answer": false,177 "posters": [178 {179 "extras": "latest single",180 "description": "Original Poster, Most Recent Poster",181 "user": {182 "id": 82566,183 "username": "evstratios",184 "name": "Evstratios Moraites",185 "avatar_template": "/user_avatar/discuss.pytorch.org/evstratios/{size}/75545_2.png",186 "trust_level": 1187 }188 }189 ]190 },191 {192 "fancy_title": "FP8 `torch.empty` doesn’t work under `inductor` of pytorch 2.4.1",193 "id": 219415,194 "title": "FP8 `torch.empty` doesn't work under `inductor` of pytorch 2.4.1",195 "slug": "fp8-torch-empty-doesnt-work-under-inductor-of-pytorch-2-4-1",196 "posts_count": 3,197 "reply_count": 1,198 "highest_post_number": 3,199 "image_url": null,200 "created_at": "2025-04-24T08:07:45.156Z",201 "last_posted_at": "2025-04-24T12:19:33.625Z",202 "bumped": true,203 "bumped_at": "2025-04-24T12:19:33.625Z",204 "archetype": "regular",205 "unseen": false,206 "pinned": false,207 "unpinned": null,208 "visible": true,209 "closed": false,210 "archived": false,211 "bookmarked": null,212 "liked": null,213 "tags_descriptions": {},214 "like_count": 0,215 "views": 104,216 "category_id": 41,217 "featured_link": null,218 "has_accepted_answer": false,219 "posters": [220 {221 "extras": "latest",222 "description": "Original Poster, Most Recent Poster",223 "user": {224 "id": 83987,225 "username": "jiwei_bd",226 "name": "",227 "avatar_template": "/letter_avatar_proxy/v4/letter/j/71e660/{size}.png",228 "trust_level": 1229 }230 },231 {232 "extras": null,233 "description": "Frequent Poster",234 "user": {235 "id": 3534,236 "username": "ptrblck",237 "name": "",238 "avatar_template": "/user_avatar/discuss.pytorch.org/ptrblck/{size}/1823_2.png",239 "admin": true,240 "moderator": true,241 "trust_level": 2242 }243 }244 ]245 },246 {247 "fancy_title": "Sharing torch compile kernels between layers",248 "id": 214118,249 "title": "Sharing torch compile kernels between layers",250 "slug": "sharing-torch-compile-kernels-between-layers",251 "posts_count": 2,252 "reply_count": 0,253 "highest_post_number": 2,254 "image_url": null,255 "created_at": "2024-12-11T19:15:43.508Z",256 "last_posted_at": "2024-12-11T19:37:46.577Z",257 "bumped": true,258 "bumped_at": "2024-12-11T19:37:46.577Z",259 "archetype": "regular",260 "unseen": false,261 "pinned": false,262 "unpinned": null,263 "visible": true,264 "closed": false,265 "archived": false,266 "bookmarked": null,267 "liked": null,268 "tags_descriptions": {},269 "like_count": 1,270 "views": 70,271 "category_id": 41,272 "featured_link": null,273 "has_accepted_answer": true,274 "posters": [275 {276 "extras": null,277 "description": "Original Poster",278 "user": {279 "id": 970,280 "username": "divinho",281 "name": "",282 "avatar_template": "/letter_avatar_proxy/v4/letter/d/9dc877/{size}.png",283 "trust_level": 2284 }285 },286 {287 "extras": "latest",288 "description": "Most Recent Poster, Accepted Answer",289 "user": {290 "id": 3534,291 "username": "ptrblck",292 "name": "",293 "avatar_template": "/user_avatar/discuss.pytorch.org/ptrblck/{size}/1823_2.png",294 "admin": true,295 "moderator": true,296 "trust_level": 2297 }298 }299 ]300 },301 {302 "fancy_title": "Torch.compile: Generated Triton kernel seems wrong",303 "id": 216192,304 "title": "Torch.compile: Generated Triton kernel seems wrong",305 "slug": "torch-compile-generated-triton-kernel-seems-wrong",306 "posts_count": 5,307 "reply_count": 3,308 "highest_post_number": 5,309 "image_url": null,310 "created_at": "2025-02-04T01:04:20.319Z",311 "last_posted_at": "2025-02-04T01:50:47.404Z",312 "bumped": true,313 "bumped_at": "2025-02-04T01:50:47.404Z",314 "archetype": "regular",315 "unseen": false,316 "pinned": false,317 "unpinned": null,318 "visible": true,319 "closed": false,320 "archived": false,321 "bookmarked": null,322 "liked": null,323 "tags_descriptions": {},324 "like_count": 0,325 "views": 193,326 "category_id": 41,327 "featured_link": null,328 "has_accepted_answer": true,329 "posters": [330 {331 "extras": "latest",332 "description": "Original Poster, Most Recent Poster, Accepted Answer",333 "user": {334 "id": 10310,335 "username": "Shihab_Shahriar",336 "name": "Shihab Shahriar",337 "avatar_template": "/user_avatar/discuss.pytorch.org/shihab_shahriar/{size}/10025_2.png",338 "trust_level": 2339 }340 },341 {342 "extras": null,343 "description": "Frequent Poster",344 "user": {345 "id": 3534,346 "username": "ptrblck",347 "name": "",348 "avatar_template": "/user_avatar/discuss.pytorch.org/ptrblck/{size}/1823_2.png",349 "admin": true,350 "moderator": true,351 "trust_level": 2352 }353 }354 ]355 },356 {357 "fancy_title": "Recompilations leading to timeout",358 "id": 219670,359 "title": "Recompilations leading to timeout",360 "slug": "recompilations-leading-to-timeout",361 "posts_count": 1,362 "reply_count": 0,363 "highest_post_number": 1,364 "image_url": null,365 "created_at": "2025-05-01T20:57:03.864Z",366 "last_posted_at": "2025-05-01T20:57:03.913Z",367 "bumped": true,368 "bumped_at": "2025-05-01T20:57:03.913Z",369 "archetype": "regular",370 "unseen": false,371 "pinned": false,372 "unpinned": null,373 "visible": true,374 "closed": false,375 "archived": false,376 "bookmarked": null,377 "liked": null,378 "tags_descriptions": {},379 "like_count": 0,380 "views": 66,381 "category_id": 41,382 "featured_link": null,383 "has_accepted_answer": false,384 "posters": [385 {386 "extras": "latest single",387 "description": "Original Poster, Most Recent Poster",388 "user": {389 "id": 83411,390 "username": "Naveen_Marri",391 "name": "Naveen Marri",392 "avatar_template": "/user_avatar/discuss.pytorch.org/naveen_marri/{size}/76283_2.png",393 "trust_level": 1394 }395 }396 ]397 }398 ],399 "tags_descriptions": {},400 "fancy_title": "How to get positional order of inputs and outputs for a graph that was exported via torch.export?",401 "id": 204947,402 "title": "How to get positional order of inputs and outputs for a graph that was exported via torch.export?",403 "posts_count": 2,404 "created_at": "2024-06-19T20:36:38.331Z",405 "views": 131,406 "reply_count": 0,407 "like_count": 1,408 "last_posted_at": "2025-05-26T14:28:35.263Z",409 "visible": true,410 "closed": false,411 "archived": false,412 "has_summary": false,413 "archetype": "regular",414 "slug": "how-to-get-positional-order-of-inputs-and-outputs-for-a-graph-that-was-exported-via-torch-export",415 "category_id": 41,416 "word_count": 499,417 "deleted_at": null,418 "user_id": 70828,419 "featured_link": null,420 "pinned_globally": false,421 "pinned_at": null,422 "pinned_until": null,423 "image_url": null,424 "slow_mode_seconds": 0,425 "draft": null,426 "draft_key": "topic_204947",427 "draft_sequence": null,428 "unpinned": null,429 "pinned": false,430 "current_post_number": 1,431 "highest_post_number": 2,432 "deleted_by": null,433 "actions_summary": [434 {435 "id": 4,436 "count": 0,437 "hidden": false,438 "can_act": false439 },440 {441 "id": 8,442 "count": 0,443 "hidden": false,444 "can_act": false445 },446 {447 "id": 10,448 "count": 0,449 "hidden": false,450 "can_act": false451 },452 {453 "id": 7,454 "count": 0,455 "hidden": false,456 "can_act": false457 }458 ],459 "chunk_size": 20,460 "bookmarked": false,461 "topic_timer": null,462 "message_bus_last_id": 0,463 "participant_count": 2,464 "show_read_indicator": false,465 "thumbnails": null,466 "slow_mode_enabled_until": null,467 "can_vote": false,468 "vote_count": 0,469 "user_voted": false,470 "discourse_zendesk_plugin_zendesk_id": null,471 "discourse_zendesk_plugin_zendesk_url": "https://your-url.zendesk.com/agent/tickets/",472 "details": {473 "can_edit": false,474 "notification_level": 1,475 "participants": [476 {477 "id": 70828,478 "username": "rvijayc",479 "name": "Vijay",480 "avatar_template": "/letter_avatar_proxy/v4/letter/r/47e85d/{size}.png",481 "post_count": 1,482 "primary_group_name": null,483 "flair_name": null,484 "flair_url": null,485 "flair_color": null,486 "flair_bg_color": null,487 "flair_group_id": null,488 "trust_level": 1489 },490 {491 "id": 82975,492 "username": "zhxchen17",493 "name": "Zhengxu Chen",494 "avatar_template": "/user_avatar/discuss.pytorch.org/zhxchen17/{size}/75908_2.png",495 "post_count": 1,496 "primary_group_name": null,497 "flair_name": null,498 "flair_url": null,499 "flair_color": null,500 "flair_bg_color": null,501 "flair_group_id": null,502 "trust_level": 1503 }504 ],505 "created_by": {506 "id": 70828,507 "username": "rvijayc",508 "name": "Vijay",509 "avatar_template": "/letter_avatar_proxy/v4/letter/r/47e85d/{size}.png"510 },511 "last_poster": {512 "id": 82975,513 "username": "zhxchen17",514 "name": "Zhengxu Chen",515 "avatar_template": "/user_avatar/discuss.pytorch.org/zhxchen17/{size}/75908_2.png"516 },517 "links": [518 {519 "url": "https://discuss.pytorch.org/t/how-to-obtain-input-variable-of-each-operand-of-a-fx-ir/220688/2",520 "title": "How to obtain input variable of each operand of a fx ir",521 "internal": true,522 "attachment": false,523 "reflection": true,524 "clicks": 1,525 "user_id": 83895,526 "domain": "discuss.pytorch.org",527 "root_domain": "pytorch.org"528 }529 ]530 },531 "bookmarks": []532 },533 {534 "post_stream": {535 "posts": [536 {537 "id": 471027,538 "name": "Victor Chen",539 "username": "Victor_Chen",540 "avatar_template": "/user_avatar/discuss.pytorch.org/victor_chen/{size}/77162_2.png",541 "created_at": "2025-05-26T03:00:33.196Z",542 "cooked": "<p>I found that on my nvidia 4090, Conv2d on bfloat16 is always slower than float16. is that an expected behaviour? as i did not find previous cases.<br>\nthis is my test script, run directly with copy-paste:</p>\n<pre data-code-wrap=\"python\"><code class=\"lang-python\">import torch\nimport torch.nn as nn\nimport time\n\n# Check if CUDA is available\nif not torch.cuda.is_available():\n print(\"CUDA is not available. This script requires a GPU to test BF16/FP16 performance.\")\n exit()\n\ndevice = torch.device(\"cuda\")\n\n# Enable cuDNN benchmark mode for potentially faster convolutions\n# This should be done after device selection and before model creation if input sizes are fixed.\ntorch.backends.cudnn.benchmark = True\nprint(f\"torch.backends.cudnn.benchmark is set to: {torch.backends.cudnn.benchmark}\")\n\n\ndef test_conv_performance(batch_size, in_channels, out_channels, input_size, kernel_size, stride, padding, dtype, num_runs=100, warmup_runs=10):\n \"\"\"\n Tests the performance of a convolutional layer with the specified data type.\n\n Args:\n batch_size (int): Input batch size.\n in_channels (int): Number of input channels.\n out_channels (int): Number of output channels.\n input_size (int): Height and width of the input image (assuming square).\n kernel_size (int): Size of the convolutional kernel.\n stride (int): Stride of the convolution.\n padding (int): Padding for the convolution.\n dtype (torch.dtype): Data type to test (torch.float32, torch.float16, torch.bfloat16).\n num_runs (int): Number of actual test runs.\n warmup_runs (int): Number of warmup runs.\n\n Returns:\n float: Average execution time per forward pass in milliseconds.\n Returns float('nan') if the dtype is not supported.\n \"\"\"\n # Check for data type support on the current GPU\n if dtype == torch.bfloat16 and not torch.cuda.is_bf16_supported():\n print(f\"Warning: BF16 is not supported on this GPU ({torch.cuda.get_device_name(0)}). Skipping BF16 test.\")\n return float('nan')\n # FP16 is generally tested on CUDA\n # (Technically, FP16 can run on CPU, but performance benefits are primarily on GPU)\n\n # Create model and input data\n try:\n model = nn.Conv2d(in_channels, out_channels, kernel_size, stride, padding).to(device, dtype=dtype)\n # Generate random input tensor on the specified device and with the target dtype\n input_tensor = torch.randn(batch_size, in_channels, input_size, input_size, device=device, dtype=dtype)\n except Exception as e:\n print(f\"Error creating model or input tensor for {dtype}: {e}\")\n return float('nan')\n\n\n # Warm-up GPU: execute the operation a few times before timing\n # This helps to ensure that the GPU is in a steady state and any one-time initialization costs are paid.\n for _ in range(warmup_runs):\n try:\n _ = model(input_tensor)\n except RuntimeError as e:\n # Catch potential runtime errors during warmup, e.g., if a dtype is truly unsupported for an op\n print(f\"Runtime error during warmup for {dtype} with kernel_size={kernel_size}, padding={padding}: {e}\")\n # This might indicate an issue with the chosen parameters for this dtype on this hardware\n return float('nan')\n\n torch.cuda.synchronize(device=device) # Wait for all CUDA cores to finish warmup operations\n\n # Start timing\n # Using torch.cuda.Event for accurate GPU timing\n start_event = torch.cuda.Event(enable_timing=True)\n end_event = torch.cuda.Event(enable_timing=True)\n\n total_time_ms = 0.0\n for _ in range(num_runs):\n try:\n start_event.record()\n _ = model(input_tensor)\n end_event.record()\n torch.cuda.synchronize(device=device) # Ensure the operation is complete for accurate timing\n total_time_ms += start_event.elapsed_time(end_event) # elapsed_time returns milliseconds\n except RuntimeError as e:\n print(f\"Runtime error during timed run for {dtype} with kernel_size={kernel_size}, padding={padding}: {e}\")\n return float('nan')\n\n\n avg_time_ms = total_time_ms / num_runs\n return avg_time_ms\n\nif __name__ == \"__main__\":\n # Define test parameters\n batch_size = 64\n in_channels = 3\n out_channels = 64\n input_size = 224 # Common input size, e.g., for ImageNet models\n kernel_size = 3\n stride = 1\n padding = 1\n num_runs = 200 # Increase runs for more stable results\n warmup_runs = 20 # Sufficient warmup\n\n print(f\"PyTorch Version: {torch.__version__}\")\n if torch.cuda.is_available():\n print(f\"CUDA Version: {torch.version.cuda}\")\n print(f\"cuDNN Version: {torch.backends.cudnn.version()}\")\n print(f\"GPU: {torch.cuda.get_device_name(0)}\")\n print(f\"CUDA BF16 Supported: {torch.cuda.is_bf16_supported()}\")\n else:\n print(\"CUDA not available, exiting.\")\n exit()\n\n print(\"-\" * 50)\n print(f\"Testing Parameters:\")\n print(f\" Batch Size: {batch_size}\")\n print(f\" Input Channels: {in_channels}\")\n print(f\" Output Channels: {out_channels}\")\n print(f\" Input Size: {input_size}x{input_size}\")\n print(f\" Kernel Size: {kernel_size}\")\n print(f\" Stride: {stride}\")\n print(f\" Padding: {padding}\")\n print(f\" Number of Runs for timing: {num_runs}\")\n print(f\" Warmup Runs: {warmup_runs}\")\n print(\"-\" * 50)\n\n results = {}\n\n # Test FP32 (Single-precision floating-point)\n print(\"Testing FP32...\")\n try:\n fp32_time = test_conv_performance(batch_size, in_channels, out_channels, input_size, kernel_size, stride, padding, torch.float32, num_runs, warmup_runs)\n if not fp32_time != fp32_time: # Check for NaN\n print(f\"FP32 Average Time: {fp32_time:.3f} ms\")\n results['fp32'] = fp32_time\n else:\n print(\"FP32 test resulted in NaN.\")\n results['fp32'] = float('nan')\n except Exception as e:\n print(f\"Error during FP32 test: {e}\")\n results['fp32'] = float('nan')\n print(\"-\" * 30)\n\n # Test FP16 (Half-precision floating-point)\n print(\"Testing FP16...\")\n try:\n fp16_time = test_conv_performance(batch_size, in_channels, out_channels, input_size, kernel_size, stride, padding, torch.float16, num_runs, warmup_runs)\n if not fp16_time != fp16_time: # Check for NaN\n print(f\"FP16 Average Time: {fp16_time:.3f} ms\")\n results['fp16'] = fp16_time\n if results.get('fp32') and not results['fp32'] != results['fp32'] and not fp16_time != fp16_time:\n print(f\" Speedup vs FP32: {results['fp32'] / fp16_time:.2f}x\")\n else:\n print(\"FP16 test resulted in NaN or was skipped.\")\n results['fp16'] = float('nan')\n except Exception as e:\n print(f\"Error during FP16 test: {e}\")\n results['fp16'] = float('nan')\n print(\"-\" * 30)\n\n # Test BF16 (BFloat16 floating-point)\n print(\"Testing BF16...\")\n if torch.cuda.is_bf16_supported():\n try:\n bf16_time = test_conv_performance(batch_size, in_channels, out_channels, input_size, kernel_size, stride, padding, torch.bfloat16, num_runs, warmup_runs)\n if not bf16_time != bf16_time: # Check for NaN\n print(f\"BF16 Average Time: {bf16_time:.3f} ms\")\n results['bf16'] = bf16_time\n if results.get('fp32') and not results['fp32'] != results['fp32'] and not bf16_time != bf16_time:\n print(f\" Speedup vs FP32: {results['fp32'] / bf16_time:.2f}x\")\n else:\n print(\"BF16 test resulted in NaN or was skipped.\")\n results['bf16'] = float('nan')\n except Exception as e:\n print(f\"Error during BF16 test: {e}\")\n results['bf16'] = float('nan')\n else:\n print(\"BF16 is not supported on this GPU. Skipping test.\")\n results['bf16'] = float('nan')\n\n print(\"-\" * 50)\n print(\"Testing complete.\")\n</code></pre>\n<p>and my output is:<br>\n<div class=\"lightbox-wrapper\"><a class=\"lightbox\" href=\"https://discuss.pytorch.org/uploads/default/original/3X/0/a/0ac97ee79302f7a697f08af8e533374b07c5cbad.png\" data-download-href=\"https://discuss.pytorch.org/uploads/default/0ac97ee79302f7a697f08af8e533374b07c5cbad\" title=\"image\"><img src=\"https://discuss.pytorch.org/uploads/default/original/3X/0/a/0ac97ee79302f7a697f08af8e533374b07c5cbad.png\" alt=\"image\" data-base62-sha1=\"1xqtAIXBXBKYgmmhQ9qA32n0Ys5\" width=\"393\" height=\"500\" data-dominant-color=\"30312B\"><div class=\"meta\"><svg class=\"fa d-icon d-icon-far-image svg-icon\" aria-hidden=\"true\"><use href=\"#far-image\"></use></svg><span class=\"filename\">image</span><span class=\"informations\">522×663 11.5 KB</span><svg class=\"fa d-icon d-icon-discourse-expand svg-icon\" aria-hidden=\"true\"><use href=\"#discourse-expand\"></use></svg></div></a></div></p>",543 "post_number": 1,544 "post_type": 1,545 "posts_count": 1,546 "updated_at": "2025-05-26T03:00:33.196Z",547 "reply_count": 0,548 "reply_to_post_number": null,549 "quote_count": 0,550 "incoming_link_count": 223,551 "reads": 5,552 "readers_count": 4,553 "score": 1106.0,554 "yours": false,555 "topic_id": 220332,556 "topic_slug": "conv2d-bfloat16-slower-than-float16-on-4090",557 "display_username": "Victor Chen",558 "primary_group_name": null,559 "flair_name": null,560 "flair_url": null,561 "flair_bg_color": null,562 "flair_color": null,563 "flair_group_id": null,564 "badges_granted": [],565 "version": 1,566 "can_edit": false,567 "can_delete": false,568 "can_recover": false,569 "can_see_hidden_post": false,570 "can_wiki": false,571 "read": true,572 "user_title": null,573 "bookmarked": false,574 "actions_summary": [],575 "moderator": false,576 "admin": false,577 "staff": false,578 "user_id": 84450,579 "hidden": false,580 "trust_level": 1,581 "deleted_at": null,582 "user_deleted": false,583 "edit_reason": null,584 "can_view_edit_history": true,585 "wiki": false,586 "post_url": "/t/conv2d-bfloat16-slower-than-float16-on-4090/220332/1",587 "can_accept_answer": false,588 "can_unaccept_answer": false,589 "accepted_answer": false,590 "topic_accepted_answer": null,591 "can_vote": false592 }593 ],594 "stream": [595 471027596 ]597 },598 "timeline_lookup": [599 [600 1,601 153602 ]603 ],604 "suggested_topics": [605 {606 "fancy_title": "Any operator is supported on fp8 tensor?",607 "id": 212371,608 "title": "Any operator is supported on fp8 tensor?",609 "slug": "any-operator-is-supported-on-fp8-tensor",610 "posts_count": 8,611 "reply_count": 6,612 "highest_post_number": 8,613 "image_url": null,614 "created_at": "2024-10-31T16:06:44.298Z",615 "last_posted_at": "2024-11-05T16:48:48.277Z",616 "bumped": true,617 "bumped_at": "2024-11-05T16:48:48.277Z",618 "archetype": "regular",619 "unseen": false,620 "pinned": false,621 "unpinned": null,622 "visible": true,623 "closed": false,624 "archived": false,625 "bookmarked": null,626 "liked": null,627 "tags_descriptions": {},628 "like_count": 1,629 "views": 3166,630 "category_id": 27,631 "featured_link": null,632 "has_accepted_answer": false,633 "posters": [634 {635 "extras": "latest",636 "description": "Original Poster, Most Recent Poster",637 "user": {638 "id": 57464,639 "username": "cokespace2",640 "name": "Vince Mo",641 "avatar_template": "/user_avatar/discuss.pytorch.org/cokespace2/{size}/51244_2.png",642 "trust_level": 2643 }644 },645 {646 "extras": null,647 "description": "Frequent Poster",648 "user": {649 "id": 43941,650 "username": "marksaroufim",651 "name": "Mark Saroufim",652 "avatar_template": "/user_avatar/discuss.pytorch.org/marksaroufim/{size}/36747_2.png",653 "trust_level": 3654 }655 },656 {657 "extras": null,658 "description": "Frequent Poster",659 "user": {660 "id": 3534,661 "username": "ptrblck",662 "name": "",663 "avatar_template": "/user_avatar/discuss.pytorch.org/ptrblck/{size}/1823_2.png",664 "admin": true,665 "moderator": true,666 "trust_level": 2667 }668 }669 ]670 },671 {672 "fancy_title": "Does autocast create copies of tensors on the fly?",673 "id": 214268,674 "title": "Does autocast create copies of tensors on the fly?",675 "slug": "does-autocast-create-copies-of-tensors-on-the-fly",676 "posts_count": 3,677 "reply_count": 1,678 "highest_post_number": 3,679 "image_url": null,680 "created_at": "2024-12-16T11:23:24.995Z",681 "last_posted_at": "2024-12-16T17:30:40.412Z",682 "bumped": true,683 "bumped_at": "2024-12-16T17:30:40.412Z",684 "archetype": "regular",685 "unseen": false,686 "pinned": false,687 "unpinned": null,688 "visible": true,689 "closed": false,690 "archived": false,691 "bookmarked": null,692 "liked": null,693 "tags_descriptions": {},694 "like_count": 1,695 "views": 58,696 "category_id": 27,697 "featured_link": null,698 "has_accepted_answer": true,699 "posters": [700 {701 "extras": "latest",702 "description": "Original Poster, Most Recent Poster",703 "user": {704 "id": 81089,705 "username": "Aknw_Fen",706 "name": "Aknw Fen",707 "avatar_template": "/user_avatar/discuss.pytorch.org/aknw_fen/{size}/74156_2.png",708 "trust_level": 2709 }710 },711 {712 "extras": null,713 "description": "Frequent Poster, Accepted Answer",714 "user": {715 "id": 41396,716 "username": "soulitzer",717 "name": "",718 "avatar_template": "/letter_avatar_proxy/v4/letter/s/839c29/{size}.png",719 "trust_level": 2720 }721 }722 ]723 },724 {725 "fancy_title": "FCN ResNet18 low precision on SUNRGBD dataset",726 "id": 213193,727 "title": "FCN ResNet18 low precision on SUNRGBD dataset",728 "slug": "fcn-resnet18-low-precision-on-sunrgbd-dataset",729 "posts_count": 1,730 "reply_count": 0,731 "highest_post_number": 1,732 "image_url": null,733 "created_at": "2024-11-20T09:38:34.082Z",734 "last_posted_at": "2024-11-20T09:38:34.146Z",735 "bumped": true,736 "bumped_at": "2024-11-20T09:38:34.146Z",737 "archetype": "regular",738 "unseen": false,739 "pinned": false,740 "unpinned": null,741 "visible": true,742 "closed": false,743 "archived": false,744 "bookmarked": null,745 "liked": null,746 "tags_descriptions": {},747 "like_count": 0,748 "views": 151,749 "category_id": 27,750 "featured_link": null,751 "has_accepted_answer": false,752 "posters": [753 {754 "extras": "latest single",755 "description": "Original Poster, Most Recent Poster",756 "user": {757 "id": 80652,758 "username": "Cupa_cups",759 "name": "Luka Tankosić",760 "avatar_template": "/letter_avatar_proxy/v4/letter/c/ed655f/{size}.png",761 "trust_level": 0762 }763 }764 ]765 },766 {767 "fancy_title": "Dtype different for eval and train loop with mixed prescison",768 "id": 214095,769 "title": "Dtype different for eval and train loop with mixed prescison",770 "slug": "dtype-different-for-eval-and-train-loop-with-mixed-prescison",771 "posts_count": 6,772 "reply_count": 2,773 "highest_post_number": 6,774 "image_url": null,775 "created_at": "2024-12-11T09:51:59.380Z",776 "last_posted_at": "2024-12-12T08:06:58.415Z",777 "bumped": true,778 "bumped_at": "2024-12-13T13:41:36.570Z",779 "archetype": "regular",780 "unseen": false,781 "pinned": false,782 "unpinned": null,783 "visible": true,784 "closed": false,785 "archived": false,786 "bookmarked": null,787 "liked": null,788 "tags_descriptions": {},789 "like_count": 0,790 "views": 317,791 "category_id": 27,792 "featured_link": null,793 "has_accepted_answer": true,794 "posters": [795 {796 "extras": "latest",797 "description": "Original Poster, Most Recent Poster",798 "user": {799 "id": 81446,800 "username": "m.span",801 "name": "",802 "avatar_template": "/user_avatar/discuss.pytorch.org/m.span/{size}/74273_2.png",803 "trust_level": 0804 }805 },806 {807 "extras": null,808 "description": "Frequent Poster, Accepted Answer",809 "user": {810 "id": 3534,811 "username": "ptrblck",812 "name": "",813 "avatar_template": "/user_avatar/discuss.pytorch.org/ptrblck/{size}/1823_2.png",814 "admin": true,815 "moderator": true,816 "trust_level": 2817 }818 }819 ]820 },821 {822 "fancy_title": "PyTorch 2.x causes divergence during training with mixed precision",823 "id": 219775,824 "title": "PyTorch 2.x causes divergence during training with mixed precision",825 "slug": "pytorch-2-x-causes-divergence-during-training-with-mixed-precision",826 "posts_count": 2,827 "reply_count": 0,828 "highest_post_number": 2,829 "image_url": null,830 "created_at": "2025-05-05T16:59:56.986Z",831 "last_posted_at": "2025-05-08T13:10:55.119Z",832 "bumped": true,833 "bumped_at": "2025-05-08T13:10:55.119Z",834 "archetype": "regular",835 "unseen": false,836 "pinned": false,837 "unpinned": null,838 "visible": true,839 "closed": false,840 "archived": false,841 "bookmarked": null,842 "liked": null,843 "tags_descriptions": {},844 "like_count": 1,845 "views": 88,846 "category_id": 27,847 "featured_link": null,848 "has_accepted_answer": false,849 "posters": [850 {851 "extras": "latest single",852 "description": "Original Poster, Most Recent Poster",853 "user": {854 "id": 84160,855 "username": "TitusPullo",856 "name": "",857 "avatar_template": "/user_avatar/discuss.pytorch.org/tituspullo/{size}/76919_2.png",858 "trust_level": 1859 }860 }861 ]862 }863 ],864 "tags_descriptions": {},865 "fancy_title": "Conv2d bfloat16 slower than float16 on 4090",866 "id": 220332,867 "title": "Conv2d bfloat16 slower than float16 on 4090",868 "posts_count": 1,869 "created_at": "2025-05-26T03:00:33.152Z",870 "views": 212,871 "reply_count": 0,872 "like_count": 0,873 "last_posted_at": "2025-05-26T03:00:33.196Z",874 "visible": true,875 "closed": false,876 "archived": false,877 "has_summary": false,878 "archetype": "regular",879 "slug": "conv2d-bfloat16-slower-than-float16-on-4090",880 "category_id": 27,881 "word_count": 982,882 "deleted_at": null,883 "user_id": 84450,884 "featured_link": null,885 "pinned_globally": false,886 "pinned_at": null,887 "pinned_until": null,888 "image_url": "https://discuss.pytorch.org/uploads/default/original/3X/0/a/0ac97ee79302f7a697f08af8e533374b07c5cbad.png",889 "slow_mode_seconds": 0,890 "draft": null,891 "draft_key": "topic_220332",892 "draft_sequence": null,893 "unpinned": null,894 "pinned": false,895 "current_post_number": 1,896 "highest_post_number": 1,897 "deleted_by": null,898 "actions_summary": [899 {900 "id": 4,901 "count": 0,902 "hidden": false,903 "can_act": false904 },905 {906 "id": 8,907 "count": 0,908 "hidden": false,909 "can_act": false910 },911 {912 "id": 10,913 "count": 0,914 "hidden": false,915 "can_act": false916 },917 {918 "id": 7,919 "count": 0,920 "hidden": false,921 "can_act": false922 }923 ],924 "chunk_size": 20,925 "bookmarked": false,926 "topic_timer": null,927 "message_bus_last_id": 0,928 "participant_count": 1,929 "show_read_indicator": false,930 "thumbnails": [931 {932 "max_width": null,933 "max_height": null,934 "width": 522,935 "height": 663,936 "url": "https://discuss.pytorch.org/uploads/default/original/3X/0/a/0ac97ee79302f7a697f08af8e533374b07c5cbad.png"937 }938 ],939 "slow_mode_enabled_until": null,940 "can_vote": false,941 "vote_count": 0,942 "user_voted": false,943 "discourse_zendesk_plugin_zendesk_id": null,944 "discourse_zendesk_plugin_zendesk_url": "https://your-url.zendesk.com/agent/tickets/",945 "details": {946 "can_edit": false,947 "notification_level": 1,948 "participants": [949 {950 "id": 84450,951 "username": "Victor_Chen",952 "name": "Victor Chen",953 "avatar_template": "/user_avatar/discuss.pytorch.org/victor_chen/{size}/77162_2.png",954 "post_count": 1,955 "primary_group_name": null,956 "flair_name": null,957 "flair_url": null,958 "flair_color": null,959 "flair_bg_color": null,960 "flair_group_id": null,961 "trust_level": 1962 }963 ],964 "created_by": {965 "id": 84450,966 "username": "Victor_Chen",967 "name": "Victor Chen",968 "avatar_template": "/user_avatar/discuss.pytorch.org/victor_chen/{size}/77162_2.png"969 },970 "last_poster": {971 "id": 84450,972 "username": "Victor_Chen",973 "name": "Victor Chen",974 "avatar_template": "/user_avatar/discuss.pytorch.org/victor_chen/{size}/77162_2.png"975 }976 },977 "bookmarks": []978 },979 {980 "post_stream": {981 "posts": [982 {983 "id": 311130,984 "name": "Abhisek",985 "username": "abhisek",986 "avatar_template": "/user_avatar/discuss.pytorch.org/abhisek/{size}/38962_2.png",987 "created_at": "2021-10-11T19:51:09.014Z",988 "cooked": "<p>In <code>numpy</code> I can do the following to avoid division by zero:</p>\n<pre><code class=\"lang-python\">a = np.random.randint(0, 10, 100)\nb = np.random.randint(0, 10, 100)\nc = np.zeros_like(a, dtype=np.float32) # It can be anything other than zero\nc = np.divide(a, b, out=c, where=(b!=0))\n</code></pre>\n<p>In <code>torch.divide</code> there is <strong>no</strong> <code>where</code> argument for masking. Only way seems to be replacing <code>inf</code> with desired value after the division takes place. Is there any better way to do this in <code>pytorch</code>?</p>",989 "post_number": 1,990 "post_type": 1,991 "posts_count": 4,992 "updated_at": "2021-10-11T19:51:42.761Z",993 "reply_count": 0,994 "reply_to_post_number": null,995 "quote_count": 0,996 "incoming_link_count": 7251,997 "reads": 134,998 "readers_count": 133,999 "score": 36231.8,1000 "yours": false,1001 "topic_id": 133968,1002 "topic_slug": "torch-divide-only-where-denominator-is-non-zero",1003 "display_username": "Abhisek",1004 "primary_group_name": null,1005 "flair_name": null,1006 "flair_url": null,1007 "flair_bg_color": null,1008 "flair_color": null,1009 "flair_group_id": null,1010 "badges_granted": [],1011 "version": 1,1012 "can_edit": false,1013 "can_delete": false,1014 "can_recover": false,1015 "can_see_hidden_post": false,1016 "can_wiki": false,1017 "read": true,1018 "user_title": "",1019 "bookmarked": false,1020 "actions_summary": [1021 {1022 "id": 2,1023 "count": 11024 }1025 ],1026 "moderator": false,1027 "admin": false,1028 "staff": false,1029 "user_id": 45949,1030 "hidden": false,1031 "trust_level": 1,1032 "deleted_at": null,1033 "user_deleted": false,1034 "edit_reason": null,1035 "can_view_edit_history": true,1036 "wiki": false,1037 "post_url": "/t/torch-divide-only-where-denominator-is-non-zero/133968/1",1038 "can_accept_answer": false,1039 "can_unaccept_answer": false,1040 "accepted_answer": false,1041 "topic_accepted_answer": null,1042 "can_vote": false1043 },1044 {1045 "id": 311408,1046 "name": "",1047 "username": "ptrblck",1048 "avatar_template": "/user_avatar/discuss.pytorch.org/ptrblck/{size}/1823_2.png",1049 "created_at": "2021-10-13T06:42:39.491Z",1050 "cooked": "<p>Would selecting the desired values in <code>a</code> and <code>b</code> before applying the division work?<br>\nE.g. you could create a mask first and use it to index both tensors where the condition would be <code>b!=0</code>.</p>",1051 "post_number": 2,1052 "post_type": 1,1053 "posts_count": 4,1054 "updated_at": "2021-10-13T06:42:39.491Z",1055 "reply_count": 1,1056 "reply_to_post_number": null,1057 "quote_count": 0,1058 "incoming_link_count": 130,1059 "reads": 133,1060 "readers_count": 132,1061 "score": 726.6,1062 "yours": false,1063 "topic_id": 133968,1064 "topic_slug": "torch-divide-only-where-denominator-is-non-zero",1065 "display_username": "",1066 "primary_group_name": null,1067 "flair_name": null,1068 "flair_url": null,1069 "flair_bg_color": null,1070 "flair_color": null,1071 "flair_group_id": null,1072 "badges_granted": [],1073 "version": 1,1074 "can_edit": false,1075 "can_delete": false,1076 "can_recover": false,1077 "can_see_hidden_post": false,1078 "can_wiki": false,1079 "read": true,1080 "user_title": "",1081 "bookmarked": false,1082 "actions_summary": [1083 {1084 "id": 2,1085 "count": 31086 }1087 ],1088 "moderator": true,1089 "admin": true,1090 "staff": true,1091 "user_id": 3534,1092 "hidden": false,1093 "trust_level": 2,1094 "deleted_at": null,1095 "user_deleted": false,1096 "edit_reason": null,1097 "can_view_edit_history": true,1098 "wiki": false,1099 "post_url": "/t/torch-divide-only-where-denominator-is-non-zero/133968/2",1100 "can_accept_answer": false,1101 "can_unaccept_answer": false,1102 "accepted_answer": false,1103 "topic_accepted_answer": null1104 },1105 {1106 "id": 311453,1107 "name": "Abhisek",1108 "username": "abhisek",1109 "avatar_template": "/user_avatar/discuss.pytorch.org/abhisek/{size}/38962_2.png",1110 "created_at": "2021-10-13T08:03:36.299Z",1111 "cooked": "<p>Yes, indeed your(<a class=\"mention\" href=\"/u/ptrblck\">@ptrblck</a>) solution works pretty well. For anyone who’s looking for solution using torch see the snippet below:</p>\n<pre><code class=\"lang-python\">import torch\n\n# numerator: tensor([2., 2., 0., 5., 7., 3., 4., 3., 6., 5.])\na = torch.randint(0, 10, (10,), dtype=torch.float32)\n\n# denominator: tensor([3., 3., 0., 4., 5., 4., 7., 8., 0., 4.])\nb = torch.randint(0, 10, (10,), dtype=torch.float32)\n\n# initialize output tensor with desired value\nc = torch.full_like(a, fill_value=float('nan'))\n\n# zero mask\nmask = (b != 0)\n\n# finally perform division\nc[mask] = a[mask] / b[mask]\n\n# output: tensor([0.6667, 0.6667, nan, 1.2500, 1.4000, 0.7500, 0.5714, 0.3750, nan, 1.2500])\n\n</code></pre>",1112 "post_number": 3,1113 "post_type": 1,1114 "posts_count": 4,1115 "updated_at": "2021-10-13T08:10:30.598Z",1116 "reply_count": 1,1117 "reply_to_post_number": 2,1118 "quote_count": 0,1119 "incoming_link_count": 139,1120 "reads": 132,1121 "readers_count": 131,1122 "score": 816.4,1123 "yours": false,1124 "topic_id": 133968,1125 "topic_slug": "torch-divide-only-where-denominator-is-non-zero",1126 "display_username": "Abhisek",1127 "primary_group_name": null,1128 "flair_name": null,1129 "flair_url": null,1130 "flair_bg_color": null,1131 "flair_color": null,1132 "flair_group_id": null,1133 "badges_granted": [],1134 "version": 2,1135 "can_edit": false,1136 "can_delete": false,1137 "can_recover": false,1138 "can_see_hidden_post": false,1139 "can_wiki": false,1140 "read": true,1141 "user_title": "",1142 "reply_to_user": {1143 "id": 3534,1144 "username": "ptrblck",1145 "name": "",1146 "avatar_template": "/user_avatar/discuss.pytorch.org/ptrblck/{size}/1823_2.png"1147 },1148 "bookmarked": false,1149 "actions_summary": [1150 {1151 "id": 2,1152 "count": 61153 }1154 ],1155 "moderator": false,1156 "admin": false,1157 "staff": false,1158 "user_id": 45949,1159 "hidden": false,1160 "trust_level": 1,1161 "deleted_at": null,1162 "user_deleted": false,1163 "edit_reason": null,1164 "can_view_edit_history": true,1165 "wiki": false,1166 "post_url": "/t/torch-divide-only-where-denominator-is-non-zero/133968/3",1167 "can_accept_answer": false,1168 "can_unaccept_answer": false,1169 "accepted_answer": false,1170 "topic_accepted_answer": null1171 },1172 {1173 "id": 461186,1174 "name": "Jake Levi",1175 "username": "jakelevi1996",1176 "avatar_template": "/user_avatar/discuss.pytorch.org/jakelevi1996/{size}/75757_2.png",1177 "created_at": "2024-12-10T18:59:58.692Z",1178 "cooked": "<p>Another solution is to use <code>where</code> before you divide (this avoids initialising <code>c</code> and then conditionally modifying its elements in-place - I think this looks more elegant, but I haven’t measured its efficiency):</p>\n<pre data-code-wrap=\"python\"><code class=\"lang-python\">good_inds = (b != 0)\nfill_value = 42 # or whatever\nc = (\n torch.where(good_inds, a, fill_value) /\n torch.where(good_inds, b, 1)\n)\n</code></pre>",1179 "post_number": 4,1180 "post_type": 1,1181 "posts_count": 4,1182 "updated_at": "2024-12-10T18:59:58.692Z",1183 "reply_count": 0,1184 "reply_to_post_number": 3,1185 "quote_count": 0,1186 "incoming_link_count": 8,1187 "reads": 13,1188 "readers_count": 12,1189 "score": 57.6,1190 "yours": false,1191 "topic_id": 133968,1192 "topic_slug": "torch-divide-only-where-denominator-is-non-zero",1193 "display_username": "Jake Levi",1194 "primary_group_name": null,1195 "flair_name": null,1196 "flair_url": null,1197 "flair_bg_color": null,1198 "flair_color": null,1199 "flair_group_id": null,1200 "badges_granted": [],