Anurag1734/cuda-error-resolution-analysis
07
1[2 {3 "post_stream": {4 "posts": [5 {6 "id": 245547,7 "name": "MirandaAgent",8 "username": "Brando_Miranda",9 "avatar_template": "/user_avatar/discuss.pytorch.org/brando_miranda/{size}/14355_2.png",10 "created_at": "2020-11-19T21:58:45.568Z",11 "cooked": "<p>I am in an unusual setting where I should not use running statistics (as that would be considered cheating e.g. meta-learning). However, I often run a forward pass on a set of points (5 in fact) and then I want to evaluate only on 1 point <strong>using the previous statistics</strong> but batch norm forgets the batch statistics it just uses. I’ve tried to hard code the value it should be but I get strange errors (even when I uncomment things like from the pytorch code itself like checking the dimension size).</p>\n<p>How do I hardcode the previous batch statistics so that batch norm works on a new single data point and then reset them for a fresh new next batch?</p>\n<p>note: I don’t want to change the batch norm layer type.</p>\n<p>Sample code I tried:</p>\n<pre><code class=\"lang-auto\">def set_tracking_running_stats(model):\n for attr in dir(model):\n if 'bn' in attr:\n target_attr = getattr(model, attr)\n target_attr.track_running_stats = True\n target_attr.running_mean = torch.nn.Parameter(torch.zeros(target_attr.num_features, requires_grad=False))\n target_attr.running_var = torch.nn.Parameter(torch.ones(target_attr.num_features, requires_grad=False))\n target_attr.num_batches_tracked = torch.nn.Parameter(torch.tensor(0, dtype=torch.long), requires_grad=False)\n # target_attr.reset_running_stats()\n return\n</code></pre>\n<p>my most comment errors:</p>\n<pre><code class=\"lang-auto\"> raise ValueError('expected 2D or 3D input (got {}D input)'\nValueError: expected 2D or 3D input (got 1D input)\n</code></pre>\n<p>and</p>\n<pre><code class=\"lang-auto\">IndexError: Dimension out of range (expected to be in range of [-1, 0], but got 1)\n</code></pre>",12 "post_number": 1,13 "post_type": 1,14 "posts_count": 4,15 "updated_at": "2020-11-19T21:58:45.568Z",16 "reply_count": 0,17 "reply_to_post_number": null,18 "quote_count": 0,19 "incoming_link_count": 498,20 "reads": 21,21 "readers_count": 20,22 "score": 2494.2,23 "yours": false,24 "topic_id": 103437,25 "topic_slug": "how-to-use-have-batch-norm-not-forget-batch-statistics-it-just-used",26 "display_username": "MirandaAgent",27 "primary_group_name": null,28 "flair_name": null,29 "flair_url": null,30 "flair_bg_color": null,31 "flair_color": null,32 "flair_group_id": null,33 "badges_granted": [],34 "version": 1,35 "can_edit": false,36 "can_delete": false,37 "can_recover": false,38 "can_see_hidden_post": false,39 "can_wiki": false,40 "link_counts": [41 {42 "url": "https://discuss.pytorch.org/t/batchnorm1d-with-batchsize-1/52136/8",43 "internal": true,44 "reflection": true,45 "title": "`BatchNorm1d()` with batchsize=1",46 "clicks": 147 }48 ],49 "read": true,50 "user_title": "",51 "bookmarked": false,52 "actions_summary": [],53 "moderator": false,54 "admin": false,55 "staff": false,56 "user_id": 2282,57 "hidden": false,58 "trust_level": 2,59 "deleted_at": null,60 "user_deleted": false,61 "edit_reason": null,62 "can_view_edit_history": true,63 "wiki": false,64 "post_url": "/t/how-to-use-have-batch-norm-not-forget-batch-statistics-it-just-used/103437/1",65 "can_accept_answer": false,66 "can_unaccept_answer": false,67 "accepted_answer": false,68 "topic_accepted_answer": true,69 "can_vote": false70 },71 {72 "id": 305723,73 "name": "MirandaAgent",74 "username": "Brando_Miranda",75 "avatar_template": "/user_avatar/discuss.pytorch.org/brando_miranda/{size}/14355_2.png",76 "created_at": "2021-09-08T19:51:25.809Z",77 "cooked": "<p>related: <a href=\"https://discuss.pytorch.org/t/how-does-pytorch-s-batch-norm-know-if-the-forward-pass-its-doing-is-for-inference-or-training/16857\" class=\"inline-onebox\">How does pytorch’s batch norm know if the forward pass its doing is for inference or training?</a></p>",78 "post_number": 2,79 "post_type": 1,80 "posts_count": 4,81 "updated_at": "2021-09-08T19:51:25.809Z",82 "reply_count": 0,83 "reply_to_post_number": null,84 "quote_count": 0,85 "incoming_link_count": 7,86 "reads": 16,87 "readers_count": 15,88 "score": 38.2,89 "yours": false,90 "topic_id": 103437,91 "topic_slug": "how-to-use-have-batch-norm-not-forget-batch-statistics-it-just-used",92 "display_username": "MirandaAgent",93 "primary_group_name": null,94 "flair_name": null,95 "flair_url": null,96 "flair_bg_color": null,97 "flair_color": null,98 "flair_group_id": null,99 "badges_granted": [],100 "version": 1,101 "can_edit": false,102 "can_delete": false,103 "can_recover": false,104 "can_see_hidden_post": false,105 "can_wiki": false,106 "link_counts": [107 {108 "url": "https://discuss.pytorch.org/t/how-does-pytorch-s-batch-norm-know-if-the-forward-pass-its-doing-is-for-inference-or-training/16857",109 "internal": true,110 "reflection": false,111 "title": "How does pytorch’s batch norm know if the forward pass its doing is for inference or training?",112 "clicks": 36113 }114 ],115 "read": true,116 "user_title": "",117 "bookmarked": false,118 "actions_summary": [],119 "moderator": false,120 "admin": false,121 "staff": false,122 "user_id": 2282,123 "hidden": false,124 "trust_level": 2,125 "deleted_at": null,126 "user_deleted": false,127 "edit_reason": null,128 "can_view_edit_history": true,129 "wiki": false,130 "post_url": "/t/how-to-use-have-batch-norm-not-forget-batch-statistics-it-just-used/103437/2",131 "can_accept_answer": false,132 "can_unaccept_answer": false,133 "accepted_answer": false,134 "topic_accepted_answer": true135 },136 {137 "id": 315442,138 "name": "MirandaAgent",139 "username": "Brando_Miranda",140 "avatar_template": "/user_avatar/discuss.pytorch.org/brando_miranda/{size}/14355_2.png",141 "created_at": "2021-11-04T21:16:04.211Z",142 "cooked": "<p>related: <a href=\"https://stackoverflow.com/questions/69845469/when-should-one-call-eval-and-train-when-doing-maml-with-the-pytorch-highe\" class=\"inline-onebox\" rel=\"noopener nofollow ugc\">machine learning - When should one call .eval() and .train() when doing MAML with the PyTorch higher library? - Stack Overflow</a></p>",143 "post_number": 3,144 "post_type": 1,145 "posts_count": 4,146 "updated_at": "2021-11-04T21:16:04.211Z",147 "reply_count": 0,148 "reply_to_post_number": null,149 "quote_count": 0,150 "incoming_link_count": 1,151 "reads": 14,152 "readers_count": 13,153 "score": 7.8,154 "yours": false,155 "topic_id": 103437,156 "topic_slug": "how-to-use-have-batch-norm-not-forget-batch-statistics-it-just-used",157 "display_username": "MirandaAgent",158 "primary_group_name": null,159 "flair_name": null,160 "flair_url": null,161 "flair_bg_color": null,162 "flair_color": null,163 "flair_group_id": null,164 "badges_granted": [],165 "version": 1,166 "can_edit": false,167 "can_delete": false,168 "can_recover": false,169 "can_see_hidden_post": false,170 "can_wiki": false,171 "link_counts": [172 {173 "url": "https://stackoverflow.com/questions/69845469/when-should-one-call-eval-and-train-when-doing-maml-with-the-pytorch-highe",174 "internal": false,175 "reflection": false,176 "title": "machine learning - When should one call .eval() and .train() when doing MAML with the PyTorch higher library? - Stack Overflow",177 "clicks": 3178 }179 ],180 "read": true,181 "user_title": "",182 "bookmarked": false,183 "actions_summary": [],184 "moderator": false,185 "admin": false,186 "staff": false,187 "user_id": 2282,188 "hidden": false,189 "trust_level": 2,190 "deleted_at": null,191 "user_deleted": false,192 "edit_reason": null,193 "can_view_edit_history": true,194 "wiki": false,195 "post_url": "/t/how-to-use-have-batch-norm-not-forget-batch-statistics-it-just-used/103437/3",196 "can_accept_answer": false,197 "can_unaccept_answer": false,198 "accepted_answer": false,199 "topic_accepted_answer": true200 },201 {202 "id": 315683,203 "name": "MirandaAgent",204 "username": "Brando_Miranda",205 "avatar_template": "/user_avatar/discuss.pytorch.org/brando_miranda/{size}/14355_2.png",206 "created_at": "2021-11-05T22:36:38.971Z",207 "cooked": "<p>Solution is to use <code>mdl.train()</code> it uses batch statistics by itself:</p>\n<blockquote>\n<p>Also by default, during training this layer keeps running estimates of its computed mean and variance, which are then used for normalization during evaluation. The running estimates are kept with a default <code>momentum</code> of 0.1.</p>\n<p>If <code>track_running_stats</code> is set to <code>False</code>, this layer then does not keep running estimates, and batch statistics are instead used during evaluation time as well.</p>\n</blockquote>\n<p><a href=\"https://pytorch.org/docs/stable/generated/torch.nn.BatchNorm2d.html\" class=\"onebox\" target=\"_blank\" rel=\"noopener nofollow ugc\">https://pytorch.org/docs/stable/generated/torch.nn.BatchNorm2d.html</a></p>",208 "post_number": 4,209 "post_type": 1,210 "posts_count": 4,211 "updated_at": "2021-11-05T22:36:38.971Z",212 "reply_count": 0,213 "reply_to_post_number": null,214 "quote_count": 0,215 "incoming_link_count": 8,216 "reads": 13,217 "readers_count": 12,218 "score": 42.6,219 "yours": false,220 "topic_id": 103437,221 "topic_slug": "how-to-use-have-batch-norm-not-forget-batch-statistics-it-just-used",222 "display_username": "MirandaAgent",223 "primary_group_name": null,224 "flair_name": null,225 "flair_url": null,226 "flair_bg_color": null,227 "flair_color": null,228 "flair_group_id": null,229 "badges_granted": [],230 "version": 1,231 "can_edit": false,232 "can_delete": false,233 "can_recover": false,234 "can_see_hidden_post": false,235 "can_wiki": false,236 "link_counts": [237 {238 "url": "https://pytorch.org/docs/stable/generated/torch.nn.BatchNorm2d.html",239 "internal": false,240 "reflection": false,241 "title": "BatchNorm2d — PyTorch 1.10.0 documentation",242 "clicks": 14243 }244 ],245 "read": true,246 "user_title": "",247 "bookmarked": false,248 "actions_summary": [],249 "moderator": false,250 "admin": false,251 "staff": false,252 "user_id": 2282,253 "hidden": false,254 "trust_level": 2,255 "deleted_at": null,256 "user_deleted": false,257 "edit_reason": null,258 "can_view_edit_history": true,259 "wiki": false,260 "post_url": "/t/how-to-use-have-batch-norm-not-forget-batch-statistics-it-just-used/103437/4",261 "can_accept_answer": false,262 "can_unaccept_answer": false,263 "accepted_answer": true,264 "topic_accepted_answer": true265 }266 ],267 "stream": [268 245547,269 305723,270 315442,271 315683272 ]273 },274 "timeline_lookup": [275 [276 1,277 1801278 ],279 [280 2,281 1508282 ],283 [284 3,285 1451286 ],287 [288 4,289 1450290 ]291 ],292 "suggested_topics": [293 {294 "fancy_title": "What is the difference between the CUDA API and CUDA HW lines in the Nsight Systems GUI?",295 "id": 216417,296 "title": "What is the difference between the CUDA API and CUDA HW lines in the Nsight Systems GUI?",297 "slug": "what-is-the-difference-between-the-cuda-api-and-cuda-hw-lines-in-the-nsight-systems-gui",298 "posts_count": 1,299 "reply_count": 0,300 "highest_post_number": 1,301 "image_url": "https://discuss.pytorch.org/uploads/default/optimized/3X/2/7/27cbe5e88c4ed57d27e04ca70978791ed7307f48_2_1024x368.png",302 "created_at": "2025-02-09T04:09:00.331Z",303 "last_posted_at": "2025-02-09T04:09:00.373Z",304 "bumped": true,305 "bumped_at": "2025-02-09T04:09:00.373Z",306 "archetype": "regular",307 "unseen": false,308 "pinned": false,309 "unpinned": null,310 "visible": true,311 "closed": false,312 "archived": false,313 "bookmarked": null,314 "liked": null,315 "tags_descriptions": {},316 "like_count": 0,317 "views": 39,318 "category_id": 1,319 "featured_link": null,320 "has_accepted_answer": false,321 "posters": [322 {323 "extras": "latest single",324 "description": "Original Poster, Most Recent Poster",325 "user": {326 "id": 79055,327 "username": "hello_e",328 "name": "hhhh",329 "avatar_template": "/letter_avatar_proxy/v4/letter/h/f9ae1b/{size}.png",330 "trust_level": 1331 }332 }333 ]334 },335 {336 "fancy_title": "RuntimeError: Input tensors need to be on the same GPU",337 "id": 217286,338 "title": "RuntimeError: Input tensors need to be on the same GPU",339 "slug": "runtimeerror-input-tensors-need-to-be-on-the-same-gpu",340 "posts_count": 1,341 "reply_count": 0,342 "highest_post_number": 1,343 "image_url": null,344 "created_at": "2025-02-28T12:44:11.593Z",345 "last_posted_at": "2025-02-28T12:44:11.634Z",346 "bumped": true,347 "bumped_at": "2025-02-28T12:44:11.634Z",348 "archetype": "regular",349 "unseen": false,350 "pinned": false,351 "unpinned": null,352 "visible": true,353 "closed": false,354 "archived": false,355 "bookmarked": null,356 "liked": null,357 "tags_descriptions": {},358 "like_count": 0,359 "views": 57,360 "category_id": 1,361 "featured_link": null,362 "has_accepted_answer": false,363 "posters": [364 {365 "extras": "latest single",366 "description": "Original Poster, Most Recent Poster",367 "user": {368 "id": 73637,369 "username": "chrathans",370 "name": "Kris Tosh",371 "avatar_template": "/user_avatar/discuss.pytorch.org/chrathans/{size}/67980_2.png",372 "trust_level": 1373 }374 }375 ]376 },377 {378 "fancy_title": "`torch.linalg.svd` uses `cudaMemcpyAsync` that syncs between host and device",379 "id": 213297,380 "title": "`torch.linalg.svd` uses `cudaMemcpyAsync` that syncs between host and device",381 "slug": "torch-linalg-svd-uses-cudamemcpyasync-that-syncs-between-host-and-device",382 "posts_count": 1,383 "reply_count": 0,384 "highest_post_number": 1,385 "image_url": null,386 "created_at": "2024-11-22T07:35:51.287Z",387 "last_posted_at": "2024-11-22T07:35:51.352Z",388 "bumped": true,389 "bumped_at": "2024-11-22T07:35:51.352Z",390 "archetype": "regular",391 "unseen": false,392 "pinned": false,393 "unpinned": null,394 "visible": true,395 "closed": false,396 "archived": false,397 "bookmarked": null,398 "liked": null,399 "tags_descriptions": {},400 "like_count": 0,401 "views": 64,402 "category_id": 1,403 "featured_link": null,404 "has_accepted_answer": false,405 "posters": [406 {407 "extras": "latest single",408 "description": "Original Poster, Most Recent Poster",409 "user": {410 "id": 14714,411 "username": "Rui_Wang",412 "name": "Rui Wang",413 "avatar_template": "/user_avatar/discuss.pytorch.org/rui_wang/{size}/37635_2.png",414 "trust_level": 1415 }416 }417 ]418 },419 {420 "fancy_title": "Cuda out of memory on 2x 32 Gb V100",421 "id": 216338,422 "title": "Cuda out of memory on 2x 32 Gb V100",423 "slug": "cuda-out-of-memory-on-2x-32-gb-v100",424 "posts_count": 1,425 "reply_count": 0,426 "highest_post_number": 1,427 "image_url": null,428 "created_at": "2025-02-07T05:24:48.257Z",429 "last_posted_at": "2025-02-07T05:24:48.313Z",430 "bumped": true,431 "bumped_at": "2025-02-07T05:24:48.313Z",432 "archetype": "regular",433 "unseen": false,434 "pinned": false,435 "unpinned": null,436 "visible": true,437 "closed": false,438 "archived": false,439 "bookmarked": null,440 "liked": null,441 "tags_descriptions": {},442 "like_count": 0,443 "views": 219,444 "category_id": 1,445 "featured_link": null,446 "has_accepted_answer": false,447 "posters": [448 {449 "extras": "latest single",450 "description": "Original Poster, Most Recent Poster",451 "user": {452 "id": 82364,453 "username": "Sourabh_Yadav",454 "name": "Sourabh Yadav",455 "avatar_template": "/user_avatar/discuss.pytorch.org/sourabh_yadav/{size}/75350_2.png",456 "trust_level": 1457 }458 }459 ]460 },461 {462 "fancy_title": ":RuntimeError: cuDNN error: CUDNN_STATUS_EXECUTION_FAILED",463 "id": 217297,464 "title": ":RuntimeError: cuDNN error: CUDNN_STATUS_EXECUTION_FAILED",465 "slug": "cudnn-error-cudnn-status-execution-failed",466 "posts_count": 3,467 "reply_count": 1,468 "highest_post_number": 3,469 "image_url": null,470 "created_at": "2025-02-28T14:38:05.225Z",471 "last_posted_at": "2025-02-28T14:52:24.878Z",472 "bumped": true,473 "bumped_at": "2025-02-28T14:58:09.597Z",474 "archetype": "regular",475 "unseen": false,476 "pinned": false,477 "unpinned": null,478 "visible": true,479 "closed": false,480 "archived": false,481 "bookmarked": null,482 "liked": null,483 "unicode_title": ":RuntimeError: cuDNN error: CUDNN_STATUS_EXECUTION_FAILED",484 "tags_descriptions": {},485 "like_count": 0,486 "views": 112,487 "category_id": 1,488 "featured_link": null,489 "has_accepted_answer": false,490 "posters": [491 {492 "extras": "latest",493 "description": "Original Poster, Most Recent Poster",494 "user": {495 "id": 82794,496 "username": "Mohan_Krishna",497 "name": "Mohan Krishna",498 "avatar_template": "/user_avatar/discuss.pytorch.org/mohan_krishna/{size}/75758_2.png",499 "trust_level": 1500 }501 },502 {503 "extras": null,504 "description": "Frequent Poster",505 "user": {506 "id": 3534,507 "username": "ptrblck",508 "name": "",509 "avatar_template": "/user_avatar/discuss.pytorch.org/ptrblck/{size}/1823_2.png",510 "admin": true,511 "moderator": true,512 "trust_level": 2513 }514 }515 ]516 }517 ],518 "tags_descriptions": {},519 "fancy_title": "How to use have batch norm not forget batch statistics it just used?",520 "id": 103437,521 "title": "How to use have batch norm not forget batch statistics it just used?",522 "posts_count": 4,523 "created_at": "2020-11-19T21:58:45.502Z",524 "views": 1599,525 "reply_count": 0,526 "like_count": 0,527 "last_posted_at": "2021-11-05T22:36:38.971Z",528 "visible": true,529 "closed": false,530 "archived": false,531 "has_summary": false,532 "archetype": "regular",533 "slug": "how-to-use-have-batch-norm-not-forget-batch-statistics-it-just-used",534 "category_id": 1,535 "word_count": 364,536 "deleted_at": null,537 "user_id": 2282,538 "featured_link": null,539 "pinned_globally": false,540 "pinned_at": null,541 "pinned_until": null,542 "image_url": null,543 "slow_mode_seconds": 0,544 "draft": null,545 "draft_key": "topic_103437",546 "draft_sequence": null,547 "unpinned": null,548 "pinned": false,549 "current_post_number": 1,550 "highest_post_number": 4,551 "deleted_by": null,552 "actions_summary": [553 {554 "id": 4,555 "count": 0,556 "hidden": false,557 "can_act": false558 },559 {560 "id": 8,561 "count": 0,562 "hidden": false,563 "can_act": false564 },565 {566 "id": 10,567 "count": 0,568 "hidden": false,569 "can_act": false570 },571 {572 "id": 7,573 "count": 0,574 "hidden": false,575 "can_act": false576 }577 ],578 "chunk_size": 20,579 "bookmarked": false,580 "topic_timer": null,581 "message_bus_last_id": 0,582 "participant_count": 1,583 "show_read_indicator": false,584 "thumbnails": null,585 "slow_mode_enabled_until": null,586 "accepted_answer": {587 "post_number": 4,588 "username": "Brando_Miranda",589 "name": "MirandaAgent",590 "excerpt": "Solution is to use mdl.train() it uses batch statistics by itself: \n\nAlso by default, during training this layer keeps running estimates of its computed mean and variance, which are then used for normalization during evaluation. The running estimates are kept with a default momentum of 0.1. \nIf trac…"591 },592 "can_vote": false,593 "vote_count": 0,594 "user_voted": false,595 "discourse_zendesk_plugin_zendesk_id": null,596 "discourse_zendesk_plugin_zendesk_url": "https://your-url.zendesk.com/agent/tickets/",597 "details": {598 "can_edit": false,599 "notification_level": 1,600 "participants": [601 {602 "id": 2282,603 "username": "Brando_Miranda",604 "name": "MirandaAgent",605 "avatar_template": "/user_avatar/discuss.pytorch.org/brando_miranda/{size}/14355_2.png",606 "post_count": 4,607 "primary_group_name": null,608 "flair_name": null,609 "flair_url": null,610 "flair_color": null,611 "flair_bg_color": null,612 "flair_group_id": null,613 "trust_level": 2614 }615 ],616 "created_by": {617 "id": 2282,618 "username": "Brando_Miranda",619 "name": "MirandaAgent",620 "avatar_template": "/user_avatar/discuss.pytorch.org/brando_miranda/{size}/14355_2.png"621 },622 "last_poster": {623 "id": 2282,624 "username": "Brando_Miranda",625 "name": "MirandaAgent",626 "avatar_template": "/user_avatar/discuss.pytorch.org/brando_miranda/{size}/14355_2.png"627 },628 "links": [629 {630 "url": "https://discuss.pytorch.org/t/how-does-pytorch-s-batch-norm-know-if-the-forward-pass-its-doing-is-for-inference-or-training/16857",631 "title": "How does pytorch’s batch norm know if the forward pass its doing is for inference or training?",632 "internal": true,633 "attachment": false,634 "reflection": false,635 "clicks": 36,636 "user_id": 2282,637 "domain": "discuss.pytorch.org",638 "root_domain": "pytorch.org"639 },640 {641 "url": "https://pytorch.org/docs/stable/generated/torch.nn.BatchNorm2d.html",642 "title": "BatchNorm2d — PyTorch 1.10.0 documentation",643 "internal": false,644 "attachment": false,645 "reflection": false,646 "clicks": 14,647 "user_id": 2282,648 "domain": "pytorch.org",649 "root_domain": "pytorch.org"650 },651 {652 "url": "https://stackoverflow.com/questions/69845469/when-should-one-call-eval-and-train-when-doing-maml-with-the-pytorch-highe",653 "title": "machine learning - When should one call .eval() and .train() when doing MAML with the PyTorch higher library? - Stack Overflow",654 "internal": false,655 "attachment": false,656 "reflection": false,657 "clicks": 3,658 "user_id": 2282,659 "domain": "stackoverflow.com",660 "root_domain": "stackoverflow.com"661 },662 {663 "url": "https://discuss.pytorch.org/t/batchnorm1d-with-batchsize-1/52136/8",664 "title": "`BatchNorm1d()` with batchsize=1",665 "internal": true,666 "attachment": false,667 "reflection": true,668 "clicks": 1,669 "user_id": 2282,670 "domain": "discuss.pytorch.org",671 "root_domain": "pytorch.org"672 }673 ]674 },675 "bookmarks": []676 },677 {678 "post_stream": {679 "posts": [680 {681 "id": 315015,682 "name": "Ming Wu",683 "username": "everlastin14",684 "avatar_template": "/letter_avatar_proxy/v4/letter/e/7ab992/{size}.png",685 "created_at": "2021-11-03T08:11:30.479Z",686 "cooked": "<p>I’m trying to run a easy deep learning model on embedding system without any framework</p>\n<p>I have trained below model with my own dataset and quantize to int8</p>\n<pre><code class=\"lang-auto\">class Net(torch.nn.Module):\n def __init__(self, n_feature, n_hidden, n_output, quant=False):\n super(Net, self).__init__()\n self.fc1 = torch.nn.Linear(n_feature, n_hidden, bias=False)\n self.fc2 = torch.nn.Linear(n_hidden, n_output, bias=False)\n self.relu = torch.nn.ReLU()\n self.quant = quant\n if self.quant:\n self.quant = torch.quantization.QuantStub()\n self.dequant = torch.quantization.DeQuantStub()\n\n def forward(self, input):\n if self.quant:\n x = self.quant(input)\n else:\n x = input\n x = self.fc1(x)\n x = self.relu(x)\n x = self.fc2(x)\n if self.quant:\n x = self.dequant(x)\n\n return x\n</code></pre>\n<p>I can get each layer’s int8 weight as fc1.weight().int_repr()<br>\nBut how to use these parameter to reproduce result like net.forward()?</p>",687 "post_number": 1,688 "post_type": 1,689 "posts_count": 2,690 "updated_at": "2021-11-03T08:11:30.479Z",691 "reply_count": 0,692 "reply_to_post_number": null,693 "quote_count": 0,694 "incoming_link_count": 29,695 "reads": 7,696 "readers_count": 6,697 "score": 146.4,698 "yours": false,699 "topic_id": 135798,700 "topic_slug": "how-to-reproduce-result",701 "display_username": "Ming Wu",702 "primary_group_name": null,703 "flair_name": null,704 "flair_url": null,705 "flair_bg_color": null,706 "flair_color": null,707 "flair_group_id": null,708 "badges_granted": [],709 "version": 1,710 "can_edit": false,711 "can_delete": false,712 "can_recover": false,713 "can_see_hidden_post": false,714 "can_wiki": false,715 "read": true,716 "user_title": "",717 "bookmarked": false,718 "actions_summary": [],719 "moderator": false,720 "admin": false,721 "staff": false,722 "user_id": 41372,723 "hidden": false,724 "trust_level": 1,725 "deleted_at": null,726 "user_deleted": false,727 "edit_reason": null,728 "can_view_edit_history": true,729 "wiki": false,730 "post_url": "/t/how-to-reproduce-result/135798/1",731 "can_accept_answer": false,732 "can_unaccept_answer": false,733 "accepted_answer": false,734 "topic_accepted_answer": null,735 "can_vote": false736 },737 {738 "id": 315681,739 "name": "Jerry Zhang",740 "username": "jerryzh168",741 "avatar_template": "/user_avatar/discuss.pytorch.org/jerryzh168/{size}/15217_2.png",742 "created_at": "2021-11-05T22:30:49.282Z",743 "cooked": "<p>Do you mean how to run the quantized pytorch model on your embedding system?</p>",744 "post_number": 2,745 "post_type": 1,746 "posts_count": 2,747 "updated_at": "2021-11-05T22:30:49.282Z",748 "reply_count": 0,749 "reply_to_post_number": null,750 "quote_count": 0,751 "incoming_link_count": 0,752 "reads": 6,753 "readers_count": 5,754 "score": 1.2,755 "yours": false,756 "topic_id": 135798,757 "topic_slug": "how-to-reproduce-result",758 "display_username": "Jerry Zhang",759 "primary_group_name": null,760 "flair_name": null,761 "flair_url": null,762 "flair_bg_color": null,763 "flair_color": null,764 "flair_group_id": null,765 "badges_granted": [],766 "version": 1,767 "can_edit": false,768 "can_delete": false,769 "can_recover": false,770 "can_see_hidden_post": false,771 "can_wiki": false,772 "read": true,773 "user_title": null,774 "bookmarked": false,775 "actions_summary": [],776 "moderator": false,777 "admin": false,778 "staff": false,779 "user_id": 21770,780 "hidden": false,781 "trust_level": 2,782 "deleted_at": null,783 "user_deleted": false,784 "edit_reason": null,785 "can_view_edit_history": true,786 "wiki": false,787 "post_url": "/t/how-to-reproduce-result/135798/2",788 "can_accept_answer": false,789 "can_unaccept_answer": false,790 "accepted_answer": false,791 "topic_accepted_answer": null792 }793 ],794 "stream": [795 315015,796 315681797 ]798 },799 "timeline_lookup": [800 [801 1,802 1453803 ],804 [805 2,806 1450807 ]808 ],809 "suggested_topics": [810 {811 "fancy_title": "JIT model is a deployment model or a quantized model?",812 "id": 216368,813 "title": "JIT model is a deployment model or a quantized model?",814 "slug": "jit-model-is-a-deployment-model-or-a-quantized-model",815 "posts_count": 1,816 "reply_count": 0,817 "highest_post_number": 1,818 "image_url": null,819 "created_at": "2025-02-07T15:48:37.971Z",820 "last_posted_at": "2025-02-07T15:48:38.008Z",821 "bumped": true,822 "bumped_at": "2025-02-07T15:48:38.008Z",823 "archetype": "regular",824 "unseen": false,825 "pinned": false,826 "unpinned": null,827 "visible": true,828 "closed": false,829 "archived": false,830 "bookmarked": null,831 "liked": null,832 "tags_descriptions": {},833 "like_count": 0,834 "views": 54,835 "category_id": 17,836 "featured_link": null,837 "has_accepted_answer": false,838 "posters": [839 {840 "extras": "latest single",841 "description": "Original Poster, Most Recent Poster",842 "user": {843 "id": 82456,844 "username": "BambooKui",845 "name": "Bamboo Kui",846 "avatar_template": "/user_avatar/discuss.pytorch.org/bambookui/{size}/75441_2.png",847 "trust_level": 1848 }849 }850 ]851 },852 {853 "fancy_title": "How to convert a QAT model to ONNX model",854 "id": 213571,855 "title": "How to convert a QAT model to ONNX model",856 "slug": "how-to-convert-a-qat-model-to-onnx-model",857 "posts_count": 4,858 "reply_count": 1,859 "highest_post_number": 5,860 "image_url": null,861 "created_at": "2024-11-28T07:36:34.183Z",862 "last_posted_at": "2024-12-19T10:03:33.919Z",863 "bumped": true,864 "bumped_at": "2024-12-19T10:03:33.919Z",865 "archetype": "regular",866 "unseen": false,867 "pinned": false,868 "unpinned": null,869 "visible": true,870 "closed": false,871 "archived": false,872 "bookmarked": null,873 "liked": null,874 "tags_descriptions": {},875 "like_count": 0,876 "views": 443,877 "category_id": 17,878 "featured_link": null,879 "has_accepted_answer": false,880 "posters": [881 {882 "extras": null,883 "description": "Original Poster",884 "user": {885 "id": 80995,886 "username": "Blue_He",887 "name": "Blue He",888 "avatar_template": "/user_avatar/discuss.pytorch.org/blue_he/{size}/74072_2.png",889 "trust_level": 1890 }891 },892 {893 "extras": null,894 "description": "Frequent Poster",895 "user": {896 "id": 81191,897 "username": "Christin602Bey",898 "name": null,899 "avatar_template": "/letter_avatar_proxy/v4/letter/c/958977/{size}.png",900 "trust_level": 0901 }902 },903 {904 "extras": "latest",905 "description": "Most Recent Poster",906 "user": {907 "id": 81150,908 "username": "iliasslasri",909 "name": "Iliass",910 "avatar_template": "/letter_avatar_proxy/v4/letter/i/67e7ee/{size}.png",911 "trust_level": 1912 }913 }914 ]915 },916 {917 "fancy_title": "Pytorch 2 Export QAT is training",918 "id": 214389,919 "title": "Pytorch 2 Export QAT is training",920 "slug": "pytorch-2-export-qat-is-training",921 "posts_count": 1,922 "reply_count": 0,923 "highest_post_number": 1,924 "image_url": null,925 "created_at": "2024-12-19T05:17:39.134Z",926 "last_posted_at": "2024-12-19T05:17:39.196Z",927 "bumped": true,928 "bumped_at": "2024-12-19T05:17:39.196Z",929 "archetype": "regular",930 "unseen": false,931 "pinned": false,932 "unpinned": null,933 "visible": true,934 "closed": false,935 "archived": false,936 "bookmarked": null,937 "liked": null,938 "tags_descriptions": {},939 "like_count": 0,940 "views": 163,941 "category_id": 17,942 "featured_link": null,943 "has_accepted_answer": false,944 "posters": [945 {946 "extras": "latest single",947 "description": "Original Poster, Most Recent Poster",948 "user": {949 "id": 34929,950 "username": "PavanMV",951 "name": "Pavan MV",952 "avatar_template": "/user_avatar/discuss.pytorch.org/pavanmv/{size}/27801_2.png",953 "trust_level": 1954 }955 }956 ]957 },958 {959 "fancy_title": "Torch.bfloat16 < how does it work? in bf 16 model",960 "id": 212450,961 "title": "Torch.bfloat16 < how does it work? in bf 16 model",962 "slug": "torch-bfloat16-how-does-it-work-in-bf-16-model",963 "posts_count": 2,964 "reply_count": 0,965 "highest_post_number": 2,966 "image_url": null,967 "created_at": "2024-11-02T14:14:26.984Z",968 "last_posted_at": "2024-11-04T17:14:31.044Z",969 "bumped": true,970 "bumped_at": "2024-11-04T17:14:31.044Z",971 "archetype": "regular",972 "unseen": false,973 "pinned": false,974 "unpinned": null,975 "visible": true,976 "closed": false,977 "archived": false,978 "bookmarked": null,979 "liked": null,980 "tags_descriptions": {},981 "like_count": 0,982 "views": 311,983 "category_id": 17,984 "featured_link": null,985 "has_accepted_answer": true,986 "posters": [987 {988 "extras": null,989 "description": "Original Poster",990 "user": {991 "id": 80013,992 "username": "minjun_jo",993 "name": "MJ_Jo",994 "avatar_template": "/user_avatar/discuss.pytorch.org/minjun_jo/{size}/63448_2.png",995 "trust_level": 1996 }997 },998 {999 "extras": "latest",1000 "description": "Most Recent Poster, Accepted Answer",1001 "user": {1002 "id": 31938,1003 "username": "Vasiliy_Kuznetsov",1004 "name": "Vasiliy Kuznetsov",1005 "avatar_template": "/user_avatar/discuss.pytorch.org/vasiliy_kuznetsov/{size}/24530_2.png",1006 "trust_level": 21007 }1008 }1009 ]1010 },1011 {1012 "fancy_title": "How to quantize my torchscript to fp8",1013 "id": 219009,1014 "title": "How to quantize my torchscript to fp8",1015 "slug": "how-to-quantize-my-torchscript-to-fp8",1016 "posts_count": 2,1017 "reply_count": 0,1018 "highest_post_number": 2,1019 "image_url": null,1020 "created_at": "2025-04-12T22:36:02.125Z",1021 "last_posted_at": "2025-04-18T03:22:19.588Z",1022 "bumped": true,1023 "bumped_at": "2025-04-18T03:22:19.588Z",1024 "archetype": "regular",1025 "unseen": false,1026 "pinned": false,1027 "unpinned": null,1028 "visible": true,1029 "closed": false,1030 "archived": false,1031 "bookmarked": null,1032 "liked": null,1033 "tags_descriptions": {},1034 "like_count": 0,1035 "views": 248,1036 "category_id": 17,1037 "featured_link": null,1038 "has_accepted_answer": false,1039 "posters": [1040 {1041 "extras": null,1042 "description": "Original Poster",1043 "user": {1044 "id": 83771,1045 "username": "mikeybydun1",1046 "name": "מייקי בר יעקב",1047 "avatar_template": "/user_avatar/discuss.pytorch.org/mikeybydun1/{size}/76583_2.png",1048 "trust_level": 01049 }1050 },1051 {1052 "extras": "latest",1053 "description": "Most Recent Poster",1054 "user": {1055 "id": 21770,1056 "username": "jerryzh168",1057 "name": "Jerry Zhang",1058 "avatar_template": "/user_avatar/discuss.pytorch.org/jerryzh168/{size}/15217_2.png",1059 "trust_level": 21060 }1061 }1062 ]1063 }1064 ],1065 "tags_descriptions": {},1066 "fancy_title": "How to reproduce result",1067 "id": 135798,1068 "title": "How to reproduce result",1069 "posts_count": 2,1070 "created_at": "2021-11-03T08:11:30.394Z",1071 "views": 460,1072 "reply_count": 0,1073 "like_count": 0,1074 "last_posted_at": "2021-11-05T22:30:49.282Z",1075 "visible": true,1076 "closed": false,1077 "archived": false,1078 "has_summary": false,1079 "archetype": "regular",1080 "slug": "how-to-reproduce-result",1081 "category_id": 17,1082 "word_count": 158,1083 "deleted_at": null,1084 "user_id": 41372,1085 "featured_link": null,1086 "pinned_globally": false,1087 "pinned_at": null,1088 "pinned_until": null,1089 "image_url": null,1090 "slow_mode_seconds": 0,1091 "draft": null,1092 "draft_key": "topic_135798",1093 "draft_sequence": null,1094 "unpinned": null,1095 "pinned": false,1096 "current_post_number": 1,1097 "highest_post_number": 2,1098 "deleted_by": null,1099 "actions_summary": [1100 {1101 "id": 4,1102 "count": 0,1103 "hidden": false,1104 "can_act": false1105 },1106 {1107 "id": 8,1108 "count": 0,1109 "hidden": false,1110 "can_act": false1111 },1112 {1113 "id": 10,1114 "count": 0,1115 "hidden": false,1116 "can_act": false1117 },1118 {1119 "id": 7,1120 "count": 0,1121 "hidden": false,1122 "can_act": false1123 }1124 ],1125 "chunk_size": 20,1126 "bookmarked": false,1127 "topic_timer": null,1128 "message_bus_last_id": 0,1129 "participant_count": 2,1130 "show_read_indicator": false,1131 "thumbnails": null,1132 "slow_mode_enabled_until": null,1133 "can_vote": false,1134 "vote_count": 0,1135 "user_voted": false,1136 "discourse_zendesk_plugin_zendesk_id": null,1137 "discourse_zendesk_plugin_zendesk_url": "https://your-url.zendesk.com/agent/tickets/",1138 "details": {1139 "can_edit": false,1140 "notification_level": 1,1141 "participants": [1142 {1143 "id": 21770,1144 "username": "jerryzh168",1145 "name": "Jerry Zhang",1146 "avatar_template": "/user_avatar/discuss.pytorch.org/jerryzh168/{size}/15217_2.png",1147 "post_count": 1,1148 "primary_group_name": null,1149 "flair_name": null,1150 "flair_url": null,1151 "flair_color": null,1152 "flair_bg_color": null,1153 "flair_group_id": null,1154 "trust_level": 21155 },1156 {1157 "id": 41372,1158 "username": "everlastin14",1159 "name": "Ming Wu",1160 "avatar_template": "/letter_avatar_proxy/v4/letter/e/7ab992/{size}.png",1161 "post_count": 1,1162 "primary_group_name": null,1163 "flair_name": null,1164 "flair_url": null,1165 "flair_color": null,1166 "flair_bg_color": null,1167 "flair_group_id": null,1168 "trust_level": 11169 }1170 ],1171 "created_by": {1172 "id": 41372,1173 "username": "everlastin14",1174 "name": "Ming Wu",1175 "avatar_template": "/letter_avatar_proxy/v4/letter/e/7ab992/{size}.png"1176 },1177 "last_poster": {1178 "id": 21770,1179 "username": "jerryzh168",1180 "name": "Jerry Zhang",1181 "avatar_template": "/user_avatar/discuss.pytorch.org/jerryzh168/{size}/15217_2.png"1182 }1183 },1184 "bookmarks": []1185 },1186 {1187 "post_stream": {1188 "posts": [1189 {1190 "id": 292541,1191 "name": "Yogeshwari",1192 "username": "anonymous1",1193 "avatar_template": "/letter_avatar_proxy/v4/letter/a/77aa72/{size}.png",1194 "created_at": "2021-06-27T09:10:58.638Z",1195 "cooked": "<p>Hi,<br>\nI performed the quantization technique on efficient net models by referring post-training static quantization method in PyTorch blogs. But I was only able to bring a reduction only by 5 MB.<br>\nAlso, I wasn’t able to perform the layer fusion step on the prebuilt layers of this model while quantizing using the existing PyTorch techniques. How do I approach this problem? Or is there an alternative method to bring down the size of the model without affecting its accuracy much?<br>\nCan someone help me with this?</p>\n<p>Thanks in advance!</p>",1196 "post_number": 1,1197 "post_type": 1,1198 "posts_count": 2,1199 "updated_at": "2021-06-27T09:10:58.638Z",1200 "reply_count": 0,