Anurag1734/cuda-error-resolution-analysis
07
1[2 {3 "post_stream": {4 "posts": [5 {6 "id": 304420,7 "name": "Ivprgrmmr",8 "username": "ivprgrmmr",9 "avatar_template": "/user_avatar/discuss.pytorch.org/ivprgrmmr/{size}/41662_2.png",10 "created_at": "2021-09-01T11:27:27.554Z",11 "cooked": "<p>Hi there,</p>\n<p>I am using a DenseNet for a two class classification problem. The last layer of my network is a linear layer, so I am using the torch.nn.BCEWithLogitsLoss() loss function.<br>\nI am debugging my network since the training loss is not decreasing. I just calculated the BCE by hand and I calculated the BCE using the pytorch function, however the outputs differ.</p>\n<p><em>Example:</em><br>\n<em>target = tensor([1.])</em><br>\n<em>linear_prediction = tensor([-0.5270], grad_fn=)</em><br>\n<em>sigmoid_prediction = torch.sigmoid(linear_prediction)</em><br>\n<em>print(sigmoid_prediction) → tensor([0.3712], grad_fn=)</em><br>\n<em>loss = torch.nn.BCELoss(sigmoid_prediction,target)</em><br>\n<em>print(loss) → tensor([0.9910], grad_fn=)</em></p>\n<p><em>However, when I calculate the loss using the following function: loss = -ylog(x), with y=1 and x=0.3712, I get a loss of 0.4304.</em></p>\n<p>Does anyone have suggestions why the two outputs could be different?</p>\n<p>Thanks in advance!</p>",12 "post_number": 1,13 "post_type": 1,14 "posts_count": 3,15 "updated_at": "2021-09-01T11:28:38.412Z",16 "reply_count": 1,17 "reply_to_post_number": null,18 "quote_count": 0,19 "incoming_link_count": 175,20 "reads": 9,21 "readers_count": 8,22 "score": 871.8,23 "yours": false,24 "topic_id": 130847,25 "topic_slug": "bce-loss-output-is-not-corresponding-to-calculation-by-hand",26 "display_username": "Ivprgrmmr",27 "primary_group_name": null,28 "flair_name": null,29 "flair_url": null,30 "flair_bg_color": null,31 "flair_color": null,32 "flair_group_id": null,33 "badges_granted": [],34 "version": 1,35 "can_edit": false,36 "can_delete": false,37 "can_recover": false,38 "can_see_hidden_post": false,39 "can_wiki": false,40 "read": true,41 "user_title": null,42 "bookmarked": false,43 "actions_summary": [],44 "moderator": false,45 "admin": false,46 "staff": false,47 "user_id": 48480,48 "hidden": false,49 "trust_level": 1,50 "deleted_at": null,51 "user_deleted": false,52 "edit_reason": null,53 "can_view_edit_history": true,54 "wiki": false,55 "post_url": "/t/bce-loss-output-is-not-corresponding-to-calculation-by-hand/130847/1",56 "can_accept_answer": false,57 "can_unaccept_answer": false,58 "accepted_answer": false,59 "topic_accepted_answer": null,60 "can_vote": false61 },62 {63 "id": 304452,64 "name": "K. Frank",65 "username": "KFrank",66 "avatar_template": "/letter_avatar_proxy/v4/letter/k/ecb155/{size}.png",67 "created_at": "2021-09-01T13:44:10.780Z",68 "cooked": "<p>Hi Iv!</p>\n<aside class=\"quote no-group\" data-username=\"ivprgrmmr\" data-post=\"1\" data-topic=\"130847\" data-full=\"true\">\n<div class=\"title\">\n<div class=\"quote-controls\"></div>\n<img loading=\"lazy\" alt=\"\" width=\"24\" height=\"24\" src=\"https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/ivprgrmmr/48/41662_2.png\" class=\"avatar\"> ivprgrmmr:</div>\n<blockquote>\n<p><em>loss = -ylog(x), with y=1 and x=0.3712, I get a loss of 0.4304.</em></p>\n</blockquote>\n</aside>\n<p>Your notation here (“<em>log(x)</em>”) is ambiguous, but you are using the<br>\nso-called <em>common logarithm</em> (log-base-10) in your calculation.<br>\nYou should be using the \"natural logarithm* (log-base-e), instead.<br>\nThe natural logarithm is the correct choice (and is what pytorch is<br>\nusing).</p>\n<p>Note that pytorch’s <code>torch.log()</code> and python’s <code>math.log()</code> give you the<br>\nnatural logarithm. (They both use <code>log10()</code> for the common logarithm.)</p>\n<p>Best.</p>\n<p>K. Frank</p>",69 "post_number": 2,70 "post_type": 1,71 "posts_count": 3,72 "updated_at": "2021-09-01T13:44:10.780Z",73 "reply_count": 1,74 "reply_to_post_number": null,75 "quote_count": 1,76 "incoming_link_count": 34,77 "reads": 8,78 "readers_count": 7,79 "score": 176.6,80 "yours": false,81 "topic_id": 130847,82 "topic_slug": "bce-loss-output-is-not-corresponding-to-calculation-by-hand",83 "display_username": "K. Frank",84 "primary_group_name": null,85 "flair_name": null,86 "flair_url": null,87 "flair_bg_color": null,88 "flair_color": null,89 "flair_group_id": null,90 "badges_granted": [],91 "version": 1,92 "can_edit": false,93 "can_delete": false,94 "can_recover": false,95 "can_see_hidden_post": false,96 "can_wiki": false,97 "read": true,98 "user_title": null,99 "bookmarked": false,100 "actions_summary": [],101 "moderator": false,102 "admin": false,103 "staff": false,104 "user_id": 18088,105 "hidden": false,106 "trust_level": 2,107 "deleted_at": null,108 "user_deleted": false,109 "edit_reason": null,110 "can_view_edit_history": true,111 "wiki": false,112 "post_url": "/t/bce-loss-output-is-not-corresponding-to-calculation-by-hand/130847/2",113 "can_accept_answer": false,114 "can_unaccept_answer": false,115 "accepted_answer": false,116 "topic_accepted_answer": null117 },118 {119 "id": 304818,120 "name": "Ivprgrmmr",121 "username": "ivprgrmmr",122 "avatar_template": "/user_avatar/discuss.pytorch.org/ivprgrmmr/{size}/41662_2.png",123 "created_at": "2021-09-03T09:11:01.823Z",124 "cooked": "<p>Ofcourse, I mixed those two up. Thanks for the reply!</p>",125 "post_number": 3,126 "post_type": 1,127 "posts_count": 3,128 "updated_at": "2021-09-03T09:11:01.823Z",129 "reply_count": 0,130 "reply_to_post_number": 2,131 "quote_count": 0,132 "incoming_link_count": 2,133 "reads": 6,134 "readers_count": 5,135 "score": 11.2,136 "yours": false,137 "topic_id": 130847,138 "topic_slug": "bce-loss-output-is-not-corresponding-to-calculation-by-hand",139 "display_username": "Ivprgrmmr",140 "primary_group_name": null,141 "flair_name": null,142 "flair_url": null,143 "flair_bg_color": null,144 "flair_color": null,145 "flair_group_id": null,146 "badges_granted": [],147 "version": 1,148 "can_edit": false,149 "can_delete": false,150 "can_recover": false,151 "can_see_hidden_post": false,152 "can_wiki": false,153 "read": true,154 "user_title": null,155 "reply_to_user": {156 "id": 18088,157 "username": "KFrank",158 "name": "K. Frank",159 "avatar_template": "/letter_avatar_proxy/v4/letter/k/ecb155/{size}.png"160 },161 "bookmarked": false,162 "actions_summary": [],163 "moderator": false,164 "admin": false,165 "staff": false,166 "user_id": 48480,167 "hidden": false,168 "trust_level": 1,169 "deleted_at": null,170 "user_deleted": false,171 "edit_reason": null,172 "can_view_edit_history": true,173 "wiki": false,174 "post_url": "/t/bce-loss-output-is-not-corresponding-to-calculation-by-hand/130847/3",175 "can_accept_answer": false,176 "can_unaccept_answer": false,177 "accepted_answer": false,178 "topic_accepted_answer": null179 }180 ],181 "stream": [182 304420,183 304452,184 304818185 ]186 },187 "timeline_lookup": [188 [189 1,190 1515191 ],192 [193 3,194 1514195 ]196 ],197 "suggested_topics": [198 {199 "fancy_title": "Training accuracy significantly decreases and doesn’t go back up when loading from a checkpoint",200 "id": 217328,201 "title": "Training accuracy significantly decreases and doesn't go back up when loading from a checkpoint",202 "slug": "training-accuracy-significantly-decreases-and-doesnt-go-back-up-when-loading-from-a-checkpoint",203 "posts_count": 3,204 "reply_count": 2,205 "highest_post_number": 4,206 "image_url": null,207 "created_at": "2025-03-02T00:10:21.117Z",208 "last_posted_at": "2025-03-02T21:49:49.170Z",209 "bumped": true,210 "bumped_at": "2025-03-02T21:49:49.170Z",211 "archetype": "regular",212 "unseen": false,213 "pinned": false,214 "unpinned": null,215 "visible": true,216 "closed": false,217 "archived": false,218 "bookmarked": null,219 "liked": null,220 "tags_descriptions": {},221 "like_count": 0,222 "views": 41,223 "category_id": 1,224 "featured_link": null,225 "has_accepted_answer": false,226 "posters": [227 {228 "extras": "latest",229 "description": "Original Poster, Most Recent Poster",230 "user": {231 "id": 83008,232 "username": "EaswarGn",233 "name": "Easwar Gn",234 "avatar_template": "/user_avatar/discuss.pytorch.org/easwargn/{size}/75936_2.png",235 "trust_level": 1236 }237 },238 {239 "extras": null,240 "description": "Frequent Poster",241 "user": {242 "id": 3534,243 "username": "ptrblck",244 "name": "",245 "avatar_template": "/user_avatar/discuss.pytorch.org/ptrblck/{size}/1823_2.png",246 "admin": true,247 "moderator": true,248 "trust_level": 2249 }250 }251 ]252 },253 {254 "fancy_title": "This is about optimizing cuDNN to avoid rebuilding the graph when batch, input dimensions change.",255 "id": 217915,256 "title": "This is about optimizing cuDNN to avoid rebuilding the graph when batch, input dimensions change.",257 "slug": "this-is-about-optimizing-cudnn-to-avoid-rebuilding-the-graph-when-batch-input-dimensions-change",258 "posts_count": 1,259 "reply_count": 0,260 "highest_post_number": 1,261 "image_url": null,262 "created_at": "2025-03-16T13:59:44.870Z",263 "last_posted_at": "2025-03-16T13:59:44.902Z",264 "bumped": true,265 "bumped_at": "2025-03-16T13:59:44.902Z",266 "archetype": "regular",267 "unseen": false,268 "pinned": false,269 "unpinned": null,270 "visible": true,271 "closed": false,272 "archived": false,273 "bookmarked": null,274 "liked": null,275 "tags_descriptions": {},276 "like_count": 0,277 "views": 36,278 "category_id": 1,279 "featured_link": null,280 "has_accepted_answer": false,281 "posters": [282 {283 "extras": "latest single",284 "description": "Original Poster, Most Recent Poster",285 "user": {286 "id": 83303,287 "username": "yhyang201",288 "name": "",289 "avatar_template": "/user_avatar/discuss.pytorch.org/yhyang201/{size}/76189_2.png",290 "trust_level": 0291 }292 }293 ]294 },295 {296 "fancy_title": "torch.OutOfMemoryError Needing Help",297 "id": 215383,298 "title": "torch.OutOfMemoryError Needing Help",299 "slug": "torch-outofmemoryerror-needing-help",300 "posts_count": 2,301 "reply_count": 0,302 "highest_post_number": 2,303 "image_url": "https://discuss.pytorch.org/uploads/default/original/3X/7/1/7177b82c5c3fa982bd5bcb76e82413cb1a9895ee.png",304 "created_at": "2025-01-14T16:13:00.335Z",305 "last_posted_at": "2025-01-14T20:48:39.772Z",306 "bumped": true,307 "bumped_at": "2025-01-14T20:48:39.772Z",308 "archetype": "regular",309 "unseen": false,310 "pinned": false,311 "unpinned": null,312 "visible": true,313 "closed": false,314 "archived": false,315 "bookmarked": null,316 "liked": null,317 "tags_descriptions": {},318 "like_count": 0,319 "views": 72,320 "category_id": 1,321 "featured_link": null,322 "has_accepted_answer": false,323 "posters": [324 {325 "extras": null,326 "description": "Original Poster",327 "user": {328 "id": 82088,329 "username": "meditrust",330 "name": "Yohan Azoulay",331 "avatar_template": "/letter_avatar_proxy/v4/letter/m/f1d935/{size}.png",332 "trust_level": 0333 }334 },335 {336 "extras": "latest",337 "description": "Most Recent Poster",338 "user": {339 "id": 3534,340 "username": "ptrblck",341 "name": "",342 "avatar_template": "/user_avatar/discuss.pytorch.org/ptrblck/{size}/1823_2.png",343 "admin": true,344 "moderator": true,345 "trust_level": 2346 }347 }348 ]349 },350 {351 "fancy_title": "Find maximum length of consecutive zeros in each row",352 "id": 213871,353 "title": "Find maximum length of consecutive zeros in each row",354 "slug": "find-maximum-length-of-consecutive-zeros-in-each-row",355 "posts_count": 3,356 "reply_count": 0,357 "highest_post_number": 3,358 "image_url": null,359 "created_at": "2024-12-05T16:46:38.153Z",360 "last_posted_at": "2024-12-08T19:58:20.444Z",361 "bumped": true,362 "bumped_at": "2024-12-08T19:58:20.444Z",363 "archetype": "regular",364 "unseen": false,365 "pinned": false,366 "unpinned": null,367 "visible": true,368 "closed": false,369 "archived": false,370 "bookmarked": null,371 "liked": null,372 "tags_descriptions": {},373 "like_count": 0,374 "views": 215,375 "category_id": 1,376 "featured_link": null,377 "has_accepted_answer": false,378 "posters": [379 {380 "extras": null,381 "description": "Original Poster",382 "user": {383 "id": 81339,384 "username": "Paulo_Nascimento",385 "name": "Paulo Nascimento",386 "avatar_template": "/user_avatar/discuss.pytorch.org/paulo_nascimento/{size}/72888_2.png",387 "trust_level": 0388 }389 },390 {391 "extras": null,392 "description": "Frequent Poster",393 "user": {394 "id": 72430,395 "username": "Eduardo_Lawson",396 "name": "Eduardo Lawson da Silva",397 "avatar_template": "/user_avatar/discuss.pytorch.org/eduardo_lawson/{size}/66899_2.png",398 "trust_level": 2399 }400 },401 {402 "extras": "latest",403 "description": "Most Recent Poster",404 "user": {405 "id": 18088,406 "username": "KFrank",407 "name": "K. Frank",408 "avatar_template": "/letter_avatar_proxy/v4/letter/k/ecb155/{size}.png",409 "trust_level": 2410 }411 }412 ]413 },414 {415 "fancy_title": "Use `torch.Tensor` as typehint",416 "id": 219334,417 "title": "Use `torch.Tensor` as typehint",418 "slug": "use-torch-tensor-as-typehint",419 "posts_count": 1,420 "reply_count": 0,421 "highest_post_number": 1,422 "image_url": null,423 "created_at": "2025-04-22T10:42:08.655Z",424 "last_posted_at": "2025-04-22T10:42:08.698Z",425 "bumped": true,426 "bumped_at": "2025-04-22T10:42:08.698Z",427 "archetype": "regular",428 "unseen": false,429 "pinned": false,430 "unpinned": null,431 "visible": true,432 "closed": false,433 "archived": false,434 "bookmarked": null,435 "liked": null,436 "tags_descriptions": {},437 "like_count": 0,438 "views": 201,439 "category_id": 1,440 "featured_link": null,441 "has_accepted_answer": false,442 "posters": [443 {444 "extras": "latest single",445 "description": "Original Poster, Most Recent Poster",446 "user": {447 "id": 45130,448 "username": "pascalm",449 "name": "",450 "avatar_template": "/user_avatar/discuss.pytorch.org/pascalm/{size}/76741_2.png",451 "trust_level": 1452 }453 }454 ]455 }456 ],457 "tags_descriptions": {},458 "fancy_title": "BCE loss output is not corresponding to calculation by hand",459 "id": 130847,460 "title": "BCE loss output is not corresponding to calculation by hand",461 "posts_count": 3,462 "created_at": "2021-09-01T11:27:27.484Z",463 "views": 738,464 "reply_count": 1,465 "like_count": 0,466 "last_posted_at": "2021-09-03T09:11:01.823Z",467 "visible": true,468 "closed": false,469 "archived": false,470 "has_summary": false,471 "archetype": "regular",472 "slug": "bce-loss-output-is-not-corresponding-to-calculation-by-hand",473 "category_id": 1,474 "word_count": 251,475 "deleted_at": null,476 "user_id": 48480,477 "featured_link": null,478 "pinned_globally": false,479 "pinned_at": null,480 "pinned_until": null,481 "image_url": null,482 "slow_mode_seconds": 0,483 "draft": null,484 "draft_key": "topic_130847",485 "draft_sequence": null,486 "unpinned": null,487 "pinned": false,488 "current_post_number": 1,489 "highest_post_number": 3,490 "deleted_by": null,491 "actions_summary": [492 {493 "id": 4,494 "count": 0,495 "hidden": false,496 "can_act": false497 },498 {499 "id": 8,500 "count": 0,501 "hidden": false,502 "can_act": false503 },504 {505 "id": 10,506 "count": 0,507 "hidden": false,508 "can_act": false509 },510 {511 "id": 7,512 "count": 0,513 "hidden": false,514 "can_act": false515 }516 ],517 "chunk_size": 20,518 "bookmarked": false,519 "topic_timer": null,520 "message_bus_last_id": 0,521 "participant_count": 2,522 "show_read_indicator": false,523 "thumbnails": null,524 "slow_mode_enabled_until": null,525 "can_vote": false,526 "vote_count": 0,527 "user_voted": false,528 "discourse_zendesk_plugin_zendesk_id": null,529 "discourse_zendesk_plugin_zendesk_url": "https://your-url.zendesk.com/agent/tickets/",530 "details": {531 "can_edit": false,532 "notification_level": 1,533 "participants": [534 {535 "id": 48480,536 "username": "ivprgrmmr",537 "name": "Ivprgrmmr",538 "avatar_template": "/user_avatar/discuss.pytorch.org/ivprgrmmr/{size}/41662_2.png",539 "post_count": 2,540 "primary_group_name": null,541 "flair_name": null,542 "flair_url": null,543 "flair_color": null,544 "flair_bg_color": null,545 "flair_group_id": null,546 "trust_level": 1547 },548 {549 "id": 18088,550 "username": "KFrank",551 "name": "K. Frank",552 "avatar_template": "/letter_avatar_proxy/v4/letter/k/ecb155/{size}.png",553 "post_count": 1,554 "primary_group_name": null,555 "flair_name": null,556 "flair_url": null,557 "flair_color": null,558 "flair_bg_color": null,559 "flair_group_id": null,560 "trust_level": 2561 }562 ],563 "created_by": {564 "id": 48480,565 "username": "ivprgrmmr",566 "name": "Ivprgrmmr",567 "avatar_template": "/user_avatar/discuss.pytorch.org/ivprgrmmr/{size}/41662_2.png"568 },569 "last_poster": {570 "id": 48480,571 "username": "ivprgrmmr",572 "name": "Ivprgrmmr",573 "avatar_template": "/user_avatar/discuss.pytorch.org/ivprgrmmr/{size}/41662_2.png"574 }575 },576 "bookmarks": []577 },578 {579 "post_stream": {580 "posts": [581 {582 "id": 304811,583 "name": "jhp",584 "username": "jhp",585 "avatar_template": "/user_avatar/discuss.pytorch.org/jhp/{size}/31228_2.png",586 "created_at": "2021-09-03T08:21:41.944Z",587 "cooked": "<p>Usually, transfer learning freezes the pre-trained model’s <code>BatchNorm running stats</code> (mean and var), so I did that but it achieves worse performance than when it was unfrozen. To be exact, the convergence is fast, but the generalization fails and the performance is not good.<br>\nFor my knowledge, I think it’s because the domain of the pre-trained model and my dataset are quite different. Is it right to unfreeze running stat in batchnorm in my use case or do other tricks exist for finetuning?<br>\nI’m still suspicious that this would make the pretrained model meaningless (cannot prevent <code>unforgetting</code>)</p>",588 "post_number": 1,589 "post_type": 1,590 "posts_count": 1,591 "updated_at": "2021-09-03T08:21:41.944Z",592 "reply_count": 0,593 "reply_to_post_number": null,594 "quote_count": 0,595 "incoming_link_count": 56,596 "reads": 4,597 "readers_count": 3,598 "score": 280.8,599 "yours": false,600 "topic_id": 131035,601 "topic_slug": "performance-drop-when-freeze-batchnorm-why",602 "display_username": "jhp",603 "primary_group_name": null,604 "flair_name": null,605 "flair_url": null,606 "flair_bg_color": null,607 "flair_color": null,608 "flair_group_id": null,609 "badges_granted": [],610 "version": 1,611 "can_edit": false,612 "can_delete": false,613 "can_recover": false,614 "can_see_hidden_post": false,615 "can_wiki": false,616 "read": true,617 "user_title": "",618 "bookmarked": false,619 "actions_summary": [],620 "moderator": false,621 "admin": false,622 "staff": false,623 "user_id": 47274,624 "hidden": false,625 "trust_level": 1,626 "deleted_at": null,627 "user_deleted": false,628 "edit_reason": null,629 "can_view_edit_history": true,630 "wiki": false,631 "post_url": "/t/performance-drop-when-freeze-batchnorm-why/131035/1",632 "can_accept_answer": false,633 "can_unaccept_answer": false,634 "accepted_answer": false,635 "topic_accepted_answer": null,636 "can_vote": false637 }638 ],639 "stream": [640 304811641 ]642 },643 "timeline_lookup": [644 [645 1,646 1514647 ]648 ],649 "suggested_topics": [650 {651 "fancy_title": "ValueError: You should supply an encoding or a list of encodings to this method that includes input_ids, but you provided [‘pixel_values’]",652 "id": 216220,653 "title": "ValueError: You should supply an encoding or a list of encodings to this method that includes input_ids, but you provided ['pixel_values']",654 "slug": "valueerror-you-should-supply-an-encoding-or-a-list-of-encodings-to-this-method-that-includes-input-ids-but-you-provided-pixel-values",655 "posts_count": 4,656 "reply_count": 1,657 "highest_post_number": 4,658 "image_url": null,659 "created_at": "2025-02-04T13:03:09.113Z",660 "last_posted_at": "2025-02-12T12:11:07.163Z",661 "bumped": true,662 "bumped_at": "2025-02-12T12:30:38.800Z",663 "archetype": "regular",664 "unseen": false,665 "pinned": false,666 "unpinned": null,667 "visible": true,668 "closed": false,669 "archived": false,670 "bookmarked": null,671 "liked": null,672 "tags_descriptions": {},673 "like_count": 0,674 "views": 445,675 "category_id": 5,676 "featured_link": null,677 "has_accepted_answer": false,678 "posters": [679 {680 "extras": "latest",681 "description": "Original Poster, Most Recent Poster",682 "user": {683 "id": 82473,684 "username": "milanalimova",685 "name": null,686 "avatar_template": "/user_avatar/discuss.pytorch.org/milanalimova/{size}/75456_2.png",687 "trust_level": 1688 }689 },690 {691 "extras": null,692 "description": "Frequent Poster",693 "user": {694 "id": 3534,695 "username": "ptrblck",696 "name": "",697 "avatar_template": "/user_avatar/discuss.pytorch.org/ptrblck/{size}/1823_2.png",698 "admin": true,699 "moderator": true,700 "trust_level": 2701 }702 }703 ]704 },705 {706 "fancy_title": "Loss becomes constant afterwarmup?",707 "id": 219517,708 "title": "Loss becomes constant afterwarmup?",709 "slug": "loss-becomes-constant-afterwarmup",710 "posts_count": 2,711 "reply_count": 0,712 "highest_post_number": 2,713 "image_url": "https://discuss.pytorch.org/uploads/default/optimized/3X/d/6/d6075778b201d499c33037d473674dbaf8d2eec7_2_1024x320.png",714 "created_at": "2025-04-27T17:21:06.202Z",715 "last_posted_at": "2025-04-27T19:51:08.959Z",716 "bumped": true,717 "bumped_at": "2025-04-27T19:51:08.959Z",718 "archetype": "regular",719 "unseen": false,720 "pinned": false,721 "unpinned": null,722 "visible": true,723 "closed": false,724 "archived": false,725 "bookmarked": null,726 "liked": null,727 "tags_descriptions": {},728 "like_count": 0,729 "views": 67,730 "category_id": 5,731 "featured_link": null,732 "has_accepted_answer": false,733 "posters": [734 {735 "extras": null,736 "description": "Original Poster",737 "user": {738 "id": 81430,739 "username": "ecoArcGaming",740 "name": "",741 "avatar_template": "/user_avatar/discuss.pytorch.org/ecoarcgaming/{size}/74453_2.png",742 "trust_level": 1743 }744 },745 {746 "extras": "latest",747 "description": "Most Recent Poster",748 "user": {749 "id": 18088,750 "username": "KFrank",751 "name": "K. Frank",752 "avatar_template": "/letter_avatar_proxy/v4/letter/k/ecb155/{size}.png",753 "trust_level": 2754 }755 }756 ]757 },758 {759 "fancy_title": "Help :frowning: Auto encoder predicts shapes ‘ok’ but lacks colours",760 "id": 214163,761 "title": "Help :( Auto encoder predicts shapes 'ok' but lacks colours",762 "slug": "help-auto-encoder-predicts-shapes-ok-but-lacks-colours",763 "posts_count": 3,764 "reply_count": 2,765 "highest_post_number": 5,766 "image_url": "https://discuss.pytorch.org/uploads/default/original/3X/5/0/501da039c27c2a1804a96da5f766dbf4ec662e9a.png",767 "created_at": "2024-12-12T18:31:06.027Z",768 "last_posted_at": "2024-12-19T14:51:00.132Z",769 "bumped": true,770 "bumped_at": "2024-12-19T14:51:00.132Z",771 "archetype": "regular",772 "unseen": false,773 "pinned": false,774 "unpinned": null,775 "visible": true,776 "closed": false,777 "archived": false,778 "bookmarked": null,779 "liked": null,780 "tags_descriptions": {},781 "like_count": 1,782 "views": 82,783 "category_id": 5,784 "featured_link": null,785 "has_accepted_answer": false,786 "posters": [787 {788 "extras": "latest",789 "description": "Original Poster, Most Recent Poster",790 "user": {791 "id": 81089,792 "username": "Aknw_Fen",793 "name": "Aknw Fen",794 "avatar_template": "/user_avatar/discuss.pytorch.org/aknw_fen/{size}/74156_2.png",795 "trust_level": 2796 }797 },798 {799 "extras": null,800 "description": "Frequent Poster",801 "user": {802 "id": 77908,803 "username": "mycul",804 "name": "",805 "avatar_template": "/user_avatar/discuss.pytorch.org/mycul/{size}/72394_2.png",806 "trust_level": 2807 }808 }809 ]810 },811 {812 "fancy_title": "How to combine pre-trained weights of components from different multimodal LLMs?",813 "id": 215119,814 "title": "How to combine pre-trained weights of components from different multimodal LLMs?",815 "slug": "how-to-combine-pre-trained-weights-of-components-from-different-multimodal-llms",816 "posts_count": 1,817 "reply_count": 0,818 "highest_post_number": 1,819 "image_url": null,820 "created_at": "2025-01-08T12:57:20.583Z",821 "last_posted_at": "2025-01-08T12:57:20.618Z",822 "bumped": true,823 "bumped_at": "2025-01-08T12:57:20.618Z",824 "archetype": "regular",825 "unseen": false,826 "pinned": false,827 "unpinned": null,828 "visible": true,829 "closed": false,830 "archived": false,831 "bookmarked": null,832 "liked": null,833 "tags_descriptions": {},834 "like_count": 0,835 "views": 42,836 "category_id": 5,837 "featured_link": null,838 "has_accepted_answer": false,839 "posters": [840 {841 "extras": "latest single",842 "description": "Original Poster, Most Recent Poster",843 "user": {844 "id": 81933,845 "username": "tcm",846 "name": "Minh Tu",847 "avatar_template": "/user_avatar/discuss.pytorch.org/tcm/{size}/74956_2.png",848 "trust_level": 1849 }850 }851 ]852 },853 {854 "fancy_title": "Adding an LSTM to a trained CNN ResNet during fine-tuning",855 "id": 222256,856 "title": "Adding an LSTM to a trained CNN ResNet during fine-tuning",857 "slug": "adding-an-lstm-to-a-trained-cnn-resnet-during-fine-tuning",858 "posts_count": 1,859 "reply_count": 0,860 "highest_post_number": 1,861 "image_url": null,862 "created_at": "2025-08-11T15:29:18.603Z",863 "last_posted_at": "2025-08-11T15:29:18.662Z",864 "bumped": true,865 "bumped_at": "2025-08-11T15:29:18.662Z",866 "archetype": "regular",867 "unseen": false,868 "pinned": false,869 "unpinned": null,870 "visible": true,871 "closed": false,872 "archived": false,873 "bookmarked": null,874 "liked": null,875 "tags_descriptions": {},876 "like_count": 0,877 "views": 29,878 "category_id": 5,879 "featured_link": null,880 "has_accepted_answer": false,881 "posters": [882 {883 "extras": "latest single",884 "description": "Original Poster, Most Recent Poster",885 "user": {886 "id": 85452,887 "username": "catb",888 "name": "catb",889 "avatar_template": "/letter_avatar_proxy/v4/letter/c/f14d63/{size}.png",890 "trust_level": 0891 }892 }893 ]894 }895 ],896 "tags_descriptions": {},897 "fancy_title": "Performance drop when freeze batchnorm. Why?",898 "id": 131035,899 "title": "Performance drop when freeze batchnorm. Why?",900 "posts_count": 1,901 "created_at": "2021-09-03T08:21:41.831Z",902 "views": 443,903 "reply_count": 0,904 "like_count": 0,905 "last_posted_at": "2021-09-03T08:21:41.944Z",906 "visible": true,907 "closed": false,908 "archived": false,909 "has_summary": false,910 "archetype": "regular",911 "slug": "performance-drop-when-freeze-batchnorm-why",912 "category_id": 5,913 "word_count": 102,914 "deleted_at": null,915 "user_id": 47274,916 "featured_link": null,917 "pinned_globally": false,918 "pinned_at": null,919 "pinned_until": null,920 "image_url": null,921 "slow_mode_seconds": 0,922 "draft": null,923 "draft_key": "topic_131035",924 "draft_sequence": null,925 "unpinned": null,926 "pinned": false,927 "current_post_number": 1,928 "highest_post_number": 1,929 "deleted_by": null,930 "actions_summary": [931 {932 "id": 4,933 "count": 0,934 "hidden": false,935 "can_act": false936 },937 {938 "id": 8,939 "count": 0,940 "hidden": false,941 "can_act": false942 },943 {944 "id": 10,945 "count": 0,946 "hidden": false,947 "can_act": false948 },949 {950 "id": 7,951 "count": 0,952 "hidden": false,953 "can_act": false954 }955 ],956 "chunk_size": 20,957 "bookmarked": false,958 "topic_timer": null,959 "message_bus_last_id": 0,960 "participant_count": 1,961 "show_read_indicator": false,962 "thumbnails": null,963 "slow_mode_enabled_until": null,964 "can_vote": false,965 "vote_count": 0,966 "user_voted": false,967 "discourse_zendesk_plugin_zendesk_id": null,968 "discourse_zendesk_plugin_zendesk_url": "https://your-url.zendesk.com/agent/tickets/",969 "details": {970 "can_edit": false,971 "notification_level": 1,972 "participants": [973 {974 "id": 47274,975 "username": "jhp",976 "name": "jhp",977 "avatar_template": "/user_avatar/discuss.pytorch.org/jhp/{size}/31228_2.png",978 "post_count": 1,979 "primary_group_name": null,980 "flair_name": null,981 "flair_url": null,982 "flair_color": null,983 "flair_bg_color": null,984 "flair_group_id": null,985 "trust_level": 1986 }987 ],988 "created_by": {989 "id": 47274,990 "username": "jhp",991 "name": "jhp",992 "avatar_template": "/user_avatar/discuss.pytorch.org/jhp/{size}/31228_2.png"993 },994 "last_poster": {995 "id": 47274,996 "username": "jhp",997 "name": "jhp",998 "avatar_template": "/user_avatar/discuss.pytorch.org/jhp/{size}/31228_2.png"999 }1000 },1001 "bookmarks": []1002 },1003 {1004 "post_stream": {1005 "posts": [1006 {1007 "id": 304665,1008 "name": "RB",1009 "username": "robeast",1010 "avatar_template": "/letter_avatar_proxy/v4/letter/r/ec9cab/{size}.png",1011 "created_at": "2021-09-02T09:19:38.245Z",1012 "cooked": "<p>Dear community,<br>\nI am working in the field of deploying PyTorch models. Our customers provide us models which have been <code>jit.scipt(...)</code>-ed and saved as <code>*.pt</code> files. This is the common model exchange interface they have defined for us.</p>\n<p>The “unholy” thing that I’m trying to achieve now is to concatenate two such scripted models in form of RecursiveScriptModules.</p>\n<pre><code class=\"lang-python\">model1 = torch.jit.load('customer_model1.pt')\nmodel2 = torch.jit.load('customer_model2.pt')\n\nprint(type(model1) # Gives me <class 'torch.jit._script.RecursiveScriptModule'>\nprint(type(model2) # Gives me <class 'torch.jit._script.RecursiveScriptModule'>\n\n# ???\n</code></pre>\n<p>The questions is whether I can somehow concatenate these two models? Every idea is highly appreciated…</p>\n<p>Edit: maybe a side note: we use the models only for inference</p>\n<p>Thank you in advance!<br>\nBest regards,<br>\nRB</p>",1013 "post_number": 1,1014 "post_type": 1,1015 "posts_count": 5,1016 "updated_at": "2021-09-02T09:22:04.486Z",1017 "reply_count": 0,1018 "reply_to_post_number": null,1019 "quote_count": 0,1020 "incoming_link_count": 236,1021 "reads": 9,1022 "readers_count": 8,1023 "score": 1181.8,1024 "yours": false,1025 "topic_id": 130956,1026 "topic_slug": "jit-concatenating-two-recursivescriptmodules",1027 "display_username": "RB",1028 "primary_group_name": null,1029 "flair_name": null,1030 "flair_url": null,1031 "flair_bg_color": null,1032 "flair_color": null,1033 "flair_group_id": null,1034 "badges_granted": [],1035 "version": 1,1036 "can_edit": false,1037 "can_delete": false,1038 "can_recover": false,1039 "can_see_hidden_post": false,1040 "can_wiki": false,1041 "read": true,1042 "user_title": null,1043 "bookmarked": false,1044 "actions_summary": [],1045 "moderator": false,1046 "admin": false,1047 "staff": false,1048 "user_id": 48683,1049 "hidden": false,1050 "trust_level": 1,1051 "deleted_at": null,1052 "user_deleted": false,1053 "edit_reason": null,1054 "can_view_edit_history": true,1055 "wiki": false,1056 "post_url": "/t/jit-concatenating-two-recursivescriptmodules/130956/1",1057 "can_accept_answer": false,1058 "can_unaccept_answer": false,1059 "accepted_answer": false,1060 "topic_accepted_answer": true,1061 "can_vote": false1062 },1063 {1064 "id": 304754,1065 "name": "",1066 "username": "ptrblck",1067 "avatar_template": "/user_avatar/discuss.pytorch.org/ptrblck/{size}/1823_2.png",1068 "created_at": "2021-09-02T23:02:19.232Z",1069 "cooked": "<p>Would you like to “concatenate” them such that the output of <code>model1</code> would be passed to <code>model2</code> in a sequential way? If so, then you might be able to create a new custom <code>nn.Module</code>, use both of the scripted models there, and script the new “parent model” again (if needed).<br>\nLet me know, if I misunderstood your question.</p>",1070 "post_number": 2,1071 "post_type": 1,1072 "posts_count": 5,1073 "updated_at": "2021-09-02T23:02:19.232Z",1074 "reply_count": 1,1075 "reply_to_post_number": null,1076 "quote_count": 0,1077 "incoming_link_count": 3,1078 "reads": 8,1079 "readers_count": 7,1080 "score": 21.6,1081 "yours": false,1082 "topic_id": 130956,1083 "topic_slug": "jit-concatenating-two-recursivescriptmodules",1084 "display_username": "",1085 "primary_group_name": null,1086 "flair_name": null,1087 "flair_url": null,1088 "flair_bg_color": null,1089 "flair_color": null,1090 "flair_group_id": null,1091 "badges_granted": [],1092 "version": 1,1093 "can_edit": false,1094 "can_delete": false,1095 "can_recover": false,1096 "can_see_hidden_post": false,1097 "can_wiki": false,1098 "read": true,1099 "user_title": "",1100 "bookmarked": false,1101 "actions_summary": [],1102 "moderator": true,1103 "admin": true,1104 "staff": true,1105 "user_id": 3534,1106 "hidden": false,1107 "trust_level": 2,1108 "deleted_at": null,1109 "user_deleted": false,1110 "edit_reason": null,1111 "can_view_edit_history": true,1112 "wiki": false,1113 "post_url": "/t/jit-concatenating-two-recursivescriptmodules/130956/2",1114 "can_accept_answer": false,1115 "can_unaccept_answer": false,1116 "accepted_answer": false,1117 "topic_accepted_answer": true1118 },1119 {1120 "id": 304788,1121 "name": "RB",1122 "username": "robeast",1123 "avatar_template": "/letter_avatar_proxy/v4/letter/r/ec9cab/{size}.png",1124 "created_at": "2021-09-03T05:55:43.378Z",1125 "cooked": "<p>Dear <a class=\"mention\" href=\"/u/ptrblck\">@ptrblck</a>,<br>\nthank you for your answer, yes that is exactly what I would like to achieve. However, I’m not sure how to convert even a single <code>RecursiveScriptModule</code> into an <code>nn.Module</code> without it’s original class definition available. I’d highly appreciate some hints please <img src=\"https://discuss.pytorch.org/images/emoji/apple/slight_smile.png?v=10\" title=\":slight_smile:\" class=\"emoji\" alt=\":slight_smile:\"></p>\n<p>Thank you & Best regards,<br>\nRB</p>",1126 "post_number": 3,1127 "post_type": 1,1128 "posts_count": 5,1129 "updated_at": "2021-09-03T05:55:43.378Z",1130 "reply_count": 1,1131 "reply_to_post_number": 2,1132 "quote_count": 0,1133 "incoming_link_count": 5,1134 "reads": 7,1135 "readers_count": 6,1136 "score": 31.4,1137 "yours": false,1138 "topic_id": 130956,1139 "topic_slug": "jit-concatenating-two-recursivescriptmodules",1140 "display_username": "RB",1141 "primary_group_name": null,1142 "flair_name": null,1143 "flair_url": null,1144 "flair_bg_color": null,1145 "flair_color": null,1146 "flair_group_id": null,1147 "badges_granted": [],1148 "version": 1,1149 "can_edit": false,1150 "can_delete": false,1151 "can_recover": false,1152 "can_see_hidden_post": false,1153 "can_wiki": false,1154 "read": true,1155 "user_title": null,1156 "reply_to_user": {1157 "id": 3534,1158 "username": "ptrblck",1159 "name": "",1160 "avatar_template": "/user_avatar/discuss.pytorch.org/ptrblck/{size}/1823_2.png"1161 },1162 "bookmarked": false,1163 "actions_summary": [],1164 "moderator": false,1165 "admin": false,1166 "staff": false,1167 "user_id": 48683,1168 "hidden": false,1169 "trust_level": 1,1170 "deleted_at": null,1171 "user_deleted": false,1172 "edit_reason": null,1173 "can_view_edit_history": true,1174 "wiki": false,1175 "post_url": "/t/jit-concatenating-two-recursivescriptmodules/130956/3",1176 "can_accept_answer": false,1177 "can_unaccept_answer": false,1178 "accepted_answer": false,1179 "topic_accepted_answer": true1180 },1181 {1182 "id": 304797,1183 "name": "",1184 "username": "ptrblck",1185 "avatar_template": "/user_avatar/discuss.pytorch.org/ptrblck/{size}/1823_2.png",1186 "created_at": "2021-09-03T06:30:40.608Z",1187 "cooked": "<p>I had something like this in mind:</p>\n<pre><code class=\"lang-python\"># save\nmodel1 = nn.Linear(10, 5)\nmodel2 = nn.Linear(5, 2)\n\nmodel1 = torch.jit.script(model1)\nmodel2 = torch.jit.script(model2)\n\ntorch.jit.save(model1, 'model1.pt')\ntorch.jit.save(model2, 'model2.pt')\n\n\n# load\nclass MyModel(nn.Module):\n def __init__(self, model1, model2):\n super().__init__()\n self.model1 = model1\n self.model2 = model2\n \n def forward(self, x):\n x = self.model1(x)\n x = self.model2(x)\n return x\n\nmodel1 = torch.jit.load('model1.pt')\nmodel2 = torch.jit.load('model2.pt')\nmodel = MyModel(model1, model2)\n\nx = torch.randn(1, 10)\nout = model(x)\nprint(out.shape)\n> torch.Size([1, 2])\n</code></pre>\n<p>I.e. just loading the models and using them wrapped in another parent model.</p>",1188 "post_number": 4,1189 "post_type": 1,1190 "posts_count": 5,1191 "updated_at": "2021-09-03T08:14:06.123Z",1192 "reply_count": 0,1193 "reply_to_post_number": 3,1194 "quote_count": 0,1195 "incoming_link_count": 4,1196 "reads": 7,1197 "readers_count": 6,1198 "score": 21.4,1199 "yours": false,1200 "topic_id": 130956,